The sharp rise in the usage of smartphones has spawned the development of countless mobile apps and in the process, expedited digital transformation. In fact, consumers downloaded a record of 204 billion apps in 2019 as cited in the 2020 State of Mobile by App Annie. As more and more mobile apps are developed and launched, the availability of alternatives for an existing app is increasing. Thus, keeping engagement levels high has become a priority requirement as Apps look at innovative ways to keep customers from churning to competition.

And integral to preventing churn is ensuring a great experience on the App. App experience is not just a function of making it bug free or designing a great User Experience (UX) but about delivering a frictionless performance. Any lag in the app can result in user abandonment.

There exist two types of methods to monitor experience – Real User Monitoring (RUM) and Synthetic Experience Monitoring (SEM).

What is Synthetic Experience Monitoring?

Synthetic Experience Monitoring is a method of performing continuous tests with the aid of robotic agents that emulate the behavior of a real user. Thus, it helps measure App performance very rapidly and under more user conditions such that identifying issues leading to a poor experience are determined easily.

It can be performed on mobile devices that are either present on the cloud or in your premise. Our SEM solution works on the device cloud of your choice. It helps your measure key metrics that define user experience and provides you with engineering insights to improve that experience.

App developers typically make use of Real User Monitoring (RUM) SDKs such as Firebase to instrument their app. While RUM is a great way to get an aggregate view of app experience it has its own drawbacks:

  • RUM is reactive and not proactive, meaning issues are found only when the user is using the app and not before.
  • The number of journeys that can be monitored is limited to the depth of instrumentation of the app. As journeys increase, the cost of instrumentation increases.
  • Monitoring of 3rd party integration performance such as authentication SDKs, payment gateways, or CDN is not possible using RUM SDKs.

Synthetic Experience Monitoring (SEM) is designed to get over these limitations. SEM is a non-intrusive method to monitor user experience.

How does Synthetic Experience Monitoring work?

With the help of SEM, you can continuously monitor user experience under various conditions such as location, time of the day, network condition, operator, and device make & model without much intervention from your side. Thus, you can quickly identify potential causes of why your app loads more slowly than competitors in a given scenario or why is content loading slowly even if the network speed is not an issue.

Our SEM solution works on Application Quality Index (AQI), a proprietary methodology for quantitatively measuring experience. It measures the App’s responsiveness or usability and content delivering experience or the way real user would perceive the app once it loads. More factors that determine AQI are shown in Figure 1. AQI varies as the app category varies with different metrics measured for more real-time transaction-based apps such as banking & payments and gaming from content-heavy apps such as OTTs and News apps.

Figure1: Sample AQI protocols for testing a Media OTT- Streaming Application

The platform focuses on three major ways to build AQI:

  • Automation Logs, which provide an understanding of on-field App Experience,
  • Device Logs, which provide details on on-device resource performance, and
  • HTTP Logs, which provide App to backend diagnostics.

SEM gives you an advantage of monitoring applications in an automated fashion, thus saving on extensive hours of manual testing and expanding the horizon from other RUM-based instrumentation tools. It lets you monitor 3rd party API and SDK integrations, performance of CDNs, and monitor for network and device anomalies that could lead to poor experience.

How using Synthetic Experience Monitoring gives you an edge?

Test App for Real World Network Conditions

Combat network congestion, network interference, and variations in wireless and wireline networks by assessing your app performance. Monitor on real devices connected to real networks across a range of device locations around the globe.

Monitor User Journeys

Persistently measure and screen every single critical user journey in real-time without any SDK integration or code changes to your app. Users usually encounter bottlenecks on the home page load, search time, video play start, etc. Hence, consistent and rigorous testing is very much needed to steer clear of customer dissatisfaction.

Identify 3rd Party Integration

Some applications are integrating around 10 3rd party SDKs or more. These include SDKs for identity management, user analytics, CDNs, fraud preventions, performance tracking, etc. It has now become imperative to diagnose SDKs bloating your app and ramping down your app experience in real-time.

Diagnose CDN performance

Perform a deep-dive into the Content Delivery Architecture to detect potential causes of lags, delays, and poor performance. Benchmark services of CDNs at a local level to build the best multi-CDN architecture for your app.

Who should use Synthetic Experience Monitoring?

  • Product Managers

    Understanding your app’s architecture well and how it impacts the content or services you deliver would allow you to accurately identify ways to improve user experience and in turn drive engagement. Rigorous, experience-focused testing from pre-launch to post-launch will help you pro-actively address customer issues as your teams would have access to deeper insights on issues that are leading to poor experience. Meanwhile, your business teams can benefit from understanding your competitive ranking in times when experience is the focal point of retaining customers.

  • App Testers

    As the QA owner of an app, it is highly necessary to spot and eliminate bugs such that the app itself works smoothly with bare minimum interruptions for customers. Achieving this though requires automated and continuous monitoring of the app on various devices, networks, and in cities so that you are able to diagnose the app as it would be perceived by a user. SEM offers that kind of capability to developers and testers.

In this day and age, SEM is the means to rapidly testing and improving your app in ways such that end-user experience is good always. With changing usage behaviors and customer expectations, user-experience is at the center of success for almost all apps. Experience now commands as much importance as content quality. As competition rises, app owners and developers must take advantage of the agility offered by SEM and stay ahead of the competition.

Learn more about how Synthetic Experience Monitoring can be your most reliant partner in delivering exceptional user experience. Sign up now and consult our QoE experts for improving your App experience:

Written By

Sourabh Chowdhury & Perdiep (Perry) Jankie