Mobile App Testing

Docs > Synthetic Testing and Monitoring > Mobile Application Testing and Monitoring > Mobile App Testing

Overview

Mobile Application Testing allows you to test and monitor key business flows for Android and iOS applications using real devices.

Mobile app tests can run on a schedule, on-demand, or directly within your CI/CD pipelines.

You can create mobile app tests in Datadog by navigating to Digital Experience > New Test and selecting Mobile Application Test.

Flakiness

Flakiness is a pain point in end-to-end testing because tests occasionally fail. When a frontend team implements a change, an identifier in your test may alert on it instead of an actual application issue.

To prevent flaky tests, Datadog uses an algorithm that leverages a set of locators to target elements in mobile app tests. A small change in the UI may modify an element (for example, moving it to another location). The mobile app test automatically locates the element again based on points of reference that are not affected by the change.

When the test runs successfully, the mobile app test recomputes (or “self heals”) any broken locators with updated values. This ensures your tests do not break from simple UI updates and your tests are automatically adapting to your mobile application’s UI.

Configuration

Define the configuration of your mobile app test.

Select a mobile application from the dropdown menu. If you haven’t created one already, create a mobile application in the Applications List section on the Synthetic Monitoring & Continuous Testing Settings page.
Select a version or click Always run the latest version to use the latest version of your mobile application whenever your test is run.
Add a name for your test.
Select environment and additional tags that relate to your test. Use the <KEY>:<VALUE> format to filter on a <VALUE> for a given <KEY>.
Select the devices to run your test on.
Set retry conditions for your test.
Set the test frequency by clicking on basic time intervals or customizing your test frequency and alert conditions for your test monitor.
Enter a name for the test monitor, select a service or team member to notify, and add a message notification.

Variables

Create local variables

To create a local variable, click Create a Local Variable. You can select one of the following available builtins to add to your variable string:

{{ numeric(n) }}: Generates a numeric string with n digits.
{{ alphabetic(n) }}: Generates an alphabetic string with n letters.
{{ alphanumeric(n) }}: Generates an alphanumeric string with n characters.
{{ date(n unit, format) }}: Generates a date in one of Datadog’s accepted formats with a value corresponding to the UTC date the test is initiated at + or - n units.
{{ timestamp(n, unit) }}: Generates a timestamp in one of Datadog’s accepted units with a value corresponding to the UTC timestamp the test is initiated at +/- n units.
{{ uuid }}: Generates a version 4 universally unique identifier (UUID).
{{ public-id }}: Injects the Public ID of your test.
{{ result-id }}: Injects the Result ID of your test run.

To obfuscate local variable values in test results, select Hide and obfuscate variable value. Once you have defined the variable string, click Add Variable.

Use global variables

You can use the global variables defined in Settings in the Starting URL and Advanced Options of your mobile app test details, as well as in your test recording to define local variables. To display a list of available variables, type {{ in the desired field.

Define the variables you want to incorporate into the user journey before you start recording.

You can inject the variables available to you while recording. For more information about using variables in your mobile test recording, see Mobile App Test Steps.

Test retries

You can specify how much time a test needs to fail before triggering a notification alert.

Retry X times after Y ms in case of failure.

Scheduling and alerts

By default, mobile app tests are set up for on-demand testing, meaning these tests can run directly in a CI pipeline.

Scheduling and alerting conditions for a mobile test

You can customize alert conditions to define how often you want to send an alert and the circumstances under which you want a test to send a notification alert.

An alert is triggered if any assertion fails for X minutes.

Configure the test monitor

A notification is sent according to the set of alerting conditions. Use this section to define how and what to message your teams.

Enter a message for the mobile app test. This field allows standard Markdown formatting and supports the following conditional variables:

Conditional Variable	Description
`{{#is_alert}}`	Show when the monitor alerts.
`{{^is_alert}}`	Show unless the monitor alerts.
`{{#is_recovery}}`	Show when the monitor recovers from `alert`.
`{{^is_recovery}}`	Show unless the monitor recovers from `alert`.
`{{#is_renotify}}`	Show when the monitor renotifies.
`{{^is_renotify}}`	Show unless the monitor renotifies.
`{{#is_priority}}`	Show when the monitor matches priority (P1 to P5).
`{{^is_priority}}`	Show unless the monitor matches priority (P1 to P5).

Notification messages include the message defined in this section and information about the failing locations.

Choose team members and services to notify.
Specify a renotification frequency. To prevent renotification on failing tests, leave the option as Never renotify if the monitor has not been resolved.
Click Save & Edit Recording to save your test configuration and record your mobile app test steps.

For more information, see Using Synthetic Test Monitors.

Run tests in CI

You can run mobile app tests in a CI pipeline by defining the mobileApplicationVersionFilePath option in a test synthetics.json file and a global configuration synthetics-ci.config file as needed. Global configuration file options take precedence over test configuration file options.

In this example, the test aaa-aaa-aaa runs with the override application version found in application/path.

// myTest.synthetics.json
{
  "tests": [
    {
      "id": "aaa-aaa-aaa",
      "config": {
        "mobileApplicationVersionFilePath": "application/path"
      }
    }
  ]
}

Then, run $ datadog-ci synthetics run-tests --config synthetics-ci.config.

For more information, see Continuous Testing and CI/CD.

Permissions

By default, only users with the Datadog Admin and Datadog Standard roles can create, edit, and delete Synthetic mobile app tests. To get create, edit, and delete access to Synthetic mobile app tests, upgrade your user to one of those two default roles.

If you are using the custom role feature, add your user to any custom role that includes synthetics_read and synthetics_write permissions.

Restrict access

Access restriction is available for customers using custom roles on their accounts.

You can restrict access to a mobile app test based on the roles in your organization. When creating a mobile app test, choose which roles (in addition to your user) can read and write your test.