Continuous Testing and CircleCI Orb

Docs > Continuous Testing > Continuous Testing and CI/CD > Continuous Testing and CircleCI Orb

Overview

Run Synthetic tests in your CircleCI pipelines using the Datadog CircleCI orb.

The CircleCI command orb installs datadog-ci and uses the datadog-ci synthetics run-tests command to execute Datadog Synthetic tests.

Setup

To get started:

Add your Datadog API and application keys as environment variables to your CircleCI project.
- For more information, see API and Application Keys.
Ensure the image running the orb is a Linux x64 base image with cURL installed.
Customize your workflow by creating a run-tests.yml file and following the naming conventions to specify inputs for your workflow.

Your workflow can be simple or complex.

Simple usage

Example orb usage using public IDs

version: 2.1

orbs:
  synthetics-ci: datadog/synthetics-ci-orb@4.2.0

jobs:
  e2e-tests:
    docker:
      - image: cimg/base:stable
    steps:
      - synthetics-ci/run-tests:
          public_ids: |
            abc-d3f-ghi
            jkl-mn0-pqr            

workflows:
  run-tests:
    jobs:
      - e2e-tests

Example orb usage using a global configuration override

This orb overrides the path to the pattern for test files.

version: 2.1

orbs:
  synthetics-ci: datadog/synthetics-ci-orb@4.2.0

jobs:
  e2e-tests:
    docker:
      - image: cimg/base:stable
    steps:
      - synthetics-ci/run-tests:
          files: e2e-tests/*.synthetics.json

workflows:
  run-tests:
    jobs:
      - e2e-tests

For another example pipeline that triggers Synthetic tests, see the simple-example.yml file.

Complex usage

Example orb usage using the `test_search_query`

version: 2.1

orbs:
  synthetics-ci: datadog/synthetics-ci-orb@4.2.0

jobs:
  e2e-tests:
    docker:
      - image: cimg/base:stable
    steps:
      - synthetics-ci/run-tests:
          test_search_query: 'tag:e2e-tests'

workflows:
  run-tests:
    jobs:
      - e2e-tests

Example orb usage using the Continuous Testing Tunnel

version: 2.1

orbs:
  synthetics-ci: datadog/synthetics-ci-orb@4.2.0

jobs:
  e2e-tests:
    docker:
      - image: your-image
    steps:
      - checkout
      - run:
          name: Running server in background
          command: npm start
          background: true
      - synthetics-ci/run-tests:
          config_path: tests/tunnel-config.json
          files: tests/*.synthetics.json
          test_search_query: 'tag:e2e-tests'
          tunnel: true

workflows:
  test-server:
    jobs:
      - build-image
      - integration-tests:
          requires:
            - build-image

For additional options such as customizing the batchTimeout for your CircleCI pipelines, see CI/CD Integrations Configuration. For another example pipeline that starts a local server and triggers Synthetic tests using the Continuous Testing Tunnel, see the advanced-example.yml file.

Inputs

To customize your workflow, you can set the following parameters in a run-tests.yml file:

Name	Description
`api_key`	The name of the environment variable containing the API key. _{Default: DATADOG_API_KEY}
`app_key`	The name of the environment variable containing the APP key. _{Default: DATADOG_APP_KEY}
`background`	Whether or not this step should run in the background. _{See official CircleCI documentation for the default value.}
`batch_timeout`	The duration (in milliseconds) after which the batch fails as timed out. _{Default: 1800000 (30 minutes)}
`config_path`	The global JSON configuration used when launching tests. _{Default: datadog-ci.json}
`fail_on_critical_errors`	Fail if tests were not triggered or results could not be fetched. _{Default: false}
`fail_on_missing_tests`	Fail if at least one specified test with a public ID (using `public_ids` or listed in a test file) is missing in a run (for example, if it has been deleted programmatically or on the Datadog site). _{Default: false}
`fail_on_timeout`	Force the CI to fail (or pass) if one of the results exceeds its test timeout. _{Default: true}
`files`	A list of glob patterns to detect Synthetic tests config files, separated by new lines. _{Default: {,!(node_modules)/*/}.synthetics.json}
`junit_report`	The filename for a JUnit report if you want to generate one. _{Default: none}
`locations`	String of locations separated by semicolons to override the locations where your tests run. _{Default: none}
`no_output_timeout`	Elapsed time the command can run without output. The string is a decimal with unit suffix, such as “20m”, “1.25h”, “5s”. _{See official CircleCI documentation for the default value.}
`public_ids`	Public IDs of Synthetic tests to run, separated by new lines or commas. If no value is provided, tests are discovered in `*.synthetics.json` files. _{Default: none}
`site`	The Datadog site to send data to. If the `DD_SITE` environment variable is set, it takes preference. _{Default: datadoghq.com}
`subdomain`	The name of the custom subdomain set to access your Datadog application. _{Default: app}
`test_search_query`	Trigger tests corresponding to a search query. _{Default: none}
`tunnel`	Use the Continuous Testing Tunnel to trigger tests. _{Default: false}
`variables`	Key-value pairs for injecting variables into tests, separated by newlines or commas. For example: `START_URL=https://example.org,MY_VARIABLE=My title`. _{Default: none}

To learn about additional options for your CircleCI pipelines, see Continuous Testing & CI/CD Integrations Configuration.