Overview

To use Datadog Quality Gates, you can define one or more rules on the Quality Gate Rules page and integrate them in your CI pipeline with the datadog-ci gate evaluate command.

Quality Gates ensures that only the code that meets your quality standards is deployed, automating your quality assurance processes and enhancing software reliability.

Create a rule

To create a Quality Gates rule in Datadog:

1. Navigate to Software Delivery > Quality Gates > Quality Gate Rules and click + New Rule.

2. Select a type of rule: Test, Pipeline, Static Analysis, or Software Composition Analysis.

3. Set the rule scope, which defines when the rule should be evaluated, by selecting Always evaluate or Select when to evaluate. You can add branches or repositories to include or exclude from the rule scope, or add a custom scope.

   You can create a rule that is evaluated only on specific repositories and branches. To customize the rule scope, click Select when to evaluate and specify the branch or repository that should be included or excluded.

   To add a custom scope (such as a team name), click + Add Filter and select Custom scope. Enter a tag name without spaces (such as documentation or team-documentation) and click Add Custom Scope. Enter values that should be included or excluded.

   When adding a custom scope to a rule, custom scopes must be passed to the datadog-ci gate evaluate command using the --scope option. For more information, see Understanding Rule Scopes.

4. Define the rule conditions. The rule condition states in which scenario the rule fails, failing the related pipeline (if the rule is blocking). You can select one of the existing rule conditions for the rule type you have selected. If the rule scope is set to always evaluate, the rule is evaluated on all repositories and branches.

   The following example demonstrates how to create a Static Analysis rule that fails when one or more Static Analysis code vulnerability violations with error status are introduced in a specific commit.

   Select Static Analysis for the rule type and click Always evaluate for the rule scope.

   In the Define rule conditions section, click New and select code vulnerabilities violations from the dropdown menu. Then, select the error status type, select above or equal to, and enter the value of 1.

5. Specify a rule name that describes the rule that you are creating.

6. Select whether the rule should block the pipeline when it fails. Non-blocking rules are helpful when you roll out a new rule and want to verify its behavior before making it blocking.

7. Integrate the Quality Gate rule into your build configuration by including the datadog-ci gate evaluate command.

8. Enable a GitHub status check for your Quality Gate rule by setting the appropriate permissions (such as Checks: Write) in your GitHub apps. To set this check as blocking in your pull requests, you must click the Required checkbox in your GitHub app’s Protected Branches settings.

9. Click Create Rule.

Integrate Quality Gates in your CI/CD pipeline

Invoke the Quality Gates evaluation by calling the datadog-ci gate evaluate command. Quality Gates requires datadog-ci version 2.27.0 or later.

This command:

1. Retrieves all the rules that have rule scopes and custom scopes that match the current pipeline context (the repository, branch, or custom scope(s) passed in the command).
2. Evaluates all the matching rules.
3. Fails if one or more blocking rules fail, blocking the pipeline.

For example:

DD_SITE= DD_API_KEY=API_KEY DD_APP_KEY=APP_KEY datadog-ci gate evaluate

Configure the behavior of the datadog-ci gate evaluate command using the following flags:

--fail-on-empty

The command fails if no matching rules are found based on the current pipeline context. By default, the command succeeds.

--fail-if-unavailable

The command fails if one or more rules cannot be evaluated because of an internal issue. By default, the command succeeds.

--timeout

The command stops its execution after the specified timeout in seconds. The default timeout is 10 minutes. The command typically completes within a few minutes, but it could take longer.

--no-wait

Skips the default time that the command waits for the events (for example, tests, static analysis violations) to arrive to Datadog. The default wait time makes sure that the events are queryable in Datadog before the rules are executed, avoiding incorrect evaluations. If, in your pipeline, the job containing the datadog-ci gate evaluate command is called several minutes after the related events are sent to Datadog, you can opt to skip this waiting time by specifying the --no-wait flag. However, if used incorrectly, this flag may result in inaccurate rule evaluations.

Add custom scopes to the command by using the --scope option one or more times:

datadog-ci gate evaluate --scope team:backend --scope team:frontend

Check the command logs to see the overall gate evaluation status and information about the rules that were evaluated.

Enable GitHub check creation

You can automatically create a GitHub check for each rule evaluated. The check contains additional information about the rule evaluation, such as the failure reason and the matching events in Datadog. When this feature is enabled, the evaluation results appear directly in GitHub.

To enable GitHub Checks:

1. Navigate to the GitHub integration tile. If you do not have this integration set up, or you don’t have a GitHub app within the integration, follow the GitHub integration documentation to set one up.
2. Grant Checks: Write access to the GitHub application.

After the permission is granted, you can see the checks in GitHub.

Note: Re-running a check does not re-run the corresponding Quality Gates rule.

Manage rules

You can edit and delete Quality Gates rules by hovering over a rule on the Quality Gates Rules page.

Alternatively, click on a rule from the list and click the Edit, Clone, or Delete icons.

Permissions

Only users with the quality_gate_rules_write permission can create and edit Quality Gate rules. Users with the quality_gate_rules_read permission can view Quality Gate rules.

For more information, see the RBAC Permissions documentation.

Further reading

Additional helpful documentation, links, and articles:

Learn about Quality GatesDOCUMENTATION