Datadog provides default rules so you can get started quickly with Scorecards, but you can also create custom rules.

Create custom rules

To add and evaluate custom rules using the Scorecards API:

1. Specify the name of the rule, the scorecard it belongs to, a rule description, and an owner to pass to /scorecard/rules.
2. Send an outcome of pass, fail, or skip for each {rule, service} tuple that you are evaluating to /scorecard/outcomes/batch.
3. View an overview of outcomes and remarks in the Scorecards dashboard.

After initial setup, rules can also be enabled or disabled through the API.

To evaluate and add custom rules in the Scorecards UI:

1. Click Create Rule on the Scorecards page.
2. Specify the name of the rule, the scorecard it belongs to, a rule description, and the owning team.
3. Navigate to the rule you created and select Edit Outcome next to the service that you want to evaluate.
4. Select the relevant outcome of pass, fail, or skip and add an optional remark describing the reason for the outcome.
5. View an overview of outcomes and remarks in the Scorecards dashboard.

Evaluate custom rules using Workflow Automation

Workflow Automation allows you to automate the evaluation of your custom rules in Datadog using the Update scorecard rule outcome action.

When you create a custom rule, you can evaluate it using one of the following options:

• Add Workflow:
  • Create a workflow from scratch.
  • Create a workflow from a Scorecards blueprint.
  • Link an existing custom workflow.
• Use the Scorecards API.

Create workflow

After creating your custom rule, choose the Add Workflow option when prompted to configure its evaluation criteria. From there, you can create a workflow from scratch or use a blueprint.

From scratch

To create a workflow from scratch:

1. Set a schedule for your workflow to run on.
2. Click the plus (+) icon to add a step.
3. Select the List service definitions action to fetch all defined services from Software Catalog.
4. Click the plus (+) icon to add a step.
5. Select the For loop to iterate over each service one-by-one.
6. Select the action needed to fetch your evaluation data (for example: “List monitors” or “Get repository content” from GitHub).
7. Transform the returned data using a custom JavaScript function to generate pass/fail outcomes for each service.
8. Use the Update scorecard rule outcome action to send results to Scorecards.
9. Run the workflow to see your evaluations appear in Scorecards for your custom rule.
10. Publish the workflow; unpublished workflows do not run automatically.

From blueprint

Instead of creating a workflow from scratch, you can start with a Scorecards blueprint:

1. Explore Scorecards blueprints and select one.

2. Edit the blueprint to create your custom workflow.

   Note: All Scorecards blueprints include the Update scorecard rule outcome action to send results to Scorecards. Do not delete this step.

3. Run the workflow to see your evaluations appear in Scorecards for your custom rule.

4. Publish the workflow; unpublished workflows do not run automatically.

After the workflow is created, it is automatically linked to the Scorecard rule. Click the link to open the workflow, make changes, and view details such as the owner, run history, and last modified date.

All Workflows created from the Scorecards page or from a Scorecards blueprint are automatically tagged with source: scorecards for searching and filtering.

Note: Clicking the red unlink button on a Scorecard rule removes the workflow from the rule but does not automatically unpublish the workflow. To prevent evaluations from running, you should also unpublish any unlinked workflows.

Link existing workflow

After creating your custom rule, choose the Add Workflow option to configure its evaluation criteria. Search for and select an existing workflow to link to the rule.

You can also link an existing workflow to any custom rule that already generates outcomes:

1. Click the custom rule.
2. Select Link Workflow.
3. Search for a workflow and select it to link it to the rule.

Further reading

추가 유용한 문서, 링크 및 기사:

Software CatalogDOCUMENTATION Scorecards APIDOCUMENTATION Prioritize and promote service observability best practices with ScorecardsBLOG Formalize best practices with custom ScorecardsBLOG Track DORA Metrics with DatadogDOCUMENTATION