Overview

Buildkite is a continuous integration and deployment platform that allows you to run builds on your own infrastructure, providing you with full control over security and customizing your build environment while managing orchestration in the cloud.

Set up tracing on Buildkite to optimize your resource usage, reduce overhead, and improve the speed and quality of your software development lifecycle.

Compatibility

Configure the Datadog integration

To set up the Datadog integration for Buildkite:

1. Go to Settings > Notification Services in Buildkite and click the Add button next to Datadog Pipeline Visibility.
2. Fill in the form with the following information:
   • Description: A description to help identify the integration in the future, such as Datadog CI Visibility integration.
   • API key: Your Datadog API Key.
   • Datadog site: datadoghq.com
   • Pipelines: Select all pipelines or the subset of pipelines you want to trace.
   • Branch filtering: Leave empty to trace all branches or select the subset of branches you want to trace.
3. Click Add Datadog Pipeline Visibility Notification to save the integration.

Advanced configuration

Set custom tags

Custom tags can be added to Buildkite traces by using the buildkite-agent meta-data set command. Any metadata tags with a key starting with dd_tags. are added to the job and pipeline spans. These tags can be used to create string facets to search and organize the pipelines.

The YAML below illustrates a simple pipeline where tags for the team name and the Go version have been set.

steps:
  - command: buildkite-agent meta-data set "dd_tags.team" "backend"
  - command: go version | buildkite-agent meta-data set "dd_tags.go.version"
    label: Go version
  - commands: go test ./...
    label: Run tests

The following tags are shown in the root span as well as the relevant job span in Datadog.

• team: backend
• go.version: go version go1.17 darwin/amd64 (output depends on the runner)

The resulting pipeline looks like the following:

Any metadata with a key starting with dd-measures. and containing a numerical value will be set as a metric tag that can be used to create numerical measures.

You can use the buildkite-agent meta-data set command to create these tags.

For example, you can measure the binary size in a pipeline with this command:

steps:
  - commands:
    - go build -o dst/binary .
    - ls -l dst/binary | awk '{print \$5}' | tr -d '\n' | buildkite-agent meta-data set "dd_measures.binary_size"
    label: Go build

The resulting pipeline will have the tags shown below in the pipeline span:

• binary_size: 502 (output depends on the file size)

In this example, you can use the value of binary_size to plot the change in the binary size over time.

Correlate infrastructure metrics to jobs

If you are using Buildkite agents, you can correlate jobs with the infrastructure that is running them. For this feature to work, install the Datadog Agent in the hosts running the Buildkite agents.

View partial and downstream pipelines

You can use the following filters to customize your search query in the CI Visibility Explorer.

You can also apply these filters using the facet panel on the left hand side of the page.

Visualize pipeline data in Datadog

The CI Pipeline List and Executions pages populate with data after the pipelines finish.

The CI Pipeline List page shows data for only the default branch of each repository. For more information, see Search and Manage CI Pipelines.

Further reading

Additional helpful documentation, links, and articles:

Explore Pipeline Execution Results and PerformanceDOCUMENTATION Troubleshooting CI VisibilityDOCUMENTATION Extend Pipeline Visibility by adding custom tags and measuresDOCUMENTATION