Correlate Infrastructure Metrics with GitLab Jobs in Datadog
Note: This method only applies to runners using the "Instance" or "Docker Autoscaler" executors.
Overview
When you click on a GitLab job in the CI Visibility Explorer, you can access an Infrastructure tab with information about the host, system, host tags, host metrics, and more.
This guide explains how to correlate infrastructure metrics with your GitLab jobs if you are using GitLab “Instance” or “Docker Autoscaler” executors and CI Visibility.
Prerequisites
The Datadog Agent must be installed in the virtual machines (VM) where the GitLab jobs will be run. This is not where the GitLab instance or the Docker Autoscaler executor is running, but in the VMs that are created with the fleeting plugin.
Ensure that the Datadog Agent is installed in your instances
If you are using an AWS Autoscaling Group, you should make sure that the machine image that is configured in the template launches with the Datadog Agent.
To test that you have performed this step successfully, you can try executing a job and you should see the host appear on the Infrastructure List page.
If you are using AWS, make sure that the host name is in the format “i-xxxxx”. If this is not the case, you should check that your instance is compatible with IMDSv1. For more information, see the official AWS documentation.
You can set this up inside the template of your AWS Autoscaling Group. The Datadog Agent uses the metadata service endpoint to resolve the host name.
Set up CI Visibility and log collection for your GitLab jobs
For instructions on setting up CI Visibility for your GitLab jobs, see Set up Pipeline Visibility on a GitLab Pipeline.
To test that you have performed the setup successfully, you can try running a GitLab pipeline and checking if it appears on the Executions page.
You are required to enable the collection of job logs. You can check if Datadog is receiving the logs correctly by clicking on the Logs tab of your pipeline execution.
After you have completed these steps, your GitLab jobs should be correlated with infrastructure metrics. The correlation is per job and not pipeline, as different jobs may run on different hosts. The Infrastructure tab appears after the job is finished and Datadog receives the logs for that job.
Further reading
Additional helpful documentation, links, and articles:
