For Agent v7.21+ / v6.21+, follow the instructions below to install the NVML check on your host. See Use Community Integrations to install with the Docker Agent or earlier versions of the Agent.
Run the following command to install the Agent integration:
For Linux:
datadog-agent integration install -t datadog-nvml==<INTEGRATION_VERSION>
# You may also need to install dependencies since those aren't packaged into the wheelsudo -u dd-agent -H /opt/datadog-agent/embedded/bin/pip3 install grpcio pynvml
For Windows (Using Powershell run as admin):
&"$env:ProgramFiles\Datadog\Datadog Agent\bin\agent.exe" integration install -t datadog-nvml==<INTEGRATION_VERSION>
# You may also need to install dependencies since those aren't packaged into the wheel&"$env:ProgramFiles\Datadog\Datadog Agent\embedded3\python" -m pip install grpcio pynvml
Configure your integration similar to core integrations.
If you are using Docker, there is an example Dockerfile in the NVML repository.
docker build -t dd-agent-nvml .
If you’re using Docker and Kubernetes, you need to expose the environment variables NVIDIA_VISIBLE_DEVICES and NVIDIA_DRIVER_CAPABILITIES. See the included Dockerfile for an example.
To correlate reserved Kubernetes NVIDIA devices with the Kubernetes pod using the device, mount the Unix domain socket /var/lib/kubelet/pod-resources/kubelet.sock into your Agent’s configuration. More information about this socket is on the Kubernetes website. Note: This device is in beta support for version 1.15.
Edit the nvml.d/conf.yaml file, in the conf.d/ folder at the root of your Agent’s configuration directory to start collecting your NVML performance data. See the sample nvml.d/conf.yaml for all available configuration options.