Event collection is enabled by default by the Datadog Operator. This can be managed in the configuration features.eventCollection.collectKubernetesEvents in your datadog-agent.yaml.
To collect Kubernetes events with the Datadog Cluster Agent, ensure that the clusterAgent.enabled, datadog.collectEvents and clusterAgent.rbac.create options are set to true in your datadog-values.yaml file.
If you don’t want to use the Cluster Agent, you can still have a Node Agent collect Kubernetes events by setting datadog.leaderElection, datadog.collectEvents, and agents.rbac.create options to true in your datadog-values.yaml file.
The Datadog Cluster Agent provides a streamlined, centralized approach to collecting cluster level monitoring data. Datadog strongly recommends using the Cluster Agent for monitoring Kubernetes.
The Datadog Operator v1.0.0+ and Helm chart v2.7.0+ enable the Cluster Agent by default. No further configuration is necessary.
The Datadog Operator v1.0.0+ enables the Cluster Agent by default. The Operator creates the necessary RBACs and deploys the Cluster Agent. Both Agents use the same API key.
The Operator automatically generates a random token in a Kubernetes Secret to be shared by the Datadog Agent and Cluster Agent for secure communication.
You can manually specify this token in the global.clusterAgentToken field in your datadog-agent.yaml:
To make use of Datadog’s Container Explorer, enable the Process Agent. The Datadog Operator and Helm chart enable the Process Agent by default. No further configuration is necessary.
In some setups, the Process Agent and Cluster Agent cannot automatically detect a Kubernetes cluster name. If this happens, the feature does not start, and the following warning displays in the Cluster Agent log: Orchestrator explorer enabled but no cluster name set: disabling. In this case, you must set spec.global.clusterName to your cluster name in datadog-agent.yaml:
The Helm chart enables the Process Agent by default.
For verification, ensure that processAgent.enabled is set to true in your datadog-values.yaml:
datadog:# (...)processAgent:enabled:true
In some setups, the Process Agent and Cluster Agent cannot automatically detect a Kubernetes cluster name. If this happens, the feature does not start, and the following warning displays in the Cluster Agent log: Orchestrator explorer enabled but no cluster name set: disabling. In this case, you must set datadog.clusterName to your cluster name in datadog-values.yaml.
In some setups, the Process Agent and Cluster Agent cannot automatically detect a Kubernetes cluster name. If this happens, the feature does not start, and the following warning displays in the Cluster Agent log: Orchestrator explorer enabled but no cluster name set: disabling. In this case, you must set spec.global.clusterName to your cluster name in datadog-agent.yaml:
In some setups, the Process Agent and Cluster Agent cannot automatically detect a Kubernetes cluster name. If this happens, the feature does not start, and the following warning displays in the Cluster Agent log: Orchestrator explorer enabled but no cluster name set: disabling. In this case, you must set datadog.clusterName to your cluster name in values.yaml.
Instead of global.credentials.apiKey, supply the name of a Kubernetes Secret containing your Datadog API key.
global.credentials.apiSecret.keyName
Instead of global.credentials.apiKey, supply the key of the Kubernetes Secret named in global.credentials.apiSecret.secretName.
global.credentials.appKey
Configures your Datadog application key. If you are using the external metrics server, you must set a Datadog application key for read access to your metrics.
global.credentials.appSecret.secretName
Instead of global.credentials.apiKey, supply the name of a Kubernetes Secret containing your Datadog app key.
global.credentials.appSecret.keyName
Instead of global.credentials.apiKey, supply the key of the Kubernetes Secret named in global.credentials.appSecret.secretName.
global.logLevel
Sets logging verbosity. This can be overridden by the container. Valid log levels are: trace, debug, info, warn, error, critical, and off. Default: info.
global.registry
Image registry to use for all Agent images. Default: gcr.io/datadoghq.
global.site
Sets the Datadog intake site to which Agent data is sent. Your site is datadoghq.com. (Ensure the correct SITE is selected on the right).
global.tags
A list of tags to attach to every metric, event, and service check collected.
For a complete list of configuration fields for the Datadog Operator, see the Operator v2alpha1 spec. For older versions, see the Operator v1alpha1 spec. Configuration fields can also be queried using kubectl explain datadogagent --recursive.
Helm
Description
datadog.apiKey
Configures your Datadog API key.
datadog.apiKeyExistingSecret
Instead of datadog.apiKey, supply the name of an existing Kubernetes Secret containing your Datadog API key, set with the key name api-key.
datadog.appKey
Configures your Datadog application key. If you are using the external metrics server, you must set a Datadog application key for read access to your metrics.
datadog.appKeyExistingSecret
Instead of datadog.appKey, supply the name of an existing Kubernetes Secret containing your Datadog app key, set with the key name app-key.
datadog.logLevel
Sets logging verbosity. This can be overridden by the container. Valid log levels are: trace, debug, info, warn, error, critical, and off. Default: info.
registry
Image registry to use for all Agent images. Default: gcr.io/datadoghq.
datadog.site
Sets the Datadog intake site to which Agent data is sent. Your site is datadoghq.com. (Ensure the correct SITE is selected on the right).
datadog.tags
A list of tags to attach to every metric, event, and service check collected.
For a complete list of environment variables for the Helm chart, see the full list of options for datadog-values.yaml.
Env Variable
Description
DD_API_KEY
Your Datadog API key (required)
DD_ENV
Sets the global env tag for all data emitted.
DD_HOSTNAME
Hostname to use for metrics (if autodetection fails)
DD_TAGS
Host tags separated by spaces. For example: simple-tag-0 tag-key-1:tag-value-1
DD_SITE
Destination site for your metrics, traces, and logs. Your DD_SITE is datadoghq.com. Defaults to datadoghq.com.
DD_DD_URL
Optional setting to override the URL for metric submission.
DD_URL (6.36+/7.36+)
Alias for DD_DD_URL. Ignored if DD_DD_URL is already set.
DD_CHECK_RUNNERS
The Agent runs all checks concurrently by default (default value = 4 runners). To run the checks sequentially, set the value to 1. If you need to run a high number of checks (or slow checks) the collector-queue component might fall behind and fail the healthcheck. You can increase the number of runners to run checks in parallel.
DD_LEADER_ELECTION
If multiple instances of the Agent are running in your cluster, set this variable to true to avoid the duplication of event collection.
The containerized Datadog Agent can be configured by using environment variables. For an extensive list of supported environment variables, see the Environment variables section of the Docker Agent documentation.
When using the Datadog Operator, you can set additional environment variables in override for a component with [key].env []object, or for a container with [key].containers.[key].env []object. The following keys are supported:
nodeAgent
clusterAgent
clusterChecksRunner
Container-level settings take priority over any component-level settings.
DogStatsD can send custom metrics over UDP with the StatsD protocol. DogStatsD is enabled by default by the Datadog Operator and Helm. See the DogStatsD documentation for more information.
You can use the following environment variables to configure DogStatsD with DaemonSet:
Env Variable
Description
DD_DOGSTATSD_NON_LOCAL_TRAFFIC
Listen to DogStatsD packets from other containers (required to send custom metrics).
DD_HISTOGRAM_PERCENTILES
The histogram percentiles to compute (separated by spaces). The default is 0.95.
DD_HISTOGRAM_AGGREGATES
The histogram aggregates to compute (separated by spaces). The default is "max median avg count".
DD_DOGSTATSD_SOCKET
Path to the Unix socket to listen to. Must be in a rw mounted volume.
DD_DOGSTATSD_ORIGIN_DETECTION
Enable container detection and tagging for Unix socket metrics.
DD_DOGSTATSD_TAGS
Additional tags to append to all metrics, events, and service checks received by this DogStatsD server, for example: "env:golden group:retrievers".
Datadog automatically collects common tags from Kubernetes.
In addition, you can map Kubernetes node labels, pod labels, and annotations to Datadog tags. Use the following environment variables to configure this mapping:
Integration credentials can be stored in Docker or Kubernetes secrets and used in Autodiscovery templates. For more information, see Secrets Management.
Exclude containers from logs collection, metrics collection, and Autodiscovery. Datadog excludes Kubernetes and OpenShift pause containers by default. These allowlists and blocklists apply to Autodiscovery only; traces and DogStatsD are not affected. These environment variables support regular expressions in their values.
Note: The kubernetes.containers.running, kubernetes.pods.running, docker.containers.running, .stopped, .running.total and .stopped.total metrics are not affected by these settings. All containers are counted.
By default, the Kubernetes State Metrics Core check waits 10 seconds for a response from the Kubernetes API server. For large clusters, the request may time out, resulting in missing metrics.
You can avoid this by setting the environment variable DD_KUBERNETES_APISERVER_CLIENT_TIMEOUT to a higher value than the default 10 seconds.
Some capabilities require that you set a Kubernetes cluster name. A valid cluster name must be unique and dot-separated, with the following restrictions:
Can contain only lowercase letters, numbers, and hyphens
Must start with a letter
Overall length is less than or equal to 80 characters
Additional Autodiscovery listeners to run. They are added in addition to the variables defined in the listeners section of the datadog.yaml configuration file.
DD_CONFIG_PROVIDERS
The providers the Agent should call to collect checks configurations. Available providers are: kubelet - Handles templates embedded in pod annotations. docker - Handles templates embedded in container labels. clusterchecks - Retrieves cluster-level check configurations from the Cluster Agent. kube_services - Watches Kubernetes services for cluster checks.
DD_EXTRA_CONFIG_PROVIDERS
Additional Autodiscovery configuration providers to use. They are added in addition to the variables defined in the config_providers section of the datadog.yaml configuration file.
Overrides container source auto-detection to force a single source. e.g "docker", "ecs_fargate", "kubelet". This is no longer needed since Agent v7.35.0.
DD_HEALTH_PORT
Set this to 5555 to expose the Agent health check at port 5555.
DD_CLUSTER_NAME
Set a custom Kubernetes cluster identifier to avoid host alias collisions. The cluster name can be up to 40 characters with the following restrictions: Lowercase letters, numbers, and hyphens only. Must start with a letter. Must end with a number or a letter.
DD_COLLECT_KUBERNETES_EVENTS
Enable event collection with the Agent. If you are running multiple instances of the Agent in your cluster, set DD_LEADER_ELECTION to true as well.