Overview

The Datadog Admission Controller is a component of the Datadog Cluster Agent. The main benefit of the Admission Controller is to simplify your application Pod configuration. For that, it has two main functionalities:

  • Inject environment variables (DD_AGENT_HOST, DD_TRACE_AGENT_URL and DD_ENTITY_ID) to configure DogStatsD and APM tracer libraries into the user’s application containers.
  • Inject Datadog standard tags (env, service, version) from application labels into the container environment variables.

Datadog’s Admission Controller is MutatingAdmissionWebhook type. For more details on admission controllers, see the Kubernetes guide on admission controllers.

Requirements

  • Datadog Cluster Agent v7.40+

Configuration

To enable the Admission Controller for the Datadog Operator, set the parameter features.admissionController.enabled to true in your DatadogAgent configuration:

datadog-agent.yaml

apiVersion: datadoghq.com/v2alpha1
kind: DatadogAgent
metadata:
  name: datadog
spec:
  #(...)
  features:
    admissionController:
      enabled: true
      mutateUnlabelled: false

Starting from Helm chart v2.35.0, Datadog Admission controller is activated by default. No extra configuration is needed to enable the Admission Controller.

To enable the Admission Controller for Helm chart v2.34.6 and earlier, set the parameter clusterAgent.admissionController.enabled to true:

datadog-values.yaml

#(...)
clusterAgent:
  #(...)
  ## @param admissionController - object - required
  ## Enable the admissionController to automatically inject APM and
  ## DogStatsD config and standard tags (env, service, version) into
  ## your pods
  #
  admissionController:
    enabled: true

    ## @param mutateUnlabelled - boolean - optional
    ## Enable injecting config without having the pod label:
    ## admission.datadoghq.com/enabled="true"
    #
    mutateUnlabelled: false

To enable the Admission Controller without using Helm or the Datadog operator, add the following to your configuration:

First, download the Cluster Agent RBAC permissions manifest, and add the following under rules:

cluster-agent-rbac.yaml

- apiGroups:
  - admissionregistration.k8s.io
  resources:
  - mutatingwebhookconfigurations
  verbs: ["get", "list", "watch", "update", "create"]
- apiGroups: [""]
  resources: ["secrets"]
  verbs: ["get", "list", "watch", "update", "create"]
- apiGroups: ["batch"]
  resources: ["jobs", "cronjobs"]
  verbs: ["get"]
- apiGroups: ["apps"]
  resources: ["statefulsets", "replicasets", "deployments"]
  verbs: ["get"]

Add the following to the bottom of agent-services.yaml:

agent-services.yaml

apiVersion: v1
kind: Service
metadata:
  name: datadog-cluster-agent-admission-controller
  labels:
    app: "datadog"
    app.kubernetes.io/name: "datadog"
spec:
  selector:
    app: datadog-cluster-agent
  ports:
  - port: 443
    targetPort: 8000

Add environment variables to the Cluster Agent deployment which enable the Admission Controller:

cluster-agent-deployment.yaml

- name: DD_ADMISSION_CONTROLLER_ENABLED
  value: "true"
- name: DD_ADMISSION_CONTROLLER_SERVICE_NAME
  value: "datadog-cluster-agent-admission-controller"

# Uncomment this to configure APM tracers automatically (see below)
# - name: DD_ADMISSION_CONTROLLER_MUTATE_UNLABELLED
#   value: "true"

Finally, run the following commands:

  • kubectl apply -f cluster-agent-rbac.yaml
  • kubectl apply -f agent-services.yaml
  • kubectl apply -f cluster-agent-deployment.yaml

Instrumentation library injection

You can configure the Cluster Agent (version 7.39 and higher) to inject instrumentation libraries. Read Instrumentation library injection with Admission Controller for more information.

APM and DogStatsD

To configure DogStatsD clients or other APM libraries that do not support library injection, inject the environment variables DD_AGENT_HOST and DD_ENTITY_ID by doing one of the following:

  • Add the label admission.datadoghq.com/enabled: "true" to your Pod.
  • Configure the Cluster Agent admission controller by setting mutateUnlabelled (or DD_ADMISSION_CONTROLLER_MUTATE_UNLABELLED, depending on your configuration method) to true.

Adding a mutateUnlabelled: true Agent config in the Helm chart causes the Cluster Agent to attempt to intercept every unlabelled Pod.

To prevent Pods from receiving environment variables, add the label admission.datadoghq.com/enabled: "false". This works even if you set mutateUnlabelled: true.

If mutateUnlabelled is set to false, the Pod label must be set to admission.datadoghq.com/enabled: "true".

Possible options:

mutateUnlabelledPod labelInjection
trueNo labelYes
trueadmission.datadoghq.com/enabled=trueYes
trueadmission.datadoghq.com/enabled=falseNo
falseNo labelNo
falseadmission.datadoghq.com/enabled=trueYes
falseadmission.datadoghq.com/enabled=falseNo

Order of priority

The Datadog Admission Controller does not inject the environment variables DD_VERSION, DD_ENV, or DD_SERVICE if they already exist.

When these environment variables are not set, the Admission Controller uses standard tags value in the following order (highest first):

  • Labels on the Pod
  • Labels on the ownerReference (ReplicaSets, DaemonSets, Deployments, etc.)

Configure APM and DogstatsD communication mode

Starting from Datadog Cluster Agent v1.20.0, the Datadog Admission Controller can be configured to inject different modes of communication between the application and Datadog agent.

This feature can be configured by setting admission_controller.inject_config.mode or by defining a Pod-specific mode using the admission.datadoghq.com/config.mode Pod label.

Starting from Helm chart v3.22.0 and Datadog Operator v1.1.0, the communication mode is automatically set to socket if either APM socket or DSD socket is enabled.

Possible options:

ModeDescription
hostip (Default)Inject the host IP in DD_AGENT_HOST environment variable
serviceInject Datadog’s local-service DNS name in DD_AGENT_HOST environment variable (available with Kubernetes v1.22+)
socketInject Unix Domain Socket path in DD_TRACE_AGENT_URL environment variable and the volume definition to access the corresponding path. Inject URL to use to connect the Datadog Agent for DogStatsD metrics in DD_DOGSTATSD_URL.

Note: Pod-specific mode takes precedence over the global mode defined at the Admission Controller level.

Troubleshooting

See Admission Controller Troubleshooting.

Further Reading

PREVIEWING: mervebolat/span-id-preprocessing