Instrumentation is the process of adding code to your application to capture and report observability data. Automatic instrumentation is a way to instrument applications and libraries without modifying their source code. Both OpenTelemetry and Datadog provide automatic instrumentation in their SDKs.

Datadog SDKs support adding OpenTelemetry instrumentation libraries to their existing automatic instrumentation. This provides observability for libraries not covered by Datadog SDKs without changing SDKs.

Prerequisites

  1. Enable OpenTelemetry support: Set the DD_TRACE_OTEL_ENABLED environment variable to true. This step isn’t required for the Datadog Go and Ruby APM SDKs.

  2. Run the Datadog Agent: Datadog SDKs provide an implementation of the OpenTelemetry API and submit spans to a Datadog Agent. Ensure the Datadog Agent is running to use OpenTelemetry instrumentation with Datadog SDKs.

  3. Disable duplicate instrumentation: When replacing a Datadog instrumentation with its OpenTelemetry equivalent, disable the Datadog instrumentation to prevent duplicate spans from appearing in the trace.

Configuration

Datadog SDKs support configuration through OpenTelemetry environment variables.

Language support

Datadog SDKs implement the OpenTelemetry API by overriding the default implementations in the OpenTelemetry SDK. However, note the following limitations:

  • Operations only supported by the OpenTelemetry SDK are not supported (for example, SpanProcessors or OTLP Trace Exporters).
  • Datadog SDKs do not support OpenTelemetry Metrics and Logs APIs. To use OpenTelemetry Logs and Metrics APIs, use OTLP Ingest.
LanguageMinimum version
Java1.35.0
Python2.10.0
Ruby2.1.0
Go1.65.0
Node.js4.3.0
PHP0.94.0
.NET2.53.0

Compatibility requirements

The Datadog Java SDK supports library instrumentations using OpenTelemetry’s instrumentation API and javaagent extension API.

Each instrumentation must be packaged as an OpenTelemetry extension in its own JAR.

OpenTelemetry provides an example extension project that registers a custom instrumentation for Servlet 3 classes.

The Datadog SDK for Java also accepts select individual instrumentation JARs produced by OpenTelemetry’s opentelemetry-java-instrumentation build, for example the CFX instrumentation JAR.

OpenTelemetry incubator APIs are not supported.

Setup

To use an OpenTelemetry instrumentation with the Datadog Java SDK:

  1. Set the dd.trace.otel.enabled system property or the DD_TRACE_OTEL_ENABLED environment variable to true.
  2. Copy the OpenTelemetry extension JAR containing the instrumentation to the same container as the application.
  3. Set the otel.javaagent.extensions system property or the OTEL_JAVAAGENT_EXTENSIONS environment variable to the extension JAR path.

Verified OpenTelemetry extensions

FrameworkVersionsOpenTelemetry ExtensionInstrumentation Names
Apache CXF (Jax-WS)3.0+opentelemetry-javaagent-jaxws-2.0-cxf-3.0cxf

Compatibility requirements

The Datadog Python SDK supports library instrumentations using the OpenTelemetry Python Trace API.

OpenTelemetry provides an example for instrumenting a sample application.

Setup

To use OpenTelemetry instrumentations with the Datadog Python SDK, perform the following steps:

  1. Follow the instructions in the OpenTelemetry API section in the Datadog Python library docs.
  2. Follow the steps for instrumenting your service with your chosen opentelemetry-python-contrib library.

Compatibility requirements

The Datadog Ruby SDK supports library instrumentation using the OpenTelemetry Ruby Trace API.

OpenTelemetry provides an example for instrumenting a sample application.

Setup

To use OpenTelemetry integrations with the Datadog Ruby SDK, perform the following steps:

  1. Follow the instructions in configuring OpenTelemetry in the Datadog Ruby SDK documentation.
  2. Follow the steps for instrumenting your service with your chosen opentelemetry-ruby-contrib library.

Compatibility requirements

The Datadog SDK for Go supports library instrumentations written using the Opentelemetry-Go Trace API, including the opentelemetry-go-contrib/instrumentation libraries.

Setup

To use OpenTelemetry integrations with the Datadog Go SDK, perform the following steps:

  1. Follow the instructions in the Imports and Setup sections of the Go Custom Instrumentation using OpenTelemetry API page.
  2. Follow the steps for instrumenting your service with your chosen opentelemetry-go-contrib library.

The following is an example instrumenting the net/http library with the Datadog Tracer and Opentelemetry’s net/http integration:

import (
	"fmt"
	"log"
	"net/http"

	ddotel "gopkg.in/DataDog/dd-trace-go.v1/ddtrace/opentelemetry"
	ddtracer "gopkg.in/DataDog/dd-trace-go.v1/ddtrace/tracer"

	"go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp"
	"go.opentelemetry.io/otel"
)

func main() {
	// register tracer
	provider := ddotel.NewTracerProvider(ddtracer.WithDebugMode(true))
	defer provider.Shutdown()
	otel.SetTracerProvider(provider)

	// configure the server with otelhttp instrumentation as you normally would using opentelemetry: https://pkg.go.dev/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp
	var mux http.ServeMux
	mux.Handle("/hello", http.HandlerFunc(hello))
	http.HandleFunc("/hello", hello)
	log.Fatal(http.ListenAndServe(":8080", otelhttp.NewHandler(&mux, "server")))
}

func hello(w http.ResponseWriter, req *http.Request) {
	fmt.Fprintf(w, "hello\n")
}
go-dd-otelhttp

Further reading

PREVIEWING: stefon.simmons/add-is_loggedin-shortcode