Compatibility requirements
- The Datadog Java SDK supports library instrumentations using OpenTelemetry’s instrumentation API and
javaagent
extension API. - Each instrumentation must be packaged as an OpenTelemetry extension in its own JAR.
- OpenTelemetry provides an example extension project that registers a custom instrumentation for Servlet 3 classes.
- The Datadog SDK for Java also accepts select individual instrumentation JARs produced by OpenTelemetry’s opentelemetry-java-instrumentation build, for example the R2DBC instrumentation JAR.
OpenTelemetry incubator APIs are not supported.
Setup
To use an OpenTelemetry instrumentation with the Datadog Java SDK:
- Set the
dd.trace.otel.enabled
system property or the DD_TRACE_OTEL_ENABLED
environment variable to true
. - Copy the OpenTelemetry extension JAR containing the instrumentation to the same container as the application.
- Set the
otel.javaagent.extensions
system property or the OTEL_JAVAAGENT_EXTENSIONS
environment variable to the extension JAR path.
Example
Here’s a step-by-step example using R2DBC in Java to illustrate how you can add OpenTelemetry instrumentation into your service and begin sending data to Datadog, ensuring you capture all the missing spans.
git clone https://github.com/eugenp/tutorials
cd tutorials/spring-reactive-modules/spring-reactive-data
curl -Lo dd-java-agent.jar 'https://dtdg.co/latest-java-tracer'
Download the OpenTelemetry R2DBC agent and run your Spring Boot application with both the Datadog Java agent and the OpenTelemetry R2DBC agent.
curl -Lo opentelemetry-javaagent-r2dbc.jar \
'https://repo1.maven.org/maven2/io/opentelemetry/javaagent/instrumentation/opentelemetry-javaagent-r2dbc-1.0/2.5.0-alpha/opentelemetry-javaagent-r2dbc-1.0-2.5.0-alpha.jar'
mvn spring-boot:run -Dstart-class=com.baeldung.pagination.PaginationApplication \
-Dspring-boot.run.jvmArguments='-javaagent:dd-java-agent.jar -Ddd.trace.otel.enabled=true -Dotel.javaagent.extensions=opentelemetry-javaagent-r2dbc.jar -Ddd.trace.split-by-tags=db.name,db.sql.table -Ddd.trace.debug=true'
Open http://127.0.0.1:8080/products
to exercise the product query. With this setup, you are using OpenTelemetry’s instrumentation to ensure full observability for R2DBC queries.
Versions 2.6.0-alpha and later of these OpenTelemetry instrumentations are not supported by the Datadog Java SDK.
Verified OpenTelemetry extensions
Compatibility requirements
- The Datadog Python SDK supports library instrumentations using the OpenTelemetry Python Trace API.
- OpenTelemetry provides an example for instrumenting a sample application.
Setup
To use OpenTelemetry instrumentations with the Datadog Python SDK, perform the following steps:
- Follow the instructions in the OpenTelemetry API section in the Datadog Python library docs.
- Follow the steps for instrumenting your service with your chosen
opentelemetry-python-contrib
library.
Example
The following is an example instrumenting the OpenTelemetry’s kafka-python library with the Datadog Python SDK:
from kafka import KafkaProducer, KafkaConsumer
from opentelemetry.instrumentation.kafka import KafkaInstrumentor
from opentelemetry import trace
# Instrument Kafka with OpenTelemetry
KafkaInstrumentor().instrument()
# Kafka configuration
KAFKA_TOPIC = 'demo-topic0'
KAFKA_BROKER = 'localhost:9092'
def produce_message():
producer = KafkaProducer(bootstrap_servers=KAFKA_BROKER)
message = b'Hello, OpenTelemetry!'
# No manual span creation, relying on automatic instrumentation
producer.send(KAFKA_TOPIC, message)
producer.flush()
print(f"Produced message: {message}")
def consume_message():
consumer = KafkaConsumer(KAFKA_TOPIC, bootstrap_servers=KAFKA_BROKER, auto_offset_reset='earliest', group_id='demo-group')
# No manual span creation, relying on automatic instrumentation
for message in consumer:
print(f"Consumed message: {message.value}")
break # For simplicity, consume just one message
if __name__ == "__main__":
# manual span here
tracer = trace.get_tracer(__name__)
with tracer.start_as_current_span("Span") as parent_span:
parent_span.set_attribute("Hello", "World")
produce_message()
consume_message()
Compatibility requirements
The Datadog SDK for Go supports library instrumentations written using the Opentelemetry-Go Trace API, including the opentelemetry-go-contrib/instrumentation
libraries.
Setup
To use OpenTelemetry integrations with the Datadog Go SDK, perform the following steps:
- Follow the instructions in the Imports and Setup sections of the Go Custom Instrumentation using OpenTelemetry API page.
- Follow the steps for instrumenting your service with your chosen
opentelemetry-go-contrib
library.
Example
The following is an example instrumenting the net/http
library with the Datadog Tracer and Opentelemetry’s net/http
integration:
import (
"fmt"
"log"
"net/http"
ddotel "gopkg.in/DataDog/dd-trace-go.v1/ddtrace/opentelemetry"
ddtracer "gopkg.in/DataDog/dd-trace-go.v1/ddtrace/tracer"
"go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp"
"go.opentelemetry.io/otel"
)
func main() {
// register tracer
provider := ddotel.NewTracerProvider(ddtracer.WithDebugMode(true))
defer provider.Shutdown()
otel.SetTracerProvider(provider)
// configure the server with otelhttp instrumentation as you normally would using opentelemetry: https://pkg.go.dev/go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp
var mux http.ServeMux
mux.Handle("/hello", http.HandlerFunc(hello))
http.HandleFunc("/hello", hello)
log.Fatal(http.ListenAndServe(":8080", otelhttp.NewHandler(&mux, "server")))
}
func hello(w http.ResponseWriter, req *http.Request) {
fmt.Fprintf(w, "hello\n")
}
Compatibility requirements
The Datadog Node.js SDK supports library instrumentations using the OpenTelemetry Node.js Trace API.
Setup
To use OpenTelemetry instrumentations with the Datadog Node.js SDK, perform the following steps:
- Follow the Setup instructions in Node.js Custom Instrumentation using OpenTelemetry API.
- Follow the steps for instrumenting your service with your chosen
opentelemetry-js-contrib
library.
Example
The following example demonstrates how to instrument the http
and express
OpenTelemetry integrations with the Datadog Node.js SDK:
const tracer = require('dd-trace').init()
const { TracerProvider } = tracer
const provider = new TracerProvider()
provider.register()
const { registerInstrumentations } = require('@opentelemetry/instrumentation')
const { HttpInstrumentation } = require('@opentelemetry/instrumentation-http')
const { ExpressInstrumentation } = require('@opentelemetry/instrumentation-express')
// Register the instrumentation with the Datadog trace provider
// and the OpenTelemetry instrumentation of your choice
registerInstrumentations({
instrumentations: [
new HttpInstrumentation({
ignoreIncomingRequestHook (req) {
// Ignore spans created from requests to the agent
return req.path === '/v0.4/traces' || req.path === '/v0.7/config' ||
req.path === '/telemetry/proxy/api/v2/apmtelemetry'
},
ignoreOutgoingRequestHook (req) {
// Ignore spans created from requests to the agent
return req.path === '/v0.4/traces' || req.path === '/v0.7/config' ||
req.path === '/telemetry/proxy/api/v2/apmtelemetry'
}
}),
new ExpressInstrumentation()
],
tracerProvider: provider
})
const express = require('express')
const http = require('http')
// app code below ....
Configuration
To avoid duplicate spans, disable the corresponding Datadog instrumentations.
Set the DD_TRACE_DISABLED_INSTRUMENTATIONS
environment variable to a comma-separated list of integration names to disable. For example, to disable Datadog instrumentations for the libraries used in the Setup example, set the following:
DD_TRACE_DISABLED_INSTRUMENTATIONS=http,dns,express,net
Compatibility requirements
The Datadog PHP SDK supports library instrumentation using the stable
OpenTelemetry PHP Trace API. OpenTelemetry provides an example for instrumenting a sample PHP application.
Setup
To use OpenTelemetry integrations with the Datadog PHP SDK:
- Follow the instructions in configuring OpenTelemetry in the Datadog PHP SDK documentation.
- Follow the steps for instrumenting your service with your chosen
opentelemetry-php-contrib
library.
Example
You can find a sample PHP application with OpenTelemetry and Datadog auto instrumentations in the DataDog/trace-examples
GitHub repository.
Configuration
To avoid duplicate spans, you can disable the corresponding Datadog integrations. Set the DD_TRACE_<INTEGRATION>_ENABLED
environment variable to 0
or false
to disable an integration(see Integration names).
Use the integration name when setting integration-specific configuration for example: Laravel is DD_TRACE_LARAVEL_ENABLED
.
DD_TRACE_LARAVEL_ENABLED=false
Compatibility requirements
The Datadog .NET SDK supports library instrumentations that come with built-in OpenTelemetry support.
Setup
To use OpenTelemetry instrumentation libraries with the Datadog .NET SDK:
- Set the
DD_TRACE_OTEL_ENABLED
environment variable to true
. - Follow the steps to configure each library, if any, to generate OpenTelemetry-compatible instrumentation via
ActivitySource
Example
The following example demonstrates how to instrument the Hangfire
OpenTelemetry integrations with the Datadog .NET SDK:
using System;
using Hangfire;
using Hangfire.MemoryStorage;
using OpenTelemetry.Trace;
using OpenTelemetry.Resources;
using OpenTelemetry.Instrumentation.Hangfire;
using OpenTelemetry;
class Program
{
static void Main(string[] args)
{
// Create an OpenTelemetry TracerProvider to initialize the OpenTelemetry Hangfire instrumentation and build the configuration
var openTelemetry = Sdk.CreateTracerProviderBuilder()
.SetResourceBuilder(ResourceBuilder.CreateDefault().AddService("hangfire-demo2"))
.AddHangfireInstrumentation() // This line generates the OpenTelemetry spans
.Build();
// Configure Hangfire to use memory storage
GlobalConfiguration.Configuration.UseMemoryStorage();
// Create a new Hangfire server
using (var server = new BackgroundJobServer())
{
// Enqueue a background job
BackgroundJob.Enqueue(() => RunBackgroundJob());
Console.WriteLine("Hangfire Server started. Press any key to exit...");
Console.ReadKey();
}
// Dispose OpenTelemetry resources
openTelemetry?.Dispose();
}
// Define the background job method
public static void RunBackgroundJob()
{
Console.WriteLine("Hello from Hangfire!");
}
}
Verified OpenTelemetry Instrumentation Libraries
Library | Versions | NuGet package | Integration Name | Setup instructions |
---|
Azure Service Bus | 7.14.0+ | Azure.Messaging.ServiceBus | AzureServiceBus | See Azure SDK section below |
Azure SDK
The Azure SDK provides built-in OpenTelemetry support. Enable it by setting the AZURE_EXPERIMENTAL_ENABLE_ACTIVITY_SOURCE
environment variable to true
or by setting the Azure.Experimental.EnableActivitySource
context switch to true
in your application code. See Azure SDK documentation for more details.
Compatibility requirements
The Datadog Ruby SDK supports library instrumentation using the OpenTelemetry Ruby Trace API.
OpenTelemetry provides an example for instrumenting a sample application.
Setup
To use OpenTelemetry integrations with the Datadog Ruby SDK, perform the following steps:
- Follow the instructions in configuring OpenTelemetry in the Datadog Ruby SDK documentation.
- Follow the steps for instrumenting your service with your chosen
opentelemetry-ruby-contrib
library.