Data Streams Monitoring for Kafka

Docs > Data Streams Monitoring > Setup Data Streams Monitoring > Data Streams Monitoring for Kafka

Prerequisites

Datadog Agent v7.34.0 or later

Language	Library	Minimal tracer version	Recommended tracer version
Java	kafka-clients (Lag generation is not supported for v3.7)	1.9.0	1.43.0
Go	confluent-kafka-go	1.56.1	1.66.0 or later
Go	Sarama	1.56.1	1.66.0 or later
Node.js	kafkajs	2.39.0 or 3.26.0 or 4.5.0	5.25.0 or later
Python	confluent-kafka	1.16.0	2.11.0 or later
.NET	Confluent.Kafka	2.28.0	2.41.0 or later

Kafka Streams is partially supported for Java, and can lead to latency measurements being missed.

Supported Kafka deployments

Instrumenting your consumers and producers with Data Streams Monitoring allows you to view your topology and track your pipelines with ready-to-go metrics independently of how Kafka is deployed. Additionally, the following Kafka deployments have further integration support, providing more insights into the health of your Kafka cluster:

Model	Integration
Self Hosted	Kafka Broker & Kafka Consumer
Confluent Platform	Confluent Platform
Confluent Cloud	Confluent Cloud
Amazon MSK	Amazon MSK or Amazon MSK (Agent)
Red Panda	Not yet integrated

Setting up Data Streams Monitoring

See setup instructions for Java, Go, Node.js, Python, or .NET.

Monitoring Kafka Pipelines

Data Streams Monitoring uses message headers to propagate context through Kafka streams. If log.message.format.version is set in the Kafka broker configuration, it must be set to 0.11.0.0 or higher. Data Streams Monitoring is not supported for versions lower than this.