Prerequisites

To start with Data Streams Monitoring, you need recent versions of the Datadog Agent and Node.js libraries:

• Datadog Agent v7.34.0 or later
• Node.js Tracer
  • Kafka: v2.39.0, v3.26.0, v4.5.0, or later
  • Amazon SQS: v4.21.0 or later
  • RabbitMQ: v3.48.0, v4.27.0, v5.3.0 or later

Installation

Node.js uses auto-instrumentation to inject and extract additional metadata required by Data Streams Monitoring for measuring end-to-end latencies and the relationship between queues and services. To enable Data Streams Monitoring, set the DD_DATA_STREAMS_ENABLED environment variable to true on services sending messages to (or consuming messages from) Kafka.

For example:

environment:
  - DD_DATA_STREAMS_ENABLED: "true"

Supported libraries

Data Streams Monitoring supports the confluent-kafka library, amqplib package, and rhea package.

Monitoring SQS pipelines

Data Streams Monitoring uses one message attribute to track a message’s path through an SQS queue. As Amazon SQS has a maximum limit of 10 message attributes allowed per message, all messages streamed through the data pipelines must have 9 or fewer message attributes set, allowing the remaining attribute for Data Streams Monitoring.

Further Reading

추가 유용한 문서, 링크 및 기사:

Kafka IntegrationDOCUMENTATION Service CatalogDOCUMENTATION