Overview

AWS Payload Extraction captures request and response data exchanged between your application and AWS services. This feature attaches the extracted information as tags to your traces, enabling you to view the data in dashboards and use it for alerting.

Requirements

The following AWS services are supported:

• Amazon Simple Notification Service (SNS)
• Amazon Simple Queue Service (SQS)
• Amazon Kinesis
• Amazon S3
• Amazon EventBridge

The following tracer versions and AWS SDK packages are supported:

How it works

AWS Payload Extraction extracts key-value pairs from hierarchical request and response bodies, converting them into dot-separated tags. For example:

Input JSON:

{  
  "Message": {  
    "foo.bar": "baz",  
    "Arr": ["a", "b"]  
  }  
}

Generated tags:

aws.request.body.Message.foo\.bar: baz  
aws.request.body.Message.Arr.0: a  
aws.request.body.Message.Arr.1: b

The tracers are configured to match JSON data nested inside JSON documents, which is a common pattern with SQS payloads.

General configuration

Enable AWS Payload Extraction

To enable AWS Payload Extraction, set these environment variables:

# Parse requests
DD_TRACE_CLOUD_REQUEST_PAYLOAD_TAGGING=all

# Parse responses  
DD_TRACE_CLOUD_RESPONSE_PAYLOAD_TAGGING=all

You can choose to parse:

• Only request bodies
• Only response bodies
• Both request and response bodies

The value all indicates that the entire body is used to generate tags. See Protect sensitive information for more configuration options.

Protect sensitive information

It’s expected that many of these payloads contain personally identifiable information (PII).

To protect sensitive information, the tracers replace common PII fields with 'redacted' (such as phone numbers in SNS). Note: You can’t disable the default redactions.

You can specify additional fields to redact using JSONPath syntax in the environment variables. For example:

DD_TRACE_CLOUD_REQUEST_PAYLOAD_TAGGING=$.Metadata.UserId,$.Attributes.0.Pass

This example:

• Redacts the UserId field within the Metadata object
• Redacts the Pass field in the first element of the Attributes array
• Applies default redactions
• Processes request bodies only

Control payload extraction depth

Control the maximum depth of payload extraction with:

DD_TRACE_CLOUD_PAYLOAD_TAGGING_MAX_DEPTH=10

The default value is 10. Nodes beyond this depth are ignored during tag generation. The main reason to modify this value is to adjust performance.

Disable AWS Payload Extraction

Setting these variables to an empty string or omitting them disables the feature:

DD_TRACE_CLOUD_REQUEST_PAYLOAD_TAGGING=""
DD_TRACE_CLOUD_RESPONSE_PAYLOAD_TAGGING=""

Language-specific configuration

Each tracer implementation provides additional configuration options specific to that language.

# Default values
DD_TRACE_CLOUD_PAYLOAD_TAGGING_SERVICES=s3,sns,sqs,kinesis,eventbridge

DD_TRACE_CLOUD_PAYLOAD_TAGGING_SERVICES=s3,sns,sqs,kinesis,eventbridge,amplify

# Default value
DD_TRACE_CLOUD_PAYLOAD_TAGGING_MAX_TAGS=758

# Default values
DD_TRACE_CLOUD_PAYLOAD_TAGGING_SERVICES=ApiGateway,ApiGatewayV2,EventBridge,Sqs,Sns,S3,Kinesis

DD_TRACE_CLOUD_PAYLOAD_TAGGING_MAX_TAGS=758

Best practices

• Different tracers use different JSONPath implementations, so test your queries with each tracer individually.
• Always verify redaction behavior in a Staging environment before enabling in Production.

Further reading

Additional helpful documentation, links, and articles:

Learn about Datadog APM tracingDOCUMENTATION APM Terminology and OverviewDOCUMENTATION