Generate metrics from 100% of ingested spans, regardless of whether the spans are indexed by a retention filter.

Use custom metrics for specific fixed queries and comparisons, while creating retention filters to allow arbitrary querying and investigation of the retained trace and its flame graph.

Billing Note: Metrics created from ingested spans are billed as Custom Metrics.

For example, you may want to use custom metrics to visualize anomalies, create dashboards and monitors, and see trends across any parameters that are important to your business context. All generated metrics are available for 15 months as Datadog custom metrics.

To generate metrics from spans, on the APM Setup and Configuration page select the Generate Metrics tab, and click the New Metric button.

Creating a span-based metric

1. Define the metric query: Start by adding a query for filtering to your required dataset. The query syntax is the same as APM Search and Analytics.

2. Define the field you want to track: Select * to generate a count of all spans matching your query or enter an attribute (for example, @cassandra_row_count) to aggregate a numeric value and create its corresponding count, minimum, maximum, sum, and average aggregated metrics. If the attribute type is a measure, the value of the metric is the value of the span attribute.

   Note: Span attributes that are not numerical values cannot be used for aggregation. To generate a metric that counts the distinct values of a span attribute (for instance count the number of user IDs hitting a specific endpoint), add this dimension to the group by selector, and use the count_nonzero function to count the number of tag values.

3. Specify the group-by dimension: By default, metrics generated from spans will not have any tags unless explicitly added. Any attribute or tag that exists in your spans can be used to create metric tags.

4. Check the Live Analytics and Search Query preview: You can view the impact of your query in real-time on the data visualization, and the matching spans considered for your query in a live preview.

5. Name your metric: Metric names must follow the metric naming convention. Metric names that start with trace.* are not permitted and will not be saved.

Updating existing span-based metrics

After a metric is created, only two fields can be updated:

Note: To change the metric type or name, create a new metric and delete the old one.

Further Reading

Additional helpful documentation, links, and articles:

Customize trace ingestion and retain important traces.DOCUMENTATION Use Analytics queries and monitors based on retained traces.DOCUMENTATION Create and manage span-based metrics with TerraformEXTERNAL SITE