Overview
Metrics without Limits™ provides you flexibility and control over your custom metrics volumes by decoupling custom metric ingestion and indexing. You only pay for custom metric tags that are valuable to your organization.
Metrics without Limits™ provides you with the ability to configure tags on all metric types in-app. You can also customize aggregations on counts, rates, and gauges without having to re-deploy or change any code. With Metrics without Limits™, you can configure an allowlist of tags in-app to remain queryable throughout the Datadog platform; this automatically drops nonessential tags attached to application-level or business metrics (for example, host). Alternatively, you can configure a blocklist of tags in-app to quickly drop and exclude tags; this automatically retains remaining essential tags that provide business value to your teams. These configuration functionalities are located in the Metrics Summary page.
This page identifies key components of Metrics without Limits™ that can help you manage your custom metrics volumes within your observability budget.
- Click any metric name to open its details side panel.
- Click Manage Tags -> “Include Tags…” to configure the tags you’d like to remain as queryable in dashboards, notebooks, monitors, and other Datadog products.
- Define your allowlist of tags.
By default, the tag configuration modal pre-populates with a Datadog recommended allowlist of tags that have been actively queried on dashboards, notebooks, monitors, or through the API in the past 30 days. Recommended tags are distinguished with a graph line icon.
- Review the Estimated New Volume of indexed custom metrics that results from this potential tag configuration.
- Click Save.
You can create, edit, delete, and estimate the impact of your tag configuration through the Metrics APIs.
- Click on any metric name to open its details side panel.
- Click Manage Tags -> “Exclude Tags…” to drop tags you don’t want to query.
- Define your blocklist of tags. Tags defined in the blocklist are not queryable on dashboards and monitors. Tags that have been actively queried on dashboards, notebooks, monitors, and through the API in the past 30 days are distinguished with a graph line icon.
- Review the Estimated New Volume of indexed custom metrics that results from this potential tag configuration.
- Click Save.
Set the parameter exclude_tags_mode: true on the Metrics API to create and edit a blocklist of tags.
When configuring tags for counts, rates, and gauges, the most frequently queried time/space aggregation combination is available for query by default.
Optimize your custom metrics volumes by using the bulk metric tag configuration feature. To specify a namespace for your metrics, click Configure Tags* on Metrics Summary. You can configure all metrics matching that namespace prefix with the same allowlist of tags under Include tags… or the same blocklist of tags under Exclude tags….
You can configure and delete tags for multiple metrics through the API. To configure a blocklist of tags for multiple metrics, set the parameter exclude_tags_mode: true on the API.
Refine and optimize your aggregations
You can further adjust your custom metrics filters by opting in to more metrics aggregations on your count, gauge, or rate metrics. To preserve the mathematical accuracy of your queries, Datadog only stores the most frequently queried time/space aggregation combination for a given metric type:
- Configured counts and rates are queryable in time/space with SUM
- Configured gauges are queryable in time/space with AVG
You can add or remove aggregations at any time with no required Agent or code-level changes.
The tag configuration modal pre-populates with an allowlist of aggregations that have been actively queried on dashboards, notebooks, monitors and through API in the past 30 days (colored in blue with an icon). You can also include your own additional aggregations.
Metrics without Limits™ billing
Configuring your tags and aggregations gives you control over which custom metrics can be queried – ultimately reducing your billable count of custom metrics. Metrics without Limits™ decouples ingestion costs from indexing costs. You can continue sending Datadog all of your data (everything is ingested) and you can specify an allowlist of tags you want to remain queryable in the Datadog platform. If the volume of data Datadog is ingesting for your configured metrics differs from the smaller, remaining volume you index, you can see two distinct volumes on your Usage page as well as the Metrics Summary page.
- Ingested Custom Metrics: The original volume of custom metrics based on all ingested tags.
- Indexed Custom Metrics: The volume of custom metrics that remains queryable in the Datadog platform (based on any Metrics without Limits™ configurations)
Note: Only configured metrics contribute to your Ingested custom metrics volume. If a metric is not configured with Metrics without Limits™, you’re only charged for its indexed custom metrics volume.
Learn more about Custom Metrics Billing.
Getting started with Metrics without Limits™
- Configure your Top 20 metrics on your Plan & Usage page from the Metrics Summary page or by using the API.
You can use bulk metric configuration (
* syntax) to quickly configure tags on multiple metrics. Datadog notifies you when the bulk configuration job is completed.
Note: If you’re using the Create Tag Configuration API, use the tag configuration cardinality estimator API first to validate the potential impact of your tag configurations prior to creating tag configurations. If the UI or the estimator API returns a resulting number of indexed that is larger than ingested, do not save your tag configuration.
Configure your unqueried metrics with empty tag configurations.
As your teams continue cleaning up noisy metrics that are never queried in the Datadog platform, you can instantly minimize the costs of these unqueried metrics by configuring them with an empty allowlist of tags.
Review your usage and billing. After configuring your metrics, the impact of your changes can be validated in three ways:
- Prior to saving your configuration, the tag configuration cardinality estimator returns the estimated resulting number of indexed custom metrics which should be lower than your ingested custom metrics volumes.
- After saving your configuration, the Metrics Summary details sidepanel should show that your indexed custom metrics are lower than your ingested custom metrics volume.
- 24 hours after you’ve saved your configuration, you can also view the impact on your Plan & Usage page’s Top Custom Metrics table. There should be reduction in the volume of custom metrics between the Month-to-Date tab and the Most Recent Day tab of this table.
Best practices
You can set up alerts on your real-time estimated custom metrics usage metric so that you can correlate spikes in custom metrics with configurations.
Role based access control for Metrics without Limits™ is also available to control which users have permissions to use this feature that has billing implications.
Audit events allow you to track any tag configurations or percentile aggregations that have been made that may correlate with custom metrics spikes. Search for “tags:audit” and “queryable tag configuration” or “percentile aggregations” on your Events Stream
*Metrics without Limits is a trademark of Datadog, Inc.
Further reading
Additional helpful documentation, links, and articles:
