- 필수 기능
- 시작하기
- Glossary
- 표준 속성
- Guides
- Agent
- 통합
- 개방형텔레메트리
- 개발자
- Administrator's Guide
- API
- Datadog Mobile App
- CoScreen
- Cloudcraft
- 앱 내
- 서비스 관리
- 인프라스트럭처
- 애플리케이션 성능
- APM
- Continuous Profiler
- 스팬 시각화
- 데이터 스트림 모니터링
- 데이터 작업 모니터링
- 디지털 경험
- 소프트웨어 제공
- 보안
- AI Observability
- 로그 관리
- 관리
",t};e.buildCustomizationMenuUi=t;function n(e){let t='
",t}function s(e){let n=e.filter.currentValue||e.filter.defaultValue,t='${e.filter.label}
`,e.filter.options.forEach(s=>{let o=s.id===n;t+=``}),t+="${e.filter.label}
`,t+=`Ingestion controls affect what traces are sent by your applications to Datadog. APM Metrics are always calculated based on all traces, and are not impacted by ingestion controls.
The Ingestion Control page provides visibility into the ingestion configuration of your applications and services. From the ingestion control page:
Use the data in the ingestion control header to monitor your trace ingestion. The header displays the total amount of data ingested over the past hour, your estimated monthly usage, and the percentage of your allocated monthly ingestion limit, calculated based on your active APM infrastructure (such as hosts, Fargate tasks, and serverless functions).
If the monthly usage is under 100%
, the projected ingested data fits within your monthly allotment. A monthly usage value over 100%
means that the monthly ingested data is projected to be over your monthly allotment.
The service table contains information about the ingested volumes and ingestion configuration, broken down by service:
sampling_service
dimension, set on the datadog.estimated_usage.apm.ingested_bytes
metrics. For more information, read APM usage metrics.Automatic
if the default head-based sampling mechanism from the Agent applies. If the ingestion was configured with trace sampling rules, the service is marked as Configured
; a Local
label is set when the sampling rule is applied from configuration in the tracing library, a Remote
label is set when the sampling rule is applied remotely, from the UI. For more information about configuring ingestion for a service, read about changing the default ingestion rate.Limited Resource
when some spans are dropped due to the Datadog Agent reaching CPU or RAM limits set in its configuration, Legacy Setup
when some spans are ingested through the legacy App Analytics mechanism, or OK
otherwise.Filter the page by environment, configuration, and status to view services for which you need to take an action. To reduce the global ingestion volume, sort the table by the Downstream Bytes/s
column to view services responsible for the largest share of your ingestion.
Note: The table is powered by the usage metrics datadog.estimated_usage.apm.ingested_spans
and datadog.estimated_usage.apm.ingested_bytes
. These metrics are tagged by service
, env
and ingestion_reason
.
The Traffic Breakdown column breaks down the destination of all traces starting from the service. It gives you an estimate of the share of traffic that is ingested and dropped, and for which reasons.
The breakdown is composed of the following parts:
Complete traces ingested (blue): The percentage of traces that have been ingested by Datadog.
Complete traces not retained (gray): The percentage of traces that have not been ingested by Datadog. Some traces might be dropped because:
Complete traces dropped by the tracer rate limiter (orange): When you choose to manually set the service ingestion rate as a percentage with trace sampling rules, a rate limiter is automatically enabled, set to 100 traces per second by default. See the rate limiter documentation to change this rate.
Traces dropped due to the Agent CPU or RAM limit (red): This mechanism may drop spans and create incomplete traces. To fix this, increase the CPU and memory allocation for the infrastructure that the Agent runs on.
Click on any service to view the Service Ingestion Summary, which provides actionable insights and configuration options for managing that service’s trace ingestion.
The table lists the applied sampling rates by resource of the service.
Ingested bytes
column surfaces the ingested bytes from spans of the service and resource, while the Downstream bytes
column surfaces the ingested bytes from spans where the sampling decision is made starting from that service and resource, including bytes from downstream services in the call chain.Configuration
column surfaces where the resource sampling rate is being applied from:Automatic
if the default head-based sampling mechanism from the Agent applies.Local Configured
if a sampling rule was set locally in the tracing library.Remote Configured
if a remote sampling rule was set from the Datadog UI. To learn how to configure sampling rules from the Ingestion Control page, read the section on remotely configuring sampling rules.Note: If the service is not making sampling decisions, the service’s resources will be collapsed under the Resources not making sampling decisions
row.
Explore the Ingestion reasons breakdown to see which mechanisms are responsible for your service ingestion. Each ingestion reason relates to one specific ingestion mechanism. After changing your service ingestion configuration, you can observe the increase or decrease of ingested bytes and spans in this timeseries graph based on the past hour of ingested data.
If most of your service ingestion volume is due to decisions taken by upstream services, investigate the detail of the Sampling decision makers top list. For example, if your service is non-root, (meaning that it never decides to sample traces), observe all upstream services responsible for your non-root service ingestion. Configure upstream root services to reduce your overall ingestion volume.
For further investigations, use the APM Trace - Estimated Usage Dashboard, which provides global ingestion information as well as breakdown graphs by service
, env
and ingestion reason
.
See the Datadog Agent and tracing library versions your service is using. Compare the versions in use to the latest released versions to make sure you are running recent and up-to-date Agents and libraries.
Note: You need to upgrade the Agent to v6.34 or v7.34 for the version information to be reported.
When configuring sampling rates for a service by resource name, you can use two main strategies:
Configurations for these strategies can be applied Remotely through the Datadog UI. This method allows changes to take effect immediately without redeploying your service. For Custom Sampling, you also have the option to apply configurations Locally by updating your service’s configuration files and redeploying.
Using Remote Configuration for service ingestion rates has specific requirements.
APM Remote Configuration Write
permissions. If you don’t have these permissions, ask your Datadog admin to update your permissions from your organization settings.Find below the minimum tracing library version required for the feature:
Language | Minimum version required |
---|---|
Java | v1.34.0 |
Go | v1.64.0 |
Python | v.2.9.0 |
Ruby | v2.0.0 |
Node.js | v5.16.0 |
PHP | v1.4.0 |
.NET | v2.53.2 |
C++ | v0.2.2 |
Specify a target monthly ingestion volume for one or multiple services while keeping visibility over all services and endpoints.
To configure adaptive sampling:
For more information, see Adaptive Sampling.
To configure custom sampling rates for the service by resource name:
*
) to match against multiple resources at the same time.This option applies the configuration using Remote Configuration, so you do not need to redeploy the service for the change to take effect. You can observe the configuration changes from the Live Search Explorer.
Click Apply to save the configuration.
Resources that have been configured remotely display as Configured Remote
in the Configuration column.
This option generates configuration for you to apply manually.
Configured Local
in the Configuration column.Click Configure Datadog Agent Ingestion to manage default head-based sampling rates, error sampling and rare sampling.
DD_APM_MAX_TPS
locally at the Agent level.DD_APM_ERROR_TPS
locally at the Agent level.DD_APM_ENABLE_RARE_SAMPLER
locally at the Agent level.With remote configuration, you don’t have to restart the Agent to update these parameters. Click Apply
to save the configuration changes, and the new configuration takes effect immediately. Remote configuration for Agent sampling parameters is available if you are using Agent version 7.42.0 or higher.
Note: The Other Ingestion Reasons
(gray) section of the pie chart represents other ingestion reasons which are not configurable at the Datadog Agent level.
Note: Remotely configured parameters take precedence over local configurations such as environment variables and datadog.yaml
configuration.
If sampling rules are set in multiple locations, the following precedence rules apply in order, where rules that appear first on the list can override lower precedence rules:
DD_TRACE_SAMPLING_RULES
)DD_TRACE_SAMPLE_RATE
)DD_APM_MAX_TPS
)To phrase it another way, Datadog uses the following precedence rules:
추가 유용한 문서, 링크 및 기사: