Remote Configuration

이 페이지는 아직 영어로 제공되지 않습니다. 번역 작업 중입니다.
현재 번역 프로젝트에 대한 질문이나 피드백이 있으신 경우 언제든지 연락주시기 바랍니다.

Overview

Remote Configuration is a Datadog capability that allows you to remotely configure and change the behavior of select product features in Datadog components such as Agents, tracing libraries, and Observability Pipelines Workers deployed in your infrastructure. Use Remote Configuration to apply configurations to Datadog components in your environment on demand, decreasing management costs, reducing friction between teams, and accelerating issue resolution times.

For Datadog security products, App and API Protection and Workload Protection, Remote Configuration-enabled Agents and compatible tracing libraries provide real-time security updates and responses, enhancing security posture for your applications and cloud infrastructure.

How it works

When Remote Configuration is enabled, Datadog components such as the Datadog Agent securely poll the configured Datadog site for configuration changes that are ready to apply. Pending changes are then automatically applied to Datadog components. For example, after you submit configuration changes in the Datadog UI for a Remote Configuration-enabled product feature, the changes are stored in Datadog.

The following diagram illustrates how Remote Configuration works:

Users configure features in the UI, the config is stored in Datadog, the Agent requests config updates.

You configure select product features in the Datadog UI.
The product feature configurations are securely stored within Datadog.
Remote-configuration enabled Datadog components in your environments securely poll, receive, and automatically apply configuration updates from Datadog. Tracing libraries that are deployed in your environments communicate with Agents to request and receive configuration updates from Datadog instead of directly polling Datadog.

Supported environments

Remote Configuration works in environments where supported Datadog components are deployed. Supported Datadog components include:

Agents
Tracers (indirectly)
Observability Pipeline Workers
Private action runners and serverless container cloud services such as AWS Fargate.

Remote Configuration does not support serverless container managed apps, such as AWS App Runner, Azure Container Apps, Google Cloud Run; or functions deployed with container packaging, such as AWS Lambda, Azure Functions, and Google Cloud Functions.

Supported products and features

The following products and features are supported with Remote Configuration.

Fleet Automation

Send flares directly from the Datadog site. Seamlessly troubleshoot the Datadog Agent without directly accessing the host.

Upgrade your Agents (Preview).

App and API Protection (AAP)

1-click AAP activation: Enable AAP in 1-click from the Datadog UI.

In-App attack patterns updates: Receive the newest Web Application Firewall (WAF) attack patterns automatically as Datadog releases them, following newly disclosed vulnerabilities or attack vectors.

Protect: Block attackers’ IPs, authenticated users, and suspicious requests that are flagged in AAP Security Signals and Traces temporarily or permanently through the Datadog UI.

Application Performance Monitoring (APM)

Configuration at runtime (Beta): Change a service’s trace sampling rate, Log Injection enablement, and HTTP header tags from within the Software Catalog UI, without having to restart the service. Read Configuration at Runtime for more information.

Remotely set Agent sampling rate (Public Beta): Remotely configure the Datadog Agent to change its trace sampling rates and set rules to scale your organization’s trace ingestion according to your needs, without needing to restart your Datadog Agent.

Dynamic Instrumentation

Send critical metrics, traces, and logs from your live applications with no code changes.

Workload Protection

Automatic default Agent rule updates: Automatically receive and update the default Agent rules maintained by Datadog as new Agent detections and enhancements are released. See Setting Up Workload Protection for more information.

Automatic deployment of custom Agent rules: Automatically deploy your custom Agent rules to designated hosts (all hosts or a defined subset of hosts).

Observability Pipelines

Remotely deploy and update Observability Pipelines Workers (OPW): Build and edit pipelines in the Datadog UI, rolling out your configuration changes to OPW instances running in your environment.

Sensitive Data Scanner (SDS) through the Datadog Agent

Redact sensitive information in your logs within your premises (Preview): Remotely configure and deploy OOTB Sensitive Data Scanning rules to the Datadog Agent in your environment. See Sensitive Data Scanner for more information.

Private action runner

Run Datadog workflows and apps that interact with services hosted on your private network without exposing your services to the public internet. For more information, see Private Actions.

Security considerations

Datadog implements the following safeguards to protect the confidentiality, integrity, and availability of configurations received and applied by your Datadog components:

Remote Configuration enabled Datadog components deployed in your infrastructure request configurations from Datadog.
Some components like private action runners are always remote configuration enabled. Others, like Agents, can be enabled or disabled using in-disk configuration options.
Datadog never sends configuration changes unless requested by Datadog components. If it does send configuration changes, Datadog only sends changes relevant to the requesting component.
The configuration requests are initiated from your infrastructure to Datadog over HTTPS (port 443). This is the same port that the Agent uses by default to send observability data.
The communication between your datadog components and Datadog is encrypted using HTTPS and is authenticated and authorized using your Datadog API key except in the case of private action runners where a JWT token is used instead.
Only users with the api_keys_write permissions are authorized to enable or disable Remote Configuration capability on API keys and use the supported product features.
Your configuration changes submitted through the Datadog UI are signed and validated by the requesting Datadog component, verifying the integrity of the configuration.

Enabling Remote Configuration

Enable Remote Configuration in your organization

Beginning April 8, 2024, Remote Configuration is enabled by default for:

New child organizations that are created by existing Datadog customers who already have enabled Remote Configuration at the parent organization level and are in the same Datadog site as their parent organization.
Organizations created by new Datadog customers.

To enable Remote Configuration:

Ensure your RBAC permissions include org_management, so you can enable Remote Configuration for your organization.
From your Organization Settings page, enable Remote Configuration. This enables Datadog components across your organization to receive configurations from Datadog.

< HH: if needed, insert section to deal with older configurations >

Product-specific configuration

After you’ve enabled Remote Configuration in your organization, consult the documentation below for instructions specific to the product you’re configuring.

Product	Setup instructions
Fleet Automation	Setup Fleet Automation
APM	Configuration at runtime
Dynamic Instrumentation	Getting started with Dynamic Instrumentation
Workload Protection	Workload Protection
Observability Pipelines	Advanced Configurations
Sensitive Data Scanner	Cloud storage
Private Action Runner	Private Actions Overview

< HH: BOOKMARK (I haven’t worked on anything below this line yet) >

Best practices

Datadog Audit Trail

Use Datadog Audit Trail to monitor organization access and Remote Configuration enabled events. Audit Trail allows your administrators and security teams to track the creation, deletion, and modification of Datadog API and application keys. After Audit Trail is configured, you can view events related to Remote Configuration enabled features and who has requested these changes. Audit Trail allows you to reconstruct sequences of events, and establish robust Datadog monitoring for Remote Configuration.

Monitors

Configure monitors to receive notifications when an event of interest is encountered.

Troubleshooting

If you experience issues using Remote Configuration, use the following troubleshooting guidelines. If you need further assistance, contact Datadog support.

Restart the Agent

After the Agent configuration is updated in the datadog.yaml file, restart the Agent for this change to take effect.

Ensure Datadog Remote Configuration endpoints are reachable from your environment

To use Remote Configuration, both the Agent and the Observability Pipelines Worker deployed in your environment communicate to Datadog Remote Configuration endpoints. For private network connection between your environment and Datadog, you can also connect to Remote Configuration Virtual Private Cloud endpoints. Ensure that outbound HTTPS has access to Remote Configuration endpoints from your environment. If you also have a proxy in between Datadog and your environment, update your proxy settings to incorporate Remote Configuration endpoints.

Enable Remote Configuration at the organization level

To enable Remote Configuration at the Organization level in the Datadog UI, go to the Remote Configuration Setup page in your Organization Settings. This allows your authenticated and authorized Datadog components to remotely receive configurations and security detection rules of supported features from Datadog. Only users who have the org_management RBAC permission can enable Remote Configuration at the Organization level.

Enable Remote Configuration on the API key

To authenticate and authorize the Agent to receive configurations and security detection rules, and to allow the Observability Pipelines Worker to receive configurations, enable Remote Configuration on the relevant API Key. Only users who have the api_keys_write RBAC permission can enable Remote Configuration on the API Key.

Note: If you have api_keys_write RBAC permission, but are missing Remote Configuration Organization level permissions, you cannot enable Remote Configuration on a new or an existing API Key. You only have permission to disable Remote Configuration on an existing API Key.

Review Remote Configuration status of Agents and Tracing libraries

Gain visibility into the Remote Configuration status of your Agent and Tracing library through the Remote Configuration UI.

The following table describes the meaning of each Agent status:

Agent Status	Description
CONNECTED	The Agent deployed in your environment is able to reach, authenticate, and authorize successfully to Datadog. This is the optimal state you want your Agents to be in for Remote Configuration.
UNAUTHORIZED	The Agent deployed in your environment is able to reach Datadog but is not able to authenticate and authorize with Datadog for Remote Configuration operation. The most likely cause is the API Key used by the Agent is not Remote Configuration-enabled. To fix the issue, enable Remote Configuration capability on the API Key used by the Agent.
CONNECTION ERROR	The Agent deployed in your environment has `remote_config.enabled` set to true in its `datadog.yaml` configuration file, however, the Agent cannot be found in the Remote Configuration service. The most likely cause is that the Agent is unable to reach Remote Configuration endpoints. To fix the issue, allow outbound HTTPS access to Remote Configuration endpoints from your environment. This status displays when the Agent version is `7.45.0` or higher.
DISABLED	The Agent deployed in your environment has `remote_config.enabled` set to false in its `datadog.yaml` configuration file. Set `remote_config.enabled` to true if you want to enable Remote Configuration on the Agent. This status displays when the Agent version is `7.45.0` or higher.
NOT CONNECTED	The Agent cannot be found in the Remote Configuration service and could have `remote_config.enabled` set to true or false in its `datadog.yaml` configuration file. Check your local Agent configuration or your proxy settings. This status displays when the Agent version is higher than `7.41.1` but lower than `7.45.0`.
UNSUPPORTED AGENT	The Agent is on a version that is not Remote Configuration capable. To fix this issue, update the Agent to the latest available version.

The following table describes the meaning of each Tracing library status:

Tracing library Status	Description
CONNECTED	The Tracing library is successfully connected to the Remote Configuration service through the associated Agent. This is the optimal state you want your Tracing library to be in for Remote Configuration.
UNAUTHORIZED	The Tracing library is associated with an Agent which doesn’t have `Remote Config Read` permission on its API key. To fix the issue, you need to enable Remote Configuration capability on the API Key used by the Agent associated with the Tracing library.
CONNECTION ERROR	The Tracing library deployed in your environment is associated with an Agent that has remote_config.enabled set to true in its `datadog.yaml` configuration file, however, the agent cannot be found in the Remote Configuration service. The most likely cause of this is that the associated Agent is unable to reach Remote Configuration endpoints. To fix the issue, you need to allow outbound HTTPS access to Remote Configuration endpoints from your environment.
DISABLED	The Tracing library deployed in your environment is associated with an Agent that has `remote_config.enabled` set to false in its `datadog.yaml` configuration file. This could be set deliberately or mistakenly. To enable Remote Configuration on the associated Agent, set `remote_config.enabled` to true.
NOT CONNECTED	The Tracing library cannot be found in the Remote Configuration service and is associated with an Agent that could have `remote_config.enabled` set to true or false in its `datadog.yaml` configuration file. Check your local Agent configuration or your proxy settings.
UNSUPPORTED AGENT	The Tracing library is associated with an Agent which is not Remote Configuration capable. To fix this issue, update the associated Agent software to the latest available version.
NOT DETECTED	The Tracing library does not support Remote Configuration. To fix this issue, update the Tracing library software to the latest available version.
UNKNOWN	The Tracing library status is unknown, and it can’t be determined if an Agent is associated with the Tracing library. For example, this could be because the Agent is deployed on a fully managed serverless container service like AWS Fargate.

Opting out of Remote Configuration

To opt out of Remote Configuration use, you can disable Remote Configuration at the organization level. Optionally, you can also disable Remote Configuration capability at the API key level and Agent level.

At the Organization level

Disable Remote Configuration at the organization level on the Remote Configuration page. This disables Datadog components across your organization to receive configurations from Datadog. You need the org_management permission to disable Remote Configuration at the organization level.

At the API key level

Disable the API key of your choice on the API Keys page. You need the api_keys_write permission to disable Remote Configuration on an API key.

At the Agent level

Starting with Agent version 7.47.0, remote_configuration.enabled is set to true by default in the Agent. This setting causes the Agent to request configuration updates from the Datadog site.

To receive configurations from Datadog, you also need to take the following steps:

Enable Remote Configuration at the organization level.
Enable Remote Configuration capability on your API Key from the Datadog UI.
Allow outbound HTTPS access to Remote Configuration endpoints from your environment.

If you don’t want your Agent to send configuration requests to Datadog, you can set remote_configuration.enabled to false in the Agent.

Change remote_configuration.enabled from true to false in your configuration YAML file:

remote_configuration:
  enabled: false

Add the following to your Datadog Agent manifest:

DD_REMOTE_CONFIGURATION_ENABLED=false

Add the following to your Helm chart:

datadog:
  remoteConfiguration:
    enabled: false