Connect to Datadog over Google Cloud Private Service Connect

This feature is not supported for the selected Datadog site.

Google Cloud Private Service Connect (PSC) allows you to send telemetry to Datadog without using the public internet.

Datadog exposes some of its data intake services in Google Cloud as Private Service Connect published services, as seen in the table of published services.

You can configure a PSC endpoint to expose a private IP address for each Datadog intake service. This IP address routes traffic to the Datadog backend. You can then configure a Google Cloud Private DNS Zone to override the DNS names corresponding to the products for each endpoint that is consumed.

Google Cloud Private Service Connect schema. On the left, a 'Customer VPC' box contains Datadog Agents sending data to a PSC endpoint. On the right, a 'Datadog VPC' box contains a service attachment in communication with Datadog services. The endpoint in the 'Customer VPC' box connects to the service attachment in the 'Datadog VPC' box through the Google Cloud backbone.

Setup

Connect an endpoint

  1. In your Google Cloud console, navigate to Network services > Private Service Connect.

  2. Go to the Endpoints section. Click on Connect endpoint.

    Screenshot of a 'Connect endpoint' page in the Google Cloud console
    • Under Target, select Published service.
    • For Target service, enter the PSC target name that corresponds to the Datadog intake service that you want to use. You can find your PSC target name in the table of published services.
    • For Endpoint name, enter a unique identifier to use for this endpoint. You can use datadog-<SERVICE>. For example: datadog-api.
    • For Network and Subnetwork, choose the network and subnetwork where you want to publish your endpoint.
    • For IP address, click the dropdown and select Create IP address to create an internal IP from your subnet dedicated to the endpoint. Select this IP.
    • Check Enable global access if you intend to connect the endpoint to virtual machines outside of the us-central1 region.

    Note: Datadog exposes PSC producer endpoints from the us-central1 region. These endpoints support global access, allowing services to connect from any region. However, the forwarding rule must be created in the us-central1 region.

  3. Click Add endpoint. Verify that your status is Accepted. Take note of the IP address, as this is used in the next section.

    Screenshot of a success message after adding an endpoint in the Google Cloud console. Includes an IP address

Create a DNS zone

  1. In your Google Cloud console, navigate to Network services > Cloud DNS.

  2. Click on Create zone.

    Screenshot of a 'Create a DNS zone' page in the Google Cloud console
    • Under Zone type, select Private.
    • For Zone name, enter a descriptive name for your zone.
    • For DNS name, enter the private DNS name that corresponds to the Datadog intake service that you want to use. You can find your DNS name in the table of published services.
  3. Next, create an A record that points to the endpoint IP. On the Zone details page of the zone you created, click on Add record set.

    Screenshot of the 'Create record set' page in the Google Cloud console.
    • For DNS name, leave the field unmodified.
    • For Resource record type, select A.
    • Under IPv4 Address, enter the IP address that was displayed at the end of the previous section.

Additional required steps for metrics and traces

There are two Datadog Intake Services that are subdomains of the (agent.) domain. Because of this, the Private Hosted Zone is slightly different from other intakes.

Create a Private Zone for (agent.), as outlined in the Create a DNS Zone section. Then add the three records below.

DNS nameResource record typeIPv4 address
(apex)AIP address for your metrics endpoint
*AIP address for your metrics endpoint
traceAIP address for your traces endpoint

Note: this zone requires a wildcard (*) record that points to the IP address for your metrics endpoint. This is because Datadog Agents submit telemetry using a versioned endpoint in the form (<version>-app.agent.).

Validation

To verify your configuration, SSH into one of your local nodes and run a dig command similar to the following:

Verify that that the wildcard is routing to the metrics endpoint

> dig +noall +answer 7-49-0-app.agent.us5.datadoghq.com

The response resembles:

7-49-0-app.agent.us5.datadoghq.com. 300 IN A        10.1.0.4

Verify that the trace subdomain is routing to the traces endpoint

> dig +noall +answer trace.agent.us5.datadoghq.com

The response resembles:

trace.agent.us5.datadoghq.com. 300 IN  A       10.1.0.9

Ensure that the IP address in the response matches the one associated with your PSC target.

Published services

Datadog intake servicePSC target namePrivate DNS name
Logs (Agent)projects/datadog-prod-us5/regions/us-central1/serviceAttachments/nlb-logs-agent-intake-pscagent-http-intake.logs.us5.datadoghq.com
Logs (User HTTP Intake)projects/datadog-prod-us5/regions/us-central1/serviceAttachments/nlb-logs-intake-pschttp-intake.logs.us5.datadoghq.com
APIprojects/datadog-prod-us5/regions/us-central1/serviceAttachments/nlb-api-pscapi.us5.datadoghq.com
Metricsprojects/datadog-prod-us5/regions/us-central1/serviceAttachments/nlb-metrics-agent-pscagent.us5.datadoghq.com
Containersprojects/datadog-prod-us5/regions/us-central1/serviceAttachments/nlb-orchestrator-pscorchestrator.us5.datadoghq.com
Processprojects/datadog-prod-us5/regions/us-central1/serviceAttachments/nlb-process-pscprocess.us5.datadoghq.com
Profilingprojects/datadog-prod-us5/regions/us-central1/serviceAttachments/nlb-logs-http-profile-pscintake.profile.us5.datadoghq.com
Tracesprojects/datadog-prod-us5/regions/us-central1/serviceAttachments/nlb-trace-edge-pscagent.us5.datadoghq.com
Database Monitoringprojects/datadog-prod-us5/regions/us-central1/serviceAttachments/nlb-dbm-metrics-pscdbm-metrics-intake.us5.datadoghq.com
Remote Configurationprojects/datadog-prod-us5/regions/us-central1/serviceAttachments/nlb-fleet-pscconfig.us5.datadoghq.com

Private Service Connect (PSC) allows you to send telemetry to Datadog without using the public internet.

Datadog exposes some of its data intake services in Google Cloud Platform as PSC published services, as seen in the table of published services.

You can configure a PSC endpoint to expose a private IP address for each Datadog intake service. This IP address routes traffic to the Datadog backend. You can then configure a Google Cloud Private DNS Zone to override the DNS names corresponding to the products for each endpoint that is consumed.

Google Cloud Private Service Connect schema. On the left, a 'Customer VPC' box contains Datadog Agents sending data to a PSC endpoint. On the right, a 'Datadog VPC' box contains a service attachment in communication with Datadog services. The PSC endpoint in the 'Customer VPC' box connects to the service attachment in the 'Datadog VPC' box through the Google Cloud backbone.

Setup

Connect an endpoint

  1. In your GCP console, navigate to Network services > Private Service Connect.

  2. Go to the Endpoints section. Click on Connect endpoint.

    Screenshot of a 'Connect endpoint' page in the Google Cloud console
    • Under Target, select Published service.
    • For Target service, enter the PSC target name that corresponds to the Datadog intake service that you want to use. You can find your PSC target name in the table of published services.
    • For Endpoint name, enter a unique identifier to use for this endpoint. You can use datadog-<SERVICE>. For example: datadog-metrics.
    • For Network and Subnetwork, choose the network and subnetwork where you want to publish your endpoint.
    • For IP address, click the dropdown and select Create IP address to create an internal IP from your subnet dedicated to the endpoint. Select this IP.
    • Check Enable global access if you intend to connect the endpoint to virtual machines outside of the europe-west3 region.

    Note: Datadog exposes PSC producer endpoints from the europe-west3 region. These endpoints support global access, allowing services to connect from any region. However, the forwarding rule must be created in the europe-west3 region.

  3. Click Add endpoint. Verify that your status is Accepted. Take note of the IP address, as this is used in the next section.

    Screenshot of a success message after adding an endpoint in the Google Cloud console. Includes an IP address

Create a DNS zone

  1. In your Google Cloud console, navigate to Network services > Cloud DNS.

  2. Click on Create zone.

    Screenshot of a 'Create a DNS zone' page in the Google Cloud console
    • Under Zone type, select Private.
    • For Zone name, enter a descriptive name for your zone.
    • For DNS name, enter the private DNS name that corresponds to the Datadog intake service that you want to use. You can find your DNS name in the table of published services.
  3. Next, create an A record that points to the endpoint IP. On the Zone details page of the zone you created, click on Add record set.

    Screenshot of the 'Create record set' page in the Google Cloud console.
    • For DNS name, leave the field unmodified.
    • For Resource record type, select A.
    • Under IPv4 Address, enter the IP address that was displayed at the end of the previous section.

Additional required steps for metrics and traces

There are two Datadog Intake Services that are subdomains of the (agent.) domain. Because of this, the Private Hosted Zone is slightly different from other intakes.

Create a Private Zone for (agent.), as outlined in the Create a DNS Zone section. Then add the three records below.

DNS nameResource record typeIPv4 address
(apex)AIP address for your metrics endpoint
*AIP address for your metrics endpoint
traceAIP address for your traces endpoint

Note: this zone requires a wildcard (*) record that points to the IP address for your metrics endpoint. This is because Datadog Agents submit telemetry using a versioned endpoint in the form (<version>-app.agent.).

Validation

To verify your configuration, SSH into one of your local nodes and run a dig command similar to the following:

Verify that that the wildcard is routing to the metrics endpoint

> dig +noall +answer 7-49-0-app.agent.datadoghq.eu

The response resembles:

7-49-0-app.agent.datadoghq.eu. 300 IN A        10.1.0.4

Verify that the trace subdomain is routing to the traces endpoint

> dig +noall +answer trace.agent.datadoghq.eu

The response resembles:

trace.agent.datadoghq.eu. 300 IN  A       10.1.0.9

Ensure that the IP address in the response matches the one associated with your PSC target.

Published services

Datadog intake servicePSC target namePrivate DNS name
Logs (Agent)projects/datadog-prod/regions/europe-west3/serviceAttachments/nlb-logs-agent-intake-pscagent-http-intake.logs.datadoghq.eu
Logs (User HTTP Intake)projects/datadog-prod/regions/europe-west3/serviceAttachments/nlb-logs-intake-pschttp-intake.logs.datadoghq.eu
APIprojects/datadog-prod/regions/europe-west3/serviceAttachments/nlb-api-pscapi.datadoghq.eu
Metricsprojects/datadog-prod/regions/europe-west3/serviceAttachments/nlb-metrics-agent-pscagent.datadoghq.eu
Containersprojects/datadog-prod/regions/europe-west3/serviceAttachments/nlb-orchestrator-pscorchestrator.datadoghq.eu
Processprojects/datadog-prod/regions/europe-west3/serviceAttachments/nlb-process-pscprocess.datadoghq.eu
Profilingprojects/datadog-prod/regions/europe-west3/serviceAttachments/nlb-logs-http-profile-pscintake.profile.datadoghq.eu
Tracesprojects/datadog-prod/regions/europe-west3/serviceAttachments/nlb-trace-edge-pscagent.datadoghq.eu
Database Monitoringprojects/datadog-prod/regions/europe-west3/serviceAttachments/nlb-dbm-metrics-pscdbm-metrics-intake.datadoghq.eu
Remote Configurationprojects/datadog-prod/regions/europe-west3/serviceAttachments/nlb-fleet-pscconfig.datadoghq.eu

Further reading

PREVIEWING: may/unit-testing