Getting Started with Google Cloud

Docs > Getting Started > Présentation des intégrations > Getting Started with Google Cloud

Cette page n'est pas encore disponible en français, sa traduction est en cours.
Si vous avez des questions ou des retours sur notre projet de traduction actuel, n'hésitez pas à nous contacter.

Overview

Use this guide to get started monitoring your Google Cloud environment at a folder or organization level. This approach simplifies the setup for Google Cloud environments with multiple projects, ensuring that there are no gaps in monitoring.

Setup

Prerequisites

Create a Datadog account
Set up a Service Account in any of your Google Cloud projects
Review these Google Cloud Prerequisites:

◆ The Google Cloud integration requires the below APIs to be enabled for the folder or organization you want to monitor:

Cloud Monitoring API: Allows Datadog to query your Google Cloud metric data.
Compute Engine API: Allows Datadog to discover compute instance data.
Cloud Asset API: Allows Datadog to request Google Cloud resources and link relevant labels to metrics as tags.
Cloud Resource Manager API: Allows Datadog to append metrics with the correct resources and tags.
IAM API: Allows Datadog to authenticate with Google Cloud.
Google Cloud Billing API: Allows developers to manage billing for their Google Cloud Platform projects programmatically.

You can confirm if these APIs are enabled by heading to Enabled APIs & Services.

Organization-level metric collection

Org-level (or folder-level) monitoring is recommended for comprehensive coverage of all projects, including any future projects that may be created in an org or folder. To set up monitoring for individual projects, see the main Google Cloud integration page.

Note: You must have the Admin role assigned to your Cloud Identity user account at the desired scope (for example, Organization Admin).

1. Create a Google Cloud service account in the default project

Open your Google Cloud console.
Navigate to IAM & Admin > Service Accounts.
Click Create service account at the top.
Give the service account a unique name.
Click Done to complete creating the service account.

2. Add the service account at the organization or folder level

In the Google Cloud console, go to the IAM page.
Select a folder or organization.
To grant a role to a principal that does not already have other roles on the resource, click Grant Access, then enter the email of the service account you created earlier.
Enter the service account’s email address.
Assign the following roles:

Compute Viewer provides read-only access to get and list Compute Engine resources
Monitoring Viewer provides read-only access to the monitoring data availabile in your Google Cloud environment
Cloud Asset Viewer provides read-only access to cloud assets metadata
Browser provides read-only access to browse the hierarchy of a project

Click Save.

Note: The Browser role is only required in the default project of the service account. Other projects require only the other listed roles.

3. Add the Datadog principal to your service account

Note: If you previously configured access using a shared Datadog principal, you can revoke the permission for that principal after you complete these steps.

In Datadog, navigate to Integrations > Google Cloud Platform.
Click Add Google Cloud Account. If you have no configured projects, you are automatically redirected to this page.
Copy your Datadog principal and keep it for the next section.

The page for adding a new Google Cloud account in Datadog's Google Cloud integration tile

Note: Keep this window open for Section 4.

In the Google Cloud console, under the Service Accounts menu, find the service account you created in Section 1.
Go to the Permissions tab and click Grant Access.

Google Cloud console interface, showing the Permissions tab under Service Accounts.

Paste your Datadog principal into the New principals text box.
Assign the role of Service Account Token Creator.
Click Save.

4. Complete the integration setup in Datadog

In your Google Cloud console, navigate to the Service Account > Details tab. On this page, find the email associated with this Google service account. It has the format <SA_NAME>@<PROJECT_ID>.iam.gserviceaccount.com.
Copy this email.
Return to the integration configuration tile in Datadog (where you copied your Datadog principal in the previous section).
In the box under Add Service Account Email, paste the email you previously copied.
Click Verify and Save Account.

After finishing these steps, metrics appear in Datadog after approximately 15 minutes.

Validation

To view your metrics, use the left menu to navigate to Metrics > Summary and search for gcp:

The Metric Summary page in Datadog filtered to metrics beginning with GCP

Google Cloud integrations

The Google Cloud integration collects all available Google Cloud metrics from your projects through the Google Cloud Monitoring API. Integrations are installed automatically when Datadog recognizes data being ingested in from your Google Cloud account. <EXAMPLE?>

See the Google Cloud integrations Datadog collects metrics from

Integration	Description
App Engine	PaaS (platform as a service) to build scalable applications
Big Query	Enterprise data warehouse
Bigtable	NoSQL Big Data database service
Cloud SQL	MySQL database service
Cloud APIs	Programmatic interfaces for all Google Cloud Platform services
Cloud Armor	Network security service to help protect against denial of service and web attacks
Cloud Composer	A fully managed workflow orchestration service
Cloud Dataproc	A cloud service for running Apache Spark and Apache Hadoop clusters
Cloud Dataflow	A fully-managed service for transforming and enriching data in stream and batch modes
Cloud Filestore	High-performance, fully managed file storage
Cloud Firestore	A flexible, scalable database for mobile, web, and server development
Cloud Interconnect	Hybrid connectivity
Cloud IoT	Secure device connection and management
Cloud Load Balancing	Distribute load-balanced compute resources
Cloud Logging	Real-time log management and analysis
Cloud Memorystore for Redis	A fully managed in-memory data store service
Cloud Router	Exchange routes between your VPC and on-premises networks by using BGP
Cloud Run	Managed compute platform that runs stateless containers through HTTP
Cloud Security Command Center	Security Command Center is a threat reporting service.
Cloud Tasks	Distributed task queues
Cloud TPU	Train and run machine learning models
Compute Engine	High performance virtual machines
Container Engine	Kubernetes, managed by google
Datastore	NoSQL database
Firebase	Mobile platform for application development
Functions	Serverless platform for building event-based microservices
Kubernetes Engine	Cluster manager and orchestration system
Machine Learning	Machine learning services
Private Service Connect	Access managed services with private VPC connections
Pub/Sub	Real-time messaging service
Spanner	Horizontally scalable, globally consistent, relational database service
Storage	Unified object storage
Vertex AI	Build, train and deploy custom machine learning (ML) models.
VPN	Managed network functionality

For deep dives into monitoring many of the more popular services, check out the blogs linked below.

Integration blogs

Cloud Armor: Google Cloud Armor is a network security service protecting against DDoS and application attacks.
BigQuery: BigQuery is a serverless and multi-cloud data warehouse that can provide you with valuable insights from your business data.
Cloud Run: Cloud Run is a fully-managed platform that lets you run your code directly on scalable infrastructure in Google Cloud.
Cloud SQL: Cloud SQL is a fully-managed relational database service that works with MySQL, PostgreSQL, and SQL Server.
Compute Engine: Compute Engine is a computing and hosting service that provides you with the ability to create and run virtual machines in Google Cloud.
Dataflow: Dataflow is a fully-managed streaming analytics service that uses autoscaling and real-time data processing.
Eventarc: Eventarc is a fully-managed service enabling you to build event-driven architectures.
Google Kubernetes Engine (GKE): GKE is a fully-managed Kubernetes service.
Private Service Connect: Private Service Connect lets you access managed Google services privately from within your VPC network.
Security Command Center: Security Command Center provides posture management and threat detection for code, identities, and data.
Vertex AI: Vertex AI is a fully-managed generative AI development platform.

Limit metric collection filters

You can limit metric collection to only the specific hosts, Cloud Run instances, or Google Cloud integrations valuable to your organization. This can help control costs by reducing the number of API calls made on your behalf.

Limit metric collection by Google Cloud integration

Under the Metric Collection tab in Datadog’s Google Cloud integration page, deselect the metric namespaces to exclude.

The metric collection tab in the Datadog Google Cloud integration page

Limit metric collection by host or Cloud Run instance

Assign a tag (such as datadog:true) to the hosts or Cloud Run instances you want to monitor with Datadog.
Under the Metric Collection tab in Datadog’s Google Cloud integration page, enter the tags in the Limit Metric Collection Filters textbox. Only hosts that match one of the defined tags are imported into Datadog. You can use wildcards (? for single character, * for multi-character) to match many hosts, or ! to exclude certain hosts. This example includes all c1* sized instances, but excludes staging hosts:

datadog:monitored,env:production,!env:staging,instance-type:c1.*

See Google’s documentation on Creating and managing labels for more details.

In the below example, only Google Cloud hosts with the label datadog:true are monitored by Datadog:

The fields to limit metric collection in the Google Cloud integration tile

Log collection

Forwarding logs from your Google Cloud environment enables near real-time monitoring of the resources and activities taking place in your organization or folder. You can set up log monitors to be notified of issues, use Cloud SIEM to detect threats, or leverage Watchdog to identify unknown issues or anomalous behavior.

Use the Datadog Dataflow template to batch and compresses your log events before forwarding them to Datadog through Google Cloud Dataflow. This is the most network-efficient way to forward your logs. To specify which logs are forwarded, configure the Google Cloud Logging sink with any inclusion or exclusion queries using Google Cloud’s Logging query language.

Follow the instructions listed here to set up Log Collection. You can also use the Stream logs from Google Cloud to Datadog guide in the Google Cloud architecture center, for a more detailed explanation of the steps and architecture involved in log forwarding. For a deep dive into the benefits of the Pub/Sub to Datadog template, read Stream your Google Cloud logs to Datadog with Dataflow in the Datadog blog.

The Dataflow API must be enabled to use Google Cloud Dataflow. See Enabling APIs in the Google Cloud documentation for more information.

Resource changes collection

Resource changes collection allows you to monitor changes in your Google Cloud environment. You receive resource events in Datadog when Google’s Cloud Asset Inventory detects changes in your cloud resources. These events are forwarded to Datadog through a Cloud Pub/Sub topic and subscription.

For detailed setup instructions, see the resource changes collection section of the Google Cloud integration documentation.

Leveraging the Datadog Agent

After the Google Cloud integration is configured, Datadog automatically starts collecting Google Cloud metrics. However, you can leverage the Datadog Agent to gather deeper insights into your infrastructure.

The Datadog Agent provides the most granular, low-latency metrics from your infrastructure, delivering real-time insights into CPU, memory, disk usage, and more for your Google Cloud hosts. The Agent can be installed on any host, including GKE.

The Agent also supports a wide range of integrations, enabling you to extend visibility into specific services and databases running on your hosts.

Traces collected through the Agent enable comprehensive Application Performance Monitoring (APM), helping you understand end-to-end service performance.

Logs collected through the Agent provide visibility into your Google Cloud resources, and the activities taking place in your Google Cloud environment.

For the full list of benefits of installing the Agent on your cloud instances, see Why should I install the Datadog Agent on my cloud instances?

Private Service Connect

Private Service Connect is only available for the US5 and EU Datadog sites.

Use the Google Cloud Private Service Connect integration to visualize connections, data transferred, and dropped packets through Private Service Connect. This gives you visibility into important metrics from your Private Service Connect connections, both for producers as well as consumers. Private Service Connect (PSC) is a Google Cloud networking product that enables you to access Google Cloud services, third-party partner services, and company-owned applications directly from your Virtual Private Cloud (VPC).

See Access Datadog privately and monitor your Google Cloud Private Service Connect usage in the Datadog blog for more information.

Google Cloud Run

Use the Google Cloud Run integration to get detailed information on your Cloud Run containers, such as metrics and audit logs.

Cloud Cost Management (CCM)

Datadog’s Google Cloud Cost Management provides insights for engineering and finance teams to understand how infrastructure changes impact costs, allocate spend across your organization, and identify potential improvements.

Security

Cloud SIEM

Cloud SIEM provides real-time analysis of operational and security logs, while using out-of-the-box integrations and rules to detect and investigate threats. To use this feature, see Getting Started with Cloud SIEM.

To view security findings from Google Cloud Security Command Center in Cloud SIEM, toggle the Enable collection of security findings option under the Security Findings tab & follow the setup instructions on the Google Cloud Security Command Center guide.

The security findings tab in the Google Cloud integration tile

Cloud Security Management

Datadog Cloud Security Management (CSM) delivers real-time threat detection and continuous configuration audits across your entire cloud infrastructure. Check out the Setting up Cloud Security Management guide to get started.

After setting up CSM, toggle the Enable Resource Collection option under the Resource Collection tab to start collecting configuration data for the Resource Catalog and CSM. Then, follow these instructions to enable Misconfigurations and Identity Risks (CIEM) on Google Cloud.