Overview
The steps required for setting up Database Monitoring in Datadog vary based on the type of database you’re using (Postgres, MySQL, SQL Server, Oracle), and the host provider (self-hosted, AWS, Google Cloud SQL, Azure, or Oracle). To use Database Monitoring for any database on any host provider, you need the following:
- A Datadog Agent
- Host for your Datadog Agent
- Read-only access for your databases
Agent
The Datadog Agent is lightweight software that monitors system metrics such as CPU, memory, and network activity. It also connects to the database as a SQL user to collect data about database performance.
For self-hosted databases, you install the agent directly onto the host that is hosting your database. For cloud-managed databases such as Amazon RDS and Azure SQL, you configure the Agent to connect to your databases remotely.
Self-hosted databases
In a self-hosted setup, the Datadog Agent collects system metrics from the operating system host, database metrics directly from the database, and log events from database logs.
For self-hosted setups, you install the Agent directly onto the database host so that you have full visibility into the health of your system running the database process.
You grant the Agent read-only access to your database, and configure the integration. The Agent must log in as a user so it can run read-only queries on your database.
Instructions for setting up Database Monitoring with a self-hosted provider:
Cloud-managed databases
If your setup is cloud-managed (with providers such as Amazon RDS or Aurora, Google Cloud SQL, or Azure), you install the Agent on a separate host and configure it to connect to each managed instance.
Database Monitoring collects system metrics such as CPU, memory, disk usage, logs, and related telemetry directly from the cloud provider using the Datadog integration with that provider.
You can install the Agent on any cloud VM (for example, EC2) provided the Agent can connect to your database instances.
If you are not running your own Kubernetes cluster, Datadog recommends using your cloud provider’s orchestration tools. For example, you can use Amazon ECS to host the Datadog Agent, as the Agent already exists as a Docker container.
Kubernetes
If you are running your apps on Kubernetes, use the Datadog Cluster Agent with Database Monitoring, which can run cluster checks across your pods.
The Cluster Agent automatically distributes the database instances across a pool of Agents. This ensures that only one instance of each check runs, as opposed to each node-based Agent pod running this corresponding check. The Cluster Agent holds the configurations and dynamically dispatches them to node-based Agents. The Agents on each node connect to the Cluster Agent every 10 seconds and retrieve the configurations to run.
If an Agent stops reporting, the Cluster Agent removes it from the active pool and dispatches the configurations to other Agents. This ensures one (and only one) instance always runs even as nodes are added and removed from the cluster. This becomes important when you have a large number of database instances — the Cluster Agent spreads the cluster checks across the different nodes.
Aurora
If you are using Aurora, the Agent must be connected to the individual Aurora instance (not the cluster endpoint) because the Agent must connect directly to the host being monitored.
For monitoring Aurora databases, the Agent should not connect to the database through a proxy, load balancer, connection pooler such as pgbouncer, or the Aurora cluster endpoint. Each Datadog Agent must have knowledge of the underlying hostname and should run on a single host for its lifetime, even in cases of failover. Otherwise, the values of metrics become incorrect.
Further Reading
Additional helpful documentation, links, and articles:
