Database Monitoring provides deep visibility into your MySQL databases by exposing query metrics, query samples, explain plans, connection data, system metrics, and telemetry for the InnoDB storage engine.
The Agent collects telemetry directly from the database by logging in as a read-only user. Do the following setup to enable Database Monitoring with your MySQL database:
Database Monitoring for MariaDB is supported with known limitations.
Supported Agent versions
7.36.1+
Performance impact
The default Agent configuration for Database Monitoring is conservative, but you can adjust settings such as the collection interval and query sampling rate to better suit your needs. For most workloads, the Agent represents less than one percent of query execution time on the database and less than one percent of CPU.
Database Monitoring runs as an integration on top of the base Agent (see benchmarks).
Proxies, load balancers, and connection poolers
The Datadog Agent must connect directly to the host being monitored. For self-hosted databases, 127.0.0.1 or the socket is preferred. The Agent should not connect to the database through a proxy, load balancer, or connection pooler. If the Agent connects to different hosts while it is running (as in the case of failover, load balancing, and so on), the Agent calculates the difference in statistics between two hosts, producing inaccurate metrics.
Data security considerations
See Sensitive information for information about what data the Agent collects from your databases and how to ensure it is secure.
To collect query metrics, samples, and explain plans, enable the MySQL Performance Schema and configure the following Performance Schema Options, either on the command line or in configuration files (for example, mysql.conf):
Recommended. Enables tracking of a larger number of recent queries across all threads. If enabled it increases the likelihood of capturing execution details from infrequent queries.
Optional. Enables tracking recent query history per thread. If enabled it increases the likelihood of capturing execution details from infrequent queries.
Parameter
Value
Description
performance_schema
ON
Required. Enables the Performance Schema.
max_digest_length
4096
Required for collection of larger queries. If left at the default value then queries longer than 1024 characters will not be collected.
Recommended. Enables tracking of a larger number of recent queries across all threads. If enabled it increases the likelihood of capturing execution details from infrequent queries.
Optional. Enables tracking recent query history per thread. If enabled it increases the likelihood of capturing execution details from infrequent queries.
Note: A recommended practice is to allow the agent to enable the performance-schema-consumer-* settings dynamically at runtime, as part of granting the Agent access. See Runtime setup consumers.
The Datadog Agent requires read-only access to the database in order to collect statistics and queries.
The following instructions grant the Agent permission to login from any host using datadog@'%'. You can restrict the datadog user to be allowed to login only from localhost by using datadog@'localhost'. See the MySQL documentation for more info.
Starting from Agent v7.65, the Datadog Agent can collect schema information from MySQL databases. See the Collecting schemas section below for more info on how to grant the Agent permissions for this collection.
Store your password using secret management software such as Vault. You can then reference this password as ENC[<SECRET_NAME>] in your Agent configuration files: for example, ENC[datadog_user_database_password]. See Secrets Management for more information.
The examples on this page use datadog_user_database_password to refer to the name of the secret where your password is stored. It is possible to reference your password in plain text, but this is not recommended.
Installing the Datadog Agent also installs the MySQL check which is required for Database Monitoring on MySQL. If you haven’t already installed the Agent for your MySQL database host, see the Agent installation instructions.
To configure this check for an Agent running on a host:
Edit the mysql.d/conf.yaml file, in the conf.d/ folder at the root of your Agent’s configuration directory to start collecting your MySQL metrics and logs. See the sample mysql.d/conf.yaml for all available configuration options, including those for custom metrics.
Add this configuration block to your mysql.d/conf.yaml to collect MySQL metrics:
init_config:instances:- dbm:truehost:127.0.0.1port:3306username:datadogpassword:'ENC[datadog_user_database_password]'# from the CREATE USER step earlier
Note that the datadog user should be set up in the MySQL integration configuration as host: 127.0.0.1 instead of localhost. Alternatively, you may also use sock.
In addition to telemetry collected from the database by the Agent, you can also choose to send your database logs directly to Datadog.
By default MySQL logs everything in /var/log/syslog which requires root access to read. To make the logs more accessible, follow these steps:
Edit /etc/mysql/conf.d/mysqld_safe_syslog.cnf and comment out all lines.
Edit /etc/mysql/my.cnf to enable the desired logging settings. For example, to enable general, error, and slow query logs, use the following configuration:
Make sure the Agent has read access to the /var/log/mysql directory and all of the files within. Double-check your logrotate configuration to make sure these files are taken into account and that the permissions are correctly set.
In /etc/logrotate.d/mysql-server there should be something similar to:
Collecting logs is disabled by default in the Datadog Agent, enable it in your datadog.yaml file:
logs_enabled:true
Add this configuration block to your mysql.d/conf.yaml file to start collecting your MySQL logs:
logs:- type:filepath:"<ERROR_LOG_FILE_PATH>"source:mysqlservice:"<SERVICE_NAME>"- type:filepath:"<SLOW_QUERY_LOG_FILE_PATH>"source:mysqlservice:"<SERVICE_NAME>"log_processing_rules:- type:multi_linename:new_slow_query_log_entrypattern:"# Time:"# If mysqld was started with `--log-short-format`, use:# pattern: "# Query_time:"# If using mysql version <5.7, use the following rules instead:# - type: multi_line# name: new_slow_query_log_entry# pattern: "# Time|# User@Host"# - type: exclude_at_match# name: exclude_timestamp_only_line# pattern: "# Time:"- type:filepath:"<GENERAL_LOG_FILE_PATH>"source:mysqlservice:"<SERVICE_NAME>"# For multiline logs, if they start by the date with the format yyyy-mm-dd uncomment the following processing rule# log_processing_rules:# - type: multi_line# name: new_log_start_with_date# pattern: \d{4}\-(0?[1-9]|1[012])\-(0?[1-9]|[12][0-9]|3[01])# If the logs start with a date with the format yymmdd but include a timestamp with each new second, rather than with each log, uncomment the following processing rule# log_processing_rules:# - type: multi_line# name: new_logs_do_not_always_start_with_timestamp# pattern: \t\t\s*\d+\s+|\d{6}\s+\d{,2}:\d{2}:\d{2}\t\s*\d+\s+
It is common to configure a single Agent host to connect to multiple remote database instances (see Agent installation architectures for DBM). To connect to multiple hosts, create an entry for each host in the MySQL integration config.
In these cases, Datadog recommends limiting the number of instances per Agent to a maximum of 10 database instances to guarantee reliable performance.
Starting from Agent v7.65, the Datadog Agent can collect schema information from MySQL databases. To enable this feature, use the schemas_collection option. Note: Schemas are only collected for tables where the Agent has SELECT access.
To grant SELECT permissions to the Datadog user, use one of the following commands:
If the Agent must connect through a proxy such as the Cloud SQL Auth proxy, all telemetry is tagged with the hostname of the proxy rather than the database instance. Use the reported_hostname option to set a custom override of the hostname detected by the Agent.