Overview
This check collects Tomcat metrics, for example:
- Overall activity metrics: error count, request count, processing times, etc.
- Thread pool metrics: thread count, number of threads busy, etc.
- Servlet processing times
Setup
Installation
The Tomcat check is included in the Datadog Agent package, so you don’t need to install anything else on your Tomcat servers.
This check is JMX-based, so you need to enable JMX Remote on your Tomcat servers. Follow the instructions in Monitoring and Managing Tomcat.
Configuration
Host
To configure this check for an Agent running on a host:
Edit the tomcat.d/conf.yaml
file, in the conf.d/
folder at the root of your Agent’s configuration directory to collect Tomcat metrics and logs. See the sample tomcat.d/conf.yaml for all available configuration options.
Restart the Agent.
See the JMX Check documentation for a list of configuration options usable by all JMX-based checks.
List of metrics
The conf
parameter is a list of metrics to be collected by the integration. Only two keys are allowed:
include
(mandatory): A dictionary of filters. Any attribute that matches these filters is collected unless it also matches the exclude
filters (see below).exclude
(optional): A dictionary of filters. Attributes that match these filters are not collected.
For a given bean, metrics get tagged in the following manner:
mydomain:attr0=val0,attr1=val1
In this example, your metric is mydomain
(or some variation depending on the attribute inside the bean) and has the tags attr0:val0
, attr1:val1
, and domain:mydomain
.
If you specify an alias in an include
key that is formatted as camel case, it is converted to snake case. For example, MyMetricName
is shown in Datadog as my_metric_name
.
The attribute filter
The attribute
filter can accept two types of values:
A dictionary whose keys are attributes names (see below). For this case, you can specify an alias for the metric that becomes the metric name in Datadog. You can also specify the metric type as a gauge or counter. If you choose counter, a rate per second is computed for the metric.
conf:
- include:
attribute:
maxThreads:
alias: tomcat.threads.max
metric_type: gauge
currentThreadCount:
alias: tomcat.threads.count
metric_type: gauge
bytesReceived:
alias: tomcat.bytes_rcvd
metric_type: counter
A list of attributes names (see below). For this case, the metric type is a gauge, and the metric name is jmx.\[DOMAIN_NAME].\[ATTRIBUTE_NAME]
.
conf:
- include:
domain: org.apache.cassandra.db
attribute:
- BloomFilterDiskSpaceUsed
- BloomFilterFalsePositives
- BloomFilterFalseRatio
- Capacity
- CompressionRatio
- CompletedTasks
- ExceptionCount
- Hits
- RecentHitRate
Log collection
To submit logs to Datadog, Tomcat uses the log4j
logger. For versions of Tomcat before 8.0, log4j
is configured by default. For Tomcat 8.0+, you must configure Tomcat to use log4j
, see Using Log4j. In the first step of those instructions, edit the log4j.properties
file in the $CATALINA_BASE/lib
directory as follows:
log4j.rootLogger = INFO, CATALINA
# Define all the appenders
log4j.appender.CATALINA = org.apache.log4j.DailyRollingFileAppender
log4j.appender.CATALINA.File = /var/log/tomcat/catalina.log
log4j.appender.CATALINA.Append = true
# Roll-over the log once per day
log4j.appender.CATALINA.layout = org.apache.log4j.PatternLayout
log4j.appender.CATALINA.layout.ConversionPattern = %d{yyyy-MM-dd HH:mm:ss} %-5p [%t] %c{1}:%L - %m%n
log4j.appender.LOCALHOST = org.apache.log4j.DailyRollingFileAppender
log4j.appender.LOCALHOST.File = /var/log/tomcat/localhost.log
log4j.appender.LOCALHOST.Append = true
log4j.appender.LOCALHOST.layout = org.apache.log4j.PatternLayout
log4j.appender.LOCALHOST.layout.ConversionPattern = %d{yyyy-MM-dd HH:mm:ss} %-5p [%t] %c{1}:%L - %m%n
log4j.appender.MANAGER = org.apache.log4j.DailyRollingFileAppender
log4j.appender.MANAGER.File = /var/log/tomcat/manager.log
log4j.appender.MANAGER.Append = true
log4j.appender.MANAGER.layout = org.apache.log4j.PatternLayout
log4j.appender.MANAGER.layout.ConversionPattern = %d{yyyy-MM-dd HH:mm:ss} %-5p [%t] %c{1}:%L - %m%n
log4j.appender.HOST-MANAGER = org.apache.log4j.DailyRollingFileAppender
log4j.appender.HOST-MANAGER.File = /var/log/tomcat/host-manager.log
log4j.appender.HOST-MANAGER.Append = true
log4j.appender.HOST-MANAGER.layout = org.apache.log4j.PatternLayout
log4j.appender.HOST-MANAGER.layout.ConversionPattern = %d{yyyy-MM-dd HH:mm:ss} %-5p [%t] %c{1}:%L - %m%n
log4j.appender.CONSOLE = org.apache.log4j.ConsoleAppender
log4j.appender.CONSOLE.layout = org.apache.log4j.PatternLayout
log4j.appender.CONSOLE.layout.ConversionPattern = %d{yyyy-MM-dd HH:mm:ss} %-5p [%t] %c{1}:%L - %m%n
# Configure which loggers log to which appenders
log4j.logger.org.apache.catalina.core.ContainerBase.[Catalina].[localhost] = INFO, LOCALHOST
log4j.logger.org.apache.catalina.core.ContainerBase.[Catalina].[localhost].[/manager] =\
INFO, MANAGER
log4j.logger.org.apache.catalina.core.ContainerBase.[Catalina].[localhost].[/host-manager] =\
INFO, HOST-MANAGER
Then follow the remaining steps in the Tomcat docs for configuring log4j
.
By default, Datadog’s integration pipeline support the following conversion patterns:
%d{yyyy-MM-dd HH:mm:ss} %-5p %c{1}:%L - %m%n
%d [%t] %-5p %c - %m%n
Clone and edit the integration pipeline if you have a different format. See Logging in Tomcat for details on Tomcat logging capabilities.
Collecting logs is disabled by default in the Datadog Agent, enable it in your datadog.yaml
file:
Add this configuration block to your tomcat.d/conf.yaml
file to start collecting your Tomcat Logs:
logs:
- type: file
path: /var/log/tomcat/*.log
source: tomcat
service: "<SERVICE>"
#To handle multi line that starts with yyyy-mm-dd use the following pattern
#log_processing_rules:
# - type: multi_line
# name: log_start_with_date
# pattern: \d{4}\-(0?[1-9]|1[012])\-(0?[1-9]|[12][0-9]|3[01])
Change the path
and service
parameter values and configure them for your environment. See the sample tomcat.yaml for all available configuration options.
Restart the Agent.
Validation
Run the Agent’s status subcommand and look for tomcat
under the Checks section.
Data Collected
Metrics
tomcat.bytes_rcvd (gauge) | Bytes per second received by all request processors. Shown as byte |
tomcat.bytes_sent (gauge) | Bytes per second sent by all the request processors. Shown as byte |
tomcat.error_count (gauge) | The number of errors per second on all request processors. Shown as error |
tomcat.jdbc.connection_pool.active (gauge) | The number of established connections in the pool that are in use. |
tomcat.jdbc.connection_pool.idle (gauge) | The number of established connections in the pool that are idle. |
tomcat.jdbc.connection_pool.max_active (gauge) | The maximum number of open connections. |
tomcat.jdbc.connection_pool.max_idle (gauge) | The maximum number of idle connections. |
tomcat.jdbc.connection_pool.min_idle (gauge) | The minimum number of idle connections. |
tomcat.jdbc.connection_pool.size (gauge) | The number of established connections in the pool, idle and in use. |
tomcat.jsp.count (gauge) | The number of JSPs per second that have been loaded in the web module. Shown as page |
tomcat.jsp.reload_count (gauge) | The number of JSPs per second that have been reloaded in the web module. Shown as page |
tomcat.max_time (gauge) | The longest request processing time (in milliseconds). Shown as millisecond |
tomcat.min_time (gauge) | The shortest request processing time (in milliseconds). Shown as millisecond |
tomcat.processing_time (gauge) | The sum of request processing times across all requests handled by the request processors (in milliseconds) per second. |
tomcat.request_count (gauge) | The number of requests per second across all request processors. Shown as request |
tomcat.servlet.error_count (gauge) | The number of erroneous requests received by the servlet per second. Shown as error |
tomcat.servlet.max_time (gauge) | The maximum processing time of a request Shown as millisecond |
tomcat.servlet.min_time (gauge) | The minimum processing time of a request Shown as millisecond |
tomcat.servlet.processing_time (gauge) | The sum of request processing times across all requests to the servlet (in milliseconds) per second. |
tomcat.servlet.request_count (gauge) | The number of requests received by the servlet per second. Shown as request |
tomcat.string_cache.access_count (gauge) | The number of accesses to the string cache per second. Shown as get |
tomcat.string_cache.cache_size (gauge) | The size of the String cache Shown as byte |
tomcat.string_cache.hit_count (gauge) | The number of string cache hits per second. Shown as hit |
tomcat.string_cache.max_size (gauge) | The maximum size of the String cache Shown as byte |
tomcat.threads.busy (gauge) | The number of threads that are in use. Shown as thread |
tomcat.threads.count (gauge) | The number of threads managed by the thread pool. Shown as thread |
tomcat.threads.max (gauge) | The maximum number of allowed worker threads. Shown as thread |
tomcat.threads.min (gauge) | The minimum number of allowed worker threads. Shown as thread |
tomcat.web.cache.hit_count (gauge) | The number of web resource cache hits per second. Shown as hit |
tomcat.web.cache.lookup_count (gauge) | The number of lookups to the web resource cache per second. Shown as get |
Events
The Tomcat check does not include any events.
Service Checks
tomcat.can_connect
Returns CRITICAL
if the Agent is unable to connect to and collect metrics from the monitored Tomcat instance, WARNING
if no metrics are collected, and OK
otherwise.
Statuses: ok, critical, warning
Troubleshooting
Missing tomcat.*
metrics
The Datadog Agent collects JMX metrics with either Catalina
or Tomcat
as bean domain names with the Datadog Agent version 7.49.0 or later. Older versions only collect metrics with Catalina
as the bean domain name.
Standalone Tomcat deployments have metrics under domain Catalina
, but embedded Tomcat deployments (such as with Spring Boot) have metrics under domain Tomcat
.
If the Datadog Agent version is older than 7.49.0, and if the exposed Tomcat metrics are prefixed with a different bean domain name such as Tomcat
, copy the default metrics from the metrics.yaml
file to the conf
section of the tomcat.d/conf.yaml
file and modify the domain
filter to use the applicable bean domain name.
- include:
domain: Tomcat
type: ThreadPool
attribute:
maxThreads:
alias: tomcat.threads.max
metric_type: gauge
currentThreadCount:
alias: tomcat.threads.count
metric_type: gauge
currentThreadsBusy:
alias: tomcat.threads.busy
metric_type: gauge
See the JMX Check documentation for more detailed information.
Commands to view the available metrics
The datadog-agent jmx
command allows you to run troubleshooting commands on JMXFetch integrations. On Linux systems, you will need to prepend the command with sudo -u dd-agent
so that the Datadog Agent runs as the correct user.
datadog-agent jmx collect
Running datadog-agent jmx collect
starts the collection of metrics based on your current configuration and displays them in the console.
datadog-agent jmx list
The datadog-agent jmx list
has a number of available subcommands:
collected
- List attributes that will actually be collected by your current instance’s configuration.everything
- List every attribute available that has a type supported by JMXFetch.limited
- List attributes that match one of your instances’ configurations but that are not being collected because it would exceed the number of metrics that can be collected.matching
- List attributes that match at least one of your instances’ configurations.not-matching
- List attributes that don’t match any of your instances’ configurations.with-metrics
- List attributes and metrics data that match at least one of your instances’ configurations.with-rate-metrics
- List attributes and metrics data that match at least one of your instances’ configurations, including rates and counters.
Further Reading
Additional helpful documentation, links, and articles: