To verify you have access to JMX, test using JConsole or equivalent if possible. If you’re unable to connect using JConsole this article may help to get you sorted. Also, if the metrics listed in your YAML aren’t 1:1 with those listed in JConsole you’ll need to correct this.

If you’re able to connect using JConsole, run the following:

java -jar /opt/datadog-agent/agent/checks/libs/jmxterm-1.0-DATADOG-uber.jar -l localhost:<PORT> -u <USER> -p <PASSWORD>

If you’re able to connect using the command above, run: beans and send to the Datadog support team a copy of the results from above along with the following information:

Note: If you’re able to see some metrics (jvm.heap_memory, jvm.non_heap_memory, etc.) it is a sign that JMXFetch is properly running. If you’re targeting another application and not seeing related metrics, the likely issue is a misconfiguration in your YAML.

Agent troubleshooting

/usr/bin/java -Xmx200m -Xms50m -classpath /usr/lib/jvm/java-8-oracle/lib/tools.jar:/opt/datadog-agent/bin/agent/dist/jmx/jmxfetch-0.18.2-jar-with-dependencies.jar org.datadog.jmxfetch.App --check <CHECK_LIST> --conf_directory /etc/datadog-agent/conf.d --log_level INFO --log_location /var/log/datadog/jmxfetch.log --reporter console <COMMAND>

/usr/bin/java -Xmx200m -Xms50m -classpath /usr/lib/jvm/java-8-oracle/lib/tools.jar:/opt/datadog-agent/bin/agent/dist/jmx/jmxfetch-0.18.2-jar-with-dependencies.jar org.datadog.jmxfetch.App --check cassandra.d/conf.yaml jmx.d/conf.yaml --conf_directory /etc/datadog-agent/conf.d --log_level INFO --log_location /var/log/datadog/jmxfetch.log --reporter console list_everything

$ docker exec -it <AGENT_CONTAINER_NAME> agent configcheck

$ docker exec -it <AGENT_CONTAINER_NAME> agent status

FAQ

The 350 metric limit

Datadog accepts a maximum of 350 metrics. A best practice is to limit your metrics to less than 350 by creating filters to refine those metrics collected; but if you need more than 350 metrics, it is possible to increase this limit by modifying a parameter in your JMX config file.

Please contact Datadog support if you would like to increase this limit.

Java path

The default Agent installation does not come with a bundled JVM and uses the one installed on your system. Therefore you must make sure that the Java home directory is present in the path of the user running the Agent.

Notes:

• The gcr.io/datadoghq/agent:latest-jmx Docker image does include a JVM, which the Agent needs to run jmxfetch. Alternatively, you can specify the JVM path in the integration’s configuration file with the java_bin_path parameter.
• Only one valid Java path needs to be specified for JMXFetch.

JVM metrics

Datadog’s Java APM library is capable of collecting JVM metrics without the JMX integration. See Runtime Metrics, for more details.

Monitoring JBoss or WildFly applications

The following instructions work on Agent v5.6.0+.

JBoss/WildFly applications expose JMX over a specific protocol (Remoting JMX) that is not bundled by default with JMXFetch. To allow JMXFetch to connect to these applications, configure it as follows:

• Locate the jboss-cli-client.jar file on your JBoss/WildFly server (by default, its path should be $JBOSS_HOME/bin/client/jboss-cli-client.jar).
• If JMXFetch is running on a different host than the JBoss/WildFly application, copy jboss-cli-client.jar to a location on the host JMXFetch is running on.
• Add the path of the jar to the init_config section of your configuration:

  # Datadog Agent >= 5.6.0

  init_config:
    custom_jar_paths:
      - /path/to/jboss-cli-client.jar

• Specify a custom URL that JMXFetch connects to, in the instances section of your configuration:

  # Datadog Agent >= 5.6.0

  # The jmx_url may be different depending on the version of JBoss/WildFly you're using
  # and the way you've set up JMX on your server
  # Refer to the relevant documentation of JBoss/WildFly for more information
  instances:
    - jmx_url: "service:jmx:remote://localhost:4447"
      name: jboss-application  # Mandatory, but can be set to any value,
                               # is used to tag the metrics pulled from that instance

• Restart the Agent.

Note: If you run into warning message like Unable to instantiate or initialize instance <instance_name> for an unknown reason.Parameter 'name' may not be null, you may need to set a wildfly user with $WILDFLY_HOME/bin/add-user.sh -a -u <user> -p <password> and specify user and password in the instances section of your configuration:

instances:
  - jmx_url: <jmx_url>
    name: <instance_name>
    user: <username>
    password: <password>

Monitoring Tomcat with JMX remote lifecycle listener enabled

The following instructions work on Agent v5.6.0+.

If you’re using Tomcat with JMX remote lifecycle listener enabled (see the Tomcat documentation for more information), JMXFetch needs an additional setup to be able to connect to your Tomcat application.

• Locate the catalina-jmx-remote.jar file on your Tomcat server (by default, its path should be $CATALINA_HOME/lib).
• If JMXFetch is running on a different host than the Tomcat application, copy catalina-jmx-remote.jar to a location on the host JMXFetch is running on.
• Add the path of the jar to the init_config section of your configuration:

# Datadog Agent >= 5.6.0

init_config:
  custom_jar_paths:
    - /path/to/catalina-jmx-remote.jar

• Specify a custom URL that JMXFetch connects to, in the instances section of your configuration:

# Datadog Agent >= 5.6.0

# The jmx_url may be different depending on the way you've set up JMX on your Tomcat server
instances:
  - jmx_url: "service:jmx:rmi://:10002/jndi/rmi://:10001/jmxrmi"
    name: tomcat-application  # Mandatory, but can be set to any arbitrary value,
                              # is used to tag the metrics pulled from that instance

• Restart the Agent.

SSL troubleshooting

Once JMX is enabled and your Agent check is successfully sending metrics to Datadog, you can secure the remote connection over an SSL Socket.

Note: You cannot secure JMX over SSL without using the JMX remote user/password authentication files. If you are using system level permissions to run your application, add these files and run them at startup.

This example shows the Datadog configuration for the Tomcat integration.

• Establish a certificate and key to apply to your Java app keystore.
• Update your Datadog Tomcat conf.yaml file located in conf.d/tomcat.d:

instances:
  - host: localhost
    port: 9000
    user: tomcat
    password: tomcat
    name: tomcat_webapp
    trust_store_path: <KEYSTORE_PATH>
    trust_store_password: <KEY_PASSWORD>

• Restart the Agent.

Further Reading

Additional helpful documentation, links, and articles:

Java integrationDOCUMENTATION