To send your logs to Datadog, log to a file and tail that file with your Datadog Agent.

Stack traces from typical Java logs are split into multiple lines, which makes them difficult to associate to the original log event. For example:

//4 events generated when only one is expected!
Exception in thread "main" java.lang.NullPointerException
        at com.example.myproject.Book.getTitle(Book.java:16)
        at com.example.myproject.Author.getBookTitles(Author.java:25)
        at com.example.myproject.Bootstrap.main(Bootstrap.java:14)

To address this issue, configure your logging library to produce your logs in JSON format. By logging to JSON, you:

• Ensure that the stack trace is properly wrapped into the log event.
• Ensure that all log event attributes (such as severity, logger name, and thread name) are properly extracted.
• Gain access to Mapped Diagnostic Context (MDC) attributes, which you can attach to any log event.
• Avoid the need for custom parsing rules.

The following instructions show setup examples for the Log4j, Log4j 2, and Logback logging libraries.

Configure your logger

JSON format

writer                     = json
writer.file                = log.json
writer.format              = LDJSON
writer.level               = info
writer.field.level         = level
writer.field.source        = {class}.{method}()
writer.field.message       = {message}
writer.field.dd.trace_id   = {context: dd.trace_id}
writer.field.dd.span_id    = {context: dd.span_id}
writer.field.dd.service    = {context: dd.service}
writer.field.dd.version    = {context: dd.version}
writer.field.dd.env        = {context: dd.env}

Inject trace IDs into your logs

If APM is enabled for this application, you can correlate logs and traces by enabling trace ID injection. See Connecting Java Logs and Traces for more information.

Raw format

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE log4j:configuration SYSTEM "log4j.dtd">
<log4j:configuration>

  <appender name="FILE" class="org.apache.log4j.FileAppender">
    <param name="File" value="logs/app.log"/>
    <param name="Append" value="true"/>

    <layout class="org.apache.log4j.PatternLayout">
      <param name="ConversionPattern" value="%d{yyyy-MM-dd HH:mm:ss} %-5p %c{1}:%L - %X{dd.trace_id} %X{dd.span_id} - %m%n"/>
    </layout>
  </appender>

  <root>
    <priority value="INFO"/>
    <appender-ref ref="FILE"/>
  </root>

</log4j:configuration>

<?xml version="1.0" encoding="UTF-8"?>
<Configuration>
  <Appenders>
    <File name="FILE" fileName="logs/app.log">
      <PatternLayout pattern="%d{yyyy-MM-dd HH:mm:ss} %-5p %c{1}:%L - %X{dd.trace_id} %X{dd.span_id} - %m%n"/>
    </File>
  </Appenders>

  <Loggers>
    <Root level="INFO">
      <AppenderRef ref="FILE"/>
    </Root>
  </Loggers>
</Configuration>

<configuration>
  <appender name="FILE" class="ch.qos.logback.core.FileAppender">
    <file>${dd.test.logfile}</file>
    <append>false</append>
    <immediateFlush>true</immediateFlush>

    <encoder>
      <pattern>%d{yyyy-MM-dd HH:mm:ss} %-5p %c{1}:%L - %X{dd.trace_id} %X{dd.span_id} - %m%n</pattern>
    </encoder>
  </appender>

  <root level="INFO">
    <appender-ref ref="FILE"/>
  </root>
</configuration>

writer          = file
writer.level    = debug
writer.format   = {level} - {message} - "dd.trace_id":{context: dd.trace_id} - "dd.span_id":{context: dd.span_id}
writer.file     = log.txt

Inject trace IDs into your logs

If APM is enabled for this application, you can correlate logs and traces by enabling trace ID injection. See Connecting Java Logs and Traces.

If you are not correlating logs and traces, you can remove the MDC placeholders (%X{dd.trace_id} %X{dd.span_id}) from the log patterns included in the above configuration examples.

Configure the Datadog Agent

Once log collection is enabled, set up custom log collection to tail your log files and send them to Datadog.

1. Create a java.d/ folder in the conf.d/ Agent configuration directory.

2. Create a conf.yaml file in java.d/ with the following content:

   #Log section
   logs:
   
     - type: file
       path: "<path_to_your_java_log>.log"
       service: <service_name>
       source: java
       sourcecategory: sourcecode
       # For multiline logs, if they start by the date with the format yyyy-mm-dd uncomment the following processing rule
       #log_processing_rules:
       #  - type: multi_line
       #    name: new_log_start_with_date
       #    pattern: \d{4}\-(0?[1-9]|1[012])\-(0?[1-9]|[12][0-9]|3[01])
   

3. Restart the Agent.

4. Run the Agent’s status subcommand and look for java under the Checks section to confirm logs are successfully submitted to Datadog.

If logs are in JSON format, Datadog automatically parses the log messages to extract log attributes. Use the Log Explorer to view and troubleshoot your logs.

Agentless logging

In the exceptional case where your application is running on a machine that cannot be accessed or cannot log to a file, it is possible to stream logs to Datadog or to the Datadog Agent directly. This is not the recommended setup, because it requires that your application handles connection issues.

To stream logs directly to Datadog:

1. Add the Logback logging library to your code, or bridge your current logger to Logback.
2. Configure Logback to send logs to Datadog.

Bridge from Java logging libraries to Logback

If you are not already using Logback, most common logging libraries can be bridged to Logback.

Configure Logback

Use the logstash-logback-encoder logging library along with Logback to stream logs directly to Datadog.

1. Configure a TCP appender in your logback.xml file. With this configuration, your api key is retrieved from the DD_API_KEY environment variable. Alternatively, you can insert your api key directly into the configuration file:

   <configuration>
     <appender name="FILE" class="ch.qos.logback.core.FileAppender">
       <file>logs/app.log</file>
       <encoder class="net.logstash.logback.encoder.LogstashEncoder" />
     </appender>
     <appender name="JSON_TCP" class="net.logstash.logback.appender.LogstashTcpSocketAppender">
       <destination>intake.logs.datadoghq.com:10516</destination>
       <keepAliveDuration>20 seconds</keepAliveDuration>
       <encoder class="net.logstash.logback.encoder.LogstashEncoder">
           <prefix class="ch.qos.logback.core.encoder.LayoutWrappingEncoder">
               <layout class="ch.qos.logback.classic.PatternLayout">
                   <pattern>${DD_API_KEY} %mdc{keyThatDoesNotExist}</pattern>
               </layout>
             </prefix>
       </encoder>
       <ssl />
     </appender>
   
     <root level="DEBUG">
       <appender-ref ref="FILE"/>
       <appender-ref ref="JSON_TCP" />
     </root>
   </configuration>
   

   <configuration>
     <appender name="FILE" class="ch.qos.logback.core.FileAppender">
       <file>logs/app.log</file>
       <encoder class="net.logstash.logback.encoder.LogstashEncoder" />
     </appender>
     <appender name="JSON_TCP" class="net.logstash.logback.appender.LogstashTcpSocketAppender">
       <destination>tcp-intake.logs.datadoghq.eu:443</destination>
       <keepAliveDuration>20 seconds</keepAliveDuration>
       <encoder class="net.logstash.logback.encoder.LogstashEncoder">
           <prefix class="ch.qos.logback.core.encoder.LayoutWrappingEncoder">
               <layout class="ch.qos.logback.classic.PatternLayout">
                   <pattern>${DD_API_KEY} %mdc{keyThatDoesNotExist}</pattern>
               </layout>
             </prefix>
       </encoder>
       <ssl />
     </appender>
   
     <root level="DEBUG">
       <appender-ref ref="FILE"/>
       <appender-ref ref="JSON_TCP" />
     </root>
   </configuration>
   

   Not supported.

   Note: %mdc{keyThatDoesNotExist} is added because the XML configuration trims whitespace. For more information about the prefix parameter, see the Logback documentation.

2. Add the Logstash encoder dependency to your pom.xml file:

   <dependency>
     <groupId>ch.qos.logback</groupId>
     <artifactId>logback-classic</artifactId>
     <version>1.2.9</version>
   </dependency>
   <dependency>
     <groupId>net.logstash.logback</groupId>
     <artifactId>logstash-logback-encoder</artifactId>
     <version>6.6</version>
   </dependency>
   

Getting further

Enrich your log events with contextual attributes.

Using the key value parser

The key value parser extracts any <KEY>=<VALUE> pattern recognized in any log event.

To enrich your log events in Java, you can re-write messages in your code and introduce <KEY>=<VALUE> sequences.

For instance if you have:

logger.info("Emitted 1001 messages during the last 93 seconds for customer scope prod30");

You can change it to:

logger.info("Emitted quantity=1001 messages during the last durationInMs=93180 ms for customer scope=prod30");

With the key value parser enabled, each pair is extracted from the JSON:

{
  "message": "Emitted quantity=1001 messages during the last durationInMs=93180 ms for customer scope=prod30",
  "scope": "prod30",
  "durationInMs": 93180,
  "quantity": 1001
}

So you can exploit scope as a field, and durationInMs and quantity as log measures.

MDC

Another option to enrich your logs is to use Java’s Mapped Diagnostic Contexts (MDC).

If you use SLF4J, use the following Java code:

...
MDC.put("scope", "prod30");
logger.info("Emitted 1001 messages during the last 93 seconds");
...

To generate this JSON:

{
  "message": "Emitted 1001 messages during the last 93 seconds",
  "scope": "prod30"
}

Note: MDC allows only string types, so don’t use them for numerical value metrics.

Further Reading

Additional helpful documentation, links, and articles:

Learn how to process your logsDOCUMENTATION Learn more about parsingDOCUMENTATION Learn how to explore your logsDOCUMENTATION Perform Log AnalyticsDOCUMENTATION Connect Logs and TracesDOCUMENTATION Log Collection Troubleshooting GuideFAQ How to collect, customize, and standardize Java logsBLOG Glossary entry for "tail"GLOSSARY