Track Backend Error Logs

Overview

If you aren’t already collecting logs with Datadog, see the Logs documentation to set up logs. Ensure that the source tag (specifying language) is properly configured. Datadog recommends setting up Agent-based log collection.

Setup

For languages such as Python, Java, and Ruby, no additional configuration is needed if the source tag in your logs is configured correctly. All required attributes are automatically tagged and sent to Datadog.

For backend languages such as C#, .NET, Go, and Node.js, the code examples in each section demonstrate how to properly configure an error log and attach the required stack trace in the log’s error.stack.

If you are already sending stack traces to Datadog but they are not in error.stack, you can set up a generic log remapper to remap the stack trace to the correct attribute in Datadog.

To configure inline code snippets in issues, set up the source code integration. Adding code snippets in Error Tracking for Logs does not require APM; the enrichment tags and linked repository is the same for both.

Attributes for Error Tracking

To enable Error Tracking, logs must include the following attributes:

  • Either an error.kind or error.stack field. Note: if using error.stack, it must be a valid stack trace.
  • A Service attribute
  • A status level of ERROR, CRITICAL, ALERT, or EMERGENCY.

The remaining attributes listed below are optional, but their presence improves error grouping.

Specific attributes have a dedicated UI display within Datadog. To enable these functionalities for Error Tracking, use the following attribute names:

AttributeDescription
error.stackActual stack trace
error.messageError message contained in the stack trace
error.kindThe type or “kind” of an error (for example, “Exception”, or “OSError”)

Note: By default, integration Pipelines attempt to remap default logging library parameters to those specific attributes and parse stack traces or traceback to automatically extract the error.message and error.kind.

For more information, see the complete source code attributes documentation.

C# and .NET

If you have not set up log collection for C#, see the C# Log Collection documentation.

To log a caught exception yourself, you may optionally use:

var log = new LoggerConfiguration()
    .WriteTo.File(new JsonFormatter(renderMessage: true), "log.json")
    .Enrich.WithExceptionDetails()
    .CreateLogger();
try {
  // …
} catch (Exception ex) {
  // pass exception as first argument of log call
  log.Error(ex, "an exception occurred");
}

If you have not set up log collection for C#, see the C# Log Collection documentation.

To log a caught exception yourself, you may optionally use:

private static Logger log = LogManager.GetCurrentClassLogger();

static void Main(string[] args)
{
  try {
    // …
  } catch (Exception ex) {
    // pass exception as second argument of log call
    log.ErrorException("an exception occurred", ex);
  }
}

If you have not set up log collection for C#, see the C# Log Collection documentation.

To log a caught exception yourself, you may optionally use:

class Program
{
  private static ILog logger = LogManager.GetLogger(typeof(Program));

  static void Main(string[] args)
  {
    try {
      // …
    } catch (Exception ex) {
      // pass exception as second argument of log call
      log.Error("an exception occurred", ex);
    }
  }
}

Go

Logrus

If you have not set up log collection for Go, see the Go Log Collection documentation.

To log a caught exception yourself, you may optionally use:

// for https://github.com/pkg/errors
type stackTracer interface {
	StackTrace() errors.StackTrace
}

type errorField struct {
  Kind    string `json:"kind"`
  Stack   string `json:"stack"`
  Message string `json:"message"`
}

func ErrorField(err error) errorField {
    var stack string
	if serr, ok := err.(stackTracer); ok {
        st := serr.StackTrace()
		stack = fmt.Sprintf("%+v", st)
		if len(stack) > 0 && stack[0] == '\n' {
			stack = stack[1:]
		}
    }
    return errorField{
        Kind: reflect.TypeOf(err).String(),
        Stack: stack,
        Message: err.Error(),
    }
}


log.WithFields(log.Fields{
    "error": ErrorField(err)
}).Error("an exception occurred")

Java (parsed)

If you have not set up log collection for Java, see the Java Log Collection documentation. Ensure your logs are tagged with source:java.

To log a caught exception yourself, you may optionally use:

Logger logger = LogManager.getLogger("HelloWorld");
try {
  // …
} catch (Exception e) {
  // pass exception as last argument of log call
  logger.error("an exception occurred", e)
}

To log a caught exception yourself, you may optionally use:

Logger logger = LoggerFactory.getLogger(NameOfTheClass.class);
try {
  // …
} catch (Exception e) {
  // pass exception as last argument of log call
  logger.error("an exception occurred", e)
}

Node.js

Winston (JSON)

If you have not set up log collection for Node.js, see the Node.js Log Collection documentation.

To log a caught exception yourself, you may optionally use:

try {
  // …
} catch (e) {
  logger.error("an exception occurred", {
    error: {
      message: e.message,
      stack: e.stack
    }
  });
}

PHP

Monolog (JSON)

If you have not set up log collection for PHP, see the PHP Log Collection documentation.

To log a caught exception yourself, you may optionally use:

try {
    // ...
} catch (\Exception $e) {
    $logger->error('An error occurred', [
        'error.message' => $e->getMessage(),
        'error.kind' => get_class($e),
        'error.stack' => $e->getTraceAsString(),
    ]);
}

Python

Logging

If you have not setup log collection for Python, see the Python Log Collection documentation. Ensure your logs are tagged with source:python.

To log a caught exception yourself, you may optionally use:

try:
  // …
except:
  logging.exception('an exception occurred')

Ruby on Rails

Custom logger formatter

If you have not set up log collection for Ruby on Rails, see the Ruby on Rails Log Collection documentation.

To manually log an error, create a formatter using JSON and map the exception values to the correct fields:

require 'json'
require 'logger'

class JsonWithErrorFieldFormatter < ::Logger::Formatter
    def call(severity, datetime, progname, message)
        log = {
            timestamp: "#{datetime.to_s}",
            level: severity,
        }

        if message.is_a?(Hash)
            log = log.merge(message)
        elsif message.is_a?(Exception)
            log['message'] = message.inspect
            log['error'] = {
                kind: message.class,
                message: message.message,
                stack: message.backtrace.join("\n"),
            }
        else
            log['message'] = message.is_a?(String) ? message : message.inspect
        end

        JSON.dump(log) + "\n"
    end
end

And use it in your logger:

logger = Logger.new(STDOUT)
logger.formatter = JsonWithErrorFieldFormatter.new

If you use Lograge, you can also set it up to send formatted error logs:

Rails.application.configure do
    jsonLogger = Logger.new(STDOUT) # STDOUT or file depending on your agent configuration
    jsonLogger.formatter = JsonWithErrorFieldFormatter.new

    # Replacing Rails default TaggedLogging logger with a new one with the json formatter.
    # TaggedLogging is incompatible with more complex json format messages
    config.logger = jsonLogger

    # Lograge config
    config.lograge.enabled = true
    config.lograge.formatter = Lograge::Formatters::Raw.new

    # Disables log coloration
    config.colorize_logging = false

    # Configure logging of exceptions to the correct fields
    config.lograge.custom_options = lambda do |event|
        if event.payload[:exception_object]
            return {
                level: 'ERROR',
                message: event.payload[:exception_object].inspect,
                error: {
                    kind: event.payload[:exception_object].class,
                    message: event.payload[:exception_object].message,
                    stack: event.payload[:exception_object].backtrace.join("\n")
                }
            }
        end
    end
end

Further Reading

Additional helpful documentation, links, and articles:

Make sense of application issues with Datadog Error TrackingBLOG more more Learn about the Error Tracking ExplorerDOCUMENTATION more more
PREVIEWING: domalessi/docs-10186