C# Log Collection

Docs > Log Management > Log Collection and Integrations > C# Log Collection

To send your C# logs to Datadog, use one of the following approaches:

Log to a file and then tail that file with your Datadog Agent.
Enable Agentless logging.
Use the Serilog sink.

File-tail logging with the Datadog Agent

The recommended approach for C# log collection is to output your logs to a file and then tail that file with your Datadog Agent. This enables the Datadog Agent to enrich the logs with additional metadata.

Datadog strongly encourages setting up your logging library to produce your logs in JSON format to avoid the need for custom parsing rules.

File-tail logging supports the following frameworks:

Serilog
NLog
log4net

Configure your logger

Like many other libraries for .NET, Serilog provides diagnostic logging into files, the console, and elsewhere. It has a clean API and is portable between recent .NET platforms.

Unlike other logging libraries, Serilog is built with powerful structured event data in mind.

To install Serilog with NuGet, run the following command in the Package Manager Console:

PM> Install-Package Serilog.Sinks.File

Then, add the following code to initialize the logger directly in your application:

// Instantiate the logger
var log = new LoggerConfiguration()  // using Serilog;

    // using Serilog.Formatting.Json;
    .WriteTo.File(new JsonFormatter(renderMessage: true), "log.json")

    // using Serilog.Formatting.Compact;
    // .WriteTo.File(new RenderedCompactJsonFormatter(), "log.json")

    .CreateLogger();

// An example
var position = new { Latitude = 25, Longitude = 134 };
var elapsedMs = 34;

log.Information("Processed {@Position} in {Elapsed:000} ms.", position, elapsedMs);

In the log.json file, confirm the logger instantiated successfully:

If using JsonFormatter(renderMessage: true), look for the following event for confirmation:

{
  "MessageTemplate": "Processed {@Position} in {Elapsed:000} ms.",
  "Level": "Information",
  "Timestamp": "2016-09-02T15:02:29.648Z",
  "Renderings": {"Elapsed": [{"Format": "000", "Rendering": "034"}]},
  "RenderedMessage":"Processed { Latitude: 25, Longitude: 134 } in 034 ms.",
  "Properties": {"Position": {"Latitude": 25, "Longitude": 134}, "Elapsed": 34}
}

If using RenderedCompactJsonFormatter(), look for the following event for confirmation:

{
  "@t": "2020-05-20T04:15:28.6898801Z",
  "@m": "Processed { Latitude: 25, Longitude: 134 } in 034 ms.",
  "@i": "d1eb2146",
  "Position": {"Latitude": 25, "Longitude": 134 },
  "Elapsed": 34
}

NLog is a logging platform for .NET with rich log routing and management capabilities. It can help you produce and manage high-quality logs for your application regardless of its size or complexity.

To install NLog using NuGet, run the following command in the Package Manager Console:

PM> Install-Package NLog

Once the library is in your classpath, attach the following layout to any target. Edit or add a NLog.config file to the project root path. Then copy/paste the following code in it (Logs are written into the application-logs.json file):

<?xml version="1.0" encoding="utf-8" ?>
<nlog xmlns="http://www.nlog-project.org/schemas/NLog.xsd"
      xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">

  <!--
  See https://github.com/nlog/nlog/wiki/Configuration-file
  for information on customizing logging rules and outputs.
   -->
  <targets async="true">
    <!-- Write logs as Json into a file -->
    <target name="json-file" xsi:type="File" fileName="application-logs.json">
      <layout xsi:type="JsonLayout">
        <attribute name="date" layout="${date:universalTime=true:format=o}" />
        <attribute name="level" layout="${level:upperCase=true}"/>
        <attribute name="message" layout="${message}" />
        <attribute name="exception" layout="${exception:format=ToString}" />
      </layout>
    </target>

  </targets>
  <rules>
    <!-- Log all events to the json-file target -->
    <logger name="*" writeTo="json-file" minlevel="Trace" />
  </rules>
</nlog>

To fire and log your first events, add this to your code:

using NLog;

namespace Datadog
{
    class Program
    {
        // Initialize a logger
        private static Logger logger = LogManager.GetCurrentClassLogger();

        static void Main(string[] args)
        {
            // Log a simple debug message
            logger.Debug("This is my first step");

            // your code continues here ...
        }
    }
}

Log4Net is a logging platform for .NET inspired from Log4j with rich log routing and management capabilities. It can help you produce and manage high-quality logs for your application regardless of its size or complexity.

To install Log4Net, run the following command in the Package Manager Console:

PM> Install-Package log4net
PM> Install-Package log4net.Ext.Json

Once the library is installed, attach the following layout to any target. Edit the App.config of your project and add the following section:

<?xml version="1.0" encoding="utf-8"?>
<configuration>

  <configSections>
    <section name="log4net" type="log4net.Config.Log4NetConfigurationSectionHandler, log4net" />
  </configSections>

  <log4net>
    <root>
      <level value="DEBUG" />
      <appender-ref ref="JsonFileAppender" />
    </root>
    <appender name="JsonFileAppender" type="log4net.Appender.FileAppender">
      <threshold value="DEBUG"/>
      <file value="application-logs.json" />
      <encoding type="System.Text.UTF8Encoding" />
      <appendToFile value="true" />
      <layout type="log4net.Layout.SerializedLayout, log4net.Ext.Json">
        <decorator type="log4net.Layout.Decorators.StandardTypesDecorator, log4net.Ext.Json" />
        <default />
        <!--explicit default members-->
        <remove value="ndc" />
        <remove value="message" />
        <!--remove the default preformatted message member-->
        <member value="message:messageobject" />
        <!--add raw message-->
      </layout>
    </appender>
  </log4net>

  <!-- The rest of your configuration starts here ... -->

Instantiate your logger and start to fire your events:

using log4net;

namespace Datadog
{
    class Program
    {
        // Get the current class logger
        private static ILog logger = LogManager.GetLogger(typeof(Program));

        static void Main(string[] args)
        {

           // Load the configure fom App.config
           XmlConfigurator.Configure();

           // Log a simple debug message
           logger.Debug("This is my first debug message");

           // your code continues here ...
        }
    }
}

If you have followed the instructions you should see in your file (for example C:\Projects\Datadog\Logs\log.json) the following event:

{
  "level": "DEBUG",
  "message": "This is my debug message",
  "date": "2016-05-24 15:53:35.7175",
  "appname": "Datadog.vshost.exe",
  "logger": "Datadog.Program",
  "thread": "10"
}

If, despite the benefits of logging in JSON, you wish to log in raw string format, try updating the log4net conversion pattern to automatically parse your logs with the C# integration Pipeline as follows:

<param name="ConversionPattern" value="%date{yyyy-MM-dd HH:mm:ss.SSS} %level [%thread] %logger %method:%line - %message%n" />

Configure the Datadog Agent

Once log collection is enabled, set up custom log collection to tail your log files and send them to Datadog.

Create a csharp.d/ folder in the conf.d/ Agent configuration directory.

Create a conf.yaml file in csharp.d/ with the following content:

init_config:

instances:

##Log section
logs:

  - type: file
    path: "<path_to_your_csharp_log>.log"
    service: <service_name>
    source: csharp
    sourcecategory: sourcecode
    # For multiline logs, if they start by the date with the format yyyy-mm-dd uncomment the following processing rule
    #log_processing_rules:
    #  - type: multi_line
    #    name: new_log_start_with_date
    #    pattern: \d{4}\-(0?[1-9]|1[012])\-(0?[1-9]|[12][0-9]|3[01])

Make sure the Agent user has read access permissions to the log file.
Restart the Agent.
Run the Agent’s status subcommand and look for csharp under the Checks section to confirm logs are successfully submitted to Datadog.

If logs are in JSON format, Datadog automatically parses the log messages to extract log attributes. Use the Log Explorer to view and troubleshoot your logs.

Connect your service across logs and traces

If APM is enabled for this application, connect your logs and traces by automatically adding trace IDs, span IDs, env, service, and version to your logs by following the APM .NET instructions

Note: If the APM tracer injects service into your logs, it overrides the value set in the agent configuration.

Agentless logging with APM

It is possible to stream logs from your application to Datadog directly, without making any code changes, using the .NET APM automatic instrumentation library. This approach sends logs directly to Datadog, so it does not benefit from features such as sensitive data scrubbing which are provided by the Datadog Agent. For that reason, we recommend using file tail logging where possible, but it is useful in environments where this is not possible (when using Azure App Service for example). It is worth noting that you will still be able to rely on server-side scrubbing capabilities performed by Sensitive Data Scanner.

Agentless logging (also known as “direct log submission”) supports the following frameworks:

Serilog (v1.0+)
NLog (v2.1+)
log4net (v1.0+)
Microsoft.Extensions.Logging (2.0+)

It does not require modifying your application code, or installing additional dependencies into your application.

Note: If you use log4net or NLog, an appender (log4net) or a logger (NLog) must be configured for Agentless logging to be enabled. In those cases, you can either add these extra dependencies, or use agentless logging with the Serilog sink instead.

Configure the APM library

Agentless logging is only available when using APM with automatic instrumentation. To get started, instrument your application as described in the following documents:

After installing, verify you are receiving traces correctly.

Enable Agentless logging

To enable Agentless logging, set the following environment variables:

DD_API_KEY: Your Datadog API Key for sending your logs to Datadog.
DD_SITE: The name of your Datadog site. Choose from one of the following examples:
Example: datadoghq.com (US1), datadoghq.eu (EU), us3.datadoghq.com (US3), us5.datadoghq.com (US5), ddog-gov.com (US1-FED)
Default: datadoghq.com (US1)
DD_LOGS_INJECTION: Enables connecting logs and traces:
Default: true
Enabled by default when using Agentless logging from Tracer version 2.7.0.
DD_LOGS_DIRECT_SUBMISSION_INTEGRATIONS: Enables Agentless logging. Enable for your logging framework by setting to Serilog, NLog, Log4Net, or ILogger (for Microsoft.Extensions.Logging). If you are using multiple logging frameworks, use a semicolon separated list of variables.
Example: Serilog;Log4Net;NLog

Note: If you are using a logging framework in conjunction with Microsoft.Extensions.Logging, you will generally need to use the framework name. For example, if you are using Serilog.Extensions.Logging, you should set DD_LOGS_DIRECT_SUBMISSION_INTEGRATIONS=Serilog.

Restart your application after setting these environment variables.

Additional configuration

You can further customize some aspects of Agentless log collection using the following environment variables:

DD_LOGS_DIRECT_SUBMISSION_MINIMUM_LEVEL: Allows filtering logs by level before they’re sent to Datadog. Set to one of the following values: Verbose, Debug, Information, Warning, Error, Critical. These correspond to the equivalent levels in the supported logging frameworks.
Default: Information
DD_LOGS_DIRECT_SUBMISSION_HOST: Set the name of the host machine associated with logs. If not provided, the host name will attempt to be found automatically.
Default: Determined automatically
DD_LOGS_DIRECT_SUBMISSION_TAGS: If specified, adds all of the specified tags to all generated spans. If not provided, will use DD_TAGS instead.
Example: layer:api, team:intake Note that the delimiter is a comma and a whitespace: , .

The following configuration values should generally not be modified, but may be set if required.

DD_LOGS_DIRECT_SUBMISSION_SOURCE: Sets the parsing rule for submitted logs. Should always be set to csharp, unless you have a custom pipeline.
Default: csharp
DD_LOGS_DIRECT_SUBMISSION_MAX_BATCH_SIZE: Sets the maximum number of logs to send at one time. Takes into account the limits in place for the API.
Default: 1000
DD_LOGS_DIRECT_SUBMISSION_MAX_QUEUE_SIZE: Sets the maximum number of logs to hold in the internal queue at any one time before dropping log messages.
Default: 100000
DD_LOGS_DIRECT_SUBMISSION_BATCH_PERIOD_SECONDS: Sets the time to wait (in seconds) before checking for new logs to send.
Default: 1

If you are using the Microsoft.Extensions.Logging integration, you can filter the logs sent to Datadog using the standard capabilities built-into ILogger. Use the key "Datadog" to identify the direct-submission provider, and set the minimum log levels for each namespace. For example, adding the following to your appSettings.json would prevent sending any logs with a level below Warning to Datadog. Introduced in the .NET tracer library v2.20.0.

{
  "Logging": {
    "Datadog": {
      "LogLevel": {
        "Microsoft.AspNetCore": "Warning"
      },
    }
  }
}

Agentless logging with Serilog sink

Since 0.2.0, you can configure the Datadog sink by using an appsettings.json file with the Serilog.Setting.Configuration package. For more information, see the `Serilog.Sinks.Datadog.Logs` package.

If it is not possible to use file-tail logging or APM Agentless logging, and you are using the Serilog framework, then you can use the Datadog Serilog sink to send logs directly to Datadog.

Install the Datadog Serilog sink into your application, which sends events and logs to Datadog. By default the sink forwards logs through HTTPS on port 443. Run the following command in the Package Manager Console:

PM> Install-Package Serilog.Sinks.Datadog.Logs

Then, initialize the logger directly in your application. Ensure that you add your <API_KEY>.

using (var log = new LoggerConfiguration()
    .WriteTo.DatadogLogs("<API_KEY>", configuration: new DatadogConfiguration(){ Url = "" })
    .CreateLogger())
{
    // Some code
}

You can also override the default behavior and forward logs in TCP by manually specifying the following required properties: url, port, useSSL, and useTCP. Optionally, specify the source, service, host, and custom tags.

For instance to forward logs to the Datadog US region in TCP you would use the following sink configuration:

var config = new DatadogConfiguration(url: "intake.logs.datadoghq.com", port: 10516, useSSL: true, useTCP: true);
using (var log = new LoggerConfiguration()
    .WriteTo.DatadogLogs(
        "<API_KEY>",
        source: "<SOURCE_NAME>",
        service: "<SERVICE_NAME>",
        host: "<HOST_NAME>",
        tags: new string[] {"<TAG_1>:<VALUE_1>", "<TAG_2>:<VALUE_2>"},
        configuration: config
    )
    .CreateLogger())
{
    // Some code
}

For instance to forward logs to the Datadog EU region in TCP you would use the following sink configuration:

var config = new DatadogConfiguration(url: "tcp-intake.logs.datadoghq.eu", port: 443, useSSL: true, useTCP: true);
using (var log = new LoggerConfiguration()
    .WriteTo.DatadogLogs(
        "<API_KEY>",
        source: "<SOURCE_NAME>",
        service: "<SERVICE_NAME>",
        host: "<HOST_NAME>",
        tags: new string[] {"<TAG_1>:<VALUE_1>", "<TAG_2>:<VALUE_2>"},
        configuration: config
    )
    .CreateLogger())
{
    // Some code
}