Instrument Azure App Service with serverless-init - Linux Containers

Overview

To instrument your Azure App Service containers with a sidecar, see Instrument Azure App Service.

This instrumentation method uses serverless-init and provides the following additional monitoring capabilities for containerized Linux Azure App Service workloads:

  • Fully distributed APM tracing using automatic instrumentation.
  • Customized APM service and trace views showing relevant Azure App Service metrics and metadata.
  • Support for manual APM instrumentation to customize spans.
  • Trace_ID injection into application logs.
  • Support for submitting custom metrics using DogStatsD.

Prerequisites

Make sure you have a Datadog API Key and are using a programming language supported by a Datadog tracing library.

Instrument your application

Dockerfile

Datadog publishes new releases of the serverless-init container image to Docker Hub, Google’s gcr.io, and AWS’s ECR:

dockerhub.iogcr.iopublic.ecr.aws
datadog/serverless-initgcr.io/datadoghq/serverless-initpublic.ecr.aws/datadog/serverless-init

Images are tagged based on semantic versioning, with each new version receiving three relevant tags:

  • 1, 1-alpine: use these to track the latest minor releases, without breaking changes
  • 1.x.x, 1.x.x-alpine: use these to pin to a precise version of the library
  • latest, latest-alpine: use these to follow the latest version release, which may include breaking changes

Add the following instructions and arguments to your Dockerfile.

# For alpine or arm64 builds, refer to the explanation section
COPY --from=datadog/serverless-init:1 / /app/
RUN chmod +x /app/dotnet.sh && /app/dotnet.sh

ENV DD_SERVICE=datadog-demo-run-dotnet
ENV DD_ENV=datadog-demo
ENV DD_VERSION=1
ENTRYPOINT ["/app/datadog-init"]
CMD ["dotnet", "helloworld.dll"]

Explanation

  1. Copy the Datadog serverless-init into your Docker image.

    COPY --from=datadog/serverless-init:1 / /app/
    
  2. Copy the Datadog .NET tracer into your Docker image. For linux/amd64, include the following:

    RUN chmod +x /app/dotnet.sh && /app/dotnet.sh
    

    For other architecture types, configure your Dockerfile like so:

    # For arm64 use datadog-dotnet-apm-2.57.0.arm64.tar.gz
    # For alpine use datadog-dotnet-apm-2.57.0-musl.tar.gz
    ARG TRACER_VERSION
    ADD https://github.com/DataDog/dd-trace-dotnet/releases/download/v${TRACER_VERSION}/datadog-dotnet-apm-${TRACER_VERSION}.tar.gz /tmp/datadog-dotnet-apm.tar.gz
    
    RUN mkdir -p /dd_tracer/dotnet/ && tar -xzvf /tmp/datadog-dotnet-apm.tar.gz -C /dd_tracer/dotnet/ && rm /tmp/datadog-dotnet-apm.tar.gz
    

    If you install the Datadog tracer library directly in your application, as outlined in the manual tracer instrumentation instructions, omit this step.

  3. (Optional) Add Datadog tags.

    ENV DD_SERVICE=datadog-demo-run-dotnet
    ENV DD_ENV=datadog-demo
    ENV DD_VERSION=1
    
  4. Change the entrypoint to wrap your application in the Datadog serverless-init process. Note: If you already have an entrypoint defined inside your Dockerfile, see the alternative configuration.

    ENTRYPOINT ["/app/datadog-init"]
    
  5. Execute your binary application wrapped in the entrypoint. Adapt this line to your needs.

    CMD ["dotnet", "helloworld.dll"]
    

Alternative configuration

If you already have an entrypoint defined inside your Dockerfile, you can instead modify the CMD argument.

# For alpine or arm64 builds, refer to tracer installation of the explanation section
COPY --from=datadog/serverless-init:1 / /app/
RUN chmod +x /app/dotnet.sh && /app/dotnet.sh

ENV DD_SERVICE=datadog-demo-run-dotnet
ENV DD_ENV=datadog-demo
ENV DD_VERSION=1
CMD ["/app/datadog-init", "dotnet", "helloworld.dll"]

If you require your entrypoint to be instrumented as well, you can swap your entrypoint and CMD arguments instead. For more information, see How serverless-init works.

# For alpine or arm64 builds, refer to tracer installation of the explanation section
COPY --from=datadog/serverless-init:1 / /app/
RUN chmod +x /app/dotnet.sh && /app/dotnet.sh

ENV DD_SERVICE=datadog-demo-run-dotnet
ENV DD_ENV=datadog-demo
ENV DD_VERSION=1
ENTRYPOINT ["/app/datadog-init"]
CMD ["your_entrypoint.sh", "dotnet", "helloworld.dll"]

As long as your command to run is passed as an argument to datadog-init, you will receive full instrumentation.

2. Configure your application

Once the container is built and pushed to your registry, the last step is to set the required environment variables for the Datadog Agent:

  • DD_API_KEY: Datadog API key, used to send data to your Datadog account. It should be configured as an Azure Secret for privacy and safety.
  • DD_SITE: Datadog endpoint and website. Select your site on the right side of this page. Your site is: datadoghq.com.
  • DD_TRACE_ENABLED: Set to true to enable tracing.

For more environment variables and their function, see Additional Configurations.

3. Results

Once the deployment is completed, your metrics and traces are sent to Datadog. In Datadog, navigate to Infrastructure->Serverless to see your serverless metrics and traces.

Deployment

To update your Datadog instrumentation with zero downtime, use deployment slots. You can create a workflow that uses GitHub Action for Azure CLI.

See the sample GitHub workflow.

Additional configurations

  • Advanced Tracing: The Datadog Agent already provides some basic tracing for popular frameworks. Follow the advanced tracing guide for more information.

  • Logs: If you use the Azure integration, your logs are already being collected. Alternatively, you can set the DD_LOGS_ENABLED environment variable to true to capture application logs through the serverless instrumentation directly.

  • Custom Metrics: You can submit custom metrics using a DogStatsD client. For monitoring Cloud Run and other serverless applications, use distribution metrics. Distributions provide avg, sum, max, min, and count aggregations by default. On the Metric Summary page, you can enable percentile aggregations (p50, p75, p90, p95, p99) and also manage tags. To monitor a distribution for a gauge metric type, use avg for both the time and space aggregations. To monitor a distribution for a count metric type, use sum for both the time and space aggregations.

  • Trace Sampling: To manage the APM traced request sampling rate for serverless applications, set the DD_TRACE_SAMPLE_RATE environment variable on the function to a value between 0.000 (no tracing of Container App requests) and 1.000 (trace all Container App requests).

Metrics are calculated based on 100% of the application’s traffic, and remain accurate regardless of any sampling configuration.

Environment Variables

VariableDescription
DD_API_KEYDatadog API Key - Required
DD_SITEDatadog site - Required
DD_LOGS_ENABLEDWhen true, send logs (stdout and stderr) to Datadog. Defaults to false.
DD_LOGS_INJECTIONWhen true, enrich all logs with trace data for supported loggers in Java, Node.js, .NET, and PHP. See additional docs for Python, Go, and Ruby.
DD_TRACE_SAMPLE_RATEControls the trace ingestion sample rate between 0.0 and 1.0.
DD_SERVICESee Unified Service Tagging.
DD_VERSIONSee Unified Service Tagging.
DD_ENVSee Unified Service Tagging.
DD_SOURCESee Unified Service Tagging.
DD_TAGSSee Unified Service Tagging.

Troubleshooting

If you are not receiving traces or custom metric data as expected, enable App Service logs to receive debugging logs.

Share the content of the Log stream with Datadog Support.

Further reading

Additional helpful documentation, links, and articles:

PREVIEWING: brett.blue/add-otel-integrations
Your Privacy Choices