Note: For .NET library injection, if the application container uses a musl-based Linux distribution (such as Alpine), you must specify a tag with the -musl suffix for the pod annotation. For example, to use library version v2.29.0, specify container tag v2.29.0-musl.
Note: If you already have an application instrumented using version X of the library, and then use library injection to instrument using version Y of the same tracer library, the tracer does not break. Rather, the library version loaded first is used. Because library injection happens at the admission controller level prior to runtime, it takes precedence over manually configured libraries.
In Python < 1.20.3, Python injection logs output to stderr. Upgrade to 1.20.3 or above to suppress the logs by default. The logs can be enabled by setting the environment variable DD_TRACE_DEBUG to 1.
Problem: The ddtrace library is already installed on the system so the injection logic aborts injecting the library to avoid introducing a breaking change in the application.
Solution: Remove the installation of ddtrace if library injection is desired. Otherwise, use the installed library (see documentation) instead of library injection.
Problem: Since Cluster Agent v1.20, the API is only served using TLS v1.3 by default. If the Kubernetes cluster is configured with TLS v1.2 or older, library injection fails.
Solution: Set DD_CLUSTER_AGENT_ALLOW_LEGACY_TLS to true for Cluster Agent.
Tracing Library Injection on a host is in beta.
When both the Agent and your services are running on a host, real or virtual, Datadog injects the tracing library by using a preload library that overrides calls to execve. Any newly started processes are intercepted and the specified instrumentation library is injected into the services.
By default, running the script installs support for Java, Node.js, Python, Ruby, and .NET all pinned to the latest major version. If you want to specify which language support is installed, also set the DD_APM_INSTRUMENTATION_LIBRARIES environment variable. The valid values are java, js, python, ruby, and dotnet. Use a comma-separated list to specify more than one language:
The following environment variables configure library injection. You can pass these in by export through the command line (export DD_CONFIG_SOURCES=BASIC), shell configuration, or launch command.
The DD_APM_INSTRUMENTATION_DEBUG environment variable is limited to the values true and false (default value false). Setting it to true sets log_level to debug and setting it to false (or not setting it at all) uses the log_level specified in the configuration file. The environment variable can only set the log level to debug, not any other log level values.
The DD_INSTRUMENT_SERVICE_WITH_APM environment variable controls whether or not injection is enabled. It defaults to TRUE. Set it to FALSE to turn off library injection altogether.
Log injection, assuming the application uses structured logging (usually JSON). For traces to appear in non-structured logs, you must change your application’s log configuration to include placeholders for trace ID and span ID. See Connect Logs and Traces for more information.
Tracing Library Injection on hosts and containers is in beta.
When your Agent is running on a host, and your services are running in containers, Datadog injects the tracing library by intercepting container creation and configuring the Docker container.
Any newly started processes are intercepted and the specified instrumentation library is injected into the services.
By default, running the script installs support for Java, Node.js, Python, Ruby, and .NET. If you want to specify which language support is installed, also set the DD_APM_INSTRUMENTATION_LIBRARIES environment variable. The valid values are java, js, python, ruby, and dotnet. Use a comma-separated list to specify more than one language:
Tracing Library Injection in containers is in beta.
When your Agent and services are running in separate Docker containers on the same host, Datadog injects the tracing library by intercepting container creation and configuring the Docker container.
Any newly started processes are intercepted and the specified instrumentation library is injected into the services.
This installs language libraries for all supported languages. To install specific languages, set the DD_APM_INSTRUMENTATION_LIBRARIES variable. The valid values are java, js, python, ruby, and dotnet:
The supported features and configuration options for the tracing library are the same for library injection as for other installation methods, and can be set with environment variables. Read the Datadog library configuration page for your language for more details.