Overview
The APM integration with Real User Monitoring allows you to link requests from your web and mobile applications to their corresponding backend traces. This combination enables you to see your full frontend and backend data through one lens.
Use frontend data from RUM, as well as backend, infrastructure, and log information from trace ID injection to pinpoint issues anywhere in your stack and understand what your users are experiencing.
To start sending just your iOS application’s traces to Datadog, see iOS Trace Collection.
Usage
Prerequisites
- You have set up APM tracing on the services targeted by your RUM applications.
- Your services use an HTTP server.
- Your HTTP servers are using a library that supports distributed tracing.
- You have the following set up based on your SDK:
- With the Browser SDK, you have added the XMLHttpRequest (XHR) or Fetch resources on the RUM Explorer to your
allowedTracingUrls
. - With the Mobile SDK, you have added the Native or XMLHttpRequest (XHR) to your
firstPartyHosts
.
- You have a corresponding trace for requests to
allowedTracingUrls
or firstPartyHosts
.
Setup RUM
Note: Configuring RUM and Traces makes use of APM paid data in RUM, which may impact your APM billing.
Set up RUM Browser Monitoring.
Initialize the RUM SDK. Configure the allowedTracingUrls
initialization parameter with the list of internal, first-party origins called by your browser application.
For npm install:
import { datadogRum } from '@datadog/browser-rum'
datadogRum.init({
clientToken: '<CLIENT_TOKEN>',
applicationId: '<APPLICATION_ID>',
site: 'datadoghq.com',
// service: 'my-web-application',
// env: 'production',
// version: '1.0.0',
allowedTracingUrls: ["https://api.example.com", /https:\/\/.*\.my-api-domain\.com/, (url) => url.startsWith("https://api.example.com")],
sessionSampleRate: 100,
sessionReplaySampleRate: 100, // if not specified, defaults to 100
trackResources: true,
trackLongTasks: true,
trackUserInteractions: true,
})
For CDN install:
window.DD_RUM.init({
clientToken: '<CLIENT_TOKEN>',
applicationId: '<APPLICATION_ID>',
site: 'datadoghq.com',
// service: 'my-web-application',
// env: 'production',
// version: '1.0.0',
allowedTracingUrls: ["https://api.example.com", /https:\/\/.*\.my-api-domain\.com/, (url) => url.startsWith("https://api.example.com")],
sessionSampleRate: 100,
sessionReplaySampleRate: 100, // if not included, the default is 100
trackResources: true,
trackLongTasks: true,
trackUserInteractions: true,
})
To connect RUM to Traces, you need to specify your browser application in the service
field.
allowedTracingUrls
matches the full URL (<scheme>://<host>[:<port>]/<path>[?<query>][#<fragment>]
). It accepts the following types:
string
: matches any URL that starts with the value, so https://api.example.com
matches https://api.example.com/v1/resource
.RegExp
: executes a test with the provided RegExp and the URL.function
: evaluates with the URL as parameter. Returning a boolean
set to true
indicates a match.
(Optional) Configure the traceSampleRate
initialization parameter to keep a defined percentage of the backend traces. If not set, 100% of the traces coming from browser requests are sent to Datadog. To keep 20% of backend traces, for example:
import { datadogRum } from '@datadog/browser-rum'
datadogRum.init({
...otherConfig,
traceSampleRate: 20
})
Note: traceSampleRate
does not impact RUM sessions sampling. Only backend traces are sampled out.
(Optional) If you set a traceSampleRate
, to ensure backend services’ sampling decisions are still applied, configure the traceContextInjection
initialization parameter to sampled
(set to all
by default).
For example, if you set the traceSampleRate
to 20% in the Browser SDK:
- When
traceContextInjection
is set to all
, 20% of backend traces are kept and 80% of backend traces are dropped.
When traceContextInjection
is set to sampled
, 20% of backend traces are kept. For the remaining 80%, the browser SDK does not inject a sampling decision. The decision is made on the server side and is based on the tracing library head-based sampling configuration. In the example below, the backend sample rate is set to 40%, and therefore 32% of the remaining backend traces are kept.
End-to-end tracing is available for requests fired after the Browser SDK is initialized. End-to-end tracing of the initial HTML document and early browser requests is not supported.
Set up RUM Android Monitoring.
Set up Android Trace Collection.
Add the Gradle dependency to the dd-sdk-android-okhttp
library in the module-level build.gradle
file:
dependencies {
implementation "com.datadoghq:dd-sdk-android-okhttp:x.x.x"
}
Configure the OkHttpClient
interceptor with the list of internal, first-party origins called by your Android application.
val tracedHosts = listOf("example.com", "example.eu")
val okHttpClient = OkHttpClient.Builder()
.addInterceptor(DatadogInterceptor(tracedHosts))
.addNetworkInterceptor(TracingInterceptor(tracedHosts))
.eventListenerFactory(DatadogEventListener.Factory())
.build()
By default, all subdomains of listed hosts are traced. For instance, if you add example.com
, you also enable the tracing for api.example.com
and foo.example.com
.
(Optional) Configure the traceSampler
parameter to keep a defined percentage of the backend traces. If not set, 20% of the traces coming from application requests are sent to Datadog. To keep 100% of backend traces:
val okHttpClient = OkHttpClient.Builder()
.addInterceptor(DatadogInterceptor(traceSampler = RateBasedSampler(100f)))
.build()
Note:
traceSampler
does not impact RUM sessions sampling. Only backend traces are sampled out.- If you define custom tracing header types in the Datadog configuration and are using a tracer registered with
GlobalTracer
, make sure the same tracing header types are set for the tracer in use.
Set up RUM iOS Monitoring.
Enable RUM
with the urlSessionTracking
option and firstPartyHostsTracing
parameter:
RUM.enable(
with: RUM.Configuration(
applicationID: "<rum application id>",
urlSessionTracking: .init(
firstPartyHostsTracing: .trace(
hosts: [
"example.com",
"api.yourdomain.com"
]
)
)
)
)
Enable URLSession instrumentation for your SessionDelegate
type, which conforms to URLSessionDataDelegate
protocol:
URLSessionInstrumentation.enable(
with: .init(
delegateClass: <YourSessionDelegate>.self
)
)
Initialize URLSession as stated in Setup:
let session = URLSession(
configuration: ...,
delegate: <YourSessionDelegate>(),
delegateQueue: ...
)
By default, all subdomains of listed hosts are traced. For instance, if you add example.com
, you also enable tracing for api.example.com
and foo.example.com
.
Trace ID injection works when you are providing a URLRequest
to the URLSession
. Distributed tracing does not work when you are using a URL
object.
(Optional) Set the sampleRate
parameter to keep a defined percentage of the backend traces. If not set, 20% of the traces coming from application requests are sent to Datadog.
To keep 100% of backend traces:
RUM.enable(
with: RUM.Configuration(
applicationID: "<rum application id>",
urlSessionTracking: .init(
firstPartyHostsTracing: .trace(
hosts: [
"example.com",
"api.yourdomain.com"
],
sampleRate: 100
)
)
)
)
Note: sampleRate
does not impact RUM sessions sampling. Only backend traces are sampled out.
Set up RUM React Native Monitoring.
Set the firstPartyHosts
initialization parameter to define the list of internal, first-party origins called by your React Native application:
const config = new DatadogProviderConfiguration(
// ...
);
config.firstPartyHosts = ["example.com", "api.yourdomain.com"];
By default, all subdomains of listed hosts are traced. For instance, if you add example.com
, you also enable tracing for api.example.com
and foo.example.com
.
(Optional) Set the resourceTracingSamplingRate
initialization parameter to keep a defined percentage of the backend traces. If not set, 20% of the traces coming from application requests are sent to Datadog.
To keep 100% of backend traces:
const config = new DatadogProviderConfiguration(
// ...
);
config.resourceTracingSamplingRate = 100;
Note: resourceTracingSamplingRate
does not impact RUM sessions sampling. Only backend traces are sampled out.
Set up RUM Flutter Monitoring.
Follow the instructions under Automatic Resource Tracking to include the Datadog Tracking HTTP Client package and enable HTTP tracking. This includes the following changes to your initialization to add a list of internal, first-party origins called by your Flutter application:
final configuration = DatadogConfiguration(
// ...
// added configuration
firstPartyHosts: ['example.com', 'api.yourdomain.com'],
)..enableHttpTracking()
RUM for Roku is not available on the US1-FED Datadog site.
Set up RUM Roku Monitoring.
Use the datadogroku_DdUrlTransfer
component to perform your network requests.
ddUrlTransfer = datadogroku_DdUrlTransfer(m.global.datadogRumAgent)
ddUrlTransfer.SetUrl(url)
ddUrlTransfer.EnablePeerVerification(false)
ddUrlTransfer.EnableHostVerification(false)
result = ddUrlTransfer.GetToString()
Verifying setup
To verify you’ve configured the APM integration with RUM, follow the steps below based on the SDK you installed RUM with.
- Visit a page in your application.
- In your browser’s developer tools, go to the Network tab.
- Check the request headers for a resource request that you expect to be correlated contains the correlation headers from Datadog.
Supported libraries
Below is a list of the supported backend libraries that need to be on the services receiving the network requests.
OpenTelemetry support
RUM supports several propagator types to connect resources with backends that are instrumented with OpenTelemetry libraries.
The default injection style is tracecontext
, Datadog
.
Note: If you are using a backend framework such as Next.js/Vercel that uses OpenTelemetry, follow these steps.
Set up RUM to connect with APM as described above.
Modify allowedTracingUrls
as follows:
import { datadogRum } from '@datadog/browser-rum'
datadogRum.init({
...otherConfig,
allowedTracingUrls: [
{ match: "https://api.example.com", propagatorTypes: ["tracecontext"]}
]
})
match
accepts the same parameter types (string
, RegExp
or function
) as when used in its simple form, described above.
propagatorTypes
accepts a list of strings for desired propagators:
Set up RUM to connect with APM as described above.
Use .traceWithHeaders(hostsWithHeaders:sampleRate:)
instead of .trace(hosts:sampleRate:)
as follows:
RUM.enable(
with: RUM.Configuration(
applicationID: "<rum application id>",
urlSessionTracking: .init(
firstPartyHostsTracing: .traceWithHeaders(
hostsWithHeaders: [
"api.example.com": [.tracecontext]
],
sampleRate: 100
)
)
)
)
.traceWithHeaders(hostsWithHeaders:sampleRate:)
takes Dictionary<String, Set<TracingHeaderType>>
as a parameter, where the key is a host and the value is a list of supported tracing header types.
TracingHeaderType
in an enum representing the following tracing header types:
Set up RUM to connect with APM as described above.
Configure the OkHttpClient
interceptor with the list of internal, first-party origins and the tracing header type to use as follows:
val tracedHosts = mapOf("example.com" to setOf(TracingHeaderType.TRACECONTEXT),
"example.eu" to setOf(TracingHeaderType.DATADOG))
val okHttpClient = OkHttpClient.Builder()
.addInterceptor(DatadogInterceptor(tracedHosts))
.addNetworkInterceptor(TracingInterceptor(tracedHosts))
.eventListenerFactory(DatadogEventListener.Factory())
.build()
TracingHeaderType
is an enum representing the following tracing header types:
Set up RUM to connect with APM.
Configure the RUM SDK with the list of internal, first-party origins and the tracing header type to use as follows:
const config = new DatadogProviderConfiguration(
// ...
);
config.firstPartyHosts = [{
match: "example.com",
propagatorTypes: [
PropagatorType.TRACECONTEXT,
PropagatorType.DATADOG
]
}];
PropagatorType
is an enum representing the following tracing header types:
Set up RUM to connect with APM as described above.
Use firstPartyHostsWithTracingHeaders
instead of firstPartyHosts
as follows:
final configuration = DatadogConfiguration(
// ...
// added configuration
firstPartyHostsWithTracingHeaders: {
'example.com': { TracingHeaderType.tracecontext },
},
)..enableHttpTracking()
firstPartyHostsWithTracingHeaders
takes Map<String, Set<TracingHeaderType>>
as a parameter, where the key is a host and the value is a list of supported tracing header types.
TracingHeaderType
in an enum representing the following tracing header types:
How are RUM resources linked to traces?
Datadog uses the distributed tracing protocol and sets up the HTTP headers below. By default, both trace context and Datadog-specific headers are used.
x-datadog-trace-id
- Generated from the Real User Monitoring SDK. Allows Datadog to link the trace with the RUM resource.
x-datadog-parent-id
- Generated from the Real User Monitoring SDK. Allows Datadog to generate the first span from the trace.
x-datadog-origin: rum
- To make sure the generated traces from Real User Monitoring don’t affect your APM Index Spans counts.
x-datadog-sampling-priority: 1
- To make sure that the Agent keeps the trace.
traceparent: [version]-[trace id]-[parent id]-[trace flags]
version
: The current specification assumes version is set to 00
.trace id
: 128 bits trace ID, hexadecimal on 32 characters. The source trace ID is 64 bits to keep compatibility with APM.parent id
: 64 bits span ID, hexadecimal on 16 characters.trace flags
: Sampled (01
) or not sampled (00
)- Example:
traceparent: 00-00000000000000008448eb211c80319c-b7ad6b7169203331s-01
b3: [trace id]-[span id]-[sampled]
trace id
: 64 bits trace ID, hexadecimal on 16 characters.span id
: 64 bits span ID, hexadecimal on 16 characters.sampled
: True (1
) or False (0
)- Example for b3 single header:
b3: 8448eb211c80319c-b7ad6b7169203331-1
- Example for b3 multiple headers:
X-B3-TraceId: 8448eb211c80319c
X-B3-SpanId: b7ad6b7169203331
X-B3-Sampled: 1
These HTTP headers are not CORS-safelisted, so you need to configure Access-Control-Allow-Headers on your server handling requests that the SDK is set up to monitor. The server must also accept preflight requests (OPTIONS requests), which are made by the SDK prior to every request.
How are APM quotas affected?
Connecting RUM and traces may significantly increase the APM-ingested volumes. Use the initialization parameter traceSampleRate
to keep a share of the backend traces starting from browser and mobile requests.
How long are traces retained?
These traces are available for 15 minutes in the Live Search explorer. To retain the traces for a longer period of time, create retention filters. Scope these retention filters on any span tag to retain traces for critical pages and user actions.
Further Reading
Additional helpful documentation, links, and articles: