Overview

This guide explains how you can use both LLM Observability and APM to link LLM Observability and APM spans in Datadog.

By instrumenting your LLM-specific operations with LLM Observability and your broader application with APM, you can accomplish the following:

• Understand end-to-end visibility: Explore upstream and downstream requests of your LLM applications within the context of your entire application.
• From APM, dive deeper into LLM Observability: Investigate whether or not an issue with your application is specific to LLM-specific applications, such as a call to OpenAI.

Setup

The LLM Observability SDK is built on APM’s dd-tracer. This allows you to use LLM Observability with Application Performance Monitoring (APM)

If you are using the LLM Observability SDK for Python along with APM’s dd-tracer, you can navigate between spans in Datadog APM and LLM Observability without additional setup.

If you are using the LLM Observability API with dd-tracer for APM:

1. Use the appropriate method to obtain the span ID from the tracer (for example, using span.Context().SpanID() for the Go tracer).
2. Include the captured span IDs in all of the LLM Observability API requests. This links APM and LLM Observability spans in Datadog.

Navigate between spans

By using this integration, you can correlate data across your application stack and understand how your LLM applications interact with other components. You can also resolve issues more quickly and optimize your application’s performance.

From LLM Observability to APM

To understand the broader context of your LLM operations within your application ecosystem, select an LLM Observability span in the LLM Observability Explorer and click APM span to navigate to the relevant APM span.

From APM to LLM Observability

To access LLM-specific insights, select an APM span in the Trace Explorer and click View Span in the LLM Observability section on the Info tab to navigate to the corresponding LLM Observability span.

Further Reading

Documentation, liens et articles supplémentaires utiles:

Learn about LLM Observability SpansDOCUMENTATION Learn about APM SpansDOCUMENTATION Troubleshooting RAG-based LLM applicationsBLOG