Cette page n'est pas encore disponible en français, sa traduction est en cours. Si vous avez des questions ou des retours sur notre projet de traduction actuel, n'hésitez pas à nous contacter.
LLM Observability is not available in the US1-FED site.
Overview
Your application can submit data to LLM Observability in two ways: with LLM Observability’s Python SDK, or with the LLM Observability API.
Each request fulfilled by your application is represented as a trace on the LLM Observability traces page in Datadog:
If you’re new to LLM Observability traces, read the Core Concepts before proceeding to decide which instrumentation options best suit your application.
Instrument your LLM application
This guide uses the LLM Observability SDK for Python. If your application is running in a serverless environment, follow the serverless setup instructions. If your application is not written in Python, you can complete the steps below with API requests instead of SDK function calls.
Datadog provides auto-instrumentation to capture LLM calls for specific LLM provider libraries. However, manually instrumenting your LLM application using the Python SDK can unlock even more of Datadog’s LLM Observability features.
Annotate your spans with input data, output data, metadata (such as temperature), metrics (such as input_tokens), and key-value tags (such as version:1.0.0).
To create a span, the LLM Observability SDK provides two options:
Decorators: Use ddtrace.llmobs.decorators.<SPAN_KIND>() as a decorator on the function you’d like to trace, replacing <SPAN_KIND> with the desired span kind.
Inline: Use ddtrace.llmobs.LLMObs.<SPAN_KIND>() as a context manager to trace any inline code, replacing <SPAN_KIND> with the desired span kind.
The examples below create a workflow span.
fromddtrace.llmobs.decoratorsimportworkflow@workflowdefprocess_message():...# user application logicreturn
fromddtrace.llmobsimportLLMObsdefprocess_message():withLLMObs.workflow()asspan:...# user application logicreturn
Nesting spans
Starting a new span before the current span is finished automatically traces a parent-child relationship between the two spans. The parent span represents the larger operation, while the child span represents a smaller nested sub-operation within it.
The examples below create a trace with two spans.
fromddtrace.llmobs.decoratorsimporttask,workflow@workflowdefprocess_message():perform_preprocessing()...# user application logicreturn@taskdefperform_preprocessing():...# user application logicreturn
fromddtrace.llmobsimportLLMObsdefprocess_message():withLLMObs.workflow(name="process_message")asworkflow_span:withLLMObs.task(name="perform_preprocessing")astask_span:...# user application logicreturn
Annotating spans
To add extra information to a span such as inputs, outputs, metadata, metrics, or tags, use the LLM Observability SDK’s LLMObs.annotate() method.
The examples below annotate the workflow span created in the above example:
fromddtrace.llmobsimportLLMObsfromddtrace.llmobs.decoratorsimportworkflow@workflow(name="process_message")defprocess_message():...# user application logicLLMObs.annotate(input_data="<ARGUMENT>",output_data="<OUTPUT>",metadata={},metrics={"input_tokens":15,"output_tokens":24},tags={},)return
fromddtrace.llmobsimportLLMObsdefprocess_message():withLLMObs.workflow()asspan:...# user application logicLLMObs.annotate(span=span,input_data="<ARGUMENT>",output_data="<OUTPUT>",metadata={},metrics={"input_tokens":15,"output_tokens":24},tags={},)return
For more information on alternative tracing methods and tracing features, see the SDK documentation.
Advanced tracing
Depending on the complexity of your LLM application, you can also: