Overview

This guide uses the LLM Observability SDK for Python. If your application is written in another language, you can create traces by calling the API instead.

Setup

Jupyter notebooks

To better understand LLM Observability terms and concepts, you can explore the examples in the LLM Observability Jupyter Notebooks repository. These notebooks provide a hands-on experience, and allow you to apply these concepts in real time.

Command line

To generate an LLM Observability trace, you can run a Python script.

Prerequisites

• LLM Observability requires a Datadog API key. For more information, see the instructions for creating an API key.
• The following example script uses OpenAI, but you can modify it to use a different provider. To run the script as written, you need:
  • An OpenAI API key stored in your environment as OPENAI_API_KEY. To create one, see Account Setup and Set up your API key in the official OpenAI documentation.
  • The OpenAI Python library installed. See Setting up Python in the official OpenAI documentation for instructions.

1. Install the SDK by adding the ddtrace and openai packages:

   Copy

      pip install ddtrace
      pip install openai
      

2. Create a Python script and save it as quickstart.py. This Python script makes a single OpenAI call.

   quickstart.py

   Copy

      import os
      from openai import OpenAI
   
      oai_client = OpenAI(api_key=os.environ.get("OPENAI_API_KEY"))
   
      completion = oai_client.chat.completions.create(
          model="gpt-3.5-turbo",
          messages=[
           {"role": "system", "content": "You are a helpful customer assistant for a furniture store."},
           {"role": "user", "content": "I'd like to buy a chair for my living room."},
       ],
      )
      

3. Run the Python script with the following shell command. This sends a trace of the OpenAI call to Datadog.

   DD_LLMOBS_ENABLED=1 DD_LLMOBS_ML_APP=onboarding-quickstart \
   DD_API_KEY=<YOUR_DATADOG_API_KEY> DD_SITE= \
   DD_LLMOBS_AGENTLESS_ENABLED=1 ddtrace-run python quickstart.py
   

   For more information about required environment variables, see the SDK documentation.

   Note: DD_LLMOBS_AGENTLESS_ENABLED is only required if you do not have the Datadog Agent running. If the Agent is running in your production environment, make sure this environment variable is unset.

4. View the trace of your LLM call on the Traces tab of the LLM Observability page in Datadog.

The trace you see is composed of a single LLM span. The ddtrace-run command automatically traces your LLM calls from Datadog’s list of supported integrations.

If your application consists of more elaborate prompting or complex chains or workflows involving LLMs, you can trace it using the Setup documentation and the SDK documentation.

Further Reading

Additional helpful documentation, links, and articles:

Learn about LLM ObservabilityDOCUMENTATION