Trace An LLM Application

Docs > LLM Observability > Trace An LLM Application

Overview

This guide uses the LLM Observability SDKs for Python and Node.js. If your application is written in another language, you can create traces by calling the API instead.

To better understand LLM Observability terms and concepts, you can explore the examples in the LLM Observability Jupyter Notebooks repository. These notebooks provide a hands-on experience, and allow you to apply these concepts in real time.

Command line

To generate an LLM Observability trace, you can run a Python or Node.js script.

Prerequisites

LLM Observability requires a Datadog API key. For more information, see the instructions for creating an API key.
The following example script uses OpenAI, but you can modify it to use a different provider. To run the script as written, you need:
- An OpenAI API key stored in your environment as OPENAI_API_KEY. To create one, see Account Setup and Set up your API key in the official OpenAI documentation.
- The OpenAI Python library installed. See Setting up Python in the official OpenAI documentation for instructions.

Install the SDK and OpenAI packages:
```
pip install ddtrace
pip install openai
```

Create a script, which makes a single OpenAI call.

import os
from openai import OpenAI

oai_client = OpenAI(api_key=os.environ.get("OPENAI_API_KEY"))

completion = oai_client.chat.completions.create(
    model="gpt-3.5-turbo",
    messages=[
     {"role": "system", "content": "You are a helpful customer assistant for a furniture store."},
     {"role": "user", "content": "I'd like to buy a chair for my living room."},
 ],
)

Run the script with the following shell command. This sends a trace of the OpenAI call to Datadog.
```
DD_LLMOBS_ENABLED=1 DD_LLMOBS_ML_APP=onboarding-quickstart \
DD_API_KEY=<YOUR_DATADOG_API_KEY> DD_SITE=<YOUR_DD_SITE> \
DD_LLMOBS_AGENTLESS_ENABLED=1 ddtrace-run python quickstart.py
```
Replace <YOUR_DATADOG_API_KEY> with your Datadog API key, and replace <YOUR_DD_SITE> with your Datadog site.
For more information about required environment variables, see the SDK documentation.

Install the SDK and OpenAI packages:

npm install dd-trace
npm install openai

Create a script, which makes a single OpenAI call.

const { OpenAI } = require('openai');

const oaiClient = new OpenAI(process.env.OPENAI_API_KEY);

function main () {
   const completion = await oaiClient.chat.completions.create({
      model: 'gpt-3.5-turbo',
      messages: [
         { role: 'system', content: 'You are a helpful customer assistant for a furniture store.' },
         { role: 'user', content: 'I\'d like to buy a chair for my living room.' },
      ]
   });
}

main();

Run the script with the following shell command. This sends a trace of the OpenAI call to Datadog.
```
DD_LLMOBS_ENABLED=1 DD_LLMOBS_ML_APP=onboarding-quickstart \
DD_API_KEY=<YOUR_DATADOG_API_KEY> DD_SITE=<YOUR_DD_SITE> \
DD_LLMOBS_AGENTLESS_ENABLED=1 NODE_OPTIONS="--import dd-trace/initialize.mjs" node quickstart.js
```
Replace <YOUR_DATADOG_API_KEY> with your Datadog API key, and replace <YOUR_DD_SITE> with your Datadog site.
For more information about required environment variables, see the SDK documentation.

Note: DD_LLMOBS_AGENTLESS_ENABLED is only required if you do not have the Datadog Agent running. If the Agent is running in your production environment, make sure this environment variable is unset.

View the trace of your LLM call on the Traces tab of the LLM Observability page in Datadog.

The trace you see is composed of a single LLM span. The ddtrace-run or NODE_OPTIONS="--import dd-trace/initialize.mjs" command automatically traces your LLM calls from Datadog’s list of supported integrations.

If your application consists of more elaborate prompting or complex chains or workflows involving LLMs, you can trace it using the Setup documentation and the SDK documentation.

Trace An LLM Application

Overview

Setup

Jupyter notebooks

Command line

Prerequisites

Further Reading