Skip to main content

[Beta] Monitor Logs in Production

note

This is in beta. Expect frequent updates, as we improve based on your feedback.

LiteLLM provides an integration to let you monitor logs in production.

👉 Jump to our sample LiteLLM Dashboard: https://admin.litellm.ai/

Debug your first logs​

Open In Colab

1. Get your LiteLLM Token​

Go to admin.litellm.ai and copy the code snippet with your unique token

2. Set up your environment​

Add it to your .env

import os 

os.env["LITELLM_TOKEN"] = "e24c4c06-d027-4c30-9e78-18bc3a50aebb" # replace with your unique token

Turn on LiteLLM Client

import litellm 
litellm.client = True

3. Make a normal completion() call​

import litellm 
from litellm import completion
import os 

# set env variables
os.environ["LITELLM_TOKEN"] = "e24c4c06-d027-4c30-9e78-18bc3a50aebb" # replace with your unique token
os.environ["OPENAI_API_KEY"] = "openai key"

litellm.use_client = True # enable logging dashboard 
messages = [{ "content": "Hello, how are you?","role": "user"}]

# openai call
response = completion(model="gpt-3.5-turbo", messages=messages)

Your completion() call print with a link to your session dashboard (https://admin.litellm.ai/<your_unique_token>)

In the above case it would be: admin.litellm.ai/e24c4c06-d027-4c30-9e78-18bc3a50aebb

Click on your personal dashboard link. Here's how you can find it 👇

👋 Tell us if you need better privacy controls

3. Review request log​

Oh! Looks like our request was made successfully. Let's click on it and see exactly what got sent to the LLM provider.

Ah! So we can see that this request was made to a Baseten (see litellm_params > custom_llm_provider) for a model with ID - 7qQNLDB (see model). The message sent was - "Hey, how's it going?" and the response received was - "As an AI language model, I don't have feelings or emotions, but I can assist you with your queries. How can I assist you today?"

info

🎉 Congratulations! You've successfully debugger your first log!