feat: add LiteLLM callback handler for all providers

[wingding12]

Summary

Fixes #1028, #1079

Implements a LiteLLM callback handler that properly tracks LLM calls across all LiteLLM-supported providers including OpenAI and Anthropic.

Root Cause Analysis

#1079 (Anthropic models not tracked):

• Anthropic models weren't being tracked because there was no LiteLLM callback handler
• Only direct provider instrumentation was available, which doesn't intercept LiteLLM's callback system

#1028 (Responses API not working):

• The litellm.responses() API wasn't supported because there was no callback handler
• The responses API uses a different format (output instead of choices) that wasn't handled

Solution

Created LiteLLMCallbackHandler that:

• Extends LiteLLM's CustomLogger interface
• Properly extracts provider from model strings (anthropic/, openai/, etc.)
• Handles both completion API (choices) and responses API (output) formats
• Records token usage with input_tokens/output_tokens mapping
• Creates proper spans with gen_ai.* semantic conventions

Usage

from agentops import LiteLLMCallbackHandler
import litellm

handler = LiteLLMCallbackHandler(api_key="your-key")
litellm.callbacks = [handler]

# Now all LiteLLM calls are tracked - both OpenAI and Anthropic!
response = litellm.completion(
    model="anthropic/claude-3-5-sonnet-20240620",
    messages=[{"role": "user", "content": "Hello!"}]
)

# Responses API also works
response = litellm.responses(
    model="gpt-4o",
    input="Tell me a story"
)

Features

• Works with litellm.callbacks = [handler]
• Supports all LiteLLM providers (OpenAI, Anthropic, Google, Mistral, Cohere, etc.)
• Handles both sync and async API calls
• Records prompt/completion tokens and cost
• Properly nests under session spans
• Handles both completion and responses API formats

Files Changed

• agentops/integration/callbacks/litellm/__init__.py (new)
• agentops/integration/callbacks/litellm/callback.py (new)
• agentops/__init__.py (export LiteLLMCallbackHandler)
• tests/unit/integration/callbacks/litellm/test_litellm_callback.py (new)

Test Plan

• Test callback handler initialization
• Test provider extraction from model strings
• Test span creation on pre-API call
• Test span ending on success
• Test exception recording on failure
• Test responses API format handling
• Test Anthropic model handling
• Test OpenAI model handling
• Test session cleanup