Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
49 changes: 49 additions & 0 deletions examples/uncommonroute/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
# AgentOps + UncommonRoute: Track and Optimize LLM Costs

[AgentOps](https://github.com/AgentOps-AI/agentops) tracks what your agents spend. [UncommonRoute](https://github.com/CommonstackAI/UncommonRoute) cuts that spend by routing each request to the cheapest model that can handle it.

Together they close the loop: **observe → optimize → verify**.

## How it works

```
Your Agent → UncommonRoute (local proxy) → cheapest capable model
AgentOps (traces every call, tracks cost per request)
```

UncommonRoute analyzes each request using three signals (metadata, embeddings, structural complexity) and routes it to the optimal model:

| Task complexity | Routed to | Example |
|----------------|-----------|---------|
| Trivial | nano model (~$0.0008) | "hello" |
| Simple | budget model (~$0.001) | "fix the typo on line 3" |
| Medium | mid-tier (~$0.03) | "refactor this function" |
| Complex | premium (~$0.05) | "design a distributed scheduler" |

**Result**: ~82% cost reduction with 93.4% task completion rate.

## Setup

```bash
# Install dependencies
pip install -r requirements.txt

# Initialize UncommonRoute (one-time setup)
uncommon-route init

# Start the local proxy
uncommon-route serve

# Set your AgentOps API key
export AGENTOPS_API_KEY="***"

# Run the example
python uncommonroute_cost_optimization.py
```

## What you'll see in the AgentOps dashboard

- Cost breakdown per LLM call
- Which model UncommonRoute selected for each task
- Total session cost (compare with always-premium routing)
4 changes: 4 additions & 0 deletions examples/uncommonroute/requirements.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
agentops
openai
python-dotenv
uncommon-route
99 changes: 99 additions & 0 deletions examples/uncommonroute/uncommonroute_cost_optimization.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,99 @@
"""
AgentOps + UncommonRoute: Track and Optimize LLM Costs

This example shows how to combine AgentOps (cost tracking) with
UncommonRoute (cost optimization) to get full visibility into your
LLM spending and automatically reduce it by ~82%.

Setup:
pip install agentops openai
pipx install uncommon-route
uncommon-route init
uncommon-route serve

Environment variables:
AGENTOPS_API_KEY - your AgentOps API key
OPENAI_API_KEY - your OpenAI API key (used by UncommonRoute upstream)
ANTHROPIC_API_KEY - your Anthropic API key (optional, for more routing options)
"""

from openai import OpenAI
import agentops
import os
from dotenv import load_dotenv

load_dotenv()

os.environ["AGENTOPS_API_KEY"] = os.getenv("AGENTOPS_API_KEY", "your_api_key_here")

# --- Initialize AgentOps FIRST (before any LLM calls) ---
agentops.init(
auto_start_session=True,
trace_name="UncommonRoute Cost Optimization",
tags=["uncommonroute", "cost-optimization", "agentops-example"],
)

tracer = agentops.start_trace(
trace_name="UncommonRoute Cost Optimization",
tags=["uncommonroute", "cost-optimization"],
)

# --- Point the OpenAI client at UncommonRoute's local proxy ---
# UncommonRoute runs on localhost:8403 and is fully OpenAI-compatible.
# It analyzes each request and routes to the cheapest capable model.
client = OpenAI(
base_url="http://localhost:8403/v1",
api_key="not-needed", # UR uses your provider keys configured during `uncommon-route init`
)

# --- Task 1: Simple greeting (routed to a cheap/fast model) ---
print("=== Task 1: Simple greeting ===")
response = client.chat.completions.create(
model="uncommon-route/auto",
messages=[{"role": "user", "content": "Say hello in three languages."}],
)
print(f"Response: {response.choices[0].message.content}\n")

# --- Task 2: Medium complexity (routed to a mid-tier model) ---
print("=== Task 2: Code explanation ===")
response = client.chat.completions.create(
model="uncommon-route/auto",
messages=[
{
"role": "user",
"content": "Explain the difference between a mutex and a semaphore. Give a short code example in Python.",
}
],
)
print(f"Response: {response.choices[0].message.content}\n")

# --- Task 3: Complex reasoning (routed to a premium model) ---
print("=== Task 3: Architecture design ===")
response = client.chat.completions.create(
model="uncommon-route/auto",
messages=[
{
"role": "system",
"content": "You are a senior distributed systems architect.",
},
{
"role": "user",
"content": (
"Design a rate limiter for a multi-region API gateway that handles "
"100k requests/second. Consider consistency vs availability tradeoffs, "
"clock skew, and burst handling. Provide a detailed architecture."
),
},
],
)
print(f"Response: {response.choices[0].message.content}\n")

# --- End trace and show results ---
agentops.end_trace(tracer, end_state="Success")

print("=" * 60)
print("Check your AgentOps dashboard to see:")
print(" - Cost breakdown per request")
print(" - Which model UncommonRoute selected for each task")
print(" - Total spend vs. what it would have cost with a single premium model")
print("=" * 60)