Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
example/aibridge.db
132 changes: 131 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
@@ -1 +1,131 @@
aibridge
# aibridge

aibridge is an HTTP gateway that sits between AI clients and upstream AI providers (Anthropic, OpenAI). It intercepts requests to record token usage, prompts, and tool invocations per user. Optionally supports centralized [MCP](https://modelcontextprotocol.io/) tool injection with allowlist/denylist filtering.

## Architecture

```
┌─────────────────┐ ┌───────────────────────────────────────────┐
│ AI Client │ │ aibridge │
│ (Claude Code, │────▶│ ┌─────────────────┐ ┌─────────────┐ │
│ Cursor, etc.) │ │ │ RequestBridge │───▶│ Providers │ │
└─────────────────┘ │ │ (http.Handler) │ │ (Anthropic │ │
│ └─────────────────┘ │ OpenAI) │ │
│ └──────┬──────┘ │
│ │ │
│ ▼ │ ┌─────────────┐
│ ┌─────────────────┐ ┌─────────────┐ │ │ Upstream │
│ │ Recorder │◀───│ Interceptor │─── ───▶│ API │
│ │ (tokens, tools, │ │ (streaming/ │ │ │ (Anthropic │
│ │ prompts) │ │ blocking) │ │ │ OpenAI) │
│ └────────┬────────┘ └──────┬──────┘ │ └─────────────┘
│ │ │ │
│ ▼ ┌──────▼──────┐ │
│ ┌ ─ ─ ─ ─ ─ ─ ─ ┐ │ MCP Proxy │ │
│ │ Database │ │ (tools) │ │
│ └ ─ ─ ─ ─ ─ ─ ─ ┘ └─────────────┘ │
└───────────────────────────────────────────┘
```

### Components

- **RequestBridge**: The main `http.Handler` that routes requests to providers
- **Provider**: Defines bridged routes (intercepted) and passthrough routes (proxied)
- **Interceptor**: Handles request/response processing and streaming
- **Recorder**: Interface for capturing usage data (tokens, prompts, tools)
- **MCP Proxy** (optional): Connects to MCP servers to list tool, inject them into requests, and invoke them in an inner agentic loop

## Request Flow

1. Client sends request to `/anthropic/v1/messages` or `/openai/v1/chat/completions`
2. **Actor extraction**: Request must have an actor in context (via `AsActor()`).
3. **Upstream call**: Request forwarded to the AI provider
4. **Response relay**: Response streamed/sent to client
5. **Recording**: Token usage, prompts, and tool invocations recorded

**With MCP enabled**: Tools from configured MCP servers are centrally defined and injected into requests (prefixed `bmcp_`). Allowlist/denylist regex patterns control which tools are available. When the model selects an injected tool, the gateway invokes it in an inner agentic loop, and continues the conversation loop until complete.

Passthrough routes (`/v1/models`, `/v1/messages/count_tokens`) are reverse-proxied directly.

## Observability

### Prometheus Metrics

Create metrics with `NewMetrics(prometheus.Registerer)`:

| Metric | Type | Description |
|--------|------|-------------|
| `interceptions_total` | Counter | Intercepted request count |
| `interceptions_inflight` | Gauge | Currently processing requests |
| `interceptions_duration_seconds` | Histogram | Request duration |
| `tokens_total` | Counter | Token usage (input/output) |
| `prompts_total` | Counter | User prompt count |
| `injected_tool_invocations_total` | Counter | MCP tool invocations |
| `passthrough_total` | Counter | Non-intercepted requests |

### Recorder Interface

Implement `Recorder` to persist usage data to your database. The example uses SQLite ([example/recorder.go](example/recorder.go)):

- `aibridge_interceptions` - request metadata (provider, model, initiator, timestamps)
- `aibridge_token_usages` - input/output token counts per response
- `aibridge_user_prompts` - user prompts
- `aibridge_tool_usages` - tool invocations (injected and client-defined)

```go
type Recorder interface {
RecordInterception(ctx context.Context, req *InterceptionRecord) error
RecordInterceptionEnded(ctx context.Context, req *InterceptionRecordEnded) error
RecordTokenUsage(ctx context.Context, req *TokenUsageRecord) error
RecordPromptUsage(ctx context.Context, req *PromptUsageRecord) error
RecordToolUsage(ctx context.Context, req *ToolUsageRecord) error
}
```

## Example

See [example/](example/) for a complete runnable example with SQLite persistence and [DeepWiki](https://mcp.deepwiki.com) MCP integration.

### Setup

1. **Get API keys** from the provider consoles:
- Anthropic: https://console.anthropic.com/settings/keys
- OpenAI: https://platform.openai.com/api-keys

2. **Set environment variables**:
```bash
export ANTHROPIC_API_KEY="sk-ant-..."
export OPENAI_API_KEY="sk-..."
```

3. **Run the example**:
```bash
cd example && go run .
```

4. **Test with curl**:
```bash
curl -X POST http://localhost:8080/anthropic/v1/messages \
-H "Content-Type: application/json" \
-d '{
"model": "claude-sonnet-4-20250514",
"max_tokens": 1024,
"messages": [{"role": "user", "content": "Hello!"}],
"stream": true
}'
```

5. **Test with Claude Code**:
Claude Code allows a base URL override via `ANTHROPIC_BASE_URL`.

![image with cloude code example](example/claude-code.png)

## Supported Routes

| Provider | Route | Type |
|----------|-------|------|
| Anthropic | `/anthropic/v1/messages` | Bridged (intercepted) |
| Anthropic | `/anthropic/v1/models` | Passthrough |
| Anthropic | `/anthropic/v1/messages/count_tokens` | Passthrough |
| OpenAI | `/openai/v1/chat/completions` | Bridged (intercepted) |
| OpenAI | `/openai/v1/models` | Passthrough |
Binary file added example/claude-code.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
77 changes: 77 additions & 0 deletions example/go.mod
Original file line number Diff line number Diff line change
@@ -0,0 +1,77 @@
module github.com/coder/aibridge/example

go 1.24.6

toolchain go1.24.10

require (
cdr.dev/slog v1.6.2-0.20250703074222-9df5e0a6c145
github.com/coder/aibridge v0.0.0
github.com/google/uuid v1.6.0
github.com/prometheus/client_golang v1.23.2
modernc.org/sqlite v1.40.1
)

require (
github.com/anthropics/anthropic-sdk-go v1.13.0 // indirect
github.com/aws/aws-sdk-go-v2 v1.30.3 // indirect
github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.6.3 // indirect
github.com/aws/aws-sdk-go-v2/config v1.27.27 // indirect
github.com/aws/aws-sdk-go-v2/credentials v1.17.27 // indirect
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.16.11 // indirect
github.com/aws/aws-sdk-go-v2/internal/configsources v1.3.15 // indirect
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.6.15 // indirect
github.com/aws/aws-sdk-go-v2/internal/ini v1.8.0 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.11.3 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.11.17 // indirect
github.com/aws/aws-sdk-go-v2/service/sso v1.22.4 // indirect
github.com/aws/aws-sdk-go-v2/service/ssooidc v1.26.4 // indirect
github.com/aws/aws-sdk-go-v2/service/sts v1.30.3 // indirect
github.com/aws/smithy-go v1.20.3 // indirect
github.com/aymanbagabas/go-osc52/v2 v2.0.1 // indirect
github.com/bahlo/generic-list-go v0.2.0 // indirect
github.com/beorn7/perks v1.0.1 // indirect
github.com/buger/jsonparser v1.1.1 // indirect
github.com/cespare/xxhash/v2 v2.3.0 // indirect
github.com/charmbracelet/lipgloss v0.7.1 // indirect
github.com/dustin/go-humanize v1.0.1 // indirect
github.com/hashicorp/errwrap v1.0.0 // indirect
github.com/hashicorp/go-multierror v1.1.1 // indirect
github.com/invopop/jsonschema v0.13.0 // indirect
github.com/lucasb-eyer/go-colorful v1.2.0 // indirect
github.com/mailru/easyjson v0.7.7 // indirect
github.com/mark3labs/mcp-go v0.38.0 // indirect
github.com/mattn/go-isatty v0.0.20 // indirect
github.com/mattn/go-runewidth v0.0.15 // indirect
github.com/muesli/reflow v0.3.0 // indirect
github.com/muesli/termenv v0.15.2 // indirect
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect
github.com/ncruces/go-strftime v0.1.9 // indirect
github.com/openai/openai-go/v2 v2.7.0 // indirect
github.com/prometheus/client_model v0.6.2 // indirect
github.com/prometheus/common v0.66.1 // indirect
github.com/prometheus/procfs v0.16.1 // indirect
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec // indirect
github.com/rivo/uniseg v0.4.4 // indirect
github.com/spf13/cast v1.7.1 // indirect
github.com/tidwall/gjson v1.18.0 // indirect
github.com/tidwall/match v1.2.0 // indirect
github.com/tidwall/pretty v1.2.1 // indirect
github.com/tidwall/sjson v1.2.5 // indirect
github.com/wk8/go-ordered-map/v2 v2.1.8 // indirect
github.com/yosida95/uritemplate/v3 v3.0.2 // indirect
go.opentelemetry.io/otel v1.33.0 // indirect
go.opentelemetry.io/otel/trace v1.33.0 // indirect
go.yaml.in/yaml/v2 v2.4.2 // indirect
golang.org/x/exp v0.0.0-20250819193227-8b4c13bb791b // indirect
golang.org/x/sys v0.36.0 // indirect
golang.org/x/term v0.34.0 // indirect
golang.org/x/xerrors v0.0.0-20240716161551-93cc26a95ae9 // indirect
google.golang.org/protobuf v1.36.8 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
modernc.org/libc v1.66.10 // indirect
modernc.org/mathutil v1.7.1 // indirect
modernc.org/memory v1.11.0 // indirect
)

replace github.com/coder/aibridge => ../
Loading