Skip to content

Route Anthropic models on Vertex AI through the native endpoint#2476

Open
dgageot wants to merge 1 commit intodocker:mainfrom
dgageot:board/fix-docker-agent-issue-2469-021fcb0c
Open

Route Anthropic models on Vertex AI through the native endpoint#2476
dgageot wants to merge 1 commit intodocker:mainfrom
dgageot:board/fix-docker-agent-issue-2469-021fcb0c

Conversation

@dgageot
Copy link
Copy Markdown
Member

@dgageot dgageot commented Apr 21, 2026

Summary

Fixes #2469.

Claude models on Google Cloud's Vertex AI Model Garden do not support the OpenAI-compatible `/chat/completions` endpoint and fail with:

`FAILED_PRECONDITION: The deployed model does not support ChatCompletions.`

When `provider_opts.publisher == "anthropic"`, requests are now routed through the Anthropic-native `:rawPredict` / `:streamRawPredict` endpoints (with the `vertex-2023-10-16` body schema) using the `anthropic-sdk-go/vertex` subpackage, authenticated via Google Application Default Credentials. Other Model Garden publishers (`meta`, `mistral`, ...) continue to use the OpenAI-compatible path, and Gemini-on-Vertex is unchanged (it never enters this code path).

Example config

```yaml
models:
claude-on-vertex:
provider: google
model: claude-sonnet-4-20250514
provider_opts:
project: my-gcp-project
location: us-east5
publisher: anthropic
```

Routing

Config Path
`provider: google`, publisher unset or `google` `gemini.NewClient` (unchanged)
`provider: google`, `publisher: anthropic` `anthropic.NewVertexClient` ( `:streamRawPredict` ) — new
`provider: google`, `publisher: meta` / `mistral` / ... OpenAI-compatible endpoint (unchanged)

Changes

  • `pkg/model/provider/anthropic/vertex.go` (new): `NewVertexClient` constructor that builds an Anthropic SDK client wired through `vertex.WithCredentials`. Credentials are resolved up front via `google.FindDefaultCredentials` so a missing ADC returns a friendly error. Explicitly clears `ANTHROPIC_API_KEY` (see in-code comment — this is required, not cosmetic).
  • `pkg/model/provider/vertexai/modelgarden.go`: single `NewClient` entry point that dispatches on publisher and returns a small `Client` interface satisfied by both `anthropic.Client` and `openai.Client`. Shared `resolveProjectLocation` helper with URL-injection-safe validation (regex blocks `..`, `/`, uppercase, etc.).
  • `pkg/model/provider/provider.go`: one-line dispatch into `vertexai.NewClient`.
  • Tests: cover publisher extraction, project/location resolution (env-var fallback, `${VAR}` expansion, URL-injection attempts, uppercase rejection).
  • Docs: `docs/providers/google/index.md` explains the two endpoint paths.
  • `go.mod` / `go.sum`: transitive `google.golang.org/api` dependency picked up by `go mod tidy` from the `anthropic-sdk-go/vertex` import.

Validation

  • `go build ./...` — clean.
  • `mise lint` — 0 issues (`golangci-lint` + custom cop over 755 files + `go mod tidy --diff` all clean).
  • `mise test` — full suite passes, including 11 new test cases covering URL-injection attempts.

Assisted-By: docker-agent

Claude models on Google Cloud's Vertex AI Model Garden do not support the OpenAI-compatible /chat/completions endpoint and fail with: 'FAILED_PRECONDITION: The deployed model does not support ChatCompletions.'

When provider_opts.publisher == "anthropic", requests are now routed through the Anthropic-native :rawPredict / :streamRawPredict endpoints (with the vertex-2023-10-16 body schema) using the anthropic-sdk-go/vertex subpackage, authenticated via Google Application Default Credentials. Other Model Garden publishers (meta, mistral, ...) continue to use the OpenAI-compatible path, and Gemini-on-Vertex is unchanged (it never enters this code path).

- pkg/model/provider/anthropic/vertex.go: new NewVertexClient constructor.

- pkg/model/provider/vertexai/modelgarden.go: single NewClient entry that dispatches on publisher and returns a small Client interface satisfied by both anthropic.Client and openai.Client; shared resolveProjectLocation helper with URL-injection-safe validation.

- pkg/model/provider/provider.go: one-line dispatch into vertexai.NewClient.

- Tests: cover publisher extraction, project/location resolution (env-var fallback, ${VAR} expansion, URL-injection attempts, uppercase rejection).

- Docs: docs/providers/google/index.md explains the two endpoint paths.

Fixes docker#2469

Assisted-By: docker-agent
@dgageot dgageot requested a review from a team as a code owner April 21, 2026 08:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Anthropic models on Vertex AI fail with FAILED_PRECONDITION (ChatCompletions not supported)

2 participants