Route Anthropic models on Vertex AI through the native endpoint#2476
Open
dgageot wants to merge 1 commit intodocker:mainfrom
Open
Route Anthropic models on Vertex AI through the native endpoint#2476dgageot wants to merge 1 commit intodocker:mainfrom
dgageot wants to merge 1 commit intodocker:mainfrom
Conversation
Claude models on Google Cloud's Vertex AI Model Garden do not support the OpenAI-compatible /chat/completions endpoint and fail with: 'FAILED_PRECONDITION: The deployed model does not support ChatCompletions.'
When provider_opts.publisher == "anthropic", requests are now routed through the Anthropic-native :rawPredict / :streamRawPredict endpoints (with the vertex-2023-10-16 body schema) using the anthropic-sdk-go/vertex subpackage, authenticated via Google Application Default Credentials. Other Model Garden publishers (meta, mistral, ...) continue to use the OpenAI-compatible path, and Gemini-on-Vertex is unchanged (it never enters this code path).
- pkg/model/provider/anthropic/vertex.go: new NewVertexClient constructor.
- pkg/model/provider/vertexai/modelgarden.go: single NewClient entry that dispatches on publisher and returns a small Client interface satisfied by both anthropic.Client and openai.Client; shared resolveProjectLocation helper with URL-injection-safe validation.
- pkg/model/provider/provider.go: one-line dispatch into vertexai.NewClient.
- Tests: cover publisher extraction, project/location resolution (env-var fallback, ${VAR} expansion, URL-injection attempts, uppercase rejection).
- Docs: docs/providers/google/index.md explains the two endpoint paths.
Fixes docker#2469
Assisted-By: docker-agent
gtardif
approved these changes
Apr 21, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #2469.
Claude models on Google Cloud's Vertex AI Model Garden do not support the OpenAI-compatible `/chat/completions` endpoint and fail with:
When `provider_opts.publisher == "anthropic"`, requests are now routed through the Anthropic-native `:rawPredict` / `:streamRawPredict` endpoints (with the `vertex-2023-10-16` body schema) using the `anthropic-sdk-go/vertex` subpackage, authenticated via Google Application Default Credentials. Other Model Garden publishers (`meta`, `mistral`, ...) continue to use the OpenAI-compatible path, and Gemini-on-Vertex is unchanged (it never enters this code path).
Example config
```yaml
models:
claude-on-vertex:
provider: google
model: claude-sonnet-4-20250514
provider_opts:
project: my-gcp-project
location: us-east5
publisher: anthropic
```
Routing
Changes
Validation
Assisted-By: docker-agent