Add support for load balancing #145
Merged
+416
−15
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
deploy/src/db.ts
Outdated
| } | ||
| } | ||
|
|
||
| // Then, fall back to the old routes format (backward compatibility) |
Member
There was a problem hiding this comment.
not sure we really need this, no one is using the old format AFAIK.
deploy/src/types.ts
Outdated
| export interface Config<ProviderKey extends string = string> { | ||
| /** @param project: record keys are the project ids */ | ||
| projects: Record<number, Project> | ||
| /** @deprecated Use routingGroups instead */ |
Member
There was a problem hiding this comment.
again, not sure we need deprecation warnings
Kludex
approved these changes
Nov 17, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Adds support for load balancing and fallback routing with priority and weight-based provider selection. This enables flexible traffic distribution across multiple providers and automatic fallback when providers fail.
New
routingGroupsConfigurationIntroduces a new
routingGroupsconfiguration option that replaces the deprecatedroutesfield. The new format supports two key parameters:priority(optional): Higher priority providers are tried first. If not specified, defaults to the negative index position (0, -1, -2, etc.)weight(optional): Controls traffic distribution among providers with the same priority. If not specified, defaults to 1. Negative weights are normalized to 0.Features
Priority-based Fallback: Providers with higher priority are attempted first. If they fail, the gateway falls back to lower priority providers.
Weighted Load Balancing: Within the same priority level, traffic is distributed probabilistically based on weights using a weighted random sampling algorithm.
Backward Compatibility: The old
routesconfiguration format is still supported and will continue to work without modification.Pydantic AI Gateway Console
We'll update the console to support specifying both a weight and priority for routing group members; the logic from this PR will be used to affect runtime behavior.