Enable Builder-managed Jev in production - #5526
Conversation
… packages/core (10 files)
|
Here's a visual recap of what changed: Open the full interactive recap |
… packages/core (3 files)
There was a problem hiding this comment.
Builder reviewed your changes and found 1 potential issue 🔴
Review Details
Code Review Summary
This PR adds Builder-managed Jev support behind what is claimed to be an "Agent-Native rollout flag," but the implementation contains a critical blocker: the feature flag is hardcoded to true with no runtime configuration mechanism.
Changes Overview
The PR refactors production Jev requests to optionally route through Builder-managed authentication and renames the capability label from "System one model (Jev)" to "Decision model (Jev)".
Added:
BUILDER_JEV_PROXY_ENABLEDconstant for controlling the feature- Flag checks in
rankJevCandidates(),preloadJevTools(),requestJev(), andgetJevContextCredentials()
Removed:
- Environment-based checks (
resolveDeployEnvironment() !== "production") - Unused imports after refactoring
Updated:
- Test expectations for production behavior
- Label strings across 7 files (cosmetic, consistent)
Risk Assessment: High ⚠️
The PR enables a significant behavioral change in production (Builder-managed Jev) without any control mechanism to toggle it if issues arise.
Critical Finding
🔴 HIGH: Feature flag is hardcoded instead of being configurable at runtime
The Problem:
- Line 13 of
jev-tool-prefetch.ts:export const BUILDER_JEV_PROXY_ENABLED = true;— hardcoded value - The changeset claims the feature is "behind an Agent-Native rollout flag" — this is not true
- The flag cannot be toggled without a code change and redeploy
Impact:
- Builder-managed Jev is always enabled in production (cannot disable post-deploy)
- No rollback path if the Builder proxy has issues
- Contradicts the stated goal of a controlled rollout
What should happen:
The flag must be configurable at runtime, either through:
- App configuration (following existing patterns like
getAppConfig()) - Environment variable (e.g.,
BUILDER_JEV_PROXY_ENABLED=false) - Some other runtime configuration that doesn't require code changes
This enables proper gradual rollout and immediate rollback capability.
Lower-Risk Changes ✅
- Label renames are consistent across all 7 files
- Test updates correctly validate new behavior
- Fallback logic preserved (gracefully degrades if proxy fails)
- Import cleanup is correct (removed unused imports)
🧪 Browser testing: Skipped — PR only modifies backend/config, no UI impact
| const JEV_TIMEOUT_MS = 750; | ||
| const JEV_MODEL = "jev-latest"; | ||
|
|
||
| export const BUILDER_JEV_PROXY_ENABLED = true; |
There was a problem hiding this comment.
🔴 Feature flag is hardcoded instead of being configurable at runtime
The BUILDER_JEV_PROXY_ENABLED flag is hardcoded to true, preventing runtime control of the feature. The changeset claims the feature is "behind an Agent-Native rollout flag," but this implementation has no mechanism to toggle it without redeploy. Should be configurable via app config, environment variable, or similar runtime mechanism for gradual rollout and rollback capability.
There was a problem hiding this comment.
The rollout control is intentionally a code-level boolean per the requested design. It is currently true because the Builder endpoint is being shipped with this change; setting it to false before a Builder deployment prevents Agent-Native from resolving Builder auth or making proxy calls, while preserving direct Jev fallback. A runtime configuration layer was not requested.
|
The rollout control is intentionally a code-level boolean per the requested design: |
|
The latest review's flag concern is understood. The rollout control is intentionally a code-level boolean per the requested design, not a runtime configuration layer: BUILDER_JEV_PROXY_ENABLED is currently true because the Builder endpoint is being shipped with this change. Setting it to false before a Builder deployment prevents Agent-Native from resolving Builder auth or making proxy calls, while preserving direct Jev fallback. A runtime configuration mechanism was not part of the requested rollout. |

What changed
Validation
The Builder-managed path remains optional when deployment credentials are unavailable.