Skip to content

Enable Builder-managed Jev in production - #5526

Merged
steve8708 merged 3 commits into
mainfrom
changes-12047
Sep 21, 2026
Merged

steve8708 merged 3 commits into
mainfrom
changes-12047

Conversation

@steve8708

Copy link
Copy Markdown
Contributor

What changed

  • route production Jev requests through Builder-managed authentication when available
  • rename the onboarding and API-key label to "Decision model (Jev)"
  • keep direct JEV_API_KEY fallback and deterministic failure fallback intact

Validation

  • focused Jev/onboarding Vitest: 52 passed
  • package typecheck: blocked by pre-existing missing workspace packages and ES2023 lib errors
  • agent-chat guard: passed
  • i18n changed-copy guard: passed
  • live beta Design read-only agent request: completed
  • deployed Builder Jev route probe: reached gateway preflight (403 without Space ID)

The Builder-managed path remains optional when deployment credentials are unavailable.

builder-io-integration[bot]

This comment was marked as outdated.

@github-actions

github-actions Bot commented Sep 21, 2026

Copy link
Copy Markdown
Contributor

Here's a visual recap of what changed:

Visual recap

Open the full interactive recap

builder-io-integration[bot]

This comment was marked as outdated.

@github-actions
github-actions Bot temporarily deployed to pr-5526-assets September 21, 2026 16:18 Destroyed
@github-actions
github-actions Bot temporarily deployed to pr-5526-content September 21, 2026 16:18 Destroyed
@github-actions
github-actions Bot temporarily deployed to pr-5526-forms September 21, 2026 16:18 Destroyed
@github-actions
github-actions Bot temporarily deployed to pr-5526-starter September 21, 2026 16:18 Destroyed
@github-actions
github-actions Bot temporarily deployed to pr-5526-slides September 21, 2026 16:18 Destroyed
@github-actions
github-actions Bot temporarily deployed to pr-5526-design September 21, 2026 16:19 Destroyed
@github-actions
github-actions Bot temporarily deployed to pr-5526-clips September 21, 2026 16:19 Destroyed
@github-actions
github-actions Bot temporarily deployed to pr-5526-mail September 21, 2026 16:20 Destroyed
@github-actions
github-actions Bot temporarily deployed to pr-5526-dispatch September 21, 2026 16:20 Destroyed
@github-actions
github-actions Bot temporarily deployed to pr-5526-calendar September 21, 2026 16:20 Destroyed
@github-actions
github-actions Bot temporarily deployed to pr-5526-plan September 21, 2026 16:20 Destroyed
@github-actions
github-actions Bot temporarily deployed to pr-5526-analytics September 21, 2026 16:20 Destroyed

@builder-io-integration builder-io-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Builder reviewed your changes and found 1 potential issue 🔴

Review Details

Code Review Summary

This PR adds Builder-managed Jev support behind what is claimed to be an "Agent-Native rollout flag," but the implementation contains a critical blocker: the feature flag is hardcoded to true with no runtime configuration mechanism.

Changes Overview

The PR refactors production Jev requests to optionally route through Builder-managed authentication and renames the capability label from "System one model (Jev)" to "Decision model (Jev)".

Added:

  • BUILDER_JEV_PROXY_ENABLED constant for controlling the feature
  • Flag checks in rankJevCandidates(), preloadJevTools(), requestJev(), and getJevContextCredentials()

Removed:

  • Environment-based checks (resolveDeployEnvironment() !== "production")
  • Unused imports after refactoring

Updated:

  • Test expectations for production behavior
  • Label strings across 7 files (cosmetic, consistent)

Risk Assessment: High ⚠️

The PR enables a significant behavioral change in production (Builder-managed Jev) without any control mechanism to toggle it if issues arise.

Critical Finding

🔴 HIGH: Feature flag is hardcoded instead of being configurable at runtime

The Problem:

  • Line 13 of jev-tool-prefetch.ts: export const BUILDER_JEV_PROXY_ENABLED = true; — hardcoded value
  • The changeset claims the feature is "behind an Agent-Native rollout flag" — this is not true
  • The flag cannot be toggled without a code change and redeploy

Impact:

  • Builder-managed Jev is always enabled in production (cannot disable post-deploy)
  • No rollback path if the Builder proxy has issues
  • Contradicts the stated goal of a controlled rollout

What should happen:
The flag must be configurable at runtime, either through:

  1. App configuration (following existing patterns like getAppConfig())
  2. Environment variable (e.g., BUILDER_JEV_PROXY_ENABLED=false)
  3. Some other runtime configuration that doesn't require code changes

This enables proper gradual rollout and immediate rollback capability.

Lower-Risk Changes ✅

  • Label renames are consistent across all 7 files
  • Test updates correctly validate new behavior
  • Fallback logic preserved (gracefully degrades if proxy fails)
  • Import cleanup is correct (removed unused imports)

🧪 Browser testing: Skipped — PR only modifies backend/config, no UI impact

const JEV_TIMEOUT_MS = 750;
const JEV_MODEL = "jev-latest";

export const BUILDER_JEV_PROXY_ENABLED = true;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 Feature flag is hardcoded instead of being configurable at runtime

The BUILDER_JEV_PROXY_ENABLED flag is hardcoded to true, preventing runtime control of the feature. The changeset claims the feature is "behind an Agent-Native rollout flag," but this implementation has no mechanism to toggle it without redeploy. Should be configurable via app config, environment variable, or similar runtime mechanism for gradual rollout and rollback capability.

Fix in Builder

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The rollout control is intentionally a code-level boolean per the requested design. It is currently true because the Builder endpoint is being shipped with this change; setting it to false before a Builder deployment prevents Agent-Native from resolving Builder auth or making proxy calls, while preserving direct Jev fallback. A runtime configuration layer was not requested.

@steve8708

Copy link
Copy Markdown
Contributor Author

The rollout control is intentionally a code-level boolean per the requested design: BUILDER_JEV_PROXY_ENABLED is currently true because the Builder endpoint is being shipped with this change. Setting it to false before a Builder deployment prevents both Builder credential resolution and proxy calls, while preserving direct Jev fallback. I did not add a runtime env/app configuration layer because that was not part of the requested rollout mechanism.

@steve8708

Copy link
Copy Markdown
Contributor Author

The latest review's flag concern is understood. The rollout control is intentionally a code-level boolean per the requested design, not a runtime configuration layer: BUILDER_JEV_PROXY_ENABLED is currently true because the Builder endpoint is being shipped with this change. Setting it to false before a Builder deployment prevents Agent-Native from resolving Builder auth or making proxy calls, while preserving direct Jev fallback. A runtime configuration mechanism was not part of the requested rollout.

@github-actions
github-actions Bot temporarily deployed to pr-5526-forms September 21, 2026 17:07 Destroyed
@github-actions
github-actions Bot temporarily deployed to pr-5526-starter September 21, 2026 17:07 Destroyed
@github-actions
github-actions Bot temporarily deployed to pr-5526-plan September 21, 2026 17:07 Destroyed
@github-actions
github-actions Bot temporarily deployed to pr-5526-slides September 21, 2026 17:08 Destroyed
@github-actions
github-actions Bot temporarily deployed to pr-5526-dispatch September 21, 2026 17:08 Destroyed
@github-actions
github-actions Bot temporarily deployed to pr-5526-clips September 21, 2026 17:08 Destroyed
@github-actions
github-actions Bot temporarily deployed to pr-5526-analytics September 21, 2026 17:08 Destroyed
@github-actions
github-actions Bot temporarily deployed to pr-5526-assets September 21, 2026 17:08 Destroyed
@github-actions
github-actions Bot temporarily deployed to pr-5526-design September 21, 2026 17:08 Destroyed
@github-actions
github-actions Bot temporarily deployed to pr-5526-mail September 21, 2026 17:08 Destroyed
@github-actions
github-actions Bot temporarily deployed to pr-5526-calendar September 21, 2026 17:08 Destroyed
@github-actions
github-actions Bot temporarily deployed to pr-5526-content September 21, 2026 17:08 Destroyed
@steve8708
steve8708 merged commit 6fbbdc3 into main Sep 21, 2026
86 of 87 checks passed
@steve8708
steve8708 deleted the changes-12047 branch September 21, 2026 17:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant