Honor reasoning effort for GPT + tools on Builder, and let automations set it - #5392
Conversation
![builder-io-integration[bot]](https://avatars.githubusercontent.com/in/353721?s=60&v=4){kind=small}
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
|
Here's a visual recap of what changed: Open the full interactive recap |
…s set it Gate the forced reasoning_effort: none for GPT reasoning models with tools on the Builder gateway behind a new agent.builderGatewayGptResponsesLane app-config field, so effort can flow through once the gateway proxies that combination to the OpenAI Responses API. Add a reasoningEffort field to job and automation frontmatter, thread it through the background automation runner, the automations service, list-automations, both agent-facing automation tools (manage-jobs, manage-automations), and Factory's automation editor UI with translated labels.
Live-verified: gpt-5.6-luna + 39 tools + reasoning_effort=xhigh returns 200 OK through api.builder.io/agent-native/gateway/v1/messages. The guard forcing "none" was based on a Chat Completions rejection actually observed on a different engine (ai-sdk:openai via a custom proxy), not the Builder gateway, which has always routed GPT reasoning models through the Responses API. Delete the now-unnecessary agent.builderGatewayGptResponsesLane app-config gate and forward the requested effort unconditionally, same as every other model family.
4bf2afc to
4ab0c40
Compare
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
- manage-automations handleDefine silently coerced an unrecognized reasoning_effort to undefined and reported a successful creation, while update rejected the same input. Pass the raw value through and validate it in defineAutomation (automations/service.ts), matching updateAutomation. - The generic manage-automation action (Agent Automations page) never exposed reasoningEffort at all, and its update path's required-field guard would reject an effort-only update outright. Add it to the schema, forward it on create/update, and include it in the required-field check. Also fixes the oxfmt formatting flagged by CI lint on the design doc.
|
Both required review findings are fixed in 656d9c0:
Also fixed the CI lint failure (oxfmt formatting on the design doc). No optional items were flagged in this pass — nothing deferred. |
There was a problem hiding this comment.
Builder reviewed your changes — looks good ✅
Review Details
Incremental Code Review Summary
The latest commits complete the two previously reported propagation fixes. The agent-facing define path now passes supplied effort values to defineAutomation, which performs the validation, and the non-agent manage-automation action now accepts nullable reasoningEffort, forwards it for create/update, supports effort-only updates, and returns the stored value. Both prior review threads were verified fixed and resolved.
The rest of the latest implementation remains coherent across Builder gateway forwarding, automation execution, frontmatter persistence, Factory save/list state, and localization. No new high-confidence bugs were found by the incremental review agents. I did not re-flag older concerns that were unchanged from the previous review cycle.
Risk assessment: Standard.
🧪 Browser testing: Attempted after this review, but all Factory UI cases were couldnt_verify; the app was reachable on a standalone Factory server, while executor sessions lacked registered Chrome/browser automation tools.
2 participants
Summary
reasoning_effort: noneonce tools were attached (i.e. on almost every real turn), based on a guard that assumed the gateway routed these requests to an OpenAI surface that rejects effort + tools together. That assumption didn't hold for this gateway — confirmed with a live request (200 OK,gpt-5.6-luna, 39 tools,reasoning_effort: xhigh) — so the guard is removed and the requested effort is now always forwarded, no flag needed.manage-jobs,manage-automations), instead of only inheriting the model's default.Test plan
xhigheffort returned200 OK, confirmed in both interactive chat and a Factory automation run (server logs:effort=xhigh...← 200 OK).pnpm guards(77/77 pass),tsc --noEmitclean forpackages/coreandtemplates/factory, and targetedvitestruns for every changed spec (239 tests across builder-engine, background-automation-runner, frontmatter, automations/service, both automation tool surfaces, and the Factory action/UI specs).