Repository navigation
fix(claude): pass --json-schema and read structured_output in claude_chat - #424
Open
JodyeJack (2vqmgh2pbd-cmyk) wants to merge 1 commit into
Open
JodyeJack (2vqmgh2pbd-cmyk) wants to merge 1 commit into
JodyeJack (2vqmgh2pbd-cmyk) wants to merge 1 commit into
Conversation
…chat The claude_chat backend passed --schema for structured calls. The Claude CLI has no such option and exits with "error: unknown option '--schema'", so every return_message=True call (OfficeQA and SpreadsheetBench rollouts) failed after its retries. The fix also reads the CLI's structured_output field. With --json-schema the CLI puts the validated object there and repeats it as JSON text in result, so reading result alone returned the reply as raw JSON with no tool calls.
Contributor
|
JodyeJack (@2vqmgh2pbd-cmyk) please read the following Contributor License Agreement(CLA). If you agree with the CLA, please reply with the following information.
Contributor License AgreementContribution License AgreementThis Contribution License Agreement (“Agreement”) is agreed to by the party signing below (“You”),
|
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
claude_chatnow passes--json-schemainstead of--schemafor structured calls and reads the CLI'sstructured_outputfield. The Claude CLI rejects--schemawitherror: unknown option '--schema', so everyreturn_message=Truecall failed after its retries.User impact: OfficeQA and SpreadsheetBench could not run on
claude_chat; every rollout turn failed.Proof: the new regression test fails on
mainat22c1ed8and passes with this change.Reproduce
With the flag the CLI accepts, the result event carries the validated object in
structured_outputand repeats it as a JSON string inresult:Claude CLI 2.1.296, macOS, Python 3.12.
Root cause
_run_claude_print()used a flag the CLI does not have._call_messages()then readresult, which under--json-schemaholds the reply as JSON text; the parsed object is instructured_output.claude_code_execalready does both (#238).Changes
--schemabecomes--json-schema._structured_payload()returnsstructured_outputwhen it is an object and otherwise falls back to the oldresultread.tests/test_claude_backend_structured_output.pyruns against a fake CLI that returns the 2.1.296 result shape. It checks that a structured call sends--json-schemawith the backend's schema and returns the requested tool call, that a result withoutstructured_outputstill returns the text reply, and that a plain call sends no schema.Tests
python -m pytest -qon macOS, rebased onmainat22c1ed8: 2345 passed (2342 onmain). The only failures are the two tests intests/test_sleep_adopt_skill_subset.pythat also fail onmainon macOS, where the temporary directory sits behind the/varsymlink.