feat(llm): opt-in strict function tools for OpenAI-compatible providers

[plombeer31]

The report

Discord #feedback-and-bugs, thegreatteacher, 2026-09-10:

I was testing mercury 2.5 from inception labs with atomic-agent. Unfortunately, the model makes a lot of tool calling mistakes without "strict": true option. Is it possible to set "strict": true through config file or do you need to add provider specific patch into atomic-agent?

Maintainer's reply:

I believe we'll need a patch to let user configure model specific parameters.

Why extraBody could not already do it

extraBody merges at the top level of the chat-completion body. OpenAI's strict mode is not a top-level field — it is tools[].function.strict, one flag per tool. And tools is in RESERVED_BODY_KEYS (src/llm/provider/openai/openai-build-body.ts:10), re-applied over the merge specifically so a passthrough can never drop the tool contract. So an operator could put strict anywhere in extraBody and it would be discarded. There was no config path to it at all.

The ToolsSupportLevel union already has a "strict" member (llm-provider.ts:35), userModels[].supportsTools accepts it (llm-config.ts:392, config-schema.ts:954), and it survives a config round-trip — but nothing consumes it. The only reads anywhere are entry.supportsTools === "none" for a display tag (format-model-details.ts:42, model-search.ts:63). This PR does not wire that level; it stays dormant. See the note on #395 below.

The knob

strictTools on the provider entry — config v63, additive, off by default:

"llm": { "providers": [{
  "id": "mercury", "kind": "openai-compatible",
  "baseUrl": "https://api.inceptionlabs.ai/v1",
  "defaultChatModel": "mercury", "strictTools": true
}] }

Parsed alongside supportsTools / maxOutputTokens in llm-config.ts, carried on LlmProviderConfigEntry, and handed to OpenAiProvider by all five OpenAI-shaped factories (openai-compatible, qwen-openai-compatible, openrouter, aimlapi, gemini — every one of them extends OpenAiProvider). It reaches buildOpenAiChatBody as a sixth argument, exactly the seam extraBody and maxOutputTokens already use, and the transform runs before the extraBody merge so the reserved-key restore keeps the transformed array. Absent, the request body is byte-identical to today.

The transform (openai-strict-tools.ts)

Getting this wrong is worse than the bug: a schema strict mode rejects 400s the whole request, not the one tool. Applied recursively to every object schema:

• additionalProperties: false;
• every key of properties present in required — an optional parameter is widened to include "null" (and null added to its enum members where it has an enum), never omitted from required;
• keyword allowlist, not a blocklist.

Kept verbatim: type, enum, description, title, $ref (+ $defs, recursed). Recursed: properties, items, anyOf. oneOf is renamed to anyOf — interchangeable for a constrained decoder, and anyOf is the spelling strict mode documents. Recomputed: required, additionalProperties.

What I stripped, and why. I scanned every keyword present at a schema position across all 85 bundled descriptors (default-tool-args-schemas.ts + github-tool-args-schemas.ts + the inline reply/finish schemas). The full set in use is type, properties, required, additionalProperties, enum, items, anyOf, minItems, maxItems, minimum — plus minLength on reply.text. So the only things actually stripped from this repo's own schemas are minItems, maxItems, minimum, minLength (fusion.delegate, vision.describe, reply). That is safe for the reason default-tool-args-schemas.ts states in its own header: these schemas guard the shape, and the runtime validators — not the provider — do the value-range checks. An empty reply.text is still rejected by validateBatch and routed to the repair prompt. Dropping a bound loosens the schema; it never invalidates a call the agent would otherwise have accepted.

The allowlist also covers what an MCP server's inputSchema may carry and this repo does not (pattern, format, default, examples, const, uniqueItems, multipleOf, exclusive*, $schema, …): those are dropped too, so a new keyword degrades to "ignored" rather than "sent and 400'd".

Refused rather than mangled. A tools array may mix strict and non-strict functions, so anything that cannot be expressed is emitted with strict: false and its schema untouched:

refused	why
os.http.request	headers is additionalProperties: {type:"string"} and body's object branch is a bare {type:"object"} — both free-form maps carrying the actual payload
mcp.prompt.get	arguments is a server-defined string map, same shape
any descriptor with no argsJsonSchema	descriptorToJsonSchema's fallback is {type:"object", properties:{}, additionalProperties:true}

That is the answer to the additionalProperties: true + empty properties question: closing it would leave a schema admitting only {}, i.e. silently deleting the tool's arguments — so the tool travels unconstrained instead. An object that declares properties and additionalProperties: true (os.fs.archive.extract.limits, the sole case) is closed: the three declared keys are its whole documented contract, so forbidding extras loses nothing a caller was entitled to send. Also refused: allOf/not/if/then/else, tuple items, an array with no items, a non-object root, a schema with nothing constraining it.

Result: 80 of 82 emitted tools are strict, the two above are not.

The response side (the part that is not just schema shuffling)

Making optionals nullable-and-required changes what the model sends: "userName": null where it used to omit the key. Several tools branch on presence, not value — os.git.init (args.userName !== undefined), memory.profile.set (rawArgs.pinned, rawArgs.keywords) — and would take the "caller asked for this" branch with nothing to put in it. So on a provider that opted in, OpenAiProvider wraps its tool-call adapter to drop top-level null arguments, leaving the parsed call identical to what those tools saw before. Nested nulls are left alone: that is data the model meant to send. Nothing changes for a provider without the flag.

Tests

src/llm/provider/openai/openai-strict-tools.test.ts (110 tests):

• The conformance sweep that matters: every descriptor in DEFAULT_TOOL_DESCRIPTORS goes through descriptorsToOpenAiTools (the same adapter a real turn uses) and then the transform, and each result is checked against every strict rule recursively — keyword allowlist, additionalProperties: false, required ⊇ and ⊆ properties, array items, anyOf branches, $defs. One it() per tool, so a failure names the tool. A new tool with an unsupported keyword fails here instead of as a field 400.
• The refusal list is pinned (["os__http__request", "mcp__prompt__get"]), so a new free-form-map tool is a decision, not a surprise.
• Table-driven unit cases: optional → nullable + required, optional enum gets null in its members, optional anyOf gains a null branch, nested objects, arrays of objects, the strip list, oneOf → anyOf, zero-argument tool, and six refusal shapes.
• Idempotence, over both the fixtures and the whole catalog: f(f(x)) deep-equals f(x). Non-mutation of the input schema.
• The null-argument drop, top-level and nested.

openai-build-body.test.ts (+4): flag absent ⇒ JSON.stringify(body) identical to the pre-flag call including key order; flag on ⇒ tools rewritten and marked; the transformed array survives an extraBody that tries to replace tools; no tools key when the request carries none.

llm-config.test.ts (+4): strictTools round-trips true/false, absent by default, rejects non-booleans. 62 added to ACCEPTED_CONFIG_VERSIONS with the v63 bump.

npm run lint                                  # tsc --noEmit, clean
npx vitest run src/llm/provider src/config    # 60 files, 936 tests, all pass

Proof the tests are not vacuous

Stash the src change, keep the tests. Reverting the six modified src files and removing openai-strict-tools.ts: 5 tests fail —

• openai-strict-tools.test.ts fails to load at all (the module is gone);
• llm-config › round-trips the opt-in flag, round-trips an explicit opt-out, rejects anything that is not a boolean;
• openai-build-body › rewrites the tools and marks them strict when the flag is on, keeps the strict tools over an extraBody that tries to replace them.

The two control cases keep passing, which is their job: leaves the body byte-identical when the flag is absent and sends no tools key at all when the request carries none.

Five targeted mutations of the restored transform, each reverted after:

mutation	result
required kept as the input's instead of every key	66 of 110 fail
keyword allowlist removed (nothing stripped)	6 fail — the schemas carrying minItems/maxItems/minimum/minLength
free-form-object refusal removed	2 fail — the pinned refusal list and the fallback case
withoutTopLevelNullArgs made a no-op	2 fail
adapter wrapper made a pass-through	1 fails

Overlap with #395

While this was being written, #395 landed on the same Discord report from another session, taking the other design: it wires the dormant supportsTools: "strict" catalog level per model, threaded through bootstrap → agent-loop → step-executor → both directions of the adapter. It is not merged, and the two collide (both touch openai-tool-call-adapter.ts and llm-config.ts). They are alternatives — please take one. The differences worth weighing:

• Granularity. feat(llm): honour supportsTools: "strict" on the OpenAI tools payload #395 is per model (userModels[].supportsTools: "strict"), which is the more correct unit and makes the dead enum member live. This is per provider entry, which is one flag on the entry the operator already edits and needs no plumbing through the agent loop — the whole change lives at the buildOpenAiChatBody seam, so it does not touch bootstrap.ts, agent-loop.ts or step-executor.ts at all.
• Coverage. feat(llm): honour supportsTools: "strict" on the OpenAI tools payload #395 converts 77 of 82 and refuses on any bound keyword. This converts 80 of 82 because it strips bounds instead of refusing on them — fusion.delegate, vision.describe and reply become strict, and reply is the tool models botch most.
• Both handle the null-argument consequence the same way and both refuse the free-form-map tools.

A merged design (per-model level as the source of truth, this transform's strip-not-refuse behaviour, one adapter wrapper) is straightforward if a maintainer wants it; say the word and I will fold them.

Not covered / could not verify

• No live provider was called. Every assertion here is about the bytes we emit. Nobody has run this against mercury-2.5, against OpenAI, or against any other endpoint. The reporter is the natural first tester. The least universal shape emitted is the anyOf: [..., {type:"null"}] widening for an optional union; if a provider rejects it, the fix is a tighter allowlist, not a looser one.
• The dormant supportsTools: "strict" level is still dormant after this PR. Deliberate — wiring it is feat(llm): honour supportsTools: "strict" on the OpenAI tools payload #395's design, and doing both would be two config surfaces for one behaviour.
• No TUI surface. strictTools is hand-edited in config.json; the Providers tab does not offer it.
• The refusal list is behaviour, not policy. A new default tool with a free-form map silently joins it; the pinned test is what turns that into a decision.
• MCP inputSchemas are handled generically, not tested against real servers. Anything the allowlist does not cover is refused, so the failure mode is "that tool is not strict", never a 400 — but I have not run this against a live MCP server's schemas.
• Nested optionals in a converted third-party schema arrive as explicit null and reach the tool that way; the null-drop is top-level only, by design.
• Nothing here touches the grammar/local path — llama-server ignores tools.
• Full suite not run (the repo's known ~10 flaky failures); the targeted src/llm/provider src/config sweep is green at 936/936.