Skip to content

Commit 9dfd9db

Browse files
authored
feat(xai): wire reasoning effort through the Grok adapter (#6627)
* feat(xai): wire reasoning effort through the Grok adapter The catalog never declared reasoningEffort for xAI and the adapter never sent reasoning_effort, so the flag was dead for every Grok model. Values are per-model and verified against the live API rather than the docs, which are wrong in three places: grok-4.5 does accept xhigh, grok-4.3 supports the parameter at all (undocumented) including none, and grok-4.20-0309-reasoning rejects it outright despite being a reasoning model. Also corrects grok-4.5's missing cachedInput and drops an inline comment the new provider TSDoc now covers. * test(xai): type the provider test helper instead of casting to any * fix(agent): correct reasoning-effort copy that still claimed GPT-5 only
1 parent afa0293 commit 9dfd9db

6 files changed

Lines changed: 256 additions & 4 deletions

File tree

apps/docs/content/docs/en/workflows/blocks/agent.mdx

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -115,6 +115,7 @@ Live tool-call chips stream for **OpenAI, Anthropic, Azure Anthropic, Google, Ve
115115
| Google | Summaries only | `gemini-3.6-flash`, `gemini-3.5-flash-lite`, `gemini-3.5-flash`, `gemini-3.1-pro-preview`, `gemini-3.1-flash-lite`, `gemini-3-flash-preview`, `gemini-2.5-pro`, `gemini-2.5-flash`, `gemini-2.5-flash-lite` |
116116
| Vertex AI | Summaries only | `vertex/gemini-3.5-flash`, `vertex/gemini-3.1-pro-preview`, `vertex/gemini-3.1-flash-lite`, `vertex/gemini-3-flash-preview`, `vertex/gemini-2.5-pro`, `vertex/gemini-2.5-flash`, `vertex/gemini-2.5-flash-lite` |
117117
| DeepSeek | Full thinking deltas | `deepseek-v4-pro`, `deepseek-v4-flash`, `deepseek-reasoner` |
118+
| xAI | Full thinking deltas | `grok-4.6`, `grok-4.5`, `grok-4.3`, `grok-4.20-multi-agent-0309` |
118119
| Groq | Full thinking deltas | `groq/openai/gpt-oss-120b`, `groq/openai/gpt-oss-20b`, `groq/openai/gpt-oss-safeguard-20b`, `groq/qwen/qwen3.6-27b` |
119120
| Meta | Not streamed | `muse-spark-1.1` |
120121
| Kimi | Full thinking deltas | `kimi-k2.6` |
@@ -142,7 +143,7 @@ The Agent reads the message from Start with `<start.input>` and returns a result
142143
{ question: "What are the memory options for the Agent block?", answer: "Four modes: None (no memory, each run is independent), Conversation (full history keyed by a conversation ID), Sliding window by messages (the N most recent messages), and Sliding window by tokens (messages up to a token budget). Memory needs a conversation ID to persist across runs." },
143144
{ question: "What is the difference between the tool usage controls (Auto, Force, None)?", answer: "In Auto, the model decides when to call a tool based on context. In Force, the model must call the tool on every run. In None, the tool is hidden from the model and never sent, which disables it without removing it from the block." },
144145
{ question: "How does the Response Format work?", answer: "It enforces structured output by providing a JSON Schema. When set, the model's response is constrained to match the schema exactly, and each field is read directly by downstream blocks using <agent.fieldName>. Without a response format, the agent returns its standard outputs: content, model, tokens, and toolCalls." },
145-
{ question: "What does the Reasoning Effort / Thinking Level setting do?", answer: "They appear only for models that support extended reasoning. Reasoning Effort (OpenAI o-series and GPT-5 models) and Thinking Level (Anthropic Claude and Gemini models with thinking) control how much compute the model spends reasoning before responding. Higher levels produce more thorough answers but cost more tokens and take longer." },
146+
{ question: "What does the Reasoning Effort / Thinking Level setting do?", answer: "They appear only for models that support extended reasoning. Reasoning Effort (OpenAI, Azure OpenAI, xAI Grok, DeepSeek, Groq, Meta, and Z.ai models that accept an effort level) and Thinking Level (Anthropic Claude and Gemini models with thinking) control how much compute the model spends reasoning before responding. Higher levels produce more thorough answers but cost more tokens and take longer." },
146147
{ question: "When should I turn on Prompt Caching?", answer: "Turn it on when the same agent runs repeatedly with a large, stable system prompt or tool set — cached input bills at a tenth of the normal input rate. Leave it off for one-off runs, because writing the cache costs 1.25x and nothing reads it back. The setting appears only for Anthropic Claude models; OpenAI and Gemini cache automatically with no setting and no write fee. Anthropic only caches a prefix of at least 1,024 tokens (2,048 on Haiku), and entries expire after five minutes of no use." },
147148
{ question: "How does max output tokens work with Anthropic models?", answer: "The Agent block uses each Anthropic model's full max output token limit by default (for example, 64,000 tokens). You can override this with the Max Output Tokens setting. For non-streaming requests that exceed the SDK's internal threshold, the provider automatically uses internal streaming to avoid timeouts." },
148149
{ question: "Can I use the Agent block with a custom or self-hosted model?", answer: "Yes. Use any Ollama or VLLM-compatible model by typing the model name directly into the model combobox, as long as it exposes a compatible API endpoint." },

apps/sim/blocks/blocks/agent.ts

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -664,7 +664,10 @@ Return ONLY the JSON array.`,
664664
},
665665
temperature: { type: 'number', description: 'Response randomness level' },
666666
maxTokens: { type: 'number', description: 'Maximum number of tokens in the response' },
667-
reasoningEffort: { type: 'string', description: 'Reasoning effort level for GPT-5 models' },
667+
reasoningEffort: {
668+
type: 'string',
669+
description: 'Reasoning effort level for models that support it',
670+
},
668671
verbosity: { type: 'string', description: 'Verbosity level for GPT-5 models' },
669672
thinkingLevel: {
670673
type: 'string',

apps/sim/providers/index.test.ts

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1405,6 +1405,26 @@ describe('executeProviderRequest — model level normalization', () => {
14051405
expect(sentRequest().verbosity).toBe('high')
14061406
})
14071407

1408+
it('keeps the reasoning effort a Grok model declares', async () => {
1409+
await executeProviderRequest('xai', {
1410+
model: 'grok-4.6',
1411+
workspaceId: 'ws-1',
1412+
reasoningEffort: 'xhigh',
1413+
})
1414+
1415+
expect(sentRequest().reasoningEffort).toBe('xhigh')
1416+
})
1417+
1418+
it('drops the reasoning effort for a Grok model that rejects the parameter', async () => {
1419+
await executeProviderRequest('xai', {
1420+
model: 'grok-4.20-0309-reasoning',
1421+
workspaceId: 'ws-1',
1422+
reasoningEffort: 'high',
1423+
})
1424+
1425+
expect(sentRequest().reasoningEffort).toBeUndefined()
1426+
})
1427+
14081428
/**
14091429
* Sim's per-model level lists drive the pickers and can lag a provider that has started
14101430
* accepting a new level, so an unrecognized level is forwarded rather than dropped: the

apps/sim/providers/models.ts

Lines changed: 14 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2153,6 +2153,9 @@ export const PROVIDER_DEFINITIONS: Record<string, ProviderDefinition> = {
21532153
},
21542154
capabilities: {
21552155
temperature: { min: 0, max: 2 },
2156+
reasoningEffort: {
2157+
values: ['low', 'medium', 'high', 'xhigh'],
2158+
},
21562159
},
21572160
contextWindow: 500000,
21582161
releaseDate: '2026-08-12',
@@ -2162,11 +2165,15 @@ export const PROVIDER_DEFINITIONS: Record<string, ProviderDefinition> = {
21622165
id: 'grok-4.5',
21632166
pricing: {
21642167
input: 2.0,
2168+
cachedInput: 0.3,
21652169
output: 6.0,
2166-
updatedAt: '2026-07-08',
2170+
updatedAt: '2026-08-12',
21672171
},
21682172
capabilities: {
21692173
temperature: { min: 0, max: 2 },
2174+
reasoningEffort: {
2175+
values: ['low', 'medium', 'high', 'xhigh'],
2176+
},
21702177
},
21712178
contextWindow: 500000,
21722179
releaseDate: '2026-07-08',
@@ -2181,6 +2188,9 @@ export const PROVIDER_DEFINITIONS: Record<string, ProviderDefinition> = {
21812188
},
21822189
capabilities: {
21832190
temperature: { min: 0, max: 2 },
2191+
reasoningEffort: {
2192+
values: ['none', 'low', 'medium', 'high', 'xhigh'],
2193+
},
21842194
},
21852195
contextWindow: 1000000,
21862196
releaseDate: '2026-04-30',
@@ -2328,6 +2338,9 @@ export const PROVIDER_DEFINITIONS: Record<string, ProviderDefinition> = {
23282338
},
23292339
capabilities: {
23302340
temperature: { min: 0, max: 2 },
2341+
reasoningEffort: {
2342+
values: ['none', 'low', 'medium', 'high', 'xhigh'],
2343+
},
23312344
},
23322345
contextWindow: 1000000,
23332346
releaseDate: '2026-03-10',
Lines changed: 197 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,197 @@
1+
/**
2+
* @vitest-environment node
3+
*/
4+
import { beforeEach, describe, expect, it, vi } from 'vitest'
5+
6+
const { mockCreate, mockExecuteProviderTool } = vi.hoisted(() => ({
7+
mockCreate: vi.fn(),
8+
mockExecuteProviderTool: vi.fn(),
9+
}))
10+
11+
vi.mock('openai', () => ({
12+
default: vi.fn().mockImplementation(
13+
class {
14+
chat = { completions: { create: mockCreate } }
15+
}
16+
),
17+
}))
18+
19+
vi.mock('@/providers', () => ({ MAX_TOOL_ITERATIONS: 20 }))
20+
21+
vi.mock('@/providers/runtime-context', () => ({
22+
executeProviderTool: mockExecuteProviderTool,
23+
}))
24+
25+
vi.mock('@/providers/models', () => ({
26+
getProviderModels: () => [],
27+
getProviderDefaultModel: () => '',
28+
}))
29+
30+
vi.mock('@/providers/attachments', () => ({
31+
formatMessagesForProvider: (messages: unknown) => messages,
32+
}))
33+
34+
vi.mock('@/providers/trace-enrichment', () => ({
35+
enrichLastModelSegmentFromChatCompletions: vi.fn(),
36+
}))
37+
38+
vi.mock('@/providers/transport', () => ({ openAICompatTransport: () => ({}) }))
39+
40+
vi.mock('@/providers/tool-schema-adapter', () => ({
41+
adaptOpenAIChatToolSchema: (tool: { id: string }) => ({
42+
type: 'function',
43+
function: { name: tool.id, parameters: {} },
44+
}),
45+
}))
46+
47+
vi.mock('@/providers/openai-compat/assistant-history', () => ({
48+
createOpenAICompatAssistantHistory: () => ({ role: 'assistant', content: '' }),
49+
}))
50+
51+
vi.mock('@/providers/openai-compat/stream-events', () => ({
52+
createOpenAICompatibleAgentEventStream: () => new ReadableStream({ start: (c) => c.close() }),
53+
}))
54+
55+
vi.mock('@/providers/stream-events', () => ({
56+
createSettledAgentEventStream: () => new ReadableStream({ start: (c) => c.close() }),
57+
}))
58+
59+
vi.mock('@/providers/streaming-execution', () => ({
60+
createStreamingExecution: vi.fn(() => ({ stream: null, execution: null })),
61+
}))
62+
63+
vi.mock('@/providers/utils', () => ({
64+
isFunctionToolCall: (toolCall: unknown) =>
65+
typeof toolCall === 'object' &&
66+
toolCall !== null &&
67+
'function' in toolCall &&
68+
(toolCall as { function?: unknown }).function != null,
69+
calculateCost: vi.fn(() => ({ input: 0, output: 0, total: 0 })),
70+
sumToolCosts: vi.fn(() => 0),
71+
prepareToolExecution: vi.fn((_tool, toolArgs) => ({
72+
toolParams: toolArgs,
73+
executionParams: toolArgs,
74+
})),
75+
prepareToolsWithUsageControl: vi.fn((tools) => ({
76+
tools,
77+
toolChoice: 'auto',
78+
forcedTools: [],
79+
hasFilteredTools: false,
80+
})),
81+
checkForForcedToolUsageOpenAI: vi.fn(() => ({
82+
hasUsedForcedTool: false,
83+
usedForcedTools: [],
84+
})),
85+
}))
86+
87+
import type { StreamingExecution } from '@/executor/types'
88+
import type { ProviderRequest, ProviderResponse, ProviderToolConfig } from '@/providers/types'
89+
import { xAIProvider } from '@/providers/xai'
90+
91+
interface ChatOptions {
92+
content?: string | null
93+
toolCalls?: Array<{ id: string; function: { name: string; arguments: string } }>
94+
}
95+
96+
function chat({ content = null, toolCalls }: ChatOptions = {}) {
97+
return {
98+
choices: [
99+
{
100+
message: { content, tool_calls: toolCalls },
101+
finish_reason: toolCalls ? 'tool_calls' : 'stop',
102+
},
103+
],
104+
usage: { prompt_tokens: 5, completion_tokens: 3, total_tokens: 8 },
105+
}
106+
}
107+
108+
function tool(name: string): ProviderToolConfig {
109+
return {
110+
id: name,
111+
name,
112+
description: 'd',
113+
params: {},
114+
parameters: { type: 'object', properties: {}, required: [] },
115+
}
116+
}
117+
118+
function run(
119+
request: Partial<ProviderRequest> = {}
120+
): Promise<ProviderResponse | StreamingExecution> {
121+
return xAIProvider.executeRequest!({
122+
model: 'grok-4.6',
123+
apiKey: 'test-key',
124+
messages: [{ role: 'user', content: 'Hi' }],
125+
...request,
126+
})
127+
}
128+
129+
const firstPayload = () => mockCreate.mock.calls[0][0]
130+
const lastPayload = () => mockCreate.mock.calls.at(-1)![0]
131+
132+
describe('xAIProvider.executeRequest', () => {
133+
beforeEach(() => {
134+
vi.clearAllMocks()
135+
mockCreate.mockResolvedValue(chat({ content: 'hello' }))
136+
mockExecuteProviderTool.mockResolvedValue({
137+
rawResponse: { success: true, output: { ok: true } },
138+
modelResponse: { success: true, output: { ok: true } },
139+
})
140+
})
141+
142+
it('maps temperature and max_completion_tokens', async () => {
143+
await run({ temperature: 0.5, maxTokens: 256 })
144+
145+
const payload = firstPayload()
146+
expect(payload.model).toBe('grok-4.6')
147+
expect(payload.temperature).toBe(0.5)
148+
expect(payload.max_completion_tokens).toBe(256)
149+
})
150+
151+
it('forwards reasoning_effort only when set to a non-default value', async () => {
152+
await run({ reasoningEffort: 'xhigh' })
153+
expect(firstPayload().reasoning_effort).toBe('xhigh')
154+
155+
mockCreate.mockClear()
156+
await run({ reasoningEffort: 'auto' })
157+
expect(firstPayload().reasoning_effort).toBeUndefined()
158+
159+
mockCreate.mockClear()
160+
await run({})
161+
expect(firstPayload().reasoning_effort).toBeUndefined()
162+
})
163+
164+
it('keeps reasoning_effort on every follow-up call in the tool loop', async () => {
165+
mockCreate
166+
.mockResolvedValueOnce(
167+
chat({ toolCalls: [{ id: 'c1', function: { name: 'known', arguments: '{"q":1}' } }] })
168+
)
169+
.mockResolvedValueOnce(chat({ content: 'done' }))
170+
171+
await run({ tools: [tool('known')], reasoningEffort: 'high' })
172+
173+
expect(mockCreate.mock.calls.length).toBeGreaterThan(1)
174+
for (const [payload] of mockCreate.mock.calls) {
175+
expect(payload.reasoning_effort).toBe('high')
176+
}
177+
})
178+
179+
it('keeps reasoning_effort on the response_format request', async () => {
180+
await run({
181+
reasoningEffort: 'low',
182+
responseFormat: { name: 'r', schema: { type: 'object', properties: {} } },
183+
})
184+
185+
const payload = lastPayload()
186+
expect(payload.response_format.type).toBe('json_schema')
187+
expect(payload.reasoning_effort).toBe('low')
188+
})
189+
190+
it('keeps reasoning_effort on the direct streaming request', async () => {
191+
await run({ reasoningEffort: 'medium', stream: true })
192+
193+
const payload = firstPayload()
194+
expect(payload.stream).toBe(true)
195+
expect(payload.reasoning_effort).toBe('medium')
196+
})
197+
})

apps/sim/providers/xai/index.ts

Lines changed: 19 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -38,6 +38,20 @@ import {
3838

3939
const logger = createLogger('XAIProvider')
4040

41+
/**
42+
* xAI's Grok models via an OpenAI-compatible chat-completions API
43+
* (`api.x.ai/v1`), with these documented deviations:
44+
* - `reasoning_effort` maps from `request.reasoningEffort`. Sim's `auto`
45+
* sentinel means "let the model pick its own default" and is never
46+
* forwarded — xAI rejects it outright with `Invalid reasoning effort`.
47+
* - Only some Grok models accept the parameter at all; the rest reject it with
48+
* `does not support parameter reasoningEffort`. Which values each model takes
49+
* is declared per-model in `capabilities.reasoningEffort`, which is also what
50+
* gates the Agent block's effort dropdown.
51+
* - Output length is capped via `max_completion_tokens`.
52+
* - `tools` and `response_format` cannot be sent in the same request, so tools
53+
* run first and the schema is applied on a follow-up pass.
54+
*/
4155
export const xAIProvider: ProviderConfig = {
4256
id: 'xai',
4357
name: 'xAI',
@@ -102,6 +116,11 @@ export const xAIProvider: ProviderConfig = {
102116

103117
if (request.temperature !== undefined) basePayload.temperature = request.temperature
104118
if (request.maxTokens != null) basePayload.max_completion_tokens = request.maxTokens
119+
120+
if (request.reasoningEffort !== undefined && request.reasoningEffort !== 'auto') {
121+
basePayload.reasoning_effort = request.reasoningEffort
122+
}
123+
105124
let preparedTools: ReturnType<typeof prepareToolsWithUsageControl> | null = null
106125

107126
if (tools?.length) {
@@ -166,7 +185,6 @@ export const xAIProvider: ProviderConfig = {
166185
try {
167186
const initialCallTime = Date.now()
168187

169-
// xAI cannot use tools and response_format together in the same request
170188
const initialPayload = { ...basePayload }
171189

172190
let originalToolChoice: any

0 commit comments

Comments
 (0)