Skip to content

Conversation

@dfischer-mw
Copy link

@dfischer-mw dfischer-mw commented Jan 9, 2026

Issue

Fixes #7488

Problem

When using Mistral or Devstral models with tool calls (especially via vLLM or openai-compatible endpoints), requests fail with:

  mistral_common.exceptions.InvalidMessageStructureException:
  Unexpected role 'user' after role 'tool'

I guess this occurs because Mistral enforces strict message ordering: a tool message must be followed by an assistant message before any user message.

I then looked in the code and saw the transform logic to fix message ordering already existed in transform.ts, but it never ran because the middleware in llm.ts line 187 was transforming the wrong parameter (args.params.prompt instead of args.params.messages) (im TS noob so please help me out here if wrong).

The changes I did:

  1. The middleware stuff in src/session/llm.ts: Changed transform to target args.params.messages instead of prompt
  2. Then included more model name variants (src/provider/transform.ts): Now detects Mistral variants including:
    • Devstral models
    • Models served via openai-compatible providers
    • Checks both model.id and model.api.id
  3. Up tests(test/provider/transform.test.ts): 6 new

The flow should now be from userassistant (tool_calls) → tooluser to userassistant (tool_calls) → toolassistant ("Done.") → user
I hope this meets your standards. :)

Fixes InvalidMessageStructureException when using Mistral models with tool
calls. Middleware was transforming wrong parameter, preventing the fix from
applying. Also improves Mistral model detection for Devstral variants.
@github-actions
Copy link
Contributor

github-actions bot commented Jan 9, 2026

The following comment was made by an LLM, it may be inaccurate:

No duplicate PRs found

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Mistral models fail with "Unexpected role 'user' after role 'tool'" error on custom hosted vLLM endpoint.

1 participant