Skip to content

Migrate /try chat to Anthropic /v1/messages + table rendering and UX polish - #5

Merged
1bcMax merged 1 commit into
masterfrom
feat/anthropic-messages-protocol
May 29, 2026
Merged

Migrate /try chat to Anthropic /v1/messages + table rendering and UX polish#5
1bcMax merged 1 commit into
masterfrom
feat/anthropic-messages-protocol

Conversation

@KillerQueen-Z

Copy link
Copy Markdown
Collaborator

What

Free NVIDIA-hosted models (DeepSeek V4 Flash, Qwen3 Coder, Llama 4 Maverick) leaked raw `<tool_call>{...}</tool_call>` text into the chat — the OpenAI-compat gateway path was lossy when translating tool schemas for them. Switching the chat endpoint to Anthropic-native `/v1/messages` — the same path Franklin CLI uses — preserves the schema end-to-end so every model (free or paid) actually calls tools.

Verified with Qwen3 Coder 480B in the live dev server: tool calls fire as native `tool_use` blocks, no `<tool_call>` text leak.

Bundles a few related reliability + rendering + UX fixes that surfaced during the migration.

Protocol migration

  • `CHAT_ENDPOINT` → `/v1/messages` with `anthropic-version: 2023-06-01` header
  • Tool schemas rewritten: OpenAI `{ type:"function", function:{ name, description, parameters } }` → Anthropic `{ name, description, input_schema }`
  • Request body: `system` lifted to top level, `max_tokens` required, `tool_choice` reshaped to `{ type:"tool", name }`
  • Response parsing: read `content[]` blocks (`tool_use` / `text`) instead of `choices[0].message.tool_calls`
  • Tool results: Anthropic `tool_result` content blocks batched into a single user message instead of OpenAI `role:"tool"` messages
  • Vision: OpenAI `image_url` → Anthropic `image.source`, auto-detecting `data:` URLs (base64 source) vs http(s) URLs (url source)
  • Proxy: `messages` added to path allowlist; `anthropic-version` added to forwarded request headers

Chat reliability

  • Payment rejection rollback — when a wallet signature is cancelled, the orphan user message is removed from history instead of persisting into the conversation list and titling a stray "New chat" that leaks to other windows on next hydrate
  • Empty conversations auto-delete — `useChatHistory.setMessages` drops a conversation (sidebar + GCS DELETE) when its message list goes to zero, instead of keeping an empty shell
  • Error path cleanup — catch block now clears `steps` + `activeTool`, so ` · thinking` no longer keeps spinning after a wallet rejection error toast

GFM table rendering

Wide assistant replies (e.g. smart-money comparison grids, vocab grids) used to fall through as raw `| col | col |` text. Now parsed and rendered.

  • `parseTableAt` detects header + `|---|---|` separator + body rows; tolerant to leading/trailing pipes and `:---` / `---:` / `:---:` alignment
  • `TableBlock` renders the parsed table inside a card with a header bar carrying Copy as Markdown / Download CSV (RFC 4180 quoted) / Expand to fullscreen overlay — matches the existing `CodeBlock` pattern
  • Horizontal scroll wrapper so wide tables don't break the chat column on narrow viewports

UI polish

  • Wallet pill restyle — single-row layout (network chip + balance + address inline), no dead zone between content and disconnect icon; quieter Base chip; ghost disconnect button with hover affordance
  • "OpenAI O3" label (was bare "O3") — aligns with same-group naming (`GPT-5.3 Codex`, `DeepSeek V4 Pro`, `Grok 4.1 Fast`)
  • 3 new prediction-market starter prompts replace the heavy-to-implement selfie / cold-call / character-consistency cases — picked to be broad, evergreen, always-returns-data queries:
    • Movers — 24h biggest odds movers leaderboard
    • Sports — hottest sports markets right now
    • Tech — most-active AI / tech markets right now

Cleanup

  • Drop the dead streaming chat path in `runChat` (`supportsTools()` was hardcoded to `true` so the streaming fallback was unreachable; it also referenced OpenAI SSE shape that wouldn't have worked anymore against the new endpoint)
  • Remove the unused `streaming` state + `setStreaming` calls
  • Drop the `casePhone` i18n key (no longer rendered)

Verification

  • `tsc --noEmit` ✅
  • `eslint` — clean on touched files; the one pre-existing `FranklinChat.tsx:66` `react-hooks/set-state-in-effect` error is unrelated to this PR (file unchanged in that area)
  • Tool-calls verified live in dev against Qwen3 Coder 480B (free NVIDIA model) — tool flow works end-to-end via Anthropic protocol
  • Table rendering verified live against assistant output with `| col |` markdown
  • Wallet rejection / orphan-msg-rollback verified in dev by cancelling MetaMask signature

Follow-ups (not in this PR)

  • Friendlier 429 error display — currently NVIDIA rate-limit responses surface as raw `{"status":429,...}` JSON to the user; `upstreamErrorMessage` already has the translation but only runs on 200-with-error-content, not on actual HTTP 429
  • Streaming over Anthropic SSE if slow models start exceeding Cloud Run / Cloudflare timeouts (the rewrite is non-streaming for now — fine for the tool-loop pattern but won't give a typewriter effect)

Free NVIDIA-hosted models (DeepSeek V4 Flash, Qwen3 Coder, Llama 4
Maverick) leaked raw <tool_call>{...}</tool_call> text into the chat
because the OpenAI-compat gateway path lost fidelity translating tool
schemas for them. Anthropic-native /v1/messages preserves the schema
end-to-end — same path Franklin CLI uses — so every model now actually
calls tools.

Protocol
- CHAT_ENDPOINT → /v1/messages with anthropic-version: 2023-06-01
- Tool schemas: OpenAI function.parameters → Anthropic input_schema
- Request body: system as top-level field, max_tokens required,
  tool_choice as {type:"tool", name}
- Response parsing: content[] blocks (tool_use / text) instead of
  choices[0].message.tool_calls
- Tool results: Anthropic tool_result content blocks batched into one
  user message instead of OpenAI role:"tool" messages
- Vision: OpenAI image_url → Anthropic image.source with auto-detect
  for data: URLs vs http(s) URLs
- Proxy allowlist: /messages + forward anthropic-version header

Chat reliability
- Payment-rejection rollback: orphan user msg no longer persists into
  the conversation list or leaks to other windows on next hydrate
- Empty conversations auto-delete (sidebar + GCS) instead of becoming
  stray "New chat" entries
- Error path clears the activity log so "<model> thinking" no longer
  keeps spinning after a wallet rejection

Table rendering
- GFM tables (| col | col | + |---|---|) now render as <table> instead
  of falling through as raw text — fixes wide assistant replies (e.g.
  smart-money comparison grids)
- TableBlock wraps in a card with header bar: Copy as Markdown,
  Download CSV (RFC 4180 quoted), Expand to fullscreen overlay
- Column alignment via :--- / ---: / :---: GFM syntax
- Horizontal scroll wrapper so wide tables don't break the chat column

UI polish
- Wallet pill: single row layout (network chip + balance + address
  inline), no dead zone between content and the disconnect icon;
  quieter Base chip, ghost disconnect button
- "OpenAI O3" label (was "O3") — aligns with same-group naming
- 3 fresh prediction-market starter prompts replace selfie/cold-call/
  character-consistency cases (broad, evergreen, always-returns-data
  queries: 24h movers, top sports markets, top AI/tech markets)

Cleanup
- Drop dead streaming chat path (supportsTools() was always true)
- Remove unused streaming state + setStreaming calls
- Drop casePhone i18n key (no longer rendered)

Verified: tsc --noEmit clean; lint clean on touched files.
@1bcMax
1bcMax merged commit aed28b6 into master May 29, 2026
1 check failed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant