Skip to content

feat(ai, ai-harness, ai-mcp, ai-acp, ai-harness-cli): durable waits in the log, at the boundary, in background runs, and on every transport - #1817

Open
AlemTuzlak wants to merge 12 commits into
feat/harness-redactionfrom
feat/harness-waits
Open

AlemTuzlak wants to merge 12 commits into
feat/harness-redactionfrom
feat/harness-waits

Conversation

@AlemTuzlak

Copy link
Copy Markdown
Contributor

A run that waits for a person now waits in the log. Any server can answer the wait, a server that restarts picks it up again, and a wait can expire. Approvals, questions, sign-ins, client tools, and human waits share one Wait shape on every transport. This is PR 7 of the harness redesign stack for #1755. It targets #1815 (feat/harness-redaction).

🎯 Changes

  • One Wait type.
    • Wait, WaitKind (approval | question | sign-in | client-tool | human), waitKind, isApproval, waitOfInterrupt, and waitOfQuestion.
    • Waits are harness.wait records in the log. SessionSnapshot.pendingWaits lists them.
    • A wait can have expiresAt. Only sign-in has a default (15 minutes). An expired wait ends expired, also on a worker.
    • session.waitFor({ runId?, signal? }) resolves with the next open wait. session.settled({ inputId, signal? }) now takes one object.
    • An operator access action answers human waits.
  • Waits at the boundary.
    • permissions() decides before the calls run and opens one approval wait per call. An answer can be true / false, 'once' | 'always' | 'reject', or { answer, message? }.
    • The question tool is its own wait.
    • step.ask(question) in durable tools, durable agent runs, and commands is a wait that only the log holds. It survives a restart, is checked against its JSON Schema, and can be secret.
    • ctx.session.ask without live throws, and its error names the places that can wait (D9). Commands that called step.ask run again from the top after a host stop.
    • Coding tools decide outside paths, links, and the formatter before the call (K4).
  • Live waits. MCP elicitation and OAuth sign-in are live waits of the calling run. Turn interrupts with expiresAt end on time.
  • Background runs wait.
    • An agent run suspends on a wait (status waiting) and resumes after the answer, also after a restart, on any host.
    • It pauses on a permissions() approval through the new core SubagentBinding.interrupts field.
  • One wait shape on every transport.
    • The view and client show pendingWaits and background waits. resolve answers turn interrupts and background-run waits. answer answers step.ask and live questions.
    • The MCP server reports waits per operation (E9). ACP maps every kind.
    • The CLI answers by the wait schema. --print exits 3 on any wait.
    • host.submit passes the wait, so an operator answer works.

Docs: a new docs/harness/waits.md, and updates to permissions, inputs, acp, cli, mcp-server, durable-tools, custom-ui, subagents, access, plugins, coding-tools, chat/subagents, and others. Changesets for @tanstack/ai, @tanstack/ai-harness, @tanstack/ai-mcp, @tanstack/ai-acp, and @tanstack/ai-harness-cli.

Decisions made while building:

  1. Core SubagentBinding.interrupts (approved by Alem). An agent's own interrupt wins on an id clash. Subagents that the model calls as tools inside a turn still cannot ask: they are refused, because their resume would ask again.
  2. The question tool stays a lead-turn tool. Agents do not get plugin tools, so a background agent has no question tool. The docs say so.
  3. The CLI exit code for "needs action" is 3 (it was 2), and it fires on any wait, including a question, where --print used to hang.
  4. A session that does not own the log reads waits and interrupts from the log. Before, a session open on host B never saw an approval that host A opened.
  5. Bug fixed on the way: every yes/no answer to a permissions() approval was refused, because the answer schema took only an object.

✅ Checklist

  • I have followed the steps in the Contributing guide.
  • I have tested code changes locally with pnpm run test:pr, or these tests do not apply to this pull request.
  • I fully understand the code in this pull request, including any code generated with AI assistance.
  • Docs: I updated docs/ for this change, or this change is not user-facing.
  • Changeset: I added a changeset (pnpm changeset), or this PR does not change a published package.

🚀 Release Impact

  • This change affects published code, and I have generated a changeset.
  • This change is docs/CI/dev-only (no release).

Testing

Commands run:

  1. Builds of ai, ai-persistence, ai-harness, and ai-mcp. test:types of those plus ai-acp, ai-harness-cli, ai-code-mode, ai-skills, and ai-sandbox. All pass.
  2. Each node ran its new tests and the tests of the source files it changed, with --maxWorkers=2. Examples: waits, wait-records, step-ask, json-schema, permissions, question, workspace-tools, live-waits, background-waits, remote-waits, view, client, protocol, access, input-lane, and the ai-mcp, ai-acp, and CLI tests. All pass.
  3. test:oxlint of ai-harness, pnpm test:docs, kiira check on the changed pages, and oxfmt --check on every changed file.

Not run locally: the full suites, knip, E2E (including the new harness-waits.spec.ts), and coverage. CI runs them.

Manual test:

  1. Run pnpm --dir packages/ai-harness exec vitest run tests/step-ask.test.ts tests/background-waits.test.ts tests/remote-waits.test.ts.
  2. Read the restart test in tests/step-ask.test.ts: a durable tool asks, the host stops, and the answer reaches the run on another host.
  3. Read tests/remote-waits.test.ts: host B sees and answers an approval that host A opened.

How this PR makes testing easy: the new test files pin each part: waits, wait-records, step-ask, live-waits, background-waits, and remote-waits. testing/e2e/tests/harness-waits.spec.ts answers an approval on a second server and lets a wait expire.

Risk / rollback

High. Every approval and question now goes through the wait records, ctx.session.ask throws without live, and settled takes an object. The harness is not released yet, and all callers in this repo are moved. To undo, revert this PR.

Public API change

Before

await session.settled(inputId)
const answer = await ctx.session.ask({ name: 'confirm', message: 'Deploy?' })

After

await session.settled({ inputId, signal })
const wait = await session.waitFor({ runId })
const answer = await step.ask({ name: 'confirm', message: 'Deploy?', expiresAt: Date.now() + 60_000 })
// core: SubagentBinding.interrupts

…itFor, settled({ inputId, signal }), and operator answers to human waits
… background-waits (stopped mid-way, not checked)
…stion tool, step.ask), live MCP and sign-in waits, and background runs that suspend and resume
…e on boundary approvals through SubagentBinding.interrupts, and every transport uses one wait shape
…on, step.ask, approvals at the boundary, background waits, and waits on every transport
@changeset-bot

changeset-bot Bot commented Oct 18, 2026 •

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: eb3279e

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 53 packages
Name Type
@tanstack/ai-harness-cli Minor
@tanstack/ai-harness Minor
@tanstack/ai-mcp Minor
@tanstack/ai Minor
@tanstack/ai-acp Minor
@tanstack/ai-code-mode Patch
@tanstack/ai-sandbox Patch
@tanstack/ai-skills Patch
@tanstack/ai-angular Patch
@tanstack/ai-anthropic Patch
@tanstack/ai-bedrock Patch
@tanstack/ai-byteplus Patch
@tanstack/ai-claude-code Patch
@tanstack/ai-client Patch
@tanstack/ai-cloudflare Patch
@tanstack/ai-code-mode-snippets Patch
@tanstack/ai-codex Patch
@tanstack/ai-cohere Patch
@tanstack/ai-compaction Patch
@tanstack/ai-devtools-core Patch
@tanstack/ai-durable-stream Patch
@tanstack/ai-elevenlabs Patch
@tanstack/ai-fal Patch
@tanstack/ai-gemini Patch
@tanstack/ai-grok-build Patch
@tanstack/ai-grok Patch
@tanstack/ai-groq Patch
@tanstack/ai-llmgateway Patch
@tanstack/ai-lovable Patch
@tanstack/ai-memory Patch
@tanstack/ai-mistral Patch
@tanstack/ai-octane Patch
@tanstack/ai-ollama Patch
@tanstack/ai-ollaya Patch
@tanstack/ai-openai Patch
@tanstack/ai-opencode Patch
@tanstack/ai-openrouter Patch
@tanstack/ai-perplexity Patch
@tanstack/ai-persistence Patch
@tanstack/ai-preact Patch
@tanstack/ai-react Patch
@tanstack/ai-reactor Patch
@tanstack/ai-remix Patch
@tanstack/ai-sandbox-cloudflare Patch
@tanstack/ai-solid Patch
@tanstack/ai-svelte Patch
@tanstack/ai-typesafe Patch
@tanstack/ai-vercel-gateway Patch
@tanstack/ai-vertex Patch
@tanstack/ai-vue Patch
@tanstack/ai-worldlabs Patch
@tanstack/openai-base Patch
ag-ui Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@coderabbitai

coderabbitai Bot commented Oct 18, 2026 •

Copy link
Copy Markdown
Contributor

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration
  • Configuration used: Repository: TanStack/ai/.coderabbit.yaml
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: a893d434-66e4-499c-b7a3-f62b6c955085

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@nx-cloud

nx-cloud Bot commented Oct 18, 2026 •

Copy link
Copy Markdown

View your CI Pipeline Execution ↗ for commit eb3279e

Command Status Duration Result
nx build @tanstack/ai-isolate-quickjs-bun ✅ Succeeded 3s View ↗

☁️ Nx Cloud last updated this comment at 2026-10-18 08:00:12 UTC

@pkg-pr-new

pkg-pr-new Bot commented Oct 18, 2026 •

Copy link
Copy Markdown

Open in StackBlitz

@tanstack/ai

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai@1817

@tanstack/ai-acp

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-acp@1817

@tanstack/ai-angular

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-angular@1817

@tanstack/ai-anthropic

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-anthropic@1817

@tanstack/ai-bedrock

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-bedrock@1817

@tanstack/ai-byteplus

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-byteplus@1817

@tanstack/ai-claude-code

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-claude-code@1817

@tanstack/ai-client

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-client@1817

@tanstack/ai-cloudflare

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-cloudflare@1817

@tanstack/ai-code-mode

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-code-mode@1817

@tanstack/ai-code-mode-snippets

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-code-mode-snippets@1817

@tanstack/ai-codex

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-codex@1817

@tanstack/ai-cohere

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-cohere@1817

@tanstack/ai-compaction

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-compaction@1817

@tanstack/ai-devtools-core

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-devtools-core@1817

@tanstack/ai-durable-stream

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-durable-stream@1817

@tanstack/ai-elevenlabs

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-elevenlabs@1817

@tanstack/ai-event-client

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-event-client@1817

@tanstack/ai-fal

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-fal@1817

@tanstack/ai-gemini

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-gemini@1817

@tanstack/ai-grok

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-grok@1817

@tanstack/ai-grok-build

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-grok-build@1817

@tanstack/ai-groq

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-groq@1817

@tanstack/ai-harness

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-harness@1817

@tanstack/ai-harness-cli

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-harness-cli@1817

@tanstack/ai-isolate-cloudflare

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-isolate-cloudflare@1817

@tanstack/ai-isolate-daytona

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-isolate-daytona@1817

@tanstack/ai-isolate-e2b

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-isolate-e2b@1817

@tanstack/ai-isolate-node

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-isolate-node@1817

@tanstack/ai-isolate-quickjs

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-isolate-quickjs@1817

@tanstack/ai-isolate-quickjs-bun

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-isolate-quickjs-bun@1817

@tanstack/ai-llmgateway

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-llmgateway@1817

@tanstack/ai-lovable

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-lovable@1817

@tanstack/ai-mcp

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-mcp@1817

@tanstack/ai-memory

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-memory@1817

@tanstack/ai-mistral

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-mistral@1817

@tanstack/ai-octane

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-octane@1817

@tanstack/ai-ollama

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-ollama@1817

@tanstack/ai-ollaya

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-ollaya@1817

@tanstack/ai-openai

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-openai@1817

@tanstack/ai-opencode

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-opencode@1817

@tanstack/ai-openrouter

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-openrouter@1817

@tanstack/ai-perplexity

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-perplexity@1817

@tanstack/ai-persistence

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-persistence@1817

@tanstack/ai-preact

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-preact@1817

@tanstack/ai-react

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-react@1817

@tanstack/ai-react-ui

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-react-ui@1817

@tanstack/ai-reactor

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-reactor@1817

@tanstack/ai-remix

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-remix@1817

@tanstack/ai-sandbox

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-sandbox@1817

@tanstack/ai-sandbox-blaxel

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-sandbox-blaxel@1817

@tanstack/ai-sandbox-boxd

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-sandbox-boxd@1817

@tanstack/ai-sandbox-cloudflare

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-sandbox-cloudflare@1817

@tanstack/ai-sandbox-daytona

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-sandbox-daytona@1817

@tanstack/ai-sandbox-docker

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-sandbox-docker@1817

@tanstack/ai-sandbox-e2b

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-sandbox-e2b@1817

@tanstack/ai-sandbox-local-process

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-sandbox-local-process@1817

@tanstack/ai-sandbox-railway

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-sandbox-railway@1817

@tanstack/ai-sandbox-sprites

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-sandbox-sprites@1817

@tanstack/ai-sandbox-upstash-box

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-sandbox-upstash-box@1817

@tanstack/ai-sandbox-vercel

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-sandbox-vercel@1817

@tanstack/ai-skills

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-skills@1817

@tanstack/ai-solid

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-solid@1817

@tanstack/ai-solid-ui

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-solid-ui@1817

@tanstack/ai-svelte

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-svelte@1817

@tanstack/ai-typesafe

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-typesafe@1817

@tanstack/ai-utils

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-utils@1817

@tanstack/ai-vercel-gateway

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-vercel-gateway@1817

@tanstack/ai-vertex

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-vertex@1817

@tanstack/ai-vue

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-vue@1817

@tanstack/ai-vue-ui

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-vue-ui@1817

@tanstack/ai-worldlabs

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-worldlabs@1817

@tanstack/openai-base

npm i https://pkg.pr.new/TanStack/ai/@tanstack/openai-base@1817

@tanstack/preact-ai-devtools

npm i https://pkg.pr.new/TanStack/ai/@tanstack/preact-ai-devtools@1817

@tanstack/react-ai-devtools

npm i https://pkg.pr.new/TanStack/ai/@tanstack/react-ai-devtools@1817

@tanstack/solid-ai-devtools

npm i https://pkg.pr.new/TanStack/ai/@tanstack/solid-ai-devtools@1817

@tanstack/svelte-ai-devtools

npm i https://pkg.pr.new/TanStack/ai/@tanstack/svelte-ai-devtools@1817

commit: eb3279e

@github-actions

Copy link
Copy Markdown
Contributor

Coverage

✅ Coverage held across 68 compared package(s).

Each package is measured twice in this job — on this PR and on its merge-base with main — and compared. A metric fails only when it is at or above 60% on the merge-base and under 60% on this PR. Other drops are listed but do not fail. Packages your PR doesn't affect are not measured and not listed.

package statements branches functions lines
ai 91.64% (0.00) 85.34% (+0.01) 92.42% (+0.02) 93.28% (0.00) ok
ai-acp 82.60% (+0.29) 71.10% (-0.54) 78.24% (+1.24) 84.54% (+0.41) ok
ai-angular 74.10% (0.00) 62.20% (0.00) 61.65% (0.00) 75.60% (0.00) ok
ai-anthropic 92.00% (0.00) 86.73% (0.00) 95.57% (0.00) 92.41% (0.00) ok
ai-bedrock 90.11% (0.00) 82.82% (0.00) 94.20% (0.00) 91.33% (0.00) ok
ai-byteplus 97.06% (0.00) 91.66% (0.00) 98.27% (0.00) 98.49% (0.00) ok
ai-claude-code 88.93% (0.00) 76.88% (0.00) 86.88% (0.00) 90.40% (0.00) ok
ai-client 87.96% (0.00) 80.38% (0.00) 87.13% (0.00) 90.26% (0.00) ok
ai-cloudflare 85.55% (0.00) 76.44% (0.00) 85.48% (0.00) 86.79% (0.00) ok
ai-code-mode 92.71% (0.00) 85.66% (0.00) 89.06% (0.00) 93.51% (0.00) ok
ai-code-mode-snippets 87.13% (0.00) 75.67% (0.00) 91.11% (0.00) 88.39% (0.00) ok
ai-codex 94.55% (0.00) 85.10% (0.00) 89.28% (0.00) 95.56% (0.00) ok
ai-cohere 88.97% (0.00) 88.37% (0.00) 85.71% (0.00) 91.80% (0.00) ok
ai-compaction 97.38% (0.00) 92.79% (0.00) 100.00% (0.00) 98.88% (0.00) ok
ai-devtools 83.13% (0.00) 72.77% (0.00) 87.07% (0.00) 85.22% (0.00) ok
ai-durable-stream 86.70% (0.00) 82.25% (0.00) 92.30% (0.00) 88.72% (0.00) ok
ai-elevenlabs 74.68% (0.00) 69.59% (0.00) 59.13% (0.00) 75.46% (0.00) ok
ai-fal 91.10% (0.00) 82.63% (0.00) 92.40% (0.00) 91.60% (0.00) ok
ai-gemini 68.33% (0.00) 67.88% (0.00) 64.62% (0.00) 68.93% (0.00) ok
ai-grok 48.85% (0.00) 53.49% (0.00) 56.46% (0.00) 49.67% (0.00) ok
ai-grok-build 79.19% (0.00) 63.56% (0.00) 77.39% (0.00) 81.02% (0.00) ok
ai-groq 52.54% (0.00) 47.73% (0.00) 67.39% (0.00) 54.66% (0.00) ok
ai-harness 93.53% (-0.25) 87.53% (-0.42) 93.56% (-0.43) 95.24% (-0.17) ok
ai-harness-cli 90.81% (-0.31) 82.69% (-2.08) 86.48% (+1.52) 91.91% (-0.60) ok
ai-isolate-cloudflare 94.11% (0.00) 70.66% (0.00) 100.00% (0.00) 93.96% (0.00) ok
ai-isolate-daytona 90.25% (0.00) 80.59% (0.00) 100.00% (0.00) 90.15% (0.00) ok
ai-isolate-e2b 92.11% (0.00) 81.48% (0.00) 97.36% (0.00) 95.05% (0.00) ok
ai-isolate-node 84.00% (0.00) 43.39% (0.00) 100.00% (0.00) 83.78% (0.00) ok
ai-isolate-quickjs 89.07% (0.00) 76.41% (0.00) 90.90% (0.00) 90.30% (0.00) ok
ai-isolate-quickjs-bun 7.76% (0.00) 8.08% (0.00) 5.00% (0.00) 8.33% (0.00) ok
ai-llmgateway 97.67% (0.00) 100.00% (0.00) 88.88% (0.00) 97.67% (0.00) ok
ai-lovable 73.21% (0.00) 55.63% (0.00) 85.07% (0.00) 73.96% (0.00) ok
ai-mcp 96.66% (+0.17) 91.32% (-0.26) 97.09% (+0.28) 98.52% (+0.12) ok
ai-memory 73.87% (0.00) 52.70% (0.00) 79.74% (0.00) 76.04% (0.00) ok
ai-mistral 84.82% (0.00) 79.88% (0.00) 92.59% (0.00) 85.46% (0.00) ok
ai-octane 90.67% (0.00) 67.74% (0.00) 86.66% (0.00) 90.79% (0.00) ok
ai-ollama 92.73% (0.00) 78.15% (0.00) 90.62% (0.00) 92.68% (0.00) ok
ai-ollaya 86.84% (0.00) 89.28% (0.00) 85.71% (0.00) 91.66% (0.00) ok
ai-openai 65.34% (0.00) 65.72% (0.00) 66.66% (0.00) 65.63% (0.00) ok
ai-opencode 69.04% (0.00) 60.82% (0.00) 65.26% (0.00) 70.87% (0.00) ok
ai-openrouter 87.88% (0.00) 73.00% (0.00) 90.78% (0.00) 89.79% (0.00) ok
ai-perplexity 91.34% (0.00) 90.43% (0.00) 100.00% (0.00) 93.87% (0.00) ok
ai-persistence 91.19% (0.00) 78.00% (0.00) 97.56% (0.00) 95.67% (0.00) ok
ai-preact 85.62% (0.00) 70.81% (0.00) 85.96% (0.00) 87.02% (0.00) ok
ai-react 72.95% (0.00) 51.89% (0.00) 73.77% (0.00) 73.07% (0.00) ok
ai-reactor 94.91% (0.00) 83.33% (0.00) 93.33% (0.00) 96.55% (0.00) ok
ai-remix 36.70% (0.00) 22.69% (0.00) 34.18% (0.00) 39.04% (0.00) ok
ai-sandbox 78.02% (0.00) 83.10% (0.00) 75.33% (0.00) 78.57% (0.00) ok
ai-sandbox-blaxel 87.64% (0.00) 77.18% (0.00) 82.26% (0.00) 89.71% (0.00) ok
ai-sandbox-boxd 95.86% (0.00) 87.12% (0.00) 95.31% (0.00) 98.17% (0.00) ok
ai-sandbox-cloudflare 67.91% (0.00) 57.47% (0.00) 71.79% (0.00) 69.62% (0.00) ok
ai-sandbox-daytona 82.70% (0.00) 81.89% (0.00) 67.14% (0.00) 85.18% (0.00) ok
ai-sandbox-docker 80.15% (0.00) 71.49% (0.00) 77.38% (0.00) 82.13% (0.00) ok
ai-sandbox-e2b 94.42% (0.00) 88.97% (0.00) 93.24% (0.00) 98.63% (0.00) ok
ai-sandbox-local-process 68.00% (0.00) 53.59% (0.00) 75.86% (0.00) 68.88% (0.00) ok
ai-sandbox-railway 94.75% (0.00) 92.68% (0.00) 90.90% (0.00) 97.59% (0.00) ok
ai-sandbox-sprites 73.23% (0.00) 59.70% (0.00) 73.83% (0.00) 76.17% (0.00) ok
ai-sandbox-upstash-box 94.82% (0.00) 91.52% (0.00) 91.04% (0.00) 97.76% (0.00) ok
ai-sandbox-vercel 44.67% (0.00) 38.18% (0.00) 35.18% (0.00) 45.76% (0.00) ok
ai-skills 83.59% (0.00) 68.58% (0.00) 87.36% (0.00) 85.63% (0.00) ok
ai-solid 52.13% (0.00) 38.29% (0.00) 49.82% (0.00) 56.87% (0.00) ok
ai-svelte 73.76% (0.00) 54.59% (0.00) 69.15% (0.00) 78.33% (0.00) ok
ai-typesafe 78.04% (0.00) 84.61% (0.00) 66.66% (0.00) 82.05% (0.00) ok
ai-vercel-gateway 75.97% (0.00) 55.63% (0.00) 85.41% (0.00) 79.38% (0.00) ok
ai-vertex 96.77% (0.00) 97.22% (0.00) 100.00% (0.00) 96.77% (0.00) ok
ai-vue 60.07% (0.00) 34.10% (0.00) 59.34% (0.00) 61.04% (0.00) ok
ai-worldlabs 79.62% (0.00) 71.50% (0.00) 91.42% (0.00) 82.66% (0.00) ok
openai-base 90.31% (0.00) 78.28% (0.00) 97.66% (0.00) 92.36% (0.00) ok

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

waiting-on: maintainer The ball is in the maintainers’ court

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants