Skip to content

feat(ai, ai-harness, ai-mcp): one redaction sink, the final tool result in the log, secret answers by reference, and single-flight OAuth refresh - #1815

Open
AlemTuzlak wants to merge 10 commits into
feat/harness-log-sessionfrom
feat/harness-redaction
Open

AlemTuzlak wants to merge 10 commits into
feat/harness-log-sessionfrom
feat/harness-redaction

Conversation

@AlemTuzlak

Copy link
Copy Markdown
Contributor

A secret that the harness hands out never lands in the log, the event stream, or the model input. The log keeps the tool result that the model really saw. A secret answer sent to another server goes through the credential store, not the log. OAuth refresh runs once per host and saves with compare-and-set. This is PR 6 of the harness redesign stack for #1755. It targets #1814 (feat/harness-log-session).

🎯 Changes

  • One redactor per session.
    • Every credential and key that the session hands out is a secret: provider keys, connector credentials, OAuth tokens, and ctx.context.run.credentials and keys.
    • The secrets become [redacted] in log records, feed events, plugin state, notes, the model context, tool results, the error frames of HTTP, the socket, and the stdio worker, and durable step results.
    • The raw, JSON-escaped, and URL-encoded forms are caught. Values shorter than 6 characters are left.
  • One scrubSecrets. The core one in @tanstack/ai/byok is the only body. The harness re-exports it.
  • The final tool result (D3).
    • A recorder at the end of the middleware list stores the result after every onAfterToolCall hook. A replacement wins, also on an error.
    • Recovery after a crash gives the model that result. The checkpoint keeps its pending and lease work where it was.
  • Secret answers across hosts.
    • An answer to a secret: true question that reaches a host that does not own the log is saved in the app's credentials store as a one-time reference. Only the reference goes into the input lane.
    • The owner reads it, deletes it, and answers. credentials.list hides the harness:answer: ids.
    • With no credentials store, such an answer is refused with secret_answer_needs_credentials.
  • OAuth refresh (C9).
    • It runs once per host per [tenantId, userId, connectorId]. A caller's abort rejects only that caller.
    • It saves with setIf when the store has it, and on a conflict it uses the winner's token. A tenant token stays in the tenant scope. previous.client is kept.
    • The new CredentialsAccess.refresh(id, update, { signal }) does this for connectors. MCP connectors save the SDK's refreshed tokens through it, and invalidateCredentials deletes only the token it saw.
  • Core CredentialStore. It has an optional getVersioned? and setIf? pair, the same shape as MetadataStore. memoryCredentialStore() implements both. The new readCredentialEntry(store, scope, id) says which scope matched, and the revision.

Docs: new sections in auth, durable-sessions, and advanced/server-keys. Updates in durable-tools, session-log, plugins, provider-keys, mcp, shared-threads, and api/ai. Changesets for @tanstack/ai, @tanstack/ai-harness, and @tanstack/ai-mcp.

Decisions made while building:

  1. The core scrubber is the only one. Core now also skips values shorter than 6 characters, like the harness one did.
  2. The recorder reads info.ok ? info.result : (info.result ?? { error }). A successful tool that returns undefined is not stored as an error.
  3. A host that does not have the question looks it up in the log. If it cannot find it, it treats the answer as a secret, so the answer is never sent in clear text.
  4. The shared refresh gets a host signal, not a caller's signal. Only host.close() aborts it.
  5. For MCP, only the save is single-flight. The MCP SDK sends its own refresh request per session, so two sessions can still both send one. A code comment and mcp.md say so.

Known limits: a secret split across two streaming deltas is not caught in those live events (the stored transcript is). A secret that is JSON-escaped twice is not caught. A secret answer reference that nobody reads stays in the store until PR 7 adds wait expiry.

✅ Checklist

  • I have followed the steps in the Contributing guide.
  • I have tested code changes locally with pnpm run test:pr, or these tests do not apply to this pull request.
  • I fully understand the code in this pull request, including any code generated with AI assistance.
  • Docs: I updated docs/ for this change, or this change is not user-facing.
  • Changeset: I added a changeset (pnpm changeset), or this PR does not change a published package.

🚀 Release Impact

  • This change affects published code, and I have generated a changeset.
  • This change is docs/CI/dev-only (no release).

Testing

Commands run:

  1. Builds of ai, ai-persistence, ai-harness, and ai-mcp. test:types of those four plus ai-acp, ai-harness-cli, ai-code-mode, ai-skills, and ai-sandbox. All pass.
  2. Each node ran its new tests and the tests of the source files it changed, with --maxWorkers=2. Examples: byok, redaction, resume, durable-tools, bound-output, input-lane, oauth-edges, auth, and the ai-mcp connector-edges and connector. All pass.
  3. test:oxlint of ai-harness, test:sherif, pnpm test:docs, kiira check on the changed pages, and oxfmt --check on every changed file.

Not run locally: the full suites, knip, E2E (including the new harness-secret-answer.spec.ts), and coverage. CI runs them.

Manual test:

  1. Run pnpm --dir packages/ai-harness exec vitest run tests/redaction.test.ts tests/input-lane.test.ts tests/oauth-edges.test.ts.
  2. Read the first test in tests/redaction.test.ts: a tool echoes a saved key in its result and its error. The key is [redacted] in the log, the feed, plugin state, and the next model call.
  3. Read refresh races in tests/oauth-edges.test.ts: two hosts refresh at once, one save wins, and both use the winner's token.

How this PR makes testing easy: the new tests/redaction.test.ts and the refresh races and a secret answer blocks pin each part. testing/e2e/tests/harness-secret-answer.spec.ts answers a secret question on a second server and checks that the secret is nowhere.

Risk / rollback

Medium. Every log record and event now passes the redactor, and the OAuth refresh path changed. The redactor skips all work when the session has no secret, and it returns the same object when nothing matches. The harness is not released yet. To undo, revert this PR.

Public API change

Before

interface CredentialStore { get; set; delete; list }
const token = await credentials.get(id) // refresh and save by hand, plain set

After

interface CredentialStore { get; set; delete; list; getVersioned?; setIf? }
const entry = await readCredentialEntry(store, scope, id) // { credential, scope, revision? }
const fresh = await credentials.refresh(id, (current, signal) => refreshToken(current, signal), { signal })

@changeset-bot

changeset-bot Bot commented Oct 17, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: a7cc5c1

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 53 packages
Name Type
@tanstack/ai Minor
@tanstack/ai-harness Minor
@tanstack/ai-mcp Minor
@tanstack/ai-acp Patch
@tanstack/ai-angular Patch
@tanstack/ai-anthropic Patch
@tanstack/ai-bedrock Patch
@tanstack/ai-byteplus Patch
@tanstack/ai-claude-code Patch
@tanstack/ai-client Patch
@tanstack/ai-cloudflare Patch
@tanstack/ai-code-mode-snippets Patch
@tanstack/ai-code-mode Patch
@tanstack/ai-codex Patch
@tanstack/ai-cohere Patch
@tanstack/ai-compaction Patch
@tanstack/ai-devtools-core Patch
@tanstack/ai-durable-stream Patch
@tanstack/ai-elevenlabs Patch
@tanstack/ai-fal Patch
@tanstack/ai-gemini Patch
@tanstack/ai-grok-build Patch
@tanstack/ai-grok Patch
@tanstack/ai-groq Patch
@tanstack/ai-harness-cli Patch
@tanstack/ai-llmgateway Patch
@tanstack/ai-lovable Patch
@tanstack/ai-memory Patch
@tanstack/ai-mistral Patch
@tanstack/ai-octane Patch
@tanstack/ai-ollama Patch
@tanstack/ai-ollaya Patch
@tanstack/ai-openai Patch
@tanstack/ai-opencode Patch
@tanstack/ai-openrouter Patch
@tanstack/ai-perplexity Patch
@tanstack/ai-persistence Patch
@tanstack/ai-preact Patch
@tanstack/ai-react Patch
@tanstack/ai-reactor Patch
@tanstack/ai-remix Patch
@tanstack/ai-sandbox-cloudflare Patch
@tanstack/ai-sandbox Patch
@tanstack/ai-skills Patch
@tanstack/ai-solid Patch
@tanstack/ai-svelte Patch
@tanstack/ai-typesafe Patch
@tanstack/ai-vercel-gateway Patch
@tanstack/ai-vertex Patch
@tanstack/ai-vue Patch
@tanstack/ai-worldlabs Patch
@tanstack/openai-base Patch
ag-ui Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@coderabbitai

coderabbitai Bot commented Oct 17, 2026 •

Copy link
Copy Markdown
Contributor

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration
  • Configuration used: Repository: TanStack/ai/.coderabbit.yaml
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 4a025448-5e71-4838-b231-7720ca17b0e6

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@nx-cloud

nx-cloud Bot commented Oct 17, 2026 •

Copy link
Copy Markdown

View your CI Pipeline Execution ↗ for commit a7cc5c1

Command Status Duration Result
nx run-many --targets=build --exclude=examples/... ✅ Succeeded 5s View ↗

☁️ Nx Cloud last updated this comment at 2026-10-17 18:08:25 UTC

@pkg-pr-new

pkg-pr-new Bot commented Oct 17, 2026

Copy link
Copy Markdown

Open in StackBlitz

@tanstack/ai

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai@1815

@tanstack/ai-acp

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-acp@1815

@tanstack/ai-angular

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-angular@1815

@tanstack/ai-anthropic

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-anthropic@1815

@tanstack/ai-bedrock

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-bedrock@1815

@tanstack/ai-byteplus

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-byteplus@1815

@tanstack/ai-claude-code

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-claude-code@1815

@tanstack/ai-client

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-client@1815

@tanstack/ai-cloudflare

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-cloudflare@1815

@tanstack/ai-code-mode

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-code-mode@1815

@tanstack/ai-code-mode-snippets

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-code-mode-snippets@1815

@tanstack/ai-codex

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-codex@1815

@tanstack/ai-cohere

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-cohere@1815

@tanstack/ai-compaction

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-compaction@1815

@tanstack/ai-devtools-core

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-devtools-core@1815

@tanstack/ai-durable-stream

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-durable-stream@1815

@tanstack/ai-elevenlabs

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-elevenlabs@1815

@tanstack/ai-event-client

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-event-client@1815

@tanstack/ai-fal

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-fal@1815

@tanstack/ai-gemini

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-gemini@1815

@tanstack/ai-grok

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-grok@1815

@tanstack/ai-grok-build

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-grok-build@1815

@tanstack/ai-groq

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-groq@1815

@tanstack/ai-harness

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-harness@1815

@tanstack/ai-harness-cli

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-harness-cli@1815

@tanstack/ai-isolate-cloudflare

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-isolate-cloudflare@1815

@tanstack/ai-isolate-daytona

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-isolate-daytona@1815

@tanstack/ai-isolate-e2b

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-isolate-e2b@1815

@tanstack/ai-isolate-node

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-isolate-node@1815

@tanstack/ai-isolate-quickjs

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-isolate-quickjs@1815

@tanstack/ai-isolate-quickjs-bun

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-isolate-quickjs-bun@1815

@tanstack/ai-llmgateway

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-llmgateway@1815

@tanstack/ai-lovable

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-lovable@1815

@tanstack/ai-mcp

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-mcp@1815

@tanstack/ai-memory

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-memory@1815

@tanstack/ai-mistral

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-mistral@1815

@tanstack/ai-octane

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-octane@1815

@tanstack/ai-ollama

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-ollama@1815

@tanstack/ai-ollaya

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-ollaya@1815

@tanstack/ai-openai

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-openai@1815

@tanstack/ai-opencode

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-opencode@1815

@tanstack/ai-openrouter

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-openrouter@1815

@tanstack/ai-perplexity

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-perplexity@1815

@tanstack/ai-persistence

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-persistence@1815

@tanstack/ai-preact

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-preact@1815

@tanstack/ai-react

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-react@1815

@tanstack/ai-react-ui

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-react-ui@1815

@tanstack/ai-reactor

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-reactor@1815

@tanstack/ai-remix

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-remix@1815

@tanstack/ai-sandbox

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-sandbox@1815

@tanstack/ai-sandbox-blaxel

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-sandbox-blaxel@1815

@tanstack/ai-sandbox-boxd

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-sandbox-boxd@1815

@tanstack/ai-sandbox-cloudflare

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-sandbox-cloudflare@1815

@tanstack/ai-sandbox-daytona

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-sandbox-daytona@1815

@tanstack/ai-sandbox-docker

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-sandbox-docker@1815

@tanstack/ai-sandbox-e2b

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-sandbox-e2b@1815

@tanstack/ai-sandbox-local-process

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-sandbox-local-process@1815

@tanstack/ai-sandbox-railway

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-sandbox-railway@1815

@tanstack/ai-sandbox-sprites

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-sandbox-sprites@1815

@tanstack/ai-sandbox-upstash-box

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-sandbox-upstash-box@1815

@tanstack/ai-sandbox-vercel

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-sandbox-vercel@1815

@tanstack/ai-skills

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-skills@1815

@tanstack/ai-solid

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-solid@1815

@tanstack/ai-solid-ui

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-solid-ui@1815

@tanstack/ai-svelte

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-svelte@1815

@tanstack/ai-typesafe

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-typesafe@1815

@tanstack/ai-utils

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-utils@1815

@tanstack/ai-vercel-gateway

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-vercel-gateway@1815

@tanstack/ai-vertex

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-vertex@1815

@tanstack/ai-vue

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-vue@1815

@tanstack/ai-vue-ui

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-vue-ui@1815

@tanstack/ai-worldlabs

npm i https://pkg.pr.new/TanStack/ai/@tanstack/ai-worldlabs@1815

@tanstack/openai-base

npm i https://pkg.pr.new/TanStack/ai/@tanstack/openai-base@1815

@tanstack/preact-ai-devtools

npm i https://pkg.pr.new/TanStack/ai/@tanstack/preact-ai-devtools@1815

@tanstack/react-ai-devtools

npm i https://pkg.pr.new/TanStack/ai/@tanstack/react-ai-devtools@1815

@tanstack/solid-ai-devtools

npm i https://pkg.pr.new/TanStack/ai/@tanstack/solid-ai-devtools@1815

@tanstack/svelte-ai-devtools

npm i https://pkg.pr.new/TanStack/ai/@tanstack/svelte-ai-devtools@1815

commit: a7cc5c1

@github-actions

Copy link
Copy Markdown
Contributor

Coverage

✅ Coverage held across 68 compared package(s).

Each package is measured twice in this job — on this PR and on its merge-base with main — and compared. A metric fails only when it is at or above 60% on the merge-base and under 60% on this PR. Other drops are listed but do not fail. Packages your PR doesn't affect are not measured and not listed.

package statements branches functions lines
ai 91.64% (+0.03) 85.33% (+0.04) 92.40% (+0.02) 93.28% (+0.02) ok
ai-acp 82.31% (0.00) 71.64% (0.00) 77.00% (0.00) 84.13% (0.00) ok
ai-angular 74.10% (0.00) 62.20% (0.00) 61.65% (0.00) 75.60% (0.00) ok
ai-anthropic 92.00% (0.00) 86.73% (0.00) 95.57% (0.00) 92.41% (0.00) ok
ai-bedrock 90.11% (0.00) 82.82% (0.00) 94.20% (0.00) 91.33% (0.00) ok
ai-byteplus 97.06% (0.00) 91.66% (0.00) 98.27% (0.00) 98.49% (0.00) ok
ai-claude-code 88.93% (0.00) 76.88% (0.00) 86.88% (0.00) 90.40% (0.00) ok
ai-client 87.96% (0.00) 80.38% (0.00) 87.13% (0.00) 90.26% (0.00) ok
ai-cloudflare 85.55% (0.00) 76.44% (0.00) 85.48% (0.00) 86.79% (0.00) ok
ai-code-mode 92.71% (0.00) 85.66% (0.00) 89.06% (0.00) 93.51% (0.00) ok
ai-code-mode-snippets 87.13% (0.00) 75.67% (0.00) 91.11% (0.00) 88.39% (0.00) ok
ai-codex 94.55% (0.00) 85.10% (0.00) 89.28% (0.00) 95.56% (0.00) ok
ai-cohere 88.97% (0.00) 88.37% (0.00) 85.71% (0.00) 91.80% (0.00) ok
ai-compaction 97.38% (0.00) 92.79% (0.00) 100.00% (0.00) 98.88% (0.00) ok
ai-devtools 83.13% (0.00) 72.77% (0.00) 87.07% (0.00) 85.22% (0.00) ok
ai-durable-stream 86.70% (0.00) 82.25% (0.00) 92.30% (0.00) 88.72% (0.00) ok
ai-elevenlabs 74.68% (0.00) 69.59% (0.00) 59.13% (0.00) 75.46% (0.00) ok
ai-fal 91.10% (0.00) 82.63% (0.00) 92.40% (0.00) 91.60% (0.00) ok
ai-gemini 68.33% (0.00) 67.88% (0.00) 64.62% (0.00) 68.93% (0.00) ok
ai-grok 48.85% (0.00) 53.49% (0.00) 56.46% (0.00) 49.67% (0.00) ok
ai-grok-build 79.19% (0.00) 63.56% (0.00) 77.39% (0.00) 81.02% (0.00) ok
ai-groq 52.54% (0.00) 47.73% (0.00) 67.39% (0.00) 54.66% (0.00) ok
ai-harness 93.78% (-0.05) 87.95% (-0.06) 93.99% (+0.07) 95.41% (-0.03) ok
ai-harness-cli 91.12% (0.00) 84.77% (0.00) 84.96% (0.00) 92.51% (0.00) ok
ai-isolate-cloudflare 94.11% (0.00) 70.66% (0.00) 100.00% (0.00) 93.96% (0.00) ok
ai-isolate-daytona 90.25% (0.00) 80.59% (0.00) 100.00% (0.00) 90.15% (0.00) ok
ai-isolate-e2b 92.11% (0.00) 81.48% (0.00) 97.36% (0.00) 95.05% (0.00) ok
ai-isolate-node 84.00% (0.00) 43.39% (0.00) 100.00% (0.00) 83.78% (0.00) ok
ai-isolate-quickjs 89.07% (0.00) 76.41% (0.00) 90.90% (0.00) 90.30% (0.00) ok
ai-isolate-quickjs-bun 7.76% (0.00) 8.08% (0.00) 5.00% (0.00) 8.33% (0.00) ok
ai-llmgateway 97.67% (0.00) 100.00% (0.00) 88.88% (0.00) 97.67% (0.00) ok
ai-lovable 73.21% (0.00) 55.63% (0.00) 85.07% (0.00) 73.96% (0.00) ok
ai-mcp 96.49% (+0.03) 91.58% (+0.07) 96.81% (+0.01) 98.40% (+0.02) ok
ai-memory 73.87% (0.00) 52.70% (0.00) 79.74% (0.00) 76.04% (0.00) ok
ai-mistral 84.82% (0.00) 79.88% (0.00) 92.59% (0.00) 85.46% (0.00) ok
ai-octane 90.67% (0.00) 67.74% (0.00) 86.66% (0.00) 90.79% (0.00) ok
ai-ollama 92.73% (0.00) 78.15% (0.00) 90.62% (0.00) 92.68% (0.00) ok
ai-ollaya 86.84% (0.00) 89.28% (0.00) 85.71% (0.00) 91.66% (0.00) ok
ai-openai 65.34% (0.00) 65.72% (0.00) 66.66% (0.00) 65.63% (0.00) ok
ai-opencode 69.04% (0.00) 60.82% (0.00) 65.26% (0.00) 70.87% (0.00) ok
ai-openrouter 87.88% (0.00) 73.00% (0.00) 90.78% (0.00) 89.79% (0.00) ok
ai-perplexity 91.34% (0.00) 90.43% (0.00) 100.00% (0.00) 93.87% (0.00) ok
ai-persistence 91.19% (0.00) 78.00% (0.00) 97.56% (0.00) 95.67% (0.00) ok
ai-preact 85.62% (0.00) 70.81% (0.00) 85.96% (0.00) 87.02% (0.00) ok
ai-react 72.95% (0.00) 51.89% (0.00) 73.77% (0.00) 73.07% (0.00) ok
ai-reactor 94.91% (0.00) 83.33% (0.00) 93.33% (0.00) 96.55% (0.00) ok
ai-remix 36.70% (0.00) 22.69% (0.00) 34.18% (0.00) 39.04% (0.00) ok
ai-sandbox 78.02% (+0.06) 83.10% (+0.04) 75.33% (+0.11) 78.57% (+0.05) ok
ai-sandbox-blaxel 87.64% (0.00) 77.18% (0.00) 82.26% (0.00) 89.71% (0.00) ok
ai-sandbox-boxd 95.86% (0.00) 87.12% (0.00) 95.31% (0.00) 98.17% (0.00) ok
ai-sandbox-cloudflare 67.91% (0.00) 57.47% (0.00) 71.79% (0.00) 69.62% (0.00) ok
ai-sandbox-daytona 82.70% (0.00) 81.89% (0.00) 67.14% (0.00) 85.18% (0.00) ok
ai-sandbox-docker 80.15% (0.00) 71.49% (0.00) 77.38% (0.00) 82.13% (0.00) ok
ai-sandbox-e2b 94.42% (0.00) 88.97% (0.00) 93.24% (0.00) 98.63% (0.00) ok
ai-sandbox-local-process 68.00% (0.00) 53.59% (0.00) 75.86% (0.00) 68.88% (0.00) ok
ai-sandbox-railway 94.75% (0.00) 92.68% (0.00) 90.90% (0.00) 97.59% (0.00) ok
ai-sandbox-sprites 73.23% (0.00) 59.70% (0.00) 73.83% (0.00) 76.17% (0.00) ok
ai-sandbox-upstash-box 94.82% (0.00) 91.52% (0.00) 91.04% (0.00) 97.76% (0.00) ok
ai-sandbox-vercel 44.67% (0.00) 38.18% (0.00) 35.18% (0.00) 45.76% (0.00) ok
ai-skills 83.59% (0.00) 68.58% (0.00) 87.36% (0.00) 85.63% (0.00) ok
ai-solid 52.13% (0.00) 38.29% (0.00) 49.82% (0.00) 56.87% (0.00) ok
ai-svelte 73.76% (0.00) 54.59% (0.00) 69.15% (0.00) 78.33% (0.00) ok
ai-typesafe 78.04% (0.00) 84.61% (0.00) 66.66% (0.00) 82.05% (0.00) ok
ai-vercel-gateway 75.97% (0.00) 55.63% (0.00) 85.41% (0.00) 79.38% (0.00) ok
ai-vertex 96.77% (0.00) 97.22% (0.00) 100.00% (0.00) 96.77% (0.00) ok
ai-vue 60.07% (0.00) 34.10% (0.00) 59.34% (0.00) 61.04% (0.00) ok
ai-worldlabs 79.62% (0.00) 71.50% (0.00) 91.42% (0.00) 82.66% (0.00) ok
openai-base 90.31% (0.00) 78.28% (0.00) 97.66% (0.00) 92.36% (0.00) ok

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

waiting-on: maintainer The ball is in the maintainers’ court

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants