Skip to content

fix(build): explain failures and persist accepted work safely (#41, #42, narrow #43) - #45

Merged
ryandmonk merged 1 commit into
mainfrom
feat/build-hardening
Aug 5, 2026
Merged

fix(build): explain failures and persist accepted work safely (#41, #42, narrow #43)#45
ryandmonk merged 1 commit into
mainfrom
feat/build-hardening

Conversation

@ryandmonk

Copy link
Copy Markdown
Contributor

Release hardening for the Phase 3 Build loop — the blockers that stand between chat and being the public-facing first experience. One PR: the failure model, the accept path, and the refine gate are the same user story (a turn either explains itself or persists itself) and share the same fold, route, and specs; splitting them would split a test file down the middle.

Blocker 1 (#41) — the structured failure model

A failed turn now says why, from the fields the pipeline already reports, never inferred from message wording:

Source (structured) Rendered as
attempt.gates[].errors (S1/S2 — those gates carry no findings) gate · code · message
attempt.findings[] (S3) gate · rule id · location · message · the owner-authored rationale, verbatim
emitted.refusal the emitter's typed refusal, verbatim
emitted.validations[].gates[].errors A-gate · name · catalog version · message
attempt.adapterError provider error with an actionable headline
exhaustion named as such, with the last attempt's reasons

composer-core.buildFailure() returns { kind, headline, stoppedAt, reasons[] }; the view renders every reason and keeps the full audit report in a disclosure. Successful gates are never hidden — the strip shows where the pipeline stopped. agent-client no longer flattens a 4xx to agent replied N: a refusal carrying findings keeps them (typed on AgentResult), and a refused Accept renders those findings in place. Vocabulary gaps keep their own panel (the Component Workshop hook), still derived from S2 evidence only, so a gap stays distinguishable from malformed output. Accept and Refine appear only for a finished, passing turn with a surface.

Bonus correctness: the fold now reconciles its streamed attempts against the audit report, so gap detection and failure presentation read one authoritative source even if a progress event is dropped mid-stream (previously vocabularyGap read the stream while buildFailure read the report).

Blocker 2 (#42) — safe worked-example persistence

  • Identity from the contract on disk (nextExampleId), never a page-local counter — a reload or a second tab cannot collide with saved work.
  • Explicit collision ⇒ 409 refusal with a pathed finding, never an overwrite. The write is append-only.
  • Truthful provenance: examplePromptFor walks the turn's parent chain and stores the original ask plus a deterministic record of the refinements — "a status screen — refined: make the title clearer" — so the saved pair describes the accepted surface, not the last edit instruction. Still one save format; still few-shot-consumable.

Narrow #43

Refine is not offered for failed/refused/adapter-error turns; scripted no longer borrows another intent's example (it refuses honestly, and the view states the absence while noting a model can still run without few-shot context); and the scripted refinement marker is monotonic, so consecutive refinements can never be byte-identical no-ops reported as success.

Fail-first evidence

  • composer-core 10/10 failed before the failure model existed (buildFailure, canAcceptTurn, examplePromptFor undefined).
  • agent 4/4 failed before id minting, collision refusal, and intent honesty.
  • 8 new DOM specs failed before the UI carried reasons.

Two spec premises had to be corrected by the product's own behavior, which is itself evidence the gates work: sabotaging the corpus to force an S3 failure originally broke emit too, and readiness correctly closed Build — so the spec now uses a governance-only violation (rule.status-report.info-card-required) that leaves emit clean.

Results

composer-core 50/50 · agent 44/44 · agent-mode DOM 42/42 (incl. a11y) · exhibit 106 passed / 4 pre-existing skips · composer production 13/13 · typecheck clean · both builds green.

Required coverage, all present: S3 rule + rationale rendered · emit refusal evidence · adapter failure explained · 422 Accept renders findings · failed turns expose neither Accept nor Refine · gap vs malformed distinguishable · two accepts across a reload keep both with unique ids · refined accept stores truthful provenance · no cross-intent borrowing · two sequential refinements non-vacuous.

Invariants held: server-side fail-closed acceptance, immutable system prompt, complete-surface regeneration, trusted-registry rendering, ledger preservation, project-source privacy, zero hosted bindings, and all existing Catalog/Scenario/governance/ledger/exhibit/production behavior.

Deferred (recorded in #43/#44, deliberately not in this PR): cancellation and timeout UI, auto-navigation refinement, readiness based on corpus quality, cross-tab coordination beyond server-side collision refusal, broader findings redesign, hosted AI, bindings, Component Workshop, README/product-marketing expansion, unrelated chat polish.

🤖 Generated with Claude Code

Release hardening for the Phase 3 Build loop: the three blockers that
stand between chat and being the public-facing first experience. One
cohesive change — the failure model, the accept path, and the refine
gate are the same user story (a turn either explains itself or persists
itself), and they share the same fold, route, and tests.

BLOCKER 1 (#41) — structured failure presentation. A failed turn now
says WHY, from the fields the pipeline already reports and never from
message wording:
  S1/S2  -> attempt.gates[].errors (those gates carry no findings)
  S3     -> attempt.findings[]: rule id, message, location, and the
            OWNER-AUTHORED rationale, verbatim
  emit   -> emitted.refusal, else emitted.validations[].gates[].errors
  adapter-> attempt.adapterError, with an actionable headline
  exhaustion -> named as such, with the last attempt's reasons
composer-core gains buildFailure() (kind + headline + stoppedAt +
reasons), and the view renders every reason with its gate, code, target,
message and rationale, keeps the full audit report in a disclosure, and
never reduces a failure to a bare outcome. Successful gates stay visible
— the strip shows where the pipeline stopped. agent-client no longer
flattens a 4xx to "agent replied N": a refusal carrying `findings`
keeps them, and a refused Accept renders those findings in place.
Vocabulary gaps keep their own explicit panel (the Component Workshop
hook), distinguishable from malformed output because it is derived from
S2 evidence only. Accept and Refine are offered only for a finished,
passing turn with a surface (canAcceptTurn / canRefineTurn).

Also: the fold now reconciles its streamed attempts against the audit
report, so gap detection and failure presentation read ONE authoritative
source even if a progress event is dropped mid-stream.

BLOCKER 2 (#42) — safe worked-example persistence. Identity is minted
from the contract ON DISK (`nextExampleId`), never from a page-local
counter, so a reload or a second tab cannot collide with saved work; an
EXPLICIT id that already exists is refused 409 with a pathed finding
rather than overwritten; the write is append-only. Provenance is
truthful: `examplePromptFor` walks the turn's parent chain and stores
the ORIGINAL ask plus a deterministic record of the refinements
("a status screen — refined: make the title clearer"), so the saved pair
describes the accepted surface rather than the last edit instruction.

#43 (narrow) — Refine is no longer offered for failed, refused, or
adapter-error turns; scripted no longer borrows another intent's worked
example (it refuses honestly and the view states the absence, while a
model may still run without few-shot context); and the scripted
refinement marker is MONOTONIC, so consecutive refinements can never be
byte-identical no-ops reported as success.

Fail-first: composer-core 10/10 failed before the failure model existed;
agent 4/4 failed before minting/collision/intent-honesty; 8 new DOM
specs failed before the UI carried reasons. Now: composer-core 50/50,
agent 44/44, DOM 42/42, exhibit 106 passed/4 skips, composer production
13/13, typecheck clean, both builds green.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Copilot AI lite review requested due to automatic review settings August 5, 2026 13:32

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The new “mint next free id from disk” save path can still lose accepted examples under concurrent writes (e.g., two tabs accepting simultaneously) because contract writes are not serialized.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

This review doesn't count toward merge requirements. Sign up for the private preview to control whether Copilot approvals count.

Pull request overview

Hardens the Phase 3 “Build” loop by making failures explain themselves from structured pipeline evidence, and by making Accept/Refine behavior safe and truthful (no overwrites, provenance preserved, and refinement gated to valid prior turns).

Changes:

  • Adds a structured build failure model (buildFailure) and uses it in the Build view to render gate/findings/refusal/adapter reasons plus an audit disclosure.
  • Makes worked-example acceptance safer: agent-minted collision-free IDs, explicit-collision refusal (409), append-only persistence, and provenance-aware prompts (examplePromptFor).
  • Tightens refine/accept eligibility to only completed, passing turns with a surface; scripted mode no longer borrows examples across intents and refinements are monotonic.
File summaries
File Description
packages/composer-core/src/index.ts Re-exports new Build helpers/types for app consumption.
packages/composer-core/src/composer-core.test.ts Adds unit coverage for structured failures, accept/refine gating, and prompt provenance.
packages/composer-core/src/build.ts Reconciles fold output with audit report; implements buildFailure, canAcceptTurn, canRefineTurn, examplePromptFor.
apps/composer/app/views/build-view.tsx Renders structured failure panels + full report disclosure; gates Accept/Refine and adjusts Accept ID UX.
apps/composer/app/state.tsx Threads parent linkage for provenance; renders refused-accept findings; tightens refinement seeding; updates accept flow and notices.
apps/composer/app/agent-client.ts Preserves structured findings/status on non-2xx responses instead of flattening to “agent replied N”.
apps/agent/src/project.ts Scripted refinement becomes monotonic; scripted no-example becomes honest; accept endpoint mints IDs, refuses collisions, and appends only.
apps/agent/src/project.test.ts Tests collision-free minting, 409 refusal, corpus preservation, and honest scripted absence.
e2e/composer-build.spec.ts End-to-end specs for failure explanations, refusal evidence, safe persistence, provenance, and refinement behavior.
e2e/composer-build-a11y.spec.ts Updates accessible-name expectation for Accept now that the ID is minted server-side.
Review details
  • Files reviewed: 10/10 changed files
  • Comments generated: 3
  • Review effort level: Lite

We're testing this review assessment. Please use 👍 or 👎 to tell us if it's correct.

Comment thread apps/agent/src/project.ts
},
};
}
const id = requested || nextExampleId(existing);
Comment on lines +124 to +125
// Prefer the report's gates; fall back to what streamed, so a
// reconciliation never LOSES detail either direction.
Comment on lines +130 to +134
<details style={{ marginTop: 6 }}>
<summary style={{ cursor: "pointer", fontSize: 11, color: "var(--fg-dim)" }}>full audit report</summary>
<pre style={{ whiteSpace: "pre-wrap", fontSize: 10, color: "var(--fg-dim)", maxHeight: 220, overflow: "auto" }}>
{JSON.stringify(turn.progress.report ?? {}, null, 1)}
</pre>
@ryandmonk
ryandmonk merged commit c114037 into main Aug 5, 2026
2 checks passed
@ryandmonk
ryandmonk deleted the feat/build-hardening branch August 5, 2026 14:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants