Skip to content

Record bounded Cell acceptance progress - #1009

Open
zlareb1 wants to merge 2 commits into
mainfrom
fix/cell-provenance-phase-evidence
Open

zlareb1 wants to merge 2 commits into
mainfrom
fix/cell-provenance-phase-evidence

Conversation

@zlareb1

@zlareb1 zlareb1 commented Nov 20, 2026 •

Copy link
Copy Markdown
Member

Changelog category (leave one):

  • CI Fix or Improvement (changelog entry is not required)

Changelog entry (one or two sentences a user of Loom can read without the diff; the release pull request collects these into the release notes):

Not required.

Why

An interrupted Cell acceptance run can end before the independent provenance proof starts. Existing request-cost spans leave part of the selected request's elapsed time outside a named phase. A terminal refusal alone cannot locate that work.

Record bounded numeric checkpoints during the proof and nine HTTP helper boundaries for the existing six selected requests. These observations identify phase entry, fully verified row counts and timed request boundaries without replacing acceptance evidence or identifying an unobserved timeout cause.

Scope and risk

Only the CI acceptance harness, its artifact exporter and focused tests change. HTTP helpers still execute once with their original arguments, results and failures. Their wrappers restore on normal exit and partial setup failure. Unselected HTTP-helper wrappers call the original directly without additional diagnostic clock, tag or writer calls. Request tags are hashed after the helper can bind them; raw tags and request data are not emitted by these diagnostics.

Queries, hashes, row comparisons, workload, selected request ordinals, deadlines, statistical thresholds and cleanup remain unchanged. Diagnostic work consumes the existing budget and adds overhead; overlapping phase durations and process CPU are not exclusive latency or CPU measurements. No extra query, request or retry is added.

The proof child stream has at most 3217 records and 823552 bytes; its parent stream has at most two records. Request-cost observations retain the existing 192-event and 65536-byte limits; the existing footer reports drops when available. The new provenance exporter accepts only closed numeric schemas from bounded owned regular files and rejects symlinks. These new diagnostic outputs contain no row data, identity, SQL, settings or exception text. Missing or invalid diagnostics cannot replace the work or cleanup result.

Compatibility, rollout and rollback

  • Wire: no operation, field or default changes.
  • Operator steps: none.
  • Rollback: reverting removes optional diagnostics; no stored-data change.

Tests

Focused regressions cover proof checkpoints and verified-row counts, budget snapshots, numeric artifact bounds, privacy, cancellation and first-error preservation, truncated input and diagnostic I/O failures. Page cases reuse the existing independent frozen row/hash oracle.

Request-phase cases cover synchronous and asynchronous delegation, original result and exception identity, bound-tag hashing, unselected requests, exact helper installation and restoration, partial setup and server startup failures, and all nine boundaries for only the existing selected requests.

On the final three-path request-phase change:

  • Ruff format and check: pass.
  • pytest tests/unit/test_cell_trigger_request_phases.py plus the three existing request-cost capture, failure and publication tests in test_cell_trigger_ci_creator.py: 25 parameterized cases pass.

The unchanged proof diagnostic has earlier component evidence: 31 proof-progress cases and 48 provenance, failure and cancellation controls passed. These are not a repeat of the full checks on the new head. Full CI remains required; the previous head's failed Cell acceptance is retained. This is diagnostic evidence, with no product or performance fix claim.

AI assistance in the code (leave one; writing this description with AI is fine):

  • AI-generated (an agent wrote most of the diff): Codex. Independent source review checks delegation, proof ordering, failure preservation and artifact privacy; focused regressions check observable behavior.

  • I can explain every line of this diff and how it interacts with the rest of Loom.

Checklist

  • make format-check, make lint, make typecheck and make test-unit pass; the integration suite passes if server behaviour changed. Full checks run in CI.
  • Generated files: none affected.
  • Migrations: none.
  • Plugin files: none affected.
  • Frontend: none affected.
  • Operator steps and wire changes: none.
  • Public text contains no deployment details.
  • Commits are signed off (git commit -s).

An interrupted proof needs its last observed page and validation phase. Numeric checkpoints retain that evidence while preserving proof checks, deadlines and the original failure.

Signed-off-by: zlareb1 <lareb.zafar@clickhouse.com>
Bound HTTP helper timings to the existing six request samples so acceptance failures retain evidence for work outside named spans. Preserve delegation, failure identity and the original workload and limits.

Signed-off-by: zlareb1 <lareb.zafar@clickhouse.com>
@zlareb1 zlareb1 changed the title Record bounded Cell provenance progress Record bounded Cell acceptance progress Nov 20, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant