Repository navigation
docs(plan): split Step 3b-2b into the REST sampler and verify; settle verify's edges (D36) - #27
Merged
Merged
Conversation
… verify's edges (D36) Step 3b-2b becomes 3b-2b-1 (the REST sampler and its recorder mode) and 3b-2b-2 (verify, its timeline lookup, and the live acceptance). 5.4 states verify's lock check, repository set under --repo, report on each exit, unverified repositories, link-total differences, and cost fields (D36, gap 94). 5.8 passes the timeline lookup's since one second before the boundary, as measured (gap 95). 3b-2b-1 gains a paging rule and recorder-mode rules (gap 96). Section 7 records the timeline measurements and Step 3b-2a-2c's acceptance file; section 8 drops the items that file resolved.
5.4: verify's untracked set follows 5.3 step 1's every-entry-resolved rule; the lock rule names a future deadline; stored rows are read per repository before its index; the unverified list names every resolve and index failure and gap 92; a verify with no repository verified is fatal; a totals-only link difference needs as many connected or disconnected events as the change; an interrupt keeps 130 when the report cannot be written; elapsed seconds are defined. 5.6: snapshot refuses a missing or stale verify report. 5.8: the lookup keeps entries at or after the boundary, and a count mismatch leaves its item unchecked. 3b-2b-1: the sampler's REST units live in their own directory, the sampler takes --repo, and the offline tests pair REST and GraphQL units from one recorder run. 3b-2b-2: matching tests, a timeline re-recording rule, a 2,500-point floor, and the exit the acceptance requires. Step 4a: issue child-count tests. Section 7: per-run failure counts, code versions, and the compare-linked accounts.
5.4: verify reads each repository's state and pending rows in one statement, then its items with their links and failures in one statement; the untracked set requires every entry to resolve without failing 5.3 step 1; the unverified list adds the full_name check; a repository error in either lookup leaves the items no lookup answered unchecked; the per-issue link rule applies only to an earlier updatedAt, and the report names each link difference's issues and totals. 5.8: the timeline lookup reads through repository(owner:, name:). 3b-2b-1: the sampler's recorder mode writes every unit of its run into its own directory. D36, gap 96, 5.6, section 8, and the 3b-2b-2 and 6a test lists follow.
An unchecked item and an unverified repository now fail verify in both modes, not only under --exact, so the scheduled verify cannot exit 0 on a database it did not check and D25 does not publish it. A timeline lookup whose entries still differ from filteredCount after its retry, or whose response the typed models reject, is an error. D37 records the rule: an error over a silent failure or silently wrong data. Gap 97, D25's wording, 5.8, and the 3b-2b-2 tests follow.
Contributor
Author
|
Head is now The description's Decisions section says a timeline lookup that still fails after one solo retry leaves its item unchecked, and that unchecked fails only
Ran on this head:
Not run: |
…nted inputs Verify's child-count checks and Step 4a's issue child-count checks are tested on their passing side, each reporting the rows it compared, equal to the counts the scenario model or the stored totals give. The timeline lookup's entries before the boundary are events the scenario model renders.
ChrisHuie
added a commit
that referenced
this pull request
Oct 22, 2026
- Step 3b-2a-2d names every test file that plants by file (the engine's patched writer and prune, user_version and alembic_version edits, dropped indexes, the checkers' planted files, the wheel copy, edited query documents), states which inputs stay as D39 inputs, tests env.py's file-held violation refusal on its passing side, and accepts on a list of every test that writes or patches, with a word search over tests/**/*.py. - D39 and its four copies except configuration as well as stubs. - 6a tests verify --db-only's unknown-revision refusal on a file from a flawless test-only revision past head. - 3b-2a-3: the foreign_keys read-back through a fake DBAPI connection, and entries for an entry that resolves to another github_id. - D25 and 5.4: an unverified repository makes the exit 2; a rate limit only waits; association drift is a deliberate exception (D38); the timeline test's half-second entry states the model's comparison order. - 3b-2a-2b-2's history, 5.7, and the Step 3 overview follow.
ChrisHuie
added a commit
that referenced
this pull request
Oct 22, 2026
… inputs (D39) (#28) * docs(plan): split Step 3b-2b into the REST sampler and verify; settle verify's edges (D36) Step 3b-2b becomes 3b-2b-1 (the REST sampler and its recorder mode) and 3b-2b-2 (verify, its timeline lookup, and the live acceptance). 5.4 states verify's lock check, repository set under --repo, report on each exit, unverified repositories, link-total differences, and cost fields (D36, gap 94). 5.8 passes the timeline lookup's since one second before the boundary, as measured (gap 95). 3b-2b-1 gains a paging rule and recorder-mode rules (gap 96). Section 7 records the timeline measurements and Step 3b-2a-2c's acceptance file; section 8 drops the items that file resolved. * docs(plan): tighten Step 3b-2b's rules after review 5.4: verify's untracked set follows 5.3 step 1's every-entry-resolved rule; the lock rule names a future deadline; stored rows are read per repository before its index; the unverified list names every resolve and index failure and gap 92; a verify with no repository verified is fatal; a totals-only link difference needs as many connected or disconnected events as the change; an interrupt keeps 130 when the report cannot be written; elapsed seconds are defined. 5.6: snapshot refuses a missing or stale verify report. 5.8: the lookup keeps entries at or after the boundary, and a count mismatch leaves its item unchecked. 3b-2b-1: the sampler's REST units live in their own directory, the sampler takes --repo, and the offline tests pair REST and GraphQL units from one recorder run. 3b-2b-2: matching tests, a timeline re-recording rule, a 2,500-point floor, and the exit the acceptance requires. Step 4a: issue child-count tests. Section 7: per-run failure counts, code versions, and the compare-linked accounts. * docs(plan): close the gaps the convergence review found in Step 3b-2b 5.4: verify reads each repository's state and pending rows in one statement, then its items with their links and failures in one statement; the untracked set requires every entry to resolve without failing 5.3 step 1; the unverified list adds the full_name check; a repository error in either lookup leaves the items no lookup answered unchecked; the per-issue link rule applies only to an earlier updatedAt, and the report names each link difference's issues and totals. 5.8: the timeline lookup reads through repository(owner:, name:). 3b-2b-1: the sampler's recorder mode writes every unit of its run into its own directory. D36, gap 96, 5.6, section 8, and the 3b-2b-2 and 6a test lists follow. * docs(plan): fail verify on anything it could not check (D37) An unchecked item and an unverified repository now fail verify in both modes, not only under --exact, so the scheduled verify cannot exit 0 on a database it did not check and D25 does not publish it. A timeline lookup whose entries still differ from filteredCount after its retry, or whose response the typed models reject, is an error. D37 records the rule: an error over a silent failure or silently wrong data. Gap 97, D25's wording, 5.8, and the 3b-2b-2 tests follow. * docs(plan): apply D37 to the sync and to Steps 4b-6 (D38) Records the decisions on the silent-failure audit as D38 and gaps 98-110: Step 3b-2a-3 makes a failed account refresh, a time-limited run with unreached items, and a hydration that contradicts the run's own index each exit 2, scopes accepted failures to an error class, re-hydrates PRs with more links than the index carries, stores every node id GitHub reports, and closes the source and run-record holes the audit found. Steps 4b, 5, and 6 and 5.6 gain the completeness, permission, staleness, and workflow rules. * docs(plan): close the gaps the review of D38 found A hydration that contradicts the run's index gets one engine retry, then fails. Any deadline makes the exit 2, and the failures a batch made final before it are recorded. The migration refuses a populated file. A close that cannot write an outcome raises before writing. code_version names which git answer it trusts. Commit-link node ids are stored; accepted failures match the class exactly and can name an account. verify --db-only writes and digests the working file, snapshot checks the job's own sync through --after-run, and 6b compares App and token reads before any App run publishes. The stopped-schedule case is stated as visible only to a consumer. * docs(plan): settle the remaining gaps in D38's rules Accepted failures match class and message exactly and can name an account by github_id or node_id; a refresh failure an account entry accepts leaves the exit code alone, and verify --exact matches entries as 5.5 does. code_version decides a checkout by a .git entry before reading git's exit codes. A stamp is refused by on_version_apply inside the transaction. Gap 110 refuses only for rows no later run reads. snapshot takes --after-run/--sync-exit or --sync-run, checks the stored row of every configured entry, and names its manifest keys. 6b's App-and-token comparison names its runs and script. Tests follow each rule. * docs: no planted, seeded, or tampered inputs, and no mutation testing (D39) D39 records the maintainer's rule: no test, review, or acceptance changes a copy of the code, a test, the plan, review material, a database, a snapshot, or a manifest to see whether something catches the change, and none measures how many such changes are found. Test inputs are recorded GitHub responses, stubs of what GitHub or the environment can send, calls and configuration the code must refuse, and files this project's own code wrote; a check only a defect could fail is tested on its passing side with the exact count it compared. - Step 3's evidence rule, the sampler, verify, Step 4a, Step 4b, and Step 6 no longer prescribe planted mismatches, planted row deletions, tampered copies, or a corrupted db-test release. - Section 7 drops the mutation result from the SQLite pragmas row. - Step 7's audit replaces merged tests that change a copy of a database or a file. - docs/testing.md, CONTRIBUTING.md, and CLAUDE.md state the same rule. * docs(plan): close the findings of the last review of D38 - 5.3 step 5: GraphQL's commit connection holds the newest 250, so above 250 commits its first page must equal compare's positions total-250..total-151 in order (Prebid.js #8858: positions 4-103); the oldest commits are checked by count alone. - 5.5: an accepted_failures entry matches the class and the cause, the first line of the failure's message, which carries no request id, count, or typed-model text; tracking.py keeps the recorded class list, held equal to the engine's classes by a test; index contradictions take the class IndexContradiction; Step 4b adds a compare entry form. - 5.6 and Step 6: a validated `repo` dispatch input (bootstrap only) for 6b's App-and-token comparison, which covers every table the sync writes from GitHub, names the cases its sample must hold, and lists App nulls; the manifest carries repo_filter, links_not_found, and Step 4b's counts; 6a tests compare_reads and lists each exit-code output for the sync.yml test. - Step 4b: completeness against an ascending read's first event, files from the first page, per-compare transactions, a stored error for a compare that still fails, deletion counters, verify's own read as unverified, halted vs not-synced classes, and merge-brought commits. - Step 5: the updatedAt measurement's instrument per kind and its counting rule, contradiction rates, the link NOT_FOUND share, and the bootstrap's projected time; the conditional pull request is 5b. - 5.1: a read version after Step 6b instead of a refusal, account_node_ids' keys and indexes, trigger `unknown`, untracked files outside -dirty, and `stamp --purge` at head. - Step 3b-2a-3: scope and tests follow the above, plus re-recording and docs/development.md. * docs(plan): close the findings of the review of 12b2452 - D39 says "no flaw is seeded" (valid seed.py rows stay allowed), lists one set of allowed inputs everywhere it is restated, and widens Step 7's replacement to tests that write a planted file, revision, call, connection, or decoy; 5.7's planted import files are marked for that replacement. - The scenario model renders a PR above 250 commits as GitHub does (newest 250), and the commit-page rule is measured live on the acceptance repositories' large PRs before it is fixed. - A 2xx REST response that is not JSON or lacks a Date is a server error whatever its headers; only a readable one without rate-limit headers is not understood. - sync_runs gains link_lookups (the NOT_FOUND limit's denominator, without accounts an earlier lookup answered NOT_FOUND) and entries (each configured entry's github_id), which snapshot --after-run checks. - D25 and publish's condition exclude a run with `repo` from release db; the sync.yml test checks publish's and report's exact values and snapshot's --out and --run-id. - verify --db-only's digest and manifest refusals are tested on snapshots the code wrote; compare_reads maps row ids to natural keys; db-test runs go in a stated order. - Step 4b: a halt exits 2, an empty log is complete when the ascending read is empty, a commit-count mismatch or later-page 404 is a compare error, the refresh runs first, verify reads stored rows before its reads, and a compare entry's not-failed rule. - 5b covers a recurring index contradiction's retry delay; the report lists each contradiction and whether the retry cleared it. * docs(plan): one D39 text everywhere it is restated; settle the last gaps - D39's rule is one text, copied word for word into Step 3, CLAUDE.md, docs/testing.md, and CONTRIBUTING.md; it allows stubs built in memory from a recorded unit, valid rows written in SQL, and configuration naming a state GitHub can produce, for tests and acceptances. - Step 3b-2a-3: the readable-2xx REST rule in its summaries, and how each refusing shape is built (replay.modify or the scenario model's set). - sync_runs.entries records the matched stored row's github_id, else the resolved one; snapshot skips an id with no stored row; a snapshot's T is the time snapshot writes it. - 6a states why verify --db-only's unknown-revision refusal is tested on its passing side; the acceptance names the refusals the 6a tests cover. - D38 and 6a state the commit-link limit as 5.2 does; Step 7 names only the view-alter revision that leaves a view in place. * docs(plan): state verify's child-count and timeline tests without planted inputs Verify's child-count checks and Step 4a's issue child-count checks are tested on their passing side, each reporting the rows it compared, equal to the counts the scenario model or the stored totals give. The timeline lookup's entries before the boundary are events the scenario model renders. * docs(plan): take planting out of every step; replace the planted tests next (3b-2a-2d) - New Step 3b-2a-2d, ahead of Step 3b-2a-3: replace every merged test that changes a file or a copy of one, or writes a file, revision, call, connection, or decoy holding a flaw, to see whether a check catches it; a check GitHub, the environment, or a caller can fail keeps a refusing side built from D39's inputs, and one only a defect could fail is tested on its passing side with the exact count it compared. Step 7's item for this is removed, and D39 and its copies name 3b-2a-2d. - The history of Steps 1, 2, 3a, 3b-1, 3b-2a-2a, and 3b-2a-2c, gaps 52, 68, and 80, and 5.7 no longer describe planted, decoy, changed-copy, or edited-entry tests, and Steps 1 and 2 no longer accept on code removed or a document changed to see a test fail. - CLAUDE.md lists Step 3b-2a-2d. * docs(plan): close the review of a2fed22 and f55ae74, and of #27's D37 - Step 3b-2a-2d names every test file that plants by file (the engine's patched writer and prune, user_version and alembic_version edits, dropped indexes, the checkers' planted files, the wheel copy, edited query documents), states which inputs stay as D39 inputs, tests env.py's file-held violation refusal on its passing side, and accepts on a list of every test that writes or patches, with a word search over tests/**/*.py. - D39 and its four copies except configuration as well as stubs. - 6a tests verify --db-only's unknown-revision refusal on a file from a flawless test-only revision past head. - 3b-2a-3: the foreign_keys read-back through a fake DBAPI connection, and entries for an entry that resolves to another github_id. - D25 and 5.4: an unverified repository makes the exit 2; a rate limit only waits; association drift is a deliberate exception (D38); the timeline test's half-second entry states the model's comparison order. - 3b-2a-2b-2's history, 5.7, and the Step 3 overview follow. * docs(plan): close the convergence check of 6dbd284 - 3b-2a-3: SQLite built without foreign-key support answers `PRAGMA foreign_keys` with no row, not 0; the listener refuses a missing read-back, tested with a fake DBAPI connection that returns no row. - 3b-2a-2d names test_snapshot.py's half-read cursor control, test_views.py's row-id projection control, the authorizer and ATTACH revisions, and the revision-rule checker's planted files; the acceptance lists every test that runs a call or query holding a flaw, and its review finds no test that holds one; the patched size limit becomes an oversized in-memory unit; the patched prune's interrupt case stays as an interrupt at a set point. - 3b-2b-2: no model entry at half a second; under the measured since and whole-second times verify's createdAt filter is tested on its passing side. - 5.4: without --max-minutes a rate-limit wait over the named constant is fatal.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Splits Step 3b-2b into two pull requests and settles what plan 5.4 left open about verify, recorded in the plan of record (
docs/redesign-plan.md) as D36 and gaps 94-96.Steps (section 6 and the status table)
3b-2b-1, REST sampler: tests,
scripts/, anddocs/testing.md; nothing undersrc/changes, sinceTransport.rest_gettakes any path and query. It keeps the old scope's sampler clauses and adds:rest_getreturns noLinkheader);recorded/, replacing that directory alone, as the first-sync mode does; the offline tests replay that directory, and the default mode and--only documentsleave it as it is;PRIVATErefusal before any request, then the public-only resolve before the first REST request;--repo, refusingprebid-agent-skillsbefore any request (D33);Acceptance: an adversarial review and CI on the exact head.
3b-2b-2, verify:
ghactivity verify, its head check, the timeline lookup and its recordings, the scenario model's event history, and the live acceptance, which runs 3b-2b-1's sampler.Every scope, test, and acceptance clause of the old 3b-2b is in one of the two parts. One acceptance clause is removed: prebid-server-java joining when none of the three repositories holds an in-coverage dismissed review, since all three do (section 7).
Decisions (D36, 5.4)
verify writes nothing and claims no run. Before any request it reads
sync_runsand exits 1, naming the run, under the claim's rule: a nulldeadline_at, a future one, or one less than 10 minutes past.Before each repository's first index request, verify reads that repository's
repo_sync_stateandsync_pendingrows in one statement, then its items with their closing links andsync_failuresrows in one statement.--repolimits verify as it limits a sync (D27). Without it, the untracked set is the one 5.3 step 1 marks, which requires every configured entry to resolve without failing step 1.Unverified:
github_idmismatch, gap 92, and afull_nameheld by another tracked repository;A verify in which every repository it attempts is unverified, with no repository stopped by the deadline, is fatal.
Report:
Link totals alone (a PR with more than the 10 links the index carries): drift when the connected events at or after the boundary number at least the increase, or the disconnected events at least the decrease. With an earlier
updatedAt, each differing issue still needs an event naming it. The report names each link difference's issues and totals.A timeline lookup that still fails after one solo retry, a
filteredCountmismatch included, leaves its item unchecked, which fails only--exact.Design
snapshotexits 1 on a missing, unreadable, or stale (last_run_id) verify report, and the manifest records the unverified repositories. Step 6a's tests gain the refusal.repository(owner:, name:), passessinceone second before the boundary, and keeps entries at or after it. Its document takes notests/query_versions.jsonentry. The recorder's default mode and--only documentsrecord the timeline units.issue_labelscheck moves to Step 4a, beside the issue comment and assignee checks 5.1 already assigns there. Step 4a's tests gain all three.verify --exactexits 0, or exits 2 with only closing-link errors, each backed by REST evidence.Evidence (section 7, measured 2026-10-21)
since: on eight events in Prebid.js and prebid-server,sinceexcluded an event created at it and included it 1 ms earlier;filteredCountappliedsince; everycreatedAtwas a whole second. Each read cost 1 point.NOT_FOUND(github_id1708813), which GraphQL also leaves unlinked;DISMISSEDreviews in all three repositories (25, 622, 3).Gaps and open items
sinceexcludes its own instant. Gap 96: the sampler's paging and the recorder's modes.References
CLAUDE.md's scope lines name both parts.Head:
cfc040e3181e159959e93f560a4390015bebb322Ran:
uv run --no-sync pre-commit run --files docs/redesign-plan.md CLAUDE.md(passed); live GraphQL reads onprebid/Prebid.jsandprebid/prebid-server; read-onlysqlite3on Step 3b-2a-2c's acceptance file.Not run:
pytest,alembic check, and the wheel build (no code changed); REST paging on the sampler's endpoints; aDisconnectedEvent(none found).