docs(plan): apply D37 to the sync and to Steps 4b-6 (D38); no planted inputs (D39) by ChrisHuie · Pull Request #28 · prebid/github-activity-db

ChrisHuie · 2026-10-22T00:43:04Z

Applies D37 (an error is better than a silent failure or silently wrong data) to the merged sync and to the steps not yet built, recorded in the plan of record (docs/redesign-plan.md) as D38 and gaps 98-110, and records D39: no planted, seeded, or tampered inputs and no mutation testing. Changes docs/redesign-plan.md, CLAUDE.md, docs/testing.md, and CONTRIBUTING.md; no code.

Builds on #27 (D37 and the verify design). Until #27 merges, this diff also shows #27's four commits; this pull request's own commits are those after 9c2b803.

New Step 3b-2a-3: D37 in the sync (before the REST sampler and verify)

Account refresh: a 24-hour refresh that still fails, or answers another databaseId, is counted in sync_runs.refresh_failures, listed in the report, and makes the exit 2 unless an account entry in [sync].accepted_failures accepts it. Today it exits 0 and a test pins that.
Deadline: a run that reaches --max-minutes exits 2 (still published under D25). Item failures a batch made final before the deadline are recorded instead of being filed as pending.
Index contradictions: a PR the run's own index listed in coverage that hydration answers NOT_FOUND, returns outside coverage, or returns with an earlier updatedAt is requested once more alone, then is an item failure of class IndexContradiction. The report lists each contradiction and whether the retry cleared it.
Accepted failures: an entry names the error class and the cause, the first line of the failure's message. Every recorded failure puts a cause without request ids, counts, or typed-model text on that line, so an entry keeps matching a permanent failure and a new cause is not accepted. tracking.py keeps the list of recorded classes, held equal to the engine's classes by a test. An entry may name an account.
Closing links: a PR with more links than the 10 the index carries is hydrated every run.
Node ids: account_node_ids stores every node id a sighting, a lookup, or a commit link reports for an account.
Commit-link NOT_FOUND: sync_runs.link_lookups and links_not_found count first lookups and their NOT_FOUND answers; from Step 6a a run above the share and minimum Step 5 sets exits 2.
Source checks (gap 104):
- a readable 2xx REST response without rate-limit headers is not understood (exit 1); one that is not JSON or lacks Date stays a retried server error;
- a GraphQL errors that is not a list, null included, is a server error;
- above 250 commits, GraphQL's first commit page must equal compare's positions total_commits - 250 to total_commits - 151, in order (GraphQL holds the newest 250; Prebid.js #8858's page is positions 4-103); the rule is measured live on the acceptance repositories' large PRs before it is fixed;
- every index page's repository databaseId is checked against the resolved one.
Run records (gap 105): code_version marks -unknown when git fails; a close given an outcome it cannot write refuses; [automation] is required; foreign_keys is read back; alembic stamp that changes the revision is refused; the wheel check compares tracking.toml bytes; the log file receives records at its level; sync_runs records repo_filter, entries (each configured entry's github_id), and trigger = unknown for other events.
The scope names the fixtures and tests these rules change, the scenario model's newest-250 commit rendering, tests/sync/expected_state.py, the re-recordings, and docs/development.md's exit codes.

Steps not built yet

Step 4b:
- compare file lists marked files_at_cap at GitHub's 300-file cap and read from the first page only;
- an activity read complete only at the floor or at the first event of an ascending read, an empty log included;
- a compare that still fails, lacks files, or returns fewer commits than total_commits is stored with its error, retried each run, and acceptable by a compare entry;
- deletions counted in sync_runs.events_deleted and history_commits_deleted; a deletion halt exits 2;
- verify marks a repository unverified when its own log or history read cannot complete, and classes merge-brought commits by their event.
Step 5: measures, per edit kind and with a stated instrument, whether body edits, base-ref changes, review dismissals, and issue label events move updatedAt, counting only cases no later change follows; records index-contradiction and commit-link NOT_FOUND rates. Pull request 5b adds the resulting rule or retry delay.
Step 6:
- before any App run publishes to db, 6b compares the App's reads with the token's over every table the sync writes from GitHub, on a sample chosen through a new repo dispatch input (bootstrap only, never published to db, D25), with row ids mapped to natural keys; any difference blocks App publication;
- the job pins the working file's SHA-256 before migrating; snapshot --after-run ties the snapshot to the job's own sync through entries;
- a test reads sync.yml and checks every exit-code condition's exact quoted values;
- verify --db-only's digest and manifest refusals are tested on snapshots the code wrote.
5.6 consumers: a pair whose manifest sync_finished_at is older than 13 hours is stale. A stopped schedule starts no run, so this repository cannot see it; the dashboard's integration builds that check (section 9).

D39

No test, review, or acceptance changes a file or a copy of one, or writes a file, row, revision, call site, or decoy holding a flaw, to see whether something catches it, and none measures a detection rate. Test inputs are GitHub's recorded responses, stubs of what GitHub or the environment can send, calls and configuration the code must refuse, valid rows (tests/db/seed.py), and files the project's own code wrote. A check only a defect of this code could fail is tested on its passing side with the exact count it compared.

The plan's test lists for the sampler, verify, Steps 4a, 4b, and 6 no longer use planted mismatches, planted row deletions, tampered copies, or corrupted db-test snapshots.
Section 7 no longer cites a mutation run.
Step 7 replaces merged tests that plant (the schema comparison's planted differences, the migration tests' planted revisions, the planted calls and decoys of the placeholder check and the suppression counter, 5.7's planted import files).
CLAUDE.md, docs/testing.md, and CONTRIBUTING.md state the same rule.

Left as they are

A commit-linked account a lookup answers NOT_FOUND: stored as D28 says, now counted per run and in the manifest.
A REST git author without a name or email: stored null, as GitHub reports none.
Deletions: counted; the snapshots kept for 7 days hold the identities.
Merge state as of hydrated_at (D32).

Head: a2fed22e7392c1a6f97107ebd45b06bac818b38f

Ran: uv run --no-sync pre-commit run --files docs/redesign-plan.md CLAUDE.md docs/testing.md CONTRIBUTING.md (passed); read-only reads of the Prebid.js #8858 recording (GraphQL page equals compare positions 4-103, in order) and of installed githubkit 0.14.2 and alembic 1.18.1.
Not run: pytest, alembic check, and the wheel build (no code changed); no GitHub behaviour Step 4b, Step 5, or Step 6 relies on was measured live (activity-log paging and direction=asc, App-token reads, the 300-file compare cap, the commit-page order on PRs with merge commits).

… verify's edges (D36) Step 3b-2b becomes 3b-2b-1 (the REST sampler and its recorder mode) and 3b-2b-2 (verify, its timeline lookup, and the live acceptance). 5.4 states verify's lock check, repository set under --repo, report on each exit, unverified repositories, link-total differences, and cost fields (D36, gap 94). 5.8 passes the timeline lookup's since one second before the boundary, as measured (gap 95). 3b-2b-1 gains a paging rule and recorder-mode rules (gap 96). Section 7 records the timeline measurements and Step 3b-2a-2c's acceptance file; section 8 drops the items that file resolved.

5.4: verify's untracked set follows 5.3 step 1's every-entry-resolved rule; the lock rule names a future deadline; stored rows are read per repository before its index; the unverified list names every resolve and index failure and gap 92; a verify with no repository verified is fatal; a totals-only link difference needs as many connected or disconnected events as the change; an interrupt keeps 130 when the report cannot be written; elapsed seconds are defined. 5.6: snapshot refuses a missing or stale verify report. 5.8: the lookup keeps entries at or after the boundary, and a count mismatch leaves its item unchecked. 3b-2b-1: the sampler's REST units live in their own directory, the sampler takes --repo, and the offline tests pair REST and GraphQL units from one recorder run. 3b-2b-2: matching tests, a timeline re-recording rule, a 2,500-point floor, and the exit the acceptance requires. Step 4a: issue child-count tests. Section 7: per-run failure counts, code versions, and the compare-linked accounts.

5.4: verify reads each repository's state and pending rows in one statement, then its items with their links and failures in one statement; the untracked set requires every entry to resolve without failing 5.3 step 1; the unverified list adds the full_name check; a repository error in either lookup leaves the items no lookup answered unchecked; the per-issue link rule applies only to an earlier updatedAt, and the report names each link difference's issues and totals. 5.8: the timeline lookup reads through repository(owner:, name:). 3b-2b-1: the sampler's recorder mode writes every unit of its run into its own directory. D36, gap 96, 5.6, section 8, and the 3b-2b-2 and 6a test lists follow.

An unchecked item and an unverified repository now fail verify in both modes, not only under --exact, so the scheduled verify cannot exit 0 on a database it did not check and D25 does not publish it. A timeline lookup whose entries still differ from filteredCount after its retry, or whose response the typed models reject, is an error. D37 records the rule: an error over a silent failure or silently wrong data. Gap 97, D25's wording, 5.8, and the 3b-2b-2 tests follow.

Records the decisions on the silent-failure audit as D38 and gaps 98-110: Step 3b-2a-3 makes a failed account refresh, a time-limited run with unreached items, and a hydration that contradicts the run's own index each exit 2, scopes accepted failures to an error class, re-hydrates PRs with more links than the index carries, stores every node id GitHub reports, and closes the source and run-record holes the audit found. Steps 4b, 5, and 6 and 5.6 gain the completeness, permission, staleness, and workflow rules.

A hydration that contradicts the run's index gets one engine retry, then fails. Any deadline makes the exit 2, and the failures a batch made final before it are recorded. The migration refuses a populated file. A close that cannot write an outcome raises before writing. code_version names which git answer it trusts. Commit-link node ids are stored; accepted failures match the class exactly and can name an account. verify --db-only writes and digests the working file, snapshot checks the job's own sync through --after-run, and 6b compares App and token reads before any App run publishes. The stopped-schedule case is stated as visible only to a consumer.

Accepted failures match class and message exactly and can name an account by github_id or node_id; a refresh failure an account entry accepts leaves the exit code alone, and verify --exact matches entries as 5.5 does. code_version decides a checkout by a .git entry before reading git's exit codes. A stamp is refused by on_version_apply inside the transaction. Gap 110 refuses only for rows no later run reads. snapshot takes --after-run/--sync-exit or --sync-run, checks the stored row of every configured entry, and names its manifest keys. 6b's App-and-token comparison names its runs and script. Tests follow each rule.

… (D39) D39 records the maintainer's rule: no test, review, or acceptance changes a copy of the code, a test, the plan, review material, a database, a snapshot, or a manifest to see whether something catches the change, and none measures how many such changes are found. Test inputs are recorded GitHub responses, stubs of what GitHub or the environment can send, calls and configuration the code must refuse, and files this project's own code wrote; a check only a defect could fail is tested on its passing side with the exact count it compared. - Step 3's evidence rule, the sampler, verify, Step 4a, Step 4b, and Step 6 no longer prescribe planted mismatches, planted row deletions, tampered copies, or a corrupted db-test release. - Section 7 drops the mutation result from the SQLite pragmas row. - Step 7's audit replaces merged tests that change a copy of a database or a file. - docs/testing.md, CONTRIBUTING.md, and CLAUDE.md state the same rule.

- 5.3 step 5: GraphQL's commit connection holds the newest 250, so above 250 commits its first page must equal compare's positions total-250..total-151 in order (Prebid.js #8858: positions 4-103); the oldest commits are checked by count alone. - 5.5: an accepted_failures entry matches the class and the cause, the first line of the failure's message, which carries no request id, count, or typed-model text; tracking.py keeps the recorded class list, held equal to the engine's classes by a test; index contradictions take the class IndexContradiction; Step 4b adds a compare entry form. - 5.6 and Step 6: a validated `repo` dispatch input (bootstrap only) for 6b's App-and-token comparison, which covers every table the sync writes from GitHub, names the cases its sample must hold, and lists App nulls; the manifest carries repo_filter, links_not_found, and Step 4b's counts; 6a tests compare_reads and lists each exit-code output for the sync.yml test. - Step 4b: completeness against an ascending read's first event, files from the first page, per-compare transactions, a stored error for a compare that still fails, deletion counters, verify's own read as unverified, halted vs not-synced classes, and merge-brought commits. - Step 5: the updatedAt measurement's instrument per kind and its counting rule, contradiction rates, the link NOT_FOUND share, and the bootstrap's projected time; the conditional pull request is 5b. - 5.1: a read version after Step 6b instead of a refusal, account_node_ids' keys and indexes, trigger `unknown`, untracked files outside -dirty, and `stamp --purge` at head. - Step 3b-2a-3: scope and tests follow the above, plus re-recording and docs/development.md.

- D39 says "no flaw is seeded" (valid seed.py rows stay allowed), lists one set of allowed inputs everywhere it is restated, and widens Step 7's replacement to tests that write a planted file, revision, call, connection, or decoy; 5.7's planted import files are marked for that replacement. - The scenario model renders a PR above 250 commits as GitHub does (newest 250), and the commit-page rule is measured live on the acceptance repositories' large PRs before it is fixed. - A 2xx REST response that is not JSON or lacks a Date is a server error whatever its headers; only a readable one without rate-limit headers is not understood. - sync_runs gains link_lookups (the NOT_FOUND limit's denominator, without accounts an earlier lookup answered NOT_FOUND) and entries (each configured entry's github_id), which snapshot --after-run checks. - D25 and publish's condition exclude a run with `repo` from release db; the sync.yml test checks publish's and report's exact values and snapshot's --out and --run-id. - verify --db-only's digest and manifest refusals are tested on snapshots the code wrote; compare_reads maps row ids to natural keys; db-test runs go in a stated order. - Step 4b: a halt exits 2, an empty log is complete when the ascending read is empty, a commit-count mismatch or later-page 404 is a compare error, the refresh runs first, verify reads stored rows before its reads, and a compare entry's not-failed rule. - 5b covers a recurring index contradiction's retry delay; the report lists each contradiction and whether the retry cleared it.

- D39's rule is one text, copied word for word into Step 3, CLAUDE.md, docs/testing.md, and CONTRIBUTING.md; it allows stubs built in memory from a recorded unit, valid rows written in SQL, and configuration naming a state GitHub can produce, for tests and acceptances. - Step 3b-2a-3: the readable-2xx REST rule in its summaries, and how each refusing shape is built (replay.modify or the scenario model's set). - sync_runs.entries records the matched stored row's github_id, else the resolved one; snapshot skips an id with no stored row; a snapshot's T is the time snapshot writes it. - 6a states why verify --db-only's unknown-revision refusal is tested on its passing side; the acceptance names the refusals the 6a tests cover. - D38 and 6a state the commit-link limit as 5.2 does; Step 7 names only the view-alter revision that leaves a view in place.

…nted inputs Verify's child-count checks and Step 4a's issue child-count checks are tested on their passing side, each reporting the rows it compared, equal to the counts the scenario model or the stored totals give. The timeline lookup's entries before the boundary are events the scenario model renders.

…ent-failures

…s next (3b-2a-2d) - New Step 3b-2a-2d, ahead of Step 3b-2a-3: replace every merged test that changes a file or a copy of one, or writes a file, revision, call, connection, or decoy holding a flaw, to see whether a check catches it; a check GitHub, the environment, or a caller can fail keeps a refusing side built from D39's inputs, and one only a defect could fail is tested on its passing side with the exact count it compared. Step 7's item for this is removed, and D39 and its copies name 3b-2a-2d. - The history of Steps 1, 2, 3a, 3b-1, 3b-2a-2a, and 3b-2a-2c, gaps 52, 68, and 80, and 5.7 no longer describe planted, decoy, changed-copy, or edited-entry tests, and Steps 1 and 2 no longer accept on code removed or a document changed to see a test fail. - CLAUDE.md lists Step 3b-2a-2d.

- Step 3b-2a-2d names every test file that plants by file (the engine's patched writer and prune, user_version and alembic_version edits, dropped indexes, the checkers' planted files, the wheel copy, edited query documents), states which inputs stay as D39 inputs, tests env.py's file-held violation refusal on its passing side, and accepts on a list of every test that writes or patches, with a word search over tests/**/*.py. - D39 and its four copies except configuration as well as stubs. - 6a tests verify --db-only's unknown-revision refusal on a file from a flawless test-only revision past head. - 3b-2a-3: the foreign_keys read-back through a fake DBAPI connection, and entries for an entry that resolves to another github_id. - D25 and 5.4: an unverified repository makes the exit 2; a rate limit only waits; association drift is a deliberate exception (D38); the timeline test's half-second entry states the model's comparison order. - 3b-2a-2b-2's history, 5.7, and the Step 3 overview follow.

- 3b-2a-3: SQLite built without foreign-key support answers `PRAGMA foreign_keys` with no row, not 0; the listener refuses a missing read-back, tested with a fake DBAPI connection that returns no row. - 3b-2a-2d names test_snapshot.py's half-read cursor control, test_views.py's row-id projection control, the authorizer and ATTACH revisions, and the revision-rule checker's planted files; the acceptance lists every test that runs a call or query holding a flaw, and its review finds no test that holds one; the patched size limit becomes an oversized in-memory unit; the patched prune's interrupt case stays as an interrupt at a set point. - 3b-2b-2: no model entry at half a second; under the measured since and whole-second times verify's createdAt filter is tested on its passing side. - 5.4: without --max-minutes a rate-limit wait over the named constant is fatal.

ChrisHuie · 2026-10-22T12:21:05Z

Head is now 8f0a759. Changes since the description, which was written at a2fed22:

docs(plan): split Step 3b-2b into the REST sampler and verify; settle verify's edges (D36) #27 merged in (49ab7d0). docs(plan): split Step 3b-2b into the REST sampler and verify; settle verify's edges (D36) #27's head 54428dc states verify's child-count and timeline tests and Step 4a's child-count tests without planted inputs. The merge changed no content here.
Planting out of every step (f55ae74):
- New Step 3b-2a-2d, ahead of Step 3b-2a-3, replaces every merged test that changes a file or a copy of one, or writes a file, revision, call, connection, or decoy holding a flaw, to see whether a check catches it. Step 7's item for this is removed.
- The history of Steps 1, 2, 3a, 3b-1, 3b-2a-2a, and 3b-2a-2c, gaps 52, 68, and 80, and 5.7 no longer describe planted, decoy, changed-copy, or edited-entry tests, and 3b-2a-2b-2's history follows in 6dbd284. Steps 1 and 2 no longer accept on code removed, or a document changed, to see a test fail.
Step 3b-2a-2d made complete (6dbd284, 8f0a759):
- It names the tests to replace by file, including those that plant without the usual words: the engine's patched writer and prune, user_version and alembic_version edits, dropped indexes, the positive controls in test_views.py and test_snapshot.py, the authorizer, ATTACH, and revision-rule revisions, the checkers' planted files, the wheel copy, and edited query documents.
- It states which inputs stay because D39 allows them: kill and interrupt points, replay.modify stubs, a caller's exception, exit-code test commands, valid scratch tables, and the tracked-file check's WAL and symlink inputs.
- Its acceptance lists every test that writes or patches something, with its D39 input or replacement. Its word search covers tests/**/*.py only, since recorded GitHub text contains "corruption" and "mutation".
D39 exempts configuration as well as stubs, in the plan and its four copies.
Other fixes:
- 6a tests verify --db-only's unknown-revision refusal on a file from a flawless test-only revision past head.
- 3b-2a-3 tests the foreign_keys read-back with a fake DBAPI connection that returns no row, as SQLite built without foreign-key support does.
- D25 and 5.4: an unverified repository makes verify's exit 2.
- 5.4: a rate limit waits, ending as a timeout or, without --max-minutes, fatal.
- D38 records association drift as a deliberate exception.
- The timeline test no longer asks the model for an entry GitHub would not return.

CI: the tests jobs fail on this head and on #27 for a reason outside these changes. tests/source/test_recorder.py replayed go-gdpr's first sync, recorded 2026-10-20, on the system clock, so from a day later the sync's 24-hour refresh asked for lookups the recording does not hold. #29 fixes it and is green. Once #29 is on main, main is merged into this branch.

Ran: uv run --no-sync pre-commit run --files docs/redesign-plan.md CLAUDE.md docs/testing.md CONTRIBUTING.md (passed).
Not run: pytest (no code changed here).

…nt-failures

…nt-failures # Conflicts: # CLAUDE.md # docs/redesign-plan.md

ChrisHuie added 16 commits October 21, 2026 13:57

Merge branch 'feature/plan-step3b2b-verify' into feature/plan-d37-sil…

49ab7d0

…ent-failures

ChrisHuie added 2 commits October 22, 2026 06:34

Merge remote-tracking branch 'origin/main' into feature/plan-d37-sile…

863b850

…nt-failures

Merge remote-tracking branch 'origin/main' into feature/plan-d37-sile…

953db70

…nt-failures # Conflicts: # CLAUDE.md # docs/redesign-plan.md

ChrisHuie merged commit 9d68f7b into main Oct 22, 2026
4 checks passed

ChrisHuie deleted the feature/plan-d37-silent-failures branch October 22, 2026 20:25

Navigation Menu

Repository navigation

docs(plan): apply D37 to the sync and to Steps 4b-6 (D38); no planted inputs (D39) - #28

docs(plan): apply D37 to the sync and to Steps 4b-6 (D38); no planted inputs (D39)#28
ChrisHuie merged 18 commits into
mainfrom
feature/plan-d37-silent-failures

ChrisHuie commented Oct 22, 2026

Uh oh!

ChrisHuie commented Oct 22, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

ChrisHuie commented Oct 22, 2026

New Step 3b-2a-3: D37 in the sync (before the REST sampler and verify)

Steps not built yet

D39

Left as they are

Uh oh!

ChrisHuie commented Oct 22, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant