Restore green CI on main: stale podwiki count pins in the content-update contract test

[alexeygrigorev]

Raw user intake — needs PM grooming. main is currently RED.

Symptom

CI run https://github.com/DataTalksClub/website/actions/runs/33621318177 for 55f6743 concluded cancelled. The quality job failed and cancelled the run; ci-gate then failed with QUALITY_RESULT=cancelled DJANGO_RESULT=cancelled PLAYWRIGHT_RESULT=cancelled.

Failing gate: quality → make test-ci → uv run --frozen pytest ci/tests tests_ci -q (Makefile:192).

ci/tests/test_content_update.py:69: AssertionError
E   {"search_documents": 2996} != {"search_documents": 2998}
E   {"graph_nodes": 1070}      != {"graph_nodes": 1072}
E   {"graph_links": 12987}     != {"graph_links": 13006}
1 failed, 616 passed in 40.00s

Reproduces locally at 55f6743. Not a flake.

Root cause

The stale pins are the podwiki block at ci/tests/test_content_update.py:56-61. Counts are computed deterministically from the checked-in projection by _family_counts (ci/content_update.py:375).

Two lanes crossed:

• 53eef66 "Add unified content update CI contract" introduced the test with pins correct at that time.
• 4b6187f "Refresh projection for organized content source" de-duplicated the podcast catalogue during route canonicalization, removing two aliased nodes (podcast:s12e08, podcast:theme-park-crowd-modeling-to-tesla-full-stack-data-engineering) and 19 graph links from content/public_projection/wiki_graph.json and wiki_search.json. It correctly refreshed five other pinned contracts but not this newly-added one.

Measured across the push:

face8e4  links=13006 nodes=1072 search=2998 wiki=282
53eef66  links=13006 nodes=1072 search=2998 wiki=282   <- pins authored here
4b6187f  links=12987 nodes=1070 search=2996 wiki=282   <- break introduced here
55f6743  links=12987 nodes=1070 search=2996 wiki=282

The projection change is intended. Only the pin is stale.

Proposed fix (PM to confirm)

Update the three integers at ci/tests/test_content_update.py:57-59 to graph_links: 12987, graph_nodes: 1070, search_documents: 2996, leaving wiki_pages: 282.

This is explicitly NOT the #302/#253 re-pinning hazard: these are plain aggregate counts asserting redaction-safe report shape, not a content-authority digest, and the drift they caught is an intended de-duplication. Grooming should still confirm that reasoning.

Design question worth deciding here

A hardcoded exact-count pin inside ci/tests/ will red-line main on every legitimate content refresh — working against the purpose of the content-update pipeline the same commit introduced. Consider asserting invariants (non-zero, bounded, consistent with manifest.json) rather than exact literals. PM to decide whether that belongs in this issue or a follow-up.

Missing evidence — do not assume the fix is sufficient

django and playwright were killed at ~2 minutes by the quality job cancelling the run (.github/workflows/ci.yml:499-503). So make test-factories, make test-migrations, make test-django-full and the full Playwright matrix have no verdict for 55f6743.

The pre-push scheduled baseline (run 33591717271) failed migrations on content/tests/test_editorial_route_migration_contract.py:48 (required_content_sha256 mismatch). 4b6187f edited exactly that test, so it may be resolved — unverified. Verification must produce fresh full Django and Playwright evidence; do not rerun failed jobs on the cancelled run.

Related, but separate

• Restore the GitHub editorial source/projection inventory validator on main #302 — the inventory validator is red independently. Explicitly ruled out as the cause here: validate_github_editorial_source_projection_inventory.py was not among this run.s selected quality targets. Confirmed still failing at both face8e4 and 55f6743, i.e. pre-existing.
• Reconcile non-reproducible public projection after face8e4 #253 — the projection reproducibility reconciliation.

Process note

31 of the 50 commits in face8e4..55f6743, including 4b6187f and 53eef66, carry no Closes #N / Refs #N trailer, contrary to _docs/PROCESS.md step 6. No owning issue existed for the content-update CI contract or the projection reorganization, which is why this issue is being filed rather than an existing one reopened.

[alexeygrigorev]

Grooming Complete

PM decision

Choose Option B — durable aggregate-shape invariant. No separate product-semantic owner decision is needed: this issue changes a CI/report contract only and does not change public content or behavior.

I independently validated the exact current tree at origin/main = 2d3c28d1b8d1c2f3844f47f664945b42849d5eb9:

• Direct enumeration of the checked Podwiki projection yields graph_links=12987, graph_nodes=1070, search_documents=2996, and wiki_pages=282.
• uv run --frozen python report construction returns those same Podwiki counts.
• The current focused pin test fails only because it still expects 13006/1072/2998; all other family values match.
• 4b6187f intentionally removed two de-duplicated podcast nodes and their 19 graph links, and reduced search documents by two. The checked projection and its manifest remain the authority.

A literal refresh (Option A) would make the incident green but would deliberately fail on every legitimate projection refresh. The durable contract must remain strict: it must not become a merely non-zero or unconstrained test.

In scope

Only this tracked file may change:

• ci/tests/test_content_update.py

The engineer shall replace today-specific aggregate literals with a durable test contract for all four families:

• exact safe key sets remain: courses {courses}; Podwiki {graph_links, graph_nodes, search_documents, wiki_pages}; FAQ {assets, courses, questions, sections}; docs {assets, pages};
• every count is a real integer (booleans rejected), positive for the currently non-empty checked families, and bounded by the corresponding report projection byte total;
• counts are independently cross-checked against the checked projection's declared/enumerated collections, rather than merely comparing build_report() to another copy of its private counting helper;
• Podwiki graph links/nodes, wiki pages, and search documents are enumerated from their checked files; search document identities are distinct and complete;
• FAQ declared counts are cross-checked against its ordered courses, sections, questions, and distinct public asset paths; every referenced asset remains bound to a report-enumerated artifact without conflating referenced FAQ images with unrelated files in the asset root;
• docs page and asset counts are cross-checked against their declared unique records and the report-enumerated checked asset files;
• existing report shape, source-pin, manifest/artifact checksum, projection validation, redaction, and deterministic-build assertions remain unchanged.

Explicit non-goals

Do not modify ci/content_update.py, content/public_projection/**, content/faq_projection.json, content/docs_projection.json, content_sync/**, source pins, manifests, workflows, Makefile, provenance, #253, #302, or any public route/render behavior. Do not hand-edit generated projections or re-pin authority digests.

Clean candidate and contamination controls

Engineering and testing must use a fresh candidate worktree based exactly on 2d3c28d1b8d1c2f3844f47f664945b42849d5eb9, not the shared root checkout.

• Prove git status --porcelain is empty before editing and contains only ci/tests/test_content_update.py afterward.
• Before collecting evidence, prove test ! -e .local/staticfiles, test ! -e staticfiles, and test ! -e test.sqlite3. Keep generated output under that candidate's .tmp/ only.
• Use uv sync --locked or the repository's locked equivalent and uv run --frozen; do not fetch a source checkout, use credentials, or use production data.
• Produce a versioned verification plan binding the actual candidate base/head, ownership graph/policy, environment, and every component disposition. Any source, graph, policy, environment, or relevant-input drift invalidates the handoff.

Acceptance criteria

• The Option B decision and this bounded scope are recorded; needs grooming is removed.
• Engineer runs DJANGO_SETTINGS_MODULE=website.settings.test uv run --frozen pytest ci/tests/test_content_update.py -q and the focused contract passes.
• make content-update-check passes for courses, Podwiki, FAQ, and docs; make test-ci passes without projection-byte or authority changes.
• Because ci/ is owned by surface.ci with test-infrastructure risk, the plan contains fresh selector, evidence-validation (make test-ci), quality, Django, container, and Playwright-smoke evidence. Compatibility and large-content invariants may be not_applicable only with the graph-derived reason. Screenshots are not_applicable because no page/render input changes.
• An independent tester recomputes the plan from the frozen base/head, verifies the one-file diff and every criterion, validates all evidence, and posts a PASS report with no pending required component.
• The PM separately performs user-perspective acceptance; no implementation criterion is checked by this grooming comment.
• Only after tester PASS and PM acceptance does the engineer create a focused commit whose body is Closes #303; the orchestrator locally merges with --no-ff and pushes main. On-call alone observes the resulting CI and records a green terminal result. A cancelled or partial run is not green.

Normative process references: _docs/PROCESS.md, content-update runbook, and selective-CI contract.

[alexeygrigorev]

Software Engineer Report — #303

Disposition: implementation complete and frozen for independent testing. The candidate
implements the PM-selected Option B durable aggregate-shape invariant without changing any
projection, source pin, report builder, application code, or public behavior. It remains
uncommitted in a dedicated clean-base worktree.

Frozen candidate and provenance

• Worktree: /home/alexey/git/dtc-website/.tmp/issue-303-engineer-20260902
• Base and candidate HEAD: 2d3c28d1b8d1c2f3844f47f664945b42849d5eb9
• Source mode: worktree; the clean candidate was created from exact origin/main and was
  empty before editing.
• Changed tracked path: ci/tests/test_content_update.py only
• Candidate diff SHA-256 (git diff --binary):
  b69a924abb6fd0bd27c741336f9e0632d3109f212b243eba763fc6693e387acd
• Verification plan: .tmp/verification/issue-303-engineer-20260902/verification-plan.json
• Verification plan SHA-256:
  2f6be3084068096136897e5bd4149a25e0d6fbe885965bbe7ec0807d62fc60df
• Plan source-tree manifest SHA-256:
  211b42cc8d916b907d508bd905b72506b9e48fa192ea10e46ecacb97312593fd
• Plan source-tree Git tree OID: f7af08192c09361a3d064e8f8282b11d568ff211
• Ownership graph SHA-256: 05ffeee99244c9becd5e98e40be08188f1c5e2a61204daece984c3beb1d1bce6
• Ownership policy version: 5
• uv sync --locked, git diff --check, and the clean-checkout controls passed. Before
  evidence collection, .local/staticfiles, staticfiles, and test.sqlite3 were absent;
  they remained absent afterward. No projection or source-authority bytes changed.

Implementation

The former today-specific literals in test_source_specific_counts_are_safe_aggregates are
replaced with a strict contract that:

• requires the exact non-sensitive report key set for each of courses, podwiki, faq, and
  docs;
• rejects booleans and non-integers, requires positive counts for the checked non-empty families,
  and bounds every count by the corresponding report projection byte total;
• independently reads the checked JSON projections rather than calling the private report-count
  helper a second time;
• enumerates Podwiki pages, graph nodes/links, and search documents, requiring distinct stable
  identities, graph endpoints bound to nodes, Wiki pages bound to wiki: nodes, and search
  graph_id values bound to graph nodes;
• recomputes FAQ ordered course/section/question totals and distinct public image paths, checks
  question relationships, and binds only referenced FAQ images to report-enumerated artifacts
  (the unrelated CSS/assets in the asset root are not counted);
• recomputes docs page and asset totals from distinct declared records and requires the exact
  report-enumerated docs asset set to match the projection's declared source paths.

The existing report shape, redaction, source-pin, checksum, manifest, projection-validation,
deterministic-build, workflow, and action assertions are unchanged. No literal current count,
projection, application/content projection code, workflow, or source pin was edited.

Versioned plan and dispositions

The plan is valid and binds the one-file worktree diff. It independently selects:

• profile full, reason test_infrastructure, browser profile smoke;
• rerun: selector, evidence validation, quality, Django, Playwright smoke, and container;
• not_applicable: compatibility and content invariants (outside the graph/large-content
  closure), plus screenshots (render_inputs_unchanged_without_reusable_visual);
• reused: none; skipped: none.

There is no render impact, so screenshots are explicitly not applicable. The graph-selected
runner was not started because the unrelated .tmp/production-prep worktree currently owns the
exclusive Chromium slot; an independent tester must recompute and run the pending Playwright
smoke component when that slot is available.

Commands and results

Passing:

• DJANGO_SETTINGS_MODULE=website.settings.test uv run --frozen pytest ci/tests/test_content_update.py -q — 10 passed
• make content-update-check CONTENT_UPDATE_OUTPUT_DIR=.tmp/content-update/issue-303-engineer-20260902 — all four families passed
• make test-ci — 617 passed
• make lint — passed
• make format-check — passed (653 files already formatted)
• make typecheck — passed (472 source files)
• make migrations-check — passed (No changes detected)
• make django-check — passed (OpenAPI, management parity, Django system check)
• make deployment-check — passed
• make security-check — passed (baseline, vulnerability, canary, artifact scan)
• make database-portability-check — passed
• make verification-container VERIFY_CONTAINER_OUTPUT=.tmp/verification/issue-303-engineer-20260902/container-check.json — passed (13 assertions)

The four content reports are below .tmp/content-update/issue-303-engineer-20260902/ and
validate with these current aggregates: courses 12; Podwiki 12987/1070/2996/282 for
links/nodes/search documents/Wiki pages; FAQ 99/6/1401/70 for assets/courses/questions/
sections; docs 39/106 for assets/pages. These values are diagnostics from direct enumeration,
not frozen literals.

Current-base limitations

• make verification-quality fails at the pre-existing terminology-check because
  _docs/runbooks/production-hosting-and-dns-migration.md contains 102 unclassified legacy-term
  findings. This candidate does not touch that document or its allowlist.
• make test-django-full starts 2,597 tests but fails the same existing terminology inventory
  assertion in core/tests/test_development_terminology.py; Django's parallel failure path then
  reports the known TypeError: cannot pickle 'traceback' object. No failure points to the
  changed test contract.
• Playwright smoke and the graph-selected verification runner remain pending because of the
  exclusive browser owner above. Screenshots are not required for this non-rendering change.

The candidate is frozen for independent tester review and separate PM acceptance after the
current-base blockers and browser ownership are resolved. No commit, push, merge, PR, deployment,
or issue closure was performed.

[alexeygrigorev]

QA Review — #303

FAIL — independent tester verification (2026-09-02). The #303 contract itself passes its
focused checks, but the required full-profile quality and Django components fail on the current
base, and the required Playwright smoke component could not be started because Chromium is owned
by active production-prep/#301 sessions. This is not a PASS or a releasable candidate.

Candidate and provenance

• Tester worktree: .tmp/verification/issue-303-tester-20260902
• Frozen base/head: 2d3c28d1b8d1c2f3844f47f664945b42849d5eb9
• Changed tracked path: ci/tests/test_content_update.py only
• Recomputed candidate diff SHA-256: b69a924abb6fd0bd27c741336f9e0632d3109f212b243eba763fc6693e387acd
• Engineer plan SHA-256: 2f6be3084068096136897e5bd4149a25e0d6fbe885965bbe7ec0807d62fc60df
• Tester plan SHA-256: 8dc8b358ae8d3dc09c41a29000d86875e3fb9110246900847571ced91654866c
• Plan comparison: exact same base/head, changed path, source manifest 211b42cc8d916b907d508bd905b72506b9e48fa192ea10e46ecacb97312593fd, tree OID f7af08192c09361a3d064e8f8282b11d568ff211, graph 05ffeee99244c9becd5e98e40be08188f1c5e2a61204daece984c3beb1d1bce6, policy 5, profile full, reason test_infrastructure, browser smoke, render false, and dispositions. The only JSON difference is the expected created_at timestamp.
• Pre/post controls: .local/staticfiles, staticfiles, and test.sqlite3 absent; git diff --check passed.

Passed candidate-specific and repository checks

• DJANGO_SETTINGS_MODULE=website.settings.test uv run --frozen pytest ci/tests/test_content_update.py -q — 10 passed.
• make content-update-check CONTENT_UPDATE_OUTPUT_DIR=.tmp/content-update/issue-303-tester-20260902 — courses, Podwiki, FAQ, and docs passed; report artifact SHA-256s: 1090c6c5b868f5416536556d905d6142d597253a931af7f75ec5b234b6a671e7, d3a0732cd57c33561ca4e12b27dad83b80f55d1f72d5fe4327058d17da0c64fa, 627c491e8e871e8b1f94c9c6a194da043861931bbd88d891cd8dc4b1e2d5fdaf, e717f747eaf6226c8ffd2c22b0d7106559ca4d3a583ff62cbe738c6a13cf6fa.
• make test-ci — 617 passed.
• make lint, make format-check (653 files), make typecheck (472 files), make migrations-check, make django-check, make deployment-check, make security-check, and make database-portability-check — passed.
• make verification-container VERIFY_CONTAINER_OUTPUT=.../manual/container-check.json — 13 assertions passed, artifact SHA-256 9d660335e2a33f2d62cdde186c709feca173d3b7d41c1e9daddab992e88a7ef6.

Blocking failures

• uv run --frozen python -m ci.quality_contract --repository . — exit 2; terminology-check reports 102 unclassified legacy-term findings in the unchanged _docs/runbooks/production-hosting-and-dns-migration.md.
• make test-django-full — exit 2; the same 102-finding terminology inventory assertion fails in core/tests/test_development_terminology.py, followed by Django's known TypeError: cannot pickle 'traceback' object failure path.
• make test-playwright-smoke — not started. Chromium is currently owned by active .tmp/production-prep and Serve public projection media from object storage instead of the git-tracked tree #301 sessions; starting another browser would violate the exclusive-slot process rule. The plan requires this component, so the candidate cannot receive tester PASS yet.

Screenshots are not_applicable under the recomputed plan because the sole change is test infrastructure and has no render impact. The interim evidence directory validates (validate-evidence-directory=0) and its failure report validates (validate-report=0); it correctly records quality/Django failures and missing Playwright evidence. No source changes, commit, push, merge, or deployment were made.

Evidence: .tmp/verification/issue-303-tester-20260902/manual/summary.txt, .tmp/verification/issue-303-tester-20260902/tester-interim-report.json, and .tmp/verification/issue-303-tester-20260902/tester-interim-report.md.

[alexeygrigorev]

PM review — HOLD / NOT ACCEPTED

I confirm the groomed Option B durable aggregate-shape invariant remains the correct scope for #303. It is a CI/report-contract change only: the exact safe key sets and projection-derived cross-checks for courses, Podwiki, FAQ, and docs belong in ci/tests/test_content_update.py; projection bytes, source pins, manifests, report construction, application behavior, and public routes remain out of scope.

Evidence disposition

The candidate-specific evidence is positive:

• the focused contract passes (10 tests);
• the four-family content-update check passes;
• the engineer reports make test-ci at 617 passed and the other static, type, migration, security, deployment, portability, and container checks green;
• the one-file candidate and verification envelope are recorded in engineer comment 5509348589.

That does not satisfy the PM gate. The independent tester report 5509523770 is explicitly FAIL, not PASS:

• the required quality component stops on 102 unclassified legacy-term findings in the unchanged production-hosting runbook;
• full Django fails on that same inventory and then the known Python 3.13 traceback-pickling path;
• the required Playwright-smoke component was not run because Chromium was owned by another active lane;
• the tester broad make test-ci result still observes the old Podwiki literals (616 passed, 1 failed).

The terminology inventory, traceback-pickling path, absent pinned dtc-main-site compatibility checkout, and unavailable browser slot are inherited/current-base or environment blockers. The stale Podwiki result is not waived as inherited: it is the exact contract owned by #303, so the candidate must be rerun from the exact candidate tree with the changed test file present and the discrepancy explained. A focused PASS cannot override a failed or missing full-profile component.

PM disposition

HOLD — NOT ACCEPTED. No acceptance checkbox is claimed. The candidate must remain uncommitted and unmerged; no commit, push, deployment, or issue closure is authorized by this review.

Required replay order

1. Complete the shared baseline/quality and compatibility provenance repairs, and release the exclusive browser slot.
2. Recompute the tester plan from the frozen Restore green CI on main: stale podwiki count pins in the content-update contract test #303 base/head and rerun the changed contract plus make test-ci, quality, full Django, Playwright smoke, evidence validation, and the required container checks. Confirm the run actually includes ci/tests/test_content_update.py from this candidate.
3. Only after an independent tester PASS with no required pending component will PM perform a fresh final acceptance; then the normal commit, local merge, push, and on-call sequence may proceed.
4. The Public course projection is stale: 2026 cohorts exist in the database but the site advertises 2025 #307 product lane is replayed separately after the AI Dev Tools renders as two separate course families, orphaning the 2026 cohort #308 freshness/data evidence is resolved; Restore green CI on main: stale podwiki count pins in the content-update contract test #303’s contract repair must be part of the clean baseline used for that replay.

[alexeygrigorev]

PM coordination clarification — shared baseline replay order

The frozen handoffs cannot be reused as-is: #303 is based on 2d3c28d1, #309 on c96578f, and local main still requires reconciliation of external/unrelated work. First freeze one clean, exact current-main baseline B0, after preserving that work and proving there are no generated artifacts or unrelated candidate changes.

Dependency order is #303 → #309:

1. Replay only Restore green CI on main: stale podwiki count pins in the content-update contract test #303 (ci/tests/test_content_update.py) on B0. An independent tester must recompute the plan from that exact candidate and PASS the focused contract, make test-ci, quality, full Django, required Playwright smoke, evidence validation, and container checks. PM acceptance, the focused Closes #303 commit, local merge/push, and a green on-call terminal result are required before proceeding.
2. Freeze the resulting exact current main as B1. Replay only the two Production hosting, DNS and content migration — tracking checklist #309 terminology files on B1, recompute its 102-occurrence ledger and all plan/policy digests, and obtain a separate independent tester PASS plus PM acceptance before its focused Refs #309 commit. The broader Production hosting, DNS and content migration — tracking checklist #309 checklist and Reconcile non-reproducible public projection after face8e4 #253 source/projection work remain separate.

No broad #303+#309 union candidate is authorized, and no tester/engineer evidence is reused across the two lanes. #309 may inherit only the accepted #303 commit through B1; its own diff and acceptance scope remain limited to the finite terminology allowlist and regression test. Any base, projection, policy, or relevant-input drift requires a new freeze and replay.