Skip to content

test: assert the data/architectures catalog ↔ records ↔ docs contract (tests/architecture-catalog-contract.test.mjs) - #260

Open
kubestellar-hive[bot] wants to merge 1 commit into
mainfrom
quality/test-architecture-catalog-contract
Open

kubestellar-hive[bot] wants to merge 1 commit into
mainfrom
quality/test-architecture-catalog-contract

Conversation

@kubestellar-hive

Copy link
Copy Markdown
Contributor

Test Improvement

Adds one new test file, tests/architecture-catalog-contract.test.mjs, and nothing else.

scripts/import-architectures.mjs writes three artifacts in a single run — data/architectures/catalog.json, data/architectures/records/<id>.json, and docs/architectures/<id>.md — but nothing re-checks that they still agree once committed. validate-architectures.mjs reads only catalog.json; validate-architecture-assets.mjs reads only static/img/architectures; no test in tests/ loads data/architectures/records/ or docs/architectures/ at all.

The new file reads the committed data directly (no fixtures, no sandbox, no network) and asserts, in 11 tests:

  • catalog.json is a non-empty array with unique lowercase-slug ids
  • every record has the string/array fields the site renders, none blank
  • sourceCommit is a full 40-hex SHA, and sourceUrl is a cncf/architecture URL pinned to that SHA
  • catalog ids ↔ data/architectures/records/*.json are one-to-one
  • each records/<id>.json deep-equals its catalog entry
  • catalog ids ↔ docs/architectures/*.md (excluding index.md) are one-to-one
  • doc frontmatter title equals the record title; sidebar_label is present and non-blank
  • each doc page cites its record's sourceCommit and retains the CC BY 4.0 attribution required by the upstream import
  • docs/architectures/index.md exists and has a title
  • assets are namespaced under /img/architectures/<id>/ and present in static/
  • no orphan directory remains under static/img/architectures/

Frontmatter is parsed with the yaml devDependency already in package.json — the generated pages use both 'single' and "double" quoted YAML scalars, so a regex reader reports false mismatches on zeiss.md and swisscom-cloud-native-telco.md. No dependency is added and package.json is not touched.

Verification

npm ci then node --test on a clean clone at parent revision 00b44df: 66 tests pass, 0 fail (55 before, 11 added). npx prettier --check is clean on the added file.

Mutation-checked rather than merely green — on a scratch copy with records/zeiss.json deleted, docs/architectures/adobe.md deleted, and one sourceCommit corrupted to abc, 6 of the 11 tests fail with messages naming the offending id.

Coverage evidence

  • Unit: node --test --experimental-test-coverage, local clone of cncf/endusers at 00b44df after npm ci, 2026-09-17. Baseline: 55 tests; data/architectures/records/ and docs/architectures/ never read by any test. After: 66 tests.
  • End-to-end: unavailable — this repository has no end-to-end or browser suite and publishes no coverage artifact from any suite (tracked in [quality] CI publishes no coverage evidence, so coverage findings cannot be verified #186). No claim is made that this contract lacks end-to-end coverage.

Disjointness

Claimed ground is the single new file tests/architecture-catalog-contract.test.mjs. It overlaps no open PR: #216 claims tests/import-architectures.test.mjs + tests/helpers-import-sandbox.mjs (script behaviour in a sandbox); #231 claims tests/helpers.mjs + tests/validate-architecture-assets.test.mjs (--fix write mode); #229 claims the JSX import path under tests/tools/; #225 claims coverage reporting for fixture-sandbox runs. None of them assert anything about the relationship between the committed catalog, records, and doc pages. tests/helpers.mjs is deliberately left untouched so this cannot conflict with #231.

Related Issue

Closes #259


Filed by quality agent (hold-gated mode). Human review required.

— hive: agent=quality backend=copilot model=claude-opus-5

scripts/import-architectures.mjs writes data/architectures/catalog.json,
data/architectures/records/<id>.json and docs/architectures/<id>.md in a
single run, but nothing re-checks that they still agree once committed:
validate-architectures.mjs reads only catalog.json and
validate-architecture-assets.mjs reads only static/img/architectures.

Add tests/architecture-catalog-contract.test.mjs, which reads the committed
data with no fixtures and no network, and fails on: a catalog entry with no
doc page, a doc page with no catalog entry, a records file drifted from its
catalog entry, a doc title that differs from the record title, a missing
sidebar_label, a doc that no longer cites its record's sourceCommit or drops
the CC BY 4.0 attribution, a sourceUrl not pinned to sourceCommit, an asset
outside its record's namespace, and an orphan asset directory.

Frontmatter is parsed with the yaml devDependency rather than by regex: the
generated pages use both single- and double-quoted YAML scalars.

Signed-off-by: kubestellar-hive[bot] <kubestellar-hive@hive.kubestellar.io>
@kubestellar-hive

Copy link
Copy Markdown
Contributor Author

Important

Held for human review by the hive's ACMM level gate.

This PR was opened by the "quality" agent while Hive policy required a human checkpoint for that agent. Non-outreach agents are held at ACMM L3–L5; the outreach agent is always held because it publishes project-facing communication.

Hive will automatically remove the hold label once current policy no longer requires a level hold for "quality". If this is an outreach PR, a human must review it and remove the label.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[quality] No test asserts the data/architectures catalog ↔ records ↔ docs contract

0 participants