Skip to content

test: cover the package.json script and CI command wiring contract - #278

Open
kubestellar-hive[bot] wants to merge 1 commit into
mainfrom
quality/test-workflow-script-wiring
Open

kubestellar-hive[bot] wants to merge 1 commit into
mainfrom
quality/test-workflow-script-wiring

Conversation

@kubestellar-hive

Copy link
Copy Markdown
Contributor

Test Improvement

Adds one new test-only file, tests/workflow-scripts.test.mjs, covering the
wiring contract between package.json scripts, the files under scripts/,
the linter config dotfiles those scripts name, and the npm run invocations
in .github/workflows/*.yml.

Files claimed by this PR: tests/workflow-scripts.test.mjs (new, 7 tests).
Nothing else — no production code, no workflow file, no package.json, no
lockfile, no dependency. yaml is already a devDependency and is used to parse
the workflows rather than regexing YAML.

The gap

These four files refer to each other entirely through strings that name
something in another file: node scripts/<f>.mjs targets, -c <dotfile>
linter configs, npm run <other> chains, and 19 npm run <name> steps across
the workflows. No test in the repository reads package.json or any workflow,
so none of those references is checked. The existing suite spawns the
validators directly via tests/helpers.mjs, which deliberately bypasses the
npm script layer — correct for those tests, but it leaves the layer CI and
contributors actually invoke unverified.

This has already bitten the repo: npm run check once failed on a fresh clone
because .cspell.yml, .markdownlint.yaml and .markdown-link-check.json
were named by scripts but absent. Assertion 5 below is the direct guard
against a repeat.

What the 7 assertions cover

  1. Every workflow parses as YAML and declares at least one job.
  2. Every npm run <name> in a workflow names a defined package.json script.
  3. Every npm run <name> inside a package.json script names a defined
    script — covers the check / fix / seq / precheck:links chains.
    Literal globs (_list:check:*) and the bare npm run --loglevel=warn
    script-listing idiom are excluded, with a comment saying why.
  4. Every node scripts/... target in package.json exists on disk.
  5. Every -c / --config dotfile passed to a linter exists.
  6. Every scripts/*.mjs entry point is reachable from some package.json
    script, so no entry point is orphaned.
  7. Any workflow running a validate:* script also runs test:unit.

Verification

Run against a local clone of cncf/endusers at 00b44df after npm ci,
node v22.

  • npm run test:unit: 62 pass, 0 fail (55 before; the 7 new tests are all
    that changed).

  • npx prettier --check tests/workflow-scripts.test.mjs: clean.

  • Mutation-checked — an assertion that cannot fail is not a test. Four
    deliberate breakages were introduced and reverted:

    mutation result
    validate:awardsnode scripts/validate-awardz.mjs assertions 4 and 6 fail
    check:spelling-c .cspell-missing.yml assertion 5 fails
    delete the validate:metrics script assertion 2 fails (workflows still invoke it)
    drop test:unit from deploy-gh-pages.yml assertion 7 fails, naming the file

    All 7 passed again after revert, and git status confirmed the working tree
    clean apart from the new test file.

Known gaps deliberately not asserted

Two real problems in this repo would make a stricter version of this file fail
on main, so they are not silently weakened into passing assertions — they
are filed instead:

Coverage evidence

  • Unit: npm run test:unit and node --test --experimental-test-coverage,
    local clone at 00b44df, 2026-09-18. package.json and workflow YAML are
    data, not executable modules, so they appear in no coverage report at any
    percentage — this gap is invisible to --experimental-test-coverage rather
    than shown as a low number, which is part of why it persisted.
  • End-to-end: unavailable — this repository has no end-to-end or browser
    suite and publishes no coverage artifact from any suite (tracked in [quality] CI publishes no coverage evidence, so coverage findings cannot be verified #186).
    No claim is made that these paths lack end-to-end coverage.

Disjointness

One new file, touched by no other open PR. Distinct from the data-file
contract tests (#235, #239, #241, #253, #257, #260, #270), the navigation
contract (#275), the per-script unit tests (#185, #197, #208, #214, #216,
#221, #231, #263), and the coverage reporter (#225) and JSX import path (#229),
both of which add lines to package.json — this PR does not modify
package.json at all, so it cannot conflict with either.

Related Issue

Closes #277 — merging this leaves nothing for that issue to track: it asks for
exactly this file, and its three completion criteria (file exists and is picked
up by npm run test:unit; each assertion mutation-checked; no production file,
workflow or dependency modified) are all met above. The ci.yml problem is
tracked separately in #276 and is not part of this issue.


Filed by quality agent (hold-gated mode). Human review required.

— hive: agent=quality backend=copilot model=claude-opus-5

Adds tests/workflow-scripts.test.mjs, asserting that every npm run target in
a workflow or in another package.json script is defined, that every
node scripts/... target and every linter config dotfile exists, that no
scripts/*.mjs entry point is orphaned, and that a workflow running a
validator also runs the unit suite.

These references are plain strings naming files in other places, and no
existing test reads package.json or any workflow, so a rename or deletion
surfaces only when a contributor or CI runs the command.

Closes #277

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: kubestellar-hive[bot] <kubestellar-hive@hive.kubestellar.io>
@kubestellar-hive

Copy link
Copy Markdown
Contributor Author

Important

Held for human review by the hive's ACMM level gate.

This PR was opened by the "quality" agent while Hive policy required a human checkpoint for that agent. Non-outreach agents are held at ACMM L3–L5; the outreach agent is always held because it publishes project-facing communication.

Hive will automatically remove the hold label once current policy no longer requires a level hold for "quality". If this is an outreach PR, a human must review it and remove the label.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[quality] package.json <-> scripts/ <-> workflows wiring contract is untested (tests/workflow-scripts.test.mjs)

0 participants