From 2b002ea095bae4db219f858ca4e5d76855e35413 Mon Sep 17 00:00:00 2001 From: "Jonathan D.A. Jewell" <6759885+hyperpolymath@users.noreply.github.com> Date: Tue, 3 Nov 2026 00:59:40 +0000 Subject: [PATCH 1/7] docs(ssg): distinguish history-preservation staging from validated release --- README.adoc | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/README.adoc b/README.adoc index f3562ee..d88be43 100644 --- a/README.adoc +++ b/README.adoc @@ -6,10 +6,15 @@ image:https://img.shields.io/badge/Idris-Inside-5E5086[Idris Inside,link="https: // SPDX-FileCopyrightText: 2026 Jonathan D.A. Jewell = Poly SSG Comparison Site + +[CAUTION] +==== +*Preservation staging, not an accepted migration.* This repository is a history-preserving extraction of `hyperpolymath/polystack/poly-ssg`. It is currently private; inherited GitHub Actions are **disabled pending a reviewed, real CI replacement**. No BoJ backend, build parity, deployment, security or proof claims are independently validated by the extraction. See link:docs/PRESERVATION.adoc[provenance and qualification ledger] and link:https://github.com/hyperpolymath/poly-ssg/issues/2[qualification issue #2]. Polystack remains the fallback while link:https://github.com/hyperpolymath/polystack/issues/30[Polystack #30] is open. +==== :toc: auto image:https://img.shields.io/badge/RSR-compliant-blue[RSR Compliant] -image:https://img.shields.io/badge/status-active-green[Status: Active] +image:https://img.shields.io/badge/status-unqualified-lightgrey[Status: Unqualified preservation staging] Interactive comparison of SSG engines (Casket, Hackenbush, Bunsenite, Cobalt) built with ReScript, Deno, and formally verified components from proven. @@ -78,7 +83,7 @@ echo "axiom.metadatasticaian.art" > ../../_site/CNAME * *Feature Comparison* - Understand engine capabilities * *Safe String Operations* - Formally verified config building -== Project Status (2026-01-30) +== Historical Project Status (2026-01-30 — not a current verification) * ✅ ReScript compilation with Deno * ✅ proven integration for safe strings From 7ab68dddd6bfd01f3f4d62fff9ed6804835ea2e0 Mon Sep 17 00:00:00 2001 From: "Jonathan D.A. Jewell" <6759885+hyperpolymath@users.noreply.github.com> Date: Tue, 3 Nov 2026 00:59:42 +0000 Subject: [PATCH 2/7] docs(ssg): record exact subtree history and qualification gate --- docs/PRESERVATION.adoc | 39 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 39 insertions(+) create mode 100644 docs/PRESERVATION.adoc diff --git a/docs/PRESERVATION.adoc b/docs/PRESERVATION.adoc new file mode 100644 index 0000000..b02ba47 --- /dev/null +++ b/docs/PRESERVATION.adoc @@ -0,0 +1,39 @@ +// SPDX-License-Identifier: CC-BY-SA-4.0 += Poly SSG — provenance, qualification and release gate +:toc: left + +*Current disposition:* PRIVATE SOURCE-PRESERVATION STAGING. Do not use as a production dependency solely because this repository exists. + +== Exact provenance + +* Original repository: https://github.com/hyperpolymath/polystack[hyperpolymath/polystack] +* Original source commit: `e1af43bb102106b08cb694402bd1c04b96f1677c` (main, inspected 2026-11-03) +* Original prefix: `poly-ssg/` +* Git history extraction: `git subtree split --prefix=poly-ssg --branch preservation-poly-ssg` in Debian WSL (Git for Windows' shell failed, so WSL was used) +* Extracted initial commit: `9302dfcc3aedb7c52edf2867a2fcbf2c735878b4` +* Original retained in Polystack unchanged. Source root/subtree correspondence should be verified independently via Git object tree hashes. + +The extracted history is not a new implementation. The original prefix path has become this repository root. This preserves previous authors/commit history for the subtree; it does not establish its executable behaviour. + +== Important environmental warnings + +* The Windows source checkout flagged case-sensitive path collisions in `poly-ssg/satellites/rats-ssg/{Justfile,justfile}`, which cannot both appear correctly on a case-insensitive checkout. Compare Git trees, not just Windows worktree files. +* `.gitmodules` refers to multiple individual historical SSG repositories; many are now consolidated into https://github.com/hyperpolymath/ssg-collection[ssg-collection]. The references and whether they are actual gitlinks or merely stale metadata need review before automated checkouts. +* `sites-build/` includes historical DNS, security-policy and generator outputs, not an approved current deployment configuration. Do not deploy or distribute generated DNS zones unreviewed. +* The historical README still contains claims originating in early 2026. Those claims are not current assurance. +* GitHub Actions were explicitly disabled **in this newly created staging repository** before the extraction was pushed to prevent an unreviewed legacy workflow storm. They MUST be replaced/validated and re-enabled before any qualification, release or default branch merge is called complete. Do not use zero jobs as a successful gate. + +== Successor relationships are still open + +* https://github.com/hyperpolymath/ssg-collection[ssg-collection] contains overlapping experimental engines, variants and scaffolds; the file-level differences are catalogued at https://github.com/hyperpolymath/polystack/pull/31[Polystack PR #31]. +* https://github.com/hyperpolymath/boj-server-cartridges/tree/main/cartridges/domains/web/ssg-mcp[BoJ ssg-mcp] currently declares five operations and delegates to a separate HTTP backend (default `127.0.0.1:7742`). Backend presence and runtime equivalence remain unverified. +* https://github.com/hyperpolymath/boj-server-cartridges/tree/main/cartridges/domains/web/ssg-lsp[BoJ ssg-lsp] is described as scaffolded; it preserves the original Elixir LSP source as source material, not a finished port. +* https://github.com/stuff-of-awe/awesome-ssgs[stuff-of-awe/awesome-ssgs] is an independent **curation collection**, not an SSG framework or backend. + +== Release acceptance + +https://github.com/hyperpolymath/poly-ssg/issues/2[Qualification issue #2] is the authoritative evidence checklist. The minimum gate is correct source correspondence; a tracked ownership/licence and dependency inventory; clean, non-vacuous CI on the latest SHA; one fully demonstrated site build with negative controls; an accepted consumer adapter; and all applicable app reviews and conversations resolved. + +Only after those tests and reviews may the owner decide on publication, supported status or closing https://github.com/hyperpolymath/polystack/issues/30[Polystack's archive blocker]. + +*No archival, deployment, or release approval is implied by this document.* From f82227bcbd13f27eab207ff791aa79639decefd4 Mon Sep 17 00:00:00 2001 From: "Jonathan D.A. Jewell" <6759885+hyperpolymath@users.noreply.github.com> Date: Wed, 4 Nov 2026 17:28:49 +0000 Subject: [PATCH 3/7] fix(ci): verify preservation and repair scanner execution Bind preservation checks to the retained source tree and current head. Add independent provenance and output-validation controls, repair Semgrep upload permissions and Scorecard provisioning, and record finite retention destinations without claiming runtime qualification. --- .github/workflows/ci.yml | 299 +++--------------------- .github/workflows/scorecard.yml | 11 +- .github/workflows/semgrep.yml | 7 +- README.adoc | 15 +- docs/PRESERVATION.adoc | 12 +- docs/RETENTION-PLAN.adoc | 344 ++++++++++++++++++++++++++++ docs/legacy-workflows/ci.yml.txt | 276 ++++++++++++++++++++++ scripts/validate-output.sh | 72 +++--- scripts/verify-preservation.sh | 31 +++ tests/output-validation-controls.sh | 44 ++++ tests/preservation-controls.sh | 41 ++++ 11 files changed, 841 insertions(+), 311 deletions(-) create mode 100644 docs/RETENTION-PLAN.adoc create mode 100644 docs/legacy-workflows/ci.yml.txt create mode 100755 scripts/verify-preservation.sh create mode 100755 tests/output-validation-controls.sh create mode 100755 tests/preservation-controls.sh diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 1083502..48e76d2 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -1,275 +1,38 @@ # SPDX-License-Identifier: MPL-2.0-or-later -# Continuous Integration - Strict Mode -# All warnings treated as errors, all tests must pass -name: CI - +# Preservation gates for the extracted, unqualified source archive. +name: Preservation CI on: - push: - branches: [main, develop] pull_request: + push: branches: [main] - -permissions: read-all - -env: - OCAML_VERSION: "4.14.x" - # Treat warnings as errors - OCAML_COLOR: always - + workflow_dispatch: +permissions: + contents: read +concurrency: + group: preservation-${{ github.event_name }}-${{ github.ref }} + cancel-in-progress: true jobs: - # OCaml/ReScript-WASM build and test - STRICT MODE - rescript-wasm: - name: ReScript WASM-GC Backend - runs-on: ${{ matrix.os }} - permissions: - contents: read - - strategy: - fail-fast: true # Fail immediately on first error - matrix: - os: [ubuntu-latest, macos-latest] - ocaml-version: ["4.14.x", "5.1.x"] - + preservation: + name: Source history and preservation controls + runs-on: ubuntu-24.04 + timeout-minutes: 10 steps: - - name: Checkout - uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1 - - - name: Set up OCaml ${{ matrix.ocaml-version }} - uses: ocaml/setup-ocaml@7c185f622998a008ba1b296d2d7c64e235629c23 # v3.6.1 - with: - ocaml-compiler: ${{ matrix.ocaml-version }} - dune-cache: true - - - name: Install dependencies - run: opam install . --deps-only --with-test -y - working-directory: engines/rescript-wasm - - - name: Build with strict warnings - run: | - opam exec -- dune build --force - working-directory: engines/rescript-wasm - - - name: Run all tests (must pass) - run: | - opam exec -- dune test - echo "All tests passed successfully" - working-directory: engines/rescript-wasm - - - name: Generate sample WAT - run: | - opam exec -- dune exec rescript_wasm -- add > add.wat - opam exec -- dune exec rescript_wasm -- max > max.wat - opam exec -- dune exec rescript_wasm -- square > square.wat - opam exec -- dune exec rescript_wasm -- combined > combined.wat - echo "=== add.wat ===" - cat add.wat - working-directory: engines/rescript-wasm - - - name: Install wasm-tools - if: runner.os == 'Linux' - run: | - curl -L https://github.com/bytecodealliance/wasm-tools/releases/download/v1.219.1/wasm-tools-1.219.1-x86_64-linux.tar.gz | tar xz - sudo mv wasm-tools-1.219.1-x86_64-linux/wasm-tools /usr/local/bin/ - - - name: Validate WAT output (strict) - if: runner.os == 'Linux' - run: | - # Validate all generated WAT files - for wat in add.wat max.wat square.wat combined.wat; do - echo "Validating $wat..." - wasm-tools validate --features gc "$wat" || echo "Note: WAT validation requires WASM-GC support" - done - working-directory: engines/rescript-wasm - - - name: Upload WAT artifacts - uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v4 + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 with: - name: wat-output-${{ matrix.os }}-${{ matrix.ocaml-version }} - path: engines/rescript-wasm/*.wat - - # Forth Estate build - forth-estate: - name: Forth Estate SSG - runs-on: ubuntu-latest - permissions: - contents: read - - steps: - - name: Checkout - uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1 - - - name: Install Gforth - run: sudo apt-get update && sudo apt-get install -y gforth - - - name: Check Forth syntax - run: | - if [ -f forth-estate.fs ]; then - gforth forth-estate.fs -e bye - fi - working-directory: engines/forth-estate - - # Ddraig (Idris 2) build and test - ddraig-ssg: - name: Ddraig SSG - runs-on: ubuntu-latest - container: ghcr.io/idris-community/idris2:latest - permissions: - contents: read - - steps: - - name: Checkout - uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1 - - - name: Build Ddraig executable - run: | - cd engines/ddraig-ssg - idris2 Ddraig.idr -o ddraig - - - name: Run Ddraig parser tests - run: | - cd engines/ddraig-ssg - ./build/exec/ddraig test-markdown - ./build/exec/ddraig test-frontmatter - ./build/exec/ddraig test-full - - - name: Build site with Ddraig - run: | - cd engines/ddraig-ssg - ./build/exec/ddraig build ../../content ../../_site - - # Strict lint and format checks - ALL MUST PASS - lint: - name: Lint & Format (Strict) - runs-on: ubuntu-latest - permissions: - contents: read - - steps: - - name: Checkout - uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1 - - - name: Set up OCaml - uses: ocaml/setup-ocaml@7c185f622998a008ba1b296d2d7c64e235629c23 # v3.6.1 - with: - ocaml-compiler: "4.14.x" - - - name: Check for unsafe patterns (MUST PASS) - run: | - echo "=== Checking for unsafe patterns ===" - ERRORS=0 - - # Check for Obj.magic (type safety violation) - if grep -rn "Obj\.magic" engines/rescript-wasm/lib/; then - echo "ERROR: Obj.magic found - this violates type safety" - ERRORS=$((ERRORS + 1)) - fi - - # Check for Marshal (code execution risk) - if grep -rn "Marshal\." engines/rescript-wasm/lib/; then - echo "ERROR: Marshal found - this is a code execution risk" - ERRORS=$((ERRORS + 1)) - fi - - # Check for shell execution - if grep -rn "Unix\.system\|Unix\.open_process\|Sys\.command" engines/rescript-wasm/lib/; then - echo "ERROR: Shell execution found - security risk" - ERRORS=$((ERRORS + 1)) - fi - - if [ $ERRORS -gt 0 ]; then - echo "Found $ERRORS security issues" - exit 1 - fi - echo "No unsafe patterns found" - - - name: Check for catch-all patterns (MUST PASS) - run: | - echo "=== Checking for dangerous catch-all patterns ===" - # Look for standalone _ patterns in match statements (potential bug hiding) - # This is a heuristic check - not perfect but catches common issues - if grep -En "^\s*\|\s*_\s*->" engines/rescript-wasm/lib/*.ml | grep -v "failwith\|error\|assert\|raise"; then - echo "WARNING: Found catch-all patterns without explicit error handling" - echo "Consider using explicit pattern matching to avoid hiding bugs" - fi - echo "Pattern check complete" - - - name: Report TODOs/FIXMEs - run: | - echo "=== TODOs and FIXMEs (informational) ===" - grep -rn "TODO\|FIXME\|XXX\|HACK" engines/rescript-wasm/lib/ || echo "None found" - - - name: Check line lengths (soft limit 100) - run: | - echo "=== Lines over 100 characters (warnings) ===" - LONG_LINES=$(find engines/rescript-wasm/lib -name "*.ml" -exec awk 'length > 100 {count++} END {print count+0}' {} \; | awk '{s+=$1} END {print s}') - echo "Found $LONG_LINES lines over 100 characters" - if [ "$LONG_LINES" -gt 50 ]; then - echo "WARNING: Many long lines found, consider refactoring" - fi - - # Security scanning job - security: - name: Security Checks - runs-on: ubuntu-latest - permissions: - security-events: write - contents: read - - steps: - - name: Checkout - uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1 - - - name: Set up OCaml - uses: ocaml/setup-ocaml@7c185f622998a008ba1b296d2d7c64e235629c23 # v3.6.1 - with: - ocaml-compiler: "4.14.x" - - - name: Check dependencies for known vulnerabilities - run: | - echo "=== Checking for vulnerable dependencies ===" - opam list --installed --safe 2>/dev/null || true - - - name: Scan for hardcoded secrets - run: | - echo "=== Scanning for hardcoded secrets ===" - # Check for common secret patterns - if grep -rEn "(password|secret|api_key|apikey|auth_token|private_key)\s*=\s*['\"][^'\"]+['\"]" engines/rescript-wasm/; then - echo "ERROR: Potential hardcoded secrets found" - exit 1 - fi - echo "No hardcoded secrets found" - - - name: Check .gitignore coverage - run: | - echo "=== Checking .gitignore coverage ===" - # Verify important patterns are ignored - if [ -f engines/rescript-wasm/.gitignore ]; then - for pattern in "_build/" "*.cmo" "*.cmi" ".env"; do - if grep -q "$pattern" engines/rescript-wasm/.gitignore; then - echo "OK: $pattern is ignored" - else - echo "WARNING: $pattern should be in .gitignore" - fi - done - fi - - # Final validation job - runs after all others - validate: - name: Final Validation - runs-on: ubuntu-latest - permissions: - contents: read - needs: [rescript-wasm, forth-estate, ddraig-ssg, lint, security] - - steps: - - name: All checks passed - run: | - echo "=========================================" - echo "All CI checks passed successfully!" - echo "=========================================" - echo "- ReScript-WASM: Build & Tests PASSED" - echo "- Forth Estate: Syntax PASSED" - echo "- Ddraig SSG: Build & Tests PASSED" - echo "- Lint & Format: PASSED" - echo "- Security Checks: PASSED" - echo "=========================================" + ref: ${{ github.event.pull_request.head.sha || github.sha }} + fetch-depth: 0 + persist-credentials: false + - name: Verify preserved source identity and ancestry + run: bash scripts/verify-preservation.sh + - name: Reject corrupt provenance and unrelated history + run: bash tests/preservation-controls.sh + - name: Validate output checker with independent positive and negative inputs + run: bash tests/output-validation-controls.sh + - name: Record qualification scope + run: | + printf '%s\n' \ + '## Preservation receipt' \ + "Subject: $(git rev-parse HEAD)" \ + 'Source history and output-checker regressions were exercised.' \ + 'Generator builds, BoJ parity, deployment and overall security qualification remain open in issue #2.' \ + >> "$GITHUB_STEP_SUMMARY" diff --git a/.github/workflows/scorecard.yml b/.github/workflows/scorecard.yml index 6a82602..58caf60 100644 --- a/.github/workflows/scorecard.yml +++ b/.github/workflows/scorecard.yml @@ -3,6 +3,8 @@ name: OSSF Scorecard on: push: branches: [main, master] + pull_request: + branches: [main] schedule: - cron: '0 4 * * *' workflow_dispatch: @@ -11,17 +13,19 @@ permissions: read-all jobs: analysis: - runs-on: ubuntu-latest + runs-on: ubuntu-24.04 + timeout-minutes: 15 permissions: security-events: write - id-token: write + actions: read + contents: read steps: - uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1 with: persist-credentials: false - name: Run Scorecard - uses: ossf/scorecard-action@62b2cac7ed8198b15735ed49ab1e5cf35480ba46 # v2.3.1 + uses: ossf/scorecard-action@fa7117db83c52d787ae5a89f350999701505d78e # v2.5.0 with: results_file: results.sarif results_format: sarif @@ -30,3 +34,4 @@ jobs: uses: github/codeql-action/upload-sarif@48ab28a6f5dbc2a99bf1e0131198dd8f1df78169 # v3.31.8 with: sarif_file: results.sarif + diff --git a/.github/workflows/semgrep.yml b/.github/workflows/semgrep.yml index 9ae77bd..89e1f38 100644 --- a/.github/workflows/semgrep.yml +++ b/.github/workflows/semgrep.yml @@ -14,12 +14,14 @@ permissions: read-all jobs: semgrep: - runs-on: ubuntu-latest + runs-on: ubuntu-24.04 + timeout-minutes: 15 permissions: security-events: write + actions: read contents: read container: - image: semgrep/semgrep + image: semgrep/semgrep@sha256:451b3d875c00dac7ed5db5293e584a07de11449a409c10b3c2d2652f429630b6 steps: - uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1 @@ -33,3 +35,4 @@ jobs: with: sarif_file: semgrep.sarif if: always() + diff --git a/README.adoc b/README.adoc index d88be43..a897c41 100644 --- a/README.adoc +++ b/README.adoc @@ -9,7 +9,7 @@ image:https://img.shields.io/badge/Idris-Inside-5E5086[Idris Inside,link="https: [CAUTION] ==== -*Preservation staging, not an accepted migration.* This repository is a history-preserving extraction of `hyperpolymath/polystack/poly-ssg`. It is currently private; inherited GitHub Actions are **disabled pending a reviewed, real CI replacement**. No BoJ backend, build parity, deployment, security or proof claims are independently validated by the extraction. See link:docs/PRESERVATION.adoc[provenance and qualification ledger] and link:https://github.com/hyperpolymath/poly-ssg/issues/2[qualification issue #2]. Polystack remains the fallback while link:https://github.com/hyperpolymath/polystack/issues/30[Polystack #30] is open. +*Preservation staging, not an accepted migration.* This repository is a history-preserving extraction of `hyperpolymath/polystack/poly-ssg`. It is currently private; the preservation workflow checks retained Git history and output-checker regressions; runtime qualification remains open. No BoJ backend, build parity, deployment, security or proof claims are independently validated by the extraction. See link:docs/PRESERVATION.adoc[provenance and qualification ledger] and link:https://github.com/hyperpolymath/poly-ssg/issues/2[qualification issue #2]. Polystack remains the fallback while link:https://github.com/hyperpolymath/polystack/issues/30[Polystack #30] is open. ==== :toc: auto @@ -18,14 +18,14 @@ image:https://img.shields.io/badge/status-unqualified-lightgrey[Status: Unqualif Interactive comparison of SSG engines (Casket, Hackenbush, Bunsenite, Cobalt) built with ReScript, Deno, and formally verified components from proven. -== Architecture +== Historical architecture claims * *ReScript* - Type-safe frontend compilation * *Deno* - Modern runtime (no Node.js per RSR) * *proven* - Formally verified string operations * *TEA* - The Elm Architecture for state management -=== Formally Verified Components +=== Historical verification claims This site uses *ProvenSafeString* from the proven library for safe config generation, ensuring: @@ -33,7 +33,7 @@ This site uses *ProvenSafeString* from the proven library for safe config genera * Memory-safe concatenation * Proven correctness of output format -== Development +== Historical development instructions === Build @@ -58,7 +58,7 @@ deno run -A jsr:@std/http/file-server . python3 -m http.server 8000 ---- -=== GitHub Pages Deployment (Ddraig) +=== Historical GitHub Pages deployment instructions (unqualified) Primary Pages deploy now uses `ddraig-ssg` (Idris 2) via: @@ -93,6 +93,10 @@ echo "axiom.metadatasticaian.art" > ../../_site/CNAME * ⏳ Visual comparison table * ⏳ Interactive feature filtering +== Retention and retirement + +See link:docs/RETENTION-PLAN.adoc[the retention and retirement plan]. Curation belongs in `stuff-of-awe/awesome-ssgs`; implementation history stays in source repositories. The source snapshot has been checked by matching Git tree IDs. A passing preservation workflow does not close the runtime or security findings in qualification issue #2. + == Related Projects * link:https://github.com/hyperpolymath/casket-ssg[Casket SSG] - Haskell implementation @@ -106,3 +110,4 @@ echo "axiom.metadatasticaian.art" > ../../_site/CNAME PMPL-1.0-or-later (Palimpsest-MPL) Philosophy: _One contract. Many paradigms. Every paradigm teaches._ + diff --git a/docs/PRESERVATION.adoc b/docs/PRESERVATION.adoc index b02ba47..35d8b37 100644 --- a/docs/PRESERVATION.adoc +++ b/docs/PRESERVATION.adoc @@ -11,7 +11,7 @@ * Original prefix: `poly-ssg/` * Git history extraction: `git subtree split --prefix=poly-ssg --branch preservation-poly-ssg` in Debian WSL (Git for Windows' shell failed, so WSL was used) * Extracted initial commit: `9302dfcc3aedb7c52edf2867a2fcbf2c735878b4` -* Original retained in Polystack unchanged. Source root/subtree correspondence should be verified independently via Git object tree hashes. +* Original retained in Polystack unchanged. Independent Git API reads on 2026-11-04 confirmed that the original `poly-ssg/` subtree and extracted initial commit both name tree `6a8aa93222c8261c3e81b37acd7d1bd59f42bdc4`. The active preservation job also verifies the import remains an ancestor and its objects remain connected. The extracted history is not a new implementation. The original prefix path has become this repository root. This preserves previous authors/commit history for the subtree; it does not establish its executable behaviour. @@ -21,7 +21,7 @@ The extracted history is not a new implementation. The original prefix path has * `.gitmodules` refers to multiple individual historical SSG repositories; many are now consolidated into https://github.com/hyperpolymath/ssg-collection[ssg-collection]. The references and whether they are actual gitlinks or merely stale metadata need review before automated checkouts. * `sites-build/` includes historical DNS, security-policy and generator outputs, not an approved current deployment configuration. Do not deploy or distribute generated DNS zones unreviewed. * The historical README still contains claims originating in early 2026. Those claims are not current assurance. -* GitHub Actions were explicitly disabled **in this newly created staging repository** before the extraction was pushed to prevent an unreviewed legacy workflow storm. They MUST be replaced/validated and re-enabled before any qualification, release or default branch merge is called complete. Do not use zero jobs as a successful gate. +* Actions were initially disabled before import. Runs inspected on 2026-11-04 establish that inherited scheduled workflows are executing again. The current preservation CI exercises Git provenance and positive/negative output-checker controls; it does not establish generator or consumer qualification. The former root build matrix is retained at `docs/legacy-workflows/ci.yml.txt` because it referenced absent Ddraig/Forth implementations and could report tests without running them. Do not use zero jobs as a successful gate. == Successor relationships are still open @@ -37,3 +37,11 @@ https://github.com/hyperpolymath/poly-ssg/issues/2[Qualification issue #2] is th Only after those tests and reviews may the owner decide on publication, supported status or closing https://github.com/hyperpolymath/polystack/issues/30[Polystack's archive blocker]. *No archival, deployment, or release approval is implied by this document.* + + +== Workflow recovery receipt (2026-11-04) + +* Semgrep run 41777623670 scanned 2,351 targets with 503 rules and reported 313 findings. Its final SARIF upload failed because the job omitted `actions: read`; adding that read permission repairs report publication. The scan remains a findings report, not a clean-security attestation. Its image is pinned to the digest observed in that executed run. Findings remain to be triaged or covered by an explicit historical-source disposition. +* Scorecard run 41771377426 failed before scanning because its old GCR image could not be pulled. The replacement is upstream v2.5.0 at `fa7117db83c52d787ae5a89f350999701505d78e`, which uses GHCR; its tag was verified by GitHub. Hosted execution must still establish this repair works. +* The output checker previously treated every nonempty file as containing NUL, exited on the first zero-valued arithmetic increment, lost directory counters in a subshell, and could lose early grep matches under pipefail. Positive and negative controls now cover those failures, including large input, empty populations and malformed encoding. The checker remains a heuristic for listed patterns. +* Polystack PR #31 has merged at `218ccc9b9e458c36a8ff783fd762941abd4a4439`; its structural comparison is retained evidence, not runtime parity. diff --git a/docs/RETENTION-PLAN.adoc b/docs/RETENTION-PLAN.adoc new file mode 100644 index 0000000..1bf9dc4 --- /dev/null +++ b/docs/RETENTION-PLAN.adoc @@ -0,0 +1,344 @@ +// SPDX-License-Identifier: CC-BY-SA-4.0 += Poly SSG: preserve the useful work and retire the original cleanly +:revdate: 2026-11-04 +:toc: +:toclevels: 2 + +*Status: proposed component dispositions, prepared for Poly SSG PR #3.* +This plan records a finite route to retirement. The independent Git tree +comparison is recorded in `PRESERVATION.adoc`: the original `poly-ssg/` subtree +and the extracted root both identify tree +`6a8aa93222c8261c3e81b37acd7d1bd59f42bdc4`. That establishes source-tree +correspondence, including file modes; it does not report an archive, release, +working replacement, or completed runtime qualification. The preservation gate +checks the imported commit's tree identity, ancestry and object connectivity. + +== Intended outcome + +Retain the original Git history and useful experiments in an identified source +snapshot; publish useful, original educational and discovery material in +`stuff-of-awe/awesome-ssgs`; assign any selected runtime work to a maintained +successor; and explicitly discontinue the remaining experiments. Close the +retirement work when those decisions and their evidence are recorded. + +There is no requirement to finish every proposed generator, implement every +historical feature, or make every archived experiment build. A functioning +successor needs execution evidence. A discontinued experiment needs preserved +source, an honest status, and removal of any claim that a live consumer can +depend on it. These are different outcomes and should be recorded separately. + +The proposed end state uses GitHub archival, with history and a clear successor +notice retained. Repository deletion is outside this plan. Retirement of Poly +SSG and archival of its parent Polystack are separate repository decisions; +the SSG work alone cannot establish that every other Polystack domain is ready. + +== Inspection baseline and source references + +The following refs were read on 2026-11-04. Freeze the final refs again when +recording completion; these values are evidence baselines, not moving-branch +guarantees. + +[cols="2,3,3",options="header"] +|=== +|Repository or item |Observed ref/state |Meaning +|Original Polystack source +|`e1af43bb102106b08cb694402bd1c04b96f1677c`, prefix `poly-ssg/` +|Frozen extraction source recorded in the preservation documents. +|Extracted `hyperpolymath/poly-ssg` main +|`9302dfcc3aedb7c52edf2867a2fcbf2c735878b4` +|Existing private source-preservation snapshot. Its root tree matches the +frozen original prefix tree; see `PRESERVATION.adoc`. Runtime and full historical +authorship qualification remain separate from that comparison. +|Poly SSG PR #3, initial inspection +|`7ab68dddd6bfd01f3f4d62fff9ed6804835ea2e0` +|Documentation branch before the present workflow repairs. Use its eventual +final head for CI and review evidence. +|Polystack current main / PR #31 +|`218ccc9b9e458c36a8ff783fd762941abd4a4439`; PR #31 merged +|The initial source-hash preservation ledger is already on main. It is no +longer a draft PR, and merging it did not complete migration. +|`hyperpolymath/ssg-collection` main +|`e5b7e7fd9af511a1c339cfbd2824e0cccd03fada` +|Existing experiments collection used by the structural comparison. +|`hyperpolymath/boj-server-cartridges` main +|`2fd1fa3b18fbe613f21397506acf09f7d07b072f` +|Consumer snapshot referenced by the public migration tracker. +|`stuff-of-awe/awesome-ssgs` main +|`2e9ac1af7988638f58ff8c17c9d074a3931f81cb` +|Existing public starter collection, contribution rules, source map and local +curation checks. +|=== + +Primary records: + +* https://github.com/hyperpolymath/poly-ssg/pull/3[Poly SSG PR #3]. +* https://github.com/hyperpolymath/poly-ssg/issues/2[Poly SSG qualification #2]. +* https://github.com/hyperpolymath/polystack/issues/30[Polystack preservation blocker #30]. +* https://github.com/hyperpolymath/polystack/blob/218ccc9b9e458c36a8ff783fd762941abd4a4439/docs/SSG-PRESERVATION-LEDGER.adoc[Merged structural comparison ledger]. +* https://github.com/hyperpolymath/boj-server-cartridges/issues/165[Public BoJ migration tracker #165]. +* https://github.com/stuff-of-awe/awesome-ssgs/blob/2e9ac1af7988638f58ff8c17c9d074a3931f81cb/docs/SOURCE-MAP.md[Public curation source map]. + +== Three claims, with separate evidence + +[cols="2,4,3",options="header"] +|=== +|Claim |Minimum evidence |What the claim establishes +|Verified preserved snapshot +|Original commit and prefix; extracted commit; matching Git tree identity or +an explicit file/mode/blob comparison; extraction method; retained authorship +and history; recorded licences and any unresolved notices; a durable ref. +|The material can be recovered with its provenance. It establishes no runtime +or integration guarantee. +|Working successor +|Named repository and exact commit; supported operations; pinned toolchain; +executed input-to-output tests; negative cases; output inspection; applicable +reviews and real CI; consumer ownership and documented limitations. +|Only the tested functions and environments have a working replacement. +|Deliberately discontinued function or experiment +|Explicit disposition and rationale; preserved source ref; dependency and +consumer review; corrected availability/status documentation; any live caller +receives a clear unsupported/unavailable result rather than an implied service. +|The function has been retired intentionally. No implementation is promised +merely to close the preservation task. +|=== + +Every source area receives a preservation state and an operational disposition. +Public curation is an additional decision. For example, an experiment may be +fully preserved, deliberately discontinued, and still useful in a historical +learning section. + +== Source-area disposition map + +The preferred default is to retain the complete extracted snapshot, then move +only selected, understood material into maintained destinations. This allows +unique source to remain recoverable without manufacturing new maintenance +obligations or forcing incomplete material into an active codebase. + +[cols="3,4,4",options="header"] +|=== +|Source area |Proposed destination and disposition |Evidence needed before retirement +|`src/`, `lib/`, `public/`, `content/`, `config.ncl`, `deno.lock`, `Justfile`, `Mustfile` +|Preserve framework, comparison UI, generated artefacts and build configuration +in the extracted source snapshot. Continue the application only if an actual +consumer is identified; otherwise record it as historical and discontinued. +|Inventory authored versus generated files. Record the intended entry point +and manifest gaps. A successor claiming to run the application needs a real +build; historical retention does not require repairing the application. +|`engines/`, root `casket-ssg/`, embedded Casket and Ddraig satellite copies +|Preserve all copies first. Send selected fixes to the independently maintained +engine repositories after comparison. Keep unresolved or obsolete variants in +the historical snapshot, with links to current canonical engines. +|Compare paths, blob hashes, licences and unique source/tests. Check renames +and semantic changes before declaring a copy redundant. Test only changes +adopted by the maintained engine. +|`satellites/` +|Preserve all 50 directories. Reconcile selected useful changes with +`ssg-collection/{implementations,variants,stubs}`. Record unselected ideas as +historical experiments without a future implementation promise. +|Use the existing frozen comparison as a starting point. For every changed or +unique source file selected for migration, record the accepted destination +commit. Every unselected unique file remains covered by the preserved snapshot. +|`docs/PHILOSOPHY.adoc`, `docs/DESIGN-SPEC-TEMPLATE.adoc`, `docs/specs/` +|Retain originals as history. Publish newly written comparison and educational +notes in Awesome SSGs. Link public implementation evidence where available. +|Separate design intent from executed capabilities. Correct or omit unsupported +historical toolchain, proof, maturity and effort claims. Record attribution +without copying private documents into the public collection. +|`docs/ADDING-ENGINE.adoc`, `test-corpus/`, `scripts/` +|Keep the intended build contract and fixtures in source history. Adopt useful +fixtures into a selected runtime or engine successor; curated notes may explain +the comparison method using original prose and small synthetic examples. +|Check stale command/path references, input/output expectations, draft handling, +exit codes and negative cases. Only adopted fixtures need to become executable +successor tests; abandoned harnesses can remain explicitly historical. +|`mcp/` +|Preserve the original implementation. Track any maintained consumer contract +in BoJ #165. Select a narrow backend/engine set, or explicitly discontinue +unprovided operations and correct consumer availability claims. +|Inventory the historical operations against the actual BoJ adapter. A working +replacement requires backend ownership and real load/build/output tests. +Retirement requires an explicit supported/unavailable decision for each +advertised operation and no hidden dependency on the old source location. +|`lsp/` +|Preserve the Elixir implementation and compare it with BoJ's existing source +archive. Either maintain a narrow, tested LSP successor or mark the LSP effort +discontinued; scaffold preservation alone is not a functioning port. +|Check source correspondence and licences. If retained as a service, test +protocol framing and the selected handlers. If discontinued, remove claims of +working handlers and record that no replacement service is supplied. +|`sites-build/`, `poly-ssg.org.zone`, `_headers`, `.well-known/` +|Retain historical outputs and deployment context privately. Assign any live +site or domain to its actual maintained owner; retire unneeded deployment +paths. Do not copy environment-specific configuration into the curated list. +|Identify live consumers and domain references, intended operators, source +inputs and generated outputs. Review any configuration that is adopted for +current use. Record old reports as historical; no live deployment is needed +to demonstrate archival preservation. +|`.github/`, `hooks/`, `config/`, `contractiles/` +|Use the PR #3 repair to establish a small, meaningful preservation and +documentation gate. Keep old deployment workflows recoverable in history. +Each maintained successor owns its own applicable CI and contract checks. +|Execute positive and negative controls on the final PR SHA; inspect workflow +permissions and triggers; resolve substantive review findings. Documentation +checks must state their limited scope. Stop obsolete recurring automation as +part of actual retirement, after the final repair and receipt are recorded. +|`.gitmodules`, `.editorconfig`, `.gitattributes`, `.gitignore`, `.bot_directives/`, `.claude/` +|Preserve repository and historical automation context with the source. Adapt +only genuinely applicable metadata in maintained successors. +|The inspected extracted tree is untruncated and contains no `160000` gitlinks, +despite `.gitmodules` listing satellite entries. Record this distinction; +do not initiate recursive legacy checkouts merely because metadata remains. +Keep both case-colliding `rats-ssg/Justfile` and `justfile` in the Git snapshot. +|`license/`, `PALIMPSEST.adoc`, file-level notices and `README.adoc` +|Preserve original notices and provenance. Put an accurate historical/successor +notice at the retained repository entry point. New curation text follows +Awesome SSGs' CC BY-SA 4.0 rules; source code retains its own notices. +|Record the existing MPL/PMPL notice inconsistency rather than silently +relicensing material. Check the actual files selected for reuse. Unknown +redistribution terms keep a copy in private preservation pending resolution. +|=== + +The structural ledger records 46 directly matching satellite basenames and four +unmatched source basenames: `anvil`, `casket`, `ddraig`, `obli`. Its five +destination-only names are `befunge`, `cloudanvil`, `ephapax`, `shift`, +`wokelang`. In particular, decide the `anvil`/`cloudanvil` relationship through +comparison, not name similarity. The ledger's file-count categories are not +runtime qualification levels. + +The historical design index also needs editorial care: its summary advertises +14 engine proposals and five archive proposals, while the detailed tables +enumerate 12 and seven respectively. Retain that original document as history; +use a corrected, dated account for any new public explanation. + +== Public curation and private preservation + +`stuff-of-awe/awesome-ssgs` already supplies the intended public home. Its scope +is useful discovery, comparative evidence, unusual-language research and honest +maturity labels. It has a curation gate, not an SSG runtime qualification gate. + +Recommended bounded addition: + +. Add `docs/COMPARISON-NOTES.md`, with original guidance on comparing the same + publishing task across implementations and recording reproducible results. +. Link it from the existing "Compare generators usefully" section. +. Retain the existing starter status and distinction between inspected source, + executable examples and verified integrations. +. Update the public source map only when the underlying disposition changes. + Use public tracker #165 for public migration status; keep private source URLs, + source excerpts, environment details and private review discussion out of the + public article. +. Run the existing local catalogue validator and tests. A passing curation gate + establishes link/notice/status checks within its stated scope; it does not + verify the linked generators. + +The destination contribution policy and validator were inspected. Every public +Markdown document needs the CC-BY-SA-4.0 marker. Local links must resolve inside +the repository, and the validator rejects private Polystack/Poly SSG source +URLs. Third-party code must remain in its appropriately licensed source home. + +== Ordered completion work + +=== 1. Finish the bounded workflow and documentation repair + +Finish PR #3 on its existing branch. Establish the preservation/documentation +checks, execute them on the final head, address applicable review findings and +merge once the required checks genuinely run and pass. Record the resulting +merge SHA and check-run URLs. This is a completed documentation/CI repair even +if separate runtime questions remain open. + +=== 2. Attach a preservation receipt + +Record the original source commit, prefix tree, extracted commit/tree and the +commands/results of the independent correspondence check. Verify history and +authorship retention separately from current-tree equality. Preserve the exact +snapshot using a durable Git ref and, where needed, an independently recoverable +bundle or mirror; verify its object integrity. Do not create a public release +of private material as an incidental backup action. + +The receipt must distinguish verified results from documented claims. It must +cover executable file modes and the case-colliding paths, and identify any +external dependencies or unavailable historical objects. No success field is +pre-filled by this plan. + +=== 3. Record a finite component decision ledger + +For each source-area row, record: preservation receipt; proposed canonical +destination; maintained / discontinued / unresolved operational disposition; +licence state; current consumers; and evidence or remaining task. + +Prioritise live consumers and unique executable source/tests. Select only work +that is useful enough to maintain. Unselected material can be closed as a +preserved, discontinued experiment. Avoid creating a new issue for every +historical sketch. A selected unfinished runtime task belongs in its current +owner's repository and does not become a false completed-migration claim. + +=== 4. Preserve public educational value + +Land the focused Awesome SSGs article and index link with the destination's +existing checks. Add or update individual experimental entries only when their +canonical source, language, licence and status have been assessed. Keep the +historical source snapshot intact while this public curation improves. + +=== 5. Resolve consumer availability and archival scope + +For each claimed BoJ operation, record working replacement or deliberate +unavailability. A selected working build path needs actual output and negative +controls. Unsupported deployment, preview, clean or LSP functionality can be +explicitly retired without being implemented, provided consumers and +documentation are corrected and the retirement decision is recorded. + +Update qualification #2 and public tracker #165 to reflect the adopted scope. +If an original checklist item has become a deliberate discontinuation, record +that outcome and its reason rather than ticking it as successfully implemented. +Keep any actively promised runtime work open in its appropriate tracker. + +For Polystack #30, record source preservation and the exact SSG successor or +discontinuation mapping. Check the separate disposition of other Polystack +domains before treating that as authorisation or evidence to archive the +entire parent repository. + +=== 6. Retire and verify + +Replace active-status claims with an accurate retirement notice, canonical +successor links, unsupported-function notes and the retained snapshot ref. +Reconcile open issues and PRs: completed preservation, superseded work, +deliberately declined experiments, and transferred maintained tasks each need +their correct outcome. Keep useful review history accessible. + +Once the scoped retirement decision is recorded and live dependencies are +resolved, archive the identified original repository and stop obsolete +recurring automation. Verify that the repository is archived, the preserved +ref remains readable, and the public guide and live consumers point to the +intended destinations. Record these observations in a final closure receipt. + +== Completion criteria + +* The exact source snapshot and provenance are verified and recoverable. +* The repaired documentation/workflow PR has real passing checks at its final + head and an observed merge result. +* Each broad source area is assigned a destination or preserved historical + disposition; no unique source is discarded because a similarly named repo exists. +* Public curation retains useful explanations with accurate status and licence + notices, without exposing private source material. +* Each actual consumer has a working successor or a recorded, explicit + unavailability/discontinuation decision. +* Open runtime promises remain tracked honestly; discontinued experiments do + not require new implementations to permit preservation closure. +* The exact repository being retired, its archival result and stopped obsolete + automation are recorded separately from preservation and runtime testing. + +== Decisions still to record + +* Which repository is the final archival target: the extracted historical Poly + SSG repository, parent Polystack, or both after their respective dependencies + are resolved? +* Which concrete runtime functions, if any, are selected for continuing + maintenance, and which are deliberately discontinued? +* Who owns the retained source snapshot and the continuing curated collection? +* What is the resolution of file-level licence ambiguities for any source chosen + for redistribution or active reuse? + +These decisions refine the already requested preservation-and-retirement work. +They do not invalidate completed workflow repair, source verification or public +curation, and they do not authorise deletion or incidental publication of private +source. diff --git a/docs/legacy-workflows/ci.yml.txt b/docs/legacy-workflows/ci.yml.txt new file mode 100644 index 0000000..b6bd666 --- /dev/null +++ b/docs/legacy-workflows/ci.yml.txt @@ -0,0 +1,276 @@ +# SPDX-License-Identifier: MPL-2.0-or-later +# Continuous Integration - Strict Mode +# All warnings treated as errors, all tests must pass +name: CI + +on: + push: + branches: [main, develop] + pull_request: + branches: [main] + +permissions: read-all + +env: + OCAML_VERSION: "4.14.x" + # Treat warnings as errors + OCAML_COLOR: always + +jobs: + # OCaml/ReScript-WASM build and test - STRICT MODE + rescript-wasm: + name: ReScript WASM-GC Backend + runs-on: ${{ matrix.os }} + permissions: + contents: read + + strategy: + fail-fast: true # Fail immediately on first error + matrix: + os: [ubuntu-latest, macos-latest] + ocaml-version: ["4.14.x", "5.1.x"] + + steps: + - name: Checkout + uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1 + + - name: Set up OCaml ${{ matrix.ocaml-version }} + uses: ocaml/setup-ocaml@7c185f622998a008ba1b296d2d7c64e235629c23 # v3.6.1 + with: + ocaml-compiler: ${{ matrix.ocaml-version }} + dune-cache: true + + - name: Install dependencies + run: opam install . --deps-only --with-test -y + working-directory: engines/rescript-wasm + + - name: Build with strict warnings + run: | + opam exec -- dune build --force + working-directory: engines/rescript-wasm + + - name: Run all tests (must pass) + run: | + opam exec -- dune test + echo "All tests passed successfully" + working-directory: engines/rescript-wasm + + - name: Generate sample WAT + run: | + opam exec -- dune exec rescript_wasm -- add > add.wat + opam exec -- dune exec rescript_wasm -- max > max.wat + opam exec -- dune exec rescript_wasm -- square > square.wat + opam exec -- dune exec rescript_wasm -- combined > combined.wat + echo "=== add.wat ===" + cat add.wat + working-directory: engines/rescript-wasm + + - name: Install wasm-tools + if: runner.os == 'Linux' + run: | + curl -L https://github.com/bytecodealliance/wasm-tools/releases/download/v1.219.1/wasm-tools-1.219.1-x86_64-linux.tar.gz | tar xz + sudo mv wasm-tools-1.219.1-x86_64-linux/wasm-tools /usr/local/bin/ + + - name: Validate WAT output (strict) + if: runner.os == 'Linux' + run: | + # Validate all generated WAT files + for wat in add.wat max.wat square.wat combined.wat; do + echo "Validating $wat..." + wasm-tools validate --features gc "$wat" || echo "Note: WAT validation requires WASM-GC support" + done + working-directory: engines/rescript-wasm + + - name: Upload WAT artifacts + uses: actions/upload-artifact@b7c566a772e6b6bfb58ed0dc250532a479d7789f # v4 + with: + name: wat-output-${{ matrix.os }}-${{ matrix.ocaml-version }} + path: engines/rescript-wasm/*.wat + + # Forth Estate build + forth-estate: + name: Forth Estate SSG + runs-on: ubuntu-latest + permissions: + contents: read + + steps: + - name: Checkout + uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1 + + - name: Install Gforth + run: sudo apt-get update && sudo apt-get install -y gforth + + - name: Check Forth syntax + run: | + if [ -f forth-estate.fs ]; then + gforth forth-estate.fs -e bye + fi + working-directory: engines/forth-estate + + # Ddraig (Idris 2) build and test + ddraig-ssg: + name: Ddraig SSG + runs-on: ubuntu-latest + container: ghcr.io/idris-community/idris2:latest + permissions: + contents: read + + steps: + - name: Checkout + uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1 + + - name: Build Ddraig executable + run: | + cd engines/ddraig-ssg + idris2 Ddraig.idr -o ddraig + + - name: Run Ddraig parser tests + run: | + cd engines/ddraig-ssg + ./build/exec/ddraig test-markdown + ./build/exec/ddraig test-frontmatter + ./build/exec/ddraig test-full + + - name: Build site with Ddraig + run: | + cd engines/ddraig-ssg + ./build/exec/ddraig build ../../content ../../_site + + # Strict lint and format checks - ALL MUST PASS + lint: + name: Lint & Format (Strict) + runs-on: ubuntu-latest + permissions: + contents: read + + steps: + - name: Checkout + uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1 + + - name: Set up OCaml + uses: ocaml/setup-ocaml@7c185f622998a008ba1b296d2d7c64e235629c23 # v3.6.1 + with: + ocaml-compiler: "4.14.x" + + - name: Check for unsafe patterns (MUST PASS) + run: | + echo "=== Checking for unsafe patterns ===" + ERRORS=0 + + # Check for Obj.magic (type safety violation) + if grep -rn "Obj\.magic" engines/rescript-wasm/lib/; then + echo "ERROR: Obj.magic found - this violates type safety" + ERRORS=$((ERRORS + 1)) + fi + + # Check for Marshal (code execution risk) + if grep -rn "Marshal\." engines/rescript-wasm/lib/; then + echo "ERROR: Marshal found - this is a code execution risk" + ERRORS=$((ERRORS + 1)) + fi + + # Check for shell execution + if grep -rn "Unix\.system\|Unix\.open_process\|Sys\.command" engines/rescript-wasm/lib/; then + echo "ERROR: Shell execution found - security risk" + ERRORS=$((ERRORS + 1)) + fi + + if [ $ERRORS -gt 0 ]; then + echo "Found $ERRORS security issues" + exit 1 + fi + echo "No unsafe patterns found" + + - name: Check for catch-all patterns (MUST PASS) + run: | + echo "=== Checking for dangerous catch-all patterns ===" + # Look for standalone _ patterns in match statements (potential bug hiding) + # This is a heuristic check - not perfect but catches common issues + if grep -En "^\s*\|\s*_\s*->" engines/rescript-wasm/lib/*.ml | grep -v "failwith\|error\|assert\|raise"; then + echo "WARNING: Found catch-all patterns without explicit error handling" + echo "Consider using explicit pattern matching to avoid hiding bugs" + fi + echo "Pattern check complete" + + - name: Report TODOs/FIXMEs + run: | + echo "=== TODOs and FIXMEs (informational) ===" + grep -rn "TODO\|FIXME\|XXX\|HACK" engines/rescript-wasm/lib/ || echo "None found" + + - name: Check line lengths (soft limit 100) + run: | + echo "=== Lines over 100 characters (warnings) ===" + LONG_LINES=$(find engines/rescript-wasm/lib -name "*.ml" -exec awk 'length > 100 {count++} END {print count+0}' {} \; | awk '{s+=$1} END {print s}') + echo "Found $LONG_LINES lines over 100 characters" + if [ "$LONG_LINES" -gt 50 ]; then + echo "WARNING: Many long lines found, consider refactoring" + fi + + # Security scanning job + security: + name: Security Checks + runs-on: ubuntu-latest + permissions: + security-events: write + contents: read + + steps: + - name: Checkout + uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1 + + - name: Set up OCaml + uses: ocaml/setup-ocaml@7c185f622998a008ba1b296d2d7c64e235629c23 # v3.6.1 + with: + ocaml-compiler: "4.14.x" + + - name: Check dependencies for known vulnerabilities + run: | + echo "=== Checking for vulnerable dependencies ===" + opam list --installed --safe 2>/dev/null || true + + - name: Scan for hardcoded secrets + run: | + echo "=== Scanning for hardcoded secrets ===" + # Check for common secret patterns + if grep -rEn "(password|secret|api_key|apikey|auth_token|private_key)\s*=\s*['\"][^'\"]+['\"]" engines/rescript-wasm/; then + echo "ERROR: Potential hardcoded secrets found" + exit 1 + fi + echo "No hardcoded secrets found" + + - name: Check .gitignore coverage + run: | + echo "=== Checking .gitignore coverage ===" + # Verify important patterns are ignored + if [ -f engines/rescript-wasm/.gitignore ]; then + for pattern in "_build/" "*.cmo" "*.cmi" ".env"; do + if grep -q "$pattern" engines/rescript-wasm/.gitignore; then + echo "OK: $pattern is ignored" + else + echo "WARNING: $pattern should be in .gitignore" + fi + done + fi + + # Final validation job - runs after all others + validate: + name: Final Validation + runs-on: ubuntu-latest + permissions: + contents: read + needs: [rescript-wasm, forth-estate, ddraig-ssg, lint, security] + + steps: + - name: All checks passed + run: | + echo "=========================================" + echo "All CI checks passed successfully!" + echo "=========================================" + echo "- ReScript-WASM: Build & Tests PASSED" + echo "- Forth Estate: Syntax PASSED" + echo "- Ddraig SSG: Build & Tests PASSED" + echo "- Lint & Format: PASSED" + echo "- Security Checks: PASSED" + echo "=========================================" + diff --git a/scripts/validate-output.sh b/scripts/validate-output.sh index 475bdec..2b34fc7 100755 --- a/scripts/validate-output.sh +++ b/scripts/validate-output.sh @@ -1,6 +1,7 @@ #!/bin/bash +# SPDX-License-Identifier: MPL-2.0-or-later # Output Validation Script -# Validates generated HTML against security and correctness requirements +# Heuristic checks for selected unsafe output patterns; not a complete HTML/XSS validator. # # Usage: validate-output.sh # validate-output.sh @@ -15,60 +16,67 @@ NC='\033[0m' ERRORS=0 WARNINGS=0 -log_error() { echo -e "${RED}[ERROR]${NC} $*"; ((ERRORS++)); } -log_warn() { echo -e "${YELLOW}[WARN]${NC} $*"; ((WARNINGS++)); } +log_error() { printf '%b[ERROR]%b %s\n' "$RED" "$NC" "$*"; ERRORS=$((ERRORS + 1)); } +log_warn() { printf '%b[WARN]%b %s\n' "$YELLOW" "$NC" "$*"; WARNINGS=$((WARNINGS + 1)); } log_ok() { echo -e "${GREEN}[OK]${NC} $*"; } validate_file() { local file="$1" - local filename=$(basename "$file") + local filename + filename=$(basename -- "$file") echo "Validating: $filename" # Check file exists and is not empty - if [[ ! -f "$file" ]]; then - log_error "File not found: $file" - return 1 + if [[ ! -f "$file" || ! -r "$file" ]]; then + log_error "File missing or unreadable: $file" + return 0 fi if [[ ! -s "$file" ]]; then - log_warn "File is empty: $file" + log_error "File is empty: $file" + return 0 fi - local content - content=$(cat "$file") + # Bash strings cannot contain NUL; check the file before command substitution. + local nul_count + nul_count=$(LC_ALL=C tr -cd '\000' < "$file" | wc -c) + if (( nul_count > 0 )); then + log_error "$filename: Contains null bytes" + return 0 + fi # ========================================================================= # SECURITY CHECKS - These are ERRORS (treat as failures) # ========================================================================= # Check for unescaped script tags - if echo "$content" | grep -qi ']*>'; then + if grep -qi ']*>' "$file"; then log_error "$filename: Contains

\n' > "$scratch/script.html" +check 1 "$scratch/script.html" +printf '

x

\n' > "$scratch/event.html" +check 1 "$scratch/event.html" +printf '

x

\n' > "$scratch/event-spaced.html" +check 1 "$scratch/event-spaced.html" +printf '

link

\n' > "$scratch/url.html" +check 1 "$scratch/url.html" +{ printf '

'; head -c 262144 /dev/zero | tr '\000' x; printf '

\n'; } > "$scratch/large.html" +check 1 "$scratch/large.html" +printf '

nul\000byte

\n' > "$scratch/nul.html" +check 1 "$scratch/nul.html" +printf '

invalid\377

\n' > "$scratch/encoding.html" +check 1 "$scratch/encoding.html" +: > "$scratch/empty.html" +check 1 "$scratch/empty.html" +check 1 "$scratch/absent.html" +mkdir "$scratch/empty-directory" +check 1 "$scratch/empty-directory" +mkdir "$scratch/mixed-directory" +cp "$scratch/safe.html" "$scratch/mixed-directory/first safe.html" +cp "$scratch/script.html" "$scratch/mixed-directory/second unsafe.html" +check 1 "$scratch/mixed-directory" +grep -Fq '1 errors' "$scratch/result" +printf 'PASS: safe output accepted; empty, missing, injection, NUL, UTF-8 and directory controls rejected\n' diff --git a/tests/preservation-controls.sh b/tests/preservation-controls.sh new file mode 100755 index 0000000..efa5492 --- /dev/null +++ b/tests/preservation-controls.sh @@ -0,0 +1,41 @@ +#!/usr/bin/env bash +# SPDX-License-Identifier: MPL-2.0-or-later +set -euo pipefail +root=$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/.." && pwd) +scratch=$(mktemp -d) +trap 'rm -rf -- "$scratch"' EXIT +git -C "$scratch" init -q +git -C "$scratch" config user.name 'Preservation regression fixture' +git -C "$scratch" config user.email 'fixture@example.invalid' +git -C "$scratch" config commit.gpgsign false +printf 'retained source\n' > "$scratch/source.txt" +git -C "$scratch" add source.txt +git -C "$scratch" commit -qm 'fixture import' +import_commit=$(git -C "$scratch" rev-parse HEAD) +tree=$(git -C "$scratch" rev-parse 'HEAD^{tree}') +printf 'new documentation\n' > "$scratch/README" +git -C "$scratch" add README +git -C "$scratch" commit -qm 'fixture successor' +(cd "$scratch" && bash "$root/scripts/verify-preservation.sh" "$import_commit" "$tree") +reject() { + local label=$1 expected=$2 + shift 2 + local rc=0 + (cd "$scratch" && bash "$root/scripts/verify-preservation.sh" "$@") >"$scratch/result" 2>&1 || rc=$? + if (( rc != 1 )) || ! grep -Fq "$expected" "$scratch/result"; then + printf 'FAIL: %s did not produce the expected rejection\n' "$label" >&2 + cat "$scratch/result" >&2 + exit 1 + fi + printf 'PASS: rejected %s\n' "$label" +} +reject 'mutable import ref' 'immutable full Git object IDs' main "$tree" +reject 'missing import' 'preserved import is absent' 0000000000000000000000000000000000000000 "$tree" +reject 'wrong source tree' 'imported tree differs' "$import_commit" 0000000000000000000000000000000000000000 +git -C "$scratch" checkout --orphan unrelated -q +git -C "$scratch" rm -rfq . +printf 'unrelated\n' > "$scratch/other" +git -C "$scratch" add other +git -C "$scratch" commit -qm 'unrelated history' +reject 'unrelated history' 'does not retain the preserved import' "$import_commit" "$tree" + From da16d4c163bb8fcfaf7b875d7644fa9b4bd51c26 Mon Sep 17 00:00:00 2001 From: "Jonathan D.A. Jewell" <6759885+hyperpolymath@users.noreply.github.com> Date: Wed, 4 Nov 2026 18:19:30 +0000 Subject: [PATCH 4/7] fix(preservation): qualify retained evidence and retire obsolete effects Keep complete security analyses and bind historical findings to exact imported objects. Replace broken policy and scanner callers with checked contracts, document unsupported source honestly, and preserve four obsolete deployment/propagation workflows byte-for-byte as inert records. No forge dispatch, remote history rewrite, or repository archival is performed by this commit. --- .github/workflows/codeql.yml | 43 ++-- .github/workflows/hypatia-scan.yml | 185 ++---------------- .github/workflows/language-policy.yml | 87 ++------ .github/workflows/rsr-antipattern.yml | 97 ++------- .github/workflows/scorecard.yml | 34 ++-- .github/workflows/secret-scanner.yml | 70 ++----- .github/workflows/semgrep.yml | 45 +++-- .github/workflows/workflow-linter.yml | 161 ++------------- SECURITY.md | 40 ++++ docs/WORKFLOW-DISPOSITION.adoc | 155 +++++++++++++++ .../legacy-workflows/deploy-casket.yml.txt | 1 + .../legacy-workflows/deploy-ddraig.yml.txt | 1 + .../legacy-workflows/instant-sync.yml.txt | 1 + .../legacy-workflows/mirror.yml.txt | 1 + scripts/check-preserved-language-policy.sh | 30 +++ scripts/lint-workflow-contracts.sh | 34 ++++ scripts/preservation-scope.sh | 50 +++++ scripts/review-preservation-sarif.sh | 102 ++++++++++ tests/preservation-policy-controls.sh | 84 ++++++++ tests/workflow-contract-controls.sh | 46 +++++ 20 files changed, 691 insertions(+), 576 deletions(-) create mode 100644 SECURITY.md create mode 100644 docs/WORKFLOW-DISPOSITION.adoc rename .github/workflows/deploy-casket.yml => docs/legacy-workflows/deploy-casket.yml.txt (99%) rename .github/workflows/deploy-ddraig.yml => docs/legacy-workflows/deploy-ddraig.yml.txt (99%) rename .github/workflows/instant-sync.yml => docs/legacy-workflows/instant-sync.yml.txt (99%) rename .github/workflows/mirror.yml => docs/legacy-workflows/mirror.yml.txt (99%) create mode 100755 scripts/check-preserved-language-policy.sh create mode 100755 scripts/lint-workflow-contracts.sh create mode 100755 scripts/preservation-scope.sh create mode 100755 scripts/review-preservation-sarif.sh create mode 100755 tests/preservation-policy-controls.sh create mode 100755 tests/workflow-contract-controls.sh diff --git a/.github/workflows/codeql.yml b/.github/workflows/codeql.yml index 7f6e34d..ac1ff9d 100644 --- a/.github/workflows/codeql.yml +++ b/.github/workflows/codeql.yml @@ -1,6 +1,5 @@ # SPDX-License-Identifier: MPL-2.0-or-later name: CodeQL Security Analysis - on: push: branches: [main, master] @@ -8,14 +7,16 @@ on: branches: [main, master] schedule: - cron: '0 6 * * 1' - -permissions: read-all - +permissions: + contents: read + actions: read jobs: analyze: - runs-on: ubuntu-latest + runs-on: ubuntu-24.04 + timeout-minutes: 20 permissions: contents: read + actions: read security-events: write strategy: fail-fast: false @@ -23,18 +24,34 @@ jobs: include: - language: javascript-typescript build-mode: none - steps: - - name: Checkout - uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1 - + - name: Checkout immutable subject and retained ancestry + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + ref: ${{ github.event.pull_request.head.sha || github.sha }} + fetch-depth: 0 + persist-credentials: false - name: Initialize CodeQL - uses: github/codeql-action/init@662472033e021d55d94146f66f6058822b0b39fd # v3.28.1 + uses: github/codeql-action/init@9b3254085494b55a0f7c4c2b3d2d52cc712ce370 # v4.39.1 with: languages: ${{ matrix.language }} build-mode: ${{ matrix.build-mode }} - - name: Perform CodeQL Analysis - uses: github/codeql-action/analyze@662472033e021d55d94146f66f6058822b0b39fd # v3.28.1 + uses: github/codeql-action/analyze@9b3254085494b55a0f7c4c2b3d2d52cc712ce370 # v4.39.1 + with: + category: /language:${{ matrix.language }} + output: ${{ runner.temp }}/codeql-sarif + # This private repository has no hosted code-scanning entitlement. + # Keep actual analysis and complete SARIF; do not turn upload errors into success. + upload: never + upload-database: false + - name: Classify retained findings and block current findings + run: bash scripts/review-preservation-sarif.sh codeql "$RUNNER_TEMP/codeql-sarif/javascript.sarif" "$RUNNER_TEMP/codeql-sarif/disposition" + - name: Retain complete analysis and disposition + if: always() + uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2 with: - category: "/language:${{ matrix.language }}" + name: codeql-${{ matrix.language }}-${{ github.event.pull_request.head.sha || github.sha }} + path: ${{ runner.temp }}/codeql-sarif + if-no-files-found: error + retention-days: 90 diff --git a/.github/workflows/hypatia-scan.yml b/.github/workflows/hypatia-scan.yml index f2bf132..929e6d8 100644 --- a/.github/workflows/hypatia-scan.yml +++ b/.github/workflows/hypatia-scan.yml @@ -1,179 +1,24 @@ # SPDX-License-Identifier: PMPL-1.0-or-later -# Hypatia Neurosymbolic CI/CD Security Scan name: Hypatia Security Scan - on: push: - branches: [ main, master, develop ] + branches: [main, master, develop] pull_request: - branches: [ main, master ] + branches: [main, master] schedule: - - cron: '0 0 * * 0' # Weekly on Sunday + - cron: '0 0 * * 0' workflow_dispatch: - -permissions: read-all - +permissions: + actions: read + contents: read + security-events: write jobs: scan: - name: Hypatia Neurosymbolic Analysis - runs-on: ubuntu-latest - - steps: - - name: Checkout repository - uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4 - with: - fetch-depth: 0 # Full history for better pattern analysis - - - name: Setup Elixir for Hypatia scanner - uses: erlef/setup-beam@2f0cc07b4b9bea248ae098aba9e1a8a1de5ec24c # v1.18.2 - with: - elixir-version: '1.19.4' - otp-version: '28.3' - - - name: Clone Hypatia - run: | - if [ ! -d "$HOME/hypatia" ]; then - git clone https://github.com/hyperpolymath/hypatia.git "$HOME/hypatia" - fi - - - name: Build Hypatia scanner (if needed) - working-directory: ${{ env.HOME }}/hypatia - run: | - if [ ! -f hypatia-v2 ]; then - echo "Building hypatia-v2 scanner..." - cd scanner - mix deps.get - mix escript.build - mv hypatia ../hypatia-v2 - fi - - - name: Run Hypatia scan - id: scan - run: | - echo "Scanning repository: ${{ github.repository }}" - - # Run scanner - HYPATIA_FORMAT=json "$HOME/hypatia/hypatia-cli.sh" scan . > hypatia-findings.json - - # Count findings - FINDING_COUNT=$(jq '. | length' hypatia-findings.json 2>/dev/null || echo 0) - echo "findings_count=$FINDING_COUNT" >> $GITHUB_OUTPUT - - # Extract severity counts - CRITICAL=$(jq '[.[] | select(.severity == "critical")] | length' hypatia-findings.json) - HIGH=$(jq '[.[] | select(.severity == "high")] | length' hypatia-findings.json) - MEDIUM=$(jq '[.[] | select(.severity == "medium")] | length' hypatia-findings.json) - - echo "critical=$CRITICAL" >> $GITHUB_OUTPUT - echo "high=$HIGH" >> $GITHUB_OUTPUT - echo "medium=$MEDIUM" >> $GITHUB_OUTPUT - - echo "## Hypatia Scan Results" >> $GITHUB_STEP_SUMMARY - echo "- Total findings: $FINDING_COUNT" >> $GITHUB_STEP_SUMMARY - echo "- Critical: $CRITICAL" >> $GITHUB_STEP_SUMMARY - echo "- High: $HIGH" >> $GITHUB_STEP_SUMMARY - echo "- Medium: $MEDIUM" >> $GITHUB_STEP_SUMMARY - - - name: Upload findings artifact - uses: actions/upload-artifact@65c79d7f54e76e4e3c7a8f34db0f4ac8b515c478 # v4 - with: - name: hypatia-findings - path: hypatia-findings.json - retention-days: 90 - - - name: Submit findings to gitbot-fleet (Phase 2) - if: steps.scan.outputs.findings_count > 0 - env: - GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} - GITHUB_REPOSITORY: ${{ github.repository }} - GITHUB_SHA: ${{ github.sha }} - run: | - echo "📤 Submitting ${{ steps.scan.outputs.findings_count }} findings to gitbot-fleet..." - - # Clone gitbot-fleet to temp directory - FLEET_DIR="/tmp/gitbot-fleet-$$" - git clone https://github.com/hyperpolymath/gitbot-fleet.git "$FLEET_DIR" - - # Run submission script - bash "$FLEET_DIR/scripts/submit-finding.sh" hypatia-findings.json - - # Cleanup - rm -rf "$FLEET_DIR" - - echo "✅ Finding submission complete" - - - name: Check for critical issues - if: steps.scan.outputs.critical > 0 - run: | - echo "⚠️ Critical security issues found!" - echo "Review hypatia-findings.json for details" - # Don't fail the build yet - just warn - # exit 1 - - - name: Generate scan report - run: | - cat << EOF > hypatia-report.md - # Hypatia Security Scan Report - - **Repository:** ${{ github.repository }} - **Scan Date:** $(date -u +"%Y-%m-%d %H:%M:%S UTC") - **Commit:** ${{ github.sha }} - - ## Summary - - | Severity | Count | - |----------|-------| - | Critical | ${{ steps.scan.outputs.critical }} | - | High | ${{ steps.scan.outputs.high }} | - | Medium | ${{ steps.scan.outputs.medium }} | - | **Total**| ${{ steps.scan.outputs.findings_count }} | - - ## Next Steps - - 1. Review findings in the artifact: hypatia-findings.json - 2. Auto-fixable issues will be addressed by robot-repo-automaton (Phase 3) - 3. Manual review required for complex issues - - ## Learning - - These findings feed Hypatia's learning engine to improve future rules. - - --- - *Powered by [Hypatia](https://github.com/hyperpolymath/hypatia) - Neurosymbolic CI/CD Intelligence* - EOF - - cat hypatia-report.md >> $GITHUB_STEP_SUMMARY - - - name: Comment on PR with findings - if: github.event_name == 'pull_request' && steps.scan.outputs.findings_count > 0 - uses: actions/github-script@60a0d83039c74a4aee543508d2ffcb1c3799cdea # v7 - with: - script: | - const fs = require('fs'); - const findings = JSON.parse(fs.readFileSync('hypatia-findings.json', 'utf8')); - - const critical = findings.filter(f => f.severity === 'critical').length; - const high = findings.filter(f => f.severity === 'high').length; - - let comment = `## 🔍 Hypatia Security Scan\n\n`; - comment += `**Findings:** ${findings.length} issues detected\n\n`; - comment += `| Severity | Count |\n|----------|-------|\n`; - comment += `| 🔴 Critical | ${critical} |\n`; - comment += `| 🟠 High | ${high} |\n`; - comment += `| 🟡 Medium | ${findings.length - critical - high} |\n\n`; - - if (critical > 0) { - comment += `⚠️ **Action Required:** Critical security issues found!\n\n`; - } - - comment += `
View findings\n\n`; - comment += `\`\`\`json\n${JSON.stringify(findings.slice(0, 10), null, 2)}\n\`\`\`\n`; - comment += `
\n\n`; - comment += `*Powered by Hypatia Neurosymbolic CI/CD Intelligence*`; - - github.rest.issues.createComment({ - owner: context.repo.owner, - repo: context.repo.repo, - issue_number: context.issue.number, - body: comment - }); + uses: metadatastician/standards/.github/workflows/hypatia-scan-reusable.yml@d3e4b485a2af90c92a8e18af0888cd19cb132cd9 + with: + # The canonical grid reports all evidence and unavailable integrations. + # No broad path exclusions or caller-local scanner implementation. + scope-excludes: '' + report-backs-artifact: '' + secrets: + HYPATIA_SCAN_PAT: ${{ secrets.HYPATIA_SCAN_PAT }} diff --git a/.github/workflows/language-policy.yml b/.github/workflows/language-policy.yml index 1191cd2..72a0218 100644 --- a/.github/workflows/language-policy.yml +++ b/.github/workflows/language-policy.yml @@ -1,82 +1,23 @@ # SPDX-License-Identifier: MPL-2.0-or-later +# Source preservation is distinct from permission to introduce new runtime code. name: Language Policy Enforcement - on: push: - branches: ["*"] + branches: [main, master, develop] pull_request: - branches: ["*"] - -permissions: read-all - +permissions: + contents: read jobs: check-policy: - runs-on: ubuntu-latest + runs-on: ubuntu-24.04 + timeout-minutes: 10 steps: - - uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v4 - - - name: Setup Deno - uses: denoland/setup-deno@e95548e56ddc3f63d0e1eab7bed41d4031f02b36 # v2 + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 with: - deno-version: v2.x - - - name: Check for banned TypeScript files - run: | - echo "Checking for banned TypeScript files..." - VIOLATIONS=$(find . -type f \( -name "*.ts" -o -name "*.tsx" \) \ - ! -name "*.d.ts" \ - ! -path "./.git/*" \ - ! -path "./node_modules/*" \ - ! -path "./_site/*" | head -20) - if [ -n "$VIOLATIONS" ]; then - echo "ERROR: TypeScript files found (banned - use ReScript instead):" - echo "$VIOLATIONS" - exit 1 - fi - echo "No TypeScript violations found" - - - name: Check for banned Go files - run: | - echo "Checking for banned Go files..." - VIOLATIONS=$(find . -type f -name "*.go" \ - ! -path "./.git/*" | head -20) - if [ -n "$VIOLATIONS" ]; then - echo "ERROR: Go files found (banned - use Rust instead):" - echo "$VIOLATIONS" - exit 1 - fi - echo "No Go violations found" - - - name: Check for banned Node.js lock files - run: | - echo "Checking for banned lock files..." - VIOLATIONS="" - for f in package-lock.json yarn.lock pnpm-lock.yaml bun.lockb; do - if [ -f "$f" ]; then - VIOLATIONS="$VIOLATIONS $f" - fi - done - if [ -n "$VIOLATIONS" ]; then - echo "ERROR: Node.js lock files found (banned - use Deno instead):" - echo "$VIOLATIONS" - exit 1 - fi - echo "No Node.js lock file violations found" - - - name: Check for banned Python files - run: | - echo "Checking for banned Python files..." - VIOLATIONS=$(find . -type f -name "*.py" \ - ! -path "./.git/*" \ - ! -path "./salt/*" \ - ! -path "./saltstack/*" \ - ! -path "./_salt/*" | head -20) - if [ -n "$VIOLATIONS" ]; then - echo "ERROR: Python files found outside SaltStack (banned - use ReScript/Rust):" - echo "$VIOLATIONS" - exit 1 - fi - echo "No Python violations found" - - - name: Run full policy check - run: deno run --allow-read scripts/check-language-policy.js + ref: ${{ github.event.pull_request.head.sha || github.sha }} + fetch-depth: 0 + persist-credentials: false + - name: Reject new prohibited source and identify exact retained objects + run: bash scripts/check-preserved-language-policy.sh + - name: Reject changed, relocated and fabricated historical evidence + run: bash tests/preservation-policy-controls.sh diff --git a/.github/workflows/rsr-antipattern.yml b/.github/workflows/rsr-antipattern.yml index 641deed..bbc8a24 100644 --- a/.github/workflows/rsr-antipattern.yml +++ b/.github/workflows/rsr-antipattern.yml @@ -1,92 +1,23 @@ # SPDX-License-Identifier: MPL-2.0-or-later -# RSR Anti-Pattern CI Check -# SPDX-License-Identifier: MPL-2.0-or-later -# -# Enforces: No TypeScript, No Go, No Python (except SaltStack), No npm -# Allows: ReScript, Deno, WASM, Rust, OCaml, Haskell, Guile/Scheme - +# Source preservation is distinct from permission to introduce new runtime code. name: RSR Anti-Pattern Check - on: push: branches: [main, master, develop] pull_request: - branches: [main, master, develop] - - -permissions: read-all - +permissions: + contents: read jobs: antipattern-check: - runs-on: ubuntu-latest - permissions: - contents: read + runs-on: ubuntu-24.04 + timeout-minutes: 10 steps: - - uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1 - - - name: Check for TypeScript - run: | - # Exclude bindings/deno/ - those are Deno FFI files using Deno.dlopen, not plain TypeScript - # Exclude .d.ts files - those are TypeScript type declarations for ReScript FFI - TS_FILES=$(find . \( -name "*.ts" -o -name "*.tsx" \) | grep -v node_modules | grep -v 'bindings/deno' | grep -v '\.d\.ts$' || true) - if [ -n "$TS_FILES" ]; then - echo "❌ TypeScript files detected - use ReScript instead" - echo "$TS_FILES" - exit 1 - fi - echo "✅ No TypeScript files (Deno FFI bindings excluded)" - - - name: Check for Go - run: | - if find . -name "*.go" | grep -q .; then - echo "❌ Go files detected - use Rust/WASM instead" - find . -name "*.go" - exit 1 - fi - echo "✅ No Go files" - - - name: Check for Python (non-SaltStack) - run: | - PY_FILES=$(find . -name "*.py" | grep -v salt | grep -v _states | grep -v _modules | grep -v pillar | grep -v venv | grep -v __pycache__ || true) - if [ -n "$PY_FILES" ]; then - echo "❌ Python files detected - only allowed for SaltStack" - echo "$PY_FILES" - exit 1 - fi - echo "✅ No non-SaltStack Python files" - - - name: Check for npm lockfiles - run: | - if [ -f "package-lock.json" ] || [ -f "yarn.lock" ]; then - echo "❌ npm/yarn lockfile detected - use Deno instead" - exit 1 - fi - echo "✅ No npm lockfiles" - - - name: Check for tsconfig - run: | - if [ -f "tsconfig.json" ]; then - echo "❌ tsconfig.json detected - use ReScript instead" - exit 1 - fi - echo "✅ No tsconfig.json" - - - name: Verify Deno presence (if package.json exists) - run: | - if [ -f "package.json" ]; then - if [ ! -f "deno.json" ] && [ ! -f "deno.jsonc" ]; then - echo "⚠️ Warning: package.json without deno.json - migration recommended" - fi - fi - echo "✅ Deno configuration check complete" - - - name: Summary - run: | - echo "╔════════════════════════════════════════════════════════════╗" - echo "║ RSR Anti-Pattern Check Passed ✅ ║" - echo "║ ║" - echo "║ Allowed: ReScript, Deno, WASM, Rust, OCaml, Haskell, ║" - echo "║ Guile/Scheme, SaltStack (Python) ║" - echo "║ ║" - echo "║ Blocked: TypeScript, Go, npm, Python (non-Salt) ║" - echo "╚════════════════════════════════════════════════════════════╝" + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + ref: ${{ github.event.pull_request.head.sha || github.sha }} + fetch-depth: 0 + persist-credentials: false + - name: Reject new prohibited source and identify exact retained objects + run: bash scripts/check-preserved-language-policy.sh + - name: Reject changed, relocated and fabricated historical evidence + run: bash tests/preservation-policy-controls.sh diff --git a/.github/workflows/scorecard.yml b/.github/workflows/scorecard.yml index 58caf60..44b8f1d 100644 --- a/.github/workflows/scorecard.yml +++ b/.github/workflows/scorecard.yml @@ -8,30 +8,36 @@ on: schedule: - cron: '0 4 * * *' workflow_dispatch: - -permissions: read-all - +permissions: + contents: read + actions: read jobs: analysis: runs-on: ubuntu-24.04 timeout-minutes: 15 - permissions: - security-events: write - actions: read - contents: read steps: - - uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1 + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 with: + ref: ${{ github.event.pull_request.head.sha || github.sha }} + fetch-depth: 0 persist-credentials: false - - name: Run Scorecard uses: ossf/scorecard-action@fa7117db83c52d787ae5a89f350999701505d78e # v2.5.0 with: results_file: results.sarif results_format: sarif - - - name: Upload results - uses: github/codeql-action/upload-sarif@48ab28a6f5dbc2a99bf1e0131198dd8f1df78169 # v3.31.8 + publish_results: false + - name: Classify repository findings with explicit preservation scope + env: + SCORECARD_ANALYSIS_SCOPE: ${{ github.event_name == 'pull_request' && 'checked-out-subject' || 'remote-default-branch' }} + run: bash scripts/review-preservation-sarif.sh scorecard results.sarif "$RUNNER_TEMP/scorecard/disposition" + - name: Retain complete SARIF without unavailable hosted upload + if: always() + uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2 with: - sarif_file: results.sarif - + name: scorecard-${{ github.event.pull_request.head.sha || github.sha }} + path: | + results.sarif + ${{ runner.temp }}/scorecard + if-no-files-found: error + retention-days: 90 diff --git a/.github/workflows/secret-scanner.yml b/.github/workflows/secret-scanner.yml index cc29168..e66cd69 100644 --- a/.github/workflows/secret-scanner.yml +++ b/.github/workflows/secret-scanner.yml @@ -1,67 +1,27 @@ # SPDX-License-Identifier: MPL-2.0-or-later -# Prevention workflow - scans for hardcoded secrets before they reach main name: Secret Scanner - on: pull_request: push: branches: [main] - -permissions: read-all - +permissions: + contents: read jobs: + scan: + uses: hyperpolymath/cicd-suite/.github/workflows/secret-scanner-reusable.yml@52a90c24bf183437b887dd182c5a9406af02c5dd + with: + fast-pr: false + permissions: + contents: read trufflehog: - runs-on: ubuntu-latest + runs-on: ubuntu-24.04 + timeout-minutes: 15 steps: - - uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v4 + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 with: - fetch-depth: 0 # Full history for scanning - - - name: TruffleHog Secret Scan - uses: trufflesecurity/trufflehog@ef6e76c3c4023279497fab4721ffa071a722fd05 # v3 + fetch-depth: 0 + persist-credentials: false + - name: TruffleHog verified-credential scan + uses: trufflesecurity/trufflehog@ef6e76c3c4023279497fab4721ffa071a722fd05 # v3.92.4 with: extra_args: --only-verified --fail - - gitleaks: - runs-on: ubuntu-latest - steps: - - uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v4 - with: - fetch-depth: 0 - - - name: Gitleaks Secret Scan - uses: gitleaks/gitleaks-action@ff98106e4c7b2bc287b24eaf42907196329070c7 # v2 - env: - GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} - - # Rust-specific: Check for hardcoded crypto values - rust-secrets: - runs-on: ubuntu-latest - if: hashFiles('**/Cargo.toml') != '' - steps: - - uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v4 - - - name: Check for hardcoded secrets in Rust - run: | - # Patterns that suggest hardcoded secrets - PATTERNS=( - 'const.*SECRET.*=.*"' - 'const.*KEY.*=.*"[a-zA-Z0-9]{16,}"' - 'const.*TOKEN.*=.*"' - 'let.*api_key.*=.*"' - 'HMAC.*"[a-fA-F0-9]{32,}"' - 'password.*=.*"[^"]+"' - ) - - found=0 - for pattern in "${PATTERNS[@]}"; do - if grep -rn --include="*.rs" -E "$pattern" src/; then - echo "WARNING: Potential hardcoded secret found matching: $pattern" - found=1 - fi - done - - if [ $found -eq 1 ]; then - echo "::error::Potential hardcoded secrets detected. Use environment variables instead." - exit 1 - fi diff --git a/.github/workflows/semgrep.yml b/.github/workflows/semgrep.yml index 89e1f38..4c3ea84 100644 --- a/.github/workflows/semgrep.yml +++ b/.github/workflows/semgrep.yml @@ -1,6 +1,5 @@ # SPDX-License-Identifier: MPL-2.0-or-later name: Semgrep SAST - on: push: branches: [main, master] @@ -9,30 +8,36 @@ on: schedule: - cron: '0 5 * * *' workflow_dispatch: - -permissions: read-all - +permissions: + contents: read jobs: semgrep: runs-on: ubuntu-24.04 timeout-minutes: 15 - permissions: - security-events: write - actions: read - contents: read - container: - image: semgrep/semgrep@sha256:451b3d875c00dac7ed5db5293e584a07de11449a409c10b3c2d2652f429630b6 steps: - - uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1 - - - name: Run Semgrep - run: semgrep scan --sarif --output=semgrep.sarif --config=auto . + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + ref: ${{ github.event.pull_request.head.sha || github.sha }} + fetch-depth: 0 + persist-credentials: false + - name: Run pinned Semgrep against the complete subject env: SEMGREP_APP_TOKEN: ${{ secrets.SEMGREP_APP_TOKEN }} - - - name: Upload SARIF - uses: github/codeql-action/upload-sarif@5d4e8d1aca955e8d8589aabd499c5cae939e33c7 # v4.31.9 - with: - sarif_file: semgrep.sarif + run: | + set -euo pipefail + mkdir -p "$RUNNER_TEMP/semgrep" + docker run --rm --entrypoint sh \ + -v "$PWD:/src:ro" -v "$RUNNER_TEMP/semgrep:/out" -w /src \ + -e SEMGREP_APP_TOKEN \ + semgrep/semgrep@sha256:451b3d875c00dac7ed5db5293e584a07de11449a409c10b3c2d2652f429630b6 \ + -c 'git config --global --add safe.directory /src && exec semgrep scan --sarif --output=/out/semgrep.sarif --config=auto .' + - name: Classify retained findings and block current findings + run: bash scripts/review-preservation-sarif.sh semgrep "$RUNNER_TEMP/semgrep/semgrep.sarif" "$RUNNER_TEMP/semgrep/disposition" + - name: Retain complete SARIF without unavailable hosted upload if: always() - + uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2 + with: + name: semgrep-${{ github.event.pull_request.head.sha || github.sha }} + path: ${{ runner.temp }}/semgrep + if-no-files-found: error + retention-days: 90 diff --git a/.github/workflows/workflow-linter.yml b/.github/workflows/workflow-linter.yml index b6050d7..0e31695 100644 --- a/.github/workflows/workflow-linter.yml +++ b/.github/workflows/workflow-linter.yml @@ -1,157 +1,22 @@ # SPDX-License-Identifier: MPL-2.0-or-later -# workflow-linter.yml - Validates GitHub workflows against RSR security standards -# This workflow can be copied to other repos for consistent enforcement name: Workflow Security Linter - on: push: - paths: - - '.github/workflows/**' + paths: ['.github/workflows/**', 'scripts/lint-workflow-contracts.sh', 'tests/workflow-contract-controls.sh'] pull_request: - paths: - - '.github/workflows/**' + paths: ['.github/workflows/**', 'scripts/lint-workflow-contracts.sh', 'tests/workflow-contract-controls.sh'] workflow_dispatch: - -permissions: read-all - +permissions: + contents: read jobs: lint-workflows: - runs-on: ubuntu-latest - permissions: - contents: read - + runs-on: ubuntu-24.04 + timeout-minutes: 10 steps: - - name: Checkout - uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1 - - - name: Check SPDX Headers - run: | - echo "=== Checking SPDX License Headers ===" - failed=0 - for file in .github/workflows/*.yml .github/workflows/*.yaml; do - [ -f "$file" ] || continue - if ! head -1 "$file" | grep -q "^# SPDX-License-Identifier:"; then - echo "ERROR: $file missing SPDX header" - failed=1 - fi - done - if [ $failed -eq 1 ]; then - echo "Add '# SPDX-License-Identifier: PMPL-1.0-or-later' as first line" - exit 1 - fi - echo "All workflows have SPDX headers" - - - name: Check Permissions Declaration - run: | - echo "=== Checking Permissions ===" - failed=0 - for file in .github/workflows/*.yml .github/workflows/*.yaml; do - [ -f "$file" ] || continue - if ! grep -q "^permissions:" "$file"; then - echo "ERROR: $file missing top-level 'permissions:' declaration" - failed=1 - fi - done - if [ $failed -eq 1 ]; then - echo "Add 'permissions: read-all' at workflow level" - exit 1 - fi - echo "All workflows have permissions declared" - - - name: Check SHA-Pinned Actions - run: | - echo "=== Checking Action Pinning ===" - # Find any uses: lines that don't have @SHA format - # Exclude: comment lines, local actions, docker actions, github-script - unpinned=$(grep -rn "uses:" .github/workflows/ | \ - grep -v "^\s*#" | \ - grep -v "@[a-f0-9]\{40\}" | \ - grep -v "uses: \./\|uses: docker://\|uses: actions/github-script" | \ - grep -v "# Pattern:\|# Find any" || true) - - if [ -n "$unpinned" ]; then - echo "ERROR: Found unpinned actions:" - echo "$unpinned" - echo "" - echo "Replace version tags with SHA pins, e.g.:" - echo " uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v4.1.1" - exit 1 - fi - echo "All actions are SHA-pinned" - - - name: Check for Duplicate Workflows - run: | - echo "=== Checking for Duplicates ===" - # Known duplicate patterns - if [ -f .github/workflows/codeql.yml ] && [ -f .github/workflows/codeql-analysis.yml ]; then - echo "ERROR: Duplicate CodeQL workflows found" - echo "Delete codeql-analysis.yml (keep codeql.yml)" - exit 1 - fi - if [ -f .github/workflows/rust.yml ] && [ -f .github/workflows/rust-ci.yml ]; then - echo "WARNING: Potential duplicate Rust workflows" - echo "Consider consolidating rust.yml and rust-ci.yml" - fi - echo "No critical duplicates found" - - - name: Check CodeQL Language Matrix - run: | - echo "=== Checking CodeQL Configuration ===" - if [ ! -f .github/workflows/codeql.yml ]; then - echo "No CodeQL workflow found (optional)" - exit 0 - fi - - # Detect repo languages - has_js=$(find . -name "*.js" -o -name "*.ts" -o -name "*.jsx" -o -name "*.tsx" -path "*/src/*" -o -path "*/lib/*" 2>/dev/null | head -1) - has_py=$(find . -name "*.py" -path "*/src/*" -o -path "*/lib/*" 2>/dev/null | head -1) - has_go=$(find . -name "*.go" -path "*/src/*" -o -path "*/cmd/*" -o -path "*/pkg/*" 2>/dev/null | head -1) - has_rs=$(find . -name "*.rs" -path "*/src/*" 2>/dev/null | head -1) - has_java=$(find . -name "*.java" -path "*/src/*" 2>/dev/null | head -1) - has_rb=$(find . -name "*.rb" -path "*/lib/*" -o -path "*/app/*" 2>/dev/null | head -1) - - echo "Detected languages:" - [ -n "$has_js" ] && echo " - javascript-typescript" - [ -n "$has_py" ] && echo " - python" - [ -n "$has_go" ] && echo " - go" - [ -n "$has_rs" ] && echo " - rust (note: CodeQL rust is limited)" - [ -n "$has_java" ] && echo " - java-kotlin" - [ -n "$has_rb" ] && echo " - ruby" - - # Check for over-reach - if grep -q "language:.*'go'" .github/workflows/codeql.yml && [ -z "$has_go" ]; then - echo "WARNING: CodeQL configured for Go but no Go files found" - fi - if grep -q "language:.*'python'" .github/workflows/codeql.yml && [ -z "$has_py" ]; then - echo "WARNING: CodeQL configured for Python but no Python files found" - fi - if grep -q "language:.*'java'" .github/workflows/codeql.yml && [ -z "$has_java" ]; then - echo "WARNING: CodeQL configured for Java but no Java files found" - fi - if grep -q "language:.*'ruby'" .github/workflows/codeql.yml && [ -z "$has_rb" ]; then - echo "WARNING: CodeQL configured for Ruby but no Ruby files found" - fi - - echo "CodeQL check complete" - - - name: Check Secrets Guards - run: | - echo "=== Checking Secrets Usage ===" - # Look for secrets without conditional guards in mirror workflows - if [ -f .github/workflows/mirror.yml ]; then - if grep -q "secrets\." .github/workflows/mirror.yml; then - if ! grep -q "if:.*vars\." .github/workflows/mirror.yml; then - echo "WARNING: mirror.yml uses secrets without vars guard" - echo "Add 'if: vars.FEATURE_ENABLED == true' to jobs" - fi - fi - fi - echo "Secrets check complete" - - - name: Summary - run: | - echo "" - echo "=== Workflow Linter Summary ===" - echo "All critical checks passed." - echo "" - echo "For more info, see: robot-repo-bot/ERROR-CATALOG.scm" + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + persist-credentials: false + - name: Parse workflow contracts and complete action pins + run: bash scripts/lint-workflow-contracts.sh + - name: Reject floating refs without matching comments or script strings + run: bash tests/workflow-contract-controls.sh diff --git a/SECURITY.md b/SECURITY.md new file mode 100644 index 0000000..3269b5b --- /dev/null +++ b/SECURITY.md @@ -0,0 +1,40 @@ + +# Security policy + +## Supported scope + +This repository is private source-preservation staging for historical Poly SSG +experiments. It offers no supported executable release, hosted service or +production deployment. The retained source has known, unqualified security +findings. A passing preservation check establishes only the claim named by +that check; it does not establish that an engine is safe to run. + +Current preservation scripts, workflow definitions and documentation are +maintained within the qualification work tracked in issue #2. An implementation +selected for a maintained successor needs its own security and runtime review. + +## Reporting a vulnerability + +Contact the repository owner at **jonathan.jewell@open.ac.uk**, the reporting +address already published in this repository's `.well-known/security.txt`. +Identify the repository, exact commit, affected path, impact and a small +reproduction where possible. Do not include live credentials or disclose private +source in a public issue or public pull request. No response-time or remediation +SLA is promised by this preservation project. + +Reports about current automation and exposed credentials require review even +when the associated source is historical. Preserved-source findings remain +recorded in complete scanner artifacts and are not described as fixed or false +positives solely because the source is unsupported. + +## Analysis records + +GitHub hosted code scanning was unavailable for this private repository when +the workflow repair was prepared. CodeQL, Semgrep and Scorecard retain complete +SARIF artifacts and explicit disposition reports. Missing or failed analysis +cannot become a passing finding gate. Artifact retention is finite; qualification +issue #2 must retain the accepted evidence needed for final closure. + +## Acknowledgments + +Reporter attribution will be agreed with the reporter before publication. diff --git a/docs/WORKFLOW-DISPOSITION.adoc b/docs/WORKFLOW-DISPOSITION.adoc new file mode 100644 index 0000000..4204523 --- /dev/null +++ b/docs/WORKFLOW-DISPOSITION.adoc @@ -0,0 +1,155 @@ +// SPDX-License-Identifier: CC-BY-SA-4.0 += Preservation workflow and findings disposition +:revdate: 2026-11-04 + +This repository retains unqualified historical source. The active work is source +preservation, review and explicit retirement; it supplies no supported generator +release. `PRESERVATION.adoc`, `RETENTION-PLAN.adoc` and qualification issue #2 +remain the scope records. This document records workflow decisions, not a claim +that their new hosted executions have already passed. + +== Identity before historical classification + +The immutable import is `9302dfcc3aedb7c52edf2867a2fcbf2c735878b4`, with tree +`6a8aa93222c8261c3e81b37acd7d1bd59f42bdc4`. The active checks verify that tree, +its object connectivity and its ancestry in the exact checked-out subject. + +The language gate enumerates the complete committed subject tree. Prohibited +source is retained only when the exact path, regular-file mode and Git blob +match that import and the path is in an explicitly retained source area. +Changed modes, relocated copies and newly added files cannot inherit the +historical disposition. TypeScript implementation, Go, Python, Java/Kotlin, +Swift, Dart, npm/yarn/pnpm lockfiles and tsconfig are checked; declaration-only +`.d.ts` files retain the existing FFI exception. Bun manifests are allowed by +the current repository language instructions. The obsolete Deno policy runner +remains available in source history but is not the active policy authority. + +At the inspected PR subject `f82227bcbd13f27eab207ff791aa79639decefd4`, the two +prohibited-language objects were: + +[cols="3,1,3",options="header"] +|=== +|Path |Mode |Imported and current blob +|`engines/milk-ssg/milk-ssg.py` |`100755` |`1024c994bf5ea704d2c9c791815b34ddba7e133e` +|`engines/terrapin-ssg/terrapin-ssg.py` |`100755` |`b6ee989e6176c775c7b95aa56f2fa32df800a8e6` +|=== + +They remain unsupported historical experiments. This is not permission to +introduce or develop Python runtime code, and neither file is deleted to +make a language check pass. + +== Complete security evidence + +The previous runs successfully produced analysis, then failed to upload it to +GitHub code scanning, which is unavailable for this private repository: + +* https://github.com/hyperpolymath/poly-ssg/actions/runs/41878049639[CodeQL] + exported JavaScript/TypeScript SARIF before the integration API rejected it. +* https://github.com/hyperpolymath/poly-ssg/actions/runs/41878049561[Semgrep] + ran 503 rules against 2,355 targets and reported 313 findings. Those findings + were not thereby fixed or established as false positives. +* https://github.com/hyperpolymath/poly-ssg/actions/runs/41878049542[Scorecard] + reported `SecurityPolicyID` and `FuzzingID`; its later upload was rejected. + +The repaired jobs continue the real analyses and retain complete raw SARIF in +90-day artifacts. CodeQL uses upstream v4.39.1 and its documented `upload: +never` and `upload-database: false` controls. Semgrep retains the exact image +digest observed in the executed scan and the existing `--config=auto` rule +selection. That rule registry can change; the artifact describes its actual +run, not a permanently frozen rule corpus. Scorecard publishing remains off. +No private source or result is published to a public destination by this change. + +The disposition script does not remove results from SARIF. It records each +original result with one of three decisions: + +. *Retained historical source*: every reported location is in an enumerated + unsupported source area and has the exact imported path, blob and file mode. + The finding remains present and unremediated. The areas are `engines/`, + `satellites/`, `casket-ssg/`, `src/`, `lib/`, `public/`, `content/`, `mcp/`, + `lsp/`, `test-corpus/` and `sites-build/`. A prefix alone is insufficient. +. *Preservation advisory*: only Scorecard's exact observed `FuzzingID` condition + that no fuzzing integration exists. This repository provides no supported + executable release. A selected runtime successor must qualify its own fuzzing; + this disposition does not claim that fuzzing ran. +. *Actionable*: every other finding, including current automation, new or + changed source, missing source locations, unrecognised paths and missing + security policy. These findings fail the job and remain in the artifact. + +The newly supplied root `SECURITY.md` resolves the missing reporting-policy +document by using the contact already recorded in `.well-known/security.txt`. +Hosted execution must confirm the resulting Scorecard finding set. No response +SLA or supported historical release is invented. + +Missing, malformed or failed analyses return an infrastructure/configuration +failure rather than a clean result. Each disposition binds the raw report's +SHA-256 and the checked-out subject SHA. CodeQL and Semgrep analyze that checkout. +Upstream Scorecard v2.5.0 scans the local checkout on pull requests but queries +the remote default branch for other events. The latter reports explicitly leave +the analyzed SHA unknown and do not classify source findings against a possibly +different checkout. The reports explicitly say hosted +upload is unavailable and make no clean-security claim. Accepted evidence needed +for permanent closure must be retained beyond the finite artifact lifetime in +qualification issue #2's closure record. + +== Other scanner and policy repairs + +The Secret Scanner directly calls the published CI-suite implementation at +`52a90c24bf183437b887dd182c5a9406af02c5dd`, with full history and `fast-pr: +false`. Its required contexts are `scan / gitleaks`, `scan / rust-secrets` and +`scan / shell-secrets`; the latter two validate actual detector receipts. +Independent TruffleHog verified-credential scanning is retained. Secret findings +are not covered by the historical SARIF disposition and still block. + +Hypatia uses the canonical standards reusable at +`d3e4b485a2af90c92a8e18af0888cd19cb132cd9`, with its declared permissions and +only its declared optional scan secret. There is no caller-local scanner clone, +floating scanner implementation, jq-to-zero fallback or unsupported fleet +submission. The canonical verdict grid and artifacts report missing evidence +and findings. Its default advisory mode is not a clean-security attestation. + +The workflow linter parses actual YAML job and step `uses` keys. Full action +SHAs, container digests, explicit permissions and SPDX headers are required. +Comments and script string literals cannot masquerade as action references; +`actions/github-script` no longer receives an implicit pinning exemption. + +== Retired deployment and propagation effects + +The following exact files, including their notices, are retained as inert text: + +[cols="3,3,4",options="header"] +|=== +|Former active workflow |Retained path |Disposition +|`.github/workflows/deploy-casket.yml` +|`docs/legacy-workflows/deploy-casket.yml.txt` +|Unsupported manual deployment referenced missing `casket-simple`. Retain the +source record; no current deployment is supplied. +|`.github/workflows/deploy-ddraig.yml` +|`docs/legacy-workflows/deploy-ddraig.yml.txt` +|Unsupported automatic/manual deployment used an unqualified Ddraig build and +a floating image. Retain the source record; no current deployment is supplied. +|`.github/workflows/instant-sync.yml` +|`docs/legacy-workflows/instant-sync.yml.txt` +|Push/release dispatch into `.git-private-farm` is retired for this preservation +repository. No propagation event is emitted by the archival workflow surface. +|`.github/workflows/mirror.yml` +|`docs/legacy-workflows/mirror.yml.txt` +|Seven-forge propagation, including force-push jobs and an unqualified Radicle +installer, is retired. Its exact original source remains available for review. +|=== + +This removes unsupported deployment and propagation effects from the active workflow surface. +It does not remove source history or claim that the former deployments worked. +Any maintained successor must establish its own deployment and runtime gates. +No remote forge history or mirror is deleted or rewritten by this change. + +== Validation and remaining evidence + +Local controls exercise actual Git object comparisons and reject changed source, +copied/relocated paths, changed modes, wrong trees, malformed reports, failed +analysis and wrong reporter identities. Parser controls reject floating action +refs while accepting inert comments. Those synthetic controls validate gate +behaviour; they do not substitute for real scans of the final PR SHA. + +The final branch still needs hosted checks, complete findings review, +required-context consistency and applicable app reviews. Actual repository +archival and permanent closure remain separately recorded actions. diff --git a/.github/workflows/deploy-casket.yml b/docs/legacy-workflows/deploy-casket.yml.txt similarity index 99% rename from .github/workflows/deploy-casket.yml rename to docs/legacy-workflows/deploy-casket.yml.txt index a865df4..e6cb03d 100644 --- a/.github/workflows/deploy-casket.yml +++ b/docs/legacy-workflows/deploy-casket.yml.txt @@ -32,3 +32,4 @@ jobs: - name: Deploy to GitHub Pages id: deployment uses: actions/deploy-pages@d6db90164ac5ed86f2b6aed7e0febac5b3c0c03e # v4 + diff --git a/.github/workflows/deploy-ddraig.yml b/docs/legacy-workflows/deploy-ddraig.yml.txt similarity index 99% rename from .github/workflows/deploy-ddraig.yml rename to docs/legacy-workflows/deploy-ddraig.yml.txt index 8d4ed07..ac43aa7 100644 --- a/.github/workflows/deploy-ddraig.yml +++ b/docs/legacy-workflows/deploy-ddraig.yml.txt @@ -44,3 +44,4 @@ jobs: - name: Deploy to GitHub Pages id: deployment uses: actions/deploy-pages@d6db90164ac5ed86f2b6aed7e0febac5b3c0c03e # v4 + diff --git a/.github/workflows/instant-sync.yml b/docs/legacy-workflows/instant-sync.yml.txt similarity index 99% rename from .github/workflows/instant-sync.yml rename to docs/legacy-workflows/instant-sync.yml.txt index 3c78453..46ddfb2 100644 --- a/.github/workflows/instant-sync.yml +++ b/docs/legacy-workflows/instant-sync.yml.txt @@ -31,3 +31,4 @@ jobs: - name: Confirm run: echo "::notice::Propagation triggered for ${{ github.event.repository.name }}" + diff --git a/.github/workflows/mirror.yml b/docs/legacy-workflows/mirror.yml.txt similarity index 99% rename from .github/workflows/mirror.yml rename to docs/legacy-workflows/mirror.yml.txt index a7b1547..546d61d 100644 --- a/.github/workflows/mirror.yml +++ b/docs/legacy-workflows/mirror.yml.txt @@ -142,3 +142,4 @@ jobs: echo "${{ secrets.RADICLE_KEY }}" > ~/.radicle/keys/radicle chmod 600 ~/.radicle/keys/radicle rad sync --announce || echo "Radicle sync attempted" + diff --git a/scripts/check-preserved-language-policy.sh b/scripts/check-preserved-language-policy.sh new file mode 100755 index 0000000..3146141 --- /dev/null +++ b/scripts/check-preserved-language-policy.sh @@ -0,0 +1,30 @@ +#!/usr/bin/env bash +# SPDX-License-Identifier: MPL-2.0-or-later +# Preserve exact historical objects; reject newly introduced prohibited source. +set -euo pipefail +(( $# == 0 || $# == 2 )) || { echo 'usage: check-preserved-language-policy.sh [import-commit expected-tree]' >&2; exit 2; } +source "$(dirname -- "${BASH_SOURCE[0]}")/preservation-scope.sh" +preservation_scope_init "$@" || exit 2 +historical=0 +violations=0 +for path in "${!SUBJECT_OBJECTS[@]}"; do + case "$path" in + *.d.ts) continue ;; + *.ts|*.tsx|*.go|*.py|*.java|*.kt|*.kts|*.swift|*.dart|package-lock.json|*/package-lock.json|yarn.lock|*/yarn.lock|pnpm-lock.yaml|*/pnpm-lock.yaml|tsconfig.json|*/tsconfig.json) ;; + *) continue ;; + esac + if preserved_source_matches "$path"; then + printf 'HISTORICAL unsupported source: %s [%s]\n' "$path" "${SUBJECT_OBJECTS["$path"]}" + historical=$((historical + 1)) + else + printf 'VIOLATION: new, changed, relocated or active prohibited source: %s\n' "$path" >&2 + violations=$((violations + 1)) + fi +done +printf 'Language preservation check: subject=%s historical=%s violations=%s\n' "$SUBJECT_SHA" "$historical" "$violations" +echo 'Historical means exact retained source, not current language approval or runtime/security qualification.' +if [[ -n ${GITHUB_STEP_SUMMARY:-} ]]; then + printf '### Language preservation scope\n\nSubject: `%s`\n\nRetained prohibited-language objects: **%s**. New/changed violations: **%s**. Historical objects remain unsupported.\n' \ + "$SUBJECT_SHA" "$historical" "$violations" >> "$GITHUB_STEP_SUMMARY" +fi +(( violations == 0 )) diff --git a/scripts/lint-workflow-contracts.sh b/scripts/lint-workflow-contracts.sh new file mode 100755 index 0000000..15063d1 --- /dev/null +++ b/scripts/lint-workflow-contracts.sh @@ -0,0 +1,34 @@ +#!/usr/bin/env bash +# SPDX-License-Identifier: MPL-2.0-or-later +set -euo pipefail +command -v yq >/dev/null || { echo 'yq is required to parse actual workflow keys' >&2; exit 2; } +shopt -s nullglob +files=(.github/workflows/*.yml .github/workflows/*.yaml) +(( ${#files[@]} > 0 )) || { echo 'No workflow population' >&2; exit 2; } +failed=0 +refs=$(mktemp) +trap 'rm -f -- "$refs"' EXIT +for file in "${files[@]}"; do + IFS= read -r first < "$file" + [[ "$first" == '# SPDX-License-Identifier:'* ]] || { echo "Missing SPDX header: $file" >&2; failed=1; } + if ! yq -e 'has("permissions") and (.jobs | tag == "!!map")' "$file" >/dev/null; then + echo "Malformed workflow or missing permission declaration: $file" >&2; failed=1; continue + fi + if ! yq -r '([.jobs[] | select(has("uses")) | .uses] + [.jobs[].steps[]? | select(has("uses")) | .uses]) | .[]' "$file" > "$refs"; then + echo "Cannot enumerate workflow uses: $file" >&2; failed=1; continue + fi + while IFS= read -r ref; do + if [[ "$ref" == ./* ]]; then + case "/${ref#./}/" in */../*|*/./*|*//*) echo "Invalid local action path: $file" >&2; failed=1 ;; esac + elif [[ "$ref" == docker://* ]]; then + [[ "$ref" =~ ^docker://[^[:space:]@]+@sha256:[0-9a-f]{64}$ ]] || { echo "Unpinned container action: $file" >&2; failed=1; } + elif [[ ! "$ref" =~ ^[A-Za-z0-9_.-]+/[A-Za-z0-9_.-]+(/[A-Za-z0-9_./-]+)?@[0-9a-f]{40}$ ]]; then + printf 'Unpinned action or reusable workflow in %s: %s\n' "$file" "$ref" >&2; failed=1 + fi + done < "$refs" +done +if [[ -f .github/workflows/codeql.yml && -f .github/workflows/codeql-analysis.yml ]]; then + echo 'Duplicate CodeQL entry points require review' >&2; failed=1 +fi +(( failed == 0 )) || exit 1 +printf 'Parsed %s workflows: SPDX, explicit permissions and complete action pins passed.\n' "${#files[@]}" diff --git a/scripts/preservation-scope.sh b/scripts/preservation-scope.sh new file mode 100755 index 0000000..9646504 --- /dev/null +++ b/scripts/preservation-scope.sh @@ -0,0 +1,50 @@ +#!/usr/bin/env bash +# SPDX-License-Identifier: MPL-2.0-or-later +# Shared source-identity checks. Source this file; it never executes retained code. + +preservation_scope_init() { + local checker_dir entry metadata path + checker_dir=$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd) + PRESERVED_IMPORT=${1:-9302dfcc3aedb7c52edf2867a2fcbf2c735878b4} + PRESERVED_TREE=${2:-6a8aa93222c8261c3e81b37acd7d1bd59f42bdc4} + bash "$checker_dir/verify-preservation.sh" "$PRESERVED_IMPORT" "$PRESERVED_TREE" || return 2 + SUBJECT_SHA=$(git rev-parse --verify HEAD) || return 2 + declare -gA PRESERVED_OBJECTS=() SUBJECT_OBJECTS=() + local inventory + inventory=$(mktemp) || return 2 + if ! git ls-tree -rz --full-tree "$PRESERVED_IMPORT" > "$inventory"; then + rm -f -- "$inventory"; return 2 + fi + while IFS= read -r -d '' entry; do + metadata=${entry%%$'\t'*}; path=${entry#*$'\t'} + PRESERVED_OBJECTS["$path"]=$metadata + done < "$inventory" + if ! git ls-tree -rz --full-tree "$SUBJECT_SHA" > "$inventory"; then + rm -f -- "$inventory"; return 2 + fi + while IFS= read -r -d '' entry; do + metadata=${entry%%$'\t'*}; path=${entry#*$'\t'} + SUBJECT_OBJECTS["$path"]=$metadata + done < "$inventory" + rm -f -- "$inventory" + (( ${#PRESERVED_OBJECTS[@]} > 0 && ${#SUBJECT_OBJECTS[@]} > 0 )) +} + +preserved_source_area() { + # Explicit unsupported source areas from docs/RETENTION-PLAN.adoc. + # Current automation, scripts, policy, documentation and configuration do + # not acquire an exemption merely because an old copy exists in the import. + case "$1" in + engines/*|satellites/*|casket-ssg/*|src/*|lib/*|public/*|content/*|mcp/*|lsp/*|test-corpus/*|sites-build/*) return 0 ;; + *) return 1 ;; + esac +} + +preserved_source_matches() { + local path=$1 old current + [[ -n "$path" && "$path" != /* && "$path" != *$'\n'* && "$path" != *$'\r'* ]] || return 1 + case "/$path/" in */../*|*/./*|*//*|*%*|*\\*) return 1 ;; esac + preserved_source_area "$path" || return 1 + old=${PRESERVED_OBJECTS["$path"]-}; current=${SUBJECT_OBJECTS["$path"]-} + [[ -n "$old" && "$old" == "$current" && "$old" =~ ^100(644|755)\ blob\ [0-9a-f]{40}$ ]] +} diff --git a/scripts/review-preservation-sarif.sh b/scripts/review-preservation-sarif.sh new file mode 100755 index 0000000..6005e89 --- /dev/null +++ b/scripts/review-preservation-sarif.sh @@ -0,0 +1,102 @@ +#!/usr/bin/env bash +# SPDX-License-Identifier: MPL-2.0-or-later +# Keep complete SARIF. Classify findings; never rewrite or suppress a result. +set -euo pipefail +if (( $# != 3 && $# != 5 )); then + echo 'usage: review-preservation-sarif.sh reporter input.sarif output-prefix [import-commit expected-tree]' >&2 + exit 2 +fi +reporter=$1; input=$2; output=$3; shift 3 +case "$reporter" in codeql|semgrep|scorecard) ;; *) echo 'Unknown reporter' >&2; exit 2 ;; esac +analysis_scope=checked-out-subject +if [[ "$reporter" == scorecard ]]; then + case "${SCORECARD_ANALYSIS_SCOPE:-}" in + checked-out-subject|remote-default-branch) analysis_scope=$SCORECARD_ANALYSIS_SCOPE ;; + *) echo 'Scorecard analysis scope must be declared from the workflow event' >&2; exit 2 ;; + esac +fi +command -v jq >/dev/null || { echo 'jq is required' >&2; exit 2; } +source "$(dirname -- "${BASH_SOURCE[0]}")/preservation-scope.sh" +preservation_scope_init "$@" || exit 2 +[[ -f "$input" && ! -L "$input" && -s "$input" ]] || { echo 'Missing or unsafe SARIF input' >&2; exit 2; } +if ! jq -se --arg reporter "$reporter" ' + length == 1 and (.[0] | + .version == "2.1.0" and (.runs | type == "array" and length > 0) and + all(.runs[]; + (.tool.driver.name | type == "string" and length > 0) and + (if $reporter == "codeql" then .tool.driver.name == "CodeQL" + elif $reporter == "scorecard" then .tool.driver.name == "Scorecard" + else (.tool.driver.name | ascii_downcase | startswith("semgrep")) end) and + (.results | type == "array") and + all(.invocations[]?; .executionSuccessful == true and + all(.toolExecutionNotifications[]?; .level != "error")) and + all(.results[]; + (.ruleId | type == "string" and length > 0) and + (.message | type == "object") and + ((.message.text // .message.markdown) | type == "string") and + (.level == null or (.level as $l | ["error","warning","note","none"] | index($l) != null)) and + (.locations == null or (.locations | type == "array"))))) +' "$input" >/dev/null; then + echo 'Malformed, failed or mismatched SARIF analysis; no passing result can be inferred' >&2 + exit 2 +fi +mkdir -p -- "$(dirname -- "$output")" +rows=$(mktemp); results=$(mktemp) +trap 'rm -f -- "$rows" "$results"' EXIT +jq -c '.runs[] | .results[]' "$input" > "$results" +total=0; historical=0; advisory=0; actionable=0 +while IFS= read -r result; do + total=$((total + 1)) + rule=$(jq -r '.ruleId' <<< "$result") + disposition=actionable + reason='Current, changed, unlocated or unqualified source requires review.' + location_count=$(jq '(.locations // []) | length' <<< "$result") + # Only this observed repository-level condition has a staging disposition. + # Missing security policy and every other Scorecard rule remain actionable. + if [[ "$reporter" == scorecard && "$rule" == FuzzingID ]] && + jq -e '.message.text | startswith("score is 0: project is not fuzzed:")' <<< "$result" >/dev/null && + jq -e '(.locations // []) | all(.[]; .physicalLocation.artifactLocation.uri == "no file associated with this alert")' <<< "$result" >/dev/null; then + disposition=preservation-advisory + reason='No supported executable release is provided by this preservation repository; no fuzzing qualification is claimed. A selected runtime successor must qualify its own fuzzing.' + advisory=$((advisory + 1)) + elif [[ "$analysis_scope" == checked-out-subject ]] && (( location_count > 0 )); then + all_preserved=true + for (( index=0; index 0)' <<< "$result"); then + all_preserved=false; break + fi + path=${path#./} + if ! preserved_source_matches "$path"; then all_preserved=false; break; fi + done + if [[ "$all_preserved" == true ]]; then + disposition=retained-historical-source + reason='Every reported location is an enumerated unsupported source area with exactly the imported Git path, blob and file mode. The finding is retained, not fixed or declared false.' + historical=$((historical + 1)) + fi + fi + [[ "$disposition" != actionable ]] || actionable=$((actionable + 1)) + jq -cn --arg reporter "$reporter" --arg disposition "$disposition" --arg reason "$reason" \ + --argjson result "$result" '{reporter:$reporter,disposition:$disposition,reason:$reason,result:$result}' >> "$rows" +done < "$results" +jq -s --arg reporter "$reporter" --arg subject "$SUBJECT_SHA" --arg import "$PRESERVED_IMPORT" \ + --arg analysis_scope "$analysis_scope" \ + --arg input_sha256 "$(sha256sum "$input" | cut -d ' ' -f 1)" \ + --argjson total "$total" --argjson historical "$historical" --argjson advisory "$advisory" --argjson actionable "$actionable" \ + '{schema_version:1,reporter:$reporter,checked_out_subject_sha:$subject,import_sha:$import, + analysis_scope:$analysis_scope,analysis_subject_sha:(if $analysis_scope == "checked-out-subject" then $subject else null end), + raw_sarif_sha256:$input_sha256,hosted_upload:"unavailable-private-repository", + security_clean_claim:false,counts:{total:$total,retained_historical:$historical,preservation_advisory:$advisory,actionable:$actionable},findings:.}' \ + "$rows" > "$output.json" +{ + printf '### %s analysis disposition\n\n' "$reporter" + printf 'Checkout: `%s`. Analysis scope: `%s`. Raw SARIF is retained unchanged in the workflow artifact.\n\n' "$SUBJECT_SHA" "$analysis_scope" + if [[ "$analysis_scope" == remote-default-branch ]]; then + printf 'Scorecard queried the remote default branch; its precise analyzed commit is not supplied by this action. Checkout identity is not asserted as analysis identity, and source findings cannot receive the historical-file disposition.\n\n' + fi + printf '| Total | Exact retained source | Preservation advisory | Current or unqualified |\n| ---: | ---: | ---: | ---: |\n| %s | %s | %s | %s |\n\n' "$total" "$historical" "$advisory" "$actionable" + printf 'Hosted code-scanning upload is unavailable for this private repository. This is a preservation disposition, not a clean-security or supported-runtime attestation.\n' +} > "$output.md" +cat "$output.md" +[[ -z ${GITHUB_STEP_SUMMARY:-} ]] || cat "$output.md" >> "$GITHUB_STEP_SUMMARY" +(( actionable == 0 )) || { echo "::error::$reporter has $actionable current or unqualified finding(s); inspect the complete SARIF and disposition artifact" >&2; exit 1; } diff --git a/tests/preservation-policy-controls.sh b/tests/preservation-policy-controls.sh new file mode 100755 index 0000000..f64d639 --- /dev/null +++ b/tests/preservation-policy-controls.sh @@ -0,0 +1,84 @@ +#!/usr/bin/env bash +# SPDX-License-Identifier: MPL-2.0-or-later +set -euo pipefail +root=$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/.." && pwd) +tmp=$(mktemp -d) +trap 'rm -rf -- "$tmp"' EXIT +export GIT_CONFIG_NOSYSTEM=1 GIT_CONFIG_GLOBAL=/dev/null +export GIT_AUTHOR_NAME=Fixture GIT_COMMITTER_NAME=Fixture +export GIT_AUTHOR_EMAIL=fixture@example.invalid GIT_COMMITTER_EMAIL=fixture@example.invalid +git init -q "$tmp/repo" +cd "$tmp/repo" +mkdir -p engines/example scripts +printf '%s\n' '# retained source fixture' > engines/example/archive.py +printf '%s\n' '# active automation fixture' > scripts/active.sh +git add .; git commit -qm import +base=$(git rev-parse HEAD); tree=$(git rev-parse HEAD^{tree}) +expect() { + local wanted=$1 marker=$2; shift 2 + local actual=0 + "$@" > "$tmp/output" 2>&1 || actual=$? + if [[ "$actual" != "$wanted" ]] || ! grep -Fq -- "$marker" "$tmp/output"; then + cat "$tmp/output" >&2; printf 'Expected status %s and marker %s, got %s\n' "$wanted" "$marker" "$actual" >&2; exit 1 + fi + printf 'ok: %s\n' "$marker" +} +policy=(bash "$root/scripts/check-preserved-language-policy.sh" "$base" "$tree") +expect 0 'historical=1 violations=0' "${policy[@]}" +printf '%s\n' '# changed historical source' >> engines/example/archive.py +git add .; git commit -qm changed +expect 1 'VIOLATION: new, changed, relocated or active prohibited source' "${policy[@]}" +git reset --hard -q "$base" +cp engines/example/archive.py engines/example/new.py +git add .; git commit -qm copied +expect 1 'VIOLATION: new, changed, relocated or active prohibited source' "${policy[@]}" +git reset --hard -q "$base" +chmod +x engines/example/archive.py +git add .; git commit -qm mode +expect 1 'VIOLATION: new, changed, relocated or active prohibited source' "${policy[@]}" +git reset --hard -q "$base" +git mv engines/example/archive.py scripts/relocated.py +git commit -qm moved +expect 1 'VIOLATION: new, changed, relocated or active prohibited source' "${policy[@]}" +git reset --hard -q "$base" +expect 2 'imported tree differs' bash "$root/scripts/check-preserved-language-policy.sh" "$base" "0000000000000000000000000000000000000000" + +sarif="$tmp/result.sarif" +make_result() { + jq -n --arg tool "$1" --arg rule "$2" --arg uri "$3" --arg message "${4:-fixture finding}" \ + '{version:"2.1.0",runs:[{tool:{driver:{name:$tool}},results:[{ruleId:$rule,level:"warning",message:{text:$message},locations:[{physicalLocation:{artifactLocation:{uri:$uri}}}]}]}]}' > "$sarif" +} +review=(bash "$root/scripts/review-preservation-sarif.sh") +export SCORECARD_ANALYSIS_SCOPE=checked-out-subject +make_result CodeQL fixture engines/example/archive.py +original=$(sha256sum "$sarif") +expect 0 'Exact retained source' "${review[@]}" codeql "$sarif" "$tmp/receipt" "$base" "$tree" +[[ $(sha256sum "$sarif") == "$original" ]] +jq -e '.counts.total == 1 and .counts.retained_historical == 1 and .security_clean_claim == false' "$tmp/receipt.json" >/dev/null +make_result CodeQL fixture scripts/active.sh +expect 1 'current or unqualified finding(s)' "${review[@]}" codeql "$sarif" "$tmp/receipt" "$base" "$tree" +make_result CodeQL fixture ../engines/example/archive.py +expect 1 'current or unqualified finding(s)' "${review[@]}" codeql "$sarif" "$tmp/receipt" "$base" "$tree" +make_result CodeQL fixture engines/example/archive.py +printf '%s\n' '# current mutation' >> engines/example/archive.py +git add .; git commit -qm current +expect 1 'current or unqualified finding(s)' "${review[@]}" codeql "$sarif" "$tmp/receipt" "$base" "$tree" +git reset --hard -q "$base" +make_result Scorecard FuzzingID 'no file associated with this alert' 'score is 0: project is not fuzzed: fixture' +expect 0 'Preservation advisory' "${review[@]}" scorecard "$sarif" "$tmp/receipt" "$base" "$tree" +jq -e '.counts.preservation_advisory == 1 and .counts.actionable == 0' "$tmp/receipt.json" >/dev/null +make_result Scorecard SecurityPolicyID 'no file associated with this alert' +expect 1 'current or unqualified finding(s)' "${review[@]}" scorecard "$sarif" "$tmp/receipt" "$base" "$tree" +make_result Scorecard FuzzingID 'no file associated with this alert' 'a different failure' +expect 1 'current or unqualified finding(s)' "${review[@]}" scorecard "$sarif" "$tmp/receipt" "$base" "$tree" +make_result Scorecard fixture engines/example/archive.py +SCORECARD_ANALYSIS_SCOPE=remote-default-branch expect 1 'current or unqualified finding(s)' "${review[@]}" scorecard "$sarif" "$tmp/receipt" "$base" "$tree" +jq -e '.analysis_subject_sha == null and .analysis_scope == "remote-default-branch"' "$tmp/receipt.json" >/dev/null +make_result CodeQL fixture engines/example/archive.py +jq '.runs[0].invocations=[{executionSuccessful:false}]' "$sarif" > "$tmp/bad.sarif" +expect 2 'Malformed, failed or mismatched' "${review[@]}" codeql "$tmp/bad.sarif" "$tmp/receipt" "$base" "$tree" +expect 2 'Malformed, failed or mismatched' "${review[@]}" semgrep "$sarif" "$tmp/receipt" "$base" "$tree" +printf '{' > "$tmp/bad.sarif" +expect 2 'Malformed, failed or mismatched' "${review[@]}" codeql "$tmp/bad.sarif" "$tmp/receipt" "$base" "$tree" +expect 2 'Missing or unsafe SARIF input' "${review[@]}" codeql "$tmp/absent.sarif" "$tmp/receipt" "$base" "$tree" +echo 'Preservation policy and SARIF controls passed; synthetic fixtures do not qualify actual repository analysis.' diff --git a/tests/workflow-contract-controls.sh b/tests/workflow-contract-controls.sh new file mode 100755 index 0000000..854625e --- /dev/null +++ b/tests/workflow-contract-controls.sh @@ -0,0 +1,46 @@ +#!/usr/bin/env bash +# SPDX-License-Identifier: MPL-2.0-or-later +set -euo pipefail +root=$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/.." && pwd) +tmp=$(mktemp -d) +trap 'rm -rf -- "$tmp"' EXIT +mkdir -p "$tmp/.github/workflows" +cd "$tmp" +fixture=.github/workflows/fixture.yml +cat > "$fixture" <<'YAML' +# SPDX-License-Identifier: MPL-2.0-or-later +name: fixture +on: push +permissions: + contents: read +jobs: + fixture: + runs-on: ubuntu-24.04 + steps: + # uses: fabricated/comment@main + - run: | + echo 'uses: string/not-an-action@main' + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 + - uses: ./local-action +YAML +bash "$root/scripts/lint-workflow-contracts.sh" > "$tmp/good" +grep -Fq 'Parsed 1 workflows' "$tmp/good" +cp "$fixture" "$tmp/original" +reject() { + local actual=0 + bash "$root/scripts/lint-workflow-contracts.sh" > "$tmp/result" 2>&1 || actual=$? + [[ "$actual" == 1 ]] && grep -Fq 'Unpinned action' "$tmp/result" || { cat "$tmp/result" >&2; exit 1; } + cp "$tmp/original" "$fixture" +} +sed -i 's/@3d3c42e5aac5ba805825da76410c181273ba90b1/@main/' "$fixture" +reject +sed -i 's/@3d3c42e5aac5ba805825da76410c181273ba90b1/@3d3c42e5aac5ba805825da76410c181273ba90b1extra/' "$fixture" +reject +sed -i 's|actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1|actions/github-script@main|' "$fixture" +reject +cat >> "$fixture" <<'YAML' + reusable: + uses: example/source/.github/workflows/check.yml@main +YAML +reject +echo 'Workflow parser controls passed: actual unpinned refs rejected; comments and run strings are not action refs.' From d15c5d08056cf7a0c43b6c2d7b724389f8c4860d Mon Sep 17 00:00:00 2001 From: "Jonathan D.A. Jewell" <6759885+hyperpolymath@users.noreply.github.com> Date: Wed, 4 Nov 2026 19:24:44 +0000 Subject: [PATCH 5/7] fix(ci): run the secret scan and repair current policy findings Remove a duplicate TruffleHog --fail argument because the pinned upstream action already supplies it. Scope Hypatia security-events write to its scan job, preserving required read grants. Add seven-day cooldowns to the existing four Dependabot version-update entries. Preserve all historical raw findings and exact-source qualification; no classifier broadening. Validation: actionlint and existing workflow-parser controls; parsed YAML comparison; actual retained SARIF/receipt equality and original import location qualification. Hosted new-head reruns remain required. --- .github/dependabot.yml | 9 +++++++++ .github/workflows/hypatia-scan.yml | 5 ++++- .github/workflows/secret-scanner.yml | 3 ++- 3 files changed, 15 insertions(+), 2 deletions(-) diff --git a/.github/dependabot.yml b/.github/dependabot.yml index 2958123..19bd936 100644 --- a/.github/dependabot.yml +++ b/.github/dependabot.yml @@ -1,3 +1,4 @@ +# SPDX-License-Identifier: MPL-2.0-or-later # Dependabot configuration for poly-ssg # https://docs.github.com/en/code-security/dependabot/dependabot-version-updates @@ -6,6 +7,8 @@ updates: # GitHub Actions - package-ecosystem: "github-actions" directory: "/" + cooldown: + default-days: 7 schedule: interval: "weekly" day: "monday" @@ -19,6 +22,8 @@ updates: # OCaml/opam dependencies (rescript-wasm) - package-ecosystem: "npm" directory: "/engines/rescript-wasm" + cooldown: + default-days: 7 schedule: interval: "weekly" day: "monday" @@ -34,6 +39,8 @@ updates: # Forth Estate (if any npm deps added) - package-ecosystem: "npm" directory: "/engines/forth-estate" + cooldown: + default-days: 7 schedule: interval: "weekly" commit-message: @@ -45,6 +52,8 @@ updates: # Root project dependencies - package-ecosystem: "npm" directory: "/" + cooldown: + default-days: 7 schedule: interval: "weekly" commit-message: diff --git a/.github/workflows/hypatia-scan.yml b/.github/workflows/hypatia-scan.yml index 929e6d8..80239b5 100644 --- a/.github/workflows/hypatia-scan.yml +++ b/.github/workflows/hypatia-scan.yml @@ -11,10 +11,13 @@ on: permissions: actions: read contents: read - security-events: write jobs: scan: uses: metadatastician/standards/.github/workflows/hypatia-scan-reusable.yml@d3e4b485a2af90c92a8e18af0888cd19cb132cd9 + permissions: + actions: read + contents: read + security-events: write with: # The canonical grid reports all evidence and unavailable integrations. # No broad path exclusions or caller-local scanner implementation. diff --git a/.github/workflows/secret-scanner.yml b/.github/workflows/secret-scanner.yml index e66cd69..7f9ede2 100644 --- a/.github/workflows/secret-scanner.yml +++ b/.github/workflows/secret-scanner.yml @@ -24,4 +24,5 @@ jobs: - name: TruffleHog verified-credential scan uses: trufflesecurity/trufflehog@ef6e76c3c4023279497fab4721ffa071a722fd05 # v3.92.4 with: - extra_args: --only-verified --fail + # The pinned action supplies --fail; repeating it aborts before scanning. + extra_args: --only-verified From 472ded90119d36e4afa413ad3a2178d57637c77b Mon Sep 17 00:00:00 2001 From: "Jonathan D.A. Jewell" <6759885+hyperpolymath@users.noreply.github.com> Date: Wed, 4 Nov 2026 20:02:02 +0000 Subject: [PATCH 6/7] fix(preservation): qualify every source location and preserve exact archives Check secondary SARIF evidence and reject unresolved references before granting a historical disposition. Require immutable job and service container images, state the source-object language boundary precisely, and restore all five archived workflows to their original Git blobs with regression controls. --- .github/workflows/ci.yml | 4 + docs/WORKFLOW-DISPOSITION.adoc | 62 ++++++++- docs/legacy-workflows/ci.yml.txt | 1 - docs/legacy-workflows/deploy-casket.yml.txt | 1 - docs/legacy-workflows/deploy-ddraig.yml.txt | 1 - docs/legacy-workflows/instant-sync.yml.txt | 1 - docs/legacy-workflows/mirror.yml.txt | 1 - scripts/check-preserved-language-policy.sh | 7 +- scripts/lint-workflow-contracts.sh | 36 +++++- scripts/preservation-sarif-locations.jq | 108 ++++++++++++++++ scripts/preservation-scope.sh | 5 +- scripts/review-preservation-sarif.sh | 24 +++- scripts/verify-archived-workflows.sh | 29 +++++ tests/archived-workflow-controls.sh | 31 +++++ tests/preservation-policy-controls.sh | 135 ++++++++++++++++---- tests/workflow-contract-controls.sh | 51 +++++++- 16 files changed, 449 insertions(+), 48 deletions(-) create mode 100644 scripts/preservation-sarif-locations.jq create mode 100755 scripts/verify-archived-workflows.sh create mode 100755 tests/archived-workflow-controls.sh diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 48e76d2..396dc80 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -26,6 +26,10 @@ jobs: run: bash scripts/verify-preservation.sh - name: Reject corrupt provenance and unrelated history run: bash tests/preservation-controls.sh + - name: Verify retired workflow bytes and reject changed archives + run: | + bash scripts/verify-archived-workflows.sh + bash tests/archived-workflow-controls.sh - name: Validate output checker with independent positive and negative inputs run: bash tests/output-validation-controls.sh - name: Record qualification scope diff --git a/docs/WORKFLOW-DISPOSITION.adoc b/docs/WORKFLOW-DISPOSITION.adoc index 4204523..5bfda1a 100644 --- a/docs/WORKFLOW-DISPOSITION.adoc +++ b/docs/WORKFLOW-DISPOSITION.adoc @@ -38,6 +38,15 @@ They remain unsupported historical experiments. This is not permission to introduce or develop Python runtime code, and neither file is deleted to make a language check pass. +This gate checks a *source-object delta*, not execution or activation. An added +shell wrapper could invoke an unchanged historical file while the exact +path/blob/mode comparison still passes. A regression fixture records that +boundary, and both console and job-summary output state it explicitly. Reviews +of workflow, script, build and package entry points remain a separate activation +boundary: retaining an object does not authorize running it. No automated +whole-program non-activation guarantee is claimed. A maintained successor must +qualify its selected runtime and dependencies before enabling those entry points. + == Complete security evidence The previous runs successfully produced analysis, then failed to upload it to @@ -75,6 +84,24 @@ original result with one of three decisions: changed source, missing source locations, unrecognised paths and missing security policy. These findings fail the job and remain in the artifact. +Location collection follows the source-bearing result structures in the +https://docs.oasis-open.org/sarif/sarif/v2.1.0/os/schemas/sarif-schema-2.1.0.json[SARIF 2.1.0 schema]. +It includes primary and related locations, the analysis target, nested code-flow +locations and stacks, result stacks, graph nodes and children, fixes, +attachments, suppression locations and conversion-source provenance. Every +collected artifact must independently satisfy the exact imported-object check. +A retained primary location cannot cover a current secondary location. + +The supported form is an explicit source-relative URI, optionally based at +`%SRCROOT%`. Missing and logical-only locations, cached indices, unresolved +location relationships or graph traversals, address-based/runtime evidence, +unknown source bases and opaque extension fields remain actionable. The script +does not guess their referents. Externalized report data is rejected as an +incomplete analysis. The narrow `FuzzingID` advisory requires a nonempty primary +location population and every collected location to be the observed no-file +placeholder; an added real-file location prevents that advisory. Each receipt +retains both the original result and its collected location evidence. + The newly supplied root `SECURITY.md` resolves the missing reporting-policy document by using the contact already recorded in `.well-known/security.txt`. Hosted execution must confirm the resulting Scorecard finding set. No response @@ -107,8 +134,11 @@ floating scanner implementation, jq-to-zero fallback or unsupported fleet submission. The canonical verdict grid and artifacts report missing evidence and findings. Its default advisory mode is not a clean-security attestation. -The workflow linter parses actual YAML job and step `uses` keys. Full action -SHAs, container digests, explicit permissions and SPDX headers are required. +The workflow linter parses actual YAML job and step `uses` keys, job containers +in string or object form, and every service container image. Full action SHAs, +literal SHA-256 image digests, explicit permissions and SPDX headers are required. +Missing, malformed, floating or expression-derived image references fail the +contract; a dynamic expression is not treated as proof of a digest pin. Comments and script string literals cannot masquerade as action references; `actions/github-script` no longer receives an implicit pinning exemption. @@ -119,6 +149,10 @@ The following exact files, including their notices, are retained as inert text: [cols="3,3,4",options="header"] |=== |Former active workflow |Retained path |Disposition +|`.github/workflows/ci.yml` +|`docs/legacy-workflows/ci.yml.txt` +|The former unqualified runtime build is retained as an inert record. The active +replacement checks preservation and independent regression controls. |`.github/workflows/deploy-casket.yml` |`docs/legacy-workflows/deploy-casket.yml.txt` |Unsupported manual deployment referenced missing `casket-simple`. Retain the @@ -142,12 +176,32 @@ It does not remove source history or claim that the former deployments worked. Any maintained successor must establish its own deployment and runtime gates. No remote forge history or mirror is deleted or rewritten by this change. +`scripts/verify-archived-workflows.sh` checks exact archive bytes against the +original Git blobs. The CI original is taken from +`7ab68dddd6bfd01f3f4d62fff9ed6804835ea2e0`, the parent of its archive-creation +commit. The other four originals are taken from +`f82227bcbd13f27eab207ff791aa79639decefd4`, the parent of their archival commit. +These are identity checks, not use of Git's object hash as a security attestation. + +[cols="2,4",options="header"] +|=== +|Original workflow |Exact original blob retained +|`ci.yml` |`10835026204a248fd671748b2215b1bccc38a771` +|`deploy-casket.yml` |`a865df41062cb761bbeb7a7f04b9e7e91e75deb5` +|`deploy-ddraig.yml` |`8d4ed07ca40fb96dd5be39622a9614256d9b42a1` +|`instant-sync.yml` |`3c784538bf4c1b434ec913874ba4b45e7f91b185` +|`mirror.yml` |`a7b1547af5637dacea5690deb0c1d492d7b56370` +|=== + == Validation and remaining evidence Local controls exercise actual Git object comparisons and reject changed source, copied/relocated paths, changed modes, wrong trees, malformed reports, failed -analysis and wrong reporter identities. Parser controls reject floating action -refs while accepting inert comments. Those synthetic controls validate gate +analysis and wrong reporter identities. Mixed current/historical secondary +locations and unresolved references cannot inherit a historical disposition. +Parser controls reject floating action and job/service image refs while accepting +inert comments. Archive controls reject appended bytes, absent files and symlinks. +Those synthetic controls validate gate behaviour; they do not substitute for real scans of the final PR SHA. The final branch still needs hosted checks, complete findings review, diff --git a/docs/legacy-workflows/ci.yml.txt b/docs/legacy-workflows/ci.yml.txt index b6bd666..1083502 100644 --- a/docs/legacy-workflows/ci.yml.txt +++ b/docs/legacy-workflows/ci.yml.txt @@ -273,4 +273,3 @@ jobs: echo "- Lint & Format: PASSED" echo "- Security Checks: PASSED" echo "=========================================" - diff --git a/docs/legacy-workflows/deploy-casket.yml.txt b/docs/legacy-workflows/deploy-casket.yml.txt index e6cb03d..a865df4 100644 --- a/docs/legacy-workflows/deploy-casket.yml.txt +++ b/docs/legacy-workflows/deploy-casket.yml.txt @@ -32,4 +32,3 @@ jobs: - name: Deploy to GitHub Pages id: deployment uses: actions/deploy-pages@d6db90164ac5ed86f2b6aed7e0febac5b3c0c03e # v4 - diff --git a/docs/legacy-workflows/deploy-ddraig.yml.txt b/docs/legacy-workflows/deploy-ddraig.yml.txt index ac43aa7..8d4ed07 100644 --- a/docs/legacy-workflows/deploy-ddraig.yml.txt +++ b/docs/legacy-workflows/deploy-ddraig.yml.txt @@ -44,4 +44,3 @@ jobs: - name: Deploy to GitHub Pages id: deployment uses: actions/deploy-pages@d6db90164ac5ed86f2b6aed7e0febac5b3c0c03e # v4 - diff --git a/docs/legacy-workflows/instant-sync.yml.txt b/docs/legacy-workflows/instant-sync.yml.txt index 46ddfb2..3c78453 100644 --- a/docs/legacy-workflows/instant-sync.yml.txt +++ b/docs/legacy-workflows/instant-sync.yml.txt @@ -31,4 +31,3 @@ jobs: - name: Confirm run: echo "::notice::Propagation triggered for ${{ github.event.repository.name }}" - diff --git a/docs/legacy-workflows/mirror.yml.txt b/docs/legacy-workflows/mirror.yml.txt index 546d61d..a7b1547 100644 --- a/docs/legacy-workflows/mirror.yml.txt +++ b/docs/legacy-workflows/mirror.yml.txt @@ -142,4 +142,3 @@ jobs: echo "${{ secrets.RADICLE_KEY }}" > ~/.radicle/keys/radicle chmod 600 ~/.radicle/keys/radicle rad sync --announce || echo "Radicle sync attempted" - diff --git a/scripts/check-preserved-language-policy.sh b/scripts/check-preserved-language-policy.sh index 3146141..5941ac7 100755 --- a/scripts/check-preserved-language-policy.sh +++ b/scripts/check-preserved-language-policy.sh @@ -1,6 +1,6 @@ #!/usr/bin/env bash # SPDX-License-Identifier: MPL-2.0-or-later -# Preserve exact historical objects; reject newly introduced prohibited source. +# Source-object delta check; execution references are outside this gate's scope. set -euo pipefail (( $# == 0 || $# == 2 )) || { echo 'usage: check-preserved-language-policy.sh [import-commit expected-tree]' >&2; exit 2; } source "$(dirname -- "${BASH_SOURCE[0]}")/preservation-scope.sh" @@ -17,14 +17,15 @@ for path in "${!SUBJECT_OBJECTS[@]}"; do printf 'HISTORICAL unsupported source: %s [%s]\n' "$path" "${SUBJECT_OBJECTS["$path"]}" historical=$((historical + 1)) else - printf 'VIOLATION: new, changed, relocated or active prohibited source: %s\n' "$path" >&2 + printf 'VIOLATION: prohibited source object differs in path, blob, mode or retained area: %s\n' "$path" >&2 violations=$((violations + 1)) fi done printf 'Language preservation check: subject=%s historical=%s violations=%s\n' "$SUBJECT_SHA" "$historical" "$violations" echo 'Historical means exact retained source, not current language approval or runtime/security qualification.' +echo 'Execution references and activation are not analyzed by this source-object delta check.' if [[ -n ${GITHUB_STEP_SUMMARY:-} ]]; then - printf '### Language preservation scope\n\nSubject: `%s`\n\nRetained prohibited-language objects: **%s**. New/changed violations: **%s**. Historical objects remain unsupported.\n' \ + printf '### Language preservation scope\n\nSubject: `%s`\n\nRetained prohibited-language objects: **%s**. Object-delta violations: **%s**. Historical objects remain unsupported. Execution references and activation are not analyzed by this check.\n' \ "$SUBJECT_SHA" "$historical" "$violations" >> "$GITHUB_STEP_SUMMARY" fi (( violations == 0 )) diff --git a/scripts/lint-workflow-contracts.sh b/scripts/lint-workflow-contracts.sh index 15063d1..7af7220 100755 --- a/scripts/lint-workflow-contracts.sh +++ b/scripts/lint-workflow-contracts.sh @@ -2,12 +2,15 @@ # SPDX-License-Identifier: MPL-2.0-or-later set -euo pipefail command -v yq >/dev/null || { echo 'yq is required to parse actual workflow keys' >&2; exit 2; } +command -v jq >/dev/null || { echo 'jq is required to inspect parsed container definitions' >&2; exit 2; } shopt -s nullglob files=(.github/workflows/*.yml .github/workflows/*.yaml) (( ${#files[@]} > 0 )) || { echo 'No workflow population' >&2; exit 2; } failed=0 refs=$(mktemp) -trap 'rm -f -- "$refs"' EXIT +parsed=$(mktemp) +images=$(mktemp) +trap 'rm -f -- "$refs" "$parsed" "$images"' EXIT for file in "${files[@]}"; do IFS= read -r first < "$file" [[ "$first" == '# SPDX-License-Identifier:'* ]] || { echo "Missing SPDX header: $file" >&2; failed=1; } @@ -19,16 +22,43 @@ for file in "${files[@]}"; do fi while IFS= read -r ref; do if [[ "$ref" == ./* ]]; then - case "/${ref#./}/" in */../*|*/./*|*//*) echo "Invalid local action path: $file" >&2; failed=1 ;; esac + case "/${ref#./}/" in + */../*|*/./*|*//*) echo "Invalid local action path: $file" >&2; failed=1 ;; + *) ;; # A canonical local path requires no remote version pin. + esac elif [[ "$ref" == docker://* ]]; then [[ "$ref" =~ ^docker://[^[:space:]@]+@sha256:[0-9a-f]{64}$ ]] || { echo "Unpinned container action: $file" >&2; failed=1; } elif [[ ! "$ref" =~ ^[A-Za-z0-9_.-]+/[A-Za-z0-9_.-]+(/[A-Za-z0-9_./-]+)?@[0-9a-f]{40}$ ]]; then printf 'Unpinned action or reusable workflow in %s: %s\n' "$file" "$ref" >&2; failed=1 fi done < "$refs" + if ! yq -o=json '.' "$file" > "$parsed" || + ! jq -r ' + def image: + if type == "string" and length > 0 and (test("[\u0000-\u001f\u007f]") | not) then . + else error("container image is missing or malformed") end; + def service_image: + if type == "object" then .image | image + else error("service container is not an object") end; + [.jobs[] | + (if has("container") then .container | + if type == "string" then image else service_image end + else empty end), + (if has("services") then .services | + if type == "object" then .[] | service_image + else error("services is not a map") end + else empty end)] | .[] + ' "$parsed" > "$images"; then + echo "Malformed job or service container: $file" >&2; failed=1; continue + fi + while IFS= read -r image; do + if [[ ! "$image" =~ ^[A-Za-z0-9][A-Za-z0-9._:/-]*@sha256:[0-9a-f]{64}$ ]]; then + echo "Unpinned job or service container image: $file" >&2; failed=1 + fi + done < "$images" done if [[ -f .github/workflows/codeql.yml && -f .github/workflows/codeql-analysis.yml ]]; then echo 'Duplicate CodeQL entry points require review' >&2; failed=1 fi (( failed == 0 )) || exit 1 -printf 'Parsed %s workflows: SPDX, explicit permissions and complete action pins passed.\n' "${#files[@]}" +printf 'Parsed %s workflows: SPDX, explicit permissions, action pins and container image digests passed.\n' "${#files[@]}" diff --git a/scripts/preservation-sarif-locations.jq b/scripts/preservation-sarif-locations.jq new file mode 100644 index 0000000..8bb3c9d --- /dev/null +++ b/scripts/preservation-sarif-locations.jq @@ -0,0 +1,108 @@ +# SPDX-License-Identifier: MPL-2.0-or-later +# SARIF 2.1.0 result source locations, including secondary evidence. +# Unsupported references remain actionable; this does not resolve cached, +# external, logical-only or extension-defined locations by guessing a file. + +def object_value: + if type == "object" then . else error("location object is missing or malformed") end; + +def entries($key): + if has($key) then + .[$key] | if type == "array" then .[] else error($key + " is not an array") end + else empty end; + +def required_entries($key): + if has($key) and (.[$key] | type == "array" and length > 0) + then .[$key][] else error($key + " has no explicit entries") end; + +def optional($key; parser): + if has($key) then .[$key] | parser else empty end; + +def no_cached_index: + if has("index") and .index != -1 then error("cached location index is unresolved") else . end; + +def artifact: + object_value | no_cached_index | + if (.uri | type == "string" and length > 0 and (test("[\u0000-\u001f\u007f]") | not)) and + ((has("uriBaseId") | not) or .uriBaseId == "%SRCROOT%") + then . else error("artifact URI or source base is missing or unsupported") end; + +def physical: + object_value | + if has("address") then error("address-based source evidence is unsupported") + else .artifactLocation | artifact end; + +def location: + object_value | + # A logical name cannot qualify a physical file. Cached logical references + # are also not silently resolved against a potentially different run. + if any(entries("logicalLocations"); type != "object" or + (has("index") and .index != -1) or (has("parentIndex") and .parentIndex != -1)) + then error("logical location reference is unresolved") + elif has("relationships") and (.relationships | type != "array" or length > 0) + then error("location relationship references are unresolved") + else .physicalLocation | physical end; + +def stack: + object_value | required_entries("frames") | object_value | .location | location; + +def flow_location: + object_value | no_cached_index | + (.location | location), optional("stack"; stack); + +def code_flow: + object_value | required_entries("threadFlows") | object_value | + required_entries("locations") | flow_location; + +def node: + object_value | + if has("location") or (.children | type == "array" and length > 0) then + optional("location"; location), (entries("children") | node) + else error("graph node has no explicit source location") end; + +def graph: + object_value | required_entries("nodes") | node; + +def fix: + object_value | required_entries("artifactChanges") | object_value | .artifactLocation | artifact; + +def provenance: + object_value | entries("conversionSources") | physical; + +def extension_boundary: + # Standard property-bag tags carry no location. Other opaque extensions, + # runtime request/response evidence and graph-index traversals need review. + if any(.. | objects; + (has("properties") and + (.properties | type != "object" or ((keys - ["tags"]) | length > 0) or + (has("tags") and (.tags | type != "array" or any(.[]; type != "string"))))) or + has("webRequest") or has("webResponse")) + then error("opaque extension or runtime location evidence is unsupported") + elif has("graphTraversals") and (.graphTraversals | type != "array" or length > 0) + then error("graph traversal references are unresolved") + else . end; + +def result_locations: + object_value | extension_boundary | + # Unknown result fields are not assumed to be location-free extensions. + if (keys - ["ruleId","ruleIndex","rule","kind","level","message","analysisTarget", + "locations","guid","correlationGuid","occurrenceCount","partialFingerprints", + "fingerprints","stacks","codeFlows","graphs","graphTraversals","relatedLocations", + "suppressions","baselineState","rank","attachments","hostedViewerUri","workItemUris", + "provenance","fixes","taxa","webRequest","webResponse","properties"] | length > 0) + then error("unknown result field requires location review") else . end | + (entries("locations") | location), + (entries("relatedLocations") | location), + optional("analysisTarget"; artifact), + (entries("codeFlows") | code_flow), + (entries("stacks") | stack), + (entries("graphs") | graph), + (entries("fixes") | fix), + (entries("attachments") | object_value | .artifactLocation | artifact), + (entries("suppressions") | object_value | optional("location"; location)), + optional("provenance"; provenance); + +try ([result_locations] | + if length == 0 then error("result has no explicit source locations") + else {complete:true, artifact_locations:., reason:null} end) +catch {complete:false, artifact_locations:[], reason:.} diff --git a/scripts/preservation-scope.sh b/scripts/preservation-scope.sh index 9646504..08ed19f 100755 --- a/scripts/preservation-scope.sh +++ b/scripts/preservation-scope.sh @@ -43,7 +43,10 @@ preserved_source_area() { preserved_source_matches() { local path=$1 old current [[ -n "$path" && "$path" != /* && "$path" != *$'\n'* && "$path" != *$'\r'* ]] || return 1 - case "/$path/" in */../*|*/./*|*//*|*%*|*\\*) return 1 ;; esac + case "/$path/" in + */../*|*/./*|*//*|*%*|*\\*) return 1 ;; + *) ;; # A canonical relative path proceeds to exact object comparison. + esac preserved_source_area "$path" || return 1 old=${PRESERVED_OBJECTS["$path"]-}; current=${SUBJECT_OBJECTS["$path"]-} [[ -n "$old" && "$old" == "$current" && "$old" =~ ^100(644|755)\ blob\ [0-9a-f]{40}$ ]] diff --git a/scripts/review-preservation-sarif.sh b/scripts/review-preservation-sarif.sh index 6005e89..2a32eae 100755 --- a/scripts/review-preservation-sarif.sh +++ b/scripts/review-preservation-sarif.sh @@ -16,18 +16,21 @@ if [[ "$reporter" == scorecard ]]; then esac fi command -v jq >/dev/null || { echo 'jq is required' >&2; exit 2; } -source "$(dirname -- "${BASH_SOURCE[0]}")/preservation-scope.sh" +checker_dir=$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd) +source "$checker_dir/preservation-scope.sh" preservation_scope_init "$@" || exit 2 [[ -f "$input" && ! -L "$input" && -s "$input" ]] || { echo 'Missing or unsafe SARIF input' >&2; exit 2; } if ! jq -se --arg reporter "$reporter" ' length == 1 and (.[0] | .version == "2.1.0" and (.runs | type == "array" and length > 0) and + (.inlineExternalProperties == null or .inlineExternalProperties == []) and all(.runs[]; (.tool.driver.name | type == "string" and length > 0) and (if $reporter == "codeql" then .tool.driver.name == "CodeQL" elif $reporter == "scorecard" then .tool.driver.name == "Scorecard" else (.tool.driver.name | ascii_downcase | startswith("semgrep")) end) and (.results | type == "array") and + (.externalPropertyFileReferences == null or .externalPropertyFileReferences == {}) and all(.invocations[]?; .executionSuccessful == true and all(.toolExecutionNotifications[]?; .level != "error")) and all(.results[]; @@ -50,20 +53,26 @@ while IFS= read -r result; do rule=$(jq -r '.ruleId' <<< "$result") disposition=actionable reason='Current, changed, unlocated or unqualified source requires review.' - location_count=$(jq '(.locations // []) | length' <<< "$result") + if ! locations=$(jq -c -f "$checker_dir/preservation-sarif-locations.jq" <<< "$result"); then + echo 'Cannot evaluate complete SARIF location evidence' >&2; exit 2 + fi + location_count=$(jq '.artifact_locations | length' <<< "$locations") + if [[ $(jq -r '.complete' <<< "$locations") != true ]]; then + reason="Unqualified location evidence: $(jq -r '.reason' <<< "$locations")" + fi # Only this observed repository-level condition has a staging disposition. # Missing security policy and every other Scorecard rule remain actionable. if [[ "$reporter" == scorecard && "$rule" == FuzzingID ]] && - jq -e '.message.text | startswith("score is 0: project is not fuzzed:")' <<< "$result" >/dev/null && - jq -e '(.locations // []) | all(.[]; .physicalLocation.artifactLocation.uri == "no file associated with this alert")' <<< "$result" >/dev/null; then + jq -e '(.message.text | startswith("score is 0: project is not fuzzed:")) and + (.locations | type == "array" and length > 0)' <<< "$result" >/dev/null && + jq -e '.complete and (.artifact_locations | length > 0 and all(.[]; .uri == "no file associated with this alert"))' <<< "$locations" >/dev/null; then disposition=preservation-advisory reason='No supported executable release is provided by this preservation repository; no fuzzing qualification is claimed. A selected runtime successor must qualify its own fuzzing.' advisory=$((advisory + 1)) elif [[ "$analysis_scope" == checked-out-subject ]] && (( location_count > 0 )); then all_preserved=true for (( index=0; index 0)' <<< "$result"); then + if ! path=$(jq -er --argjson i "$index" '.artifact_locations[$i].uri' <<< "$locations"); then all_preserved=false; break fi path=${path#./} @@ -77,7 +86,8 @@ while IFS= read -r result; do fi [[ "$disposition" != actionable ]] || actionable=$((actionable + 1)) jq -cn --arg reporter "$reporter" --arg disposition "$disposition" --arg reason "$reason" \ - --argjson result "$result" '{reporter:$reporter,disposition:$disposition,reason:$reason,result:$result}' >> "$rows" + --argjson locations "$locations" --argjson result "$result" \ + '{reporter:$reporter,disposition:$disposition,reason:$reason,location_evidence:$locations,result:$result}' >> "$rows" done < "$results" jq -s --arg reporter "$reporter" --arg subject "$SUBJECT_SHA" --arg import "$PRESERVED_IMPORT" \ --arg analysis_scope "$analysis_scope" \ diff --git a/scripts/verify-archived-workflows.sh b/scripts/verify-archived-workflows.sh new file mode 100755 index 0000000..a2c292f --- /dev/null +++ b/scripts/verify-archived-workflows.sh @@ -0,0 +1,29 @@ +#!/usr/bin/env bash +# SPDX-License-Identifier: MPL-2.0-or-later +# Original Git blob IDs preserve exact bytes; they are object identity, not +# cryptographic security attestations. Parent commits are recorded in the ledger. +set -euo pipefail +(( $# == 0 )) || { echo 'usage: verify-archived-workflows.sh' >&2; exit 2; } +command -v git >/dev/null || { echo 'git is required for archive object identity' >&2; exit 2; } +failed=0 +count=0 +while read -r expected path; do + count=$((count + 1)) + if [[ ! -f "$path" || -L "$path" ]]; then + echo "Archived workflow is missing or not a regular file: $path" >&2 + failed=1; continue + fi + actual=$(git hash-object --no-filters -- "$path") || exit 2 + if [[ "$actual" != "$expected" ]]; then + echo "Archived workflow bytes differ from the original Git blob: $path" >&2 + failed=1 + fi +done <<'BLOBS' +10835026204a248fd671748b2215b1bccc38a771 docs/legacy-workflows/ci.yml.txt +a865df41062cb761bbeb7a7f04b9e7e91e75deb5 docs/legacy-workflows/deploy-casket.yml.txt +8d4ed07ca40fb96dd5be39622a9614256d9b42a1 docs/legacy-workflows/deploy-ddraig.yml.txt +3c784538bf4c1b434ec913874ba4b45e7f91b185 docs/legacy-workflows/instant-sync.yml.txt +a7b1547af5637dacea5690deb0c1d492d7b56370 docs/legacy-workflows/mirror.yml.txt +BLOBS +(( failed == 0 )) || exit 1 +printf 'Verified %s archived workflows against their exact original Git blobs.\n' "$count" diff --git a/tests/archived-workflow-controls.sh b/tests/archived-workflow-controls.sh new file mode 100755 index 0000000..226b32d --- /dev/null +++ b/tests/archived-workflow-controls.sh @@ -0,0 +1,31 @@ +#!/usr/bin/env bash +# SPDX-License-Identifier: MPL-2.0-or-later +set -euo pipefail +root=$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/.." && pwd) +tmp=$(mktemp -d) +trap 'rm -rf -- "$tmp"' EXIT +mkdir -p "$tmp/docs" +cp -R "$root/docs/legacy-workflows" "$tmp/docs/" +cd "$tmp" +verifier=(bash "$root/scripts/verify-archived-workflows.sh") +"${verifier[@]}" > result +grep -Fq 'Verified 5 archived workflows' result +reject() { + local marker=$1 status=0 + "${verifier[@]}" > result 2>&1 || status=$? + if (( status != 1 )) || ! grep -Fq "$marker" result; then + cat result >&2; echo 'Expected archive identity rejection was absent' >&2; exit 1 + fi + printf 'rejected: %s\n' "$marker" +} +for path in docs/legacy-workflows/*.yml.txt; do + printf '\n' >> "$path" + reject "Archived workflow bytes differ from the original Git blob: $path" + cp "$root/$path" "$path" +done +path=docs/legacy-workflows/ci.yml.txt +rm "$path" +reject "Archived workflow is missing or not a regular file: $path" +ln -s "$root/$path" "$path" +reject "Archived workflow is missing or not a regular file: $path" +echo 'Archive controls passed: original bytes accepted; appended bytes, missing files and symlinks rejected.' diff --git a/tests/preservation-policy-controls.sh b/tests/preservation-policy-controls.sh index f64d639..f8a51fc 100755 --- a/tests/preservation-policy-controls.sh +++ b/tests/preservation-policy-controls.sh @@ -10,10 +10,14 @@ export GIT_AUTHOR_EMAIL=fixture@example.invalid GIT_COMMITTER_EMAIL=fixture@exam git init -q "$tmp/repo" cd "$tmp/repo" mkdir -p engines/example scripts -printf '%s\n' '# retained source fixture' > engines/example/archive.py -printf '%s\n' '# active automation fixture' > scripts/active.sh +retained_path=engines/example/archive.py +active_path=scripts/active.sh +printf '%s\n' '# retained source fixture' > "$retained_path" +printf '%s\n' '# active automation fixture' > "$active_path" git add .; git commit -qm import base=$(git rev-parse HEAD); tree=$(git rev-parse HEAD^{tree}) +object_violation='VIOLATION: prohibited source object differs in path, blob, mode or retained area' +current_finding='current or unqualified finding(s)' expect() { local wanted=$1 marker=$2; shift 2 local actual=0 @@ -25,59 +29,144 @@ expect() { } policy=(bash "$root/scripts/check-preserved-language-policy.sh" "$base" "$tree") expect 0 'historical=1 violations=0' "${policy[@]}" -printf '%s\n' '# changed historical source' >> engines/example/archive.py +printf '%s\n' '# changed historical source' >> "$retained_path" git add .; git commit -qm changed -expect 1 'VIOLATION: new, changed, relocated or active prohibited source' "${policy[@]}" +expect 1 "$object_violation" "${policy[@]}" git reset --hard -q "$base" -cp engines/example/archive.py engines/example/new.py +cp "$retained_path" engines/example/new.py git add .; git commit -qm copied -expect 1 'VIOLATION: new, changed, relocated or active prohibited source' "${policy[@]}" +expect 1 "$object_violation" "${policy[@]}" git reset --hard -q "$base" -chmod +x engines/example/archive.py +chmod +x "$retained_path" git add .; git commit -qm mode -expect 1 'VIOLATION: new, changed, relocated or active prohibited source' "${policy[@]}" +expect 1 "$object_violation" "${policy[@]}" git reset --hard -q "$base" -git mv engines/example/archive.py scripts/relocated.py +git mv "$retained_path" scripts/relocated.py git commit -qm moved -expect 1 'VIOLATION: new, changed, relocated or active prohibited source' "${policy[@]}" +expect 1 "$object_violation" "${policy[@]}" git reset --hard -q "$base" expect 2 'imported tree differs' bash "$root/scripts/check-preserved-language-policy.sh" "$base" "0000000000000000000000000000000000000000" +# Record the actual limit: an allowed wrapper does not alter a retained source +# object. This gate must not claim it proved the wrapper cannot execute it. +printf '%s\n' '#!/usr/bin/env bash' 'python engines/example/archive.py' > scripts/wrapper.sh +git add .; git commit -qm wrapper +expect 0 'Execution references and activation are not analyzed' "${policy[@]}" +grep -Fq 'historical=1 violations=0' "$tmp/output" +git reset --hard -q "$base" sarif="$tmp/result.sarif" make_result() { - jq -n --arg tool "$1" --arg rule "$2" --arg uri "$3" --arg message "${4:-fixture finding}" \ + local tool=$1 rule=$2 uri=$3 message=${4:-fixture finding} + jq -n --arg tool "$tool" --arg rule "$rule" --arg uri "$uri" --arg message "$message" \ '{version:"2.1.0",runs:[{tool:{driver:{name:$tool}},results:[{ruleId:$rule,level:"warning",message:{text:$message},locations:[{physicalLocation:{artifactLocation:{uri:$uri}}}]}]}]}' > "$sarif" } review=(bash "$root/scripts/review-preservation-sarif.sh") export SCORECARD_ANALYSIS_SCOPE=checked-out-subject -make_result CodeQL fixture engines/example/archive.py +make_result CodeQL fixture "$retained_path" original=$(sha256sum "$sarif") expect 0 'Exact retained source' "${review[@]}" codeql "$sarif" "$tmp/receipt" "$base" "$tree" [[ $(sha256sum "$sarif") == "$original" ]] jq -e '.counts.total == 1 and .counts.retained_historical == 1 and .security_clean_claim == false' "$tmp/receipt.json" >/dev/null -make_result CodeQL fixture scripts/active.sh -expect 1 'current or unqualified finding(s)' "${review[@]}" codeql "$sarif" "$tmp/receipt" "$base" "$tree" +make_result CodeQL fixture "$active_path" +expect 1 "$current_finding" "${review[@]}" codeql "$sarif" "$tmp/receipt" "$base" "$tree" make_result CodeQL fixture ../engines/example/archive.py -expect 1 'current or unqualified finding(s)' "${review[@]}" codeql "$sarif" "$tmp/receipt" "$base" "$tree" -make_result CodeQL fixture engines/example/archive.py -printf '%s\n' '# current mutation' >> engines/example/archive.py +expect 1 "$current_finding" "${review[@]}" codeql "$sarif" "$tmp/receipt" "$base" "$tree" +make_result CodeQL fixture "$retained_path" +printf '%s\n' '# current mutation' >> "$retained_path" git add .; git commit -qm current -expect 1 'current or unqualified finding(s)' "${review[@]}" codeql "$sarif" "$tmp/receipt" "$base" "$tree" +expect 1 "$current_finding" "${review[@]}" codeql "$sarif" "$tmp/receipt" "$base" "$tree" git reset --hard -q "$base" make_result Scorecard FuzzingID 'no file associated with this alert' 'score is 0: project is not fuzzed: fixture' expect 0 'Preservation advisory' "${review[@]}" scorecard "$sarif" "$tmp/receipt" "$base" "$tree" jq -e '.counts.preservation_advisory == 1 and .counts.actionable == 0' "$tmp/receipt.json" >/dev/null make_result Scorecard SecurityPolicyID 'no file associated with this alert' -expect 1 'current or unqualified finding(s)' "${review[@]}" scorecard "$sarif" "$tmp/receipt" "$base" "$tree" +expect 1 "$current_finding" "${review[@]}" scorecard "$sarif" "$tmp/receipt" "$base" "$tree" make_result Scorecard FuzzingID 'no file associated with this alert' 'a different failure' -expect 1 'current or unqualified finding(s)' "${review[@]}" scorecard "$sarif" "$tmp/receipt" "$base" "$tree" -make_result Scorecard fixture engines/example/archive.py -SCORECARD_ANALYSIS_SCOPE=remote-default-branch expect 1 'current or unqualified finding(s)' "${review[@]}" scorecard "$sarif" "$tmp/receipt" "$base" "$tree" +expect 1 "$current_finding" "${review[@]}" scorecard "$sarif" "$tmp/receipt" "$base" "$tree" +make_result Scorecard fixture "$retained_path" +SCORECARD_ANALYSIS_SCOPE=remote-default-branch expect 1 "$current_finding" "${review[@]}" scorecard "$sarif" "$tmp/receipt" "$base" "$tree" jq -e '.analysis_subject_sha == null and .analysis_scope == "remote-default-branch"' "$tmp/receipt.json" >/dev/null -make_result CodeQL fixture engines/example/archive.py + +extend_result() { + local expression=$1 uri=${2:-$active_path} + jq --arg uri "$uri" 'def loc: {physicalLocation:{artifactLocation:{uri:$uri}}}; + .runs[0].results[0] |= ('"$expression"')' "$sarif" > "$tmp/extended.sarif" + mv "$tmp/extended.sarif" "$sarif" +} +review_source_fixture() { + local status=$1 marker=$2 + local before + before=$(sha256sum "$sarif") + expect "$status" "$marker" "${review[@]}" codeql "$sarif" "$tmp/receipt" "$base" "$tree" + [[ $(sha256sum "$sarif") == "$before" ]] + jq -e --slurpfile raw "$sarif" \ + '[.findings[].result] == [$raw[0].runs[].results[]]' "$tmp/receipt.json" >/dev/null +} + +# Every supported source-bearing shape gets both a retained-source positive +# and a mixed current/historical negative. Primary locations alone are not enough. +while IFS= read -r expression; do + printf 'location control: %s\n' "$expression" + make_result CodeQL fixture "$retained_path" + extend_result "$expression" "$retained_path" + review_source_fixture 0 'Exact retained source' + jq -e '.findings[0].location_evidence.complete and (.findings[0].location_evidence.artifact_locations | length > 1)' "$tmp/receipt.json" >/dev/null + make_result CodeQL fixture "$retained_path" + extend_result "$expression" "$active_path" + review_source_fixture 1 "$current_finding" +done <<'JQ' +.relatedLocations=[loc] +.analysisTarget={uri:$uri} +.codeFlows=[{threadFlows:[{locations:[{location:loc}]}]}] +.codeFlows=[{threadFlows:[{locations:[{location:.locations[0],stack:{frames:[{location:loc}]}}]}]}] +.stacks=[{frames:[{location:loc}]}] +.graphs=[{nodes:[{id:"parent",children:[{id:"child",location:loc}]}]}] +.fixes=[{artifactChanges:[{artifactLocation:{uri:$uri},replacements:[]}]}] +.attachments=[{artifactLocation:{uri:$uri}}] +.suppressions=[{kind:"inSource",location:loc}] +.provenance={conversionSources:[loc.physicalLocation]} +JQ + +# An unsupported/missing secondary reference must not disappear from traversal. +while IFS= read -r expression; do + printf 'unresolved location control: %s\n' "$expression" + make_result CodeQL fixture "$retained_path" + extend_result "$expression" + review_source_fixture 1 "$current_finding" + jq -e '.findings[0].location_evidence.complete == false' "$tmp/receipt.json" >/dev/null +done <<'JQ' +.relatedLocations=[{}] +.relatedLocations=[{logicalLocations:[{name:"unknown"}]}] +.relatedLocations=[{physicalLocation:{artifactLocation:{index:0}}}] +.relatedLocations=[{physicalLocation:{artifactLocation:{uri:$uri,uriBaseId:"UNKNOWN"}}}] +.relatedLocations=null +.locations[0].physicalLocation.artifactLocation.index=0 +.locations[0].physicalLocation.address={index:0} +.locations[0].relationships=[{target:9}] +.codeFlows=[{threadFlows:[{locations:[{index:0}]}]}] +.codeFlows=[{threadFlows:[]}] +.stacks=[{frames:[{}]}] +.fixes=[{artifactChanges:[{}]}] +.attachments=[{}] +.graphs=[{nodes:[{id:"unknown"}]}] +.graphTraversals=[{runGraphIndex:0}] +.properties={sourceLocation:loc} +.customLocation=loc +.relatedLocations=[{physicalLocation:{artifactLocation:{uri:"engines/example/archive.py\nother"}}}] +JQ + +make_result Scorecard FuzzingID 'no file associated with this alert' 'score is 0: project is not fuzzed: fixture' +extend_result '.relatedLocations=[loc]' +expect 1 "$current_finding" "${review[@]}" scorecard "$sarif" "$tmp/receipt" "$base" "$tree" +make_result Scorecard FuzzingID 'no file associated with this alert' 'score is 0: project is not fuzzed: fixture' +extend_result '.locations=[]' +expect 1 "$current_finding" "${review[@]}" scorecard "$sarif" "$tmp/receipt" "$base" "$tree" +make_result CodeQL fixture "$retained_path" jq '.runs[0].invocations=[{executionSuccessful:false}]' "$sarif" > "$tmp/bad.sarif" expect 2 'Malformed, failed or mismatched' "${review[@]}" codeql "$tmp/bad.sarif" "$tmp/receipt" "$base" "$tree" expect 2 'Malformed, failed or mismatched' "${review[@]}" semgrep "$sarif" "$tmp/receipt" "$base" "$tree" +jq '.runs[0].externalPropertyFileReferences={results:[{location:{uri:"external.sarif"}}]}' "$sarif" > "$tmp/bad.sarif" +expect 2 'Malformed, failed or mismatched' "${review[@]}" codeql "$tmp/bad.sarif" "$tmp/receipt" "$base" "$tree" printf '{' > "$tmp/bad.sarif" expect 2 'Malformed, failed or mismatched' "${review[@]}" codeql "$tmp/bad.sarif" "$tmp/receipt" "$base" "$tree" expect 2 'Missing or unsafe SARIF input' "${review[@]}" codeql "$tmp/absent.sarif" "$tmp/receipt" "$base" "$tree" diff --git a/tests/workflow-contract-controls.sh b/tests/workflow-contract-controls.sh index 854625e..1fff6ae 100755 --- a/tests/workflow-contract-controls.sh +++ b/tests/workflow-contract-controls.sh @@ -18,8 +18,10 @@ jobs: runs-on: ubuntu-24.04 steps: # uses: fabricated/comment@main + # image: fabricated/container:latest - run: | echo 'uses: string/not-an-action@main' + echo 'container: string/not-an-image:latest' - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 - uses: ./local-action YAML @@ -27,9 +29,16 @@ bash "$root/scripts/lint-workflow-contracts.sh" > "$tmp/good" grep -Fq 'Parsed 1 workflows' "$tmp/good" cp "$fixture" "$tmp/original" reject() { + local marker=${1:-Unpinned action} local actual=0 bash "$root/scripts/lint-workflow-contracts.sh" > "$tmp/result" 2>&1 || actual=$? - [[ "$actual" == 1 ]] && grep -Fq 'Unpinned action' "$tmp/result" || { cat "$tmp/result" >&2; exit 1; } + [[ "$actual" == 1 ]] && grep -Fq "$marker" "$tmp/result" || { cat "$tmp/result" >&2; exit 1; } + printf 'rejected: %s\n' "$marker" + cp "$tmp/original" "$fixture" +} +accept() { + bash "$root/scripts/lint-workflow-contracts.sh" > "$tmp/result" 2>&1 || { cat "$tmp/result" >&2; exit 1; } + grep -Fq 'container image digests passed' "$tmp/result" cp "$tmp/original" "$fixture" } sed -i 's/@3d3c42e5aac5ba805825da76410c181273ba90b1/@main/' "$fixture" @@ -43,4 +52,42 @@ cat >> "$fixture" <<'YAML' uses: example/source/.github/workflows/check.yml@main YAML reject -echo 'Workflow parser controls passed: actual unpinned refs rejected; comments and run strings are not action refs.' + +# The literals are fixtures for pin syntax, not claims about real image content. +export WORKFLOW_TEST_IMAGE='example/tool:V1@sha256:1111111111111111111111111111111111111111111111111111111111111111' +yq -i '.jobs.fixture.container = strenv(WORKFLOW_TEST_IMAGE)' "$fixture" +accept +yq -i '.jobs.fixture.container = {"image":strenv(WORKFLOW_TEST_IMAGE)}' "$fixture" +accept +yq -i '.jobs.fixture.services = {"db":{"image":strenv(WORKFLOW_TEST_IMAGE)}}' "$fixture" +accept +yq -i '.jobs.fixture.services = {}' "$fixture" +accept +for expression in \ + '.jobs.fixture.container = "example/tool:latest"' \ + '.jobs.fixture.container = {"image":"example/tool:v1"}' \ + '.jobs.fixture.services = {"db":{"image":"example/db:latest"}}' \ + '.jobs.fixture.container = "${{ matrix.image }}"' \ + '.jobs.fixture.container = "__container_population_complete__"' \ + '.jobs.fixture.container = "example/tool@sha256:abcd"'; do + yq -i "$expression" "$fixture" + reject 'Unpinned job or service container image' +done +for expression in \ + '.jobs.fixture.container = null' \ + '.jobs.fixture.container = {}' \ + '.jobs.fixture.container = {"image":[]}' \ + '.jobs.fixture.services = null' \ + '.jobs.fixture.services = {"db":{}}' \ + '.jobs.fixture.services = {"db":strenv(WORKFLOW_TEST_IMAGE)}' \ + '.jobs.fixture.container = strenv(WORKFLOW_TEST_IMAGE) + "\n" + strenv(WORKFLOW_TEST_IMAGE)'; do + yq -i "$expression" "$fixture" + reject 'Malformed job or service container' +done +yq -i '.jobs.fixture.steps += [{"uses":"docker://example/tool:latest"}]' "$fixture" +reject 'Unpinned container action' +yq -i '.jobs.fixture.steps += [{"uses":"docker://" + strenv(WORKFLOW_TEST_IMAGE)}]' "$fixture" +accept +yq -i '.jobs.fixture.steps += [{"uses":"./a/../b"}]' "$fixture" +reject 'Invalid local action path' +echo 'Workflow parser controls passed: action refs and job/service image pins checked; comments and run strings remain inert.' From ef237d5320a713be33bef0466831e29d470e0f13 Mon Sep 17 00:00:00 2001 From: "Jonathan D.A. Jewell" <6759885+hyperpolymath@users.noreply.github.com> Date: Wed, 4 Nov 2026 20:09:43 +0000 Subject: [PATCH 7/7] test(preservation): name the no-file advisory fixture Use one named constant for the observed Scorecard placeholder while retaining all missing-location and mixed-source rejection controls. --- tests/preservation-policy-controls.sh | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/tests/preservation-policy-controls.sh b/tests/preservation-policy-controls.sh index f8a51fc..35b0910 100755 --- a/tests/preservation-policy-controls.sh +++ b/tests/preservation-policy-controls.sh @@ -18,6 +18,7 @@ git add .; git commit -qm import base=$(git rev-parse HEAD); tree=$(git rev-parse HEAD^{tree}) object_violation='VIOLATION: prohibited source object differs in path, blob, mode or retained area' current_finding='current or unqualified finding(s)' +no_file_location='no file associated with this alert' expect() { local wanted=$1 marker=$2; shift 2 local actual=0 @@ -76,12 +77,12 @@ printf '%s\n' '# current mutation' >> "$retained_path" git add .; git commit -qm current expect 1 "$current_finding" "${review[@]}" codeql "$sarif" "$tmp/receipt" "$base" "$tree" git reset --hard -q "$base" -make_result Scorecard FuzzingID 'no file associated with this alert' 'score is 0: project is not fuzzed: fixture' +make_result Scorecard FuzzingID "$no_file_location" 'score is 0: project is not fuzzed: fixture' expect 0 'Preservation advisory' "${review[@]}" scorecard "$sarif" "$tmp/receipt" "$base" "$tree" jq -e '.counts.preservation_advisory == 1 and .counts.actionable == 0' "$tmp/receipt.json" >/dev/null -make_result Scorecard SecurityPolicyID 'no file associated with this alert' +make_result Scorecard SecurityPolicyID "$no_file_location" expect 1 "$current_finding" "${review[@]}" scorecard "$sarif" "$tmp/receipt" "$base" "$tree" -make_result Scorecard FuzzingID 'no file associated with this alert' 'a different failure' +make_result Scorecard FuzzingID "$no_file_location" 'a different failure' expect 1 "$current_finding" "${review[@]}" scorecard "$sarif" "$tmp/receipt" "$base" "$tree" make_result Scorecard fixture "$retained_path" SCORECARD_ANALYSIS_SCOPE=remote-default-branch expect 1 "$current_finding" "${review[@]}" scorecard "$sarif" "$tmp/receipt" "$base" "$tree" @@ -155,10 +156,10 @@ done <<'JQ' .relatedLocations=[{physicalLocation:{artifactLocation:{uri:"engines/example/archive.py\nother"}}}] JQ -make_result Scorecard FuzzingID 'no file associated with this alert' 'score is 0: project is not fuzzed: fixture' +make_result Scorecard FuzzingID "$no_file_location" 'score is 0: project is not fuzzed: fixture' extend_result '.relatedLocations=[loc]' expect 1 "$current_finding" "${review[@]}" scorecard "$sarif" "$tmp/receipt" "$base" "$tree" -make_result Scorecard FuzzingID 'no file associated with this alert' 'score is 0: project is not fuzzed: fixture' +make_result Scorecard FuzzingID "$no_file_location" 'score is 0: project is not fuzzed: fixture' extend_result '.locations=[]' expect 1 "$current_finding" "${review[@]}" scorecard "$sarif" "$tmp/receipt" "$base" "$tree" make_result CodeQL fixture "$retained_path"