Promote the Hub-Hosted Secret and Env Readers to Main

.github/actions/repo-gate/action.yml

@@ -1,5 +1,5 @@
 name: Repository gate
-description: Check action pins, line-ending contracts, and composite action shell and schema.
+description: Check action pins, line-ending contracts, composite action shell and schema, and carried tree currency.
 
 inputs:
   exclude-globs:

.github/actions/repo-gate/repo_gate.py

@@ -8,6 +8,7 @@
   eol           .editorconfig <-> .gitattributes disagreement  40 findings
   eol-coverage  Git attribute resolution differs from policy   count not recorded
   composite-actions  shellcheck and schema coverage of action.yml  count not recorded
+  carried-trees  a manifest-owned tree differing from the hub's  count not recorded
 
 `eol-coverage` asks Git how representative paths resolve. This proves the global text default
 reaches Python, shell, Dockerfiles, workflow YAML, and extensionless scripts without maintaining
@@ -35,12 +36,15 @@
 from __future__ import annotations
 
 import argparse
+import importlib.util
 import json
+import os
 import re
 import shutil
 import subprocess
 import sys
 from pathlib import Path
+from types import ModuleType
 
 # GOVERNANCE.md documents exactly one floating-ref exception.
 SHA_EXCEPTIONS = {"dotnet/nbgv"}
@@ -769,11 +773,146 @@ def check_composite_actions(root: Path, files: list[str]) -> list[str]:
     return hits
 
 
+def gate_hub_root() -> Path | None:
+    """The hub tree this gate file sits in, or None where it was lifted out of one.
+
+    The pinned action runs from the whole hub tree at its pin, and so does a run from a hub
+    checkout. A single-file fetch has no tree around it, and whatever happens to sit three levels
+    above the file is not the hub, so the path is trusted only where its own parents name it.
+    """
+    here = Path(__file__).resolve()
+    if [part.name for part in here.parents[:3]] != ["repo-gate", "actions", ".github"]:
+        return None
+    root = here.parents[3]
+    return root if (root / "spec" / "files.json").is_file() else None
+
+
+def hub_carry(hub: Path) -> ModuleType:
+    """The hub's own `scripts/carry.py`, whose tree inventory and comparison this check reuses.
+
+    It imports only the standard library until a section is read, so plain `python3` loads it.
+    """
+    spec = importlib.util.spec_from_file_location("carry", hub / "scripts" / "carry.py")
+    assert spec is not None and spec.loader is not None
+    module = importlib.util.module_from_spec(spec)
+    spec.loader.exec_module(module)
+    return module
+
+
+def warn_carried_tree(message: str) -> None:
+    """Show `message` on the pull request under Actions, built from manifest values alone so nothing a scanned repository names reaches the command."""
+    if os.environ.get("GITHUB_ACTIONS") == "true":
+        print(f"::warning title=carried-trees::{message}")
+
+
+def check_carried_trees(root: Path, files: list[str]) -> list[str]:
+    """Report each manifest-owned tree the scanned repository carries that differs from the hub's.
+
+    `spec/files.json` declares the trees a repository carries byte-for-byte, and a carried copy
+    goes stale silently at the next hub fix. Comparing against the hub tree this gate runs from
+    makes the answer reproducible at the action's pin, so a Dependabot bump of that pin is what
+    surfaces a refresh the resync then applies.
+
+    A difference is a note rather than a finding, so it never fails the gate. Failing would break
+    every downstream pull request on each hub fix, and a byte difference cannot tell a stale copy
+    from an edited one, which the audit tells apart. Which trees a repository owes is the audit's
+    too, read from its declared types, so a tree absent here is not compared. The copy is read
+    from its tracked files alone, so an untracked file is not a difference.
+    """
+    hub = gate_hub_root()
+    if hub is None:
+        NOTES.append(
+            "no hub tree is beside this gate, as with a single-file fetch, "
+            "so no carried tree was compared."
+        )
+        return []
+    # A hub tracks its manifest, which no carrying repository does, so a hub worktree on another branch is not read as a stale copy.
+    if "spec/files.json" in files:
+        NOTES.append("the scanned repository is a hub, whose trees are sources rather than copies.")
+        return []
+    # Each failure is a note rather than a traceback, since one here would also skip every check sorted after this one.
+    # The catches are broad because carry.py runs under whatever `python3` the host has, so a newer interpreter's API in it fails here as any exception.
+    try:
+        carry = hub_carry(hub)
+    except Exception as error:  # noqa: BLE001
+        NOTES.append(
+            f"the hub's scripts/carry.py could not be loaded, so no carried tree was compared: {error}"
+        )
+        return []
+    try:
+        trees = carry.load_json(hub / "spec" / "files.json")["trees"]
+    except Exception as error:  # noqa: BLE001
+        NOTES.append(f"no carried tree was compared: {error}")
+        return []
+    carried = 0
+    for tree in trees:
+        target = tree["target"]
+        prefix = target + "/"
+        # Git tracks a link to a directory as one path, so a target or an ancestor tracked that way hides every file beneath it.
+        # A top-level ancestor such as `lib` is generic, so one tracked that way says nothing about a carry and is passed over like an uncarried tree.
+        blocking = next(
+            (
+                path
+                for path in files
+                if (path == target or "/" in path) and prefix.startswith(path + "/")
+            ),
+            None,
+        )
+        if blocking is not None:
+            warn_carried_tree(
+                f"{target} sits at or below a tracked link or file, which a carry refuses"
+            )
+            NOTES.append(
+                f"{target} was not compared, since {blocking} is tracked as a link or a file."
+            )
+            continue
+        tracked_here = [path[len(prefix) :] for path in files if path.startswith(prefix)]
+        if not tracked_here:
+            continue
+        try:
+            owned = [p for p in tracked_here if carry.included(p, carry.target_patterns(tree))]
+            # A tree that does not prune shares its target with the repository's own files, so those alone do not make it carried.
+            if not owned:
+                continue
+            # A carry refuses a link anywhere under a tree's target, so the gate refuses one too rather than judging it by a rule of its own.
+            link = next((p for p in sorted(tracked_here) if (root / target / p).is_symlink()), None)
+            if link is not None:
+                warn_carried_tree(f"{target} holds a link, which a carry refuses")
+                NOTES.append(
+                    f"{target} could not be compared, since {link} is a link, which a carry refuses."
+                )
+                continue
+            # A tracked file absent from the working tree is left out, so it reads as missing.
+            copy = {
+                relative: (root / target / relative).read_bytes()
+                for relative in owned
+                if (root / target / relative).is_file()
+            }
+            source = carry.inventory(hub / tree["source"], tree["include"])
+            result = carry.compare_declaration(tree, source, carry.Inventory(copy, frozenset(), ""))
+        except Exception as error:  # noqa: BLE001
+            NOTES.append(f"{target} could not be compared: {error}")
+            continue
+        carried += 1
+        changed, missing, extra = result["modified"], result["missing"], result["extra"]
+        if changed or missing or extra:
+            warn_carried_tree(f"{target} differs from the hub's {tree['source']}")
+            NOTES.append(
+                f"{target} differs from the hub's {tree['source']}: {len(changed)} changed, "
+                f"{len(missing)} missing, {len(extra)} extra, first "
+                f"{(changed + missing + extra)[0]}. Refresh it from the hub, or let the audit say "
+                "whether it is a stale copy or an edited one."
+            )
+    NOTES.append(f"compared {carried} carried tree(s) against the hub tree beside this gate.")
+    return []
+
+
 CHECKS = {
     "sha-pin": check_sha_pin,
     "eol": check_eol,
     "eol-coverage": check_eol_coverage,
     "composite-actions": check_composite_actions,
+    "carried-trees": check_carried_trees,
 }
 
 

.github/workflows/validate-task.yml

@@ -23,7 +23,7 @@ on:
         default: ''
       # Threaded to the repo-gate composite action's own exclude-globs input, unvalidated.
       repo-gate-exclude-globs:
-        description: Git pathspec patterns to exclude, one per line (for example 'themes/PaperMod/**'), narrowing the sha-pin, eol-coverage, and composite-actions scans via repo_gate.py's --exclude.
+        description: Git pathspec patterns to exclude, one per line (for example 'themes/PaperMod/**'), narrowing the sha-pin, eol-coverage, composite-actions, and carried-trees scans via repo_gate.py's --exclude.
         required: false
         type: string
         default: ''

AUDIT.md

@@ -58,6 +58,7 @@ Otherwise read its `types[]`. If the entry is `classificationPending` (a backlog
 - `Dockerfile` + a docker build/push workflow -> `docker`, an `upstream-version.json` tracker -> `upstream-wrapper`.
 - `custom_components/*/manifest.json` + `hacs.json` -> `homeassistant`, a codegen workflow -> `codegen`, no `build-*` task -> `source-only`, governance-only -> `docs`.
 - `hugo.yaml` / `hugo.toml` / `config/_default/hugo.yaml` -> `hugo`. A repo may carry it alongside `source-only`, since a site deploy leaf is not a `build-*` task and both declarations stay true.
+- a carried `lib/fleet/sh/secrets.sh` -> `host-secrets`, and so does a shell script the repo runs that reads a credential file under `.secrets/` or `~/.secrets/`.
 
 ## 3. Applicability Gate
 

GOVERNANCE.md

@@ -123,6 +123,7 @@ A repo whose own stacks or scripts read local runtime credentials from disk, mos
 
   A fresh checkout then documents its own required shape without ever exposing a real value. This negation keeps a real secret file out of a **new** commit. It does not remove one already tracked: `.gitignore` has no effect on a path git already follows. A real secret file found tracked is removed from the index (`git rm --cached <path>`) and its credential is rotated, not just added to `.gitignore` going forward.
 - **`.secrets/README.md` is a catalog**, one row per file naming what it holds and what consumes it, plus a short note on how the directory relates to `~/.secrets/` where the repo also touches that.
+- **A repository whose shell scripts read a credential from disk declares the `host-secrets` type, and reads each one through the carried readers.** A credential is a secret file or an env file holding one, in `.secrets/` or in the host-scoped `~/.secrets/`, and this bullet reaches both. A single inline read counts, since it is the read a bare existence check and a sourced file get wrong. A script sources its carried `lib/fleet/sh/secrets.sh`, per "Hub-Hosted Tooling", and checks the file before reading it. A private file takes `require_secret` or `secret_set_verdict`, and one a container reads as a Docker secret takes `secret_verdict` with `shared`. It reads one value from an env file with `read_env_value` rather than sourcing the file into its own shell.
 
 ## Repository Onboarding and Conformance
 
@@ -139,6 +140,8 @@ The fleet's tooling lives in the hub once and a repository runs it from there ra
 
 **What a repository carries and what it reaches is decided by what the content is.** It carries the content it is audited against and the configuration that describes it, meaning its rule text and the files the manifest declares. It reaches machinery whose content is identical in every repository, meaning the prose and repository gates, the review digest, and the configuration script, because a file holding no per-repo content is a copy whose only future is to go stale. A tool named in a carried rule is therefore named as the hub's, since the alternative is a pointer to a path the reader does not have, and a pointer that resolves nowhere teaches the reader that a pointer in carried text is decorative.
 
+**Code a repository's own runtime executes is the exception, and it is carried.** Such a library runs on a host the project owns, often unattended and sometimes as root. Fetched from the hub at run time, it would let a GitHub outage or a bad `main` commit stop a scheduled host job. It would also run code the repository never reviewed against its own secrets. So offline operation and review in the repository that runs it outweigh the staleness of a copy. A gate differs, since "the tool did not run" is an acceptable answer there and CI is the backstop. The hub hosts the library once. A repository whose declared types select it carries it byte-for-byte, as a tree the hub's `spec/files.json` declares. The hub owns the carried directory whole. The repository extends the library in a file of its own outside it that sources it, never by editing the copy. The staleness is made visible rather than accepted. The repository gate compares each carried tree with the hub tree its pinned action holds. It reports a difference as a warning that never fails the gate. A failure would break every downstream pull request on each hub fix. The audit reports a stale copy, and the resync refreshes it.
+
 **Reaching it is a checkout of the hub rather than a copy of one file.** A tool reads the payloads, tables, and sibling modules beside it, so a single file lifted out of the tree runs against whatever the caller happens to have, which is the copy problem again in a shorter loop. Read `main`, the promoted and gated state, and fetch immediately before running, because a clone is whatever it last fetched rather than the branch it names, and a stale clone answers confidently instead of failing. Name the tool by its path in that checkout and name the target explicitly, since a tool that defaults to the current directory or the current repository resolves somewhere either way, and a result computed against the wrong repository is well-formed. Which directory the command runs in is the tool's own contract rather than a rule here, so a gate reading a working tree runs in the repository under test while a tool taking its target as an argument runs anywhere and is given one. What the rule forbids is letting a default decide which repository the answer is about.
 
 **A loader is outside this section rather than exempt from it.** The rule above governs a tool that reads hub content, because a tool reads the payloads, tables, and sibling modules beside it. A loader reads none of them: it obtains a tree and hands control to a tool inside that tree, on a host that cannot yet obtain one. The bound is what it may contain rather than who runs it, and it is one line: a loader references no path inside the tree it fetches except the single entry point it hands control to, and everything else it touches is the machine or the network. A loader that grows a second path into that tree has become a tool and is governed above.
@@ -374,6 +377,7 @@ Every repo's GitHub repository details (the About panel) follow a fixed conventi
 - [`.github/skills/`](./.github/skills/): generated in the hub from `.agents/skills/` and carried verbatim to every repo, so a fix to a Skill belongs at that source rather than in any copy of this tree.
 - [`.claude-plugin/fleet-skills/`](./.claude-plugin/fleet-skills/): hub-only, the same Skills as a Claude plugin. The `marketplace.json` beside it names the plugin rather than the Skills, is hand-authored, and `build_dist.py` never writes it.
 - [`scripts/`](./scripts/): the fleet's lint and review tooling, with the deterministic checks also gating CI, for the documented rules no CI linter checks. Hosted here and reached rather than carried, per "Hub-Hosted Tooling".
+- [`lib/`](./lib/): runtime libraries the hub hosts and a repository carries byte-for-byte, per "Hub-Hosted Tooling", each under `lib/fleet/` in the repository whose declared types select it. `lib/sh/` is the shell secret and env readers, carried to a `host-secrets` repository.
 - [`tests/`](./tests/): hub-only, the unit tests for the hub's own tooling, which the shared validator runs under coverage as the hub's declared Python directory.
 - [`docs/`](./docs/): hub-only procedure and measurement docs (host setup, repository configuration, signing, agent token cost), none of them carried downstream.
 - [`host-setup/`](./host-setup/): hub-only, the installers that bring a machine up to the host contract `docs/host-setup.md` states, one tree each for Linux and Windows, with the cross-platform agent-safety install beside them.