Slice 17: auto-ingest a witnessed Run's scan file over SSH

[xmap]

Why

The 2-BM pilot has recorded 133 witnessed Runs and zero Datasets. ingest_scan is a POST route plus an MCP tool that a human invokes, by design: run/aggregates/run/capture_path.py keeps the observed path unredacted precisely so an operator can read it to locate the file. That was tenable when scans were supervised one at a time. It is not now, with vertical_lens_series.sh running 510 unattended scans (~24 GB each, ~2 min apart, ~12 TB per series).

CaptureScanIngestor sweeps terminated witnessed Runs whose capture path resolved and issues the existing IngestScan command for each, attributed to a new pinned agent principal, behind its own kill switch.

The bytes do not move

The scan files live on the detector host (tomdet), not on CORA's. Measured 2026-08-18:

Fact	Value
Files in the active scan dir	801, 19.2 TB, mean 24.0 GB
sha256 on tomdet, local disk	~920 MB/s, ~26 s per file
arcturus link to tomdet	1 GbE, measured ~105 MB/s
Pulling one file to arcturus	~230 s vs a ~120 s scan cadence

A mount falls behind ~2x permanently and saturates the same link the witness uses for EPICS. So the computation moves to the bytes instead: the read and the digest run remotely over SSH and ship back a ~200-byte JSON verdict. This is cheap because tomdet already runs CORA's own venv (python 3.13.12, h5py 3.16.0).

Two round trips (describe, then checksum) are kept deliberately separate. Collapsing them would make the handler's changed-under-read guard agree by construction, and therefore decorative.

The locator is untrusted

full_file_name comes from 2bmSP2:HDF1:FullFileName_RBV, writable by anyone with Channel Access. Today CORA only ever opens that string; this slice makes it reach a remote host, so:

• it never enters argv (the remote shell would re-parse it) and travels over the SSH child's stdin as JSON;
• a client-side prefix and charset pre-check refuses early, explicitly non-authoritative;
• the authoritative check is the same resolve_confined_file_uri (realpath + commonpath) the local adapters already use, run on whichever host actually holds the bytes;
• fixed ssh argv with BatchMode=yes, a connect timeout with a 1 s floor, and -- before the host.

A sweep, not a terminal hook

The slice-17 memo said "fires off the witnessed terminal". A ~30 s remote probe inline in _record_outcome would stall the single sequential witness loop for 30 s of every 120 s, and missing a BEGUN is the one failure this system cannot afford. _run_witness.py is not touched at all. A sweep is also restart-safe with no queue and no in-memory state, and it backfills runs that already have a resolved path.

Gate review

4-agent panel (architecture / test-coverage / cross-BC consistency / security specialist, the last declared for the untrusted-locator-to-SSH surface). 2 P0s, both fixed here:

• Head-of-line blocking (found independently by two reviewers): the original one-oldest-candidate-per-tick design let a single permanently-stuck run block every newer run forever. tick() now tries up to 10 candidates, excluding each one it gives up on, stopping at the first success or the first systemic failure.
• A previously-empty, previously-inviolate cross-BC-SQL fitness test did not fire on an earlier full-suite run because the new file was untracked at the time; the AST scanner walks tracked_python_files() only. Allowlisted with a citation, the escape hatch the test itself names for a query neither BC's own store can express alone.

A third boundary violation surfaced only at commit time from tach-check: cora.api depending on cora.data.aggregates to catch IngestScan's own composed errors is the identical blessed-cross-BC-access pattern already granted to five other BCs for the same reason.

Also fixed: a PII leak (the vaulted observed_path reaching log lines across four call sites); a broken never-raise contract on a malformed remote response; a missing startup authorization probe plus per-tick log spam on a standing denial; untested settings validators and the new boot-gate branch; two test-naming fitness violations; and several P2s. 34 tests added in the fix pass, 76 total for the slice.

Risk

Inert on merge. capture_scan_ingestor_enabled defaults off, and the boot gate refuses to start if it is on without capture_path_recording_enabled.

Every green test on the SSH path uses a fake invoker or a loopback; the loopback does exercise real h5py against a real HDF5 file, so the composition is proven, but no byte has yet crossed to tomdet. Live verification against the real host, and the operator's consent to run a read-only probe there, are tracked outside this PR.

Open before switch-on, not before merge

Auto-ingest puts the vaulted observed_path (personal data: 2-BM's directory layout embeds a surname and proposal number) onto immutable DatasetRegistered.uri / DistributionRegistered.uri events. Not new to this slice, since a human could already do this via the POST route, but now automatic and unconditional. Recorded as a policy decision owed before the switch goes on.

🤖 Generated with Claude Code

[github-actions]

Coverage report

Click to see where and how coverage changed

File	Statements	Missing	Coverage	Coverage (new stmts)	Lines missing
apps/api/src/cora/agent
__init__.py
seed_capture_scan_ingestor.py
apps/api/src/cora/api
_capture_scan_ingestor.py
main.py
apps/api/src/cora/data
_remote_scan_probe.py
wire.py
apps/api/src/cora/data/adapters
_ssh_probe.py
posix_checksum.py
ssh_data_exchange_scan_reader.py
ssh_posix_checksum_computer.py
apps/api/src/cora/infrastructure
config.py
Project Total

This report was generated by python-coverage-comment-action