UK input-mass evidence through the shared Frame-typed helpers (#611 A4)

[juaristi22]

A4 of the plan on #611: the weighted-integrity dedup. The UK-only input-mass table walkers (uk_dataset_input_mass_totals, _uk_entity_weights, _entity_table, _UK_ENTITY_STRUCTURAL_COLUMNS) are deleted in favor of the shared Frame-typed input_mass_totals, and the QRF-tail evidence helper plus the legacy uk_terminal_gate_report become Frame-typed with them. Two commits: the code migration (digest constants deliberately untouched — that commit is operationally stale by design and says so), then the keys-only digest re-freeze that moves every pin together.

What changes

• uk_input_mass_totals(frame) wraps the shared helper and removes the exported weight columns (UK_EXPORTED_WEIGHT_COLUMNS, a new named constant in national_frame.py, which owns the export-contract doctrine). The post-filter is load-bearing: household_weight is a real, engine-known column on the UK frame, so neither the shared helper's schema-derived structural set nor a caller's columns= allowlist can exclude it — without the wrapper the totals would gain a Σw² pseudo-column on both the candidate and reference sides.
• Totals keys are flat frame column names instead of entity.column: the frame's global column-uniqueness invariant makes the namespace unnecessary, and the gate-battery contract doc's worked example already used flat keys. The degenerate-surface gate keeps its own dotted convention — that surface is untouched.
• _UKGateSurface / _uk_gate_surface move to national_frame.py (the Frame-carrier seam) as the single copy shared by the battery bindings and the legacy report, with a fail-closed type guard. Five of the seven binding call sites still need the duck surface; only input-mass and QRF-tail now read the frame directly. The legacy report builds the surface once and must hand it (never a raw Frame) to the coverage gate, which reads household_weight_kind/time_period/mass_log via getattr-with-default.
• Both tools migrate. build_uk_efrs_parity_reference.py --emit-weighted-totals now constructs the gate-surface Frame from the licensed H5 — subset to structural+requested columns, group ids sorted, cross-entity collision guard — so the reference emitter validates exactly what the totals are measured from. measure_uk_weighted_integrity_baselines.py drops its engine_tables downgrade (and the "One country-agnostic gate battery: phased evaluation, unconditional reporting, per-country gate selection (#578) #611 owns their Frame typing" IOU comment with it).
• Benunit failure taxonomy moves to the construction seam (named behavior change): orphan/memberless benunits are unrepresentable as a Frame, and resolve_weights refuses only unequal-weight household spans — the old helper refused any span. Real-frame paths are unaffected; the rewritten tests pin the new seam, including the equal-weight-span success.

The digest re-freeze (keys-only, for review)

The reviewed enhanced-FRS reference digest binds the totals' key shape, so it moves. The re-freeze is keys-only against the #610 freeze: the saved measurement totals reproduce the prior digest 11b22dd4… byte-for-byte; stripping the entity prefixes yields 131→131 columns with zero cross-entity collisions and every value untouched; identity unchanged. Verified end-to-end by loading the re-keyed totals through load_uk_input_mass_reference unpatched against the new constants — and the dotted-era file is now refused. The adjudication note lives in the weighted_integrity.py receipt comment, with a one-sentence pointer in the uk/gates.json entry notes.

Every pin moves in one commit: the module constant, the gates.json declared pin (held equal at runtime by the binding), the data shard's schema-3 mirror and schema-4 wrapped pin, and — because a gates.json parameter changed — the three vintage pins (policy_sha256, gates_manifest_sha256, spec_fingerprint), recomputed from a fresh GateBatteryRun over the committed spec. The build-shard pins tests and the contract lockstep test verify all of them against the live producer.

Verification

• Full microcosm-build + microcosm-data suites green locally with policyengine-uk installed (5045 passed, 108 skipped), including the battery-vs-legacy differential with both sides fed one Frame, and the repurposed wrapper-equivalence anchor (uk_input_mass_totals == input_mass_totals minus household_weight, hand-computed values).
• Two deselected tests are pre-existing environmental failures unrelated to this PR: the cached-artifact regeneration tests fail on any machine with the licensed HF cache and an engine newer than the committed 2.89.0 vintage (871 vs 866 known variables); CI skips them, and they fail identically on main.

Delete-or-justify

Deleted: the three UK table walkers and their structural-column map (the shared helper + wrapper is the single implementation), the Frame→duck downgrade in the two migrated binding call sites and the measurement tool, and the collision test whose colliding layout is unrepresentable as a Frame (the kernel pins that invariant in microcosm-frame's own tests). Kept: _uk_gate_surface (five legacy gates still read the duck surface), terminal_gates._entity_tables (degenerate gate's duck contract, untouched), and the schema-3 contract path (vintage dispatch, per #663).

Part of #611. Depends on #663 (merged).

🤖 Generated with Claude Code

[vahid-ahmadi]

A dedup PR that deletes more than it adds (+404/−437) while moving a frozen digest is the kind that deserves a careful read, and the structure makes that possible: separating the code migration from the keys-only re-freeze, and saying out loud that the first commit is operationally stale by design, is the right way to stage it.

I checked the two claims the whole thing rests on.

Flat keys are safe, and deleting the collision test is justified. The kernel really does enforce it — microcosm-frame/bundle.py:292-301 raises "Column names must be globally unique across entity tables (the flattening rule)", and it's documented as a class invariant at line 15. So the layout test_uk_totals_handle_shuffled_ids_benunits_and_entity_name_collisions was constructing is genuinely unrepresentable as a Frame, and the namespace was buying nothing. Good.

The re-freeze discipline is right. Keys-only, prior digest reproduced byte-for-byte from the saved totals, 131→131 with zero collisions, identity unchanged, and verified by loading the re-keyed totals through load_uk_input_mass_reference unpatched against the new constants rather than through a test seam. Refusing the dotted-era file closes the ambiguity rather than tolerating both. Moving all six pins in one commit — module constant, gates.json pin, both data-shard mirrors, and the three vintage pins recomputed from a fresh GateBatteryRun — is what keeps the tripwire meaningful.

The household_weight post-filter reasoning is subtle and worth the docstring it has: a real engine-known column that neither the schema-derived structural set nor a caller allowlist can be trusted to exclude, so it has to come off after the shared helper runs. Agreed, and the digest pin is the backstop if a future exported weight column is missed.

One thing I'd push back on: the benunit span narrowing removes the only check for a real defect.

orphan/memberless benunits are unrepresentable as a Frame, and resolve_weights refuses only unequal-weight household spans — the old helper refused any span. Real-frame paths are unaffected.

The first clause is true — the kernel does make orphans unrepresentable. But spanning isn't orphaned, and I don't think anything catches it now:

• Frame.resolve_weights refuses on unequal weights within a group (bundle.py:663-664), not on nesting.
• The kernel's nesting check (bundle.py:1070-1085, "does not nest in") only fires inside explicit group→group placement operations, which this path never performs.
• validate_uk_national_frame doesn't check it — I read it through; there's no nesting assertion anywhere in national_frame.py.

So a benunit spanning two households with equal household weights now totals cleanly where it previously raised. Your own test pins exactly that: uk_input_mass_totals(spanning_frame([2.0, 2.0])) succeeds.

A benefit unit spanning households is a genuine FRS-level data error, not a representational edge case — and the equal-weight case is the likely accidental one, not the exotic one. Pre-calibration frames, staging builds, and synthetic fixtures all commonly carry uniform weights, which is precisely when the remaining check goes quiet. "Real-frame paths are unaffected" is true of correct real frames, but the check existed for the incorrect ones.

I'd not restore it where it was, though. Living inside a totals helper made it a side effect of computing input mass — if you'd migrated a different consumer first, it would have disappeared just the same. It belongs in validate_uk_national_frame, which every UK stage already runs, as a stated frame invariant: a benunit's members share a household. That's both a stronger guarantee than the old one and a better home for it, and it makes the deletion here unambiguously correct rather than a narrowing.

If the span really is unreachable for some reason upstream of all this, that reasoning is worth a sentence in the changelog's named-behaviour-change paragraph — right now it asserts the conclusion without the argument.

Smaller

• uk_input_mass_totals is the only thing applying the weight-column filter, so a caller reaching for the shared input_mass_totals directly on a UK frame gets the Σw² pseudo-column silently. The digest catches it on the gate path; a new consumer computing totals for a diagnostic wouldn't be. Worth a line in the shared helper pointing at the UK wrapper.
• Moving _UKGateSurface to national_frame.py as the single copy, with the fail-closed type guard and the note that five of seven binding call sites still need the duck surface, reads as a genuine consolidation rather than a shuffle. Good that the retirement condition is stated.
• Having the reference emitter construct the gate-surface Frame from the licensed H5 — so the emitter validates exactly what the totals are measured from — closes a gap the old --emit-weighted-totals path had. That's a real improvement buried in the middle of a refactor.