UK: port the unblocked uk-data fixes tracked in #1095

[juaristi22]

Refs #1095. Stacked on #1089 (uk-spine-followups-1063).

Summary

This PR ports to the microcosm UK spine the uk-data fixes from the post-AB2026 wave that #1095 tracks and that need no policyengine-uk change. It covers nine of the tracker's items in 13 commits, plus a register retirement and the receipts note. Each commit carries its own changelog fragment with the measured before-state:

• Benefit target scope (uk-data#490 follow-up).
  • f5fffc55f: the DWP and OBR table 4.9 benefit rows now bind the households their publisher counts. That is Great Britain for the DWP social security rows; England and Wales for Attendance Allowance, Carer's Allowance, PIP and Winter Fuel Payment, since their Scottish successors transferred.
  • 7acd6f7aa: obr.pip binds the engine's pip + dla, the EFO line's "DLA and PIP".
• Scottish disability payments (uk-data#500).
  • 7c44e7f49: FRS codes 117/118 (Adult Disability Payment) and 121/122 (Child Disability Payment) land in the PIP and DLA columns they replace.
• FRS dividends (uk-data#498).
  • d90450ae6: the SPI income stage no longer redraws FRS respondents' dividend_income from the SPI forest.
• Dependants aged 16 to 19 (uk-data#504).
  • 6444dc8f6: SPI draws go only to FRS claimants and partners (is_uc_claimant), so dependants keep their FRS twin's values.
  • Every other recipient keeps its exact draw.
  • Band-donor carriers are claimants and partners too.
• Council tax (uk-data#496/Adjudicate the JCX-35-25 OBBBA no-tax anchors: fenced parity exclusion + ledgered probe receipts (#451 items 3-4) #499).
  • 22bfc7e81: council_tax is the bill before council tax reduction, and Scottish water and sewerage are netted at the gross cells.
  • 856de4abe: the Scottish water charge is the amount paid: gross, less the status discount, at 65% for reduction recipients.
• SPI benefit coherence (uk-data#514).
  • 0fb1e1179: new stage spi_benefit_coherence after uc_reporter_redraw, with the release-blocking gate uk_stage_spi_benefit_coherence.
  • It zeroes the SPI rows' legacy and out-of-work reports, restores IIDB, AFCS and bereavement support from the FRS twin, and re-derives the disability flags and receives_benefits_in_own_right.
  • It redraws would_claim_uc for SPI units as frs_take_up draws it.
• SPI capital (uk-data#495).
  • d55f317da: every SPI benefit unit's TOTCAPB4 is redrawn conditioned on its financial investment income, with declared coarsening.
• WAS predictors (uk-data#486/UK dense/local epic: rowwise constituency-grain dataset by cloning the national staging pool (US dense-arm analog) #495).
  • 606f13793: the donor reads DVCTaxAmtAnnualR8 and NumChildR8; the recipient counts come from is_uc_claimant; and a rental_income predictor is added.
• Buy-to-let (uk-data#501).
  • f07c779af: other_residential_property_value includes DVBltValR8_sum.
• Age-18 child elsewhere (uk-data#486).
  • 9b9243078: the ETB VAT and services adult and child counts come from FRS roles.
  • 9c7394086: constituency UC child bands use the national UC family-type composition, through one shared helper.
• Target-fit register.
  • a4fb0e303: retires UK spine follow-ups: SPI donors funded, WAS coherence, residential split, Child Benefit, release-cut blockers (#1063) #1089's deferral of the South East £12,570–15,000 income-tax cell.
  • The ports bring that cell back inside the bound (see Measurement), so uk_target_fit fails the deferral as stale. The register is now empty.
• Receipts.
  • f36af232d: experiments/1095-uk-data-ports-receipts.md.

María's rulings of 2 October are applied: buy-to-let into other_residential_property_value; the two-rule Scottish council tax netting and charge; obr.pip = PIP + DLA over England and Wales; and the full SPI coherence stage including the UC take-up redraw.

Not in this PR

• Housing Benefit pension-age spending row (planned C4c), dropped.
  • dwp.hb.amount_pension_age cannot compile: calendar_year_window needs both fiscal years from one series, but the DWP forecast-table facts carry a measure_id per year (expenditure_2024, expenditure_2025).
  • It needs either a Chronicle series-identity change or a ruling to bind FY2025-26 alone.
• Property income is in review as UK FRS spine: stop counting lodgers' rent and property losses as property income; sub-letting rent for every tenure #1081, BRMA as Weight UK BRMA assignment by census private-rented households #1087, HBAI validation folds into UK scorecard: populace_uk_2023 vs enhanced_frs_2024_25 vs admin benchmarks (2026) #731, and reproducibility is uk-data#507.
• The items that need policyengine-uk changes stay listed in UK: track the defects uk-data is fixing that microcosm still has #1095.

Needs María's signature at review (not committed)

Four texts change what a signed register says, so they wait for María's signature and none is committed:

• a new council-tax-gross-of-reduction signed difference for C3a;
• an amended scottish-water-sewerage-successor-level for C3b, since the amount netted from council tax no longer equals the charge;
• reworded text for the uc_unit_vs_household_grain adjudication;
• reworded text for its census fence, since the constituency bands now use the national composition instead of num_children.

Proposed texts

New entry: council-tax-gross-of-reduction (C3a, commit 22bfc7e)

{
 "id": "council-tax-gross-of-reduction",
 "class": "mechanism_change",
 "scope": {
  "surface": "weighted_totals",
  "columns": ["council_tax"],
  "entities": ["household"]
 },
 "expectation": "column_differs",
 "magnitude_evidence": "FRS 2024-25 CTANNUAL is the annual bill after discounts and the reported council tax reduction (SN 9563 DV summary), while the engine reads council_tax as the liability before the reduction, so the incumbent and our earlier build both entered reduction recipients net of their reduction: 20.6% of English FRS-channel reduction reporters on the 30 September national build had a GBP 0 bill and 25.6% a bill below their own reduction. A household reporting a reduction (CTREB 1) now gets its CTREBAMT added back, a full reduction included; a recipient with no positive amount takes the larger of its bill and its cell mean; the region, band and single-adult cells pool non-recipients only; and Scottish water and sewerage charges are netted at the gross cells, in full for non-recipients and at 65% for recipients, the shares DWP's CTANNUAL carries (uk-data#496/#499). The add-back on the FRS channel is GBP 0.78bn in England, 0.05bn in Wales and 0.06bn in Scotland; on the microcosm#1095 arm spines, council_tax at design weights moves from GBP 42.27bn to 43.57bn across England, Wales and Scotland (England 37.0bn to 38.3bn, Wales 2.08bn to 2.18bn, Scotland 3.21bn to 3.06bn, where the gross water and sewerage charges now come off non-recipients' bills in full), and the reduction reporters with a GBP 0 bill fall from 10.6% to 0.",
 "evidence": "experiments/1095-uk-data-ports-receipts.md#spine-receipts-by-commit",
 "adjudicator": "juaristi22",
 "adjudicated_on": "[on signature]",
 "quantitative": {
  "weighted_totals": {
   "expected_columns": ["council_tax"]
  }
 }
}

Amended entry: scottish-water-sewerage-successor-level (C3b, commit 856de4a)

Only magnitude_evidence changes. The current text says the amount netted
from council_tax equals the charge. That stops being true at C3a, because
the netting follows DWP's derivation and the charge follows what the
household pays.

Proposed magnitude_evidence:

The Scottish charge is assembled from the successors FRS 2024-25 published for the cells it retired. CWATAMT1 and CSEWAMT1 are the gross charges on the bill; the household pays them less the status discount its council tax records (CTDISC with CT25D50D: 25% or 50%), at 65% for council tax reduction recipients, the Water Charges Reduction Scheme's maximum 35% reduction, and a household with no gross water cell pays its recorded CWATAMTD (uk-data#499, microcosm#1095). The earlier rule read CWATAMTD/CWATAMT1 as the household's discount factor, but CWATAMTD sits at a median 0.83 of the gross water charge where no discount applies, which put the charge about 13-16% below the gross charges less the discount. Weighted annual per Scottish household with a charge moves from about GBP 404 to about 465 on the microcosm#1095 arm spines (A3 to A4), against roughly GBP 490 for England and Wales on WATSEWRT; the incumbent sits at zero because of the separate NaN defect. council_tax nets the gross charges at the shares DWP's CTANNUAL carries (in full for non-recipients, 65% for recipients), which is not the amount charged, so the two columns no longer move by one construction; council_tax's own move is signed under council-tax-gross-of-reduction. The nonzero share is unaffected, so this entry deliberately does not sign the share surface.

Scope and quantitative block: unchanged (weighted_totals on
water_and_sewerage_charges and council_tax). Narrowing the columns to
water_and_sewerage_charges alone is an option, since the new entry signs
council_tax; that is María's call.

Amended adjudication: uc_unit_vs_household_grain (C9b, commit 9c73940)

packages/microcosm-build/src/microcosm/build/uk/local_binding_adjudications.json,
key uc_unit_vs_household_grain. Only reason changes; approved_by,
approved_on and expires_on follow the signature. The current text says
the bands use each unit's num_children and match the national re-bind.
They did not match until C9b. Now they match, but no longer through
num_children.

Proposed reason:

Replace the stated basis with the named alternative: the uc_households family binds at benefit-unit grain. uc_households per area counts UC benefit units mapped to their households (the DWP claim-household definition), and the four children bands classify each UC benefit unit by the members DWP's UC family type counts as children (uc_relationships.uc_family_child_member: never a claimant or partner; any UC child or qualifying young person, or any other member under 20), summed benefit unit to household. That is the composition the national dwp.uc.households* re-bind of PR #813 binds (microcosm#1095; before it the bands read the engine's age-18 num_children, which put 2.77% of UC-reporting units in a different band). The household-children basis is retired. Fence evidence: uk_local_target_census.json#/binding_fences/uc_unit_vs_household_grain; runtime: microcosm.build.uk_runtime.local_targets.compute_household_metrics.

Amended census fence rule: uc_unit_vs_household_grain (C9b)

packages/microcosm-build/src/microcosm/build/uk_runtime/local_target_census.py
(the rule of the _UC_GRAIN_FENCE_ID fence; regenerated into
uk/uk_local_target_census.json). Proposed rule:

The runtime maps benefit-unit-level UC receipt and each UC benefit unit's UC family-type child-count band (uc_relationships.uc_family_child_member, the national UC-by-children composition) onto household rows by sum: a physical household containing two UC benefit units contributes 2 to uc_households and one contribution to each unit's child band. This aligns the runtime with DWP UC claim households, which correspond to benefit units rather than physical FRS households; binding must preserve this reviewed grain crosswalk.

Measurement

Eight cumulative licensed arms ran, A0 to A7, with an extra arm at the rebased head:

• Build. Each arm is a spine build followed by the national calibration against Chronicle 825406f, with local staging only. Nothing was released.

• Base. The arms ran on the pre-rebase chain on 747313ae7. A0 is UK spine follow-ups: SPI donors funded, WAS coherence, residential split, Child Benefit, release-cut blockers (#1063) #1089's stack-2 spine recalibrated at 747313ae7.

• Write-up. The full receipts are in experiments/1095-uk-data-ports-receipts.md.

• Overall fit. Loss falls from 0.00720 (A0) to 0.00702 (A7). Targets within 10% rise from 98.3% to 98.5% of 1,168.

• South East income-tax cell. The cell UK spine follow-ups: SPI donors funded, WAS coherence, residential split, Child Benefit, release-cut blockers (#1063) #1089 excludes (hmrc.spi_region.income_tax_by_region_12570_15000@E12000008@2025) calibrates at +31.3% in A0.

  • It falls to +8.0% once FRS dividends stand as reported and dependants stop taking SPI draws (A3), and ends at +12.1% (A7).
  • The redrawn dividends were behind the pull on that cell. The reviewed exclusion goes stale, and uk_target_fit asks for its removal, which is the only gate failure at A6 and A7.

• Council tax. obr.council_tax moves from −11.1% to −9.9%, and all 98 council-tax rows are within 10% from A6.

  • Among households reporting a reduction, those with a £0 bill fall from 10.6% to 0. The mean Scottish water charge goes from £404 to £465.

• VAT. obr.vat sits at +24.4% to +25.2% from A3 on, against +22.8% at A0. It crosses the 25% bound in A3 and A5 but not in A7, so it is a risk for later spines.

• Weight ratio. uk_weight_ratio exceeds its reviewed maximum in A5 only: 1,161.8 against 1,151.3.

• Spine receipts (design weights):

  • Scottish PIP daily-living reporters per 1,000 rise from 13.2 to 43.9 (C5).
  • FRS-channel dividends fall from £40.5bn to £5.1bn, the respondents' reports (C1).
  • SPI dependants aged 16 to 19 with earnings fall from 94% to none, and dependant band carriers from 40 to 0 (C2).
  • SPI rows with legacy or out-of-work reports, 435 of them carrying £2.50bn, fall to none (C6).
  • SPI units whose investment income implies more than £16,000 while they hold no more fall from 1.86m to 0.18m (C7).
  • Rent-receiving households holding other property rise from 11.6% to 88.3% (C8b).

• A8, the rebased PR head 9c7394086.

  • Loss 0.00693 and 98.8% of targets within 10%, the best of the arms.
  • Every calibrated-seam gate passes except uk_target_fit, and that gate fails only on the stale South East deferral, which a4fb0e303 retires.

• Risk to watch: VAT. At the head obr.vat calibrates at +24.998% against the 25% bound, so it passes by 0.002 points.

  • The overshoot predates this PR: design VAT is 25–28% over the OBR line in every arm, the control included. The ports move the calibrated value from +22.8% to the bound.
  • Any later spine change can tip it, so the VAT imputation (ETB rate times LCFS spending) is the next thing to look at if it does.

Verification

There is no CI on stacked PRs, so these local lanes are the record. The pinned feed (CHRONICLE_UK_FACTS, chronicle-uk 825406f) was set throughout.

• Touched surface: 74 test files, chosen as every test the commits change, every test importing a touched runtime module or its test-support helper, the H2 parity test (microcosm-graph engine/uk/test_acceptance_h_parity.py) and the integration-uk lane, plus the incumbent-name guard.
  • 1,452 tests: 1,440 passed, 7 skipped, 5 failed.
  • One failure was this branch's: an E6 receipt fixture lacked is_uc_claimant after the ETB change. It is fixed in that commit; the file's 27 tests pass.
  • The other four fail identically on UK spine follow-ups: SPI donors funded, WAS coherence, residential split, Child Benefit, release-cut blockers (#1063) #1089's head 45014f48c, so they predate this branch:
    • test_uk_target_references.py::test_committed_surfaces_regenerate_from_pinned_feed[national]: the regenerated target_references.json hashes e8f829… against the committed bf7549…, the same on both trees;
    • test_uk_target_references.py::test_current_national_compile_parity_regenerates_and_passes_with_pinned_feed: regenerating both national compile-parity resources gives byte-identical files on the two trees, so the contract changes here do not move them;
    • test_uk_national_graph.py::test_national_gates_evaluate_the_seam_scope_and_record_the_evidence (uk_target_fit);
    • test_uk_uc_family.py::test_full_uc_payment_registry_matches_independent_relationship_and_band_cases (81 payment rows against 83).
• tools/ci_test_plan.py verify: ok.
• tools/build_uk_release_input_coverage_manifest.py --check: ok.
• tools/build_uk_was_wealth_support_bounds.py --check and the LISA bounds check on the licensed tabs: ok, unchanged.

Notes

• The etb_services stage notes were a plain YAML scalar containing #890, which YAML reads as a comment, so everything after it was dropped at load. They are now quoted. lcfs_consumption has the same defect and is left for a separate fix.
• Rebased onto UK spine follow-ups: SPI donors funded, WAS coherence, residential split, Child Benefit, release-cut blockers (#1063) #1089's head 45014f48c with no conflicts. Arms A0–A7 ran on the pre-rebase chain on 747313ae7; A8 ran on the rebased head.

🤖 Generated with Claude Code