Split additional State Pension by the period's State Pension type

[MaxGhenis]

Fixes #1921

Stacked on #1899 (base state-pension-age-67-phase-in); GitHub retargets it to main when #1899 merges.

Summary

basic_state_pension and new_state_pension split the data year's state_pension_reported by state_pension_type in the period simulated, but additional_state_pension split it by the type in the data year. Survey ages are held fixed when a dataset is projected (extend_single_year_dataset copies the data year), so a record's birth cohort moves one year later for each year projected and its type can change. A man aged 75 in the 2024-25 data reached State Pension age in 2014 (basic); held at 75 in 2030-31, the record stands for a man who reached it in 2021 (new). For such records new_state_pension paid the reported amount up to the new State Pension's full rate and additional_state_pension paid everything above the basic State Pension's full rate, so the band between the two flat rates was paid twice.

• additional_state_pension now reads state_pension_type for the period, like the other two components. Basic, new and additional State Pension then add up to the reported amount uprated by the period type's flat rate.
• The is_SP_age factor Set State Pension age from date of birth, including the rise to 67 #1899 added stays as a guard (review). A computed type is already NONE below State Pension age, so it changes nothing there (a rerun of the full data gives identical results); it also holds if a dataset or reform sets state_pension_type directly.
• Docs: docs/book/programs/gov/dwp/state-pension.md explains the split across years. It replaces the stale aggregate-gap section (it quoted ~£127.5bn against ~£140bn, and an unsourced mechanism for it) with a sourced comparison to DWP's and the OBR's forecasts and links to the two tracking issues.

The reverse case, new State Pension in the data year and basic in the period, would leave the band unpaid. With ages held fixed, cohorts only move later, so it never occurs in projections from the FRS (0 people in every year below); the tests cover it with datasets that carry other ages.

Invariants (stated and tested)

For every person and every year, with T the person's state_pension_type in that year, F_T the full weekly rate for T, r the weekly amount reported in the data year d, and U_T the add-on's uprating:

1. Components partition the reported amount. Over State Pension age: the flat-rate component (basic or new) = min(r, F_T(d)) × 52 × F_T(year)/F_T(d); and, deflating each component by its own uprating, flat/(F_T(year)/F_T(d)) + additional/U_T = 52r, with no overlap and no gap. The model sets U_T = F_T(year)/F_T(d), so the components add up to 52r × F_T(year)/F_T(d). In law, additional pensions and protected payments rise with CPI (SI 2026/148 arts 4(3), 6(3): 3.8% against 4.8% for the full rates in April 2026). That is a separate change, tracked in Uprate additional State Pension and protected payments by CPI, not the flat rate #1941. The tests keep U_T in one helper (add_on_uprating), so fixing Uprate additional State Pension and protected payments by CPI, not the flat rate #1941 changes only that helper.
2. Nothing below State Pension age. T is NONE exactly when is_SP_age is false, and all three components are 0.
3. One flat-rate component, within bounds. Basic is 0 unless T is BASIC, and new is 0 unless T is NEW. Basic ≤ 52 × F_BASIC(year), new ≤ 52 × F_NEW(year), and additional ≥ 0.

test_state_pension_components.py builds simulations from in-memory data:

• Property tests (Hypothesis) for 1-3. One uses up to 30 random people (ages 55-100, any birthday position, both sexes, weekly amounts 0-£600 plus each flat rate), projected through extend_single_year_dataset with ages held fixed, and checks every year from 2024-25 to 2030-31. The other gives the period arbitrary ages, so every pair of data-year and period types occurs.
• Examples at hand-worked cohorts:
  • basic then new, for a man and a woman, including a protected payment above the new State Pension's full rate;
  • unchanged basic and unchanged new;
  • new then below State Pension age after the rise to 67 (a control);
  • new then basic, and below State Pension age then new.
• Mutation check. On the pre-fix formula, both property tests and the three transition examples fail. The three controls (unchanged basic, unchanged new, and new then below State Pension age, which the is_SP_age guard also covers) pass either way. The minimal counterexamples are a woman aged 72 reporting £170 a week (basic in 2024-25, new later) and a woman below State Pension age in the data year reporting £170 a week who is on the basic State Pension in the period.
• On the full data. After the fix, the components add up to 52r × F_T(year)/F_T(d) for every person over State Pension age in every year, to within £0.004 (float32 rounding; observed maximum £0.0031). Before it, they exceeded it by up to £3,266 a year. The excess, re-derived independently from the formulas' own terms for records that are basic in the data year and new in the period, matches the observed change to within £1,000 in every year (table below).

Why pytest, not YAML: the YAML runner builds a simulation without a dataset (policyengine_core/tools/test_runner.py uses SimulationBuilder.build_from_dict, and a test has no dataset key). There simulation.dataset is None, so the data year is the period and the old and new code agree. Setting state_pension_reported at an earlier year in a YAML test doesn't change that; the value just carries forward to the period. The existing YAML tests in tests/policy/baseline/gov/dwp/ (121) pass unchanged.

Microsimulation impact

Enhanced FRS 2024-25 (policyengine-uk-data-private 1.57.4, snapshot ace89433; data year 2024-25). Every figure comes from a full microsimulation run of #1899's head (2466f44), without and with this change. Nothing is scaled; every figure is read from the model's own outputs. The "before" run matches #1899's recorded run of c46893e on all 96 State Pension, Pension Credit and State Pension age aggregates. The others differ because 2466f44 also merges #1901 (Housing Benefit) from main.

Change from the fix (after minus before)

		2024-25	2025-26	2026-27	2027-28	2028-29	2029-30	2030-31
State Pension	£bn	+0.00	-0.84	-2.17	-3.28	-4.29	-5.49	-6.77
of which additional State Pension	£bn	+0.00	-0.84	-2.17	-3.28	-4.29	-5.49	-6.77
of which basic and new State Pension	£bn	+0.00	+0.00	+0.00	+0.00	+0.00	+0.00	+0.00
Pension Credit entitlement (before take-up)	£bn	+0.000	+0.022	+0.053	+0.232	+0.334	+0.430	+0.483
of which Guarantee Credit	£bn	+0.000	+0.021	+0.052	+0.243	+0.346	+0.438	+0.485
of which Savings Credit	£bn	+0.000	+0.001	+0.001	-0.010	-0.012	-0.008	-0.002
Pension Credit (after take-up)	£bn	+0.000	+0.021	+0.047	+0.066	+0.160	+0.178	+0.225
Pension Credit eligible benefit units	k	+0.0	+4.9	+13.6	+17.4	+23.1	+63.9	+92.1
Pension Credit recipient benefit units	k	+0.0	+4.6	+14.6	+16.2	+19.3	+21.7	+49.8
Housing Benefit	£bn	+0.000	+0.005	+0.013	+0.032	+0.045	+0.046	+0.095
Council Tax Reduction	£bn	+0.000	+0.002	+0.003	+0.005	+0.006	+0.010	+0.025
Winter Fuel Payment	£bn	+0.000	+0.000	+0.002	+0.003	+0.003	+0.004	+0.006
Universal Credit	£bn	+0.000	+0.000	-0.000	-0.000	-0.000	-0.000	-0.000
Income tax	£bn	+0.00	-0.16	-0.42	-0.65	-0.85	-1.14	-1.42
Household net income	£bn	+0.00	-0.65	-1.68	-2.52	-3.22	-4.11	-5.00
Government balance	£bn	+0.00	+0.65	+1.68	+2.52	+3.22	+4.11	+4.99
People in absolute poverty, BHC	k	+0	+24	+26	+55	+85	+104	+121
of whom over State Pension age	k	+0	+24	+25	+55	+82	+101	+118
People in absolute poverty, AHC	k	+0	+14	+29	+53	+70	+84	+99
of whom over State Pension age	k	+0	+13	+28	+51	+68	+81	+96

Guarantee Credit and Savings Credit are gated by is_pension_credit_eligible. The guarantee_credit variable on its own has no pension-age condition and sums to £104-126bn a year over 2024-25 to 2030-31, so don't read it as an entitlement.

Where the overlap came from (before the fix)

		2024-25	2025-26	2026-27	2027-28	2028-29	2029-30	2030-31
People on basic State Pension in the data year and new in the year	m	0.00	0.59	1.31	2.02	2.59	3.18	3.84
of whom reported more than the basic full rate	m	0.00	0.56	1.24	1.70	2.17	2.65	3.13
Overlap from the formulas' own terms	£bn	0.00	0.84	2.17	3.28	4.29	5.49	6.77
Components minus reported amount uprated by flat rate, before	£bn	-0.00	0.84	2.17	3.28	4.29	5.49	6.77
Components minus reported amount uprated by flat rate, after	£bn	-0.00	-0.00	0.00	0.00	0.00	-0.00	0.00
People on new State Pension in the data year and basic in the year	m	0.00	0.00	0.00	0.00	0.00	0.00	0.00
People below State Pension age in the data year and over it in the year	m	0.00	0.00	0.00	0.00	0.00	0.00	0.00

The fix widens the gap to the OBR's forecast

The model was already below the OBR's State Pension forecast. Removing the double count moves it further below: by £0.8bn in 2025-26, rising to £6.8bn in 2030-31. The table compares with DWP's Spring Forecast 2026, which is consistent with the OBR's March 2026 forecast: the two differ by about £2m a year from 2026-27. The OBR's figures are £1.4bn higher in 2024-25, which the OBR records as outturn, and £0.1bn higher in 2025-26.

Against the OBR and DWP State Pension forecast

		2024-25	2025-26	2026-27	2027-28	2028-29	2029-30	2030-31
Model State Pension, before	£bn	119.0	125.6	131.8	134.2	137.2	142.4	147.7
Model State Pension, after	£bn	119.0	124.8	129.6	130.9	132.9	136.9	140.9
OBR March 2026 EFO, State Pension	£bn	138.0	146.2	154.2	158.9	164.0	172.2	180.7
DWP Spring Forecast 2026, State Pension	£bn	136.6	146.1	154.2	158.9	164.0	172.2	180.7
of which paid outside the UK	£bn	5.3	5.6	5.9	6.1	6.2	6.4	6.6
DWP less paid outside the UK	£bn	131.3	140.4	148.3	152.8	157.8	165.8	174.1
Model minus that, before	£bn	-12.2	-14.8	-16.5	-18.7	-20.5	-23.4	-26.4
Model minus that, after	£bn	-12.2	-15.6	-18.6	-21.9	-24.8	-28.9	-33.1
Model State Pension recipients	m	11.36	11.44	11.34	11.10	11.00	11.04	11.09
DWP caseload less paid outside the UK	m	11.88	12.12	12.15	12.04	12.14	12.42	12.69
Model State Pension per recipient, before	£/yr	10,483	10,986	11,628	12,094	12,480	12,889	13,312
Model State Pension per recipient, after	£/yr	10,483	10,912	11,437	11,798	12,090	12,392	12,702
DWP spending per recipient, less paid outside the UK	£/yr	11,044	11,591	12,201	12,694	12,993	13,351	13,714

Growth from 2025-26 to 2030-31: model recipients -3.0% vs DWP caseload less paid outside the UK +4.8%; model State Pension per recipient +21.2% before and +16.4% after, vs DWP +18.3%; State Pension +17.5% before and +12.9% after, vs DWP less paid outside the UK +23.9%.

What the comparison covers:

• Sources.
  • OBR March 2026 EFO detailed expenditure tables, sheet 4.9, "State pension".
  • DWP benefit expenditure and caseload tables, Spring Forecast 2026, sheet "State Pension": row 4 total, row 13 paid outside the UK, rows 29 and 38 caseloads.
  • Each figure was re-checked cell by cell by a second reader.
• Geography.
  • DWP's figures cover Great Britain and UK pensions paid abroad, not Northern Ireland. The model covers UK private households, including Northern Ireland, so the like-for-like gap is larger by Northern Ireland's State Pension (not sourced here).
  • The FRS covers private households only, not nursing homes and other communal establishments.
• The data year (policyengine-uk-data#493).
  • In 2024-25 the model has 11.36m recipients averaging £201.6 a week.
  • The FRS's own grossed figures are 11.53m and £212, and DWP's administrative figures 11.88m and £212 (FRS 2024-25 methodology tables M.6a/M.6b).
  • The FRS's admin-linked grossed UK total is £130.6bn, against £135.1bn administrative (FRS transformation tables, sheet 16). So most of the model's £16.1bn shortfall against the UK administrative total arises in building the enhanced FRS, not in the survey.
  • For FRS respondents, state_pension_reported is the FRS benefits-table weekly amount for State Pension × 52 (policyengine-uk-data datasets/frs.py, 1.57.4 = b45c373, lines 1151-1182). On the Survey of Personal Incomes donor rows the enhanced FRS adds, a second-stage QRF re-imputes it (datasets/imputations/frs_only.py:85, called from imputations/income.py).
  • The docs page's old explanation said a "single weekly figure (SRP)" caps SERPS. It is removed: SRP is an HMRC Survey of Personal Incomes field (policyengine-uk-data datasets/spi.py), not the source of the FRS amount, and the FRS's mean amount matches DWP's.

Why the gap widens in projected years (#1929), computed from the runs and the DWP tables:

• Caseload. Ages are held fixed and household_weight grows with total population (policyengine_uk/data/uprating_indices.yaml), so the pension-age population keeps the 2024-25 age structure. From 2025-26 to 2030-31, model recipients fall 3.0% as State Pension age rises to 67. DWP's caseload less those paid abroad rises 4.8%.
• Awards. Each record keeps its reported amount, uprated by the flat rate, even when its cohort moves from basic to new State Pension.
  • DWP's spending per recipient (less those paid abroad) grows 18.3%. Its new State Pension spending rises from £56.1bn to £102.4bn, and basic falls from £66.7bn to £57.8bn.
  • The model's per-recipient amount grew 21.2% before this fix: the double count overshot DWP's award growth, for the wrong reason. After the fix it grows 16.4%.

So the widening after the data year comes from the model not ageing the pension-age population and not giving new cohorts new awards (#1929). The data-year shortfall is in the enhanced FRS (policyengine-uk-data#493). Neither is a reason to keep an overlap that pays part of one amount twice.

Out of scope, found in review. A State Pension rate reform without a date, such as {"gov.dwp.state_pension.new_state_pension.amount": 300}, also rewrites the data year's rate. The uprating ratio is then 1, so a new State Pension record gets its un-uprated reported amount. For a man aged 70 reporting £240 a week in 2024-25, that is £12,480 in 2027 against £14,077 at baseline. With {"2026-01-01.2100-12-31": 300} it is £16,926. The same split does this on main and on the #1899 base; it is not changed here and is being split off as a follow-up.

Tests run

• policyengine-core test policyengine_uk/tests/policy/baseline/gov/dwp/ -c policyengine_uk: 121 passed.
• pytest policyengine_uk/tests/test_state_pension_components.py policyengine_uk/tests/test_state_pension_age.py policyengine_uk/tests/test_triple_lock_outturn.py: 43 passed (13 min) at 0ca9bec. After the review changes, test_state_pension_components.py again: 8 passed.
• ruff format --check . and ruff check on the changed files: clean.
• Not run locally: the full make test (limited to targeted tests while the host's memory is in use elsewhere; CI runs the suite) and tests/microsimulation/ (needs the private dataset; the before/after runs above cover it).

axiom: n/a: microsimulation data-handling. This changes how the survey-reported State Pension is split across PolicyEngine's components in projected years; no statutory rule changes. rulespec-uk's uk/policies/govuk/state-pension.yaml computes entitlement from qualifying years and never reads a reported amount.

🤖 Generated with Claude Code

[vahid-ahmadi]

Reviewed: approve with changes. Removing the double count is correct and well tested. The new tests pass locally (8/8), and restoring #1899's formula fails exactly the 5 tests claimed, with the 3 controls passing.

1. Should fix: additional pension is uprated by the flat rate, not CPI. At variables/gov/dwp/additional_state_pension.py:56-62, the amount above the flat rate is uprated by F_T(year)/F_T(data year), i.e. the triple-lock rate.
   • In law, the additional pension, protected payment, deferral increments and inherited amounts rise by CPI. DWP's rates show it: in 2025-26 additional pension and protected payment rose 1.7% while the flat rates rose 4.1%; in 2026-27, 3.8% against 4.8%.
   • This isn't new here. But invariant 1, uprated_reported() (test:79), assert_components_add_up (test:225) and the docs line "uprate it by that type's flat rate" (docs:33) now treat it as correct, so a later CPI fix would break them.
   • Scale: DWP's 2024-25 figures put S2P at £19.9bn, graduated retirement benefit at £1.9bn and protected payments at £1.2bn, so roughly £0.5bn too much in 2025-26, growing yearly (rough, not run).
   • Fix: split the invariant into the flat part × the flat-rate ratio plus the excess × a CPI index, or at least add a docs caveat and an issue.
2. Nit: keep is_SP_age * as a cheap guard. If a dataset or reform sets state_pension_type directly (its default is BASIC), additional pension would be paid below State Pension age.
3. Nit (docs:121/132): the OBR and DWP figures differ by £1.4bn in 2024-25 (OBR 137.98, DWP outturn 136.58). The table's 2024-25 gap mixes the two, and the prose uses DWP's £131.2bn. Say so, or use one source.
4. Should fix (can follow later): a record that moves from basic to new keeps a basic-era amount relabelled as a partial new State Pension. It's acknowledged under State pension undershoot: -£12bn vs OBR, ASP/Protected Payment under-captured #1632, but the docs should say it understates new cohorts.

Verified against the sources

• OBR March 2026 table 4.9 "State pension", every year (137.98 to 180.70).
• DWP Spring Forecast 2026 State Pension sheet: payments abroad, caseloads, per-recipient amounts, and the new and basic totals.

I didn't check the £131.2bn or the FRS £130.6bn / £135.1bn sources.

[MaxGhenis]

Thanks @vahid-ahmadi. Changes are in dce3fbc (and eca6134 for point 3).

1. CPI uprating of the add-on. Confirmed in the Social Security Benefits Up-rating Order 2026 (SI 2026/148). Art. 4(3) (additional pensions, deferral increases, etc.) and art. 6(3)(b) (new State Pension above the full rate) rise by 3.8%; the full rates in arts 4(2) and 6(1) rise by 4.8%. I've kept the fix out of this PR because it's a separate change to how the law is encoded, with its own Axiom parity entry and aggregate impact. It's filed as Uprate additional State Pension and protected payments by CPI, not the flat rate #1941, with the law, the scale and a suggested index (gov.benefit_uprating_cpi). This PR no longer treats flat-rate uprating as correct:
   • Tests: they now check that the components partition the reported amount. The flat-rate part is min(r, F_T(d)) uprated by the full rate. Deflating each component by its own uprating gives back exactly r.
   • One helper: the add-on's uprating lives in add_on_uprating, which for now is the model's flat-rate ratio, so Uprate additional State Pension and protected payments by CPI, not the flat rate #1941 only needs to change that helper.
   • Docs: the Components section says the model uprates add-ons by the full rate, while the law uses CPI.
2. is_SP_age guard: restored. A rerun on the full enhanced FRS gives identical results in every year.
3. OBR vs DWP: fixed in eca6134. The table now compares with DWP's Spring Forecast 2026 less payments abroad: -£12.2bn in 2024-25, matching the text. The OBR row is shown too, and the docs say the OBR is £1.4bn higher in 2024-25 and £0.1bn higher in 2025-26, and within about £2m a year from 2026-27. On the two figures you didn't check:
   • The £131.2bn GB-resident outturn is no longer used.
   • The £130.6bn and £135.1bn are from the FRS "integrating administrative data for benefits" tables, sheet 16, FYE2025 UK block (linked in the references).
4. New cohorts: the docs now say a record that moves to a new State Pension cohort keeps its reported amount. That's one reason the model's State Pension per recipient grows 16.4% from 2025-26 to 2030-31, against DWP's 18.3%. The projection itself is tracked in Project the pension-age population and new-cohort State Pension awards beyond the data year #1929.