Charge Class 4 NICs on Chapter 2 profits, after capital allowances, the trading allowance and Schedule 2 losses

[MaxGhenis]

Fixes #1916

#1881 (the trading allowance fix, which adds self_employment_gross_receipts) has merged, so this PR's diff is its own change.

What was wrong

ni_class_4, ni_class_4_main and ni_class_4_maximum took profits as raw self_employment_income. That ignores capital allowances, trading losses and, when receipts are supplied, the trading allowance. #1886 already removed the Class 1 deduction, and this PR does not bring it back.

The law

• SSCBA 1992 s. 15(1)(b) and Sch. 2 para. 2. Class 4 is charged on profits "chargeable to income tax under Chapter 2 of Part 2" of ITTOIA 2005, computed in accordance with Sch. 2 (s. 15(3)).
• Capital allowances are given effect as a trade expense (CAA 2001 s. 247).
• The trading allowance recomputes the trade's profits. They are nil where receipts are within the allowance (ITTOIA 2005 s. 783AF), and receipts less the allowance on partial relief (s. 783AI). HMRC BIM86052 says it "is deducted in determining the profits chargeable to income tax under Chapter 2, Part 2, ITTOIA ... so no Class 4 NICs will be due on income covered by the allowance".
• Losses: Sch. 2 para. 3. Relief is given under ITA 2007 ss. 64, 72, 83 and 89, but only against profits that count for Class 4. Para. 3(4) is where it differs from income tax. Where income tax deducts a trade loss in computing net income, even against other income, that deduction is "treated ... as reducing the person's profits" of the trade for Class 4 in that year. Any excess reduces the next year's profits, "and so on". So for Class 4 a loss only ever reduces trade profits, and it carries forward until used. ITA 2007 s. 84 says the same about order: the earliest year comes first.
• Regulation 100 (SI 2001/1004) works out the annual maximum on "the amount of profits for the year", which is the same measure.

The fix

Variable	Formula
ni_class_4_profits_before_losses (new)	max(0, self_employment_income − capital_allowances − trading_allowance_deduction). These are the Chapter 2 profits. trading_allowance_deduction (#1881) is already net of expenses and capital allowances, so the two never stack.
ni_class_4_trading_loss (new)	The year's trading_loss where one is supplied for that year, and nil otherwise. trading_loss is an input the engine carries into later years that are not set; for Class 4 a loss is relieved once, so a carried-over value is not a new loss.
ni_class_4_loss_relief (new)	min(ni_class_4_trading_loss + losses brought forward, profits before losses)
ni_class_4_losses_brought_forward (new)	Unrelieved losses from earlier years, folded year by year from the first year a loss or opening balance is supplied for. It reads supplied inputs only (policyengine_uk/utils/supplied_inputs.py, which uses the engine's record of set_input calls), so the result does not depend on what was calculated first. It can also be supplied as an input for any year, and later years build on that balance.
ni_class_4_losses_carried_forward (new)	Losses available less losses relieved
ni_class_4_profits (new)	Profits before losses less loss relief
ni_class_4, ni_class_4_main, ni_class_4_maximum	Now use ni_class_4_profits in place of self_employment_income. #1884's float32 handling and regulation 100 steps are unchanged.

Personal reliefs, interest relief and pension contributions are not deducted (para. 3(2)). Income tax is untouched.

Why a loop and not a formula on the previous year. The first version recursed through period.last_year. The engine's spiral guard (max_spiral_loops = 10) returns the default past ten levels, so a 2016 loss simply vanished from 2030. The fold has no self-reference, so there is no limit. It stops at the first year for which any loss data is known, so microsimulation runs one extra year at most. test_losses_carry_forward_over_many_years checks up to 16 years, with the years calculated in random order.

Invariants (stated and property-tested)

policyengine_uk/tests/test_ni_class_4_profits_properties.py uses Hypothesis. Each example simulates 24 cases twice, as drawn and varied, over two years, for 2022–23 and 2025–26. The draws are weighted towards the £1,000 allowance, the LPL and the UPL. The reference is an independent exact-rational (Fraction) implementation of the statute.

1. Differential. Profits before losses, loss relief, losses brought and carried forward, and Class 4 profits equal the reference: the better of actual deductions and the allowance where receipts are known, and losses set against each year's profits with the rest carried forward.
2. Accounting identity. trading_loss + brought forward = relief + carried forward. Brought forward equals last year's carried forward, and 0 ≤ relief ≤ profits before losses.
3. Bound. 0 ≤ ni_class_4 ≤ the s. 15(3) amount on the reference Class 4 profits, which is at most the s. 15(3) amount on unrelieved profit. Where regulation 100 cannot apply, ni_class_4 equals the s. 15(3) amount.
4. Monotonicity. ni_class_4 is non-decreasing in profit, at fixed expenses (or unknown receipts), capital allowances, losses and employment income, wherever Class 2 is unchanged. It is non-increasing in capital allowances and in trading losses.
5. Carry-forward. Over 2–16 years, with or without a supplied balance at any year and in any calculation order, the result matches the reference fold.

test_ni_class_4_properties.py still checks s. 15(3) and the exact regulation 100 steps for arbitrary thresholds and rates. It now takes ni_class_4_profits as its input, and asserts that with no reliefs this equals the full profit, or nil within the trading allowance. That keeps #1885's no-Class-1-deduction invariant.

Mutation checks. Each mutant below fails the new YAML file, the new property file, or both:

• dropping capital allowances;
• dropping the trading allowance;
• stacking the allowance on expenses;
• income-tax-style one-year loss lookback;
• one-year carry-forward;
• ignoring a supplied balance;
• no loss relief.

Revenue effect

Real Microsimulation runs on the Enhanced FRS 2024-25 (policyengine-uk-data 1.57.4, SHA-256 03fe15e4…68d4), comparing main 5232f9af (which includes #1881) with this branch at 9ca5d6eb, which merges that main: no total moves in 2025, 2026, 2027 or 2028. Class 4, National Insurance, income tax and the government balance are unchanged, and no household's net income changes. Main bfc5fea1 fed raw self_employment_income into Class 4; this branch uses adjusted Class 4 profits. Liabilities are equal on this dataset because it has no capital_allowances, trading_loss, self_employment_gross_receipts or ni_class_4_losses_brought_forward column, so none of this PR's loss or allowance inputs are present.

Known limits (not changed here)

• Carry-back claims are not modelled. These are s. 64(2)(b) (previous year), s. 72 (early years) and s. 89 (terminal loss). Under para. 3(4) they would reduce an earlier year's Class 4 profits. The model sets losses against the current year and then carries them forward.
• Para. 3(5) deductions are not modelled. These are trade-related interest and certain royalty payments. There are no inputs for them.
• Allowances in excess of profit do not create a loss. Capital allowances above profit floor Chapter 2 profits at nil; supply any resulting loss through trading_loss. Losses of a trade under trading-allowance full relief are not nilled (s. 783AF), as in Stop stacking the trading allowance on expenses already netted from profit #1881.
• trading_loss carries over for income tax. It is an input the engine carries to later years that are not set (auto_carry_over_input_variables). Class 4 now counts a loss only in the year it is supplied for (ni_class_4_trading_loss), but income tax's loss_relief still reads the carried value, so a one-off loss recurs there unless later years are set to zero (follow-up).
• Class 2 "relevant profits" (SSCBA s. 11(3)) are also Class 4 profits, but ni_class_2 still uses self_employment_income. See follow-up; Class 2 NICs wrong for 2022-23 and 2023-24 (rate, lower profits threshold) and 2015-17 (small profits threshold) #1887 covers other Class 2 issues.
• Negative profit. A negative self_employment_income creates no loss, for Class 4 or for income tax in this model; supply losses through trading_loss.
• Losses are pooled across trades. ITA 2007 s. 83 carries a loss forward only against the same trade, but the model has no identity for each trade.
• Engine input semantics. Supplying ni_class_4_losses_brought_forward for one person in a multi-person situation sets it, as zero, for everyone else in that year. This is standard for any formula variable supplied as an input.

Tests run

On the current head 9ca5d6eb (main merged, policyengine-core 3.32.13): the four Class 4 and trading allowance test files (42 tests) and the code-health tests pass, 1,830 in all; the 518 YAML cases under tests/policy/baseline/gov/hmrc pass; lint is clean. Before that merge, the review corrections were validated from fb192e1f13ac on core 3.32.9 with all Python commands through uv run --frozen --extra dev and one test process at a time:

• Before the provenance fix, the plain-clone regression and both delete-then-recalculate cases failed with £20,000 Class 4 profits and £445.80 liability; after the fix they pass with £30,000 and £1,045.80. The clone alone has £25,000 of profits, and neither simulation repeats the loss in 2027.
• Focused clone, deletion, missing-array and historical allowance regressions: 5 passed.
• pytest policyengine_uk/tests/test_ni_class_4_maximum.py policyengine_uk/tests/test_ni_class_4_profits_properties.py policyengine_uk/tests/test_ni_class_4_properties.py policyengine_uk/tests/test_trading_allowance_properties.py -v: 37 passed.
• pytest policyengine_uk/tests/code_health -q: 1,768 passed.
• policyengine-core test policyengine_uk/tests/policy/baseline/gov/hmrc -c policyengine_uk: 508 passed.
• make format: 1,149 files unchanged; lint passed.

Earlier validation, retained for context:

On ddfdfff6 (before the last merge of main, which touched no file here):

• pytest test_ni_class_4_maximum.py test_ni_class_4_profits_properties.py test_ni_class_4_properties.py test_trading_allowance_properties.py tests/code_health: 1,788 passed.
• policyengine-core test policyengine_uk/tests/policy/baseline/gov/hmrc: 508 passed (on eb4f3c9d), including the Class 4 cases and the new one-off loss case.
• ruff format --check . and ruff check .: clean.

Independent review

An Opus 5.5 reviewer ran on a Subfleet review/standard lane (read-only; it could not execute code) and returned APPROVE. It read s. 15, Sch. 2 paras. 2–3, ITA 2007 s. 83 and the engine source, and hand-checked all 15 YAML expectations, including the reg. 100 Case 3 steps. It also checked the revenue JSON: totals, counts and ni_class_4_profits = taxable_self_employment_income are identical in every year. Its minor findings were addressed in 70cbd58e:

1. trading_loss recurs in later years — superseded for Class 4 by the current fix. This earlier review assigned the persistence repair to an income-tax follow-up. The current PR now fixes Class 4 itself: ni_class_4_trading_loss counts a loss only in its supplied year, and the loss fold carries forward only the unused balance. Income tax's loss_relief still reads the carried value and remains a follow-up, as described under Known limits.
2. Negative profit creates no Class 4 loss. This is kept deliberately and documented. Income tax in this model also ignores negative self_employment_income (adjusted_net_income uses the floored taxable_self_employment_income), so losses come only through trading_loss for both.
3. Carry-backs and claim choice are documented as not modelled, as is pooling across trades (finding 5).
4. Ordering and para. 3(5). The wording now follows s. 83(4): brought-forward losses come first. This makes no numerical difference, because the pool is combined. The para. 3(5) deductions are noted as not modelled.
5. Spiral-limit regression. There is now a fixed test: a 2016 loss queried in 2030. Restoring the recursive formula makes it fail (brought forward 0 instead of 20,000).
6. Docs intro. It no longer says Class 4 is paid alongside Class 2 after April 2024.

Finding 7 (the trading_loss label) is handled by the new documentation field on trading_loss.

A delta re-review of 70cbd58e on a second review/standard lane returned APPROVE. It found every finding fixed or reasonably documented, and it hand-checked the spiral test's arithmetic, including that the values are exact in float32. Its one nit, the Class 2 bullets in the NI docs, predates this PR and belongs with the Class 2 follow-up (#1887).

axiom: TheAxiomFoundation/rulespec-uk#379 queued

🤖 Generated with Claude Code

Review of 30 September (vahid-ahmadi) and 2 October (Codex backlog review)

1. A one-off loss was relieved again every later year. Fixed: ni_class_4_trading_loss is the loss supplied for the year itself, and relief, the carry-forward and the brought-forward fold read it. The YAML case asked for (a £10,000 loss in 2025 only, £40,000 profit in 2026 and 2027) now gives Class 4 profits of £30,000 then £40,000, and the property tests cover sparse years, calculation order, cache warming and branch inputs. A neutralized trading_loss is honoured. (An abolition test was dropped: PolicyEngine-UK builds no gov.abolitions parameters, so there is no abolition switch to honour.)
2. Stop stacking the trading allowance on expenses already netted from profit #1881 merged, and this branch has merged main since.
3. Class 4 rates and limits in some years are pre-existing and tracked separately: Class 4 lower profits limit is £11,908 in 2023-24; NICA 2022 s.2 sets £12,570 from 6 April 2023 #1968 and Fix Class 2 and Class 4 NICs against SSCBA 1992 for 2015-16 to 2026-27 #2058 (2022-23 blended rates, 2023-24 lower profits limit); NICs thresholds are uprated before the Budget 2025 freeze ends in April 2031 #1879's freeze was fixed by Hold NICs thresholds at 2026-27 levels through 2030-31 #1959.
4. Class 2 relevant profits use raw self-employment income: Class 2 NICs wrong for 2022-23 and 2023-24 (rate, lower profits threshold) and 2015-17 (small profits threshold) #1887 and Class 2 NICs test raw self-employment income, not the Class 4 profits s.11(3) makes relevant profits #2086.

Review corrections of 3 October

• Plain UK simulation clones now have separate supplied-input provenance and input-context state. Public input deletion removes provenance for the deleted arrays before carry-over can replace them, and the supplied-input helpers ignore missing stored arrays. Existing named-branch lookup is retained. The regressions reproduce both lifecycle defects on the pre-fix code.
• The independent Class 4 profit reference uses each year's trading-allowance parameter. The fixed £500-profit/£1,000-loss case for 2015 followed by £500 profit and no loss in 2016 consumes £500 of loss each year and carries nothing forward after 2016.
• A value deleted straight from its holder (Holder.delete_arrays) keeps core's record of it as an input (policyengine-core#559; core's own fix, policyengine-core#561, is open). The engine could then carry an earlier loss into the emptied year, and the stale record made Class 4 count that carried value as a second loss: the reviewer's case gave £20,000 of profits and £445.80 where £30,000 and £1,045.80 are right. Simulation.calculate now forgets such records for the variables the supplied-input helpers read (SUPPLIED_INPUT_VARIABLES) before the engine can refill the year, and the helpers raise for any variable outside that set. The new tests cover the default simulation and a branch, and a deleted loss both different from and equal to the carried one (9ca5d6eb).
• The revenue explanation distinguishes main's raw profit base from this PR's adjusted base, and the older review's Class 4 follow-up narrative is explicitly superseded above.

🤖 Generated with Claude Code

[vahid-ahmadi]

Reviewed. The approach is right, with one issue I'd fix before merging. It matches SSCBA 1992 s.15(1)(b) and Sch 2 para 3: Class 4 on Chapter 2 profits, after capital allowances, the trading allowance and the losses para 3 allows, and nothing is deducted twice. Hand probes matched: partial trading allowance relief; £20k profit less a £5k loss giving £15k and £145.80; a loss carried forward from 2014. Locally, NI and income tax YAML pass (227), the property modules pass (29) and the policy YAML suite passes (1,292).

1. Should fix (arguably blocking): a one-off loss is relieved again every later year. trading_loss carries over to later years automatically, and the new carry-forward loop carries the same loss forward too. The loop is at ni_class_4_losses_brought_forward.py:54; ni_class_4_loss_relief.py:29 reads it as well.
   • Probe: a £10k loss set only in 2025, with £40k profit in 2026 and 2027, should give Class 4 profits of £30k then £40k. The model gives £20k then £30k.
   • The property tests set a loss every year, so they can't see this.
   • Fix: read only explicitly set loss values from the holder, as the loop already does for the supplied balance. Then add a YAML case with a loss in one year only.
2. Should fix: the PR still includes all of Stop stacking the trading allowance on expenses already netted from profit #1881. Rebase once Stop stacking the trading allowance on expenses already netted from profit #1881 merges and run the full make test.
3. Pre-existing, for separate issues: Class 4 parameters in some years.
   • 2022-23: rates should be the blended 9.73% and 2.73% after the levy repeal.
   • 2023-24: the additional rate should be 2% and the lower profits limit £12,570.
   • 2027 onwards: the limits are CPI-uprated despite the freeze to 2031.
   • The property tests read the model's own parameters, so they can't catch these.
4. Nit: Class 2 "relevant profits" still use raw self-employment income (already noted as Class 2 NICs wrong for 2022-23 and 2023-24 (rate, lower profits threshold) and 2015-17 (small profits threshold) #1887).

[juaristi22]

Reviewed commit 70cbd58eb7add067d9688697bd501e6a9142bc70. Needs changes; review coverage is partial. Retain the Class 4 profit-and-loss correction. It needs an explicit annual-loss input contract and a regression for a loss entered in only one year; the later rate/threshold PR #2058 does not replace this work.

Outstanding loss-persistence concern. The carry-forward loop at ni_class_4_losses_brought_forward.py:54 reads person("trading_loss", year) for each intervening year. trading_loss.py:8 documents that an omitted later year inherits the input and instructs users to enter zeros for a one-off loss. That leaves the existing review's counterexample dependent on manual zeros: the reviewer reported that a £10,000 loss entered only in 2025 with £40,000 profit in each of 2026 and 2027 produced £20,000 and £30,000 of Class 4 profits, rather than £30,000 and £40,000. This audit rechecked the code path but did not rerun that numerical probe.

Repair and acceptance steps:

1. Define separately the current year's loss flow and the stock of unused losses. For the requested one-off semantics, read loss values explicitly supplied for that year rather than treating an inherited input value as another newly incurred loss. The holder-based handling already used for an explicitly supplied brought-forward balance provides a pattern to inspect; ensure current-year relief follows the same contract.
2. Add the 2025-only loss case without later zero entries. Assert annual profits before losses, loss relief, remaining carried balance and final Class 4 liability in 2026 and 2027. Derive liability from that year's parameters rather than copying the historical review's NIC figures.
3. Retain cases with genuine losses explicitly entered in consecutive years, partial exhaustion, no loss, a supplied opening balance and a loss more than ten years earlier. Check evaluation order and branch/cache behaviour so a prior calculation does not change what counts as explicitly supplied input.
4. Coordinate the included trading-allowance work with Stop stacking the trading allowance on expenses already netted from profit #1881 and the separate rates/thresholds work in Fix Class 2 and Class 4 NICs against SSCBA 1992 for 2015-16 to 2026-27 #2058. Rebase after dependencies are available and preserve their regression tests; do not close this PR as superseded solely because those rates changed.

Suggested post-repair checks:

uv run --no-sync pytest policyengine_uk/tests/test_ni_class_4_profits_properties.py policyengine_uk/tests/test_ni_class_4_properties.py policyengine_uk/tests/test_ni_class_4_maximum.py -q
uv run --no-sync policyengine-core test policyengine_uk/tests/policy/baseline/gov/hmrc/national_insurance -c policyengine_uk

No new execution occurred during this audit. The historical probe and passing suites belong to the linked prior review. The repaired one-off case, interaction with Income Tax loss handling, current-main conflict resolution and fresh hosted CI remain unverified.

Live posting check (2026-10-02T15:12:35.823669+00:00): same reviewed commit; GitHub reports CONFLICTING. All reported checks passed. These checks do not replace the remaining validation listed above.

[MaxGhenis]

Merged under decision d708 (merge after #1881 once CI is green on the head, an independent review approves it and the impact is stated) and Max's PE-UK merge-on-gates rule.

• Independent reviews: subfleet 20261003-074058-mt-review-1919 and 20261003-090728-mt-review-1919b raised the clone and deletion provenance defects, the per-year trading allowance in the property reference, and description fixes; build 20261003-120724-mt-build-1919b2 and a7abbd43 fixed them. 20261003-144124-mt-review-1919c cleared those at f377239a with one remaining finding (a loss deleted straight from its holder kept its input record); 9ca5d6eb fixed it, and the resumed review 20261003-152530-mt-review-1919c APPROVEs 9ca5d6eb with no remaining findings.
• Tests: on 9ca5d6eb, the four Class 4 and trading allowance test files and the code-health tests (1,830) and the 518 YAML cases under tests/policy/baseline/gov/hmrc pass locally; CI passes on 9ca5d6eb.
• Impact: real runs on the Enhanced FRS 2024-25 (1.57.4), main 5232f9af against 9ca5d6eb: no total moves in 2025-2028. The dataset carries none of the capital allowance, trading loss or loss-brought-forward inputs, so the change shows only where those are supplied.
• Follow-up: core's own fix for input records left behind by deletion is policyengine-core#561 (open); this PR's guard covers the three variables it reads.