Set UC gainful self-employment from the FRS main-job status

[MaxGhenis]

Model side: PolicyEngine/policyengine-uk#2081 (stacked on #1973), which adds the person input uc_is_in_gainful_self_employment (UC Regs 2013 reg 64). Merge after #2081 merges, so the variable name is final. policyengine-uk releases without the variable skip the column, so outputs on today's lock (2.93.0) are unchanged.

Summary

Without an input, #2081 decides gainful self-employment from income: any self-employment profit or loss counts, and none means none. This build floors losses at zero, so in the data "none" covers break-even traders and loss-makers, and the minimum income floor never applies to them. The formula also gives the floor to people whose main job is as an employee but who have a small side trade.

This PR sets the input from the FRS. It is a survey proxy for reg 64 that overrides #2081's income-based default:

Person	uc_is_in_gainful_self_employment
Main job is self-employment (EMPSTATI 3 or 4), whatever its profit	true
Main job is not self-employment, but self-employment profit is above employment income	true
Anyone else	false

The SPI copy that impute_income stacks keeps its FRS donor's employment status but draws new incomes. The copy re-derives the flag from its own imputed incomes, so every row follows the same rule.

What the FRS says

All figures are aggregates of the FRS 2024-25 end-user files (UKDS SN 9563) at survey weights, before uprating.

The variables

• EMPSTATI is "Adult - Employment Status - ILO definition". Values: 1/2 full/part-time employee, 3/4 full/part-time self-employed, 5-11 unemployed or inactive (data dictionary, adult table).
• The FRS methodology glossary defines the main job as the one "the respondent says is the dominant activity. Where they cannot decide, the number of hours worked will determine which is the main job."
• Its definition of self-employment "include[s] anyone doing work for their own business, but which is currently unpaid."
• SEINCAM2 is "Adult - Gross Earn from Self-Emp Opt 2". It is derived, so it is never missing: it is zero for people with no self-employment.
• The glossary says self-employed income is "based on profits (where the individual considers themselves as running a business) or on estimated drawings otherwise ... Any losses are recorded as such."
• The question instructions ask for the profit or loss from the accounts or tax return (Profit1), and whether it was a profit or a loss (Profit2).

Adults whose main job is self-employment (EMPSTATI 3/4): 2,110 adults, 4.20m weighted

SEINCAM2	Adults	Weighted	What the job table shows
Positive	1,893	3.82m
Zero	176	0.31m	21% give a profit-or-loss figure of exactly zero (Profit1 = 0). 21% report zero income from the business (SEIncAmt = 0).
Negative	41	0.07m	Losses, which the glossary says are "recorded as such"

• Here self_employment_income = max(0, SEINCAM2), so the 41 loss-makers become zero.
• In total, 217 adults (0.38m weighted) with a self-employed main job reach the data with no self-employment income.

Side trades. 227 adults (0.49m weighted) report a self-employment profit although their main job is not self-employment. For 65 of them (0.14m), all employees, that income is above their employment income.

Why these rules

From DWP's Advice for Decision Making (ADM), chapter H4:

• A trade can be gainful at zero profit or a loss.
  • The test is "whether there is a profit seeking motive (regardless of whether or not a profit is actually made)" (H4013, with an example of a loss-making trade).
  • A business "not generating much by way of an income ... does not mean that they are no longer gainfully S/E" (H4054).
  • "If a self-employed claimant has a loss in any AP, the level of earnings for that AP will be nil and the minimum income floor will be applied" (H4503).
• Main employment.
  • Hours are the starting point (H4031). Earnings can outweigh them: self-employment is likely to be the main employment where the claimant "works more hours on employed earners employment but receives a greater proportion of their income from S/E activity" (H4034, example 1).
  • Where the job brings in more than the side trade, self-employment is unlikely to be the main employment (H4035(2); example 2, Ann, whose pub job earns more than her sewing). ADM weighs this as a likelihood; the proxy turns it into a rule.
  • The FRS main-job flag stands in for the activity and goal tests. The earnings comparison adds H4034.

What the survey cannot see. These limits are known:

• The decision maker's view of an individual business. A dormant trade with no work in the pipeline can fail reg 64(c) (H4055, example "Ira").
• A stated goal of finding a job instead (ADM Appendix 2, example 3).
• Hours and earnings in the same period. The flag is fixed from survey-year incomes (or the SPI copy's imputed incomes). Uprating reprices profit and pay by different indices, so in a projected year a flagged side trade can earn less than the job, or the reverse; the flag does not follow.

Whether the floor then applies is decided by #2081, not by this input:

• the all-work-related-requirements condition, reg 62(1)(b);
• the start-up period, reg 63.

Not in this PR

• Keeping losses in self_employment_income.
  • policyengine-uk adds self_employment_income into several incomes:
    • total_income;
    • the Housing Benefit, Income Support, tax credit and Council Tax Reduction incomes;
    • Pension Credit earnings;
    • the benefit cap earnings tests;
    • in_work.
      A negative value would set a loss against other income in every one of them.
  • The law differs by programme:
    • Means-tested benefits do not let a loss offset other earnings. Examples: HB Regs 2006 reg 38(10), UC Regs 2013 reg 57(2).
    • Tax credits and income tax do, with conditions and caps: SI 2002/2006 reg 3, ITA 2007 ss.64, 66 and 24A.
  • tests/test_non_negative_incomes.py keeps incomes non-negative in this repo.
  • For the floor, the loss only matters through the gainful test, and this input now answers that.
  • A separate loss input, read by each programme under its own rule, is follow-up work.
• The reg 63 start-up period (uc_is_in_startup_period, never set by this build). Since 23 September 2020 (SI 2019/1152), reg 63(1)(a) asks whether the floor has previously applied to the trade, not whether the trade is new. The claimant must also be taking active steps to raise earnings (63(1)(b)), and a second start-up period needs a different trade and five years since the first (63(2)). Follow-up.
• The SPI copy's status and income. The SPI income model draws incomes from age, sex and region alone, while the copy keeps its donor's employment status.
  • About 7% of adult copy rows with a self-employed status draw any profit.
  • About 5% of the other adult copy rows draw a profit.
  • This PR keeps the flag consistent with each row's own values. Making status and income agree is follow-up work.

Invariants (property-tested with Hypothesis)

1. Every self-employed main job is flagged.
2. A flagged person either has a self-employed main job or has self-employment profit above their employment income.
3. More profit never removes the flag, and more pay never adds it.
4. The vectorised rule equals an independently written per-person oracle, for every employment status and any container type.
5. After impute_income, every row (FRS and SPI copy) carries the rule applied to its own status and incomes.
6. A built base FRS and a built enhanced FRS must carry the column (the test fails, not skips, if a build drops it). On the base FRS the column equals the rule; on the enhanced FRS invariants 1 and 2 hold. Later stages reprice incomes, so the exact rule is checked on the base build only.
7. The FRS-only imputation stage, which rewrites columns on the SPI copy after the flag is set, touches neither the flag nor its inputs.

A mutation check made seven deliberate defects in the rule and in the SPI step, including one from the independent review (dropping full-time employees from the earnings test). All seven fail the tests.

Impact

All figures come from real policyengine-uk microsimulations, one run per row.

Setup

• Model: policyengine-uk#2081's head (94b7129b4), the only model that reads the input.
• Branch build: the enhanced FRS built from this branch (e05539e; the code at dc2abf7 differs only in tests, docs and the changelog). It used production settings: 512 epochs, one OA clone, seed 0, and cached calibration targets. It used uk-data's policyengine-uk 2.93.0 lock, so the build and calibration did not see the input. Build time 22 minutes, exit 0.
• Base: the same file without the column, so #2081's income-based default decides.
• In every run with the column, each year's uc_is_in_gainful_self_employment was checked to equal the dataset's value, for 2025-26 to 2030-31.
• On policyengine-uk 2.93.0, the file with the column and the file without it give identical Universal Credit, household net income, income tax and Housing Benefit for 2025-26 and 2026-27. Today's lock is unaffected.

Universal Credit spending change against the base, £bn

	2025-26	2026-27	2027-28	2028-29	2029-30	2030-31
This PR	-0.48	-0.50	-0.50	-0.49	-0.49	-0.49
Sensitivity: main-job status only, no H4034 earnings test	-0.41	-0.43	-0.43	-0.43	-0.43	-0.43

2026-27 in detail (this PR)

	Base	This PR
Universal Credit spending	£78.37bn	£77.88bn
People in gainful self-employment (uc_is_in_gainful_self_employment)	4.45m	5.71m
People the floor applies to (uc_mif_applies)	3.59m	4.57m
... of whom in benefit units receiving UC	302k	349k

• UC falls for 97k benefit units. The UC caseload falls by 37k benefit units, net.
• Where the change sits.
  • The net fall in the SPI copy's households is £0.34bn. It is driven by the floor newly applying to SPI-copy rows whose employment status says self-employed but whose SPI-imputed profit is zero. That status and income mismatch comes from the SPI income model (see "Not in this PR"). Smaller offsetting changes are inside the net figure and are not shown.
  • The net fall in the original FRS households is £0.16bn.
• Other programmes. Pension Credit, Housing Benefit and Council Tax Reduction do not change. Household net income falls by the same amount as UC.
• Survey support. Each published figure rests on at least 10 distinct survey households. A survey household counts once across its FRS row, SPI copy and clones. Exact support counts are not given, because differences between them could reveal smaller cells.
• Suppressed. Poverty changes and the split of benefit units whose UC rises rest on fewer than 10 survey households, so they are not shown.

Checklist

• Data: no record-level values in this PR. Cells under 10 survey households are suppressed.
• axiom: n/a: survey-data proxy for a policyengine-uk input; encoding reg 64 itself belongs to policyengine-uk#2081.

🤖 Generated with Claude Code

[MaxGhenis]

Hand-off to the UK hub (session 888dad4b → local_60f5f051)

• Head: dc2abf7b4cd52b9e63c3631020af6e7e0c79c6da.
• Checks: 4/4 pass on that head. MERGEABLE, out of draft.
• Independent review: four Subfleet rounds on Opus 5.5. Round 4 says APPROVE at dc2abf7 with the current body. Rounds 1-3 asked for changes to the code and body; all were made at this head or in this body.
• Impact: real policyengine-uk runs on #2081's head 94b7129b4: the branch build against the same file without the column. Universal Credit falls by £0.50bn in 2026-27 (see Impact above). On today's lock, policyengine-uk 2.93.0, the column changes nothing; checked on the real build.

Waiting on

1. Max's choice of rule, decision d865: A, as built; B, status only, −£0.43bn; or C, status or any income. If he picks B or C, change derive_uc_is_in_gainful_self_employment and rerun the impact.
2. The uk-data batch release on Max's go, decision d833.
3. Apply the UC minimum income floor only to claimants subject to all work-related requirements policyengine-uk#2081 merging first, which fixes the variable name. #2081 is stacked on #1973 and #1949. If #2081's head changes the variable or its formula before it merges, rerun the impact. The scripts are in the session's review folder (impact/run_impact.sh). The pinned model worktree is ~/PolicyEngine/_worktrees/pe-uk-2081-gainful-se-impact.

Merge note. #528 rewrites the self_employment_income line in datasets/frs.py, and this PR adds its call directly after that line. Whichever lands second resolves the conflict by keeping #528's split and then this PR's call.

Follow-ups already split out: #526 (EMPSTATI 11), #527 (start-up period), #528 (trading losses), and the SPI-status work (#529 earnings groups).