UK cut: the personal allowance, from statute to a Manchester family to the whole UK

.github/workflows/ci.yml

@@ -9,11 +9,11 @@ permissions:
   contents: read
 
 env:
-  PYDEPS: --with pytest==9.1.1 --with hypothesis==6.168.1 --with numpy==2.5.3 --with scipy==1.18.1 --with pyloudnorm==0.2.0 --with pyyaml==6.0.3
+  PYDEPS: --with pytest==9.1.1 --with hypothesis==6.168.1 --with numpy==2.5.3 --with scipy==1.18.1 --with pyloudnorm==0.2.0 --with pyyaml==6.0.3 --with pyproj==3.8.0 --with shapely==2.1.2 --with pypdf==6.19.0
 
 jobs:
   invariants:
-    name: Invariants (published numbers, family curve, timeline helpers, score)
+    name: Invariants (published numbers, family curves, timeline helpers, score, UK geography)
     runs-on: ubuntu-latest
     steps:
       - uses: actions/checkout@v7
@@ -32,11 +32,13 @@ jobs:
           uv run --with pyyaml==6.0.3 tools/adapt_outputs.py > /dev/null
           uv run --with pyyaml==6.0.3 tools/build_video_data.py
           test -z "$(git status --porcelain -- data/)" || { git status; git diff; exit 1; }
-      - name: Published numbers, family curve, score (pytest + Hypothesis)
-        run: uv run $PYDEPS pytest -m "not slow"
+      - name: Published numbers, family curves, score, UK geography (pytest + Hypothesis)
+        run: |
+          uv run $PYDEPS pytest -m "not slow"
+          test -z "$(git status --porcelain -- data/)" || { git status; git diff; exit 1; }
 
   determinism:
-    name: Frames depend only on t (both layouts)
+    name: Frames depend only on t (both layouts, US and UK)
     runs-on: ubuntu-latest
     steps:
       - uses: actions/checkout@v7
@@ -50,3 +52,34 @@ jobs:
       - run: bunx playwright install --with-deps chromium
       - run: bun run build
       - run: bun run determinism
+      - name: Keep the mismatched frames
+        if: failure()
+        uses: actions/upload-artifact@v7
+        with:
+          name: determinism-frames
+          path: frames/determinism/
+
+  us-film-unchanged:
+    # Work on another country must not move a single US pixel. A PR that means to change the US film
+    # carries the us-film-changes label, and the job is skipped.
+    name: The US film is unchanged against the base branch
+    if: github.event_name == 'pull_request' && !contains(github.event.pull_request.labels.*.name, 'us-film-changes')
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v7
+        with:
+          fetch-depth: 0
+      - uses: actions/setup-node@v7
+        with:
+          node-version: 22
+      - uses: oven-sh/setup-bun@v2
+        with:
+          bun-version: 1.3.12
+      - run: bun install --frozen-lockfile
+      - run: bunx playwright install --with-deps chromium
+      - run: bun run build
+      - name: Build the base branch alongside
+        run: |
+          git worktree add ../base ${{ github.event.pull_request.base.sha }}
+          cd ../base && bun install --frozen-lockfile && bun build site/main.js --outfile site/bundle.js --format iife
+      - run: node tools/compare_revisions.mjs ../base us 1920x1080,1080x1920 0.2

.gitignore

@@ -19,3 +19,10 @@ review/*.png
 .DS_Store
 .hypothesis/
 data/raw/
+
+# UK: raw downloads (re-fetched by tools/uk/fetch_sources.py and the geography build)
+data/uk/raw/
+audio/**/*.wav
+
+# UK survey microdata and anything record-level derived from it (UKDS End User Licence: never redistributed)
+data/uk/private/

README.md

@@ -11,6 +11,8 @@ The rendered files are attached to the [latest release](../../releases/latest);
 | `out/policyengine-30s-vertical-1080x1920.mp4` | 1080×1920, 60 fps (Reels, Shorts, TikTok) |
 | `out/posters-16x9/`, `out/posters-9x16/` | 2× stills at 12.9, 18.9, 23.6 and 28.8 s for thumbnails |
 
+A UK version (`?country=uk`) runs the same timeline on UK law: the Income Tax Act 2007 personal allowance, the policyengine-uk parameter file, a single parent in Manchester across earnings, and the 650 Westminster constituencies. Its national figures come from policyengine.py 6.2.0 (policyengine-uk 2.102.3) on the enhanced Family Resources Survey 2024-25; the survey microdata and the map's record-level draws stay out of the repo (`data/uk/private/`, git-ignored), so a fresh clone renders the UK map with placeholder dots under the MOCK DATA banner. Sources and checks for every UK string are in [`data/uk/README.md`](data/uk/README.md); `tools/render_all.sh uk` writes the files to `out/uk/`.
+
 ## The story
 
 | Time | Scene | Source of what's on screen |
@@ -77,6 +79,7 @@ These hold for every input, and `bun run test` plus CI check them (`tests/`, `.g
 - `site/` — a deterministic HTML/canvas timeline. `window.renderAt(t)` draws the frame at time `t`; nothing depends on wall-clock time or `Math.random`.
 - `tools/render.mjs` — Playwright drives headless Chromium frame by frame. `--stream out.mp4` pipes PNG screenshots straight into parallel ffmpeg encoders and joins the segments, so no frames touch the disk.
 - `tools/soundtrack.py` — the score is synthesized with numpy/scipy at 120 BPM (D minor; Bb → C → F under the wordmark), mastered to −14 LUFS integrated with a 4× oversampled true-peak limiter at −2 dBTP. The loud transients are shaped where they are made, so the limiter only shaves strays: the drum bus is soft-clipped (4× oversampled, found through the master's high-pass) with kick and clap eased back on the backbeats, and the hit and drop smoothly dip themselves and the music (and the drums for longer) instead of rippling the whole mix with the limiter. Sound effects come from `events.json`, which the page exports from the same timeline, so each click and soft mallet note lands on the frame that causes it; gainer notes rise in pitch with the $800 step, and the family curve brightens a held chord as the gain rises.
+- `tools/compare_revisions.mjs` — renders a film from this checkout and from another (in CI, the PR's base) every 0.2 s in both layouts and compares them. On pull requests CI runs it for the US film, so work on another country cannot move a US pixel; a PR meant to change the US film carries the `us-film-changes` label.
 - `tools/determinism.mjs` — proves a fresh page and a page that played the film up to *t* render identical pixels at 11 timestamps, in both layouts. Streaming workers start mid-film, so this is what makes parallel rendering safe.
 - `tools/adapt_outputs.py` → `tools/build_video_data.py` — turn the raw PolicyEngine outputs into `data/video.json`, the only data the page reads. If any input is missing the page paints a striped MOCK DATA banner on every frame.
 
@@ -90,12 +93,14 @@ bun run score         # timeline cues -> audio/events.json -> audio/score.wav
 bun run determinism   # fresh-page frames == played-through frames, both layouts
 bun run render        # 4K/60 master, 1080p/60, 9:16 (supersampled from 2x), posters
 bun run check         # specs, loudness, true peak, single-frame glitch scan
-bun run test          # invariants (Vitest + fast-check, pytest + Hypothesis)
+bun run test          # invariants (Vitest + fast-check, pytest + Hypothesis), US and UK
 ```
 
+UK: `uv run --no-project --with pyyaml python tools/uk/build_uk_video.py` asserts every on-screen UK statement and writes `data/uk/video.json`; the score comes from `node tools/render.mjs --country uk --stills 12 --out frames/probe --events audio/uk/events.json` and then `tools/soundtrack.py audio/uk/events.json audio/uk/score.wav`, and `tools/render_all.sh uk` renders.
+
 Recomputing the PolicyEngine outputs themselves (`data/compute/*.py`) needs the pinned environment in `data/compute/requirements.lock.txt` and the dataset from Hugging Face; each script's docstring gives its run command.
 
-Preview any moment in a browser: `node tools/serve.mjs 4317`, then open `http://127.0.0.1:4317/site/index.html?t=17.5` (or `?play=1`, or `?w=1080&h=1920` for portrait).
+Preview any moment in a browser: `node tools/serve.mjs 4317`, then open `http://127.0.0.1:4317/site/index.html?t=17.5` (or `?play=1`, or `?w=1080&h=1920` for portrait, or `&country=uk`).
 
 ## Credits
 
@@ -106,8 +111,15 @@ Preview any moment in a browser: `node tools/serve.mjs 4317`, then open `http://
 - Microdata: PolicyEngine `populace-us` (`populace_us_2024.h5@populace-us-2024-spm-20260915`, MIT, huggingface.co/datasets/policyengine/populace-us). `data/households_sample.json` holds derived fields for 6,976 of its household records.
 - Congressional district boundaries (`data/district_geography.json`, used to place dots): US Census Bureau cartographic boundary file `cb_2024_us_cd119_20m` (public domain), via `PolicyEngine/policyengine-app-v2`.
 - Congressional district hex layout (`data/district_layout.json`; kept for the record, not shown in the video): House hexmap v3.1 by Daniel Donner, Daily Kos Elections / The Downballot (the-downballot.com; original release dkel.ec/map), via `PolicyEngine/snap-district-map` and `PolicyEngine/policyengine-app-v2`, licensed CC BY 4.0. Changes: district IDs re-keyed to PolicyEngine GEOIDs, centroids and bounding boxes added; polygons unmodified.
+- UK version (`data/uk/`, details and checks in `data/uk/README.md`):
+  - `data/uk/amount.yaml`: verbatim from `PolicyEngine/policyengine-uk` (`policyengine_uk/parameters/gov/hmrc/income_tax/allowances/personal_allowance/amount.yaml`) at commit 412b25aa, AGPL-3.0.
+  - Statute text (`data/uk/statute.json`): Income Tax Act 2007, Part 3, and Finance Act 2021 s. 5, from legislation.gov.uk. Contains public sector information licensed under the Open Government Licence v3.0.
+  - Edward VI, "Discourse on the Reformation of Abuses" (1551, public domain), as printed in J. G. Nichols (ed.), *Literary Remains of King Edward the Sixth*, vol. 2 (Roxburghe Club, 1857), p. 486, via the Internet Archive.
+  - UK national figures: Department for Work and Pensions. (2026). Family Resources Survey, 2024-2025. [data collection]. UK Data Service. SN: 9563, DOI: http://doi.org/10.5255/UKDA-SN-9563-1 (© Crown copyright; UK Data Service End User Licence), as enhanced by PolicyEngine's `policyengine-uk-data` 1.56.16 (`enhanced_frs_2024_25`), which also imputes from the Wealth and Assets Survey, the Living Costs and Food Survey, the Effects of Taxes and Benefits data and HMRC's Survey of Personal Incomes (see `data/uk/README.md`). The data creators, depositors and the UK Data Service bear no responsibility for the analysis here. Only aggregates are published; no record-level data is in this repository.
+  - Westminster constituency boundaries, July 2024 (`data/uk/geography.json`, simplified), and the ONS region and country lookups. Source: Office for National Statistics licensed under the Open Government Licence v.3.0. Contains OS data © Crown copyright and database right 2024.
+  - The UK family curve is computed from single households defined in `tools/uk/uk_family.py`. The national figures come from the enhanced Family Resources Survey 2024-25 (policyengine-uk-data 1.56.16) through policyengine.py 6.2.0; the FRS is UK Data Service data under its End User Licence, so only aggregates are committed.
 - Fonts: Inter, JetBrains Mono and Newsreader (SIL Open Font License, via Fontsource). US state shapes: `us-atlas` (Census cartographic boundaries). The score is synthesized by `tools/soundtrack.py`.
 
 ## License
 
-Code in this repository is released under the [MIT License](LICENSE). Original text and figures are released under [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/) with attribution to PolicyEngine. Third-party material keeps its own terms (see Credits): `data/base.yaml` (AGPL-3.0), `data/district_layout.json` (CC BY 4.0, Daily Kos Elections / The Downballot), and the PolicyEngine name and logo.
+Code in this repository is released under the [MIT License](LICENSE). Original text and figures are released under [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/) with attribution to PolicyEngine. Third-party material keeps its own terms (see Credits): `data/base.yaml` and `data/uk/amount.yaml` (AGPL-3.0), the UK statute text and boundaries (Open Government Licence v3.0), `data/district_layout.json` (CC BY 4.0, Daily Kos Elections / The Downballot), and the PolicyEngine name and logo.

audio/uk/events.json

@@ -0,0 +1 @@
+[{"t":0,"type":"pour","dur":1.4},{"t":0.2,"type":"word","i":0},{"t":0.3,"type":"word","i":1},{"t":0.4,"type":"word","i":2},{"t":0.5,"type":"word","i":3},{"t":0.6,"type":"word","i":4},{"t":0.7,"type":"word","i":5},{"t":0.8,"type":"word","i":6},{"t":0.9,"type":"word","i":7},{"t":1,"type":"word","i":8},{"t":1.1,"type":"word","i":9},{"t":1.2,"type":"word","i":10},{"t":1.3,"type":"word","i":11},{"t":1.4,"type":"word","i":12},{"t":1.5,"type":"word","i":13},{"t":2.75,"type":"whoosh","dur":0.75},{"t":3.5,"type":"lock"},{"t":4,"type":"whoosh","dur":0.8499999999999996},{"t":4.25,"type":"key","i":0},{"t":4.282,"type":"key","i":1},{"t":4.314,"type":"key","i":2},{"t":4.346,"type":"key","i":3},{"t":4.378,"type":"key","i":4},{"t":4.41,"type":"key","i":5},{"t":4.442,"type":"key","i":6},{"t":4.474,"type":"key","i":7},{"t":4.506,"type":"key","i":8},{"t":4.538,"type":"key","i":9},{"t":4.57,"type":"key","i":10},{"t":4.602,"type":"key","i":11},{"t":4.634,"type":"key","i":12},{"t":4.666,"type":"key","i":13},{"t":4.698,"type":"key","i":14},{"t":4.73,"type":"key","i":15},{"t":4.762,"type":"key","i":16},{"t":4.794,"type":"key","i":17},{"t":4.826,"type":"key","i":18},{"t":4.85,"type":"land"},{"t":4.858,"type":"key","i":19},{"t":4.89,"type":"key","i":20},{"t":5.3,"type":"ping","note":0},{"t":6.85,"type":"key","i":40},{"t":6.91,"type":"key","i":41},{"t":6.97,"type":"key","i":42},{"t":7.03,"type":"key","i":43},{"t":7.09,"type":"key","i":44},{"t":7.25,"type":"ping","note":4},{"t":9.1,"type":"curve","samples":[[9.1,0],[9.145,0],[9.19,0],[9.235,0],[9.28,0],[9.325,0],[9.37,0],[9.415,0],[9.46,0],[9.505,0],[9.55,0],[9.595,128.7],[9.64,218.7],[9.685,218.7],[9.73,218.7],[9.775,218.7],[9.82,218.7],[9.865,218.7],[9.91,218.7],[9.955,218.7],[10,218.7],[10.045,218.7],[10.09,218.7],[10.135,486],[10.18,882],[10.225,972],[10.27,972],[10.315,972],[10.36,972],[10.405,972],[10.45,972],[10.495,972],[10.54,972],[10.585,972],[10.63,972],[10.675,972],[10.72,972],[10.765,972],[10.81,972],[10.855,972],[10.9,972]],"gmax":972},{"t":11,"type":"ping","note":0},{"t":11,"type":"rise","dur":0.9499999999999993},{"t":12,"type":"hit"},{"t":14,"type":"drop"},{"t":14.88,"type":"plink","n":109,"step":1},{"t":14.94,"type":"plink","n":599,"step":1},{"t":15,"type":"plink","n":643,"step":1},{"t":15.06,"type":"plink","n":689,"step":1},{"t":15.12,"type":"plink","n":640,"step":1},{"t":15.18,"type":"plink","n":613,"step":1},{"t":15.24,"type":"plink","n":425,"step":1},{"t":15.42,"type":"plink","n":384,"step":2},{"t":15.48,"type":"plink","n":592,"step":2},{"t":15.54,"type":"plink","n":556,"step":2},{"t":15.6,"type":"plink","n":605,"step":2},{"t":15.66,"type":"plink","n":545,"step":2},{"t":15.72,"type":"plink","n":561,"step":2},{"t":15.78,"type":"plink","n":63,"step":2},{"t":15.9,"type":"plink","n":110,"step":3},{"t":15.96,"type":"plink","n":348,"step":3},{"t":16.02,"type":"plink","n":350,"step":3},{"t":16.08,"type":"plink","n":357,"step":3},{"t":16.14,"type":"plink","n":324,"step":3},{"t":16.2,"type":"plink","n":389,"step":3},{"t":16.26,"type":"plink","n":216,"step":3},{"t":16.5,"type":"stat","i":0},{"t":17,"type":"stat","i":1},{"t":17.5,"type":"stat","i":2},{"t":20,"type":"whoosh","dur":0.9},{"t":21,"type":"decile","n":3,"v":0.10256410256410256},{"t":21.07,"type":"decile","n":4,"v":0.16056166056166057},{"t":21.14,"type":"decile","n":5,"v":0.2081807081807082},{"t":21.21,"type":"decile","n":6,"v":0.38461538461538464},{"t":21.28,"type":"decile","n":7,"v":0.4719169719169719},{"t":21.35,"type":"decile","n":8,"v":0.5317460317460317},{"t":21.42,"type":"decile","n":9,"v":0.6514041514041514},{"t":21.49,"type":"decile","n":10,"v":0.8034188034188035},{"t":21.56,"type":"decile","n":11,"v":0.8962148962148963},{"t":21.63,"type":"decile","n":12,"v":1},{"t":25,"type":"swell","dur":1},{"t":26,"type":"logo"}]