Skip to content

Migrate to policyengine.py 4.3.0 for household and microsim#5

Draft
PavelMakarchuk wants to merge 4 commits intomainfrom
policyengine-py-transparency
Draft

Migrate to policyengine.py 4.3.0 for household and microsim#5
PavelMakarchuk wants to merge 4 commits intomainfrom
policyengine-py-transparency

Conversation

@PavelMakarchuk
Copy link
Copy Markdown
Contributor

Summary

  • Installs policyengine[us]==4.3.0 (pinning policyengine-us==1.653.3) through Modal.
  • Household path: parametric-only Reform.from_dict + server-side eitc_child_count > 0 patch, replacing the custom eitc_maximum Variable override. Kept the 401-point axes sweep (~11s) because looping calculate_household per point extrapolates to >1 hour per request.
  • Microsim rewritten on policyengine.core.Simulation with ensure_datasets for the enhanced CPS and plain-dict policy. Uses calculate_us_poverty_rates with age filters for poverty-by-age, own weighted Gini/top-share computation, and direct output_dataset.data access for winners/losers, decile averages, intra-decile gain/loss groups, and AGI-bracket breakdown. Public calculate_aggregate_impact return shape preserved so scripts/pipeline.py and the frontend CSV extractors don't change.
  • Drops the create_microsimulation_compatibility_reform custom-Variable patches (MTR override, DE/VA branches, VT subtractions) in the new microsim path. create_aspen_reform() is still exported in its original 5-tuple form so the existing tests/test_aspen_reform.py cases that exercise DE refundable-EITC branches keep passing.
  • Policy Overview page notes the pinned versions for transparency.

Household parity (spot-checked on 3 example households × 7 income points)

Household calc is byte-identical to the old path except a single $139 gap at CA childless, $25k-reform, which is exactly the case the child-count patch now covers. Full comparison written up in the branch conversation.

National numbers will shift

Expected: the new engine is on policyengine-us==1.653.3 (was 1.661.0) and no longer applies the MTR/DE/VA compatibility overrides. One-year smoke-run (static, 2026):

  • Budgetary impact: −$47.0B (federal −$48.9B, state +$1.9B, benefits ~$0)
  • Winners 31M / losers 17.7M; avg benefit $+966
  • Child poverty 23.20% → 21.70% (−1.50pp); Gini 0.7511 → 0.7487

Still need to rerun the full 10-year × static/dynamic pipeline and commit the regenerated CSVs in frontend/public/data/. That's being kicked off now and will land as a follow-up commit on this branch.

Test plan

  • python -m pytest tests/test_aspen_reform.py — 5/5 pass
  • calculate_household_impact returns correct baseline/reform/diff for all 3 example households (CA childless diff = $0 preserved)
  • Full python scripts/pipeline.py run completes without OOM across all 10 years × 2 variants
  • Frontend National Impact tab renders correctly against regenerated CSVs
  • Frontend Household Impact tab matches old curves for an example household (verify via dev server before merge)

🤖 Generated with Claude Code

PavelMakarchuk and others added 4 commits April 23, 2026 18:18
Installs policyengine[us]==4.3.0 (pinning policyengine-us==1.653.3)
via Modal. Household path keeps the 401-point axes sweep through
policyengine_us.Simulation (fastest viable) but swaps the Aspen reform
to a parametric-only Reform.from_dict plus a server-side
eitc_child_count > 0 patch that preserves current-law childless EITC,
replacing the custom eitc_maximum Variable override.

Microsim rewritten on policyengine.core.Simulation with ensure_datasets
for the enhanced CPS, plain dict policy (compiled internally), and
economic_impact_analysis-style outputs (calculate_us_poverty_rates with
age filters, direct output_dataset.data access for winners/losers,
decile, intra-decile, and AGI-bracket breakdowns). The public return
shape is preserved so pipeline.py and the frontend CSV extractors work
unchanged. National numbers will shift from the prior run since the
new engine drops the MTR/DE/VA compatibility Variable overrides.

Policy Overview page notes the pinned versions for transparency.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Drops the hand-rolled Lorenz/Gini loop and ad-hoc spm_unit→person
broadcast in favour of policyengine.py's canonical helpers. Matches the
methodology used in policyengine-app and policyengine-api.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Output of scripts/pipeline.py against the rewritten microsimulation.py
(policyengine-us 1.653.3 pinned via policyengine[us]==4.3.0, with
USInequalityPreset.CBO_COMPARABLE inequality and canonical
calculate_us_poverty_rates). 10 years x static/dynamic variants.

Example 2026 static: budget -$47.04B (fed -$48.94B, state +$1.90B),
Gini 0.7607→0.7583, child SPM 23.20→21.70%.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Drops the policyengine-us pin and source-file reference; the
policyengine package version is the single thing users need to see.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant