Migrate to policyengine.py 4.3.0 for household and microsim#5
Draft
PavelMakarchuk wants to merge 4 commits intomainfrom
Draft
Migrate to policyengine.py 4.3.0 for household and microsim#5PavelMakarchuk wants to merge 4 commits intomainfrom
PavelMakarchuk wants to merge 4 commits intomainfrom
Conversation
Installs policyengine[us]==4.3.0 (pinning policyengine-us==1.653.3) via Modal. Household path keeps the 401-point axes sweep through policyengine_us.Simulation (fastest viable) but swaps the Aspen reform to a parametric-only Reform.from_dict plus a server-side eitc_child_count > 0 patch that preserves current-law childless EITC, replacing the custom eitc_maximum Variable override. Microsim rewritten on policyengine.core.Simulation with ensure_datasets for the enhanced CPS, plain dict policy (compiled internally), and economic_impact_analysis-style outputs (calculate_us_poverty_rates with age filters, direct output_dataset.data access for winners/losers, decile, intra-decile, and AGI-bracket breakdowns). The public return shape is preserved so pipeline.py and the frontend CSV extractors work unchanged. National numbers will shift from the prior run since the new engine drops the MTR/DE/VA compatibility Variable overrides. Policy Overview page notes the pinned versions for transparency. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Drops the hand-rolled Lorenz/Gini loop and ad-hoc spm_unit→person broadcast in favour of policyengine.py's canonical helpers. Matches the methodology used in policyengine-app and policyengine-api. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Output of scripts/pipeline.py against the rewritten microsimulation.py (policyengine-us 1.653.3 pinned via policyengine[us]==4.3.0, with USInequalityPreset.CBO_COMPARABLE inequality and canonical calculate_us_poverty_rates). 10 years x static/dynamic variants. Example 2026 static: budget -$47.04B (fed -$48.94B, state +$1.90B), Gini 0.7607→0.7583, child SPM 23.20→21.70%. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Drops the policyengine-us pin and source-file reference; the policyengine package version is the single thing users need to see. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
policyengine[us]==4.3.0(pinningpolicyengine-us==1.653.3) through Modal.Reform.from_dict+ server-sideeitc_child_count > 0patch, replacing the customeitc_maximumVariable override. Kept the 401-point axes sweep (~11s) because loopingcalculate_householdper point extrapolates to >1 hour per request.policyengine.core.Simulationwithensure_datasetsfor the enhanced CPS and plain-dict policy. Usescalculate_us_poverty_rateswith age filters for poverty-by-age, own weighted Gini/top-share computation, and directoutput_dataset.dataaccess for winners/losers, decile averages, intra-decile gain/loss groups, and AGI-bracket breakdown. Publiccalculate_aggregate_impactreturn shape preserved soscripts/pipeline.pyand the frontend CSV extractors don't change.create_microsimulation_compatibility_reformcustom-Variable patches (MTR override, DE/VA branches, VT subtractions) in the new microsim path.create_aspen_reform()is still exported in its original 5-tuple form so the existingtests/test_aspen_reform.pycases that exercise DE refundable-EITC branches keep passing.Household parity (spot-checked on 3 example households × 7 income points)
Household calc is byte-identical to the old path except a single $139 gap at CA childless, $25k-reform, which is exactly the case the child-count patch now covers. Full comparison written up in the branch conversation.
National numbers will shift
Expected: the new engine is on
policyengine-us==1.653.3(was1.661.0) and no longer applies the MTR/DE/VA compatibility overrides. One-year smoke-run (static, 2026):Still need to rerun the full 10-year × static/dynamic pipeline and commit the regenerated CSVs in
frontend/public/data/. That's being kicked off now and will land as a follow-up commit on this branch.Test plan
python -m pytest tests/test_aspen_reform.py— 5/5 passcalculate_household_impactreturns correct baseline/reform/diff for all 3 example households (CA childless diff = $0 preserved)python scripts/pipeline.pyrun completes without OOM across all 10 years × 2 variants🤖 Generated with Claude Code