Observed firm frame: real sponsor records calibrated to SUSB (#192) - #390
Open
vahid-ahmadi wants to merge 2 commits into
Open
Observed firm frame: real sponsor records calibrated to SUSB (#192)#390vahid-ahmadi wants to merge 2 commits into
vahid-ahmadi wants to merge 2 commits into
Conversation
Builds the firm population from observed Form 5500 sponsor records instead of generating one. No synthetic firm row is created. Form 5500 alone populates all 97 SUSB sector x canonical-band cells with at least ten records each, so the frame uses a single consistent unit (plan sponsor) and OSHA ITA stays an independent measurement reference. Unioning the two would silently mix a sponsor unit with an establishment unit: they agree on the canonical band for only 66.3% of the 43,001 EINs they share. The calibration took three attempts and the first two are recorded because the failures are informative: 1. post_stratify (firm-count margin only) matches firms exactly and overshoots employment by 2.06x. A single weight per cell treats a 307,086-participant enterprise as representative of twenty ordinary 500+ firms. 2. Finer stratification on SUSB's 22 detail size classes makes it worse (2.52x), because SUSB's top class is also unbounded. 3. calibrate_dual_margin lets weights vary within a cell via a maximum-entropy tilt, matching firm and employment margins at once. Employment ratio 0.973 across 92 of 97 cells. The five remaining cells fail closed rather than being forced, and they are a SUSB data-quality artifact: noise infusion distorts thin published cells enough to make them arithmetically impossible for their own size class. NAICS 11's 2,000-2,499 class reports 5 firms and 292 employees (flag H); its 5,000+ class reports 28 firms and 6,778 (flag J). Where the implied cell mean falls outside the band it belongs to, no reweighting can reach it. Post-stratification is retained alongside the working method because its employment_coverage diagnostic is what falsified the naive design. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Follow-up to the frame's first commit, resolving the two defects it
left open. Both choices are measured rather than assumed.
## Weight bound: 140, from the sensitivity curve
The unbounded exponential tilt put a weight of 3,866 on one NAICS 99
record in a 35-record cell, beside another at 1.3e-08. Bounded
calibration is the Deville-Sarndal (1992) remedy the project already
cites. The bound is not a guess — p99 is 76.5, so bounds above ~140
never bind and give identical margins, while tighter ones degrade the
firm margin sharply:
bound firm ratio emp ratio
50 0.7023 0.9766
100 0.9558 1.0151
130 0.9960 1.0226
140 0.9988 1.0237
250 0.9988 1.0237
1000 0.9990 1.0237
140 is the tightest bound that costs nothing on either margin. It is a
referee parameter, like the OSHA employment cap, not a default to
inherit silently.
Fixed along the way: the first bounded implementation clipped and then
rescaled to hit the firm total, which pushes weights straight back over
the bound — it left two records at 4,138 while the bound was nominally
250. The final clip no longer rescales, and the firm-margin residual
that leaves is reported instead.
## NAICS 55 employment is excluded, firms are kept
NAICS 55 (Management of Companies) is a holding-company sector: one
Form 5500 covers the whole enterprise's workforce, while SUSB
attributes those workers to the operating sectors and records only
head-office staff under 55. Counting them again double-counts, and the
effect is not marginal: 42,676,524 weighted employees against SUSB's
3,661,977 (11.7x), which is 24% of all weighted employment. It is
systematic across every band (1.0x, 2.6x, 4.2x, 6.0x, 12.4x), not a
thin-cell artifact.
These sponsors stay in the firm margin — SUSB does count 25,413 of
them — but carry employment_in_scope=False. The flag travels on the
frame rather than only in attrs, so a downstream consumer cannot sum
double-counted employment by accident.
Also corrected: the five impossible cells were earlier attributed to
SUSB noise infusion. That was wrong — 507 of 532 rows carry the
low-noise G flag. They are a documented SUSB scope rule: NAICS 55 as
above, and NAICS 11 because SUSB excludes crop and animal production.
## Result, and a held-out check
firms 6,453,598 / 6,461,497 = 0.9988
employment (in-scope) 1.0237
weights min 0.100 median 2.99 max 140.0
Per band, in-scope: 0.9993 / 1.0000 / 1.0000 / 1.0024 / 1.0440.
Validated against BDS 2022, which is never used in the calibration,
with SUSB-vs-BDS as a control to separate our error from source
disagreement. Firm counts: our/BDS 1.176, 1.270, 1.892 against
SUSB/BDS 1.177, 1.270, 1.888 — our own error is 0.999, 1.000, 1.002.
The whole gap is SUSB and BDS disagreeing, not the calibration.
No random draw, sampling or generation anywhere in the firm path:
every row traces to a real DOL filing.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #386. Builds the firm population from observed Form 5500 sponsor records instead of generating one. No synthetic firm row is created; every row traces to a real DOL filing.
Why the frame can be observed rather than generated
Form 5500 alone populates all 97 SUSB sector x canonical-band cells with at least ten records each, so the frame uses a single consistent unit and OSHA ITA stays an independent measurement reference. Unioning the two would silently mix a plan-sponsor unit with an establishment unit — they agree on the canonical band for only 66.3% of the 43,001 EINs they share.
The calibration took three attempts; the failures are recorded
post_stratify— firm margin onlycalibrate_dual_margin— bounded max-entropy tiltPost-stratification matches firms exactly and overshoots employment by 2x: a single weight per cell treats a 307,086-participant enterprise as representative of twenty ordinary
500+firms. Finer stratification makes it worse, because SUSB's top class is also unbounded. The fix is to let weights vary within a cell so both margins are met at once.post_stratifyis retained because its diagnostic is what falsified the naive design.Two parameters, both measured
Weight bound 140. p99 is 76.5, so bounds above ~140 never bind and give identical margins; below it the firm margin degrades sharply (0.956 at 100, 0.702 at 50). 140 is the tightest bound that costs nothing. A referee parameter, like the OSHA employment cap.
NAICS 55 excluded from the employment margin, kept in the firm margin. Holding companies file one plan covering the whole enterprise while SUSB attributes those workers to the operating sectors — 42,676,524 weighted employees against SUSB's 3,661,977 (11.7x), 24% of all weighted employment, systematic across every band. Rows carry
employment_in_scope=Falseso a consumer cannot sum double-counted employment by accident.Result
Held-out validation against BDS 2022
BDS is never used in the calibration. SUSB-vs-BDS acts as a control separating our error from source disagreement:
Firm counts are faithful to three decimals; the entire gap vs BDS is the two sources disagreeing.
Corrections to earlier claims on #192
Gflag). They are a documented SUSB scope rule: NAICS 55 as above, NAICS 11 because SUSB excludes crop and animal production.Scope
Does not modify
gates.yaml, IC3 targets, floors, thresholds, partitions, or candidate fitting. The worker-to-firm assignment is unchanged and still constructed — no public source links workers to employers, and the phase-2 claims boundary on #282 stands.Relates to #192, #282, #386.