PR 7: adopt CosmoSIS native SACC likelihood (sacc_like), validated against the converter path#255
Draft
cailmdaley wants to merge 10 commits into
Draft
PR 7: adopt CosmoSIS native SACC likelihood (sacc_like), validated against the converter path#255cailmdaley wants to merge 10 commits into
cailmdaley wants to merge 10 commits into
Conversation
…sis_fitting
Add sp_validation.twopoint_convert.sacc_to_twopoint_fits, converting an analysis
SACC into the CosmoSIS 2pt-FITS that 2pt_like (and Sacha's rho/tau
2pt_like_xi_sys fork) reads. Output reproduces today's cosmo_inference
cosmosis_fitting.py assembly HDU-for-HDU and byte-for-byte: NZDATA, XI_PLUS/MINUS,
CELL_EE, the blocked COVMAT (STRT_i offsets) and separate COVMAT_CELL, and the
TAU_{0,2}_PLUS + verbatim RHO_STATS tables.
The data vector and covariance are permuted from SACC's pair-major order to the
2pt-FITS type-major layout via s.indices. The tau covariance is laid in as one
contiguous [tau_0+; tau_2+] block, preserving the tau_0<->tau_2 cross-correlation
that covdat_to_fits carries. Rho/tau HDUs need the rho/tau sidecar FITS: the
RHO_STATS varrho_* variance columns are not stored in the analysis SACC, so the
converter copies them verbatim (never fabricates them); xi, Cl, n(z) and the
covariance are fully reconstructed from SACC alone.
Tests:
- test_twopoint_convert.py: byte-equal vs current cosmosis_fitting.py builders on
deterministic synthetic inputs for all three product shapes (plain xi; xi+Cl;
xi+rho/tau), plus the tau_0<->tau_2 cross-correlation and perturbation teeth.
- test_twopoint_convert_realdata.py (skipif on candide paths): build a SACC from a
real product's own contents, convert, byte-compare vs the current writer.
Observed byte-equal for SP_v1.4.6_leak_corr and a glass_mock sibling; the stale
on-disk files differ only by an extra CELL_BB HDU (older script version) and
~1e-17 float noise on bin edges, documented and guarded.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Add sp_validation.one_covariance_io, the two small pieces of glue between the SACC layout and OneCovariance (which knows nothing of SACC and only exchanges files): - write_nz / nz_table / nz_config_stanza: the source_i NZ tracers of an analysis SACC -> OneCovariance's combined whitespace n(z) file (column 0 = shared z grid, one n(z) column per tomographic bin, no bin edges) plus the [redshift] config stanza pointing at it. Config keys verified against upstream OneCovariance config.ini (zlens_directory/zlens_file/value_loc_in_lensbin); the UNIONS template (cosmo_val/pseudo_cl.py._modify_onecov_config) uses the z_directory alias, exposed via dir_key. Fails fast on missing bins or disagreeing z grids. - covariance_blocks: OneCovariance's flat covariance_list_*.dat output -> dense square block(s) paired with SACC selectors for sacc_io.assemble_covariance, reusing statistics.cov_from_one_covariance for the per-block reshape (col 10 gaussian / col 9 gauss+non-gaussian). Single-block and multi-block (tomography-ready) forms. Tests (test_one_covariance_io.py, 9 passing): reshape to a hand-built matrix + gaussian/gauss+ng column selection + perturbation teeth; blocks feed assemble_covariance and round-trip s.covariance.dense; n(z) write/read round-trip, UNIONS dir_key, header/no-header, and fail-fast on mismatched grids / missing bin / bad value_loc. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ather
Adversarial-review hardening (one HIGH, one MEDIUM, one coverage gap):
- HIGH: n_bins>1 silently truncated — n_bins drove only the NZDATA column
count while data/covariance were read from bin (0, 0), so a 2-bin SACC
emitted a plausible-looking FITS carrying 1/3 of the data. The converter
now fails fast unless n_bins == 1 and the ξ tracer pairs are exactly
{(source_0, source_0)}; tomographic emission lands with the tomographic
round.
- MEDIUM: a ξ-less SACC wrote an empty XI_PLUS and a (0, 0) COVMAT
silently; now a loud ValueError.
- Coverage: every prior covariance test exercised the identity permutation
(single-pair SACC order is already type-major). A (row, col)-encoded
covariance test now pins the exact np.ix_ gather of COVMAT (block-diag
ξ + joint non-adjacent [τ0+; τ2+]) and COVMAT_CELL — any transposition,
offset, or swapped block fails.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016Ko89uZF84ez6HEDcTWbJe
Adopt CosmoSIS's native SACC likelihood for the ξ± inference path through a thin
sp_validation-owned subclass that fixes two upstream defects:
1. arcmin→rad theta-tag conversion — upstream sacc_like builds its theory
spline on block[section,"theta"] (radians) but evaluates it at raw SACC
theta tags (arcmin), silently returning theory≈0 and collapsing χ². 2pt_like
converts to radians for exactly this reason; sacc_like never does.
2. an ordering guard — the data vector (get_mean, insertion order) and the
theory loop (get_data_types × tracer_combinations × points) agree only for
type-major files; a pair-major tomographic file would silently misalign.
The guard reconstructs the theory-loop order and requires it == arange.
Factory pattern (setup imports the upstream class from csl_dir and subclasses at
call time) so importing the module never needs cosmosis; not imported by
__init__. build_data calls super() first (scale cuts run in arcmin, matching
2pt_like's angle_range grammar) then converts + guards.
Tests (cosmosis + CSL_DIR gated, skip cleanly otherwise) prove in-process
equality against 2pt_like on the PR-3 converter FITS: machine-zero equality on
synthetic (χ²=1.18829133107, Δχ²=0) and real data (χ²=420310.753739, Δχ²=0, N=40),
the unit-gap tripwire (raw sacc_like 631× off), scale-cut equivalence, identical
perturbation response, the ordering guard, and real-type-scoped theta conversion.
pyproject: cosmosis>=3.25 extra (inert in CI, which installs [test,glass,blinding]).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EnV8NPWhkxS2SxGVgGSJyt
Add the CosmoSIS pipeline ini template for the native SACC path (cosmosis_pipeline_A_ia_sacc.ini): copies the A_ia template, swapping load_nz_fits→load_nz_sacc (native n(z) from the SACC's source_i NZ tracers) and 2pt_like→sacc_like (the shim, via SP_VALIDATION_MODULES, csl_dir=COSMOSIS_DIR, like_name=2pt_like for block-key parity). Same numeric scale cuts as A_ia, in the sacc_like angle_range grammar (full data-type names + source_0 pairs). generate_inference_config.py fills a template's [DEFAULT] section with concrete paths (SCRATCH, FITS_FILE|SACC_FILE, COSMOSIS_DIR, SP_VALIDATION_MODULES resolved from sp_validation.__file__). Dual-mode (snakemake object OR argparse CLI) like assemble_sacc.py; plain text processing (the pipeline.sh sed idiom) that REPLACES an existing DEFAULT key in place and prepends only new ones — no configparser round-trip (which would strip comments + %(...)s interpolation) and no duplicate DEFAULT key (which CosmoSIS's strict parser rejects). Tests (no cosmosis needed): both templates fill + interpolate cleanly, the referenced module file exists, no %(...)s placeholder survives, missing-DEFAULT raises. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EnV8NPWhkxS2SxGVgGSJyt
Revive the real-data inference_prep from its dormant, pre-SACC state. It now
consumes the assembled analysis {version}.sacc (cosmo_val.smk's assemble_sacc,
bound lazily through cv_analysis_sacc) and emits the A_ia (IA-only, ξ±) file-prep
products:
(a) the converter 2pt-FITS (sacc_to_twopoint_fits, pure ξ — no rho/tau
sidecars, A_ia scope) + a generated 2pt_like ini (the validating/legacy
path), and
(b) a generated sacc_like ini pointing at the SACC (the native path, validated
bit-for-bit against (a)).
The cosmosis_fitting.py real-data assembly is retired from this rule; the
glass-mock rules keep it (their SACC migration is out of scope, noted inline).
inference_fiducial extends to all three prep outputs.
CSL_DIR is read lazily (inference.smk is parsed by every paper workflow, but only
cosmo_val carries inference.csl_dir) — a missing key still fails loudly, just at
DAG time rather than at parse time for an unrelated (bmodes) workflow.
config.yaml: add inference.csl_dir. The workflow dry-run guard gains a test
covering the revived inference_prep DAG (SACC in, converter FITS + both engine
inis out, no cosmosis_fitting.py real-data assembly).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EnV8NPWhkxS2SxGVgGSJyt
…rcmin tags Review finding (MEDIUM): the shim converted theta tags arcmin→rad in place on self.sacc_data. Upstream do_likelihood's save_theory / save_realization paths copy self.sacc_data and overwrite only point values, so they were writing radian theta tags into the saved SACC — any arcmin-assuming consumer (sacc_io.get_xi, or re-ingesting the file as data_file, which double-converts to ~8.5e-8) is then silently off by 3437×. Keep self.sacc_data in arcmin always. build_data now builds a separate self._sacc_data_rad copy with the real-category theta tags converted, and extract_theory_points swaps it in around super().extract_theory_points(block) in a try/finally — so the theory spline sees radians while everything after (save_theory / save_realization) sees the untouched arcmin original. The ordering guard runs on the arcmin object (ordering is unit-independent). Theory equality is unchanged (the swap produces identical results): synthetic χ²=1.18829133107, real-data χ²=420310.753739, both Δχ²=0 vs 2pt_like. New test: run with save_theory set, reload, assert the saved theta tags equal the input file's arcmin tags exactly (and are NOT the radian conversion) and the saved values equal the theory vector; assert a second execute() yields the identical χ² (guards against a double-conversion regression). The theta-scoping test now checks the rad copy carries radians AND the original stays arcmin. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EnV8NPWhkxS2SxGVgGSJyt
Review finding (MEDIUM): the two pipeline ini templates were referenced as params, so there was no DAG edge — editing a template never regenerated the per-version config files. Bind both as rule inputs (referenced via input.* in the run block). The templates are source files, so anchor them on the running checkout (INFERENCE_TEMPLATE_DIR, off the workflow dir's parent) rather than the env-overridable COSMO_INFERENCE output root; the generated per-version configs still land in COSMO_INFERENCE (INFERENCE_CONFIG_OUT). In a normal run the two roots coincide; the split is what lets a template edit in this checkout drive the DAG. The dry-run guard now asserts both templates appear as inference_prep inputs. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EnV8NPWhkxS2SxGVgGSJyt
…cosmosis CI (test_imports.py::test_package_module_imports[sp_validation.sacc_like_unions]) failed: the CI image has the science stack but no cosmosis, and the shim imported `from cosmosis.datablock import SectionOptions, option_section` at module level. cosmosis is an optional dependency, so a bare `import sp_validation.sacc_like_unions` must succeed without it (test_imports bare-imports every top-level module). Move the cosmosis import into setup() (call time). Nothing at top level touches cosmosis now — only os/sys/numpy (all in the image). The factory's subclass of the upstream SaccClLikelihood already deferred (setup imports it from csl_dir), so setup() is the single cosmosis entry point. Verified: in the container (numpy present, cosmosis genuinely absent — the CI condition), `import sp_validation.sacc_like_unions` succeeds with setup/execute/ cleanup/ARCMIN_TO_RAD all present, and sacc_like_unions is absent from the import-sweep failures. Full function intact: test_sacc_like.py 8 passed under the venv (cosmosis present). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EnV8NPWhkxS2SxGVgGSJyt
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #254. Row 7 of PRD #241 — the final PR of the SACC/Smokescreen series: the inference pipeline consumes the assembled
{version}.saccnatively through CosmoSIS'slikelihood/sacc/sacc_like.py, with the PR-3 converter path (SACC → 2pt-FITS →2pt_like) retained as the cross-validating twin. Each path validates the other, per PRD §2.The "prototype-grade" verdict, grounded
The PRD flagged
sacc_like's ξ± path as prototype-grade. A source read of CSL @4fd2f1cfound the concrete mechanisms:sacc_likelihoods/twopoint.pybuilds the theory spline onblock[section, "theta"](radians, CosmoSIS convention) and evaluates it at raw SACCthetatags (arcmin, SACC/firecrown convention).2pt_likeexplicitly converts everything to radians (2pt_like.pyL179–183);sacc_likenever does — the spline is evaluated ~3437× outside its grid,SpectrumInterpreturns 0 there, and χ² silently collapses to dᵀC⁻¹d. Its only production use (HSC Y3) was the ℓ-space Cℓ path, which is unit-free — hence never caught.get_mean()(insertion order); theory is a type-major loop. A comment claimsto_canonical_orderwas called on load — it never is. Single-pair files align by construction; tomographic pair-major files would silently misalign.sacc_like.pyL106 references undefinedt1, t2in thekeep_tracersremoval path.)Adoption = subclass shim, not fork
src/sp_validation/sacc_like_unions.pysubclasses upstreamSaccClLikelihood(factory-loaded fromcsl_dir), containing exactly two fixes: theory extraction runs against a radian-θ copy of the loaded SACC (self.sacc_datastays arcmin, so scale cuts keep arcmin ini ergonomics and upstream'ssave_theory/save_realizationnever write radian tags), and a loud ordering guard that reconstructs the upstream theory-loop order and requires it to match insertion order. Everything else — scale-cut grammar, Sellentin/Hartlap, windows for Cℓ — rides upstream unmodified. n(z) loads natively via CSL'snumber_density/load_nz_sacc.like_name = 2pt_likekeeps every block key identical across engines, so chain post-processing is engine-agnostic.Validation (all observed, in-process, no sampler/CAMB needed)
Both likelihood modules run standalone against identical synthetic theory DataBlocks:
2pt_like-on-converter-FITS: synthetic (realistic covariance) χ² = 1.18829133107 both, Δχ² = 0, max|Δtheory| = 0; real data (SP_v1.4.6_leak_corr, ξ-only, N=40) χ² = 420310.753739 both, Δχ² = 0.sacc_likeon the same arcmin SACC is ~631× off — this test failing one day means upstream fixed units and the shim should be retired.n, spectrumelluntouched);save_theoryoutput verified to carry arcmin tags and theory values, with re-execution χ²-stable.Workflow revival
inference_prephonors PR 4's dormant-note contract: input = the assembled{version}.sacc(+ the two ini templates, DAG-tracked), outputs = converter 2pt-FITS + generated2pt_likeini + generatedsacc_likeini (workflow/scripts/generate_inference_config.py, dual-mode likeassemble_sacc.py).cosmosis_fitting.pyis retired from the real-data path; the glass-mock rules keep it, and the PSF/xi_sys pipeline variant stays on2pt_like(both flagged in comments — follow-up work, not silent scope creep).snakemake -n inference_fiducialresolves the full chain: parts →assemble_sacc→inference_prep→inference_fiducial.Environment
New inert
[cosmosis]extra (cosmosis>=3.25; pip-installs cleanly on py3.12). Tests gate onimportorskip("cosmosis")+ aCSL_DIRenv var pointing at a CSL checkout — they skip in CI (image carries neither) and run on candide. CI runs the rest of the suite as usual.Adversarial review
Fresh-context multi-lens review (5 lenses, 3 refuters per finding) before this draft opened; two MEDIUM findings confirmed and fixed: (1)
save_theory/save_realizationwrote radian θ tags into saved SACCs (in-place mutation leak — now a swap-in copy, regression-tested); (2) the ini templates were ruleparamsrather thaninputs, so template edits didn't propagate to generated configs (now DAG edges, asserted by the dry-run guard).Stack note
Base =
feat/sacc-4-cosmo-val-sacc(PR #251), withfeat/sacc-3-2pt-converter(PR #249) merged in — this PR needs both PR 4's workflow surface and PR 3's converter, so its diff shows #249's files until the stack collapses onto develop. Net-new in THIS PR:sacc_like_unions.py,test_sacc_like.py,test_generate_inference_config.py,cosmosis_pipeline_A_ia_sacc.ini,generate_inference_config.py; changed:inference.smk,config.yaml,pyproject.toml,test_bmodes_workflow_dry_run.py.Recommended follow-ups (team call): file the unit gap + the L106
keep_tracersbug upstream at joezuntz/cosmosis-standard-library; migrate the PSF/xi_sys variant and glass-mock prep; tomographic support in both the converter and the shim's guard-relaxation.— Claude (Fable) on behalf of Cail.
🤖 Generated with Claude Code
https://claude.ai/code/session_01EnV8NPWhkxS2SxGVgGSJyt