Smokescreen blinding: per-catalogue custody, blind drawn once, parts sealed at birth

[cailmdaley]

This PR blinds UNIONS cosmic-shear data vectors with the UNIONS-WL Smokescreen fork. On a blinded catalogue, ξ± and pseudo-Cℓ_EE are shifted by t(hidden) − t(fiducial) before they are first written. That is the theory at a secret cosmology minus the theory at the fiducial, so the true values never reach disk. Everything computed from them inherits the shift: COSEBIs, pure-E/B, ⟨M_ap²⟩ and the assembled per-version SACC. The aim is to prevent accidents in an honest collaboration, not to resist an adversary.

What blinding does to v1.4.6.3

The figure shows SP_v1.4.6.3_leak_corr under a throwaway blind, now revealed (ΔS8 = −0.060, ΔΩm = −0.043). The shift is a smooth, coherent change of amplitude:

• up to 2.7σ per bin in ξ±;
• up to 1.9σ in Cℓ_EE;
• in COSEBIs, only E₁–E₃.

Cℓ_BB, B_n and the pure-mode ξ^B are unchanged. The one exception is a pure-mode B bin at 1.5′ (see Testing).

Data vectors: ξ±, Cℓ, E and B modes

The pure-E residuals look jagged because the pure-mode covariance diagonal alternates from bin to bin. The shift itself is smooth.

Design

• A blind is a named secret file, <paths.blinds>/<name>.blind.json. It holds a seed, the envelope the hidden point is drawn in (S8 ± 0.075, Ωm ± 0.1) and the fiducial point. It is drawn once (python -m sp_validation.blinding init <name>) and never committed: *.blind.json is gitignored.
• Each catalogue declares its blind in cosmo_val/cat_config.yaml, as blind: none or blind: <name>. An entry without one doesn't run, so nobody measures a new catalogue before someone has decided. _leak_corr and _seed<N> versions take their entry's blind, and entries reading the same shear file must declare the same one.
• A theory is a function theory(params, s) that returns the prediction for every row of SACC s, as an array the length of s.mean, zero where cosmology has no effect. Smokescreen draws the hidden point from the seed, calls the theory at both points, and the difference is added to the data.

Calculate, then save. A measurement function computes its data vector and saves it in the same function, with sacc_io.save(s, path, blind=cv.blind(version)). It returns the saved part, which is shifted if the catalogue is blinded, so raw signal never leaves the function. calculate_2pcf follows this pattern.

Statistics computed from parts are saved with derived_from=parts and carry the parts' stamp (s.metadata["blind"]). Assembly refuses a part whose stamp isn't the catalogue's declared blind.

Theories.

• The default, blinding.shear, computes ξ± and Cℓ_EE with pyccl through cs_util.cosmo.get_cosmo, on the same CAMB HMCode2020-feedback route as inference. It returns zero for Cℓ_BB and Cℓ_EB.
• blinding.no_signal returns zeros, for ρ/τ.
• Any other statistic passes its own:

def my_theory(params, s):
    out = np.zeros(len(s.mean))
    ee = s.indices(sacc_io.CL_EE)
    window = s.get_bandpower_windows(ee)
    out[ee] = window.weight.T @ my_cl_ee(params, window.values)  # CCL, an emulator, ...
    return out                                                    # BB, EB: zero

sacc_io.save(s, path, blind=cv.blind(version), theory=my_theory)

• params is a plain dict: S8, Omega_m, Omega_b, h, n_s, m_nu, w0, wa, logT_AGN and A_IA.
• s is the SACC being saved. Its rows carry a data type, a tracer pair, a θ or ℓ tag and, for Cℓ, a window; its tracers carry the n(z).
• The default theory raises on a data type it doesn't model, so such a type is never saved unshifted.

Unblinding means flipping the declaration to blind: none in a reviewed PR and rerunning. Producer rules carry the blind's name in params.blind, so Snakemake reruns exactly what the flip touches.

Other changes for users

• sacc_io.save takes blind= for a new measurement (shifted if blinded) or derived_from= for a derived statistic. sacc_io.load reads any SACC.
• calculate_2pcf returns the saved ξ± part, which sacc_io.xi_correlation reads like a GGCorrelation. ξ± is no longer written to text files.
• ⟨M_ap²⟩ is computed from the ξ± part, matching TreeCorr's calculateMapSq to 1e-12.
• Pure E/B reads the ξ± integration part and carries its stamp; the modes and covariance are develop's operator (Pure E/B: fixed-quadrature operator with exact covariance #373) applied to it.
• scripts/xip_xim.py and scripts/map2.py are deleted. They computed ξ± and ⟨M_ap²⟩ outside cosmo_val, so on a blinded catalogue they would write true values. Nothing in the repo calls them, and cosmo_val computes both.

Open questions

1. Where do blinds live? On a candide path (the default, /n17data/UNIONS/WL/blinds, isn't created yet) or in a private repo such as UNIONS-WL/blinds?
2. Who may draw and hold a blind?
3. Has v1.6.x been seen unblinded? For now every entry is blind: none. If v1.6.x has been seen, blinding it protects little, and the first real blind belongs on the tomographic catalogue.
4. Should cosmo_val refuse to overlay unblinded v1.4.6.3 on a blinded tomographic catalogue? That overlay shows the shift, which is a few σ per point. Otherwise we don't police plots.
5. Envelope and fiducial: keep S8 ± 0.075 and Ωm ± 0.1 around Planck18, or centre on the inference prior?
6. Nuisance parameters: Smokescreen shifts only S8 and Ωm. Is that enough once γt and galaxy bias enter? We expect the effect to be second order.
7. Reveal criteria (PRD: SACC data format + Smokescreen blinding #241 §7).

A blind file doesn't record how Smokescreen draws the hidden point. If the fork changed its draw, an existing blind would give a different shift. The lock pins the fork.

Testing

• Container suite including the slow tests: 317 passed, 1 xfailed (GLASS maps, unrelated). Host workflow tests: 15 passed, 1 skipped (the SLURM submission smoke test).
• Synthetic SACCs:
  • ξ± and Cℓ_EE move by the theory difference; BB, EB and ρ/τ don't move.
  • A custom theory blinds a new data type, and the default theory refuses an unknown one.
  • Assembly refuses mixed stamps, and a saved part can't be shifted twice.
  • Flipping a blind reruns exactly the ξ± parts, and in the apptainer toy run the ξ± parts carry the blind's stamp.
• B-modes on synthetic data vectors with SP_v1.4.6.3 shape noise, at each corner of the envelope: COSEBIs B_n move by at most 0.034σ.
• End to end on SP_v1.4.6.3 (the figures above):
  • On every ξ± and Cℓ_EE part, blinded − true equals the shift to 1e-12. BB and EB are unchanged.
  • No true ξ± value, seed or hidden parameter appears in any byte of the blinded tree or its logs.
  • Rerun on this head under the same blind, the blinded tree reproduces the earlier one to 1e-12 in every ξ±, Cℓ and COSEBI part.
• The pure-mode B bin at 1.5′ moves 1.15σ. cosmo_numba's pure-E/B integral doesn't converge on noisy ξ±, so the transform isn't additive in that bin, blinded or not. The bin is already an outlier in the unblinded data, and the fix belongs to the B-mode estimator work.

Closes #252. Part of #280: the default theory's settings were matched to cosmosis_pipeline_A_*.ini by reading, and the numerical comparison with CosmoSIS isn't done yet.

— Claude (Opus) on behalf of Cail.

🤖 Generated with Claude Code

[cailmdaley]

@LisaGoh and @sachaguer this is ready for feedback now. some open questions to settle and some oversights i'm sure, but i worked hard to get this as simple as possible. i think it's pretty nice. make sure to click the dropdown in the PR description to appreciate the full shifted data vector plots!

[cailmdaley]

@AstroMike also happy to hear your opinion about this and whether it's flexible enough for your delta sigma example. i think the only work required to blind a new data vector is to a) get your data vector into sacc format and b) pass a function that calculates a theory curve when you save the data vector to disk. the data vector itself needn't be calculated by sp_validation as long as it's in SACC format.

[LisaGoh]

Thank you for this Cail! I think I'd need to try running the whole thing to fully understand the logic of the pipeline, but for now I think it's good to go! With this we should also start thinking about blinding protocol (linked to the open questions you posted): when it happens, by whom, and on which catalogues.