Skip to content

Two-page white paper for high-throughput screening deployment - #596

Open
talonchandler wants to merge 8 commits into
mainfrom
wo-hcs-whitepaper
Open

talonchandler wants to merge 8 commits into
mainfrom
wo-hcs-whitepaper

Conversation

@talonchandler

@talonchandler talonchandler commented Nov 14, 2026 •

Copy link
Copy Markdown
Collaborator
  • Adds a 2-page label-free screening tutorial on OPS atlas data, viewable as markdown or PDF.
  • Adds z_focus_offset: auto, an autofocus algorith matching the OPS production approach (MBSF from 3D phase recon). The resolved offset is used for the run and printed; no new config is written for it.
  • The tutorial folder is self-contained: figures/make_figures.py and figures/koehler/make_figure.py regenerate every figure from files in the repo (no HPC access needed) with the packages pinned in figures/requirements.txt, make_pdf.sh builds the PDF from whitepaper.tex, and the configs shown in the tutorial are included. See the folder's README.
  • Adds Figure 2 (realign the condenser) and shows the Micro-Manager z-stack settings in Figure 1.

Bare 'auto' resolves the in-focus slice with a discrete per-slice search
over 3D reconstructions of several structured crops (median across crops),
matching the OPS production autofocus. '{init: auto, lr: ...}' seeds the
gradient optimizer with the searched value for sub-slice co-optimization
with tilt angles. The resolved value is frozen into a sibling
*_autofocus.yml config for auditability.

Also fix the optimization path to differentiate the same SVD-based
reconstruction it returns: the norm-based surrogate (previously forced
whenever gradients were enabled) drops absorption/phase coupling and its
gradients disagree with the true loss landscape, stalling tilt and focus
fits. The surrogate now remains only as a CUDA fallback where complex SVD
backward is unreliable.
Two-page case study on OPS atlas data: acquisition settings (Koehler,
condenser NA, z spacing), a one-config reconstruction with
z_focus_offset: auto, and four boxed failure modes with their fixes
(flipped z axis, incorrect regularization, autofocus drift, well-edge
illumination tilt).

The folder is self-contained: the docs-site page (labelfree-screening.md,
wired into the guide toctree), the 2-page whitepaper (whitepaper.tex +
whitepaper.pdf, built by make_pdf.sh with base texlive), vector and raster
figures with their generator (figures/make_figures.py, runs on the Biohub
HPC against the atlas data), and the configs the tutorial teaches.
@talonchandler talonchandler changed the title Label-free high-throughput screening tutorial and z_focus_offset: auto Two-page white paper for high-throughput screening deployment Nov 14, 2026
@talonchandler
talonchandler marked this pull request as ready for review November 14, 2026 00:05
last_good was saved after optimizer.step(), so it held unvalidated
parameter values. A degenerate SVD gradient can step parameters to
non-finite values; those were checkpointed, the next iteration's
reconstruction raised, the loop rolled back to the same bad values, and
the final reconstruction crashed with linalg.svd non-finite errors.
Checkpoint the values that produced the validated loss instead, and
fail fast when a step produces non-finite parameters.
torch.linalg.svd backward produces non-finite gradients where singular
values are degenerate, which is ubiquitous beyond the band limit; one bad
step then poisons the optimization (seen as CI failures in
test_optimization_cli). Replace the gradient-mode path with a closed-form
eigendecomposition of the 2x2 Gram matrix: the same decomposition as the
SVD up to gauge (reconstructions agree; regression-tested), with
relative-epsilon smoothing so gradients stay finite at degeneracies, and
working on all devices including CUDA.
…n-repo

- Add Figure 2 (realign the condenser) from a parametric cartoon microscope
- Show the Micro-Manager z-stack settings and the centered field stop in Figure 1
- Draw all data figures from crops in figures/data/ instead of HPC paths
- Rebalance the whitepaper layout; add n = 1 to the acquisition table
- Rename failure mode 3 to "Focus drift"
- Ask for a z stack centered on focus, else set z_focus_offset
- Lock figure dependencies and make the PDF build deterministic
- Draw failure-mode figures at print size in two equal columns
- Show real config keys in Figure 1; number figures and table in the docs page
- Advise separate config and output per parallel job
- Remove development leftovers from the Koehler figure scripts
- Shrink whitepaper.pdf under the 1 MB file limit
- Keep the auto-resolved offset in memory for the run; no *_autofocus.yml
- Name the transfer function after the config the user passed
- Add a CLI test; update the tutorial text
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant