Repository navigation
Conversation
The package is rebuilt on torch modules under src/waveorder in the commits that follow. This commit removes the old top-level waveorder/ package (the functional models, the api and cli layers, the calibration and optim modules, the napari plugin and GUI) and the tests written against it. BREAKING CHANGE: the 3.x Python API and the napari plugin are gone. The new API is waveorder.models; README.md lists what each 3.x name becomes. Signed-off-by: Sricharan Reddy Varra <sricharan.varra@biohub.org>
Collaborator
|
Exciting stuff! I just took a very brief first pass through the filetrees:
|
Contributor
Author
Yup will do
👍 makes sense to me, I'll put it on it's own branch and in waveorder-design.
nope, will remove them.
They'll be added back in a future commit. |
The package moves under src/ and requires Python 3.12 to 3.14 and torch 2.15. tensordict stores transfer functions, and the napari plugin keeps its visual extra. Signed-off-by: Sricharan Reddy Varra <sricharan.varra@biohub.org>
waveorder 3.x computed transfer functions and inverses in free functions, one set per contrast. These are the shared parts every imaging system and solver is built from: Fourier-diagonal operators, pupils and apertures, named physical parameters with bounds, sampling, and the errors and registry they report through. Signed-off-by: Sricharan Reddy Varra <sricharan.varra@biohub.org>
Each contrast is one module that turns its optical parameters into a transfer function on any sampling. Transfer functions are built in slabs of planes and cropped to the data grid as they go, so a build needs about 4 times a tile's memory instead of 40 and stays under cuFFT's 2^31 element limit. The stored transfer is the Hermitian part of the crop, which makes the forward model the same real operator as wo-4.0's on every grid. Signed-off-by: Sricharan Reddy Varra <sricharan.varra@biohub.org>
A prior, a noise model and a solver are separate parts, so a new one works with every contrast. Covers the closed-form Wiener solve, FISTA, ADMM and Richardson-Lucy with gradient consensus, which stops once every voxel is frozen as wo-4.0 does. Signed-off-by: Sricharan Reddy Varra <sricharan.varra@biohub.org>
Calibration is a search over free optical parameters scored on the data: grid and gradient stages, the evidence and contrast scores, automatic regularization weight, and bead calibration. Signed-off-by: Sricharan Reddy Varra <sricharan.varra@biohub.org>
waveorder.testing checks that a new system, solver, prior or noise model obeys the rules the solvers and models rely on (adjoint, units, gradients, device and dtype), so parts written outside the package can be verified the same way. Signed-off-by: Sricharan Reddy Varra <sricharan.varra@biohub.org>
All OME-Zarr access goes through iohub. A single-position store is accepted as input and gives a single-position output. Transfer functions are stored in tf.zarr through tensordict so a fixed configuration computes them once. Signed-off-by: Sricharan Reddy Varra <sricharan.varra@biohub.org>
Volumes too large for one solve are reconstructed tile by tile with cross-faded overlaps, reading ahead and writing behind the GPU. Signed-off-by: Sricharan Reddy Varra <sricharan.varra@biohub.org>
Every version 4 file that wo-4.0 accepts loads here with wo-4.0's meaning, or with a warning at the key where a value has no effect. wo config migrate turns a 3.x file into a version 4 file, and solver.padding.z accepts auto, which picks the padding from the optics. Signed-off-by: Sricharan Reddy Varra <sricharan.varra@biohub.org>
…ate, simulate waveorder.models is the user API: one model per contrast with the same methods, built from a config or in Python. Reconstruction runs over positions and time points on several GPUs, in tiles when a volume is large, and a new output keeps the OME-Zarr version of its input. BREAKING CHANGE: the 3.x Python API is removed. waveorder.focus.focus_from_transverse_band stays and warns. Signed-off-by: Sricharan Reddy Varra <sricharan.varra@biohub.org>
The reconstruction tab builds its forms from the version 4 schema and runs wo reconstruct jobs, for phase and fluorescence. The birefringence checkbox and the LC calibration tab are shown disabled, because birefringence is not in this version yet. Signed-off-by: Sricharan Reddy Varra <sricharan.varra@biohub.org>
wo rec, compute-tf, apply-inv-tf, calibrate-tf, sim, config, interactive and bm call waveorder.models, with the flags of wo-4.0. Signed-off-by: Sricharan Reddy Varra <sricharan.varra@biohub.org>
The 19 regression cases run through the new models and compare against wo-4.0. The job no longer pins a CPU vendor. Signed-off-by: Sricharan Reddy Varra <sricharan.varra@biohub.org>
The README keeps its text. Its example runs on the new models, and it says which methods are not in this version yet. Signed-off-by: Sricharan Reddy Varra <sricharan.varra@biohub.org>
srivarra
force-pushed
the
wo-4.0-nn-modules
branch
from
November 16, 2026 19:22
8803a67 to
c614878
Compare
A weight chosen on a geometric range was reported as ending on a bound, because the test was linear. Per-tile calibration never started the noise level from the data, so sigma stayed capped at 1 on every tile. The random stage now draws wo-4.0's points for a seed, a constant-fit focus search no longer crashes, a degenerate focus peak warns, and ADMM factors its system once. Signed-off-by: Sricharan Reddy Varra <sricharan.varra@biohub.org>
A migrated gradient calibration stepped by 0.01 times each quantity's span where wo-4.0 steps by 0.01. A file that writes solver.reconstruction: closed_form was refused for bead calibration and simulation, although wo-4.0 writes the key in every file. A Poisson noise block with a Tikhonov prior was refused where wo-4.0 ignores it. wo config migrate now accepts the wo-4.0 files the loader accepts, and wo config show writes a file that loads. Signed-off-by: Sricharan Reddy Varra <sricharan.varra@biohub.org>
… keys in errors A config with object.prior.denoiser reconstructed with no prior. It now loads and is refused when a reconstruction is asked for, as in wo-4.0. Refusals raised while building from a config name config keys, not Python arguments. Signed-off-by: Sricharan Reddy Varra <sricharan.varra@biohub.org>
… warnings wo-4.0 has the flag; every case is migrated here, so it is accepted and does nothing. With --device the pixel-size warning lost the name of the config file. Signed-off-by: Sricharan Reddy Varra <sricharan.varra@biohub.org>
The log box sat inside the disabled calibration tab. Validate checked the schema only, so a model the build refuses got a check mark. Signed-off-by: Sricharan Reddy Varra <sricharan.varra@biohub.org>
The package imports them, and they arrived only through iohub and typer. Signed-off-by: Sricharan Reddy Varra <sricharan.varra@biohub.org>
A new output takes the OME-Zarr version of its input, and the tile merger needs shards, which v0.4 lacks, so wo rec --tile on a v0.4 input was refused. A new tiled output is now v0.5. Adding tiles to an existing v0.4 output is still refused. Signed-off-by: Sricharan Reddy Varra <sricharan.varra@biohub.org>
wo-4.0 opens stores, transfer functions and benchmark runs in napari with these two commands, and its tutorial relies on wo view. They read this branch's stores. A 2D transfer-function store now holds its per-plane transfer functions, as a 3D store holds its transfer, so wo view has something to show for a 2D reconstruction. Signed-off-by: Sricharan Reddy Varra <sricharan.varra@biohub.org>
The scripts under .skills are byte-identical copies of what the Biohub MCP serves, and .skills/ruff.toml extends this pyproject. The rules this branch adds for the package (no print, no numpy, annotations, docstrings) therefore reported 42 errors in them, and `ruff check` failed at the repository root, which is how the pull-request workflow runs it. Signed-off-by: Sricharan Reddy Varra <sricharan.varra@biohub.org>
Each tile was read with a halo beyond its overlap, sized to hold 99% of the solver's filter, and the halo was discarded after the solve. wo-4.0's tile-stitch does not do that: a tile is its size and its overlap, cross-faded with its neighbours. For 3D phase the filter is as wide as a tile, so the tiled regression case read 672 x 672 pixels per tile where wo-4.0 reads 384 x 384, and took twice wo-4.0's time (48.8 s against 24.2 s on one node). TilePlan loses `halo`, and `tiling.halo_for` and `Solver.tile_halo` are removed, so no solver can ask for one. Field tiles are planned the same way. The tiled regression case now reads 384 x 384 per tile and takes 21.4 s against wo-4.0's 23.1 s on one node, within the suite's bounds. Stitching accuracy depends on the overlap alone; the tests measure it against the whole-volume reconstruction. Signed-off-by: Sricharan Reddy Varra <sricharan.varra@biohub.org>
wo-4.0 was updated from main (#599) and the pull request no longer merged cleanly. This takes LICENSE.md, the code of conduct, the security policy, the pull-request template and the README's status, community and security notes. Conflicts: pyproject.toml keeps this branch's dependencies and points the license at LICENSE.md; CONTRIBUTING.md gains the two policy paragraphs above its setup section; uv.lock is unchanged by a new lock. wo-4.0's fix that apply-inv-tf runs on the configuration's device (#595) was made in a file this branch replaced. The rewritten command already does that, and a test ported from wo-4.0 now checks it for phase and fluorescence. Signed-off-by: Sricharan Reddy Varra <sricharan.varra@biohub.org>
The suite carried more than it needed. This removes: - tests/config/data: 15 YAML example configs. Tests with a subject of their own build a small config inline; tests that only checked that an example file migrates or loads are gone. - The copies of 3.x code under tests/nn/legacy_*.py and tests/tiling/legacy_zernikes.py, with the tests that compared the modules against them. Parity with 3.x is checked by the regression suite on real data. Tests that only took an input from those files build it inline. - Benchmark-harness tests that pinned numbers wo-4.0 computed or tested internal helpers; the reference check, the wo bm commands and the runner keep their tests, now in three files. - Tests of source text, signatures and class flags, duplicates, and helpers that no test called. Signed-off-by: Sricharan Reddy Varra <sricharan.varra@biohub.org>
The widefield and confocal transfer functions clamp the point-spread function at zero after cropping the OTF. The clamped lobes alone set the transfer function at zero lateral frequency off k_z = 0, and the float32 inverse transform before the clamp left that column 1e-4 to 2e-4 off. A Tikhonov inverse passes it into every plane's mean, which showed as the minimum of a549_fluor_3d differing from wo-4.0 by 9e-4. The inverse transform and the clamp now run in float64 (float32 on MPS). The minimum agrees with wo-4.0 to 1e-5 on the CPU and 5e-5 on CUDA, and the volume to 5e-7 where it was 5e-6. The float32 spectrum is released first, so the peak memory of the build is unchanged. Signed-off-by: Sricharan Reddy Varra <sricharan.varra@biohub.org>
wo-4.0's wo rec ran a configuration with both a rule-based weight
(object.weight: {auto: ...}) and a sharpness calibration: it calibrated on
the whole frame at weight 1e-3, then swept the weight on the most structured
crop with the values it had found. This branch refused the combination.
The stages before a weight sweep now run first, on the whole frame, with the
swept weight held at its start; nn.calibrate gains hold= for that. Under an
evidence or gcv calibration the combination is still refused, as wo-4.0's
calibrate-tf refused it, and the message says to write object.weight: auto.
Signed-off-by: Sricharan Reddy Varra <sricharan.varra@biohub.org>
Several settings of a wo-4.0 file were read differently without a warning, and four warnings gave a wrong reason. Now, for a file wo-4.0's wo rec would have run (sharpness, one gradient stage, free values with tolerance: free): - advanced.use_gradients: false keeps the free values at nominal, as wo-4.0 took no step. - advanced.optimizer: grid_search searches wo-4.0's grid, grid_points values one learning rate apart around the nominal, not the bounds. - advanced.optimizer: lbfgs warns that wo-4.0 stepped without a line search. - Without advanced.optimizer wo-4.0 ran 10 adam steps whatever max_iter said: a direct load warns, and wo config migrate writes max_iter: 10. - A free aperture written without bounds gets [0.02, n] from a 3.x or wo-4.0 file, so a gradient step stays in aperture units as it was there. Under an evidence or gcv calibration advanced.learning_rates is not applied, as wo-4.0's calibrate-tf did not read it. The warnings for convergence_tol, convergence_patience, use_gradients and log_dir say what wo-4.0 did with them. The file-level warning names only the defaults that differ in the loaded file, now including the Richardson-Lucy start. Signed-off-by: Sricharan Reddy Varra <sricharan.varra@biohub.org>
The process-wide observer in waveorder.observe had three listeners: the command line's progress display, a W&B adapter behind --wandb that nothing in this repository or waveorder-design used, and the relay of worker events. It made results depend on ambient state (SolveResult.history was filled only inside an observe block), used a private torch call to stay silent inside transforms, and needed a class swap in __init__ to keep waveorder.observe a function. - Iterative solvers take iterations, tolerance and check_every directly; the Loop class and one nesting level of a saved solver are gone. - A solver call takes callback(iteration, estimate), calibrate takes callback(index, stage, best), and reconstruct_positions takes on_start(total) and on_unit(result). - The command line draws its progress from on_start and on_unit and logs calibration stages from CalibrationResult.trace. - waveorder.observe, waveorder.integrations, --wandb and the wandb extra are removed; callers that track runs log from the callbacks and results. The regression suite gives identical metrics before and after on one node. Signed-off-by: Sricharan Reddy Varra <sricharan.varra@biohub.org>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
waveorder becomes a set of torch modules, with one model per contrast. A model calibrates its optics, reconstructs, and simulates. Most
wocommands and wo-4.0 configs work as before.What changes for users
wo recreads single-position stores, reconstructs large volumes in tiles with--tile, spreads time points over GPUs with--device cuda:0,cuda:1, and reuses a stored transfer function with-t tf.zarr.Breaking changes
waveorder.focus.focus_from_transverse_bandremains and warns.wo config migrate, which replaceswo migrate-config.wo tile-stitchis nowwo rec --tile.wo compute-tfandwo config migrateneed--overwriteto replace an existing output.wo simon a config without asimulateblock. The plugin shows its birefringence and LC calibration tabs disabled.Status
I tested the reconstructions on neuromast and zebrafish data. I'm working on a cleanup pass.