Refactor cli, use zarr instead of N5 #73

mzouink · 2026-01-21T14:03:05Z

Cli :

Smart detector the the existing models and configuartion
Stream virtual zarr instead of N5
Fix not standard data bugs
Support anistropic data

…arr array.

add single-scale ome-ngff (0.4) metadata generation when creating a …

🐛 write unit into a .zattrs metadata as a string, not as a list

mzouink · 2026-01-21T14:06:25Z

@krokicki

Copilot

Pull request overview

This PR refactors the CLI architecture and migrates from N5 to Zarr format. The changes introduce dynamic CLI generation based on ModelConfig subclasses and consolidate model configuration classes into a dedicated module.

Changes:

Migrated from N5 to Zarr format across server endpoints and neuroglancer integrations
Refactored CLI to use dynamic command generation based on ModelConfig subclasses
Moved model configuration classes from utils/data.py to models/models_config.py
Introduced new utility modules for CLI generation, configuration serialization, and multi-cloud job extensions
Updated documentation with comprehensive CLI reference guide

Reviewed changes

Copilot reviewed 41 out of 41 changed files in this pull request and generated 18 comments.

Show a summary per file

File	Description
cellmap_flow/server.py	Changed from N5 to Zarr format for API endpoints and metadata
cellmap_flow/cli/cli.py	Refactored to use dynamic command generation from ModelConfig classes
cellmap_flow/cli/server_cli.py	Updated to dynamically create server commands
cellmap_flow/cli/yaml_cli.py	New YAML-based CLI with dynamic model discovery
cellmap_flow/models/models_config.py	New module consolidating all ModelConfig classes
cellmap_flow/utils/cli_utils.py	New utilities for dynamic CLI generation
cellmap_flow/utils/bsub_utils.py	Refactored with Job abstraction and cloud extensions
cellmap_flow/utils/ds.py	Added axes_names tracking and removed swap_axes logic
docs/source/cli_reference.rst	New comprehensive CLI documentation
neuroglancer_utils.py	Root-level file with Zarr migration

Comments suppressed due to low confidence (2)

cellmap_flow/utils/ds.py:766

get_ds_info returns tuple of size 5 and tuple of size 6.
get_ds_info returns tuple of size 5 and tuple of size 6.
get_ds_info returns tuple of size 5 and tuple of size 6.
get_ds_info returns tuple of size 5 and tuple of size 6.
get_ds_info returns tuple of size 2 and tuple of size 5.
get_ds_info returns tuple of size 2 and tuple of size 6.
get_ds_info returns tuple of size 2 and tuple of size 6.
get_ds_info returns tuple of size 2 and tuple of size 6.
get_ds_info returns tuple of size 2 and tuple of size 6.

def get_ds_info(path: str, mode: str = "r"):

example/server_check.py:8

Import of 'ModelConfig' is not used.
Import of 'BioModelConfig' is not used.
Import of 'DaCapoModelConfig' is not used.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-01-21T14:07:24Z

cellmap_flow/utils/serialize_config.py

+            # if isinstance(v, np.ndarray):
+            #     elms.append(f"{k}: type={type(v)} shape={v.shape}\n")
+            # elif inspect.ismodule(v):
+            #     elms.append(f"{k}: <module '{v.__name__}'>\n")
+            # elif k=="checkpoint" or k=="model":
+            #     elms.append(f"{k}\n")
+            # else:


This comment appears to contain commented-out code.

Copilot · 2026-01-21T14:07:25Z

cellmap_flow/cli/cli.py

+
+    # Get type hints if available
+    try:
+        type_hints = get_type_hints(config_class.__init__)


Variable type_hints is not used.

Copilot · 2026-01-21T14:07:25Z

cellmap_flow/cli/cli.py

+    # Get type hints if available
+    try:
+        type_hints = get_type_hints(config_class.__init__)
+    except:
+        type_hints = {}
+


Variable type_hints is not used.

Suggested change

# Get type hints if available

try:

type_hints = get_type_hints(config_class.__init__)

except:

type_hints = {}

Copilot · 2026-01-21T14:07:25Z

neuroglancer_utils.py

+        for job in g.jobs:
+            model = job.model_name
+            host = job.host
+            color = next(color_cycle)


Variable color is not used.

Copilot · 2026-01-21T14:07:26Z

cellmap_flow/cli/server_cli.py

+
+    # Get type hints if available
+    try:
+        type_hints = get_type_hints(config_class.__init__)


Variable type_hints is not used.

Copilot · 2026-01-21T14:07:28Z

cellmap_flow/cli/server_cli.py

+from cellmap_flow.utils.cli_utils import (
+    get_all_subclasses,
+    create_click_option_from_param,
+    process_constructor_args,
+    get_all_model_configs,
+    print_available_models,
+)


Import of 'get_all_subclasses' is not used.

Copilot · 2026-01-21T14:07:29Z

cellmap_flow/cli/cli.py

+    # Get type hints if available
+    try:
+        type_hints = get_type_hints(config_class.__init__)
+    except:


Except block directly handles BaseException.

Suggested change

except:

except (TypeError, NameError, AttributeError):

Copilot · 2026-01-21T14:07:29Z

cellmap_flow/utils/cli_utils.py

+
+    try:
+        type_hints = get_type_hints(config_class.__init__)
+    except:


Except block directly handles BaseException.

Suggested change

except:

except Exception:

Copilot · 2026-01-21T14:07:29Z

cellmap_flow/cli/server_cli.py

+    # Get type hints if available
+    try:
+        type_hints = get_type_hints(config_class.__init__)
+    except:


Except block directly handles BaseException.

Suggested change

except:

except Exception:

Copilot · 2026-01-21T14:07:30Z

cellmap_flow/image_data_interface.py

+            self.filetype,
+        ) = get_ds_info(dataset_path)


Left hand side of assignment contains 6 variables, but right hand side is a tuple of length 5.

Suggested change

self.filetype,

) = get_ds_info(dataset_path)

*maybe_filetype,

) = get_ds_info(dataset_path)

self.filetype = maybe_filetype[0] if maybe_filetype else None

yuriyzubov and others added 19 commits August 28, 2025 10:58

add single-scale ome-ngff (0.4) metadata generation when creating a z…

6b29f16

…arr array.

track progress

4e32157

support diff size fly

8f7b963

Merge branch 'optimize_blockwise_fly' into metadata

3ebac87

Merge pull request #69 from janelia-cellmap/metadata

2e1e420

add single-scale ome-ngff (0.4) metadata generation when creating a …

fix processor

da16bef

optimize gui

4a78771

🐛 write unit into a .zattrs metadata as a string, not as a list

d40bb06

Merge pull request #70 from janelia-cellmap/metadata

d960974

🐛 write unit into a .zattrs metadata as a string, not as a list

support 4d output

b58c171

easier cli

b1215e3

fix zarr n5 reading

705e1f6

support 4d

8e43e77

Merge remote-tracking branch 'origin/main' into optimize_blockwise_fly

9235a85

Standarize cli

29878b5

dashboard ordering

ed5b976

support multi model fix bug

35eb0cc

clean cli

fa766c2

use zarr and start cli

b9afa83

Copilot AI review requested due to automatic review settings January 21, 2026 14:03

Copilot started reviewing on behalf of mzouink January 21, 2026 14:03 View session

Merge branch 'main' into refactor_cli

f2260b4

Copilot AI reviewed Jan 21, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Refactor cli, use zarr instead of N5 #73

Refactor cli, use zarr instead of N5 #73

Uh oh!

mzouink commented Jan 21, 2026 •

edited

Loading

Uh oh!

mzouink commented Jan 21, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Jan 21, 2026

Uh oh!

Copilot AI Jan 21, 2026

Uh oh!

Copilot AI Jan 21, 2026

Uh oh!

Copilot AI Jan 21, 2026

Uh oh!

Copilot AI Jan 21, 2026

Uh oh!

Copilot AI Jan 21, 2026

Uh oh!

Copilot AI Jan 21, 2026

Uh oh!

Copilot AI Jan 21, 2026

Uh oh!

Copilot AI Jan 21, 2026

Uh oh!

Copilot AI Jan 21, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Refactor cli, use zarr instead of N5 #73

Are you sure you want to change the base?

Refactor cli, use zarr instead of N5 #73

Uh oh!

Conversation

mzouink commented Jan 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mzouink commented Jan 21, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Jan 21, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 21, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 21, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 21, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 21, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 21, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 21, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 21, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 21, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 21, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

mzouink commented Jan 21, 2026 •

edited

Loading