Add Facebook's Regnet Models

README.md

@@ -121,6 +121,7 @@ The Markdown sources live in [`docs/`](docs/) if you would rather read them in t
     | PoolFormer | [MetaFormer is Actually What You Need for Vision](https://arxiv.org/abs/2111.11418) | `timm` |
     | PVT | [Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions](https://arxiv.org/abs/2102.12122) | `transformers` |
     | PVTv2 | [PVTv2: Improved Baselines with Pyramid Vision Transformer](https://arxiv.org/abs/2106.13797) | `transformers` |
+    | RegNet | [Designing Network Design Spaces](https://arxiv.org/abs/2003.13678) | `transformers` |
     | Res2Net | [Res2Net: A New Multi-scale Backbone Architecture](https://arxiv.org/abs/1904.01169) | `timm` |
     | ResMLP | [ResMLP: Feedforward networks for image classification with data-efficient training](https://arxiv.org/abs/2105.03404) | `timm` |
     | ResNet | [Deep Residual Learning for Image Recognition](https://arxiv.org/abs/1512.03385) | `timm` |

docs/regnet.md

@@ -0,0 +1,165 @@
+# RegNet
+
+<div class="kf-note kf-note--weights">
+<b>Weights:</b> pretrained Keras weights live on Hugging Face under
+<a href="https://huggingface.co/zeromodels">zeromodels/regnet-&lt;variant&gt;</a>
+(12 <b>X</b> + 12 <b>Y</b> variants; each repo carries <code>zm_config.json</code> +
+<code>model.weights.h5</code>). Load with
+<code>from_weights("zeromodels/regnet-y-040")</code>.
+</div>
+
+RegNet (Designing Network Design Spaces) is a family of ConvNets whose per-stage widths and
+depths follow a simple **quantized-linear rule** found by searching design spaces rather than
+tuning individual architectures. It is a 3x3 stride-2 stem followed by **four stages** of
+residual blocks; each block is a `1x1 -> 3x3 grouped -> 1x1` bottleneck, and the **RegNet-Y**
+variant inserts a **Squeeze-and-Excitation** module. The 3x3 convolution is split into
+`out_channels // groups_width` groups. Width grows and resolution halves each stage, so a
+single backbone yields a standard CNN feature pyramid usable for classification and dense
+prediction.
+
+**Paper**: [Designing Network Design Spaces](https://arxiv.org/abs/2003.13678)
+
+RegNet comes in two families: **X** (plain bottleneck) and **Y** (+ Squeeze-and-Excitation,
+the stronger and more common one).
+
+## API
+
+### RegNetImageClassify
+
+```python
+RegNetImageClassify(
+    embedding_size=32,
+    hidden_sizes=(128, 192, 512, 1088),
+    depths=(2, 6, 12, 2),
+    groups_width=64,
+    layer_type="y",
+    downsample_in_first_stage=True,
+    image_size=224,
+    include_normalization=True,
+    normalization_mode="imagenet",
+    num_classes=1000,
+    classifier_activation="linear",
+    name="RegNetImageClassify",
+)
+```
+
+The classifier: the backbone plus a GlobalAveragePooling2D + dense head.
+`include_normalization=True` means the model takes **raw `[0, 255]` pixels** and applies
+ImageNet mean/std internally, so there is no separate image processor to construct.
+
+**Parameters**
+
+- **embedding_size** (`int`): output width of the 3x3 stride-2 stem.
+- **hidden_sizes** / **depths** (`tuple`): per-stage output width and block count.
+- **groups_width** (`int`): channels per group of the 3x3 grouped convolution (a block's group count is `out_channels // groups_width`).
+- **layer_type** (`str`): `"y"` (adds Squeeze-and-Excitation) or `"x"`.
+- **downsample_in_first_stage** (`bool`): whether the first stage downsamples. `True` for the standard checkpoints (RegNet has no pooling stem).
+- **image_size** (`int`, *optional*, defaults to `224`): resolution the model is built for.
+- **include_normalization** (`bool`, *optional*, defaults to `True`): bake ImageNet normalization into the graph.
+- **num_classes** (`int`, *optional*, defaults to `1000`): classifier outputs.
+
+`from_weights` fills the architectural fields from the variant's config, so you normally pass
+only the repo id.
+
+**Call** `model(pixel_values, training=False)`. **Returns** class logits of shape
+`(B, num_classes)`.
+
+### RegNetModel
+
+The backbone alone. With `as_backbone=True` it returns the four stage feature maps
+(the pyramid) instead of just the last one, for detection or segmentation necks.
+
+```python
+RegNetModel(as_backbone=False, layer_type="y", groups_width=64, ..., include_normalization=True)
+```
+
+### RegNetConfig
+
+Typed config (`model_type="regnet"`) holding the fields above; serialized into each Hub repo's
+`zm_config.json`.
+
+## Model Variants
+
+For `RegNetImageClassify.from_weights("zeromodels/regnet-<variant>")`. The number is the
+model's compute in units of 0.1 GFLOPs (`002` = 0.2 GF ... `320` = 32 GF).
+
+| Family                 | Variants                                                                 |
+|------------------------|--------------------------------------------------------------------------|
+| **RegNet-X** (plain)   | `regnet-x-{002,004,006,008,016,032,040,064,080,120,160,320}`             |
+| **RegNet-Y** (+ SE)    | `regnet-y-{002,004,006,008,016,032,040,064,080,120,160,320}`             |
+
+At matched compute the Y family (with Squeeze-and-Excitation) is generally stronger; e.g.
+`regnet-y-320` reaches ~80.9% ImageNet-1k top-1. All are 224x224, 1000 classes. The larger
+self-supervised `facebook/regnet-y-{320,640,1280,10b}-seer` checkpoints are not mirrored here
+but load on the fly with the `hf:` prefix (see [below](#loading-fine-tuned-and-community-weights)).
+
+## Basic Usage
+
+```python
+import keras
+import numpy as np
+from PIL import Image
+from zeromodels.models.regnet import RegNetImageClassify
+
+model = RegNetImageClassify.from_weights("zeromodels/regnet-y-040")
+
+image = Image.open("assets/data/coco_bear.jpg").convert("RGB").resize((224, 224))
+pixels = np.asarray(image, "float32")[None]  # (1, 224, 224, 3), raw [0, 255]
+
+logits = model(pixels, training=False)
+top5 = np.argsort(keras.ops.convert_to_numpy(logits)[0])[-5:][::-1]
+print("top-5 ImageNet-1k class ids:", top5.tolist())
+```
+
+Normalization is inside the model, so pass raw pixels. Map the class ids to the
+[ImageNet-1k label list](https://huggingface.co/datasets/imagenet-1k) to read names.
+
+## Feature Pyramid
+
+For detection / segmentation, take the four stage outputs:
+
+```python
+from zeromodels.models.regnet import RegNetModel
+
+backbone = RegNetModel.from_weights("zeromodels/regnet-y-040", as_backbone=True)
+feats = backbone(np.zeros((1, 224, 224, 3), "float32"), training=False)
+print([tuple(f.shape) for f in feats])
+# spatial 56 / 28 / 14 / 7 (strides 4, 8, 16, 32); channels are the variant's hidden_sizes
+```
+
+The strides are 4, 8, 16, 32, matching a standard CNN backbone. RegNet is fully
+convolutional (no learned position embeddings), so any input resolution works with no
+weight interpolation: build the model at the target `image_size`.
+
+## Data Format
+
+**The model supports both `channels_last` and `channels_first`, and the two are
+bit-exact.** A model reads `keras.config.image_data_format()` when it is **constructed**
+(there is no `data_format` argument); set the format before building.
+
+```python
+import keras
+
+keras.config.set_image_data_format("channels_first")
+model = RegNetImageClassify.from_weights(
+    "zeromodels/regnet-y-040"
+)  # expects (B, 3, H, W)
+```
+
+## Loading Fine-tuned and Community Weights
+
+The `zeromodels/regnet-*` repos above are pre-converted. Any **other** Hugging Face repo whose
+`model_type` is `"regnet"` (the upstream `facebook/regnet-*` and `-seer` checkpoints, or any
+fine-tune) loads with the `hf:` prefix, converting on the fly:
+
+```python
+from zeromodels.models.regnet import RegNetImageClassify
+
+model = RegNetImageClassify.from_weights("hf:facebook/regnet-x-320")
+model = RegNetImageClassify.from_weights("hf:<user>/regnet-finetuned-on-my-data")
+
+# Architecture only, randomly initialized
+model = RegNetImageClassify.from_weights("hf:facebook/regnet-y-040", load_weights=False)
+```
+
+`RegNetModel` accepts `hf:` the same way.

tests/base/model_test_registry.py

@@ -423,6 +423,22 @@
         "input_shape": (2, 32, 32, 3),
         "expected_output_shape": (2, 1000),
     },
+    "RegNetImageClassify": {
+        "module": "zeromodels.models.regnet",
+        "model_cls": "RegNetImageClassify",
+        "model_type": "classification",
+        "init_kwargs": {
+            "embedding_size": 8,
+            "hidden_sizes": (16, 32, 64, 128),
+            "depths": (1, 1, 1, 1),
+            "groups_width": 8,
+            "layer_type": "y",
+            "image_size": (32, 32, 3),
+            "num_classes": 1000,
+        },
+        "input_shape": (2, 32, 32, 3),
+        "expected_output_shape": (2, 1000),
+    },
     "Res2NetImageClassify": {
         "module": "zeromodels.models.res2net",
         "model_cls": "Res2NetImageClassify",

website/mkdocs.yml

@@ -184,6 +184,7 @@ nav:
           - MobileViTV2: mobilevitv2.md
           - PVT: pvt.md
           - PVTv2: pvt_v2.md
+          - RegNet: regnet.md
           - RF-DETR: rf_detr.md
           - RT-DETR: rt_detr.md
           - RT-DETRv2: rt_detr_v2.md

zeromodels/conversion/weight_transfer_util.py

@@ -212,6 +212,7 @@ def transform_conv_weights(
             "sr",
             "reduction",  # spatial_reduction / sequence_reduction (PVT SRA conv)
             "proj",  # patch-embed proj / projection conv (only reached for 4D weights)
+            "attention",  # Squeeze-and-Excitation 1x1 gate conv (RegNet-Y SE)
         ]
     ):
         # Standard 2D convolution

zeromodels/models/__init__.py

@@ -100,6 +100,7 @@
     qwen3_next,
     qwen3_vl,
     qwen3_vl_moe,
+    regnet,
     res2net,
     resmlp,
     resnet,

zeromodels/models/regnet/__init__.py

@@ -0,0 +1,4 @@
+from zeromodels.models.regnet.regnet_config import RegNetConfig
+from zeromodels.models.regnet.regnet_model import RegNetImageClassify, RegNetModel
+
+__all__ = ["RegNetImageClassify", "RegNetModel", "RegNetConfig"]

zeromodels/models/regnet/convert_regnet_hf_to_keras.py

@@ -0,0 +1,136 @@
+import gc
+import json
+
+import keras
+from tqdm import tqdm
+
+from zeromodels.conversion import verify_cls_model_equivalence
+from zeromodels.conversion.exceptions import (
+    WeightMappingError,
+    WeightShapeMismatchError,
+)
+from zeromodels.conversion.hf_download_utils import download_hf_state_dict
+from zeromodels.conversion.weight_split_util import split_model_weights
+from zeromodels.conversion.weight_transfer_util import (
+    compare_keras_torch_names,
+    transfer_weights,
+)
+from zeromodels.models.regnet import RegNetImageClassify
+
+# Hosted variant -> HF (transformers) repo id, for every standard RegNet X/Y FLOP
+# variant (002=0.2 GF ... 320=32 GF). The arch is read from each repo's config.json
+# via RegNetImageClassify.config_from_hf, so no per-variant arch table is needed
+# here; ``hf:facebook/regnet-*`` also loads any of these on the fly. (The larger
+# self-supervised "-seer" checkpoints are not listed but load the same way.)
+REGNET_FLOPS = (
+    "002",
+    "004",
+    "006",
+    "008",
+    "016",
+    "032",
+    "040",
+    "064",
+    "080",
+    "120",
+    "160",
+    "320",
+)
+REGNET_VARIANTS = {
+    f"regnet_{layer_type}_{flops}": f"facebook/regnet-{layer_type}-{flops}"
+    for layer_type in ("x", "y")
+    for flops in REGNET_FLOPS
+}
+
+# keras weight name ``{layer.name}_{weight.name}`` -> HF (torch) name. The keras
+# layers are named as the HF module path with ``.`` replaced by ``_``, so ``_ ->
+# .`` alone reconstructs the path; the rest renames Keras weight suffixes.
+WEIGHT_NAME_MAPPING = {
+    "_": ".",
+    "kernel": "weight",
+    "gamma": "weight",
+    "beta": "bias",
+    "moving.mean": "running_mean",
+    "moving.variance": "running_var",
+}
+
+
+def transfer_regnet_weights(keras_model, state_dict):
+    trainable, non_trainable = split_model_weights(keras_model)
+
+    for keras_weight, keras_name in tqdm(
+        trainable + non_trainable, desc="Transferring weights to Keras"
+    ):
+        torch_name = keras_name
+        for old, new in WEIGHT_NAME_MAPPING.items():
+            torch_name = torch_name.replace(old, new)
+
+        if torch_name not in state_dict:
+            raise WeightMappingError(keras_name, torch_name)
+
+        torch_weight = state_dict[torch_name]
+        if not compare_keras_torch_names(
+            keras_name, keras_weight, torch_name, torch_weight
+        ):
+            raise WeightShapeMismatchError(
+                keras_name, keras_weight.shape, torch_name, torch_weight.shape
+            )
+        transfer_weights(keras_name, keras_weight, torch_weight)
+
+
+if __name__ == "__main__":
+    import importlib.metadata as _meta
+
+    _orig_version = _meta.version
+    _meta.version = lambda name: (
+        "0.23.0" if name == "tokenizers" else _orig_version(name)
+    )
+    import torch
+    import transformers
+    from huggingface_hub import hf_hub_download
+
+    # Compare in true float32: cuDNN TF32 on a GPU inflates the conv/BN diff to
+    # ~1e-2 (HF runs on CPU), which can spuriously trip the 1e-2 parity gate on the
+    # SE ("y") variants. Disabling TF32 restores the real ~5e-6 conversion fidelity.
+    torch.backends.cudnn.allow_tf32 = False
+    torch.backends.cuda.matmul.allow_tf32 = False
+
+    for variant, hf_id in REGNET_VARIANTS.items():
+        print(f"\n{'=' * 60}")
+        print(f"Converting: {variant}  <-  {hf_id}")
+        print(f"{'=' * 60}")
+
+        with open(hf_hub_download(hf_id, "config.json"), encoding="utf-8") as f:
+            hf_config = json.load(f)
+        state = download_hf_state_dict(hf_id)
+        keras_model = RegNetImageClassify(
+            **RegNetImageClassify.config_from_hf(hf_config),
+            include_normalization=False,
+        )
+        transfer_regnet_weights(keras_model, state)
+
+        hf_model = transformers.RegNetForImageClassification.from_pretrained(
+            hf_id
+        ).eval()
+        results = verify_cls_model_equivalence(
+            model_a=hf_model,
+            model_b=keras_model,
+            input_shape=keras_model.input_shape[1:],
+            output_specs={"num_classes": keras_model.output_shape[-1]},
+            comparison_type="hf_to_keras",
+            run_performance=False,
+            atol=1e-2,
+            rtol=1e-2,
+        )
+        if not results["standard_input"]:
+            raise ValueError(
+                "Model equivalence test failed - outputs do not match for standard input"
+            )
+
+        out_path = f"{variant}.weights.h5"
+        keras_model.save_weights(out_path)
+        print(f"  Saved -> {out_path}")
+
+        del keras_model, state, hf_model
+        keras.backend.clear_session()
+        gc.collect()