Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 8 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@

## 📖 Introduction

ZeroModels is a collection of models with pretrained weights, built entirely with Keras 3. It supports a range of tasks, including classification, object detection (DETR, RT-DETR, RT-DETRv2, RF-DETR, D-FINE, EfficientDet, OWL-ViT, OWLv2, Grounding DINO), segmentation (SAM, SAM2, SAM3, SegFormer, DeepLabV3, EoMT, MaskFormer, Mask2Former, OneFormer, MobileViT-DeepLabV3, RF-DETR), monocular depth estimation (Depth Anything V1, Depth Anything V2, TIPSv2-DPT), feature extraction (DINO, DINOv2, DINOv3), vision-language modeling (CLIP, SigLIP, SigLIP2, MetaCLIP 2, TIPSv2), speech recognition (Whisper, Speech2Text, Moonshine, Granite Speech 5), speech-aware language modeling (Granite Speech, Granite Speech Plus), text encoding and masked language modeling (BERT, ModernBERT, ELECTRA, RoBERTa, XLM-RoBERTa, DeBERTa, DeBERTa-v2, DeBERTa-v3), text generation with large language models (GPT, GPT-2, Qwen2, Qwen2-MoE, Qwen3, Qwen3-MoE, Qwen3-Next, Qwen3.5, GPT-OSS, Llama 2, Llama 3, Llama 4, Mistral, Mixtral, Gemma, Gemma 2, MiniMax-Text-01, MiniMax-M2, DeepSeek-V2, DeepSeek-V3, DeepSeek-V4, GLM-4, GLM-4-0414, GLM-4.5/GLM-4.6, GLM-5/GLM-5.1/GLM-5.2), text-to-text encoder-decoder modeling (T5), multimodal vision-language generation (Qwen2-VL, Qwen2.5-VL, Qwen3-VL, Qwen3-VL-MoE, Qwen3.5-MoE, InternVL3, Gemma 3, Gemma 3n, Gemma 4, Gemma 4 Unified, Mistral 3, DeepSeek-VL, Janus-Pro, MiniMax-M3-VL, GLM-4V, GLM-4.5V, Kimi K2.5, Kimi K2.6, Kimi K2.7-Code), vision-language grounding across object detection, OCR, pointing, and referring (LocateAnything), and more. It includes hybrid architectures like MaxViT alongside traditional CNNs and pure transformers. zeromodels includes custom layers and backbone support, providing flexibility and efficiency across various applications. For backbones, there are various weight variants like `in1k`, `in21k`, `fb_dist_in1k`, `ms_in22k`, `fb_in22k_ft_in1k`, `ns_jft_in1k`, `aa_in1k`, `cvnets_in1k`, `augreg_in21k_ft_in1k`, `augreg_in21k`, and many more.
ZeroModels is a collection of pretrained models built entirely in Keras 3. It spans a broad range of tasks, including image classification, object detection, segmentation, monocular depth estimation, feature extraction, vision-language modeling (VLMs), speech recognition, speech-aware language modeling, text encoding and masked language modeling, large language models (LLMs), text-to-text encoder-decoder modeling, multimodal vision-language generation, and more.

## ⚡ Installation

Expand All @@ -31,11 +31,14 @@ pip install -U git+https://github.com/IMvision12/ZeroModels

## 📑 Documentation

📖 **[imvision12.github.io/ZeroModels](https://imvision12.github.io/ZeroModels/)** — the rendered docs, with search.
**[ZeroModels Documentation](https://imvision12.github.io/ZeroModels/)**

Per-model guides - with architecture notes, usage examples, and available pretrained weights, cover one page per model across every supported task (classification, object detection, segmentation, depth estimation, feature extraction, vision-language, speech recognition, text encoding, and language modeling). Classification backbones share a single [page](https://imvision12.github.io/ZeroModels/classification_backbones/) since they all follow the same `XModel` / `XImageClassify` two-class structure; each other model has its own. Every example on those pages prints its real, measured output.
Detailed guides are available for all supported tasks, with architecture notes, usage examples, pretrained weights, and real model outputs.

Classification backbones share a single [documentation page](https://imvision12.github.io/ZeroModels/classification_backbones/), while other model families have dedicated pages.

Documentation sources are also available in [`docs/`](docs/).

The Markdown sources live in [`docs/`](docs/) if you would rather read them in the repo.

## 📑 Models

Expand Down Expand Up @@ -142,6 +145,7 @@ The Markdown sources live in [`docs/`](docs/) if you would rather read them in t
|---------------|-------------------|---------------------|
| D-FINE | [D-FINE: Redefine Regression Task of DETRs as Fine-grained Distribution Refinement](https://arxiv.org/abs/2410.13842) | `transformers` |
| DETR | [End-to-End Object Detection with Transformers](https://arxiv.org/abs/2005.12872) | `transformers` |
| Table Transformer | [PubTables-1M: Towards comprehensive table extraction from unstructured documents](https://arxiv.org/abs/2110.00061) | `transformers` |
| EfficientDet | [EfficientDet: Scalable and Efficient Object Detection](https://arxiv.org/abs/1911.09070) | `automl` |
| RT-DETR | [DETRs Beat YOLOs on Real-time Object Detection](https://arxiv.org/abs/2304.08069) | `transformers` |
| RT-DETRv2 | [RT-DETRv2: Improved Baseline with Bag-of-Freebies for Real-Time Detection Transformers](https://arxiv.org/abs/2407.17140) | `transformers` |
Expand Down
Binary file added assets/data/attention_table2.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added assets/data/attention_table3.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added assets/table_transformer_batch_output.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added assets/table_transformer_output.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
3 changes: 2 additions & 1 deletion docs/classification_backbones.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ backbone = ResNetModel.from_weights("zeromodels/resnet50_a1_in1k")
feature_map = backbone(images) # (B, H/32, W/32, 2048)
```

The same pattern works for every classification arch - swap `ResNet` for `CaiT`, `ViT`, `ConvNeXt`, `EfficientNet`, `Swin`, `MobileNetV3`, etc.
The same pattern works for every classification arch - swap `ResNet` for `CaiT`, `ViT`, `ConvNeXt`, `RegNet`, `EfficientNet`, `Swin`, `MobileNetV3`, etc.

## Data Format

Expand Down Expand Up @@ -65,6 +65,7 @@ The number of stages and their semantics depend on the architecture:
| Family | Stages when `as_backbone=True` |
|-----------------------------------------------------|-----------------------------------------------------------------|
| ResNet / ResNetV2 / Res2Net / ResNeXt / SENet | 4 (one per residual stage) |
| RegNet | 4 (one per stage; strides 4, 8, 16, 32) |
| ConvNeXt / ConvNeXtV2 | 4 |
| EfficientNet / EfficientNet-Lite / EfficientNetV2 | 5 (at stride-2 boundaries; head conv excluded) |
| EfficientFormer | 4 |
Expand Down
165 changes: 0 additions & 165 deletions docs/regnet.md

This file was deleted.

Loading
Loading