OASIS-DC: Generalizable Depth Completion via Output-level Alignment of Sparse-Integrated Monocular Pseudo Depth

Jaehyeon Cho · Jhonhyun An
ICRA 2026

📄 Paper 💻 Source Code

---

🔥 News

• [2026] OASIS-DC has been accepted to ICRA 2026.
• Pretrained models will be released soon. (This repository will be updated.)

---

Overview

Recent monocular foundation models excel at zero-shot depth estimation, yet their outputs are inherently relative rather than metric, limiting direct use in robotics and autonomous driving. We leverage the fact that relative depth preserves global layout and boundaries: by calibrating it with sparse range measurements, we transform it into a pseudo metric depth prior. Building on this prior, we design a refinement network that follows the prior where reliable and deviates where necessary, enabling accurate metric predictions from very few labeled samples. The resulting system is particularly effective when curated validation data are unavailable, sustaining stable scale and sharp edges across few-shot regimes. These findings suggest that coupling foundation priors with sparse anchors is a practical route to robust, deployment-ready depth completion under real-world label scarcity.

Note: This repository is under preparation.
Training code, evaluation scripts, and checkpoints will be uploaded soon.

---

What this repository contains

• Official implementation of OASIS-DC
• Training & evaluation code for NYU Depth V2
• Training & evaluation code for KITTI Depth Completion
• Pretrained models & logs
• Reproducible scripts (dataset preparation, evaluation, visualization)

---

Dataset Setup

NYU Depth V2 Dataset Structure

NYUv2_root/
├── train/
│   ├── rgb/                   # RGB images
│   │   ├── 0001.png
│   │   └── ...
│   ├── depth_inpainted_mm/    # Ground truth depth (16-bit PNG, millimeters)
│   │   ├── 0001.png
│   │   └── ...
│   └── rgb_da/                # Precomputed monocular prior (e.g. Depth Anything V2)
│       ├── 0001.png
│       └── ...
└── test/                      # Same layout as train/

NYU Dataset Requirements:

• Sample Lists: one row per sample, <rgb> <depth> <id> <prior>, paths relative to data.data_root (e.g. train/rgb/0001.png train/depth_inpainted_mm/0001.png 0001 train/rgb_da/0001.png, generated with make_list.py). Rows without a prior column read <data_root>/<pre_mono_dir>/<id>.png
• Geometry: resized to data.target_size (240×320), then center-cropped to data.crop_size (228×304)
• Ground Truth: 16-bit PNG in millimeters (max 10m)
• Monocular Prior: 8/16-bit grayscale or color-mapped RGB image, normalized to [0,1] and oriented with the sparse depth
• Sparse Points: Sampled from ground truth at runtime (default: 500 points, identical for a given seed and sample id)
• Splits: data.train_list, data.test_list and optional data.val_list

KITTI Depth Completion Dataset Structure

KITTI_root/
├── data_depth_selection/
│   └── depth_selection/
│       └── val_selection_cropped/
│           ├── image/                  # RGB images
│           │   ├── 2011_09_26_drive_0001_sync_image_0000000000_02.png
│           │   ├── 2011_09_26_drive_0001_sync_image_0000000001_02.png
│           │   └── ...
│           ├── velodyne_raw/           # Sparse LiDAR depth  
│           │   ├── 2011_09_26_drive_0001_sync_velodyne_raw_0000000000_02.png
│           │   ├── 2011_09_26_drive_0001_sync_velodyne_raw_0000000001_02.png
│           │   └── ...
│           └── groundtruth_depth/      # Ground truth depth
│               ├── 2011_09_26_drive_0001_sync_groundtruth_depth_0000000000_02.png
│               ├── 2011_09_26_drive_0001_sync_groundtruth_depth_0000000001_02.png
│               └── ...
└── [est_dir]/                         # Precomputed depth estimates (optional)
    ├── 2011_09_26_drive_0001_sync_image_0000000000_02.png
    └── ...

KITTI Dataset Requirements:

• Evaluation Split (data.data_root): the official val_selection_cropped folder above
• Training Split (data.train_root): KITTI training frames flattened into the same layout (image/, velodyne_raw/, groundtruth_depth/, est/); validation frames are never used for training
• Depth PNGs: 16-bit, meters × 256 as in the official release (data.depth_scale: 256.0; use 1000.0 for millimeter exports)
• Depth Estimates: est/ (data.pre_est_dir), 8/16-bit grayscale or color-mapped RGB named like the image
• Frame Lists (optional): data.train_list / data.val_list / data.test_list with make_list.py rows (<sparse> <gt>) or image file stems; without a list every frame under the root is used
• File Format: Long descriptive naming with date_drive_sync_type_frame_camera format

Creating Dataset Lists

# NYU list: rows "<rgb> <depth> <id> <prior>" relative to --root
python make_list.py --dataset nyuv2 --root /path/to/NYUv2 --out lists/nyu_train_full.txt \
    --nyu-rgb-dir /path/to/NYUv2/train/rgb \
    --nyu-gt-dir /path/to/NYUv2/train/depth_inpainted_mm \
    --nyu-est-dir /path/to/NYUv2/train/rgb_da

# NYU list with automatic detection of rgb/, depth_inpainted_mm/ and rgb_da/ (one split under --root)
python make_list.py --dataset nyuv2 --root /path/to/NYUv2/test --out nyu_test.txt

# KITTI frame list: rows "<sparse> <gt>"
python make_list.py --dataset kitti --root /path/to/kitti_train_flattened --out lists/kitti_train_full.txt

Note: When data.train_list is not set, lists/<dataset>_train_<shots>shot_seed<seed>.txt is used and, if missing, sampled from lists/<dataset>_train_full.txt. File names must match across all directories using stem matching (filename without extension).

Training

Basic Training Commands

# Train NYU n-shot model
python train.py --dataset nyu --config configs/nyu_nshot.yaml --shots 10 --seed 0 --save-dir runs/nyu_10shot

# Train KITTI n-shot model (requires data.train_root)
python train.py --dataset kitti --config configs/kitti_nshot.yaml --shots 100 --seed 0 --save-dir runs/kitti_100shot

# Resume from checkpoint
python train.py --dataset nyu --config configs/nyu_nshot.yaml --shots 1 \
    --resume runs/nyu_1shot/checkpoint_last.pth --save-dir runs/nyu_1shot_resume

# Override configuration parameters
python train.py --dataset nyu --config configs/nyu_nshot.yaml --shots 10 --seed 42 \
    --save-dir runs/nyu_10shot_seed42

Using Training Scripts

# N-shot training scripts with flexible parameters
./scripts/train_nyu_nshot.sh 10 0 runs/nyu_10shot     # Train NYU 10-shot, seed 0
./scripts/train_kitti_nshot.sh 100 1 runs/kitti_100shot_s1  # Train KITTI 100-shot, seed 1

Training Features

• Automatic N-shot Sampling: Training lists are automatically generated by sampling from the full dataset
• Reproducible Seeds: Use different seeds for different n-shot samples
• Flexible Shot Counts: Support for 1-shot, 10-shot, 100-shot, and custom counts
• Checkpoint Management: checkpoint_last.pth after the last epoch, checkpoint_best.pth for the best validation RMSE (only with data.val_list), and checkpoint_epoch_XXX.pth every train.save_every epochs
• Validation: Evaluation during training when data.val_list is set

Evaluation

Basic Evaluation Commands

# Evaluate all modes (full, residual_off, poisson_only)
python eval.py --dataset kitti --config configs/kitti_eval.yaml --mode all \
    --checkpoint runs/kitti_100shot/checkpoint_last.pth --output results.json

# Evaluate specific mode with checkpoint
python eval.py --dataset nyu --config configs/nyu_eval.yaml --mode full \
    --checkpoint runs/nyu_10shot/checkpoint_last.pth --output nyu_results.json

# Poisson-only evaluation (no checkpoint needed)
python eval.py --dataset kitti --config configs/kitti_eval.yaml --mode poisson_only \
    --output poisson_results.json

The model is rebuilt from the model configuration stored in the checkpoint, so the evaluation config only needs to provide data and evaluation settings.

Using Evaluation Scripts

# N-shot evaluation script (supports both datasets)
./scripts/eval_nshot.sh nyu 10 runs/nyu_10shot/checkpoint_last.pth all results/nyu_10shot.json
./scripts/eval_nshot.sh kitti 1 runs/kitti_1shot/checkpoint_last.pth poisson_only results/kitti_1shot.json

Evaluation Modes

1. full: Complete model pipeline including all components
2. residual_off: Network output without residual refinement
3. poisson_only: Pseudo depth P from the Poisson solver, computed exactly as the network receives it (no neural network)
4. all: Run all above modes and compare results

Evaluation Features

• Metrics: RMSE, MAE and δ1
• Timing Analysis: Solver timing statistics for Poisson-only mode
• JSON Output: Structured results saving for analysis
• Progress Tracking: Real-time progress bars with metric updates

Model Options

The default configuration follows the paper: Poincaré-ball affinities over K = {3, 5, 7} computed from the encoder features, per-pixel softmax kernel gates, learnable per-pixel sensor anchoring, the L1+L2 + scale-invariant log loss, and an iteration budget that depends only on the shot count.

Key	Values	Description
model.geometry	poincare (default), hyper, ellip	poincare: Poincaré-ball embedding with geodesic-distance softmax affinities (paper Sec. III-C); hyper: FiLM-modulated convolution approximating hyperbolic affinity (pre-refactor MCPropNet); ellip: cosine affinity
model.kernels	[3, 5, 7] (default)	Propagation kernel sizes
model.use_p_affinity	false (default), true	false: affinities from the encoder features F(I, P, E, M_L) (paper); true: separate pseudo-depth-only encoder (pre-refactor MCPropNet)
model.anchor_learnable, model.anchor_mode	true, map (default)	Learnable per-pixel sensor anchoring α(p)
train.loss	paper (default), l1_si	paper: L1+L2 + scale-invariant log loss on dmax-normalized depth (paper Sec. IV-B); l1_si: L1 + train.mu_scaleinv × scale-invariant loss in meters
train.iterations	set per shot in shot_adaptations	Optimizer steps: 100 (1-shot), 1,000 (10-shot), 3,000 (100-shot, 1-sequence); overrides train.epochs

Pre-refactor MCPropNet checkpoints store no configuration: evaluate them with geometry: hyper, use_p_affinity: true and the kernel/anchor settings they were trained with.

Tests

python -m unittest discover -s tests -t . -v

The pipeline tests (tests/test_pipeline.py) need PyYAML and tqdm and are skipped without them.

---

Experiment

KITTI Depth Completion Benchmark

Method	1-shot		10-shot		100-shot		1-Sequence
	RMSE (m)	MAE (m)	RMSE (m)	MAE (m)	RMSE (m)	MAE (m)	RMSE (m)	MAE (m)
CSPN	9.2748	3.5921	2.0222	0.7825	1.4510	0.5184	2.6289	0.8355
S2D	8.8479	5.6022	5.0500	3.1469	4.2799	2.6633	4.7950	2.5610
NLSPN	7.2899	4.7422	4.0070	2.2588	2.4979	1.1710	4.0290	1.7881
DySPN	2.6350	0.8870	2.2701	0.9150	1.8777	0.6188	2.8530	0.7980
CompletionFormer	4.7212	2.3789	3.1601	1.4740	2.6122	1.3299	4.5588	1.9603
BPNet	5.4000	1.0740	1.8799	0.5559	1.3001	0.3910	2.1322	0.6420
DepthPrompting	2.9840	1.1430	2.3988	1.1290	1.8249	0.6240	2.9468	0.9869
OASIS-DC (Ours)	1.4190	0.5073	1.2830	0.4001	1.2455	0.3548	1.5782	0.5540

---

Contact

• Jaehyeon Cho — jjh000503@gachon.ac.kr
• Jhonhyun An — jhonghyun@gachon.ac.kr

If you have questions, please open an issue or contact us via email.

---

Related Works

We are deeply grateful for the following outstanding opensource work; without them, our work would not have been possible.

• UniDC
• DepthPrompting
• Depth Anything V2
• CSPN