fix: handle empty batches in detection and end-to-end predictors

[linhongyu510]

DetectionPredictor, OCRPredictor and KIEPredictor raise on an empty page list instead of returning an empty result. RecognitionPredictor has always short-circuited on empty input (recognition/predictor/pytorch.py:50), and OrientationPredictor gained the same behaviour in #2069 — these three were the remaining gap, so this follows that precedent rather than introducing a new convention.

Filtering a batch down to nothing is ordinary caller code (skip already-processed pages, drop files that failed a check upstream), and today it fails from internals that never mention the empty input.

Reproduction

from doctr.models import ocr_predictor, kie_predictor, detection

ocr_predictor(pretrained=False)([])       # IndexError: list index out of range
kie_predictor(pretrained=False)([])       # IndexError: list index out of range
detection.detection_predictor(pretrained=False)([])  # IndexError

There are three separate failure points on the way down, which is why the fix is not a one-liner in a single helper:

Location	Failure
preprocessor/pytorch.py:73	num_batches is correctly 0, then samples[0] is read to pick the tuple/tensor branch → IndexError
utils/geometry.py:124	zip(*(_detach(box) for box in boxes)) over no boxes → ValueError: not enough values to unpack (expected 2, got 0)
models/_utils.py:276	KIE path only: {k: ... for k in x[0]} → IndexError

I confirmed the ordering by fixing them one at a time: guarding batch_inputs moved the crash to detach_scores, and guarding that one let the whole call return Document(pages=[]).

Fix

Guard at the three public entry points instead of patching each internal, so batch_inputs, detach_scores and invert_data_structure keep their non-empty precondition and stay simple.

Two details worth flagging:

• DetectionPredictor returns the shape its own return_maps contract promises: [] normally, ([], []) when return_maps=True.
• KIEPredictor returns KIEDocument(pages=[]), not Document(pages=[]). KIEDocument subclasses Document, so returning the base class would type-check and pass an isinstance assertion while silently dropping the per-class page shape. The test asserts the exact type for this reason — I verified it catches the degradation by deliberately returning Document there and watching the test go red.

Tests

Added test_predictors_on_empty_batch, which covers detection (both return_maps values), recognition (already-correct, asserted so the three stay consistent), and both end-to-end predictors.

The test is load-bearing: reverting the guards fails it at preprocessor/pytorch.py:73.

tests/pytorch/test_models_zoo_pt.py::test_predictors_on_empty_batch  1 passed
tests/pytorch/test_models_zoo_pt.py                                 14 passed, 6 errors
tests/common/                                                        523 passed, 1 failed, 7 errors
ruff check . / ruff format --check                                   clean
mypy doctr/                                                          clean (171 source files)

The errors and the one failure are pre-existing on this machine and unrelated: the errors are requests.exceptions.ConnectionError from tests that download weights or fixtures, and tests/common/test_io.py::test_read_html fails with OSError: cannot load library (missing WeasyPrint system libs). I verified the test_read_html failure reproduces on a clean checkout via git stash.

AI assistance was used for this change; I reviewed every changed line and ran the commands above locally.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 97.21%. Comparing base (c7f7218) to head (05d0615).

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2138      +/-   ##
==========================================
- Coverage   97.23%   97.21%   -0.02%     
==========================================
  Files         169      169              
  Lines       10039    10045       +6     
==========================================
+ Hits         9761     9765       +4     
- Misses        278      280       +2     

Flag	Coverage Δ
unittests	97.21% <100.00%> (-0.02%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[felixdittrich92]

Hi @linhongyu510 👋,

Thanks for the PR. Left one comment.

[felixdittrich92]

Please remove the docstring afterwards fine to merge 👍

[linhongyu510]

Docstring removed in 05d0615 — thanks for catching it. I checked the convention before deleting rather than just complying: 3 of 371 test functions in tests/ carry a docstring, and the other two are single-line, so the 10-line block here was the outlier. The load-bearing context was already in the inline comments (the return_maps shape contract and the reason for type(out) is over isinstance), so nothing was lost by dropping it.

Also rebased onto current main (c7f7218), which only touches vocabs.py / docs / pre-commit — no overlap with the four files here, and 0 commits behind now.

Re-verified after the rebase rather than assuming the earlier run still held:

pytest tests/pytorch/test_models_zoo_pt.py::test_predictors_on_empty_batch   1 passed
pytest tests/pytorch/test_models_zoo_pt.py                                  14 passed, 6 errors
ruff check / ruff format --check (4 changed files)                           clean
mypy doctr/                                                                 clean, 171 files

The 6 errors are fixture downloads timing out on this machine (3.bp.blogspot.com unreachable); I confirmed the attribution by git stash-ing the change and re-running — identical 14 passed / 6 errors with the same error set, so they are not from this diff.

I also re-confirmed the test still earns its place after the edit, by reverting each guard one at a time:

reverted guard	failure
DetectionPredictor	IndexError at preprocessor/pytorch.py:73 (samples[0])
OCRPredictor	ValueError: not enough values to unpack at utils/geometry.py:124
KIEPredictor	IndexError at models/_utils.py:276 (x[0])

Three distinct failure points, matching the three in the description. All guards restored afterwards.

[linhongyu510]

One thing that needs your click: the nine workflows on 05d0615 are all sitting at action_required — the fork-PR approval gate, not a failure on my side. Compare with the previous head a11910c, where the same nine came back green once approved:

workflow	a11910c (approved)	05d0615 (current)
tests / style / builds / demo / references / scripts / pull_requests / Docker image on ghcr.io	success	action_required
docker	cancelled	action_required

So CI here needs an "Approve and run" whenever I push. Since the only change between the two heads is the 10-line docstring deletion plus the rebase onto c7f7218, I would expect the same result, but I would rather point at the gate than claim a green run I cannot produce myself.

[linhongyu510]

The docstring is gone and CI is green now that you approved the run — 56 of the 57 checks pass, including tests, mypy, ruff, all 15 build matrix jobs, and codecov/patch at 100.00% of diff hit.

The one red check is codecov/project (97.21%, -0.02%), and I dug into it rather than asking you to wave it through. It is not caused by this PR — it is a pre-existing nondeterminism in the coverage of a file this PR never touches.

Codecov's own per-file data says so. Of the 169 files, exactly one has a changed miss count, and it is not one of mine:

file	in this PR's diff?	misses base → head
models/detection/predictor/pytorch.py	yes	1 → 1
models/kie_predictor/pytorch.py	yes	0 → 0
models/predictor/pytorch.py	yes	42 → 42
transforms/functional/base.py	no	4 → 6

For that last file Codecov reports has_diff: false, patch: null, and identical line counts (68 → 68) — same code, 2 more misses. The arithmetic matches the project delta exactly: 278 → 280 misses, and 9761/10039 − 9765/10045 = 0.018%.

Root cause. create_shadow_mask in that file draws from an unseeded RNG and branches on it:

_params = np.random.rand(1)
quad_idx = int(_params[0] / 0.25)   # 0,1,2,3 -> four different branches
if quad_idx % 2 == 0:              # line 173
    ...
    if quad_idx == 0: ...          # line 175
else:                              # line 178
    ...
    if quad_idx == 1: ...          # line 180

Each run reaches only some of those lines, so the miss count for the file moves on its own. Measured with coverage.py, changing nothing but the seed:

seed=0 : 31 missing lines   missing in 165-185: [173, 174, 175]
seed=1 : 31 missing lines   missing in 165-185: [178, 179, 180]
seed=7 : 36 missing lines   missing in 165-185: [175, 178, 179, 180]
seed=3 : 31 missing lines   missing in 165-185: [178, 179, 180]

Over 400 seeds the four quadrants come up 104 / 99 / 95 / 102 times, so which lines get counted is close to a coin flip per run.

This also shows up on main without any PR involved. From Codecov's commit history:

commit on main	what it changed	misses	coverage
60cb879	Correct some vocabs	276	97.25%
1a6f4c1	Update docs — no code at all	278	97.23%
c7f7218	Correct vocabs (#2139)	278	97.23%

A docs-only commit moved it by the same 2 misses / 0.02%. My own earlier head a11910c happened to land on the lucky side and reported +<0.01%; after the rebase onto c7f7218 it landed on the unlucky side. The diff between those two heads is the 10-line docstring deletion.

So this PR adds 7 executable lines, all 7 covered, and touches no other file. Happy to seed the RNG in the shadow-mask tests as a separate PR if you'd like that flake fixed — it is unrelated to this change, so I did not fold it in here.

One request: the review is still marked changes_requested, which is what mergeable_state: blocked is reporting now that the checks are done. The thread is already showing as outdated against 05d0615.

[linhongyu510]

Opened #2144 for the coverage flake described above — it is independent of this PR and touches only tests/pytorch/test_transforms_pt.py (+37/-0, no source change), so the two can be reviewed and merged in either order.

If #2144 goes in first, the codecov/project noise on this PR should settle on its own, since that file stops contributing a random delta.

[linhongyu510]

@felixdittrich92 Two small maintainer actions would unblock this pair — no code change needed on either.

Here (#2138): the docstring you asked to remove is gone (05d0615), and that thread is now marked outdated. CI on this head is 56 of 57 green, including tests, mypy, ruff, all 15 build matrix jobs, and codecov/patch at 100% of diff hit. The single red is codecov/project (-0.02%), which I traced to a pre-existing coverage flake in transforms/functional/base.py — a file this PR never touches — rather than asking you to wave it through. So the only thing holding mergeable_state at BLOCKED is the changes_requested from 09-15; a dismissal or re-review would clear it.

And #2144, the fix for that flake, currently shows only 1 check run — the fork-PR approval gate has not been triggered on it at all, so there is no result to look at. An "Approve and run" would let it report. It is test-only, +37/-0, no source change.

Worth noting the two are order-independent: if #2144 lands first, the codecov/project noise here should settle on its own, since that file stops contributing a random delta.