Skip to content

Recover truncated video decodes via an ffmpeg fallback - #1

Merged
JoshW-dev merged 1 commit into
mainfrom
fix/opencv-partial-decode-fallback
Jul 17, 2026
Merged

Recover truncated video decodes via an ffmpeg fallback#1
JoshW-dev merged 1 commit into
mainfrom
fix/opencv-partial-decode-fallback

Conversation

@JoshW-dev

Copy link
Copy Markdown
Owner

What

extract_visual lost the tail of any clip that OpenCV stopped decoding partway through. Some variable-rate AVIs make cv2.VideoCapture give up early and report no error, so features covered only the front of the clip and every later time bin came out NaN, while the audio features (decoded separately by ffmpeg) looked fine.

Found while extracting Dr. Lin's shared movie clips: 10a4.avi declares 614 frames, OpenCV decoded 330, and the back half of the clip was blank.

Fix

  • Compare OpenCV's decoded frame count against the container's declared count.
  • On a short read (< 98%), retry with ffmpeg, which reads these files whole.
  • Keep whichever backend got further; raise a RuntimeWarning instead of silently emitting NaN.
  • Decode loop split into _iter_frames_cv2 / _iter_frames_ffmpeg / _visual_rows so both backends share one feature path.

Verification

  • 10a4.avi now recovers all 614 frames straight from the raw AVI, zero NaNs. Cross-checked against an independent clean transcode: luminance r=0.99, faces r=0.99, audio r=1.00.
  • Full 165-clip corpus re-extracted from the raw AVIs: 0 NaN cells, uniform schema, fallback fired on exactly the one broken clip (no false positives).
  • New regression test truncates the OpenCV path and asserts full recovery + the warning; verified it fails without the fix. Green under both pytest and the bare python tests/test_pipeline.py runner (16/16).

Some variable-rate AVIs make OpenCV stop decoding partway through and
report no error. extract_visual then saw frames only for the front of the
clip, so every later time bin came out NaN while the audio features (decoded
separately by ffmpeg) looked fine. One of Dr. Lin's shared clips (10a4) hit
this: OpenCV read 330 of 614 frames and the back half of the clip was blank.

Check the decode against the container's declared frame count and, on a short
read, retry with ffmpeg, which reads these files whole. Keep whichever backend
got further and raise a RuntimeWarning rather than emitting NaN silently. The
decode loop is split into _iter_frames_cv2 / _iter_frames_ffmpeg / _visual_rows
so both backends share one feature path.
@JoshW-dev
JoshW-dev merged commit 70520ec into main Jul 17, 2026
3 checks passed
@JoshW-dev
JoshW-dev deleted the fix/opencv-partial-decode-fallback branch July 17, 2026 16:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant