Recover truncated video decodes via an ffmpeg fallback - #1
Merged
Conversation
Some variable-rate AVIs make OpenCV stop decoding partway through and report no error. extract_visual then saw frames only for the front of the clip, so every later time bin came out NaN while the audio features (decoded separately by ffmpeg) looked fine. One of Dr. Lin's shared clips (10a4) hit this: OpenCV read 330 of 614 frames and the back half of the clip was blank. Check the decode against the container's declared frame count and, on a short read, retry with ffmpeg, which reads these files whole. Keep whichever backend got further and raise a RuntimeWarning rather than emitting NaN silently. The decode loop is split into _iter_frames_cv2 / _iter_frames_ffmpeg / _visual_rows so both backends share one feature path.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
extract_visuallost the tail of any clip that OpenCV stopped decoding partway through. Some variable-rate AVIs makecv2.VideoCapturegive up early and report no error, so features covered only the front of the clip and every later time bin came out NaN, while the audio features (decoded separately by ffmpeg) looked fine.Found while extracting Dr. Lin's shared movie clips:
10a4.avideclares 614 frames, OpenCV decoded 330, and the back half of the clip was blank.Fix
RuntimeWarninginstead of silently emitting NaN._iter_frames_cv2/_iter_frames_ffmpeg/_visual_rowsso both backends share one feature path.Verification
10a4.avinow recovers all 614 frames straight from the raw AVI, zero NaNs. Cross-checked against an independent clean transcode: luminance r=0.99, faces r=0.99, audio r=1.00.pytestand the barepython tests/test_pipeline.pyrunner (16/16).