Skip to content

Fix non-monotonic output timestamps from c2.mtk.hevc.decoder - #3352

Draft
2bitoperations wants to merge 2 commits into
androidx:releasefrom
2bitoperations:fix/mediatek-hevc-output-timestamp-reorder
Draft

Fix non-monotonic output timestamps from c2.mtk.hevc.decoder#3352
2bitoperations wants to merge 2 commits into
androidx:releasefrom
2bitoperations:fix/mediatek-hevc-output-timestamp-reorder

Conversation

@2bitoperations

@2bitoperations 2bitoperations commented Jul 29, 2026

Copy link
Copy Markdown

Fixes #3347

Problem

On Google TV Streamer 4K (MediaTek MT8696, c2.mtk.hevc.decoder),
decoding specific HEVC Main10 content causes the codec to return
output buffers with non-monotonic presentationTimeUs, while the
order in which those buffers are released remains correct — confirmed
by comparing each buffer's release-order rank against its
timestamp-sorted rank: the two never diverge by more than a few
positions, and the timestamp values themselves are individually
correct for the frame they belong to.

Reproduced independently of ExoPlayer, using a minimal test harness
built directly on AMediaCodec/AMediaExtractor (no ExoPlayer, no
Java MediaCodec) against the same file: identical non-monotonic
timestamp pattern. This confirms the defect is in the decoder/HAL
output, not in ExoPlayer's handling of it.

MediaCodecRenderer processes output buffers in raw release order
but uses the codec-reported presentationTimeUs directly for the
render/drop decision in MediaCodecVideoRenderer. When a buffer's
timestamp is behind the previous one, it's treated as arriving too
late and dropped. Measured on affected content: ~30% of decoded video
frames dropped, sustained throughout playback, reproduced in the
Media3 demo app as well as Plex, Jellyfin, and other ExoPlayer-based
players on the same device/file.

Fix

Re-derive each output buffer's presentationTimeUs from its release
order and the stream's known frame duration (from Format.frameRate),
instead of trusting the codec-reported value. Buffer release order is
left untouched — buffers are not held or reordered.

Testing

Verified on the affected device (Google TV Streamer 4K) against three
files that reliably reproduce the issue on unpatched main,
including a 90-second real-world clip: dropped-frame count goes from
~30% to 0, and the fix was visually confirmed on-device — direct
observation of smooth playback, not just log metrics. HOWEVER, with this approach, video and audio lose sync.

An earlier approach that buffered and released output buffers in
sorted-timestamp order was also tried and rejected: it also eliminated
logged drops, but produced visible playback artifacts on-device. That
result is why this PR does not reorder buffers.

On Google TV Streamer 4K (MT8696, c2.mtk.hevc.decoder), specific HEVC
Main10 content causes the codec to return output buffers with
non-monotonic presentationTimeUs, while the buffer release order
itself remains correct. MediaCodecRenderer processes output buffers
in raw release order and uses the codec-reported timestamp directly
for the render/drop decision, causing MediaCodecVideoRenderer to treat
these buffers as arriving too late and drop them.

Measured on affected content: ~30% of decoded video frames dropped,
sustained throughout playback.

This re-derives each output buffer's presentationTimeUs from its
release order and the stream's known frame duration, instead of
trusting the codec-reported value. Buffer release order is untouched.

Verified on-device on the affected hardware/content: dropped frames
go to 0, playback confirmed visually smooth by direct observation.
@google-cla

google-cla Bot commented Jul 29, 2026

Copy link
Copy Markdown

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

The per-frame duration was rounded to the nearest microsecond once,
then multiplied by a growing frame index. Any fractional-microsecond
remainder in the true frame duration (e.g. 41708.333...us for
24000/1001fps content) was silently dropped every frame and never
recovered, so the error accumulated linearly with runtime: ~0.7ms
over a 90-second clip (undetectable, and the only length tested
before this fix), but ~58ms by the end of a 2-hour movie — enough to
produce audible/visible A/V desync that gets worse the longer
playback continues.

Fix: keep the per-frame duration unrounded and compute each buffer's
offset independently as round(frameIndex * unroundedDurationUs),
rounding only the final result. Error no longer accumulates; it stays
bounded to at most +/-0.5us for the life of the stream.

Found via real-world testing in a third-party player (Plezy) on full-
length movies, where the 90-second on-device test clips used to
validate the original fix could never have revealed it.
@2bitoperations

Copy link
Copy Markdown
Author

Pushed a fix for a second bug: audio/video would slowly drift out of
sync over long playback. Cause was an accumulating rounding error in
the per-frame timestamp math — invisible on a 90s test clip, ~58ms by
the end of a 2-hour movie. Fixed by rounding only the final offset,
not the per-frame duration.

Tested in two real players on the affected hardware (Jellyfin Android
TV, and Plezy). 40 minutes of real playback, no drift, no stutter.

@FongMi

FongMi commented Aug 1, 2026

Copy link
Copy Markdown

Thanks for the detailed investigation and for providing a legal,
reproducible sample. I agree that changing the late-frame threshold
does not address the underlying problem.

However, I don't think the current evidence isolates this as a
MediaTek decoder/HAL defect yet. The repro MP4 itself appears to have
a questionable presentation timeline for reordered frames.

I downloaded tos_x265_3.3_final.mp4 and inspected it with ffprobe.
All 360 video packets have PTS == DTS, even though the HEVC stream
uses B-frames and B-pyramid. For example:

packet PTS/DTS:
-0.080000 / -0.080000
-0.041667 / -0.041667
 0.000000 /  0.000000
 0.041667 /  0.041667
...

After decoding, the propagated frame PTS becomes non-monotonic:

frame PTS:
0.041667
0.083333
0.000000
0.125000
0.250000
0.291667
0.208333
...

while FFmpeg's best_effort_timestamp reconstructs approximately:

0.041667
0.083333
0.125000
0.166667
0.250000
0.291667
0.333333
...

This looks consistent with missing or incorrect composition offsets
in the MP4, possibly introduced by the raw-HEVC stream-copy muxing
pipeline in the repro README.

Android documents BufferInfo.presentationTimeUs as being derived
from the timestamp queued with the corresponding input buffer:

https://developer.android.com/reference/android/media/MediaCodec.BufferInfo#presentationTimeUs

Therefore, reproducing the same output timestamp sequence with
AMediaCodec/AMediaExtractor proves that ExoPlayer did not introduce
the sequence, but it does not by itself prove that the decoder/HAL is
behaving incorrectly. It may simply be propagating input timestamps
that do not describe the HEVC presentation order correctly.

Could we rule this out before changing MediaCodecRenderer?

  1. Generate the same encoded content in a container with correct
    composition timestamps and verify that some packet PTS values
    differ from DTS where reordering requires it.
  2. Log each timestamp queued into MediaCodec and its corresponding
    output BufferInfo.presentationTimeUs.
  3. Compare packet PTS/DTS for the affected x265 3.3 file and the
    unaffected x265 4.1 file.

There is also a separate scope concern with the current patch: despite
being motivated by one MTK decoder, it rewrites timestamps for every
video decoder whenever Format.frameRate is known. This would replace
valid VFR timing, intentional gaps, duplicate frames, and format
transitions with a synthetic CFR timeline. The frame duration is also
not reset when the input frame rate changes.

The patch explains why this particular CFR sample becomes smooth, but
I think the source timeline needs to be ruled out before treating it
as a general Media3 renderer fix. If a player-side workaround is still
needed afterward, it should probably be gated to the affected codec
and device, activated only after detecting the known timestamp pattern,
and kept out of the generic MediaCodecRenderer path.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

2 participants