Repository navigation
Read a video's start time from its GPS clock, and tell whether creation_time marks the start or the end - #832
Merged
Conversation
… duration A video's start time decides where its sampled frames land on a GPS track. We derived it as creation_time - duration, which pushed every video one full duration into the past unless its camera happened to stamp the creation time at the end of the recording. That subtraction came from 644701c (2019), whose removed comment says why: "Blackvue actually reports endtime in the created_at exif field". The workaround was later generalised to every video. ISO/IEC 14496-12 defines creation_time as the creation time of the presentation, which for a recording is the moment it started, and that is what cameras and ffmpeg write. Take the start time from the video's own telemetry when it has some. That clock is absolute UTC, so it is right for BlackVue, whose container time is the end of the recording, and for GoPro, whose container time is in local time (mapillary#819). Fall back to creation_time for the plain MP4s that get geotagged from a GPX, which are exactly the videos that have no telemetry to sync against. Reported for a 10:49 dashcam video paired with a GPX starting at the video's creation time: every sampled frame came out 648.88s early, so the documented GPX workflow failed with MapillaryOutsideGPXTrackError on all of them. Where the track does reach back far enough -- a GPX covering a recording split into several files -- the frames were silently placed up to a kilometre off instead. With this change all 289 sampled frames land on the track exactly.
The previous commit reads a video's start time from its telemetry and otherwise takes creation_time as the start. That fixes the reported dashcam, and Sony and Insta360, which also stamp the start. But it moves every video from a camera that stamps the end one full duration late. Those are most dashcams: Viofo, Vantrue, AZDome and others without GPS telemetry, which people geotag from a GPX, plus the Ricoh Theta X and the Labpano PanoX V2. With interval sampling and a GPX of their own track, sample videos from a Viofo A129 and an AZDome GS63H went from 119 and 57 geotagged frames to 1 and 0. Nothing in the metadata says which of the two a camera writes, so look for evidence, in this order: - a camera known to stamp the end: the Theta X and the PanoX V2 by their make and model tags, BlackVue by the boxes it writes its GPS log into, which it does whether or not it ever gets a fix - a date and time in the file name that matches only one of the two, allowing for a whole UTC offset because cameras name files in local time - otherwise the start, with a warning that prints the --video_start_time value to use if the camera stamps the end Also: - Take the median of the first 5 plausible GPS timestamps instead of the first one. The PanoX V2 can record a stale first fix seconds older than the rest, and a GPS clock that was never set (GoPro writes 2000-01-01 until its first fix) must not become the start time. The hero8 fixture moves by 0.1s because the GPS5 samples of one payload share a timestamp. - Fall back to the creation time when the telemetry fails to parse, instead of failing to sample the video. - Measure BlackVue point times from the first NMEA line, which is the start of the recording, instead of from the first valid fix. Until the receiver gets a fix the camera logs lines without positions, so every point, and with the previous commit every frame sampled by interval, was late by the time to first fix. Across 433 BlackVue sample videos 7 took more than 5s to get a fix, one 60s. - Distance sampling resolves the start time only when a sampled frame has no absolute GPS time of its own, instead of requiring a creation time up front for every video. - Ignore a malformed creation_time with a warning instead of raising. On top of mapillary#839 and mapillary#841: the start time raises a plain MapillaryVideoError, which the sample summary reports, as mapillary#839's other sampling errors do. mapillary#839's test of an unreadable start time patched probe_video_start_time(), which this removes; it now covers interval sampling through _extract_video_start_time(), and distance sampling through a track without a GPS clock, which is when distance sampling looks the start time up. mapillary#841's comment on why only --video_start_time places a GPX gave creation_time - duration as the reason, which no longer holds; the reason now is that a video without a GPS clock would get its start time from its creation time, which is stamped at the start or the end of the recording, some in local time.
caglarpir
force-pushed
the
fix-video-start-time
branch
from
October 20, 2026 11:17
2fb148d to
2567ade
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The bug
A community report (Mapillary forum + FB group): a video recorded at 4.91 fps, paired with a GPX
of the same drive, comes out with its frames on the wrong part of the route. The same file + GPX
works in Google Street View.
The reporter described it as the frame rate being misread. It isn't — the container is parsed
correctly (
mdhdtimescale 15712,stts3186 × 3200 → exactly 4.91 fps, andmp4_sample_parserreproduces every composition time to the microsecond). What is wrong isthe video's start time, and since the frames are located along the GPX by their absolute
timestamps, getting that wrong moves all of them together.
Probe.extract_stream_start_time()returnedcreation_time - duration. For the reported file(
creation_time2026-08-08T08:47:32Z, duration 648.879837s) that is 08:36:43.120Z, while theGPX starts at 08:47:32Z — the creation time, to the second. Every sampled frame came out one
full video duration early, and the workflow the README documents fails outright:
The hard failure is the lucky case. When the track does reach back far enough — a GPX covering
a drive that the camera split across several files — the frames from every file after the first
are silently placed one file-length back along the route. On this track that is up to ~1 km,
and the images end up bunched onto road they were not shot on, which is what the report
describes.
Where it came from
The subtraction was introduced in 644701c (Feb 2019). The comment it deleted says what it was
for:
# Blackvue actually reports endtime in the created_at exif fieldIt was a BlackVue workaround that later got generalised to every video. But neither reading is
right for every camera, and nothing in the file says which one applies. Checked against sample
videos, by comparing the creation time with the camera's own GPS clock or with the neighbouring
clips of the same drive:
Ricoh Theta X and the Labpano PanoX V2.
start, but in local time (Video Timestamping Bugs with GoPro MAX #819).
The fix
Resolve the start time in this order:
--video_start_time, if given (unchanged).GoPro: the median of the first 5 plausible GPS timestamps, each mapped back to the video's
time 0. Timestamps before 2010 (a GoPro writes 2000-01-01 until its first fix) or more than
a day in the future are skipped, and the median keeps a stale first fix (seen on the PanoX
V2, seconds older than the rest) from shifting the result.
creation_time, read as the start or the end of the recording bylooking for evidence:
free/gpsandfree/cprtboxes it writes whether or not it ever got a fix, and the Theta X and PanoX V2, by their
make and model tags;
20230106_150558_00422F.MP4,2023_0725_141524_103.MP4, …) that matches only one ofcreation_timeandcreation_time - duration, allowing for a whole UTC offset because cameras name files inlocal time;
--video_start_timevalue to pass ifthe camera stamps the end.
A start-time source that fails (telemetry that does not parse, a malformed
creation_time)now falls through to the next one with a warning instead of failing the video.
Behaviour changes
timestamped one duration later than before, which is when they were shot.
identifies them. The others now get the start reading, with the warning, where every camera
used to get the end reading silently.
hero8.mp4fixture that moves the first sample from2019_11_18_15_41_12_354(the camera'slocal RTC, minus the duration, labelled UTC) to
2019_11_18_23_42_08_539(real UTC, matchingthe GPS positions written into the same frames). This is Video Timestamping Bugs with GoPro MAX #819 items 2 and 4.
start of the recording. Until the receiver gets a fix the camera logs NMEA lines without
positions, so every point was late by the time to first fix. They are now measured from the
first NMEA line, which changes BlackVue point times wherever they are used, not only the
start time. For most videos the two coincide; on 433 BlackVue sample videos, 7 took more than
5 s to get a fix, the slowest 60 s.
time from it. The start time is resolved, in the order above, only for frames without one, so
a video without a usable creation time no longer fails when it does not need one. That includes
every frame placed along a GPX, whose points carry their own time: since Honour --geotag_source in distance-based sampling #841 reads the GPX in
distance sampling, such a video gets as far as the GPX instead of failing before it is read.
On top of #839 and #841
This was rebased once both had landed, rather than having them rebase onto it:
MapillaryVideoError, which Fail the sample instead of silently producing no frames #839's sample summary reports withthe other sampling errors.
probe_video_start_time(), which this PRremoves. It now covers interval sampling through
_extract_video_start_time(), and distancesampling through a track without a GPS clock, which is the only case in which distance
sampling looks the start time up.
--video_start_timeplaces a GPX on a video without GPS gavecreation_time - durationas the reason. It now says why that still holds: such a video wouldget its start time from its creation time, which cameras stamp at the start or the end of the
recording, some in local time.
Verification
RICOH/RICOH THETA Z1, which is not on theend-stamping list, and no date in its name) resolves to 08:47:32Z, the GPX start, and gets the
warning. That is the start time the first revision of this PR used, with which
video_process --geotag_source gpx --video_sample_distance -1 --video_sample_interval 2yielded 289 geotagged frames spanning 08:47:32 → 08:58:18, every one 0.0000 m from the GPX
position at its true video time. Before: 0 frames, all rejected as outside the track.
11 sample videos from 10 cameras:
main. Reading everycreation time as the start had dropped them to 1 and 0.
and after;
--video_start_timeis the way out there.ruff check,ruff format --check,usort diffandmypy(for Python 3.9, as CI checks) all pass.Not addressed here
process --video_geotag_source gpxstill places a GPX at video time 0 on a video without GPS,so a GPX covering several video files is injected whole into each of them (Honour --geotag_source in distance-based sampling #841 lets
video_processplace it by--video_start_timeinstead). That is the documented limitationunder "Limitations of
--video_geotag_source" and is a separate change.