Skip to content

Read a video's start time from its GPS clock, and tell whether creation_time marks the start or the end - #832

Merged
caglarpir merged 2 commits into
mapillary:mainfrom
caglarpir:fix-video-start-time
Oct 20, 2026
Merged

caglarpir merged 2 commits into
mapillary:mainfrom
caglarpir:fix-video-start-time

Conversation

@caglarpir

@caglarpir caglarpir commented Sep 18, 2026 •

Copy link
Copy Markdown
Contributor

The bug

A community report (Mapillary forum + FB group): a video recorded at 4.91 fps, paired with a GPX
of the same drive, comes out with its frames on the wrong part of the route. The same file + GPX
works in Google Street View.

The reporter described it as the frame rate being misread. It isn't — the container is parsed
correctly (mdhd timescale 15712, stts 3186 × 3200 → exactly 4.91 fps, and
mp4_sample_parser reproduces every composition time to the microsecond). What is wrong is
the video's start time, and since the frames are located along the GPX by their absolute
timestamps, getting that wrong moves all of them together.

Probe.extract_stream_start_time() returned creation_time - duration. For the reported file
(creation_time 2026-08-08T08:47:32Z, duration 648.879837s) that is 08:36:43.120Z, while the
GPX starts at 08:47:32Z — the creation time, to the second. Every sampled frame came out one
full video duration early, and the workflow the README documents fails outright:

$ mapillary_tools video_process VIDEO_DIR --geotag_source gpx \
      --geotag_source_path R0024681.gpx --video_sample_distance -1 --video_sample_interval 20
...
32 image read in total
        32 (81.0 MB) MapillaryOutsideGPXTrackError
MapillaryProcessError: Failed to process 32 files.

The hard failure is the lucky case. When the track does reach back far enough — a GPX covering
a drive that the camera split across several files — the frames from every file after the first
are silently placed one file-length back along the route. On this track that is up to ~1 km,
and the images end up bunched onto road they were not shot on, which is what the report
describes.

Where it came from

The subtraction was introduced in 644701c (Feb 2019). The comment it deleted says what it was
for:

# Blackvue actually reports endtime in the created_at exif field

It was a BlackVue workaround that later got generalised to every video. But neither reading is
right for every camera, and nothing in the file says which one applies. Checked against sample
videos, by comparing the creation time with the camera's own GPS clock or with the neighbouring
clips of the same drive:

  • End of the recording: BlackVue, Viofo, Vantrue, AZDome and most other dashcams, the
    Ricoh Theta X and the Labpano PanoX V2.
  • Start of the recording: Sony, Insta360, and the reported file. GoPro also writes the
    start, but in local time (Video Timestamping Bugs with GoPro MAX #819).

The fix

Resolve the start time in this order:

  1. --video_start_time, if given (unchanged).
  2. The video's own GPS clock, which is absolute UTC and so right for both BlackVue and
    GoPro: the median of the first 5 plausible GPS timestamps, each mapped back to the video's
    time 0. Timestamps before 2010 (a GoPro writes 2000-01-01 until its first fix) or more than
    a day in the future are skipped, and the median keeps a stale first fix (seen on the PanoX
    V2, seconds older than the rest) from shifting the result.
  3. The container's creation_time, read as the start or the end of the recording by
    looking for evidence:
    • a camera known to stamp the end: BlackVue, detected by the free/gps and free/cprt
      boxes it writes whether or not it ever got a fix, and the Theta X and PanoX V2, by their
      make and model tags;
    • otherwise a date and time in the file name (20230106_150558_00422F.MP4,
      2023_0725_141524_103.MP4, …) that matches only one of creation_time and
      creation_time - duration, allowing for a whole UTC offset because cameras name files in
      local time;
    • otherwise the start, with a warning that prints the --video_start_time value to pass if
      the camera stamps the end.

A start-time source that fails (telemetry that does not parse, a malformed creation_time)
now falls through to the next one with a warning instead of failing the video.

Behaviour changes

  • Start-stamping cameras without telemetry (the reported case, Sony, Insta360): frames are
    timestamped one duration later than before, which is when they were shot.
  • End-stamping cameras without telemetry: unchanged when the camera model or the file name
    identifies them. The others now get the start reading, with the warning, where every camera
    used to get the end reading silently.
  • GoPro / CAMM with interval sampling: frames are timestamped from the GPS clock. For the
    hero8.mp4 fixture that moves the first sample from 2019_11_18_15_41_12_354 (the camera's
    local RTC, minus the duration, labelled UTC) to 2019_11_18_23_42_08_539 (real UTC, matching
    the GPS positions written into the same frames). This is Video Timestamping Bugs with GoPro MAX #819 items 2 and 4.
  • BlackVue: GPS point times were measured from the first valid fix instead of from the
    start of the recording. Until the receiver gets a fix the camera logs NMEA lines without
    positions, so every point was late by the time to first fix. They are now measured from the
    first NMEA line, which changes BlackVue point times wherever they are used, not only the
    start time. For most videos the two coincide; on 433 BlackVue sample videos, 7 took more than
    5 s to get a fix, the slowest 60 s.
  • Distance-based sampling: frames at GPS points with an absolute timestamp still take their
    time from it. The start time is resolved, in the order above, only for frames without one, so
    a video without a usable creation time no longer fails when it does not need one. That includes
    every frame placed along a GPX, whose points carry their own time: since Honour --geotag_source in distance-based sampling #841 reads the GPX in
    distance sampling, such a video gets as far as the GPX instead of failing before it is read.

On top of #839 and #841

This was rebased once both had landed, rather than having them rebase onto it:

  • The start time raises a plain MapillaryVideoError, which Fail the sample instead of silently producing no frames #839's sample summary reports with
    the other sampling errors.
  • Fail the sample instead of silently producing no frames #839's test of an unreadable start time patched probe_video_start_time(), which this PR
    removes. It now covers interval sampling through _extract_video_start_time(), and distance
    sampling through a track without a GPS clock, which is the only case in which distance
    sampling looks the start time up.
  • Honour --geotag_source in distance-based sampling #841's comment on why only --video_start_time places a GPX on a video without GPS gave
    creation_time - duration as the reason. It now says why that still holds: such a video would
    get its start time from its creation time, which cameras stamp at the start or the end of the
    recording, some in local time.

Verification

  • The reported file (make/model RICOH / RICOH THETA Z1, which is not on the
    end-stamping list, and no date in its name) resolves to 08:47:32Z, the GPX start, and gets the
    warning. That is the start time the first revision of this PR used, with which
    video_process --geotag_source gpx --video_sample_distance -1 --video_sample_interval 2
    yielded 289 geotagged frames spanning 08:47:32 → 08:58:18, every one 0.0000 m from the GPX
    position at its true video time. Before: 0 frames, all rejected as outside the track.
  • Interval sampling with a GPX, built from the camera's own track where it has one, on
    11 sample videos from 10 cameras:
    • Viofo A129 and AZDome GS63H: 119 and 57 frames geotagged, as on main. Reading every
      creation time as the start had dropped them to 1 and 0.
    • Sony FDR-X3000 with its converted GPX: 0 → 129 frames geotagged.
    • BlackVue DR900M-2CH: frames move from 5.2 s late to on time.
    • The samples whose camera clock is in local time or simply wrong fail the same way before
      and after; --video_start_time is the way out there.
  • The full suite (946 passed, 18 skipped, 1 xfailed), ruff check, ruff format --check,
    usort diff and mypy (for Python 3.9, as CI checks) all pass.

Not addressed here

  • process --video_geotag_source gpx still places a GPX at video time 0 on a video without GPS,
    so a GPX covering several video files is injected whole into each of them (Honour --geotag_source in distance-based sampling #841 lets
    video_process place it by --video_start_time instead). That is the documented limitation
    under "Limitations of --video_geotag_source" and is a separate change.

@meta-cla meta-cla Bot added the cla signed label Sep 18, 2026
@caglarpir caglarpir changed the title Read a video's start time from its own clock, not creation time minus duration Read a video's start time from its GPS clock, and tell whether creation_time marks the start or the end Sep 23, 2026
… duration

A video's start time decides where its sampled frames land on a GPS track.
We derived it as creation_time - duration, which pushed every video one full
duration into the past unless its camera happened to stamp the creation time
at the end of the recording.

That subtraction came from 644701c (2019), whose removed comment says why:
"Blackvue actually reports endtime in the created_at exif field". The
workaround was later generalised to every video. ISO/IEC 14496-12 defines
creation_time as the creation time of the presentation, which for a recording
is the moment it started, and that is what cameras and ffmpeg write.

Take the start time from the video's own telemetry when it has some. That
clock is absolute UTC, so it is right for BlackVue, whose container time is
the end of the recording, and for GoPro, whose container time is in local
time (mapillary#819). Fall back to creation_time for the plain MP4s that get geotagged
from a GPX, which are exactly the videos that have no telemetry to sync
against.

Reported for a 10:49 dashcam video paired with a GPX starting at the video's
creation time: every sampled frame came out 648.88s early, so the documented
GPX workflow failed with MapillaryOutsideGPXTrackError on all of them. Where
the track does reach back far enough -- a GPX covering a recording split into
several files -- the frames were silently placed up to a kilometre off
instead. With this change all 289 sampled frames land on the track exactly.
The previous commit reads a video's start time from its telemetry and
otherwise takes creation_time as the start. That fixes the reported
dashcam, and Sony and Insta360, which also stamp the start. But it moves
every video from a camera that stamps the end one full duration late.
Those are most dashcams: Viofo, Vantrue, AZDome and others without GPS
telemetry, which people geotag from a GPX, plus the Ricoh Theta X and
the Labpano PanoX V2. With interval sampling and a GPX of their own
track, sample videos from a Viofo A129 and an AZDome GS63H went from 119
and 57 geotagged frames to 1 and 0.

Nothing in the metadata says which of the two a camera writes, so look
for evidence, in this order:

- a camera known to stamp the end: the Theta X and the PanoX V2 by their
  make and model tags, BlackVue by the boxes it writes its GPS log into,
  which it does whether or not it ever gets a fix
- a date and time in the file name that matches only one of the two,
  allowing for a whole UTC offset because cameras name files in local
  time
- otherwise the start, with a warning that prints the
  --video_start_time value to use if the camera stamps the end

Also:

- Take the median of the first 5 plausible GPS timestamps instead of the
  first one. The PanoX V2 can record a stale first fix seconds older
  than the rest, and a GPS clock that was never set (GoPro writes
  2000-01-01 until its first fix) must not become the start time. The
  hero8 fixture moves by 0.1s because the GPS5 samples of one payload
  share a timestamp.
- Fall back to the creation time when the telemetry fails to parse,
  instead of failing to sample the video.
- Measure BlackVue point times from the first NMEA line, which is the
  start of the recording, instead of from the first valid fix. Until the
  receiver gets a fix the camera logs lines without positions, so every
  point, and with the previous commit every frame sampled by interval,
  was late by the time to first fix. Across 433 BlackVue sample videos
  7 took more than 5s to get a fix, one 60s.
- Distance sampling resolves the start time only when a sampled frame
  has no absolute GPS time of its own, instead of requiring a creation
  time up front for every video.
- Ignore a malformed creation_time with a warning instead of raising.

On top of mapillary#839 and mapillary#841: the start time raises a plain
MapillaryVideoError, which the sample summary reports, as mapillary#839's other
sampling errors do. mapillary#839's test of an unreadable start time patched
probe_video_start_time(), which this removes; it now covers interval
sampling through _extract_video_start_time(), and distance sampling
through a track without a GPS clock, which is when distance sampling
looks the start time up. mapillary#841's comment on why only --video_start_time
places a GPX gave creation_time - duration as the reason, which no longer
holds; the reason now is that a video without a GPS clock would get its
start time from its creation time, which is stamped at the start or the
end of the recording, some in local time.
@caglarpir
caglarpir force-pushed the fix-video-start-time branch from 2fb148d to 2567ade Compare October 20, 2026 11:17
@caglarpir
caglarpir merged commit af91624 into mapillary:main Oct 20, 2026
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant