Skip to content

Sync a GPX to a video without a GPS clock at --video_duration_ratio - #856

Open
caglarpir wants to merge 3 commits into
mapillary:mainfrom
caglarpir:video-duration-ratio-for-gpx
Open

caglarpir wants to merge 3 commits into
mapillary:mainfrom
caglarpir:video-duration-ratio-for-gpx

Conversation

@caglarpir

Copy link
Copy Markdown
Contributor

Stacked on #852 and #853; only the last commit is new.

Problem

A timelapse packs several seconds of capture into each second of video. Without a GPS clock of its own, nothing in the video tells how many, and its GPX is laid out at 1x, so the frames get only the start of the track. An Insta360 ONE X timelapse stitched by Insta360 Studio (1.8 s of video, 110 s GPX) gives 1 frame with --interpolation_use_gpx_start_time, the option the current error suggests.

Changes

  • --video_duration_ratio (real seconds per video second) used to apply to interval sampling only, and was marked deprecated. It now also paces the GPX of a video without a GPS clock in distance sampling, however the GPX is placed: by --video_start_time, by --interpolation_use_gpx_start_time, or by assuming it starts with the video. A video's own GPS clock still wins.
  • The ratio is required where a track runs on long after the video and nothing else accounts for it:
    • Without a start time, a track more than twice as long as the video was already refused. It is now measured against the time the video spans at the ratio, and the error names the ratio.
    • With --interpolation_use_gpx_start_time, such a track is now refused too (MapillaryGPXVideoStartUnknownError), in interval sampling as well. The error suggests the ratio for a timelapse, or --video_start_time for a track recorded for more than this video.
    • With --video_start_time, nothing is required, so one GPX can serve many videos.
  • process is unchanged.

Behaviour change

--interpolation_use_gpx_start_time with a GPX more than twice as long as a video without a GPS clock used to be accepted, with the frames along the start of the track. It is now refused, and --video_start_time places such a video instead. The integration test for the README's example now uses a GPX that ends with the video, and a new one checks the refusal.

Verification

The ONE X timelapse with its GPX:

main this PR
distance, --interpolation_use_gpx_start_time 1 frame refused
same, --video_duration_ratio 60 1 frame 33 frames over 129 m
distance, --video_start_time, --video_duration_ratio 60 1 frame 33 frames over 129 m
interval, --interpolation_use_gpx_start_time 10 frames, 8 duplicates, over 1 m refused
interval, same, --video_duration_ratio 60 18 frames over 131 m the same

Normal-speed videos (GoPro HERO7 and HERO8, BlackVue, Insta360 Pro) give output identical to main. Unit: 932 passed, 18 skipped, 1 xfailed. Integration: 70 passed. Doctests, mypy, ruff check, ruff format --check and usort diff are clean.

With #851: VideoSyncOption keeps start_time and duration_ratio, and the interval check reads use_gpx_start_time from each GPX source's interpolation.

Interval sampling stamped each frame with the start time of the video plus
its time in the video. A timelapse packs several seconds of capture into
each second of video, so its frames were stamped that many times too close
together. Geotagged from a GPX by those stamps, they all landed near the
start of the track. Geotagged from the video's own track, they moved so far
per second that the speed check could reject them: a 30x timelapse of a
drive read as over 1,000 km/h.

Interval sampling now reads the rate from the video's GPS clock, as GPX sync
does, and stamps the frames at it. The start time is mapped back from the
first GPS timestamps at the same rate. --video_duration_ratio, which used to
apply to every video, now paces only the videos without a GPS clock, so that
a ratio given for those leaves the other videos in the batch alone.

The samples are then geotagged from the video's own track (the gopro, camm
and blackvue sources) by their time since the first sample, while the track
is timed in video seconds. It is now timed in GPS seconds, at the rate of its
clock. That also fixes distance samples of a timelapse, which were already
stamped with their capture times.
--interpolation_use_gpx_start_time places the GPX from the start of the
video instead of by the video's GPS clock. It skipped reading the clock
altogether, and with it the clock's rate, so a timelapse got its GPX laid
out at 1x: a 5 s, 30x timelapse got 157 s of GPX, and distance sampling
picked 5 frames along its first 12 m instead of 148 along the 1.4 km route.

The clock is now read in that mode too, and placed at the start of the GPX,
so that only its rate is used. A video that is not a timelapse has a rate of
1, and is placed as before.

The coverage check raises when the clock a track was placed by says that
the track misses the video. A clock placed at the start of the GPX says
nothing of the kind, so a track placed in that mode still only warns, as
it did when it had no clock.
A timelapse packs several seconds of capture into each second of video.
With a GPS clock of its own, a GPX is synced at the clock's rate. Without
one, nothing in the video tells the rate, and the GPX was laid out at 1x,
so the frames got only the start of the track: an Insta360 ONE X timelapse,
1.8 s of video with a 110 s GPX, gave 1 frame.

--video_duration_ratio, the seconds of real time per second of video, used
to apply to interval sampling only, and was marked deprecated. It now paces
the GPX of a video without a GPS clock in distance sampling too, whether the
GPX is placed by --video_start_time, by --interpolation_use_gpx_start_time
or by assuming that it starts with the video. The GPS clock of a video, if
it has one, still wins.

The ratio is required where a track runs on long after the video and
nothing else accounts for it. Without a start time, a track more than twice
as long as the video was already refused; it is now measured against the
time the video spans at the ratio, and the error names the ratio. With
--interpolation_use_gpx_start_time, such a track is now refused as well, in
interval sampling too, as it belongs to a timelapse or to more than this
video. With --video_start_time nothing is required, so that one GPX can
serve many videos, each placed by its start time.

process is unchanged.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant