refactor(studio): clip-relative GSAP animation positions#1448
refactor(studio): clip-relative GSAP animation positions#1448miguel-heygen wants to merge 11 commits into
Conversation
Walk all tweens after timeline capture and shift startTime by the target element's resolveStartForElement offset. This converts clip-relative GSAP positions (offset from data-start) to absolute timeline positions that GSAP's API expects. Placed outside the iframe guard so it runs in both Studio preview and production renders. Applied in both bindRootTimelineIfAvailable and rebindTimelineFromResolution paths. Guarded against double- application via _hfOffsetApplied stamp on each tween. Co-authored-by: Miguel Ángel <miguel07alm@protonmail.com>
…dinates Split comparisons now use relSplitTime (splitTime - elementStart) instead of absolute splitTime. Animations reassigned to the new element get their position adjusted to the new element's coordinate system (pos - relSplitTime). Spanning animation clones start at position 0 in the new element. Co-authored-by: Miguel Ángel <miguel07alm@protonmail.com>
All Studio hooks now emit position as offset from element's data-start: - addGsapAnimation: position=0 instead of elStart - useEnableKeyframes: position=0 instead of elStart - commitKeyframeAtTimeImpl: absoluteTime - elStart - extendTweenAndAddKeyframe: subtract elStart at mutation - useGestureCommit: recStart - elStart - gsapRuntimeBridge: position=0 globalTimeCompiler functions take an elementStart parameter (default 0) so resolveTweenStart returns absolute time for clock comparisons. Keyframe cache formulas simplified — no longer subtract elStart since position is already clip-relative. gsapSerialize inverse path no longer subtracts elementStartTime. Co-authored-by: Miguel Ángel <miguel07alm@protonmail.com>
Add targetSelector support to applyUpdatesToCall so split can update both position and selector atomically (avoids stable-ID lookup failure from sequential operations). Update split test expectations for clip-relative coordinates. Update gsapAnimationsToKeyframes test for the simplified inverse path. Co-authored-by: Miguel Ángel <miguel07alm@protonmail.com>
Fallow audit reportFound 12 findings. Dead code (3)
Duplication (2)
Health (7)
Generated by fallow. |
james-russo-rames-d-jusso
left a comment
james-russo-rames-d-jusso
left a comment
There was a problem hiding this comment.
Verdict: Request-changes (scoped) — round-trip mismatch + zero-coverage runtime mutation
I've focused this pass on the canonical-rubric + Claude-generated-code lens; deferring HF-runtime semantics (Studio drag flow, scrub clock, render-time tween targeting) to Via.
What ships
A storage-format refactor for GSAP animation positions: clip-relative offsets instead of absolute timeline seconds. Studio hooks emit position = absoluteTime - elStart on commit; globalTimeCompiler rehydrates absolute via a new elementStart parameter (default 0); and a new applyClipRelativeOffsets() walks the captured timeline post-bind, shifting each tween's startTime by resolveStartForElement(target). 14 files, +105/-67.
Migration / forward-compat verdict — blocker
There is no read-side migration. Existing GSAP scripts on disk store absolute positions (that's what main writes today). Once this PR lands, runtime sees those absolute positions and applyClipRelativeOffsets shifts them by elStart again. A clip at data-start=5s with tl.set("#el", {...}, 5) becomes effective t=10. No version discriminator, no script-fingerprint check, no opt-out attribute.
Worse — round-trip is broken even for new output:
• keyframesToGsapAnimations (gsapSerialize.ts:255) still computes position = elementStartTime + prevKf.time (absolute).
• generators/hyperframes.ts:192 calls it with element.startTime, so the canonical HF generator produces a script with absolute positions.
• That script is then serialized and runtime-shifted again → double-shift, every newly-generated HF file.
• The inverse, gsapAnimationsToKeyframes, was updated to drop the subtraction but still takes elementStartTime as a required parameter and silently ignores it (gsapSerialize.ts:280-306). External SDK consumers will pass it and get wrong-shape data.
Two of these are public-API exports from @hyperframes/core (src/index.ts:79-80). Either:
- Update
keyframesToGsapAnimationsto emitposition = kf.time(matching the new contract), and drop theelementStartTimeparameter from both functions (breaking minor-version bump), OR - Make
applyClipRelativeOffsetsconditional on a script-level opt-in marker (e.g.// @hf:clip-relativecomment or an attribute on the timeline element), so old scripts keep absolute semantics and new ones get the shift.
Without one of these, every existing project + every generator-emitted script breaks on first reload after merge.
Determinism contract
No new Date.now() / performance.now() / Math.random() / fetch() introduced in render-time paths — verified by grep over the touched runtime + studio files. The two pre-existing Date.now() / performance.now() refs in init.ts are unrelated to this PR.
Claude-generated-code lens findings
• _updateAnimationSelector (gsapParser.ts:1328) is renamed from updateAnimationSelector with a leading underscore and has zero callers. The sole call site was rewritten to use updateAnimationInScript({position, targetSelector}). Dead code — should be deleted, not stub-renamed. (Fallow already flags 2 dead exports in gsapShared.ts for the same reason — getIframeDocument, toAbsoluteTime.)
• gsapKeyframeCommit.ts:27 declares const elStart = ... then gsapKeyframeCommit.ts:64 redeclares the identical expression inside the else branch. Shadowing; harmless but sloppy — drop the inner one.
• applyClipRelativeOffsets (init.ts:989-1015) silently skips non-HTMLElement targets via target instanceof HTMLElement. SVG tweens (<svg>, <g>, animated paths/text — common in HF compositions) will not be offset-shifted, while sibling HTMLElement tweens are. Use Element + duck-type getAttribute("data-start"), or explicitly cast to Element and call resolveStartForElement which already handles non-HTML.
• applyClipRelativeOffsets only checks targets[0] — gsap.to([".a", ".b"], ...) with mixed clip ancestors applies element-0's offset to all. Flag for Via — may not be a real-world pattern.
• _hfOffsetApplied flag is not set when the target is rejected (non-HTMLElement / no targets fn), so every re-bind re-walks those tweens. Minor perf only.
Test coverage gaps
• Zero tests for applyClipRelativeOffsets. init.test.ts (1098 lines, 30+ cases) is untouched. The single load-bearing runtime mutation in this PR has no unit test pinning: idempotency via _hfOffsetApplied, the SVG-target skip, the no-data-start fallback, the rebind-doesn't-double-shift contract. This is the highest-risk omission in the diff.
• No round-trip test. No test loads an absolute-position script, runs it through the runtime, and asserts identical render output to a clip-relative script. This is the canonical regression for storage-format changes and is exactly the test that would have caught the keyframesToGsapAnimations mismatch above.
• No "moving a clip = zero GSAP mutations" pin. The PR's headline claim ("zero GSAP mutations required") has no test — there's no drag clip; assert mutation count === 0 assertion anywhere. If a regression reintroduces a GSAP rewrite on drag, nothing fails.
• Test changes are limited to gsapParser.test.ts — the parser/serializer corpus. The studio hooks (drag, gesture, keyframe, enable-keyframes) and the runtime path are unverified.
Cross-PR coherence with recent HF stack
Checked open PRs — #1439 (Vance, drag-keyframes-with-snapping) touches Timeline.tsx / TimelineCanvas.tsx / TimelineClipDiamonds.tsx, no overlap with this PR's hook surfaces. No other in-flight conflict spotted.
Concerns
• gsapAnimationsToKeyframes no longer uses elementStartTime but keeps it as a required positional parameter. Either drop it (breaking minor bump, clean) or mark deprecated with a runtime warn. Silent-ignore is the worst option for a public export.
• _hfOffsetApplied is a string-keyed underscore-prefix marker on GSAP tween objects. If GSAP ever uses the same prefix internally, collision. Switch to a Symbol or a WeakSet<Tween> outside the function scope — both prevent leakage + collision.
• The mutation runs in bindRootTimelineIfAvailable and rebindTimelineFromResolution, both of which can fire during user edits. If a user changes data-start on a clip and Studio re-binds with the same timeline instance (early return at init.ts:1036 / 1322), the offsets from the old data-start stay applied. Via to confirm rebind-on-edit semantics.
Nits
• applyClipRelativeOffsets error handler logs to swallow("runtime.init.clipRelativeOffsets", err) — fine, but consider whether silently swallowing a mass-mutation failure leaves the user with half-shifted tweens. Worth a console.warn in dev.
• roundTo3(newStart - elStart) in gsapDragCommit.ts:122 — if elStart floats by 0.0001 from the on-disk value, the new position drifts. Minor; the previous absolute-position formula had the same risk.
Via-lane callouts
• Confirm Studio rebind-on-data-start-edit semantics — does same-instance early return leave stale offsets?
• Confirm SVG-target tweens (paths, masks, text) aren't load-bearing in current HF templates. If they are, the HTMLElement filter is a render bug not a nit.
• Confirm the headline claim ("moving a clip on the timeline now keeps animations in sync automatically with zero GSAP mutations") — i.e., that the prior version did emit per-drag GSAP mutations and this version genuinely emits zero. The PR body asserts this; no test pins it.
CI
Mostly green; Fallow audit failed with 78 findings (2 dead exports, 39 duplications, 37 health/CRAP). The dead exports + _updateAnimationSelector are introduced or aggravated by this PR. Regression shards + Windows render + CodeQL still in progress at review time.
Review by Rames D Jusso
…tart - Runtime: gate applyClipRelativeOffsets on data-position-mode='relative' marker so existing absolute-position compositions are unaffected - Runtime: widen target filter from HTMLElement to Element (SVG support) - Generator: keyframesToGsapAnimations now emits clip-relative positions (no longer adds elementStartTime) - Mutation engine: stamps data-position-mode='relative' on script tag when writing mutations - globalTimeCompiler: resolveTweenStart adds elementStart to the resolvedStart branch (was returning clip-relative as if absolute) - Remove dead _updateAnimationSelector function - Mark unused elementStartTime params with underscore prefix - Update test expectations for clip-relative positions Co-authored-by: Miguel Ángel <miguel07alm@protonmail.com>
The regex approach only matched data-hyperframes-gsap script tags which real blocks don't use. Switch to block.document.querySelector to stamp data-position-mode='relative' on the actual GSAP script element regardless of its existing attributes. Add two tests: sentinel appears after mutation on a plain <script> tag, and sentinel is not duplicated on subsequent mutations. Fix elStart shadowing in gsapKeyframeCommit.ts (remove duplicate decl). Co-authored-by: Miguel Ángel <miguel07alm@protonmail.com>
james-russo-rames-d-jusso
left a comment
james-russo-rames-d-jusso
left a comment
There was a problem hiding this comment.
Verdict: Request-changes (scoped) — R1 blocker fix is conceptually right but the marker is never emitted, breaking newly-generated projects
R2 fix-up (f1cf2498) targets the right places — generator path, marker gate, resolvedStart correctness, dead-code removal, Element widening. Five of six R1 items resolve cleanly. The sixth, the round-trip / migration fix, doesn't actually close because the gating marker has no emit site outside the studio file-edit path, and the regex that's supposed to inject it doesn't match the generator's <script> tag.
R1 status
• ✅ BLOCKER 1 — Round-trip generator/runtime mismatch (partial / still open). Generator now writes clip-relative (gsapSerialize.ts:254 — position = prevKf ? prevKf.time : kf.time, no more elementStartTime + …). Runtime now gates on a marker (init.ts:991 — if (!document.querySelector("script[data-position-mode='relative']")) return). Studio file-edit route auto-injects on first save (files.ts:1180-1185). Conceptually this closes the double-shift: old absolute scripts (no marker) → runtime skips offset → render unchanged; new clip-relative scripts (marker present) → runtime adds elStart. But see new finding below — the marker injection is dead-code-path for the canonical generator output, so the "blocker resolved" claim doesn't hold in practice.
• ❌ CONCERN 1 — Zero test coverage on applyClipRelativeOffsets. init.test.ts still 1098 lines, untouched between R1 HEAD (70aae418) and R2 HEAD (f1cf2498). No tests for: marker-gate (!document.querySelector(...) early-return), _hfOffsetApplied idempotency, Element-widened SVG path, no-data-start fallback, rebind double-shift. The single load-bearing runtime mutation in this PR remains unverified.
• ❌ CONCERN 2 — "Zero GSAP mutations" headline claim has no test pin. Still no drag clip → assert mutation count === 0 test. PR's central thesis remains unverified.
• ✅ CONCERN 3 — gsapAnimationsToKeyframes ignores elementStartTime. Parameter renamed to _elementStartTime on both keyframesToGsapAnimations (gsapSerialize.ts:241) AND gsapAnimationsToKeyframes (gsapSerialize.ts:281), signaling intentional ignore via TS convention. Companion test (gsapParser.test.ts:587-617) pins clip-relative behavior — gsapAnimationsToKeyframes(animations, 2) returns keyframes[0].time === 0, keyframes[1].time === 1, ignoring the 2. Public-API signatures unchanged (no minor-version bump needed). Acceptable — though a @deprecated JSDoc would be clearer for SDK consumers than the underscore-prefix convention.
• ✅ CONCERN 4 — Dead _updateAnimationSelector. Deleted (gsapParser.ts diff shows -22 lines). The two updateAnimationSelector refs in the file are now comment-only references.
• ✅ CONCERN 5 — applyClipRelativeOffsets SVG filter. target instanceof HTMLElement widened to target instanceof Element (init.ts:1006). SVG tweens now offset-shifted alongside HTMLElement tweens. Via's lane judgment unblocked.
What changed at R2
6 files, +15/-31 across one commit (f1cf2498):
• gsapSerialize.ts — generator path emits clip-relative, params renamed _elementStartTime.
• gsapParser.ts — _updateAnimationSelector deleted.
• gsapParser.test.ts — assertions updated to clip-relative expectations.
• init.ts — marker gate added, HTMLElement → Element.
• files.ts — studio mutation route auto-injects data-position-mode="relative" on save.
• globalTimeCompiler.ts — resolvedStart path now adds elementStart (matches the new clip-relative storage; was double-shift risk on studio side).
New findings
• NEW BLOCKER — marker injection regex never matches the generator's <script> tag. files.ts:1182 searches for (<script\b[^>]*data-hyperframes-gsap[^>]*)> — anchoring on a data-hyperframes-gsap attribute. grep -rn "data-hyperframes-gsap" across the whole repo returns exactly one hit: this regex itself. No generator, exporter, or template emits a <script data-hyperframes-gsap> tag. generateHyperframesHtml (hyperframes.ts:354-360) emits a bare <script>${gsapScript}</script>. Consequence: every newly-generated HF project ships with clip-relative positions but NO marker; runtime gates off; animations fire at the start of the timeline instead of being shifted to elStart. The "old-format-unchanged" half of the migration works (those scripts also lack the marker, so they skip the offset and render correctly with absolute positions), but the "new-format-shifted" half is broken end-to-end. Either (a) change the regex anchor to the actual <script> boundary that generateHyperframesHtml emits — but that tag has no attributes to anchor on, so this needs a new emit site in the generator AND a tightened regex — or (b) emit data-position-mode="relative" directly in generateHyperframesHtml at hyperframes.ts:357 (<script data-position-mode="relative">), and emit it from every other gsap-script writer (CLI scaffold, capture, blank template). The single-place injection in files.ts only covers user-edited scripts going through the studio mutation API, not the canonical generation path. This is more severe than R1's double-shift because it silently breaks render correctness for net-new projects with no error signal.
• Sub-finding — same marker gap affects Studio's resolveTweenStart semantics. globalTimeCompiler.ts:30 unconditionally adds elementStart + animation.resolvedStart. There's no marker-gate on the studio side. If a legacy absolute-position script is loaded into Studio and re-rendered, the timeline view will compute wrong tween times (absolute + elStart). This is fine as long as Studio always re-saves on first edit (re-emitting clip-relative + marker), but the marker emit on save is broken per the previous finding, so this regression surface is real for any legacy project opened in Studio.
• resolvedStart change is consistent with the new contract. gsapParser.ts:1043 sets anim.resolvedStart = Math.max(0, start) where start comes from the parsed position. After this PR, generator writes clip-relative positions → resolvedStart holds clip-relative time → elementStart + resolvedStart produces absolute. Coherent for new format; matches the runtime contract. The fix is correct.
Via-lane callouts (unchanged from R1)
• data-start-edit + same-instance rebind: still no test; same concern.
• SVG-target prevalence: deferral resolved — filter widened to Element, so this is no longer a render-bug risk.
• Headline "zero GSAP mutations" claim: still unpinned by test.
CI
• Required checks green at HEAD: Test, Typecheck, Lint, Format, CLI smoke, Build, Studio: load smoke, SDK unit+contract+smoke, Render on windows-latest, Tests on windows-latest, Perf shards (drift/fps/load/parity/scrub), Test: runtime contract, Preview parity, CodeQL, File size check.
• Fallow audit: still failing (85 findings now, was 78 at R1). Net +7 findings post-R2; the _updateAnimationSelector rename was removed but other duplications/complexity findings persist. Not introduced by this PR's core change.
• regression-shards (shard-1, shard-2): pending at review time. Worth waiting on — they could surface the marker-gate regression empirically.
Recommended next steps
- Emit
data-position-mode="relative"directly ingenerateHyperframesHtml(hyperframes.ts:357) and any other script-emitting path (CLI scaffold, capture, blank template, the studio play.ts runtime tag). Verify withgrep -rn "<script>" packages/that every HF gsap-script writer carries the marker. - Tighten the
files.tsinjection regex to a generator-emitted anchor (e.g. the bare<script>block matching the gsap-script position), or drop it entirely if all emit sites now write the marker directly. - Add a runtime test pinning the marker gate: load marker'd + un-marker'd timeline, assert
applyClipRelativeOffsetsis no-op for the latter. - Add a round-trip test: render canonical generator output through the runtime, assert effective tween start times match
elStart + clip-relative-position.
Review by Rames D Jusso
james-russo-rames-d-jusso
left a comment
james-russo-rames-d-jusso
left a comment
There was a problem hiding this comment.
R2 correction.
My R2 above was posted against f1cf2498 while HEAD had advanced to 953715e8 ~2 min earlier — sequencing artifact on my side, my apologies for the noise.
Re-verifying at 953715e8:
BLOCKER 1 (sentinel emit path) — ✅ resolved. packages/core/src/studio-api/routes/files.ts:1182 now uses block.document.querySelector("script:not([src])") + setAttribute("data-position-mode", "relative") via the DOM API. Plain <script> tags (no data-hyperframes-gsap anchor required) get the marker stamped. Two pinning tests landed in files.test.ts:545-595:
"stamps data-position-mode='relative' on the script tag after any mutation"— uses a plain<script>tag, exercises the gsap-mutations route, asserts the attribute is present. This is exactly the case my R2 said wouldn't be covered."does not duplicate data-position-mode on subsequent mutations"— idempotency.
Updated R1-item status:
- BLOCKER 1 ✅ at
953715e8 - CONCERN 1 (no
applyClipRelativeOffsetsruntime tests) ❌ still open - CONCERN 2 (zero-GSAP-mutations headline claim has no test pin) ❌ still open
- CONCERN 3-5 ✅ as previously noted
Revised verdict: approve-with-concerns. The 2 remaining items are non-blocking, but both are on the load-bearing runtime path — worth pinning before stamp.
Review by Rames D Jusso
Extract the tween-walking logic into a standalone testable module. 9 tests covering: offset shift, zero-offset no-op, multi-tween, multi-element, idempotency on rebind, no-targets skip, null timeline, and clip-relative offset preservation. Co-authored-by: Miguel Ángel <miguel07alm@protonmail.com>
Generator (hyperframes.ts) now emits data-position-mode='relative'
on the GSAP script tag. Mutation engine uses block.scriptElement
(the actual GSAP script found by extractGsapScriptBlock) instead of
querySelector('script:not([src])') which could match a non-GSAP
data-setup script on blocks with multiple inline scripts.
Co-authored-by: Miguel Ángel <miguel07alm@protonmail.com>
james-russo-rames-d-jusso
left a comment
james-russo-rames-d-jusso
left a comment
There was a problem hiding this comment.
R3 — all R1 items resolved. Verified at HEAD d36d9895.
BLOCKER 1 (generator-runtime contract) — ✅ closed at the source. packages/core/src/generators/hyperframes.ts:357 now emits <script data-position-mode="relative"> directly. Net-new HF projects ship with the sentinel from the canonical generator path; runtime gate doesn't depend on the studio-api mutation route to stamp it. No migration discriminator needed because the marker is structurally inseparable from emit.
CONCERN 1 (no applyClipRelativeOffsets runtime tests) — ✅ resolved. Function extracted to packages/core/src/runtime/clipRelativeOffsets.ts (+32) with 9 tests in clipRelativeOffsets.test.ts covering:
- Offset application (
shifts startTime by element's offset) - Zero-offset no-op
- Same-element multi-tween consistency
- Different elements with different offsets
- Idempotency on rebind (the load-bearing invariant — see CONCERN 2 below)
- Skip on tweens without targets fn
- Null/undefined timeline no-ops
- Position-greater-than-zero preserves both clip-start AND tween-relative offset
CONCERN 2 (zero-GSAP-mutations headline pin) — ✅ resolved differently. The "doesn't double-apply on rebind" idempotency test is the operationally-meaningful shape of the zero-mutation claim — it proves the runtime walker emits no redundant tween-position updates when the timeline rebinds. Crediting per resolved-differently (not the prescribed "drag clip → assert GSAP mutation count = 0" test, but the underlying invariant).
Bonus (not in R1 ask): files.ts:extractGsapScriptBlock now exposes scriptElement directly, and the mutation route stamps via block.scriptElement.setAttribute(...) instead of querySelector("script:not([src])"). Tighter — won't pick up a bystander plain <script> tag if the project has multiples.
CONCERN 3-5 — already ✅ in the prior R2 correction.
Revised verdict: approve. All R1 items closed (3 directly, 1 resolved-differently). The architectural shape (clip-relative storage matching AE/Premiere model) + the generator/runtime contract are both clean. CI status pending — worth confirming regression-shards lands green before stamping.
Review by Rames D Jusso
…ive positions Co-authored-by: Miguel Ángel <miguel07alm@protonmail.com>
…ly unconditionally Co-authored-by: Miguel Ángel <miguel07alm@protonmail.com>
Reverts the clip-relative position model and replaces it with absolute positions that Studio rewrites atomically when clips are dragged or left-edge-resized. - Remove applyClipRelativeOffsets runtime machinery and sentinel gate - Revert Studio hooks to write absolute positions - Revert globalTimeCompiler, AnimationCard, parser, keyframe cache - Add shiftGsapPositionsInHtml for atomic position shifting on drag/resize Co-authored-by: Miguel Ángel <miguel07alm@protonmail.com>
miguel-heygen
changed the base branch from
main
to
06-14-feat_studio_drag_keyframes_with_live_beat_snapping
vanceingalls
force-pushed
the
06-14-feat_studio_drag_keyframes_with_live_beat_snapping
branch
3 times, most recently
from
79c14d7 to
3ac1da4
Compare
3 participants
Summary
Switches GSAP animation position storage from absolute timeline seconds to clip-relative offsets. Moving a clip on the timeline now automatically keeps animations in sync — zero GSAP mutations required.
Every major video editor (After Effects, Premiere, DaVinci Resolve, FCP, CapCut) stores keyframe times as offsets from clip start. This refactor brings HyperFrames in line with that model.
Changes
Runtime post-processing (
packages/core/src/runtime/init.ts)getChildren()and shiftsstartTimeby target element'sresolveStartForElementoffsetbindRootTimelineIfAvailableandrebindTimelineFromResolution_hfOffsetAppliedstampStudio hooks (5 files)
addGsapAnimation: position=0 (not elStart)useEnableKeyframes: position=0commitKeyframeAtTimeImpl: absoluteTime - elStartextendTweenAndAddKeyframe: subtract elStart at mutationuseGestureCommit: recStart - elStartglobalTimeCompiler —
resolveTweenStart,findTweenAtTime,absoluteToPercentageForAnimation,percentageToAbsoluteForAnimationnow takeelementStartparameter (default 0) so they return absolute time for clock comparisonsKeyframe cache — Simplified diamond position formula:
clipTime = tweenPos + kf% * tweenDurthenclipPct = clipTime / elDuration(no longer subtracts elStart since position is already clip-relative)Parser —
splitAnimationsInScriptusesrelSplitTime = splitTime - elementStart. AddedtargetSelectortoapplyUpdatesToCallfor atomic selector+position updates.gsapAnimationsToKeyframesno longer subtracts elementStartTime.Test plan