Skip to content

fix(core): scrubbing back across a scene no longer leaves its content hidden - #5766

Merged
miguel-heygen merged 3 commits into
mainfrom
fix/scrub-holds-last-good-frame
Oct 17, 2026
Merged

miguel-heygen merged 3 commits into
mainfrom
fix/scrub-holds-last-good-frame

Conversation

@miguel-heygen

@miguel-heygen miguel-heygen commented Oct 17, 2026 •

Copy link
Copy Markdown
Collaborator

What

Scrubbing the Studio preview past a scene, back before it, and then forward into it again left that scene's background on screen with its content hidden. It stayed that way until the scene's next step. The rendered export was always right. With this change the preview matches the export on every scrub path.

Why

To reproduce: take a sub-composition whose script reveals its content with two steps at its own time 0, such as tl.set(field, { visibility: "hidden" }, 0) followed by tl.set(field, { visibility: "visible" }, 0) (common when a scene builds clones at load and shows them from its first frame). Seek past the scene, seek back to just before its start, then scrub forward into it. Before this change the field stays hidden until the scene's next visibility step. On the test fixture below that is about 1.2 s of an empty dark frame.

Cause: on every seek the runtime also seeks each registered scene timeline to its own time (seekStandaloneRegisteredTimelines). A scene that has not started yet is held at local time 0. When GSAP lands a timeline exactly on a time coming from a later time, it applies the zero-duration steps at that time in reverse authored order and marks them done. So "hidden, then visible" ends hidden, and the next forward move never applies them again. Plain GSAP 3.14.2 and 3.15.0 do this:

tl.totalTime(1); tl.totalTime(0);   // field: hidden (steps at 0 applied in reverse)
tl.totalTime(0.5);                  // field: still hidden

The same happens at any later time a scene lands on exactly from above, e.g. two steps on a whole second. The root timeline already re-renders after every eventful seek, from just below the playhead, except at 0, where it steps down from above and so applies the steps at 0 in reverse too.

Related work

Refs #5614, which made seekGsapTimeline the path every runtime GSAP seek takes and added the never-rendered redraw this extends.

How

One function owns the repair, reapplyStepsInAuthoredOrder: it takes the zero-duration steps sitting exactly on the landing time from the cached step index (landingMarksOf's structure-keyed index from #5674), renders each in authored order, and restores GSAP's own bookkeeping on it (ratio, _zTime, _act), so callbacks fire exactly as often as in plain GSAP. Two callers:

  • seekGsapTimeline, when a timeline lands backward (a scene parked at 0, or scrubbed back onto a step). Nothing else is redrawn, so a backward seek costs what it did on main, plus a binary search.
  • rerenderGsapTimelineAt at 0, after its existing step down from above and its keyframe priming, which stay as they were.

Both seek paths in init.ts now pass the same structure key (gsapStructureOf). Render capture keeps passing none, as before.

Test plan

New cases in packages/core/src/runtime/adapters/gsap.steps.test.ts, run with real GSAP:

  • a scene's steps at 0 after the playhead comes back to 0 from later (the empty dark frame)
  • a scene nested in a root timeline, parked at 0 by the same seek order the runtime uses
  • two steps on one later time, after the playhead comes back onto it
  • the root's re-render at 0 keeps them
  • keyframed tweens starting at 0 (array and percent forms) match plain GSAP across five root and scene seek orders
  • a call on the landing time fires as often as in plain GSAP across back-and-forth seeks
  • a backward landing never redraws the whole scene

Against main's gsap.ts and init.ts, the four symptom tests fail (exit 1, 4 failed). On this branch: 74 passed, green on every run. Mutations: removing the re-application on backward landings turns 3 tests red, removing it at 0 turns 1 red, a full redraw on backward landings turns 1 red, and dropping the bookkeeping restore turns the callback test red (7 calls vs 5).

packages/core test:hyperframe-runtime-ci (typecheck, build, contract, behavior, seek, parity, security, coverage): exit 0, 83 files, 2104 passed. packages/producer firstFrameStartingState.integration.test.ts (real render): exit 0, 2 passed. Studio edit accuracy gate in CI: 271 accurate, same as main.

What I measured

Fixture: a 12 s, four-scene composition with generated images. The third scene clones a 3x3 field of windows full of pictures at load and reveals it with steps at 0, and the second ends in a zoom push with motion blur. Driven through the published Studio's own timeline with a real mouse drag (park at 8 s, drag from 4.5 s to 8 s), headless Chrome, GPU, device scale 2, recording the compositor's frames:

runtime frames recorded frames showing the empty dark scene
main 185 60
this branch 176 0

The same back-then-forward seek order on Linux (headless Chrome 152): main shows the empty scene from 6.1 s and no new frame through 7.0 s, while the fix shows the field at every step.

Cost in Chrome on the same fixture, main vs this branch, three alternating rounds. Per-seek main-thread time over forward, backward and forward sweeps at 0.1 s steps, then 5 s of playback:

seek p50 seek p95 backward-sweep seek p95 playback fps frame gap p95
main 0.3-0.4 ms 0.5-0.6 ms 0.4-0.5 ms 60.2 16.9-17.8 ms
this branch 0.3 ms 0.5 ms 0.4-0.6 ms 60.2 16.9-18.4 ms

On a 38,000-step timeline (the long-caption case from #5674), a backward root seek costs a median 12.07 ms on this branch vs 11.89 ms on main over 8 alternating rounds, and a backward scene seek costs 1.5-2.6 ms on both.

Before / After

Studio preview during the drag, same times. Before: the third scene shows only its background. After: its window field.

Before and after: Studio preview while scrubbing forward after coming back

The frame that looks low-res during the second scene's exit is authored motion blur, and the export has it too. Studio preview with this change next to the rendered export:

Studio preview with the fix next to the rendered export

What I did NOT exercise

  • The fix in an embedding host's preview after a release: it reaches embedders when they pick up the next core release.
  • Real-GPU Linux: the Linux run used headless Chrome without a GPU, and ran on the first version of this fix, not this head.
  • The edit accuracy suite locally: it passed in CI at this change.
  • The screenshots below come from the first version of this fix. I re-ran the drag at this head (0 empty frames) and the frames match them.

@github-actions

github-actions Bot commented Oct 17, 2026 •

Copy link
Copy Markdown
Contributor

Edit accuracy: accurate 271 (base branch 271), smooth 177 of those

The gate passes.
Smoothness is reported in the artifact, not gated. A case fails only if it fails 2 of 3 runs.

Quarantined, measured but not gated (0)

Unstable (1)

  • seqfastkeys-none-px-r0-root-z100: tracking 7.76, pressJump 0.05, drop 0, reload 0, render 0.02, renderKey -, undo true, teleport true / tracking 0.06, pressJump 0.05, drop 0, reload 0, render 0.02, renderKey -, undo true, teleport true / tracking 0.06, pressJump 0.05, drop 0, reload 0, render 0.02, renderKey -, undo true, teleport true

@jrusso1020 jrusso1020 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requesting changes at e38913f9 because the required Studio: edit accuracy gate fails. It failed on the first run (39011185991) and again on the rerun (39011237683). The gate counts a case as failing only when it fails 2 of 3 runs, so this isn't flakiness.

The accurate count is 255, down from 271 on main. Sixteen cases that pass on main fail here:

  • move: fromto-…-mid, keys-…-on, keys-…-mid, keys-…-after
  • nudge: fromto-…-mid, keys-…-on, keys-…-mid, keys-…-after
  • resize: fromto-…-mid, scale-…-mid, size-…-mid, keys-…-on, keys-…-mid, keys-…-after, keys-px-r0-nested-z100-on
  • rotate: spin-…-mid

(… = px-r0-root-z100; the full list is in the sticky edit-accuracy comment.)

Every one of them is a px-r0 case, and all but one are root-level. That points at the seek path this PR changes, not at one edit kind. I haven't reproduced it locally, so these two suspects are unconfirmed:

  1. seekGsapTimeline now redraws whenever the seek lands behind from. The editor seeks repeatedly while it measures an edit at a fixed time. If one of those seeks counts as "backward", the redraw could repaint the element at a state other than the one the edit gate reads back.
  2. renderOntoInAuthoredOrder now calls timeline.render(t, true, true) for t < 1 ms. With suppressEvents and force both set, the render can produce a different start state than stepping up from below did, for tweens whose from values the editor has just rewritten.

The code itself reads correctly to me, so this is the only blocker. Please run the edit-accuracy suite locally on one of the keys-…-mid cases and diff the measured geometry against main. Then either fix the seek path or quarantine the cases with a stated reason. I'll re-review as soon as the gate is green.

— Rames

@miguel-heygen

Copy link
Copy Markdown
Collaborator Author

Thanks for the precise list. Both suspects were real, and the head now removes both code paths instead of tuning them.

  1. The redraw on every backward seek is gone. A backward landing now only re-applies the zero-duration steps sitting exactly on the landing time, in authored order, found through the cached step index from perf(core): preview runtime scales with what plays, not with the film's clip count #5674. GSAP's own bookkeeping on each step is restored afterwards, so callbacks fire exactly as often as in plain GSAP (covered by a test). Nothing else is redrawn, so the editor's repeated seeks at a fixed time leave every tween as main does.
  2. The forced render(t, true, true) below 1 ms is gone too, along with the render member it needed on the timeline type. At 0 the re-render steps down from above exactly as on main, then re-applies the steps at 0 in order.

The edit accuracy gate at ec03c777 reports 271 accurate, the same as main, with no case quarantined. ebc5d810 on top fixes a type error that CI's stricter runtime typecheck caught. I didn't run the suite locally; CI's run is the evidence. The PR body has the new design, tests, mutations and numbers.

@jrusso1020 jrusso1020 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving at ebc5d810. This clears my change request from e38913f9: both seek-path changes I suspected for the edit-accuracy drop are gone.

  • No whole-timeline redraw on a backward seek. seekGsapTimeline now re-applies only the steps placed exactly at the landing time (stepsAt, within float noise), in authored order. Each one is rendered at its start silently, then its ratio / _zTime / _act are restored, so GSAP's own bookkeeping for the next seek is unchanged. Landings between steps cost nothing extra, and the new test never redraws a whole scene for a backward landing covers that.
  • No forced timeline.render(t, true, true) at 0. renderOntoInAuthoredOrder goes back to stepping up from just above 0, then re-applies the steps at 0 in order. The render binding is removed from every caller.
  • The new structure key comes from gsapStructureOf, which returns undefined once render capture starts. So the extra work is preview-only, and export seeks take the same path as before.

The tests compare against plain GSAP, which is the right oracle here: keyframes arrays and percent keyframes match plain GSAP at 0 across the root and scene seek sequences, and calls on the landing time fire exactly as often as plain GSAP across back-and-forth seeks. Required CI is green at this head, including Studio: edit accuracy gate.

No live run.

— Rames

@miguel-heygen
miguel-heygen added this pull request to the merge queue Oct 17, 2026
Merged via the queue into main with commit 4cf0e3a Oct 17, 2026
122 checks passed
@miguel-heygen
miguel-heygen deleted the fix/scrub-holds-last-good-frame branch October 17, 2026 09:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants