AI Video to Frame Sequence for Scroll Animation

Scroll-driven image sequences are the effect where a product rotates, a liquid pours or a scene unfolds as the visitor scrolls, frame by frame, in perfect sync with the page. AI video makes the footage cheap to produce: a 5 or 10 second Runway clip is all you need. The hard part is everything after generation, turning video into frames, keeping file sizes sane and playing them smoothly. This guide covers the whole path, from export to the few lines of logic that tie frames to scroll.

Plan the clip for scrolling

A clip made for scrolling is different from a clip made for watching. The visitor controls time, so the footage must make sense played slowly, quickly or backwards.

• One continuous move. A rotation, a camera push or a material flowing. Cuts look broken when scrubbed.
• Locked-off or very smooth camera. Shaky movement feels worse under scroll control.
• A clear first and last frame. The animation rests on these at the start and end of the section, so both should look finished. See first and last frame product animation.
• Short is fine. Five seconds at 24 frames per second is 120 frames, plenty for a full-screen section.

Abstract material clips work especially well here; the material animation guide has prompts.

Getting frames out of Runway

There are two routes.

1. PNG sequence export. Runway's help centre says that when generating with Gen-4.5 in Tool Mode, or editing with Aleph 2.0 in Edit Studio, you can choose MP4, ProRes or PNG Sequence as the output format. The PNG sequence arrives as a .zip with every frame as a separate PNG, with no compression loss. The option is chosen at generation time, adds 5 credits per second to the cost, and is currently limited to Max, Unlimited (Legacy) and Enterprise plans. Workflows and Agent generations deliver MP4 only.
2. Extract from the MP4. On any plan, download the MP4 and extract frames yourself with ffmpeg, a free command-line tool. The MP4 is compressed, so frames carry some compression, but at web sizes it is rarely visible.

Runway's video models output 24 frames per second (Gen-4.5 also offers 25), at 720p-class sizes such as 1280 x 720 for 16:9. That sets the ceiling: for a larger display, upscale the video before extracting, not the frames after.

Extracting and resizing frames with ffmpeg

These commands assume the clip is called clip.mp4 and a folder called frames exists.

ffmpeg -i clip.mp4 -vf "fps=24,scale=1280:-2" frames/f_%04d.png

This writes every frame at 1280 pixels wide as numbered PNGs (f_0001.png, f_0002.png…). The fps filter sets how many frames per second to keep; lower it to thin the sequence:

ffmpeg -i clip.mp4 -vf "fps=12,scale=1280:-2" -c:v libwebp -quality 75 frames/f_%04d.webp

This keeps 12 frames per second, half the frames, and writes compressed WebP directly. For scroll animation you rarely need every frame: the visitor scrolls in jumps, and 60 to 150 frames over a section usually look smooth.

Frame budgets

Do the arithmetic before you design: 120 frames at 60 KB each is about 7 MB, which is heavy on a phone. Serve a lighter set to small screens (fewer frames, narrower width) and consider a static image for visitors who prefer reduced motion.

Playing frames on scroll

The most robust technique is a canvas. The page has a tall section (say three screens high) with a sticky canvas inside it. As the visitor scrolls through the section, you work out how far they are, from 0 to 1, multiply by the number of frames, and draw that frame:

1. Preload the images into an array, showing the first frame as soon as it arrives.
2. On scroll, compute progress = how far the section has scrolled ÷ (section height − viewport height), clamped between 0 and 1.
3. Pick frame = round(progress × (frame count − 1)).
4. Draw it on the canvas inside requestAnimationFrame, and only when the frame number changes.

Two alternatives exist. You can scrub a <video> element by setting its currentTime on scroll, which needs only one file; it seeks smoothly only if the video is encoded with very frequent keyframes (for example -g 1 in ffmpeg), which makes the file much larger. And CSS scroll-driven animations (animation-timeline: scroll() or view()) can drive simple effects without JavaScript; browser support has been growing, so check current support before relying on it for the whole effect.

Checks before launch

• Scroll quickly up and down: frames should never flash blank. If they do, delay the effect until enough frames have loaded.
• Test on a mid-range phone, not only a laptop.
• Respect reduced-motion preferences with a static frame.
• Keep the source clip, prompt, seed and model with the frames, so the sequence can be regenerated at a new size later.

The Gen-4 image to video guide covers making the clip itself.

Try it

Locked-off camera as the matte black headphones rotate slowly clockwise a quarter turn on a pale grey background, ending with the ear cup facing the camera. Smooth, even motion, soft studio light.

Generate it at 5 seconds from a clean product still, extract at 12 frames per second to get 60 frames, and wire them to a section two to three screens tall. If the rotation feels too fast under scroll, make the section taller rather than adding frames.