Apple researchers say FastFrames generates video in 36 milliseconds per frame
The 480p research models target video that responds as users direct it. The larger version needs eight B200 GPUs to reach its reported 45-millisecond latency.
Loading page…
The 480p research models target video that responds as users direct it. The larger version needs eight B200 GPUs to reach its reported 45-millisecond latency.
Listen to this story
Apple- and Cornell-affiliated researchers introduced FastFrames, two causal video models designed to respond frame by frame to changing inputs; the paper was submitted to arXiv on November 17, and Apple provides model and training checkpoints. At 480p, the 1.3B model reaches 36 milliseconds per frame on one NVIDIA B200, while the 14B model’s 45-millisecond result requires eight B200s. The work targets interactive generation, but its headline speeds depend on high-end NVIDIA hardware and are reported as steady-state measurements.
Apple’s model card breaks FastFrames-S’s 35.4-millisecond total into two generator evaluations (16.1 and 16.2 ms), KV refinement (2.2 ms), and pixel decoding (0.9 ms).
The team reports FastFrames-S preserves Causal Forcing++ visual quality while reducing latency from 272 to 36 milliseconds on one GPU.
FastFrames-L is listed at 158 milliseconds on one B200 GPU, compared with 45 milliseconds on eight.
AI-generated video can respond to new inputs in 36 milliseconds per frame, according to new research from Apple- and Cornell-affiliated authors. Their FastFrames paper, submitted to arXiv on November 17, introduces models designed for interactive visual experiences, reporting that speed at 480p on a single NVIDIA B200 GPU rather than a consumer device.
FastFrames generates video one frame after another, using new observations and inputs as it goes. That makes the research about responsiveness, not just finishing a prerecorded clip sooner. The authors demonstrate camera-directed and physics-directed generation, where the next image must reflect a changing instruction. Their two models have 1.3 billion and 14 billion parameters, respectively.
The researchers began by measuring where existing models spent their time during generation. More than half of end-to-end latency came from two operations: VAE decoding and updates to the key/value, or KV, cache. These were components often treated as fixed implementation details, the authors write. FastFrames instead replaces them with lighter alternatives and adds systems optimizations to diffusion sampling.
For FastFrames-S, the smaller model, the team keeps the existing two-step Causal Forcing++ generator frozen and trains the networks around it. The larger FastFrames-L starts from Wan2.1-14B, which the team distills into a two-step causal model before training its pixel decoder and KV refinement network. The approach therefore combines changes around an existing generator with a separately adapted larger model.
Apple’s model card shows how the smaller model’s remaining work is distributed. In an example at 480p with a full KV cache, its two generator evaluations take 16.1 and 16.2 milliseconds. KV refinement adds 2.2 milliseconds, and the pixel decoder adds 0.9 milliseconds. The listed total is 35.4 milliseconds, alongside the paper’s rounded 36-millisecond result.
The authors report 28 responses per second for the 1.3B model on one B200 GPU.
The authors report 22 responses per second for the 14B model on eight B200 GPUs.
The team says FastFrames-S preserves the visual quality of Causal Forcing++, while reducing latency from 272 to 36 milliseconds on one GPU. For FastFrames-L, the project page lists 158 milliseconds on one GPU and 45 milliseconds on eight. Its headline interactive result therefore depends on the larger hardware configuration. The page identifies these timings as steady-state measurements at 480p.
The authors describe the larger model as producing more realistic lighting and shadows, more coherent compositions and more legible text than the 1.3B models. Their project page also shows paired interactive demonstrations: keyboard navigation with LingBot and physics-based dragging with RealWonder. Each pair receives identical inputs at the same time. The team claims its optimized LingBot implementation is 16 times faster than the official version.
The official Apple model card includes both FastFrames models, the optimized LingBot version, and instructions for generation, latency profiling and VBench quality evaluation. Installation requires Linux on x86-64 hardware, an NVIDIA GPU and a CUDA 12.8-compatible driver. Its text-to-video example produces 832-by-480 video with 81 frames at a playback rate of 16 frames per second.
For developers studying how the larger model was trained, the release extends beyond its final generator. Apple lists checkpoints covering the following stages and components:
Reuse requires attention to the component licenses. FastFrames code uses the Apple Sample Code License, while LingBot-World v2 code and weights retain their upstream CC BY-NC-SA 4.0 license. For researchers seeking to check the speed and quality claims, the model card provides profiling commands and evaluation instructions alongside the checkpoints.
Loading discussion...
Join the conversation
Explain where faster interaction would justify the extra hardware.
Be the first to share a perspective or an experience.
Reader comments
Newest comments first. Replies stay oldest first.