Meta Introduces Live Video Avatars Synced to AI Speech

A portrait can now respond with expressions as Meta’s AI speaks; an illustrated character can gesture through the same live exchange. Meta introduced Muse Realtime Avatar on September 23, extending Muse Realtime Voice with video generated from reference media. The company says the system keeps speech and movement synchronized while delivering the first part of a response in under a second.

One stream drives the voice and the video

Muse Realtime Voice produces a stream of tokens that carry both the words and how they should sound. An audio decoder turns those tokens into speech. The avatar model uses the same stream, along with reference media, to generate the matching visual performance. That shared input is how Meta aims to keep lip motion and expressions aligned with the voice.

The system generates video in short chunks rather than making an entire scene in advance. It feeds recent generated frames back into the next chunk, giving the model a record of how the character has looked and moved. Meta says this helps an avatar retain its appearance and mannerisms across conversational turns. Its examples range from a photographic portrait to a full-body illustration, animals and everyday objects.

Cutting the work needed for each chunk

Generating each new piece of video quickly is a separate challenge from keeping the character consistent. Meta trained a faster model, called a student, using a slower, higher-quality teacher model. The student also trains on its own generated context, an approach intended to limit the visual drift that could build up when one imperfect chunk influences the next.

That reduction describes model evaluations, not a 60-fold improvement in the time a person waits for a reply. Delivering a live call also means managing the voice stream, video generation and multiple sessions at once. Meta says it redesigned its serving system to reuse recent context, route sessions and group work dynamically. It also used four-bit quantization, which reduces the precision of numbers used in the model, and collaborated with NVIDIA on model optimizations.

What Meta measured in live use

Meta reports 448-by-768 portrait video at 25 frames per second. It measured about 870 milliseconds from the end of a user’s turn to the first byte of the synchronized voice-and-video response. That is a precise starting point, not a measure of how long every subsequent moment of the conversation takes to reach the viewer.

The company says its serving changes raise capacity eightfold against its two-step BF16 baseline, allowing video generation for 12 simultaneous real-time sessions on one NVIDIA GB200. The comparison is to Meta’s specified baseline, not to another company’s hardware setup. It matters because a live avatar must keep generating frames for every active conversation, rather than producing a single finished clip.

A preference test with a short horizon

Meta also tested Muse against Runway Characters and HeyGen LiveAvatar through each product’s live-call experience. Raters held two- to three-minute conversations using matched avatar identities. Meta says they preferred Muse overall and on every evaluated dimension, including visual quality, synchronization and character consistency. One result was less decisive: its mannerism comparison with Runway Characters was not statistically distinguishable from an even split.

Those calls offer a test of the immediate experience, but a narrower one than Meta’s goal of sustaining a consistent avatar throughout a conversation. Whether the same preference holds over longer exchanges remains open. Meta’s introduction describes the system and its results, but does not specify when people or developers will be able to use it.