NVIDIA says VFlash runs robot AI pipelines five times faster in research tests

Robot AI can spend less time writing out a movement plan—and produce a more accurate one—according to new research from NVIDIA and collaborators. Their VFlash framework generates whole spatial predictions in parallel. NVIDIA reports roughly five times faster robot model-pipeline inference on Jetson Thor and DGX Spark, alongside lower trajectory errors.

The published research targets a bottleneck between seeing a scene and choosing an action. Vision-language models, which process images and language, can describe where a robot should grasp or move using coordinates. But generating those coordinates sequentially becomes costly as plans grow longer or include more objects. VFlash changes how those predictions are generated, rather than retraining the entire underlying model.

VFlash starts with the structure of the requested output. A bounding box has a known coordinate layout; a trajectory has a requested number of points. A structured tool call specifies the output length, while the type of prediction determines where each coordinate belongs. A small drafting model then predicts the complete geometric unit in one parallel pass.

That requires keeping positions predictable. Written coordinate values can occupy different numbers of tokens—the pieces of output a model generates—and shift everything that follows. VFlash uses dedicated coordinate tokens or fixed-width numeric fields to preserve the layout. Its training and decoding blocks also follow complete geometric boundaries, rather than cutting a path into arbitrary token windows.

The larger vision-language model stays frozen while the lightweight drafter learns from annotated geometry. Crucially, visual predictions do not have to match the larger model token for token. Text receives different treatment: the larger model checks proposed text blocks and retains the matching portion. Predicted visual plans are passed through the larger model to produce features for downstream action control.

NVIDIA reports a 42.6% reduction in mean 3D displacement error across three robot datasets and three target-model variants. That measures how far predicted points stray from their reference positions. The headline reduction of up to 61% is a best-case result, not the average across those evaluations.

The researchers also tested whether those plans helped a downstream robot-control model complete tasks. On MolmoSpaces, VFlash’s combined 2D and 3D plans yielded 70.2% average success across four task suites, versus 62.6% for the baseline. Plans generated sequentially by the larger model reached 61.0%. All variants used the same action-model architecture and training budget.

A further comparison separated the contribution of geometry from language. Replacing trajectory features with image features, while retaining the generated sub-task description, lowered success to 63.7%. Using only VFlash’s 3D plans reached 69.3%. Those tests support the researchers’ argument that the visual plan contributes more than a written instruction about the next step.

Sequential decoding accounted for 75–87% of measured model-stage latency on Thor, giving VFlash a large bottleneck to attack. But the remaining stages still take time. NVIDIA’s timing covers the model pipeline, not the full interval from sensor input to actuator response. Faster inference therefore should not be read as an equivalent increase in a robot’s physical working speed.

Measurements used customized NVIDIA runtimes at batch size one, processing one request at a time. They include model-stage work and preparation of the target model’s features, but exclude loading, warmup, file access, CPU preprocessing outside the measured modules and conversion of output tokens into text. Robot timing samples covered 24 records; Thor used ten passes, while Spark and H100 used one.

The framework also handled driving and other visual tasks. NVIDIA reports more than three times faster NAVSIM driving inference on Thor and Spark. For dense 3D motion forecasting, it reports 27% lower displacement error and approximately 25× faster single-rollout generation on H100. Object-localization tests produced 2.6–2.8× more boxes per second, with mean F1 accuracy close to the sequential target.