Google Research Publishes Retrieve-for-Train to Speed Up Multi-Query Search

Google Research has published Retrieve-for-Train, a framework designed to replace slow, live language-model planning with a compact model that generates a complete set of search directions in one pass. The target is a common search problem: broad requests need a varied, complementary group of results, not ten near-duplicates. Google reports that its diffusion retriever was 12–20 times faster than autoregressive approaches in its tests while improving several measures of retrieval quality.

The work appears in the ICML 2026 paper “Efficient, Property-Aligned Fan-Out Retrieval via RL-Compiled Diffusion.” Its central bet is that a system should do the hard work of discovering good query expansions before deployment, then reuse that learned behavior at search time rather than asking an LLM to reason through every request token by token.

One broad request, two very different paths

Conventional fan-out retrieval can use an autoregressive language model to turn one broad request into multiple related sub-queries. That can improve coverage, but Google argues it creates a latency problem: the model must generate its reasoning and outputs sequentially. The company also says generic models can fall into “paraphrastic collapse,” producing nearly synonymous searches instead of distinct directions.

Retrieve-for-Train separates exploration from serving. First, reinforcement learning trains a fan-out language model against a reward for three properties: remaining close to retrievable database content, staying aligned with the original request, and generating a diverse set. That trained model then creates query-to-target-set examples offline, without human labels. Finally, a 53.9-million-parameter diffusion retriever learns from those examples.

A smaller model inherits a larger model’s search behavior

The deployed model does not write search terms as text. It maps a query embedding, a numerical representation of the request, directly to a set of target embeddings. Google says that parallel generation is what avoids the text-based chain-of-thought processing used by a live language-model fan-out system.

Quality claims depend on narrow retrieval tests

Google evaluated the framework in two settings: open-ended abstract retrieval, where quality is judged through properties of the whole result set, and weakly supervised compositional retrieval, where a reference set is only one plausible answer. The experiments used a fashion dataset with CLIP-based retrieval and a proprietary dataset of expert-generated music playlists with MuLan retrieval.

In those evaluations, Google says Retrieve-for-Train outperformed single-query search, zero-shot expansion and a Best-of-N baseline on diversity, alignment and recall. The company fine-tuned Gemma3-4B and Qwen3-4B to generate 10 sub-queries per main prompt during the fan-out stage.

The unresolved question is portability. The paper presents a way to compile carefully defined search objectives into training data, but its reported results come from fashion and music retrieval experiments. Whether the same reward design can preserve useful diversity, relevance and grounding in other databases will depend on the properties a search operator can actually specify and measure.