Salesforce says AI research agents improved a retrieval model in about 80 hours

Salesforce says its research agents completed a model-improvement cycle in about 80 hours, doing work that can take human researchers several weeks. In a November 11 research update, the company described SFR-AutoR&D: agents that propose changes, implement them and test whether they improve the software and AI models behind enterprise services.

The work targets embedding models, which turn text into numerical representations so software can find relevant information. Salesforce uses SFR-Embedding V3 for retrieval in Data Cloud. When an Agentforce agent answers a question using customer data, that retrieval step helps determine which records or documents inform its answer.

Better retrieval, then a faster model

Salesforce reports that SFR-AutoR&D improved SFR-Embedding V3. The research team then applied the system to the open-source EmbeddingGemma 2 model, producing SFR-Embedding v4 with improvements in both quality and latency—the time a model takes to respond. These are company-reported results, not an independent assessment of the system.

In its model-quality evaluation, Salesforce reports a 6.13-point improvement in nDCG@10 across 10 public retrieval tasks in MTEB, a benchmark suite. That measure scores the relevance of the first 10 search results, giving more weight to useful results placed nearer the top. The gain is in points, not a percentage increase.

The system then ran a separate optimization session lasting roughly nine hours, focused on how efficiently the model runs. Salesforce reports 36% higher throughput—the amount of work processed over time—and 17% lower latency, with almost no loss in quality. That session targeted operating efficiency rather than another large retrieval-quality gain.

Two agents divide the experimental work

SFR-AutoR&D starts with a goal, a system to improve and a compute budget. Its agents formulate hypotheses, implement proposed methods and run controlled experiments against a baseline. Salesforce describes this as a research loop, rather than simply asking an agent to write code and accepting whatever it produces.

• AutoR&D-Engineer searches existing code for algorithmic and systems improvements.
• TrainForge explores training methods, runs experiments and evaluates candidates on held-out benchmarks: tests whose data the system never trains on.

Both agents run on AutoInfra, Salesforce’s training infrastructure for experiments across shared GPU capacity. The company says the system also improved pandas, a Python library for working with tables of data, and the similarity-search libraries hnswlib and USearch. It says those code changes came with benchmark evidence supporting the gains.

The agents do not set their own rules

Researchers still set the goals, compute budget, evaluations and data constraints. The system explores within those boundaries; without a verified improvement over the baseline, the existing model stays in place. Salesforce presents that requirement as a limit on autonomy: the agents can pursue changes, but a proposal alone is not enough to replace a model.

Salesforce also says models trained for product use draw only on datasets that have passed ethical and legal review. Strong evaluation scores do not finish the deployment process: models must still meet production requirements before customers see them, and that work is underway. The update describes research results and internal use, not a general release of SFR-AutoR&D.