Google Releases ToolGrad, Reporting a 99.8% Pass Rate for Tool-Use Data

Google Research has released ToolGrad, a framework for training AI systems to use software tools by building a working API-call chain before writing the user request that would lead to it. The change targets a costly part of agent development: generating examples where a model must choose and execute the right sequence of tools.

The conventional approach starts with a made-up instruction, then sends an agent to search for a way to satisfy it. Google says that query-first process can waste compute when a depth-first search follows a dead end and the resulting sample is discarded. ToolGrad reverses that order: it starts with a ground-truth sequence of successful tool calls, then has a language model produce the matching prompt and answer.

A workflow grows from execution feedback

The framework treats the results of attempted API calls as guidance for the next step. Google describes this as a use of textual gradients: plain-language execution feedback that helps the system extend a valid workflow instead of optimizing model weights. Each completed sample contains a synthetic user query, a verified workflow and a final response.

The four parts of ToolGrad

• The API Proposer narrows a sampled pool to calls that may extend the current workflow.
• API Executors test selected calls in parallel and produce execution reports.
• The API Selector chooses one successful call to append to the chain.
• The LLM Updater revises the synthetic request and response to fit the expanded workflow.

The reported gain is in usable examples

Google tested ToolGrad against a depth-first-search baseline on ToolBench, a database with more than 16,000 real-world APIs. Google reports a 99.8% pass rate and lower generation cost for ToolGrad. A public summary of the paper puts the baseline pass rate at 63.8%, and says ToolGrad increased successful tool uses per sample from 2.1 to 3.4 while reducing tool-use steps from 34.3 to 20.0.

That distinction matters. The pass rate measures how often the data-generation pipeline produces a sample that clears its checks; it is not itself a measure of how reliably a finished agent performs for users. Google’s central claim is that finding a valid chain first makes the subsequent language-writing step more bounded and less ambiguous than searching from an invented request.

A compact model nears the reported proprietary benchmark score

For its downstream test, Google generated a 500-example dataset with Gemini 2.5 Flash-Lite and used it to fine-tune 1B, 4B and 12B-parameter Gemma 3 models. The evaluation used the Berkeley Function Calling Leaderboard, whose tools differ from ToolBench’s. That makes the test a check of transfer to unfamiliar tools, rather than recall of the APIs used to create the training set.

Google reports that ToolGrad-12B scored 83.1 on BFCL, compared with 83.2 for Gemini 2.5 Pro, 82.8 for Claude 4.5 Opus and 74.4 for GPT-5 in its comparison. Google also says the 12B model outperformed the Flash-Lite model that generated the data. Those are company-reported benchmark results, but they point to the practical promise of the method: a smaller model may gain specialized tool-use behavior from a relatively small set of verified examples.

The next test is a changing tool ecosystem

ToolGrad has been presented at ACL 2026, with its implementation available on GitHub. Google frames future work around larger and more dynamic API ecosystems, as well as continuous, on-the-fly learning for personalization. The immediate research result is clear: successful executions can become the starting material for agent training. The open question is how well that recipe holds as tool libraries, permissions and real-world workflows keep changing.