Google’s WikiSkill Lifts Agent Benchmarks by Giving Models a Memory of Failed Work

AI agents in Google Research’s WikiSkill tests improved sharply when they could retain lessons from earlier attempts: Gemini 3.5 Flash’s average score rose from 49.5% to 68.1% across five benchmarks. The result comes from a new framework that stores traces of completed work, converts them into a durable knowledge base, and uses that record to revise an agent’s operating instructions.

Google Research introduced WikiSkill as a way to give agents persistent knowledge and improve their procedural skills over successive runs. It does not retrain the underlying model. Instead, the system records what happened during a task and generates better reusable instructions for the next attempt.

Three layers separate evidence from action

The framework divides the agent’s workspace into three layers. The Raw Layer keeps complete execution traces, including tool calls and results. A persistent Wiki Layer distills those records into documented failure patterns and successful strategies. The Skill Layer holds the procedural instructions used on new tasks.

That separation is the design’s central safeguard. A Wiki Maintainer analyzes traces and adds findings to the wiki; a Skill Proposer then recommends targeted changes. A gating mechanism checks each proposed change on a separate validation set. If the update reduces performance, the system can restore the prior skill while keeping the experience that informed the failed proposal.

The strongest scores came in math and spreadsheets

Researchers evaluated WikiSkill on benchmarks for math reasoning, web search, spreadsheet manipulation, document question-answering, and interactive tasks in a virtual environment. The breadth is useful, but the results were not uniform across those task types.

• Gemini 3.5 Flash rose from 33.0% to 72.6% on LiveMath.
• The same model rose from 50.5% to 76.6% on SpreadSheet.
• Qwen-3.6-27B’s five-benchmark average increased from 39.4% without skills to 63.3% with WikiSkill.

The study found smaller improvements on OfficeQA, the long-document question-answering task. It also found that larger models generally gained more from evolved skills, while smaller models could struggle to carry out multi-step search strategies over long contexts and revert to their default behavior.

A reusable record, not a universal upgrade

WikiSkill’s notable promise is that a skill developed with one model can sometimes help another model, and may occasionally outperform a skill created by the receiving model itself. That transfer was not reliable in every case, so the researchers’ results leave model-to-model portability as something to test rather than assume.

The framework therefore offers a different route to agent improvement than changing model weights: preserve operational evidence, summarize it, and test whether a revised instruction set works better. Its demonstrated gains make that route compelling for the evaluated tasks; the uneven performance across benchmarks and models is the limit on treating the wiki as a general-purpose cure for agent reliability.