MIT uses AI to lift a model's math score to 48.1%
The repair curbed repetition rather than adding math knowledge—and took 20.3 hours to find.
By Saeed Ezzati8 min read
The audio edition
Listen to this newsletter
0:005:32
Read transcript
MIT researchers say OpenAI’s GPT-6 Astra raised Qwen’s score on the MATH-500 benchmark from 7.9 percent to 48.1 percent—not by teaching it new mathematics, but by editing the model to stop it repeating itself. That distinction matters: the repair let Qwen finish more answers using capabilities it already had. Astra changed 131,072 weights, about 0.016 percent of the small Qwen3.5-0.8B model. The researchers traced much of the gain to suppressing a numerical direction associated with repetition in parts of layers 15 and 19. Under their evaluation conditions, accuracy also rose from 13.8 to 67.1 percent on GSM8K, and from 21 to 62.3 percent on IFEval, an instruction-following test. But the tiny final patch came after a substantial search. The strongest attempt ran for 20.3 hours, including 4.7 hours waiting through a service outage. It involved 158 conversational turns, 80 Python runs and up to 50 evaluation requests. So this wasn’t an instant fix from one clever prompt; it was a lengthy process of inspecting, changing and testing the model. And the repair didn’t transfer to every model. In a test with Gemma 3 270M, Astra produced no established improvement in math. That model did not have Qwen’s severe repetition problem, reinforcing the researchers’ narrower explanation: removing a bottleneck can lift measured performance without adding reasoning ability. The work also doesn’t show that weight editing reliably beats conventional training on cost, speed or effectiveness, or that it enables recursive self-improvement. The researchers have released experiment logs and editing artifacts, and say they’re developing a benchmark and supporting framework. The practical question is whether this can become a repeatable repair method, rather than an impressive result that depends on a long search for just the right patch. That same gap between capability and operational limits shows up in the infrastructure behind coding agents. OpenAI says its new Git backend, Pando, now serves all development in the company’s shared code repository. The change follows a surge in parallel work enabled by Codex: more branches, uploads, merges and automated checks were putting pressure on the old system. OpenAI reports nearly fivefold lower median push latency, with Pando serving nearly a petabyte of Git data a day. The key design choice is to let unrelated branches update independently, rather than making every repository change wait in one shared queue. Pando also reuses download bundles, reducing repeated server work when many clients request the same code. Engineers and agents keep using Git workflows; OpenAI says the rollout did not interrupt development. There’s a tradeoff: reading several branches at once does not give users one synchronized snapshot of the whole repository. That may be acceptable for the company’s workload, but it’s a reminder that scaling parallel agents often means revisiting the guarantees underneath their tools. The security story raises a different operational question: who is accountable when systems and automation make large-scale attacks easier? Investigators say North Korea’s TraderTraitor unit is using AI to help identify cryptocurrency businesses that could yield large payouts. The assessment, reported by Bloomberg, is based on techniques seen in recent hacks; the reports do not describe which AI systems the hackers may be using. The FBI attributes a recent 6.3-billion-dollar Binance theft to the unit. This is what researchers call “Big Game Hunting”: going after companies rather than taking small amounts from individual investors. Investigators describe attacks that study a business’s infrastructure, then target that infrastructure or its money-transfer processes. Elliptic’s chief scientist separately described the laundering activity as automated and complex: about 12,000 wallets across 11 blockchains and more than 40,000 transactions in under four days. He said most of the funds were likely still held by North Korea. ISACA’s Chris Dimitriadis argues that a theft on this scale belongs in the boardroom as well as the security team. The useful issue to watch is not just whether AI assists attackers, but whether companies treat the resulting financial exposure as a governance responsibility. And in serving AI agents, Anthropic is trying to remove repeated upload work—not reduce the amount of material Claude processes. Its message threads beta lets developers send a conversation once, then follow up with only new messages and a prior response ID. Anthropic reports faster time to first token for Claude Code requests containing 500,000 to one million tokens. The full conversation is still rebuilt and read on every turn, so token usage, prices and rate limits do not change. Threads also don’t automatically enable prompt caching. They’re temporary, best-effort storage: developers need to keep their own conversation records and be ready to resend the full history if a thread fails. The feature is aimed at long sessions where repeatedly uploading a growing history adds delay, not cost. Across these stories, the practical dividing line is between a promising capability and the work required to make it dependable. Watch for evidence that model repair becomes repeatable, that agent infrastructure handles parallel work without weakening needed guarantees, and that companies plan for both the cost and security consequences of long-running AI systems.
