MIT researchers use AI to patch a model, raising its math score from 7.9% to 48.1%

MIT CSAIL researchers reported on November 2 that OpenAI’s GPT-6 Astra directly edited another AI model’s internal parameters, lifting its MATH-500 math-test accuracy from 7.9% to 48.1%. The gain mainly came from stopping repetitive output—not adding mathematical knowledge, according to the researchers’ findings described by VentureBeat. That makes the result a striking repair, rather than proof of broadly smarter AI.

Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab call the approach “weight-space programming.” Their November 2 research post describes experiments in which Astra inspected, repaired and constructed neural networks. The central question was whether an already trained language model could reason about another network’s numerical machinery and deliberately change it.

A tiny patch, a long investigation

Weights are the numerical parameters that govern a model’s behavior. Conventional training calculates adjustments from examples, errors or feedback. Here, Astra instead investigated the target network and devised direct edits. For Qwen3.5-0.8B, it received practice questions, access to weights through Python and a limited allowance of model evaluations. This was not improvement without computation or testing.

The strongest attempt allowed 50 evaluation requests, each potentially covering hundreds of examples. It involved 158 conversational turns and 80 Python runs over 20.3 hours, including 4.7 hours waiting through a service outage. The final edit was small, but finding it required a substantial search—not a single instruction followed by an instant fix.

Astra ultimately changed 131,072 weights, approximately 0.016% of the model. The researchers traced the improvement mainly to removing a numerical direction associated with repetition from selected components in layers 15 and 19. Before the patch, Qwen often repeated text until its answer allowance ran out. Reducing that behavior let it finish more answers using abilities it already had.

The benefits extended beyond that test. Under the researchers’ evaluation conditions, accuracy on the GSM8K math benchmark rose from 13.8% to 67.1%. On IFEval, which measures instruction following, it increased from 21.0% to 62.3%. Those results broadened the observed benefit beyond one math score.

The repair did not generalize to Gemma

A follow-up with Gemma 3 270M drew a sharper boundary around the result. That model did not exhibit Qwen’s severe repetition problem, and Astra’s edits produced no established mathematical improvement. The contrast supports the researchers’ narrower explanation: correcting an obstruction can improve measured performance without demonstrating that the editor has added new reasoning abilities.

The same project tested direct work on weights outside language models, with mixed comparisons:

• Simulated driving: Given a controller’s weights, a scene description and one failed trajectory, Astra returned a patch in one response. The controller had repeatedly stalled at a ramp. The researchers reported that changing 3.5% of its weights produced a 100% success rate.
• Game-network construction: Astra built networks from simplified game rules and source code without trying them before evaluation. Its Breakout network scored 517.4, versus 25.3 for a matching network trained for 500 million steps. But it trailed those training baselines in Seaquest and slightly in Asterix.

From successful edits to a reliable method

Inspection also had limits. Astra initially failed to distinguish a model trained to agree with users from one trained to answer correctly. It succeeded only after receiving measurements of attention during execution. Access to raw weights alone did not consistently provide enough information to explain the behavior.

The team released experiment logs and editing artifacts and said it was developing a benchmark and supporting framework. The work has not established that direct editing consistently beats conventional training on speed, cost or effectiveness. Nor did it demonstrate recursive self-improvement: a cycle in which an AI’s improvements help it generate further improvements. Reliability and comparative value remain open engineering questions.