MIT uses AI to lift a model's math score to 48.1%

Welcome to The Signal from Superpower Daily

With Maya and Theo

we are previewing a wild story today

MIT researchers just used an advanced AI model to physically patch the internal weights of another AI model

Yeah and they radically boosted its math score

Right

from 7 9 to

48 1 So let's dive right into this

lead story The team at MIT CSAIL used OpenAI's

GPT 6 ASTRA and they used it to directly edit the parameters of an open

source model called QN3

5

0 8b And QN was honestly just terrible at math before

this patch

Completely terrible I mean it scored a 7 9 on the

Math 500 test But after ASTRA tweaked

its internal weights that score

skyrocketed to 48 1 That

is a massive jump And it wasn't just on that

one

test either

No it wasn't It also lifted its GSM 8k score from about 13 8

up

to 67 1 Wow Yeah And its instruction following score on the IFEVAL metric went from

21 to 62 3 So if you

just read the headline you might think ASTRA basically taught QN

how to do calculus Right Like we have finally hit that sci

fi singularity where AI

is actively you know evolving itself But the reality of what ASTRA

actually did is

much more mechanical Yeah

And honestly it's a lot weirder ASTRA

did not teach QN new math It really didn't To understand what happened you have to look at

how we normally improve

these

models Right With conventional training Exactly Conventional training

is sort of a blunt instrument If you want a

model to get better at math you

feed it thousands of

math problems You let it guess

And when it gets them wrong

an algorithm just calculates the error Right And it mathematically adjusts millions of

digital dials which are called

weights across the entire neural

network It's

a very slow iterative grind This

MIT experiment is different though

This

is weight space programming Yes

So instead of conventional training ASTRA essentially investigated the

neural network like a mechanic It diagnosed a specific behavioral bottleneck and then it manually tweaked

the numbers to fix

it It's fascinating detective work ASTRA suppressed a very specific numerical direction

that was linked to repetition Because QN was previously just repeating text over and over until it completely

ran out of token space Exactly

It would just ramble endlessly Fixing this didn't teach it new skills it just unblocked the

math skills it already possessed I really want to push back on the

whole

self

improving

AI narrative here

Please do Because this isn't like uploading a

calculus textbook into someone's brain It is like untying a

kink in a garden hose That's a

great

way to put it Right You untie the kink so

the water can finally

flow It is a repair

It

is not an evolution No

It's really just a repair ASTRA found

the kink in the hose specifically in layers 15

and 19 of the network And it

only changed exactly 131 072

weights Which sounds like a lot But it's only

about 0 016

of the entire model It is surgical But we have

to mention the most important caveat here

This was not

some fast magical fix Not at all

The

strongest attempt took ASTRA 20 3

hours Yeah And it took

158 conversational turns It ran

80 Python scripts It made 50 evaluation requests It even had to sit there

and wait through a 4 7 hour service outage Which is just funny to think about An

advanced AI just

waiting for a server to reboot It really is But crucially when they

tested this exact same method on a different

model GEMMA

3 270m it produced no established mathematical improvement

None

whatsoever Because GEMMA didn't have Quinn's specific repetition glitch

Right

It didn't have that specific kink in the hose Exactly So this does not prove

that this method beats conventional training on speed or cost

And it certainly doesn't prove runaway recursive

self improvement is here So what should listeners watch for next You should watch

for the benchmark and supporting framework the MIT team is developing We need to see

if this weight patching technique can actually generalize across different architectures In other

news as AI models get better at writing and editing code OpenAI has had to completely rebuild its

Git

backend just to

survive the sheer volume of AI generated work Yeah OpenAI introduced

Pando It is a

newly rebuilt backend powering

their shared code repository which is their monorepo Because late last year the

massive output

from their Codex coding agents was literally

breaking their systems

It was Fetches and pushes

slowed to a crawl Merges failed

outright

Their reliability dropped below 90 Which is catastrophic for version control Absolutely So

Pando stepped in and cut the median push latency by nearly fivefold

And now it serves almost a petabyte

of Git data per day Which is a staggering amount of data And they

did this all without requiring engineers to change

their

standard Git workflows So why

does

this matter beyond

the headline Well standard Git queues

updates sequentially Right one by one But Pando coordinates

each branch independently It stores compressed bundles of code called pack files And

it reuses those pack files across requests to save server processing power It is just a stark

look at the invisible infrastructure toll of deploying AI agents

at scale Human

systems simply buckle under the parallel output

of AI They really do But there is a massive

tradeoff here Pando gives up a single synchronized

snapshot across multiple branches Right It offers something called causal

consistency instead So readers get the perfect history for a specific branch But they can't necessarily see

a perfect snapshot of the whole repository at a single exact moment I think of it like replacing

a single checkout line

at a grocery store

with 50 self checkout terminals That makes perfect sense It is vastly faster But the store manager no longer has a single

perfectly synced

ledger of exactly what left the

store at exactly

12 01pm Exactly It trades that global state for sheer speed Listeners should watch to see if OpenAI's planned upstream contributions of

these client side changes are adopted by the broader open source Git community

Next up the massive scale of AI automation isn't just straining code repositories North Korean hackers are

now using AI

to automate the targeting and laundering of high value crypto heists According to experts a

state sponsored North Korean unit called TraderTrader is using

AI to hunt

crypto companies And this unit only has about a dozen members Just a dozen

But they are pulling in massive payouts with a strategy

dubbed

Big

Game

Hunting The FBI blames

them

for a recent 6 3 billion

Binance theft A billion with a B Yeah And the

laundering was shockingly fast They moved the funds through

12 000 wallets across 11 blockchains And they did 40

000 transactions in under four

days Which breaks down to one

new wallet every 25 seconds A

human cannot do that The attack surface has completely changed

They aren't just

phishing individuals anymore They are studying a company's entire infrastructure and fund transfer processes ISATCA

actually argues this elevates cybersecurity from an IT problem to

a board level governance mandate But we need to highlight the most important caveat here

This

AI assessment rests entirely on inferred attack techniques observed by TRM Labs and Zscaler Right There

is no demonstrated account or seized system showing AI

carrying

out the entire theft autonomously It's an educated deduction based on the inhuman speed Exactly So watch how

boards of directors at major

crypto firms restructure their risk oversight in response to these automated

high velocity laundering networks Meanwhile managing these complex automated AI workflows is getting really

expensive This prompted Anthropic to launch a public beta of

message threads for cloud agents This is a really clever update to their messages API Now

agents can send a conversation once

and then

only

upload what changed on subsequent turns So Anthropic temporarily stores the conversation server side Right Normally

coding agents

doing hundreds

of requests

have

to re upload the entire

history every single time The code the files the results

Everything And it causes severe latency But by sending only new

messages and a response ID Anthropic drastically improved the time to

first token for massive prompts We are talking

about prompts with 500 000 to 1 million tokens Now Cloud

starts typing its answer much sooner

I really want

to emphasize what this does not do though It does not save you money No it doesn't Your token usage and

billing

stay exactly the same Because Cloud still

has to read the full context to

answer correctly Exactly You are just saving network

upload time And furthermore the storage

is

strictly temporary It expires between 5 minutes and 1 hour And

it is strictly best effort Right So if a continuation fails and you get

a 404 error the developer must have a recovery plan to replay the entire history from scratch Watch

how developers build robust recovery systems to handle those

API failures when relying on this temporary storage We are moving to our Quick

Reads section now

Alphabet shareholder Phil Rosen filed a derivative

suit in Chicago against Sundar Pichai

and 12 executives He alleges Google trained

systems

like Lyria Mulan and MusicLM on copyrighted music Including 5 million

clips for

MusicLM alone Yeah And he also

alleges Gemini Live used voice prints violating Illinois BIPA law Rosen claims this inflated Alphabet stock

linking it

to

169 6 billion in share repurchases And he is seeking damages for the company not himself Next OpenAI

launched an internal benchmark called LifeVal to measure AI assistance with everyday errands like budgeting or childcare It is based

on a survey of nearly 4 000 people with

100 experienced users writing the tasks and grading criteria It shifts

focus from workplace skills to personal context Notably current

examples

show

the AI can prep a shortlist but critical final decisions still remain with the

user Finally Apple's

J595 project is a cost 1 000 or more according to reports It features a 9

inch screen on a motorized arm

that swivels to face speakers during FaceTime and it

controls smart home functions It likely

will not travel around the house though Launches aim for

2027 or 2028 but Apple is

reportedly struggling to find enough everyday utility to justify

that

premium price point We are moving to our three takeaways

from today First if you

look at the MIT and OpenAI Git stories AI isn't

just

writing code It is forcing a fundamental

redesign

of how we store patch and manage

software at an infrastructure level

Second looking at the security and scale stories speed is the new weapon Whether it's cloud processing a

million tokens or hackers spinning up a crypto wallet every

25

seconds the velocity of AI agents is outpacing legacy human defenses

Third from the consumer stories like LifeVal and the Apple robot

the final mile of AI is proving to be the hardest from deciding on a

child care

schedule to justifying a 1 000 robotic screen

Human preference and physical reality are

stubborn bottlenecks For one development to watch tomorrow keep an eye on

the

enterprise fallout from the Anthropic Thread update Specifically watch how fast developers can rewrite their agent workflows to take advantage of the

reduced latency without breaking their error

recovery systems Head over

to superpowerdaily com for