Gemini 4 Argon scores 97.5% in one ARC-AGI-3 setup, while standard runs remain incomplete

Google’s Gemini 4 Argon posted sharply different ARC-AGI-3 results depending on the software used to run the evaluation. ARC Prize’s results, dated October 13, 2026, show 49.8% with the Standard harness and 97.5% with the Provider Adapter at low reasoning. But API problems left the Standard evaluation incomplete, complicating a direct comparison.

The split also appears at medium reasoning: the Standard harness scored 46.3%, while the Provider Adapter scored 97.3%. The adapter’s high-reasoning result was also 97.3%. ARC Prize omitted the high-reasoning Standard point from its chart because too few runs finished. The page therefore does not offer a complete Standard-harness comparison across all three reasoning settings.

Two ways to carry work forward

A harness is the software setup around the model during a test. Here, the two setups differ in how they carry previous work into later requests. The Standard harness lets a model carry forward notes it chooses to keep. The Provider Adapter instead preserves opaque reasoning state between requests. It also uses compaction for longer conversations, allowing the model to reuse prior work.

ARC-AGI-3 at low reasoning

0149.8%

Standard harness

Runs finished on 42 of 55 tasks; the score retains partial credit from incomplete runs.

0297.5%

Provider Adapter

The setup preserves reasoning state between requests and uses compaction for longer conversations.

Unfinished runs still count

Completion coverage varied substantially by reasoning level, with the fewest finished runs at high reasoning.

• Low reasoning: runs completed on 42 of 55 tasks.
• Medium reasoning: runs completed on 46 of 55 tasks.
• High reasoning: runs completed on 19 of 55 tasks.

The scoring method does not discard unfinished tasks and calculate a result only from successful completions. Reported scores include all 55 tasks, retaining whatever partial credit incomplete runs earned. The published Standard figures therefore describe an interrupted evaluation, not a completed run over a smaller selection of tasks.

Earlier tests show a different reasoning pattern

On ARC-AGI-1 and ARC-AGI-2, the verified-scores table shows stronger results at higher reasoning settings. Argon scored 76.5% on ARC-AGI-1 at low reasoning, then 98.5% at both medium and high. ARC-AGI-2 rose from 42.1% at low reasoning to 87.1% at medium and 91.3% at high. These are separate benchmark results, rather than additional ARC-AGI-3 configurations.

There is a small internal discrepancy in ARC Prize’s page: its introductory prose gives the high-reasoning ARC-AGI-2 score as 91.2%, while the table lists 91.3%. The figures above follow the table. That mismatch is distinct from the incomplete-run issue, which the page identifies specifically for ARC-AGI-3’s Standard harness.

ARC Prize also specifies the pricing behind its reported costs: Google’s introductory rates of $2 per million input tokens and $10 per million output tokens. Those are usage rates for model input and output, not a disclosed total bill for either of the two ARC-AGI-3 setups.