Full archive

Benchmark: Latest AI News

Search reporting on AI markets, models, products, companies, policy, and culture.

Showing 1-9 of 9Results matching "Agent harness"

Yesterday

1 story

InferenceX Puts GB300 First for MiniMax M2.7, but Only on a Fixed Chat Test

The new ranking gives infrastructure teams a comparable speed and cost snapshot, but its single-turn design does not test the multi-step agent work MiniMax highlights for M2.7.

15 hours ago

2 min readRead story

Earlier this week

2 stories

Benchmark Models

Grok 4.6 Clears 50% on Biosecurity Refusal and Routine Biology Tasks

The reported result rewards a difficult balance: recognizing hazards hidden inside ordinary-looking research requests without shutting down legitimate biological work. Its surveillance score shows that refusal performance is not the whole biosecurity test.

Sep 1, 2026

3 min readRead story

Benchmark Models

Qwen3.8-Max Leads Open-Weight Commerce Agent Bench by Two Passes, With Audit Limits

The benchmark checks whether agents leave commerce software and files in the required state, rather than simply narrate a workflow. But configuration choices and unavailable immutable result bundles limit what the narrow ranking can settle.

Aug 31, 2026

2 min readRead story

Last week

3 stories

Benchmark Tools

Perplexity Search API Sweeps Artificial Analysis Test, but Medium Context Wins

The result gives developers a concrete retrieval-quality lead to test, while showing that sending an agent more page text is not automatically the best route to better answers.

Aug 28, 2026

3 min readRead story

Benchmark Models

Meta’s EvoHarness-RL Takes Qwen3-8B to 96.9% on ALFWorld, Near Claude Opus 4.5

The result is a strong benchmark showing for learned state management, not yet evidence that the framework improves the long-running enterprise workflows it is designed to target.

Aug 28, 2026

3 min readRead story

Benchmark Models

NVIDIA Says AVO Took Claude Opus 5 From 30% to Perfect on ARC-AGI-3

The result shifts attention from training larger models to the software that gives them memory, tools and recovery loops. Its clearest limitation is equally important: the perfect score came on ARC-AGI-3’s public environments, while private sets remain the harder test.

Aug 24, 2026

3 min readRead story

August 2026

3 stories

Benchmark Models

Stanford Won Databricks’ Agent Cup, but 18.8% of Questions Stumped Every Team

The live contest tested whether agents developed on one benchmark could handle an unfamiliar Treasury archive. Stanford led the field, but the unsolved questions show the limits of reliable document reasoning.

Aug 22, 2026

2 min readRead story

Benchmark Models

Nvidia’s AVO Clears ARC-AGI-3’s Public Set. Withheld Tests Still Matter.

The result makes a strong case that long-running agent design can change benchmark outcomes. It does not yet show whether AVO transfers to ARC-AGI-3’s withheld competition environments.

Aug 21, 2026

3 min readRead story

Benchmark Models

Open Models Are Catching the Frontier Faster. Benchmark Scores Aren’t the Whole Contest.

SemiAnalysis sees the lag from closed-model breakthroughs to open-model benchmark parity shrinking across three AI eras. But a comparable model score still leaves a harder product question: who can turn capability into dependable agent work?

Aug 21, 2026

4 min readRead story