Full archive

Benchmark: Latest AI News

Search reporting on AI markets, models, products, companies, policy, and culture.

Showing 1-9 of 9Results matching "Claude Code"

Earlier this week

2 stories

Anthropic’s Fable 5.1 Takes Benchmark Lead, but Its Top Setting Costs More Per Task

Artificial Analysis found Anthropic’s newest general model reached its highest measured intelligence score, while the token use required at maximum effort raised the cost of its own evaluation tasks.

Yesterday

4 min readRead story

Benchmark Models

Qwen3.8-Max Leads Open-Weight Commerce Agent Bench by Two Passes, With Audit Limits

The benchmark checks whether agents leave commerce software and files in the required state, rather than simply narrate a workflow. But configuration choices and unavailable immutable result bundles limit what the narrow ranking can settle.

Aug 31, 2026

2 min readRead story

Last week

2 stories

Benchmark Tools

Lemmalog Turns Agent Memory Into Datalog State, Cutting LongMemEval Context 38-Fold

The experimental system moves long-running agent work away from recalling old notes and toward maintaining conclusions with explicit support. Its early benchmark results suggest a substantial context-saving path, while leaving the probabilistic extraction step as the central weakness.

Aug 28, 2026

3 min readRead story

Benchmark Models

NVIDIA Says AVO Took Claude Opus 5 From 30% to Perfect on ARC-AGI-3

The result shifts attention from training larger models to the software that gives them memory, tools and recovery loops. Its clearest limitation is equally important: the perfect score came on ARC-AGI-3’s public environments, while private sets remain the harder test.

Aug 24, 2026

3 min readRead story

August 2026

5 stories

Benchmark Tools

InferenceX Adds 11 Telemetry Views to AgentX, Exposing What Benchmark Curves Hide

The new exploration layer makes cache setup, bursty subagents and queue behavior inspectable, but its best-curve design means readers must still examine each point’s underlying configuration before treating a curve as a like-for-like comparison.

Aug 23, 2026

3 min readRead story

Benchmark Models

Inherent Says Its 27B Science Agent Beat OpenAI and Anthropic on Paper Replication

The company’s result centers on reproducing known findings, not making discoveries. Its next test is whether that training approach can move from replication to reliable new science.

Aug 22, 2026

2 min readRead story

Benchmark Models

GLM-5.3’s Cheap Retries Put Fable 5’s Coding Premium Under Pressure

Together AI’s benchmark run suggests first-attempt parity is not enough to justify Fable 5’s price for general coding. But a Rust and serialization advantage, plus benchmark-specific cost assumptions, leave room for targeted routing.

Aug 22, 2026

2 min readRead story

Benchmark Models

Nvidia’s AVO Clears ARC-AGI-3’s Public Set. Withheld Tests Still Matter.

The result makes a strong case that long-running agent design can change benchmark outcomes. It does not yet show whether AVO transfers to ARC-AGI-3’s withheld competition environments.

Aug 21, 2026

3 min readRead story

Benchmark Infrastructure

AgentX Replays Claude Code Sessions to Test AI Serving Systems

The replay dataset is designed to measure infrastructure behind agentic coding workloads, not whether a model writes better code. Its value depends on whether synthetic traces preserve the traffic patterns operators need to serve.

Aug 18, 2026

3 min readRead story