Full archive

Benchmark: Models AI News

Search reporting on AI markets, models, products, companies, policy, and culture.

Showing 1-18 of 26

Today

1 story

13 hours ago

Anthropic’s Fable 5.1 Takes Benchmark Lead, but Its Top Setting Costs More Per Task

Artificial Analysis found Anthropic’s newest general model reached its highest measured intelligence score, while the token use required at maximum effort raised the cost of its own evaluation tasks.

4 min readRead story

Yesterday

2 stories

Benchmark Models

Yesterday

GPT-5 and Gemini-3 Recovered Up to 65% of Missed Facts by Thinking Longer

The Google Research and Technion benchmark shifts the diagnosis for some factual errors from missing training data to unreliable access—while leaving open whether the result holds beyond Wikipedia facts.

3 min readRead story

Benchmark Models

Yesterday

Grok 4.6 Clears 50% on Biosecurity Refusal and Routine Biology Tasks

The reported result rewards a difficult balance: recognizing hazards hidden inside ordinary-looking research requests without shutting down legitimate biological work. Its surveillance score shows that refusal performance is not the whole biosecurity test.

3 min readRead story

Earlier this week

2 stories

Benchmark Models

Yesterday

Qwen3.8-Max Leads Open-Weight Commerce Agent Bench by Two Passes, With Audit Limits

The benchmark checks whether agents leave commerce software and files in the required state, rather than simply narrate a workflow. But configuration choices and unavailable immutable result bundles limit what the narrow ranking can settle.

2 min readRead story

Benchmark Models

Aug 31, 2026

ChatGPT o3 Handles Simulated Liquidity Choices, but Consistency Slips in Harder Cases

The simulated wholesale-payments exercise defines a narrow starting point for delegated cash management: routine payment priorities, with human oversight for potentially anomalous patterns.

2 min readRead story

Last week

12 stories

Benchmark Models

Aug 30, 2026

Hack The Box Benchmark: Best Human Team Cleared 36 Challenges; Top AI Team Reached 32

The latest benchmark points to a useful division of labor in cyber work: agents can accelerate solving, but elite performance still depends on people choosing paths and checking results.

3 min readRead story

Benchmark Models

Aug 30, 2026

Chatbots Debunked Foreign Falsehoods About 75% of the Time, Beating Search Results

The result makes chatbots a potentially stronger starting point for investigating state-spread claims than a search-results page. It does not make their answers self-validating: language, citations and product design still shape the outcome.

4 min readRead story

Benchmark Models

Aug 29, 2026

Google’s WikiSkill Lifts Agent Benchmarks by Giving Models a Memory of Failed Work

The framework turns task traces into reusable instructions instead of changing a model’s training, offering a practical route to more capable agents while leaving uneven task gains and cross-model transfer as constraints.

3 min readRead story

Benchmark Models

Aug 28, 2026

Meta’s EvoHarness-RL Takes Qwen3-8B to 96.9% on ALFWorld, Near Claude Opus 4.5

The result is a strong benchmark showing for learned state management, not yet evidence that the framework improves the long-running enterprise workflows it is designed to target.

3 min readRead story

Benchmark Models

Aug 27, 2026

ReViSQL-K2.6 Hits 91.37% on Cleaned SQL Test, but Not a Live Database

The result argues that a focused model, repaired examples, and domain-specific rewards can rival elaborate SQL-agent stacks. Its remaining test is whether that recipe transfers from curated academic data to corporate databases.

3 min readRead story

Benchmark Models

Aug 27, 2026

Databricks Says 300M Chart-JSON Pipeline Tops Four Multimodal Baselines on Answer Accuracy

The company’s test suggests chart values can become a useful retrieval layer instead of relying solely on page-image embeddings, though its comparison measures answer correctness and includes a synthetic benchmark it built.

3 min readRead story

Benchmark Models

Aug 27, 2026

Google DeepMind Puts Gemini Flash Lite in a Double-Blind Test to Guard Secret Benchmarks

The pilot replaces a longstanding choice between exposing confidential tests and exposing proprietary model weights. Its value now rests on whether cryptographic separation can make external testing more credible for sensitive use cases.

3 min readRead story

Benchmark Models

Aug 26, 2026

Google and UNSW’s GlucoFM Beats a CGM Baseline by 4.1 PR-AUC Points, but Has No Public Checkpoint

The result suggests that separating slow glucose patterns from short-lived deviations can improve reusable wearable-data representations. For now, it is a retrospective research result rather than a clinical product or downloadable model.

3 min readRead story

Benchmark Models

Aug 25, 2026

LLMScholarBench Tests 22 Models and Finds an Accuracy-Representation Trade-Off

The benchmark treats AI-generated expert lists as recommender systems to audit, showing that factual correctness and balanced representation cannot yet be improved together reliably.

3 min readRead story

Benchmark Models

Aug 25, 2026

HiDream-O1-World Tops WBench Navi at 80.9, Betting on Persistent 3D Scenes

HiDream.ai’s new model is designed to make generated environments hold together through movement and edits. Its strongest disclosed evidence is a navigation benchmark; its larger film, robotics and production ambitions remain proposed uses.

3 min readRead story

Benchmark Models

Aug 24, 2026

NVIDIA Says AVO Took Claude Opus 5 From 30% to Perfect on ARC-AGI-3

The result shifts attention from training larger models to the software that gives them memory, tools and recovery loops. Its clearest limitation is equally important: the perfect score came on ARC-AGI-3’s public environments, while private sets remain the harder test.

3 min readRead story

Benchmark Models

Aug 24, 2026

GPT-BERT Beats Llama 2 70B on One Grammar Test With 100 Million Words

The narrow result does not make child-scale models competitive with frontier chatbots. It does show that massive pretraining corpora do not settle every language-learning test.

3 min readRead story

August 2026

1 story

Benchmark Models

Aug 22, 2026

Inherent Says Its 27B Science Agent Beat OpenAI and Anthropic on Paper Replication

The company’s result centers on reproducing known findings, not making discoveries. Its next test is whether that training approach can move from replication to reliable new science.

2 min readRead story