Full archive

Benchmark: Models AI News

Search reporting on AI markets, models, products, companies, policy, and culture.

Showing 1-18 of 26

Today

1 story

Anthropic’s Fable 5.1 Takes Benchmark Lead, but Its Top Setting Costs More Per Task

Artificial Analysis found Anthropic’s newest general model reached its highest measured intelligence score, while the token use required at maximum effort raised the cost of its own evaluation tasks.

13 hours ago

4 min readRead story

Yesterday

2 stories

Benchmark Models

GPT-5 and Gemini-3 Recovered Up to 65% of Missed Facts by Thinking Longer

The Google Research and Technion benchmark shifts the diagnosis for some factual errors from missing training data to unreliable access—while leaving open whether the result holds beyond Wikipedia facts.

Yesterday

3 min readRead story

Benchmark Models

Grok 4.6 Clears 50% on Biosecurity Refusal and Routine Biology Tasks

The reported result rewards a difficult balance: recognizing hazards hidden inside ordinary-looking research requests without shutting down legitimate biological work. Its surveillance score shows that refusal performance is not the whole biosecurity test.

Yesterday

3 min readRead story

Earlier this week

2 stories

Benchmark Models

Qwen3.8-Max Leads Open-Weight Commerce Agent Bench by Two Passes, With Audit Limits

The benchmark checks whether agents leave commerce software and files in the required state, rather than simply narrate a workflow. But configuration choices and unavailable immutable result bundles limit what the narrow ranking can settle.

Yesterday

2 min readRead story

Benchmark Models

ChatGPT o3 Handles Simulated Liquidity Choices, but Consistency Slips in Harder Cases

The simulated wholesale-payments exercise defines a narrow starting point for delegated cash management: routine payment priorities, with human oversight for potentially anomalous patterns.

Aug 31, 2026

2 min readRead story

Last week

12 stories

Benchmark Models

Hack The Box Benchmark: Best Human Team Cleared 36 Challenges; Top AI Team Reached 32

The latest benchmark points to a useful division of labor in cyber work: agents can accelerate solving, but elite performance still depends on people choosing paths and checking results.

Aug 30, 2026

3 min readRead story

Benchmark Models

Chatbots Debunked Foreign Falsehoods About 75% of the Time, Beating Search Results

The result makes chatbots a potentially stronger starting point for investigating state-spread claims than a search-results page. It does not make their answers self-validating: language, citations and product design still shape the outcome.

Aug 30, 2026

4 min readRead story

Benchmark Models

Google’s WikiSkill Lifts Agent Benchmarks by Giving Models a Memory of Failed Work

The framework turns task traces into reusable instructions instead of changing a model’s training, offering a practical route to more capable agents while leaving uneven task gains and cross-model transfer as constraints.

Aug 29, 2026

3 min readRead story

Benchmark Models

Meta’s EvoHarness-RL Takes Qwen3-8B to 96.9% on ALFWorld, Near Claude Opus 4.5

The result is a strong benchmark showing for learned state management, not yet evidence that the framework improves the long-running enterprise workflows it is designed to target.

Aug 28, 2026

3 min readRead story

Benchmark Models

ReViSQL-K2.6 Hits 91.37% on Cleaned SQL Test, but Not a Live Database

The result argues that a focused model, repaired examples, and domain-specific rewards can rival elaborate SQL-agent stacks. Its remaining test is whether that recipe transfers from curated academic data to corporate databases.

Aug 27, 2026

3 min readRead story

Benchmark Models

Databricks Says 300M Chart-JSON Pipeline Tops Four Multimodal Baselines on Answer Accuracy

The company’s test suggests chart values can become a useful retrieval layer instead of relying solely on page-image embeddings, though its comparison measures answer correctness and includes a synthetic benchmark it built.

Aug 27, 2026

3 min readRead story

Benchmark Models

Google DeepMind Puts Gemini Flash Lite in a Double-Blind Test to Guard Secret Benchmarks

The pilot replaces a longstanding choice between exposing confidential tests and exposing proprietary model weights. Its value now rests on whether cryptographic separation can make external testing more credible for sensitive use cases.

Aug 27, 2026

3 min readRead story

Benchmark Models

Google and UNSW’s GlucoFM Beats a CGM Baseline by 4.1 PR-AUC Points, but Has No Public Checkpoint

The result suggests that separating slow glucose patterns from short-lived deviations can improve reusable wearable-data representations. For now, it is a retrospective research result rather than a clinical product or downloadable model.

Aug 26, 2026

3 min readRead story

Benchmark Models

LLMScholarBench Tests 22 Models and Finds an Accuracy-Representation Trade-Off

The benchmark treats AI-generated expert lists as recommender systems to audit, showing that factual correctness and balanced representation cannot yet be improved together reliably.

Aug 25, 2026

3 min readRead story

Benchmark Models

HiDream-O1-World Tops WBench Navi at 80.9, Betting on Persistent 3D Scenes

HiDream.ai’s new model is designed to make generated environments hold together through movement and edits. Its strongest disclosed evidence is a navigation benchmark; its larger film, robotics and production ambitions remain proposed uses.

Aug 25, 2026

3 min readRead story

Benchmark Models

NVIDIA Says AVO Took Claude Opus 5 From 30% to Perfect on ARC-AGI-3

The result shifts attention from training larger models to the software that gives them memory, tools and recovery loops. Its clearest limitation is equally important: the perfect score came on ARC-AGI-3’s public environments, while private sets remain the harder test.

Aug 24, 2026

3 min readRead story

Benchmark Models

GPT-BERT Beats Llama 2 70B on One Grammar Test With 100 Million Words

The narrow result does not make child-scale models competitive with frontier chatbots. It does show that massive pretraining corpora do not settle every language-learning test.

Aug 24, 2026

3 min readRead story

August 2026

1 story

Benchmark Models

Inherent Says Its 27B Science Agent Beat OpenAI and Anthropic on Paper Replication

The company’s result centers on reproducing known findings, not making discoveries. Its next test is whether that training approach can move from replication to reliable new science.

Aug 22, 2026

2 min readRead story

Load more stories8 more next