Full archive

Benchmark: Latest AI News

Search reporting on AI markets, models, products, companies, policy, and culture.

Showing 19-36 of 45

Last week

12 stories

Aug 26, 2026

Google and UNSW’s GlucoFM Beats a CGM Baseline by 4.1 PR-AUC Points, but Has No Public Checkpoint

The result suggests that separating slow glucose patterns from short-lived deviations can improve reusable wearable-data representations. For now, it is a retrospective research result rather than a clinical product or downloadable model.

3 min readRead story

Benchmark Products

Aug 26, 2026

Tiangong Ultra Runs 100 Meters in 8.86 Seconds, While Safe Stopping Remains Unresolved

The Beijing competition made athletic progress visible, but its practical events point to the work still needed on deceleration, dexterity and autonomous operation before broader deployment.

2 min readRead story

Benchmark Models

Aug 25, 2026

LLMScholarBench Tests 22 Models and Finds an Accuracy-Representation Trade-Off

The benchmark treats AI-generated expert lists as recommender systems to audit, showing that factual correctness and balanced representation cannot yet be improved together reliably.

3 min readRead story

Benchmark Business

Aug 25, 2026

Amazon Puts $25B Standalone Marker on Chip Business While Remaining a Top Nvidia Customer

Jassy’s hypothetical revenue benchmark does not describe a separately reported company, but it gives Amazon’s internal chip effort a new measure of scale as it both buys from and competes with Nvidia.

2 min readRead story

Benchmark Products

Aug 25, 2026

Beijing Humanoid Games Put Practical Work to the Test; Only 3 of 12 Finish Fire Course

The games paired spectacle with multi-step chores and emergency work, where task switching, human control and offboard computing remain central constraints.

2 min readRead story

Benchmark Infrastructure

Aug 25, 2026

OpenAI’s Jalapeño Claims Up to 1.9x Efficiency Gain, but Leaves Nvidia Training Intact

The Broadcom co-developed inference processor is headed for deployment by year-end. Its published gains are meaningful, but they exclude model training and Nvidia’s Vera Rubin platform.

3 min readRead story

Benchmark Products

Aug 25, 2026

Tiangong Ultra Runs 100m in 8.86 Seconds, Then Hits a Stopping Mat

The Beijing race shows a sharp jump in humanoid sprint performance, but its finish also exposed a basic operational constraint: Tiangong Ultra needed a thick mat to stop and caught fire after impact.

3 min readRead story

Benchmark Infrastructure

Aug 25, 2026

OpenAI’s Jalapeño Claims 1.5–1.9x More AI Work Per Watt, Faces 2027 Scale Test

The custom chip gives OpenAI an early efficiency claim against available Nvidia systems, but production qualification, broader workloads and a small initial rollout will determine whether it becomes a meaningful serving platform.

4 min readRead story

Benchmark Models

Aug 25, 2026

HiDream-O1-World Tops WBench Navi at 80.9, Betting on Persistent 3D Scenes

HiDream.ai’s new model is designed to make generated environments hold together through movement and edits. Its strongest disclosed evidence is a navigation benchmark; its larger film, robotics and production ambitions remain proposed uses.

3 min readRead story

Benchmark Models

Aug 24, 2026

NVIDIA Says AVO Took Claude Opus 5 From 30% to Perfect on ARC-AGI-3

The result shifts attention from training larger models to the software that gives them memory, tools and recovery loops. Its clearest limitation is equally important: the perfect score came on ARC-AGI-3’s public environments, while private sets remain the harder test.

3 min readRead story

Benchmark Infrastructure

Aug 24, 2026

NVIDIA Says Vera Rubin Delivers 30x More Agentic Throughput per Megawatt, Pending Review

The preliminary comparison shifts attention from fixed prompt benchmarks toward the power required to keep long, tool-using AI sessions responsive.

3 min readRead story

Benchmark Models

Aug 24, 2026

GPT-BERT Beats Llama 2 70B on One Grammar Test With 100 Million Words

The narrow result does not make child-scale models competitive with frontier chatbots. It does show that massive pretraining corpora do not settle every language-learning test.

3 min readRead story

August 2026

6 stories

Benchmark Tools

Aug 23, 2026

InferenceX Adds 11 Telemetry Views to AgentX, Exposing What Benchmark Curves Hide

The new exploration layer makes cache setup, bursty subagents and queue behavior inspectable, but its best-curve design means readers must still examine each point’s underlying configuration before treating a curve as a like-for-like comparison.

3 min readRead story

Benchmark Tools

Aug 22, 2026

SemiAnalysis Says AgentX Drove 50-Plus Upstream Fixes for AI Agents

The claimed contribution is not a faster model kernel. It is a test workload that makes state retention, routing and data movement visible—and leaves open whether its fixes generalize beyond AgentX’s replay matrix.

3 min readRead story

Benchmark Models

Aug 22, 2026

Inherent Says Its 27B Science Agent Beat OpenAI and Anthropic on Paper Replication

The company’s result centers on reproducing known findings, not making discoveries. Its next test is whether that training approach can move from replication to reliable new science.

2 min readRead story

Benchmark Models

Aug 22, 2026

AI Safety Scores Can Reward Models for Refusing Too Much

A study involving the UK AI Security Institute argues that combined benchmark scores can hide a direct trade-off between blocking harmful requests and answering harmless ones. Its proposed fixes are cheaper tests and anomaly checks, but its sandbagging evidence comes from models directly told to act cautious.

3 min readRead story

Benchmark Tools

Aug 22, 2026

LinkedIn Measures AI Code Review Against Merged Code

The company’s reported acceptance rate is a useful outcome measure, but the more transferable design is a controllable review pipeline: local rules, pre-posting filters and infrastructure built for monitoring.

2 min readRead story

Benchmark Models

Aug 22, 2026

Stanford Won Databricks’ Agent Cup, but 18.8% of Questions Stumped Every Team

The live contest tested whether agents developed on one benchmark could handle an unfamiliar Treasury archive. Stanford led the field, but the unsolved questions show the limits of reliable document reasoning.

2 min readRead story