Benchmark: Latest AI News

Google and UNSW’s GlucoFM Beats a CGM Baseline by 4.1 PR-AUC Points, but Has No Public Checkpoint

The result suggests that separating slow glucose patterns from short-lived deviations can improve reusable wearable-data representations. For now, it is a retrospective research result rather than a clinical product or downloadable model.

Aug 26, 2026Products

Benchmark Products

Tiangong Ultra Runs 100 Meters in 8.86 Seconds, While Safe Stopping Remains Unresolved

The Beijing competition made athletic progress visible, but its practical events point to the work still needed on deceleration, dexterity and autonomous operation before broader deployment.

Aug 25, 2026Models

LLMScholarBench Tests 22 Models and Finds an Accuracy-Representation Trade-Off

The benchmark treats AI-generated expert lists as recommender systems to audit, showing that factual correctness and balanced representation cannot yet be improved together reliably.

Aug 25, 2026Business

Benchmark Business

Amazon Puts $25B Standalone Marker on Chip Business While Remaining a Top Nvidia Customer

Jassy’s hypothetical revenue benchmark does not describe a separately reported company, but it gives Amazon’s internal chip effort a new measure of scale as it both buys from and competes with Nvidia.

Aug 25, 2026Products

Benchmark Products

Beijing Humanoid Games Put Practical Work to the Test; Only 3 of 12 Finish Fire Course

The games paired spectacle with multi-step chores and emergency work, where task switching, human control and offboard computing remain central constraints.

Aug 25, 2026Infrastructure

Benchmark Infrastructure

OpenAI’s Jalapeño Claims Up to 1.9x Efficiency Gain, but Leaves Nvidia Training Intact

The Broadcom co-developed inference processor is headed for deployment by year-end. Its published gains are meaningful, but they exclude model training and Nvidia’s Vera Rubin platform.

Aug 25, 2026Products

Benchmark Products

Tiangong Ultra Runs 100m in 8.86 Seconds, Then Hits a Stopping Mat

The Beijing race shows a sharp jump in humanoid sprint performance, but its finish also exposed a basic operational constraint: Tiangong Ultra needed a thick mat to stop and caught fire after impact.

Aug 25, 2026Infrastructure

Benchmark Infrastructure

OpenAI’s Jalapeño Claims 1.5–1.9x More AI Work Per Watt, Faces 2027 Scale Test

The custom chip gives OpenAI an early efficiency claim against available Nvidia systems, but production qualification, broader workloads and a small initial rollout will determine whether it becomes a meaningful serving platform.

4 min

Aug 25, 2026Models

HiDream-O1-World Tops WBench Navi at 80.9, Betting on Persistent 3D Scenes

HiDream.ai’s new model is designed to make generated environments hold together through movement and edits. Its strongest disclosed evidence is a navigation benchmark; its larger film, robotics and production ambitions remain proposed uses.

Aug 24, 2026Models

NVIDIA Says AVO Took Claude Opus 5 From 30% to Perfect on ARC-AGI-3

The result shifts attention from training larger models to the software that gives them memory, tools and recovery loops. Its clearest limitation is equally important: the perfect score came on ARC-AGI-3’s public environments, while private sets remain the harder test.

Aug 24, 2026Infrastructure

Benchmark Infrastructure

NVIDIA Says Vera Rubin Delivers 30x More Agentic Throughput per Megawatt, Pending Review

The preliminary comparison shifts attention from fixed prompt benchmarks toward the power required to keep long, tool-using AI sessions responsive.

Aug 24, 2026Models

GPT-BERT Beats Llama 2 70B on One Grammar Test With 100 Million Words

The narrow result does not make child-scale models competitive with frontier chatbots. It does show that massive pretraining corpora do not settle every language-learning test.

August 2026

6 stories

Aug 23, 2026Tools

Benchmark Tools

InferenceX Adds 11 Telemetry Views to AgentX, Exposing What Benchmark Curves Hide

The new exploration layer makes cache setup, bursty subagents and queue behavior inspectable, but its best-curve design means readers must still examine each point’s underlying configuration before treating a curve as a like-for-like comparison.

Aug 22, 2026Tools

Benchmark Tools

SemiAnalysis Says AgentX Drove 50-Plus Upstream Fixes for AI Agents

The claimed contribution is not a faster model kernel. It is a test workload that makes state retention, routing and data movement visible—and leaves open whether its fixes generalize beyond AgentX’s replay matrix.

Aug 22, 2026Models

Inherent Says Its 27B Science Agent Beat OpenAI and Anthropic on Paper Replication

The company’s result centers on reproducing known findings, not making discoveries. Its next test is whether that training approach can move from replication to reliable new science.

Aug 22, 2026Models

AI Safety Scores Can Reward Models for Refusing Too Much

A study involving the UK AI Security Institute argues that combined benchmark scores can hide a direct trade-off between blocking harmful requests and answering harmless ones. Its proposed fixes are cheaper tests and anomaly checks, but its sandbagging evidence comes from models directly told to act cautious.

Aug 22, 2026Tools

Benchmark Tools

LinkedIn Measures AI Code Review Against Merged Code

The company’s reported acceptance rate is a useful outcome measure, but the more transferable design is a controllable review pipeline: local rules, pre-posting filters and infrastructure built for monitoring.

Aug 22, 2026Models

Stanford Won Databricks’ Agent Cup, but 18.8% of Questions Stumped Every Team

The live contest tested whether agents developed on one benchmark could handle an unfamiliar Treasury archive. Stanford led the field, but the unsolved questions show the limits of reliable document reasoning.