Full archive

Benchmark: Latest AI News

Search reporting on AI markets, models, products, companies, policy, and culture.

Showing 1-11 of 11Results matching "Figure AI"

Yesterday

2 stories

20 hours ago

LlamaIndex and Kaggle Launch a Document Benchmark for Cost and Long Files

ExtractBench publishes a broader test for enterprise document extraction, while its launch results include company-reported figures for LlamaIndex’s own product. LlamaIndex and Kaggle have launched ExtractBench, an open leaderboard for systems that turn enterprise documents into structured data. A broader extraction test ExtractBench evaluates 14 systems on 370 enterprise documents totaling 4,869 pages.

2 min readRead story

Benchmark Models

Yesterday

Anthropic’s Fable 5.1 Takes Benchmark Lead, but Its Top Setting Costs More Per Task

Artificial Analysis found Anthropic’s newest general model reached its highest measured intelligence score, while the token use required at maximum effort raised the cost of its own evaluation tasks. Anthropic ’s Claude Fable 5.1 has taken the top measured position on Artificial Analysis’ Intelligence Index, scoring 66 at its maximum effort setting. Related on X Claude Fable 5.1 is now available in Cursor!

4 min readRead story

Last week

6 stories

Benchmark Products

Aug 29, 2026

Sanctuary AI Puts Its Robot Brain on Existing Machines After a 99.5%-Plus Test

The company is pitching dexterous automation that can fit installed robot systems, but its automotive wire-plugging result has not yet become a production deployment. Sanctuary AI’s latest industrial pitch leaves its humanoid body off the factory floor. That is an expansion, not an abandonment, of Phoenix.

3 min readRead story

Benchmark Models

Aug 28, 2026

Meta’s EvoHarness-RL Takes Qwen3-8B to 96.9% on ALFWorld, Near Claude Opus 4.5

The result is a strong benchmark showing for learned state management, not yet evidence that the framework improves the long-running enterprise workflows it is designed to target. Meta AI and the University of Illinois Urbana–Champaign say their EvoHarness-RL framework let Qwen3-8B reach 96.9% average success on ALFWorld, edging Claude Opus 4.5’s reported 96.4% score. Source: venturebeat.com.

3 min readRead story

Benchmark Infrastructure

Aug 27, 2026

Databricks Says Async Save Cut 20B-Parameter PyTorch Checkpoints to 9 Seconds on 32 H100s

The AI Runtime components pair distributed recovery with local-NVMe data caching. The remaining operational test is whether training jobs save their input position as well as their model state. Large PyTorch training jobs do not just lose time when a GPU fails.

3 min readRead story

Benchmark Models

Aug 27, 2026

Databricks Says 300M Chart-JSON Pipeline Tops Four Multimodal Baselines on Answer Accuracy

The company’s test suggests chart values can become a useful retrieval layer instead of relying solely on page-image embeddings, though its comparison measures answer correctness and includes a synthetic benchmark it built. The company plans to add the extraction automatically to ai_parse_document, turning chart values into structured text without changing that tool’s interface. Source: databricks.com.

3 min readRead story

Benchmark Products

Aug 25, 2026

Tiangong Ultra Runs 100m in 8.86 Seconds, Then Hits a Stopping Mat

The Beijing race shows a sharp jump in humanoid sprint performance, but its finish also exposed a basic operational constraint: Tiangong Ultra needed a thick mat to stop and caught fire after impact. Tiangong Ultra, a humanoid robot from the Beijing Humanoid Robot Innovation Center, ran 100 metres in 8.86 seconds in a large-size division semifinal at Beijing’s World Humanoid Robot Games. High jump: 2.88 metres.

3 min readRead story

Benchmark Infrastructure

Aug 24, 2026

NVIDIA Says Vera Rubin Delivers 30x More Agentic Throughput per Megawatt, Pending Review

The preliminary comparison shifts attention from fixed prompt benchmarks toward the power required to keep long, tool-using AI sessions responsive. NVIDIA says Vera Rubin NVL72 delivered up to 30 times the throughput per megawatt of GB300 NVL72 for an agentic AI workload at 160 output tokens per second per user. NVIDIA reports the DeepSeek V4 Pro 1.6T comparison on the AgentX workload.

3 min readRead story

August 2026

3 stories

Benchmark Models

Aug 22, 2026

Stanford Won Databricks’ Agent Cup, but 18.8% of Questions Stumped Every Team

The live contest tested whether agents developed on one benchmark could handle an unfamiliar Treasury archive. Stanford led the field, but the unsolved questions show the limits of reliable document reasoning. Stanford won Databricks’ inaugural Grounded Reasoning Cup with 63.3% accuracy, but 18.8% of questions were unsolved by every team.

2 min readRead story

Benchmark Infrastructure

Aug 18, 2026

LMCache Reworks the Cache Plumbing That Can Stall Long-Running Agents

The changes address a specific failure mode in long, concurrent agent sessions: reusable model state can fill a shared pool before any request makes progress. The reported gains are promising, but they come from targeted validations rather than a broad production benchmark. Long-running AI agents do not just demand more model compute.

3 min readRead story

Benchmark Models

Aug 17, 2026

Zhipu Says GLM-5.3 Can Find Bugs Across an Exploitation Chain

The company reports strong vulnerability-discovery results on one benchmark and in 269 real-world projects. It also acknowledges weaker results elsewhere, leaving its wider standing—and the practical force of its access controls—unsettled. Zhipu is presenting GLM-5.3 as a cybersecurity model that does more than spot isolated software flaws.

4 min readRead story