Full archive

Benchmark: Infrastructure AI News

Search reporting on AI markets, models, products, companies, policy, and culture.

Showing 1-8 of 8

Last week

5 stories

Aug 27, 2026

Databricks Says Async Save Cut 20B-Parameter PyTorch Checkpoints to 9 Seconds on 32 H100s

The AI Runtime components pair distributed recovery with local-NVMe data caching. The remaining operational test is whether training jobs save their input position as well as their model state.

3 min readRead story

Benchmark Infrastructure

Aug 27, 2026

Heidi Cuts ASR GPUs From 16 to 4 With CUDA MPS, Holding Sub-Second Latency

The production design turns spare capacity in small speech-inference requests into concurrency, but it depends on workload-specific scheduling and protections against MPS failure modes.

3 min readRead story

Benchmark Infrastructure

Aug 25, 2026

OpenAI’s Jalapeño Claims Up to 1.9x Efficiency Gain, but Leaves Nvidia Training Intact

The Broadcom co-developed inference processor is headed for deployment by year-end. Its published gains are meaningful, but they exclude model training and Nvidia’s Vera Rubin platform.

3 min readRead story

Benchmark Infrastructure

Aug 25, 2026

OpenAI’s Jalapeño Claims 1.5–1.9x More AI Work Per Watt, Faces 2027 Scale Test

The custom chip gives OpenAI an early efficiency claim against available Nvidia systems, but production qualification, broader workloads and a small initial rollout will determine whether it becomes a meaningful serving platform.

4 min readRead story

Benchmark Infrastructure

Aug 24, 2026

NVIDIA Says Vera Rubin Delivers 30x More Agentic Throughput per Megawatt, Pending Review

The preliminary comparison shifts attention from fixed prompt benchmarks toward the power required to keep long, tool-using AI sessions responsive.

3 min readRead story

August 2026

3 stories

Benchmark Infrastructure

Aug 18, 2026

LMCache Reworks the Cache Plumbing That Can Stall Long-Running Agents

The changes address a specific failure mode in long, concurrent agent sessions: reusable model state can fill a shared pool before any request makes progress. The reported gains are promising, but they come from targeted validations rather than a broad production benchmark.

3 min readRead story

Benchmark Infrastructure

Aug 18, 2026

Nvidia Dynamo’s Agentic Speedups Target the Work Around the Model

The reported gains come from changing how serving systems route, retain and stream state under concurrency. The next challenge is whether these AgentX-specific improvements hold across production mixes and unfinished scheduling work.

4 min readRead story

Benchmark Infrastructure

Aug 18, 2026

AgentX Replays Claude Code Sessions to Test AI Serving Systems

The replay dataset is designed to measure infrastructure behind agentic coding workloads, not whether a model writes better code. Its value depends on whether synthetic traces preserve the traffic patterns operators need to serve.

3 min readRead story