Full archive

Benchmark: Latest AI News

Search reporting on AI markets, models, products, companies, policy, and culture.

Showing 1-6 of 6Results matching "vLLM"

Earlier this week

1 story

Yesterday

InferenceX Puts GB300 First for MiniMax M2.7, but Only on a Fixed Chat Test

The new ranking gives infrastructure teams a comparable speed and cost snapshot, but its single-turn design does not test the multi-step agent work MiniMax highlights for M2.7. MiniMax pitches M2.7 for complex agent harnesses and software-engineering work. GB200 NVL72 came second at 12,515 tokens per second per GPU, a 10% gap.

2 min readRead story

Last week

1 story

Benchmark Infrastructure

Aug 24, 2026

NVIDIA Says Vera Rubin Delivers 30x More Agentic Throughput per Megawatt, Pending Review

The preliminary comparison shifts attention from fixed prompt benchmarks toward the power required to keep long, tool-using AI sessions responsive. NVIDIA says Vera Rubin NVL72 delivered up to 30 times the throughput per megawatt of GB300 NVL72 for an agentic AI workload at 160 output tokens per second per user. NVIDIA reports the DeepSeek V4 Pro 1.6T comparison on the AgentX workload.

3 min readRead story

August 2026

4 stories

Benchmark Tools

Aug 22, 2026

SemiAnalysis Says AgentX Drove 50-Plus Upstream Fixes for AI Agents

The claimed contribution is not a faster model kernel. It is a test workload that makes state retention, routing and data movement visible—and leaves open whether its fixes generalize beyond AgentX’s replay matrix. SemiAnalysis says its AgentX benchmark has helped produce more than 50 upstream pull requests across eight layers of AI inference software.

3 min readRead story

Benchmark Models

Aug 21, 2026

Open Models Are Catching the Frontier Faster. Benchmark Scores Aren’t the Whole Contest.

SemiAnalysis sees the lag from closed-model breakthroughs to open-model benchmark parity shrinking across three AI eras. But a comparable model score still leaves a harder product question: who can turn capability into dependable agent work? In the reasoning and agentic eras, open AI models reached closed-model benchmark parity in months, according to SemiAnalysis ’s cross-era analysis.

4 min readRead story

Benchmark Infrastructure

Aug 18, 2026

LMCache Reworks the Cache Plumbing That Can Stall Long-Running Agents

The changes address a specific failure mode in long, concurrent agent sessions: reusable model state can fill a shared pool before any request makes progress. The reported gains are promising, but they come from targeted validations rather than a broad production benchmark. Long-running AI agents do not just demand more model compute.

3 min readRead story

Benchmark Infrastructure

Aug 18, 2026

Nvidia Dynamo’s Agentic Speedups Target the Work Around the Model

The reported gains come from changing how serving systems route, retain and stream state under concurrency. The next challenge is whether these AgentX-specific improvements hold across production mixes and unfinished scheduling work. NVIDIA Dynamo’s agentic-serving optimization work treats the router, request plane and cache-ownership ledger as performance-critical infrastructure—not background plumbing.

4 min readRead story