Full archive

Benchmark: Latest AI News

Search reporting on AI markets, models, products, companies, policy, and culture.

Showing 1-10 of 10Results matching "SemiAnalysis"

Today

1 story

2 hours ago

InferenceX Ranks GB200 First for MiniMax M3, While MI355X Has Lower Token Cost

The live AgentX result gives operators a comparable speed-and-price snapshot at one responsiveness target, but the ranking can shift with serving software and does not settle the wider configuration trade-offs. Infrastructure teams serving MiniMax M3 now face a clean split in InferenceX ’s latest ranking: GB200 NVL72 delivers more throughput, while MI355X carries the lower reported price per million tokens.

3 min readRead story

Earlier this week

1 story

Benchmark Infrastructure

Sep 2, 2026

InferenceX Puts GB300 First for MiniMax M2.7, but Only on a Fixed Chat Test

The new ranking gives infrastructure teams a comparable speed and cost snapshot, but its single-turn design does not test the multi-step agent work MiniMax highlights for M2.7. MiniMax pitches M2.7 for complex agent harnesses and software-engineering work. GB200 NVL72 came second at 12,515 tokens per second per GPU, a 10% gap.

2 min readRead story

Last week

2 stories

Benchmark Infrastructure

Aug 25, 2026

OpenAI’s Jalapeño Claims Up to 1.9x Efficiency Gain, but Leaves Nvidia Training Intact

The Broadcom co-developed inference processor is headed for deployment by year-end. Its published gains are meaningful, but they exclude model training and Nvidia’s Vera Rubin platform. OpenAI says Jalapeño, its custom chip co-developed with Broadcom, delivered up to 1.9 times higher throughput per kilowatt and 1.7 to 3.6 times lower end-to-end latency than Nvidia GB200 and GB300 systems in published tests.

3 min readRead story

Benchmark Infrastructure

Aug 24, 2026

NVIDIA Says Vera Rubin Delivers 30x More Agentic Throughput per Megawatt, Pending Review

The preliminary comparison shifts attention from fixed prompt benchmarks toward the power required to keep long, tool-using AI sessions responsive. NVIDIA says Vera Rubin NVL72 delivered up to 30 times the throughput per megawatt of GB300 NVL72 for an agentic AI workload at 160 output tokens per second per user. NVIDIA reports the DeepSeek V4 Pro 1.6T comparison on the AgentX workload.

3 min readRead story

August 2026

6 stories

Benchmark Tools

Aug 23, 2026

InferenceX Adds 11 Telemetry Views to AgentX, Exposing What Benchmark Curves Hide

The new exploration layer makes cache setup, bursty subagents and queue behavior inspectable, but its best-curve design means readers must still examine each point’s underlying configuration before treating a curve as a like-for-like comparison. A throughput curve can make two inference systems look nearly identical. The distinction reflects the workload AgentX is designed to measure.

3 min readRead story

Benchmark Tools

Aug 22, 2026

SemiAnalysis Says AgentX Drove 50-Plus Upstream Fixes for AI Agents

The claimed contribution is not a faster model kernel. It is a test workload that makes state retention, routing and data movement visible—and leaves open whether its fixes generalize beyond AgentX’s replay matrix. SemiAnalysis says its AgentX benchmark has helped produce more than 50 upstream pull requests across eight layers of AI inference software.

3 min readRead story

Benchmark Models

Aug 21, 2026

Open Models Are Catching the Frontier Faster. Benchmark Scores Aren’t the Whole Contest.

SemiAnalysis sees the lag from closed-model breakthroughs to open-model benchmark parity shrinking across three AI eras. But a comparable model score still leaves a harder product question: who can turn capability into dependable agent work? In the reasoning and agentic eras, open AI models reached closed-model benchmark parity in months, according to SemiAnalysis ’s cross-era analysis.

4 min readRead story

Benchmark Infrastructure

Aug 18, 2026

LMCache Reworks the Cache Plumbing That Can Stall Long-Running Agents

The changes address a specific failure mode in long, concurrent agent sessions: reusable model state can fill a shared pool before any request makes progress. The reported gains are promising, but they come from targeted validations rather than a broad production benchmark. Long-running AI agents do not just demand more model compute.

3 min readRead story

Benchmark Infrastructure

Aug 18, 2026

Nvidia Dynamo’s Agentic Speedups Target the Work Around the Model

The reported gains come from changing how serving systems route, retain and stream state under concurrency. The next challenge is whether these AgentX-specific improvements hold across production mixes and unfinished scheduling work. NVIDIA Dynamo’s agentic-serving optimization work treats the router, request plane and cache-ownership ledger as performance-critical infrastructure—not background plumbing.

4 min readRead story

Benchmark Infrastructure

Aug 18, 2026

AgentX Replays Claude Code Sessions to Test AI Serving Systems

The replay dataset is designed to measure infrastructure behind agentic coding workloads, not whether a model writes better code. Its value depends on whether synthetic traces preserve the traffic patterns operators need to serve. Long conversations, repeated context and parallel subagents create a different kind of serving workload than isolated prompts.

3 min readRead story