Full archive

Latest AI News

Search reporting on AI markets, models, products, companies, policy, and culture.

Showing 1-18 of 19Results matching "vLLM"

Today

2 stories

1 hour agoTools

Research Tools

Databricks Says Proteus Generated Qwen Kernels Up to 5.2× Faster Than vLLM

Proteus specializes GPU code for particular runtime shapes rather than seeking one universal kernel. Databricks’ results highlight both the potential of that approach and the validation work needed to trust it.

3 min

3 hours agoStartups

Funding Startups

Resect AI Raises $25M to Try Changing Enterprise Models Before Bad Answers Reach Users

The startup has public fact-checking models and an ambitious runtime product pitch. Its next test is showing that model-level intervention can improve reliability without disrupting useful answers.

3 min

Yesterday

2 stories

YesterdayModels

Launch Models

IFM Releases Six K2 Horizon Models With Weights, Code and Training Data

The new lineup gives developers models from 0.9B to 375B parameters under Apache 2.0, while putting IFM’s performance claims and unusually broad training record in public view.

3 min

YesterdayProducts

Launch Products

NVIDIA Adds PAIR to Spread Local AI Jobs Across Nearby PCs

The free beta is designed to keep separate agent tasks from queuing behind one GPU, while its utility depends on having compatible machines with spare local capacity.

3 min

Earlier this week

5 stories

YesterdayInfrastructure

Benchmark Infrastructure

InferenceX Puts GB300 First for MiniMax M2.7, but Only on a Fixed Chat Test

The new ranking gives infrastructure teams a comparable speed and cost snapshot, but its single-turn design does not test the multi-step agent work MiniMax highlights for M2.7.

2 min

YesterdayInfrastructure

Enterprise adoption Infrastructure

Enterprise Buyers Put Non-Nvidia Chip Evaluations Ahead of Nvidia’s Next GPUs

The July survey signals a broader search across AWS, Google, AMD, Intel and custom hardware. It does not show a migration away from Nvidia, but it does show buyers weighing performance, reliability and integration at the workload level.

3 min

Sep 1, 2026Infrastructure

Research Infrastructure

Baseten’s New Inference Guide Separates Configuration Tradeoffs From True Efficiency Gains

The guide’s useful test for infrastructure teams is whether a change merely reallocates latency, throughput and cost—or reduces the work required to serve every token.

3 min

Aug 31, 2026Models

Open release Models

DeepSeek Releases 305B Vision Weights Under MIT License, but Local Use Needs Serious Hardware

The release gives developers the model files and serving references needed to run or adapt DeepSeek-V4-Flash-Vision-Exp independently. Its 168GB size means that freedom is aimed more at infrastructure-equipped teams than typical individual users.

3 min

Aug 31, 2026Infrastructure

Launch Infrastructure

Broadcom Validates Five AI Model Families for VMware’s On-Premises Model Service

The update gives enterprises a path to run models from NVIDIA, Google, NEC, Alibaba Cloud and Z.ai inside their own infrastructure, with new sharing, routing and code-isolation services around them.

2 min

Last week

6 stories

Aug 28, 2026Tools

Open release Tools

Intel Releases 20 Agent Skills for Arc GPUs, With an Open-Port Security Catch

The open repository turns Intel’s GPU deployment steps into reusable instructions for coding agents, but several model-server launchers bind broadly without authentication by default.

3 min

Aug 27, 2026Models

Research Models

Netflix’s GenRec Pairs a 0.006% Gain With Catalog-Bound LLM Ranking

The online result is statistically significant but too opaque to measure as a product outcome. The clearer contribution is a recommendation architecture that constrains an LLM to available titles while managing inference cost.

3 min

Aug 27, 2026Infrastructure

Open release Infrastructure

Snowflake Open-Sources Semi-Persistence With Sub-Second Single-GPU vLLM Swaps

The release treats GPU model copies as disposable while retaining weights in CPU memory, aiming to let many specialized models share costly accelerator capacity without long reload delays.

3 min

Aug 26, 2026Models

Open release Models

IBM Releases Granite 4.2 Open Weights, Reserving Agentic Training for 8B and 30B

The new family gives self-hosted deployments a choice between a smaller tool-capable model and larger variants trained for code, search and other tool-use tasks.

2 min

Aug 24, 2026Models

Open release Models

MiniMax Maps H3 to 24GB PCs and Server Stacks, but Keeps Context Layer Hosted

The index gives developers more routes to run and adapt H3, yet the layer that turns mixed media into model-ready context remains an API product—and local use is restricted in several major markets.

3 min

Aug 24, 2026Infrastructure

Benchmark Infrastructure

NVIDIA Says Vera Rubin Delivers 30x More Agentic Throughput per Megawatt, Pending Review

The preliminary comparison shifts attention from fixed prompt benchmarks toward the power required to keep long, tool-using AI sessions responsive.

3 min

August 2026

3 stories

Aug 22, 2026Tools

Benchmark Tools

SemiAnalysis Says AgentX Drove 50-Plus Upstream Fixes for AI Agents

The claimed contribution is not a faster model kernel. It is a test workload that makes state retention, routing and data movement visible—and leaves open whether its fixes generalize beyond AgentX’s replay matrix.

3 min

Aug 21, 2026Models

Benchmark Models

Open Models Are Catching the Frontier Faster. Benchmark Scores Aren’t the Whole Contest.

SemiAnalysis sees the lag from closed-model breakthroughs to open-model benchmark parity shrinking across three AI eras. But a comparable model score still leaves a harder product question: who can turn capability into dependable agent work?

4 min

Aug 18, 2026Infrastructure

Benchmark Infrastructure

LMCache Reworks the Cache Plumbing That Can Stall Long-Running Agents

The changes address a specific failure mode in long, concurrent agent sessions: reusable model state can fill a shared pool before any request makes progress. The reported gains are promising, but they come from targeted validations rather than a broad production benchmark.

3 min

Load more stories1 more next