Databricks Says Proteus Generated Qwen Kernels Up to 5.2× Faster Than vLLM

Databricks has described Proteus, an agent-driven system for producing specialized GPU kernels, with company-reported Qwen 3.5 122B results of 1.8–5.2× faster than the best implementations available in vLLM. Its defining feature is not merely automated code generation: candidates must match a controlled reference implementation before they are benchmarked.

Conventional inference systems use generic kernels across varied workloads. Proteus instead specializes a kernel to the runtime shape of an operation, which combines fixed model parameters with request-time conditions such as token count. The trade is clear: a local optimization can improve one layout without serving as a replacement for every shape.

A narrow kernel versus broad coverage

The Gated DeltaNet packed-decode case study shows that distinction. On NVIDIA B200 GPUs with a Triton backend, Databricks measured a 0.025 ms reference latency and a lowest candidate latency of 0.018 ms. Its best speedup, 1.6×, applied to a Batch=4, Key=128, Value=128 serving-decode layout.

0.025 msReference latency

Databricks measured the Gated DeltaNet packed-decode reference on NVIDIA B200 GPUs using Triton.

0.018 msLowest candidate latency

The lowest measured candidate latency came from the same company case study.

1.6×Best shape-specific speedup

The result applied to the stated Batch=4, Key=128, Value=128 layout.

That serving fragment processed the value dimension in 64-wide chunks. Databricks presents it as a result attached to its specific shape, not evidence that one kernel improves all requests.

The benchmark is part of the product

Proteus generates candidates, validates them against a controlled reference, benchmarks successful ones, and iterates from measured results. Its checks use consistent timing, clear leftover compiled state, hold back test sizes, and flag speedups that exceed theoretical GPU bandwidth or compute limits.

Those controls address ways a candidate may appear faster without a fair comparison, including reuse of prior compiled work or tuning only to visible inputs. Databricks’ design shifts the constraint from generating many drafts to establishing which kernels can be trusted.

Memory can slow the search loop

Databricks found that, in one long run, most model tokens went to retrieving and routing memory rather than writing kernels. Its proposed remedy is a smaller knowledge layer: concise, actionable lessons retrieved through hierarchical tag filtering plus keyword and semantic search.

The case study also records an unfinished path: later C++ kernel-generation attempts hit build and generation failures before their branch exhausted its attempt budget. The fastest result is therefore only one part of the system’s output; failed paths and verified limits remain part of the optimization record.