Superpower Daily
OpenAI Extends GPT-6 Astra to Workplace Users With Tool-Use Mon...Microsoft Opens MAI-Image-2.6 in Foundry Preview and Adds a Low...Databricks Says Proteus Generated Qwen Kernels Up to 5.2× Faste...Lutnick Ties G20 AI Infrastructure Pitch to U.S. Tariff ReliefGitHub Launches HydraFusion, Citing 67% Lower Cost on Terminal-...xAI Loses Bid to Halt Minnesota’s AI Fake-Nude Law During Court...OpenAI Extends GPT-6 Astra to Workplace Users With Tool-Use Mon...Microsoft Opens MAI-Image-2.6 in Foundry Preview and Adds a Low...Databricks Says Proteus Generated Qwen Kernels Up to 5.2× Faste...Lutnick Ties G20 AI Infrastructure Pitch to U.S. Tariff ReliefGitHub Launches HydraFusion, Citing 67% Lower Cost on Terminal-...xAI Loses Bid to Halt Minnesota’s AI Fake-Nude Law During Court...
Toolspublished

Databricks Says Proteus Generated Qwen Kernels Up to 5.2× Faster Than vLLM

Proteus specializes GPU code for particular runtime shapes rather than seeking one universal kernel. Databricks’ results highlight both the potential of that approach and the validation work needed to trust it.

By 3 min read
Databricks Says Proteus Generated Qwen Kernels Up to 5.2× Faster Than vLLM
Databricks Says Proteus Generated Qwen Kernels Up to 5.2× Faster Than vLLM

Listen to this story

The audio brief

About 1:41
0:001:41
Read transcript
Databricks says its Proteus system generated GPU kernels that ran up to 5.2 times faster than the best available vLLM implementations in tests on Qwen 3.5, a 122-billion-parameter model. The important detail is that Proteus is not trying to find one universal kernel. It specializes code for the exact runtime shape of an operation—combining fixed model settings with conditions such as how many tokens a request is handling. That makes the result powerful, but narrow. In one Gated DeltaNet packed-decode case on NVIDIA B200 GPUs, using the Triton backend, Databricks measured reference latency of 0.025 milliseconds. The best candidate came in at 0.018 milliseconds, or a 1.6-times speedup. But that gain applied specifically to batch four, key 128, value 128. It is evidence for that serving layout, not proof that the same kernel will improve every request. Proteus also puts verification ahead of raw code generation. A candidate must match a controlled reference before it can be benchmarked. The system uses consistent timing, clears leftover compiled state, holds back test sizes, and flags results that exceed plausible hardware limits. That validation comes with a practical cost. In one long run, retrieving and routing memory consumed most of the model’s tokens, rather than writing kernels. Databricks also recorded C++ generation attempts that failed during generation or build. So the result to watch is not just the fastest kernel, but whether specialized gains can be found, verified, and searched efficiently across more shapes.

Story brief

3 key points

Databricks’ Proteus specializes GPU kernels for exact inference shapes instead of seeking one broadly reusable implementation. In company-reported Qwen 3.5 122B tests, it reached 1.8–5.2× the performance of the best vLLM implementations, but those gains are conditional. A B200 case study lowered Gated DeltaNet latency from 0.025 ms to 0.018 ms—1.6× for one layout. Proteus validates candidates against a controlled...

  1. 01

    The 1.6× Gated DeltaNet gain applied specifically to Batch=4, Key=128, Value=128 packed decode.

  2. 02

    Proteus rejects optimization candidates unless outputs match a controlled reference before benchmarking.

  3. 03

    Databricks found memory retrieval and routing consumed most tokens in one long optimization run.

Databricks has described Proteus, an agent-driven system for producing specialized GPU kernels, with company-reported Qwen 3.5 122B results of 1.8–5.2× faster than the best implementations available in vLLM. Its defining feature is not merely automated code generation: candidates must match a controlled reference implementation before they are benchmarked.

Conventional inference systems use generic kernels across varied workloads. Proteus instead specializes a kernel to the runtime shape of an operation, which combines fixed model parameters with request-time conditions such as token count. The trade is clear: a local optimization can improve one layout without serving as a replacement for every shape.

A narrow kernel versus broad coverage

The Gated DeltaNet packed-decode case study shows that distinction. On NVIDIA B200 GPUs with a Triton backend, Databricks measured a 0.025 ms reference latency and a lowest candidate latency of 0.018 ms. Its best speedup, 1.6×, applied to a Batch=4, Key=128, Value=128 serving-decode layout.

Packed-decode case study
0.025 msReference latency

Databricks measured the Gated DeltaNet packed-decode reference on NVIDIA B200 GPUs using Triton.

0.018 msLowest candidate latency

The lowest measured candidate latency came from the same company case study.

1.6×Best shape-specific speedup

The result applied to the stated Batch=4, Key=128, Value=128 layout.

That serving fragment processed the value dimension in 64-wide chunks. Databricks presents it as a result attached to its specific shape, not evidence that one kernel improves all requests.

Diagram showing a packed-decode baseline and subsequent shape-specific kernel candidates.
The case-study trace distinguishes measured candidates, shape-specific gains, and later failed C++ generation attempts. Source: databricks.com.

The benchmark is part of the product

Proteus generates candidates, validates them against a controlled reference, benchmarks successful ones, and iterates from measured results. Its checks use consistent timing, clear leftover compiled state, hold back test sizes, and flag speedups that exceed theoretical GPU bandwidth or compute limits.

Those controls address ways a candidate may appear faster without a fair comparison, including reuse of prior compiled work or tuning only to visible inputs. Databricks’ design shifts the constraint from generating many drafts to establishing which kernels can be trusted.

Memory can slow the search loop

Databricks found that, in one long run, most model tokens went to retrieving and routing memory rather than writing kernels. Its proposed remedy is a smaller knowledge layer: concise, actionable lessons retrieved through hierarchical tag filtering plus keyword and semantic search.

The case study also records an unfinished path: later C++ kernel-generation attempts hit build and generation failures before their branch exhausted its attempt budget. The fastest result is therefore only one part of the system’s output; failed paths and verified limits remain part of the optimization record.

Sources

  1. databricks.comAchieving Extreme Efficiency through Specialized GPU Kernel Generation

Related coverage

Continue reading