Microsoft nearly doubles coding AI output speed in its RTX Spark test

Microsoft says it has nearly doubled a coding model’s output speed on an RTX Spark PC by having a smaller model draft text for it to check. In results published November 4, MAI-Code-1.1-Flash reached 51.12 tokens per second with a 120,000-token input—but fitting the system into memory required more than compressing the model.

Smaller weights leave room for the rest

MAI-Code-1.1-Flash has 137.9 billion parameters, the numerical values that encode a model’s learned behavior. It uses a mixture-of-experts design, which routes work through specialized components. Microsoft reduced its weight footprint from roughly 276 GB at 16-bit precision to a released checkpoint of 53.0 GiB.

That reduction uses quantization: storing weights at lower numerical precision. Microsoft did not compress every component equally. Experts account for about 96.6% of the parameters, making them the main target. Other components retain higher precision where needed, rather than forcing the entire model into one low-bit format.

Choosing those settings required testing, not just measuring compression. Microsoft used coding activity to identify components sensitive to reduced precision, then evaluated candidates across coding and non-coding benchmarks. More aggressive compression at individual layers did not consistently produce a better balance of quality and size.

The team developed two versions: Q2-XS at 53.0 GiB and Q2-XXS at 46.1 GiB. Their relative quality varied by benchmark, so neither represented a universally best compression recipe. Those sizes describe model weights, not the full memory needed to run them; working memory and the software runtime also need space.

Draft first, verify in batches

Compression addresses fit. To accelerate output, Microsoft trained a smaller model using the DFlash2 architecture. This draft proposes blocks of tokens—pieces of text—which the main model verifies in one pass. Called speculative decoding, the approach lets the larger model check proposed output in batches.

But the helper creates its own memory demand. Both models maintain a KV cache, stored attention information used as a sequence grows. At a 262,000-token context, an unrestricted draft cache approached 70% of the main model’s cache size. Microsoft limited four of the draft’s five layers to an 8,000-token sliding window, keeping full-context attention in the final layer.

Without retraining, that change cut reserved draft-cache memory from 5,120 MiB to 1,156 MiB, or 77.4%. Microsoft says draft acceptance and output throughput differed by no more than 2% from the full-attention version. Pass/fail results matched on 29 of 30 SWE-Bench tasks—a limited check, not proof of identical behavior.

Faster output, a smaller cut in waiting

Microsoft tested Q2-XS on RTX Spark with 128 GB of unified memory and a 64 GB GPU allocation. It compared runs with and without the draft across six input lengths, from 1,000 to 120,000 tokens. Each request generated 8,192 tokens, with three repetitions per case; the published results are medians.

The draft increased output throughput by 1.52–1.98 times across those input lengths. At the longest input, output speed nearly doubled, but total request time fell by 27%. These are Microsoft’s measurements for this specific model-and-serving setup, not a promise of equivalent gains under different conditions.

Memory remained tight: peak GPU use with the draft rose from 59.42 GB to 63.50 GB as input grew across the tested range. The released checkpoint is available through Microsoft Foundry and locally in GitHub Copilot on RTX Spark PCs. Microsoft positions that local route as a way to experiment without per-token cloud charges, while retaining cloud models for tasks needing greater capability.