Meta’s 30B Muse Glimmer Tries to Make 131K Context Fit in 24 GB

Meta’s Muse Glimmer is a 30-billion-parameter agent model built around an unusually tight local-memory target: a quantized configuration that Meta says can run in 24 GB or 32 GB while supporting a 131,072-token context window. The design puts the main constraint of long-running agents in focus: not just model weights, but the memory retained for every prior token.

Meta released Muse Glimmer on August 10 under the Apache 2.0 license. It is a dense, multimodal model intended for local coding, function calls, tool use, and extended agent workflows; its roughly 1.8-billion-parameter perception encoder handles images alongside text. The model is also designed to make sequential tool calls across turns and recover from failed calls or retries.

The memory budget begins with the weights

The uncompressed checkpoint would take more than 55 GB. Meta says approximately four-bit quantization brings the language model below 20 GB, leaving memory for the image encoder, the attention cache, and a speculative-decoding drafter. That is the first step in getting a 30B-class system into the stated hardware envelope.

Most layers do not keep the entire past

The second step is limiting the key-value cache, the stored attention state that normally grows as a conversation or tool trace lengthens. Of Muse Glimmer’s 52 text layers, 39 use a 2,048-token sliding window. Thirteen layers can attend across the full sequence, creating a recurring global layer every fourth layer.

The model also uses 32 query heads but only two key-value heads. That grouped-query arrangement reduces the state retained for each active sequence. At the full context length, an estimate puts an FP16 or BF16 cache at about 1.70 GiB: about 1.625 GiB for the global layers and roughly 78 MiB for local layers once their windows are full.

Where the apparent savings can disappear

• The serving engine must evict old entries or reuse space circularly for sliding-window layers; a cache statically allocated across the full sequence would use substantially more memory.
• Cache quantization, page sizes, fragmentation, and runtime workspaces can change measured memory use.
• DFlash speculative decoding is meant to lower generation latency by having a smaller drafter propose token blocks for the main model to verify.

A long context can fit without feeling short

The architecture reduces memory pressure; it does not make prompt processing cheap. The 13 global-attention layers still perform full attention while reading a prompt, and that work grows quadratically as the sequence grows. Memory-efficient kernels can avoid storing the full attention matrix, but they do not remove that compute cost.

The local-global split also changes how distant material is retrieved. Local layers preserve nearby ordered information, while global layers search representations shaped by prior local layers. The global layers omit rotary positional embeddings and primarily match distant content by content, which can make similar passages harder to distinguish when nearby context does not separate them.

The next test is the full agent loop

Meta tested its K-Quant-17GB configuration with the DFlash drafter on M4 Max and M5 Max MacBooks and an Nvidia RTX 5090. Meta reports decoding speedups of 1.5 times on M4 Max, 1.8 times on M5 Max, and 3.1 times on the RTX 5090. Those are decoding figures, not a measure of how quickly the system ingests a maximum-length prompt.

For agent builders, the unresolved issue is whether their serving stack realizes the sliding-cache design while maintaining useful long-context retrieval and acceptable prompt-processing time. Fireworks AI has made Muse Glimmer available in serverless and on-demand deployments, giving teams another route to test that tradeoff beyond local hardware.