Tomas Koutsky explains how Meta fits Muse Glimmer into 24 GB

Why it matters

Muse Glimmer shows how a 30B local agent can preserve a 131K-token context without allowing its attention cache to consume most device memory. Actual performance still depends on prompt-processing time and whether the serving engine implements the sliding-window cache efficiently.

Meta has released Muse Glimmer, a 30-billion-parameter multimodal agent designed to run on consumer hardware with 24 GB or 32 GB of memory. Tomas Koutsky, co-founder of AI workflow startup Byteline, examined how Meta makes room for the model, a 131,072-token context, a vision encoder and a speculative-decoding model on the same device.

Koutsky approaches the architecture as a builder of agent systems. He co-founded Byteline in 2025 to let users coordinate several AI agents through natural-language conversations. He previously led engineering at BRAHMA AI and Metaphysic.ai after engineering roles at Pipedrive and Streetbees. His Muse Glimmer analysis, published August 18, describes the model as a memory hierarchy built around a narrow key-value cache.

Meta compresses the weights below 20 GB

Meta released Muse Glimmer on August 10 under the Apache 2.0 license. Meta describes it as a dense, multimodal model for local coding, function calls, tool use and extended agent workflows without a cloud connection. RuntimeWire covered the open-weight release, including Meta's claim that its quantized configurations can run within 24 GB or 32 GB memory envelopes.

The uncompressed checkpoint would consume more than 55 GB. Meta says approximately four-bit quantization reduces the language model to less than 20 GB, leaving capacity for the key-value cache, perception encoder and DFlash speculative drafter.

Koutsky calculates that Muse Glimmer contains about 29.777 billion parameters. The 52 text Transformer blocks account for roughly 25.165 billion parameters, while the vision tower contains about 1.853 billion. He estimates total BF16 storage at 55.46 GiB, including the embeddings, untied output head and bridge between the visual and language components.

Meta tested its K-Quant-17GB configuration with the quantized DFlash drafter on M4 Max and M5 Max MacBooks and an Nvidia RTX 5090. The drafter proposes blocks of tokens that the main model verifies in parallel. Meta reports decode speedups of 3.1 times on the RTX 5090, 1.8 times on the M5 Max and 1.5 times on the M4 Max.

A narrow cache leaves room for long context

Muse Glimmer limits the memory that grows with each token. Across its 52 text layers, 39 attend to a sliding window of 2,048 tokens and 13 can inspect the full sequence. The model also uses 32 query heads with two key-value heads. Queries for the current token can be discarded, while keys and values for earlier tokens remain available during generation. Storing two key-value banks instead of 32 reduces the state accumulated for each active sequence.

Koutsky estimates an active BF16 or FP16 key-value cache of about 1.70 GiB at the full 131,072-token context. The 13 global layers account for about 1.625 GiB, with roughly 78 MiB allocated to the local layers after their windows fill.

That estimate depends on the serving implementation. Koutsky says an engine must evict or circularly reuse obsolete entries from the sliding-window layers to realize the savings. Static allocation across the full sequence would consume substantially more memory. Cache quantization, page sizes, fragmentation and runtime workspaces can also change measured usage.

Thirty-nine layers use a 2,048-token sliding window, while every fourth layer attends across the full context.

Long context still carries a compute charge

Muse Glimmer's 13 global-attention layers perform full attention during prompt processing. Their attention work grows quadratically with sequence length, even when memory-efficient kernels avoid storing the full attention matrix. Koutsky's analysis therefore separates memory capacity from processing time: the architecture can make a 131K-token context fit without making its prefill equivalent to a short prompt.

The local and global layers divide the retrieval work. Local layers preserve ordered information within bounded windows. Periodic global layers search representations that already contain nearby context. Koutsky notes that the global layers omit rotary positional embeddings, or RoPE, and match distant material primarily by content.

Order still reaches those layers through the causal mask and through representations produced by three preceding RoPE-equipped local layers. Koutsky argues that the arrangement can support content-based retrieval across long distances. Similar passages may be harder to distinguish when their local context does not provide enough separation.

Meta reports limited quantization loss

Meta says its quantization produces "minimal to no degradation on agentic tasks." Its release presents the accuracy results in a graphic comparing full precision, K-Quant-Dynamic and K-Quant-17GB configurations, without stating a single aggregate degradation figure in the accompanying text.

Meta compares Muse Glimmer with Gemma 4 31B and Qwen3.6 27B. Its evaluation methodology report describes the framework behind those comparisons. The reported results establish Meta's chosen comparison, while performance in other agent scaffolds and extended personal workflows remains outside the scope of those results.

Koutsky's teardown explains the memory arithmetic behind Meta's hardware claims. Quantization reduces the fixed weight cost, grouped-query attention narrows the persistent cache, and the local-global layer schedule limits how much history most layers retain. Prompt processing and the serving implementation still determine how the model behaves near its maximum context.