Moonshot releases Kimi K3 report, detailing how a 2.8T open model runs

Why it matters

K3 shows that open weights alone do not make a frontier model practical. Draft models, cache systems and accelerator-scale serving now shape distribution and cost.

Zhilin Yang's Moonshot AI released the full weights and technical report for Kimi K3 on July 27th, documenting the architecture and infrastructure behind a 2.8 trillion-parameter model that activates 104 billion parameters for each token.

The release follows K3's July 16th introduction, when Moonshot AI began offering the model through Kimi, Kimi Work, Kimi Code and its API. The Beijing developer had promised to publish the weights and report by July 27th. Its public GitHub repository now includes the weights' license, model specifications and deployment instructions.

Yang has spent much of his research career working on the problem K3 is designed to attack: making language models retain and retrieve information over long sequences. Before founding Moonshot AI in 2023, he co-authored Transformer-XL and XLNet, earned his doctorate at Carnegie Mellon University, and worked at Meta AI and Google Brain. Moonshot AI co-founder Yuxin Wu previously worked on foundation models at Google Brain and created Meta's Detectron2 computer-vision library.

Meta AI researcher Zhuokai Zhao (@zhuokaiz) drew attention on July 30th to five algorithm-level choices in the report that he argued deserved closer examination. His first example exposed an important boundary around the phrase "open model": Moonshot AI released K3's main weights while keeping its own speculative-decoding draft model, Zhao wrote.

An independent implementation has already filled that gap. Inferact published an open Kimi K3 DSpark draft model, and vLLM added production support for it. Speculative decoding uses a smaller model to propose several tokens before the main model verifies them in parallel, reducing the delay users experience between generated tokens.

The architecture is built around inference cost

K3 combines two attention systems. Most layers use Kimi Delta Attention, or KDA, which carries a fixed-size recurrent state rather than preserving a conventional key-value cache that grows with every token. Periodic full-attention layers retain exact access to the wider sequence. Moonshot AI says this hybrid structure supports a context window of 1,048,576 tokens without the memory growth of an all-full-attention model.

Attention Residuals change how information moves between layers. Instead of repeatedly adding each layer's output to one accumulating residual stream, K3 learns how heavily to weight earlier residual states. The architecture is intended to retrieve useful representations from different depths rather than treating every prior contribution equally.

Moonshot AI also expanded the model's mixture-of-experts structure to 896 routed experts, while selecting 16 for each token. Stable LatentMoE performs expert computation in a narrower latent dimension to reduce weight movement and communication. A technique called Quantile Balancing assigns work using the distribution of router scores, removing a manually tuned balancing parameter used by other expert-routing approaches.

These decisions let Moonshot AI claim a roughly 2.5x improvement in scaling efficiency over Kimi K2. That figure is Moonshot AI's measurement, and it combines architecture, training and data changes rather than isolating the contribution of any single technique.

K3's apparent efficiency also has a hard infrastructure floor. Moonshot AI recommends deployments with at least 64 accelerators for high-bandwidth communication. vLLM's deployment guide says serving the model requires at least one eight-GPU B300 or GB300 NVL72 node, with 16 B200 GPUs also supported. Running K3 locally remains outside the reach of ordinary developer workstations despite the public weights.

A third party tripled single-user decoding speed

On 16 Nvidia GB300 GPUs, vLLM measured K3 at 118 tokens per second for a single user without speculative decoding. Adding Inferact's DSpark draft increased that result to 370 tokens per second, a claimed 3.14x gain. Coding and other predictable tasks averaged 4.73 accepted draft tokens per step, compared with 2.61 for higher-entropy work such as creative writing.

The test measures a specific hardware configuration and workload rather than the cost or throughput of a typical production service. It still shows why the draft model matters. Releasing a frontier-scale checkpoint gives developers access to the model's capabilities, while the surrounding cache managers, communication kernels, draft models and schedulers determine whether anyone can serve it economically.

Moonshot AI has priced that serving work aggressively. Its K3 API charges $0.30 per million cached input tokens, $3 per million uncached input tokens and $15 per million output tokens. Moonshot AI claims its disaggregated Mooncake inference system reaches cache-hit rates above 90% on coding workloads, an assertion that would make the lowest input rate the relevant one for many long-running agent sessions.

The model remains subject to limitations documented by Moonshot AI. K3 expects agent frameworks to preserve its earlier reasoning history across turns, and quality can become unstable when that history is missing. Moonshot AI also warns that K3 may take unexpected actions when instructions are ambiguous, and says its overall user experience still trails Claude Fable 5 and GPT-5.6 Sol.

The report arrives less than three months after Moonshot AI raised about $2 billion at a valuation above $20 billion in a round led by Meituan's Long-Z Investments, according to reporting by TechCrunch. That financing gives Yang the capital to keep training at trillion-parameter scale. K3's release shows the other half of the bet: Moonshot AI is using open weights to recruit infrastructure vendors and developers to absorb part of the engineering required to distribute its models.