Unsloth 表示，Dynamic 3.0 的量化团队在相同规模下比竞争对手高出 10%

Why it matters

The Han brothers are turning quantization into a distribution business: every quality gain at a fixed file size expands the hardware that can run open models locally.

Daniel Han and Michael Han are releasing Unsloth Dynamic 3.0, an update to the brothers' compression method for Qwen3.8-27B with GGUF files designed to preserve more of the original model's behavior at sizes suited to local hardware.

Daniel Han and Michael Han 正在发布 Unsloth Dynamic 3.0，这是对两兄弟用于 Qwen3.8-27B 的压缩方法的更新，采用 GGUF 文件，旨在以适合本地硬件的尺寸保留更多原模型的行为。

The release extends a technical bet the Hans have pursued since founding Unsloth in 2023: model optimization can serve as a distribution layer for open AI. Each reduction in memory and storage requirements puts a model within reach of another tier of consumer machines, giving developers an alternative to paying an inference provider whenever they want to run it.

此发布延续了 Hans 兄弟自 2023 年创立 Unsloth 以来一直在追求的一个技术押注：模型优化可以作为开放 AI 的分发层。每一次内存和存储需求的降低都会使模型进入另一类消费级机器的可运行范围，为开发者在想要运行模型时提供一种替代付费推理提供商的选择。

Daniel Han previously worked at Nvidia and says he made the t-SNE algorithm 2,000 times faster and fixed more than 20 bugs across open models including Llama, Gemma, Mistral and Phi. Michael Han handles product, design and engineering. The brothers took Unsloth through Y Combinator's Summer 2024 batch, and YC currently lists the San Francisco operation at eight people. Lightspeed identifies Daniel as CEO and Michael as co-founder.

Daniel Han 曾在 Nvidia 工作，并表示他使 t-SNE 算法快了 2000 倍，并修复了包括 Llama、Gemma、Mistral 和 Phi 在内的开源模型中的 20 多个 bug。Michael Han 负责产品、设计和工程工作。兄弟二人带领 Unsloth 参加了 Y Combinator 的 2024 年夏季班，YC 当前列出其旧金山团队人数为八人。Lightspeed 将 Daniel 标识为 CEO，Michael 为联合创始人。

A larger claim than a smaller file

比更小文件更重要的主张

Unsloth says Dynamic 3.0's Qwen3.8-27B quants deliver more than 10% better top-1 accuracy at the same disk size than competing files. That figure comes from Unsloth's own benchmarks and has not been independently reproduced.

Unsloth 表示，Dynamic 3.0 的 Qwen3.8-27B 量化文件在相同磁盘大小下，其 top-1 精度比竞品文件高出超过 10%。该数据来自 Unsloth 自己的基准测试，尚未被独立复现。

The underlying method is post-training quantization. Unsloth says it neither retrained Qwen3.8 on its calibration data nor used quantization-aware training or quantization-aware distillation. Instead, Dynamic 3.0 applies a higher-quality importance matrix, revised layer selection and additional quantization techniques after the base model has already been trained.

其基本方法是训练后量化（post-training quantization）。Unsloth 表示他们既没有在自己的校准数据上对 Qwen3.8 进行重新训练，也没有使用量化感知训练或量化感知蒸馏。相反，Dynamic 3.0 在基础模型训练完成后，应用了更高质量的重要性矩阵、修订的层选择以及额外的量化技术。

Unsloth assembled the calibration material around coding agents, chat and multilingual tasks. That choice matters because the data used to decide which weights require greater precision can shape where a compressed model holds up and where it deteriorates. A quant tuned around coding prompts may behave differently on other workloads, even when an aggregate accuracy figure looks strong.

Unsloth 围绕编码代理、聊天和多语言任务组建了校准素材。这个选择很重要，因为用于决定哪些权重需要更高精度的数据，会影响压缩模型在哪些场景下表现良好、在哪些场景下退化。以编码提示为中心调优的量化模型在其他工作负载上可能表现不同，即使汇总的准确率看起来很优秀。

Unsloth also acknowledges that its headline top-1 measurement has limits. It records whether a quantized model selects the same highest-probability next token as the BF16 reference, leaving the rest of the generated trajectory unmeasured.

Unsloth 也承认其醒目的 top-1 测量存在局限。该指标记录量化模型是否选择了与 BF16 参考模型相同的最高概率的下一个 token，但并未测量随后生成轨迹的其余部分。

Dynamic 3.0 therefore adds a company-designed test called Divergence-300 @32. Unsloth used 300 prompts drawn from Terminal-Bench 2.1, DeepSWE, Harbor, MathArena 2025-26 and a collection of non-Latin and long-document tasks. It then compared 32 tokens of greedy decoding from each quant against the BF16 model. The release also reports KL divergence, which measures how far the quantized model's output distribution moves from the higher-precision reference.

因此，Dynamic 3.0 添加了一个公司自设计的测试，称为 Divergence-300 @32。Unsloth 使用了来自 Terminal-Bench 2.1、DeepSWE、Harbor、MathArena 2025-26 以及一组非拉丁文和长文档任务的 300 条提示，然后比较了每个量化模型进行贪婪解码时的 32 个 token 与 BF16 模型的差异。该发布还报告了 KL 散度（KL divergence），用于衡量量化模型的输出分布与更高精度参考模型偏离的程度。

Those additions make the evaluation more useful than a single next-token score. The broad performance claim still rests on prompts, calibration choices and comparison settings selected by Unsloth. Developers deciding whether to replace an existing quant will need to test the files against their own applications, especially outside coding, chat and the languages represented in the calibration set.

这些补充使得评估比单一的下一个 token 分数更有用。但广泛的性能主张仍然取决于 Unsloth 所选择的提示、校准选择和比较设置。开发者在决定是否替换现有量化文件时，仍需将这些文件在自己的应用中进行测试，尤其是在编码、聊天以及校准集所涵盖的语言之外的场景。

Unsloth has published its importance matrix for community testing and variations. That gives outside developers a path to inspect the method, create fine-tunes and run evaluations beyond Unsloth's benchmark suite.

Unsloth 已经发布了其重要性矩阵供社区测试和变体使用。这为外部开发者提供了一条途径，可以检查该方法、创建微调并在 Unsloth 的基准套件之外进行评估。

The hardware trade

硬件权衡

The most aggressive file is the 6.2GB UD-IQ1_S one-bit quant. Unsloth says it is 89% smaller than the reference and retains about 72% top-1 accuracy. That reduction puts a 27-billion-parameter model within the storage and memory range of machines that could not approach the full-precision version, with a substantial quality trade attached.

最激进的文件是 6.2GB 的 UD-IQ1_S 一位量化。Unsloth 表示它比参考模型小 89%，并保留了约 72% 的 top-1 精度。这种缩减使一个 270 亿参数的模型进入了那些无法运行全精度版本的机器的存储和内存范围，但伴随的是显著的质量权衡。

The 9.83GB UD-Q2_K_XL offers a less severe compromise. Unsloth reports that it beats the next-best comparison by about 8% on top-1 accuracy and can produce a working HTML program, albeit with a JavaScript bug in the example shown in the documentation.

9.83GB 的 UD-Q2_K_XL 提供了一个不那么激烈的折衷。Unsloth 报告称其在 top-1 精度上比次优比较高约 8%，并且可以生成一个可运行的 HTML 程序，尽管文档中展示的示例存在一个 JavaScript 错误。

For quants below UD-Q2_K_XL, Unsloth removed the model's multi-token prediction module to save roughly 500MB. Users who want that capability can load a separate Q4_0 MTP module. The decision turns storage into an explicit option rather than packaging the speculative-decoding component into every smaller file.

对于比 UD-Q2_K_XL 更低的量化版本，Unsloth 为了节省约 500MB，移除了模型的多 token 预测模块。需要该功能的用户可以加载单独的 Q4_0 MTP module。该决定将存储变成了一个显式选项，而不是将猜测解码组件打包到每个较小的文件中。

The Qwen3.8-27B GGUF files work with engines including llama.cpp and Unsloth Desktop. Unsloth says the Qwen3.8 repositories recorded 5.1 million downloads in five days. That metric represents file requests rather than unique users: Hugging Face's counting rules count GGUF file requests and can count multiple files pulled during a repository clone.

Qwen3.8-27B GGUF files 可与包括 llama.cpp 和 Unsloth Desktop 在内的引擎配合使用。Unsloth 表示 Qwen3.8 的代码仓在五天内记录了 510 万次下载。该指标表示文件请求次数而非唯一用户数： Hugging Face 的计数规则 统计 GGUF 文件请求，并且在克隆仓库时可能计算多次拉取的文件。

The demand still explains the timing. Open-model releases now produce a second race among toolmakers to package weights for the machines developers already own. The Hans are positioning Unsloth between model labs and local runtimes, where file format, quantization quality and day-one availability determine which weights get tested and adopted.

这种需求仍然解释了发布时机。开源模型的发布现在会在工具提供者之间产生第二场竞赛，争相为开发者已经拥有的机器打包权重文件。Hans 兄弟将 Unsloth 定位在模型实验室与本地运行时之间，在这里文件格式、量化质量和首日可用性决定了哪些权重会被测试和采用。

RuntimeWire reported in June that Unsloth had applied the same distribution strategy to Z.ai's GLM-5.2, using GGUF compression and Unsloth Desktop to make a frontier-scale open model more accessible outside data centers. Dynamic 3.0 turns that recurring release work into a named methodology that Unsloth can apply across model families.

RuntimeWire 在六月报道，称 Unsloth 已经将相同的分发策略应用到 Z.ai 的 GLM-5.2，使用 GGUF 压缩和 Unsloth Desktop 使一个前沿规模的开源模型在数据中心之外更易获取。Dynamic 3.0 将这种反复的发布工作命名为一种方法论，Unsloth 可以在不同的模型家族之间应用此方法。

That approach gives the brothers a useful place in the open-model supply chain. Labs can publish increasingly capable weights, while hardware constraints still decide who can run them. Unsloth's credibility will depend on how well Dynamic 3.0's gains carry across models and workloads once developers begin measuring the files outside Unsloth's test suite.

这种方法为兄弟二人在开源模型供应链中提供了一个有用的位置。实验室可以发布日益强大的权重文件，而硬件限制仍决定谁能够运行这些模型。Unsloth 的信誉将取决于当开发者开始在 Unsloth 的测试套件之外测量这些文件时，Dynamic 3.0 的提升能在多大程度上跨模型和工作负载持续有效。