Archsloth fixes two AutoRound flags, says Qwen3-4B drifts less

Why it matters

Local-model distributors can ship files with the same size and format that preserve materially different behavior. Quantization recipes and calibration data deserve the same scrutiny as model weights and benchmarks.

The contributors behind Archsloth published a correction on September 14th to a quantization workflow they had used for months, saying two changed AutoRound settings produced a version of Alibaba's Qwen3-4B that stayed closer to the original model without adding a single byte.

The work came from SeaWolf-AI, writing through the FINAL-Bench community article on Hugging Face. Archsloth publishes quantized model files under the Archsloth name.

That approach matters here because Archsloth is making a narrow, testable claim. Its Qwen3-4B GGUF repository contains the calibration corpus, evaluation texts and per-run logs used for the comparison. The project also publishes the commands needed to download the competing file, convert the original BF16 weights and rerun its KL-divergence measurements.

The same format concealed a different rounding process

Quantization compresses a model's weights into lower-precision values, reducing the memory and storage required for local inference. Developers commonly choose files by model name, quantization label and size. Archsloth's test shows why those three labels can conceal a consequential part of the build process.

Archsloth compared its Q4_K_M file with Unsloth's Qwen3-4B Q4_K_M repository. Both files were exactly 2,497,280,800 bytes. Both used the same architecture and the same tensor map: 216 Q4_K tensors, 37 Q6_K tensors and 145 F32 tensors. Both ran through stock llama.cpp.

The integer selected for each rounded weight was the reported difference.

Archsloth measured that difference using KL divergence from Alibaba's original BF16 model. Lower divergence means the compressed model's output distribution remained closer to the reference model on the evaluation text. Across 10 axes covering seven writing systems, Archsloth reported lower divergence ranging from 20.9% for Hindi to 54.4% for Korean. The reported reductions included 52.9% for source code, 33.7% for Japanese, 33.2% for English and 31.1% for Chinese.

Three axes - Korean, English and source code - appeared in Archsloth's calibration set. The remaining seven served as checks against buying an improvement in the calibrated languages at the expense of others. The published results did not show that pattern on the 4B test.

These remain Archsloth's measurements. KL divergence captures how closely a quantized model preserves the reference model's output distribution. It does not establish a matching gain in downstream task accuracy, safety or application quality. Archsloth reported essentially tied inference speed, which fits the identical file size and tensor layout.

Two settings survived because nothing crashed

The first error sat in the quantization scheme. Archsloth had passed W4A16 to AutoRound while exporting the result as gguf:q4_k_m. W4A16 and GGUF Q4_K_M use different grouping, scaling and zero-point rules. The search therefore optimized rounding decisions for one quantizer before the export process rebuilt them for another.

The correction changed the scheme to GGUF:Q4_K_M, matching the format used at export. Intel's AutoRound documentation lists W4A16 and GGUF:Q4_K_M as separate supported schemes.

The second change enabled --enable_alg_ext, an experimental AutoRound option that is disabled by default. Archsloth linked the setting to SignRoundV2, a sign-gradient quantization method published by Intel researchers. The project said the option increased tuning time for the 4B model from about six minutes to 11 minutes on one H100.

The failure mode was mundane enough to persist. AutoRound produced a valid file, its optimization loss declined and llama.cpp could run the result. A working artifact gave the maintainers little reason to suspect that much of the rounding search was being discarded during export.

For developers building deployment tooling, that is the useful part of Archsloth's correction. A successful conversion only proves that a file can be loaded. It says little about whether the optimization procedure actually targeted the format that shipped.

Calibration text became a product decision

Archsloth's more consequential finding concerned the text used during quantization. AutoRound chooses whether individual weights round up or down by minimizing reconstruction error on calibration activations. The calibration corpus therefore participates directly in those choices.

In a controlled Qwen3.5-9B comparison, Archsloth reported a Korean mean KL divergence of 0.019255 with interleaved Korean and English calibration samples, compared with 0.027336 using English alone. That amounted to a reported 29.6% reduction from changing the text while holding the model, code, settings, bit widths and byte count constant. A 27B experiment produced a reported 31.9% difference.

Sample order also mattered. Archsloth said alternating Korean and English samples improved the English result by 11.4% compared with grouping the languages, despite keeping the overall character mix constant. Adding source code improved the code axis by 52.9%, while Korean and English results deteriorated by 3.4% and 5.3%, respectively. Archsloth accepted and documented that trade.

The project also published failed experiments. Four times as many calibration samples increased compute cost without moving the result. Quantization-aware healing worsened the measured divergence by 57% after the corrected rounding search was enabled. Per-layer bit allocation ranged from no measurable gain to a 10% deterioration.

Those results deepen a pattern RuntimeWire covered in Bartowski's work on per-tensor GGUF maps on September 11th. Local model packaging is becoming an engineering discipline of its own. Tensor-level bit choices, calibration samples, optimization settings and evaluation methods can alter the artifact even when the model card presents the same familiar quantization label.

Alibaba released Qwen3 on April 29th, 2025, with dense and mixture-of-experts models intended for open deployment. The work happening downstream now determines which version many users actually run on laptops, workstations and local servers. Archsloth's contribution is a reminder that model distribution does not end when the original weights reach Hugging Face. The last few lines of the build command can still decide how much of the model survives the trip.