Unsloth says it cannot reproduce GSQ-RCO's recovery numbers and wants the methods
Daniel Han says Unsloth measured GSQ-RCO IQ1_M at 97.24% recovery against a claimed 98.7%, and wants ISTA-DASLab's benchmark protocol published.
By Ryan Merket · Published
Primary source: x.com
Why it matters
Quantized GGUF files are chosen on published recovery scores, so a public gap between ISTA-DASLab's claimed 98.7 to 100% recovery and Unsloth's measured 97.24 to 99.54% decides which files developers download for local DeepSeek and Qwen runs. The dispute also sets a practical standard: Unsloth has posted run counts, context lengths, graders and confidence intervals for its rerun, and the same disclosure from ISTA-DASLab would let any third party settle the numbers instead of picking a side.
Daniel Han (@danielhanchen), Unsloth's co-founder, said on November 2nd, 2026 that Unsloth asked the GSQ-RCO team for more detail on how its benchmarks were made, and got told to reproduce the numbers itself. In the post on X, Han reproduced comments he attributed to a now-deleted Reddit account: "Please just try to reproduce our numbers yourself. Even if your hardware doesn't allow you to run DeepSeek, you can still check our benchmarks on QFN and Qwen3.8-27B.…
When you purchase through links in our articles, we may earn a small commission. This doesn’t affect our editorial independence.