vLLM shrinks a 970 MB output head to 300 MB and gains 12% on Strix Halo
Xiao Yu's merged patch compresses Qwen3.6's 16-bit lm_head at load, cuts median decode time by 2.82 ms, and publishes the perplexity cost beside the gain.
By Ryan Merket · Published

Xiao Yu's merged patch compresses Qwen3.6's 16-bit lm_head at load, cuts median decode time by 2.82 ms, and publishes the perplexity cost beside the gain.