llama.cpp adds an x86 VNNI Q2_0 kernel and triples tested CPU throughput
Contributor Saejin Kim-Heinert's group-64 kernel was merged on November 15th, with Bonsai gains from 3.0x to 3.64x and a small perplexity drift attributed to compiler behavior.
By RuntimeWire Staff · Published

Contributor Saejin Kim-Heinert's group-64 kernel was merged on November 15th, with Bonsai gains from 3.0x to 3.64x and a small perplexity drift attributed to compiler behavior.