llama.cpp patch speeds up one RTX 5090 matrix multiply 4.5x

Jeff Bolz's narrow tiles cut the KHR cooperative-matrix case at M=4096, K=14336, N=16 from 325.49 to 72.36 microseconds on an RTX 5090.

By · Published

AI-generated illustration of a llama beside a laptop displaying code, with a mug and notebook on the desk and plants and books in the background.

Jeff Bolz's narrow tiles cut the KHR cooperative-matrix case at M=4096, K=14336, N=16 from 325.49 to 72.36 microseconds on an RTX 5090.