NVIDIA's TensorRT-LLM more than doubles DeepSeek cache capacity in one test

NVIDIA's batch-based allocator gave MLA layers 38.9 GiB of a 43 GiB DeepSeek-R1 cache reservation, up from 16.1 GiB in one reported layout.

By · Published

Glowing blue and orange cubes stream into a metallic machine beside blue-lit server racks marked “Rw.”

NVIDIA's batch-based allocator gave MLA layers 38.9 GiB of a 43 GiB DeepSeek-R1 cache reservation, up from 16.1 GiB in one reported layout.