NVIDIA says new optimizations make local agents up to 1.9x faster
The llama.cpp and vLLM updates target RTX 5090, RTX PRO 6000 and dual DGX Spark systems, with support through LM Studio and Ollama.
By Ryan Merket · Published
Primary source: NVIDIA RTX Spark on X
Why it matters
Local agents make repeated inference calls, so backend gains can reduce latency and cloud usage without requiring developers to replace their existing NVIDIA hardware.
NVIDIA said Friday in a post on X that new llama.cpp and vLLM optimizations can raise local large-language-model throughput by as much as 1.9x across its Blackwell consumer, workstation and desktop AI systems.
The largest gain in NVIDIA's benchmark came from llama.cpp running Qwen3.6-35B on a GeForce RTX 5090, where throughput reached 1.9x the baseline NVIDIA used at Computex. Qwen3.6-27B reached 1.5x the same baseline on the card.…
When you purchase through links in our articles, we may earn a small commission. This doesn’t affect our editorial independence.