Kog says it hit 3,000 tokens/s per request on standard GPUs
Kog opened a live coding playground and argues single-request decode speed, not FLOPS, is the bottleneck that matters for autonomous agents.
By Ryan Merket · Published
Primary source: Kog
Why it matters
If agents are gated by sequential token generation, the product frontier shifts from bigger models to faster loops. Hitting 3,000 tokens/s on commodity GPUs could unlock real-time agent UX without new silicon.
Kog says it has reached 3,000 tokens per second per request on standard datacenter GPUs with a latency-first inference stack, unveiling the results in a blog post and a live demo in the Kog live coding playground.…