Tokenless routes LLM requests mid-generation to cut inference costs
Tokenless founders Rohit Agarwal, Andrew Liu and Kevin W. are betting that watching several models begin a task beats choosing one upfront.
By Ryan Merket · Published
Primary source: Tokenless
Why it matters
Inference routing is becoming a valuable control point in the AI stack. Tokenless is betting that partial model output can guide routing more accurately than prompt inspection alone, while its economics depend on making that decision fast enough to offset parallel inference costs.
Rohit Agarwal, Andrew Liu and Kevin W. are building Tokenless, an LLM router that starts each request on several models, watches their early work and keeps the model that appears most likely to finish successfully. Kevin W.…
When you purchase through links in our articles, we may earn a small commission. This doesn’t affect our editorial independence.