Tokenless 在生成过程中对 LLM 请求进行路由以降低推理成本
Tokenless 创始人 Rohit Agarwal、Andrew Liu 和 Kevin W. 押注，观察多个模型开始执行一项任务胜过事先选择其中一个。
By Ryan Merket · Published
Primary source: Tokenless
为何重要
推理路由正在成为 AI 堆栈中的一个重要控制点。Tokenless 正押注于部分模型输出可以比仅通过检查提示更准确地指导路由，而其经济性取决于能否足够快地做出该决策以抵消并行推理的成本。
Rohit Agarwal, Andrew Liu and Kevin W. are building Tokenless, an LLM router that starts each request on several models, watches their early work and keeps the model that appears most likely to finish successfully. Kevin W.…
When you purchase through links in our articles, we may earn a small commission. This doesn’t affect our editorial independence.