Perplexity 发布了用于搜索嵌入和排序的三层堆栈
Ivy 准备请求，Tulip 对它们进行批处理，ROSE 运行模型；Perplexity 表示，这套定制堆栈使在线和批量嵌入更快且更便宜，但未披露生产基准。
By RuntimeWire Staff · Published
Primary source: Perplexity
为何重要
Perplexity 正在将模型服务内部化，以处理嵌入、排序、分类、评分以及大型语言模型的工作负载。此类掌控可能会降低其面向消费者的产品和开发者 API 在延迟和成本方面的开销，尽管该公司尚未发布支持这些说法的生产基准测试。
Perplexity, the AI search company co-founded by CEO Aravind Srinivas, published the architecture behind the embedding and ranking workloads that feed its search product. The three-layer internal stack is designed to prevent request preparation, batch scheduling and GPU execution from holding one another up.
When you purchase through links in our articles, we may earn a small commission. This doesn’t affect our editorial independence.