Perplexity publishes its three-layer stack for search embeddings and ranking
Ivy prepares requests, Tulip batches them and ROSE runs models; Perplexity says the custom stack makes online and batch embeddings faster and cheaper, without disclosing production benchmarks.
By RuntimeWire Staff · Published
Primary source: Perplexity
Why it matters
Perplexity is bringing model serving in-house for embedding, ranking, classification, scoring and large language model workloads. That control could lower latency and cost across its consumer product and developer APIs, although the company has not published production benchmarks supporting those claims.
Perplexity, the AI search company co-founded by CEO Aravind Srinivas, published the architecture behind the embedding and ranking workloads that feed its search product. The three-layer internal stack is designed to prevent request preparation, batch scheduling and GPU execution from holding one another up.
When you purchase through links in our articles, we may earn a small commission. This doesn’t affect our editorial independence.