OpenRelay launches one inference API across GPUs, TPUs and Trainium
OpenRelay says its network handles 100B tokens weekly across 22 locations while routing jobs by cost, latency and availability.
By Ryan Merket · Published
Primary source: Y Combinator
Why it matters
OpenRelay is betting that AI accelerators become interchangeable supply. Reliable routing could reduce cloud lock-in while giving hardware owners a market for unused capacity.
Earlier in August, Jaden Wang and Prashant Patel launched OpenRelay, a hardware-agnostic inference network that routes AI workloads across accelerators from Nvidia, AMD, Google and Amazon through a single endpoint.…