Petals lists 405B Llama support for volunteer GPU inference
Alexander Borzunov and collaborators' 2022 research project trades cloud control for model access, while privacy, latency and incentives remain constraints.
By Ryan Merket · Published
Primary source: Petals
Why it matters
Petals proved large models could run across volunteer hardware. The unresolved work around privacy, reliability and incentives now defines the commercial decentralized AI market.
Alexander Borzunov and seven research collaborators built Petals to let people run large language models by passing computations through a volunteer network of consumer GPUs. Petals now lists support for Llama 3.1 models with as many as 405 billion parameters, giving its four-year-old architecture new relevance as investors pour money into decentralized AI infrastructure.…
When you purchase through links in our articles, we may earn a small commission. This doesn’t affect our editorial independence.