Edge0 open-sources a 35B model runtime that streams weights from storage
Samuel Zeng says Edge0 runs on an iPhone in 1-2.5 GB, though the released MLX code currently supports Apple Silicon Macs.
By Ryan Merket · Published
Primary source: X
Why it matters
Edge0 shows how sparse models can trade storage traffic for sharply lower memory use, but its most consequential claim - a usable 35B iPhone model - is not yet reproducible from the released code.
Samuel Zeng (@SamuelZengML) open-sourced Edge0 on September 10th, releasing two language models and an inference framework designed to run large mixture-of-experts models without loading their full weight sets into memory.
https://x.com/SamuelZengML/status/2097861839287927139
…
When you purchase through links in our articles, we may earn a small commission. This doesn’t affect our editorial independence.