Hugo Vergnes trained a 3.8B model for a reported $998
The independent little-lm project processed 65.3B tokens and scored 0.384 on CORE after an earlier six-day failure.
By RuntimeWire Staff · Published
Primary source: Hugo Vergnes
Why it matters
Vergnes credits ClimbMix, Muon, FP8, vocabulary padding and rented B200 GPUs for the convergence and throughput behind his reported 3.8B-parameter run. The $998 figure is a reported run cost, not a documented all-in project cost, because the write-up does not itemize development labor, failed experiments, storage, data preparation or other overhead.
Hugo Vergnes trained a 3.848 billion-parameter language model from random weights for a reported $998 in rented GPU time. He says he wrote the project in the evenings.…
When you purchase through links in our articles, we may earn a small commission. This doesn’t affect our editorial independence.