Berkeley researchers synthesize Mac-specific LLM engines up to 2.72x faster than tested alternatives
UC Berkeley's Sky Computing Lab used coding agents to synthesize six model-and-Mac-specific serving engines, reporting up to 2.72x faster single-user decoding.
By Ryan Merket · Published
Primary source: x.com
Why it matters
JITEngine tests whether coding agents can replace months of specialist inference engineering with a repeatable synthesis loop tuned to one model, one chip and one memory budget. If the approach generalizes beyond these six builds, the advantage in local AI shifts from owning the best general-purpose server to generating the right engine for the hardware already on a developer's desk.
Mert Cemri (@mertcemri), a UC Berkeley EECS PhD student, and six co-authors released JITEngine on November 4th, a set of LLM serving engines synthesized for specific Apple Silicon Macs and specific mixture-of-experts models. Cemri said in a thread on X that the engines run up to 2.7x faster than the fastest alternatives the team tested. The project comes from UC Berkeley's Sky Computing Lab and is released as open source, not as a commercial product.
When you purchase through links in our articles, we may earn a small commission. This doesn’t affect our editorial independence.