Osaurus ships a 6.3 GB agent model for 8GB and 16GB Macs

Why it matters

Raptor 0.5 tests whether a narrowly trained, small-active model can make useful local agents practical on ordinary Macs, where memory limits matter more than headline parameter counts.

Osaurus co-founder Terence Pae (@tpae) released Raptor 0.5 on September 3rd, a small local agent model designed to remain loaded on Apple Silicon Macs with 8GB or 16GB of unified memory.

https://x.com/OsaurusAI/status/2095563382716002653

The release addresses a constraint Pae can see inside his own user base. Osaurus said in its launch thread that about 40% of its users have machines with no more than 16GB of memory, making many larger local models impractical once macOS and everyday applications take their share. That 40% figure is Osaurus's own measurement and does not describe Mac ownership overall.

Pae, previously a software engineer at Tesla and Netflix, started Osaurus after building Dinoki, a pixel-art desktop companion. He told TechCrunch in May that Dinoki users questioned why they had to buy an application and continue paying an AI provider for inference. Osaurus became Pae's attempt to move the model, memory, files and agent tools onto hardware the user already owns.

Raptor 0.5 brings that argument down to the base MacBook Air and Mac mini rather than assuming every local AI user owns a workstation with tens of gigabytes of spare memory.

An 8B model with about 1B parameters active

Raptor 0.5 is based on the MIT-licensed Ling-3.0-tiny model. It has 7.9 billion total parameters in a mixture-of-experts architecture, with about 1 billion active for each token. Osaurus lists a 6.3 GB bundle size and a 131,000-token context window.

The architecture uses 128 experts per layer, with eight routed experts and one shared expert active for each token. Eighteen of its 24 layers use Kimi Delta Attention, whose recurrent decode state stays fixed as a conversation grows. The remaining six layers use a compressed key-value cache. Osaurus estimates that a 32,000-token session requires about 0.3 GB of cache.

Osaurus also streams the session cache to SSD instead of retaining all of it in memory. The disk cache persists across application restarts, allowing a session to resume without processing its entire history again. The tradeoff is greater dependence on local storage performance and an implementation tailored closely to the Osaurus runtime.

The shipped model uses mixed-precision JANG quantization rather than applying one bit width across every tensor. Parameters judged less sensitive were compressed to five or six bits, while selected tensors remained at seven, eight or full precision.

Osaurus reports 95.8% top-1 agreement between the 6.3 GB package and the BF16 reference model on a held-out corpus. That percentage measures whether the compressed and reference models select the same next token. It is a quantization-fidelity result, rather than a general measure of reasoning or task accuracy.

Trained for the harness, not every job

Jinho Jang (@dealignai), the independent researcher behind dealign.ai and the JANG quantization work, built the fine-tune, training corpus and non-uniform quantization used for Raptor 0.5. The fine-tune contains 1,150 examples and 230,000 supervised tokens drawn from Osaurus's actual tool interface, including 81 tool schemas and 24 system-prompt variants.

That narrow training target defines Raptor 0.5's job. Osaurus positions it for reading folders, checking claims in documents against spreadsheets, creating backups before editing files, drafting emails and handling multi-step tool workflows. Osaurus explicitly says Raptor 0.5 is not intended to be a coding specialist or a research model for synthesizing many external sources.

In one launch demonstration, Raptor 0.5 opened a workbook and verified that shipping cost per order rose from $6.10 to $8.40. Another prompt instructed the model to extract five figures from a business review while making a dated backup first. A third asked it to identify a gross-margin decline caused by an unpassed packaging-cost increase and calculate the price change needed to restore the earlier margin.

Osaurus reports zero schema violations and zero parsing failures across 281 tool calls in a 40-task internal benchmark. The test focuses on whether Raptor 0.5 follows Osaurus's tool contracts and completes selected agent workflows. Osaurus also lists weak points including exact grouped database counts, byte-exact file targeting and recovery from prompts containing an incorrect file path.

The model weights and training corpus are available under the MIT license. Raptor 0.5 is also included in the model list inside the open-source Osaurus application.

Raptor 0.5 can load in other MLX runtimes, although its tool calls use Ling's XML argument format rather than a commonly autodetected JSON format. Users running it outside Osaurus need a compatible parser. That dependence exposes the central bet behind the release: small models become useful agents when they are trained for one concrete software environment instead of being asked to improvise across every harness.