Mia publishes a $14,000 desktop deployment for GLM-5.2

Why it matters

Mia's stack pushes a 753-billion-parameter model onto desktop hardware, showing how community quantization and serving work can move frontier-scale inference outside cloud data centers.

Mia (@MiaAI_lab) published an open-source deployment stack on July 23rd that runs Z.ai's GLM-5.2 across three Nvidia DGX Spark computers, bringing a 753-billion-parameter model onto a desktop cluster with a 248,000-token context window.

Mia, who describes Mia's AI Lab as a local AI and large-model builder, released the configuration in a 10-post thread on X alongside the scripts and instructions on GitHub. Mia's GitHub account has 46 public repositories, including earlier deployment work for DeepSeek models and a monitoring dashboard for multi-DGX Spark clusters.

The new stack recorded roughly 15 to 19 generated tokens per second during mixed interactive use and about 20 tokens per second on structured output, according to measurements in the repository. Those figures cover warm, single-stream decoding and remain content-dependent. They are measurements from Mia's three-machine rig, rather than independently reproduced benchmarks.

The setup exposes an OpenAI-compatible API and uses an approximately 272 GB NVFP4 and AQLM hybrid checkpoint. The compression is what makes the model practical within the cluster's memory limits. Z.ai's official GLM-5.2 release contains 753 billion parameters, carries an MIT license and supports a one-million-token context window.

Mia's configuration reaches roughly one-quarter of that native context ceiling. It allocates an eight-gibibyte key-value cache on each node and caps a single session at 248,000 tokens. The repository reports a measured cache pool of 248,896 tokens, leaving enough capacity for one request at the stated maximum.

A $14,000 desktop cluster

Each DGX Spark contains 128 GB of unified memory, a GB10 Grace Blackwell chip and a ConnectX-7 network interface rated at 200 gigabits per second. Nvidia lists each unit at $4,699, putting three machines at $14,097 before cables. Nvidia's three-node configuration requires three QSFP cables, bringing the listed hardware total to about $14,397.

That price makes "at home" technically accurate while placing the setup well outside the normal consumer PC market. The trade is control: developers can keep model weights, prompts and generated code on hardware they operate, without paying a hosted inference provider for each request.

Nvidia published an official three-Spark ring-topology guide on March 19th. The guide connects the systems directly over 200-gigabit Ethernet, configures their network interfaces and establishes passwordless SSH. Mia's scripts build on that layout, using one Spark as the head node and the other two as workers.

The software remains an engineering project rather than a one-command consumer installation. Mia's repository requires Docker, Nvidia Container Toolkit, SSH access between all three nodes, at least about 280 GB of free storage per machine and a recent NCCL installation. The deployment also depends on a modified vLLM fork; stock vLLM cannot load this hybrid checkpoint with the required three-way tensor-parallel padding.

Memory is the sharpest constraint. The repository says the maximum-context configuration consumes about 120 GB of the approximately 121 GB available to workloads on each Spark and can spill into swap. Mia recommends disabling the operating system's early out-of-memory service because it can terminate Ray workers while CUDA graphs are being captured. That leaves little room for other processes and raises the operational cost of holding the advertised context window.

Context and speed pull in opposite directions

The default configuration uses GLM-5.2's built-in multi-token prediction layer for speculative decoding. That choice preserves the 248,000-token ceiling while accelerating generation by attempting several future tokens during each decoding step.

Mia also documented an optional external draft-model path called DSpark. It can reach about 20 tokens per second in the tested configuration, according to the repository, but the additional model consumes several gigabytes of memory. Mia recommends limiting DSpark deployments to roughly 100,000 to 150,000 tokens and using the built-in predictor when maximum context matters.

The result is a usable local endpoint for long coding sessions, repository analysis and agent experiments, provided the operator accepts the cost and tuning burden. Nvidia markets one DGX Spark for inference on models up to 200 billion parameters and two connected systems for models up to 405 billion parameters. Mia's work pushes three units into a larger class through aggressive quantization, a custom serving fork and a memory configuration operating close to the hardware's limit.

The release also shows where local model deployment is heading. Open weights alone do not make a frontier-scale model accessible. Quantized checkpoints, networking recipes and serving code determine whether those weights can run outside a data center. Mia has packaged those pieces into a reproducible stack, while documenting the compromises behind the headline numbers.