Wafer says AMD's MI355X beats Nvidia B300 on Kimi K3 cost efficiency

Why it matters

Kimi K3's size turns GPU memory into a deployment constraint. Wafer's self-reported benchmark shows how software tuning and larger HBM capacity could make AMD economical for frontier-scale open models, even when Nvidia retains the raw-speed lead.

Wafer, founded by Emilio Andere (@gpuemi) and Steven Arellano (@gpusteve), says it ran Kimi K3 on eight AMD MI355X GPUs at better performance per dollar than an Nvidia B300 system, using memory capacity and a pair of software fixes to narrow AMD's inference gap.

The San Francisco AI infrastructure provider reported 952 output tokens per second in peak aggregate throughput and 118 tokens per second on a single stream. Wafer disclosed the results in a July 31st technical post written by technical staff member Ian Ye.

Wafer's absolute throughput remained below the B300 configuration, which reached 1,568 tokens per second in the same test. The economic result went the other way: Wafer calculated 48 tokens per second per dollar for the MI355X, compared with 33 for the B300 and 7 for a two-node B200 deployment.

Those figures use Wafer's assumed hourly rental prices of $2.50 per MI355X GPU, $6 for a B300 and $4.25 for a B200. Wafer produced the figures using its own test setup.

Memory changes the hardware choice

Kimi K3's technical report describes a mixture-of-experts model with 2.8 trillion total parameters, 104 billion active parameters and a 1-million-token context window. Wafer estimates that the weights require more than 1.5 terabytes of GPU memory before allocating memory for the key-value cache used during inference.

That size creates the opening Wafer is exploiting. An AMD MI355X carries 288 GB of HBM3E, giving an eight-GPU node roughly 2.3 TB of high-bandwidth memory. Wafer says the same model and 1-million-token cache would not fit on one eight-GPU B200 node, forcing its B200 test across 16 GPUs in two nodes.

The configurations matter. Wafer's B200 deployment paid the latency cost of communicating across nodes on the decode path, while the MI355X and B300 systems stayed within one node. Wafer reported 498 aggregate tokens per second across the full 16-GPU B200 deployment, or about 249 per node.

That makes the B200 comparison structurally unfavorable. It also captures the practical constraint Wafer wants customers to notice: a GPU with enough memory can avoid the inter-node coordination required by a nominally faster accelerator. As open-weight models grow, memory capacity can determine the serving architecture before raw compute enters the calculation.

Arellano wrote on X that Wafer achieved "3.8x higher throughput and 71% lower cost" on the MI355X compared with its B200 setup. The throughput multiple is measured per node, while the cost percentage follows Wafer's listed hourly rental assumptions.

Steven Arellano on X

Two small fixes produced most of the gain

Kimi K3 could run on AMD's ROCm software without a ground-up port, according to Wafer, but the initial deployment left substantial performance unused.

Wafer first added speculative decoding using RadixArk's Kimi-K3-DSpark, an external draft model designed to propose several tokens that Kimi K3 can verify together. The CUDA version worked directly. On ROCm, a missing top_k_renorm_prob definition in SGLang's accept-sampling verifier caused the scheduler to fail.

Wafer implemented the operation in PyTorch by selecting the highest-probability tokens, masking the rest and rescaling the retained values. Wafer says that fix increased single-stream performance about 2.2x, improved per-stream performance about 1.7x at moderate load and lifted peak aggregate throughput by 18%.

The second issue appeared during prefill, the stage when the model processes the user's input before producing its first output token. Wafer initially measured about 51 seconds for a 172,000-token cold prefill on the MI355X, compared with about 23 seconds on the B300.

Wafer traced most of the delay to an attention kernel shape mismatch. Kimi K3 produced 12 attention heads per rank under tensor parallelism across eight GPUs, while AMD's faster AITER path supported four, eight or multiples of 16. Wafer padded the head count from 12 to 16, ran the faster kernel and discarded the extra outputs.

The change raised steady-state prefill performance from roughly 4,000 to 7,000 tokens per second to about 13,000, according to Wafer. It improved the time before the first token appeared.

A benchmark built around Wafer's founding thesis

Andere and Arellano met during their first year at the University of Chicago, became roommates and developed Wafer around the idea that scarce performance engineers were limiting how efficiently AI hardware could be used. Andere studied mathematics, researched machine-learning security at UChicago and worked on weather models at Argonne National Laboratory. Arellano studied computer science and worked on high-performance computing and AI infrastructure at Google, Two Sigma and Sei Labs.

Wafer now pitches software agents that profile workloads and search across models, inference engines, kernels and hardware configurations. Y Combinator lists Wafer as a Summer 2025 company with seven people in San Francisco.

The MI355X work is a direct demonstration of the founders' claim that software optimization can make alternative accelerators economically competitive for demanding models. It is also a sales argument for Wafer's serverless APIs and dedicated inference deployments: customers can hand Wafer a model and workload instead of committing in advance to a particular chip vendor.

Wafer raised a $4 million seed round on April 14th, 2026, led by Fifty Years with participation from Liquid 2 and Y Combinator. The financing announcement framed Wafer's goal as maximizing "intelligence per watt" by automating work traditionally performed by a small pool of kernel and systems specialists.

Kimi K3 gives that pitch a useful stress test. Its size rewards AMD's 288 GB memory design, while the software problems Wafer encountered were narrow enough to fix without writing new custom kernels. That combination favors Wafer and AMD: AMD gets evidence that its hardware can serve a frontier-scale open model, and Wafer gets a concrete example of why inference buyers may need optimization across chip vendors.

The result does not establish the MI355X as the faster GPU. Wafer's own B300 run delivered 65% more aggregate throughput and a faster single stream. Wafer's finding is narrower and commercially relevant: under its configurations and rental prices, one MI355X node handled Kimi K3 more cheaply per unit of throughput.