SGLang publishes one-GPU Qwen3.8-27B recipes, claims 206.1 tokens per second

Why it matters

Open weights gain practical value when developers can serve them on attainable hardware. SGLang's single-GPU recipes make Qwen3.8-27B more accessible, while its project-published speed figures still require independent replication and output-quality checks.

The SGLang open-source inference project, associated with Ying Sheng, has added deployment recipes for Alibaba's Qwen3.8-27B, giving developers a documented route to run the 27-billion-parameter model with NVFP4 quantization and DFlash2 speculative decoding on a single GPU.

Support circulated around August 20, 2026, when the cookbook update and Qwen announcement appeared. The update followed Alibaba's release of the model's open weights by about five days, extending the launch from downloadable parameters into configurations that developers can attempt to reproduce on Blackwell hardware.

Qwen's post announcing the SGLang recipes

SGLang's original post about the Qwen3.8-27B cookbook update

That handoff matters because model weights alone rarely amount to a usable deployment. Sheng and SGLang's contributors are handling the less visible serving layer of the release cycle.

SGLang's January 2024 overview paired a structured language for model programs with RadixAttention, a runtime technique for reusing cached prompt prefixes. Sheng later co-founded RadixArk, a commercial entity built around SGLang, with Banghua Zhu. Zhu is publicly associated with SGLang and RadixArk.

Compressing a 27B model for one GPU

The new cookbook page provides Qwen3.8-27B deployment recipes using NVFP4 and DFlash2. Alibaba's model card describes a dense native multimodal model with 27 billion parameters, image and video support, a 262,144-token native context window and an extension path to 1 million tokens using YaRN. The weights carry an Apache 2.0 license.

The Qwen model card reports a 48.0152 score on WildClawBench; that figure does not by itself establish a ranking against other hosted systems. Hugging Face's evaluation metadata lists the model's WildClawBench overall rank as 8. The same model card reports an Agents' Last Exam score of 42.9 and a Pass@1 rate of 20.4. Those model-published and platform-listed evaluations are separate from SGLang's serving measurements.

NVFP4 stores key model operations in a four-bit floating-point format designed for NVIDIA's Blackwell generation. Cutting numerical precision can reduce the memory and compute required to serve a model, though operators still need to test the resulting checkpoint and runtime combination for output quality and stability.

The speed claims still need independent validation

DFlash2 uses a smaller draft model to propose future tokens before Qwen3.8-27B verifies them. Accepted proposals allow the target model to produce several tokens during one verification pass. Actual gains depend on the prompt, draft quality, hardware, batch size and memory configuration.

The cookbook's DFlash2 recipe uses the separate incoai/Qwen3.8-27B-DFlash2 draft checkpoint and sets --speculative-num-draft-tokens 8. That adds another checkpoint and another compatibility surface to an already hardware-specific deployment.

The cookbook's benchmark table says SGLang validated the RTX 5090 and RTX PRO 6000 configurations with an 8,192-token input, a 1,024-token output and concurrency of one. It says the DGX Spark configurations booted and served under those settings, while clarifying that the project did not take DGX Spark throughput or acceptance-length measurements in that validation.

SGLang reports 206.1 tokens per second on one RTX 5090 and 38.28 tokens per second on a DGX Spark. Those are project-published measurements, not independently replicated benchmarks. SGLang has not published acceptance-rate data or output-quality results for those measurements.

A separate SGLang issue reported severely repetitive output from an unofficial NVFP4 conversion of Qwen3.8-27B because an FP8 scaling value for the model's output head was not loaded. The report concerns a different checkpoint from the RadixArk NVFP4 model in the cookbook and does not establish a defect in the new recipe. It does show why exact checkpoint and runtime combinations matter when deploying mixed-precision models.

RadixArk's distribution strategy

RadixArk reportedly launched with a $100M seed round at a $400M post-money valuation. TechCrunch reported the financing, while RadixArk's financing release identified Accel as the lead, with Spark Capital and angel investors including Intel CEO Lip-Bu Tan and xAI co-founder Igor Babuschkin participating. The financing belongs to RadixArk, not Alibaba's Qwen project.

The funding gives Sheng and Zhu room to turn rapid model support into a distribution channel. Each cookbook entry puts SGLang between an open-model lab and developers deciding how to run its weights.

Alibaba benefits when outside inference projects make Qwen easier to deploy. Qwen is part of Alibaba's broader AI and cloud strategy, where wider model use could support demand for paid compute and related services.

Sheng's larger bet is that open models need an open serving layer to become credible building blocks. The Qwen3.8-27B recipes are a modest release measured in configuration files and project-published performance claims. RadixArk needs to repeat that work with each model cycle, translating a lab's weights into software another engineering team can run.