Vercel's Shu Ding ships a 27.5KB syntax highlighter with no language grammars

Why it matters

gpu-lexer shows how browser GPUs can replace large catalogs of language definitions with one compact model, opening a smaller deployment path for code previews and generated interfaces. Its uneven accuracy also draws a clear boundary between acceptable visual guesses and tooling that must understand code exactly.

Shu Ding (@shuding), a designer and developer at Vercel, has published gpu-lexer through Vercel Labs, an experimental 27.5KB syntax highlighter that uses WebGPU and a small classification model instead of maintaining a separate grammar for every programming language.

The project turns a familiar developer-tooling problem into a compact machine-learning task. gpu-lexer divides source code into words, whitespace, newlines and symbols, then labels those pieces as plain text, comments, strings, numbers, keywords, types, functions, constants or operators. It returns spans with their start and end positions, leaving rendering to the application.

Ding has spent much of his career working where interface design meets developer infrastructure. He earned a computer science degree from Fudan University in 2017, joined Microsoft as a software engineer, and joined ZEIT, now Vercel, in 2018. His portfolio credits him with work across Vercel's core platform, Next.js, AI SDK and v0, alongside open-source projects including SWR, Satori and Nextra.

With gpu-lexer, Ding is testing whether a browser can infer enough about unfamiliar code to make grammar files optional for basic highlighting. That is a narrower goal than parsing or understanding a program, and the distinction determines where the experiment is useful.

One model instead of hundreds of grammars

Established syntax highlighters generally ship definitions for the languages they support. Shiki, which gpu-lexer uses as its reference, relies on TextMate grammars. Prism.js and Highlight.js also depend on language-specific definitions.

gpu-lexer packages one model for every language. Ding lists the model at 41,321 parameters and 31KB when packed with six-bit integers. The complete browser bundle is 27.5KB after minification and Brotli compression.

The project homepage's bundle-size comparison shows that bundle is heavier than narrowly configured alternatives. Ding measured Sugar High at 5.4KB for six major web languages, Prism.js at 8.6KB and Highlight.js at 14.9KB. The comparison changes when broad language coverage is included: Prism.js reached 162.1KB for 297 languages, Highlight.js reached 240.4KB for 193, Shiki reached 991.5KB for 242 grammars and Starry Night reached 1.46MB for 710.

The advantage is operational simplicity. A developer does not have to identify a language before highlighting it or decide which grammar payloads to ship. The model also handles mixed-language documents, including JavaScript and CSS embedded inside HTML, Vue and Svelte files. That makes gpu-lexer an interesting fit for previews, snippets, generated artifacts and other interfaces where inputs may arrive without trustworthy language metadata.

The benchmark is useful, and narrow

gpu-lexer's benchmark ran on September 8th, 2026, using Chrome 152 on a MacBook Pro with an Apple M4 Pro, a 20-core GPU and 24GB of memory. After one warm-up, gpu-lexer processed 10 concatenated copies of three.min.js, totaling 5.56 million characters, in 471.7 milliseconds.

Sugar High took 859.4 milliseconds, Prism.js took 1.20 seconds, Highlight.js took 1.32 seconds, Starry Night took 11.1 seconds and Shiki took 30.4 seconds. Each library ran in a dedicated worker and the test excluded DOM rendering.

Those numbers came from one run, on one machine, with a large minified JavaScript input. The tools also produced different output formats: gpu-lexer and Shiki returned token data, Starry Night returned a HAST tree, while Sugar High, Prism.js and Highlight.js returned HTML. The result establishes a promising path for Ding's implementation. It does not settle performance across browsers, hardware, file sizes or normal application workloads.

The use of WebGPU is the central technical bet. The browser API gives web applications direct access to general-purpose GPU computation, allowing the classifier to run beside the interface instead of calling a server. That keeps source code local and removes network latency, while tying the library to environments with working WebGPU support.

Accuracy remains the bill that comes due

Ding is direct about the model's limits. According to the project's held-out evaluation, 12.57% of gpu-lexer's token labels differed from Shiki. The project describes that figure as agreement with Shiki rather than objective correctness, and warns that unfamiliar languages and production code could perform worse.

A second test weighted 1,069 held-out files using the top 25 languages in the GitHub Innovation Graph. In the aggregate comparison, gpu-lexer reached 90.20% agreement with Shiki after Shiki was fixed as the 100% reference. Prism.js scored 87.01%, Starry Night 85.07%, Highlight.js 84.65% and Sugar High 76.33%. Unsupported languages received zero, which rewards gpu-lexer's broad coverage, but the score still measures resemblance to one reference highlighter.

The project should therefore be read as a highlighter for presentation, not a source of program truth. Ding explicitly says gpu-lexer is not a parser, compiler, linter or security-analysis system. Probabilistic labeling is acceptable when the failure produces an oddly colored token. It is unsuitable when a mistaken classification changes program behavior or a security decision.

Ding's experiment is especially credible because it exposes the uncomfortable numbers. The homepage gives bundle sizes, benchmark hardware and browser details, identifies the held-out evaluation set, and reports aggregate agreement results. It does not disclose the full training corpus, training procedure or a complete language-by-language accuracy table, and its scores measure agreement with Shiki rather than objective accuracy. That makes the 27.5KB figure easier to evaluate and harder to mistake for a universal replacement claim.

gpu-lexer's immediate value will depend on whether developers have interfaces where breadth, local execution and package size matter more than exact grammar reproduction. Ding has built a compact demonstration that such a trade can work. The held-out disagreement rate shows how much certainty it costs.