Charlie Ruan leads WebLLM, which runs open models locally in your browser

Why it matters

WebLLM gives developers a credible path to ship private, offline-capable AI features without paying for every generated token. Its limits also force product decisions around model size, device support and first-run downloads.

Charlie F. Ruan leads WebLLM, an open-source runtime that lets developers run language models inside a browser without an inference server. WebLLM was publicly documented by April 2023.

Ruan is WebLLM's public technical lead and a contributor to MLC-LLM, the compiler-based deployment project underneath it. Carnegie Mellon identified him in a May 2025 thesis record as a computer science master's student studying on-device LLM inference using compilers and web technologies.

That work informs WebLLM's defining choice. Ruan and the MLC contributors approached browser AI as a systems and compiler problem. Instead of asking application developers to write directly against low-level browser graphics interfaces, WebLLM compiles models and optimized GPU kernels ahead of time, then exposes them through a JavaScript package with an API patterned after OpenAI's chat-completions interface.

The browser becomes the inference server

WebLLM uses WebGPU for accelerated model operations and WebAssembly for supporting computation. According to the project's technical architecture, MLC-LLM and Apache TVM compile each supported model into quantized weights and a WebAssembly library containing WebGPU kernels. The browser downloads those artifacts, caches them locally and processes prompts on the user's device.

The practical result is a different cost and data path from a conventional AI web app. A developer can build chat, local-document search or structured-generation features without routing every prompt and token through a hosted GPU endpoint. Once the model files are available locally, WebLLM itself does not need a remote inference server to generate a response.

Ruan's contribution sits within the broader MLC community associated with Tianqi Chen, a Carnegie Mellon associate professor and NVIDIA distinguished engineer. Chen previously served as chief technologist at OctoAI, which NVIDIA acquired, and has worked on open-source systems including XGBoost, Apache TVM, Apache MXNet and MLC-LLM. His technical thesis is that compilers can translate models across hardware targets while reducing the deployment work required from developers.

WebLLM applies that thesis to web deployment. Its compiler generates WebGPU kernels ahead of time and packages them for browser applications.

A runtime developers can actually use

According to the project's official repository, WebLLM supports streaming responses, structured JSON generation, model caching, Web Workers, Service Workers and Chrome-extension examples. The repository also documents a JavaScript interface that accepts OpenAI-style messages through chat-completions methods.

The interface lowers the switching cost for teams already building around OpenAI-shaped APIs. The model still has to fit the user's hardware, and developers must handle a potentially lengthy first download. Once loaded, however, the application can preserve the interaction pattern of a hosted model while moving inference onto the client device.

The WebLLM repository had approximately 18,600 GitHub stars when checked on September 2, 2026. That dynamic figure does not establish production deployment, though it indicates sustained developer interest in browser-side inference.

Performance still belongs to the device

The research paper reports up to roughly 80% of native inference performance in its tests, including 41.1 tokens per second for Llama 3.1 8B and 71.1 tokens per second for Phi-3.5-mini on an M3 Max MacBook Pro. The paper was first posted in December 2024 and revised in April 2026.

Those figures describe specific models, quantization settings and hardware. Browser support, GPU memory and driver behavior determine what an individual user will see. Larger models can require large downloads, creating a poor first-run experience on slower connections or mobile devices. A model that fits comfortably on a high-end Mac may fail or run slowly on an older laptop.

WebLLM also competes with broader browser runtimes. Hugging Face's Transformers.js covers text, vision and audio tasks through ONNX Runtime, while Microsoft's ONNX Runtime Web supports several browser execution backends.

That focus gives Ruan a defined technical lane. Browser inference will not replace cloud models for every workload, particularly where developers need frontier-scale capabilities, centralized observability or consistent performance across devices. WebLLM's official materials document privacy-oriented applications, offline use and Chrome extensions as targets for local browser inference.

Ruan's personal site describes him as a UC Berkeley CS PhD student working on post-training and building SkyRL. A WebLLM roadmap he opened in July 2025 includes work on tool use, image and audio inputs, faster generation, crash recovery and simpler model conversion.

WebLLM's bet remains unusually concrete: the browser can serve as an AI runtime, the user's GPU can carry the inference bill, and web developers should not need to become compiler engineers to make either happen.