Nvidia ships Nemotron 3.5 Lightning for single-GPU AI agents

Why it matters

Nemotron 3.5 Lightning gives startups a commercially usable agent model that can run on one GPU, while advancing Nvidia's strategy of turning free model access into hardware demand.

Nvidia released Nemotron 3.5 Lightning on August 11th, giving developers an open-weight reasoning model designed to run agent workloads on a single Nvidia GPU.

The release follows a public campaign by CEO Jensen Huang, who entered Washington's fight over open AI models in late July. Huang has a direct commercial reason to defend models that developers can download and operate themselves: every locally deployed model still needs compute, and Nvidia sells the hardware and software used to provide it.

Nemotron 3.5 Lightning is built around that calculation. According to Nvidia's official model card, it contains 30 billion parameters while activating 3 billion for each token. The sparse design reduces the amount of computation required during inference compared with a dense model of the same total size.

Nvidia lists a single DGX Spark system or H100 GPU as supported deployment configurations. The model card also names the GeForce RTX 5090, H200 and GB200, with support for Ampere-generation GPUs through W4A16 kernels. That puts Nemotron 3.5 Lightning within reach of startups that want to operate an agent model on their own hardware without assembling a multi-GPU server.

Nvidia built for inference efficiency

Nemotron 3.5 Lightning uses a hybrid architecture combining Mamba-2 layers, mixture-of-experts layers and selected attention layers. It supports context windows of up to 1 million tokens, although reaching that limit in a production deployment will depend on available memory, concurrency and serving configuration.

The released checkpoint uses NVFP4, Nvidia's low-precision format for reducing the memory and compute costs of inference. Nvidia also shipped three approaches for generating multiple candidate tokens at once: Multi-Token Prediction, DFlash and a separate DSpark draft model optimized for DGX Spark and low-concurrency data center workloads.

Those deployment details matter more than the headline parameter count. Agent systems can make repeated model calls for planning, tool selection and intermediate work. A smaller number of active parameters and faster token generation can reduce the cost of those loops, particularly when the model is serving as a worker beneath a larger frontier model.

Nvidia says the model was pretrained on more than 20 trillion tokens and then tuned for code, math, science, tool calling, structured output and long-document retrieval. Its pretraining data cutoff was September 2025, while post-training data extended through May 2026.

Nvidia reported a score of 52.8 on SWE-bench Verified for the NVFP4 checkpoint and 75.57 on GPQA Diamond without tools. Those are Nvidia's measurements rather than independent results. Nvidia published its evaluation recipes through NeMo Gym, including prompts, containers and serving settings, so developers can attempt to reproduce the scores. Nvidia did not publish head-to-head results against current Llama, Qwen or DeepSeek models in the main benchmark table.

Open weights feed Nvidia's hardware business

Nemotron 3.5 Lightning is available for commercial use under the OpenMDW 1.1 license. The license permits users to work with the model materials without paying Nvidia and places no restrictions on modifying or sharing outputs. Distributors must retain the license and applicable notices. The grant can also terminate if a user initiates certain patent or copyright litigation involving the model materials.

That makes "open weight" the more precise label than the broad "open source" description used in some coverage. Nvidia provides downloadable weights, deployment code and evaluation recipes, while the model card also identifies private datasets and multiple training-data categories with undisclosed sizes.

Huang laid out the policy case in a July 24th paper, arguing that downloadable models give businesses control over deployment and reduce dependence on a single API provider. He also urged policymakers to avoid restrictions that could push open-model development outside the United States.

The commercial case is simpler. Proprietary AI vendors collect revenue when developers call their models. Nvidia can earn from the compute layer regardless of which model a developer chooses. Free weights expand the number of applications that can justify running inference, while optimization for NVFP4, TensorRT-LLM and Nvidia-specific hardware keeps that activity close to Nvidia's stack.

For builders, Nemotron 3.5 Lightning offers another deployable base for coding agents, retrieval systems and tool-using applications. Its competitive standing will depend on independent testing under real workloads. Nvidia has already made its intended role clear: a fast, commercially usable worker model that turns open-weight adoption into additional demand for GPUs.