Newsletter
Join the Community
Subscribe to our newsletter for the latest news and updates
Official SGLang launch card. Source: the NVIDIA Nemotron and SGLang team announcement.
NVIDIA Nemotron 3.5 Lightning is built for the repetitive steps that make AI agents expensive: tool calls, classification, coding edits, and multi-turn follow-through. Its weights are downloadable, SGLang supports it on day one, and only part of the model activates per token. The real question is whether that efficiency survives your workload.
This is an evidence review, not a hands-on model test. NVIDIA's public launch page establishes the release and local-AI positioning. The joint NVIDIA Nemotron and SGLang technical post documents the architecture, weights, serving commands, reasoning controls, and original PinchBench figure. An independent The Decoder report confirms the release and frames it as an efficiency-first model, but it does not independently reproduce the launch benchmarks.
NVIDIA Nemotron 3.5 Lightning is a downloadable text model designed for high-volume agent work rather than maximum general intelligence. The joint NVIDIA and SGLang release describes a 30-billion-parameter hybrid mixture-of-experts model that activates 3 billion parameters per token, supports up to a 1-million-token context, and ships in BF16 and NVFP4 formats. SGLang provides day-one serving through an OpenAI-compatible interface, plus controls that can turn reasoning on or off for each step. NVIDIA says the model was distilled from Nemotron 3 Ultra and trained for coding, tool use, instruction following, and multi-turn agent workflows. Those are vendor and partner claims, not an independent production result. The practical reason to test it is narrower: a small active footprint and adjustable reasoning could reduce repeated agent work, while downloadable weights give teams more deployment control. Builders should benchmark their own tasks before replacing a stronger general model.
The model uses a hybrid mixture-of-experts design. In plain English, the full parameter set exists in the checkpoint, but only a smaller subset participates in each token. That is why “30B total” and “3B active” can both be true. It does not mean the full model has the memory footprint of a 3B dense model.
The other change is inference choice. SGLang documents three speculative-decoding paths: multi-token prediction (MTP), DFlash, and DSpark. Each drafts multiple future tokens before the main model verifies them. The technical post recommends different paths by hardware and even notes that maximum throughput may currently come from running without speculative decoding, which is a useful warning against assuming every accelerator is automatically faster.
There are three layers. First, sparse activation limits how much of the model runs per token. Second, controllable reasoning lets an agent harness use deeper thought for planning and direct answers for routine extraction or classification. Third, speculative decoding can propose multiple tokens and verify them in parallel, trading extra draft machinery for lower generation latency.
That combination is a good fit for an agent that already separates work into steps. A planner might receive a larger reasoning budget, while a ticket classifier, JSON transformer, or tool-result summarizer could run with reasoning disabled. The model does not create that routing policy for you; your harness still has to decide which steps deserve more compute.
The easiest first test is not a chatbot. Pick one narrow agent stage with a clear pass/fail result: tool selection, structured extraction, patch classification, or a small code edit. Serve the documented BF16 or NVFP4 checkpoint through SGLang, then compare it with your current model on completed-task rate, invalid tool calls, latency, GPU memory, and total retries.
Do not treat the 1-million-token limit as a reason to stuff an entire repository into every request. Long-context support is an advertised ceiling, not proof of reliable retrieval across that window. Test the context lengths you actually use, including cache warm-up and multi-turn state.
For normal users, this release is less about a new chat personality and more about who controls the machine. Downloadable weights and local deployment targets mean a capable assistant can live closer to a workstation, edge device, or private server instead of sending every step to a hosted frontier API.
That control has a cost. The launch path is still infrastructure-heavy, hardware support varies, and quantized weights may trade quality for memory and speed. If you want a polished consumer app, wait for integrations. If you already run local models or build agent backends, this is worth a bounded test now.
SGLang's original PinchBench figure places Nemotron 3.5 Lightning near Qwen3.6-35B accuracy while reporting roughly 30% less H100 time to complete 10,000 tasks. The same post says the model can deliver up to four times the throughput of similarly sized open models. Both are vendor/partner-reported results from the launch setup, not independent measurements.
Original PinchBench accuracy-versus-H100-time figure from the NVIDIA Nemotron and SGLang technical post. The chart is vendor/partner-reported, point-in-time, and was not independently reproduced for this article.
The chart is useful evidence of what NVIDIA optimized for: task throughput at acceptable agent accuracy, not a universal “smartest model” claim. It does not tell you how the model behaves on a different GPU, a longer context, a tool schema, or your retry policy. The independent report reaches a similar high-level conclusion—speed over maximum intelligence—but its cited comparison is also not a substitute for your production harness.
My decision rule: test Nemotron 3.5 Lightning now if you already operate NVIDIA inference and have high-volume, measurable agent stages. Wait if you need CPU-first deployment, a polished end-user app, independently reproduced long-context quality, or a universal replacement for a frontier model.
The model is interesting because it makes agent efficiency configurable. It is not interesting merely because another benchmark dot moved left.
What is NVIDIA Nemotron 3.5 Lightning? It is a downloadable 30B hybrid mixture-of-experts text model with 3B active parameters per token. NVIDIA and SGLang position it for coding, tool use, instruction following, and high-volume agent workflows rather than as the strongest general model for every task.
Is Nemotron 3.5 Lightning available now? Yes. The August 11, 2026 technical post links BF16 and NVFP4 checkpoints and a SGLang cookbook. Availability does not guarantee that your hardware fits the model, so check the current checkpoint files, runtime support, and memory requirements before downloading.
Can Nemotron 3.5 Lightning run locally? NVIDIA lists local and edge deployment targets including DGX Spark, RTX systems, and Jetson, alongside datacenter GPUs. “Local” still means suitable NVIDIA hardware and a supported runtime; it should not be read as a claim that an ordinary laptop can run the full checkpoint well.
How should developers test Nemotron 3.5 Lightning? Start with one measurable agent stage and compare completed-task rate, schema validity, tool-call success, latency, GPU memory, and retries against your current model. Test reasoning-on and reasoning-off modes separately, then evaluate speculative decoding only after establishing a stable baseline.
What is still unverified about Nemotron 3.5 Lightning? The launch benchmark has not been independently reproduced in this review. Real performance across other GPUs, longer contexts, quantization formats, and production tool loops remains workload-specific. Treat the published chart as a testable launch claim, not a standing guarantee.
Discover practical AI products and emerging tools at AIToolHunt.
