Official SGLang launch card. Source: the NVIDIA Nemotron and SGLang team announcement.
NVIDIA Nemotron 3.5 Lightning is built for the repetitive steps that make AI agents expensive: tool calls, classification, coding edits, and multi-turn follow-through. Its weights are downloadable, SGLang supports it on day one, and only part of the model activates per token. The real question is whether that efficiency survives your workload.
Quick Navigation
- What is Nemotron 3.5 Lightning? A 30B open model with 3B active parameters
- What changed? Agent-focused distillation, controllable reasoning, and three speculative decoders
- How does it work? Sparse activation plus faster token drafting
- Why should developers care? A practical test plan for specialized agents
- What remains unclear? Hardware costs and independent production evidence
- FAQ: Five decision-focused questions
What We Know So Far
This is an evidence review, not a hands-on model test. NVIDIA's public launch page establishes the release and local-AI positioning. The joint NVIDIA Nemotron and SGLang technical post documents the architecture, weights, serving commands, reasoning controls, and original PinchBench figure. An independent The Decoder report confirms the release and frames it as an efficiency-first model, but it does not independently reproduce the launch benchmarks.
What is NVIDIA Nemotron 3.5 Lightning?
NVIDIA Nemotron 3.5 Lightning is a downloadable text model designed for high-volume agent work rather than maximum general intelligence. The joint NVIDIA and SGLang release describes a 30-billion-parameter hybrid mixture-of-experts model that activates 3 billion parameters per token, supports up to a 1-million-token context, and ships in BF16 and NVFP4 formats. SGLang provides day-one serving through an OpenAI-compatible interface, plus controls that can turn reasoning on or off for each step. NVIDIA says the model was distilled from Nemotron 3 Ultra and trained for coding, tool use, instruction following, and multi-turn agent workflows. Those are vendor and partner claims, not an independent production result. The practical reason to test it is narrower: a small active footprint and adjustable reasoning could reduce repeated agent work, while downloadable weights give teams more deployment control. Builders should benchmark their own tasks before replacing a stronger general model.
What changed with Nemotron 3.5 Lightning?
- Use a large frontier model for every agent step: New public option: Route routine steps to a 30B model with 3B active parameters; Why it matters: Sparse activation may lower repeated inference work
- Keep reasoning behavior fixed for the whole workflow: New public option: Turn reasoning off or set a token budget per request; Why it matters: Agents can spend more compute only on ambiguous steps
- Wait for serving runtimes to catch up: New public option: Use SGLang's day-one BF16 and NVFP4 paths; Why it matters: Teams can begin controlled evaluation immediately
The model uses a hybrid mixture-of-experts design. In plain English, the full parameter set exists in the checkpoint, but only a smaller subset participates in each token. That is why “30B total” and “3B active” can both be true. It does not mean the full model has the memory footprint of a 3B dense model.
The other change is inference choice. SGLang documents three speculative-decoding paths: multi-token prediction (MTP), DFlash, and DSpark. Each drafts multiple future tokens before the main model verifies them. The technical post recommends different paths by hardware and even notes that maximum throughput may currently come from running without speculative decoding, which is a useful warning against assuming every accelerator is automatically faster.
How does Nemotron 3.5 Lightning work for agents?
There are three layers. First, sparse activation limits how much of the model runs per token. Second, controllable reasoning lets an agent harness use deeper thought for planning and direct answers for routine extraction or classification. Third, speculative decoding can propose multiple tokens and verify them in parallel, trading extra draft machinery for lower generation latency.
That combination is a good fit for an agent that already separates work into steps. A planner might receive a larger reasoning budget, while a ticket classifier, JSON transformer, or tool-result summarizer could run with reasoning disabled. The model does not create that routing policy for you; your harness still has to decide which steps deserve more compute.
The developer view
The easiest first test is not a chatbot. Pick one narrow agent stage with a clear pass/fail result: tool selection, structured extraction, patch classification, or a small code edit. Serve the documented BF16 or NVFP4 checkpoint through SGLang, then compare it with your current model on completed-task rate, invalid tool calls, latency, GPU memory, and total retries.
Do not treat the 1-million-token limit as a reason to stuff an entire repository into every request. Long-context support is an advertised ceiling, not proof of reliable retrieval across that window. Test the context lengths you actually use, including cache warm-up and multi-turn state.
The product enthusiast view
For normal users, this release is less about a new chat personality and more about who controls the machine. Downloadable weights and local deployment targets mean a capable assistant can live closer to a workstation, edge device, or private server instead of sending every step to a hosted frontier API.
That control has a cost. The launch path is still infrastructure-heavy, hardware support varies, and quantized weights may trade quality for memory and speed. If you want a polished consumer app, wait for integrations. If you already run local models or build agent backends, this is worth a bounded test now.
What does the launch benchmark actually show?
SGLang's original PinchBench figure places Nemotron 3.5 Lightning near Qwen3.6-35B accuracy while reporting roughly 30% less H100 time to complete 10,000 tasks. The same post says the model can deliver up to four times the throughput of similarly sized open models. Both are vendor/partner-reported results from the launch setup, not independent measurements.
Original PinchBench accuracy-versus-H100-time figure from the NVIDIA Nemotron and SGLang technical post. The chart is vendor/partner-reported, point-in-time, and was not independently reproduced for this article.
The chart is useful evidence of what NVIDIA optimized for: task throughput at acceptable agent accuracy, not a universal “smartest model” claim. It does not tell you how the model behaves on a different GPU, a longer context, a tool schema, or your retry policy. The independent report reaches a similar high-level conclusion—speed over maximum intelligence—but its cited comparison is also not a substitute for your production harness.
Why should developers care?
- Routine agent steps get a credible open candidate — downloadable weights and SGLang support make the model testable without waiting for a managed provider. 2. Reasoning becomes a budget you can route — the harness can reserve deeper reasoning for planning and turn it down for mechanical steps. 3. The performance claim is falsifiable — PinchBench provides a launch baseline, while your own completed-task cost should decide adoption.
My decision rule: test Nemotron 3.5 Lightning now if you already operate NVIDIA inference and have high-volume, measurable agent stages. Wait if you need CPU-first deployment, a polished end-user app, independently reproduced long-context quality, or a universal replacement for a frontier model.
What remains unclear?
- The launch materials do not provide independent production measurements across varied agent harnesses.
- Hardware-specific memory, latency, and throughput can differ sharply between BF16, NVFP4, and speculative-decoding configurations.
- The practical quality of the advertised 1-million-token context is not established by the launch chart.
- “Open model” and downloadable weights do not remove the need to review the model's current license and acceptable-use terms before commercial deployment.
- Community demonstrations were not included because the available X search did not return a verifiable canonical example during this run.
Quick Take
- What is actually new?: Evidence-backed answer: An efficiency-focused Nemotron release with 3B active parameters, controllable reasoning, downloadable weights, and day-one SGLang support.
- Who can use it now?: Evidence-backed answer: Teams able to run the documented BF16 or NVFP4 checkpoints on supported NVIDIA hardware.
- What is the strongest evidence?: Evidence-backed answer: The joint technical post, serving instructions, downloadable checkpoint links, and original PinchBench figure.
- What should developers verify?: Evidence-backed answer: Completed-task quality, invalid tool calls, memory, latency, and retries on their own hardware.
- What is still unknown?: Evidence-backed answer: Independent production performance and reliable quality across the full advertised context.
The model is interesting because it makes agent efficiency configurable. It is not interesting merely because another benchmark dot moved left.
FAQ
What is NVIDIA Nemotron 3.5 Lightning? It is a downloadable 30B hybrid mixture-of-experts text model with 3B active parameters per token. NVIDIA and SGLang position it for coding, tool use, instruction following, and high-volume agent workflows rather than as the strongest general model for every task.
Is Nemotron 3.5 Lightning available now? Yes. The August 11, 2026 technical post links BF16 and NVFP4 checkpoints and a SGLang cookbook. Availability does not guarantee that your hardware fits the model, so check the current checkpoint files, runtime support, and memory requirements before downloading.
Can Nemotron 3.5 Lightning run locally? NVIDIA lists local and edge deployment targets including DGX Spark, RTX systems, and Jetson, alongside datacenter GPUs. “Local” still means suitable NVIDIA hardware and a supported runtime; it should not be read as a claim that an ordinary laptop can run the full checkpoint well.
How should developers test Nemotron 3.5 Lightning? Start with one measurable agent stage and compare completed-task rate, schema validity, tool-call success, latency, GPU memory, and retries against your current model. Test reasoning-on and reasoning-off modes separately, then evaluate speculative decoding only after establishing a stable baseline.
What is still unverified about Nemotron 3.5 Lightning? The launch benchmark has not been independently reproduced in this review. Real performance across other GPUs, longer contexts, quantization formats, and production tool loops remains workload-specific. Treat the published chart as a testable launch claim, not a standing guarantee.
Discover practical AI products and emerging tools at AIToolHunt.