Announcing Nemotron 3.5 Lightning — reasoning at 1,200 tokens/sec

1,200 tokens/sec changes the agent loop

Nemotron 3.5 Lightning is available today for Enterprise on BLACKBOX AI—a 30B-parameter open reasoning model, 3B active, that NVIDIA built for the execution layer of an agent. In our preview eval it reasoned at 1,200.57 tokens per second at p50, nearly 29× Gemma 4 26B on the same prompt. Blink, and the reasoning has already started.

For agents, this speed compounds. An agent is a loop—plan, call a tool, check the result, correct—and Lightning collapses the wait at every turn of it. The model can afford to think hard on every step, and your agent still moves like a product instead of a queue.

Built for agents that work all day

Most of what an agent actually does is execution—writing code, calling tools, checking results, handing work to subagents. Lightning is shaped for that work. Four things set it apart:

30B parameters, 3B active. A sparse mixture-of-experts with a 256K context window. Distilled from Nemotron 3 Ultra. Trained for agent harnesses, including OpenClaw. Weights, data, and recipes all released openly under OpenMDW-1.1.

Built for the execution layer. More reasoning on every tool call, result check, and delegated task—without turning a single step into a wait.

Terminal-Bench 2.0: 58.4 → 69.7. We published the case for the orchestrator-executor split in June: pair a strong orchestrator with a cheaper executor, and the executor becomes the constraint. Lightning removes it. Plans route up to Ultra, execution routes down to Lightning, and every step goes to the model that wins it.

NVFP4 on NVIDIA B300. Lightning's native NVFP4 format runs on infrastructure built for it. Four-bit weights move less memory per token—hardware efficiency you feel as speed and keep as scale.

Reasoning speed you can use

That throughput came out of a task built to be unforgiving: parse ordered hour, minute, and second combinations; reject invalid input; and return only the function. No partial credit. Nemotron's validated answer passed all 12 checks.

Both models ran the same prompt at the same settings, temperature 0, on August 10, 2026—and we published the prompt so you can run it yourself.

Write a JavaScript function parseDuration(s) that parses duration strings into total seconds. It must handle: "2h", "45m", "30s", "1h30m", "1h2m3s", "90m", and combinations in h/m/s order. Invalid input (empty string, unknown units, no digits) must return null. Return only the function in a code block.

20 runs. A 108 millisecond spread.

Fast is a headline. Repeatable is a budget. Across all 20 runs, every Lightning response landed between 7.82 and 7.93 seconds—the slowest run and the fastest run were 108 milliseconds apart.

Throughput held just as tightly: 1,200.57 tokens per second at p50, 1,202.12 at p95—a tail that sits a tenth of a percent off the median. Gemma ranged from 23.57 to 107.43 tokens per second over the same prompt, and its slowest run took 7.71 seconds longer than its fastest.

That is what makes speed bankable. You can size a queue, promise a customer a response time, and set an alert threshold that means something—because the number you measured on Monday is the number you get on Friday.

We published both models' answers. Open either one and read the code it wrote:

function parseDuration(s) {
  if (s === '') return null;
  const match = s.match(/^(\d+h)?(\d+m)?(\d+s)?$/);
  if (!match) return null;
  const hours = match[1] ? parseInt(match[1]) : 0;
  const mins = match[2] ? parseInt(match[2]) : 0;
  const secs = match[3] ? parseInt(match[3]) : 0;
  return hours * 3600 + mins * 60 + secs;
}

function parseDuration(s) {
  if (!s || typeof s !== 'string') return null;

  const regex = /^(\d+h)?(\d+m)?(\d+s)?$/;
  const match = s.match(regex);
  if (!match || s === "") return null;

  let totalSeconds = 0;
  let hasValue = false;
  const units = { '1': 3600, '2': 60, '3': 1 };

  for (let i = 1; i <= 3; i++) {
    const val = match[i];
    if (val) {
      hasValue = true;
      const num = parseInt(val.slice(0, -1), 10);
      totalSeconds += num * units[i];
    }
  }
  return hasValue ? totalSeconds : null;
}

function parseDuration(s) {
  if (!s || typeof s !== 'string') return null;

  const regex = /^(\d+h)?(\d+m)?(\d+s)?$/;
  const match = s.match(regex);
  if (!match || s === "") return null;

  let totalSeconds = 0;
  let hasValue = false;
  const units = { '1': 3600, '2': 60, '3': 1 };

  for (let i = 1; i <= 3; i++) {
    if (match[i]) {
      hasValue = true;
      const value = parseInt(match[i], 10);
      totalSeconds += value * units[i];
    }
  }
  return hasValue ? totalSeconds : null;
}

function parseDuration(s) {
  if (!s || typeof s !== 'string') return null;

  const regex = /^(\d+h)?(\d+m)?(\d+s)?$/;
  const match = s.match(regex);
  if (!match || s === "") return null;

  let totalSeconds = 0;
  let hasDigits = false;
  const units = { '1': 3600, '2': 60, '3': 1 };

  for (let i = 1; i <= 3; i++) {
    const val = match[i];
    if (val) {
      hasDigits = true;
      const num = parseInt(val.slice(0, -1), 10);
      totalSeconds += num * units[i];
    }
  }
  return hasDigits ? totalSeconds : null;
}

function parseDuration(s) {
  if (!s || typeof s !== 'string') return null;

  const regex = /^(\d+h)?(\d+m)?(\d+s)?$/;
  const match = s.match(regex);
  if (!match || s === "") return null;

  let totalSeconds = 0;
  let hasValue = false;

  for (let i = 1; i <= 3; i++) {
    const part = match[i];
    if (part) {
      hasValue = true;
      const value = parseInt(part, 10);
      const unit = part.slice(-1);
      if (unit === 'h') totalSeconds += value * 3600;
      if (unit === 'm') totalSeconds += value * 60;
      if (unit === 's') totalSeconds += value;
    }
  }
  return hasValue ? totalSeconds : null;
}

Steady speed is what makes speed useful

Speed makes the agent loop fast; holding that speed run after run is what makes the loop something you can build a product on.

Size the system once. When p95 sits on top of p50, capacity planning is arithmetic instead of guesswork. Provision for the number you measured, not for the worst hour you can imagine.

Set alerts that mean something. A tight band makes a real outlier obvious, so your pager fires when something is actually wrong instead of every time a long tail wanders past the threshold.

Promise a response time and keep it. Put a number in the SLA, then put the same number in front of the customer. A ten step agent chain stays predictable because every step is predictable.

Scale without surprises. The throughput you benchmarked is the throughput you get across every customer, every queue, every shift—so the ten thousandth request costs what the first one did.

Five jobs you can hand over

Most teams do not stall on AI because it cannot do the work. They stall because they cannot predict it. You cannot put a process into production when you do not know what it will cost you in time.

The numbers above close that gap: a model fast enough to sit inside the process and steady enough to plan around.

Available for Enterprise, encrypted end-to-end

Lightning ships to Enterprise customers as a dedicated single-tenant deployment. The model runs on hardware reserved for you, and the inference path is encrypted end-to-end—from your application, to the GPU, and back.

End-to-end encrypted inference. Your prompts and your completions are encrypted the whole way through. You run the workload; nobody else reads it.

Single tenant, airgapped on request. Hardware reserved for you alone. No shared capacity, no noisy neighbors, and no route to the public internet unless you ask for one.

Zero data retention by default. Nothing you send is kept after the response returns, and nothing you send trains a model—not ours, not anyone's.

Customer-managed keys. Bring your own keys and keep them. Rotate on your schedule. Revoke on your terms.

Open weights under OpenMDW-1.1, running where only you can reach them—speed you can plan around, on infrastructure you control.

Point your SDK at it.

Your Enterprise deployment speaks the Blackbox API — OpenAI compatible — so the migration is one line: set the model to blackboxai/nvidia/nemotron-3.5-lightning. Nothing else changes.

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ.get("BLACKBOX_API_KEY"),
    base_url="https://api.blackbox.ai",
)

stream = client.chat.completions.create(
    model="blackboxai/nvidia/nemotron-3.5-lightning",
    messages=[{"role": "user", "content": "Validate this tool output."}],
    stream=True,
)

for chunk in stream:
    if not chunk.choices:
        continue
    delta = chunk.choices[0].delta.content or ""
    print(delta, end="", flush=True)

import OpenAI from 'openai';

const client = new OpenAI({
  apiKey: process.env.BLACKBOX_API_KEY,
  baseURL: 'https://api.blackbox.ai',
});

async function run() {
  const stream = await client.chat.completions.create({
    model: 'blackboxai/nvidia/nemotron-3.5-lightning',
    messages: [{ role: 'user', content: 'Validate this tool output.' }],
    stream: true,
  });

  for await (const chunk of stream) {
    const delta = chunk.choices[0]?.delta?.content || '';
    process.stdout.write(delta);
  }
}

run();

Nemotron 3.5 Lightning is built for agents that work all day, every day, at a cost that holds up at scale. Open, encrypted, and fast.