- MODEL:
- FIXED
- HARDWARE:
- FIXED
- ENGINE:
- TUNABLE
The model is fixed.
The engine is not.
Most inference deployments arrive at the same problem. What is left to tune is the engine, and the right settings depend on your traffic.
- WORKLOAD:
- YOUR TRAFFIC
- TARGET:
- YOUR NUMBERS
- QUALITY:
- YOUR EVALS
You set three things.
Agents do the rest.
You give us three inputs. Agents propose changes to the engine, a fixed harness replays your workload and scores each one, and only a change that beats the current best becomes the new baseline. The loop runs until the target is reached.
Your workload
A dataset that matches your real traffic: the requests you send, their lengths, concurrency and arrival pattern. The agents tune for your traffic, not a generic benchmark.
Your targets
The numbers that matter to you: throughput, time to first token, latency, cost, or a mix of them.
Your quality bar
The evals the model must keep passing. Every change is measured against the original model, so speed never costs quality.
01 PROPOSE
Each agent proposes one change to the engine, from a flag to a kernel.
02 MEASURE
A fixed harness replays your workload and scores the change. The agents cannot modify it.
03 KEEP OR DISCARD
A change that beats the current best becomes the new baseline. The harness discards the rest.
04 SHARE
Every result goes to a shared log, so each agent starts its next try from what all of them learned.
- MODEL:
- GPT-OSS-120B
- HARDWARE:
- 1× NVIDIA B200
- HUMAN EDITS:
- 0
One GPU. 1.5× faster.
No human in the loop.
We gave 4 agents one NVIDIA B200 and the open gpt-oss-120b weights. From a strong standard recipe at about 480 tokens per second, they reached 723.2: past 2 custom inference chips, with software alone. No human touched the engine during the run.
Single-stream tokens/sec. Each step up is a change the harness kept.
| Agent run, start to latest verified result | Value |
|---|---|
| Start | ≈480 |
| Latest | 723.2 |
Single stream, 10k-token prompts. One agent-tuned GPU against GPU clouds and custom inference chips.
| Entry | Value |
|---|---|
| GPUs BLACKBOX AI 1× NVIDIA B200 · agent-tuned | 723.2 |
| GPUs Azure GPU · ours 2.0× | 361.0 |
| GPUs Nebius (Base) GPU · ours 2.0× | 357.7 |
| Custom inference chips Cerebras Wafer-scale engine · next target | 1,729.0 |
| Custom inference chips SambaNova RDU · ours +2.3% | 706.8 |
| Custom inference chips Groq LPU · ours +53% | 473.5 |
Providers: Artificial Analysis, P50 over the past 72 hours, October 16, 2026. BLACKBOX AI: best verified result from our agent run, measured on the same methodology by a fixed harness.
- HARNESS:
- READ-ONLY
- WINS:
- REPRODUCED
- KERNELS:
- HUMAN REVIEW
The judge is never an agent.
Agents propose changes. They never grade their own work, so every number on this page is one the harness measured.
- 01
Scoring is fixed
Agents submit changes to a harness they cannot modify.
- 02
Wins must reproduce
A new configuration must beat the current best when both are measured again.
- 03
Quality comes first
A faster result that fails the quality benchmarks does not count.
- 04
You stay in control
In the gpt-oss run, no human touched the engine. On your deployment, deeper engine changes go through human review before they reach production.
- AGENTS:
- 4
- EXPERIMENTS:
- 83
- LOG:
- SHARED
One shared log.
No repeated work.
Agents run the loop in parallel and read each other's results, so no two of them try the same idea. They start with settings and go deeper into the engine as the easy wins run out.
- WORKLOADS:
- YOURS
- TARGET:
- YOU SET IT
Your workload.
Your target.
The loop is the same for every workload. What changes is the target you set and the part of the engine the agents work on.
- 01
Real-time chat and agents
TARGET: TIME TO FIRST TOKEN
Short, bursty requests where every user waits on the first token. The agents tune speculative decoding, scheduling and prefill for that shape.
- 02
High-volume batch jobs
TARGET: THROUGHPUT AND COST
Large offline runs where cost per token matters more than one request. The agents push batch size, concurrency and cache layout until the GPUs stay full.
- 03
Long context and RAG
TARGET: LATENCY ON LONG PROMPTS
Prompts that carry long, shared documents. The agents tune chunked prefill, prefix caching and the KV cache for that traffic.
- 04
A new open model
TARGET: THE SPEED YOU SET
A model with no tuned recipe yet. The agents start from a standard setup and tune for its architecture, as they did for gpt-oss-120b.
- MODELS:
- OPEN WEIGHT
- START:
- CONVERSATION
Common questions.
What do you need from us?
Three inputs: a dataset that matches your real traffic, the numbers you need to reach (throughput, time to first token, latency or cost), and the evals the model must keep passing.
Which models can the agents tune?
Open-weight models, including weights that you fine-tuned. The agents start from a standard setup for the model and tune for its architecture, as they did for gpt-oss-120b.
Can tuning lower the model's quality?
Every change is measured against the original model on your evals. A faster result that fails them does not count, and the harness discards it.
Where does the tuned engine run?
On your dedicated deployment: a private endpoint on GPUs that no other customer shares, or an air-gapped instance on hardware that you control. See the deployment options.
Who approves a change before production?
You review what the agents found. Deeper engine changes, such as new kernels, also go through human review before they reach production.
How does it start?
With a conversation, not a signup. Talk to us about the workload, the target and the quality bar.
