In our comparison with Fireworks AI, BLACKBOX AI delivered 3.6× the peak throughput on the same eight B300 GPUs, with median time to first token below a second through 48 concurrent coding conversations.
On a single node with eight NVIDIA B300 GPUs, BLACKBOX AI served Kimi K3.1 Pro at 20.23M total tokens per minute. Fireworks AI’s dedicated deployment reached a peak of 5.66M on the same GPU count and type. That is 3.6× the peak throughput for the same coding workload.
20.23M
total TPM at 48 conversations, BLACKBOX AI
0.65 s
median TTFT at 48 conversations, BLACKBOX AI
125
completed requests/min at 48 conversations, BLACKBOX AI
More throughput as concurrency grows
Total token throughput on 8× B300
Input + output tokens, including cached input · same number of concurrent conversations
Higher is better
Total token throughput on 8× B300
Entry
Value
8 concurrent coding conversations Fireworks AI Dedicated deployment
3.95M TPM
8 concurrent coding conversations BLACKBOX AI 1.3× Fireworks throughput
5.14M TPM
16 concurrent coding conversations Fireworks AI Dedicated deployment
5.66M TPM
16 concurrent coding conversations BLACKBOX AI 1.9× Fireworks throughput
10.90M TPM
32 concurrent coding conversations Fireworks AI Dedicated deployment
1.33M TPM
32 concurrent coding conversations BLACKBOX AI 12.6× Fireworks throughput
16.79M TPM
48 concurrent coding conversations Fireworks AI Dedicated deployment
0.48M TPM
48 concurrent coding conversations BLACKBOX AI 42.1× Fireworks throughput
20.23M TPM
64 concurrent coding conversations Fireworks AI Dedicated deployment
0.65M TPM
64 concurrent coding conversations BLACKBOX AI 29.5× Fireworks throughput
Kimi K3.1 Pro · same coding workload · one 8× B300 node per deployment. Each pair compares the same number of concurrent coding conversations.
Blackbox throughput rose from 5.14M tokens per minute (TPM) at 8 conversations to 20.23M at 48. Fireworks peaked at 16 conversations. At 32, its throughput fell 77%, to 1.33M TPM, while Blackbox reached 16.79M TPM.
The gap widened with more concurrent coding conversations: 12.6× Fireworks’ throughput at 32 conversations and 42.1× at 48. The 3.6× peak-to-peak result compares Blackbox at 48 conversations with Fireworks at 16, each deployment’s strongest result.
At 64 conversations, Blackbox served 19.19M TPM, about 5% below its peak. For this workload, 48 conversations delivered the highest throughput.
Shorter waits for a response
Higher throughput matters alongside response time. At 32 conversations, the median wait for a first token was 0.52 s on Blackbox and 51.50 s on Fireworks. Once generation began, median output speed was 41 versus 4 tokens per second per request.
Median time to first token
Client-observed latency · p95 shown below each deployment
Lower is better
Median time to first token
Entry
Value
8 concurrent coding conversations Fireworks AI p95 20.53 s
2.40 s
8 concurrent coding conversations BLACKBOX AI p95 7.12 s
0.37 s
16 concurrent coding conversations Fireworks AI p95 30.78 s
3.09 s
16 concurrent coding conversations BLACKBOX AI p95 8.01 s
0.39 s
32 concurrent coding conversations Fireworks AI p95 221.35 s
51.50 s
32 concurrent coding conversations BLACKBOX AI p95 9.58 s
0.52 s
48 concurrent coding conversations Fireworks AI p95 327.45 s
110.84 s
48 concurrent coding conversations BLACKBOX AI p95 16.00 s
0.65 s
64 concurrent coding conversations Fireworks AI p95 464.75 s
159.00 s
64 concurrent coding conversations BLACKBOX AI p95 16.89 s
Client-observed latency. Blackbox was accessed locally and Fireworks through its hosted API.
Blackbox kept median time to first token (TTFT) below a second through 48 conversations, where p95 was 16.00 s. At 64, the median rose to 5.97 s, compared with 159.00 s on Fireworks. Neither deployment reported request errors in these results.