MiniMax M3-class (~400B MoE + vision): 6.4 → 34.5 tok/s
8× GB10 · 100 Gbps switch · w4a16 · 512k context · two serving profiles · per the methodology
This is the most instructive entry in the database, because it went through nearly every trap on the index on its way from 6.4 tok/s out of the box to 34.5 tok/s delivered on code — a 5.4× journey, every step measured.
| metric | result | notes |
|---|---|---|
| decode · code (spec on) | 34.5 tok/s | out-of-box was 6.4 |
| decode · prose (spec off) | ~22.5 tok/s | speculation pays on code, not prose — two profiles |
| aggregate · 16 streams | 170 tok/s | concurrency and spec are mutually exclusive modes |
| prefill | 699 → 1,520 tok/s | one collective-protocol line; TTFT 14.6 → 6.7 s |
| prefix caching | up to 67× | on repeated-prefix agent workloads |
| context | 512k | validated deep into the window; fp8 KV wins at long context |
The traps this model hit (all documented in the field notes)
- Graph capture was the difference between 6.4 and 22+ — and enabling speculation silently disabled it (the +58% recovery).
- Vision was silently disabled by a default limit nobody reads — a vision model shipping text-only, no errors.
- The prefill fix was one configuration line worth 2.2× (the 4.4× protocol tax).
- Code and prose want different serving configs — one profile cannot be optimal for both; we ship two and switch by workload.
Why this matters beyond this model
The ~400B class is the sweet spot of the multi-node Spark question: too big for one or two boxes, small enough that a tuned 8-node cluster serves it at genuinely usable speed. Nobody else publishes this class measured, in writing, with the failure path included.
Your 400B-class deployment stuck in the single digits?
This model started at 6.4 tok/s too. The diagnostic finds which of these traps your cluster is in — reproduced on our lab, delivered in writing, or it's free.
Book a 1-day diagnostic →Free to read and share. Numbers are from hardware we've measured directly; your engine versions and configs will differ — measure your own, and count only delivered tokens.