GLM-5.3-Flash-class (~320B MoE) on 4× DGX Spark: 47.7 tok/s
4× GB10 · 100 Gbps switch · 4-bit · 1M-token context configured · measured 2026-10-25 · per the methodology
A ~320-billion-parameter MoE (about 18B active per token) on half of our cluster: 47.7 tok/s on code and 41.3 on prose, single stream, delivered tokens — and more than 420 tok/s across 64 simultaneous users. The first working launch on this hardware ran at about 19.6 tok/s; configuration took it to roughly 2.4× that.
| metric | result | notes |
|---|---|---|
| decode · code | 47.7 tok/s | ± 2.7 (n=15, 95% CI) |
| decode · prose | 41.3 tok/s | ± 3.0 (n=15, 95% CI) |
| aggregate · 64 streams | 420–427 tok/s | 3 bursts, 64 × 1,024 tokens each |
| first working launch | ~19.6 tok/s | September 2026, shorter protocol |
| long context | 97k-token prompt | decode held within ~6% of short-context speed |
Delivered tokens include the model's reasoning tokens (this model always thinks before answering). Measured on one server instance; two earlier, separate instances landed at 47.4 and 50.1 on the same code workload — the restart is the unit of replication.
What stood between the first launch and this number
- Hand-tuning lost to the shipped recipe. Our own tuned configuration never finished a 97k-token prompt (killed after 75+ minutes); the vendor's documented path did it in 45 seconds — and then needed a small number of chip-specific corrections to run correctly on sm_121.
- A crash that left memory stranded. One launch path failed deterministically mid-startup and left ~86 GB per node held with no owning process — recoverable without a reboot, if you know the sequence (trap index).
What this entry teaches
- Four nodes is enough for this class. A ~320B sparse model serves at chat-ready speed on half the cluster, leaving the other half free for a second model.
- Read the shipped docs before tuning. The biggest single jump came from running the documented configuration exactly, then fixing only what this chip needs.
Running a 300B-class model on Spark nodes?
The diagnostic reproduces your setup on our 8-node lab, measures where you really are, and ranks your recoverable headroom in writing — or it's free.
Book a 1-day diagnostic →Free to read and share. Numbers are from hardware we've measured directly; your engine versions and configs will differ — measure your own, and count only delivered tokens.