GLM-5-class (753B MoE) on 8× DGX Spark: 9.1 → 16.5 tok/s
8× GB10 · 100 Gbps switch · mixed-precision 4-bit · TP8 · per the methodology
A 753-billion-parameter MoE across eight GB10 nodes: 9.1 tok/s out of the box, 16.5 tok/s after tuning — +81%, from the same two levers that keep winning on this platform: native multi-token prediction (k=2) and the right partial graph-capture mode. Delivered tokens, single stream.
| config | decode |
|---|---|
| out of the box | 9.1 tok/s |
| + MTP k=2 + partial graph capture | 16.5 tok/s |
Honest status: headroom remains
This entry is not finished, and we say so: a meaningful slice of this network still runs at full precision because its layers resist the standard quantization path — the next measured lever, worth an estimated double-digit percentage. We publish work-in-progress states because a benchmark database that only shows finished victories is marketing.
What this class teaches
- At 753B across 8 nodes, the collective-communication layer is a first-order cost — the same protocol trap that cost 4.4× on prefill elsewhere (that story) hunts this class too.
- Per-model quantization is not transferable: the recipe that made the 1T model fly does not port here — each checkpoint's layout has to be measured, not assumed.
Serving a 500B+ model across Spark nodes?
This weight class is where defaults hurt most and nobody publishes numbers. The diagnostic reproduces your setup on our 8-node lab and ranks your recoverable headroom in writing — or it's free.
Book a 1-day diagnostic →Free to read and share. Numbers are from hardware we've measured directly; your engine versions and configs will differ — measure your own, and count only delivered tokens.