GLM-5.3-Flash on four GB10 nodes: 47.7 → 98.8 tok/s on code, with an MIT drafter

Published 28 Oct 2026

Field note · measured on 4× GB10 (DGX Spark-class) · 100 Gbps switch · n=15 per workload

On 25 October we published GLM-5.3-Flash on four GB10 (DGX Spark-class) nodes at 47.7 tok/s on code and 41.3 on prose. Two days later we found knapcio's results, published a month earlier, on the same four-node GB10 setup at roughly twice our speed (on a different, shorter probe). Their post told us something we needed to hear: we weren't getting the most out of our setup. A careful measurement of a setup that leaves half its speed on the table is still a measurement of the wrong setup. So we went back and re-examined ours, layer by layer.

Where we are now

Single stream, temperature 0, delivered tokens including reasoning, 95% CI, warm-up excluded: the methodology and prompts of the original entry. One difference to know about: on 25 Oct, 28 of 30 answers hit the 1,024-token cap mid-reasoning. On 28 Oct, with NVIDIA's checkpoint and its chat template, answers finished on their own at a mean of 650 (code) and 444 (prose) tokens. Shorter answers spend proportionally more time on the prompt (about 3%), which works against the new numbers. The answers also differ in content (more answer, less capped reasoning), and we can't separate that effect out.

Thank you, knapcio

We owe this one to knapcio, who published their GLM-5.3-Flash 4× DGX Spark setup openly, and to tonyd2wild, whose runbook documents how to build it. Their write-ups showed exactly where our setup was leaving speed behind. Building on tonyd2wild's image, their setup combines NVIDIA's own NVFP4 checkpoint, 8-bit dense layers, 4-bit Marlin expert kernels, a one-shot RDMA all-reduce, and speculative decoding that checks only the guesses likely to be accepted. knapcio's own code (MIT and Apache-2.0) is now the base we build on, as its license invites, and we're grateful it's public. These four nodes have no internet access, so we built every layer from source offline. Running their software unchanged, our nodes came in about 3–7% faster per step than the published numbers, most likely from hardware and clock differences, not anything we did. The hardware was never the gap; our configuration was.

The licensing trap

The piece that adds the most speed is its drafter, the small model that guesses ahead: Inco's DFlash2, licensed CC BY-NC-ND, non-commercial. knapcio's README says so clearly. It's fine for personal use, but not for anything we'd run for a client. Check the license of every component before you build a business on a benchmark.

Red Hat publishes an MIT-licensed drafter for the same model (DSpark), and the same engine build runs it unmodified. Out of the box, though, it was slower: it guesses well, but the big model checked all nine guessed positions every step.

What we changed

• Checking only what's likely to pass, on the MIT drafter. knapcio's setup trims each step's guesses using a fitted table. DSpark has its own confidence head that scores each guessed position. We wired that trimming logic to the head, so the engine verifies only the guesses worth checking.
• 4-bit dense layers. The attention and shared-expert weights went from 8-bit to 4-bit: about 10% faster per step, with no measurable exam loss. The full quality test.
• An 8-bit drafter, as knapcio's stack uses for its own. Worth about 1 ms per step, once we found that the drafter's layers are numbered after the main model's inside the engine. Our first exclusion list named the wrong layers.
• A setting that was silently dropped. A kernel-split option was missing from our launch configuration. Restoring it took 1.4 ms off every step.

What didn't help (so you can skip it)

• Both network rails instead of one: about 0. The network isn't the bottleneck at this step size.
• Re-tuning the 8-bit kernels for the drafter's shapes: no gain. The stock kernel already runs near memory bandwidth.
• An 8-bit draft head, and deciding the trim one step late to hide host time: no gain, or worse. The step is GPU-bound.
• Training our own drafter on about 2,500 of the model's own answers: a tie with Red Hat's. That is far too little data to beat a drafter trained on about 1.7 million samples. The pipeline works; the data budget is the next step.

Where we still stand, honestly

Running their full stack (with the non-commercial drafter) on our four nodes with their own probe script, a different and shorter test than our n=15 table, we measure about 70 / 112 / 140 tok/s on prose / code / JSON. With their probe script on our nodes, our setup with the MIT drafter does 64 / 98 / 128. Their drafter plus our 4-bit dense layers gives 77 / 123 / 150. Like for like, the MIT drafter costs 15–20% today; our 4-bit dense change wins back about half of that, which leaves us roughly 10% behind their full setup. Closing that gap is our next job. (For reference, their README publishes 90 / 124 on prose / code using a different, 256-token, thinking-off probe.)

The 264–271 tok/s at 64 users is lower than before for a reason: this setup is tuned for one user. With speculation off, the same four nodes serve 535 tok/s across 128 users (fixed 512-token answers, a different test). Why one config can't do both, measured.

On licenses: the drafter, the engine and knapcio's own code are MIT or Apache-2.0 (some vendored kernel files add NVIDIA's BSD-3-Clause), and NVIDIA's model checkpoint is MIT per its model card. Some patches in the setup we started from come from the build recipe credited above, whose repositories didn't declare a license when we checked; knapcio's NOTICE file flags the same open question. Rather than rely on them, we wrote our own versions from descriptions of what each patch does, then re-ran our tests. The tables above were measured on the original build. On the new build: code 98.9 ± 3.2 tok/s (was 98.8), prose 73.1 ± 3.7 (was 73.2), 64 users 258–270 (was 264–271), all n=15 results within their error bars; knapcio's probe 64 / 94–97 / 128 (was 63–64 / 97–99 / 127–128). Exams: GSM8K 247 of 250 on both, HumanEval 153 vs 154 of 164 (a one-problem difference, p = 1.0). We did not re-run the other comparisons in this note. We read licenses carefully, but we are not lawyers and this is not legal advice: check them for your own use.

Free to read and share. Numbers are from hardware we've measured directly; your engine versions and configs will differ — measure your own, and count only delivered tokens.