GLM-5.3-Flash on four GB10 nodes: 47.7 → 98.8 tok/s on code, with an MIT drafter

Published 28 Oct 2026

Field note · measured on 4× GB10 (DGX Spark-class) · 100 Gbps switch · n=15 per workload

On 25 October we published GLM-5.3-Flash on four GB10 (DGX Spark-class) nodes at 47.7 tok/s on code and 41.3 on prose. Two days later we found knapcio's results, published a month earlier, on the same four-node GB10 setup at roughly twice our speed (on a different, shorter probe). Their post told us something we needed to hear: we weren't getting the most out of our setup. A careful measurement of a setup that leaves half its speed on the table is still a measurement of the wrong setup. So we went back and re-examined ours, layer by layer.

Where we are now

Single stream, temperature 0, delivered tokens including reasoning, 95% CI, warm-up excluded: the methodology and prompts of the original entry. The benchmark entry itself is on the GLM-5.3-Flash benchmark page. One difference to know about: on 25 Oct, 28 of 30 answers hit the 1,024-token cap mid-reasoning. On 28 Oct, with NVIDIA's checkpoint and its chat template, answers finished on their own at a mean of 650 (code) and 444 (prose) tokens. Shorter answers spend proportionally more time on the prompt (about 3%), which works against the new numbers. The answers also differ in content (more answer, less capped reasoning), and we can't separate that effect out.

Thank you, knapcio

We owe this one to knapcio, who published their GLM-5.3-Flash 4× DGX Spark setup openly, and to tonyd2wild, whose runbook documents how to build it. Their write-ups showed exactly where our setup was leaving speed behind. Building on tonyd2wild's image, their setup combines NVIDIA's own NVFP4 checkpoint, 8-bit dense layers, 4-bit Marlin expert kernels, a one-shot RDMA all-reduce, and speculative decoding that checks only the guesses likely to be accepted. knapcio's own code (MIT and Apache-2.0) is now the base we build on, as its license invites, and we're grateful it's public. These four nodes have no internet access, so we built every layer from source offline. Running their software unchanged, our nodes came in about 3–7% faster per step than the published numbers, most likely from hardware and clock differences, not anything we did. The hardware was never the gap; our configuration was.

The licensing trap

The piece that adds the most speed is its drafter, the small model that guesses ahead: Inco's DFlash2, licensed CC BY-NC-ND 4.0, non-commercial. knapcio's README says so clearly. It's fine for personal use, but not for anything we'd run for a client. Check the license of every component before you build a business on a benchmark.

Red Hat publishes an MIT-licensed drafter for the same model (DSpark), and the same engine build runs it unmodified. Out of the box, though, it was slower: it guesses well, but every step the big model checked all eight guesses plus the current token (nine rows), whether or not the later guesses were likely to pass.

What we changed

• Checking only what's likely to pass, on the MIT drafter. knapcio's setup trims each step's guesses using a fitted table. DSpark has its own confidence head that scores each guessed position. We wired that trimming logic to the head, so the engine verifies only the guesses worth checking.
• 4-bit dense layers. The attention and shared-expert weights went from 8-bit to 4-bit: about 10% faster per step, with no measurable exam loss. The full quality test.
• An 8-bit drafter, as knapcio's stack uses for its own. Worth about 1 ms per step, once we found that the drafter's layers are numbered after the main model's inside the engine. Our first exclusion list named the wrong layers.
• A setting that was silently dropped. A kernel-split option was missing from our launch configuration. Restoring it took 1.4 ms off every step.

What didn't help (so you can skip it)

• Both network rails instead of one: about 0. The network isn't the bottleneck at this step size.
• Re-tuning the 8-bit kernels for the drafter's shapes: no gain. The stock kernel already runs near memory bandwidth.
• An 8-bit draft head, and deciding the trim one step late to hide host time: no gain, or worse. The step is GPU-bound.
• Training our own drafter on about 2,500 of the model's own answers: a tie with Red Hat's. That is far too little data to beat a drafter trained on about 1.7 million samples. The pipeline works; the data budget is the next step.

Where we still stand, honestly

Running their full stack (with the non-commercial drafter) on our four nodes with their own probe script, a different and shorter test than our n=15 table, we measure about 70 / 112 / 140 tok/s on prose / code / JSON. The tok/s differ from their README's 90 / 124 on prose / code because the README numbers use a thinking-off, 256-token probe with higher acceptance. With their probe script on our nodes, our setup with the MIT drafter does 64 / 98 / 128. Their drafter plus our 4-bit dense layers gives 77 / 123 / 150. Like for like, the MIT drafter costs 15–20% today; our 4-bit dense change wins back about half of that, which leaves us roughly 10% behind their full setup. Closing that gap is our next job.

The 264–271 tok/s at 64 users is lower than before for a reason: this setup is tuned for one user. With speculation off, the same four nodes serve 535 tok/s across 128 users (fixed 512-token answers, a different test). Why one config can't do both, measured.

On licenses: the drafter, the engine and knapcio's own code are MIT or Apache-2.0 (some vendored kernel files add NVIDIA's BSD-3-Clause), and NVIDIA's model checkpoint is MIT per its model card. Some patches in the setup we started from come from the build recipe credited above, whose repositories, as of 28 Oct 2026, didn't declare a license; knapcio's NOTICE file flags the same open question. We re-implemented those changes ourselves and re-ran our tests. The tables above were measured on the original build. On the new build: code 98.9 ± 3.2 tok/s (was 98.8), prose 73.1 ± 3.7 (was 73.2), 64 users 258–270 (was 264–271), all n=15 results within their error bars; knapcio's probe 64 / 94–97 / 128 (was 63–64 / 97–99 / 127–128). Exams: GSM8K 247 of 250 on both, HumanEval 153 vs 154 of 164 (a one-problem difference, p = 1.0). We did not re-run the other comparisons in this note. We read licenses carefully, but we are not lawyers and this is not legal advice: check them for your own use.

Free to read and share. Numbers are from hardware we've measured directly; your engine versions and configs will differ — measure your own, and count only delivered tokens.