Is 4-bit safe for the dense layers? The probe said check; the exam said yes

Published 28 Oct 2026

Field note · measured on 4× GB10 (DGX Spark-class) · GLM-5.3-Flash · GSM8K + HumanEval, paired

GLM-5.3-Flash's routed experts already ship in 4-bit (NVFP4). The rest of the weights, the attention projections and the always-on shared experts, are converted to 8-bit in the stack we build on (the checkpoint ships them at higher precision). Every one of those weights is read on every token, so on bandwidth-bound GB10 (DGX Spark-class) hardware they set a hard floor under each decoding step. We moved them to 4-bit as well and asked the obvious question: does the model get dumber?

Speed: about 10% per token

Same server, same speculative drafter, only the weight file swapped: the decoding step went from 32.4 ms to 29.3 ms on prose. On knapcio's single-user probe, with their (non-commercial) drafter, that took prose from ~70 to 77 tok/s, code from ~112 to 123, and JSON from ~140 to 150. Speed was measured with that drafter; quality (below) on our MIT-drafter setup. With greedy decoding, speculative decoding in principle doesn't change the output, so the exam applies to both. On our MIT-drafter setup, measured before our later drafter and kernel-split fixes, the gain was 6–8% (prose 55.6 → 60, code 86 → 92.6, JSON 112.8 → 120).

Quality: the drift probe said "check"

The first test compares the two models' next-token probabilities, position by position, over about 134,000 positions. The 4-bit version picked the same top token as the 8-bit version 92.7% of the time, with a mean KL divergence of 0.038. For scale, knapcio measured 0.029 between the 8-bit version and a 16-bit reference (over fewer positions than ours). So 4-bit moves the model slightly further again. That doesn't prove harm. Many disagreements are near-ties that don't change an answer. But it isn't proof of safety either.

Quality: the exam said "no measurable loss"

So we gave both versions the same exam: the first 250 GSM8K math problems and all 164 HumanEval coding problems. Greedy decoding, low reasoning effort, generated code run in a sandbox with no network. Before scoring either model, we fed the scorer the reference answers to check it: 250/250 and 164/164.

On GSM8K, the 4-bit version lost one problem and gained two. On HumanEval, it lost two and gained two. A paired McNemar test on the flips gives p = 1.0: nothing to distinguish them. (knapcio publishes a higher HumanEval score for their 8-bit stack, with different settings: thinking off vs our low effort. The paired comparison is what matters here.)

The method, which is the real lesson

• Speed claims need a quality claim attached. Every quantization step is a trade. Publishing the tok/s without the exam is publishing half the result.
• Use two kinds of test. A distribution probe catches drift cheaply. A scored exam tells you whether the drift changes answers. Here they disagreed in tone, and the exam decided.
• Test the scorer first. A grader that can't award full marks to the reference answers will invent differences between models.
• Compare paired, not pooled. Same questions, same settings, and count the flips. Two percentages side by side can hide a model that broke ten problems and fixed ten others.

The full speed story this is part of: how GLM-5.3-Flash on our four GB10 nodes went from 47.7 to 98.8 tok/s on code.

Free to read and share. Numbers are from hardware we've measured directly; your engine versions and configs will differ — measure your own, and count only delivered tokens.