Why is my DGX Spark slow? A checklist
Published 28 Oct 2026
Field-note hub · GB10 / DGX Spark-class · symptom → check → measured note
A slow DGX Spark or other GB10 machine usually has one of a small number of causes. This checklist puts them in a sensible order: single-machine checks first, then how you serve the model, then multi-node, then the measurement itself. Each item is a symptom, a quick check, and a link to the field note with our measurements.
Before you start: stop other jobs on the GPU, note your engine and version, and write down the exact number you're unhappy with (decode tokens/s for one user, total tokens/s for many, or time to first token).
1 · Every model on one machine runs at about half speed
Symptom: Under full load, the GPU clock and power draw sit far below normal, utilization still reads high, no throttle reason is reported, and a reboot doesn't help.
Check: Run our free 45-second check on that machine with the GPU otherwise idle. It reports throughput, clock and power under load, and compares them with healthy and stuck reference units.
Read: Is your GB10 stuck in low-power mode? A 45-second check, and the fix · the check script itself (one file, no network access)
2 · The tokens/s number doesn't match what users feel
Symptom: The engine log, a benchmark tool and your users report different speeds for the same server, especially with speculative decoding on.
Check: Check which tokens each number counts: proposed, accepted or actually delivered to the client. Compare only delivered tokens, measured at the client.
Read: Your server log says 40.7 tok/s. A careful benchmark says 27.8. Here's why both appear.
3 · Out-of-memory errors while memory looks free, or a server that wedges at startup
Symptom: The machine reports plenty of available memory, yet the model server fails to start, stalls, or the whole system becomes unresponsive.
Check: CPU and GPU share one pool of memory on GB10. Compare what CUDA can actually get (torch.cuda.mem_get_info()) with what free -g reports. Before loading, run sync, drop the page cache and compact memory (drop_caches, compact_memory). Then check the memory fraction you've given the engine.
Read: Where your Spark's 128 GB actually goes, and why the memory numbers lie
4 · Output is garbage or repeated characters, with no error
Symptom: The server runs and answers, but the text is nonsense or a single repeated character, often with a 4-bit (FP4) model.
Check: Try eager mode first (CUDA graphs off): one garbage-output case we hit was CUDA-graph replay, not quantization. Then run the same prompt with a different quantization or kernel backend. If that path is fine, suspect the low-precision kernel on this chip, not your prompt or the model.
Read: When your Spark outputs garbage, check the FP4 kernel path before your prompt
5 · The server sits at 'Loading weights' for a long time
Symptom: Startup takes many minutes or appears hung, on one node or across several.
Check: Watch whether the node is actually reading data during the stall, and whether every node in a multi-node launch has reached the same point. 'Slow to load' and 'waiting for another node' look identical in the log.
Read: Why your model hangs "loading weights" on GB10: three causes we measured
6 · Speculative decoding made it slower, not faster
Symptom: Turning on a draft model or multi-token prediction lowers single-stream speed instead of raising it.
Check: Check whether CUDA graphs are still being captured after you enable speculation. Look in the engine's startup log and compare it with a run without speculation. Then measure code and prose separately: speculation can win on one and lose on the other.
Read: Speculative decoding made our model slower. It had switched off CUDA graphs.
7 · Fast for one user, slow or long waits for a team
Symptom: One person gets good speed, but with many simultaneous users total throughput stalls and some requests wait a long time before the first word.
Check: Measure total throughput and the worst time to first token at your real number of users, and watch how many requests are queued rather than running.
Read: Fast for one user, slow for a team: the speculative-decoding trap
8 · Multi-node: transfers between machines are far below your link speed
Symptom: A multi-node setup is much slower than the same model's single-node numbers suggest, or one node-to-node transfer runs at a small fraction of the link rate.
Check: Run ib_write_bw between two nodes: about 93.5 Gb/s per node is healthy on our hardware. Read the negotiated active_mtu with ibv_devinfo on every node, not the interface MTU, and check that each port is on the network you expect.
Read: Your GB10 RDMA at ~94 Gbps isn't broken: that's the ceiling we measured
9 · Multi-node: long prompts take far too long to start, short ones are fine
Symptom: Decode speed looks normal, but time to first token grows badly with prompt length on a multi-node setup.
Check: Look for NCCL algorithm or protocol settings forced in your launch scripts or container environment (for example NCCL_PROTO or NCCL_ALGO). A setting tuned for small decode messages can also apply to prefill's large ones.
Read: The NCCL override that cost us 4.4× on prefill
10 · Adding nodes made it no faster, or slower
Symptom: Two nodes work well, but four or eight give little gain, or less speed than fewer nodes.
Check: Compare node counts with CUDA graphs on, in milliseconds per token: in eager mode, launch overhead can hide the gain from more nodes. Check that your tensor-parallel size divides the model's attention head count.
Read: vLLM tensor parallelism across Spark nodes: when more nodes help
None of these? GB10 hides other slowdowns too. The trap index lists every problem class we've measured on our own cluster, and the benchmarks give you reference numbers to compare against.
Checked them all and it's still slow?
The diagnostic reproduces your setup on our 8-node GB10 lab and tells you, in writing and with measurements, where the speed is going, or it's free.
Book a 1-day diagnostic →Free to read and share. Numbers are from hardware we've measured directly; your engine versions and configs will differ — measure your own, and count only delivered tokens.