Why your model hangs "loading weights" on GB10: three causes we measured

Published 25 Oct 2026 · Updated 28 Oct 2026

Field note · measured on an 8× GB10 cluster, Kimi K2.7 NVFP4 (~71 GiB per node), vLLM; one case on SGLang

You start the server, it prints something like loading weights..., and then nothing. No error, no traceback. At its worst, our Kimi K2.7 startup took 27 minutes from launch to serving. It now takes under 3 minutes. Three different causes were behind it, and from the outside they all look like "a big model being slow."

It's rarely "the model is big"

Reading all 71 GiB of a node's weights off local NVMe takes about 15 seconds. If your load takes many minutes, something other than the size of the model is in the way.

The three causes, and how they differ

• Cause 1: the bytes crawl from the file to the GPU. safetensors hands back tensors that are views into a memory-mapped file. Copying such a view straight to the GPU forced every page through a slow bounce path, one tensor at a time: we measured 167 MB/s, against 4,054 MB/s when the tensor is first cloned into ordinary memory. Adding tensor.clone() before the copy in vLLM's sharded-state loader took the weight phase from 452 s to 34 s on all 8 ranks, and launch-to-serving from 10 min 46 s to 2 min 45 s. The same defect sat in the default loader path (the MoE expert loaders and the default weight loader) and needed the same fix there.
  Signature: the disk reads almost nothing, one CPU core sits near 100%, and a profiler shows the time inside the weight-copy frames. We first read that signature as a single-threaded, CPU-bound loader and planned to parallelize it. That was wrong: the loader's own bookkeeping was 15.3 s of the roughly 460 s. The pegged core was the copy path.
• Cause 2: one storage device feeding eight nodes. With weights on one NFS export, every node reads from the same disk at once. After the Cause 1 fix, loading over NFS still took 278–384 s per rank, while per-node local copies of each rank's slice loaded in 34–38 s. The order of fixes matters: before Cause 1 was fixed, local copies saved only about 11% (452 s vs about 507 s), because the copy path was the bigger wall. On SGLang, the same shared-NFS setup made peer nodes load nothing at all for 200 s.
  Signature: per-rank load times differ widely, and the node that hosts the files finishes first.
• Cause 3: it isn't loading, it's waiting. On multi-node, every rank waits at a barrier for the slowest one, so one bad rank looks like a cluster-wide hang. In one vLLM launch, five ranks finished allocating their KV cache in 3.7–5.5 s; the other three took 57.0, 314.2 and 556.1 s, and everyone waited for the slowest. Which rank was slow changed from run to run. (That stall disappeared with the Cause 1 fix: five launches in a row came in at 32.6–37.0 s.) On SGLang, an unset SGLANG_HOST_IP stalled startup before any weights loaded: the engine finds its own IP address by trying an outbound route, our nodes have none, and setup waited. Setting each node's fabric IP fixed it. Ray does something similar and needs --node-ip-address on such networks.
  Signature: little CPU or disk activity, and the logs stop at different points on different ranks.

Each cause has a different fix, and the exact place to apply it depends on your framework version, your model's shard layout, and your topology. Get the diagnosis right first.

Free to read and share. Numbers are from hardware we've measured directly; your setup will differ with framework version, model layout, and topology, so measure your own.