Repository navigation
Fresh jarvis-gb10 install fails at image/packs: the built pins predate the edit pack from #78 #84
Description
Hi Jay, thanks for LocalRouter. We're really enjoying getting it running on our Spark.
We hit a snag on a fresh install today, and it looks like it affects anyone installing from scratch since #78 landed.
What happened
On d9a0734, a fresh tools/install/bootstrap.sh --profile jarvis-gb10 --accept on a DGX Spark (GB10, driver 580.178.04, CUDA 13.0) stops at image/packs:
FAIL te/manifest.json: size 131480, want 67306
FAIL te/weights.safetensors: size 16289653928, want 15136839544
FAIL vae/manifest.json: size 62806, want 33861
FAIL vae/weights.safetensors: size 662002120, want 506156560
fail: 6/10 files match
Every other step passed, including the chat and Drama 3 GPU qualifications.
What we think is going on
tools/spark/prepare.sh:105 (added in #78) runs pack --precision edit, which rewrites te/ (749 tensors) and vae/ (230 tensors, now with the encoder). image/packs then checks them against tools/twin/packs/built/qwen-image-2.1-turbo.json, which was last changed in c82bd87 on 22 October and still pins the pre-edit te/ and vae/. A box that already has an image-packs stamp from before #78 won't notice, which is probably why it didn't show up on DGX-01.
Possible fix
Re-pin te/ and vae/ in built/qwen-image-2.1-turbo.json from an edit-pack build, and pin edit.json too.
As a stopgap we did exactly that locally, from our own build on the Spark. The source checkpoints had already passed their pins in image/download. With these values verify_assets.py passes 10/10 and the install goes on. The two manifest hashes are the same ones the edit pack printed when it wrote them. Here they are in case they're useful to compare against a build on DGX-01:
| Path | Size (bytes) | SHA-256 |
|---|---|---|
te/manifest.json |
131480 | a70146666c4990c0a66a9774fab991d7f1dbd687efaaa45eac5cc1928e836519 |
te/weights.safetensors |
16289653928 | 62d2543cdfb919f0c67f2265fd87483a6cd0d9e7b7143440199ec2f4c87f71bc |
vae/manifest.json |
62806 | 191c1441c61496f30d3f5b4baa2433e712ba13e15d1e80f167ed82353c54e6a7 |
vae/weights.safetensors |
662002120 | ed03972f15842905e6ee1e96f22b426ef99be57c15921b81088f7369617487b4 |
edit.json (not pinned today) |
221 | 79baea3189bf6bcd6f6bfa38866978a3d7a1da6ada17cf60ef9228d5806a3a15 |
We haven't checked whether the edit pack is byte-for-byte reproducible across boxes, so please treat these as a comparison point rather than the pins to ship.
Happy to test a fix on our box if that helps.
Raised by Claude Opus on behalf of GUITARAHOLIC.
Activity
jayleaton commentedon Oct 28, 2026
Sorry about that. Let me take a look at this. Thanks for the report
jayleaton commentedon Oct 29, 2026
Thanks for such a clear report, and for the hashes. They pinned the problem down exactly.
You were right about the first part. The built pins (
tools/twin/packs/built/qwen-image-2.1-turbo.json) still describedte/andvae/from before the edit pack, so every fresh install failedimage/packs. #118 re-pinned them from a build on our second Spark.Your table showed a second problem. Your
te/weights.safetensors,vae/weights.safetensorsandvae/manifest.jsonmatch ours byte for byte, so the weights are reproducible across boxes. Yourte/manifest.jsonandedit.jsondidn't match.The cause was one field. The edit step copied the vision tower's whole transformers config into
te/manifest.json, and that includestransformers_version. Our twin venv installedtransformers>=4.57, which gave your box 5.20.2 and ours 5.21.0. When we put "5.20.2" into our manifest, it reproduced youra7014666…exactly.edit.jsonrecords that manifest's hash, so it reproduced your79baea31…too.The fix is in #130 (commit
d2fe032, merged as059b3ad):tools/spark/prepare.shrewrites an existing edit pack'ste/manifest.jsonandedit.jsonthe same way. Only metadata changes, so there is no re-pack, and your stopgap pins can go.te/manifest.json130922 bytes,91fd46ff…;edit.json64638c30…). The weight pins are unchanged.transformers==5.21.0.You also mentioned a box with an older
image-packsstamp. The stamp is the sha256 of the pin file, so the new pins makeimage/packsrun again on such a box. It doesn't silently pass.Could you check it on your Spark? Revert your local change to the pin file, pull main (
059b3ador later), and run the same command again:git checkout -- tools/twin/packs/built/qwen-image-2.1-turbo.json # your local stopgap pins git pull tools/install/bootstrap.sh --profile jarvis-gb10 --acceptimage/packsshould run again. The log should showedit pack metadata normalised (#84), thenpass: 11/11 files match, and the install should go on from there. Like anyimage/packsrun, the step wants 64 GiB of free memory. If it still fails, please paste theFAILlines frominstall/logs/image-packs.log.Thanks again, and especially for offering to test.