From a33e39d4226d63896caa0785c8073d81d0b8448c Mon Sep 17 00:00:00 2001 From: vickydotbat Date: Fri, 31 Jul 2026 12:51:14 +0200 Subject: [PATCH] docs: describe the on-disk shape of an emitted NWSync tree Blob filenames are the SHA-1 of a resource's original bytes, but the file on disk is NWCompressedBuffer framing: a 24-byte NSYC header then a zstd frame. Hashing the file directly is the first thing anyone tries when checking a zone by hand, and it never matches. Record the layout and the one-line recipe that does match, plus where the uncompressed length lives in the header. Verified against a real emit and against internal/nwsync/compressedbuf.go. Co-Authored-By: Claude Opus 5 (1M context) --- docs/command-surface.md | 29 +++++++++++++++++++++++++++++ 1 file changed, 29 insertions(+) diff --git a/docs/command-surface.md b/docs/command-surface.md index 15128c9..33162a5 100644 --- a/docs/command-surface.md +++ b/docs/command-surface.md @@ -70,6 +70,35 @@ the zone and no index. Blob names are content hashes, so re-running skips whatever already landed, and `assemble` fails closed on an artifact with no index rather than publishing a manifest that is missing a hak. +### What an emitted tree looks like + +`--out DIR` produces the same tree `emit` would upload, which makes it the way +to check a zone by hand without touching one: + +``` +.nsym binary index +.nsym.json the same index, readable +data/sha1/a7/4a/a74aa84a... one blob per resource, two-level fanout +``` + +A blob's name is the SHA-1 of the resource's **original** bytes, but the file on +disk is not those bytes: each blob is wrapped in NWCompressedBuffer framing, a +24-byte `NSYC` header followed by a zstd frame. Hashing the file directly will +not match its name, which is the obvious +first thing to try and the obvious first thing to be confused by. Strip the +header first: + +``` +tail -c +25 | zstd -dc | sha1sum # == the blob's filename +``` + +The header carries the uncompressed length as a little-endian `uint32` at offset +12, so the decompressed size is checkable without decompressing. Compression is +worth roughly a 4:1 +saving on hak content: a 250 MB hak emitted 2296 blobs totalling 59 MB on disk +against 249 MB of resources, as recorded in the sidecar's `on_disk_bytes` and +`total_bytes`. + ## Hidden compatibility aliases Existing scripts may continue using these names indefinitely. They are accepted