docs: describe the on-disk shape of an emitted NWSync tree #77
@@ -70,6 +70,35 @@ the zone and no index. Blob names are content hashes, so re-running skips
|
|||||||
whatever already landed, and `assemble` fails closed on an artifact with no
|
whatever already landed, and `assemble` fails closed on an artifact with no
|
||||||
index rather than publishing a manifest that is missing a hak.
|
index rather than publishing a manifest that is missing a hak.
|
||||||
|
|
||||||
|
### What an emitted tree looks like
|
||||||
|
|
||||||
|
`--out DIR` produces the same tree `emit` would upload, which makes it the way
|
||||||
|
to check a zone by hand without touching one:
|
||||||
|
|
||||||
|
```
|
||||||
|
<artifact-sha>.nsym binary index
|
||||||
|
<artifact-sha>.nsym.json the same index, readable
|
||||||
|
data/sha1/a7/4a/a74aa84a... one blob per resource, two-level fanout
|
||||||
|
```
|
||||||
|
|
||||||
|
A blob's name is the SHA-1 of the resource's **original** bytes, but the file on
|
||||||
|
disk is not those bytes: each blob is wrapped in NWCompressedBuffer framing, a
|
||||||
|
24-byte `NSYC` header followed by a zstd frame. Hashing the file directly will
|
||||||
|
not match its name, which is the obvious
|
||||||
|
first thing to try and the obvious first thing to be confused by. Strip the
|
||||||
|
header first:
|
||||||
|
|
||||||
|
```
|
||||||
|
tail -c +25 <blob> | zstd -dc | sha1sum # == the blob's filename
|
||||||
|
```
|
||||||
|
|
||||||
|
The header carries the uncompressed length as a little-endian `uint32` at offset
|
||||||
|
12, so the decompressed size is checkable without decompressing. Compression is
|
||||||
|
worth roughly a 4:1
|
||||||
|
saving on hak content: a 250 MB hak emitted 2296 blobs totalling 59 MB on disk
|
||||||
|
against 249 MB of resources, as recorded in the sidecar's `on_disk_bytes` and
|
||||||
|
`total_bytes`.
|
||||||
|
|
||||||
## Hidden compatibility aliases
|
## Hidden compatibility aliases
|
||||||
|
|
||||||
Existing scripts may continue using these names indefinitely. They are accepted
|
Existing scripts may continue using these names indefinitely. They are accepted
|
||||||
|
|||||||
Reference in New Issue
Block a user