Compare commits

..
4 Commits
Author SHA1 Message Date
archvillainette f23009ed50 verify is what tells you which keys to purge (#91)
Running the repair disproved the advice #90 landed an hour earlier.

"Purge the zone, then believe `verify`" assumed the stale set was unknowable. It is not. `verify` reads the edge, so the run straight after a repair names every key the edge is still serving stale — a survey, not a verdict. Purge those, re-run, and the second run is the verdict.

The measured numbers are the whole argument:

| | |
| --- | --- |
| blobs rewritten at the origin | 2,603 |
| blobs stale at the edge | **8** |

All eight were ones a failed player sync had pulled ninety minutes before the backfill. The edge only caches what someone fetched, so purging the zone would have cooled 69,169 objects to fix 8.

Full sweep after the targeted purge: `verified 69177 of 69177 blobs behind 72544 resources: 0 failures, 14887519535 bytes checked`. #75's gate is met.

Docs only. Runbook side in sow-platform.

Refs #88, #89, #75.

🤖 Generated with [Claude Code](https://claude.com/claude-code)Reviewed-on: #91
Reviewed-by: xtul <mpiasecki720@protonmail.com>
Co-authored-by: vickydotbat <vickydotbat@tutamail.com>
2026-08-01 09:20:16 +00:00
archvillainette 3cac6e9484 fix(project): a module resref names a file, not a resource (#93)
`module.resref` is the name of the built `.mod` on disk, so the 16-byte resref limit never applied to it — NWN:EE module file names are routinely longer. The blanket check rejected `ShadowsOverWestgate` (19 characters) and blocked sow-module#60:

```
crucible module build
  module.resref "ShadowsOverWestgate" exceeds 16 characters
```

## What changed

`internal/project/project.go` — `module.resref` is validated as a **file name** now, which still rejects a resref that is a path or empty.

The 16-character limit is kept where the value really does become a resref: a project with `paths.assets` and no `haks[]` names its single generated HAK after the module resref (`build.go:1144`), and a HAK name is a resref the engine loads. That case now says what to do about it instead of refusing every long module name.

## Verified

- 3 new tests in `internal/project`: a long module name validates and produces `ShadowsOverWestgate.mod`; a resref containing a path is rejected; a long resref that would name a generated HAK is still rejected.
- `make check` green.
- `crucible module build` in sow-module writes `module/ShadowsOverWestgate.mod` (70 resources).
- `crucible topdata validate` in sow-topdata still passes — its assets live under `topdata.assets`, not `paths.assets`, so the HAK guard does not bite.

Merge this **first**: sow-module's rename PR cannot go green in CI until this lands and its `flake.lock` is bumped.

Refs ShadowsOverWestgate/sow-module#60

🤖 Generated with [Claude Code](https://claude.com/claude-code)Reviewed-on: #93

Co-authored-by: vickydotbat <vickydotbat@tutamail.com>
2026-08-01 08:59:25 +00:00
archvillainette 3f78197f0a Purge the edge after a repair, or verify answers per PoP (#89) (#90)
build-binaries / build-binaries (push) Successful in 2m33s
#89 asked for a decision. This is it, and it is the laziest of the three options listed there: **purge the whole pull zone by hand after a repair, one call, documented in the repair procedure.**

Why not the other two:

- Purging from `emit --verify` needs a CDN credential `emit` deliberately does not hold, and `emit` reports how many blobs it wrote, never which ones — so it could not target the keys anyway.
- Waiting out the TTL means 30 days.

Whole-zone rather than per-key costs a cold cache on a zone whose objects are mostly cold, and a repair scatters thousands of keys across the tree regardless.

Also answers the question #89 left open: **the zone does not negative-cache.** A missing key answers 404 with `cache-control: no-cache` and `cdn-cache: MISS`, still MISS on an immediate retry (checked credential-free, 2026-08-01).

Docs only — `docs/command-surface.md` and `nwsync`'s usage text. The procedure itself lives in sow-platform's NWSync runbook, next to the zone it acts on.

Closes #89.

🤖 Generated with [Claude Code](https://claude.com/claude-code)Reviewed-on: #90
Reviewed-by: xtul <mpiasecki720@protonmail.com>
Co-authored-by: vickydotbat <vickydotbat@tutamail.com>
2026-07-31 23:13:55 +00:00
archvillainette 7cc53aeb68 fix(nwsync): declare Frame_Content_Size on every blob, and verify what is published (#87)
build-binaries / build-binaries (push) Successful in 2m40s
Closes #86. Closes #85.

These land together on purpose. Fixing the encoder alone changes nothing for the blobs already in the zone, because `emit` skips whatever is already present.

## #86 — the framing fix

`klauspost/compress` omits the zstd `Frame_Content_Size` field for inputs under 256 bytes, which the format permits. Reference libzstd never does, so the NWN client — which sizes its output buffer from `ZSTD_getFrameContentSize` and has therefore never met a frame without one — rejected roughly 6% of our blobs outright. Any single one stops a sync dead, so no client could complete a sync of the live manifest.

No encoder option changes this, so `compressBlob` re-headers the affected frames into the shape libzstd itself emits: `Single_Segment_flag` set, `Window_Descriptor` dropped, and the freed byte spent on a one-byte `Frame_Content_Size`. Same length in, same length out, and the same descriptor byte (`0x24`) the issue recorded from libzstd.

`compressBlob` then asserts its own output. An encoder upgrade that finds another way to omit the field would otherwise reproduce #86 in silence, and a blob is skipped by every later emit once written.

`emitter_version` goes to `2`, so `assemble` refuses to merge an index written by the encoder that omitted the field.

**Proved against the reference decoder, not just a round trip.** A real emitted 175-byte blob:

```
Frames  Skips  Compressed  Uncompressed  Ratio  Check  Filename
     1      0      48   B       175   B  3.646  XXH64  frame.zst
c59d6620d4ffd4bf3fe73df43b19b7afcfe8fea4  -            <- zstd -dc | sha1sum
c59d6620d4ffd4bf3fe73df43b19b7afcfe8fea4               <- the blob's own name
```

Before the fix that `Uncompressed` column was blank.

## #85 — `nwsync verify`

`crucible nwsync verify <manifest-sha1>` reads a manifest and its blobs back through the **public pull zone**, with no credential, because what matters is the bytes a client is served, edge behaviour included. Every distinct blob is decompressed and hashed; failures are reported per blob as missing / malformed framing / size mismatch / hash mismatch, and the exit code is 1.

- `--sample N` makes a routine check cheap against a manifest that is ~69,000 blobs and 15 GB; the default is a full sweep.
- `--base URL` / `NWSYNC_PULL_BASE` overrides the public host.
- The manifest is checked against its own sha1 before a single blob is fetched.
- `emit --verify` applies the same check where `emit` would otherwise trust presence, and replaces a stored blob that is not what its name claims. This is what makes the #86 blobs repairable.

## Why the existing checks missed this

Both new checks assert the **frame property**, not just a round trip. The conformance suite (#59) compares decompressed bytes, so a frame that decodes correctly passes regardless of its header; and the earlier zone audit decompressed 68 blobs with the `zstd` CLI, a *more* capable decoder than the client's, which certified exactly the blobs the client rejects.

## Checks

`make check` and `make smoke` green. Second commit is the fixes from a two-axis review of the first.

## Not in this PR

Three follow-ups, filed separately: the backfill has not been run, replacing a blob does not purge the pull-zone edge cache, and #85's runbook line belongs to `sow-platform`.

🤖 Generated with [Claude Code](https://claude.com/claude-code)Reviewed-on: #87

Co-authored-by: vickydotbat <vickydotbat@tutamail.com>
2026-07-31 22:33:12 +00:00
4 changed files with 104 additions and 2 deletions
+17
View File
@@ -97,6 +97,23 @@ broken — is skipped by every later run forever and no backfill repairs it. Wit
and replaced when it does not match. It costs a full GET per existing blob, so
it is a repair pass, not the default.
**After a repair, `verify` is what tells you which keys to purge.** A repair is
the one thing that makes a key serve different bytes than it did before, and the
edge caches these objects for 30 days precisely because that normally cannot
happen. The two commands look at different copies on purpose: `emit --verify`
repairs the **origin**, `verify` reads the **edge**. So a `verify` run straight
after a repair is not a verdict — it is a survey, and every blob it still calls
bad is one the edge is serving stale. Purge exactly those, then re-run it; only
that second run is the verdict.
Purging the keys `verify` names beats purging the zone, because the edge only
ever cached what somebody actually fetched: the 2026-08-01 repair rewrote 2,603
blobs at the origin and left 8 stale at the edge. The purge belongs in the
repair procedure rather than in `emit`, which reports how many blobs it wrote
and never which ones — so it could not target one even with a CDN credential,
which it deliberately does not hold (#89; the procedure itself is in
sow-platform's NWSync runbook).
`emit` uploads blobs first and the index last, so the presence of an index is
the publication marker: an artifact whose emit died halfway leaves real blobs in
the zone and no index. Blob names are content hashes, so re-running skips
+3
View File
@@ -67,6 +67,9 @@ check on a published blob upstream of a player's client.
--verify makes emit hash what it would otherwise skip. emit normally treats a
blob's presence as proof of its contents, so without this an object written
truncated, or written by an emitter since found broken, is skipped forever.
--verify repairs the storage zone, while verify reads the edge in front of it.
So a verify run right after a repair is a survey, not a verdict: it names the
keys the edge still serves stale. Purge those, then run it again.
--out DIR writes to a local repository tree instead of uploading, which is the
conformance path against upstream nwn_nwsync_write. Without it, the zone comes
+12 -2
View File
@@ -560,8 +560,18 @@ func (p *Project) ValidateLayout() error {
if strings.TrimSpace(p.Config.Module.ResRef) == "" {
failures = append(failures, errors.New("module.resref is required"))
}
if len(p.Config.Module.ResRef) > 16 {
failures = append(failures, fmt.Errorf("module.resref %q exceeds 16 characters", p.Config.Module.ResRef))
// module.resref names the built .mod FILE, so the 16-byte resref limit does not
// apply to it — NWN:EE module file names are routinely longer. It is validated as
// a file name instead. The limit still binds when the same value has to be a real
// resref: with no haks configured, an asset project names its single generated HAK
// after it, and a HAK name is a resref the engine loads.
if err := validateOutputFileName("module.resref", p.Config.Module.ResRef+".mod", ".mod"); err != nil {
failures = append(failures, err)
}
if len(p.Config.Module.ResRef) > 16 && strings.TrimSpace(p.Config.Paths.Assets) != "" && len(p.Config.HAKs) == 0 {
failures = append(failures, fmt.Errorf(
"module.resref %q exceeds 16 characters and would name this project's generated HAK; configure haks[] with a shorter name",
p.Config.Module.ResRef))
}
if strings.TrimSpace(p.Config.Paths.Source) == "" && strings.TrimSpace(p.Config.Paths.Assets) == "" && !p.HasTopData() {
failures = append(failures, errors.New("at least one of paths.source, paths.assets, or topdata.source is required"))
+72
View File
@@ -1093,6 +1093,78 @@ func TestValidateLayoutAllowsMissingAssetsDir(t *testing.T) {
}
}
// module.resref names the built .mod FILE, not a resource inside an archive, so the
// 16-byte resref limit does not apply to it. NWN:EE module file names are commonly
// longer (ShadowsOverWestgate.mod is 19). The limit still binds everywhere a resref
// really is a resref — see TestValidateLayoutRejectsLongResRefWhenItNamesAHAK.
func TestValidateLayoutAllowsLongModuleResRef(t *testing.T) {
root := t.TempDir()
mkdirAll(t, filepath.Join(root, "src"))
mkdirAll(t, filepath.Join(root, "build"))
proj := &Project{
Root: root,
Config: Config{
Module: ModuleConfig{Name: "Shadows Over Westgate", ResRef: "ShadowsOverWestgate"},
Paths: PathConfig{Source: "src", Build: "build"},
},
}
if err := proj.ValidateLayout(); err != nil {
t.Fatalf("ValidateLayout rejected a 19-character module file name: %v", err)
}
if got, want := filepath.Base(proj.ModuleArchivePath()), "ShadowsOverWestgate.mod"; got != want {
t.Fatalf("ModuleArchivePath() = %q, want %q", got, want)
}
}
// A module.resref that is not a usable file name is still rejected.
func TestValidateLayoutRejectsModuleResRefThatIsAPath(t *testing.T) {
root := t.TempDir()
mkdirAll(t, filepath.Join(root, "src"))
proj := &Project{
Root: root,
Config: Config{
Module: ModuleConfig{Name: "Test", ResRef: "../escape/mod"},
Paths: PathConfig{Source: "src", Build: "build"},
},
}
err := proj.ValidateLayout()
if err == nil {
t.Fatal("ValidateLayout accepted a module.resref containing a path")
}
if !strings.Contains(err.Error(), "module.resref") {
t.Fatalf("error does not name the offending field: %v", err)
}
}
// When a project declares no haks, the module resref becomes the name of the single
// generated HAK — and a HAK name IS a resref the engine loads. The limit applies
// there, so a long name is only allowed for projects that build no HAKs.
func TestValidateLayoutRejectsLongResRefWhenItNamesAHAK(t *testing.T) {
root := t.TempDir()
mkdirAll(t, filepath.Join(root, "src"))
mkdirAll(t, filepath.Join(root, "assets"))
proj := &Project{
Root: root,
Config: Config{
Module: ModuleConfig{Name: "Shadows Over Westgate", ResRef: "ShadowsOverWestgate"},
Paths: PathConfig{Source: "src", Assets: "assets", Build: "build"},
},
}
err := proj.ValidateLayout()
if err == nil {
t.Fatal("ValidateLayout accepted a 19-character name for a generated HAK")
}
if !strings.Contains(err.Error(), "16") {
t.Fatalf("error does not explain the resref limit: %v", err)
}
}
// paths.build is an OUTPUT dir the builder creates (MkdirAll) before writing, so
// a bare clone with no build dir yet must still validate/build with no pre-step
// (R2/parity). Only a build path that exists but is not a directory is an error.