From f23009ed5031f9031ed5a1a8a32ffba3bebaf822 Mon Sep 17 00:00:00 2001 From: vickydotbat Date: Sat, 1 Aug 2026 09:20:16 +0000 Subject: [PATCH] verify is what tells you which keys to purge (#91) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Running the repair disproved the advice #90 landed an hour earlier. "Purge the zone, then believe `verify`" assumed the stale set was unknowable. It is not. `verify` reads the edge, so the run straight after a repair names every key the edge is still serving stale — a survey, not a verdict. Purge those, re-run, and the second run is the verdict. The measured numbers are the whole argument: | | | | --- | --- | | blobs rewritten at the origin | 2,603 | | blobs stale at the edge | **8** | All eight were ones a failed player sync had pulled ninety minutes before the backfill. The edge only caches what someone fetched, so purging the zone would have cooled 69,169 objects to fix 8. Full sweep after the targeted purge: `verified 69177 of 69177 blobs behind 72544 resources: 0 failures, 14887519535 bytes checked`. #75's gate is met. Docs only. Runbook side in sow-platform. Refs #88, #89, #75. 🤖 Generated with [Claude Code](https://claude.com/claude-code)Reviewed-on: https://git.westgate.pw/ShadowsOverWestgate/sow-tools/pulls/91 Reviewed-by: xtul Co-authored-by: vickydotbat --- docs/command-surface.md | 23 ++++++++++++++--------- internal/nwsync/run.go | 5 +++-- 2 files changed, 17 insertions(+), 11 deletions(-) diff --git a/docs/command-surface.md b/docs/command-surface.md index 05d3329..cf18ea0 100644 --- a/docs/command-surface.md +++ b/docs/command-surface.md @@ -97,17 +97,22 @@ broken — is skipped by every later run forever and no backfill repairs it. Wit and replaced when it does not match. It costs a full GET per existing blob, so it is a repair pass, not the default. -**After a repair, purge the pull zone before believing `verify`.** A repair is +**After a repair, `verify` is what tells you which keys to purge.** A repair is the one thing that makes a key serve different bytes than it did before, and the edge caches these objects for 30 days precisely because that normally cannot -happen. The two commands therefore look at different copies on purpose: `emit ---verify` repairs the **origin**, `verify` reads the **edge**, and in between a -warm PoP still answers with the old bytes while a cold one answers with the new. -Until the zone is purged `verify`'s verdict is per-PoP and settles nothing — a -pass is not proof, and a failure is not the repair having failed. The purge is -one call against the pull zone; it belongs in the repair procedure rather than -in `emit`, which holds a storage credential and no CDN one (sow-tools#89, and -the procedure itself is in sow-platform's NWSync runbook). +happen. The two commands look at different copies on purpose: `emit --verify` +repairs the **origin**, `verify` reads the **edge**. So a `verify` run straight +after a repair is not a verdict — it is a survey, and every blob it still calls +bad is one the edge is serving stale. Purge exactly those, then re-run it; only +that second run is the verdict. + +Purging the keys `verify` names beats purging the zone, because the edge only +ever cached what somebody actually fetched: the 2026-08-01 repair rewrote 2,603 +blobs at the origin and left 8 stale at the edge. The purge belongs in the +repair procedure rather than in `emit`, which reports how many blobs it wrote +and never which ones — so it could not target one even with a CDN credential, +which it deliberately does not hold (#89; the procedure itself is in +sow-platform's NWSync runbook). `emit` uploads blobs first and the index last, so the presence of an index is the publication marker: an artifact whose emit died halfway leaves real blobs in diff --git a/internal/nwsync/run.go b/internal/nwsync/run.go index c221797..0617853 100644 --- a/internal/nwsync/run.go +++ b/internal/nwsync/run.go @@ -67,8 +67,9 @@ check on a published blob upstream of a player's client. --verify makes emit hash what it would otherwise skip. emit normally treats a blob's presence as proof of its contents, so without this an object written truncated, or written by an emitter since found broken, is skipped forever. ---verify repairs the storage zone, while verify reads the edge in front of it, -so purge the pull zone after a repair or verify answers differently per PoP. +--verify repairs the storage zone, while verify reads the edge in front of it. +So a verify run right after a repair is a survey, not a verdict: it names the +keys the edge still serves stale. Purge those, then run it again. --out DIR writes to a local repository tree instead of uploading, which is the conformance path against upstream nwn_nwsync_write. Without it, the zone comes