crucible depot: Increment 1 core (status/push/verify/get/pull + local/cdn/bunny backends) (#30)

Implements Increment 1 of `docs/superpowers/specs/2026-07-04-crucible-depot-core-design.md`: a new stdlib-only `internal/depot` package wired into the dispatcher.

- `crucible depot status|push|verify|get|pull` with backends `local`/`cdn`/`bunny`; exit contract `0` clean / `1` drift / `2` unconfirmed-only / `64` usage / `70` internal.
- Presence is always probed against the real target (IPv4 1-byte range GET; HEAD is banned with a regression-tripwire test). `unconfirmed` is a distinct state, never collapsed into `missing`.
- No prompting anywhere: missing `BUNNY_STORAGE_*` env fails closed (read path included), enforced by a no-stdin test.
- Uploads: probe-then-PUT with `Checksum: <UPPER-sha>`; read/write key split (`BUNNY_STORAGE_READ_PASSWORD` falls back to `BUNNY_STORAGE_PASSWORD`).
- Field-driven fix included: per-probe transient retry (curl `--retry 2` equivalent) — without it a real 1490-blob CDN sweep reported 1222 false-unconfirmed; with it, 1490/1490 present in 74s, exit 0.
- Registry: depot `Wired: true`, joins the interactive menu; stale "(SeaweedFS)" wording removed.

Tests: unit + httptest fake-Bunny (probe sequence, Checksum header, key split, 428 throttling → exit 2) + local→bunny integration (drift → push → clean → idempotent no-second-PUT; incremental pull). `make check` green.

**Merge ordering:** this merges FIRST; the companion `sow-assets-manifest#crucible-depot-cutover` PR needs its flake input bumped to include this.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Reviewed-on: #30
Reviewed-by: xtul <mpiasecki720@protonmail.com>
Co-authored-by: vickydotbat <vickydotbat@tutamail.com>
Co-committed-by: vickydotbat <vickydotbat@tutamail.com>

docs/superpowers/plans/2026-07-04-crucible-depot-core.md

@@ -0,0 +1,330 @@
+# Crucible Depot Core (Increment 1) Implementation Plan
+
+> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
+
+**Goal:** Implement `crucible depot {status,push,verify,get,pull}` with local/cdn/bunny backends in sow-tools, then cut sow-assets-manifest over to it and delete the bash it replaces.
+
+**Architecture:** New stdlib-only `internal/depot` package with its own `Run(args) int` entrypoint (exit contract 0/1/2/70 is richer than `internal/app.Run`'s 0/1, so dispatch special-cases depot instead of routing through `app.Run`). Backends implement a small interface; a sweep engine does parallel range-GET probes with serial re-confirm. Cutover repo work happens in sow-assets-manifest on its own branch.
+
+**Tech Stack:** Go 1.26, stdlib `net/http`/`crypto/sha256`/`sync`, `gopkg.in/yaml.v3` (already a dep — add nothing to go.mod, vendorHash must not change).
+
+**Spec:** `docs/superpowers/specs/2026-07-04-crucible-depot-core-design.md` — read it before any task; it is authoritative.
+
+## Global Constraints
+
+- **No new Go dependencies.** stdlib + existing `gopkg.in/yaml.v3` only.
+- **HEAD is banned** for existence checks. All probes: IPv4, `GET` with `Range: bytes=0-0`, expect `206` (accept any 2xx). A test must fail if HEAD is introduced.
+- **Never prompt, never read stdin** for credentials — read path included. Missing key → clear error on stderr, exit `70`.
+- **No cache of "what I uploaded"** — presence is always probed against the real target backend.
+- `unconfirmed` is a distinct state from `missing`; exit `2` when unconfirmed-only.
+- Exit codes: `0` clean, `1` drift (referenced-but-absent), `2` unconfirmed-only, `64` usage, `70` internal/backend error.
+- Blob key: `sha256/<sha[0:2]>/<sha[2:4]>/<sha>`; sha must match `^[0-9a-f]{64}$`.
+- Env names (unchanged from bash): `DEPOT_CDN_BASE`/`BUNNY_CDN_BASE` (cdn read base, default `https://cdn-a7f3k9.westgate.pw`), `BUNNY_STORAGE_HOST` (**required for bunny, no default** — spec overrides bash), `BUNNY_STORAGE_ZONE` (default `sow-assets-depot`), `BUNNY_STORAGE_READ_PASSWORD` (falls back to `BUNNY_STORAGE_PASSWORD`) for reads/probes, `BUNNY_STORAGE_PASSWORD` for PUT, `DEPOT_PROBE_JOBS` (default 16), `DEPOT_JOBS` (default 16), `DEPOT_CONNECT_TIMEOUT` (default 10s), `DEPOT_PROBE_MAX_TIME` (default 30s), `DEPOT_CONFIRM_RETRIES` (default 3), `DEPOT_CONFIRM_MAX` (default 200, 0 = unlimited).
+- Upload: `PUT` with `AccessKey: <write key>` and `Checksum: <UPPERCASE sha>` headers; each sha uploaded at most once per run; probe-before-put.
+- Every downloaded blob is re-hashed; deleted on mismatch.
+- `cdn` backend writes fail closed (error, never fake success).
+- Verify with `make vet test` in sow-tools (run inside `nix develop` if go missing from PATH).
+- Commit after each task. **Never commit to main.** sow-tools branch: `depot-core-increment-1`. sow-assets-manifest branch: `crucible-depot-cutover`. sow-tools has an unrelated modified spec file on main — leave it out of commits unless the task says otherwise.
+
+---
+
+### Task 1: Package skeleton — config + manifest parsing
+
+**Files:**
+- Create: `internal/depot/config.go`, `internal/depot/config_test.go`
+- Create: `internal/depot/manifest.go`, `internal/depot/manifest_test.go`
+
+**Interfaces (Produces):**
+
+```go
+package depot
+
+// config.go
+type Config struct {
+    CDNBase        string        // DEPOT_CDN_BASE || BUNNY_CDN_BASE || default
+    StorageHost    string        // BUNNY_STORAGE_HOST, no default
+    StorageZone    string        // BUNNY_STORAGE_ZONE, default "sow-assets-depot"
+    ReadKey        string        // BUNNY_STORAGE_READ_PASSWORD || BUNNY_STORAGE_PASSWORD
+    WriteKey       string        // BUNNY_STORAGE_PASSWORD
+    ProbeJobs      int           // DEPOT_PROBE_JOBS, default 16
+    Jobs           int           // DEPOT_JOBS, default 16
+    ConnectTimeout time.Duration // DEPOT_CONNECT_TIMEOUT seconds, default 10s
+    ProbeMaxTime   time.Duration // DEPOT_PROBE_MAX_TIME seconds, default 30s
+    ConfirmRetries int           // DEPOT_CONFIRM_RETRIES, default 3
+    ConfirmMax     int           // DEPOT_CONFIRM_MAX, default 200 (0 = unlimited)
+}
+func LoadConfig(getenv func(string) string) Config
+
+// manifest.go
+// ReferencedSHAs parses every *.yml in dir (schema: assets[].{path,sha256,size,restype,hak})
+// and returns the deduplicated sha set plus per-sha size (for pull).
+func ReferencedSHAs(dir string) (map[string]int64, error)
+func ValidSHA(s string) bool // ^[0-9a-f]{64}$
+func BlobKey(sha string) string // "sha256/ab/cd/<sha>"
+```
+
+`LoadConfig` takes `getenv` so tests inject env without `t.Setenv` races. Invalid ints/durations in env → fall back to default (do not error). `ReferencedSHAs` errors on: unreadable dir, no `*.yml` files, YAML parse failure, or an entry whose `sha256` fails `ValidSHA` (report file + path). Manifest struct:
+
+```go
+type manifestFile struct {
+    Assets []struct {
+        Path   string `yaml:"path"`
+        SHA256 string `yaml:"sha256"`
+        Size   int64  `yaml:"size"`
+    } `yaml:"assets"`
+}
+```
+
+- [ ] **Step 1:** `git -C sow-tools switch -c depot-core-increment-1` (from main).
+- [ ] **Step 2:** Write failing tests: `TestLoadConfigDefaults`, `TestLoadConfigReadKeyFallback` (READ unset → WriteKey value; READ set → its own), `TestLoadConfigBadIntFallsBack`, `TestBlobKey` (`BlobKey("0b1d…")=="sha256/0b/1d/0b1d…"`), `TestValidSHA` (rejects uppercase, short, non-hex), `TestReferencedSHAs` (temp dir with two yml files sharing one sha → dedup; bad sha → error naming the file; empty dir → error).
+- [ ] **Step 3:** `go test ./internal/depot/` — verify FAIL (compile error is fine).
+- [ ] **Step 4:** Implement `config.go` + `manifest.go` minimally.
+- [ ] **Step 5:** `go test ./internal/depot/` PASS; `gofmt -l -w internal/depot`.
+- [ ] **Step 6:** Commit: `feat(depot): config resolution and manifest sha parsing`.
+
+### Task 2: Local backend
+
+**Files:**
+- Create: `internal/depot/backend.go`, `internal/depot/local.go`, `internal/depot/local_test.go`
+
+**Interfaces (Produces):**
+
+```go
+// backend.go
+type ProbeState int
+const (
+    Present ProbeState = iota
+    Absent
+    Unconfirmed
+)
+type Backend interface {
+    Name() string
+    // Probe returns the existence state of one sha. transient=true means a
+    // retry might change the answer (feeds the serial confirm loop).
+    Probe(ctx context.Context, sha string) (state ProbeState, transient bool, err error)
+    // Get fetches sha into dest (temp file + rename), re-hashes, deletes on mismatch.
+    Get(ctx context.Context, sha, dest string) error
+    // Put uploads bytes from src for sha. Read-only backends return an error.
+    Put(ctx context.Context, sha, src string) error
+}
+
+// local.go
+type LocalBackend struct{ Root string }  // implements Backend; Name()=="local"
+```
+
+Local semantics: `Probe` = `os.Stat(Root/BlobKey(sha))`, never transient. `Put` = copy to temp file in dest dir then `os.Rename` (MkdirAll parents). `Get` = copy out + re-hash (hash the source bytes while copying via `io.TeeReader` into `sha256.New()`), delete dest and error on mismatch. `err` from Probe only for real I/O errors (permission), not absence.
+
+- [ ] **Step 1:** Failing tests: `TestLocalRoundTrip` (Put→Probe Present→Get→bytes equal), `TestLocalProbeAbsent`, `TestLocalGetHashMismatch` (hand-write a corrupt blob file at the keyed path, Get returns error and dest does not exist), `TestLocalPutIdempotent`.
+- [ ] **Step 2:** Verify FAIL. **Step 3:** Implement. **Step 4:** PASS + gofmt.
+- [ ] **Step 5:** Commit: `feat(depot): backend interface and local backend`.
+
+### Task 3: Remote backends (cdn, bunny) with fake-Bunny tests
+
+**Files:**
+- Create: `internal/depot/remote.go`, `internal/depot/remote_test.go`
+
+**Interfaces (Consumes):** `Backend`, `ProbeState`, `Config`, `BlobKey`.
+**Produces:**
+
+```go
+// NewBackend returns the backend for name ("local"|"cdn"|"bunny").
+// localRoot is used only for "local". Fails closed: "bunny" with empty
+// StorageHost or empty ReadKey returns an error (never prompts);
+// unknown name returns an error.
+func NewBackend(name, localRoot string, cfg Config) (Backend, error)
+
+type httpBackend struct { ... } // shared by cdn and bunny
+```
+
+Implementation requirements:
+
+```go
+// IPv4-only transport — REQUIRED, spec bans IPv6/HEAD probe hazards.
+transport := &http.Transport{
+    DialContext: func(ctx context.Context, network, addr string) (net.Conn, error) {
+        d := net.Dialer{Timeout: cfg.ConnectTimeout}
+        return d.DialContext(ctx, "tcp4", addr)
+    },
+    MaxIdleConnsPerHost: cfg.ProbeJobs,
+}
+client := &http.Client{Transport: transport, Timeout: cfg.ProbeMaxTime}
+```
+
+- **Probe** (both remotes): `GET <base>/<BlobKey(sha)>` with header `Range: bytes=0-0`; drain/close body. 2xx → `Present`; 404/410 → `Absent`; anything else (incl. transport error) → `Unconfirmed, transient=true`. **Never http.Head / MethodHead.**
+- cdn: base = `cfg.CDNBase`, no auth headers, `Put` returns `errors.New("cdn backend is read-only")`.
+- bunny: probe/read against storage `https://<StorageHost>/<StorageZone>/<BlobKey(sha)>` with `AccessKey: <ReadKey>`. `Get`: try CDN base first, on non-2xx fall back to storage URL with ReadKey (mirrors bash `_depot_bunny_get`); re-hash, delete on mismatch. `Put`: probe first, skip if Present; else `PUT` storage URL, headers `AccessKey: <WriteKey>`, `Checksum: <strings.ToUpper(sha)>`, body streamed from src file; non-2xx → error. `Put` with empty WriteKey → error before any I/O.
+- Both `Get`s write via temp file + rename and re-hash exactly like local.
+
+httptest note: `httptest.NewServer` listens on `127.0.0.1`, which the tcp4 dialer reaches — no test shim needed.
+
+- [ ] **Step 1:** Failing tests against an `httptest` fake Bunny (a handler recording method+path+headers, serving a blob map):
+  - `TestProbeUsesRangeGetNotHead` — probe a present sha; assert recorded request has `Method=="GET"` and `Range=="bytes=0-0"`; **also** assert the depot source contains no HEAD probe: the handler returns 500 for any HEAD request, and the test asserts zero HEAD requests were received across the whole test run (this is the anti-HEAD regression tripwire from field note 3).
+  - `TestProbeStates` — 200/206→Present, 404→Absent, 428→Unconfirmed+transient, 503→Unconfirmed+transient.
+  - `TestBunnyPutSequence` — Put of an absent sha: recorded sequence is [GET range probe, PUT]; PUT has `Checksum` = uppercase sha and `AccessKey` = write key; Put of a present sha: probe only, no PUT.
+  - `TestReadWriteKeySplit` — probe uses ReadKey, PUT uses WriteKey (distinct values asserted).
+  - `TestBunnyNoReadKeyFailsClosed` — `NewBackend("bunny",…)` with empty read key returns error **without reading stdin**: replace `os.Stdin` with the read end of an `os.Pipe()` for the test; after the call, write+close and confirm the pipe is undrained (or simply assert error occurs with no stdin interaction because the code path has no stdin reference — grep-based assertion acceptable: test fails if `internal/depot` sources mention `os.Stdin` or `/dev/tty`).
+  - `TestBunnyNoStorageHostFailsClosed`.
+  - `TestGetHashMismatchDeletes` — server returns wrong bytes; Get errors, dest removed.
+  - `TestCDNPutReadOnly`.
+- [ ] **Step 2:** Verify FAIL. **Step 3:** Implement `remote.go`. **Step 4:** PASS + gofmt + `go vet ./...`.
+- [ ] **Step 5:** Commit: `feat(depot): cdn and bunny HTTP backends with fake-Bunny tests`.
+
+### Task 4: Sweep engine — parallel probe + serial confirm
+
+**Files:**
+- Create: `internal/depot/sweep.go`, `internal/depot/sweep_test.go`
+
+**Interfaces (Consumes):** `Backend`, `Config`, `ProbeState`.
+**Produces:**
+
+```go
+type SweepResult struct {
+    Present     []string
+    Missing     []string // confirmed absent (404/410 after retries)
+    Unconfirmed []string // probe budget exhausted, state unknown
+}
+// Sweep probes every sha against b: one parallel pass (cfg.ProbeJobs workers,
+// sync.WaitGroup + buffered-channel work queue), then any non-Present result
+// is re-probed serially with linear backoff (attempt t sleeps t seconds,
+// cfg.ConfirmRetries attempts) — unless the non-Present count exceeds
+// cfg.ConfirmMax (>0), in which case the remainder is reported Unconfirmed
+// without serial re-probe (never collapsed into Missing).
+// A transient result that stays non-2xx after retries is Unconfirmed;
+// a 404/410 is Missing. Backend errors (non-transient) abort with error.
+func Sweep(ctx context.Context, b Backend, shas []string, cfg Config, progress io.Writer) (SweepResult, error)
+```
+
+Sort `shas` before sweeping (deterministic output). Progress: print `probed X/N` roughly every 1000 blobs to `progress` (may be `io.Discard`). For tests, make the backoff sleeper injectable: package-level `var confirmSleep = time.Sleep`, overridden in tests.
+
+- [ ] **Step 1:** Failing tests using a stub Backend (function fields):
+  - `TestSweepAllPresent`, `TestSweepMissing` (404 → Missing, no retries beyond first serial confirm).
+  - `TestSweepThrottledUnconfirmed` — stub always returns Unconfirmed/transient → after retries lands in `Unconfirmed`, never `Missing`; assert `confirmSleep` called with 1s then 2s.
+  - `TestSweepTransientRecovers` — first probe 428, serial re-probe 206 → Present.
+  - `TestSweepConfirmMaxCap` — 5 non-present, ConfirmMax=2 → all 5 Unconfirmed, zero serial re-probes.
+  - `TestSweepParallelBounded` — 100 shas, ProbeJobs=4; stub counts concurrent in-flight probes (atomic), assert max ≤ 4.
+- [ ] **Step 2:** FAIL. **Step 3:** Implement. **Step 4:** PASS + gofmt.
+- [ ] **Step 5:** Commit: `feat(depot): parallel probe sweep with serial confirm and unconfirmed state`.
+
+### Task 5: Commands + Run entrypoint + integration test
+
+**Files:**
+- Create: `internal/depot/run.go`, `internal/depot/run_test.go`, `internal/depot/integration_test.go`
+
+**Interfaces (Consumes):** everything above.
+**Produces:**
+
+```go
+// Run executes a depot subcommand. args[0] is the subcommand
+// (status|push|verify|get|pull); returns the process exit code.
+// Exit: 0 clean, 1 drift, 2 unconfirmed-only, 64 usage, 70 internal.
+func Run(args []string, stdout, stderr io.Writer, getenv func(string) string) int
+```
+
+Flag parsing: one `flag.FlagSet` per subcommand (stdlib; `internal/depot` is a fresh package, the hand-rolled loop in `internal/app` is not a constraint here). Flags per spec:
+
+```
+status  [--manifests DIR] [--source LOCAL_DEPOT] --target bunny|cdn|local
+push    [--manifests DIR] --source LOCAL_DEPOT   --target bunny
+verify  [--manifests DIR] --target bunny|cdn     [--sample N]
+get     <sha> <dest>       --target cdn|bunny|local
+pull    [--manifests DIR] --dest DIR             --target cdn|bunny
+```
+
+`--manifests` default `assets`. `--target local` uses `--source` as root for status, and `--target`-side root comes from `DEPOT_DIR` env when target is local (matches bash `DEPOT_DIR`); document in usage string. Missing required flag / unknown subcommand / no args → usage on stderr, exit 64.
+
+Command semantics (all build shas via `ReferencedSHAs`, backend via `NewBackend`, sweep via `Sweep`):
+
+- **status**: sweep; print `referenced=<n> present=<n> missing=<n> unconfirmed=<n>` to stdout, then each missing sha on its own line prefixed `missing ` and each unconfirmed prefixed `unconfirmed `. Exit 1 if missing>0; else 2 if unconfirmed>0; else 0.
+- **push**: target must be `bunny` (enforce). Sweep; treat missing∪unconfirmed as to-upload (unconfirmed→re-upload is safe/idempotent per spec). For each, read bytes from `--source` local root; a sha absent from source → report and count as failed. Upload with `cfg.Jobs` parallel workers, each sha at most once. Exit 70 on upload error, 1 if any sha was missing from source, else 0. Print `uploaded=<n> failed=<n>`.
+- **verify**: sweep (exit 1/2 as status) **plus**, when clean, download `--sample` N (default 3, 0=skip) random blobs (`math/rand` seeded — pick via deterministic-enough `rand.New(rand.NewSource(time.Now().UnixNano()))`) to temp files via `Get` (which re-hashes); any failure → exit 1.
+- **get**: `Get(sha, dest)`; validate sha; exit 0/70.
+- **pull**: for every referenced sha, dest path `--dest/BlobKey(sha)`. Incremental: if dest file exists, re-hash it; matching → skip, mismatch → re-download. Download absent/mismatched with `cfg.Jobs` workers via `Get`. Print `pulled=<n> present=<n>`; any failed download → exit 70 after finishing the batch (report each failure). Non-zero missing at the source (404) → exit 1 listing them.
+
+Wire the binary: **Modify** `internal/dispatch/dispatch.go` — in `runBuilder`, before the `Wired` check/`delegateLegacy` routing, special-case:
+
+```go
+if b.Name == "depot" && b.Wired {
+    return depot.Run(args[1:], os.Stdout, errw, os.Getenv)
+}
+```
+
+(Exact insertion point: the implementer must read `runBuilder` and place this after subcommand resolution is bypassed — depot does its own subcommand parsing, so pass the raw args through, i.e. `args[1:]` where `args[0]=="depot"` at the dispatcher level, or the full arg slice in `RunBuilder("depot", …)` context. Verify both `crucible depot status …` and `crucible-depot status …` paths hit `depot.Run`.)
+
+- [ ] **Step 1:** Failing unit tests (`run_test.go`): `TestRunUsage` (no args→64, unknown cmd→64, push --target cdn→64), `TestStatusExitCodes` (drift→1, unconfirmed-only→2 using a bunny backend pointed at an httptest server that 428s), `TestGetInvalidSHA` (→64 or 70, pick 64), `TestNoKeyStatusBunnyFailsClosedNoStdin` (getenv returns empty keys → exit 70, stderr mentions the env var name).
+- [ ] **Step 2:** Failing integration test (`integration_test.go`), pure local→local in temp dirs: build manifests yml referencing 3 blobs; source depot has all 3; target depot has 1. Then: `status --target local` → exit 1, missing=2 → `push` (local→local: for the test, allow `push --target local` **only** via an unexported hook? No — keep the spec surface: instead run the integration through bunny backend backed by httptest fake-Bunny with an in-memory blob map, which exercises the real push path) → `status` → 0 → `push` again → 0 uploads (idempotent, assert fake recorded no second PUT). Also `pull` into a pre-populated dest: present-and-hash-ok skipped (fake records no GET for it), corrupt pre-existing file re-downloaded.
+- [ ] **Step 3:** FAIL. **Step 4:** Implement `run.go`. **Step 5:** `make vet test` PASS + gofmt.
+- [ ] **Step 6:** Commit: `feat(depot): status/push/verify/get/pull commands with exit-code contract`.
+
+### Task 6: Registry wiring + docs
+
+**Files:**
+- Modify: `internal/dispatch/dispatch.go` (depot Registry entry)
+- Modify: `internal/dispatch/dispatch_test.go` (if it asserts wired/unwired sets)
+
+Flip the depot entry:
+
+```go
+{
+    Name:    "depot",
+    Bin:     "crucible-depot",
+    Summary: "content-addressed asset depot (local/cdn/bunny)",
+    Commands: []Command{
+        {Name: "status", Summary: "report referenced-vs-present drift against a backend", Usage: "crucible depot status [--manifests DIR] [--source DIR] --target bunny|cdn|local"},
+        {Name: "push", Summary: "upload referenced-but-absent blobs from a local depot", Usage: "crucible depot push [--manifests DIR] --source DIR --target bunny"},
+        {Name: "verify", Summary: "existence sweep plus sampled download re-hash", Usage: "crucible depot verify [--manifests DIR] --target bunny|cdn [--sample N]"},
+        {Name: "get", Summary: "fetch one blob with sha re-verify", Usage: "crucible depot get <sha> <dest> --target cdn|bunny|local"},
+        {Name: "pull", Summary: "incremental verified pull of every referenced blob", Usage: "crucible depot pull [--manifests DIR] --dest DIR --target cdn|bunny"},
+    },
+    Wired: true,
+},
+```
+
+The stale "(SeaweedFS)" wording must not survive anywhere: `grep -ri seaweedfs` across the repo and fix hits.
+
+- [ ] **Step 1:** Update registry; run `go test ./internal/dispatch/` — fix any test that pinned depot as unwired; add/extend a test asserting `crucible depot` routes to `depot.Run` (e.g. `RunBuilder("depot", []string{"status"})` with no manifests dir returns 70/64, not the unwired 70 message — assert on stderr text).
+- [ ] **Step 2:** `make check` (vet+test+shellcheck+yamllint) PASS. Also `make build && ./bin/crucible depot` prints depot usage; `./bin/crucible` interactive menu shows depot entries (manual eyeball via `printf 'q\n' | ./bin/crucible` or equivalent).
+- [ ] **Step 3:** Commit: `feat(depot): wire depot into dispatch registry and menu`.
+
+### Task 7: sow-assets-manifest cutover
+
+**Repo:** `/home/vicky/Projects/westgate/repositories/migration/sow-assets-manifest`, new branch `crucible-depot-cutover` from main.
+
+**Files:**
+- Delete: `scripts/mirror.sh`
+- Modify: `scripts/build-haks.sh` (the `--full` depot-verify block, ~L138-192), `Makefile` (`mirror` target, `haks` target), `.gitea/workflows/release.yml`, `.gitea/workflows/publish-haks.yml`, `.gitea/workflows/build-haks.yml`, `scripts/lib.sh` (header comments only), `AGENTS.md`/`README.md` (author-facing docs)
+
+**Consumes:** the `crucible depot` CLI from Tasks 1–6. NOTE: CI resolves crucible via the flake input tracking sow-tools' default branch — the sow-tools branch must be merged before this repo's CI can pass. Local verification uses a locally built binary (`sow-tools/bin/crucible` on PATH). Do not run `nix flake update` here; note in the PR description that a `nix flake update sow-tools` (or lock bump) must land with/after the sow-tools merge.
+
+Changes:
+
+1. **`release.yml`** "Warm depot cache (verified full pull)" step body (keep name, job env unchanged; drop the ticker/compgen bash):
+```yaml
+      - name: Warm depot cache (verified full pull)
+        env:
+          BUNNY_STORAGE_READ_PASSWORD: ${{ secrets.BUNNY_STORAGE_READ_PASSWORD }}
+          BUNNY_STORAGE_HOST: storage.bunnycdn.com
+          BUNNY_STORAGE_ZONE: sow-assets-depot
+        run: |
+          nix develop --command bash -c '
+            set -euo pipefail
+            crucible depot status --manifests assets --target bunny
+            crucible depot pull --manifests assets --dest "$DEPOT_CACHE_DIR" --target cdn
+          '
+```
+   (If the `BUNNY_STORAGE_READ_PASSWORD` secret does not exist in the repo, use `secrets.BUNNY_STORAGE_PASSWORD` and say so in the PR body. Check what other steps use.)
+2. **`publish-haks.yml`** — same swap of the identical step.
+3. **`build-haks.yml`** — the PR-check step keeps `build-haks.sh --dry-run --no-verify --base "$base"` (structural check) and **adds** `crucible depot verify --manifests assets --target cdn --sample 3` as the cdn-backend verification.
+4. **`scripts/build-haks.sh`** — delete the depot existence-sweep + sample-download block inside `--dry-run` (the `depot_has_many`/`depot_get` verify at ~L138-185); replace with a comment pointing at `crucible depot verify`. Remove now-unused locals. `shellcheck scripts/build-haks.sh` must pass.
+5. **`Makefile`** — `mirror:` target now runs `crucible depot pull --manifests assets --dest "$(DEPOT_DIR)" --target cdn`. `haks:` gains a trailing non-fatal drift report: `-crucible depot status --manifests assets --target cdn` (leading `-` so drift never fails a local build) and an echo pointing at `crucible depot push --source workspace/depot --target bunny` as the standard post-edit step.
+6. **`scripts/lib.sh`** — no function deletions (caller analysis: every depot function used by mirror.sh/build-haks.sh retains Increment 2–3 callers). Add a header comment block at the top of the depot section: `# NOTE: this bash depot layer is being retired. Increment 1 moved mirror/verify to 'crucible depot' (sow-tools). import/sync go in Increment 2; export/pack/publish/artifact/cdn_purge in Increment 3. Do not add new callers.` **Do not touch the backend-keyed stat-cache in import.sh** (spec forbids it).
+7. **Docs** — in README.md (or AGENTS.md where depot workflow is documented): document `crucible depot status --manifests assets --target cdn` as the canonical credential-free "is my content live?" author check, and `crucible depot push --source workspace/depot --target bunny` as the standard post-edit publish step.
+
+- [ ] **Step 1:** Branch; make all edits above.
+- [ ] **Step 2:** Verify: `shellcheck scripts/*.sh`; `yamllint .gitea`; `grep -rn "mirror.sh" . --exclude-dir=.git` returns nothing; with sow-tools' `bin` on PATH run `crucible depot status --manifests assets --target cdn` for real (credential-free) and confirm it produces a sane report (exit 0 or 2 acceptable; exit 1 means real drift — report it, don't hide it).
+- [ ] **Step 3:** Commit: `feat: cut depot mirror/verify over to crucible depot, retire mirror.sh`.
+
+## Self-Review Notes
+
+- Spec coverage: command surface (T5), drift model + unconfirmed/exit 2 (T4/T5), backends + env + no-prompt (T1/T3), IPv4 range probe + HEAD ban test (T3), Checksum PUT (T3), pull incremental (T5), cutover of all three workflows + Makefile + lib.sh headers + author docs (T7), registry/SeaweedFS fix + menu (T6), testing section requirements all mapped (T1–T5), interim contract `make haks` drift report (T7.5).
+- Deliberate deviations from spec, justified: `status --target local` root via `DEPOT_DIR`; push treats source-missing blobs as exit 1; pull download failure exits 70 (backend error) but source-404 exits 1 (drift).
+- Out of scope, per spec non-goals: import/sync, prune/gc/export, mdl checks, rich TUI, cdn_purge.

docs/superpowers/specs/2026-07-04-crucible-depot-core-design.md

@@ -124,9 +124,10 @@ Increment 1.
 
 This spec covers **Increment 1** only. Roadmap for context:
 
-- **Increment 1 (this spec):** `crucible depot {status, push, verify, get}` +
-  blob backends (`local` / `cdn` / `bunny`). Cut over `mirror`/verify/release
-  gate. Delete the corresponding `lib.sh` depot layer.
+- **Increment 1 (this spec):** `crucible depot {status, push, verify, get, pull}`
+  + blob backends (`local` / `cdn` / `bunny`). Cut over `mirror`/verify/release
+  gate. Delete the `lib.sh` depot functions that lose their last caller (see
+  Cutover for what actually remains until Increments 2–3).
 - **Increment 2:** `crucible depot import` / `sync` (edit-tree → manifest+depot),
   replacing `import.sh` / `sync-assets.sh`. Retire the stat-cache concept.
 - **Increment 3:** mdl integrity checks, `mirror`/`pull` ergonomics, `prune`,
@@ -141,8 +142,17 @@ crucible depot status  [--manifests DIR] [--source LOCAL_DEPOT] --target bunny|c
 crucible depot push    [--manifests DIR] --source LOCAL_DEPOT   --target bunny
 crucible depot verify  [--manifests DIR] --target bunny|cdn     [--sample N]
 crucible depot get     <sha> <dest>       --target cdn|bunny|local
+crucible depot pull    [--manifests DIR] --dest DIR             --target cdn|bunny
 ```
 
+- `pull` is the bulk fetch that replaces `mirror.sh`: every referenced sha into
+  `--dest/sha256/ab/cd/<sha>`, **incremental** (skip blobs already present with a
+  matching hash — the release warm step re-runs against a persistent
+  `/var/cache` dir holding ~69k blobs; a non-incremental pull would re-download
+  the corpus every release), sha re-verified on download, parallel with the same
+  probe/transfer bounds as `status`. Without `pull`, Increment 1 cannot delete
+  `mirror.sh` (single-blob `get` is not a replacement).
+
 - `--manifests` defaults to `assets/` (the repo's manifest dir). Confirmed
   decision: `crucible depot` parses `assets/*.yml` directly (same coupling model
   as `crucible-hak` reading hak manifests); the caller does not pre-extract sha
@@ -159,8 +169,26 @@ no cache of "what I uploaded."
 
 - **`status`** — collect every `sha256` referenced by `--manifests/*.yml`, run
   one parallel existence sweep against `--target`, report
-  `{referenced, present, missing}`. Non-zero exit if any referenced blob is
-  missing. This is the release pre-flight gate.
+  `{referenced, present, missing, unconfirmed}`. Non-zero exit if any referenced
+  blob is missing. This is the release pre-flight gate.
+
+  **`unconfirmed` is a distinct state, not `missing`.** Any non-2xx from the
+  parallel sweep is re-probed serially with backoff (the bash
+  `depot_confirm_absent` behavior). If the serial re-probe budget is exhausted
+  (bash caps at 200; keep a cap, `DEPOT_CONFIRM_MAX`), the remainder is reported
+  as `unconfirmed` — **not** collapsed into `missing`. For `push` the
+  distinction is harmless (unconfirmed → treat as absent → re-upload; wasteful
+  but idempotent and safe). For `status` as the release gate it is critical: a
+  throttled CDN edge must produce "N unconfirmed, retry" (distinct exit code,
+  proposed `2`), never a false "thousands missing" hard-fail — exactly the
+  over-reporting the corrected v0.1.5 drift measurement documented above.
+
+  **Target authority split:** `bunny` probes storage — the origin, authoritative,
+  needs the read key; this is what the release gate uses. `cdn` probes the edge —
+  credential-free, what players actually fetch, the author-facing "is my content
+  live?" check; the edge cache can briefly lag storage (bash carries `cdn_purge`
+  for this reason). A blob `present` on bunny but `missing` on cdn is
+  propagation lag, not drift.
 - **`push`** — run `status`, then for each missing sha read the bytes from
   `--source` and upload to `--target`. Stateless and idempotent; re-running does
   nothing once clean. **Cannot silently skip an upload** — "should I upload this?"
@@ -189,13 +217,25 @@ untouched; flags are local aliases):
 - `BUNNY_STORAGE_HOST`, `BUNNY_STORAGE_ZONE` — Bunny storage endpoint.
 - **Read/write key split (preserved):** `BUNNY_STORAGE_READ_PASSWORD` (falls back
   to `BUNNY_STORAGE_PASSWORD`) for probes/reads; `BUNNY_STORAGE_PASSWORD` for
-  `PUT`.
+  `PUT`. Note for the `release.yml` cutover: `status --target bunny` as the gate
+  step needs at least the read key exported in that step's env.
+- **No prompting, read path included.** The no-prompt rule from field note 1
+  applies to every credentialed operation, not just `push`: `status`/`verify`/
+  `get` against `bunny` with no read key fail closed with a clear message —
+  never a hang, never a tty read.
+- `BUNNY_STORAGE_HOST` stays **env-required, no baked default** — the bash never
+  defaulted it (`depot_require_write` defaults only the zone
+  `sow-assets-depot` and the CDN base); resolving the earlier open question.
 - Concurrency: `DEPOT_PROBE_JOBS` (read probes), `DEPOT_JOBS` (uploads);
   `DEPOT_CONNECT_TIMEOUT`, `DEPOT_PROBE_MAX_TIME` per-transfer bounds.
 
 Invariants carried over exactly:
 
-- **Existence probe** is a 1-byte range request (no body download).
+- **Existence probe** is an IPv4 1-byte range request (no body download) — for
+  **all** backends. This is a deliberate deviation from `lib.sh`, whose `cdn`
+  backend probes with HEAD (`_depot_cdn_probe_code`); the "faithful port"
+  clause above does not extend to the probe method. HEAD is banned (see field
+  note 3 and the test requirement there).
 - **Upload** is `PUT` with `Checksum: <UPPER-sha>` so Bunny rejects corrupt
   writes server-side; each sha uploaded at most once per run.
 - **Fail-safe:** a blob with no confirmed-present reply is treated as absent and
@@ -223,26 +263,68 @@ assets/*.yml ──parse──▶ referenced sha set ─┐
 
 Increment 1 deletes the bash it replaces (no wrappers left behind):
 
-- `scripts/mirror.sh` → `crucible depot get`/pull path; script removed.
+- `scripts/mirror.sh` → `crucible depot pull`; script removed.
 - `build-haks.sh`'s `--full` depot-verify block → `crucible depot verify`.
-- `release.yml` step "Warm depot cache" → `crucible depot status` (gate) then the
-  fetch path; the inline bash ticker/compgen logic goes away.
-- `lib.sh` depot layer (`depot_require`, `depot_require_write`, `_depot_*_has`,
-  `_depot_*_get`, `_depot_*_put*`, `_depot_*_probe*`, `blob_key`, backend
-  dispatch — ~250+ lines) is removed once no remaining bash references it.
-  Anything still needed by Increments 2–3 is deleted when those land, not kept as
-  a compatibility shim.
+- **All three depot-touching workflows switch in the same change** (not just
+  `release.yml`):
+  - `release.yml` "Warm depot cache (verified full pull)" →
+    `crucible depot status --target bunny` (gate) then
+    `crucible depot pull --dest $DEPOT_CACHE_DIR`; the inline bash
+    ticker/compgen logic goes away.
+  - `publish-haks.yml` mirrors the same warm+verify+pack flow — same swap.
+  - `build-haks.yml` (PR check, cdn-backend verification) →
+    `crucible depot verify --target cdn`.
+- Registry wiring: the `depot` entry in `internal/dispatch` flips to
+  `Wired: true` with its `Commands` list populated (status/push/verify/get/pull)
+  and the stale "(SeaweedFS)" summary corrected. This automatically puts depot
+  in the interactive `crucible` menu alongside the other wired tools — no
+  menu-specific code needed.
+
+**What `lib.sh` deletion actually looks like.** The depot layer cannot be fully
+removed in Increment 1 — it is still referenced by scripts outside this
+increment's scope:
+
+- `import.sh` / `sync-assets.sh` (`make haks`, `make import`, `make sync`) —
+  Increment 2.
+- `export.sh` (`make pull`), `pack-haks.sh` (release-time `DEPOT_BACKEND=bunny`
+  packing), `publish-release.sh` + `release_put`, and the path-addressed
+  `artifact_has/get/put` + `cdn_purge` helpers — Increment 3 (these were
+  previously unlisted; they are now explicitly Increment 3 scope).
+
+Increment 1 deletes the functions whose last caller it removes (the
+mirror/verify probe-and-get paths) and leaves the rest with a header comment
+marking them scheduled for deletion in Increments 2–3. No compatibility shims,
+no new wrappers — but no pretending `lib.sh` dies before its last caller does.
 
 Consumers call `crucible depot …` directly. Nothing new is wrapped.
 
+### Interim contract for local builds (until Increment 2)
+
+`make haks` keeps running bash `sync-assets.sh` with `DEPOT_BACKEND=local` until
+Increment 2 replaces import/sync. The gap this leaves — local imports creating
+blobs Bunny never sees — is held closed by two things:
+
+- The backend-keyed stat-cache
+  ([`sow-assets-manifest#29`](https://git.westgate.pw/ShadowsOverWestgate/sow-assets-manifest/pulls/29))
+  stays exactly as it is; do not "simplify" it (see the 2026-07-04 update above).
+- `make haks` gains a trailing `crucible depot status --manifests assets
+  --target cdn` (credential-free, report-only, non-fatal) so drift is visible
+  the moment it is created instead of at release time; and
+  `crucible depot push --source workspace/depot --target bunny` is documented
+  as the standard post-edit step (field note 4's ask, landed here). Upload when
+  you want the content live; build local-only when you don't — but either way
+  the drift is *reported*, never masked.
+
 ## Exit-code contract
 
 Matches Crucible's fail-closed convention:
 
 - `0` — clean / success.
-- distinct non-zero (proposed `1`) — **drift found** (`status`/`push` saw
-  referenced-but-absent blobs), so `release.yml` can gate on `status` before
-  packing.
+- `1` — **drift found** (`status`/`push` saw referenced-but-absent blobs), so
+  `release.yml` can gate on `status` before packing.
+- `2` — **unconfirmed only**: no confirmed-missing blobs, but the probe budget
+  ran out before every blob got a confirmed reply (throttled edge). The gate
+  should retry, not report drift.
 - `70` — internal/backend error; never a faked result.
 
 ## Testing
@@ -255,6 +337,10 @@ Matches Crucible's fail-closed convention:
   `Checksum: <UPPER-sha>` header, and the read/write key split — no real Bunny.
 - **Integration (local→local)**: `status` shows drift → `push` clears it →
   `status` clean → `push` again is a no-op (idempotency).
+- **New-surface tests**: fake-Bunny throttling (`428`) → blobs land in
+  `unconfirmed`, exit `2`, never `missing`; `pull` into a pre-populated dest
+  skips present-and-hash-ok blobs (incremental); credentialed op with no key
+  exits non-zero without reading stdin (read path and write path both).
 
 ## Non-goals (Increment 1)
 
@@ -262,7 +348,11 @@ Matches Crucible's fail-closed convention:
 - mdl integrity checks, `prune`, `gc`, `export`, `mirror` ergonomics
   (Increment 3).
 - Any change to the depot wire protocol, blob layout, or Bunny account config.
-- A TUI.
+- A depot-specific rich TUI. **Direction note:** Crucible is headed toward being
+  a TUI overall — the existing interactive dispatcher menu is the first step,
+  and depot joins it for free via the registry wiring above. What Increment 1
+  does not build is anything richer than that menu (progress UIs, pickers,
+  dashboards); those come with the Crucible-wide TUI work, not per-tool.
 
 ## Immediate operational payoff
 
@@ -276,8 +366,8 @@ Increment 1 first both fixes the tooling and clears the current outage.
 
 ## Risks / open questions
 
-- `BUNNY_STORAGE_HOST` default: confirm the canonical storage host to bake as the
-  fallback (bash referenced it without a visible default in the reviewed range).
+- ~~`BUNNY_STORAGE_HOST` default~~ — resolved: env-required, no baked default
+  (see Backends & config).
 - Manifest schema coupling: `crucible depot` now depends on the `assets/*.yml`
   shape (`assets[].{path,sha256,size,restype,hak}`). Acceptable and intentional,
   but a schema change now touches Crucible.