docs: depot core increment 1 plan + spec updates

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

docs/superpowers/specs/2026-07-04-crucible-depot-core-design.md

@@ -124,9 +124,10 @@ Increment 1.
 
 This spec covers **Increment 1** only. Roadmap for context:
 
-- **Increment 1 (this spec):** `crucible depot {status, push, verify, get}` +
-  blob backends (`local` / `cdn` / `bunny`). Cut over `mirror`/verify/release
-  gate. Delete the corresponding `lib.sh` depot layer.
+- **Increment 1 (this spec):** `crucible depot {status, push, verify, get, pull}`
+  + blob backends (`local` / `cdn` / `bunny`). Cut over `mirror`/verify/release
+  gate. Delete the `lib.sh` depot functions that lose their last caller (see
+  Cutover for what actually remains until Increments 2–3).
 - **Increment 2:** `crucible depot import` / `sync` (edit-tree → manifest+depot),
   replacing `import.sh` / `sync-assets.sh`. Retire the stat-cache concept.
 - **Increment 3:** mdl integrity checks, `mirror`/`pull` ergonomics, `prune`,
@@ -141,8 +142,17 @@ crucible depot status  [--manifests DIR] [--source LOCAL_DEPOT] --target bunny|c
 crucible depot push    [--manifests DIR] --source LOCAL_DEPOT   --target bunny
 crucible depot verify  [--manifests DIR] --target bunny|cdn     [--sample N]
 crucible depot get     <sha> <dest>       --target cdn|bunny|local
+crucible depot pull    [--manifests DIR] --dest DIR             --target cdn|bunny
 ```
 
+- `pull` is the bulk fetch that replaces `mirror.sh`: every referenced sha into
+  `--dest/sha256/ab/cd/<sha>`, **incremental** (skip blobs already present with a
+  matching hash — the release warm step re-runs against a persistent
+  `/var/cache` dir holding ~69k blobs; a non-incremental pull would re-download
+  the corpus every release), sha re-verified on download, parallel with the same
+  probe/transfer bounds as `status`. Without `pull`, Increment 1 cannot delete
+  `mirror.sh` (single-blob `get` is not a replacement).
+
 - `--manifests` defaults to `assets/` (the repo's manifest dir). Confirmed
   decision: `crucible depot` parses `assets/*.yml` directly (same coupling model
   as `crucible-hak` reading hak manifests); the caller does not pre-extract sha
@@ -159,8 +169,26 @@ no cache of "what I uploaded."
 
 - **`status`** — collect every `sha256` referenced by `--manifests/*.yml`, run
   one parallel existence sweep against `--target`, report
-  `{referenced, present, missing}`. Non-zero exit if any referenced blob is
-  missing. This is the release pre-flight gate.
+  `{referenced, present, missing, unconfirmed}`. Non-zero exit if any referenced
+  blob is missing. This is the release pre-flight gate.
+
+  **`unconfirmed` is a distinct state, not `missing`.** Any non-2xx from the
+  parallel sweep is re-probed serially with backoff (the bash
+  `depot_confirm_absent` behavior). If the serial re-probe budget is exhausted
+  (bash caps at 200; keep a cap, `DEPOT_CONFIRM_MAX`), the remainder is reported
+  as `unconfirmed` — **not** collapsed into `missing`. For `push` the
+  distinction is harmless (unconfirmed → treat as absent → re-upload; wasteful
+  but idempotent and safe). For `status` as the release gate it is critical: a
+  throttled CDN edge must produce "N unconfirmed, retry" (distinct exit code,
+  proposed `2`), never a false "thousands missing" hard-fail — exactly the
+  over-reporting the corrected v0.1.5 drift measurement documented above.
+
+  **Target authority split:** `bunny` probes storage — the origin, authoritative,
+  needs the read key; this is what the release gate uses. `cdn` probes the edge —
+  credential-free, what players actually fetch, the author-facing "is my content
+  live?" check; the edge cache can briefly lag storage (bash carries `cdn_purge`
+  for this reason). A blob `present` on bunny but `missing` on cdn is
+  propagation lag, not drift.
 - **`push`** — run `status`, then for each missing sha read the bytes from
   `--source` and upload to `--target`. Stateless and idempotent; re-running does
   nothing once clean. **Cannot silently skip an upload** — "should I upload this?"
@@ -189,13 +217,25 @@ untouched; flags are local aliases):
 - `BUNNY_STORAGE_HOST`, `BUNNY_STORAGE_ZONE` — Bunny storage endpoint.
 - **Read/write key split (preserved):** `BUNNY_STORAGE_READ_PASSWORD` (falls back
   to `BUNNY_STORAGE_PASSWORD`) for probes/reads; `BUNNY_STORAGE_PASSWORD` for
-  `PUT`.
+  `PUT`. Note for the `release.yml` cutover: `status --target bunny` as the gate
+  step needs at least the read key exported in that step's env.
+- **No prompting, read path included.** The no-prompt rule from field note 1
+  applies to every credentialed operation, not just `push`: `status`/`verify`/
+  `get` against `bunny` with no read key fail closed with a clear message —
+  never a hang, never a tty read.
+- `BUNNY_STORAGE_HOST` stays **env-required, no baked default** — the bash never
+  defaulted it (`depot_require_write` defaults only the zone
+  `sow-assets-depot` and the CDN base); resolving the earlier open question.
 - Concurrency: `DEPOT_PROBE_JOBS` (read probes), `DEPOT_JOBS` (uploads);
   `DEPOT_CONNECT_TIMEOUT`, `DEPOT_PROBE_MAX_TIME` per-transfer bounds.
 
 Invariants carried over exactly:
 
-- **Existence probe** is a 1-byte range request (no body download).
+- **Existence probe** is an IPv4 1-byte range request (no body download) — for
+  **all** backends. This is a deliberate deviation from `lib.sh`, whose `cdn`
+  backend probes with HEAD (`_depot_cdn_probe_code`); the "faithful port"
+  clause above does not extend to the probe method. HEAD is banned (see field
+  note 3 and the test requirement there).
 - **Upload** is `PUT` with `Checksum: <UPPER-sha>` so Bunny rejects corrupt
   writes server-side; each sha uploaded at most once per run.
 - **Fail-safe:** a blob with no confirmed-present reply is treated as absent and
@@ -223,26 +263,68 @@ assets/*.yml ──parse──▶ referenced sha set ─┐
 
 Increment 1 deletes the bash it replaces (no wrappers left behind):
 
-- `scripts/mirror.sh` → `crucible depot get`/pull path; script removed.
+- `scripts/mirror.sh` → `crucible depot pull`; script removed.
 - `build-haks.sh`'s `--full` depot-verify block → `crucible depot verify`.
-- `release.yml` step "Warm depot cache" → `crucible depot status` (gate) then the
-  fetch path; the inline bash ticker/compgen logic goes away.
-- `lib.sh` depot layer (`depot_require`, `depot_require_write`, `_depot_*_has`,
-  `_depot_*_get`, `_depot_*_put*`, `_depot_*_probe*`, `blob_key`, backend
-  dispatch — ~250+ lines) is removed once no remaining bash references it.
-  Anything still needed by Increments 2–3 is deleted when those land, not kept as
-  a compatibility shim.
+- **All three depot-touching workflows switch in the same change** (not just
+  `release.yml`):
+  - `release.yml` "Warm depot cache (verified full pull)" →
+    `crucible depot status --target bunny` (gate) then
+    `crucible depot pull --dest $DEPOT_CACHE_DIR`; the inline bash
+    ticker/compgen logic goes away.
+  - `publish-haks.yml` mirrors the same warm+verify+pack flow — same swap.
+  - `build-haks.yml` (PR check, cdn-backend verification) →
+    `crucible depot verify --target cdn`.
+- Registry wiring: the `depot` entry in `internal/dispatch` flips to
+  `Wired: true` with its `Commands` list populated (status/push/verify/get/pull)
+  and the stale "(SeaweedFS)" summary corrected. This automatically puts depot
+  in the interactive `crucible` menu alongside the other wired tools — no
+  menu-specific code needed.
+
+**What `lib.sh` deletion actually looks like.** The depot layer cannot be fully
+removed in Increment 1 — it is still referenced by scripts outside this
+increment's scope:
+
+- `import.sh` / `sync-assets.sh` (`make haks`, `make import`, `make sync`) —
+  Increment 2.
+- `export.sh` (`make pull`), `pack-haks.sh` (release-time `DEPOT_BACKEND=bunny`
+  packing), `publish-release.sh` + `release_put`, and the path-addressed
+  `artifact_has/get/put` + `cdn_purge` helpers — Increment 3 (these were
+  previously unlisted; they are now explicitly Increment 3 scope).
+
+Increment 1 deletes the functions whose last caller it removes (the
+mirror/verify probe-and-get paths) and leaves the rest with a header comment
+marking them scheduled for deletion in Increments 2–3. No compatibility shims,
+no new wrappers — but no pretending `lib.sh` dies before its last caller does.
 
 Consumers call `crucible depot …` directly. Nothing new is wrapped.
 
+### Interim contract for local builds (until Increment 2)
+
+`make haks` keeps running bash `sync-assets.sh` with `DEPOT_BACKEND=local` until
+Increment 2 replaces import/sync. The gap this leaves — local imports creating
+blobs Bunny never sees — is held closed by two things:
+
+- The backend-keyed stat-cache
+  ([`sow-assets-manifest#29`](https://git.westgate.pw/ShadowsOverWestgate/sow-assets-manifest/pulls/29))
+  stays exactly as it is; do not "simplify" it (see the 2026-07-04 update above).
+- `make haks` gains a trailing `crucible depot status --manifests assets
+  --target cdn` (credential-free, report-only, non-fatal) so drift is visible
+  the moment it is created instead of at release time; and
+  `crucible depot push --source workspace/depot --target bunny` is documented
+  as the standard post-edit step (field note 4's ask, landed here). Upload when
+  you want the content live; build local-only when you don't — but either way
+  the drift is *reported*, never masked.
+
 ## Exit-code contract
 
 Matches Crucible's fail-closed convention:
 
 - `0` — clean / success.
-- distinct non-zero (proposed `1`) — **drift found** (`status`/`push` saw
-  referenced-but-absent blobs), so `release.yml` can gate on `status` before
-  packing.
+- `1` — **drift found** (`status`/`push` saw referenced-but-absent blobs), so
+  `release.yml` can gate on `status` before packing.
+- `2` — **unconfirmed only**: no confirmed-missing blobs, but the probe budget
+  ran out before every blob got a confirmed reply (throttled edge). The gate
+  should retry, not report drift.
 - `70` — internal/backend error; never a faked result.
 
 ## Testing
@@ -255,6 +337,10 @@ Matches Crucible's fail-closed convention:
   `Checksum: <UPPER-sha>` header, and the read/write key split — no real Bunny.
 - **Integration (local→local)**: `status` shows drift → `push` clears it →
   `status` clean → `push` again is a no-op (idempotency).
+- **New-surface tests**: fake-Bunny throttling (`428`) → blobs land in
+  `unconfirmed`, exit `2`, never `missing`; `pull` into a pre-populated dest
+  skips present-and-hash-ok blobs (incremental); credentialed op with no key
+  exits non-zero without reading stdin (read path and write path both).
 
 ## Non-goals (Increment 1)
 
@@ -262,7 +348,11 @@ Matches Crucible's fail-closed convention:
 - mdl integrity checks, `prune`, `gc`, `export`, `mirror` ergonomics
   (Increment 3).
 - Any change to the depot wire protocol, blob layout, or Bunny account config.
-- A TUI.
+- A depot-specific rich TUI. **Direction note:** Crucible is headed toward being
+  a TUI overall — the existing interactive dispatcher menu is the first step,
+  and depot joins it for free via the registry wiring above. What Increment 1
+  does not build is anything richer than that menu (progress UIs, pickers,
+  dashboards); those come with the Crucible-wide TUI work, not per-tool.
 
 ## Immediate operational payoff
 
@@ -276,8 +366,8 @@ Increment 1 first both fixes the tooling and clears the current outage.
 
 ## Risks / open questions
 
-- `BUNNY_STORAGE_HOST` default: confirm the canonical storage host to bake as the
-  fallback (bash referenced it without a visible default in the reviewed range).
+- ~~`BUNNY_STORAGE_HOST` default~~ — resolved: env-required, no baked default
+  (see Backends & config).
 - Manifest schema coupling: `crucible depot` now depends on the `assets/*.yml`
   shape (`assets[].{path,sha256,size,restype,hak}`). Acceptable and intentional,
   but a schema change now touches Crucible.