Files
sow-tools/docs/superpowers/plans/2026-07-04-crucible-depot-core.md
archvillainette 9357b30994
test / test (push) Successful in 1m31s
build-binaries / build-binaries (push) Successful in 2m27s
build-image / publish (push) Successful in 42s
crucible depot: Increment 1 core (status/push/verify/get/pull + local/cdn/bunny backends) (#30)
Implements Increment 1 of `docs/superpowers/specs/2026-07-04-crucible-depot-core-design.md`: a new stdlib-only `internal/depot` package wired into the dispatcher.

- `crucible depot status|push|verify|get|pull` with backends `local`/`cdn`/`bunny`; exit contract `0` clean / `1` drift / `2` unconfirmed-only / `64` usage / `70` internal.
- Presence is always probed against the real target (IPv4 1-byte range GET; HEAD is banned with a regression-tripwire test). `unconfirmed` is a distinct state, never collapsed into `missing`.
- No prompting anywhere: missing `BUNNY_STORAGE_*` env fails closed (read path included), enforced by a no-stdin test.
- Uploads: probe-then-PUT with `Checksum: <UPPER-sha>`; read/write key split (`BUNNY_STORAGE_READ_PASSWORD` falls back to `BUNNY_STORAGE_PASSWORD`).
- Field-driven fix included: per-probe transient retry (curl `--retry 2` equivalent) — without it a real 1490-blob CDN sweep reported 1222 false-unconfirmed; with it, 1490/1490 present in 74s, exit 0.
- Registry: depot `Wired: true`, joins the interactive menu; stale "(SeaweedFS)" wording removed.

Tests: unit + httptest fake-Bunny (probe sequence, Checksum header, key split, 428 throttling → exit 2) + local→bunny integration (drift → push → clean → idempotent no-second-PUT; incremental pull). `make check` green.

**Merge ordering:** this merges FIRST; the companion `sow-assets-manifest#crucible-depot-cutover` PR needs its flake input bumped to include this.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Reviewed-on: #30
Reviewed-by: xtul <mpiasecki720@protonmail.com>
Co-authored-by: vickydotbat <vickydotbat@tutamail.com>
Co-committed-by: vickydotbat <vickydotbat@tutamail.com>
2026-07-04 23:30:54 +00:00

25 KiB
Raw Permalink Blame History

Crucible Depot Core (Increment 1) Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Implement crucible depot {status,push,verify,get,pull} with local/cdn/bunny backends in sow-tools, then cut sow-assets-manifest over to it and delete the bash it replaces.

Architecture: New stdlib-only internal/depot package with its own Run(args) int entrypoint (exit contract 0/1/2/70 is richer than internal/app.Run's 0/1, so dispatch special-cases depot instead of routing through app.Run). Backends implement a small interface; a sweep engine does parallel range-GET probes with serial re-confirm. Cutover repo work happens in sow-assets-manifest on its own branch.

Tech Stack: Go 1.26, stdlib net/http/crypto/sha256/sync, gopkg.in/yaml.v3 (already a dep — add nothing to go.mod, vendorHash must not change).

Spec: docs/superpowers/specs/2026-07-04-crucible-depot-core-design.md — read it before any task; it is authoritative.

Global Constraints

  • No new Go dependencies. stdlib + existing gopkg.in/yaml.v3 only.
  • HEAD is banned for existence checks. All probes: IPv4, GET with Range: bytes=0-0, expect 206 (accept any 2xx). A test must fail if HEAD is introduced.
  • Never prompt, never read stdin for credentials — read path included. Missing key → clear error on stderr, exit 70.
  • No cache of "what I uploaded" — presence is always probed against the real target backend.
  • unconfirmed is a distinct state from missing; exit 2 when unconfirmed-only.
  • Exit codes: 0 clean, 1 drift (referenced-but-absent), 2 unconfirmed-only, 64 usage, 70 internal/backend error.
  • Blob key: sha256/<sha[0:2]>/<sha[2:4]>/<sha>; sha must match ^[0-9a-f]{64}$.
  • Env names (unchanged from bash): DEPOT_CDN_BASE/BUNNY_CDN_BASE (cdn read base, default https://cdn-a7f3k9.westgate.pw), BUNNY_STORAGE_HOST (required for bunny, no default — spec overrides bash), BUNNY_STORAGE_ZONE (default sow-assets-depot), BUNNY_STORAGE_READ_PASSWORD (falls back to BUNNY_STORAGE_PASSWORD) for reads/probes, BUNNY_STORAGE_PASSWORD for PUT, DEPOT_PROBE_JOBS (default 16), DEPOT_JOBS (default 16), DEPOT_CONNECT_TIMEOUT (default 10s), DEPOT_PROBE_MAX_TIME (default 30s), DEPOT_CONFIRM_RETRIES (default 3), DEPOT_CONFIRM_MAX (default 200, 0 = unlimited).
  • Upload: PUT with AccessKey: <write key> and Checksum: <UPPERCASE sha> headers; each sha uploaded at most once per run; probe-before-put.
  • Every downloaded blob is re-hashed; deleted on mismatch.
  • cdn backend writes fail closed (error, never fake success).
  • Verify with make vet test in sow-tools (run inside nix develop if go missing from PATH).
  • Commit after each task. Never commit to main. sow-tools branch: depot-core-increment-1. sow-assets-manifest branch: crucible-depot-cutover. sow-tools has an unrelated modified spec file on main — leave it out of commits unless the task says otherwise.

Task 1: Package skeleton — config + manifest parsing

Files:

  • Create: internal/depot/config.go, internal/depot/config_test.go
  • Create: internal/depot/manifest.go, internal/depot/manifest_test.go

Interfaces (Produces):

package depot

// config.go
type Config struct {
    CDNBase        string        // DEPOT_CDN_BASE || BUNNY_CDN_BASE || default
    StorageHost    string        // BUNNY_STORAGE_HOST, no default
    StorageZone    string        // BUNNY_STORAGE_ZONE, default "sow-assets-depot"
    ReadKey        string        // BUNNY_STORAGE_READ_PASSWORD || BUNNY_STORAGE_PASSWORD
    WriteKey       string        // BUNNY_STORAGE_PASSWORD
    ProbeJobs      int           // DEPOT_PROBE_JOBS, default 16
    Jobs           int           // DEPOT_JOBS, default 16
    ConnectTimeout time.Duration // DEPOT_CONNECT_TIMEOUT seconds, default 10s
    ProbeMaxTime   time.Duration // DEPOT_PROBE_MAX_TIME seconds, default 30s
    ConfirmRetries int           // DEPOT_CONFIRM_RETRIES, default 3
    ConfirmMax     int           // DEPOT_CONFIRM_MAX, default 200 (0 = unlimited)
}
func LoadConfig(getenv func(string) string) Config

// manifest.go
// ReferencedSHAs parses every *.yml in dir (schema: assets[].{path,sha256,size,restype,hak})
// and returns the deduplicated sha set plus per-sha size (for pull).
func ReferencedSHAs(dir string) (map[string]int64, error)
func ValidSHA(s string) bool // ^[0-9a-f]{64}$
func BlobKey(sha string) string // "sha256/ab/cd/<sha>"

LoadConfig takes getenv so tests inject env without t.Setenv races. Invalid ints/durations in env → fall back to default (do not error). ReferencedSHAs errors on: unreadable dir, no *.yml files, YAML parse failure, or an entry whose sha256 fails ValidSHA (report file + path). Manifest struct:

type manifestFile struct {
    Assets []struct {
        Path   string `yaml:"path"`
        SHA256 string `yaml:"sha256"`
        Size   int64  `yaml:"size"`
    } `yaml:"assets"`
}
  • Step 1: git -C sow-tools switch -c depot-core-increment-1 (from main).
  • Step 2: Write failing tests: TestLoadConfigDefaults, TestLoadConfigReadKeyFallback (READ unset → WriteKey value; READ set → its own), TestLoadConfigBadIntFallsBack, TestBlobKey (BlobKey("0b1d…")=="sha256/0b/1d/0b1d…"), TestValidSHA (rejects uppercase, short, non-hex), TestReferencedSHAs (temp dir with two yml files sharing one sha → dedup; bad sha → error naming the file; empty dir → error).
  • Step 3: go test ./internal/depot/ — verify FAIL (compile error is fine).
  • Step 4: Implement config.go + manifest.go minimally.
  • Step 5: go test ./internal/depot/ PASS; gofmt -l -w internal/depot.
  • Step 6: Commit: feat(depot): config resolution and manifest sha parsing.

Task 2: Local backend

Files:

  • Create: internal/depot/backend.go, internal/depot/local.go, internal/depot/local_test.go

Interfaces (Produces):

// backend.go
type ProbeState int
const (
    Present ProbeState = iota
    Absent
    Unconfirmed
)
type Backend interface {
    Name() string
    // Probe returns the existence state of one sha. transient=true means a
    // retry might change the answer (feeds the serial confirm loop).
    Probe(ctx context.Context, sha string) (state ProbeState, transient bool, err error)
    // Get fetches sha into dest (temp file + rename), re-hashes, deletes on mismatch.
    Get(ctx context.Context, sha, dest string) error
    // Put uploads bytes from src for sha. Read-only backends return an error.
    Put(ctx context.Context, sha, src string) error
}

// local.go
type LocalBackend struct{ Root string }  // implements Backend; Name()=="local"

Local semantics: Probe = os.Stat(Root/BlobKey(sha)), never transient. Put = copy to temp file in dest dir then os.Rename (MkdirAll parents). Get = copy out + re-hash (hash the source bytes while copying via io.TeeReader into sha256.New()), delete dest and error on mismatch. err from Probe only for real I/O errors (permission), not absence.

  • Step 1: Failing tests: TestLocalRoundTrip (Put→Probe Present→Get→bytes equal), TestLocalProbeAbsent, TestLocalGetHashMismatch (hand-write a corrupt blob file at the keyed path, Get returns error and dest does not exist), TestLocalPutIdempotent.
  • Step 2: Verify FAIL. Step 3: Implement. Step 4: PASS + gofmt.
  • Step 5: Commit: feat(depot): backend interface and local backend.

Task 3: Remote backends (cdn, bunny) with fake-Bunny tests

Files:

  • Create: internal/depot/remote.go, internal/depot/remote_test.go

Interfaces (Consumes): Backend, ProbeState, Config, BlobKey. Produces:

// NewBackend returns the backend for name ("local"|"cdn"|"bunny").
// localRoot is used only for "local". Fails closed: "bunny" with empty
// StorageHost or empty ReadKey returns an error (never prompts);
// unknown name returns an error.
func NewBackend(name, localRoot string, cfg Config) (Backend, error)

type httpBackend struct { ... } // shared by cdn and bunny

Implementation requirements:

// IPv4-only transport — REQUIRED, spec bans IPv6/HEAD probe hazards.
transport := &http.Transport{
    DialContext: func(ctx context.Context, network, addr string) (net.Conn, error) {
        d := net.Dialer{Timeout: cfg.ConnectTimeout}
        return d.DialContext(ctx, "tcp4", addr)
    },
    MaxIdleConnsPerHost: cfg.ProbeJobs,
}
client := &http.Client{Transport: transport, Timeout: cfg.ProbeMaxTime}
  • Probe (both remotes): GET <base>/<BlobKey(sha)> with header Range: bytes=0-0; drain/close body. 2xx → Present; 404/410 → Absent; anything else (incl. transport error) → Unconfirmed, transient=true. Never http.Head / MethodHead.
  • cdn: base = cfg.CDNBase, no auth headers, Put returns errors.New("cdn backend is read-only").
  • bunny: probe/read against storage https://<StorageHost>/<StorageZone>/<BlobKey(sha)> with AccessKey: <ReadKey>. Get: try CDN base first, on non-2xx fall back to storage URL with ReadKey (mirrors bash _depot_bunny_get); re-hash, delete on mismatch. Put: probe first, skip if Present; else PUT storage URL, headers AccessKey: <WriteKey>, Checksum: <strings.ToUpper(sha)>, body streamed from src file; non-2xx → error. Put with empty WriteKey → error before any I/O.
  • Both Gets write via temp file + rename and re-hash exactly like local.

httptest note: httptest.NewServer listens on 127.0.0.1, which the tcp4 dialer reaches — no test shim needed.

  • Step 1: Failing tests against an httptest fake Bunny (a handler recording method+path+headers, serving a blob map):
    • TestProbeUsesRangeGetNotHead — probe a present sha; assert recorded request has Method=="GET" and Range=="bytes=0-0"; also assert the depot source contains no HEAD probe: the handler returns 500 for any HEAD request, and the test asserts zero HEAD requests were received across the whole test run (this is the anti-HEAD regression tripwire from field note 3).
    • TestProbeStates — 200/206→Present, 404→Absent, 428→Unconfirmed+transient, 503→Unconfirmed+transient.
    • TestBunnyPutSequence — Put of an absent sha: recorded sequence is [GET range probe, PUT]; PUT has Checksum = uppercase sha and AccessKey = write key; Put of a present sha: probe only, no PUT.
    • TestReadWriteKeySplit — probe uses ReadKey, PUT uses WriteKey (distinct values asserted).
    • TestBunnyNoReadKeyFailsClosedNewBackend("bunny",…) with empty read key returns error without reading stdin: replace os.Stdin with the read end of an os.Pipe() for the test; after the call, write+close and confirm the pipe is undrained (or simply assert error occurs with no stdin interaction because the code path has no stdin reference — grep-based assertion acceptable: test fails if internal/depot sources mention os.Stdin or /dev/tty).
    • TestBunnyNoStorageHostFailsClosed.
    • TestGetHashMismatchDeletes — server returns wrong bytes; Get errors, dest removed.
    • TestCDNPutReadOnly.
  • Step 2: Verify FAIL. Step 3: Implement remote.go. Step 4: PASS + gofmt + go vet ./....
  • Step 5: Commit: feat(depot): cdn and bunny HTTP backends with fake-Bunny tests.

Task 4: Sweep engine — parallel probe + serial confirm

Files:

  • Create: internal/depot/sweep.go, internal/depot/sweep_test.go

Interfaces (Consumes): Backend, Config, ProbeState. Produces:

type SweepResult struct {
    Present     []string
    Missing     []string // confirmed absent (404/410 after retries)
    Unconfirmed []string // probe budget exhausted, state unknown
}
// Sweep probes every sha against b: one parallel pass (cfg.ProbeJobs workers,
// sync.WaitGroup + buffered-channel work queue), then any non-Present result
// is re-probed serially with linear backoff (attempt t sleeps t seconds,
// cfg.ConfirmRetries attempts) — unless the non-Present count exceeds
// cfg.ConfirmMax (>0), in which case the remainder is reported Unconfirmed
// without serial re-probe (never collapsed into Missing).
// A transient result that stays non-2xx after retries is Unconfirmed;
// a 404/410 is Missing. Backend errors (non-transient) abort with error.
func Sweep(ctx context.Context, b Backend, shas []string, cfg Config, progress io.Writer) (SweepResult, error)

Sort shas before sweeping (deterministic output). Progress: print probed X/N roughly every 1000 blobs to progress (may be io.Discard). For tests, make the backoff sleeper injectable: package-level var confirmSleep = time.Sleep, overridden in tests.

  • Step 1: Failing tests using a stub Backend (function fields):
    • TestSweepAllPresent, TestSweepMissing (404 → Missing, no retries beyond first serial confirm).
    • TestSweepThrottledUnconfirmed — stub always returns Unconfirmed/transient → after retries lands in Unconfirmed, never Missing; assert confirmSleep called with 1s then 2s.
    • TestSweepTransientRecovers — first probe 428, serial re-probe 206 → Present.
    • TestSweepConfirmMaxCap — 5 non-present, ConfirmMax=2 → all 5 Unconfirmed, zero serial re-probes.
    • TestSweepParallelBounded — 100 shas, ProbeJobs=4; stub counts concurrent in-flight probes (atomic), assert max ≤ 4.
  • Step 2: FAIL. Step 3: Implement. Step 4: PASS + gofmt.
  • Step 5: Commit: feat(depot): parallel probe sweep with serial confirm and unconfirmed state.

Task 5: Commands + Run entrypoint + integration test

Files:

  • Create: internal/depot/run.go, internal/depot/run_test.go, internal/depot/integration_test.go

Interfaces (Consumes): everything above. Produces:

// Run executes a depot subcommand. args[0] is the subcommand
// (status|push|verify|get|pull); returns the process exit code.
// Exit: 0 clean, 1 drift, 2 unconfirmed-only, 64 usage, 70 internal.
func Run(args []string, stdout, stderr io.Writer, getenv func(string) string) int

Flag parsing: one flag.FlagSet per subcommand (stdlib; internal/depot is a fresh package, the hand-rolled loop in internal/app is not a constraint here). Flags per spec:

status  [--manifests DIR] [--source LOCAL_DEPOT] --target bunny|cdn|local
push    [--manifests DIR] --source LOCAL_DEPOT   --target bunny
verify  [--manifests DIR] --target bunny|cdn     [--sample N]
get     <sha> <dest>       --target cdn|bunny|local
pull    [--manifests DIR] --dest DIR             --target cdn|bunny

--manifests default assets. --target local uses --source as root for status, and --target-side root comes from DEPOT_DIR env when target is local (matches bash DEPOT_DIR); document in usage string. Missing required flag / unknown subcommand / no args → usage on stderr, exit 64.

Command semantics (all build shas via ReferencedSHAs, backend via NewBackend, sweep via Sweep):

  • status: sweep; print referenced=<n> present=<n> missing=<n> unconfirmed=<n> to stdout, then each missing sha on its own line prefixed missing and each unconfirmed prefixed unconfirmed . Exit 1 if missing>0; else 2 if unconfirmed>0; else 0.
  • push: target must be bunny (enforce). Sweep; treat missingunconfirmed as to-upload (unconfirmed→re-upload is safe/idempotent per spec). For each, read bytes from --source local root; a sha absent from source → report and count as failed. Upload with cfg.Jobs parallel workers, each sha at most once. Exit 70 on upload error, 1 if any sha was missing from source, else 0. Print uploaded=<n> failed=<n>.
  • verify: sweep (exit 1/2 as status) plus, when clean, download --sample N (default 3, 0=skip) random blobs (math/rand seeded — pick via deterministic-enough rand.New(rand.NewSource(time.Now().UnixNano()))) to temp files via Get (which re-hashes); any failure → exit 1.
  • get: Get(sha, dest); validate sha; exit 0/70.
  • pull: for every referenced sha, dest path --dest/BlobKey(sha). Incremental: if dest file exists, re-hash it; matching → skip, mismatch → re-download. Download absent/mismatched with cfg.Jobs workers via Get. Print pulled=<n> present=<n>; any failed download → exit 70 after finishing the batch (report each failure). Non-zero missing at the source (404) → exit 1 listing them.

Wire the binary: Modify internal/dispatch/dispatch.go — in runBuilder, before the Wired check/delegateLegacy routing, special-case:

if b.Name == "depot" && b.Wired {
    return depot.Run(args[1:], os.Stdout, errw, os.Getenv)
}

(Exact insertion point: the implementer must read runBuilder and place this after subcommand resolution is bypassed — depot does its own subcommand parsing, so pass the raw args through, i.e. args[1:] where args[0]=="depot" at the dispatcher level, or the full arg slice in RunBuilder("depot", …) context. Verify both crucible depot status … and crucible-depot status … paths hit depot.Run.)

  • Step 1: Failing unit tests (run_test.go): TestRunUsage (no args→64, unknown cmd→64, push --target cdn→64), TestStatusExitCodes (drift→1, unconfirmed-only→2 using a bunny backend pointed at an httptest server that 428s), TestGetInvalidSHA (→64 or 70, pick 64), TestNoKeyStatusBunnyFailsClosedNoStdin (getenv returns empty keys → exit 70, stderr mentions the env var name).
  • Step 2: Failing integration test (integration_test.go), pure local→local in temp dirs: build manifests yml referencing 3 blobs; source depot has all 3; target depot has 1. Then: status --target local → exit 1, missing=2 → push (local→local: for the test, allow push --target local only via an unexported hook? No — keep the spec surface: instead run the integration through bunny backend backed by httptest fake-Bunny with an in-memory blob map, which exercises the real push path) → status → 0 → push again → 0 uploads (idempotent, assert fake recorded no second PUT). Also pull into a pre-populated dest: present-and-hash-ok skipped (fake records no GET for it), corrupt pre-existing file re-downloaded.
  • Step 3: FAIL. Step 4: Implement run.go. Step 5: make vet test PASS + gofmt.
  • Step 6: Commit: feat(depot): status/push/verify/get/pull commands with exit-code contract.

Task 6: Registry wiring + docs

Files:

  • Modify: internal/dispatch/dispatch.go (depot Registry entry)
  • Modify: internal/dispatch/dispatch_test.go (if it asserts wired/unwired sets)

Flip the depot entry:

{
    Name:    "depot",
    Bin:     "crucible-depot",
    Summary: "content-addressed asset depot (local/cdn/bunny)",
    Commands: []Command{
        {Name: "status", Summary: "report referenced-vs-present drift against a backend", Usage: "crucible depot status [--manifests DIR] [--source DIR] --target bunny|cdn|local"},
        {Name: "push", Summary: "upload referenced-but-absent blobs from a local depot", Usage: "crucible depot push [--manifests DIR] --source DIR --target bunny"},
        {Name: "verify", Summary: "existence sweep plus sampled download re-hash", Usage: "crucible depot verify [--manifests DIR] --target bunny|cdn [--sample N]"},
        {Name: "get", Summary: "fetch one blob with sha re-verify", Usage: "crucible depot get <sha> <dest> --target cdn|bunny|local"},
        {Name: "pull", Summary: "incremental verified pull of every referenced blob", Usage: "crucible depot pull [--manifests DIR] --dest DIR --target cdn|bunny"},
    },
    Wired: true,
},

The stale "(SeaweedFS)" wording must not survive anywhere: grep -ri seaweedfs across the repo and fix hits.

  • Step 1: Update registry; run go test ./internal/dispatch/ — fix any test that pinned depot as unwired; add/extend a test asserting crucible depot routes to depot.Run (e.g. RunBuilder("depot", []string{"status"}) with no manifests dir returns 70/64, not the unwired 70 message — assert on stderr text).
  • Step 2: make check (vet+test+shellcheck+yamllint) PASS. Also make build && ./bin/crucible depot prints depot usage; ./bin/crucible interactive menu shows depot entries (manual eyeball via printf 'q\n' | ./bin/crucible or equivalent).
  • Step 3: Commit: feat(depot): wire depot into dispatch registry and menu.

Task 7: sow-assets-manifest cutover

Repo: /home/vicky/Projects/westgate/repositories/migration/sow-assets-manifest, new branch crucible-depot-cutover from main.

Files:

  • Delete: scripts/mirror.sh
  • Modify: scripts/build-haks.sh (the --full depot-verify block, ~L138-192), Makefile (mirror target, haks target), .gitea/workflows/release.yml, .gitea/workflows/publish-haks.yml, .gitea/workflows/build-haks.yml, scripts/lib.sh (header comments only), AGENTS.md/README.md (author-facing docs)

Consumes: the crucible depot CLI from Tasks 16. NOTE: CI resolves crucible via the flake input tracking sow-tools' default branch — the sow-tools branch must be merged before this repo's CI can pass. Local verification uses a locally built binary (sow-tools/bin/crucible on PATH). Do not run nix flake update here; note in the PR description that a nix flake update sow-tools (or lock bump) must land with/after the sow-tools merge.

Changes:

  1. release.yml "Warm depot cache (verified full pull)" step body (keep name, job env unchanged; drop the ticker/compgen bash):
      - name: Warm depot cache (verified full pull)
        env:
          BUNNY_STORAGE_READ_PASSWORD: ${{ secrets.BUNNY_STORAGE_READ_PASSWORD }}
          BUNNY_STORAGE_HOST: storage.bunnycdn.com
          BUNNY_STORAGE_ZONE: sow-assets-depot
        run: |
          nix develop --command bash -c '
            set -euo pipefail
            crucible depot status --manifests assets --target bunny
            crucible depot pull --manifests assets --dest "$DEPOT_CACHE_DIR" --target cdn
          '

(If the BUNNY_STORAGE_READ_PASSWORD secret does not exist in the repo, use secrets.BUNNY_STORAGE_PASSWORD and say so in the PR body. Check what other steps use.) 2. publish-haks.yml — same swap of the identical step. 3. build-haks.yml — the PR-check step keeps build-haks.sh --dry-run --no-verify --base "$base" (structural check) and adds crucible depot verify --manifests assets --target cdn --sample 3 as the cdn-backend verification. 4. scripts/build-haks.sh — delete the depot existence-sweep + sample-download block inside --dry-run (the depot_has_many/depot_get verify at ~L138-185); replace with a comment pointing at crucible depot verify. Remove now-unused locals. shellcheck scripts/build-haks.sh must pass. 5. Makefilemirror: target now runs crucible depot pull --manifests assets --dest "$(DEPOT_DIR)" --target cdn. haks: gains a trailing non-fatal drift report: -crucible depot status --manifests assets --target cdn (leading - so drift never fails a local build) and an echo pointing at crucible depot push --source workspace/depot --target bunny as the standard post-edit step. 6. scripts/lib.sh — no function deletions (caller analysis: every depot function used by mirror.sh/build-haks.sh retains Increment 23 callers). Add a header comment block at the top of the depot section: # NOTE: this bash depot layer is being retired. Increment 1 moved mirror/verify to 'crucible depot' (sow-tools). import/sync go in Increment 2; export/pack/publish/artifact/cdn_purge in Increment 3. Do not add new callers. Do not touch the backend-keyed stat-cache in import.sh (spec forbids it). 7. Docs — in README.md (or AGENTS.md where depot workflow is documented): document crucible depot status --manifests assets --target cdn as the canonical credential-free "is my content live?" author check, and crucible depot push --source workspace/depot --target bunny as the standard post-edit publish step.

  • Step 1: Branch; make all edits above.
  • Step 2: Verify: shellcheck scripts/*.sh; yamllint .gitea; grep -rn "mirror.sh" . --exclude-dir=.git returns nothing; with sow-tools' bin on PATH run crucible depot status --manifests assets --target cdn for real (credential-free) and confirm it produces a sane report (exit 0 or 2 acceptable; exit 1 means real drift — report it, don't hide it).
  • Step 3: Commit: feat: cut depot mirror/verify over to crucible depot, retire mirror.sh.

Self-Review Notes

  • Spec coverage: command surface (T5), drift model + unconfirmed/exit 2 (T4/T5), backends + env + no-prompt (T1/T3), IPv4 range probe + HEAD ban test (T3), Checksum PUT (T3), pull incremental (T5), cutover of all three workflows + Makefile + lib.sh headers + author docs (T7), registry/SeaweedFS fix + menu (T6), testing section requirements all mapped (T1T5), interim contract make haks drift report (T7.5).
  • Deliberate deviations from spec, justified: status --target local root via DEPOT_DIR; push treats source-missing blobs as exit 1; pull download failure exits 70 (backend error) but source-404 exits 1 (drift).
  • Out of scope, per spec non-goals: import/sync, prune/gc/export, mdl checks, rich TUI, cdn_purge.