# Crucible `depot` pipeline migration — design (charter / draft)

Date: 2026-07-12
Status: **draft — needs its own brainstorming pass before implementation.**
Depends on: `2026-07-12-crucible-assets-builder-design.md` (the `assets` builder
and `internal/assets/mdl` module) landing first.

This is a scoping charter, not an implementation-ready design. The scripts it
covers are nontrivial and lean heavily on `sow-assets-manifest/scripts/lib.sh`
(~35 KB). A full design requires studying `lib.sh` and the manifest format in
depth; that work happens in this spec's own brainstorming before any code.

## Why this exists

`sow-assets-manifest` is the last consumer still carrying real builder logic in
shell. The `assets` spec folded its standalone integrity tools into Crucible but
deliberately left the **depot-interacting pipeline** behind, because those
scripts read/write the content-addressed depot and the asset manifests — depot
domain, and a bigger lift. This charter completes the migration so
`sow-assets-manifest` becomes what the other consumers already are: a thin
caller of released `crucible` binaries with no vendored toolkit logic.

## What moves into `crucible depot`

The depot builder already owns `status/push/verify/get/pull`. This adds the
authoring side:

| Script | Proposed command | What it does today |
| --- | --- | --- |
| `import.sh` (318 lines) | `crucible depot import <edit-dir> [prefix…]` | Import an edit tree into the depot + manifests: sticky per-path member assignment, new paths to the tail member, per-category `max_bytes` rebalance (spill overflow forward, no full repack), a local stat-cache to skip unchanged files, batched hashing + parallel `depot_put_many`. |
| `sync-assets.sh` (61 lines) | `crucible depot sync <tree> [prefix…]` | Full reconcile = import + prune manifest entries whose path no longer exists (scoped by prefix when given). Delegates to `import.sh`. |
| `export.sh` (96 lines) | `crucible depot export [glob] [--hak m] [--restype ext] --to <dir>` | Materialize a manifest-selected subset out of the depot into a clean edit tree (category paths preserved, no metadata). |
| `check-duplicate-names.sh --manifests` | `crucible depot check-dupes` (name TBD) | Hak-scoped basename collisions read from `assets/*.yml` (`.assets[].path` grouped by `.assets[].hak`). Manifest-structure check, not files on disk. |

The inline **pre-import mdl gate** in `import.sh` (batched `mdl-scan.awk` pass
that aborts on an H/B model-name mismatch before any blob is hashed/uploaded)
is reimplemented with the in-process `internal/assets/mdl` module from the
`assets` spec — same H/B contract, no subprocess.

## What retires when this lands

Deleted from `sow-assets-manifest/scripts/` once the commands above are wired
and the Makefile is repointed:

- `import.sh`, `sync-assets.sh`, `export.sh`
- `mdl-scan.awk`, `mdl-name-lib.sh` (only remaining consumer was `import.sh`)
- `check-duplicate-names.sh` (its directory mode already superseded by
  `crucible assets check-dupes`; the `--manifests` mode moves here)
- whatever else in `lib.sh` becomes dead once the above are gone (assess during
  design — `lib.sh` also serves `build-haks.sh`, `pack-haks.sh`, `promote.sh`,
  etc., which are **not** in scope here, so `lib.sh` likely shrinks, not dies)

Makefile targets `import`, `sync`, `haks` (its sync/import/dupe steps), `export`
repoint to `./crucible.sh depot …`.

## Known hard parts (resolve in brainstorming)

- **Manifest read/write parity.** Byte-stable output, member `max_bytes`
  rebalance semantics, sticky path→member assignment. Must match the current
  awk/`lib.sh` behavior or diff-review of manifests becomes noise. Crucible
  already parses these manifests (`internal/pipeline`, hak build) — reuse, don't
  reinvent.
- **Depot backends.** `import`/`export` drive bunny/cdn/local through
  `lib.sh`'s `depot_put_many`/`depot_require_write` (incl. the
  `BUNNY_STORAGE_PASSWORD` prompt and cdn→bunny write normalization). Crucible's
  `internal/depot` already speaks these backends for pull/push — the authoring
  side should share that client, not a second implementation.
- **Stat-cache.** `import.sh`'s `.cache/import/<target>/<cat>.tsv` skip-unchanged
  optimization is the dominant re-import speedup. Decide whether Crucible
  reproduces it, replaces it (content-addressing already dedupes uploads), or
  drops it with a measured justification.
- **Throughput.** The shell versions are already batched to avoid per-file
  forks; a Go port should be at least as fast (in-process, no forks) — verify,
  don't assume.
- **Scoping semantics.** Prefix-scoped import/prune (segment-boundary match) and
  export glob matching (`**`/`*`/literal dots, no shell expansion) must port
  exactly.

## Out of scope

The HAK build/pack/promote/publish scripts (`build-haks.sh`, `pack-haks.sh`,
`promote.sh`, `publish-release*.sh`, `hak-artifact-record.sh`) and their
`lib.sh` support. `crucible hak build` already exists; whether these thin
release wrappers also migrate is a separate question, not this charter.

## Next step

Brainstorm this charter into a full design: read `lib.sh` and the manifest
format, settle the hard parts above, then write the implementation-ready spec
and plan. Do not implement from this document.