docs: design for crucible assets builder

Spec for folding the NWN:EE model-compile, texture-convert, and
texture-upscale tools out of the Python toolkit into a self-contained
`crucible assets` builder. Arguments only, no config, in place.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-07-12 09:12:09 +02:00
co-authored by Claude Opus 4.8
parent e9860c12c6
commit 2742f8937e
@@ -0,0 +1,192 @@
# Crucible `assets` builder — design
Date: 2026-07-12
Status: approved, pre-implementation
## Goal
Fold three NWN:EE asset tools out of the standalone Python toolkit
(`nwnee-asset-processing`) and into Crucible as a new self-contained builder,
`crucible assets`. The Python scripts are references only; they are broken as-is
and carry a heavy config/staging/report layer that we drop.
Three commands:
- `crucible assets compile` — compile every ASCII `.mdl` in a directory to
binary, in place.
- `crucible assets convert` — convert every texture in a directory to NWN:EE
DDS (or back to PNG/TGA), flipping vertically **every time** so a DDS is
always upside-down relative to its source.
- `crucible assets upscale` — upscale every texture in a directory through
whatever upscaling backend is installed.
## Principles
- **No configuration files. Arguments only.** No `processing/` staging dirs, no
discards, no YAML, no reports. Operates in place on the target directory.
- **Discover, don't configure.** External tools (the NWN engine, ImageMagick, an
upscaler) are found on `PATH` and at standard install locations. Each has a
single override flag when discovery is not enough.
- **Fail closed, never fake** (Crucible rule #1). A missing backend is a clear
error naming what to install and a non-zero exit — never a placeholder
artifact.
- **Simplify.** Drop the reference tools' padding / power-of-two / `nwn-safe` /
explicit-format knobs (convert auto-picks DXT1 vs DXT5) and the small-texture
staging (upscale). These can return later if wanted.
## Architecture
A new self-contained builder that follows the existing `depot` pattern exactly
(delegates straight to its own internal package, bypassing the legacy
`internal/app` surface):
- `internal/assets/run.go` — `func Run(args []string, stdout, stderr io.Writer,
getenv func(string) string) int`. Parses the subcommand
(`compile|convert|upscale`) and dispatches. Returns a sysexits-style code.
- `internal/assets/` — one file per command plus small shared helpers
(recursion/file-selection, external-command runner, backend discovery).
- `cmd/crucible-assets/main.go` — one-line shim:
`func main() { os.Exit(dispatch.RunBuilder("assets", os.Args[1:])) }`.
- `internal/dispatch`:
- one `Registry` entry for `assets` (`Wired: true`) listing the three
commands with `Usage`/`Options`;
- extend the direct-delegate branch that today reads
`if b.Name == "depot" && b.Wired` so `assets` also routes to
`assets.Run(...)` instead of the legacy `app.Run`.
- `docs/command-surface.md` — three visible-command rows.
- `wrappers/consumers.txt` / wrappers are unchanged (no new consumer).
### Shared helpers (`internal/assets`)
- `walk(roots, exts)` — collect files under each root (recursive by default)
whose extension is in `exts`, case-insensitively. Reject nothing fancy; these
are arguments the caller chose.
- `run(cmd, args...)` — thin `exec.Command` wrapper capturing combined output,
returning `(output, error)`. A single indirection point so tests can observe
invocations. cwd and env overrides passed as needed.
- `look(names...)` — return the first name found via `exec.LookPath`, or the
first path that exists from a supplied list of candidates. Used by every
backend discovery.
## `crucible assets compile <dir>…`
Usage: `crucible assets compile [--nwn <install>] [--non-recursive] <dir>…`
Recursively find ASCII `.mdl` files under each `<dir>` and compile each to a
binary `.mdl` in place (the compiled result replaces the source).
The NWN:EE engine is the only real ASCII→binary MDL compiler, so this drives it
exactly as the reference does:
- **Discovery.** User data dir is fixed on Linux at
`~/.local/share/Neverwinter Nights` (holds the flat `development/` and
`modelcompiler/` folders the engine uses). The `nwmain` binary is found by
probing standard install roots, first hit wins:
- `~/.local/share/Steam/steamapps/common/Neverwinter Nights/bin/linux-x86/nwmain-linux`
- GOG: `~/GOG Games/Neverwinter Nights Enhanced Edition/.../nwmain-linux`
- Beamdog install dirs.
- `--nwn <install>` overrides (path to the install root or directly to the
binary).
- **Headless.** The engine is a GUI binary. If there is no `DISPLAY` and
`xvfb-run` is present, wrap the call:
`xvfb-run -a --server-args=-screen 0 1024x768x24 nwmain-linux compilemodel <stem>`.
- **Per model (one at a time — the engine's `development/` and `modelcompiler/`
folders are flat and single-slot):**
1. Skip if the file is not ASCII (binary/compiled `.mdl`) — reported, not an
error.
2. Copy source into `development/<name>`.
3. Run `nwmain-linux compilemodel <stem>` with cwd = the binary dir.
4. Collect the compiled artifact from `modelcompiler/` matched
case-insensitively by stem.
5. Move it back over the source path, lowercased.
6. Clean the temp files created in `development/` and `modelcompiler/`.
- **Collisions.** A pre-existing `development/<name>` or `modelcompiler/<stem>`
aborts that model before any mutation.
Exit: non-zero if any model fails; each failure prints a short excerpt of the
engine log (`~/.local/share/Neverwinter Nights/logs/nwengineLog.txt`) as
advisory diagnostics.
Dropped vs reference: internal-name-mismatch pre-checks and the
`Model Names Differ` fail rule are kept (cheap, prevents silent wrong output);
the discard-manifest, dry-run/report layer, and `processing/in|out` staging are
dropped.
## `crucible assets convert <dir>…`
Usage: `crucible assets convert [--to dds|png|tga] [--backend <path>]
[--non-recursive] <dir>…`
Recursively convert textures in place. `--to` defaults to `dds`. The source set
is every texture under the roots whose format is not already the target, among
`{.dds, .png, .tga}`.
**Every conversion flips vertically exactly once.** Converting to DDS produces
an upside-down DDS (NWN's convention); converting a DDS back to PNG/TGA flips it
upright again. The flip is unconditional — it is the defining behavior, not a
policy.
Backend: ImageMagick (`magick`), one tool for decode + flip + encode across all
three formats:
- to DDS: `magick <src> -flip -define dds:mipmaps=<n> -define
dds:compression=<dxt1|dxt5> <dst>.dds`, choosing DXT1 when the source is
opaque and DXT5 when it has alpha (detected with `magick identify`).
- from DDS: `magick <src>.dds -flip <dst>.<png|tga>`.
- `--backend <path>` points at a different ImageMagick-compatible binary or a
dedicated encoder for higher-quality DXT (e.g. compressonator); default is
`magick`.
The original file is replaced in place — canonical DDS is the point. Exit
non-zero if any file fails.
Dropped vs reference: `--pad-color`, POT/legacy-safe/`nwn-safe`/`--strict`
sizing, explicit `--format`, alpha-detect toggles, discard staging. Auto DXT
selection replaces the format knobs.
## `crucible assets upscale <dir>…`
Usage: `crucible assets upscale [--scale N] [--backend <path>]
[--non-recursive] <dir>…`
Recursively upscale textures in place.
- **Backend discovery**, first found wins: `upscayl-bin`, `upscayl`,
`realesrgan-ncnn-vulkan`, `waifu2x-ncnn-vulkan`. These share a compatible
ncnn-vulkan CLI shape (`-i <in> -o <out> -s <scale>`). `--backend <path>`
overrides.
- `--scale N` default 4.
- Backends operate on PNG; real NWN textures are DDS, so for a `.dds` input the
command reuses the convert helpers: dds→png (flip), run the backend, png→dds
(flip back) — yielding a correctly-flipped upscaled DDS. PNG/TGA inputs are
upscaled directly in place.
- No backend found → fail closed with a message naming the supported backends.
Dropped vs reference: min-dimension small-texture staging, the separate
`--dds-backend` plumbing (it reuses convert), configured backend-path table.
## Error handling
- Missing external tool (engine, `magick`, upscaler) → non-zero exit, message
names the tool and how to get it. Never a faked artifact.
- Per-file failures are collected and printed as a summary line at the end; the
command exits non-zero if any file failed but still processes the rest.
- Unknown subcommand / bad flags → usage error, exit 64.
## Testing
Go tests in `internal/assets`:
- flag/subcommand parsing and usage errors;
- backend discovery against a fabricated `PATH` (temp dir with stub
executables) — asserts first-match ordering and the `--backend` override;
- `walk` recursion and extension filtering, including `--non-recursive`;
- one real convert round-trip using `magick` (present in the dev shell):
encode a small PNG to DDS and back, asserting the pixels are vertically
flipped after a single conversion and restored after the round trip.
External engine/upscaler calls are exercised through the `run` indirection with
a stub in tests; they are not invoked for real in CI.
`make check` must stay green; add an `assets` expectation to `make smoke` for
the wired exit.