docs: design for crucible assets builder
Spec for folding the NWN:EE model-compile, texture-convert, and texture-upscale tools out of the Python toolkit into a self-contained `crucible assets` builder. Arguments only, no config, in place. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,192 @@
|
||||
# Crucible `assets` builder — design
|
||||
|
||||
Date: 2026-07-12
|
||||
Status: approved, pre-implementation
|
||||
|
||||
## Goal
|
||||
|
||||
Fold three NWN:EE asset tools out of the standalone Python toolkit
|
||||
(`nwnee-asset-processing`) and into Crucible as a new self-contained builder,
|
||||
`crucible assets`. The Python scripts are references only; they are broken as-is
|
||||
and carry a heavy config/staging/report layer that we drop.
|
||||
|
||||
Three commands:
|
||||
|
||||
- `crucible assets compile` — compile every ASCII `.mdl` in a directory to
|
||||
binary, in place.
|
||||
- `crucible assets convert` — convert every texture in a directory to NWN:EE
|
||||
DDS (or back to PNG/TGA), flipping vertically **every time** so a DDS is
|
||||
always upside-down relative to its source.
|
||||
- `crucible assets upscale` — upscale every texture in a directory through
|
||||
whatever upscaling backend is installed.
|
||||
|
||||
## Principles
|
||||
|
||||
- **No configuration files. Arguments only.** No `processing/` staging dirs, no
|
||||
discards, no YAML, no reports. Operates in place on the target directory.
|
||||
- **Discover, don't configure.** External tools (the NWN engine, ImageMagick, an
|
||||
upscaler) are found on `PATH` and at standard install locations. Each has a
|
||||
single override flag when discovery is not enough.
|
||||
- **Fail closed, never fake** (Crucible rule #1). A missing backend is a clear
|
||||
error naming what to install and a non-zero exit — never a placeholder
|
||||
artifact.
|
||||
- **Simplify.** Drop the reference tools' padding / power-of-two / `nwn-safe` /
|
||||
explicit-format knobs (convert auto-picks DXT1 vs DXT5) and the small-texture
|
||||
staging (upscale). These can return later if wanted.
|
||||
|
||||
## Architecture
|
||||
|
||||
A new self-contained builder that follows the existing `depot` pattern exactly
|
||||
(delegates straight to its own internal package, bypassing the legacy
|
||||
`internal/app` surface):
|
||||
|
||||
- `internal/assets/run.go` — `func Run(args []string, stdout, stderr io.Writer,
|
||||
getenv func(string) string) int`. Parses the subcommand
|
||||
(`compile|convert|upscale`) and dispatches. Returns a sysexits-style code.
|
||||
- `internal/assets/` — one file per command plus small shared helpers
|
||||
(recursion/file-selection, external-command runner, backend discovery).
|
||||
- `cmd/crucible-assets/main.go` — one-line shim:
|
||||
`func main() { os.Exit(dispatch.RunBuilder("assets", os.Args[1:])) }`.
|
||||
- `internal/dispatch`:
|
||||
- one `Registry` entry for `assets` (`Wired: true`) listing the three
|
||||
commands with `Usage`/`Options`;
|
||||
- extend the direct-delegate branch that today reads
|
||||
`if b.Name == "depot" && b.Wired` so `assets` also routes to
|
||||
`assets.Run(...)` instead of the legacy `app.Run`.
|
||||
- `docs/command-surface.md` — three visible-command rows.
|
||||
- `wrappers/consumers.txt` / wrappers are unchanged (no new consumer).
|
||||
|
||||
### Shared helpers (`internal/assets`)
|
||||
|
||||
- `walk(roots, exts)` — collect files under each root (recursive by default)
|
||||
whose extension is in `exts`, case-insensitively. Reject nothing fancy; these
|
||||
are arguments the caller chose.
|
||||
- `run(cmd, args...)` — thin `exec.Command` wrapper capturing combined output,
|
||||
returning `(output, error)`. A single indirection point so tests can observe
|
||||
invocations. cwd and env overrides passed as needed.
|
||||
- `look(names...)` — return the first name found via `exec.LookPath`, or the
|
||||
first path that exists from a supplied list of candidates. Used by every
|
||||
backend discovery.
|
||||
|
||||
## `crucible assets compile <dir>…`
|
||||
|
||||
Usage: `crucible assets compile [--nwn <install>] [--non-recursive] <dir>…`
|
||||
|
||||
Recursively find ASCII `.mdl` files under each `<dir>` and compile each to a
|
||||
binary `.mdl` in place (the compiled result replaces the source).
|
||||
|
||||
The NWN:EE engine is the only real ASCII→binary MDL compiler, so this drives it
|
||||
exactly as the reference does:
|
||||
|
||||
- **Discovery.** User data dir is fixed on Linux at
|
||||
`~/.local/share/Neverwinter Nights` (holds the flat `development/` and
|
||||
`modelcompiler/` folders the engine uses). The `nwmain` binary is found by
|
||||
probing standard install roots, first hit wins:
|
||||
- `~/.local/share/Steam/steamapps/common/Neverwinter Nights/bin/linux-x86/nwmain-linux`
|
||||
- GOG: `~/GOG Games/Neverwinter Nights Enhanced Edition/.../nwmain-linux`
|
||||
- Beamdog install dirs.
|
||||
- `--nwn <install>` overrides (path to the install root or directly to the
|
||||
binary).
|
||||
- **Headless.** The engine is a GUI binary. If there is no `DISPLAY` and
|
||||
`xvfb-run` is present, wrap the call:
|
||||
`xvfb-run -a --server-args=-screen 0 1024x768x24 nwmain-linux compilemodel <stem>`.
|
||||
- **Per model (one at a time — the engine's `development/` and `modelcompiler/`
|
||||
folders are flat and single-slot):**
|
||||
1. Skip if the file is not ASCII (binary/compiled `.mdl`) — reported, not an
|
||||
error.
|
||||
2. Copy source into `development/<name>`.
|
||||
3. Run `nwmain-linux compilemodel <stem>` with cwd = the binary dir.
|
||||
4. Collect the compiled artifact from `modelcompiler/` matched
|
||||
case-insensitively by stem.
|
||||
5. Move it back over the source path, lowercased.
|
||||
6. Clean the temp files created in `development/` and `modelcompiler/`.
|
||||
- **Collisions.** A pre-existing `development/<name>` or `modelcompiler/<stem>`
|
||||
aborts that model before any mutation.
|
||||
|
||||
Exit: non-zero if any model fails; each failure prints a short excerpt of the
|
||||
engine log (`~/.local/share/Neverwinter Nights/logs/nwengineLog.txt`) as
|
||||
advisory diagnostics.
|
||||
|
||||
Dropped vs reference: internal-name-mismatch pre-checks and the
|
||||
`Model Names Differ` fail rule are kept (cheap, prevents silent wrong output);
|
||||
the discard-manifest, dry-run/report layer, and `processing/in|out` staging are
|
||||
dropped.
|
||||
|
||||
## `crucible assets convert <dir>…`
|
||||
|
||||
Usage: `crucible assets convert [--to dds|png|tga] [--backend <path>]
|
||||
[--non-recursive] <dir>…`
|
||||
|
||||
Recursively convert textures in place. `--to` defaults to `dds`. The source set
|
||||
is every texture under the roots whose format is not already the target, among
|
||||
`{.dds, .png, .tga}`.
|
||||
|
||||
**Every conversion flips vertically exactly once.** Converting to DDS produces
|
||||
an upside-down DDS (NWN's convention); converting a DDS back to PNG/TGA flips it
|
||||
upright again. The flip is unconditional — it is the defining behavior, not a
|
||||
policy.
|
||||
|
||||
Backend: ImageMagick (`magick`), one tool for decode + flip + encode across all
|
||||
three formats:
|
||||
|
||||
- to DDS: `magick <src> -flip -define dds:mipmaps=<n> -define
|
||||
dds:compression=<dxt1|dxt5> <dst>.dds`, choosing DXT1 when the source is
|
||||
opaque and DXT5 when it has alpha (detected with `magick identify`).
|
||||
- from DDS: `magick <src>.dds -flip <dst>.<png|tga>`.
|
||||
- `--backend <path>` points at a different ImageMagick-compatible binary or a
|
||||
dedicated encoder for higher-quality DXT (e.g. compressonator); default is
|
||||
`magick`.
|
||||
|
||||
The original file is replaced in place — canonical DDS is the point. Exit
|
||||
non-zero if any file fails.
|
||||
|
||||
Dropped vs reference: `--pad-color`, POT/legacy-safe/`nwn-safe`/`--strict`
|
||||
sizing, explicit `--format`, alpha-detect toggles, discard staging. Auto DXT
|
||||
selection replaces the format knobs.
|
||||
|
||||
## `crucible assets upscale <dir>…`
|
||||
|
||||
Usage: `crucible assets upscale [--scale N] [--backend <path>]
|
||||
[--non-recursive] <dir>…`
|
||||
|
||||
Recursively upscale textures in place.
|
||||
|
||||
- **Backend discovery**, first found wins: `upscayl-bin`, `upscayl`,
|
||||
`realesrgan-ncnn-vulkan`, `waifu2x-ncnn-vulkan`. These share a compatible
|
||||
ncnn-vulkan CLI shape (`-i <in> -o <out> -s <scale>`). `--backend <path>`
|
||||
overrides.
|
||||
- `--scale N` default 4.
|
||||
- Backends operate on PNG; real NWN textures are DDS, so for a `.dds` input the
|
||||
command reuses the convert helpers: dds→png (flip), run the backend, png→dds
|
||||
(flip back) — yielding a correctly-flipped upscaled DDS. PNG/TGA inputs are
|
||||
upscaled directly in place.
|
||||
- No backend found → fail closed with a message naming the supported backends.
|
||||
|
||||
Dropped vs reference: min-dimension small-texture staging, the separate
|
||||
`--dds-backend` plumbing (it reuses convert), configured backend-path table.
|
||||
|
||||
## Error handling
|
||||
|
||||
- Missing external tool (engine, `magick`, upscaler) → non-zero exit, message
|
||||
names the tool and how to get it. Never a faked artifact.
|
||||
- Per-file failures are collected and printed as a summary line at the end; the
|
||||
command exits non-zero if any file failed but still processes the rest.
|
||||
- Unknown subcommand / bad flags → usage error, exit 64.
|
||||
|
||||
## Testing
|
||||
|
||||
Go tests in `internal/assets`:
|
||||
|
||||
- flag/subcommand parsing and usage errors;
|
||||
- backend discovery against a fabricated `PATH` (temp dir with stub
|
||||
executables) — asserts first-match ordering and the `--backend` override;
|
||||
- `walk` recursion and extension filtering, including `--non-recursive`;
|
||||
- one real convert round-trip using `magick` (present in the dev shell):
|
||||
encode a small PNG to DDS and back, asserting the pixels are vertically
|
||||
flipped after a single conversion and restored after the round trip.
|
||||
|
||||
External engine/upscaler calls are exercised through the `run` indirection with
|
||||
a stub in tests; they are not invoked for real in CI.
|
||||
|
||||
`make check` must stay green; add an `assets` expectation to `make smoke` for
|
||||
the wired exit.
|
||||
Reference in New Issue
Block a user