assets builder tools (#39)
build-binaries / build-binaries (push) Successful in 2m5s
test / test (push) Successful in 1m23s

Reviewed-on: #39
Reviewed-by: xtul <mpiasecki720@protonmail.com>
This commit was merged in pull request #39.
This commit is contained in:
2026-07-12 12:51:17 +00:00
parent 87fd3d8c04
commit 5ebef57160
26 changed files with 4625 additions and 10 deletions
+11
View File
@@ -0,0 +1,11 @@
// Command crucible-assets is the standalone assets builder (equivalent to
// `crucible assets`). A single-token binary keeps consumer wrapper scripts simple.
package main
import (
"os"
"git.westgate.pw/ShadowsOverWestgate/sow-tools/internal/dispatch"
)
func main() { os.Exit(dispatch.RunBuilder("assets", os.Args[1:])) }
+7
View File
@@ -22,6 +22,13 @@ aliases.
| `topdata` | `convert` | Convert between 2DA and native JSON/module formats. |
| `wiki` | `build` | Render wiki page drafts from compiled topdata. |
| `wiki` | `deploy` | Deploy generated wiki pages to NodeBB. |
| `assets` | `compile` | Compile ASCII `.mdl` models to binary in place. |
| `assets` | `convert` | Convert textures to/from NWN DDS (flips vertically). |
| `assets` | `upscale` | Upscale textures through an installed backend. |
| `assets` | `check-mdl` | Report uncompiled ASCII `.mdl` and model-name mismatches. |
| `assets` | `fix-mdl` | Lowercase `.mdl` names and rewrite model identity to match. |
| `assets` | `check-dupes` | Report runtime-name (basename) collisions across dirs. |
| `assets` | `clean-dupes` | Delete clean-tree files whose basename collides with primary. |
`crucible-depot` remains registered but unwired. It fails closed with exit `70`
and never emits placeholder artifacts.
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,342 @@
# Crucible `assets` builder — design
Date: 2026-07-12
Status: approved, pre-implementation
## Goal
Fold the NWN:EE asset tools out of two standalone toolkits into Crucible as a
new self-contained builder, `crucible assets`:
- from the Python toolkit (`nwnee-asset-processing`) — model compile, texture
convert, texture upscale. Those scripts are references only; broken as-is and
carrying a heavy config/staging/report layer that we drop.
- from `sow-assets-manifest/scripts/` — the model-name and duplicate-name
integrity tools (`check-ascii-mdl.sh`, `fix-mdl-model-names.sh`,
`check-duplicate-names.sh`, `clean-duplicate-names.sh`). These are ported into
Go and the shell drivers removed; `sow-assets-manifest` calls
`./crucible.sh assets …` through its bootstrap wrapper instead.
Seven commands:
- `crucible assets compile` — compile every ASCII `.mdl` in a directory to
binary, in place.
- `crucible assets convert` — convert every texture in a directory to NWN:EE
DDS (or back to PNG/TGA), flipping vertically **every time** so a DDS is
always upside-down relative to its source.
- `crucible assets upscale` — upscale every texture in a directory through
whatever upscaling backend is installed.
- `crucible assets check-mdl` — report uncompiled ASCII `.mdl` files and
model-name mismatches (read-only).
- `crucible assets fix-mdl` — lowercase `.mdl` filenames and rewrite ASCII
model/root names to match each file's stem.
- `crucible assets check-dupes` — report runtime-name (basename) collisions
across directories.
- `crucible assets clean-dupes` — delete files from a "clean" tree whose
basename collides with anything in a "primary" tree.
## Principles
- **No configuration files. Arguments only.** No `processing/` staging dirs, no
discards, no YAML, no reports. Operates in place on the target directory.
- **Discover, don't configure.** External tools (the NWN engine, ImageMagick, an
upscaler) are found on `PATH` and at standard install locations. Each has a
single override flag when discovery is not enough.
- **Fail closed, never fake** (Crucible rule #1). A missing backend is a clear
error naming what to install and a non-zero exit — never a placeholder
artifact.
- **Simplify.** Drop the reference tools' padding / power-of-two / `nwn-safe` /
explicit-format knobs (convert auto-picks DXT1 vs DXT5) and the small-texture
staging (upscale). These can return later if wanted.
## Architecture
A new self-contained builder that follows the existing `depot` pattern exactly
(delegates straight to its own internal package, bypassing the legacy
`internal/app` surface):
- `internal/assets/run.go` — `func Run(args []string, stdout, stderr io.Writer,
getenv func(string) string) int`. Parses the subcommand
(`compile|convert|upscale|check-mdl|fix-mdl|check-dupes|clean-dupes`) and
dispatches. Returns a sysexits-style code.
- `internal/assets/` — one file per command plus small shared helpers
(recursion/file-selection, external-command runner, backend discovery).
- `cmd/crucible-assets/main.go` — one-line shim:
`func main() { os.Exit(dispatch.RunBuilder("assets", os.Args[1:])) }`.
- `internal/dispatch`:
- one `Registry` entry for `assets` (`Wired: true`) listing the seven
commands with `Usage`/`Options`;
- extend the direct-delegate branch that today reads
`if b.Name == "depot" && b.Wired` so `assets` also routes to
`assets.Run(...)` instead of the legacy `app.Run`.
- `docs/command-surface.md` — seven visible-command rows.
- `wrappers/consumers.txt` / wrappers are unchanged (no new consumer; the
consumers listed already vendor the wrapper).
### Shared helpers (`internal/assets`)
- `walk(roots, exts)` — collect files under each root (recursive by default)
whose extension is in `exts`, case-insensitively. Reject nothing fancy; these
are arguments the caller chose.
- `run(cmd, args...)` — thin `exec.Command` wrapper capturing combined output,
returning `(output, error)`. A single indirection point so tests can observe
invocations. cwd and env overrides passed as needed.
- `look(names...)` — return the first name found via `exec.LookPath`, or the
first path that exists from a supplied list of candidates. Used by every
backend discovery.
### Shared MDL name module (`internal/assets/mdl`)
A pure-Go port of `mdl-name-lib.sh` + `mdl-scan.awk`, no external tools. One
place three commands share (`compile`'s pre-check, `check-mdl`, `fix-mdl`):
- `IsASCII(path)` — NUL in the first 256 bytes → binary; else the leading
keyword must be `#`/`newmodel`/`node`/`setsupermodel`. Mirrors `mdl_is_ascii`.
- `ExpectedName(path)` — the file stem (sans `.mdl`).
- `CheckNames(path) []Mismatch` — for an ASCII model, report header-token
mismatches (`newmodel`, `setsupermodel`, `beginmodelgeom`, `endmodelgeom`,
`donemodel`, `newanim`, `doneanim` — case-insensitive vs the stem) and the
geometry **base node** mismatch (the node inside `beginmodelgeom` whose parent
is `null`; empty if none). Mirrors `mdl_check_model_name` + `mdl_base_name`.
- `FixNames(in) (out []byte, changed bool)` — rewrite only the model identity:
the header tokens above, plus the base node's declaration / any `parent` /
`animroot` pointing at the *old* base name → the expected name. Animation bone
names and supermodel animroots are left untouched so inheritance keeps working.
Mirrors `mdl_fix_model_name` byte-for-byte in intent.
Detection must stay identical to the awk classifier so behavior does not drift
while `import.sh` still runs the awk version (see Consumer migration).
## `crucible assets compile <dir>…`
Usage: `crucible assets compile [--nwn <install>] [--non-recursive] <dir>…`
Recursively find ASCII `.mdl` files under each `<dir>` and compile each to a
binary `.mdl` in place (the compiled result replaces the source).
The NWN:EE engine is the only real ASCII→binary MDL compiler, so this drives it
exactly as the reference does:
- **Discovery.** User data dir is fixed on Linux at
`~/.local/share/Neverwinter Nights` (holds the flat `development/` and
`modelcompiler/` folders the engine uses). The `nwmain` binary is found by
probing standard install roots, first hit wins:
- `~/.local/share/Steam/steamapps/common/Neverwinter Nights/bin/linux-x86/nwmain-linux`
- GOG: `~/GOG Games/Neverwinter Nights Enhanced Edition/.../nwmain-linux`
- Beamdog install dirs.
- `--nwn <install>` overrides (path to the install root or directly to the
binary).
- **Headless.** The engine is a GUI binary. If there is no `DISPLAY` and
`xvfb-run` is present, wrap the call:
`xvfb-run -a --server-args=-screen 0 1024x768x24 nwmain-linux compilemodel <stem>`.
- **Per model (one at a time — the engine's `development/` and `modelcompiler/`
folders are flat and single-slot):**
1. Skip if the file is not ASCII (binary/compiled `.mdl`) — reported, not an
error.
2. Copy source into `development/<name>`.
3. Run `nwmain-linux compilemodel <stem>` with cwd = the binary dir.
4. Collect the compiled artifact from `modelcompiler/` matched
case-insensitively by stem.
5. Move it back over the source path, lowercased.
6. Clean the temp files created in `development/` and `modelcompiler/`.
- **Collisions.** A pre-existing `development/<name>` or `modelcompiler/<stem>`
aborts that model before any mutation.
Exit: non-zero if any model fails; each failure prints a short excerpt of the
engine log (`~/.local/share/Neverwinter Nights/logs/nwengineLog.txt`) as
advisory diagnostics.
Dropped vs reference: internal-name-mismatch pre-checks and the
`Model Names Differ` fail rule are kept (cheap, prevents silent wrong output),
using the shared `internal/assets/mdl` module — `compile` aborts a model whose
names differ and tells the user to run `crucible assets fix-mdl`. It does **not**
auto-fix (kept separate, by decision). The discard-manifest, dry-run/report
layer, and `processing/in|out` staging are dropped.
## `crucible assets convert <dir>…`
Usage: `crucible assets convert [--to dds|png|tga] [--backend <path>]
[--non-recursive] <dir>…`
Recursively convert textures in place. `--to` defaults to `dds`. The source set
is every texture under the roots whose format is not already the target, among
`{.dds, .png, .tga}`.
**Every conversion flips vertically exactly once.** Converting to DDS produces
an upside-down DDS (NWN's convention); converting a DDS back to PNG/TGA flips it
upright again. The flip is unconditional — it is the defining behavior, not a
policy.
Backend: ImageMagick (`magick`), one tool for decode + flip + encode across all
three formats:
- to DDS: `magick <src> -flip -define dds:mipmaps=<n> -define
dds:compression=<dxt1|dxt5> <dst>.dds`, choosing DXT1 when the source is
opaque and DXT5 when it has alpha (detected with `magick identify`).
- from DDS: `magick <src>.dds -flip <dst>.<png|tga>`.
- `--backend <path>` points at a different ImageMagick-compatible binary or a
dedicated encoder for higher-quality DXT (e.g. compressonator); default is
`magick`.
The original file is replaced in place — canonical DDS is the point. Exit
non-zero if any file fails.
Dropped vs reference: `--pad-color`, POT/legacy-safe/`nwn-safe`/`--strict`
sizing, explicit `--format`, alpha-detect toggles, discard staging. Auto DXT
selection replaces the format knobs.
## `crucible assets upscale <dir>…`
Usage: `crucible assets upscale [--scale N] [--backend <path>]
[--non-recursive] <dir>…`
Recursively upscale textures in place.
- **Backend discovery**, first found wins: `upscayl-bin`, `upscayl`,
`realesrgan-ncnn-vulkan`, `waifu2x-ncnn-vulkan`. These share a compatible
ncnn-vulkan CLI shape (`-i <in> -o <out> -s <scale>`). `--backend <path>`
overrides.
- `--scale N` default 4.
- Backends operate on PNG; real NWN textures are DDS, so for a `.dds` input the
command reuses the convert helpers: dds→png (flip), run the backend, png→dds
(flip back) — yielding a correctly-flipped upscaled DDS. PNG/TGA inputs are
upscaled directly in place.
- No backend found → fail closed with a message naming the supported backends.
Dropped vs reference: min-dimension small-texture staging, the separate
`--dds-backend` plumbing (it reuses convert), configured backend-path table.
## `crucible assets check-mdl <path>…`
Port of `check-ascii-mdl.sh`. Read-only. Each `<path>` is a file or a directory
(recursed for `*.mdl`). For every model, using `internal/assets/mdl`:
- report an **uncompiled ASCII** `.mdl` (a binary/compiled one is fine, skipped);
- report every model-name mismatch (header tokens and geometry base node) with
file, line, the offending token, and the expected stem.
Exit non-zero if any ASCII model or mismatch is found; zero and an `OK` line
otherwise. No mutation.
## `crucible assets fix-mdl [--dry-run] <path>…`
Port of `fix-mdl-model-names.sh`. For each `*.mdl` under the paths:
1. **Lowercase the filename** if it is not already lowercase (`mv`; abort that
file on a case-collision with an existing target).
2. For ASCII models with a name mismatch, rewrite the model identity in place
via `mdl.FixNames` (binary models are skipped — the engine owns those).
`--dry-run` reports every "would lowercase" / "would fix" without touching disk.
Prints a per-file log and a final `no broken mdl model names found` when clean.
Selection optimization from the reference is preserved: a file is a fix
candidate only if it needs a content fix (name mismatch) or a filename
lowercase, so the expensive per-file read/rewrite runs only on the few that
matter, not the whole tree.
## `crucible assets check-dupes <dir>…`
Port of `check-duplicate-names.sh`'s **directory mode only**. Read-only. NWN
packs hak entries by basename, not path, so files at different paths with the
same basename (case-insensitively) silently clobber on pack. Across all `<dir>`
args, report any two files whose lowercased basename matches.
Exit non-zero if any collision is found.
The script's `--manifests` mode (hak-scoped collisions read from `assets/*.yml`)
is **not** ported here — it inspects manifest structure, not files on disk, so
it belongs with the manifest/depot tooling. It moves in the follow-up spec (see
Out of scope); until then `sow-assets-manifest` keeps
`check-duplicate-names.sh` solely for that mode.
## `crucible assets clean-dupes [--dry-run] <primary> <clean>`
Port of `clean-duplicate-names.sh`. Deletes files from the `<clean>` tree whose
basename collides (case-insensitively) with any file in the `<primary>` tree.
Mutates only `<clean>`. Same rule as `check-dupes` directory mode, but resolves
the collision by removal instead of reporting.
- Refuses to run if `<primary>` and `<clean>` overlap (either contains the
other), matching the script's guard.
- `--dry-run` reports every "would delete" without removing.
Exit zero on success with a count of removed (or would-remove) files.
## Consumer migration (`sow-assets-manifest`)
`sow-assets-manifest` already ships the `crucible` bootstrap wrapper
(`crucible.sh`) and uses `crucible depot` from its Makefile, so it resolves a
released `crucible` with zero extra setup.
- **Delete** the three fully-folded driver scripts: `scripts/check-ascii-mdl.sh`,
`scripts/fix-mdl-model-names.sh`, `scripts/clean-duplicate-names.sh`.
- **Repoint** the Makefile targets to the wrapper:
- `check-mdl` → `./crucible.sh assets check-mdl …`
- `fix-mdl` → `./crucible.sh assets fix-mdl [--dry-run] …`
- `check-dupes` — its directory scan → `./crucible.sh assets check-dupes …`;
the `check-duplicate-names.sh --manifests` line stays as-is.
- `clean-dupes` → `./crucible.sh assets clean-dupes [--dry-run] <PRIMARY> <CLEAN>`
- **Keep for now:**
- `scripts/check-duplicate-names.sh` — kept **only** for its `--manifests`
mode, still called by the `check` and `haks` targets. Its directory mode is
superseded by `crucible assets check-dupes`. Ported and deleted in the
follow-up.
- `scripts/mdl-scan.awk` and `scripts/mdl-name-lib.sh` — still sourced by
`import.sh` for its batched pre-import gate (~28× faster than per-file, on a
hot path). They retire in the follow-up too. Because both the awk classifier
and the Go `mdl` module must agree, the port keeps detection byte-for-byte
identical (a shared fixture set checks this).
Nothing in `sow-assets-manifest` is committed by this project's plan except the
script deletions and Makefile edits; that repo's change lands as its own commit
once the new `crucible` release with `assets` is available.
## Out of scope — follow-up spec
Moving the **depot-interacting pipeline** into `crucible depot` is a separate
project with its own spec
(`2026-07-12-crucible-depot-pipeline-migration-design.md`). It covers
`import.sh`, `sync-assets.sh`, `export.sh`, the inline mdl gate, and the
`check-duplicate-names.sh --manifests` manifest-collision check. When that lands:
its gate uses the in-process `internal/assets/mdl` module, and
`scripts/mdl-scan.awk`, `scripts/mdl-name-lib.sh`, and
`scripts/check-duplicate-names.sh` are all deleted. This spec deliberately stops
at the standalone, file-on-disk integrity tools to keep one focused plan.
## Error handling
- Missing external tool (engine, `magick`, upscaler) → non-zero exit, message
names the tool and how to get it. Never a faked artifact. The pure-Go
integrity commands (`check-mdl`, `fix-mdl`, `check-dupes`, `clean-dupes`) need
no external tool.
- Per-file failures are collected and printed as a summary line at the end; the
command exits non-zero if any file failed but still processes the rest.
- Report commands (`check-mdl`, `check-dupes`) exit non-zero when they *find*
problems — that is their contract as CI/pre-import gates, not a tool error.
- Unknown subcommand / bad flags → usage error, exit 64.
## Testing
Go tests in `internal/assets`:
- flag/subcommand parsing and usage errors;
- backend discovery against a fabricated `PATH` (temp dir with stub
executables) — asserts first-match ordering and the `--backend` override;
- `walk` recursion and extension filtering, including `--non-recursive`;
- one real convert round-trip using `magick` (present in the dev shell):
encode a small PNG to DDS and back, asserting the pixels are vertically
flipped after a single conversion and restored after the round trip.
- `internal/assets/mdl`: table-driven fixtures — ASCII vs binary detection,
each header-token mismatch, the base-node mismatch, and `FixNames` producing
a model that then passes `CheckNames`. A parity fixture set shared in intent
with `mdl-scan.awk` so the Go port and the still-present awk agree.
- `check-dupes`/`clean-dupes`: temp trees asserting basename-collision
detection (case-insensitive), the overlap guard, and `--dry-run` mutating
nothing.
External engine/upscaler calls are exercised through the `run` indirection with
a stub in tests; they are not invoked for real in CI.
`make check` must stay green; add an `assets` expectation to `make smoke` for
the wired exit.
@@ -0,0 +1,90 @@
# Crucible `depot` pipeline migration — design (charter / draft)
Date: 2026-07-12
Status: **draft — needs its own brainstorming pass before implementation.**
Depends on: `2026-07-12-crucible-assets-builder-design.md` (the `assets` builder
and `internal/assets/mdl` module) landing first.
This is a scoping charter, not an implementation-ready design. The scripts it
covers are nontrivial and lean heavily on `sow-assets-manifest/scripts/lib.sh`
(~35 KB). A full design requires studying `lib.sh` and the manifest format in
depth; that work happens in this spec's own brainstorming before any code.
## Why this exists
`sow-assets-manifest` is the last consumer still carrying real builder logic in
shell. The `assets` spec folded its standalone integrity tools into Crucible but
deliberately left the **depot-interacting pipeline** behind, because those
scripts read/write the content-addressed depot and the asset manifests — depot
domain, and a bigger lift. This charter completes the migration so
`sow-assets-manifest` becomes what the other consumers already are: a thin
caller of released `crucible` binaries with no vendored toolkit logic.
## What moves into `crucible depot`
The depot builder already owns `status/push/verify/get/pull`. This adds the
authoring side:
| Script | Proposed command | What it does today |
| --- | --- | --- |
| `import.sh` (318 lines) | `crucible depot import <edit-dir> [prefix…]` | Import an edit tree into the depot + manifests: sticky per-path member assignment, new paths to the tail member, per-category `max_bytes` rebalance (spill overflow forward, no full repack), a local stat-cache to skip unchanged files, batched hashing + parallel `depot_put_many`. |
| `sync-assets.sh` (61 lines) | `crucible depot sync <tree> [prefix…]` | Full reconcile = import + prune manifest entries whose path no longer exists (scoped by prefix when given). Delegates to `import.sh`. |
| `export.sh` (96 lines) | `crucible depot export [glob] [--hak m] [--restype ext] --to <dir>` | Materialize a manifest-selected subset out of the depot into a clean edit tree (category paths preserved, no metadata). |
| `check-duplicate-names.sh --manifests` | `crucible depot check-dupes` (name TBD) | Hak-scoped basename collisions read from `assets/*.yml` (`.assets[].path` grouped by `.assets[].hak`). Manifest-structure check, not files on disk. |
The inline **pre-import mdl gate** in `import.sh` (batched `mdl-scan.awk` pass
that aborts on an H/B model-name mismatch before any blob is hashed/uploaded)
is reimplemented with the in-process `internal/assets/mdl` module from the
`assets` spec — same H/B contract, no subprocess.
## What retires when this lands
Deleted from `sow-assets-manifest/scripts/` once the commands above are wired
and the Makefile is repointed:
- `import.sh`, `sync-assets.sh`, `export.sh`
- `mdl-scan.awk`, `mdl-name-lib.sh` (only remaining consumer was `import.sh`)
- `check-duplicate-names.sh` (its directory mode already superseded by
`crucible assets check-dupes`; the `--manifests` mode moves here)
- whatever else in `lib.sh` becomes dead once the above are gone (assess during
design — `lib.sh` also serves `build-haks.sh`, `pack-haks.sh`, `promote.sh`,
etc., which are **not** in scope here, so `lib.sh` likely shrinks, not dies)
Makefile targets `import`, `sync`, `haks` (its sync/import/dupe steps), `export`
repoint to `./crucible.sh depot …`.
## Known hard parts (resolve in brainstorming)
- **Manifest read/write parity.** Byte-stable output, member `max_bytes`
rebalance semantics, sticky path→member assignment. Must match the current
awk/`lib.sh` behavior or diff-review of manifests becomes noise. Crucible
already parses these manifests (`internal/pipeline`, hak build) — reuse, don't
reinvent.
- **Depot backends.** `import`/`export` drive bunny/cdn/local through
`lib.sh`'s `depot_put_many`/`depot_require_write` (incl. the
`BUNNY_STORAGE_PASSWORD` prompt and cdn→bunny write normalization). Crucible's
`internal/depot` already speaks these backends for pull/push — the authoring
side should share that client, not a second implementation.
- **Stat-cache.** `import.sh`'s `.cache/import/<target>/<cat>.tsv` skip-unchanged
optimization is the dominant re-import speedup. Decide whether Crucible
reproduces it, replaces it (content-addressing already dedupes uploads), or
drops it with a measured justification.
- **Throughput.** The shell versions are already batched to avoid per-file
forks; a Go port should be at least as fast (in-process, no forks) — verify,
don't assume.
- **Scoping semantics.** Prefix-scoped import/prune (segment-boundary match) and
export glob matching (`**`/`*`/literal dots, no shell expansion) must port
exactly.
## Out of scope
The HAK build/pack/promote/publish scripts (`build-haks.sh`, `pack-haks.sh`,
`promote.sh`, `publish-release*.sh`, `hak-artifact-record.sh`) and their
`lib.sh` support. `crucible hak build` already exists; whether these thin
release wrappers also migrate is a separate question, not this charter.
## Next step
Brainstorm this charter into a full design: read `lib.sh` and the manifest
format, settle the hard parts above, then write the implementation-ready spec
and plan. Do not implement from this document.
+67
View File
@@ -0,0 +1,67 @@
package assets
import (
"flag"
"fmt"
"io"
"git.westgate.pw/ShadowsOverWestgate/sow-tools/internal/assets/mdl"
)
// mdlExt is the file set shared by the mdl commands.
var mdlExt = map[string]bool{".mdl": true}
// runCheckMDL reports uncompiled ASCII .mdl files and model-name mismatches.
// Read-only. Exit exitFail if any problem is found.
func runCheckMDL(args []string, stdout, stderr io.Writer) int {
fs := flag.NewFlagSet("check-mdl", flag.ContinueOnError)
fs.SetOutput(stderr)
nonRecursive := fs.Bool("non-recursive", false, "do not descend into subdirectories")
if err := fs.Parse(args); err != nil {
return exitUsage
}
paths := fs.Args()
if len(paths) == 0 {
fmt.Fprintln(stderr, "assets check-mdl: usage: check-mdl <path>...")
return exitUsage
}
files, err := walk(paths, mdlExt, !*nonRecursive)
if err != nil {
fmt.Fprintln(stderr, "assets check-mdl:", err)
return exitUsage
}
fail := false
for _, f := range files {
ascii, err := mdl.IsASCII(f)
if err != nil {
fmt.Fprintln(stderr, "assets check-mdl:", err)
return exitTool
}
if ascii {
fmt.Fprintf(stderr, "uncompiled ascii mdl: %s\n", f)
fail = true
}
mismatches, err := mdl.CheckNames(f)
if err != nil {
fmt.Fprintln(stderr, "assets check-mdl:", err)
return exitTool
}
expected := mdl.ExpectedName(f)
for _, m := range mismatches {
if m.Line == 0 {
fmt.Fprintf(stderr, "%s: mdl model-name mismatch: root node is %s, expected %s\n", f, m.Got, expected)
} else {
fmt.Fprintf(stderr, "%s: mdl model-name mismatch: line %d: %s is %s, expected %s\n", f, m.Line, m.What, m.Got, expected)
}
fail = true
}
}
if fail {
fmt.Fprintln(stderr, "check-mdl: found broken or uncompiled .mdl file(s)")
return exitFail
}
fmt.Fprintln(stdout, "check-mdl: OK")
return exitOK
}
+32
View File
@@ -0,0 +1,32 @@
package assets
import (
"bytes"
"os"
"path/filepath"
"testing"
)
func TestCheckMDL(t *testing.T) {
dir := t.TempDir()
// Uncompiled ASCII model with a name mismatch.
bad := filepath.Join(dir, "foo.mdl")
if err := os.WriteFile(bad, []byte("newmodel wrong\nbeginmodelgeom wrong\nendmodelgeom wrong\ndonemodel wrong\n"), 0o644); err != nil {
t.Fatal(err)
}
var out, errw bytes.Buffer
if code := runCheckMDL([]string{dir}, &out, &errw); code != exitFail {
t.Fatalf("bad mdl exit = %d, want %d\n%s", code, exitFail, errw.String())
}
// A binary (compiled) model is fine.
good := t.TempDir()
if err := os.WriteFile(filepath.Join(good, "bar.mdl"), []byte("\x00\x00compiled binary blob"), 0o644); err != nil {
t.Fatal(err)
}
out.Reset()
errw.Reset()
if code := runCheckMDL([]string{good}, &out, &errw); code != exitOK {
t.Fatalf("binary mdl exit = %d, want %d\n%s", code, exitOK, errw.String())
}
}
+190
View File
@@ -0,0 +1,190 @@
package assets
import (
"flag"
"fmt"
"io"
"os"
"path/filepath"
"strings"
"git.westgate.pw/ShadowsOverWestgate/sow-tools/internal/assets/mdl"
)
// runCompile compiles ASCII .mdl models to binary in place, one at a time (the
// engine's development/ and modelcompiler/ folders are flat, single-slot).
func runCompile(args []string, stdout, stderr io.Writer, getenv func(string) string) int {
fs := flag.NewFlagSet("compile", flag.ContinueOnError)
fs.SetOutput(stderr)
nwn := fs.String("nwn", "", "path to the NWN install root or nwmain-linux binary")
nonRecursive := fs.Bool("non-recursive", false, "do not descend into subdirectories")
if err := fs.Parse(args); err != nil {
return exitUsage
}
dirs := fs.Args()
if len(dirs) == 0 {
fmt.Fprintln(stderr, "assets compile: usage: compile [--nwn INSTALL] <dir>...")
return exitUsage
}
home := getenv("HOME")
userData := filepath.Join(home, ".local", "share", "Neverwinter Nights")
dev := filepath.Join(userData, "development")
mc := filepath.Join(userData, "modelcompiler")
nwmain := findNWMain(*nwn, home)
if nwmain == "" {
fmt.Fprintln(stderr, "assets compile: nwmain-linux not found — pass --nwn <install> "+
"(Steam/GOG/Beamdog install root or the nwmain-linux binary)")
return exitTool
}
binDir := filepath.Dir(nwmain)
// Headless wrap: no DISPLAY + xvfb-run present -> run under a virtual X.
var wrap []string
if getenv("DISPLAY") == "" {
if xvfb := look("xvfb-run"); xvfb != "" {
wrap = []string{xvfb, "-a", "--server-args=-screen 0 1024x768x24"}
}
}
files, err := walk(dirs, mdlExt, !*nonRecursive)
if err != nil {
fmt.Fprintln(stderr, "assets compile:", err)
return exitUsage
}
failed := false
for _, src := range files {
if err := compileOne(src, binDir, nwmain, dev, mc, wrap, userData, stdout, stderr); err != nil {
fmt.Fprintf(stderr, "assets compile: %s: %v\n", src, err)
failed = true
}
}
if failed {
return exitFail
}
return exitOK
}
// findNWMain resolves the nwmain-linux binary from --nwn or standard roots.
func findNWMain(override, home string) string {
if override != "" {
if fi, err := os.Stat(override); err == nil && !fi.IsDir() {
return override
}
cand := filepath.Join(override, "bin", "linux-x86", "nwmain-linux")
if fi, err := os.Stat(cand); err == nil && !fi.IsDir() {
return cand
}
return ""
}
return look(
filepath.Join(home, ".local/share/Steam/steamapps/common/Neverwinter Nights/bin/linux-x86/nwmain-linux"),
filepath.Join(home, "GOG Games/Neverwinter Nights Enhanced Edition/game/bin/linux-x86/nwmain-linux"),
filepath.Join(home, ".steam/steam/steamapps/common/Neverwinter Nights/bin/linux-x86/nwmain-linux"),
)
}
// compileOne compiles a single model. Binary models are reported and skipped
// (not an error). A name-mismatched model is aborted with guidance.
func compileOne(src, binDir, nwmain, dev, mc string, wrap []string, userData string, stdout, stderr io.Writer) error {
ascii, err := mdl.IsASCII(src)
if err != nil {
return err
}
if !ascii {
fmt.Fprintf(stdout, "skip (already compiled): %s\n", src)
return nil
}
mismatches, err := mdl.CheckNames(src)
if err != nil {
return err
}
if len(mismatches) > 0 {
return fmt.Errorf("model names differ from the file stem; run `crucible assets fix-mdl` first")
}
stem := mdl.ExpectedNameStem(src)
name := strings.ToLower(filepath.Base(src))
devFile := filepath.Join(dev, name)
// Collision guard: a pre-existing slot aborts before any mutation.
if _, err := os.Stat(devFile); err == nil {
return fmt.Errorf("development slot already occupied: %s", devFile)
}
data, err := os.ReadFile(src)
if err != nil {
return err
}
if err := os.WriteFile(devFile, data, 0o644); err != nil {
return err
}
defer os.Remove(devFile)
// Run the engine with cwd = the binary dir.
cmd := append(append([]string{}, wrap...), nwmain, "compilemodel", stem)
if out, err := runner(binDir, nil, cmd[0], cmd[1:]...); err != nil {
printEngineLog(userData, stderr)
return fmt.Errorf("engine: %v: %s", err, strings.TrimSpace(string(out)))
}
// Collect the compiled artifact from modelcompiler/ (match by stem, any case).
compiled, err := findCompiled(mc, stem)
if err != nil {
printEngineLog(userData, stderr)
return err
}
defer os.Remove(compiled)
out, err := os.ReadFile(compiled)
if err != nil {
return err
}
// Move it back over the source path, lowercased.
dst := filepath.Join(filepath.Dir(src), name)
if err := os.WriteFile(dst, out, 0o644); err != nil {
return err
}
if dst != src {
_ = os.Remove(src)
}
fmt.Fprintf(stdout, "compiled: %s\n", dst)
return nil
}
// findCompiled returns the modelcompiler/ artifact whose stem matches (case-
// insensitively).
func findCompiled(mc, stem string) (string, error) {
entries, err := os.ReadDir(mc)
if err != nil {
return "", err
}
want := strings.ToLower(stem) + ".mdl"
for _, e := range entries {
if e.IsDir() {
continue
}
if strings.ToLower(e.Name()) == want {
return filepath.Join(mc, e.Name()), nil
}
}
return "", fmt.Errorf("engine produced no compiled model for %q", stem)
}
// printEngineLog appends a short tail of the engine log as advisory diagnostics.
func printEngineLog(userData string, stderr io.Writer) {
log := filepath.Join(userData, "logs", "nwengineLog.txt")
data, err := os.ReadFile(log)
if err != nil {
return
}
lines := strings.Split(strings.TrimRight(string(data), "\n"), "\n")
if len(lines) > 8 {
lines = lines[len(lines)-8:]
}
fmt.Fprintf(stderr, " engine log tail (%s):\n", log)
for _, l := range lines {
fmt.Fprintf(stderr, " %s\n", l)
}
}
+110
View File
@@ -0,0 +1,110 @@
package assets
import (
"bytes"
"os"
"path/filepath"
"testing"
)
func TestCompileDrivesEngineAndReplacesInPlace(t *testing.T) {
home := t.TempDir()
userData := filepath.Join(home, ".local", "share", "Neverwinter Nights")
dev := filepath.Join(userData, "development")
mc := filepath.Join(userData, "modelcompiler")
for _, d := range []string{dev, mc} {
if err := os.MkdirAll(d, 0o755); err != nil {
t.Fatal(err)
}
}
// Fake nwmain binary for discovery via --nwn.
binDir := t.TempDir()
nwmain := filepath.Join(binDir, "nwmain-linux")
if err := os.WriteFile(nwmain, []byte("#!/bin/sh\n"), 0o755); err != nil {
t.Fatal(err)
}
getenv := func(k string) string {
switch k {
case "HOME":
return home
case "DISPLAY":
return ":0" // pretend a display exists so no xvfb wrap is needed
}
return ""
}
// Stub the engine: writes a binary compiled model into modelcompiler/.
orig := runner
defer func() { runner = orig }()
runner = func(dir string, env []string, name string, args ...string) ([]byte, error) {
// args: compilemodel <stem>
stem := args[len(args)-1]
compiled := filepath.Join(mc, stem+".mdl")
return nil, os.WriteFile(compiled, []byte("\x00\x00compiled"), 0o644)
}
// Source tree with one ASCII model whose names already match its stem.
srcDir := t.TempDir()
src := filepath.Join(srcDir, "foo.mdl")
body := "newmodel foo\nbeginmodelgeom foo\n node dummy foo\n parent null\n endnode\nendmodelgeom foo\ndonemodel foo\n"
if err := os.WriteFile(src, []byte(body), 0o644); err != nil {
t.Fatal(err)
}
var stdout, stderr bytes.Buffer
code := runCompile([]string{"--nwn", nwmain, srcDir}, &stdout, &stderr, getenv)
if code != exitOK {
t.Fatalf("compile exit = %d\n%s", code, stderr.String())
}
// The source is now binary (the compiled artifact moved back over it).
data, err := os.ReadFile(src)
if err != nil {
t.Fatalf("compiled model missing: %v", err)
}
if string(data) != "\x00\x00compiled" {
t.Fatalf("source not replaced by compiled binary: %q", data)
}
}
func TestCompileAbortsOnNameMismatch(t *testing.T) {
home := t.TempDir()
if err := os.MkdirAll(filepath.Join(home, ".local", "share", "Neverwinter Nights", "development"), 0o755); err != nil {
t.Fatal(err)
}
binDir := t.TempDir()
nwmain := filepath.Join(binDir, "nwmain-linux")
if err := os.WriteFile(nwmain, []byte("#!/bin/sh\n"), 0o755); err != nil {
t.Fatal(err)
}
getenv := func(k string) string {
if k == "HOME" {
return home
}
if k == "DISPLAY" {
return ":0"
}
return ""
}
orig := runner
defer func() { runner = orig }()
engineCalled := false
runner = func(string, []string, string, ...string) ([]byte, error) {
engineCalled = true
return nil, nil
}
srcDir := t.TempDir()
// Internal name "wrong" != stem "foo": must abort before touching the engine.
if err := os.WriteFile(filepath.Join(srcDir, "foo.mdl"),
[]byte("newmodel wrong\ndonemodel wrong\n"), 0o644); err != nil {
t.Fatal(err)
}
var stdout, stderr bytes.Buffer
if code := runCompile([]string{"--nwn", nwmain, srcDir}, &stdout, &stderr, getenv); code != exitFail {
t.Fatalf("mismatch exit = %d, want %d", code, exitFail)
}
if engineCalled {
t.Fatal("engine should not run for a name-mismatched model")
}
}
+116
View File
@@ -0,0 +1,116 @@
package assets
import (
"flag"
"fmt"
"io"
"os"
"path/filepath"
"strings"
)
var textureExts = map[string]bool{".dds": true, ".png": true, ".tga": true}
// runConvert converts textures in place, flipping vertically exactly once per
// conversion. --to defaults to dds. Files already in the target format are
// skipped.
func runConvert(args []string, stdout, stderr io.Writer) int {
fs := flag.NewFlagSet("convert", flag.ContinueOnError)
fs.SetOutput(stderr)
to := fs.String("to", "dds", "target format: dds|png|tga")
backend := fs.String("backend", "magick", "ImageMagick-compatible binary")
nonRecursive := fs.Bool("non-recursive", false, "do not descend into subdirectories")
if err := fs.Parse(args); err != nil {
return exitUsage
}
target := strings.ToLower(*to)
if target != "dds" && target != "png" && target != "tga" {
fmt.Fprintln(stderr, "assets convert: --to must be dds, png, or tga")
return exitUsage
}
dirs := fs.Args()
if len(dirs) == 0 {
fmt.Fprintln(stderr, "assets convert: usage: convert [--to dds|png|tga] <dir>...")
return exitUsage
}
magick := look(*backend)
if magick == "" {
fmt.Fprintf(stderr, "assets convert: backend %q not found — install ImageMagick (magick)\n", *backend)
return exitTool
}
files, err := walk(dirs, textureExts, !*nonRecursive)
if err != nil {
fmt.Fprintln(stderr, "assets convert:", err)
return exitUsage
}
targetExt := "." + target
failed := false
for _, src := range files {
srcExt := strings.ToLower(filepath.Ext(src))
if srcExt == targetExt {
continue // already in the target format
}
dst := strings.TrimSuffix(src, filepath.Ext(src)) + targetExt
var convErr error
if target == "dds" {
convErr = pngToDDS(magick, src, dst)
} else {
convErr = ddsToPNG(magick, src, dst) // works for any decodable source
}
if convErr != nil {
fmt.Fprintf(stderr, "assets convert: %s: %v\n", src, convErr)
failed = true
continue
}
if dst != src {
if err := os.Remove(src); err != nil {
fmt.Fprintf(stderr, "assets convert: %s: %v\n", src, err)
failed = true
}
}
}
if failed {
return exitFail
}
return exitOK
}
// ddsToPNG decodes src to dst (PNG/TGA by dst's extension), flipping vertically
// once. The single flip is the defining NWN behavior.
func ddsToPNG(magickBin, src, dst string) error {
if out, err := runner("", nil, magickBin, src, "-flip", dst); err != nil {
return fmt.Errorf("magick: %v: %s", err, strings.TrimSpace(string(out)))
}
return nil
}
// pngToDDS encodes src to a DDS at dst, flipping vertically once and choosing
// DXT1 for opaque images, DXT5 for images with alpha.
func pngToDDS(magickBin, src, dst string) error {
compression := "dxt1"
if hasAlpha(magickBin, src) {
compression = "dxt5"
}
// ponytail: mipmaps rely on magick's default full chain (NWN wants a
// pyramid, which magick writes by default). Add `-define dds:mipmaps=N`
// here if a specific count is ever required.
out, err := runner("", nil, magickBin, src, "-flip",
"-define", "dds:compression="+compression, dst)
if err != nil {
return fmt.Errorf("magick: %v: %s", err, strings.TrimSpace(string(out)))
}
return nil
}
// hasAlpha reports whether src has any non-opaque pixel, via `magick identify`.
// On any probe error it assumes alpha (DXT5), the safe/lossless-alpha default.
func hasAlpha(magickBin, src string) bool {
out, err := runner("", nil, magickBin, "identify", "-format", "%[opaque]", src)
if err != nil {
return true
}
return !strings.EqualFold(strings.TrimSpace(string(out)), "true")
}
+108
View File
@@ -0,0 +1,108 @@
package assets
import (
"bytes"
"image"
"image/color"
"image/png"
"os"
"os/exec"
"path/filepath"
"testing"
)
// writeTestPNG writes a 16x16 image: top half red, bottom half blue. The size
// and the half-way split keep every 4x4 DXT block a single flat color, so the
// DXT1 round trip stays lossless and only the flip is under test. (A 4x4 image
// is one mixed DXT block that magick's encoder collapses to a single color.)
func writeTestPNG(t *testing.T, path string) {
t.Helper()
const n = 16
img := image.NewRGBA(image.Rect(0, 0, n, n))
for y := 0; y < n; y++ {
c := color.RGBA{255, 0, 0, 255} // red
if y >= n/2 {
c = color.RGBA{0, 0, 255, 255} // blue
}
for x := 0; x < n; x++ {
img.Set(x, y, c)
}
}
f, err := os.Create(path)
if err != nil {
t.Fatal(err)
}
defer f.Close()
if err := png.Encode(f, img); err != nil {
t.Fatal(err)
}
}
func topRowIsBlue(t *testing.T, pngPath string) bool {
t.Helper()
f, err := os.Open(pngPath)
if err != nil {
t.Fatal(err)
}
defer f.Close()
img, err := png.Decode(f)
if err != nil {
t.Fatal(err)
}
r, _, b, _ := img.At(0, 0).RGBA()
return b > r // blue dominates the top-left after a flip
}
func TestConvertFlipsOnceAndRoundTrips(t *testing.T) {
magick := look("magick")
if magick == "" {
t.Skip("magick not on PATH")
}
dir := t.TempDir()
src := filepath.Join(dir, "tex.png")
writeTestPNG(t, src)
// Convert PNG -> DDS (in place: tex.png becomes tex.dds, original removed).
var out, errw bytes.Buffer
if code := runConvert([]string{"--to", "dds", dir}, &out, &errw); code != exitOK {
t.Fatalf("to-dds exit = %d\n%s", code, errw.String())
}
dds := filepath.Join(dir, "tex.dds")
if _, err := os.Stat(dds); err != nil {
t.Fatalf("dds not produced: %v", err)
}
if _, err := os.Stat(src); !os.IsNotExist(err) {
t.Fatal("source png was not replaced")
}
// Decode the DDS RAW (no extra flip) and confirm a single flip happened:
// the source top row was red, so the DDS top row must now be blue.
raw := filepath.Join(dir, "raw.png")
if err := exec.Command(magick, dds, raw).Run(); err != nil {
t.Fatalf("raw decode: %v", err)
}
if !topRowIsBlue(t, raw) {
t.Fatal("expected the DDS to be vertically flipped vs the source")
}
// Convert DDS -> PNG (another flip). Result should match the original.
out.Reset()
errw.Reset()
if code := runConvert([]string{"--to", "png", dir}, &out, &errw); code != exitOK {
t.Fatalf("to-png exit = %d\n%s", code, errw.String())
}
restored := filepath.Join(dir, "tex.png")
if topRowIsBlue(t, restored) {
t.Fatal("round trip did not restore the original orientation")
}
}
func TestConvertMissingBackendFailsClosed(t *testing.T) {
dir := t.TempDir()
writeTestPNG(t, filepath.Join(dir, "tex.png"))
var out, errw bytes.Buffer
code := runConvert([]string{"--backend", "/nonexistent/magick", dir}, &out, &errw)
if code != exitTool {
t.Fatalf("missing backend exit = %d, want %d", code, exitTool)
}
}
+151
View File
@@ -0,0 +1,151 @@
package assets
import (
"flag"
"fmt"
"io"
"os"
"path/filepath"
"sort"
"strings"
)
// allFiles returns every regular file under root (recursive).
func allFiles(root string) ([]string, error) {
var out []string
err := filepath.WalkDir(root, func(p string, d os.DirEntry, err error) error {
if err != nil {
return err
}
if !d.IsDir() {
out = append(out, p)
}
return nil
})
sort.Strings(out)
return out, err
}
// runCheckDupes reports files whose lower-cased basename collides across the
// given dirs. Read-only. Exit exitFail if any collision is found.
func runCheckDupes(args []string, stdout, stderr io.Writer) int {
fs := flag.NewFlagSet("check-dupes", flag.ContinueOnError)
fs.SetOutput(stderr)
if err := fs.Parse(args); err != nil {
return exitUsage
}
dirs := fs.Args()
if len(dirs) == 0 {
fmt.Fprintln(stderr, "assets check-dupes: usage: check-dupes <dir>...")
return exitUsage
}
first := map[string]string{}
fail := false
for _, dir := range dirs {
if fi, err := os.Stat(dir); err != nil || !fi.IsDir() {
fmt.Fprintf(stderr, "assets check-dupes: no such dir: %s\n", dir)
return exitUsage
}
files, err := allFiles(dir)
if err != nil {
fmt.Fprintln(stderr, "assets check-dupes:", err)
return exitTool
}
for _, f := range files {
key := strings.ToLower(filepath.Base(f))
if prev, ok := first[key]; ok {
fmt.Fprintf(stderr, "runtime-name collision: %s collides with %s\n", f, prev)
fail = true
} else {
first[key] = f
}
}
}
if fail {
fmt.Fprintln(stderr, "check-dupes: found runtime-name collision(s)")
return exitFail
}
fmt.Fprintln(stdout, "check-dupes: OK")
return exitOK
}
// runCleanDupes deletes files from <clean> whose lower-cased basename collides
// with any file in <primary>. Mutates only <clean>. Refuses to run if the two
// trees overlap.
func runCleanDupes(args []string, stdout, stderr io.Writer) int {
fs := flag.NewFlagSet("clean-dupes", flag.ContinueOnError)
fs.SetOutput(stderr)
dryRun := fs.Bool("dry-run", false, "report deletions without removing")
if err := fs.Parse(args); err != nil {
return exitUsage
}
pos := fs.Args()
if len(pos) != 2 {
fmt.Fprintln(stderr, "assets clean-dupes: usage: clean-dupes [--dry-run] <primary> <clean>")
return exitUsage
}
primary, clean := pos[0], pos[1]
for _, d := range []string{primary, clean} {
if fi, err := os.Stat(d); err != nil || !fi.IsDir() {
fmt.Fprintf(stderr, "assets clean-dupes: no such dir: %s\n", d)
return exitUsage
}
}
primaryReal, err1 := filepath.EvalSymlinks(primary)
cleanReal, err2 := filepath.EvalSymlinks(clean)
if err1 != nil || err2 != nil {
fmt.Fprintln(stderr, "assets clean-dupes: cannot resolve dirs")
return exitTool
}
if overlaps(primaryReal, cleanReal) {
fmt.Fprintln(stderr, "assets clean-dupes: primary and clean dirs must not overlap")
return exitUsage
}
primaryFiles, err := allFiles(primary)
if err != nil {
fmt.Fprintln(stderr, "assets clean-dupes:", err)
return exitTool
}
names := map[string]bool{}
for _, f := range primaryFiles {
names[strings.ToLower(filepath.Base(f))] = true
}
cleanFiles, err := allFiles(clean)
if err != nil {
fmt.Fprintln(stderr, "assets clean-dupes:", err)
return exitTool
}
removed := 0
for _, f := range cleanFiles {
if !names[strings.ToLower(filepath.Base(f))] {
continue
}
if *dryRun {
fmt.Fprintf(stderr, "would delete clean-tree collision: %s\n", f)
} else {
if err := os.Remove(f); err != nil {
fmt.Fprintln(stderr, "assets clean-dupes:", err)
return exitTool
}
fmt.Fprintf(stderr, "deleted clean-tree collision: %s\n", f)
}
removed++
}
verb := "removed"
if *dryRun {
verb = "would remove"
}
fmt.Fprintf(stdout, "clean-dupes: %s %d file(s)\n", verb, removed)
return exitOK
}
// overlaps reports whether either directory contains the other (or they are
// equal), using cleaned absolute-ish paths with a trailing separator.
func overlaps(a, b string) bool {
as := a + string(filepath.Separator)
bs := b + string(filepath.Separator)
return strings.HasPrefix(as, bs) || strings.HasPrefix(bs, as)
}
+80
View File
@@ -0,0 +1,80 @@
package assets
import (
"bytes"
"os"
"path/filepath"
"testing"
)
func touch(t *testing.T, path string) {
t.Helper()
if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(path, []byte("x"), 0o644); err != nil {
t.Fatal(err)
}
}
func TestCheckDupes(t *testing.T) {
root := t.TempDir()
touch(t, filepath.Join(root, "tex", "Foo.tga"))
touch(t, filepath.Join(root, "plc", "foo.tga")) // basename collision (case-insensitive)
var out, errw bytes.Buffer
if code := runCheckDupes([]string{root}, &out, &errw); code != exitFail {
t.Fatalf("collision exit = %d, want %d\n%s", code, exitFail, errw.String())
}
clean := t.TempDir()
touch(t, filepath.Join(clean, "a.tga"))
touch(t, filepath.Join(clean, "sub", "b.tga"))
out.Reset()
errw.Reset()
if code := runCheckDupes([]string{clean}, &out, &errw); code != exitOK {
t.Fatalf("no-collision exit = %d, want %d\n%s", code, exitOK, errw.String())
}
}
func TestCleanDupes(t *testing.T) {
primary := t.TempDir()
clean := t.TempDir()
touch(t, filepath.Join(primary, "keep.tga"))
collide := filepath.Join(clean, "sub", "Keep.tga")
survive := filepath.Join(clean, "unique.tga")
touch(t, collide)
touch(t, survive)
// dry-run removes nothing.
var out, errw bytes.Buffer
if code := runCleanDupes([]string{"--dry-run", primary, clean}, &out, &errw); code != exitOK {
t.Fatalf("dry-run exit = %d\n%s", code, errw.String())
}
if _, err := os.Stat(collide); err != nil {
t.Fatal("dry-run deleted a file")
}
// real run deletes the collision, keeps the unique file.
out.Reset()
errw.Reset()
if code := runCleanDupes([]string{primary, clean}, &out, &errw); code != exitOK {
t.Fatalf("clean exit = %d\n%s", code, errw.String())
}
if _, err := os.Stat(collide); !os.IsNotExist(err) {
t.Fatal("collision file was not deleted")
}
if _, err := os.Stat(survive); err != nil {
t.Fatal("unique file was wrongly deleted")
}
}
func TestCleanDupesRejectsOverlap(t *testing.T) {
root := t.TempDir()
sub := filepath.Join(root, "child")
touch(t, filepath.Join(sub, "x.tga"))
var out, errw bytes.Buffer
if code := runCleanDupes([]string{root, sub}, &out, &errw); code != exitUsage {
t.Fatalf("overlap exit = %d, want %d", code, exitUsage)
}
}
+103
View File
@@ -0,0 +1,103 @@
package assets
import (
"flag"
"fmt"
"io"
"os"
"path/filepath"
"strings"
"git.westgate.pw/ShadowsOverWestgate/sow-tools/internal/assets/mdl"
)
// runFixMDL lowercases .mdl filenames and rewrites ASCII model identity to
// match each file's stem. Binary models are skipped (the engine owns those).
func runFixMDL(args []string, stdout, stderr io.Writer) int {
fs := flag.NewFlagSet("fix-mdl", flag.ContinueOnError)
fs.SetOutput(stderr)
dryRun := fs.Bool("dry-run", false, "report changes without touching disk")
nonRecursive := fs.Bool("non-recursive", false, "do not descend into subdirectories")
if err := fs.Parse(args); err != nil {
return exitUsage
}
paths := fs.Args()
if len(paths) == 0 {
fmt.Fprintln(stderr, "assets fix-mdl: usage: fix-mdl [--dry-run] <path>...")
return exitUsage
}
files, err := walk(paths, mdlExt, !*nonRecursive)
if err != nil {
fmt.Fprintln(stderr, "assets fix-mdl:", err)
return exitUsage
}
changed := 0
failed := false
for _, f := range files {
// 1. Lowercase the basename if needed.
//
// ponytail: 35k files are each read once here (CheckNames + the rewrite
// on candidates). The shell reference batched an awk pass to avoid
// per-file subprocess spawns; in Go a plain read is cheap, so the batch
// is not worth porting. If profiling ever shows this hot, parallelize
// the loop.
lower := f
if base := filepath.Base(f); base != strings.ToLower(base) {
lower = filepath.Join(filepath.Dir(f), strings.ToLower(base))
if _, err := os.Stat(lower); err == nil {
fmt.Fprintf(stderr, "assets fix-mdl: lowercase collision: %s -> %s\n", f, lower)
failed = true
continue
}
if *dryRun {
fmt.Fprintf(stdout, "would lowercase: %s -> %s\n", f, lower)
} else {
if err := os.Rename(f, lower); err != nil {
fmt.Fprintf(stderr, "assets fix-mdl: failed to lowercase %s: %v\n", f, err)
failed = true
continue
}
fmt.Fprintf(stdout, "lowercased: %s -> %s\n", f, lower)
}
changed++
}
// 2. Rewrite model identity if the (lowercased) file has a mismatch.
// In dry-run the on-disk file is still the original path.
readPath := lower
if *dryRun && lower != f {
readPath = f
}
data, err := os.ReadFile(readPath)
if err != nil {
fmt.Fprintln(stderr, "assets fix-mdl:", err)
failed = true
continue
}
out, wasChanged := mdl.FixNames(data, mdl.ExpectedName(lower))
if !wasChanged {
continue
}
if *dryRun {
fmt.Fprintf(stdout, "would fix: %s\n", lower)
} else {
if err := os.WriteFile(lower, out, 0o644); err != nil {
fmt.Fprintln(stderr, "assets fix-mdl:", err)
failed = true
continue
}
fmt.Fprintf(stdout, "fixed: %s\n", lower)
}
changed++
}
if changed == 0 {
fmt.Fprintln(stdout, "no broken mdl model names found")
}
if failed {
return exitFail
}
return exitOK
}
+48
View File
@@ -0,0 +1,48 @@
package assets
import (
"bytes"
"os"
"path/filepath"
"testing"
"git.westgate.pw/ShadowsOverWestgate/sow-tools/internal/assets/mdl"
)
func TestFixMDL(t *testing.T) {
dir := t.TempDir()
// Uppercase name + internal mismatch.
upper := filepath.Join(dir, "Foo.MDL")
body := "newmodel wrong\nbeginmodelgeom wrong\nendmodelgeom wrong\ndonemodel wrong\n"
if err := os.WriteFile(upper, []byte(body), 0o644); err != nil {
t.Fatal(err)
}
// dry-run: nothing changes on disk.
var out, errw bytes.Buffer
if code := runFixMDL([]string{"--dry-run", dir}, &out, &errw); code != exitOK {
t.Fatalf("dry-run exit = %d\n%s", code, errw.String())
}
if _, err := os.Stat(upper); err != nil {
t.Fatal("dry-run renamed a file")
}
// real run: file is lowercased and its identity rewritten to match.
out.Reset()
errw.Reset()
if code := runFixMDL([]string{dir}, &out, &errw); code != exitOK {
t.Fatalf("fix exit = %d\n%s", code, errw.String())
}
lowered := filepath.Join(dir, "foo.mdl")
if _, err := os.Stat(lowered); err != nil {
t.Fatalf("file was not lowercased: %v", err)
}
mismatches, err := mdl.CheckNames(lowered)
if err != nil {
t.Fatal(err)
}
if len(mismatches) != 0 {
data, _ := os.ReadFile(lowered)
t.Fatalf("model still has mismatches after fix: %+v\n%s", mismatches, data)
}
}
+84
View File
@@ -0,0 +1,84 @@
// Package assets is the crucible `assets` builder: NWN:EE model/texture tools
// that operate in place on a target directory. Mirrors internal/depot's
// Run(args, stdout, stderr, getenv) shape and bypasses the legacy internal/app
// surface entirely.
package assets
import (
"os"
"os/exec"
"path/filepath"
"sort"
"strings"
)
// runner is the single external-command indirection so tests can observe and
// stub every engine / ImageMagick / upscaler invocation. dir is the working
// directory ("" = inherit); env replaces the child environment when non-nil.
var runner = func(dir string, env []string, name string, args ...string) ([]byte, error) {
cmd := exec.Command(name, args...)
cmd.Dir = dir
if env != nil {
cmd.Env = env
}
return cmd.CombinedOutput()
}
// walk collects files under each root whose lower-cased extension is in exts. A
// root that is itself a matching file is included. recursive controls descent
// into subdirectories. Results are sorted and deduplicated.
func walk(roots []string, exts map[string]bool, recursive bool) ([]string, error) {
seen := map[string]bool{}
var out []string
add := func(p string) {
if exts[strings.ToLower(filepath.Ext(p))] && !seen[p] {
seen[p] = true
out = append(out, p)
}
}
for _, root := range roots {
fi, err := os.Stat(root)
if err != nil {
return nil, err
}
if !fi.IsDir() {
add(root)
continue
}
err = filepath.WalkDir(root, func(p string, d os.DirEntry, err error) error {
if err != nil {
return err
}
if d.IsDir() {
if !recursive && p != root {
return filepath.SkipDir
}
return nil
}
add(p)
return nil
})
if err != nil {
return nil, err
}
}
sort.Strings(out)
return out, nil
}
// look returns the first bare name found on PATH, or the first candidate that
// is an existing path. Returns "" if none resolve.
func look(candidates ...string) string {
for _, c := range candidates {
if strings.ContainsRune(c, filepath.Separator) {
if fi, err := os.Stat(c); err == nil && !fi.IsDir() {
return c
}
continue
}
if p, err := exec.LookPath(c); err == nil {
return p
}
}
return ""
}
+63
View File
@@ -0,0 +1,63 @@
package assets
import (
"os"
"path/filepath"
"testing"
)
func TestWalk(t *testing.T) {
root := t.TempDir()
must := func(rel string) {
p := filepath.Join(root, rel)
if err := os.MkdirAll(filepath.Dir(p), 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(p, []byte("x"), 0o644); err != nil {
t.Fatal(err)
}
}
must("a.MDL")
must("sub/b.mdl")
must("sub/c.txt")
exts := map[string]bool{".mdl": true}
got, err := walk([]string{root}, exts, true)
if err != nil {
t.Fatal(err)
}
if len(got) != 2 {
t.Fatalf("recursive walk = %v, want 2 mdl files", got)
}
got, _ = walk([]string{root}, exts, false)
if len(got) != 1 || filepath.Base(got[0]) != "a.MDL" {
t.Fatalf("non-recursive walk = %v, want only a.MDL", got)
}
// A file argument that matches is included directly.
got, _ = walk([]string{filepath.Join(root, "sub", "b.mdl")}, exts, true)
if len(got) != 1 {
t.Fatalf("file arg walk = %v, want the file itself", got)
}
}
func TestLook(t *testing.T) {
dir := t.TempDir()
stub := filepath.Join(dir, "mytool")
if err := os.WriteFile(stub, []byte("#!/bin/sh\n"), 0o755); err != nil {
t.Fatal(err)
}
t.Setenv("PATH", dir)
if got := look("nope-not-here", "mytool"); got == "" {
t.Fatal("look should find mytool on PATH")
}
// Explicit existing path candidate.
if got := look(stub); got != stub {
t.Fatalf("look(%q) = %q, want the path itself", stub, got)
}
if got := look("definitely-absent-binary-xyz"); got != "" {
t.Fatalf("look = %q, want empty", got)
}
}
+218
View File
@@ -0,0 +1,218 @@
// Package mdl ports the ASCII MDL model-name checks from sow-assets-manifest's
// mdl-name-lib.sh + mdl-scan.awk to pure Go. Binary (compiled) MDLs are opaque
// here; the NWN engine is the validator for those. Detection mirrors
// mdl-scan.awk exactly so behavior does not drift while that awk still runs in
// sow-assets-manifest.
package mdl
import (
"bytes"
"os"
"path/filepath"
"strings"
)
// Mismatch is one model-name problem. Line is 1-based; 0 means the geometry
// base-node mismatch, which the source has no single line for.
type Mismatch struct {
Line int
What string
Got string
}
// ExpectedName is the file stem with a trailing ".mdl" (any case) removed.
func ExpectedName(path string) string {
base := filepath.Base(path)
if len(base) >= 4 && strings.EqualFold(base[len(base)-4:], ".mdl") {
return base[:len(base)-4]
}
return base
}
// ExpectedNameStem is an alias of ExpectedName kept for call-site clarity where
// the value is used as the engine's model stem argument.
func ExpectedNameStem(path string) string { return ExpectedName(path) }
// IsASCII reports whether path is an uncompiled ASCII model: no NUL in the
// first 256 bytes, and a leading keyword of '#' / newmodel / node /
// setsupermodel. Mirrors mdl_is_ascii.
func IsASCII(path string) (bool, error) {
data, err := os.ReadFile(path)
if err != nil {
return false, err
}
return isASCII(data), nil
}
func isASCII(data []byte) bool {
head := data
if len(head) > 256 {
head = head[:256]
}
if bytes.IndexByte(head, 0) >= 0 {
return false
}
line := head
if i := bytes.IndexByte(line, '\n'); i >= 0 {
line = line[:i]
}
trimmed := strings.TrimLeft(string(line), " \t\r")
if strings.HasPrefix(trimmed, "#") {
return true
}
fields := strings.Fields(trimmed)
if len(fields) == 0 {
return false
}
tok := strings.ToLower(fields[0])
return strings.HasPrefix(tok, "newmodel") ||
strings.HasPrefix(tok, "node") ||
strings.HasPrefix(tok, "setsupermodel")
}
// lineFields splits a line into whitespace-delimited tokens with the trailing
// CR removed, matching awk's default field split + \r strip.
func lineFields(line string) []string {
return strings.Fields(strings.TrimRight(line, "\r"))
}
// field returns the 1-based nth field or "" if absent.
func field(f []string, n int) string {
if n >= 1 && n <= len(f) {
return f[n-1]
}
return ""
}
// CheckNames returns every model-name mismatch in an ASCII model. Returns nil
// for a binary model or one with no newmodel line (not a real model).
func CheckNames(path string) ([]Mismatch, error) {
data, err := os.ReadFile(path)
if err != nil {
return nil, err
}
if !isASCII(data) {
return nil, nil
}
return checkNames(data, ExpectedName(path)), nil
}
func checkNames(data []byte, expected string) []Mismatch {
expLower := strings.ToLower(expected)
var out []Mismatch
seenModel := false
inGeom := false
curNode := ""
base := ""
baseFound := false
// header records one header-token mismatch (field n, 1-based) if the token
// is present and differs case-insensitively from the stem.
lines := strings.Split(string(data), "\n")
for i, raw := range lines {
f := lineFields(raw)
if len(f) == 0 {
continue
}
key := strings.ToLower(f[0])
mis := func(what string, n int) {
got := field(f, n)
if got != "" && !strings.EqualFold(got, expected) {
out = append(out, Mismatch{Line: i + 1, What: what, Got: got})
}
}
switch key {
case "newmodel":
seenModel = true
mis("newmodel", 2)
case "setsupermodel":
mis("setsupermodel model", 2)
case "beginmodelgeom":
inGeom = true
mis("beginmodelgeom", 2)
case "endmodelgeom":
inGeom = false
mis("endmodelgeom", 2)
case "donemodel":
mis("donemodel", 2)
case "newanim":
mis("newanim model", 3)
case "doneanim":
mis("doneanim model", 3)
case "node":
if inGeom {
curNode = field(f, 3)
}
case "parent":
if inGeom && !baseFound && strings.EqualFold(field(f, 2), "null") {
base = curNode
baseFound = true
}
}
}
if !seenModel {
return nil
}
if baseFound && base != "" && !strings.EqualFold(base, expLower) {
out = append(out, Mismatch{Line: 0, What: "root node", Got: base})
}
return out
}
// FixNames rewrites only the model identity to expected: the header tokens plus
// the geometry base node's declaration / any parent / animroot pointing at the
// old base name. Animation bone names and supermodel animroots are left alone
// so inheritance keeps working. Mirrors mdl_fix_model_name in intent.
//
// ponytail: a rewritten line is rebuilt joining fields with single spaces and
// drops a trailing CR, exactly like the awk it replaces; unchanged lines are
// byte-identical. The engine ignores identity-line whitespace, and FixNames'
// contract is only "produces a model that passes CheckNames", not byte parity.
func FixNames(in []byte, expected string) (out []byte, changed bool) {
if !isASCII(in) {
return in, false
}
// oldbase: the base-node name to rename, only if it differs from expected.
oldbase := ""
for _, m := range checkNames(in, expected) {
if m.What == "root node" {
oldbase = m.Got
}
}
lines := strings.Split(string(in), "\n")
for idx, raw := range lines {
f := lineFields(raw)
if len(f) == 0 {
continue
}
key := strings.ToLower(f[0])
set := func(n int) bool {
if field(f, n) == "" {
return false
}
f[n-1] = expected
return true
}
modified := false
switch key {
case "newmodel", "setsupermodel", "beginmodelgeom", "endmodelgeom", "donemodel":
modified = set(2)
case "newanim", "doneanim":
modified = set(3)
case "node":
if oldbase != "" && strings.EqualFold(field(f, 3), oldbase) {
modified = set(3)
}
case "parent", "animroot":
if oldbase != "" && strings.EqualFold(field(f, 2), oldbase) {
modified = set(2)
}
}
if modified {
lines[idx] = strings.Join(f, " ")
}
}
out = []byte(strings.Join(lines, "\n"))
return out, !bytes.Equal(out, in)
}
+114
View File
@@ -0,0 +1,114 @@
package mdl
import (
"os"
"path/filepath"
"testing"
)
func writeMDL(t *testing.T, name, body string) string {
t.Helper()
p := filepath.Join(t.TempDir(), name)
if err := os.WriteFile(p, []byte(body), 0o644); err != nil {
t.Fatal(err)
}
return p
}
func TestIsASCII(t *testing.T) {
ascii := writeMDL(t, "foo.mdl", "newmodel foo\nbeginmodelgeom foo\n")
if ok, err := IsASCII(ascii); err != nil || !ok {
t.Fatalf("ascii: ok=%v err=%v, want true nil", ok, err)
}
bin := writeMDL(t, "bar.mdl", "\x00\x01binary\x00garbage")
if ok, err := IsASCII(bin); err != nil || ok {
t.Fatalf("binary: ok=%v err=%v, want false nil", ok, err)
}
hash := writeMDL(t, "baz.mdl", "# a comment\nnewmodel baz\n")
if ok, _ := IsASCII(hash); !ok {
t.Fatal("leading # should be ascii")
}
}
func TestExpectedName(t *testing.T) {
for in, want := range map[string]string{
"a/b/Foo.MDL": "Foo",
"waxbt_b_091.mdl": "waxbt_b_091",
"x.mdl": "x",
} {
if got := ExpectedName(in); got != want {
t.Errorf("ExpectedName(%q)=%q want %q", in, got, want)
}
}
}
func TestCheckNames(t *testing.T) {
// Header token mismatch + base-node mismatch in one file.
body := "newmodel wrong\n" +
"setsupermodel wrong a_base\n" +
"beginmodelgeom foo\n" +
" node dummy Wmgst_m_081\n" +
" parent null\n" +
" endnode\n" +
"endmodelgeom foo\n" +
"donemodel foo\n"
p := writeMDL(t, "foo.mdl", body)
got, err := CheckNames(p)
if err != nil {
t.Fatal(err)
}
// Expected stem "foo": newmodel(wrong), setsupermodel(wrong), base node(Wmgst_m_081).
if len(got) != 3 {
t.Fatalf("got %d mismatches, want 3: %+v", len(got), got)
}
if got[0].What != "newmodel" || got[0].Got != "wrong" || got[0].Line != 1 {
t.Errorf("first mismatch = %+v", got[0])
}
base := got[len(got)-1]
if base.What != "root node" || base.Got != "Wmgst_m_081" || base.Line != 0 {
t.Errorf("base mismatch = %+v", base)
}
// Clean model -> no mismatches.
clean := writeMDL(t, "bar.mdl",
"newmodel bar\nbeginmodelgeom bar\n node dummy bar\n parent null\n endnode\nendmodelgeom bar\ndonemodel bar\n")
if got, _ := CheckNames(clean); len(got) != 0 {
t.Fatalf("clean model reported mismatches: %+v", got)
}
// Binary model -> nil, no error.
bin := writeMDL(t, "b.mdl", "\x00\x00binary")
if got, _ := CheckNames(bin); got != nil {
t.Fatalf("binary reported mismatches: %+v", got)
}
}
func TestFixNamesRoundTrips(t *testing.T) {
body := "newmodel wrong\n" +
"setsupermodel wrong\n" +
"beginmodelgeom wrong\n" +
" node dummy Wmgst_m_081\n" +
" parent null\n" +
" endnode\n" +
"endmodelgeom wrong\n" +
"donemodel wrong\n"
out, changed := FixNames([]byte(body), "foo")
if !changed {
t.Fatal("expected changed=true")
}
p := filepath.Join(t.TempDir(), "foo.mdl")
if err := os.WriteFile(p, out, 0o644); err != nil {
t.Fatal(err)
}
got, err := CheckNames(p)
if err != nil {
t.Fatal(err)
}
if len(got) != 0 {
t.Fatalf("fixed model still has mismatches: %+v\n%s", got, out)
}
// Idempotent: fixing a clean model changes nothing.
if _, changed := FixNames(out, "foo"); changed {
t.Fatal("second fix reported a change")
}
}
+55
View File
@@ -0,0 +1,55 @@
package assets
import (
"fmt"
"io"
)
const (
exitOK = 0
exitFail = 1 // problems found / per-file failures
exitUsage = 64 // bad invocation / unknown subcommand / bad flags
exitTool = 70 // missing external tool / internal error
)
// Run executes an assets subcommand. args[0] is the subcommand; returns the
// process exit code.
func Run(args []string, stdout, stderr io.Writer, getenv func(string) string) int {
if len(args) == 0 {
printUsage(stderr)
return exitUsage
}
rest := args[1:]
switch args[0] {
case "check-dupes":
return runCheckDupes(rest, stdout, stderr)
case "clean-dupes":
return runCleanDupes(rest, stdout, stderr)
case "check-mdl":
return runCheckMDL(rest, stdout, stderr)
case "fix-mdl":
return runFixMDL(rest, stdout, stderr)
case "convert":
return runConvert(rest, stdout, stderr)
case "upscale":
return runUpscale(rest, stdout, stderr)
case "compile":
return runCompile(rest, stdout, stderr, getenv)
default:
fmt.Fprintf(stderr, "assets: unknown subcommand %q\n\n", args[0])
printUsage(stderr)
return exitUsage
}
}
func printUsage(w io.Writer) {
fmt.Fprint(w, `usage:
assets compile [--nwn INSTALL] [--non-recursive] <dir>...
assets convert [--to dds|png|tga] [--backend PATH] [--non-recursive] <dir>...
assets upscale [--scale N] [--backend PATH] [--non-recursive] <dir>...
assets check-mdl <path>...
assets fix-mdl [--dry-run] <path>...
assets check-dupes <dir>...
assets clean-dupes [--dry-run] <primary> <clean>
`)
}
+26
View File
@@ -0,0 +1,26 @@
package assets
import (
"bytes"
"strings"
"testing"
)
func env(string) string { return "" }
func TestRunNoArgsIsUsage(t *testing.T) {
var out, errw bytes.Buffer
if code := Run(nil, &out, &errw, env); code != exitUsage {
t.Fatalf("no args exit = %d, want %d", code, exitUsage)
}
if !strings.Contains(errw.String(), "usage") {
t.Fatalf("no args should print usage, got: %q", errw.String())
}
}
func TestRunUnknownSubcommandIsUsage(t *testing.T) {
var out, errw bytes.Buffer
if code := Run([]string{"frobnicate"}, &out, &errw, env); code != exitUsage {
t.Fatalf("unknown subcommand exit = %d, want %d", code, exitUsage)
}
}
+98
View File
@@ -0,0 +1,98 @@
package assets
import (
"flag"
"fmt"
"io"
"os"
"path/filepath"
"strconv"
"strings"
)
var upscaleBackends = []string{"upscayl-bin", "upscayl", "realesrgan-ncnn-vulkan", "waifu2x-ncnn-vulkan"}
// runUpscale upscales textures in place through an installed ncnn-vulkan
// backend. DDS inputs are bridged through PNG so the NWN flip stays correct.
func runUpscale(args []string, stdout, stderr io.Writer) int {
fs := flag.NewFlagSet("upscale", flag.ContinueOnError)
fs.SetOutput(stderr)
scale := fs.Int("scale", 4, "upscale factor")
backend := fs.String("backend", "", "override the upscaler binary")
nonRecursive := fs.Bool("non-recursive", false, "do not descend into subdirectories")
if err := fs.Parse(args); err != nil {
return exitUsage
}
dirs := fs.Args()
if len(dirs) == 0 {
fmt.Fprintln(stderr, "assets upscale: usage: upscale [--scale N] <dir>...")
return exitUsage
}
candidates := upscaleBackends
if *backend != "" {
candidates = []string{*backend}
}
tool := look(candidates...)
if tool == "" {
fmt.Fprintf(stderr, "assets upscale: no upscaler found — install one of: %s\n", strings.Join(upscaleBackends, ", "))
return exitTool
}
files, err := walk(dirs, textureExts, !*nonRecursive)
if err != nil {
fmt.Fprintln(stderr, "assets upscale:", err)
return exitUsage
}
// magick is only needed if a .dds input is present; resolve lazily.
magick := ""
failed := false
for _, src := range files {
var upErr error
if strings.EqualFold(filepath.Ext(src), ".dds") {
if magick == "" {
if magick = look("magick"); magick == "" {
fmt.Fprintln(stderr, "assets upscale: .dds input needs ImageMagick (magick) for the png bridge")
return exitTool
}
}
upErr = upscaleDDS(tool, magick, src, *scale)
} else {
upErr = upscaleImage(tool, src, src, *scale)
}
if upErr != nil {
fmt.Fprintf(stderr, "assets upscale: %s: %v\n", src, upErr)
failed = true
}
}
if failed {
return exitFail
}
return exitOK
}
// upscaleImage runs the ncnn-vulkan backend to upscale src into dst (may be the
// same path).
func upscaleImage(tool, src, dst string, scale int) error {
tmp := dst + ".upscaled.png"
out, err := runner("", nil, tool, "-i", src, "-o", tmp, "-s", strconv.Itoa(scale))
if err != nil {
return fmt.Errorf("%s: %v: %s", filepath.Base(tool), err, strings.TrimSpace(string(out)))
}
return os.Rename(tmp, dst)
}
// upscaleDDS bridges a DDS through PNG: dds->png (flip), upscale, png->dds
// (flip back), yielding a correctly-flipped upscaled DDS.
func upscaleDDS(tool, magick, src string, scale int) error {
tmpPNG := src + ".bridge.png"
if err := ddsToPNG(magick, src, tmpPNG); err != nil {
return err
}
defer os.Remove(tmpPNG)
if err := upscaleImage(tool, tmpPNG, tmpPNG, scale); err != nil {
return err
}
return pngToDDS(magick, tmpPNG, src)
}
+69
View File
@@ -0,0 +1,69 @@
package assets
import (
"bytes"
"os"
"path/filepath"
"testing"
)
func TestUpscalePNGUsesBackend(t *testing.T) {
// A fake backend binary on PATH so look() resolves it.
binDir := t.TempDir()
fake := filepath.Join(binDir, "upscayl-bin")
if err := os.WriteFile(fake, []byte("#!/bin/sh\n"), 0o755); err != nil {
t.Fatal(err)
}
t.Setenv("PATH", binDir)
// Stub runner: emulate `-i in -o out -s scale` by copying in->out.
orig := runner
defer func() { runner = orig }()
var gotScale string
runner = func(dir string, env []string, name string, args ...string) ([]byte, error) {
var in, out string
for i := 0; i < len(args)-1; i++ {
switch args[i] {
case "-i":
in = args[i+1]
case "-o":
out = args[i+1]
case "-s":
gotScale = args[i+1]
}
}
data, err := os.ReadFile(in)
if err != nil {
return nil, err
}
return nil, os.WriteFile(out, data, 0o644)
}
dir := t.TempDir()
src := filepath.Join(dir, "tex.png")
if err := os.WriteFile(src, []byte("pngdata"), 0o644); err != nil {
t.Fatal(err)
}
var stdout, stderr bytes.Buffer
if code := runUpscale([]string{"--scale", "2", dir}, &stdout, &stderr); code != exitOK {
t.Fatalf("upscale exit = %d\n%s", code, stderr.String())
}
if gotScale != "2" {
t.Fatalf("scale passed to backend = %q, want 2", gotScale)
}
if data, _ := os.ReadFile(src); string(data) != "pngdata" {
t.Fatalf("upscaled file content = %q, want the backend output", data)
}
}
func TestUpscaleNoBackendFailsClosed(t *testing.T) {
t.Setenv("PATH", t.TempDir()) // empty PATH: no backend resolvable
dir := t.TempDir()
if err := os.WriteFile(filepath.Join(dir, "tex.png"), []byte("x"), 0o644); err != nil {
t.Fatal(err)
}
var stdout, stderr bytes.Buffer
if code := runUpscale([]string{dir}, &stdout, &stderr); code != exitTool {
t.Fatalf("no-backend exit = %d, want %d", code, exitTool)
}
}
+26 -5
View File
@@ -17,6 +17,7 @@ import (
"os"
"git.westgate.pw/ShadowsOverWestgate/sow-tools/internal/app"
"git.westgate.pw/ShadowsOverWestgate/sow-tools/internal/assets"
"git.westgate.pw/ShadowsOverWestgate/sow-tools/internal/buildinfo"
"git.westgate.pw/ShadowsOverWestgate/sow-tools/internal/depot"
"git.westgate.pw/ShadowsOverWestgate/sow-tools/internal/menu"
@@ -103,6 +104,21 @@ var Registry = []Builder{
},
Wired: true,
},
{
Name: "assets",
Bin: "crucible-assets",
Summary: "compile/convert/upscale NWN assets + mdl/dupe integrity",
Commands: []Command{
{Name: "compile", Summary: "compile ASCII .mdl models to binary in place", Usage: "crucible assets compile [--nwn INSTALL] [--non-recursive] <dir>..."},
{Name: "convert", Summary: "convert textures to/from NWN DDS (flips vertically)", Usage: "crucible assets convert [--to dds|png|tga] [--backend PATH] [--non-recursive] <dir>..."},
{Name: "upscale", Summary: "upscale textures through an installed backend", Usage: "crucible assets upscale [--scale N] [--backend PATH] [--non-recursive] <dir>..."},
{Name: "check-mdl", Summary: "report uncompiled ASCII .mdl and model-name mismatches", Usage: "crucible assets check-mdl <path>..."},
{Name: "fix-mdl", Summary: "lowercase .mdl names and rewrite model identity to match", Usage: "crucible assets fix-mdl [--dry-run] <path>..."},
{Name: "check-dupes", Summary: "report runtime-name (basename) collisions across dirs", Usage: "crucible assets check-dupes <dir>..."},
{Name: "clean-dupes", Summary: "delete clean-tree files whose basename collides with primary", Usage: "crucible assets clean-dupes [--dry-run] <primary> <clean>"},
},
Wired: true,
},
{
Name: "hak",
Bin: "crucible-hak",
@@ -375,11 +391,16 @@ func runBuilder(name string, args []string, out, errw io.Writer) int {
return exitOK
}
}
if b.Name == "depot" && b.Wired {
// depot parses its own subcommand (status/push/verify/get/pull) and
// has its own richer exit contract (0/1/2/64/70), so it bypasses the
// b.Commands/delegateLegacy routing entirely.
return depot.Run(args, out, errw, os.Getenv)
if b.Wired {
// depot and assets are self-contained builders: they parse their own
// subcommands and own their exit contract, bypassing the
// b.Commands/delegateLegacy legacy routing entirely.
switch b.Name {
case "depot":
return depot.Run(args, out, errw, os.Getenv)
case "assets":
return assets.Run(args, out, errw, os.Getenv)
}
}
if !b.Wired {
// No migrated logic yet (depot): fail closed, never fake an artifact.
+5 -4
View File
@@ -151,6 +151,7 @@ func TestBuilderHelpIsOK(t *testing.T) {
func TestCanonicalCommandSurface(t *testing.T) {
want := map[string][]string{
"depot": {"status", "push", "verify", "get", "pull"},
"assets": {"compile", "convert", "upscale", "check-mdl", "fix-mdl", "check-dupes", "clean-dupes"},
"hak": {"build", "manifest"},
"module": {"build", "extract", "validate", "compare", "manifest"},
"topdata": {"validate", "build", "package", "compare", "convert"},
@@ -172,10 +173,10 @@ func TestRegistryCommandNamesAndAliasesAreUnambiguous(t *testing.T) {
for _, builder := range Registry {
seen := map[string]bool{}
for _, command := range builder.Commands {
// depot parses its own subcommands and bypasses AppCommand routing
// entirely (see the depot special-case in runBuilder), so its
// Commands carry no AppCommand.
requireAppCommand := builder.Name != "depot"
// depot and assets parse their own subcommands and bypass AppCommand
// routing entirely (see the self-contained-builder branch in
// runBuilder), so their Commands carry no AppCommand.
requireAppCommand := builder.Name != "depot" && builder.Name != "assets"
if command.Name == "" || command.Summary == "" || command.Usage == "" || (requireAppCommand && command.AppCommand == "") {
t.Errorf("%s has incomplete command metadata: %#v", builder.Name, command)
}
+1 -1
View File
@@ -18,7 +18,7 @@ bin=bin
# Keep in sync with internal/dispatch.Registry (Wired flag).
unwired=()
wired=(depot hak module topdata wiki)
wired=(assets depot hak module topdata wiki)
exit_of() { set +e; "$@" >/dev/null 2>&1; local c=$?; set -e; echo "${c}"; }