command and help ux pass #21

Merged
archvillainette merged 1 commits from crucible-ux-pass into main 2026-06-25 09:29:40 +00:00
29 changed files with 1077 additions and 3151 deletions
Showing only changes of commit d3ee546e33 - Show all commits
+2 -2
View File
@@ -12,8 +12,8 @@ dispatcher and the `crucible-<name>` binaries.
## What this repo owns / does not own
Owns: build/extract/validate/compare pipeline, ERF/HAK packing, topdata 2da/tlk
compilation, wiki rendering/deploy, depot blob verify, music conversion,
changelog. Does **not** own authored game content (that is `sow-module` /
compilation, wiki rendering/deploy, depot blob verify, changelog. Does **not**
own authored game content (that is `sow-module` /
`sow-topdata` / `sow-assets-manifest`) or any production deploy authority (that
is `sow-platform`).
+3 -6
View File
@@ -29,7 +29,7 @@ command lands is mapped in [`docs/command-surface.md`](docs/command-surface.md).
## Status (cutover performed)
The internal `app`/`pipeline`/`project`/`erf`/`gff`/`topdata`/`music`/`changelog`/
The internal `app`/`pipeline`/`project`/`erf`/`gff`/`topdata`/`changelog`/
`validator` packages from `gitea/sow-tools` have been migrated into this tree, and
the `module`, `topdata`, `hak`, and `wiki` builders now **delegate to the migrated
`nwn-tool` command surface** (mapped in
@@ -49,7 +49,7 @@ which downloads the latest released `crucible` for your OS and runs it:
```bash
./crucible # interactive menu (pick a command)
./crucible module build
./crucible topdata validate-topdata
./crucible topdata validate
```
Windows (PowerShell):
@@ -62,15 +62,12 @@ The binary is cached under `~/.cache/crucible/<version>/` (`%LOCALAPPDATA%\cruci
on Windows); `--repo-local` caches inside the repo instead. Private releases:
set `CRUCIBLE_TOKEN` or write the token to `~/.config/crucible/token`.
Only the **music** builder needs `ffmpeg`; everything else has zero dependencies.
If you run a music command without it, Crucible prints a per-OS install hint.
## Develop
Self-contained (D8) — a host with only Nix can run everything:
```bash
nix develop # Go + ffmpeg + shellcheck + yamllint + make
nix develop # Go + shellcheck + yamllint + make
make check # go vet + go test + shellcheck + yamllint
make build # build every cmd/* into ./bin (gitignored)
make smoke # build + assert the fail-closed contract
+1 -6
View File
@@ -6,10 +6,6 @@
# and ships them on a static base. The `crucible` dispatcher is the entrypoint;
# consumer CI can also call the standalone crucible-<name> binaries by path.
#
# NOTE: the internal pipeline is migrated and the music conversion path is wired,
# so the runtime stage is debian-slim with ffmpeg on PATH (BMU encode/decode).
# See docs/migration-from-nwn-tool.md.
FROM golang:1.26-alpine AS build
WORKDIR /src
RUN apk add --no-cache git
@@ -30,11 +26,10 @@ RUN set -eux; \
done
FROM debian:12-slim
# ffmpeg: the migrated music pipeline shells out to it for BMU conversion.
# ca-certificates: builders fetch published manifests over HTTPS.
RUN set -eux; \
apt-get update; \
apt-get install -y --no-install-recommends ffmpeg ca-certificates; \
apt-get install -y --no-install-recommends ca-certificates; \
rm -rf /var/lib/apt/lists/*; \
useradd --system --create-home --uid 65532 nonroot
COPY --from=build /out/ /usr/local/bin/
+56 -46
View File
@@ -1,67 +1,77 @@
# Crucible command surface
Crucible re-homes the single `nwn-tool` (a.k.a. `sow-toolkit`) command surface
into five builders plus a dispatcher (D11). This table maps every legacy command
to its Crucible home. It is the migration contract — keep it in sync with
`internal/dispatch.Registry`.
`internal/dispatch.Registry` is the single source of truth for Crucible's
builders, visible commands, descriptions, help, and hidden compatibility
aliases.
## Builders
## Visible commands
| Builder | Legacy `nwn-tool` commands subsumed |
| ------------------ | ------------------------------------------------------------------------------------- |
| `crucible-module` | `build`, `build-module`, `extract`, `validate`, `compare`, `apply-hak-manifest` |
| `crucible-topdata` | `validate-topdata`, `build-topdata`, `build-top-package`, `compare-topdata`, `convert-topdata` |
| `crucible-hak` | `build-haks`, `apply-hak-manifest` |
| `crucible-wiki` | `build-wiki`, `deploy-wiki` |
| `crucible-depot` | (new) depot blob verify/move — was shell `mc`/S3 in `sow-assets-manifest` |
| Builder | Command | Purpose |
| --- | --- | --- |
| `hak` | `build` | Build configured HAK archives and their manifest. |
| `hak` | `manifest` | Apply a generated HAK list to module source. |
| `module` | `build` | Build the module and configured project outputs. |
| `module` | `extract` | Extract built archives into configured source trees. |
| `module` | `validate` | Validate project layout, config, and source inventory. |
| `module` | `compare` | Compare source content with built archives. |
| `module` | `manifest` | Apply a generated HAK list to module source. |
| `topdata` | `validate` | Validate topdata source and canonical JSON compatibility. |
| `topdata` | `build` | Compile topdata and refresh the configured package. |
| `topdata` | `package` | Package cached outputs into HAK and TLK files. |
| `topdata` | `compare` | Compare current output with a fresh native build. |
| `topdata` | `convert` | Convert between 2DA and native JSON/module formats. |
| `wiki` | `build` | Render wiki page drafts from compiled topdata. |
| `wiki` | `deploy` | Deploy generated wiki pages to NodeBB. |
`apply-hak-manifest` is HAK-list maintenance on a module; it is reachable from
both `crucible-module` and `crucible-hak`. Canonical home is `crucible-hak`.
`crucible-depot` remains registered but unwired. It fails closed with exit `70`
and never emits placeholder artifacts.
## Folded / global commands
## Hidden compatibility aliases
These legacy commands are **not** top-level builders:
Existing scripts may continue using these names indefinitely. They are accepted
but omitted from routine help and the interactive menu:
- `music *` (`list-datasets`, `scan`, `build`, `credits`, `validate`, `manifest`,
`normalize`) → folds into the **module/hak build pipeline** (music BMUs are
packed into HAKs at build time). Exposed as `crucible-module music ...` /
`crucible-hak music ...`. Needs `ffmpeg` at runtime (in the image).
- `config *` (`validate`, `effective`, `inspect`, `explain`, `sources`) → a
**global** concern shared by every builder; exposed as the global
`crucible config ...`, not its own binary.
- `build-changelog` → release tooling; exposed as the global `crucible changelog`
rather than a content builder.
| Builder | Hidden alias | Implementation |
| --- | --- | --- |
| `hak` | `build-haks` | `build-haks` |
| `hak` | `apply-hak-manifest` | `apply-hak-manifest` |
| `module` | `build-module` | module-only `build-module` |
| `module` | `apply-hak-manifest` | `apply-hak-manifest` |
| `topdata` | `validate-topdata` | `validate-topdata` |
| `topdata` | `build-topdata` | `build-topdata` |
| `topdata` | `build-top-package` | `build-top-package` |
| `topdata` | `compare-topdata` | `compare-topdata` |
| `topdata` | `convert-topdata` | `convert-topdata` |
| `wiki` | `build-wiki` | `build-wiki` |
| `wiki` | `deploy-wiki` | `deploy-wiki` |
## Global commands (implemented)
`module build-module` deliberately retains its narrower module-only behavior;
it is not redirected to the broader `module build`.
## Global commands
```text
crucible help | -h usage + builder list
crucible version | -V build version (git sha via -ldflags)
crucible list builders, one per line: name<TAB>bin<TAB>summary
crucible config [args] -> legacy `config` command group
crucible changelog [args] -> legacy `build-changelog`
crucible version | -V build version
crucible list name<TAB>binary<TAB>summary
crucible config [args] inspect and validate effective configuration
crucible changelog [args] generate a release changelog
```
`crucible list` is machine-readable so CI can enumerate builders without parsing
help text.
## HAK builder source modes
## HAK build options
```text
build-haks [--hak <hak-name> ...] [--archive <archive-name> ...]
[--source-manifest <path>] [--content-addressed-root <path>]
[--plan-only] [--skip-music] [--music-dataset <id> ...]
crucible hak build
[--hak <hak-name> ...]
[--archive <archive-name> ...]
[--source-manifest <path>]
[--content-addressed-root <path>]
[--plan-only]
[--quiet|--verbose|--debug]
```
When a source manifest contains `asset_sources`, pass
`--content-addressed-root <directory>` (or
`--content-addressed-root=<directory>`). Crucible resolves each declared SHA-256
`--content-addressed-root <directory>`. Crucible resolves each declared SHA-256
under `sha256/<first-two>/<next-two>/<full-sha256>` and verifies streamed bytes
while writing the selected HAK archives. Source manifests without
`asset_sources` continue to use the configured assets tree.
## Global flags (planned, at wiring time)
`--quiet`, `--verbose`, `--debug` (the legacy verbosity model), passed after the
builder name, e.g. `crucible topdata build-topdata --verbose`.
while writing selected HAK archives. Authored `.bmu` files are ordinary assets
and require no conversion pipeline.
+2 -5
View File
@@ -27,7 +27,7 @@ checklist is kept as the record of what was done.
## Cutover steps
1. **Move the internal packages.** Bring `internal/{app,pipeline,project,erf,
gff,topdata,music,changelog,validator}` from `gitea/sow-tools` into this repo.
gff,topdata,changelog,validator}` from `gitea/sow-tools` into this repo.
Re-home `internal/app` command wiring behind `internal/dispatch` instead of
the old flag parser in `cmd/nwn-tool`.
2. **Restore dependencies.** Add `golang.org/x/text` and `gopkg.in/yaml.v3` back
@@ -43,10 +43,7 @@ checklist is kept as the record of what was done.
artifacts and image layers only.
6. **Carry the determinism test.** Port
`TestBuildNativeKeepsGeneratedIDsStableAcrossRepeatedBuilds` and keep it green.
7. **Runtime base for music.** If music conversion is wired, switch the final
Docker stage from `distroless/static` to a debian-slim base that includes
`ffmpeg`.
8. **Update `make smoke`.** Once a builder is wired, change the smoke
7. **Update `make smoke`.** Once a builder is wired, change the smoke
expectation for it from exit `70` to a real success path.
## Wrapper contract carried over
@@ -0,0 +1,346 @@
# Crucible Command Surface Cleanup Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use
> `superpowers:subagent-driven-development` or
> `superpowers:executing-plans` to implement this plan task-by-task. Steps use
> checkbox (`- [ ]`) syntax for tracking.
**Goal:** Replace repetitive Crucible command names with a descriptive,
registry-driven public surface, retain long names as hidden aliases, and remove
the unused music conversion pipeline without affecting authored `.bmu` assets.
**Architecture:** `internal/dispatch.Registry` gains explicit command records
that drive lookup, help, and the interactive menu while translating to unchanged
`internal/app` implementation names. The unused music subsystem is removed
through its app, pipeline, project-config, validator, packaging, and consumer
boundaries. The two active `sow-assets-manifest` call sites are decoupled in the
same worktree session.
**Tech Stack:** Go 1.26, Go tests, Nix flakes, Dockerfile, Bash/Bats.
## Global Constraints
- Never commit or push.
- Do not touch secrets or `.sops` data.
- Keep `internal/dispatch.Registry` synchronized with
`docs/command-surface.md`.
- Preserve deterministic artifact behavior.
- Preserve every existing long command as a hidden alias except all music
commands, which are removed.
- Preserve `.bmu` as an ordinary HAK asset.
- Do not retain generated binaries or build output.
---
### Task 1: Registry-driven canonical commands and hidden aliases
**Files:**
- Modify: `internal/dispatch/dispatch.go`
- Modify: `internal/dispatch/dispatch_test.go`
**Interfaces:**
- Produces a `Command` record owned by each `Builder`.
- Produces lookup that returns the command record and exact app target.
- Preserves `Builder.Legacy` only if required by migration tests; otherwise
replaces it with command-derived compatibility metadata.
- [ ] Add failing tests covering the visible command table:
`hak build/manifest`, `module build/extract/validate/compare/manifest`,
`topdata validate/build/package/compare/convert`, and `wiki build/deploy`.
- [ ] Add failing tests proving long aliases resolve to the same app target and
that `module build-module` resolves to `build-module`, not `build`.
- [ ] Add a failing test proving all music commands are rejected.
- [ ] Run:
```bash
nix develop --command go test ./internal/dispatch -run 'TestCanonical|TestHidden|TestMusic' -v
```
Expected: failures caused by the old registry shape and exposed music
commands.
- [ ] Introduce command records with fields equivalent to:
```go
type Command struct {
Name string
Summary string
AppCommand string
Usage string
Options []string
HiddenAlias []string
}
```
Use the minimum final field names that keep lookup, menu, and help readable.
- [ ] Replace builder-level subcommand acceptance with command lookup. Translate
canonical and hidden aliases to `AppCommand` before calling `app.Run`.
- [ ] Define the approved command table and remove dispatcher music entries.
- [ ] Re-run the focused dispatch tests and the full dispatch package:
```bash
nix develop --command go test ./internal/dispatch -v
```
Expected: pass.
### Task 2: Descriptive menu and command-specific help
**Files:**
- Modify: `internal/menu/menu.go`
- Modify: `internal/menu/menu_test.go`
- Modify: `internal/dispatch/dispatch.go`
- Modify: `internal/dispatch/dispatch_test.go`
**Interfaces:**
- `menu.Item` carries display label, summary, dispatcher args, usage, and option
lines.
- `menu.Select` returns selected dispatcher args plus entered arguments.
- Dispatcher command help uses the same registry guidance as the menu.
- [ ] Add failing menu tests proving descriptions start at one aligned column
for labels of different lengths and hidden aliases never enter `menuItems()`.
- [ ] Add a failing test selecting a command and asserting output contains its
usage, option guidance, and
`arguments (press Enter to use defaults):`.
- [ ] Add failing dispatcher tests proving
`crucible topdata build --help` returns `0`, prints canonical usage/options,
and does not delegate into project loading.
- [ ] Run:
```bash
nix develop --command go test ./internal/menu ./internal/dispatch -run 'Test.*Menu|Test.*Help' -v
```
Expected: fail against the current fixed-width menu and builder-only help.
- [ ] Compute menu label width from visible items and render descriptions with
that width.
- [ ] Print command guidance immediately after a selection and before reading
optional arguments.
- [ ] Add command-level help handling before app delegation.
- [ ] Ensure builder help lists only canonical names and their summaries.
- [ ] Run:
```bash
nix develop --command go test ./internal/menu ./internal/dispatch -v
```
Expected: pass.
### Task 3: Remove music from HAK app and pipeline
**Files:**
- Delete: `internal/pipeline/music.go`
- Modify: `internal/pipeline/build.go`
- Modify: `internal/pipeline/pipeline_test.go`
- Modify: `internal/app/app.go`
- Modify: `internal/app/app_test.go`
- Delete: `internal/music/config.go`
- Delete: `internal/music/credits.go`
- Delete: `internal/music/metadata.go`
- Delete: `internal/music/music.go`
- Delete: `internal/music/music_test.go`
- Delete: `internal/music/naming.go`
**Interfaces:**
- `pipeline.BuildHAKOptions` retains only HAK selection, source-manifest,
content-addressed-root, and progress controls.
- `pipeline.BuildResult` retains artifact fields unrelated to generated music
credits.
- `build-haks` accepts no music-specific flags.
- [ ] Replace music pipeline tests with a failing contract test that builds a
HAK containing a pre-authored `.bmu` and proves the file appears in the
manifest/archive without conversion setup.
- [ ] Add failing parser tests proving `--skip-music` and `--music-dataset`
return unknown-argument errors and help does not mention them.
- [ ] Run:
```bash
nix develop --command go test ./internal/app ./internal/pipeline -run 'Test.*Music|Test.*BMU|TestParseBuildHAK' -v
```
Expected: at least the removed-flag assertions fail.
- [ ] Remove the app `music` command, argument parser, emitters, music HAK
options, and credits console output.
- [ ] Remove music fields from `BuildHAKOptions` and `BuildResult`.
- [ ] Remove preparation/cleanup calls from HAK builds so asset collection reads
authored resources directly.
- [ ] Delete the music implementation packages and obsolete conversion tests.
- [ ] Keep `.bmu` handling in ordinary ERF/resource and asset collection paths.
- [ ] Run:
```bash
nix develop --command go test ./internal/app ./internal/pipeline -v
```
Expected: pass.
### Task 4: Remove music configuration and validation
**Files:**
- Modify: `internal/project/project.go`
- Modify: `internal/project/effective.go`
- Modify: `internal/project/project_test.go`
- Modify: `internal/validator/validator.go`
- Modify: `internal/validator/validator_test.go`
**Interfaces:**
- `project.Config` and `project.EffectiveConfig` contain no music section.
- Project scanning uses only configured inventory extensions.
- `.bmu` and `.wav` remain default asset extensions; `.mp3` and `.ogg` do not.
- [ ] Add or adjust failing project tests asserting default extensions include
`.bmu` and `.wav` but exclude `.mp3` and `.ogg`.
- [ ] Add a failing strict-decoder test proving a top-level `music:` field is
rejected as unknown.
- [ ] Run:
```bash
nix develop --command go test ./internal/project ./internal/validator -run 'Test.*Music|Test.*AssetExtension|Test.*BMU' -v
```
Expected: fail while music config is still accepted.
- [ ] Remove music config types, defaults, normalization, validation, accessors,
effective config, provenance defaults, and environment overrides.
- [ ] Simplify `Project.Scan` to accept assets solely from
`effective.Inventory.AssetExtensions`.
- [ ] Remove validator logic that exempts conversion source files.
- [ ] Remove conversion-only `.mp3` and `.ogg` from default asset extensions;
retain `.bmu` and `.wav`.
- [ ] Remove obsolete music-focused project and validator tests.
- [ ] Run:
```bash
nix develop --command go test ./internal/project ./internal/validator -v
```
Expected: pass.
### Task 5: Remove ffmpeg packaging and stale documentation
**Files:**
- Modify: `flake.nix`
- Modify: `docker/Dockerfile`
- Modify: `wrappers/crucible.sh`
- Modify: `README.md`
- Modify: `AGENTS.md`
- Modify: `docs/command-surface.md`
- Modify: `docs/migration-from-nwn-tool.md`
**Interfaces:**
- Nix development shell and image no longer contain ffmpeg.
- Docker runtime contains only certificates and the non-root runtime support
needed by current builders.
- User docs advertise canonical short commands.
- [ ] Add a shell verification that fails while active packaging still mentions
ffmpeg or removed command names:
```bash
! rg -n 'ffmpeg|ffprobe|skip-music|music-dataset|module music|hak music' \
flake.nix docker wrappers README.md AGENTS.md docs/command-surface.md
```
- [ ] Remove `pkgs.ffmpeg-headless` and `ffmpeg` from Nix image/dev-shell
contents and update comments.
- [ ] Return the Docker runtime to a minimal base without ffmpeg while retaining
CA certificates and non-root execution.
- [ ] Remove the wrapper's `ffmpeg-free` wording.
- [ ] Rewrite command-surface documentation around canonical names and a
concise hidden-alias compatibility section.
- [ ] Update README examples and ownership/status prose.
- [ ] Re-run the search. Expected: no matches in active files.
### Task 6: Decouple active consumer scripts
**Files in `../sow-assets-manifest`:**
- Modify: `scripts/pack-haks.sh`
- Modify: `scripts/build-local-haks.sh`
- Modify: `tests/pack-haks.bats`
- Modify only directly related active comments/tests if a focused test proves
they require adjustment.
**Interfaces:**
- Both scripts call the hidden-compatible `hak build-haks` or canonical
`hak build` without `--skip-music`.
- [ ] Read `../sow-assets-manifest/AGENTS.md` before editing.
- [ ] Add or adjust a focused test to reject `--skip-music` in generated
Crucible invocations.
- [ ] Run the focused test and confirm it fails while the flag is present.
- [ ] Remove `--skip-music` and its lockstep comments from both scripts.
- [ ] Update the HAK parser test invocation to omit the removed flag.
- [ ] Run:
```bash
cd ../sow-assets-manifest
nix develop --command bats tests/pack-haks.bats
```
Expected: pass.
### Task 7: Full verification and scratchpad cleanup
**Files:**
- Modify: `../SCRATCHPAD.md`
- Do not retain: `bin/`, `.cache/go-build/`, result symlinks, image tarballs, or
other generated outputs.
- [ ] Run formatting:
```bash
nix develop --command gofmt -w internal/app internal/dispatch internal/menu internal/pipeline internal/project internal/validator
```
- [ ] Run full repository checks:
```bash
nix develop --command make check
make build
make smoke
```
- [ ] Run focused consumer verification from Task 6.
- [ ] Verify deleted pipeline identifiers are absent from active code:
```bash
rg -n 'BuildMusic|CreditsSummary|MusicDataset|skip-music|music-dataset|ffmpeg|ffprobe' \
--glob '!docs/superpowers/**' .
```
Expected: no matches.
- [ ] Verify active command docs/examples do not advertise long names:
```bash
rg -n 'topdata (validate-topdata|build-topdata|build-top-package|compare-topdata|convert-topdata)|wiki (build-wiki|deploy-wiki)|hak (build-haks|apply-hak-manifest)' \
README.md docs/command-surface.md
```
Expected: matches only inside the explicitly labeled hidden-alias
compatibility section.
- [ ] Inspect `git diff --check`, `git status --short`, and both repository
diffs. Remove only generated files created by this work.
- [ ] Remove the completed “Simplify crucible commands” to-do and the full
“Crucible commands issue” section from `../SCRATCHPAD.md`; do not archive it.
- [ ] Record exact verification commands and outcomes in the final handoff.
@@ -0,0 +1,147 @@
# Crucible Command Surface Cleanup Design
**Date:** 2026-06-25
**Status:** Approved
**Scope:** `sow-tools`, with the two active `sow-assets-manifest` call sites
updated in lockstep
## Problem
Crucible exposes migrated `nwn-tool` implementation names directly. Several
public commands repeat their builder name (`topdata build-topdata`,
`wiki build-wiki`), the interactive menu repeats builder-level descriptions
instead of explaining each action, and selected commands only prompt for
unexplained "extra args".
The registry also exposes a migrated music-conversion subsystem that is not part
of the current project workflow. Live HAK builds explicitly bypass it with
`--skip-music`; authored `.bmu` files are already packed as ordinary assets.
## Decisions
### Canonical command names
The visible command surface is:
| Builder | Visible commands |
| --- | --- |
| `hak` | `build`, `manifest` |
| `module` | `build`, `extract`, `validate`, `compare`, `manifest` |
| `topdata` | `validate`, `build`, `package`, `compare`, `convert` |
| `wiki` | `build`, `deploy` |
The dispatcher translates these names to the existing `internal/app`
implementation commands. The app implementation names do not need to be
renamed.
### Hidden compatibility aliases
Existing long names remain callable indefinitely but are omitted from the
interactive menu and routine builder help:
- `hak build-haks`
- `hak apply-hak-manifest`
- `module build-module`
- `module apply-hak-manifest`
- `topdata validate-topdata`
- `topdata build-topdata`
- `topdata build-top-package`
- `topdata compare-topdata`
- `topdata convert-topdata`
- `wiki build-wiki`
- `wiki deploy-wiki`
Aliases preserve their existing implementation target. In particular,
`module build-module` continues to run the module-only implementation; it must
not silently become the broader `module build`.
### Registry model
`internal/dispatch.Registry` remains the single source of truth. Each wired
builder owns command records containing:
- visible command name;
- action-oriented summary;
- underlying `internal/app` command;
- command usage and option guidance;
- hidden compatibility aliases, where applicable.
Execution, menu items, and builder help are generated from these records.
Unknown commands continue to fail with exit `64`.
### Interactive behavior
The main menu:
- lists only visible commands;
- aligns the description column from the longest visible label;
- shows a distinct description for every command.
After selection, it prints the selected command's usage and available options,
then prompts:
```text
arguments (press Enter to use defaults):
```
Arguments continue to use shell-like whitespace splitting, matching current
behavior. Quoted argument parsing is out of scope.
`crucible <builder> <command> --help` prints the same command-specific guidance,
returns success, and performs no project loading or build work.
### Remove the unused music pipeline
The music conversion subsystem is deleted rather than hidden:
- remove the `music` app and dispatcher commands;
- remove `internal/music` and `internal/pipeline/music.go`;
- remove music config, effective config, validation, environment overrides, and
project accessors;
- remove HAK music preparation, conversion, credits, generated-manifest result
fields, `--skip-music`, and `--music-dataset`;
- remove ffmpeg/ffprobe runtime and development dependencies;
- remove music-pipeline documentation and wrapper wording.
`.bmu` remains a supported ordinary asset extension and continues through the
same content-addressed HAK packing path as other authored files. Actual game
content, category names, or prose that happens to use the word "music" is not
part of this deletion.
The default asset-extension list drops conversion-only `.mp3` and `.ogg`.
`.wav` remains because it is an NWN resource type independent of the deleted
conversion pipeline.
Because `sow-assets-manifest` currently passes `--skip-music`, its
`scripts/pack-haks.sh` and `scripts/build-local-haks.sh` call sites are updated
in the same change before verification.
## Error handling
- Canonical and hidden alias lookup is deterministic within one builder.
- Registry tests reject duplicate visible names or aliases.
- Command help never delegates to `internal/app`.
- Removed music commands and flags fail as unknown usage.
- Existing project files containing a top-level `music:` configuration become
invalid under strict YAML decoding. No active consumer configuration contains
that field.
## Testing
Tests assert public contracts rather than exact incidental formatting:
- visible command names and hidden alias translation;
- preservation of the distinct `module build-module` target;
- hidden aliases absent from menu and normal help;
- command-specific help exits successfully without project setup;
- menu columns align and selected-command guidance is displayed;
- `.bmu` remains accepted as an asset;
- `.mp3`/`.ogg` are no longer default packable assets;
- HAK builds no longer expose or execute music preparation;
- ffmpeg is absent from package/image definitions;
- both consumer scripts no longer pass `--skip-music`.
Final verification is `nix develop --command make check`, `make build`,
`make smoke`, focused consumer tests, and repository-wide searches for deleted
pipeline identifiers.
+3 -5
View File
@@ -10,12 +10,11 @@
in
{
# Self-contained (D8): a host with ONLY nix can enter this shell and run
# `make check`. ffmpeg is present because the migrated music pipeline needs
# it; the scaffold itself is pure stdlib Go.
# `make check`.
# Daemonless image build (D8): `nix build .#image` produces an OCI tarball
# with no docker/podman daemon. CI loads/pushes it with skopeo. This is the
# Nix-native replacement for `docker build` on the host-mode runner, which
# has no container runtime. ffmpeg-headless ships the BMU codec path.
# has no container runtime.
packages = forAllSystems (pkgs:
let
version = self.shortRev or self.dirtyShortRev or "unknown";
@@ -49,7 +48,7 @@
image = pkgs.dockerTools.buildLayeredImage {
name = "registry.westgate.pw/deployment/crucible";
tag = version;
contents = [ crucible pkgs.cacert pkgs.ffmpeg-headless pkgs.fakeNss ];
contents = [ crucible pkgs.cacert pkgs.fakeNss ];
config = {
Entrypoint = [ "/bin/crucible" ];
Cmd = [ "help" ];
@@ -69,7 +68,6 @@
go
gopls
gotools
ffmpeg
git
tea
gnumake
+1 -207
View File
@@ -9,7 +9,6 @@ import (
"os/exec"
"path/filepath"
"runtime"
"sort"
"strconv"
"strings"
"sync"
@@ -184,11 +183,6 @@ var commands = []command{
description: "Inspect, explain, and validate the resolved project configuration.",
run: runConfig,
},
{
name: "music",
description: "List, scan, build, and validate configured music datasets.",
run: runMusic,
},
{
name: "compare",
description: "Compare current source content against the built archives.",
@@ -470,8 +464,6 @@ type buildHAKOptions struct {
filteredArchives []string
sourceManifest string
contentAddressedRoot string
musicDatasets []string
skipMusic bool
planOnly bool
logLevel logLevel
}
@@ -721,46 +713,11 @@ func (c *buildHAKConsole) emitResult(result pipeline.BuildResult) {
spin.linebreak()
fmt.Fprintln(c.stdout, "Build HAKs ----------")
if c.level != logLevelQuiet {
c.emitCreditsSummary(result)
c.emitCoreSteps(result)
}
c.emitFinalSummary(result)
}
func (c *buildHAKConsole) emitCreditsSummary(result pipeline.BuildResult) {
summary := result.CreditsSummary
if len(summary.ArtifactPaths) == 0 {
return
}
fmt.Fprintln(c.stdout, "Refreshing credits cache")
if len(summary.ScannedDirs) > 0 {
fmt.Fprintf(c.stdout, " scanned: %s\n", strings.Join(summary.ScannedDirs, ", "))
}
fmt.Fprintf(c.stdout, " tracks: %d\n", summary.TrackCount)
if len(summary.GeneratedPaths) > 0 {
for _, path := range summary.GeneratedPaths {
fmt.Fprintf(c.stdout, " output: %s\n", c.relPath(path))
}
}
switch {
case summary.ChangedFiles == 0:
fmt.Fprintln(c.stdout, " status: unchanged")
default:
fmt.Fprintf(c.stdout, " updated: %d artifact(s)\n", summary.ChangedFiles)
}
if summary.TrackCount > 0 {
if c.level >= logLevelVerbose {
fmt.Fprintln(c.stdout, " mappings:")
for _, mapping := range summary.Mappings {
fmt.Fprintf(c.stdout, " %s -> %s\n", mapping.Source, mapping.Output)
}
} else {
fmt.Fprintf(c.stdout, " mapped: %d music file(s); use --verbose to list mappings\n", summary.TrackCount)
}
}
fmt.Fprintln(c.stdout)
}
func (c *buildHAKConsole) emitCoreSteps(result pipeline.BuildResult) {
fmt.Fprintln(c.stdout, "Validating project")
fmt.Fprintln(c.stdout, "Collecting asset resources")
@@ -808,9 +765,6 @@ func (c *buildHAKConsole) emitFinalSummary(result pipeline.BuildResult) {
if result.Manifest != "" {
fmt.Fprintf(c.stdout, "manifest: %s\n", c.relPath(result.Manifest))
}
if len(result.CreditsArtifactPaths) > 0 {
fmt.Fprintf(c.stdout, "credits artifacts: %d\n", len(result.CreditsArtifactPaths))
}
if len(result.AutogenManifestPaths) > 0 {
fmt.Fprintf(c.stdout, "autogen manifests: %d\n", len(result.AutogenManifestPaths))
}
@@ -1050,8 +1004,6 @@ func runBuildHAKs(ctx context) error {
ArchiveNames: opts.filteredArchives,
SourceManifestPath: opts.sourceManifest,
ContentAddressedRoot: opts.contentAddressedRoot,
SkipMusic: opts.skipMusic,
MusicDatasetIDs: opts.musicDatasets,
}
if opts.planOnly {
result, err = pipeline.PlanHAKsWithOptions(p, pipelineOpts)
@@ -1076,7 +1028,7 @@ func parseBuildHAKArgs(args []string) (buildHAKOptions, error) {
arg := args[index]
switch arg {
case "-h", "--help":
return opts, errors.New("usage: build-haks [--hak <hak-name> ...] [--archive <archive-name> ...] [--source-manifest <path>] [--content-addressed-root <path>] [--plan-only] [--skip-music] [--music-dataset <id> ...] [--quiet|--verbose|--debug]")
return opts, errors.New("usage: build-haks [--hak <hak-name> ...] [--archive <archive-name> ...] [--source-manifest <path>] [--content-addressed-root <path>] [--plan-only] [--quiet|--verbose|--debug]")
case "--hak":
index++
if index >= len(args) {
@@ -1091,14 +1043,6 @@ func parseBuildHAKArgs(args []string) (buildHAKOptions, error) {
opts.filteredArchives = append(opts.filteredArchives, args[index])
case "--plan-only":
opts.planOnly = true
case "--skip-music":
opts.skipMusic = true
case "--music-dataset":
index++
if index >= len(args) {
return opts, errors.New("--music-dataset requires a value")
}
opts.musicDatasets = append(opts.musicDatasets, args[index])
case "--quiet":
opts.logLevel = logLevelQuiet
case "--verbose":
@@ -1146,13 +1090,6 @@ func parseBuildHAKArgs(args []string) (buildHAKOptions, error) {
opts.contentAddressedRoot = value
continue
}
if value, ok, err := requireInlineFlagValue(arg, "--music-dataset"); ok || err != nil {
if err != nil {
return opts, err
}
opts.musicDatasets = append(opts.musicDatasets, value)
continue
}
if arg == "--quiet=true" {
opts.logLevel = logLevelQuiet
continue
@@ -1182,149 +1119,6 @@ func requireInlineFlagValue(arg, name string) (string, bool, error) {
return value, true, nil
}
type musicCommandOptions struct {
datasets []string
json bool
dryRun bool
check bool
force bool
}
func runMusic(ctx context) error {
if len(ctx.args) < 2 {
return errors.New("usage: music <list-datasets|scan|build|credits|validate|manifest|normalize> [--dataset <id>] [--json] [--dry-run] [--check] [--force]")
}
p, err := loadProject(ctx)
if err != nil {
return err
}
if err := p.Scan(); err != nil {
return err
}
subcommand := ctx.args[1]
opts, err := parseMusicCommandArgs(ctx.args[2:])
if err != nil {
return err
}
switch subcommand {
case "list-datasets":
return emitMusicDatasets(ctx, p, opts)
case "scan":
result, err := pipeline.BuildMusic(p, pipeline.MusicBuildOptions{DatasetIDs: opts.datasets, Write: false})
if err != nil {
return err
}
return emitMusicResult(ctx, "scan", result, opts)
case "build", "credits":
write := true
if subcommand == "build" && opts.dryRun {
write = false
}
if subcommand == "credits" && opts.check {
write = false
}
result, err := pipeline.BuildMusic(p, pipeline.MusicBuildOptions{DatasetIDs: opts.datasets, Write: write})
if err != nil {
return err
}
return emitMusicResult(ctx, subcommand, result, opts)
case "manifest", "normalize":
result, err := pipeline.BuildMusic(p, pipeline.MusicBuildOptions{DatasetIDs: opts.datasets, Write: false})
if err != nil {
return err
}
return emitMusicResult(ctx, subcommand, result, opts)
case "validate":
if err := p.ValidateLayout(); err != nil {
return err
}
result, err := pipeline.BuildMusic(p, pipeline.MusicBuildOptions{DatasetIDs: opts.datasets, Write: false})
if err != nil {
return err
}
return emitMusicResult(ctx, "validate", result, opts)
default:
return fmt.Errorf("unknown music subcommand %q", subcommand)
}
}
func parseMusicCommandArgs(args []string) (musicCommandOptions, error) {
opts := musicCommandOptions{}
for index := 0; index < len(args); index++ {
arg := args[index]
switch arg {
case "--dataset":
index++
if index >= len(args) || strings.TrimSpace(args[index]) == "" {
return opts, errors.New("--dataset requires a value")
}
opts.datasets = append(opts.datasets, args[index])
case "--json":
opts.json = true
case "--dry-run":
opts.dryRun = true
case "--check":
opts.check = true
case "--force":
opts.force = true
default:
if value, ok, err := requireInlineFlagValue(arg, "--dataset"); ok || err != nil {
if err != nil {
return opts, err
}
opts.datasets = append(opts.datasets, value)
continue
}
return opts, fmt.Errorf("unknown music argument %q", arg)
}
}
return opts, nil
}
func emitMusicDatasets(ctx context, p *project.Project, opts musicCommandOptions) error {
effective := p.EffectiveConfig()
if opts.json {
return emitConfigValue(ctx.stdout, effective.Music.Datasets, "json")
}
if len(effective.Music.Datasets) == 0 {
fmt.Fprintln(ctx.stdout, "music datasets: none")
return nil
}
fmt.Fprintln(ctx.stdout, "music datasets:")
ids := make([]string, 0, len(effective.Music.Datasets))
for id := range effective.Music.Datasets {
ids = append(ids, id)
}
sort.Strings(ids)
for _, id := range ids {
ds := effective.Music.Datasets[id]
fmt.Fprintf(ctx.stdout, " %s: source=%s output=%s prefix=%s\n", id, ds.Source, ds.Output, ds.Prefix)
}
return nil
}
func emitMusicResult(ctx context, action string, result pipeline.BuildResult, opts musicCommandOptions) error {
if opts.json {
return emitConfigValue(ctx.stdout, result.CreditsSummary, "json")
}
fmt.Fprintf(ctx.stdout, "music %s\n", action)
if len(result.CreditsSummary.ScannedDirs) > 0 {
fmt.Fprintf(ctx.stdout, "scanned: %s\n", strings.Join(result.CreditsSummary.ScannedDirs, ", "))
}
fmt.Fprintf(ctx.stdout, "tracks: %d\n", result.CreditsSummary.TrackCount)
if action != "scan" && len(result.CreditsArtifactPaths) > 0 {
fmt.Fprintf(ctx.stdout, "credits artifacts: %d\n", len(result.CreditsArtifactPaths))
}
if len(result.CreditsSummary.Mappings) > 0 && ctx.logLevel >= logLevelVerbose {
fmt.Fprintln(ctx.stdout, "mappings:")
for _, mapping := range result.CreditsSummary.Mappings {
fmt.Fprintf(ctx.stdout, " %s -> %s\n", mapping.Source, mapping.Output)
}
}
fmt.Fprintln(ctx.stdout, "status: ok")
return nil
}
func runExtract(ctx context) error {
p, err := loadProject(ctx)
if err != nil {
+20 -155
View File
@@ -54,8 +54,6 @@ func TestParseBuildHAKArgsHelpListsContentAddressedRoot(t *testing.T) {
"--source-manifest",
"--content-addressed-root",
"--plan-only",
"--skip-music",
"--music-dataset",
"--quiet",
"--verbose",
"--debug",
@@ -66,6 +64,18 @@ func TestParseBuildHAKArgsHelpListsContentAddressedRoot(t *testing.T) {
}
}
func TestParseBuildHAKArgsRejectsRemovedMusicFlags(t *testing.T) {
for _, args := range [][]string{
{"--skip-music"},
{"--music-dataset", "westgate"},
{"--music-dataset=westgate"},
} {
if _, err := parseBuildHAKArgs(args); err == nil {
t.Errorf("expected removed music arguments %v to fail", args)
}
}
}
func TestRunBuildTopPackageUsesCachedCompiledOutputs(t *testing.T) {
root := t.TempDir()
mkdirAll(t, filepath.Join(root, "build"))
@@ -113,7 +123,7 @@ topdata:
}
}
func TestRunBuildHAKsNormalOutputOmitsPerFileMappings(t *testing.T) {
func TestRunBuildHAKsPacksAuthoredBMU(t *testing.T) {
root := t.TempDir()
mkdirAll(t, filepath.Join(root, "assets", "envi", "music", "westgate"))
mkdirAll(t, filepath.Join(root, "build"))
@@ -135,23 +145,7 @@ haks:
include:
- envi/**
`)
writeFile(t, filepath.Join(root, "assets", "envi", "music", "westgate", "AleandAnecdotes.mp3"), "source-mp3")
writeFile(t, filepath.Join(root, "assets", "envi", "music", "westgate", "CREDITS.md"), "# NWN Music Pack Credits\n\n| Artist | Title | Output File | Original File | Album | Date | License / Copyright | Notes |\n|---|---|---|---|---|---|---|---|\n| Darren Curtis | Ale and Anecdotes | `mus_wg_andnc.bmu` | `AleandAnecdotes.mp3` | Darren's Commercially Free Music | | Darren Curtis Music | |\n")
ffprobePath := filepath.Join(root, "ffprobe")
writeFile(t, ffprobePath, "#!/bin/sh\nprintf '%s\\n' '{\"format\":{\"tags\":{\"artist\":\"Unknown\",\"title\":\"Broken Metadata\"}}}'\n")
if err := os.Chmod(ffprobePath, 0o755); err != nil {
t.Fatalf("chmod ffprobe: %v", err)
}
ffmpegPath := filepath.Join(root, "ffmpeg")
writeFile(t, ffmpegPath, "#!/bin/sh\nfor last; do :; done\nprintf 'bmu-data' > \"$last\"\n")
if err := os.Chmod(ffmpegPath, 0o755); err != nil {
t.Fatalf("chmod ffmpeg: %v", err)
}
t.Setenv("SOW_FFMPEG", ffmpegPath)
t.Setenv("SOW_FFPROBE", ffprobePath)
writeFile(t, filepath.Join(root, "assets", "envi", "music", "westgate", "westgate_theme.bmu"), "authored-bmu")
var stdout bytes.Buffer
ctx := context{
@@ -165,9 +159,12 @@ haks:
t.Fatalf("runBuildHAKs failed: %v", err)
}
output := stdout.String()
if strings.Contains(output, "AleandAnecdotes.mp3") || strings.Contains(output, "mus_wg_andnc.bmu") {
t.Fatalf("did not expect verbose mapping in normal mode, got %q", output)
manifest, err := os.ReadFile(filepath.Join(root, "build", "haks.json"))
if err != nil {
t.Fatalf("read HAK manifest: %v", err)
}
if !strings.Contains(string(manifest), "envi/music/westgate/westgate_theme.bmu") {
t.Fatalf("authored BMU missing from HAK manifest:\n%s", manifest)
}
}
@@ -184,64 +181,6 @@ func setTreeTime(t *testing.T, root string, modTime time.Time) {
}
}
func TestRunBuildHAKsVerboseListsMappings(t *testing.T) {
root := t.TempDir()
mkdirAll(t, filepath.Join(root, "assets", "envi", "music", "westgate"))
mkdirAll(t, filepath.Join(root, "build"))
mkdirAll(t, filepath.Join(root, "src"))
writeFile(t, filepath.Join(root, "nwn-tool.yaml"), `
module:
name: Test Module
resref: testmod
paths:
source: src
assets: assets
build: build
haks:
- name: envi
priority: 1
max_bytes: 1048576
split: false
include:
- envi/**
`)
writeFile(t, filepath.Join(root, "assets", "envi", "music", "westgate", "AleandAnecdotes.mp3"), "source-mp3")
writeFile(t, filepath.Join(root, "assets", "envi", "music", "westgate", "CREDITS.md"), "# NWN Music Pack Credits\n\n| Artist | Title | Output File | Original File | Album | Date | License / Copyright | Notes |\n|---|---|---|---|---|---|---|---|\n| Darren Curtis | Ale and Anecdotes | `mus_wg_andnc.bmu` | `AleandAnecdotes.mp3` | Darren's Commercially Free Music | | Darren Curtis Music | |\n")
ffprobePath := filepath.Join(root, "ffprobe")
writeFile(t, ffprobePath, "#!/bin/sh\nprintf '%s\\n' '{\"format\":{\"tags\":{\"artist\":\"Unknown\",\"title\":\"Broken Metadata\"}}}'\n")
if err := os.Chmod(ffprobePath, 0o755); err != nil {
t.Fatalf("chmod ffprobe: %v", err)
}
ffmpegPath := filepath.Join(root, "ffmpeg")
writeFile(t, ffmpegPath, "#!/bin/sh\nfor last; do :; done\nprintf 'bmu-data' > \"$last\"\n")
if err := os.Chmod(ffmpegPath, 0o755); err != nil {
t.Fatalf("chmod ffmpeg: %v", err)
}
t.Setenv("SOW_FFMPEG", ffmpegPath)
t.Setenv("SOW_FFPROBE", ffprobePath)
var stdout bytes.Buffer
ctx := context{
stdout: &stdout,
stderr: &bytes.Buffer{},
cwd: root,
args: []string{"build-haks", "--verbose"},
}
if err := runBuildHAKs(ctx); err != nil {
t.Fatalf("runBuildHAKs failed: %v", err)
}
output := stdout.String()
if !strings.Contains(output, "AleandAnecdotes.mp3") || !strings.Contains(output, "mus_wg_andnc.bmu") {
t.Fatalf("expected verbose mapping output, got %q", output)
}
}
func TestTopdataConsoleSuppressesProgressInNormalMode(t *testing.T) {
var stdout bytes.Buffer
console := &topdataConsole{
@@ -348,77 +287,6 @@ func TestProjectConsoleEmitsRelativePaths(t *testing.T) {
}
}
func TestRunMusicScanUsesConfiguredDatasetsWithoutWriting(t *testing.T) {
root := t.TempDir()
mkdirAll(t, filepath.Join(root, "assets", "audio", "westgate"))
mkdirAll(t, filepath.Join(root, "build"))
writeFile(t, filepath.Join(root, "nwn-tool.yaml"), `
module:
name: Test Module
resref: testmod
paths:
assets: assets
build: build
music:
datasets:
westgate_audio:
source: audio/westgate
output: generated/music
prefix: wg_
`)
writeFile(t, filepath.Join(root, "assets", "audio", "westgate", "Theme Song.mp3"), "source-mp3")
ctx := context{
stdout: &bytes.Buffer{},
stderr: &bytes.Buffer{},
cwd: root,
args: []string{"music", "scan", "--dataset", "westgate_audio"},
}
if err := runMusic(ctx); err != nil {
t.Fatalf("runMusic failed: %v", err)
}
if _, err := os.Stat(filepath.Join(root, ".cache", "credits")); !os.IsNotExist(err) {
t.Fatalf("music scan should not write credits artifacts, stat err=%v", err)
}
}
func TestRunMusicListDatasets(t *testing.T) {
root := t.TempDir()
mkdirAll(t, filepath.Join(root, "assets"))
mkdirAll(t, filepath.Join(root, "build"))
writeFile(t, filepath.Join(root, "nwn-tool.yaml"), `
module:
name: Test Module
resref: testmod
paths:
assets: assets
music:
datasets:
westgate_audio:
source: audio/westgate
output: generated/music
prefix: wg_
`)
var stdout bytes.Buffer
ctx := context{
stdout: &stdout,
stderr: &bytes.Buffer{},
cwd: root,
args: []string{"music", "list-datasets"},
}
if err := runMusic(ctx); err != nil {
t.Fatalf("runMusic failed: %v", err)
}
for _, value := range []string{"westgate_audio", "audio/westgate", "generated/music", "wg_"} {
if !strings.Contains(stdout.String(), value) {
t.Errorf("dataset list missing configured value %q: %q", value, stdout.String())
}
}
}
func TestRunConfigEffectiveJSONShowsDefaultsAndProvenance(t *testing.T) {
root := t.TempDir()
writeFile(t, filepath.Join(root, "nwn-tool.yaml"), `
@@ -539,9 +407,6 @@ func TestInlineFlagParsersRejectEmptyValues(t *testing.T) {
if _, err := parseBuildHAKArgs([]string{"--hak="}); err == nil {
t.Fatalf("expected empty --hak inline value error, got %v", err)
}
if _, err := parseMusicCommandArgs([]string{"--dataset="}); err == nil {
t.Fatalf("expected empty --dataset inline value error, got %v", err)
}
if _, err := parseDeployWikiArgs("deploy-wiki", []string{"--endpoint="}); err == nil {
t.Fatalf("expected empty --endpoint inline value error, got %v", err)
}
+224 -28
View File
@@ -2,7 +2,7 @@
// builders shared by the `crucible` dispatcher and the standalone
// `crucible-<name>` shims.
//
// Cutover state (Phase 5 → 6): the internal pipeline/topdata/erf/wiki/music
// Cutover state (Phase 5 → 6): the internal pipeline/topdata/erf/wiki
// packages migrated from gitea/sow-tools now live in this tree, so wired
// builders delegate to internal/app's legacy command surface (mapped per
// docs/command-surface.md). A builder with no migrated logic yet (depot) keeps
@@ -32,22 +32,56 @@ type Builder struct {
Name string // canonical dispatcher subcommand, e.g. "module"
Bin string // standalone binary name, e.g. "crucible-module"
Summary string // one-line description
Legacy []string // nwn-tool commands this subsumes (migration aid; see docs/command-surface.md)
Extra []string // additional callable subcommands beyond Legacy (e.g. cross-listed "music")
Commands []Command // visible commands plus hidden compatibility aliases
Wired bool // true once the builder delegates to migrated internal/app logic
}
// subcommands returns every legacy command a wired builder accepts: the
// canonical Legacy set plus any cross-listed Extra commands.
func (b Builder) subcommands() []string { return append(append([]string{}, b.Legacy...), b.Extra...) }
// Command is one public builder action. AppCommand names the unchanged
// internal/app implementation command. Aliases remain callable for compatibility
// but are omitted from routine help and the interactive menu.
type Command struct {
Name string
Summary string
AppCommand string
Usage string
Options []string
Aliases []CommandAlias
}
type CommandAlias struct {
Name string
AppCommand string
}
func (b Builder) subcommands() []string {
commands := make([]string, 0, len(b.Commands))
for _, command := range b.Commands {
commands = append(commands, command.Name)
}
return commands
}
func (b Builder) command(name string) (Command, string, bool) {
for _, command := range b.Commands {
if command.Name == name {
return command, command.AppCommand, true
}
for _, alias := range command.Aliases {
if alias.Name == name {
target := alias.AppCommand
if target == "" {
target = command.AppCommand
}
return command, target, true
}
}
}
return Command{}, "", false
}
func (b Builder) accepts(sub string) bool {
for _, s := range b.subcommands() {
if s == sub {
return true
}
}
return false
_, _, ok := b.command(sub)
return ok
}
// Registry is the single source of truth for the Crucible command surface.
@@ -57,39 +91,171 @@ var Registry = []Builder{
Name: "depot",
Bin: "crucible-depot",
Summary: "verify/move content-addressed depot blobs (SeaweedFS)",
Legacy: nil,
Wired: false, // no migrated logic yet; fails closed (exit 70)
},
{
Name: "hak",
Bin: "crucible-hak",
Summary: "pack/unpack ERF/HAK archives + hak manifests",
Legacy: []string{"build-haks", "apply-hak-manifest"},
Extra: []string{"music"}, // music BMUs are packed into HAKs (command-surface.md)
Commands: []Command{
{
Name: "build",
Summary: "build configured HAK archives and their manifest",
AppCommand: "build-haks",
Usage: "crucible hak build [options]",
Options: []string{
"--hak <name> build selected configured HAKs",
"--archive <name> build selected archive members",
"--source-manifest <path> read a generated source manifest",
"--content-addressed-root <path> resolve source-manifest blobs here",
"--plan-only plan outputs without writing archives",
"--quiet | --verbose | --debug select output detail",
},
Aliases: []CommandAlias{{Name: "build-haks"}},
},
{
Name: "manifest",
Summary: "apply a generated HAK list to the module source",
AppCommand: "apply-hak-manifest",
Usage: "crucible hak manifest [manifest-path]",
Aliases: []CommandAlias{{Name: "apply-hak-manifest"}},
},
},
Wired: true,
},
{
Name: "module",
Bin: "crucible-module",
Summary: "build/extract/validate/compare the .mod",
Legacy: []string{"build", "build-module", "extract", "validate", "compare"},
// apply-hak-manifest is canonically a hak command but reachable here too;
// music is folded into the module build pipeline (command-surface.md).
Extra: []string{"apply-hak-manifest", "music"},
Commands: []Command{
{
Name: "build",
Summary: "build the module and configured project outputs",
AppCommand: "build",
Usage: "crucible module build [--quiet|--verbose|--debug]",
Aliases: []CommandAlias{{Name: "build-module", AppCommand: "build-module"}},
},
{
Name: "extract",
Summary: "extract built archives into configured source trees",
AppCommand: "extract",
Usage: "crucible module extract [resource ...]",
},
{
Name: "validate",
Summary: "validate project layout, config, and source inventory",
AppCommand: "validate",
Usage: "crucible module validate",
},
{
Name: "compare",
Summary: "compare source content with built module and HAK archives",
AppCommand: "compare",
Usage: "crucible module compare",
},
{
Name: "manifest",
Summary: "apply a generated HAK list to the module source",
AppCommand: "apply-hak-manifest",
Usage: "crucible module manifest [manifest-path]",
Aliases: []CommandAlias{{Name: "apply-hak-manifest"}},
},
},
Wired: true,
},
{
Name: "topdata",
Bin: "crucible-topdata",
Summary: "compile 2da/tlk topdata + packages",
Legacy: []string{"validate-topdata", "build-topdata", "build-top-package", "compare-topdata", "convert-topdata"},
Commands: []Command{
{
Name: "validate",
Summary: "validate topdata layout and canonical JSON compatibility",
AppCommand: "validate-topdata",
Usage: "crucible topdata validate",
Aliases: []CommandAlias{{Name: "validate-topdata"}},
},
{
Name: "build",
Summary: "compile topdata and refresh the configured package",
AppCommand: "build-topdata",
Usage: "crucible topdata build [--force] [--wiki] [--skip-lfs]",
Options: []string{
"--force rebuild outputs even when current",
"--wiki build wiki drafts after compilation",
"--skip-lfs skip Git LFS materialization",
},
Aliases: []CommandAlias{{Name: "build-topdata"}},
},
{
Name: "package",
Summary: "package cached topdata outputs into HAK and TLK files",
AppCommand: "build-top-package",
Usage: "crucible topdata package [--force] [--skip-lfs]",
Options: []string{
"--force rebuild outputs even when current",
"--skip-lfs skip Git LFS materialization",
},
Aliases: []CommandAlias{{Name: "build-top-package"}},
},
{
Name: "compare",
Summary: "compare current output with a fresh native build",
AppCommand: "compare-topdata",
Usage: "crucible topdata compare",
Aliases: []CommandAlias{{Name: "compare-topdata"}},
},
{
Name: "convert",
Summary: "convert between 2DA and native JSON/module formats",
AppCommand: "convert-topdata",
Usage: "crucible topdata convert <2da-to-json|2da-to-module|json-to-2da> ...",
Options: []string{
"2da-to-json <input.2da> <output.json>",
"2da-to-module [flags] <input.2da> [output.json]",
"json-to-2da <input.json> <output.2da>",
},
Aliases: []CommandAlias{{Name: "convert-topdata"}},
},
},
Wired: true,
},
{
Name: "wiki",
Bin: "crucible-wiki",
Summary: "render + deploy mechanical wiki pages",
Legacy: []string{"build-wiki", "deploy-wiki"},
Commands: []Command{
{
Name: "build",
Summary: "render wiki page drafts from compiled topdata",
AppCommand: "build-wiki",
Usage: "crucible wiki build [--force]",
Options: []string{"--force rebuild drafts even when current"},
Aliases: []CommandAlias{{Name: "build-wiki"}},
},
{
Name: "deploy",
Summary: "deploy generated wiki pages to NodeBB",
AppCommand: "deploy-wiki",
Usage: "crucible wiki deploy [options]",
Options: []string{
"--source-dir <path> read generated pages here",
"--endpoint <url> override the NodeBB endpoint",
"--token <token> authenticate with an API token",
"--username/--password <value> authenticate with credentials",
"--version <version> label the deployed revision",
"--namespace <name> deploy selected namespaces",
"--category <namespace=id> override a namespace category",
"--manifest <path> write deployment state here",
"--stale-policy <policy> report, archive, or purge",
"--dry-run report changes without writing",
"--create allow missing pages to be created",
"--force update unchanged pages",
"--reset-managed-namespaces reset managed namespace state",
},
Aliases: []CommandAlias{{Name: "deploy-wiki"}},
},
},
Wired: true,
},
}
@@ -128,10 +294,14 @@ func menuItems() []menu.Item {
continue
}
for _, sub := range b.subcommands() {
command, _, _ := b.command(sub)
items = append(items, menu.Item{
Label: b.Name + " " + sub,
Desc: b.Summary,
Desc: command.Summary,
Args: []string{b.Name, sub},
Usage: command.Usage,
Options: append([]string(nil),
command.Options...),
})
}
}
@@ -206,14 +376,21 @@ func runBuilder(name string, args []string, out, errw io.Writer) int {
return exitUsage
}
sub := args[0]
if !b.accepts(sub) {
command, appCommand, ok := b.command(sub)
if !ok {
fmt.Fprintf(errw, "crucible %s: unknown subcommand %q\n\n", b.Name, sub)
builderHelp(errw, b)
return exitUsage
}
// Delegate to the migrated legacy command surface. args[0] is already the
// legacy command name, so it is forwarded unchanged.
return delegateLegacy(args, errw)
if len(args) > 1 {
switch args[1] {
case "-h", "--help", "help":
commandHelp(out, command)
return exitOK
}
}
legacyArgs := append([]string{appCommand}, args[1:]...)
return delegateLegacy(legacyArgs, errw)
}
// delegateLegacy runs a migrated nwn-tool command via internal/app and maps its
@@ -269,8 +446,27 @@ func builderHelp(w io.Writer, b Builder) {
}
fmt.Fprintf(w, "usage: crucible %s <subcommand> [args] (or: %s <subcommand> [args])\n\n", b.Name, b.Bin)
fmt.Fprintf(w, "subcommands:\n")
for _, c := range b.subcommands() {
fmt.Fprintf(w, " %s\n", c)
width := 0
for _, command := range b.Commands {
if len(command.Name) > width {
width = len(command.Name)
}
}
for _, command := range b.Commands {
fmt.Fprintf(w, " %-*s %s\n", width, command.Name, command.Summary)
}
fmt.Fprintf(w, "\nstatus: wired to migrated nwn-tool logic. See docs/command-surface.md.\n")
}
func commandHelp(w io.Writer, command Command) {
fmt.Fprintf(w, "%s\n", command.Summary)
if command.Usage != "" {
fmt.Fprintf(w, "\nusage: %s\n", command.Usage)
}
if len(command.Options) > 0 {
fmt.Fprintln(w, "\noptions:")
for _, option := range command.Options {
fmt.Fprintf(w, " %s\n", option)
}
}
}
+127 -17
View File
@@ -148,28 +148,138 @@ func TestBuilderHelpIsOK(t *testing.T) {
}
}
// Every legacy command named in command-surface.md must have exactly one
// canonical home (Builder.Legacy), so the migrated surface has no gaps or
// ambiguous homes.
func TestLegacyCommandsHaveExactlyOneHome(t *testing.T) {
legacy := []string{
"build", "build-module", "extract", "validate", "compare",
"build-haks", "apply-hak-manifest",
"validate-topdata", "build-topdata", "build-top-package", "compare-topdata", "convert-topdata",
"build-wiki", "deploy-wiki",
func TestCanonicalCommandSurface(t *testing.T) {
want := map[string][]string{
"depot": nil,
"hak": {"build", "manifest"},
"module": {"build", "extract", "validate", "compare", "manifest"},
"topdata": {"validate", "build", "package", "compare", "convert"},
"wiki": {"build", "deploy"},
}
for _, cmd := range legacy {
homes := 0
for _, b := range Registry {
for _, c := range b.Legacy {
if c == cmd {
homes++
for _, builder := range Registry {
got := builder.subcommands()
expected, ok := want[builder.Name]
if !ok {
t.Fatalf("unexpected builder %q", builder.Name)
}
if strings.Join(got, ",") != strings.Join(expected, ",") {
t.Errorf("%s commands = %v, want %v", builder.Name, got, expected)
}
}
}
if homes != 1 {
t.Errorf("legacy command %q has %d canonical homes, want 1", cmd, homes)
func TestRegistryCommandNamesAndAliasesAreUnambiguous(t *testing.T) {
for _, builder := range Registry {
seen := map[string]bool{}
for _, command := range builder.Commands {
if command.Name == "" || command.Summary == "" || command.AppCommand == "" || command.Usage == "" {
t.Errorf("%s has incomplete command metadata: %#v", builder.Name, command)
}
if seen[command.Name] {
t.Errorf("%s repeats command name %q", builder.Name, command.Name)
}
seen[command.Name] = true
for _, alias := range command.Aliases {
if alias.Name == "" {
t.Errorf("%s %s has an empty alias", builder.Name, command.Name)
continue
}
if seen[alias.Name] {
t.Errorf("%s repeats command or alias %q", builder.Name, alias.Name)
}
seen[alias.Name] = true
}
}
}
}
func TestMusicCommandsAreNotAccepted(t *testing.T) {
for _, builderName := range []string{"hak", "module"} {
builder, ok := find(builderName)
if !ok {
t.Fatalf("missing builder %q", builderName)
}
if builder.accepts("music") {
t.Errorf("%s must not accept the removed music command", builderName)
}
}
}
func TestHiddenAliasesPreserveImplementationTargets(t *testing.T) {
tests := []struct {
builder string
alias string
target string
}{
{"hak", "build-haks", "build-haks"},
{"hak", "apply-hak-manifest", "apply-hak-manifest"},
{"module", "build-module", "build-module"},
{"module", "apply-hak-manifest", "apply-hak-manifest"},
{"topdata", "validate-topdata", "validate-topdata"},
{"topdata", "build-topdata", "build-topdata"},
{"topdata", "build-top-package", "build-top-package"},
{"topdata", "compare-topdata", "compare-topdata"},
{"topdata", "convert-topdata", "convert-topdata"},
{"wiki", "build-wiki", "build-wiki"},
{"wiki", "deploy-wiki", "deploy-wiki"},
}
for _, tt := range tests {
t.Run(tt.builder+"/"+tt.alias, func(t *testing.T) {
builder, ok := find(tt.builder)
if !ok {
t.Fatalf("missing builder %q", tt.builder)
}
command, target, ok := builder.command(tt.alias)
if !ok {
t.Fatalf("hidden alias %q is not accepted", tt.alias)
}
if target != tt.target {
t.Fatalf("alias %q target = %q, want %q", tt.alias, target, tt.target)
}
if command.Name == tt.alias {
t.Fatalf("alias %q must not be a visible command", tt.alias)
}
})
}
}
func TestBuilderHelpHidesCompatibilityAliases(t *testing.T) {
aliases := map[string][]string{
"hak": {"build-haks", "apply-hak-manifest"},
"module": {"build-module", "apply-hak-manifest"},
"topdata": {"validate-topdata", "build-topdata", "build-top-package", "compare-topdata", "convert-topdata"},
"wiki": {"build-wiki", "deploy-wiki"},
}
for builderName, hidden := range aliases {
var out, errw bytes.Buffer
if code := runBuilder(builderName, []string{"--help"}, &out, &errw); code != exitOK {
t.Fatalf("%s help exit = %d", builderName, code)
}
for _, alias := range hidden {
if strings.Contains(out.String(), alias) {
t.Errorf("%s help exposed hidden alias %q:\n%s", builderName, alias, out.String())
}
}
}
}
func TestCommandHelpUsesCanonicalGuidanceWithoutLoadingProject(t *testing.T) {
var out, errw bytes.Buffer
if code := runBuilder("topdata", []string{"build", "--help"}, &out, &errw); code != exitOK {
t.Fatalf("command help exit = %d, stderr:\n%s", code, errw.String())
}
for _, want := range []string{
"compile topdata",
"usage: crucible topdata build",
"--force",
"--skip-lfs",
} {
if !strings.Contains(out.String(), want) {
t.Errorf("command help missing %q:\n%s", want, out.String())
}
}
if errw.Len() != 0 {
t.Fatalf("command help must not load a project or emit errors:\n%s", errw.String())
}
}
+21 -4
View File
@@ -15,8 +15,10 @@ import (
// Item is one selectable command.
type Item struct {
Label string // e.g. "module build"
Desc string // e.g. "build/extract/validate/compare the .mod"
Desc string // e.g. "build the configured module"
Args []string // dispatcher args, e.g. {"module", "build"}
Usage string
Options []string
}
// ANSI styling; terminals that don't support it simply ignore the codes.
@@ -35,9 +37,15 @@ func Select(out io.Writer, in io.Reader, items []Item) ([]string, error) {
return nil, fmt.Errorf("no commands available")
}
fmt.Fprintf(out, "%s%sCrucible%s — pick a command:\n\n", cBold, cCyan, cReset)
labelWidth := 0
for _, item := range items {
if len(item.Label) > labelWidth {
labelWidth = len(item.Label)
}
}
for i, it := range items {
fmt.Fprintf(out, " %s%2d%s %s%-24s%s %s%s%s\n",
cCyan, i+1, cReset, cBold, it.Label, cReset, cDim, it.Desc, cReset)
fmt.Fprintf(out, " %s%2d%s %s%-*s%s %s%s%s\n",
cCyan, i+1, cReset, cBold, labelWidth, it.Label, cReset, cDim, it.Desc, cReset)
}
fmt.Fprintf(out, " %sq%s quit\n\nselection: ", cCyan, cReset)
@@ -52,7 +60,16 @@ func Select(out io.Writer, in io.Reader, items []Item) ([]string, error) {
return nil, fmt.Errorf("invalid selection %q", choice)
}
it := items[n-1]
fmt.Fprintf(out, "extra args for %q (optional): ", it.Label)
if it.Usage != "" {
fmt.Fprintf(out, "\nusage: %s\n", it.Usage)
}
if len(it.Options) > 0 {
fmt.Fprintln(out, "options:")
for _, option := range it.Options {
fmt.Fprintf(out, " %s\n", option)
}
}
fmt.Fprint(out, "\narguments (press Enter to use defaults): ")
argLine, _ := r.ReadString('\n')
extra := strings.Fields(strings.TrimSpace(argLine))
return append(append([]string{}, it.Args...), extra...), nil
+57
View File
@@ -1,8 +1,10 @@
package menu
import (
"bytes"
"io"
"reflect"
"regexp"
"strings"
"testing"
)
@@ -45,3 +47,58 @@ func TestSelectInvalidErrors(t *testing.T) {
t.Fatal("out-of-range selection should error")
}
}
func TestSelectAlignsDescriptionsAfterLongestLabel(t *testing.T) {
items := []Item{
{Label: "hak build", Desc: "build haks", Args: []string{"hak", "build"}},
{Label: "topdata exceptionally-long-command", Desc: "long command", Args: []string{"topdata", "long"}},
}
var out bytes.Buffer
if _, err := Select(&out, strings.NewReader("q\n"), items); err != nil {
t.Fatalf("select: %v", err)
}
text := stripANSI(out.String())
var positions []int
for _, line := range strings.Split(text, "\n") {
for _, desc := range []string{"build haks", "long command"} {
if index := strings.Index(line, desc); index >= 0 {
positions = append(positions, index)
}
}
}
if len(positions) != 2 {
t.Fatalf("expected two menu descriptions, got %d:\n%s", len(positions), text)
}
if positions[0] != positions[1] {
t.Fatalf("descriptions are not aligned: columns %v\n%s", positions, text)
}
}
func TestSelectShowsCommandGuidanceBeforeArgumentsPrompt(t *testing.T) {
items := []Item{{
Label: "topdata build",
Desc: "compile topdata",
Args: []string{"topdata", "build"},
Usage: "crucible topdata build [--force]",
Options: []string{"--force rebuild outputs"},
}}
var out bytes.Buffer
if _, err := Select(&out, strings.NewReader("1\n\n"), items); err != nil {
t.Fatalf("select: %v", err)
}
text := stripANSI(out.String())
for _, want := range []string{
"usage: crucible topdata build [--force]",
"options:",
"--force rebuild outputs",
"arguments (press Enter to use defaults):",
} {
if !strings.Contains(text, want) {
t.Errorf("selected command guidance missing %q:\n%s", want, text)
}
}
}
func stripANSI(value string) string {
return regexp.MustCompile(`\x1b\[[0-9;]*m`).ReplaceAllString(value, "")
}
-47
View File
@@ -1,47 +0,0 @@
package music
import (
"fmt"
"path/filepath"
"strings"
)
type DatasetConfig struct {
Source string `json:"source" yaml:"source"`
Prefix string `json:"prefix,omitempty" yaml:"prefix,omitempty"`
StageRoot string `json:"stage_root,omitempty" yaml:"stage_root,omitempty"`
CreditsRoot string `json:"credits_root,omitempty" yaml:"credits_root,omitempty"`
}
type DefaultsConfig struct {
StageRoot string `json:"stage_root,omitempty" yaml:"stage_root,omitempty"`
CreditsRoot string `json:"credits_root,omitempty" yaml:"credits_root,omitempty"`
CreditsOverlay string `json:"credits_overlay,omitempty" yaml:"credits_overlay,omitempty"`
MaxStemLength *int `json:"max_stem_length,omitempty" yaml:"max_stem_length,omitempty"`
}
func ValidateDatasetSource(field, source string) error {
trimmed := strings.TrimSpace(source)
if trimmed == "" {
return fmt.Errorf("%s.source is required", field)
}
if strings.Contains(trimmed, "\x00") {
return fmt.Errorf("%s.source must not contain NUL bytes", field)
}
clean := filepath.Clean(filepath.FromSlash(trimmed))
if clean == ".." || strings.HasPrefix(clean, ".."+string(filepath.Separator)) {
return fmt.Errorf("%s.source must not escape project root", field)
}
return nil
}
func ValidateDatasetID(id string) error {
trimmed := strings.TrimSpace(id)
if trimmed == "" {
return fmt.Errorf("dataset id must not be empty")
}
if strings.Contains(trimmed, "/") || strings.Contains(trimmed, "\\") {
return fmt.Errorf("dataset id %q must not contain path separators", trimmed)
}
return nil
}
-292
View File
@@ -1,292 +0,0 @@
package music
import (
"fmt"
"os"
"path/filepath"
"strings"
)
func ParseCreditsOverlay(path string) (CreditsOverlay, error) {
entries, err := ParseCreditsMarkdown(path)
if err != nil {
return CreditsOverlay{}, err
}
overlay := CreditsOverlay{
ByOriginal: make(map[string]CreditsEntry),
ByOutput: make(map[string]CreditsEntry),
Entries: entries,
}
for _, entry := range entries {
if entry.OriginalFile != "" {
key := strings.ToLower(strings.TrimSpace(entry.OriginalFile))
if _, exists := overlay.ByOriginal[key]; exists {
return CreditsOverlay{}, fmt.Errorf("duplicate manual credits row for original file %s", entry.OriginalFile)
}
overlay.ByOriginal[key] = entry
}
if entry.OutputFile != "" {
key := strings.ToLower(strings.TrimSpace(entry.OutputFile))
if _, exists := overlay.ByOutput[key]; exists {
return CreditsOverlay{}, fmt.Errorf("duplicate manual credits row for output file %s", entry.OutputFile)
}
overlay.ByOutput[key] = entry
}
}
return overlay, nil
}
func (o CreditsOverlay) Match(originalFile, outputFile string) *CreditsEntry {
if entry, ok := o.ByOutput[strings.ToLower(strings.TrimSpace(outputFile))]; ok {
entryCopy := entry
return &entryCopy
}
if entry, ok := o.ByOriginal[strings.ToLower(strings.TrimSpace(originalFile))]; ok {
entryCopy := entry
return &entryCopy
}
return nil
}
func ValidateOverlayEntries(dir string, overlay CreditsOverlay, generated []CreditsEntry) error {
if len(overlay.Entries) == 0 {
return nil
}
validOriginal := make(map[string]struct{}, len(generated))
validOutput := make(map[string]struct{}, len(generated))
for _, entry := range generated {
validOriginal[strings.ToLower(entry.OriginalFile)] = struct{}{}
validOutput[strings.ToLower(entry.OutputFile)] = struct{}{}
}
for _, entry := range overlay.Entries {
if entry.OriginalFile != "" {
if _, ok := validOriginal[strings.ToLower(entry.OriginalFile)]; !ok {
return fmt.Errorf("manual credits entry in %s references unknown original file %s", dir, entry.OriginalFile)
}
}
if entry.OutputFile != "" {
if _, ok := validOutput[strings.ToLower(entry.OutputFile)]; !ok {
return fmt.Errorf("manual credits entry in %s references unknown output file %s", dir, entry.OutputFile)
}
}
}
return nil
}
func ParseCreditsMarkdown(path string) ([]CreditsEntry, error) {
raw, err := os.ReadFile(path)
if err != nil {
if os.IsNotExist(err) {
return nil, nil
}
return nil, err
}
lines := strings.Split(strings.ReplaceAll(string(raw), "\r\n", "\n"), "\n")
header := []string(nil)
entries := make([]CreditsEntry, 0)
for _, line := range lines {
line = strings.TrimSpace(line)
if !strings.HasPrefix(line, "|") {
continue
}
cells := ParseMarkdownTableRow(line)
if len(cells) == 0 {
continue
}
if header == nil {
header = make([]string, len(cells))
for i, cell := range cells {
header[i] = NormalizeCreditsHeader(cell)
}
continue
}
if IsMarkdownSeparatorRow(cells) {
continue
}
entry := CreditsEntry{}
for i, key := range header {
if i >= len(cells) {
continue
}
value := CleanCreditsCell(cells[i])
switch key {
case "artist":
entry.Artist = value
case "title":
entry.Title = value
case "output_file":
entry.OutputFile = value
case "original_file":
entry.OriginalFile = value
case "album":
entry.Album = value
case "date":
entry.Date = value
case "rights":
entry.Rights = value
case "notes":
entry.Notes = value
}
}
if entry.Artist == "" && entry.Title == "" && entry.OutputFile == "" && entry.OriginalFile == "" {
continue
}
entries = append(entries, entry)
}
return entries, nil
}
func ParseMarkdownTableRow(line string) []string {
trimmed := strings.TrimSpace(line)
trimmed = strings.TrimPrefix(trimmed, "|")
trimmed = strings.TrimSuffix(trimmed, "|")
parts := strings.Split(trimmed, "|")
cells := make([]string, 0, len(parts))
for _, part := range parts {
cells = append(cells, strings.TrimSpace(strings.ReplaceAll(part, `\|`, "|")))
}
return cells
}
func IsMarkdownSeparatorRow(cells []string) bool {
if len(cells) == 0 {
return false
}
for _, cell := range cells {
cell = strings.TrimSpace(cell)
cell = strings.ReplaceAll(cell, "-", "")
cell = strings.ReplaceAll(cell, ":", "")
if cell != "" {
return false
}
}
return true
}
func NormalizeCreditsHeader(value string) string {
switch strings.ToLower(CleanCreditsCell(value)) {
case "artist":
return "artist"
case "title":
return "title"
case "output file":
return "output_file"
case "original file":
return "original_file"
case "album":
return "album"
case "date":
return "date"
case "license / copyright":
return "rights"
case "notes":
return "notes"
default:
return ""
}
}
func CleanCreditsCell(value string) string {
value = strings.TrimSpace(value)
value = strings.Trim(value, "`")
value = strings.ReplaceAll(value, "<br>", "\n")
return strings.TrimSpace(value)
}
func RenderCreditsMarkdown(entries []CreditsEntry) string {
var builder strings.Builder
builder.WriteString(CreditsHeader)
for _, entry := range entries {
builder.WriteString("| ")
builder.WriteString(EscapeMarkdownCell(entry.Artist))
builder.WriteString(" | ")
builder.WriteString(EscapeMarkdownCell(entry.Title))
builder.WriteString(" | ")
builder.WriteString(EscapeMarkdownCell(WrapCodeCell(entry.OutputFile)))
builder.WriteString(" | ")
builder.WriteString(EscapeMarkdownCell(WrapCodeCell(entry.OriginalFile)))
builder.WriteString(" | ")
builder.WriteString(EscapeMarkdownCell(entry.Album))
builder.WriteString(" | ")
builder.WriteString(EscapeMarkdownCell(entry.Date))
builder.WriteString(" | ")
builder.WriteString(EscapeMarkdownCell(entry.Rights))
builder.WriteString(" | ")
builder.WriteString(EscapeMarkdownCell(entry.Notes))
builder.WriteString(" |\n")
}
return builder.String()
}
func WrapCodeCell(value string) string {
if strings.TrimSpace(value) == "" {
return ""
}
return "`" + value + "`"
}
func EscapeMarkdownCell(value string) string {
value = strings.ReplaceAll(value, `\`, `\\`)
value = strings.ReplaceAll(value, "|", `\|`)
value = strings.ReplaceAll(value, "\n", "<br>")
return value
}
func ApplyCreditsOverlay(entry *CreditsEntry, overlay *CreditsEntry) {
if overlay == nil {
return
}
entry.Artist = FirstNonEmpty(overlay.Artist, entry.Artist)
entry.Title = FirstNonEmpty(overlay.Title, entry.Title)
entry.Album = FirstNonEmpty(overlay.Album, entry.Album)
entry.Date = FirstNonEmpty(overlay.Date, entry.Date)
entry.Rights = FirstNonEmpty(overlay.Rights, entry.Rights)
entry.Notes = FirstNonEmpty(overlay.Notes, entry.Notes)
}
func SyncGeneratedCreditsArtifacts(root string, desiredFiles map[string][]byte) (int, error) {
changedFiles := 0
existingFiles := make(map[string]struct{})
if err := filepath.WalkDir(root, func(path string, d os.DirEntry, walkErr error) error {
if walkErr != nil {
if os.IsNotExist(walkErr) {
return nil
}
return walkErr
}
if d.IsDir() {
return nil
}
existingFiles[path] = struct{}{}
return nil
}); err != nil {
return 0, fmt.Errorf("scan credits dir: %w", err)
}
for path, content := range desiredFiles {
current, err := os.ReadFile(path)
if err != nil && !os.IsNotExist(err) {
return 0, fmt.Errorf("read credits artifact %s: %w", path, err)
}
if err == nil && string(current) == string(content) {
delete(existingFiles, path)
continue
}
if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil {
return 0, fmt.Errorf("create generated credits dir: %w", err)
}
if err := os.WriteFile(path, content, 0o644); err != nil {
return 0, fmt.Errorf("write credits artifact %s: %w", path, err)
}
changedFiles++
delete(existingFiles, path)
}
for path := range existingFiles {
if err := os.Remove(path); err != nil && !os.IsNotExist(err) {
return 0, fmt.Errorf("remove stale credits artifact %s: %w", path, err)
}
changedFiles++
}
return changedFiles, nil
}
-62
View File
@@ -1,62 +0,0 @@
package music
import (
"encoding/json"
"fmt"
"os/exec"
"strings"
)
func ProbeMetadata(ffprobePath, sourcePath string) (Metadata, error) {
cmd := exec.Command(ffprobePath, "-v", "quiet", "-print_format", "json", "-show_format", sourcePath)
output, err := cmd.Output()
if err != nil {
return Metadata{}, err
}
var payload struct {
Format struct {
Tags map[string]string `json:"tags"`
} `json:"format"`
}
if err := json.Unmarshal(output, &payload); err != nil {
return Metadata{}, err
}
tags := make(map[string]string, len(payload.Format.Tags))
for key, value := range payload.Format.Tags {
tags[strings.ToLower(strings.TrimSpace(key))] = strings.TrimSpace(value)
}
return Metadata{
Title: FirstNonEmpty(tags["title"]),
Artist: FirstNonEmpty(tags["artist"], tags["album_artist"], tags["composer"]),
Album: FirstNonEmpty(tags["album"]),
Date: FirstNonEmpty(tags["date"], tags["year"]),
Comment: FirstNonEmpty(tags["comment"], tags["description"]),
Copyright: FirstNonEmpty(tags["copyright"]),
License: FirstNonEmpty(tags["license"], tags["license_url"]),
}, nil
}
func ConvertSource(ffmpegPath, sourcePath, outputPath string) error {
cmd := exec.Command(
ffmpegPath, "-y",
"-i", sourcePath,
"-vn",
"-map_metadata", "-1",
"-ar", "44100",
"-ac", "2",
"-b:a", "192k",
"-codec:a", "libmp3lame",
"-write_xing", "0",
"-f", "mp3",
outputPath,
)
output, err := cmd.CombinedOutput()
if err != nil {
message := strings.TrimSpace(string(output))
if message == "" {
message = err.Error()
}
return fmt.Errorf("%s", message)
}
return nil
}
-239
View File
@@ -1,239 +0,0 @@
package music
import (
"fmt"
"os"
"os/exec"
"path/filepath"
"regexp"
"sort"
"strings"
)
const (
MaxStemLen = 16
CreditsHeader = "# NWN Music Pack Credits\n\nOriginal rights remain with the respective composers and licensors.\n\n## Processed Files\n\n| Artist | Title | Output File | Original File | Album | Date | License / Copyright | Notes |\n|---|---|---|---|---|---|---|---|\n"
)
var (
defaultConvertExtensions = []string{
".flac",
".m4a",
".mp3",
".ogg",
".wav",
}
ConvertExtensions = map[string]struct{}{
".flac": {},
".m4a": {},
".mp3": {},
".ogg": {},
".wav": {},
}
PassthroughExtensions = map[string]struct{}{
".bmu": {},
".wav": {},
}
DropWords = map[string]struct{}{
"mp3": {}, "wav": {}, "flac": {}, "ogg": {}, "m4a": {},
"music": {}, "track": {}, "audio": {}, "song": {}, "soundtrack": {},
"final": {}, "version": {}, "ver": {}, "v": {}, "mix": {}, "master": {}, "remaster": {},
"free": {}, "royalty": {}, "copyright": {}, "background": {}, "bgm": {},
"loop": {}, "loops": {}, "loopable": {}, "seamless": {},
"official": {}, "download": {}, "preview": {},
"the": {}, "a": {}, "an": {}, "and": {}, "of": {}, "to": {}, "in": {}, "for": {}, "with": {}, "by": {},
}
BracketRE = regexp.MustCompile(`\[[^\]]*\]|\([^)]*\)|\{[^}]*\}`)
SplitRE = regexp.MustCompile(`[^A-Za-z0-9]+`)
YearRE = regexp.MustCompile(`^\d{4,}$`)
VowelRE = regexp.MustCompile(`[aeiou]`)
)
func DefaultConvertExtensions() []string {
return append([]string(nil), defaultConvertExtensions...)
}
type Metadata struct {
Title string
Artist string
Album string
Date string
Rights string
Notes string
Comment string
Copyright string
License string
}
type CreditsEntry struct {
Artist string `json:"artist"`
Title string `json:"title"`
OutputFile string `json:"output_file"`
OriginalFile string `json:"original_file"`
Album string `json:"album"`
Date string `json:"date"`
Rights string `json:"rights"`
Notes string `json:"notes"`
}
type CreditsSource struct {
Path string `json:"path"`
Kind string `json:"kind"`
Entries []CreditsEntry `json:"entries"`
}
type CreditsInventory struct {
Sources []CreditsSource `json:"sources"`
}
type CreditsOverlay struct {
ByOriginal map[string]CreditsEntry
ByOutput map[string]CreditsEntry
Entries []CreditsEntry
}
func ResolveFFmpeg() (string, error) {
return ResolveFFmpegWithConfig("")
}
func ResolveFFmpegWithConfig(configuredPath string) (string, error) {
if configured := strings.TrimSpace(configuredPath); configured != "" {
return configured, nil
}
if configured := strings.TrimSpace(os.Getenv("SOW_FFMPEG")); configured != "" {
return configured, nil
}
path, err := exec.LookPath("ffmpeg")
if err != nil {
return "", ffmpegMissingErr("ffmpeg")
}
return path, nil
}
func ResolveFFprobe() (string, error) {
return ResolveFFprobeWithConfig("")
}
func ResolveFFprobeWithConfig(configuredPath string) (string, error) {
if configured := strings.TrimSpace(configuredPath); configured != "" {
return configured, nil
}
if configured := strings.TrimSpace(os.Getenv("SOW_FFPROBE")); configured != "" {
return configured, nil
}
path, err := exec.LookPath("ffprobe")
if err != nil {
return "", ffmpegMissingErr("ffprobe")
}
return path, nil
}
// ffmpegMissingErr renders an actionable, cross-platform "tool not found"
// message. Only the music builder needs ffmpeg/ffprobe; every other builder
// runs with zero external dependencies.
func ffmpegMissingErr(tool string) error {
return fmt.Errorf(`%[1]s not found on PATH.
Windows: winget install Gyan.FFmpeg
macOS: brew install ffmpeg
Linux: apt install ffmpeg (or your distro's package manager)
Only the music builder needs %[1]s; set SOW_%[2]s to a full path to override.`,
tool, strings.ToUpper(tool))
}
func IsMusicAssetPath(rel string) bool {
rel = filepath.ToSlash(strings.TrimSpace(rel))
return rel == "envi/music" || strings.HasPrefix(rel, "envi/music/")
}
func PathIsUnder(root, rel string) bool {
root = TrimSlashes(filepath.ToSlash(root))
rel = TrimSlashes(filepath.ToSlash(rel))
if root == "" || rel == "" {
return false
}
return rel == root || strings.HasPrefix(rel, root+"/")
}
func PrefixForDir(prefixes map[string]string, dir string) string {
dir = TrimSlashes(filepath.ToSlash(dir))
bestPrefix := ""
bestLen := -1
keys := make([]string, 0, len(prefixes))
for key := range prefixes {
keys = append(keys, key)
}
sort.Strings(keys)
for _, key := range keys {
value := prefixes[key]
normalizedKey := TrimSlashes(filepath.ToSlash(key))
if normalizedKey == "" {
continue
}
if dir != normalizedKey && !strings.HasPrefix(dir, normalizedKey+"/") {
continue
}
if len(normalizedKey) > bestLen {
bestLen = len(normalizedKey)
bestPrefix = SanitizePrefix(value)
}
}
return bestPrefix
}
func SanitizePrefix(prefix string) string {
prefix = strings.ToLower(strings.TrimSpace(prefix))
var builder strings.Builder
for _, r := range prefix {
switch {
case r >= 'a' && r <= 'z':
builder.WriteRune(r)
case r >= '0' && r <= '9':
builder.WriteRune(r)
case r == '_':
builder.WriteRune(r)
}
}
result := builder.String()
if strings.TrimSpace(prefix) != "" && strings.HasSuffix(strings.TrimSpace(prefix), "_") && !strings.HasSuffix(result, "_") {
result += "_"
}
return result
}
func TrimSlashes(value string) string {
return strings.Trim(strings.TrimSpace(value), "/")
}
func Min(a, b int) int {
if a < b {
return a
}
return b
}
func Max(a, b int) int {
if a > b {
return a
}
return b
}
func FirstNonEmpty(values ...string) string {
for _, value := range values {
if strings.TrimSpace(value) != "" {
return strings.TrimSpace(value)
}
}
return ""
}
func JoinNonEmpty(sep string, values ...string) string {
parts := make([]string, 0, len(values))
for _, value := range values {
value = strings.TrimSpace(value)
if value != "" {
parts = append(parts, value)
}
}
return strings.Join(parts, sep)
}
-368
View File
@@ -1,368 +0,0 @@
package music
import (
"os"
"path/filepath"
"strings"
"testing"
)
func TestUniqueNameHandlesShortStemCollisions(t *testing.T) {
used := map[string]struct{}{
"mus_wg_mystc": {},
}
got := UniqueName("mus_wg_mystc", used)
if got == "mus_wg_mystc" || len(got) > MaxStemLen {
t.Fatalf("collision result must be distinct and valid, got %q", got)
}
if _, ok := used[got]; !ok {
t.Fatalf("collision result was not reserved: %q", got)
}
}
func TestUniqueNameNoCollision(t *testing.T) {
used := map[string]struct{}{}
got := UniqueName("new_stem", used)
if got != "new_stem" {
t.Fatalf("expected 'new_stem', got %q", got)
}
}
func TestUniqueNameMultipleCollisions(t *testing.T) {
used := map[string]struct{}{
"stem": {},
"stem_1": {},
"stem_2": {},
}
got := UniqueName("stem", used)
if _, collided := map[string]bool{"stem": true, "stem_1": true, "stem_2": true}[got]; collided {
t.Fatalf("expected an unused collision result, got %q", got)
}
if _, ok := used[got]; !ok {
t.Fatalf("collision result was not reserved: %q", got)
}
}
func TestGenerateStem(t *testing.T) {
used := map[string]struct{}{
"existing": {},
}
stem, err := GenerateStem("My Cool Track.mp3", "mus_", used)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if stem == "" {
t.Fatal("expected non-empty stem")
}
if len(stem) > MaxStemLen {
t.Fatalf("stem %q exceeds max length %d", stem, MaxStemLen)
}
if _, exists := used[stem]; !exists {
t.Fatal("expected stem to be registered in used set")
}
}
func TestGenerateStemPrefixTooLong(t *testing.T) {
_, err := GenerateStem("test.mp3", "this_prefix_is_way_too_long_for_stem_", nil)
if err == nil {
t.Fatal("expected error for too-long prefix")
}
}
func TestSlugWordsStripsBrackets(t *testing.T) {
words := SlugWords("Song [Explicit] (Remix)")
for _, w := range words {
if w == "explicit" || w == "remix" {
t.Fatalf("bracket content should be removed, got word %q", w)
}
}
}
func TestSanitizePrefix(t *testing.T) {
for _, input := range []string{"Mus_WG_", " Test "} {
got := SanitizePrefix(input)
if got == "" || got != strings.ToLower(got) || strings.ContainsAny(got, " \t\r\n") {
t.Errorf("sanitized prefix contains uppercase or whitespace: %q", got)
}
}
if got := SanitizePrefix(""); got != "" {
t.Fatalf("expected empty prefix, got %q", got)
}
}
func TestReserveName(t *testing.T) {
used := map[string]struct{}{}
if err := ReserveName("testname", used); err != nil {
t.Fatalf("unexpected error: %v", err)
}
if _, exists := used["testname"]; !exists {
t.Fatal("expected name to be reserved")
}
if err := ReserveName("testname", used); err == nil {
t.Fatal("expected error for duplicate name")
}
}
func TestValidateManualOutputFile(t *testing.T) {
if err := ValidateManualOutputFile("test.bmu"); err != nil {
t.Fatalf("unexpected error: %v", err)
}
if err := ValidateManualOutputFile("test.txt"); err == nil {
t.Fatal("expected error for non-bmu extension")
}
if err := ValidateManualOutputFile(".bmu"); err == nil {
t.Fatal("expected error for empty stem")
}
}
func TestParseCreditsMarkdownMissingFile(t *testing.T) {
entries, err := ParseCreditsMarkdown("nonexistent.md")
if err != nil {
t.Fatalf("unexpected error for missing file: %v", err)
}
if entries != nil {
t.Fatalf("expected nil entries for missing file, got %v", entries)
}
}
func TestParseCreditsMarkdownRealContent(t *testing.T) {
content := "# Header\n\n| Artist | Title | Output File | Original File | Album | Date | License / Copyright | Notes |\n|---|---|---|---|---|---|---|---|\n| Test Artist | Test Title | `output.bmu` | `original.mp3` | Test Album | 2024 | MIT | test |\n"
path := filepath.Join(t.TempDir(), "CREDITS.md")
if err := os.WriteFile(path, []byte(content), 0o644); err != nil {
t.Fatalf("write test credits: %v", err)
}
entries, err := ParseCreditsMarkdown(path)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if len(entries) != 1 {
t.Fatalf("expected 1 entry, got %d", len(entries))
}
if entries[0].Artist != "Test Artist" {
t.Fatalf("expected 'Test Artist', got %q", entries[0].Artist)
}
if entries[0].Title != "Test Title" {
t.Fatalf("expected 'Test Title', got %q", entries[0].Title)
}
if entries[0].OutputFile != "output.bmu" {
t.Fatalf("expected 'output.bmu', got %q", entries[0].OutputFile)
}
}
func TestRenderCreditsMarkdown(t *testing.T) {
entries := []CreditsEntry{
{
Artist: "Test Artist",
Title: "Test Title",
OutputFile: "output.bmu",
OriginalFile: "original.mp3",
Album: "Test Album",
Date: "2024",
Rights: "MIT",
Notes: "test note",
},
}
result := RenderCreditsMarkdown(entries)
if !strings.Contains(result, "Test Artist") {
t.Fatal("expected rendered output to contain artist")
}
if !strings.Contains(result, "output.bmu") {
t.Fatal("expected rendered output to contain output file")
}
}
func TestParseCreditsOverlay(t *testing.T) {
content := "# Header\n\n| Artist | Title | Output File | Original File |\n|---|---|---|---|\n| Overlay Artist | Ovr Title | `override.bmu` | `original.mp3` |\n"
path := filepath.Join(t.TempDir(), "CREDITS.md")
if err := os.WriteFile(path, []byte(content), 0o644); err != nil {
t.Fatalf("write test overlay: %v", err)
}
overlay, err := ParseCreditsOverlay(path)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if len(overlay.Entries) != 1 {
t.Fatalf("expected 1 overlay entry, got %d", len(overlay.Entries))
}
matched := overlay.Match("original.mp3", "")
if matched == nil {
t.Fatal("expected overlay match by original file")
}
if matched.Artist != "Overlay Artist" {
t.Fatalf("expected 'Overlay Artist', got %q", matched.Artist)
}
}
func TestApplyCreditsOverlay(t *testing.T) {
entry := &CreditsEntry{
Artist: "Original Artist",
Title: "Original Title",
}
overlay := &CreditsEntry{
Artist: "Overlay Artist",
}
ApplyCreditsOverlay(entry, overlay)
if entry.Artist != "Overlay Artist" {
t.Fatalf("expected 'Overlay Artist', got %q", entry.Artist)
}
if entry.Title != "Original Title" {
t.Fatalf("expected 'Original Title' preserved, got %q", entry.Title)
}
ApplyCreditsOverlay(entry, nil)
if entry.Artist != "Overlay Artist" {
t.Fatalf("expected nil overlay to be no-op, got %q", entry.Artist)
}
}
func TestValidateOverlayEntries(t *testing.T) {
generated := []CreditsEntry{
{OriginalFile: "track1.mp3", OutputFile: "track1.bmu"},
}
overlay := CreditsOverlay{
Entries: []CreditsEntry{
{OriginalFile: "track1.mp3"},
},
}
if err := ValidateOverlayEntries("test", overlay, generated); err != nil {
t.Fatalf("unexpected validation error: %v", err)
}
badOverlay := CreditsOverlay{
Entries: []CreditsEntry{
{OriginalFile: "nonexistent.mp3"},
},
}
if err := ValidateOverlayEntries("test", badOverlay, generated); err == nil {
t.Fatal("expected validation error for unknown original file")
}
}
func TestIsMusicAssetPath(t *testing.T) {
if !IsMusicAssetPath("envi/music") {
t.Fatal("expected 'envi/music' to be music path")
}
if !IsMusicAssetPath("envi/music/westgate") {
t.Fatal("expected 'envi/music/westgate' to be music path")
}
if IsMusicAssetPath("envi/textures") {
t.Fatal("expected 'envi/textures' not to be music path")
}
if IsMusicAssetPath("") {
t.Fatal("expected empty string not to be music path")
}
}
func TestEscapeMarkdownCell(t *testing.T) {
result := EscapeMarkdownCell("a|b\nc")
if !strings.Contains(result, `\|`) {
t.Fatal("expected pipe to be escaped")
}
if !strings.Contains(result, "<br>") {
t.Fatal("expected newline to be replaced with <br>")
}
}
func TestSyncGeneratedCreditsArtifactsWritesNewFiles(t *testing.T) {
root := t.TempDir()
desired := map[string][]byte{
filepath.Join(root, "test.md"): []byte("content"),
}
changed, err := SyncGeneratedCreditsArtifacts(root, desired)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if changed != 1 {
t.Fatalf("expected 1 changed file, got %d", changed)
}
}
func TestSyncGeneratedCreditsArtifactsNoChanges(t *testing.T) {
root := t.TempDir()
path := filepath.Join(root, "test.md")
if err := os.WriteFile(path, []byte("content"), 0o644); err != nil {
t.Fatalf("write file: %v", err)
}
desired := map[string][]byte{
path: []byte("content"),
}
changed, err := SyncGeneratedCreditsArtifacts(root, desired)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if changed != 0 {
t.Fatalf("expected 0 changed files, got %d", changed)
}
}
func TestSyncGeneratedCreditsArtifactsRemovesStale(t *testing.T) {
root := t.TempDir()
stalePath := filepath.Join(root, "stale.md")
if err := os.WriteFile(stalePath, []byte("stale"), 0o644); err != nil {
t.Fatalf("write stale file: %v", err)
}
desired := map[string][]byte{}
changed, err := SyncGeneratedCreditsArtifacts(root, desired)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if changed != 1 {
t.Fatalf("expected 1 removed file, got %d", changed)
}
}
func TestPrefixForDir(t *testing.T) {
prefixes := map[string]string{
"envi/music/westgate": "mus_wg_",
}
if got := PrefixForDir(prefixes, "envi/music/westgate"); got != "mus_wg_" {
t.Fatalf("expected 'mus_wg_', got %q", got)
}
if got := PrefixForDir(prefixes, "envi/music/other"); got != "" {
t.Fatalf("expected empty prefix, got %q", got)
}
if got := PrefixForDir(nil, "envi/music"); got != "" {
t.Fatalf("expected empty prefix for nil map, got %q", got)
}
}
func TestResolveFFmpegEnv(t *testing.T) {
t.Setenv("SOW_FFMPEG", "/custom/ffmpeg")
path, err := ResolveFFmpeg()
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if path != "/custom/ffmpeg" {
t.Fatalf("expected '/custom/ffmpeg', got %q", path)
}
}
func TestResolveFFprobeEnv(t *testing.T) {
t.Setenv("SOW_FFPROBE", "/custom/ffprobe")
path, err := ResolveFFprobe()
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if path != "/custom/ffprobe" {
t.Fatalf("expected '/custom/ffprobe', got %q", path)
}
}
func TestFFmpegMissingErrHasInstallHints(t *testing.T) {
msg := ffmpegMissingErr("ffmpeg").Error()
for _, want := range []string{"ffmpeg", "SOW_FFMPEG"} {
if !strings.Contains(msg, want) {
t.Errorf("ffmpeg error missing %q; got:\n%s", want, msg)
}
}
probe := ffmpegMissingErr("ffprobe").Error()
if !strings.Contains(probe, "SOW_FFPROBE") {
t.Errorf("ffprobe error should mention SOW_FFPROBE; got:\n%s", probe)
}
}
-197
View File
@@ -1,197 +0,0 @@
package music
import (
"crypto/sha1"
"fmt"
"path/filepath"
"strings"
)
func GenerateStem(fileName, prefix string, used map[string]struct{}) (string, error) {
return GenerateStemWithMax(fileName, prefix, MaxStemLen, used)
}
func GenerateStemWithMax(fileName, prefix string, maxStemLen int, used map[string]struct{}) (string, error) {
if maxStemLen <= 0 {
maxStemLen = MaxStemLen
}
maxCore := maxStemLen - len(prefix)
if maxCore < 3 {
return "", fmt.Errorf("prefix too long: %q", prefix)
}
base := strings.TrimSuffix(fileName, filepath.Ext(fileName))
words := SlugWords(base)
core := MakeMusicCore(words, maxCore)
if core == "" {
sum := sha1.Sum([]byte(fileName))
core = fmt.Sprintf("%x", sum[:])[:maxCore]
}
stem := strings.TrimRight((prefix + core)[:Min(maxStemLen, len(prefix)+len(core))], "_")
if stem == "" {
return "", fmt.Errorf("could not derive music stem for %s", fileName)
}
return UniqueNameWithMax(stem, maxStemLen, used), nil
}
func SlugWords(value string) []string {
value = BracketRE.ReplaceAllString(value, " ")
value = strings.ReplaceAll(value, "&", " and ")
value = strings.ReplaceAll(value, "'", "")
value = strings.ReplaceAll(value, "\u2019", "")
value = strings.ToLower(value)
var ascii strings.Builder
for _, r := range value {
switch {
case r >= 'a' && r <= 'z':
ascii.WriteRune(r)
case r >= '0' && r <= '9':
ascii.WriteRune(r)
default:
ascii.WriteRune(' ')
}
}
rawWords := SplitRE.Split(ascii.String(), -1)
words := make([]string, 0, len(rawWords))
for _, word := range rawWords {
if word == "" {
continue
}
if _, drop := DropWords[word]; drop {
continue
}
if YearRE.MatchString(word) {
continue
}
words = append(words, word)
}
return words
}
func MakeMusicCore(words []string, maxCore int) string {
core := ""
for _, word := range words {
candidate := word
if core != "" {
candidate = core + "_" + word
}
if len(candidate) <= maxCore {
core = candidate
continue
}
break
}
if core != "" {
return core
}
abbrev := make([]string, 0, Min(4, len(words)))
for _, word := range words {
if len(abbrev) == 4 {
break
}
abbrev = append(abbrev, AbbreviateWord(word, 5))
}
for _, word := range abbrev {
candidate := word
if core != "" {
candidate = core + "_" + word
}
if len(candidate) <= maxCore {
core = candidate
continue
}
break
}
if core != "" {
return core
}
if len(words) > 0 {
return strings.Trim(words[0][:Min(maxCore, len(words[0]))], "_")
}
return ""
}
func AbbreviateWord(word string, maxLen int) string {
if len(word) <= maxLen {
return word
}
consonants := VowelRE.ReplaceAllString(word, "")
candidate := word[:1]
if len(consonants) > 1 {
candidate += consonants[1:]
}
if len(candidate) >= 3 {
return candidate[:Min(maxLen, len(candidate))]
}
return word[:Min(maxLen, len(word))]
}
func UniqueName(stem string, used map[string]struct{}) string {
return UniqueNameWithMax(stem, MaxStemLen, used)
}
func UniqueNameWithMax(stem string, maxStemLen int, used map[string]struct{}) string {
if maxStemLen <= 0 {
maxStemLen = MaxStemLen
}
if _, exists := used[stem]; !exists {
used[stem] = struct{}{}
return stem
}
for index := 1; ; index++ {
suffix := fmt.Sprintf("_%d", index)
prefixLen := Min(len(stem), Max(0, maxStemLen-len(suffix)))
candidate := strings.TrimRight(stem[:prefixLen], "_") + suffix
if _, exists := used[candidate]; exists {
continue
}
used[candidate] = struct{}{}
return candidate
}
}
func ReserveName(stem string, used map[string]struct{}) error {
stem = strings.ToLower(strings.TrimSpace(stem))
if stem == "" {
return fmt.Errorf("output filename is empty")
}
if _, exists := used[stem]; exists {
return fmt.Errorf("output filename %s collides with an existing asset", stem)
}
used[stem] = struct{}{}
return nil
}
func ValidateManualOutputFile(name string) error {
return ValidateManualOutputFileWithRules(name, ".bmu", MaxStemLen)
}
func ValidateManualOutputFileWithRules(name, extension string, maxStemLen int) error {
name = strings.ToLower(strings.TrimSpace(name))
extension = strings.ToLower(strings.TrimSpace(extension))
if extension == "" {
extension = ".bmu"
}
if maxStemLen <= 0 {
maxStemLen = MaxStemLen
}
if !strings.HasSuffix(name, extension) {
return fmt.Errorf("output file must end with %s", extension)
}
stem := strings.TrimSuffix(name, extension)
if stem == "" {
return fmt.Errorf("output file stem is empty")
}
if len(stem) > maxStemLen {
return fmt.Errorf("output file stem %q exceeds %d characters", stem, maxStemLen)
}
for _, r := range stem {
switch {
case r >= 'a' && r <= 'z':
case r >= '0' && r <= '9':
case r == '_':
default:
return fmt.Errorf("output file stem %q contains unsupported character %q", stem, string(r))
}
}
return nil
}
+5 -65
View File
@@ -26,8 +26,6 @@ type BuildResult struct {
HAKPaths []string
Manifest string
AutogenManifestPaths []string
CreditsArtifactPaths []string
CreditsSummary CreditsRefreshSummary
HAKSummary HAKArchiveSummary
Resources int
HAKAssets int
@@ -36,21 +34,6 @@ type BuildResult struct {
TopPackageFiles int
}
type CreditsRefreshSummary struct {
ScannedDirs []string
TrackCount int
ArtifactPaths []string
GeneratedPaths []string
InventoryPath string
ChangedFiles int
Mappings []MusicFileMapping
}
type MusicFileMapping struct {
Source string
Output string
}
type HAKArchiveSummary struct {
Total int
Reused int
@@ -110,8 +93,6 @@ type BuildHAKOptions struct {
ArchiveNames []string
SourceManifestPath string
ContentAddressedRoot string
SkipMusic bool
MusicDatasetIDs []string
}
func Build(p *project.Project) (BuildResult, error) {
@@ -241,26 +222,11 @@ func planOrBuildHAKs(p *project.Project, progress ProgressFunc, writeArchives bo
if err != nil {
return BuildResult{}, err
}
musicAssets := &preparedMusicAssets{SkipSourceRel: map[string]struct{}{}}
if !opts.SkipMusic {
musicAssets, err = prepareMusicAssetsWithOptions(p, musicPrepareOptions{
datasetIDs: opts.MusicDatasetIDs,
write: writeArchives,
})
if err != nil {
return BuildResult{}, err
}
}
defer func() {
if cleanupErr := cleanupPreparedMusicAssets(musicAssets); cleanupErr != nil && err == nil {
err = cleanupErr
}
}()
generated2DAAssets, err := topdata.BuildGenerated2DAAssets(p, progress)
if err != nil {
return BuildResult{}, err
}
assetResources, err := collectAssetResources(p, false, allowedAssets, musicAssets, generated2DAAssets)
assetResources, err := collectAssetResources(p, false, allowedAssets, generated2DAAssets)
if err != nil {
return BuildResult{}, err
}
@@ -277,8 +243,6 @@ func planOrBuildHAKs(p *project.Project, progress ProgressFunc, writeArchives bo
}
result = BuildResult{HAKAssets: len(assetResources)}
result.CreditsArtifactPaths = append(result.CreditsArtifactPaths, musicAssets.Artifacts...)
result.CreditsSummary = musicAssets.Summary
if err := writeAutogenManifestOutputs(progress, autogenManifests); err != nil {
return BuildResult{}, err
}
@@ -410,8 +374,8 @@ func planOrBuildHAKs(p *project.Project, progress ProgressFunc, writeArchives bo
}
// buildDirectHAKs packages HAKs directly from the content-addressed source blob
// cache. It skips music preparation, generated 2da discovery, git creation-time
// lookups, LFS discovery, and any materialized asset tree scan.
// cache. It skips generated 2da discovery, git creation-time lookups, LFS
// discovery, and any materialized asset tree scan.
func buildDirectHAKs(p *project.Project, progress ProgressFunc, writeArchives bool, manifest *SourceBuildManifest, opts BuildHAKOptions) (BuildResult, error) {
progressf(progress, "Collecting asset resources...")
assetResources, err := collectDirectSourceResources(manifest, opts.ContentAddressedRoot)
@@ -651,7 +615,7 @@ func collectModuleResources(p *project.Project, moduleHakOrder []string) ([]erf.
return moduleResources, nil
}
func collectAssetResources(p *project.Project, requireContent bool, allowed map[string]struct{}, musicAssets *preparedMusicAssets, generated2DAAssets []topdata.Generated2DAAsset) ([]assetResource, error) {
func collectAssetResources(p *project.Project, requireContent bool, allowed map[string]struct{}, generated2DAAssets []topdata.Generated2DAAsset) ([]assetResource, error) {
var hakResources []assetResource
assetCreatedAt, err := collectGitAssetCreationTimes(p)
if err != nil {
@@ -674,11 +638,6 @@ func collectAssetResources(p *project.Project, requireContent bool, allowed map[
}
for _, rel := range p.Inventory.AssetFiles {
if musicAssets != nil {
if _, skip := musicAssets.SkipSourceRel[filepath.ToSlash(rel)]; skip {
continue
}
}
if len(allowed) > 0 {
if _, ok := allowed[filepath.ToSlash(rel)]; !ok {
continue
@@ -706,16 +665,6 @@ func collectAssetResources(p *project.Project, requireContent bool, allowed map[
ContentID: resourceInfo.ContentID,
})
}
if musicAssets != nil {
for _, asset := range musicAssets.Generated {
if len(allowed) > 0 {
if _, ok := allowed[filepath.ToSlash(asset.Rel)]; !ok {
continue
}
}
hakResources = append(hakResources, asset)
}
}
for _, asset := range generated2DAAssets {
rel := filepath.ToSlash(asset.Rel)
if len(allowed) > 0 {
@@ -1617,20 +1566,11 @@ func plannedModuleHAKOrder(p *project.Project) (order []string, err error) {
return append([]string(nil), manifest.ModuleHAKs...), nil
}
musicAssets, err := prepareMusicAssets(p)
if err != nil {
return nil, err
}
defer func() {
if cleanupErr := cleanupPreparedMusicAssets(musicAssets); cleanupErr != nil && err == nil {
err = cleanupErr
}
}()
generated2DAAssets, err := topdata.BuildGenerated2DAAssets(p, nil)
if err != nil {
return nil, err
}
assetResources, err := collectAssetResources(p, false, nil, musicAssets, generated2DAAssets)
assetResources, err := collectAssetResources(p, false, nil, generated2DAAssets)
if err != nil {
return nil, err
}
+1 -1
View File
@@ -223,7 +223,7 @@ func ensureBuildIsCurrent(p *project.Project, modulePath string, hakPaths []stri
}
if !newestSource.IsZero() && newestSource.After(oldestArchive) {
return fmt.Errorf("built archives are older than source file %s; run build-module or build-haks before compare", newestPath)
return fmt.Errorf("built archives are older than source file %s; run crucible module build or crucible hak build before compare", newestPath)
}
return nil
}
-529
View File
@@ -1,529 +0,0 @@
package pipeline
import (
"encoding/json"
"fmt"
"os"
"path/filepath"
"sort"
"strings"
"git.westgate.pw/ShadowsOverWestgate/sow-tools/internal/erf"
"git.westgate.pw/ShadowsOverWestgate/sow-tools/internal/music"
"git.westgate.pw/ShadowsOverWestgate/sow-tools/internal/project"
)
type preparedMusicAssets struct {
Generated []assetResource
SkipSourceRel map[string]struct{}
Artifacts []string
TempRoots []string
Summary CreditsRefreshSummary
}
type MusicBuildOptions struct {
DatasetIDs []string
Write bool
}
type musicPrepareOptions struct {
datasetIDs []string
write bool
}
type musicSourceGroup struct {
DatasetID string
Dataset project.EffectiveMusicDataset
SourceDir string
OutputDir string
Sources []string
}
type musicCreditGroup struct {
CreditsRoot string
SourceDir string
Entries []music.CreditsEntry
}
func musicRootPath(p *project.Project, path string) string {
if strings.TrimSpace(path) == "" {
return p.Root
}
if filepath.IsAbs(path) {
return path
}
return filepath.Join(p.Root, filepath.FromSlash(path))
}
func effectiveMusicDatasets(p *project.Project, onlyIDs []string) (map[string]project.EffectiveMusicDataset, error) {
effective := p.EffectiveConfig()
wanted := map[string]struct{}{}
for _, id := range onlyIDs {
id = strings.TrimSpace(id)
if id != "" {
wanted[id] = struct{}{}
}
}
datasets := map[string]project.EffectiveMusicDataset{}
for id, dataset := range effective.Music.Datasets {
if len(wanted) > 0 {
if _, ok := wanted[id]; !ok {
continue
}
delete(wanted, id)
}
datasets[id] = dataset
}
if len(wanted) > 0 {
unknown := make([]string, 0, len(wanted))
for id := range wanted {
unknown = append(unknown, id)
}
sort.Strings(unknown)
return nil, fmt.Errorf("unknown music dataset(s): %s", strings.Join(unknown, ", "))
}
if len(datasets) == 0 && len(wanted) == 0 {
for _, rel := range p.Inventory.AssetFiles {
ext := strings.ToLower(filepath.Ext(rel))
if !music.PathIsUnder("envi/music", rel) {
continue
}
if _, ok := extensionSet(effective.Music.ConvertExtensions)[ext]; ok {
datasets["legacy_envi_music"] = project.EffectiveMusicDataset{
Source: "envi/music",
Output: "envi/music",
StageRoot: effective.Music.StageRoot,
CreditsRoot: effective.Music.CreditsRoot,
NamingScheme: effective.Music.NamingScheme,
OutputExtension: effective.Music.OutputExtension,
MaxStemLength: effective.Music.MaxStemLength,
PackageMode: "hak_asset",
ConvertExtensions: effective.Music.ConvertExtensions,
}
break
}
}
}
return datasets, nil
}
func extensionSet(values []string) map[string]struct{} {
out := make(map[string]struct{}, len(values))
for _, value := range values {
value = strings.ToLower(strings.TrimSpace(value))
if value != "" {
out[value] = struct{}{}
}
}
return out
}
func datasetForMusicSource(datasets map[string]project.EffectiveMusicDataset, rel, ext string) (string, project.EffectiveMusicDataset, bool) {
bestID := ""
var best project.EffectiveMusicDataset
bestLen := -1
for id, dataset := range datasets {
if _, ok := extensionSet(dataset.ConvertExtensions)[ext]; !ok {
continue
}
if !music.PathIsUnder(dataset.Source, rel) {
continue
}
if len(dataset.Source) > bestLen {
bestID = id
best = dataset
bestLen = len(dataset.Source)
}
}
return bestID, best, bestID != ""
}
func plannedMusicAsset(outputRel, sourceRel string) (assetResource, error) {
extension := strings.TrimPrefix(strings.ToLower(filepath.Ext(outputRel)), ".")
resourceType, ok := erf.HAKResourceTypeForExtension(extension)
if !ok {
return assetResource{}, fmt.Errorf("unsupported generated music resource extension %q", filepath.Ext(outputRel))
}
name := strings.ToLower(strings.TrimSuffix(filepath.Base(outputRel), filepath.Ext(outputRel)))
return assetResource{
Rel: outputRel,
Resource: erf.Resource{
Name: name,
Type: resourceType,
},
ContentID: "music-plan:" + filepath.ToSlash(sourceRel),
}, nil
}
func BuildMusic(p *project.Project, opts MusicBuildOptions) (BuildResult, error) {
prepared, err := prepareMusicAssetsWithOptions(p, musicPrepareOptions{datasetIDs: opts.DatasetIDs, write: opts.Write})
if err != nil {
return BuildResult{}, err
}
if err := cleanupPreparedMusicAssets(prepared); err != nil {
return BuildResult{}, err
}
return BuildResult{
CreditsArtifactPaths: prepared.Artifacts,
CreditsSummary: prepared.Summary,
HAKAssets: len(prepared.Generated),
}, nil
}
func prepareMusicAssets(p *project.Project) (*preparedMusicAssets, error) {
return prepareMusicAssetsWithOptions(p, musicPrepareOptions{write: true})
}
func prepareMusicAssetsWithOptions(p *project.Project, opts musicPrepareOptions) (*preparedMusicAssets, error) {
usedNames := make(map[string]struct{})
groupsByKey := make(map[string]*musicSourceGroup)
datasets, err := effectiveMusicDatasets(p, opts.datasetIDs)
if err != nil {
return nil, err
}
for _, rel := range p.Inventory.AssetFiles {
rel = filepath.ToSlash(rel)
ext := strings.ToLower(filepath.Ext(rel))
name := strings.ToLower(strings.TrimSuffix(filepath.Base(rel), filepath.Ext(rel)))
if datasetID, dataset, ok := datasetForMusicSource(datasets, rel, ext); ok {
sourceDir := filepath.ToSlash(filepath.Dir(rel))
relDir, err := filepath.Rel(filepath.FromSlash(dataset.Source), filepath.FromSlash(sourceDir))
if err != nil {
return nil, fmt.Errorf("resolve music source directory %s: %w", sourceDir, err)
}
relDir = filepath.ToSlash(relDir)
outputDir := dataset.Output
if relDir != "." && relDir != "" {
outputDir = filepath.ToSlash(filepath.Join(outputDir, filepath.FromSlash(relDir)))
}
key := datasetID + "\x00" + sourceDir
group := groupsByKey[key]
if group == nil {
group = &musicSourceGroup{
DatasetID: datasetID,
Dataset: dataset,
SourceDir: sourceDir,
OutputDir: outputDir,
}
groupsByKey[key] = group
}
group.Sources = append(group.Sources, rel)
continue
}
if name != "" {
usedNames[name] = struct{}{}
}
}
result := &preparedMusicAssets{
SkipSourceRel: make(map[string]struct{}),
}
if len(groupsByKey) == 0 {
if !opts.write {
return result, nil
}
writeResult, err := writeCreditsArtifacts(p, nil)
if err != nil {
return nil, err
}
result.Artifacts = writeResult.Artifacts
result.Summary = writeResult.Summary
return result, nil
}
groups := make([]*musicSourceGroup, 0, len(groupsByKey))
for _, group := range groupsByKey {
sort.Strings(group.Sources)
groups = append(groups, group)
}
sort.Slice(groups, func(i, j int) bool {
if groups[i].SourceDir != groups[j].SourceDir {
return groups[i].SourceDir < groups[j].SourceDir
}
return groups[i].DatasetID < groups[j].DatasetID
})
creditGroups := make([]musicCreditGroup, 0, len(groups))
for _, group := range groups {
result.Summary.ScannedDirs = append(result.Summary.ScannedDirs, group.SourceDir)
result.Summary.TrackCount += len(group.Sources)
}
ffmpegPath := ""
ffprobePath := ""
if opts.write {
var err error
ffmpegPath, err = music.ResolveFFmpegWithConfig(p.MusicFFmpegPath())
if err != nil {
return nil, err
}
ffprobePath, err = music.ResolveFFprobeWithConfig(p.MusicFFprobePath())
if err != nil {
return nil, err
}
}
for _, group := range groups {
stageRoot := musicRootPath(p, group.Dataset.StageRoot)
result.TempRoots = append(result.TempRoots, stageRoot)
prefix := music.SanitizePrefix(group.Dataset.Prefix)
overlayPath := filepath.Join(p.AssetsDir(), filepath.FromSlash(group.SourceDir), p.EffectiveConfig().Music.CreditsOverlay)
overlay, err := music.ParseCreditsOverlay(overlayPath)
if err != nil {
return nil, fmt.Errorf("parse credits overlay %s: %w", overlayPath, err)
}
entries := make([]music.CreditsEntry, 0, len(group.Sources))
for _, rel := range group.Sources {
base := filepath.Base(rel)
manual := overlay.Match(base, "")
outputFile := ""
switch {
case manual != nil && strings.TrimSpace(manual.OutputFile) != "":
outputFile = strings.ToLower(strings.TrimSpace(manual.OutputFile))
if err := music.ValidateManualOutputFileWithRules(outputFile, group.Dataset.OutputExtension, group.Dataset.MaxStemLength); err != nil {
return nil, fmt.Errorf("manual credits output for %s: %w", rel, err)
}
if err := music.ReserveName(strings.TrimSuffix(outputFile, filepath.Ext(outputFile)), usedNames); err != nil {
return nil, fmt.Errorf("manual credits output for %s: %w", rel, err)
}
default:
stem, err := music.GenerateStemWithMax(base, prefix, group.Dataset.MaxStemLength, usedNames)
if err != nil {
return nil, fmt.Errorf("generate music name for %s: %w", rel, err)
}
outputFile = stem + group.Dataset.OutputExtension
}
outputRel := filepath.ToSlash(filepath.Join(group.OutputDir, outputFile))
result.Summary.Mappings = append(result.Summary.Mappings, MusicFileMapping{
Source: rel,
Output: outputRel,
})
result.SkipSourceRel[rel] = struct{}{}
if !opts.write {
if group.Dataset.PackageMode == "hak_asset" {
asset, err := plannedMusicAsset(outputRel, rel)
if err != nil {
return nil, err
}
result.Generated = append(result.Generated, asset)
}
continue
}
metadata, err := music.ProbeMetadata(ffprobePath, filepath.Join(p.AssetsDir(), filepath.FromSlash(rel)))
if err != nil {
return nil, fmt.Errorf("probe music metadata for %s: %w", rel, err)
}
entry := music.CreditsEntry{
Artist: metadata.Artist,
Title: music.FirstNonEmpty(metadata.Title, strings.TrimSuffix(base, filepath.Ext(base))),
OutputFile: outputFile,
OriginalFile: base,
Album: metadata.Album,
Date: metadata.Date,
Rights: music.FirstNonEmpty(metadata.Rights, music.JoinNonEmpty("; ", metadata.License, metadata.Copyright)),
Notes: music.FirstNonEmpty(metadata.Notes, metadata.Comment),
}
music.ApplyCreditsOverlay(&entry, manual)
stagePath := filepath.Join(stageRoot, filepath.FromSlash(outputRel))
if err := os.MkdirAll(filepath.Dir(stagePath), 0o755); err != nil {
return nil, fmt.Errorf("create music stage dir: %w", err)
}
if err := music.ConvertSource(ffmpegPath, filepath.Join(p.AssetsDir(), filepath.FromSlash(rel)), stagePath); err != nil {
return nil, fmt.Errorf("convert music %s: %w", rel, err)
}
if group.Dataset.PackageMode == "hak_asset" {
resourceInfo, err := assetResourceFromPath(stagePath, outputRel, lfsAssetInfo{}, false)
if err != nil {
return nil, fmt.Errorf("load converted music %s: %w", outputRel, err)
}
info, err := os.Stat(stagePath)
if err != nil {
return nil, fmt.Errorf("stat converted music %s: %w", stagePath, err)
}
result.Generated = append(result.Generated, assetResource{
Rel: outputRel,
Resource: resourceInfo.Resource,
Size: erf.ArchiveSize([]erf.Resource{resourceInfo.Resource}),
CreatedAt: info.ModTime(),
ContentID: resourceInfo.ContentID,
})
}
entries = append(entries, entry)
}
if opts.write {
if err := music.ValidateOverlayEntries(group.SourceDir, overlay, entries); err != nil {
return nil, err
}
creditGroups = append(creditGroups, musicCreditGroup{
CreditsRoot: group.Dataset.CreditsRoot,
SourceDir: group.SourceDir,
Entries: entries,
})
}
}
if !opts.write {
sort.Slice(result.Generated, func(i, j int) bool {
return compareResourceKeys(result.Generated[i].Resource, result.Generated[j].Resource) < 0
})
return result, nil
}
writeResult, err := writeCreditsArtifacts(p, creditGroups)
if err != nil {
return nil, err
}
writeResult.Summary.ScannedDirs = append(writeResult.Summary.ScannedDirs, result.Summary.ScannedDirs...)
writeResult.Summary.TrackCount = result.Summary.TrackCount
result.Artifacts = writeResult.Artifacts
result.Summary = writeResult.Summary
sort.Slice(result.Generated, func(i, j int) bool {
return compareResourceKeys(result.Generated[i].Resource, result.Generated[j].Resource) < 0
})
return result, nil
}
type creditsArtifactWriteResult struct {
Artifacts []string
Summary CreditsRefreshSummary
}
func writeCreditsArtifacts(p *project.Project, generated []musicCreditGroup) (creditsArtifactWriteResult, error) {
defaultCreditsRoot := p.CreditsDir()
sources := make([]music.CreditsSource, 0)
desiredByRoot := map[string]map[string][]byte{}
summary := CreditsRefreshSummary{}
if generated != nil {
sort.Slice(generated, func(i, j int) bool {
if generated[i].SourceDir != generated[j].SourceDir {
return generated[i].SourceDir < generated[j].SourceDir
}
return generated[i].CreditsRoot < generated[j].CreditsRoot
})
for _, group := range generated {
creditsRoot := musicRootPath(p, group.CreditsRoot)
if desiredByRoot[creditsRoot] == nil {
desiredByRoot[creditsRoot] = map[string][]byte{}
}
entries := append([]music.CreditsEntry(nil), group.Entries...)
sort.Slice(entries, func(i, j int) bool {
if strings.ToLower(entries[i].Artist) != strings.ToLower(entries[j].Artist) {
return strings.ToLower(entries[i].Artist) < strings.ToLower(entries[j].Artist)
}
if strings.ToLower(entries[i].Title) != strings.ToLower(entries[j].Title) {
return strings.ToLower(entries[i].Title) < strings.ToLower(entries[j].Title)
}
return strings.ToLower(entries[i].OutputFile) < strings.ToLower(entries[j].OutputFile)
})
outputPath := filepath.Join(creditsRoot, filepath.FromSlash(group.SourceDir), p.EffectiveConfig().Music.CreditsOverlay)
desiredByRoot[creditsRoot][outputPath] = []byte(music.RenderCreditsMarkdown(entries))
summary.GeneratedPaths = append(summary.GeneratedPaths, outputPath)
for _, entry := range entries {
summary.Mappings = append(summary.Mappings, MusicFileMapping{
Source: entry.OriginalFile,
Output: entry.OutputFile,
})
}
sources = append(sources, music.CreditsSource{
Path: filepath.ToSlash(outputPath),
Kind: "generated_music",
Entries: entries,
})
}
}
// Authored credits.md files live under the assets tree. A project with no
// configured assets dir (e.g. a module that consumes prebuilt HAKs) has none
// to collect — walking "" would fail with `lstat : no such file or directory`.
if assetsDir := p.AssetsDir(); assetsDir != "" {
err := filepath.WalkDir(assetsDir, func(path string, d os.DirEntry, walkErr error) error {
if walkErr != nil {
return walkErr
}
if d.IsDir() {
return nil
}
if strings.ToLower(d.Name()) != "credits.md" {
return nil
}
entries, err := music.ParseCreditsMarkdown(path)
if err != nil {
return fmt.Errorf("parse credits %s: %w", path, err)
}
rel, err := filepath.Rel(p.Root, path)
if err != nil {
return err
}
sources = append(sources, music.CreditsSource{
Path: filepath.ToSlash(rel),
Kind: "authored",
Entries: entries,
})
return nil
})
if err != nil {
return creditsArtifactWriteResult{}, err
}
}
sort.Slice(sources, func(i, j int) bool { return sources[i].Path < sources[j].Path })
payload, err := json.MarshalIndent(music.CreditsInventory{Sources: sources}, "", " ")
if err != nil {
return creditsArtifactWriteResult{}, fmt.Errorf("marshal credits inventory: %w", err)
}
payload = append(payload, '\n')
if len(desiredByRoot) == 0 {
desiredByRoot[defaultCreditsRoot] = map[string][]byte{}
}
artifacts := make([]string, 0)
changedFiles := 0
roots := make([]string, 0, len(desiredByRoot))
for root := range desiredByRoot {
roots = append(roots, root)
}
sort.Strings(roots)
for _, creditsRoot := range roots {
desiredFiles := desiredByRoot[creditsRoot]
inventoryPath := filepath.Join(creditsRoot, "credits.json")
desiredFiles[inventoryPath] = payload
if err := os.MkdirAll(creditsRoot, 0o755); err != nil {
return creditsArtifactWriteResult{}, fmt.Errorf("create credits dir: %w", err)
}
changed, err := music.SyncGeneratedCreditsArtifacts(creditsRoot, desiredFiles)
if err != nil {
return creditsArtifactWriteResult{}, err
}
changedFiles += changed
for path := range desiredFiles {
artifacts = append(artifacts, path)
}
if summary.InventoryPath == "" {
summary.InventoryPath = inventoryPath
}
}
sort.Strings(artifacts)
summary.ArtifactPaths = append(summary.ArtifactPaths, artifacts...)
summary.ChangedFiles = changedFiles
return creditsArtifactWriteResult{
Artifacts: artifacts,
Summary: summary,
}, nil
}
func cleanupPreparedMusicAssets(prepared *preparedMusicAssets) error {
if prepared == nil {
return nil
}
for _, root := range prepared.TempRoots {
if strings.TrimSpace(root) == "" {
continue
}
if err := os.RemoveAll(root); err != nil {
return fmt.Errorf("remove temporary music artifact root %s: %w", root, err)
}
}
return nil
}
+3 -412
View File
@@ -2929,7 +2929,7 @@ func TestPlanHAKChunksPutsNewestAssetsInLastChunk(t *testing.T) {
t.Fatalf("scan: %v", err)
}
assets, err := collectAssetResources(p, false, nil, nil, nil)
assets, err := collectAssetResources(p, false, nil, nil)
if err != nil {
t.Fatalf("collect asset resources: %v", err)
}
@@ -3676,418 +3676,9 @@ func TestBuildHAKsFailsForDuplicateResourcesInSameHAK(t *testing.T) {
}
}
func TestBuildHAKsConvertsMusicSourcesAndWritesCreditsArtifacts(t *testing.T) {
root := t.TempDir()
mustMkdir(t, filepath.Join(root, "assets", "envi", "music", "westgate"))
mustMkdir(t, filepath.Join(root, "build"))
mustWriteFile(t, filepath.Join(root, "nwn-tool.json"), `{
"module": {
"name": "Test Assets",
"resref": "testassets"
},
"paths": {
"assets": "assets",
"build": "build"
},
"music": {
"prefixes": {
"envi/music/westgate": "mus_wg_"
}
},
"haks": [
{
"name": "envi",
"priority": 1,
"max_bytes": 1048576,
"split": false,
"include": ["envi/**"]
}
]
}
`)
mustWriteFile(t, filepath.Join(root, "assets", "envi", "music", "westgate", "AleandAnecdotes.mp3"), "source-mp3")
mustWriteFile(t, filepath.Join(root, "assets", "envi", "music", "westgate", "CREDITS.md"), "# NWN Music Pack Credits\n\n| Artist | Title | Output File | Original File | Album | Date | License / Copyright | Notes |\n|---|---|---|---|---|---|---|---|\n| Darren Curtis | Ale and Anecdotes | `mus_wg_andnc.bmu` | `AleandAnecdotes.mp3` | Darren's Commercially Free Music | | Darren Curtis Music | |\n")
ffprobePath := filepath.Join(root, "ffprobe")
mustWriteFile(t, ffprobePath, "#!/bin/sh\nprintf '%s\\n' '{\"format\":{\"tags\":{\"artist\":\"Unknown\",\"title\":\"Broken Metadata\"}}}'\n")
if err := os.Chmod(ffprobePath, 0o755); err != nil {
t.Fatalf("chmod ffprobe: %v", err)
}
ffmpegPath := filepath.Join(root, "ffmpeg")
mustWriteFile(t, ffmpegPath, "#!/bin/sh\nfor last; do :; done\nprintf 'bmu-data' > \"$last\"\n")
if err := os.Chmod(ffmpegPath, 0o755); err != nil {
t.Fatalf("chmod ffmpeg: %v", err)
}
t.Setenv("SOW_FFMPEG", ffmpegPath)
t.Setenv("SOW_FFPROBE", ffprobePath)
p, err := project.Load(root)
if err != nil {
t.Fatalf("load project: %v", err)
}
if err := p.ValidateLayout(); err != nil {
t.Fatalf("validate layout: %v", err)
}
if err := p.Scan(); err != nil {
t.Fatalf("scan: %v", err)
}
result, err := BuildHAKs(p)
if err != nil {
t.Fatalf("build haks: %v", err)
}
if len(result.CreditsArtifactPaths) == 0 {
t.Fatal("expected credits artifacts to be written")
}
manifestRaw, err := os.ReadFile(filepath.Join(root, "build", "haks.json"))
if err != nil {
t.Fatalf("read manifest: %v", err)
}
if !strings.Contains(string(manifestRaw), "envi/music/westgate/mus_wg_andnc.bmu") {
t.Fatalf("expected converted bmu in manifest, got %s", string(manifestRaw))
}
if strings.Contains(string(manifestRaw), "AleandAnecdotes.mp3") {
t.Fatalf("did not expect source mp3 in manifest, got %s", string(manifestRaw))
}
creditsRaw, err := os.ReadFile(filepath.Join(root, ".cache", "credits", "envi", "music", "westgate", "CREDITS.md"))
if err != nil {
t.Fatalf("read generated credits: %v", err)
}
creditsText := string(creditsRaw)
if !strings.Contains(creditsText, "Darren Curtis") || !strings.Contains(creditsText, "mus_wg_andnc.bmu") {
t.Fatalf("unexpected generated credits contents: %s", creditsText)
}
if strings.Contains(creditsText, "Broken Metadata") {
t.Fatalf("manual credits overlay should override bad metadata: %s", creditsText)
}
inventoryRaw, err := os.ReadFile(filepath.Join(root, ".cache", "credits", "credits.json"))
if err != nil {
t.Fatalf("read credits inventory: %v", err)
}
inventoryText := string(inventoryRaw)
if !strings.Contains(inventoryText, "assets/envi/music/westgate/CREDITS.md") {
t.Fatalf("expected authored credits source in inventory: %s", inventoryText)
}
if !strings.Contains(inventoryText, ".cache/credits/envi/music/westgate/CREDITS.md") {
t.Fatalf("expected generated credits source in inventory: %s", inventoryText)
}
if _, err := os.Stat(filepath.Join(root, ".cache", "music")); !os.IsNotExist(err) {
t.Fatalf("expected temporary music staging to be cleaned, got err=%v", err)
}
secondResult, err := BuildHAKs(p)
if err != nil {
t.Fatalf("build haks second pass: %v", err)
}
if secondResult.CreditsSummary.ChangedFiles != 0 {
t.Fatalf("expected second credits refresh to be unchanged, got %d changed files", secondResult.CreditsSummary.ChangedFiles)
}
if secondResult.CreditsSummary.TrackCount != 1 {
t.Fatalf("expected one tracked music conversion, got %d", secondResult.CreditsSummary.TrackCount)
}
}
func TestBuildHAKsUsesExplicitMusicDatasetOutsideLegacyRoot(t *testing.T) {
root := t.TempDir()
mustMkdir(t, filepath.Join(root, "assets", "audio", "westgate"))
mustMkdir(t, filepath.Join(root, "build"))
ffprobePath := filepath.Join(root, "tools", "ffprobe")
ffmpegPath := filepath.Join(root, "tools", "ffmpeg")
mustMkdir(t, filepath.Dir(ffprobePath))
mustWriteFile(t, ffprobePath, "#!/bin/sh\nprintf '%s\\n' '{\"format\":{\"tags\":{\"artist\":\"Dataset Artist\",\"title\":\"Dataset Title\"}}}'\n")
mustWriteFile(t, ffmpegPath, "#!/bin/sh\nfor last; do :; done\nprintf 'dataset-bmu' > \"$last\"\n")
if err := os.Chmod(ffprobePath, 0o755); err != nil {
t.Fatalf("chmod ffprobe: %v", err)
}
if err := os.Chmod(ffmpegPath, 0o755); err != nil {
t.Fatalf("chmod ffmpeg: %v", err)
}
mustWriteFile(t, filepath.Join(root, "nwn-tool.yaml"), `
module:
name: Test Assets
resref: testassets
paths:
assets: assets
build: build
music:
tools:
ffmpeg: tools/ffmpeg
ffprobe: tools/ffprobe
defaults:
credits_root: custom-credits
datasets:
westgate_audio:
source: audio/westgate
output: generated/music
prefix: wg_
haks:
- name: music
priority: 1
max_bytes: 1048576
split: false
include:
- generated/music/**
`)
mustWriteFile(t, filepath.Join(root, "assets", "audio", "westgate", "Theme Song.mp3"), "source-mp3")
p, err := project.Load(root)
if err != nil {
t.Fatalf("load project: %v", err)
}
if err := p.ValidateLayout(); err != nil {
t.Fatalf("validate layout: %v", err)
}
if err := p.Scan(); err != nil {
t.Fatalf("scan: %v", err)
}
result, err := BuildHAKs(p)
if err != nil {
t.Fatalf("build haks: %v", err)
}
if result.CreditsSummary.TrackCount != 1 {
t.Fatalf("expected one music track, got %#v", result.CreditsSummary)
}
manifestRaw, err := os.ReadFile(filepath.Join(root, "build", "haks.json"))
if err != nil {
t.Fatalf("read manifest: %v", err)
}
manifestText := string(manifestRaw)
if !strings.Contains(manifestText, "generated/music/wg_theme.bmu") {
t.Fatalf("expected dataset output resource in manifest, got:\n%s", manifestText)
}
if strings.Contains(manifestText, "audio/westgate/Theme Song.mp3") {
t.Fatalf("did not expect source mp3 in manifest, got:\n%s", manifestText)
}
creditsRaw, err := os.ReadFile(filepath.Join(root, "custom-credits", "audio", "westgate", "CREDITS.md"))
if err != nil {
t.Fatalf("read generated dataset credits: %v", err)
}
if !strings.Contains(string(creditsRaw), "Dataset Artist") || !strings.Contains(string(creditsRaw), "wg_theme.bmu") {
t.Fatalf("unexpected generated credits:\n%s", string(creditsRaw))
}
}
func TestBuildHAKsConvertsFLACMusicSourcesByDefault(t *testing.T) {
root := t.TempDir()
mustMkdir(t, filepath.Join(root, "assets", "audio", "westgate"))
mustMkdir(t, filepath.Join(root, "build"))
ffprobePath := filepath.Join(root, "tools", "ffprobe")
ffmpegPath := filepath.Join(root, "tools", "ffmpeg")
mustMkdir(t, filepath.Dir(ffprobePath))
mustWriteFile(t, ffprobePath, "#!/bin/sh\nprintf '%s\\n' '{\"format\":{\"tags\":{\"artist\":\"FLAC Artist\",\"title\":\"FLAC Title\"}}}'\n")
mustWriteFile(t, ffmpegPath, "#!/bin/sh\nfor last; do :; done\nprintf 'dataset-bmu' > \"$last\"\n")
if err := os.Chmod(ffprobePath, 0o755); err != nil {
t.Fatalf("chmod ffprobe: %v", err)
}
if err := os.Chmod(ffmpegPath, 0o755); err != nil {
t.Fatalf("chmod ffmpeg: %v", err)
}
mustWriteFile(t, filepath.Join(root, "nwn-tool.yaml"), `
module:
name: Test Assets
resref: testassets
paths:
assets: assets
build: build
music:
tools:
ffmpeg: tools/ffmpeg
ffprobe: tools/ffprobe
datasets:
westgate_audio:
source: audio/westgate
output: generated/music
prefix: wg_
haks:
- name: music
priority: 1
max_bytes: 1048576
split: false
include:
- generated/music/**
`)
mustWriteFile(t, filepath.Join(root, "assets", "audio", "westgate", "Theme Song.flac"), "source-flac")
p, err := project.Load(root)
if err != nil {
t.Fatalf("load project: %v", err)
}
if err := p.ValidateLayout(); err != nil {
t.Fatalf("validate layout: %v", err)
}
if err := p.Scan(); err != nil {
t.Fatalf("scan: %v", err)
}
result, err := BuildHAKs(p)
if err != nil {
t.Fatalf("build haks: %v", err)
}
if result.CreditsSummary.TrackCount != 1 {
t.Fatalf("expected one music track, got %#v", result.CreditsSummary)
}
manifestRaw, err := os.ReadFile(filepath.Join(root, "build", "haks.json"))
if err != nil {
t.Fatalf("read manifest: %v", err)
}
manifestText := string(manifestRaw)
if !strings.Contains(manifestText, "generated/music/wg_theme.bmu") {
t.Fatalf("expected generated FLAC music in manifest, got:\n%s", manifestText)
}
if strings.Contains(manifestText, "audio/westgate/Theme Song.flac") {
t.Fatalf("did not expect source flac in manifest, got:\n%s", manifestText)
}
}
func TestBuildHAKsUsesYAMLMusicConvertExtensionsForDiscovery(t *testing.T) {
root := t.TempDir()
mustMkdir(t, filepath.Join(root, "assets", "audio", "westgate"))
mustMkdir(t, filepath.Join(root, "build"))
ffprobePath := filepath.Join(root, "tools", "ffprobe")
ffmpegPath := filepath.Join(root, "tools", "ffmpeg")
mustMkdir(t, filepath.Dir(ffprobePath))
mustWriteFile(t, ffprobePath, "#!/bin/sh\nprintf '%s\\n' '{\"format\":{\"tags\":{\"artist\":\"AAC Artist\",\"title\":\"AAC Title\"}}}'\n")
mustWriteFile(t, ffmpegPath, "#!/bin/sh\nfor last; do :; done\nprintf 'dataset-bmu' > \"$last\"\n")
if err := os.Chmod(ffprobePath, 0o755); err != nil {
t.Fatalf("chmod ffprobe: %v", err)
}
if err := os.Chmod(ffmpegPath, 0o755); err != nil {
t.Fatalf("chmod ffmpeg: %v", err)
}
mustWriteFile(t, filepath.Join(root, "nwn-tool.yaml"), `
module:
name: Test Assets
resref: testassets
paths:
assets: assets
build: build
music:
defaults:
convert_extensions:
- .aac
tools:
ffmpeg: tools/ffmpeg
ffprobe: tools/ffprobe
datasets:
westgate_audio:
source: audio/westgate
output: generated/music
prefix: wg_
haks:
- name: music
priority: 1
max_bytes: 1048576
split: false
include:
- generated/music/**
`)
mustWriteFile(t, filepath.Join(root, "assets", "audio", "westgate", "Theme Song.aac"), "source-aac")
p, err := project.Load(root)
if err != nil {
t.Fatalf("load project: %v", err)
}
if err := p.ValidateLayout(); err != nil {
t.Fatalf("validate layout: %v", err)
}
if err := p.Scan(); err != nil {
t.Fatalf("scan: %v", err)
}
result, err := BuildHAKs(p)
if err != nil {
t.Fatalf("build haks: %v", err)
}
if result.CreditsSummary.TrackCount != 1 {
t.Fatalf("expected one music track discovered through YAML convert_extensions, got %#v", result.CreditsSummary)
}
}
func TestPlanHAKsDoesNotTranscodeMusicSources(t *testing.T) {
root := t.TempDir()
mustMkdir(t, filepath.Join(root, "assets", "audio", "westgate"))
mustMkdir(t, filepath.Join(root, "build"))
ffmpegPath := filepath.Join(root, "tools", "ffmpeg")
mustMkdir(t, filepath.Dir(ffmpegPath))
mustWriteFile(t, ffmpegPath, "#!/bin/sh\necho should-not-run >&2\nexit 42\n")
if err := os.Chmod(ffmpegPath, 0o755); err != nil {
t.Fatalf("chmod ffmpeg: %v", err)
}
mustWriteFile(t, filepath.Join(root, "nwn-tool.yaml"), `
module:
name: Test Assets
resref: testassets
paths:
assets: assets
build: build
music:
tools:
ffmpeg: tools/ffmpeg
ffprobe: tools/ffprobe
datasets:
westgate_audio:
source: audio/westgate
output: generated/music
prefix: wg_
haks:
- name: music
priority: 1
max_bytes: 1048576
split: false
include:
- generated/music/**
`)
mustWriteFile(t, filepath.Join(root, "assets", "audio", "westgate", "Theme Song.mp3"), "source-mp3")
p, err := project.Load(root)
if err != nil {
t.Fatalf("load project: %v", err)
}
if err := p.ValidateLayout(); err != nil {
t.Fatalf("validate layout: %v", err)
}
if err := p.Scan(); err != nil {
t.Fatalf("scan: %v", err)
}
result, err := PlanHAKs(p)
if err != nil {
t.Fatalf("plan haks: %v", err)
}
if result.CreditsSummary.TrackCount != 1 {
t.Fatalf("expected one planned music track, got %#v", result.CreditsSummary)
}
manifestRaw, err := os.ReadFile(filepath.Join(root, "build", "haks.json"))
if err != nil {
t.Fatalf("read manifest: %v", err)
}
if !strings.Contains(string(manifestRaw), "generated/music/wg_theme.bmu") {
t.Fatalf("expected planned generated music in manifest, got:\n%s", string(manifestRaw))
}
if _, err := os.Stat(filepath.Join(root, ".cache", "music")); !os.IsNotExist(err) {
t.Fatalf("plan-only should not stage music, stat err=%v", err)
}
}
// A module-only project (consumes prebuilt HAKs, no paths.assets) must build via
// the full `crucible module build` (pipeline.Build) without an assets tree. The
// HAK + music stages used to crash on the unset assets dir
// the full `crucible module build` (pipeline.Build) without an assets tree. HAK
// collection used to crash on the unset assets dir
// (`lstat : no such file or directory` / `Rel: can't make "" relative`).
func TestBuildModuleOnlyProjectWithoutAssetsDir(t *testing.T) {
root := t.TempDir()
-130
View File
@@ -8,8 +8,6 @@ import (
"slices"
"strings"
"time"
"git.westgate.pw/ShadowsOverWestgate/sow-tools/internal/music"
)
const (
@@ -21,11 +19,6 @@ const (
DefaultHAKArchiveTemplate = "{hak.name}.hak"
DefaultSourceJSONPattern = "{resref}.{extension}.json"
DefaultValidationProfile = "nwn_module"
DefaultMusicStageRoot = "{paths.cache}/music"
DefaultCreditsRoot = "{paths.cache}/credits"
DefaultCreditsOverlayFile = "CREDITS.md"
DefaultMusicNamingScheme = "nwn_bmu"
DefaultMusicOutputExtension = ".bmu"
DefaultTopDataBuild = "{paths.build}/topdata"
DefaultTopDataCompiled2DADir = "2da"
DefaultTopDataCompiledTLK = "sow_tlk.tlk"
@@ -85,7 +78,6 @@ type EffectiveConfig struct {
Generated GeneratedConfig `json:"generated_assets" yaml:"generated_assets"`
Inventory InventoryConfig `json:"inventory" yaml:"inventory"`
Validation ValidationConfig `json:"validation" yaml:"validation"`
Music EffectiveMusicConfig `json:"music" yaml:"music"`
TopData EffectiveTopDataConfig `json:"topdata" yaml:"topdata"`
Extract ExtractConfig `json:"extract" yaml:"extract"`
Autogen EffectiveAutogenConfig `json:"autogen" yaml:"autogen"`
@@ -108,32 +100,6 @@ type EffectiveOutputConfig struct {
HAKArchive string `json:"hak_archive" yaml:"hak_archive"`
}
type EffectiveMusicConfig struct {
Prefixes map[string]string `json:"prefixes" yaml:"prefixes"`
Tools MusicToolsConfig `json:"tools" yaml:"tools"`
StageRoot string `json:"stage_root" yaml:"stage_root"`
CreditsRoot string `json:"credits_root" yaml:"credits_root"`
CreditsOverlay string `json:"credits_overlay" yaml:"credits_overlay"`
MaxStemLength int `json:"max_stem_length" yaml:"max_stem_length"`
NamingScheme string `json:"naming_scheme" yaml:"naming_scheme"`
OutputExtension string `json:"output_extension" yaml:"output_extension"`
ConvertExtensions []string `json:"convert_extensions" yaml:"convert_extensions"`
Datasets map[string]EffectiveMusicDataset `json:"datasets,omitempty" yaml:"datasets,omitempty"`
}
type EffectiveMusicDataset struct {
Source string `json:"source" yaml:"source"`
Output string `json:"output" yaml:"output"`
Prefix string `json:"prefix,omitempty" yaml:"prefix,omitempty"`
StageRoot string `json:"stage_root,omitempty" yaml:"stage_root,omitempty"`
CreditsRoot string `json:"credits_root,omitempty" yaml:"credits_root,omitempty"`
NamingScheme string `json:"naming_scheme" yaml:"naming_scheme"`
OutputExtension string `json:"output_extension" yaml:"output_extension"`
MaxStemLength int `json:"max_stem_length" yaml:"max_stem_length"`
PackageMode string `json:"package_mode" yaml:"package_mode"`
ConvertExtensions []string `json:"convert_extensions" yaml:"convert_extensions"`
}
type EffectiveTopDataConfig struct {
Source string `json:"source" yaml:"source"`
Build string `json:"build" yaml:"build"`
@@ -275,81 +241,6 @@ func (p *Project) EffectiveConfig() EffectiveConfig {
extract.ConsumeArchives = &defaultConsume
}
musicStageRoot := expandPathTemplate(DefaultMusicStageRoot, paths)
musicCreditsRoot := expandPathTemplate(DefaultCreditsRoot, paths)
musicCreditsOverlay := DefaultCreditsOverlayFile
musicMaxStemLength := 16
musicNamingScheme := DefaultMusicNamingScheme
musicOutputExtension := DefaultMusicOutputExtension
musicConvertExtensions := music.DefaultConvertExtensions()
if d := p.Config.Music.Defaults; d != nil {
if d.StageRoot != "" {
musicStageRoot = expandPathTemplate(d.StageRoot, paths)
}
if d.CreditsRoot != "" {
musicCreditsRoot = expandPathTemplate(d.CreditsRoot, paths)
}
if d.CreditsOverlay != "" {
musicCreditsOverlay = d.CreditsOverlay
}
if d.MaxStemLength != nil {
musicMaxStemLength = *d.MaxStemLength
}
if d.NamingScheme != "" {
musicNamingScheme = d.NamingScheme
}
if d.OutputExtension != "" {
musicOutputExtension = normalizeExtension(d.OutputExtension)
}
if len(d.ConvertExtensions) > 0 {
musicConvertExtensions = normalizeExtensionSlice(d.ConvertExtensions)
}
}
musicPrefixes := cloneStringMap(p.Config.Music.Prefixes)
if len(musicPrefixes) == 0 {
musicPrefixes = make(map[string]string, len(p.Config.Music.Datasets))
for _, ds := range p.Config.Music.Datasets {
if ds.Source != "" && ds.Prefix != "" {
musicPrefixes[ds.Source] = ds.Prefix
}
}
}
effectiveDatasets := make(map[string]EffectiveMusicDataset, len(p.Config.Music.Datasets))
for id, ds := range p.Config.Music.Datasets {
dsStageRoot := ds.StageRoot
if dsStageRoot == "" {
dsStageRoot = musicStageRoot
}
dsCreditsRoot := ds.CreditsRoot
if dsCreditsRoot == "" {
dsCreditsRoot = musicCreditsRoot
}
dsOutput := strings.TrimSpace(ds.Output)
if dsOutput == "" {
dsOutput = ds.Source
}
dsNamingScheme := defaultString(ds.NamingScheme, musicNamingScheme)
dsOutputExtension := musicOutputExtension
if ds.OutputExtension != "" {
dsOutputExtension = normalizeExtension(ds.OutputExtension)
}
dsConvertExtensions := musicConvertExtensions
if len(ds.ConvertExtensions) > 0 {
dsConvertExtensions = normalizeExtensionSlice(ds.ConvertExtensions)
}
effectiveDatasets[id] = EffectiveMusicDataset{
Source: ds.Source,
Output: dsOutput,
Prefix: ds.Prefix,
StageRoot: dsStageRoot,
CreditsRoot: dsCreditsRoot,
NamingScheme: dsNamingScheme,
OutputExtension: dsOutputExtension,
MaxStemLength: musicMaxStemLength,
PackageMode: defaultString(ds.PackageMode, "hak_asset"),
ConvertExtensions: dsConvertExtensions,
}
}
effective := EffectiveConfig{
ConfigSource: p.ConfigSource,
Module: p.Config.Module,
@@ -359,18 +250,6 @@ func (p *Project) EffectiveConfig() EffectiveConfig {
Generated: generated,
Inventory: inventory,
Validation: validation,
Music: EffectiveMusicConfig{
Prefixes: musicPrefixes,
Tools: p.Config.Music.Tools,
StageRoot: musicStageRoot,
CreditsRoot: musicCreditsRoot,
CreditsOverlay: musicCreditsOverlay,
MaxStemLength: musicMaxStemLength,
NamingScheme: musicNamingScheme,
OutputExtension: musicOutputExtension,
ConvertExtensions: musicConvertExtensions,
Datasets: effectiveDatasets,
},
TopData: top,
Extract: extract,
Autogen: EffectiveAutogenConfig{
@@ -517,13 +396,6 @@ func markMissingDefaults(provenance ConfigProvenance) {
"validation.profile": DefaultValidationProfile,
"validation.builtin_script_prefixes": "NWN built-in script prefixes",
"validation.required_fields": "NWN required GFF fields",
"music.stage_root": DefaultMusicStageRoot,
"music.credits_root": DefaultCreditsRoot,
"music.credits_overlay": DefaultCreditsOverlayFile,
"music.max_stem_length": "16",
"music.naming_scheme": DefaultMusicNamingScheme,
"music.output_extension": DefaultMusicOutputExtension,
"music.convert_extensions": strings.Join(music.DefaultConvertExtensions(), ", "),
"topdata.build": DefaultTopDataBuild,
"topdata.compiled_2da_dir": DefaultTopDataCompiled2DADir,
"topdata.compiled_tlk": DefaultTopDataCompiledTLK,
@@ -690,8 +562,6 @@ func activeOverrides(effective EffectiveConfig) []ConfigOverride {
"autogen.cache.parts_manifest_refresh": effective.Autogen.Cache.PartsManifestRefreshEnv,
"autogen.release_source.server_url": effective.Autogen.ReleaseSource.ServerURLEnv,
"autogen.release_source.repo": effective.Autogen.ReleaseSource.RepoEnv,
"music.ffmpeg": "SOW_FFMPEG",
"music.ffprobe": "SOW_FFPROBE",
"wiki.endpoint": "NODEBB_API_ENDPOINT",
"wiki.token": "NODEBB_API_TOKEN",
"wiki.categories": "NODEBB_WIKI_CATEGORIES",
+2 -195
View File
@@ -31,7 +31,7 @@ var SourceExtensions = []string{
}
var AssetExtensions = []string{
".2da", ".bik", ".bmp", ".bmu", ".dds", ".dwk", ".gr2", ".itp", ".jpg", ".lod", ".lyt", ".mdb", ".mdl", ".mdx", ".mp3", ".mtr", ".ogg", ".plt", ".png", ".pwk", ".set", ".shd", ".tga", ".txi", ".uti", ".vis", ".wav", ".wlk", ".wok", ".xml",
".2da", ".bik", ".bmp", ".bmu", ".dds", ".dwk", ".gr2", ".itp", ".jpg", ".lod", ".lyt", ".mdb", ".mdl", ".mdx", ".mtr", ".plt", ".png", ".pwk", ".set", ".shd", ".tga", ".txi", ".uti", ".vis", ".wav", ".wlk", ".wok", ".xml",
}
var BuiltinScriptPrefixes = []string{
@@ -88,7 +88,6 @@ type Config struct {
HAKs []HAKConfig `json:"haks" yaml:"haks"`
Inventory InventoryConfig `json:"inventory" yaml:"inventory"`
Validation ValidationConfig `json:"validation" yaml:"validation"`
Music MusicConfig `json:"music" yaml:"music"`
TopData TopDataConfig `json:"topdata" yaml:"topdata"`
Extract ExtractConfig `json:"extract" yaml:"extract"`
Autogen AutogenConfig `json:"autogen" yaml:"autogen"`
@@ -153,40 +152,6 @@ type HAKConfig struct {
Include []string `json:"include" yaml:"include"`
}
type MusicConfig struct {
Prefixes map[string]string `json:"prefixes,omitempty" yaml:"prefixes,omitempty"`
Tools MusicToolsConfig `json:"tools,omitempty" yaml:"tools,omitempty"`
Defaults *MusicDefaultsConfig `json:"defaults,omitempty" yaml:"defaults,omitempty"`
Datasets map[string]MusicDatasetConfig `json:"datasets,omitempty" yaml:"datasets,omitempty"`
}
type MusicToolsConfig struct {
FFmpeg string `json:"ffmpeg,omitempty" yaml:"ffmpeg,omitempty"`
FFprobe string `json:"ffprobe,omitempty" yaml:"ffprobe,omitempty"`
}
type MusicDefaultsConfig struct {
StageRoot string `json:"stage_root,omitempty" yaml:"stage_root,omitempty"`
CreditsRoot string `json:"credits_root,omitempty" yaml:"credits_root,omitempty"`
CreditsOverlay string `json:"credits_overlay,omitempty" yaml:"credits_overlay,omitempty"`
MaxStemLength *int `json:"max_stem_length,omitempty" yaml:"max_stem_length,omitempty"`
NamingScheme string `json:"naming_scheme,omitempty" yaml:"naming_scheme,omitempty"`
OutputExtension string `json:"output_extension,omitempty" yaml:"output_extension,omitempty"`
ConvertExtensions []string `json:"convert_extensions,omitempty" yaml:"convert_extensions,omitempty"`
}
type MusicDatasetConfig struct {
Source string `json:"source" yaml:"source"`
Output string `json:"output,omitempty" yaml:"output,omitempty"`
Prefix string `json:"prefix,omitempty" yaml:"prefix,omitempty"`
StageRoot string `json:"stage_root,omitempty" yaml:"stage_root,omitempty"`
CreditsRoot string `json:"credits_root,omitempty" yaml:"credits_root,omitempty"`
NamingScheme string `json:"naming_scheme,omitempty" yaml:"naming_scheme,omitempty"`
OutputExtension string `json:"output_extension,omitempty" yaml:"output_extension,omitempty"`
PackageMode string `json:"package_mode,omitempty" yaml:"package_mode,omitempty"`
ConvertExtensions []string `json:"convert_extensions,omitempty" yaml:"convert_extensions,omitempty"`
}
type TopDataConfig struct {
Source string `json:"source" yaml:"source"`
Build string `json:"build" yaml:"build"`
@@ -750,8 +715,6 @@ func (p *Project) ValidateLayout() error {
}
}
failures = append(failures, validateAutogenConfig(p.Config.Autogen)...)
failures = append(failures, validateMusicConfig(p.Config.Music)...)
hakNames := map[string]struct{}{}
for index, hak := range p.Config.HAKs {
name := strings.TrimSpace(hak.Name)
@@ -1059,46 +1022,9 @@ func (p *Project) Scan() error {
assetDir := p.AssetsDir()
var assetFiles []string
if assetDir != "" && filepath.Clean(assetDir) != filepath.Clean(p.Root) {
musicSourceExts := make(map[string]struct{})
for _, ext := range effective.Music.ConvertExtensions {
musicSourceExts[ext] = struct{}{}
}
musicSourceRoots := make(map[string]map[string]struct{})
for _, ds := range effective.Music.Datasets {
if strings.TrimSpace(ds.Source) == "" {
continue
}
exts := musicSourceRoots[ds.Source]
if exts == nil {
exts = make(map[string]struct{})
musicSourceRoots[ds.Source] = exts
}
for _, ext := range ds.ConvertExtensions {
exts[ext] = struct{}{}
}
for ext := range musicSourceExts {
exts[ext] = struct{}{}
}
}
var err error
assetFiles, _, err = scanDir(assetDir, func(path string) bool {
rel, err := filepath.Rel(assetDir, path)
if err != nil {
return false
}
rel = filepath.ToSlash(rel)
ext := strings.ToLower(filepath.Ext(rel))
if slices.Contains(effective.Inventory.AssetExtensions, ext) {
return true
}
for root, exts := range musicSourceRoots {
if !pathIsUnder(root, rel) {
continue
}
_, ok := exts[ext]
return ok
}
return false
return slices.Contains(effective.Inventory.AssetExtensions, strings.ToLower(filepath.Ext(path)))
})
if err != nil {
if errors.Is(err, os.ErrNotExist) {
@@ -1343,33 +1269,6 @@ func (p *Project) TopDataPackageTLKPath() string {
return filepath.Join(p.BuildDir(), p.TopDataPackageTLKName())
}
func (p *Project) MusicStageDir() string {
return p.rootPath(p.EffectiveConfig().Music.StageRoot)
}
func (p *Project) CreditsDir() string {
return p.rootPath(p.EffectiveConfig().Music.CreditsRoot)
}
func (p *Project) MusicToolPath(path string) string {
path = strings.TrimSpace(path)
if path == "" {
return ""
}
if filepath.IsAbs(path) {
return path
}
return filepath.Join(p.Root, filepath.FromSlash(path))
}
func (p *Project) MusicFFmpegPath() string {
return p.MusicToolPath(p.EffectiveConfig().Music.Tools.FFmpeg)
}
func (p *Project) MusicFFprobePath() string {
return p.MusicToolPath(p.EffectiveConfig().Music.Tools.FFprobe)
}
func (p *Project) AutogenCacheDir() string {
return p.rootPath(p.EffectiveConfig().Autogen.Cache.Root)
}
@@ -1459,36 +1358,6 @@ func normalizeConfig(cfg *Config) {
for i := range cfg.Autogen.Consumers {
cfg.Autogen.Consumers[i].Include = normalizeStringSlice(cfg.Autogen.Consumers[i].Include)
}
normalizeMusicConfig(&cfg.Music)
}
func normalizeMusicConfig(music *MusicConfig) {
if music.Defaults != nil {
music.Defaults.ConvertExtensions = normalizeExtensionSlice(music.Defaults.ConvertExtensions)
music.Defaults.OutputExtension = normalizeExtension(music.Defaults.OutputExtension)
}
if len(music.Prefixes) > 0 && len(music.Datasets) == 0 {
datasets := make(map[string]MusicDatasetConfig, len(music.Prefixes))
for dir, prefix := range music.Prefixes {
source := trimConfigSlashes(filepath.ToSlash(strings.TrimSpace(dir)))
if source == "" {
continue
}
key := strings.ReplaceAll(source, "/", "_")
datasets[key] = MusicDatasetConfig{
Source: source,
Prefix: strings.TrimSpace(prefix),
}
}
music.Datasets = datasets
}
for id, ds := range music.Datasets {
ds.Source = trimConfigSlashes(filepath.ToSlash(strings.TrimSpace(ds.Source)))
ds.Output = trimConfigSlashes(filepath.ToSlash(strings.TrimSpace(ds.Output)))
ds.OutputExtension = normalizeExtension(ds.OutputExtension)
ds.ConvertExtensions = normalizeExtensionSlice(ds.ConvertExtensions)
music.Datasets[id] = ds
}
}
func normalizeStringSlice(input []string) []string {
@@ -1982,68 +1851,6 @@ func validatePartsRowsFormat(format string) error {
return nil
}
func validateMusicConfig(cfg MusicConfig) []error {
var failures []error
if cfg.Defaults != nil {
if cfg.Defaults.MaxStemLength != nil && *cfg.Defaults.MaxStemLength < 3 {
failures = append(failures, errors.New("music.defaults.max_stem_length must be at least 3"))
}
if cfg.Defaults.NamingScheme != "" && cfg.Defaults.NamingScheme != "nwn_bmu" && cfg.Defaults.NamingScheme != "passthrough" {
failures = append(failures, fmt.Errorf("music.defaults.naming_scheme %q is not supported", cfg.Defaults.NamingScheme))
}
if cfg.Defaults.OutputExtension != "" && !strings.HasPrefix(cfg.Defaults.OutputExtension, ".") {
failures = append(failures, errors.New("music.defaults.output_extension must start with ."))
}
}
normalizedRoots := map[string]string{}
for root, prefix := range cfg.Prefixes {
normalizedRoot := trimConfigSlashes(filepath.ToSlash(strings.TrimSpace(root)))
if normalizedRoot == "" {
failures = append(failures, errors.New("music.prefixes contains an empty root"))
continue
}
if prior, exists := normalizedRoots[normalizedRoot]; exists {
failures = append(failures, fmt.Errorf("music.prefixes roots %q and %q normalize to the same path", prior, root))
}
normalizedRoots[normalizedRoot] = root
if strings.TrimSpace(prefix) == "" {
failures = append(failures, fmt.Errorf("music.prefixes[%q] must not be empty", root))
}
}
seenSources := map[string]string{}
for id, ds := range cfg.Datasets {
if strings.TrimSpace(id) == "" {
failures = append(failures, errors.New("music.datasets contains an empty key"))
} else if strings.Contains(id, "/") || strings.Contains(id, "\\") {
failures = append(failures, fmt.Errorf("music.datasets id %q must not contain path separators", id))
}
source := trimConfigSlashes(filepath.ToSlash(strings.TrimSpace(ds.Source)))
if source == "" {
failures = append(failures, fmt.Errorf("music.datasets[%q].source is required", id))
} else if pathEscapesRoot(source) {
failures = append(failures, fmt.Errorf("music.datasets[%q].source must not escape project root", id))
} else if prior, exists := seenSources[source]; exists {
failures = append(failures, fmt.Errorf("music.datasets[%q].source %q conflicts with dataset %q", id, ds.Source, prior))
} else {
seenSources[source] = id
}
output := trimConfigSlashes(filepath.ToSlash(strings.TrimSpace(ds.Output)))
if output != "" && pathEscapesRoot(output) {
failures = append(failures, fmt.Errorf("music.datasets[%q].output must not escape project root", id))
}
if ds.NamingScheme != "" && ds.NamingScheme != "nwn_bmu" && ds.NamingScheme != "passthrough" {
failures = append(failures, fmt.Errorf("music.datasets[%q].naming_scheme %q is not supported", id, ds.NamingScheme))
}
if ds.PackageMode != "" && ds.PackageMode != "hak_asset" && ds.PackageMode != "none" {
failures = append(failures, fmt.Errorf("music.datasets[%q].package_mode %q is not supported", id, ds.PackageMode))
}
if ds.OutputExtension != "" && !strings.HasPrefix(ds.OutputExtension, ".") {
failures = append(failures, fmt.Errorf("music.datasets[%q].output_extension must start with .", id))
}
}
return failures
}
func pathEscapesRoot(path string) bool {
clean := filepath.Clean(filepath.FromSlash(path))
return clean == ".." || strings.HasPrefix(clean, ".."+string(filepath.Separator))
+17 -65
View File
@@ -384,50 +384,30 @@ func TestValidateLayoutRejectsInvalidTopDataClassFeatInjections(t *testing.T) {
}
}
func TestScanUsesYAMLMusicConvertExtensionsWithinConfiguredDatasetRoots(t *testing.T) {
func TestDefaultAssetExtensionsKeepAuthoredBMUWithoutConversionSources(t *testing.T) {
for _, extension := range []string{".bmu", ".wav"} {
if !slices.Contains(AssetExtensions, extension) {
t.Errorf("default asset extensions must retain %s", extension)
}
}
for _, extension := range []string{".mp3", ".ogg"} {
if slices.Contains(AssetExtensions, extension) {
t.Errorf("default asset extensions must not include conversion source %s", extension)
}
}
}
func TestLoadRejectsRemovedMusicConfig(t *testing.T) {
root := t.TempDir()
if err := os.MkdirAll(filepath.Join(root, "assets", "audio", "westgate"), 0o755); err != nil {
t.Fatalf("mkdir westgate audio: %v", err)
}
if err := os.MkdirAll(filepath.Join(root, "assets", "audio", "other"), 0o755); err != nil {
t.Fatalf("mkdir other audio: %v", err)
}
writeProjectFile(t, filepath.Join(root, ConfigFile), `
module:
name: Test Module
resref: testmod
paths:
source: src
assets: assets
music:
defaults:
convert_extensions:
- .flac
datasets:
westgate_audio:
source: audio/westgate
convert_extensions:
- .aac
datasets: {}
`)
writeProjectFile(t, filepath.Join(root, "assets", "audio", "westgate", "theme.aac"), "aac")
writeProjectFile(t, filepath.Join(root, "assets", "audio", "westgate", "theme.flac"), "flac")
writeProjectFile(t, filepath.Join(root, "assets", "audio", "other", "ignored.aac"), "aac")
proj, err := Load(root)
if err != nil {
t.Fatalf("Load returned error: %v", err)
}
if err := proj.Scan(); err != nil {
t.Fatalf("Scan returned error: %v", err)
}
if !slices.Contains(proj.Inventory.AssetFiles, "audio/westgate/theme.aac") {
t.Fatalf("expected configured dataset AAC file to be discovered, got %#v", proj.Inventory.AssetFiles)
}
if !slices.Contains(proj.Inventory.AssetFiles, "audio/westgate/theme.flac") {
t.Fatalf("expected configured dataset FLAC file to be discovered, got %#v", proj.Inventory.AssetFiles)
}
if slices.Contains(proj.Inventory.AssetFiles, "audio/other/ignored.aac") {
t.Fatalf("did not expect non-dataset AAC file to be discovered, got %#v", proj.Inventory.AssetFiles)
if _, err := Load(root); err == nil || !strings.Contains(err.Error(), "music") {
t.Fatalf("expected strict YAML decoding to reject removed music config, got %v", err)
}
}
@@ -1780,34 +1760,6 @@ func TestValidateLayoutRejectsDuplicateHAKNames(t *testing.T) {
}
}
func TestValidateLayoutRejectsAmbiguousMusicPrefixes(t *testing.T) {
root := t.TempDir()
mkdirAll(t, filepath.Join(root, "assets"))
mkdirAll(t, filepath.Join(root, "build"))
proj := &Project{
Root: root,
Config: Config{
Module: ModuleConfig{Name: "Test", ResRef: "test"},
Paths: PathConfig{Assets: "assets", Build: "build"},
Music: MusicConfig{
Prefixes: map[string]string{
"envi/music": "mus_",
"/envi/music": "mus2_",
},
},
},
}
err := proj.ValidateLayout()
if err == nil {
t.Fatal("expected music prefix validation error")
}
if !strings.Contains(err.Error(), "normalize to the same path") {
t.Fatalf("expected ambiguous prefix validation error, got %v", err)
}
}
func TestScanAllowsMissingAssetsDir(t *testing.T) {
root := t.TempDir()
mkdirAll(t, filepath.Join(root, "src"))
+1 -30
View File
@@ -10,7 +10,6 @@ import (
"git.westgate.pw/ShadowsOverWestgate/sow-tools/internal/erf"
"git.westgate.pw/ShadowsOverWestgate/sow-tools/internal/gff"
"git.westgate.pw/ShadowsOverWestgate/sow-tools/internal/music"
"git.westgate.pw/ShadowsOverWestgate/sow-tools/internal/project"
)
@@ -101,8 +100,7 @@ func ValidateProject(p *project.Project) Report {
}
for _, rel := range p.Inventory.AssetFiles {
musicSource := isMusicSourceAsset(p, rel)
if hasUppercaseResourceName(rel) && !musicSource {
if hasUppercaseResourceName(rel) {
report.add(rel, "resource filenames should be lowercase", SeverityWarning)
}
base := strings.ToLower(strings.TrimSuffix(filepath.Base(rel), filepath.Ext(rel)))
@@ -116,9 +114,6 @@ func ValidateProject(p *project.Project) Report {
if _, exists := assetIndex[key]; !exists {
assetIndex[key] = rel
}
if musicSource {
continue
}
group, err := resolver.groupFor(rel)
if err != nil {
@@ -507,30 +502,6 @@ func hasAsset(name string, extensions []string, assets map[string]string) bool {
return false
}
func isMusicSourceAsset(p *project.Project, rel string) bool {
rel = filepath.ToSlash(strings.TrimSpace(rel))
ext := strings.ToLower(filepath.Ext(rel))
effective := p.EffectiveConfig()
for _, dataset := range effective.Music.Datasets {
if !music.PathIsUnder(dataset.Source, rel) {
continue
}
for _, candidate := range dataset.ConvertExtensions {
if ext == strings.ToLower(strings.TrimSpace(candidate)) {
return true
}
}
}
if music.PathIsUnder("envi/music", rel) {
for _, candidate := range effective.Music.ConvertExtensions {
if ext == strings.ToLower(strings.TrimSpace(candidate)) {
return true
}
}
}
return false
}
func isBuiltinScript(name string, builtinScriptPrefixes []string) bool {
lower := strings.ToLower(name)
for _, prefix := range builtinScriptPrefixes {
+1 -1
View File
@@ -29,7 +29,7 @@ os="$(uname -s)"; arch="$(uname -m)"
case "$os" in
Linux) os=linux ;;
Darwin) os=darwin ;;
*) echo "crucible: unsupported OS '$os' — install ffmpeg-free crucible manually" >&2; exit 1 ;;
*) echo "crucible: unsupported OS '$os' — install crucible manually" >&2; exit 1 ;;
esac
case "$arch" in
x86_64|amd64) arch=amd64 ;;