claude: fold in old sow-tools

AGENTS.md

@@ -8,8 +8,7 @@ alwaysApply: true
 This repo owns the **builder logic** for the migration: one Go module
 (`gitea.westgate.pw/ShadowsOverWestgate/sow-tools`) producing the `crucible`
 dispatcher and the `crucible-<name>` binaries (D11). Read
-[`../AGENTS.md`](../AGENTS.md) (migration hub) and
-[`../../KICKOFF_PROMPT.md`](../../KICKOFF_PROMPT.md) first.
+`../AGENTS.md` (migration hub) and `../../KICKOFF_PROMPT.md` first.
 
 ## What this repo owns / does not own
 
@@ -19,29 +18,33 @@ changelog. Does **not** own authored game content (that is `sow-module` /
 `sow-topdata` / `sow-assets-manifest`) or any production deploy authority (that
 is `sow-platform`).
 
-## Scaffold rules (Phase 5)
+## Rules
 
-1. **Source is not transplanted.** The migration hard rules forbid copying the
-   `internal/` packages from `gitea/sow-tools` automatically. The operator
-   migrates them at cutover; see [`docs/migration-from-nwn-tool.md`](docs/migration-from-nwn-tool.md).
-2. **Fail closed, never fake.** Unwired builders exit `70`. Do not stub a
-   builder to emit a placeholder artifact.
+1. **Migrated logic, not a fresh rewrite.** The `internal/` packages (`app`,
+   `pipeline`, `project`, `erf`, `gff`, `topdata`, `music`, `changelog`,
+   `validator`) were folded in from `gitea/sow-tools` at cutover; wired builders
+   delegate to `internal/app`'s command surface. Keep them in step with upstream
+   fixes rather than diverging silently.
+2. **Fail closed, never fake.** A builder with no migrated logic yet (`depot`)
+   exits `70`. Do not stub a builder to emit a placeholder artifact.
 3. **Binaries are not committed.** They are CI artifacts / image layers (D19).
    `/bin/`, `*.exe`, `nwn-tool`, `sow-toolkit` are gitignored.
 4. **No home-dir / `NWN_ROOT` guessing.** Builders take roots explicitly via
-   flag or env. See [`docs/consumer-contract.md`](docs/consumer-contract.md).
+   flag or env (project resolution is CWD-based, never `$HOME`). See
+   [`docs/consumer-contract.md`](docs/consumer-contract.md).
 5. **The registry is the command surface.** `internal/dispatch.Registry` is the
    single source of truth; keep it in sync with `cmd/` and
    [`docs/command-surface.md`](docs/command-surface.md). Adding a builder = a
    `cmd/crucible-<name>/main.go` shim + a `Registry` entry + a doc row.
 
-## Wiring a builder (operator cutover)
+## Wiring a builder
 
-1. Migrate the relevant `internal/` package(s) from `gitea/sow-tools`.
-2. Register a handler and flip the builder off the unwired path in
-   `internal/dispatch`.
+1. Ensure the relevant `internal/` package(s) cover the work.
+2. Add the legacy command(s) to the builder's `Legacy`/`Extra` set and set
+   `Wired: true` in `internal/dispatch`; the dispatcher delegates to
+   `app.Run`. `depot` is the remaining unwired builder.
 3. Add tests; keep outputs deterministic (same input → same bytes).
-4. `make check` must stay green; `make smoke` is updated to expect a wired exit.
+4. `make check` must stay green; update `make smoke` to expect the wired exit.
 
 ## Commands
 

README.md

@@ -27,16 +27,19 @@ The dispatcher and the standalone shims share one registry
 single-token command. The full legacy `nwn-tool` command surface and where each
 command lands is mapped in [`docs/command-surface.md`](docs/command-surface.md).
 
-## Status (Phase 5 scaffold)
+## Status (cutover performed)
 
-The suite **builds, vets, tests, and runs**, but every builder is **unwired**:
-running one fails closed with exit `70` and never fakes an artifact. The internal
-pipeline/topdata/erf/wiki/music packages from `gitea/sow-tools` are migrated by
-the operator at cutover — the workspace hard rules forbid transplanting that
-source automatically. See [`docs/migration-from-nwn-tool.md`](docs/migration-from-nwn-tool.md).
+The internal `app`/`pipeline`/`project`/`erf`/`gff`/`topdata`/`music`/`changelog`/
+`validator` packages from `gitea/sow-tools` have been migrated into this tree, and
+the `module`, `topdata`, `hak`, and `wiki` builders now **delegate to the migrated
+`nwn-tool` command surface** (mapped in
+[`docs/command-surface.md`](docs/command-surface.md)). `config` and `changelog`
+are global commands on the dispatcher. `depot` has no migrated logic yet, so it
+keeps the fail-closed path: exit `70`, never a faked artifact.
 
-This matches the downstream skeletons: `sow-module` / `sow-topdata` package
-scripts already fail closed until they can resolve a Crucible binary.
+See [`docs/migration-from-nwn-tool.md`](docs/migration-from-nwn-tool.md) for what
+was done and what remains (the consumer `--manifest/--source/--out` flag contract
+is the open Phase-6 item).
 
 ## Develop
 

docker/Dockerfile

@@ -6,15 +6,15 @@
 # and ships them on a static base. The `crucible` dispatcher is the entrypoint;
 # consumer CI can also call the standalone crucible-<name> binaries by path.
 #
-# NOTE (Phase 5 scaffold): the binaries are pure stdlib and fail closed until
-# the operator migrates the internal pipeline. When the music pipeline is wired
-# it needs ffmpeg at runtime — switch the final stage to a debian-slim base with
-# ffmpeg then (see docs/migration-from-nwn-tool.md).
+# NOTE: the internal pipeline is migrated and the music conversion path is wired,
+# so the runtime stage is debian-slim with ffmpeg on PATH (BMU encode/decode).
+# See docs/migration-from-nwn-tool.md.
 
 FROM golang:1.26-alpine AS build
 WORKDIR /src
 RUN apk add --no-cache git
-# go.sum is optional while the scaffold has no external deps.
+# go.sum is committed now that the migrated packages pull golang.org/x/text and
+# gopkg.in/yaml.v3; the glob keeps the build working if it is ever absent.
 COPY go.mod go.sum* ./
 RUN go mod download
 COPY . .
@@ -29,7 +29,14 @@ RUN set -eux; \
         -o "/out/${name}" "${dir}"; \
     done
 
-FROM gcr.io/distroless/static-debian12:nonroot
+FROM debian:12-slim
+# ffmpeg: the migrated music pipeline shells out to it for BMU conversion.
+# ca-certificates: builders fetch published manifests over HTTPS.
+RUN set -eux; \
+    apt-get update; \
+    apt-get install -y --no-install-recommends ffmpeg ca-certificates; \
+    rm -rf /var/lib/apt/lists/*; \
+    useradd --system --create-home --uid 65532 nonroot
 COPY --from=build /out/ /usr/local/bin/
 USER nonroot
 ENTRYPOINT ["/usr/local/bin/crucible"]

docs/command-surface.md

@@ -25,19 +25,21 @@ These legacy commands are **not** top-level builders:
 - `music *` (`list-datasets`, `scan`, `build`, `credits`, `validate`, `manifest`,
   `normalize`) → folds into the **module/hak build pipeline** (music BMUs are
   packed into HAKs at build time). Exposed as `crucible-module music ...` /
-  `crucible-hak music ...` when wired. Needs `ffmpeg` at runtime.
+  `crucible-hak music ...`. Needs `ffmpeg` at runtime (in the image).
 - `config *` (`validate`, `effective`, `inspect`, `explain`, `sources`) → a
-  **global** concern shared by every builder; exposed as a global subcommand
-  group, not its own binary.
-- `build-changelog` → release tooling; lands as a global `crucible changelog`
+  **global** concern shared by every builder; exposed as the global
+  `crucible config ...`, not its own binary.
+- `build-changelog` → release tooling; exposed as the global `crucible changelog`
   rather than a content builder.
 
-## Global commands (implemented in the scaffold)
+## Global commands (implemented)
 
 ```text
 crucible help | -h          usage + builder list
 crucible version | -V       build version (git sha via -ldflags)
 crucible list               builders, one per line: name<TAB>bin<TAB>summary
+crucible config [args]      -> legacy `config` command group
+crucible changelog [args]   -> legacy `build-changelog`
 ```
 
 `crucible list` is machine-readable so CI can enumerate builders without parsing

docs/migration-from-nwn-tool.md

@@ -1,9 +1,18 @@
-# Migrating from `nwn-tool` to Crucible (operator cutover)
+# Migrating from `nwn-tool` to Crucible (cutover — DONE)
 
 The legacy toolchain lives at `gitea/sow-tools` as a single `nwn-tool` binary
 (entry `cmd/nwn-tool`, logic under `internal/`). This repo is the Crucible
-rewrite. Per the migration hard rules, the `internal/` source is **not**
-transplanted automatically — the operator migrates it. This is the checklist.
+rewrite. The cutover below has been **performed**: the `internal/` packages are
+migrated and the `module`/`topdata`/`hak`/`wiki` builders delegate to them. The
+checklist is kept as the record of what was done.
+
+> **Status.** Steps 1–8 are complete. `depot` is the one remaining unwired
+> builder (no legacy source — it was shell `mc`/S3). The open follow-up is the
+> consumer flag contract: `sow-module` / `sow-topdata` / `sow-assets-manifest`
+> wrappers invoke `crucible-<name>` with `--manifest/--source/--out` style flags
+> (see [`consumer-contract.md`](consumer-contract.md)), whereas the migrated
+> commands use the legacy CWD/project model. Reconciling the two is the Phase-6
+> consumer task and is **not** part of this cutover.
 
 ## Why a clean scaffold first
 

go.mod

@@ -1,3 +1,8 @@
 module git.westgate.pw/ShadowsOverWestgate/sow-tools
 
 go 1.26.0
+
+require (
+	golang.org/x/text v0.35.0
+	gopkg.in/yaml.v3 v3.0.1
+)

go.sum

@@ -0,0 +1,6 @@
+golang.org/x/text v0.35.0 h1:JOVx6vVDFokkpaq1AEptVzLTpDe9KGpj5tR4/X+ybL8=
+golang.org/x/text v0.35.0/go.mod h1:khi/HExzZJ2pGnjenulevKNX1W67CUy0AsXcNubPGCA=
+gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM=
+gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
+gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
+gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=

internal/app/app_test.go

@@ -0,0 +1,574 @@
+package app
+
+import (
+	"bytes"
+	"os"
+	"path/filepath"
+	"strings"
+	"testing"
+	"time"
+
+	"git.westgate.pw/ShadowsOverWestgate/sow-tools/internal/pipeline"
+)
+
+func TestRunBuildTopPackageUsesCachedCompiledOutputs(t *testing.T) {
+	root := t.TempDir()
+	mkdirAll(t, filepath.Join(root, "build"))
+	mkdirAll(t, filepath.Join(root, ".cache", "2da"))
+	mkdirAll(t, filepath.Join(root, "topdata", "assets", "gui"))
+	mkdirAll(t, filepath.Join(root, "topdata", "data", "repadjust"))
+
+	writeFile(t, filepath.Join(root, "nwn-tool.yaml"), `
+module:
+  name: Test Module
+  resref: test_module
+topdata:
+  source: topdata
+  build: .cache
+  package_hak: sow_top.hak
+  package_tlk: sow_tlk.tlk
+`)
+	writeFile(t, filepath.Join(root, ".cache", "2da", "repadjust.2da"), "2DA V2.0\n\n      Label\n0     TEST_LABEL\n")
+	writeFile(t, filepath.Join(root, "build", "sow_tlk.tlk"), "compiled tlk")
+	writeFile(t, filepath.Join(root, "topdata", "assets", "gui", "testicon.png"), "icon-data")
+	writeFile(t, filepath.Join(root, "topdata", "data", "repadjust", "base.json"), `{
+  "output": "repadjust.2da",
+  "columns": ["Label"],
+  "rows": [{"id": 0, "Label": "TEST_LABEL"}]
+}`+"\n")
+
+	sourceTime := time.Now().Add(-2 * time.Hour)
+	outputTime := time.Now().Add(-1 * time.Hour)
+	setTreeTime(t, filepath.Join(root, "topdata"), sourceTime)
+	setFileTime(t, filepath.Join(root, ".cache", "2da", "repadjust.2da"), outputTime)
+	setFileTime(t, filepath.Join(root, "build", "sow_tlk.tlk"), outputTime)
+
+	var stdout bytes.Buffer
+	ctx := context{
+		stdout: &stdout,
+		stderr: &bytes.Buffer{},
+		cwd:    root,
+		args:   []string{"build-top-package"},
+	}
+
+	if err := runBuildTopPackage(ctx); err != nil {
+		t.Fatalf("runBuildTopPackage failed: %v", err)
+	}
+	output := stdout.String()
+	if !strings.Contains(output, "top package hak: build/sow_top.hak") {
+		t.Fatalf("expected build-top-package output, got %q", output)
+	}
+	if strings.Contains(output, "[build-top-package]") {
+		t.Fatalf("did not expect raw progress lines in normal output, got %q", output)
+	}
+	if _, err := os.Stat(filepath.Join(root, "build", "sow_top.hak")); err != nil {
+		t.Fatalf("expected packaged hak output: %v", err)
+	}
+}
+
+func TestRunBuildHAKsEmitsCompactSummary(t *testing.T) {
+	root := t.TempDir()
+	mkdirAll(t, filepath.Join(root, "assets", "envi", "music", "westgate"))
+	mkdirAll(t, filepath.Join(root, "build"))
+	mkdirAll(t, filepath.Join(root, "src"))
+
+	writeFile(t, filepath.Join(root, "nwn-tool.yaml"), `
+module:
+  name: Test Module
+  resref: testmod
+paths:
+  source: src
+  assets: assets
+  build: build
+haks:
+  - name: envi
+    priority: 1
+    max_bytes: 1048576
+    split: false
+    include:
+      - envi/**
+`)
+	writeFile(t, filepath.Join(root, "assets", "envi", "music", "westgate", "AleandAnecdotes.mp3"), "source-mp3")
+	writeFile(t, filepath.Join(root, "assets", "envi", "music", "westgate", "CREDITS.md"), "# NWN Music Pack Credits\n\n| Artist | Title | Output File | Original File | Album | Date | License / Copyright | Notes |\n|---|---|---|---|---|---|---|---|\n| Darren Curtis | Ale and Anecdotes | `mus_wg_andnc.bmu` | `AleandAnecdotes.mp3` | Darren's Commercially Free Music |  | Darren Curtis Music |  |\n")
+
+	ffprobePath := filepath.Join(root, "ffprobe")
+	writeFile(t, ffprobePath, "#!/bin/sh\nprintf '%s\\n' '{\"format\":{\"tags\":{\"artist\":\"Unknown\",\"title\":\"Broken Metadata\"}}}'\n")
+	if err := os.Chmod(ffprobePath, 0o755); err != nil {
+		t.Fatalf("chmod ffprobe: %v", err)
+	}
+
+	ffmpegPath := filepath.Join(root, "ffmpeg")
+	writeFile(t, ffmpegPath, "#!/bin/sh\nfor last; do :; done\nprintf 'bmu-data' > \"$last\"\n")
+	if err := os.Chmod(ffmpegPath, 0o755); err != nil {
+		t.Fatalf("chmod ffmpeg: %v", err)
+	}
+
+	t.Setenv("SOW_FFMPEG", ffmpegPath)
+	t.Setenv("SOW_FFPROBE", ffprobePath)
+
+	var stdout bytes.Buffer
+	ctx := context{
+		stdout: &stdout,
+		stderr: &bytes.Buffer{},
+		cwd:    root,
+		args:   []string{"build-haks"},
+	}
+
+	if err := runBuildHAKs(ctx); err != nil {
+		t.Fatalf("runBuildHAKs failed: %v", err)
+	}
+
+	output := stdout.String()
+	if !strings.Contains(output, "Build HAKs ----------") {
+		t.Fatalf("expected build header, got %q", output)
+	}
+	if !strings.Contains(output, "mapped: 1 music file(s); use --verbose to list mappings") {
+		t.Fatalf("expected compact mapping summary, got %q", output)
+	}
+	if strings.Contains(output, "AleandAnecdotes.mp3 -> mus_wg_andnc.bmu") {
+		t.Fatalf("did not expect verbose mapping in normal mode, got %q", output)
+	}
+	if !strings.Contains(output, "manifest: build/haks.json") {
+		t.Fatalf("expected relative manifest path, got %q", output)
+	}
+}
+
+func setTreeTime(t *testing.T, root string, modTime time.Time) {
+	t.Helper()
+	err := filepath.WalkDir(root, func(path string, d os.DirEntry, walkErr error) error {
+		if walkErr != nil {
+			return walkErr
+		}
+		return os.Chtimes(path, modTime, modTime)
+	})
+	if err != nil {
+		t.Fatalf("set tree time under %s: %v", root, err)
+	}
+}
+
+func TestRunBuildHAKsVerboseListsMappings(t *testing.T) {
+	root := t.TempDir()
+	mkdirAll(t, filepath.Join(root, "assets", "envi", "music", "westgate"))
+	mkdirAll(t, filepath.Join(root, "build"))
+	mkdirAll(t, filepath.Join(root, "src"))
+
+	writeFile(t, filepath.Join(root, "nwn-tool.yaml"), `
+module:
+  name: Test Module
+  resref: testmod
+paths:
+  source: src
+  assets: assets
+  build: build
+haks:
+  - name: envi
+    priority: 1
+    max_bytes: 1048576
+    split: false
+    include:
+      - envi/**
+`)
+	writeFile(t, filepath.Join(root, "assets", "envi", "music", "westgate", "AleandAnecdotes.mp3"), "source-mp3")
+	writeFile(t, filepath.Join(root, "assets", "envi", "music", "westgate", "CREDITS.md"), "# NWN Music Pack Credits\n\n| Artist | Title | Output File | Original File | Album | Date | License / Copyright | Notes |\n|---|---|---|---|---|---|---|---|\n| Darren Curtis | Ale and Anecdotes | `mus_wg_andnc.bmu` | `AleandAnecdotes.mp3` | Darren's Commercially Free Music |  | Darren Curtis Music |  |\n")
+
+	ffprobePath := filepath.Join(root, "ffprobe")
+	writeFile(t, ffprobePath, "#!/bin/sh\nprintf '%s\\n' '{\"format\":{\"tags\":{\"artist\":\"Unknown\",\"title\":\"Broken Metadata\"}}}'\n")
+	if err := os.Chmod(ffprobePath, 0o755); err != nil {
+		t.Fatalf("chmod ffprobe: %v", err)
+	}
+
+	ffmpegPath := filepath.Join(root, "ffmpeg")
+	writeFile(t, ffmpegPath, "#!/bin/sh\nfor last; do :; done\nprintf 'bmu-data' > \"$last\"\n")
+	if err := os.Chmod(ffmpegPath, 0o755); err != nil {
+		t.Fatalf("chmod ffmpeg: %v", err)
+	}
+
+	t.Setenv("SOW_FFMPEG", ffmpegPath)
+	t.Setenv("SOW_FFPROBE", ffprobePath)
+
+	var stdout bytes.Buffer
+	ctx := context{
+		stdout: &stdout,
+		stderr: &bytes.Buffer{},
+		cwd:    root,
+		args:   []string{"build-haks", "--verbose"},
+	}
+
+	if err := runBuildHAKs(ctx); err != nil {
+		t.Fatalf("runBuildHAKs failed: %v", err)
+	}
+
+	output := stdout.String()
+	if !strings.Contains(output, "mappings:") {
+		t.Fatalf("expected verbose mappings header, got %q", output)
+	}
+	if !strings.Contains(output, "AleandAnecdotes.mp3 -> mus_wg_andnc.bmu") {
+		t.Fatalf("expected verbose mapping output, got %q", output)
+	}
+	if !strings.Contains(output, "wrote: envi (1 assets)") {
+		t.Fatalf("expected verbose archive action, got %q", output)
+	}
+}
+
+func TestTopdataConsoleSuppressesProgressInNormalMode(t *testing.T) {
+	var stdout bytes.Buffer
+	console := &topdataConsole{
+		stdout:       &stdout,
+		projectRoot:  "/workspace/project",
+		projectName:  "Test Module",
+		commandName:  "build-topdata",
+		commandLabel: "Build Topdata",
+		level:        logLevelNormal,
+	}
+
+	console.progress("Packaging compiled topdata resources into sow_top.hak...")
+
+	if stdout.String() != "" {
+		t.Fatalf("expected no normal-mode progress output, got %q", stdout.String())
+	}
+}
+
+func TestSpinnerEnabledForHonorsPlainTTYMode(t *testing.T) {
+	t.Setenv("SOW_TOOLS_TTY_MODE", "plain")
+	if spinnerEnabledFor(&bytes.Buffer{}, logLevelNormal) {
+		t.Fatal("expected plain tty mode to disable spinner output")
+	}
+}
+
+func TestTopdataConsoleDebugProgressAndRelativePaths(t *testing.T) {
+	var stdout bytes.Buffer
+	console := &topdataConsole{
+		stdout:       &stdout,
+		projectRoot:  "/workspace/project",
+		projectName:  "Test Module",
+		commandName:  "deploy-wiki",
+		commandLabel: "Deploy Wiki",
+		level:        logLevelDebug,
+	}
+
+	console.progress("NodeBB wiki plan: create 1, update 2, skip 3, stale 4, archive 5, purge 6, drift 0")
+	console.emitWikiDeployResult(10, 1, 2, 3, 4, 5, 6, 0, "/workspace/project/build/wiki/deploy-manifest.json")
+
+	output := stdout.String()
+	if !strings.Contains(output, "[debug] NodeBB wiki plan: create 1, update 2, skip 3, stale 4, archive 5, purge 6, drift 0") {
+		t.Fatalf("expected debug progress line, got %q", output)
+	}
+	if !strings.Contains(output, "archived: 5") {
+		t.Fatalf("expected archived deploy count, got %q", output)
+	}
+	if !strings.Contains(output, "purged: 6") {
+		t.Fatalf("expected purged deploy count, got %q", output)
+	}
+	if !strings.Contains(output, "manifest: build/wiki/deploy-manifest.json") {
+		t.Fatalf("expected relative deploy manifest path, got %q", output)
+	}
+}
+
+func TestParseDeployWikiHelpListsPurgeStalePolicy(t *testing.T) {
+	_, err := parseDeployWikiArgs("deploy-wiki", []string{"--help"})
+	if err == nil || !strings.Contains(err.Error(), "--stale-policy <report|archive|purge>") {
+		t.Fatalf("expected deploy-wiki help to list purge stale policy, got %v", err)
+	}
+}
+
+func TestParseDeployWikiResetManagedNamespacesFlag(t *testing.T) {
+	opts, err := parseDeployWikiArgs("deploy-wiki", []string{"--reset-managed-namespaces"})
+	if err != nil {
+		t.Fatalf("parse deploy wiki reset flag: %v", err)
+	}
+	if !opts.ResetManagedNamespaces {
+		t.Fatalf("expected --reset-managed-namespaces to enable namespace reset")
+	}
+}
+
+func TestProjectConsoleSuppressesBuildModuleProgressInNormalMode(t *testing.T) {
+	var stdout bytes.Buffer
+	console := &projectConsole{
+		stdout:       &stdout,
+		projectRoot:  "/workspace/project",
+		projectName:  "Test Module",
+		commandName:  "build-module",
+		commandLabel: "Build Module",
+		level:        logLevelNormal,
+	}
+
+	console.progress("Writing module archive...")
+
+	if stdout.String() != "" {
+		t.Fatalf("expected no normal-mode progress output, got %q", stdout.String())
+	}
+}
+
+func TestProjectConsoleEmitsRelativePaths(t *testing.T) {
+	var stdout bytes.Buffer
+	console := &projectConsole{
+		stdout:       &stdout,
+		projectRoot:  "/workspace/project",
+		projectName:  "Test Module",
+		commandName:  "compare",
+		commandLabel: "Compare",
+		level:        logLevelNormal,
+	}
+
+	console.emitCompareResult(pipeline.CompareResult{
+		ModulePath: "/workspace/project/build/test.mod",
+		HAKPaths:   []string{"/workspace/project/build/core.hak"},
+		Checked:    42,
+	})
+
+	output := stdout.String()
+	if !strings.Contains(output, "module: build/test.mod") {
+		t.Fatalf("expected relative module path, got %q", output)
+	}
+}
+
+func TestRunMusicScanUsesConfiguredDatasetsWithoutWriting(t *testing.T) {
+	root := t.TempDir()
+	mkdirAll(t, filepath.Join(root, "assets", "audio", "westgate"))
+	mkdirAll(t, filepath.Join(root, "build"))
+	writeFile(t, filepath.Join(root, "nwn-tool.yaml"), `
+module:
+  name: Test Module
+  resref: testmod
+paths:
+  assets: assets
+  build: build
+music:
+  datasets:
+    westgate_audio:
+      source: audio/westgate
+      output: generated/music
+      prefix: wg_
+`)
+	writeFile(t, filepath.Join(root, "assets", "audio", "westgate", "Theme Song.mp3"), "source-mp3")
+
+	var stdout bytes.Buffer
+	ctx := context{
+		stdout: &stdout,
+		stderr: &bytes.Buffer{},
+		cwd:    root,
+		args:   []string{"music", "scan", "--dataset", "westgate_audio"},
+	}
+
+	if err := runMusic(ctx); err != nil {
+		t.Fatalf("runMusic failed: %v", err)
+	}
+	output := stdout.String()
+	if !strings.Contains(output, "music scan") || !strings.Contains(output, "tracks: 1") {
+		t.Fatalf("unexpected music scan output: %q", output)
+	}
+	if _, err := os.Stat(filepath.Join(root, ".cache", "credits")); !os.IsNotExist(err) {
+		t.Fatalf("music scan should not write credits artifacts, stat err=%v", err)
+	}
+}
+
+func TestRunMusicListDatasets(t *testing.T) {
+	root := t.TempDir()
+	mkdirAll(t, filepath.Join(root, "assets"))
+	mkdirAll(t, filepath.Join(root, "build"))
+	writeFile(t, filepath.Join(root, "nwn-tool.yaml"), `
+module:
+  name: Test Module
+  resref: testmod
+paths:
+  assets: assets
+music:
+  datasets:
+    westgate_audio:
+      source: audio/westgate
+      output: generated/music
+      prefix: wg_
+`)
+
+	var stdout bytes.Buffer
+	ctx := context{
+		stdout: &stdout,
+		stderr: &bytes.Buffer{},
+		cwd:    root,
+		args:   []string{"music", "list-datasets"},
+	}
+
+	if err := runMusic(ctx); err != nil {
+		t.Fatalf("runMusic failed: %v", err)
+	}
+	if !strings.Contains(stdout.String(), "westgate_audio: source=audio/westgate output=generated/music prefix=wg_") {
+		t.Fatalf("unexpected dataset list: %q", stdout.String())
+	}
+}
+
+func TestRunConfigEffectiveJSONShowsDefaultsAndProvenance(t *testing.T) {
+	root := t.TempDir()
+	writeFile(t, filepath.Join(root, "nwn-tool.yaml"), `
+module:
+  name: Test Module
+  resref: testmod
+paths:
+  source: src
+`)
+
+	var stdout bytes.Buffer
+	ctx := context{
+		stdout: &stdout,
+		stderr: &bytes.Buffer{},
+		cwd:    root,
+		args:   []string{"config", "effective", "--json"},
+	}
+
+	if err := runConfig(ctx); err != nil {
+		t.Fatalf("runConfig failed: %v", err)
+	}
+	output := stdout.String()
+	if !strings.Contains(output, `"hak_manifest": "haks.json"`) {
+		t.Fatalf("expected HAK manifest default in effective config, got %q", output)
+	}
+	if !strings.Contains(output, `"paths.build":`) || !strings.Contains(output, `"toolkit default"`) {
+		t.Fatalf("expected default provenance in effective config, got %q", output)
+	}
+}
+
+func TestRunConfigExplainReportsYAMLSource(t *testing.T) {
+	root := t.TempDir()
+	writeFile(t, filepath.Join(root, "nwn-tool.yaml"), `
+module:
+  name: Test Module
+  resref: testmod
+paths:
+  build: output
+`)
+
+	var stdout bytes.Buffer
+	ctx := context{
+		stdout: &stdout,
+		stderr: &bytes.Buffer{},
+		cwd:    root,
+		args:   []string{"config", "explain", "paths.build"},
+	}
+
+	if err := runConfig(ctx); err != nil {
+		t.Fatalf("runConfig failed: %v", err)
+	}
+	output := stdout.String()
+	if !strings.Contains(output, "value: \"output\"") {
+		t.Fatalf("expected configured build value, got %q", output)
+	}
+	if !strings.Contains(output, "source: yaml") {
+		t.Fatalf("expected YAML source, got %q", output)
+	}
+}
+
+func TestRunConfigValidateDoesNotScanSourceInventory(t *testing.T) {
+	root := t.TempDir()
+	mkdirAll(t, filepath.Join(root, "build"))
+	writeFile(t, filepath.Join(root, "nwn-tool.yaml"), `
+module:
+  name: Test Module
+  resref: testmod
+paths:
+  source: missing-src
+  build: build
+`)
+
+	var stdout bytes.Buffer
+	ctx := context{
+		stdout: &stdout,
+		stderr: &bytes.Buffer{},
+		cwd:    root,
+		args:   []string{"config", "validate"},
+	}
+
+	if err := runConfig(ctx); err != nil {
+		t.Fatalf("runConfig failed: %v", err)
+	}
+	if !strings.Contains(stdout.String(), "config: ok") {
+		t.Fatalf("expected config validation output, got %q", stdout.String())
+	}
+}
+
+func TestRunConfigSourcesListsActiveOverrides(t *testing.T) {
+	root := t.TempDir()
+	writeFile(t, filepath.Join(root, "nwn-tool.yaml"), `
+module:
+  name: Test Module
+  resref: testmod
+paths:
+  source: src
+`)
+	t.Setenv("SOW_BUILD_HAKS_KEEP_EXISTING", "1")
+
+	var stdout bytes.Buffer
+	ctx := context{
+		stdout: &stdout,
+		stderr: &bytes.Buffer{},
+		cwd:    root,
+		args:   []string{"config", "sources"},
+	}
+
+	if err := runConfig(ctx); err != nil {
+		t.Fatalf("runConfig failed: %v", err)
+	}
+	output := stdout.String()
+	if !strings.Contains(output, "active overrides:") {
+		t.Fatalf("expected active overrides section, got %q", output)
+	}
+	if !strings.Contains(output, "build.keep_existing_haks=1") {
+		t.Fatalf("expected keep existing override, got %q", output)
+	}
+}
+
+func TestInlineFlagParsersRejectEmptyValues(t *testing.T) {
+	if _, err := parseBuildHAKArgs([]string{"--hak="}); err == nil || !strings.Contains(err.Error(), "--hak requires a value") {
+		t.Fatalf("expected empty --hak inline value error, got %v", err)
+	}
+	if _, err := parseMusicCommandArgs([]string{"--dataset="}); err == nil || !strings.Contains(err.Error(), "--dataset requires a value") {
+		t.Fatalf("expected empty --dataset inline value error, got %v", err)
+	}
+	if _, err := parseDeployWikiArgs("deploy-wiki", []string{"--endpoint="}); err == nil || !strings.Contains(err.Error(), "--endpoint requires a value") {
+		t.Fatalf("expected empty --endpoint inline value error, got %v", err)
+	}
+	if _, err := parseBuildChangelogArgs("build-changelog", []string{"--output="}); err == nil || !strings.Contains(err.Error(), "--output requires a value") {
+		t.Fatalf("expected empty --output inline value error, got %v", err)
+	}
+}
+
+func TestParseBuildChangelogArgsAcceptsInlineValues(t *testing.T) {
+	opts, err := parseBuildChangelogArgs("build-changelog", []string{
+		"--config=scripts/changelog.json",
+		"--output=CHANGELOG.md",
+		"--current-tag=v2",
+		"--previous-tag=v1",
+		"--api-base-url=https://gitea.example/api/v1",
+		"--token=secret",
+	})
+	if err != nil {
+		t.Fatalf("parseBuildChangelogArgs failed: %v", err)
+	}
+	if opts.configPath != "scripts/changelog.json" ||
+		opts.outputPath != "CHANGELOG.md" ||
+		opts.currentTag != "v2" ||
+		opts.previousTag != "v1" ||
+		opts.apiBaseURL != "https://gitea.example/api/v1" ||
+		opts.token != "secret" {
+		t.Fatalf("unexpected changelog opts: %#v", opts)
+	}
+}
+
+func mkdirAll(t *testing.T, path string) {
+	t.Helper()
+	if err := os.MkdirAll(path, 0o755); err != nil {
+		t.Fatalf("mkdir %s: %v", path, err)
+	}
+}
+
+func writeFile(t *testing.T, path, content string) {
+	t.Helper()
+	if err := os.WriteFile(path, []byte(content), 0o644); err != nil {
+		t.Fatalf("write %s: %v", path, err)
+	}
+}
+
+func setFileTime(t *testing.T, path string, modTime time.Time) {
+	t.Helper()
+	if err := os.Chtimes(path, modTime, modTime); err != nil {
+		t.Fatalf("set file time %s: %v", path, err)
+	}
+}

internal/changelog/changelog.go

@@ -0,0 +1,455 @@
+package changelog
+
+import (
+	"bytes"
+	"encoding/json"
+	"fmt"
+	"io"
+	"net/http"
+	"net/url"
+	"os"
+	"os/exec"
+	"path/filepath"
+	"regexp"
+	"sort"
+	"strings"
+)
+
+type Config struct {
+	RepoURL    string     `json:"repo_url"`
+	Categories []Category `json:"categories"`
+}
+
+type Category struct {
+	Title    string    `json:"title"`
+	Sections []Section `json:"sections"`
+}
+
+type Section struct {
+	Title string   `json:"title"`
+	Paths []string `json:"paths"`
+}
+
+type Options struct {
+	RepoRoot    string
+	ConfigPath  string
+	OutputPath  string
+	CurrentTag  string
+	PreviousTag string
+	APIBaseURL  string
+	Token       string
+	Stdout      io.Writer
+}
+
+type pullInfo struct {
+	AuthorName string
+}
+
+type generator struct {
+	repoRoot    string
+	config      Config
+	currentTag  string
+	previousTag string
+	apiBaseURL  string
+	repoURL     string
+	owner       string
+	name        string
+	token       string
+	httpClient  *http.Client
+	pullCache   map[string]pullInfo
+}
+
+type changelogEntry struct {
+	Title          string
+	ReferenceLabel string
+	ReferenceURL   string
+	ReferenceSort  string
+	AuthorName     string
+}
+
+var pullNumberPattern = regexp.MustCompile(`\(#([0-9]+)\)`)
+
+func Generate(opts Options) error {
+	repoRoot := strings.TrimSpace(opts.RepoRoot)
+	if repoRoot == "" {
+		detectedRepoRoot, err := gitOutput("", "rev-parse", "--show-toplevel")
+		if err != nil {
+			return fmt.Errorf("detect repo root: %w", err)
+		}
+		repoRoot = strings.TrimSpace(detectedRepoRoot)
+	}
+
+	configPath := opts.ConfigPath
+	if configPath == "" {
+		configPath = filepath.Join(repoRoot, "scripts", "changelog.json")
+	}
+	config, err := loadConfig(configPath)
+	if err != nil {
+		return err
+	}
+
+	currentTag := strings.TrimSpace(opts.CurrentTag)
+	if currentTag == "" {
+		currentTag, err = gitOutput(repoRoot, "describe", "--tags", "--exact-match")
+		if err != nil {
+			return fmt.Errorf("detect current tag: %w", err)
+		}
+		currentTag = strings.TrimSpace(currentTag)
+	}
+	if currentTag == "" {
+		return fmt.Errorf("could not determine current tag")
+	}
+
+	previousTag := strings.TrimSpace(opts.PreviousTag)
+	if previousTag == "" {
+		previousTag, err = previousTagFor(repoRoot, currentTag)
+		if err != nil {
+			return err
+		}
+	}
+
+	repoURL := strings.TrimSpace(config.RepoURL)
+	if repoURL == "" {
+		return fmt.Errorf("config %s is missing repo_url", configPath)
+	}
+	apiBaseURL := strings.TrimSpace(opts.APIBaseURL)
+	if apiBaseURL == "" {
+		if apiBaseURL = strings.TrimSpace(os.Getenv("GITEA_SERVER_URL")); apiBaseURL == "" {
+			apiBaseURL = deriveAPIBaseURL(repoURL)
+		}
+	}
+	apiBaseURL = normalizeAPIBaseURL(apiBaseURL)
+	if apiBaseURL == "" {
+		return fmt.Errorf("could not determine Gitea API base URL")
+	}
+
+	owner, name, err := ownerAndRepoFromURL(repoURL)
+	if err != nil {
+		return err
+	}
+
+	g := generator{
+		repoRoot:    repoRoot,
+		config:      config,
+		currentTag:  currentTag,
+		previousTag: previousTag,
+		apiBaseURL:  strings.TrimRight(apiBaseURL, "/"),
+		repoURL:     strings.TrimRight(repoURL, "/"),
+		owner:       owner,
+		name:        name,
+		token:       firstNonEmpty(opts.Token, os.Getenv("GITEA_API_TOKEN"), os.Getenv("GITEA_TOKEN"), os.Getenv("SOW_TOOLS_TOKEN")),
+		httpClient:  http.DefaultClient,
+		pullCache:   map[string]pullInfo{},
+	}
+
+	rendered, err := g.render()
+	if err != nil {
+		return err
+	}
+
+	if opts.OutputPath != "" {
+		if err := os.MkdirAll(filepath.Dir(opts.OutputPath), 0o755); err != nil {
+			return fmt.Errorf("create output directory: %w", err)
+		}
+		if err := os.WriteFile(opts.OutputPath, rendered, 0o644); err != nil {
+			return fmt.Errorf("write changelog: %w", err)
+		}
+		return nil
+	}
+
+	stdout := opts.Stdout
+	if stdout == nil {
+		stdout = os.Stdout
+	}
+	_, err = stdout.Write(rendered)
+	return err
+}
+
+func loadConfig(path string) (Config, error) {
+	var config Config
+	data, err := os.ReadFile(path)
+	if err != nil {
+		return Config{}, fmt.Errorf("read changelog config %s: %w", path, err)
+	}
+	if err := json.Unmarshal(data, &config); err != nil {
+		return Config{}, fmt.Errorf("parse changelog config %s: %w", path, err)
+	}
+	if len(config.Categories) == 0 {
+		return Config{}, fmt.Errorf("changelog config %s does not define any categories", path)
+	}
+	return config, nil
+}
+
+func previousTagFor(repoRoot string, currentTag string) (string, error) {
+	output, err := gitOutput(repoRoot, "for-each-ref", "--sort=-creatordate", "--format=%(refname:strip=2)", "refs/tags")
+	if err != nil {
+		return "", fmt.Errorf("list tags: %w", err)
+	}
+
+	tags := strings.Fields(output)
+	for idx, tag := range tags {
+		if tag != currentTag {
+			continue
+		}
+		if idx+1 < len(tags) {
+			return tags[idx+1], nil
+		}
+		return "", nil
+	}
+
+	return "", fmt.Errorf("current tag %s was not found in repository tag list", currentTag)
+}
+
+func deriveAPIBaseURL(repoURL string) string {
+	parsed, err := url.Parse(repoURL)
+	if err != nil || parsed.Scheme == "" || parsed.Host == "" {
+		return ""
+	}
+	return fmt.Sprintf("%s://%s/api/v1", parsed.Scheme, parsed.Host)
+}
+
+func normalizeAPIBaseURL(raw string) string {
+	raw = strings.TrimRight(strings.TrimSpace(raw), "/")
+	if raw == "" {
+		return ""
+	}
+	if strings.HasSuffix(raw, "/api/v1") {
+		return raw
+	}
+	return raw + "/api/v1"
+}
+
+func ownerAndRepoFromURL(repoURL string) (string, string, error) {
+	parsed, err := url.Parse(repoURL)
+	if err != nil {
+		return "", "", fmt.Errorf("parse repo_url %s: %w", repoURL, err)
+	}
+
+	parts := strings.Split(strings.Trim(parsed.Path, "/"), "/")
+	if len(parts) < 2 {
+		return "", "", fmt.Errorf("repo_url %s does not include owner/repo", repoURL)
+	}
+
+	owner := strings.TrimSpace(parts[0])
+	name := strings.TrimSuffix(strings.TrimSpace(parts[1]), ".git")
+	if owner == "" || name == "" {
+		return "", "", fmt.Errorf("repo_url %s does not include owner/repo", repoURL)
+	}
+	return owner, name, nil
+}
+
+func (g generator) render() ([]byte, error) {
+	publishDate, err := gitOutput(g.repoRoot, "log", "-1", "--format=%cs", g.currentTag)
+	if err != nil {
+		return nil, fmt.Errorf("resolve publish date: %w", err)
+	}
+	publishDate = strings.TrimSpace(publishDate)
+
+	var buf bytes.Buffer
+	fmt.Fprintf(&buf, "# %s - %s\n\n", g.currentTag, publishDate)
+	if g.previousTag != "" {
+		fmt.Fprintf(&buf, "_Compared against %s_\n\n", g.previousTag)
+	} else {
+		buf.WriteString("_Initial tagged release_\n\n")
+	}
+
+	for _, category := range g.config.Categories {
+		categoryText, err := g.renderCategory(category)
+		if err != nil {
+			return nil, err
+		}
+		if categoryText == "" {
+			continue
+		}
+		fmt.Fprintf(&buf, "## %s\n\n", category.Title)
+		buf.WriteString(categoryText)
+	}
+
+	return buf.Bytes(), nil
+}
+
+func (g generator) renderCategory(category Category) (string, error) {
+	var buf bytes.Buffer
+	for _, section := range category.Sections {
+		entries, err := g.sectionEntries(section)
+		if err != nil {
+			return "", err
+		}
+		if len(entries) == 0 {
+			continue
+		}
+		fmt.Fprintf(&buf, "### %s\n", section.Title)
+		for _, entry := range entries {
+			fmt.Fprintf(&buf, "- %s ([%s](%s)) - From %s\n", entry.Title, entry.ReferenceLabel, entry.ReferenceURL, entry.AuthorName)
+		}
+		buf.WriteString("\n")
+	}
+	return buf.String(), nil
+}
+
+func (g generator) sectionEntries(section Section) ([]changelogEntry, error) {
+	if len(section.Paths) == 0 {
+		return nil, nil
+	}
+
+	args := []string{"log", "--first-parent", "--pretty=format:%H%x1f%s%x1f%an"}
+	if g.previousTag != "" {
+		args = append(args, fmt.Sprintf("%s..%s", g.previousTag, g.currentTag))
+	} else {
+		args = append(args, g.currentTag)
+	}
+	args = append(args, "--")
+	args = append(args, section.Paths...)
+
+	output, err := gitOutput(g.repoRoot, args...)
+	if err != nil {
+		return nil, fmt.Errorf("list git log for section %s: %w", section.Title, err)
+	}
+
+	seen := map[string]struct{}{}
+	var entries []changelogEntry
+	for _, line := range strings.Split(output, "\n") {
+		if strings.TrimSpace(line) == "" {
+			continue
+		}
+		parts := strings.SplitN(line, "\x1f", 3)
+		commitHash := strings.TrimSpace(parts[0])
+		subject := ""
+		if len(parts) > 1 {
+			subject = strings.TrimSpace(parts[1])
+		}
+		fallbackAuthor := ""
+		if len(parts) > 2 {
+			fallbackAuthor = strings.TrimSpace(parts[2])
+		}
+		matches := pullNumberPattern.FindStringSubmatch(subject)
+		entryKey := commitHash
+		if len(matches) == 2 {
+			entryKey = "pr:" + matches[1]
+		}
+		if _, exists := seen[entryKey]; exists {
+			continue
+		}
+		seen[entryKey] = struct{}{}
+
+		entry := changelogEntry{
+			Title:      sanitizeSubject(subject),
+			AuthorName: fallbackAuthor,
+		}
+
+		if len(matches) == 2 {
+			pullNumber := matches[1]
+			pullInfo, err := g.pullDetails(pullNumber)
+			if err != nil {
+				return nil, err
+			}
+			authorName := strings.TrimSpace(pullInfo.AuthorName)
+			if authorName == "" {
+				authorName = fallbackAuthor
+			}
+			entry.ReferenceLabel = "#" + pullNumber
+			entry.ReferenceURL = fmt.Sprintf("%s/pulls/%s", g.repoURL, pullNumber)
+			entry.ReferenceSort = "pr:" + pullNumber
+			entry.AuthorName = authorName
+		} else {
+			entry.ReferenceLabel = shortCommitHash(commitHash)
+			entry.ReferenceURL = fmt.Sprintf("%s/commit/%s", g.repoURL, commitHash)
+			entry.ReferenceSort = "commit:" + commitHash
+		}
+
+		entries = append(entries, entry)
+	}
+
+	sort.Slice(entries, func(i, j int) bool {
+		if entries[i].Title == entries[j].Title {
+			return entries[i].ReferenceSort < entries[j].ReferenceSort
+		}
+		return entries[i].Title < entries[j].Title
+	})
+
+	return entries, nil
+}
+
+func sanitizeSubject(subject string) string {
+	title := strings.TrimSpace(subject)
+	title = pullNumberPattern.ReplaceAllString(title, "")
+	title = strings.TrimSpace(title)
+	title = strings.TrimPrefix(title, "Merge pull request")
+	title = strings.TrimSpace(title)
+	title = strings.Trim(title, `"'`)
+	title = strings.TrimSpace(title)
+	if title == "" {
+		return strings.TrimSpace(subject)
+	}
+	return title
+}
+
+func shortCommitHash(commitHash string) string {
+	commitHash = strings.TrimSpace(commitHash)
+	if len(commitHash) <= 7 {
+		return commitHash
+	}
+	return commitHash[:7]
+}
+
+func (g generator) pullDetails(pullNumber string) (pullInfo, error) {
+	if cached, ok := g.pullCache[pullNumber]; ok {
+		return cached, nil
+	}
+
+	endpoint := fmt.Sprintf("%s/repos/%s/%s/pulls/%s", g.apiBaseURL, url.PathEscape(g.owner), url.PathEscape(g.name), url.PathEscape(pullNumber))
+	req, err := http.NewRequest(http.MethodGet, endpoint, nil)
+	if err != nil {
+		return pullInfo{}, fmt.Errorf("build Gitea request for pull #%s: %w", pullNumber, err)
+	}
+	req.Header.Set("Accept", "application/json")
+	if g.token != "" {
+		req.Header.Set("Authorization", "token "+g.token)
+	}
+
+	resp, err := g.httpClient.Do(req)
+	if err != nil {
+		return pullInfo{}, fmt.Errorf("request Gitea pull #%s: %w", pullNumber, err)
+	}
+	defer resp.Body.Close()
+
+	if resp.StatusCode != http.StatusOK {
+		body, _ := io.ReadAll(io.LimitReader(resp.Body, 512))
+		return pullInfo{}, fmt.Errorf("request Gitea pull #%s: unexpected HTTP %d: %s", pullNumber, resp.StatusCode, strings.TrimSpace(string(body)))
+	}
+
+	var payload struct {
+		User struct {
+			FullName string `json:"full_name"`
+			Login    string `json:"login"`
+		} `json:"user"`
+	}
+	if err := json.NewDecoder(resp.Body).Decode(&payload); err != nil {
+		return pullInfo{}, fmt.Errorf("decode Gitea pull #%s: %w", pullNumber, err)
+	}
+
+	info := pullInfo{AuthorName: firstNonEmpty(strings.TrimSpace(payload.User.FullName), strings.TrimSpace(payload.User.Login))}
+	g.pullCache[pullNumber] = info
+	return info, nil
+}
+
+func gitOutput(repoRoot string, args ...string) (string, error) {
+	cmd := exec.Command("git", args...)
+	if repoRoot != "" {
+		cmd.Dir = repoRoot
+	}
+	output, err := cmd.CombinedOutput()
+	if err != nil {
+		return "", fmt.Errorf("%s: %s", strings.Join(cmd.Args, " "), strings.TrimSpace(string(output)))
+	}
+	return string(output), nil
+}
+
+func firstNonEmpty(values ...string) string {
+	for _, value := range values {
+		if strings.TrimSpace(value) != "" {
+			return value
+		}
+	}
+	return ""
+}

internal/changelog/changelog_test.go

@@ -0,0 +1,213 @@
+package changelog
+
+import (
+	"bytes"
+	"fmt"
+	"net/http"
+	"net/http/httptest"
+	"os"
+	"os/exec"
+	"path/filepath"
+	"strings"
+	"testing"
+)
+
+func TestSanitizeSubject(t *testing.T) {
+	t.Parallel()
+
+	tests := []struct {
+		name    string
+		subject string
+		want    string
+	}{
+		{
+			name:    "merge prefix with quotes",
+			subject: "Merge pull request 'Add classes overhaul (#123)'",
+			want:    "Add classes overhaul",
+		},
+		{
+			name:    "plain subject",
+			subject: "Add classes overhaul (#123)",
+			want:    "Add classes overhaul",
+		},
+		{
+			name:    "double quotes",
+			subject: `Merge pull request "Add classes overhaul (#123)"`,
+			want:    "Add classes overhaul",
+		},
+		{
+			name:    "direct push",
+			subject: "Fix direct push handling",
+			want:    "Fix direct push handling",
+		},
+	}
+
+	for _, tt := range tests {
+		tt := tt
+		t.Run(tt.name, func(t *testing.T) {
+			t.Parallel()
+			if got := sanitizeSubject(tt.subject); got != tt.want {
+				t.Fatalf("sanitizeSubject(%q) = %q, want %q", tt.subject, got, tt.want)
+			}
+		})
+	}
+}
+
+func TestShortCommitHash(t *testing.T) {
+	t.Parallel()
+
+	if got := shortCommitHash("1234567890abcdef"); got != "1234567" {
+		t.Fatalf("shortCommitHash() = %q, want %q", got, "1234567")
+	}
+	if got := shortCommitHash("1234567"); got != "1234567" {
+		t.Fatalf("shortCommitHash() preserved %q unexpectedly as %q", "1234567", got)
+	}
+}
+
+func TestOwnerAndRepoFromURL(t *testing.T) {
+	t.Parallel()
+
+	owner, repo, err := ownerAndRepoFromURL("https://gitea.example.test/org/repo.git")
+	if err != nil {
+		t.Fatalf("ownerAndRepoFromURL returned error: %v", err)
+	}
+	if owner != "org" || repo != "repo" {
+		t.Fatalf("ownerAndRepoFromURL returned %q/%q", owner, repo)
+	}
+}
+
+func TestDeriveAPIBaseURL(t *testing.T) {
+	t.Parallel()
+
+	got := deriveAPIBaseURL("https://gitea.example.test/org/repo")
+	want := "https://gitea.example.test/api/v1"
+	if got != want {
+		t.Fatalf("deriveAPIBaseURL() = %q, want %q", got, want)
+	}
+}
+
+func TestNormalizeAPIBaseURL(t *testing.T) {
+	t.Parallel()
+
+	tests := []struct {
+		input string
+		want  string
+	}{
+		{input: "https://gitea.example.test", want: "https://gitea.example.test/api/v1"},
+		{input: "https://gitea.example.test/api/v1", want: "https://gitea.example.test/api/v1"},
+	}
+
+	for _, tt := range tests {
+		if got := normalizeAPIBaseURL(tt.input); got != tt.want {
+			t.Fatalf("normalizeAPIBaseURL(%q) = %q, want %q", tt.input, got, tt.want)
+		}
+	}
+}
+
+func TestGenerateIncludesPullRequestsAndDirectPushes(t *testing.T) {
+	t.Parallel()
+
+	repoRoot := t.TempDir()
+	runGit(t, repoRoot, "init")
+	runGit(t, repoRoot, "config", "user.name", "Test User")
+	runGit(t, repoRoot, "config", "user.email", "test@example.com")
+
+	writeFile(t, filepath.Join(repoRoot, "content", "entry.txt"), "initial\n")
+	runGit(t, repoRoot, "add", ".")
+	runGit(t, repoRoot, "commit", "-m", "Initial import")
+	runGit(t, repoRoot, "tag", "v1.0.0")
+
+	writeFile(t, filepath.Join(repoRoot, "content", "entry.txt"), "direct\n")
+	runGit(t, repoRoot, "add", ".")
+	runGit(t, repoRoot, "commit", "-m", "Fix direct push handling")
+	directHash := strings.TrimSpace(runGit(t, repoRoot, "rev-parse", "HEAD"))
+
+	writeFile(t, filepath.Join(repoRoot, "content", "entry.txt"), "pull\n")
+	runGit(t, repoRoot, "add", ".")
+	runGit(t, repoRoot, "commit", "-m", "Add release summary (#12)")
+	runGit(t, repoRoot, "tag", "v1.1.0")
+
+	configPath := filepath.Join(repoRoot, "scripts", "changelog.json")
+	writeFile(t, configPath, `{
+  "repo_url": "REPO_URL",
+  "categories": [
+    {
+      "title": "Content",
+      "sections": [
+        {
+          "title": "Entries",
+          "paths": ["content/"]
+        }
+      ]
+    }
+  ]
+}`)
+
+	var pullRequests int
+	server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		if r.URL.Path != "/api/v1/repos/org/repo/pulls/12" {
+			t.Fatalf("unexpected API path %q", r.URL.Path)
+		}
+		pullRequests++
+		w.Header().Set("Content-Type", "application/json")
+		fmt.Fprint(w, `{"user":{"full_name":"Patch Author","login":"patch-author"}}`)
+	}))
+	defer server.Close()
+
+	repoURL := server.URL + "/org/repo"
+	configData, err := os.ReadFile(configPath)
+	if err != nil {
+		t.Fatalf("read changelog config: %v", err)
+	}
+	configData = bytes.ReplaceAll(configData, []byte("REPO_URL"), []byte(repoURL))
+	if err := os.WriteFile(configPath, configData, 0o644); err != nil {
+		t.Fatalf("write changelog config: %v", err)
+	}
+
+	var stdout bytes.Buffer
+	err = Generate(Options{
+		RepoRoot:    repoRoot,
+		ConfigPath:  configPath,
+		CurrentTag:  "v1.1.0",
+		PreviousTag: "v1.0.0",
+		APIBaseURL:  server.URL + "/api/v1",
+		Stdout:      &stdout,
+	})
+	if err != nil {
+		t.Fatalf("Generate() returned error: %v", err)
+	}
+
+	rendered := stdout.String()
+	if !strings.Contains(rendered, "- Add release summary ([#12]("+repoURL+"/pulls/12)) - From Patch Author") {
+		t.Fatalf("rendered changelog missing pull entry:\n%s", rendered)
+	}
+	if !strings.Contains(rendered, "- Fix direct push handling (["+shortCommitHash(directHash)+"]("+repoURL+"/commit/"+directHash+")) - From Test User") {
+		t.Fatalf("rendered changelog missing direct-push entry:\n%s", rendered)
+	}
+	if pullRequests != 1 {
+		t.Fatalf("expected 1 pull lookup, got %d", pullRequests)
+	}
+}
+
+func runGit(t *testing.T, repoRoot string, args ...string) string {
+	t.Helper()
+
+	cmd := exec.Command("git", args...)
+	cmd.Dir = repoRoot
+	output, err := cmd.CombinedOutput()
+	if err != nil {
+		t.Fatalf("git %s failed: %v\n%s", strings.Join(args, " "), err, output)
+	}
+	return string(output)
+}
+
+func writeFile(t *testing.T, path string, content string) {
+	t.Helper()
+
+	if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil {
+		t.Fatalf("MkdirAll(%q): %v", path, err)
+	}
+	if err := os.WriteFile(path, []byte(content), 0o644); err != nil {
+		t.Fatalf("WriteFile(%q): %v", path, err)
+	}
+}

internal/dispatch/dispatch.go

@@ -2,12 +2,11 @@
 // builders shared by the `crucible` dispatcher and the standalone
 // `crucible-<name>` shims.
 //
-// Scaffold contract (Phase 5): every builder is registered but UNWIRED. Running
-// a builder fails closed (exit 70) with an operator-cutover message; it never
-// fakes an artifact. The internal pipeline/topdata/erf/wiki/music packages from
-// gitea/sow-tools are migrated by the operator at cutover (the workspace hard
-// rules forbid transplanting source here). Once a builder's handler is wired,
-// flip its Wired flag and register the handler.
+// Cutover state (Phase 5 → 6): the internal pipeline/topdata/erf/wiki/music
+// packages migrated from gitea/sow-tools now live in this tree, so wired
+// builders delegate to internal/app's legacy command surface (mapped per
+// docs/command-surface.md). A builder with no migrated logic yet (depot) keeps
+// the Wired=false fail-closed path: exit 70, never a faked artifact.
 package dispatch
 
 import (
@@ -16,6 +15,7 @@ import (
 	"os"
 	"text/tabwriter"
 
+	"git.westgate.pw/ShadowsOverWestgate/sow-tools/internal/app"
 	"git.westgate.pw/ShadowsOverWestgate/sow-tools/internal/buildinfo"
 )
 
@@ -33,6 +33,21 @@ type Builder struct {
 	Bin     string   // standalone binary name, e.g. "crucible-module"
 	Summary string   // one-line description
 	Legacy  []string // nwn-tool commands this subsumes (migration aid; see docs/command-surface.md)
+	Extra   []string // additional callable subcommands beyond Legacy (e.g. cross-listed "music")
+	Wired   bool     // true once the builder delegates to migrated internal/app logic
+}
+
+// subcommands returns every legacy command a wired builder accepts: the
+// canonical Legacy set plus any cross-listed Extra commands.
+func (b Builder) subcommands() []string { return append(append([]string{}, b.Legacy...), b.Extra...) }
+
+func (b Builder) accepts(sub string) bool {
+	for _, s := range b.subcommands() {
+		if s == sub {
+			return true
+		}
+	}
+	return false
 }
 
 // Registry is the single source of truth for the Crucible command surface.
@@ -43,30 +58,39 @@ var Registry = []Builder{
 		Bin:     "crucible-depot",
 		Summary: "verify/move content-addressed depot blobs (SeaweedFS)",
 		Legacy:  nil,
+		Wired:   false, // no migrated logic yet; fails closed (exit 70)
 	},
 	{
 		Name:    "hak",
 		Bin:     "crucible-hak",
 		Summary: "pack/unpack ERF/HAK archives + hak manifests",
 		Legacy:  []string{"build-haks", "apply-hak-manifest"},
+		Extra:   []string{"music"}, // music BMUs are packed into HAKs (command-surface.md)
+		Wired:   true,
 	},
 	{
 		Name:    "module",
 		Bin:     "crucible-module",
 		Summary: "build/extract/validate/compare the .mod",
 		Legacy:  []string{"build", "build-module", "extract", "validate", "compare"},
+		// apply-hak-manifest is canonically a hak command but reachable here too;
+		// music is folded into the module build pipeline (command-surface.md).
+		Extra: []string{"apply-hak-manifest", "music"},
+		Wired: true,
 	},
 	{
 		Name:    "topdata",
 		Bin:     "crucible-topdata",
 		Summary: "compile 2da/tlk topdata + packages",
 		Legacy:  []string{"validate-topdata", "build-topdata", "build-top-package", "compare-topdata", "convert-topdata"},
+		Wired:   true,
 	},
 	{
 		Name:    "wiki",
 		Bin:     "crucible-wiki",
 		Summary: "render + deploy mechanical wiki pages",
 		Legacy:  []string{"build-wiki", "deploy-wiki"},
+		Wired:   true,
 	},
 }
 
@@ -102,6 +126,13 @@ func run(args []string, out, errw io.Writer) int {
 	case "list":
 		list(out)
 		return exitOK
+	case "config":
+		// Global concern shared by every builder (command-surface.md): the legacy
+		// `config` command group, surfaced verbatim.
+		return delegateLegacy(args, errw)
+	case "changelog":
+		// Release tooling: global alias for the legacy `build-changelog` command.
+		return delegateLegacy(append([]string{"build-changelog"}, args[1:]...), errw)
 	}
 	return runBuilder(args[0], args[1:], out, errw)
 }
@@ -120,19 +151,43 @@ func runBuilder(name string, args []string, out, errw io.Writer) int {
 			return exitOK
 		}
 	}
-	// Every builder is unwired in the Phase 5 scaffold: fail closed, never fake.
-	fmt.Fprintf(errw, unwiredMsg, b.Name, b.Bin)
-	return exitUnwired
+	if !b.Wired {
+		// No migrated logic yet (depot): fail closed, never fake an artifact.
+		fmt.Fprintf(errw, unwiredMsg, b.Name, b.Bin)
+		return exitUnwired
+	}
+	if len(args) == 0 {
+		fmt.Fprintf(errw, "crucible %s: a subcommand is required\n\n", b.Name)
+		builderHelp(errw, b)
+		return exitUsage
+	}
+	sub := args[0]
+	if !b.accepts(sub) {
+		fmt.Fprintf(errw, "crucible %s: unknown subcommand %q\n\n", b.Name, sub)
+		builderHelp(errw, b)
+		return exitUsage
+	}
+	// Delegate to the migrated legacy command surface. args[0] is already the
+	// legacy command name, so it is forwarded unchanged.
+	return delegateLegacy(args, errw)
+}
+
+// delegateLegacy runs a migrated nwn-tool command via internal/app and maps its
+// (code, err) result onto the dispatcher's exit code. app writes its own output
+// to os.Stdout/os.Stderr (the writers the entrypoints pass through), so only the
+// returned error is surfaced here.
+func delegateLegacy(args []string, errw io.Writer) int {
+	code, err := app.Run(args)
+	if err != nil {
+		fmt.Fprintln(errw, err)
+	}
+	return code
 }
 
 const unwiredMsg = `crucible %[1]s: not wired yet.
 
-The Crucible suite is scaffolded (Phase 5). The %[2]s implementation is migrated
-from gitea/sow-tools internal packages at operator cutover; the workspace hard
-rules forbid transplanting that source into this tree automatically.
-
-This command fails closed (exit 70) rather than producing a fake artifact.
-See docs/migration-from-nwn-tool.md.
+The %[2]s builder has no migrated logic in this tree yet, so it fails closed
+(exit 70) rather than producing a fake artifact. See docs/command-surface.md.
 `
 
 func usage(w io.Writer) {
@@ -147,6 +202,8 @@ func usage(w io.Writer) {
 	fmt.Fprintf(w, "  help | -h         show this help\n")
 	fmt.Fprintf(w, "  version | -V      show the build version\n")
 	fmt.Fprintf(w, "  list              list builders (one per line: name<TAB>bin<TAB>summary)\n")
+	fmt.Fprintf(w, "  config [args]     inspect/validate effective configuration\n")
+	fmt.Fprintf(w, "  changelog [args]  generate the release changelog\n")
 	fmt.Fprintf(w, "\nNWN_ROOT must be passed explicitly (env or flag); crucible never guesses\n")
 	fmt.Fprintf(w, "from $HOME. See docs/consumer-contract.md.\n")
 }
@@ -161,15 +218,15 @@ func list(w io.Writer) {
 
 func builderHelp(w io.Writer, b Builder) {
 	fmt.Fprintf(w, "crucible %s (%s) — %s\n\n", b.Name, b.Bin, b.Summary)
-	if len(b.Legacy) > 0 {
-		fmt.Fprintf(w, "subsumes nwn-tool commands: ")
-		for i, c := range b.Legacy {
-			if i > 0 {
-				fmt.Fprintf(w, ", ")
-			}
-			fmt.Fprintf(w, "%s", c)
-		}
-		fmt.Fprintf(w, "\n\n")
+	if !b.Wired {
+		fmt.Fprintf(w, "status: not wired — no migrated logic yet; fails closed (exit 70).\n")
+		fmt.Fprintf(w, "See docs/command-surface.md.\n")
+		return
 	}
-	fmt.Fprintf(w, "status: scaffolded, not wired (Phase 5). See docs/command-surface.md.\n")
+	fmt.Fprintf(w, "usage: crucible %s <subcommand> [args]   (or: %s <subcommand> [args])\n\n", b.Name, b.Bin)
+	fmt.Fprintf(w, "subcommands:\n")
+	for _, c := range b.subcommands() {
+		fmt.Fprintf(w, "  %s\n", c)
+	}
+	fmt.Fprintf(w, "\nstatus: wired to migrated nwn-tool logic. See docs/command-surface.md.\n")
 }

internal/dispatch/dispatch_test.go

@@ -55,8 +55,11 @@ func TestUnknownBuilderFailsUsage(t *testing.T) {
 	}
 }
 
-func TestEveryBuilderFailsClosed(t *testing.T) {
+func TestUnwiredBuilderFailsClosed(t *testing.T) {
 	for _, b := range Registry {
+		if b.Wired {
+			continue
+		}
 		var out, errw bytes.Buffer
 		// Via dispatcher.
 		if code := run([]string{b.Name}, &out, &errw); code != exitUnwired {
@@ -74,6 +77,28 @@ func TestEveryBuilderFailsClosed(t *testing.T) {
 	}
 }
 
+func TestWiredBuilderRejectsBadInvocation(t *testing.T) {
+	for _, b := range Registry {
+		if !b.Wired {
+			continue
+		}
+		// No subcommand is a usage error (it must never silently delegate).
+		var out, errw bytes.Buffer
+		if code := runBuilder(b.Name, nil, &out, &errw); code != exitUsage {
+			t.Errorf("crucible %s (no subcommand): exit=%d want %d", b.Name, code, exitUsage)
+		}
+		// Unknown subcommand is a usage error, not a delegate.
+		out.Reset()
+		errw.Reset()
+		if code := runBuilder(b.Name, []string{"frobnicate"}, &out, &errw); code != exitUsage {
+			t.Errorf("crucible %s frobnicate: exit=%d want %d", b.Name, code, exitUsage)
+		}
+		if !strings.Contains(errw.String(), "unknown subcommand") {
+			t.Errorf("crucible %s frobnicate: stderr=%q", b.Name, errw.String())
+		}
+	}
+}
+
 func TestBuilderHelpIsOK(t *testing.T) {
 	for _, b := range Registry {
 		var out, errw bytes.Buffer
@@ -85,3 +110,28 @@ func TestBuilderHelpIsOK(t *testing.T) {
 		}
 	}
 }
+
+// Every legacy command named in command-surface.md must have exactly one
+// canonical home (Builder.Legacy), so the migrated surface has no gaps or
+// ambiguous homes.
+func TestLegacyCommandsHaveExactlyOneHome(t *testing.T) {
+	legacy := []string{
+		"build", "build-module", "extract", "validate", "compare",
+		"build-haks", "apply-hak-manifest",
+		"validate-topdata", "build-topdata", "build-top-package", "compare-topdata", "convert-topdata",
+		"build-wiki", "deploy-wiki",
+	}
+	for _, cmd := range legacy {
+		homes := 0
+		for _, b := range Registry {
+			for _, c := range b.Legacy {
+				if c == cmd {
+					homes++
+				}
+			}
+		}
+		if homes != 1 {
+			t.Errorf("legacy command %q has %d canonical homes, want 1", cmd, homes)
+		}
+	}
+}

internal/erf/erf.go

@@ -0,0 +1,424 @@
+package erf
+
+import (
+	"bytes"
+	"encoding/binary"
+	"fmt"
+	"io"
+	"os"
+	"sort"
+	"strings"
+)
+
+const (
+	headerSize = 160
+	versionV10 = "V1.0"
+	strRefNone = 0xFFFFFFFF
+	typeMOD    = "MOD "
+	typeHAK    = "HAK "
+)
+
+type Archive struct {
+	FileType  string
+	Version   string
+	Resources []Resource
+}
+
+type Resource struct {
+	Name       string
+	Type       uint16
+	Data       []byte
+	SourcePath string
+	Size       int64
+}
+
+type header struct {
+	FileType              [4]byte
+	Version               [4]byte
+	LanguageCount         uint32
+	LocalizedStringSize   uint32
+	EntryCount            uint32
+	LocalizedStringOffset uint32
+	KeyListOffset         uint32
+	ResourceListOffset    uint32
+	BuildYear             uint32
+	BuildDay              uint32
+	DescriptionStrRef     uint32
+	Reserved              [116]byte
+}
+
+type keyEntry struct {
+	ResRef       [16]byte
+	ResourceID   uint32
+	ResourceType uint16
+	Unused       uint16
+}
+
+type resourceEntry struct {
+	Offset uint32
+	Size   uint32
+}
+
+var extensionTypes = map[string]uint16{
+	"res": 0x0000,
+	"bmp": 0x0001,
+	"mve": 0x0002,
+	"tga": 0x0003,
+	"wav": 0x0004,
+	"plt": 0x0006,
+	"ini": 0x0007,
+	"bmu": 0x0008,
+	"txt": 0x000A,
+	"mdl": 0x07D2,
+	"nss": 0x07D9,
+	"ncs": 0x07DA,
+	"are": 0x07DC,
+	"set": 0x07DD,
+	"ifo": 0x07DE,
+	"bic": 0x07DF,
+	"wok": 0x07E0,
+	"2da": 0x07E1,
+	"tlk": 0x07E2,
+	"txi": 0x07E6,
+	"git": 0x07E7,
+	"uti": 0x07E9,
+	"utc": 0x07EB,
+	"dlg": 0x07ED,
+	"itp": 0x07EE,
+	"utt": 0x07F0,
+	"dds": 0x07F1,
+	"uts": 0x07F3,
+	"fac": 0x07F6,
+	"gff": 0x07F7,
+	"ute": 0x07F8,
+	"utd": 0x07FA,
+	"utp": 0x07FC,
+	"dfa": 0x07FD,
+	"gic": 0x07FE,
+	"gui": 0x07FF,
+	"utm": 0x0803,
+	"dwk": 0x0804,
+	"pwk": 0x0805,
+	"utg": 0x0807,
+	"jrl": 0x0808,
+	"utw": 0x080A,
+	"ssf": 0x080C,
+	"hak": 0x080D,
+	"nwm": 0x080E,
+	"bik": 0x080F,
+	"ndb": 0x0810,
+	"ptm": 0x0811,
+	"ptt": 0x0812,
+	"ltr": 0x0813,
+	"shd": 0x0815,
+	"mdb": 0x0816,
+	"mtr": 0x0818,
+	"jpg": 0x081C,
+	"lod": 0x081E,
+	"png": 0x0820,
+	"lyt": 0x0BB8,
+	"vis": 0x0BB9,
+	"mdx": 0x0BC0,
+	"wlk": 0x0BCC,
+	"xml": 0x0BCD,
+	"gr2": 0x0FA3,
+}
+
+var typeExtensions = map[uint16]string{
+	0x0000: "res",
+	0x0001: "bmp",
+	0x0002: "mve",
+	0x0003: "tga",
+	0x0004: "wav",
+	0x0006: "plt",
+	0x0007: "ini",
+	0x0008: "bmu",
+	0x000A: "txt",
+	0x07D2: "mdl",
+	0x07D9: "nss",
+	0x07DA: "ncs",
+	0x07DC: "are",
+	0x07DD: "set",
+	0x07DE: "ifo",
+	0x07DF: "bic",
+	0x07E0: "wok",
+	0x07E1: "2da",
+	0x07E2: "tlk",
+	0x07E6: "txi",
+	0x07E7: "git",
+	0x07E9: "uti",
+	0x07EB: "utc",
+	0x07ED: "dlg",
+	0x07EE: "itp",
+	0x07F0: "utt",
+	0x07F1: "dds",
+	0x07F3: "uts",
+	0x07F6: "fac",
+	0x07F7: "gff",
+	0x07F8: "ute",
+	0x07FA: "utd",
+	0x07FC: "utp",
+	0x07FD: "dfa",
+	0x07FE: "gic",
+	0x07FF: "gui",
+	0x0803: "utm",
+	0x0804: "dwk",
+	0x0805: "pwk",
+	0x0807: "utg",
+	0x0808: "jrl",
+	0x080A: "utw",
+	0x080C: "ssf",
+	0x080D: "hak",
+	0x080E: "nwm",
+	0x080F: "bik",
+	0x0810: "ndb",
+	0x0811: "ptm",
+	0x0812: "ptt",
+	0x0813: "ltr",
+	0x0815: "shd",
+	0x0816: "mdb",
+	0x0818: "mtr",
+	0x081C: "jpg",
+	0x081E: "lod",
+	0x0820: "png",
+	0x0BB8: "lyt",
+	0x0BB9: "vis",
+	0x0BC0: "mdx",
+	0x0BCC: "wlk",
+	0x0BCD: "xml",
+	0x0FA3: "gr2",
+}
+
+func init() {
+	for ext, resourceType := range extensionTypes {
+		canonicalExt, ok := typeExtensions[resourceType]
+		if !ok {
+			panic(fmt.Sprintf("missing canonical extension for resource type 0x%04X", resourceType))
+		}
+		if canonicalExt == ext {
+			continue
+		}
+	}
+}
+
+func New(fileType string, resources []Resource) Archive {
+	normalized := make([]Resource, len(resources))
+	copy(normalized, resources)
+	sort.Slice(normalized, func(i, j int) bool {
+		if normalized[i].Name == normalized[j].Name {
+			return normalized[i].Type < normalized[j].Type
+		}
+		return normalized[i].Name < normalized[j].Name
+	})
+
+	return Archive{
+		FileType:  padFour(fileType),
+		Version:   versionV10,
+		Resources: normalized,
+	}
+}
+
+func ArchiveSize(resources []Resource) int64 {
+	return int64(headerSize+len(resources)*24+len(resources)*8) + totalResourceBytes(resources)
+}
+
+func Write(w io.Writer, archive Archive) error {
+	if archive.Version == "" {
+		archive.Version = versionV10
+	}
+	if archive.FileType == "" {
+		archive.FileType = typeMOD
+	}
+
+	keys := make([]keyEntry, 0, len(archive.Resources))
+	entries := make([]resourceEntry, 0, len(archive.Resources))
+
+	dataOffset := uint32(headerSize + len(archive.Resources)*24 + len(archive.Resources)*8)
+	for index, resource := range archive.Resources {
+		if len(resource.Name) > 16 {
+			return fmt.Errorf("resource %q exceeds 16-byte resref limit", resource.Name)
+		}
+		size, err := payloadSize(resource)
+		if err != nil {
+			return err
+		}
+		if size > int64(^uint32(0)) {
+			return fmt.Errorf("resource %q exceeds 4 GiB resource size limit", resource.Name)
+		}
+
+		entry := resourceEntry{
+			Offset: dataOffset,
+			Size:   uint32(size),
+		}
+		entries = append(entries, entry)
+		dataOffset += uint32(size)
+
+		var key keyEntry
+		copy(key.ResRef[:], []byte(resource.Name))
+		key.ResourceID = uint32(index)
+		key.ResourceType = resource.Type
+		keys = append(keys, key)
+	}
+
+	hdr := header{
+		LanguageCount:         0,
+		LocalizedStringSize:   0,
+		EntryCount:            uint32(len(archive.Resources)),
+		LocalizedStringOffset: headerSize,
+		KeyListOffset:         headerSize,
+		ResourceListOffset:    uint32(headerSize + len(keys)*24),
+		BuildYear:             0,
+		BuildDay:              0,
+		DescriptionStrRef:     strRefNone,
+	}
+	copy(hdr.FileType[:], []byte(padFour(archive.FileType)))
+	copy(hdr.Version[:], []byte(padFour(archive.Version)))
+
+	if err := binary.Write(w, binary.LittleEndian, hdr); err != nil {
+		return err
+	}
+	if err := binary.Write(w, binary.LittleEndian, keys); err != nil {
+		return err
+	}
+	if err := binary.Write(w, binary.LittleEndian, entries); err != nil {
+		return err
+	}
+	for _, resource := range archive.Resources {
+		if err := writeResourceData(w, resource); err != nil {
+			return err
+		}
+	}
+	return nil
+}
+
+func Read(r io.Reader) (Archive, error) {
+	data, err := io.ReadAll(r)
+	if err != nil {
+		return Archive{}, fmt.Errorf("read erf: %w", err)
+	}
+	if len(data) < headerSize {
+		return Archive{}, fmt.Errorf("erf file too small: %d bytes", len(data))
+	}
+
+	var hdr header
+	if err := binary.Read(bytes.NewReader(data[:headerSize]), binary.LittleEndian, &hdr); err != nil {
+		return Archive{}, fmt.Errorf("decode erf header: %w", err)
+	}
+
+	keyStart := int(hdr.KeyListOffset)
+	keyEnd := keyStart + int(hdr.EntryCount)*24
+	if keyEnd > len(data) {
+		return Archive{}, fmt.Errorf("erf key list exceeds file bounds")
+	}
+	keys := make([]keyEntry, hdr.EntryCount)
+	if err := binary.Read(bytes.NewReader(data[keyStart:keyEnd]), binary.LittleEndian, &keys); err != nil {
+		return Archive{}, fmt.Errorf("decode key list: %w", err)
+	}
+
+	resourceStart := int(hdr.ResourceListOffset)
+	resourceEnd := resourceStart + int(hdr.EntryCount)*8
+	if resourceEnd > len(data) {
+		return Archive{}, fmt.Errorf("erf resource list exceeds file bounds")
+	}
+	entries := make([]resourceEntry, hdr.EntryCount)
+	if err := binary.Read(bytes.NewReader(data[resourceStart:resourceEnd]), binary.LittleEndian, &entries); err != nil {
+		return Archive{}, fmt.Errorf("decode resource list: %w", err)
+	}
+
+	resources := make([]Resource, 0, hdr.EntryCount)
+	for index, key := range keys {
+		entry := entries[index]
+		start := int(entry.Offset)
+		end := start + int(entry.Size)
+		if end > len(data) {
+			return Archive{}, fmt.Errorf("resource %d exceeds file bounds", index)
+		}
+
+		resref := string(bytes.TrimRight(key.ResRef[:], "\x00"))
+		payload := make([]byte, entry.Size)
+		copy(payload, data[start:end])
+		resources = append(resources, Resource{
+			Name: resref,
+			Type: key.ResourceType,
+			Data: payload,
+			Size: int64(entry.Size),
+		})
+	}
+
+	return Archive{
+		FileType:  string(hdr.FileType[:]),
+		Version:   string(hdr.Version[:]),
+		Resources: resources,
+	}, nil
+}
+
+func ResourceTypeForExtension(extension string) (uint16, bool) {
+	resourceType, ok := extensionTypes[strings.TrimPrefix(strings.ToLower(extension), ".")]
+	return resourceType, ok
+}
+
+func HAKResourceTypeForExtension(extension string) (uint16, bool) {
+	return ResourceTypeForExtension(extension)
+}
+
+func ExtensionForResourceType(resourceType uint16) (string, bool) {
+	extension, ok := typeExtensions[resourceType]
+	return extension, ok
+}
+
+func padFour(value string) string {
+	value = strings.ToUpper(value)
+	if len(value) >= 4 {
+		return value[:4]
+	}
+	return value + strings.Repeat(" ", 4-len(value))
+}
+
+func totalResourceBytes(resources []Resource) int64 {
+	var total int64
+	for _, resource := range resources {
+		switch {
+		case resource.Size > 0:
+			total += resource.Size
+		default:
+			total += int64(len(resource.Data))
+		}
+	}
+	return total
+}
+
+func payloadSize(resource Resource) (int64, error) {
+	if resource.Size > 0 {
+		return resource.Size, nil
+	}
+	if resource.SourcePath != "" && len(resource.Data) == 0 {
+		info, err := os.Stat(resource.SourcePath)
+		if err != nil {
+			return 0, fmt.Errorf("stat resource %q: %w", resource.SourcePath, err)
+		}
+		return info.Size(), nil
+	}
+	return int64(len(resource.Data)), nil
+}
+
+func writeResourceData(w io.Writer, resource Resource) error {
+	if len(resource.Data) > 0 || resource.SourcePath == "" {
+		_, err := w.Write(resource.Data)
+		return err
+	}
+
+	file, err := os.Open(resource.SourcePath)
+	if err != nil {
+		return fmt.Errorf("open resource %q: %w", resource.SourcePath, err)
+	}
+	defer file.Close()
+
+	written, err := io.Copy(w, file)
+	if err != nil {
+		return fmt.Errorf("copy resource %q: %w", resource.SourcePath, err)
+	}
+	if resource.Size > 0 && written != resource.Size {
+		return fmt.Errorf("copy resource %q: expected %d bytes, wrote %d", resource.SourcePath, resource.Size, written)
+	}
+	return nil
+}

internal/erf/erf_test.go

@@ -0,0 +1,130 @@
+package erf
+
+import (
+	"bytes"
+	"testing"
+)
+
+func TestArchiveRoundTrip(t *testing.T) {
+	archive := New("MOD ", []Resource{
+		{Name: "module", Type: 0x07DE, Data: []byte("ifo")},
+		{Name: "start", Type: 0x07DC, Data: []byte("are")},
+	})
+
+	var buf bytes.Buffer
+	if err := Write(&buf, archive); err != nil {
+		t.Fatalf("write: %v", err)
+	}
+
+	decoded, err := Read(bytes.NewReader(buf.Bytes()))
+	if err != nil {
+		t.Fatalf("read: %v", err)
+	}
+
+	if len(decoded.Resources) != 2 {
+		t.Fatalf("expected 2 resources, got %d", len(decoded.Resources))
+	}
+	if decoded.Resources[0].Name != "module" || string(decoded.Resources[0].Data) != "ifo" {
+		t.Fatalf("unexpected first resource: %#v", decoded.Resources[0])
+	}
+}
+
+func TestExtensionMappingsKeep2DAAndTLKDistinct(t *testing.T) {
+	resourceType, ok := ResourceTypeForExtension("2da")
+	if !ok {
+		t.Fatal("expected 2da resource type")
+	}
+	if resourceType != 0x07E1 {
+		t.Fatalf("expected 2da type 0x07E1, got 0x%04X", resourceType)
+	}
+	extension, ok := ExtensionForResourceType(resourceType)
+	if !ok {
+		t.Fatal("expected canonical extension for 2da type")
+	}
+	if extension != "2da" {
+		t.Fatalf("expected 2da extension, got %q", extension)
+	}
+
+	tlkType, ok := ResourceTypeForExtension("tlk")
+	if !ok {
+		t.Fatal("expected tlk resource type")
+	}
+	if tlkType != 0x07E2 {
+		t.Fatalf("expected tlk type 0x07E2, got 0x%04X", tlkType)
+	}
+	if tlkType == resourceType {
+		t.Fatal("expected 2da and tlk resource types to stay distinct")
+	}
+}
+
+func TestHAKResourceTypeForExtensionSupportsPNG(t *testing.T) {
+	if resourceType, ok := HAKResourceTypeForExtension("png"); !ok || resourceType != 0x0820 {
+		t.Fatalf("expected png restype 0x0820 (2080) for HAK generation, got ok=%v type=0x%04X", ok, resourceType)
+	}
+	if resourceType, ok := HAKResourceTypeForExtension("2da"); !ok || resourceType != 0x07E1 {
+		t.Fatalf("expected 2da to stay valid for HAK generation, got ok=%v type=0x%04X", ok, resourceType)
+	}
+}
+
+func TestExtensionMappingsSupportModernAssetTypes(t *testing.T) {
+	cases := map[string]uint16{
+		"mtr": 0x0818,
+		"shd": 0x0815,
+		"txi": 0x07E6,
+		"jpg": 0x081C,
+		"mdb": 0x0816,
+		"lyt": 0x0BB8,
+		"vis": 0x0BB9,
+		"mdx": 0x0BC0,
+		"xml": 0x0BCD,
+		"wlk": 0x0BCC,
+		"gr2": 0x0FA3,
+	}
+	for ext, wantType := range cases {
+		gotType, ok := ResourceTypeForExtension(ext)
+		if !ok {
+			t.Fatalf("expected %s to resolve to a resource type", ext)
+		}
+		if gotType != wantType {
+			t.Fatalf("expected %s type 0x%04X, got 0x%04X", ext, wantType, gotType)
+		}
+		gotExt, ok := ExtensionForResourceType(wantType)
+		if !ok {
+			t.Fatalf("expected type 0x%04X to resolve back to an extension", wantType)
+		}
+		if gotExt != ext {
+			t.Fatalf("expected type 0x%04X to resolve to %s, got %s", wantType, ext, gotExt)
+		}
+	}
+}
+
+func TestExtensionMappingsSupportAllUTBlueprintTypes(t *testing.T) {
+	cases := map[string]uint16{
+		"uti": 0x07E9,
+		"utc": 0x07EB,
+		"utt": 0x07F0,
+		"uts": 0x07F3,
+		"ute": 0x07F8,
+		"utd": 0x07FA,
+		"utp": 0x07FC,
+		"utm": 0x0803,
+		"utg": 0x0807,
+		"utw": 0x080A,
+	}
+	for ext, wantType := range cases {
+		gotType, ok := ResourceTypeForExtension(ext)
+		if !ok {
+			t.Fatalf("expected %s to resolve to a resource type", ext)
+		}
+		if gotType != wantType {
+			t.Fatalf("expected %s type 0x%04X, got 0x%04X", ext, wantType, gotType)
+		}
+		gotExt, ok := ExtensionForResourceType(wantType)
+		if !ok {
+			t.Fatalf("expected type 0x%04X to resolve back to an extension", wantType)
+		}
+		if gotExt != ext {
+			t.Fatalf("expected type 0x%04X to resolve to %s, got %s", wantType, ext, gotExt)
+		}
+	}
+}

internal/gff/binary.go

@@ -0,0 +1,675 @@
+package gff
+
+import (
+	"bytes"
+	"encoding/binary"
+	"fmt"
+	"io"
+	"math"
+)
+
+const (
+	headerSize = 56
+)
+
+type header struct {
+	FileType           [4]byte
+	FileVersion        [4]byte
+	StructOffset       uint32
+	StructCount        uint32
+	FieldOffset        uint32
+	FieldCount         uint32
+	LabelOffset        uint32
+	LabelCount         uint32
+	FieldDataOffset    uint32
+	FieldDataCount     uint32
+	FieldIndicesOffset uint32
+	FieldIndicesCount  uint32
+	ListIndicesOffset  uint32
+	ListIndicesCount   uint32
+}
+
+type rawStruct struct {
+	Type         uint32
+	DataOrOffset uint32
+	FieldCount   uint32
+}
+
+type rawField struct {
+	Type         uint32
+	LabelIndex   uint32
+	DataOrOffset uint32
+}
+
+func Read(r io.Reader) (Document, error) {
+	data, err := io.ReadAll(r)
+	if err != nil {
+		return Document{}, fmt.Errorf("read gff: %w", err)
+	}
+	if len(data) < headerSize {
+		return Document{}, fmt.Errorf("gff file too small: %d bytes", len(data))
+	}
+
+	var hdr header
+	if err := binary.Read(bytes.NewReader(data[:headerSize]), binary.LittleEndian, &hdr); err != nil {
+		return Document{}, fmt.Errorf("decode header: %w", err)
+	}
+
+	reader := binaryReader{data: data, hdr: hdr}
+	rawStructs, err := reader.readStructs()
+	if err != nil {
+		return Document{}, err
+	}
+	rawFields, err := reader.readFields()
+	if err != nil {
+		return Document{}, err
+	}
+	labels, err := reader.readLabels()
+	if err != nil {
+		return Document{}, err
+	}
+
+	root, err := reader.decodeStruct(0, rawStructs, rawFields, labels)
+	if err != nil {
+		return Document{}, err
+	}
+
+	return Document{
+		FileType:    string(hdr.FileType[:]),
+		FileVersion: string(hdr.FileVersion[:]),
+		Root:        root,
+	}, nil
+}
+
+func Write(w io.Writer, doc Document) error {
+	encoder := newBinaryEncoder(doc)
+	data, err := encoder.encode()
+	if err != nil {
+		return err
+	}
+	_, err = w.Write(data)
+	return err
+}
+
+type binaryReader struct {
+	data []byte
+	hdr  header
+}
+
+func (r binaryReader) readStructs() ([]rawStruct, error) {
+	return readTable[rawStruct](r.data, r.hdr.StructOffset, r.hdr.StructCount)
+}
+
+func (r binaryReader) readFields() ([]rawField, error) {
+	return readTable[rawField](r.data, r.hdr.FieldOffset, r.hdr.FieldCount)
+}
+
+func (r binaryReader) readLabels() ([]string, error) {
+	start := int(r.hdr.LabelOffset)
+	end := start + int(r.hdr.LabelCount)*16
+	if end > len(r.data) {
+		return nil, fmt.Errorf("label table exceeds file bounds")
+	}
+
+	labels := make([]string, 0, r.hdr.LabelCount)
+	for offset := start; offset < end; offset += 16 {
+		chunk := r.data[offset : offset+16]
+		n := bytes.IndexByte(chunk, 0)
+		if n == -1 {
+			n = len(chunk)
+		}
+		labels = append(labels, string(chunk[:n]))
+	}
+	return labels, nil
+}
+
+func (r binaryReader) decodeStruct(index uint32, structs []rawStruct, fields []rawField, labels []string) (Struct, error) {
+	if index >= uint32(len(structs)) {
+		return Struct{}, fmt.Errorf("struct index %d out of range", index)
+	}
+
+	raw := structs[index]
+	fieldIndices, err := r.fieldIndices(raw)
+	if err != nil {
+		return Struct{}, err
+	}
+
+	out := Struct{
+		Type:   raw.Type,
+		Fields: make([]Field, 0, len(fieldIndices)),
+	}
+	for _, fieldIndex := range fieldIndices {
+		if fieldIndex >= uint32(len(fields)) {
+			return Struct{}, fmt.Errorf("field index %d out of range", fieldIndex)
+		}
+		field, err := r.decodeField(fields[fieldIndex], structs, fields, labels)
+		if err != nil {
+			return Struct{}, err
+		}
+		out.Fields = append(out.Fields, field)
+	}
+
+	return out, nil
+}
+
+func (r binaryReader) fieldIndices(s rawStruct) ([]uint32, error) {
+	switch s.FieldCount {
+	case 0:
+		return nil, nil
+	case 1:
+		return []uint32{s.DataOrOffset}, nil
+	default:
+		start := int(r.hdr.FieldIndicesOffset + s.DataOrOffset)
+		end := start + int(s.FieldCount)*4
+		if end > len(r.data) {
+			return nil, fmt.Errorf("field indices exceed file bounds")
+		}
+
+		out := make([]uint32, s.FieldCount)
+		reader := bytes.NewReader(r.data[start:end])
+		if err := binary.Read(reader, binary.LittleEndian, &out); err != nil {
+			return nil, fmt.Errorf("decode field indices: %w", err)
+		}
+		return out, nil
+	}
+}
+
+func (r binaryReader) decodeField(raw rawField, structs []rawStruct, fields []rawField, labels []string) (Field, error) {
+	if raw.LabelIndex >= uint32(len(labels)) {
+		return Field{}, fmt.Errorf("label index %d out of range", raw.LabelIndex)
+	}
+	fieldType := FieldType(raw.Type)
+	value, err := r.decodeValue(fieldType, raw.DataOrOffset, structs, fields, labels)
+	if err != nil {
+		return Field{}, fmt.Errorf("decode field %q: %w", labels[raw.LabelIndex], err)
+	}
+
+	return Field{
+		Label: labels[raw.LabelIndex],
+		Type:  fieldType,
+		Value: value,
+	}, nil
+}
+
+func (r binaryReader) decodeValue(fieldType FieldType, data uint32, structs []rawStruct, fields []rawField, labels []string) (Value, error) {
+	switch fieldType {
+	case TypeByte:
+		return ByteValue(uint8(data)), nil
+	case TypeChar:
+		return CharValue(int8(data)), nil
+	case TypeWord:
+		return WordValue(uint16(data)), nil
+	case TypeShort:
+		return ShortValue(int16(data)), nil
+	case TypeDWord:
+		return DWordValue(data), nil
+	case TypeInt:
+		return IntValue(int32(data)), nil
+	case TypeFloat:
+		return FloatValue(math.Float32frombits(data)), nil
+	case TypeDWord64:
+		raw, err := r.sliceFieldData(data, 8)
+		if err != nil {
+			return nil, err
+		}
+		return DWord64Value(binary.LittleEndian.Uint64(raw)), nil
+	case TypeInt64:
+		raw, err := r.sliceFieldData(data, 8)
+		if err != nil {
+			return nil, err
+		}
+		return Int64Value(int64(binary.LittleEndian.Uint64(raw))), nil
+	case TypeDouble:
+		raw, err := r.sliceFieldData(data, 8)
+		if err != nil {
+			return nil, err
+		}
+		return DoubleValue(math.Float64frombits(binary.LittleEndian.Uint64(raw))), nil
+	case TypeCExoString:
+		raw, err := r.prefixedFieldData(data)
+		if err != nil {
+			return nil, err
+		}
+		return StringValue(string(raw)), nil
+	case TypeResRef:
+		raw, err := r.prefixedByteFieldData(data, 1)
+		if err != nil {
+			return nil, err
+		}
+		return ResRefValue(string(raw)), nil
+	case TypeCExoLocString:
+		raw, err := r.locStringFieldData(data)
+		if err != nil {
+			return nil, err
+		}
+		return decodeLocString(raw)
+	case TypeVoid:
+		raw, err := r.prefixedFieldData(data)
+		if err != nil {
+			return nil, err
+		}
+		return VoidValue(raw), nil
+	case TypeStruct:
+		return r.decodeStruct(data, structs, fields, labels)
+	case TypeList:
+		return r.decodeList(data, structs, fields, labels)
+	case TypeOrientation:
+		raw, err := r.sliceFieldData(data, 16)
+		if err != nil {
+			return nil, err
+		}
+		return Orientation{
+			X: math.Float32frombits(binary.LittleEndian.Uint32(raw[0:4])),
+			Y: math.Float32frombits(binary.LittleEndian.Uint32(raw[4:8])),
+			Z: math.Float32frombits(binary.LittleEndian.Uint32(raw[8:12])),
+			W: math.Float32frombits(binary.LittleEndian.Uint32(raw[12:16])),
+		}, nil
+	case TypeVector:
+		raw, err := r.sliceFieldData(data, 12)
+		if err != nil {
+			return nil, err
+		}
+		return Vector{
+			X: math.Float32frombits(binary.LittleEndian.Uint32(raw[0:4])),
+			Y: math.Float32frombits(binary.LittleEndian.Uint32(raw[4:8])),
+			Z: math.Float32frombits(binary.LittleEndian.Uint32(raw[8:12])),
+		}, nil
+	default:
+		return nil, fmt.Errorf("unsupported field type %d", fieldType)
+	}
+}
+
+func (r binaryReader) decodeList(offset uint32, structs []rawStruct, fields []rawField, labels []string) (ListValue, error) {
+	start := int(r.hdr.ListIndicesOffset + offset)
+	if start+4 > len(r.data) {
+		return nil, fmt.Errorf("list data exceeds file bounds")
+	}
+	count := binary.LittleEndian.Uint32(r.data[start : start+4])
+	start += 4
+	end := start + int(count)*4
+	if end > len(r.data) {
+		return nil, fmt.Errorf("list indices exceed file bounds")
+	}
+
+	list := make(ListValue, 0, count)
+	for pos := start; pos < end; pos += 4 {
+		index := binary.LittleEndian.Uint32(r.data[pos : pos+4])
+		child, err := r.decodeStruct(index, structs, fields, labels)
+		if err != nil {
+			return nil, err
+		}
+		list = append(list, child)
+	}
+	return list, nil
+}
+
+func (r binaryReader) sliceFieldData(offset uint32, size int) ([]byte, error) {
+	start := int(r.hdr.FieldDataOffset + offset)
+	end := start + size
+	if end > len(r.data) {
+		return nil, fmt.Errorf("field data exceeds file bounds")
+	}
+	return r.data[start:end], nil
+}
+
+func (r binaryReader) prefixedFieldData(offset uint32) ([]byte, error) {
+	sizeRaw, err := r.sliceFieldData(offset, 4)
+	if err != nil {
+		return nil, err
+	}
+	size := int(binary.LittleEndian.Uint32(sizeRaw))
+	return r.sliceFieldData(offset+4, size)
+}
+
+func (r binaryReader) prefixedByteFieldData(offset uint32, prefixBytes uint32) ([]byte, error) {
+	header, err := r.sliceFieldData(offset, int(prefixBytes))
+	if err != nil {
+		return nil, err
+	}
+	size := int(header[0])
+	return r.sliceFieldData(offset+prefixBytes, size)
+}
+
+func (r binaryReader) locStringFieldData(offset uint32) ([]byte, error) {
+	sizeRaw, err := r.sliceFieldData(offset, 4)
+	if err != nil {
+		return nil, err
+	}
+	size := int(binary.LittleEndian.Uint32(sizeRaw))
+	return r.sliceFieldData(offset, size+4)
+}
+
+func readTable[T any](data []byte, offset uint32, count uint32) ([]T, error) {
+	if count == 0 {
+		return nil, nil
+	}
+	var row T
+	size := binary.Size(row)
+	start := int(offset)
+	end := start + int(count)*size
+	if end > len(data) {
+		return nil, fmt.Errorf("table exceeds file bounds")
+	}
+
+	out := make([]T, count)
+	reader := bytes.NewReader(data[start:end])
+	if err := binary.Read(reader, binary.LittleEndian, &out); err != nil {
+		return nil, fmt.Errorf("decode table: %w", err)
+	}
+	return out, nil
+}
+
+func decodeLocString(data []byte) (LocString, error) {
+	if len(data) < 12 {
+		return LocString{}, fmt.Errorf("locstring too small")
+	}
+	totalSize := binary.LittleEndian.Uint32(data[0:4])
+	if totalSize+4 != uint32(len(data)) {
+		return LocString{}, fmt.Errorf("locstring size mismatch")
+	}
+	stringRef := binary.LittleEndian.Uint32(data[4:8])
+	count := binary.LittleEndian.Uint32(data[8:12])
+	pos := 12
+	entries := make([]LocStringEntry, 0, count)
+	for i := uint32(0); i < count; i++ {
+		if pos+8 > len(data) {
+			return LocString{}, fmt.Errorf("locstring entry header truncated")
+		}
+		id := binary.LittleEndian.Uint32(data[pos : pos+4])
+		length := binary.LittleEndian.Uint32(data[pos+4 : pos+8])
+		pos += 8
+		if pos+int(length) > len(data) {
+			return LocString{}, fmt.Errorf("locstring entry %d truncated", i)
+		}
+		entries = append(entries, LocStringEntry{
+			ID:    id,
+			Value: string(data[pos : pos+int(length)]),
+		})
+		pos += int(length)
+	}
+
+	return LocString{
+		StringRef: stringRef,
+		Entries:   entries,
+	}, nil
+}
+
+type binaryEncoder struct {
+	document     Document
+	structs      []rawStruct
+	fields       []rawField
+	labels       []string
+	labelIndex   map[string]uint32
+	fieldData    bytes.Buffer
+	fieldIndices []uint32
+	listIndices  []uint32
+}
+
+func newBinaryEncoder(doc Document) *binaryEncoder {
+	return &binaryEncoder{
+		document:   doc,
+		labelIndex: map[string]uint32{},
+	}
+}
+
+func (e *binaryEncoder) encode() ([]byte, error) {
+	if err := e.addStruct(e.document.Root); err != nil {
+		return nil, err
+	}
+
+	var buf bytes.Buffer
+	hdr := header{}
+	copy(hdr.FileType[:], []byte(padFour(e.document.FileType)))
+	copy(hdr.FileVersion[:], []byte(padFour(defaultString(e.document.FileVersion, "V3.2"))))
+
+	hdr.StructOffset = headerSize
+	hdr.StructCount = uint32(len(e.structs))
+	hdr.FieldOffset = hdr.StructOffset + uint32(len(e.structs))*12
+	hdr.FieldCount = uint32(len(e.fields))
+	hdr.LabelOffset = hdr.FieldOffset + uint32(len(e.fields))*12
+	hdr.LabelCount = uint32(len(e.labels))
+	hdr.FieldDataOffset = hdr.LabelOffset + uint32(len(e.labels))*16
+	hdr.FieldDataCount = uint32(e.fieldData.Len())
+	hdr.FieldIndicesOffset = hdr.FieldDataOffset + uint32(e.fieldData.Len())
+	hdr.FieldIndicesCount = uint32(len(e.fieldIndices) * 4)
+	hdr.ListIndicesOffset = hdr.FieldIndicesOffset + uint32(len(e.fieldIndices))*4
+	hdr.ListIndicesCount = uint32(len(e.listIndices) * 4)
+
+	if err := binary.Write(&buf, binary.LittleEndian, hdr); err != nil {
+		return nil, err
+	}
+	if err := binary.Write(&buf, binary.LittleEndian, e.structs); err != nil {
+		return nil, err
+	}
+	if err := binary.Write(&buf, binary.LittleEndian, e.fields); err != nil {
+		return nil, err
+	}
+	for _, label := range e.labels {
+		var raw [16]byte
+		copy(raw[:], []byte(label))
+		if _, err := buf.Write(raw[:]); err != nil {
+			return nil, err
+		}
+	}
+	if _, err := buf.Write(e.fieldData.Bytes()); err != nil {
+		return nil, err
+	}
+	if err := binary.Write(&buf, binary.LittleEndian, e.fieldIndices); err != nil {
+		return nil, err
+	}
+	if err := binary.Write(&buf, binary.LittleEndian, e.listIndices); err != nil {
+		return nil, err
+	}
+
+	return buf.Bytes(), nil
+}
+
+func (e *binaryEncoder) addStruct(s Struct) error {
+	index := uint32(len(e.structs))
+	e.structs = append(e.structs, rawStruct{Type: s.Type})
+
+	fieldIndices := make([]uint32, 0, len(s.Fields))
+	for _, field := range s.Fields {
+		fieldIndex, err := e.addField(field)
+		if err != nil {
+			return err
+		}
+		fieldIndices = append(fieldIndices, fieldIndex)
+	}
+
+	entry := &e.structs[index]
+	entry.FieldCount = uint32(len(fieldIndices))
+	switch len(fieldIndices) {
+	case 0:
+		entry.DataOrOffset = 0
+	case 1:
+		entry.DataOrOffset = fieldIndices[0]
+	default:
+		entry.DataOrOffset = uint32(len(e.fieldIndices) * 4)
+		e.fieldIndices = append(e.fieldIndices, fieldIndices...)
+	}
+	return nil
+}
+
+func (e *binaryEncoder) addField(field Field) (uint32, error) {
+	labelIndex, err := e.indexLabel(field.Label)
+	if err != nil {
+		return 0, err
+	}
+	data, err := e.encodeValue(field.Value)
+	if err != nil {
+		return 0, fmt.Errorf("encode field %q: %w", field.Label, err)
+	}
+
+	entry := rawField{
+		Type:         uint32(field.Type),
+		LabelIndex:   labelIndex,
+		DataOrOffset: data,
+	}
+	index := uint32(len(e.fields))
+	e.fields = append(e.fields, entry)
+	return index, nil
+}
+
+func (e *binaryEncoder) encodeValue(value Value) (uint32, error) {
+	switch typed := value.(type) {
+	case ByteValue:
+		return uint32(uint8(typed)), nil
+	case CharValue:
+		return uint32(uint8(typed)), nil
+	case WordValue:
+		return uint32(uint16(typed)), nil
+	case ShortValue:
+		return uint32(uint16(typed)), nil
+	case DWordValue:
+		return uint32(typed), nil
+	case IntValue:
+		return uint32(int32(typed)), nil
+	case FloatValue:
+		return math.Float32bits(float32(typed)), nil
+	case DWord64Value:
+		return e.writeFieldData(func(buf *bytes.Buffer) error {
+			return binary.Write(buf, binary.LittleEndian, uint64(typed))
+		}), nil
+	case Int64Value:
+		return e.writeFieldData(func(buf *bytes.Buffer) error {
+			return binary.Write(buf, binary.LittleEndian, int64(typed))
+		}), nil
+	case DoubleValue:
+		return e.writeFieldData(func(buf *bytes.Buffer) error {
+			return binary.Write(buf, binary.LittleEndian, math.Float64bits(float64(typed)))
+		}), nil
+	case StringValue:
+		return e.writeLengthPrefixed([]byte(typed)), nil
+	case ResRefValue:
+		if len(typed) > 255 {
+			return 0, fmt.Errorf("resref exceeds 255 bytes")
+		}
+		return e.writeFieldData(func(buf *bytes.Buffer) error {
+			if err := buf.WriteByte(byte(len(typed))); err != nil {
+				return err
+			}
+			_, err := buf.Write([]byte(typed))
+			return err
+		}), nil
+	case LocString:
+		return e.writeFieldData(func(buf *bytes.Buffer) error {
+			payload := bytes.Buffer{}
+			if err := binary.Write(&payload, binary.LittleEndian, typed.StringRef); err != nil {
+				return err
+			}
+			if err := binary.Write(&payload, binary.LittleEndian, uint32(len(typed.Entries))); err != nil {
+				return err
+			}
+			for _, entry := range typed.Entries {
+				if err := binary.Write(&payload, binary.LittleEndian, entry.ID); err != nil {
+					return err
+				}
+				if err := binary.Write(&payload, binary.LittleEndian, uint32(len(entry.Value))); err != nil {
+					return err
+				}
+				if _, err := payload.Write([]byte(entry.Value)); err != nil {
+					return err
+				}
+			}
+			if err := binary.Write(buf, binary.LittleEndian, uint32(payload.Len())); err != nil {
+				return err
+			}
+			if _, err := buf.Write(payload.Bytes()); err != nil {
+				return err
+			}
+			return nil
+		}), nil
+	case VoidValue:
+		return e.writeLengthPrefixed([]byte(typed)), nil
+	case Struct:
+		index := uint32(len(e.structs))
+		if err := e.addStruct(typed); err != nil {
+			return 0, err
+		}
+		return index, nil
+	case ListValue:
+		return e.writeList(typed)
+	case Orientation:
+		return e.writeFieldData(func(buf *bytes.Buffer) error {
+			for _, component := range []float32{typed.X, typed.Y, typed.Z, typed.W} {
+				if err := binary.Write(buf, binary.LittleEndian, math.Float32bits(component)); err != nil {
+					return err
+				}
+			}
+			return nil
+		}), nil
+	case Vector:
+		return e.writeFieldData(func(buf *bytes.Buffer) error {
+			for _, component := range []float32{typed.X, typed.Y, typed.Z} {
+				if err := binary.Write(buf, binary.LittleEndian, math.Float32bits(component)); err != nil {
+					return err
+				}
+			}
+			return nil
+		}), nil
+	default:
+		return 0, fmt.Errorf("unsupported value type %T", value)
+	}
+}
+
+func (e *binaryEncoder) writeList(list ListValue) (uint32, error) {
+	indices := make([]uint32, 0, len(list))
+	for _, item := range list {
+		index := uint32(len(e.structs))
+		if err := e.addStruct(item); err != nil {
+			return 0, err
+		}
+		indices = append(indices, index)
+	}
+
+	offset := uint32(len(e.listIndices) * 4)
+	e.listIndices = append(e.listIndices, uint32(len(list)))
+	e.listIndices = append(e.listIndices, indices...)
+	return offset, nil
+}
+
+func (e *binaryEncoder) writeFieldData(write func(*bytes.Buffer) error) uint32 {
+	offset := uint32(e.fieldData.Len())
+	if err := write(&e.fieldData); err != nil {
+		panic(err)
+	}
+	return offset
+}
+
+func (e *binaryEncoder) writeLengthPrefixed(data []byte) uint32 {
+	return e.writeFieldData(func(buf *bytes.Buffer) error {
+		if err := binary.Write(buf, binary.LittleEndian, uint32(len(data))); err != nil {
+			return err
+		}
+		_, err := buf.Write(data)
+		return err
+	})
+}
+
+func (e *binaryEncoder) indexLabel(label string) (uint32, error) {
+	if len(label) > 16 {
+		return 0, fmt.Errorf("label %q exceeds 16 bytes", label)
+	}
+	if index, ok := e.labelIndex[label]; ok {
+		return index, nil
+	}
+	index := uint32(len(e.labels))
+	e.labels = append(e.labels, label)
+	e.labelIndex[label] = index
+	return index, nil
+}
+
+func padFour(value string) string {
+	if len(value) >= 4 {
+		return value[:4]
+	}
+	return value + string(bytes.Repeat([]byte(" "), 4-len(value)))
+}
+
+func defaultString(value, fallback string) string {
+	if value == "" {
+		return fallback
+	}
+	return value
+}

internal/gff/binary_test.go

@@ -0,0 +1,55 @@
+package gff
+
+import (
+	"bytes"
+	"encoding/json"
+	"testing"
+)
+
+func TestRoundTripBinaryAndJSON(t *testing.T) {
+	original := Document{
+		FileType:    "UTC ",
+		FileVersion: "V3.2",
+		Root: Struct{
+			Type: 0,
+			Fields: []Field{
+				NewField("Tag", StringValue("test_creature")),
+				NewField("TemplateResRef", ResRefValue("nw_test")),
+				NewField("HP", IntValue(12)),
+				NewField("Position", Vector{X: 1.25, Y: 2.5, Z: 3.75}),
+				NewField("Inventory", ListValue{
+					{
+						Type: 1,
+						Fields: []Field{
+							NewField("Slot", DWordValue(0)),
+							NewField("ResRef", ResRefValue("itm_sword")),
+						},
+					},
+				}),
+			},
+		},
+	}
+
+	var binaryBuf bytes.Buffer
+	if err := Write(&binaryBuf, original); err != nil {
+		t.Fatalf("write binary: %v", err)
+	}
+
+	decoded, err := Read(bytes.NewReader(binaryBuf.Bytes()))
+	if err != nil {
+		t.Fatalf("read binary: %v", err)
+	}
+
+	left, err := json.Marshal(original)
+	if err != nil {
+		t.Fatalf("marshal original json: %v", err)
+	}
+	right, err := json.Marshal(decoded)
+	if err != nil {
+		t.Fatalf("marshal decoded json: %v", err)
+	}
+
+	if string(left) != string(right) {
+		t.Fatalf("roundtrip mismatch\noriginal: %s\ndecoded:  %s", left, right)
+	}
+}

internal/gff/json.go

@@ -0,0 +1,317 @@
+package gff
+
+import (
+	"encoding/base64"
+	"encoding/json"
+	"fmt"
+)
+
+type jsonDocument struct {
+	FileType    string     `json:"file_type"`
+	FileVersion string     `json:"file_version"`
+	Root        jsonStruct `json:"root"`
+}
+
+type jsonStruct struct {
+	StructType uint32      `json:"struct_type"`
+	Fields     []jsonField `json:"fields"`
+}
+
+type jsonField struct {
+	Label string          `json:"label"`
+	Type  string          `json:"type"`
+	Value json.RawMessage `json:"value"`
+}
+
+type jsonLocString struct {
+	StringRef uint32           `json:"string_ref"`
+	Entries   []LocStringEntry `json:"entries"`
+}
+
+func (s Struct) MarshalJSON() ([]byte, error) {
+	payload, err := marshalStruct(s)
+	if err != nil {
+		return nil, err
+	}
+	return json.Marshal(payload)
+}
+
+func (s *Struct) UnmarshalJSON(data []byte) error {
+	var payload jsonStruct
+	if err := json.Unmarshal(data, &payload); err != nil {
+		return err
+	}
+	decoded, err := unmarshalStruct(payload)
+	if err != nil {
+		return err
+	}
+	*s = decoded
+	return nil
+}
+
+func (d Document) MarshalJSON() ([]byte, error) {
+	root, err := marshalStruct(d.Root)
+	if err != nil {
+		return nil, err
+	}
+
+	return json.Marshal(jsonDocument{
+		FileType:    d.FileType,
+		FileVersion: d.FileVersion,
+		Root:        root,
+	})
+}
+
+func (d *Document) UnmarshalJSON(data []byte) error {
+	var payload jsonDocument
+	if err := json.Unmarshal(data, &payload); err != nil {
+		return err
+	}
+
+	root, err := unmarshalStruct(payload.Root)
+	if err != nil {
+		return err
+	}
+
+	d.FileType = payload.FileType
+	d.FileVersion = payload.FileVersion
+	d.Root = root
+	return nil
+}
+
+func marshalStruct(in Struct) (jsonStruct, error) {
+	fields := make([]jsonField, 0, len(in.Fields))
+	for _, field := range in.Fields {
+		payload, err := marshalValue(field.Value)
+		if err != nil {
+			return jsonStruct{}, fmt.Errorf("marshal field %q: %w", field.Label, err)
+		}
+		fields = append(fields, jsonField{
+			Label: field.Label,
+			Type:  field.Type.String(),
+			Value: payload,
+		})
+	}
+
+	return jsonStruct{
+		StructType: in.Type,
+		Fields:     fields,
+	}, nil
+}
+
+func unmarshalStruct(in jsonStruct) (Struct, error) {
+	fields := make([]Field, 0, len(in.Fields))
+	for _, field := range in.Fields {
+		ft, err := parseFieldType(field.Type)
+		if err != nil {
+			return Struct{}, fmt.Errorf("field %q: %w", field.Label, err)
+		}
+
+		value, err := unmarshalValue(ft, field.Value)
+		if err != nil {
+			return Struct{}, fmt.Errorf("field %q: %w", field.Label, err)
+		}
+
+		fields = append(fields, Field{
+			Label: field.Label,
+			Type:  ft,
+			Value: value,
+		})
+	}
+
+	return Struct{
+		Type:   in.StructType,
+		Fields: fields,
+	}, nil
+}
+
+func marshalValue(value Value) (json.RawMessage, error) {
+	switch typed := value.(type) {
+	case ByteValue:
+		return json.Marshal(uint8(typed))
+	case CharValue:
+		return json.Marshal(int8(typed))
+	case WordValue:
+		return json.Marshal(uint16(typed))
+	case ShortValue:
+		return json.Marshal(int16(typed))
+	case DWordValue:
+		return json.Marshal(uint32(typed))
+	case IntValue:
+		return json.Marshal(int32(typed))
+	case DWord64Value:
+		return json.Marshal(uint64(typed))
+	case Int64Value:
+		return json.Marshal(int64(typed))
+	case FloatValue:
+		return json.Marshal(float32(typed))
+	case DoubleValue:
+		return json.Marshal(float64(typed))
+	case StringValue:
+		return json.Marshal(string(typed))
+	case ResRefValue:
+		return json.Marshal(string(typed))
+	case LocString:
+		return json.Marshal(jsonLocString(typed))
+	case VoidValue:
+		return json.Marshal(base64.StdEncoding.EncodeToString([]byte(typed)))
+	case Struct:
+		payload, err := marshalStruct(typed)
+		if err != nil {
+			return nil, err
+		}
+		return json.Marshal(payload)
+	case ListValue:
+		payload := make([]jsonStruct, 0, len(typed))
+		for _, item := range typed {
+			entry, err := marshalStruct(item)
+			if err != nil {
+				return nil, err
+			}
+			payload = append(payload, entry)
+		}
+		return json.Marshal(payload)
+	case Orientation:
+		return json.Marshal(typed)
+	case Vector:
+		return json.Marshal(typed)
+	default:
+		return nil, fmt.Errorf("unsupported value type %T", value)
+	}
+}
+
+func unmarshalValue(ft FieldType, data []byte) (Value, error) {
+	switch ft {
+	case TypeByte:
+		var out uint8
+		if err := json.Unmarshal(data, &out); err != nil {
+			return nil, err
+		}
+		return ByteValue(out), nil
+	case TypeChar:
+		var out int8
+		if err := json.Unmarshal(data, &out); err != nil {
+			return nil, err
+		}
+		return CharValue(out), nil
+	case TypeWord:
+		var out uint16
+		if err := json.Unmarshal(data, &out); err != nil {
+			return nil, err
+		}
+		return WordValue(out), nil
+	case TypeShort:
+		var out int16
+		if err := json.Unmarshal(data, &out); err != nil {
+			return nil, err
+		}
+		return ShortValue(out), nil
+	case TypeDWord:
+		var out uint32
+		if err := json.Unmarshal(data, &out); err != nil {
+			return nil, err
+		}
+		return DWordValue(out), nil
+	case TypeInt:
+		var out int32
+		if err := json.Unmarshal(data, &out); err != nil {
+			return nil, err
+		}
+		return IntValue(out), nil
+	case TypeDWord64:
+		var out uint64
+		if err := json.Unmarshal(data, &out); err != nil {
+			return nil, err
+		}
+		return DWord64Value(out), nil
+	case TypeInt64:
+		var out int64
+		if err := json.Unmarshal(data, &out); err != nil {
+			return nil, err
+		}
+		return Int64Value(out), nil
+	case TypeFloat:
+		var out float32
+		if err := json.Unmarshal(data, &out); err != nil {
+			return nil, err
+		}
+		return FloatValue(out), nil
+	case TypeDouble:
+		var out float64
+		if err := json.Unmarshal(data, &out); err != nil {
+			return nil, err
+		}
+		return DoubleValue(out), nil
+	case TypeCExoString:
+		var out string
+		if err := json.Unmarshal(data, &out); err != nil {
+			return nil, err
+		}
+		return StringValue(out), nil
+	case TypeResRef:
+		var out string
+		if err := json.Unmarshal(data, &out); err != nil {
+			return nil, err
+		}
+		return ResRefValue(out), nil
+	case TypeCExoLocString:
+		var out jsonLocString
+		if err := json.Unmarshal(data, &out); err != nil {
+			return nil, err
+		}
+		return LocString(out), nil
+	case TypeVoid:
+		var out string
+		if err := json.Unmarshal(data, &out); err != nil {
+			return nil, err
+		}
+		decoded, err := base64.StdEncoding.DecodeString(out)
+		if err != nil {
+			return nil, fmt.Errorf("decode base64 void: %w", err)
+		}
+		return VoidValue(decoded), nil
+	case TypeStruct:
+		var payload jsonStruct
+		if err := json.Unmarshal(data, &payload); err != nil {
+			return nil, err
+		}
+		return unmarshalStruct(payload)
+	case TypeList:
+		var payload []jsonStruct
+		if err := json.Unmarshal(data, &payload); err != nil {
+			return nil, err
+		}
+		out := make(ListValue, 0, len(payload))
+		for _, item := range payload {
+			decoded, err := unmarshalStruct(item)
+			if err != nil {
+				return nil, err
+			}
+			out = append(out, decoded)
+		}
+		return out, nil
+	case TypeOrientation:
+		var out Orientation
+		if err := json.Unmarshal(data, &out); err != nil {
+			return nil, err
+		}
+		return out, nil
+	case TypeVector:
+		var out Vector
+		if err := json.Unmarshal(data, &out); err != nil {
+			return nil, err
+		}
+		return out, nil
+	default:
+		return nil, fmt.Errorf("unsupported field type %d", ft)
+	}
+}
+
+func parseFieldType(name string) (FieldType, error) {
+	for kind, candidate := range fieldTypeNames {
+		if candidate == name {
+			return kind, nil
+		}
+	}
+	return 0, fmt.Errorf("unknown field type %q", name)
+}

internal/gff/types.go

@@ -0,0 +1,140 @@
+package gff
+
+import "fmt"
+
+type FieldType uint32
+
+const (
+	TypeByte FieldType = iota
+	TypeChar
+	TypeWord
+	TypeShort
+	TypeDWord
+	TypeInt
+	TypeDWord64
+	TypeInt64
+	TypeFloat
+	TypeDouble
+	TypeCExoString
+	TypeResRef
+	TypeCExoLocString
+	TypeVoid
+	TypeStruct
+	TypeList
+	TypeOrientation
+	TypeVector
+)
+
+var fieldTypeNames = map[FieldType]string{
+	TypeByte:          "Byte",
+	TypeChar:          "Char",
+	TypeWord:          "Word",
+	TypeShort:         "Short",
+	TypeDWord:         "DWord",
+	TypeInt:           "Int",
+	TypeDWord64:       "DWord64",
+	TypeInt64:         "Int64",
+	TypeFloat:         "Float",
+	TypeDouble:        "Double",
+	TypeCExoString:    "CExoString",
+	TypeResRef:        "ResRef",
+	TypeCExoLocString: "CExoLocString",
+	TypeVoid:          "Void",
+	TypeStruct:        "Struct",
+	TypeList:          "List",
+	TypeOrientation:   "Orientation",
+	TypeVector:        "Vector",
+}
+
+type Document struct {
+	FileType    string `json:"file_type"`
+	FileVersion string `json:"file_version"`
+	Root        Struct `json:"root"`
+}
+
+type Struct struct {
+	Type   uint32  `json:"struct_type"`
+	Fields []Field `json:"fields"`
+}
+
+type Field struct {
+	Label string    `json:"label"`
+	Type  FieldType `json:"-"`
+	Value Value     `json:"-"`
+}
+
+type Value interface {
+	fieldType() FieldType
+}
+
+type LocString struct {
+	StringRef uint32           `json:"string_ref"`
+	Entries   []LocStringEntry `json:"entries"`
+}
+
+type LocStringEntry struct {
+	ID    uint32 `json:"id"`
+	Value string `json:"value"`
+}
+
+type Vector struct {
+	X float32 `json:"x"`
+	Y float32 `json:"y"`
+	Z float32 `json:"z"`
+}
+
+type Orientation struct {
+	X float32 `json:"x"`
+	Y float32 `json:"y"`
+	Z float32 `json:"z"`
+	W float32 `json:"w"`
+}
+
+type ByteValue uint8
+type CharValue int8
+type WordValue uint16
+type ShortValue int16
+type DWordValue uint32
+type IntValue int32
+type DWord64Value uint64
+type Int64Value int64
+type FloatValue float32
+type DoubleValue float64
+type StringValue string
+type ResRefValue string
+type VoidValue []byte
+type ListValue []Struct
+
+func (ByteValue) fieldType() FieldType    { return TypeByte }
+func (CharValue) fieldType() FieldType    { return TypeChar }
+func (WordValue) fieldType() FieldType    { return TypeWord }
+func (ShortValue) fieldType() FieldType   { return TypeShort }
+func (DWordValue) fieldType() FieldType   { return TypeDWord }
+func (IntValue) fieldType() FieldType     { return TypeInt }
+func (DWord64Value) fieldType() FieldType { return TypeDWord64 }
+func (Int64Value) fieldType() FieldType   { return TypeInt64 }
+func (FloatValue) fieldType() FieldType   { return TypeFloat }
+func (DoubleValue) fieldType() FieldType  { return TypeDouble }
+func (StringValue) fieldType() FieldType  { return TypeCExoString }
+func (ResRefValue) fieldType() FieldType  { return TypeResRef }
+func (LocString) fieldType() FieldType    { return TypeCExoLocString }
+func (VoidValue) fieldType() FieldType    { return TypeVoid }
+func (Struct) fieldType() FieldType       { return TypeStruct }
+func (ListValue) fieldType() FieldType    { return TypeList }
+func (Orientation) fieldType() FieldType  { return TypeOrientation }
+func (Vector) fieldType() FieldType       { return TypeVector }
+
+func (t FieldType) String() string {
+	if name, ok := fieldTypeNames[t]; ok {
+		return name
+	}
+	return fmt.Sprintf("FieldType(%d)", t)
+}
+
+func NewField(label string, value Value) Field {
+	return Field{
+		Label: label,
+		Type:  value.fieldType(),
+		Value: value,
+	}
+}

internal/music/config.go

@@ -0,0 +1,47 @@
+package music
+
+import (
+	"fmt"
+	"path/filepath"
+	"strings"
+)
+
+type DatasetConfig struct {
+	Source      string `json:"source" yaml:"source"`
+	Prefix      string `json:"prefix,omitempty" yaml:"prefix,omitempty"`
+	StageRoot   string `json:"stage_root,omitempty" yaml:"stage_root,omitempty"`
+	CreditsRoot string `json:"credits_root,omitempty" yaml:"credits_root,omitempty"`
+}
+
+type DefaultsConfig struct {
+	StageRoot      string `json:"stage_root,omitempty" yaml:"stage_root,omitempty"`
+	CreditsRoot    string `json:"credits_root,omitempty" yaml:"credits_root,omitempty"`
+	CreditsOverlay string `json:"credits_overlay,omitempty" yaml:"credits_overlay,omitempty"`
+	MaxStemLength  *int   `json:"max_stem_length,omitempty" yaml:"max_stem_length,omitempty"`
+}
+
+func ValidateDatasetSource(field, source string) error {
+	trimmed := strings.TrimSpace(source)
+	if trimmed == "" {
+		return fmt.Errorf("%s.source is required", field)
+	}
+	if strings.Contains(trimmed, "\x00") {
+		return fmt.Errorf("%s.source must not contain NUL bytes", field)
+	}
+	clean := filepath.Clean(filepath.FromSlash(trimmed))
+	if clean == ".." || strings.HasPrefix(clean, ".."+string(filepath.Separator)) {
+		return fmt.Errorf("%s.source must not escape project root", field)
+	}
+	return nil
+}
+
+func ValidateDatasetID(id string) error {
+	trimmed := strings.TrimSpace(id)
+	if trimmed == "" {
+		return fmt.Errorf("dataset id must not be empty")
+	}
+	if strings.Contains(trimmed, "/") || strings.Contains(trimmed, "\\") {
+		return fmt.Errorf("dataset id %q must not contain path separators", trimmed)
+	}
+	return nil
+}

internal/music/credits.go

@@ -0,0 +1,292 @@
+package music
+
+import (
+	"fmt"
+	"os"
+	"path/filepath"
+	"strings"
+)
+
+func ParseCreditsOverlay(path string) (CreditsOverlay, error) {
+	entries, err := ParseCreditsMarkdown(path)
+	if err != nil {
+		return CreditsOverlay{}, err
+	}
+	overlay := CreditsOverlay{
+		ByOriginal: make(map[string]CreditsEntry),
+		ByOutput:   make(map[string]CreditsEntry),
+		Entries:    entries,
+	}
+	for _, entry := range entries {
+		if entry.OriginalFile != "" {
+			key := strings.ToLower(strings.TrimSpace(entry.OriginalFile))
+			if _, exists := overlay.ByOriginal[key]; exists {
+				return CreditsOverlay{}, fmt.Errorf("duplicate manual credits row for original file %s", entry.OriginalFile)
+			}
+			overlay.ByOriginal[key] = entry
+		}
+		if entry.OutputFile != "" {
+			key := strings.ToLower(strings.TrimSpace(entry.OutputFile))
+			if _, exists := overlay.ByOutput[key]; exists {
+				return CreditsOverlay{}, fmt.Errorf("duplicate manual credits row for output file %s", entry.OutputFile)
+			}
+			overlay.ByOutput[key] = entry
+		}
+	}
+	return overlay, nil
+}
+
+func (o CreditsOverlay) Match(originalFile, outputFile string) *CreditsEntry {
+	if entry, ok := o.ByOutput[strings.ToLower(strings.TrimSpace(outputFile))]; ok {
+		entryCopy := entry
+		return &entryCopy
+	}
+	if entry, ok := o.ByOriginal[strings.ToLower(strings.TrimSpace(originalFile))]; ok {
+		entryCopy := entry
+		return &entryCopy
+	}
+	return nil
+}
+
+func ValidateOverlayEntries(dir string, overlay CreditsOverlay, generated []CreditsEntry) error {
+	if len(overlay.Entries) == 0 {
+		return nil
+	}
+	validOriginal := make(map[string]struct{}, len(generated))
+	validOutput := make(map[string]struct{}, len(generated))
+	for _, entry := range generated {
+		validOriginal[strings.ToLower(entry.OriginalFile)] = struct{}{}
+		validOutput[strings.ToLower(entry.OutputFile)] = struct{}{}
+	}
+	for _, entry := range overlay.Entries {
+		if entry.OriginalFile != "" {
+			if _, ok := validOriginal[strings.ToLower(entry.OriginalFile)]; !ok {
+				return fmt.Errorf("manual credits entry in %s references unknown original file %s", dir, entry.OriginalFile)
+			}
+		}
+		if entry.OutputFile != "" {
+			if _, ok := validOutput[strings.ToLower(entry.OutputFile)]; !ok {
+				return fmt.Errorf("manual credits entry in %s references unknown output file %s", dir, entry.OutputFile)
+			}
+		}
+	}
+	return nil
+}
+
+func ParseCreditsMarkdown(path string) ([]CreditsEntry, error) {
+	raw, err := os.ReadFile(path)
+	if err != nil {
+		if os.IsNotExist(err) {
+			return nil, nil
+		}
+		return nil, err
+	}
+	lines := strings.Split(strings.ReplaceAll(string(raw), "\r\n", "\n"), "\n")
+	header := []string(nil)
+	entries := make([]CreditsEntry, 0)
+	for _, line := range lines {
+		line = strings.TrimSpace(line)
+		if !strings.HasPrefix(line, "|") {
+			continue
+		}
+		cells := ParseMarkdownTableRow(line)
+		if len(cells) == 0 {
+			continue
+		}
+		if header == nil {
+			header = make([]string, len(cells))
+			for i, cell := range cells {
+				header[i] = NormalizeCreditsHeader(cell)
+			}
+			continue
+		}
+		if IsMarkdownSeparatorRow(cells) {
+			continue
+		}
+		entry := CreditsEntry{}
+		for i, key := range header {
+			if i >= len(cells) {
+				continue
+			}
+			value := CleanCreditsCell(cells[i])
+			switch key {
+			case "artist":
+				entry.Artist = value
+			case "title":
+				entry.Title = value
+			case "output_file":
+				entry.OutputFile = value
+			case "original_file":
+				entry.OriginalFile = value
+			case "album":
+				entry.Album = value
+			case "date":
+				entry.Date = value
+			case "rights":
+				entry.Rights = value
+			case "notes":
+				entry.Notes = value
+			}
+		}
+		if entry.Artist == "" && entry.Title == "" && entry.OutputFile == "" && entry.OriginalFile == "" {
+			continue
+		}
+		entries = append(entries, entry)
+	}
+	return entries, nil
+}
+
+func ParseMarkdownTableRow(line string) []string {
+	trimmed := strings.TrimSpace(line)
+	trimmed = strings.TrimPrefix(trimmed, "|")
+	trimmed = strings.TrimSuffix(trimmed, "|")
+	parts := strings.Split(trimmed, "|")
+	cells := make([]string, 0, len(parts))
+	for _, part := range parts {
+		cells = append(cells, strings.TrimSpace(strings.ReplaceAll(part, `\|`, "|")))
+	}
+	return cells
+}
+
+func IsMarkdownSeparatorRow(cells []string) bool {
+	if len(cells) == 0 {
+		return false
+	}
+	for _, cell := range cells {
+		cell = strings.TrimSpace(cell)
+		cell = strings.ReplaceAll(cell, "-", "")
+		cell = strings.ReplaceAll(cell, ":", "")
+		if cell != "" {
+			return false
+		}
+	}
+	return true
+}
+
+func NormalizeCreditsHeader(value string) string {
+	switch strings.ToLower(CleanCreditsCell(value)) {
+	case "artist":
+		return "artist"
+	case "title":
+		return "title"
+	case "output file":
+		return "output_file"
+	case "original file":
+		return "original_file"
+	case "album":
+		return "album"
+	case "date":
+		return "date"
+	case "license / copyright":
+		return "rights"
+	case "notes":
+		return "notes"
+	default:
+		return ""
+	}
+}
+
+func CleanCreditsCell(value string) string {
+	value = strings.TrimSpace(value)
+	value = strings.Trim(value, "`")
+	value = strings.ReplaceAll(value, "<br>", "\n")
+	return strings.TrimSpace(value)
+}
+
+func RenderCreditsMarkdown(entries []CreditsEntry) string {
+	var builder strings.Builder
+	builder.WriteString(CreditsHeader)
+	for _, entry := range entries {
+		builder.WriteString("| ")
+		builder.WriteString(EscapeMarkdownCell(entry.Artist))
+		builder.WriteString(" | ")
+		builder.WriteString(EscapeMarkdownCell(entry.Title))
+		builder.WriteString(" | ")
+		builder.WriteString(EscapeMarkdownCell(WrapCodeCell(entry.OutputFile)))
+		builder.WriteString(" | ")
+		builder.WriteString(EscapeMarkdownCell(WrapCodeCell(entry.OriginalFile)))
+		builder.WriteString(" | ")
+		builder.WriteString(EscapeMarkdownCell(entry.Album))
+		builder.WriteString(" | ")
+		builder.WriteString(EscapeMarkdownCell(entry.Date))
+		builder.WriteString(" | ")
+		builder.WriteString(EscapeMarkdownCell(entry.Rights))
+		builder.WriteString(" | ")
+		builder.WriteString(EscapeMarkdownCell(entry.Notes))
+		builder.WriteString(" |\n")
+	}
+	return builder.String()
+}
+
+func WrapCodeCell(value string) string {
+	if strings.TrimSpace(value) == "" {
+		return ""
+	}
+	return "`" + value + "`"
+}
+
+func EscapeMarkdownCell(value string) string {
+	value = strings.ReplaceAll(value, `\`, `\\`)
+	value = strings.ReplaceAll(value, "|", `\|`)
+	value = strings.ReplaceAll(value, "\n", "<br>")
+	return value
+}
+
+func ApplyCreditsOverlay(entry *CreditsEntry, overlay *CreditsEntry) {
+	if overlay == nil {
+		return
+	}
+	entry.Artist = FirstNonEmpty(overlay.Artist, entry.Artist)
+	entry.Title = FirstNonEmpty(overlay.Title, entry.Title)
+	entry.Album = FirstNonEmpty(overlay.Album, entry.Album)
+	entry.Date = FirstNonEmpty(overlay.Date, entry.Date)
+	entry.Rights = FirstNonEmpty(overlay.Rights, entry.Rights)
+	entry.Notes = FirstNonEmpty(overlay.Notes, entry.Notes)
+}
+
+func SyncGeneratedCreditsArtifacts(root string, desiredFiles map[string][]byte) (int, error) {
+	changedFiles := 0
+	existingFiles := make(map[string]struct{})
+	if err := filepath.WalkDir(root, func(path string, d os.DirEntry, walkErr error) error {
+		if walkErr != nil {
+			if os.IsNotExist(walkErr) {
+				return nil
+			}
+			return walkErr
+		}
+		if d.IsDir() {
+			return nil
+		}
+		existingFiles[path] = struct{}{}
+		return nil
+	}); err != nil {
+		return 0, fmt.Errorf("scan credits dir: %w", err)
+	}
+
+	for path, content := range desiredFiles {
+		current, err := os.ReadFile(path)
+		if err != nil && !os.IsNotExist(err) {
+			return 0, fmt.Errorf("read credits artifact %s: %w", path, err)
+		}
+		if err == nil && string(current) == string(content) {
+			delete(existingFiles, path)
+			continue
+		}
+		if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil {
+			return 0, fmt.Errorf("create generated credits dir: %w", err)
+		}
+		if err := os.WriteFile(path, content, 0o644); err != nil {
+			return 0, fmt.Errorf("write credits artifact %s: %w", path, err)
+		}
+		changedFiles++
+		delete(existingFiles, path)
+	}
+
+	for path := range existingFiles {
+		if err := os.Remove(path); err != nil && !os.IsNotExist(err) {
+			return 0, fmt.Errorf("remove stale credits artifact %s: %w", path, err)
+		}
+		changedFiles++
+	}
+	return changedFiles, nil
+}

internal/music/metadata.go

@@ -0,0 +1,62 @@
+package music
+
+import (
+	"encoding/json"
+	"fmt"
+	"os/exec"
+	"strings"
+)
+
+func ProbeMetadata(ffprobePath, sourcePath string) (Metadata, error) {
+	cmd := exec.Command(ffprobePath, "-v", "quiet", "-print_format", "json", "-show_format", sourcePath)
+	output, err := cmd.Output()
+	if err != nil {
+		return Metadata{}, err
+	}
+	var payload struct {
+		Format struct {
+			Tags map[string]string `json:"tags"`
+		} `json:"format"`
+	}
+	if err := json.Unmarshal(output, &payload); err != nil {
+		return Metadata{}, err
+	}
+	tags := make(map[string]string, len(payload.Format.Tags))
+	for key, value := range payload.Format.Tags {
+		tags[strings.ToLower(strings.TrimSpace(key))] = strings.TrimSpace(value)
+	}
+	return Metadata{
+		Title:     FirstNonEmpty(tags["title"]),
+		Artist:    FirstNonEmpty(tags["artist"], tags["album_artist"], tags["composer"]),
+		Album:     FirstNonEmpty(tags["album"]),
+		Date:      FirstNonEmpty(tags["date"], tags["year"]),
+		Comment:   FirstNonEmpty(tags["comment"], tags["description"]),
+		Copyright: FirstNonEmpty(tags["copyright"]),
+		License:   FirstNonEmpty(tags["license"], tags["license_url"]),
+	}, nil
+}
+
+func ConvertSource(ffmpegPath, sourcePath, outputPath string) error {
+	cmd := exec.Command(
+		ffmpegPath, "-y",
+		"-i", sourcePath,
+		"-vn",
+		"-map_metadata", "-1",
+		"-ar", "44100",
+		"-ac", "2",
+		"-b:a", "192k",
+		"-codec:a", "libmp3lame",
+		"-write_xing", "0",
+		"-f", "mp3",
+		outputPath,
+	)
+	output, err := cmd.CombinedOutput()
+	if err != nil {
+		message := strings.TrimSpace(string(output))
+		if message == "" {
+			message = err.Error()
+		}
+		return fmt.Errorf("%s", message)
+	}
+	return nil
+}

internal/music/music.go

@@ -0,0 +1,226 @@
+package music
+
+import (
+	"os"
+	"os/exec"
+	"path/filepath"
+	"regexp"
+	"sort"
+	"strings"
+)
+
+const (
+	MaxStemLen    = 16
+	CreditsHeader = "# NWN Music Pack Credits\n\nOriginal rights remain with the respective composers and licensors.\n\n## Processed Files\n\n| Artist | Title | Output File | Original File | Album | Date | License / Copyright | Notes |\n|---|---|---|---|---|---|---|---|\n"
+)
+
+var (
+	defaultConvertExtensions = []string{
+		".flac",
+		".m4a",
+		".mp3",
+		".ogg",
+		".wav",
+	}
+	ConvertExtensions = map[string]struct{}{
+		".flac": {},
+		".m4a":  {},
+		".mp3":  {},
+		".ogg":  {},
+		".wav":  {},
+	}
+	PassthroughExtensions = map[string]struct{}{
+		".bmu": {},
+		".wav": {},
+	}
+	DropWords = map[string]struct{}{
+		"mp3": {}, "wav": {}, "flac": {}, "ogg": {}, "m4a": {},
+		"music": {}, "track": {}, "audio": {}, "song": {}, "soundtrack": {},
+		"final": {}, "version": {}, "ver": {}, "v": {}, "mix": {}, "master": {}, "remaster": {},
+		"free": {}, "royalty": {}, "copyright": {}, "background": {}, "bgm": {},
+		"loop": {}, "loops": {}, "loopable": {}, "seamless": {},
+		"official": {}, "download": {}, "preview": {},
+		"the": {}, "a": {}, "an": {}, "and": {}, "of": {}, "to": {}, "in": {}, "for": {}, "with": {}, "by": {},
+	}
+	BracketRE = regexp.MustCompile(`\[[^\]]*\]|\([^)]*\)|\{[^}]*\}`)
+	SplitRE   = regexp.MustCompile(`[^A-Za-z0-9]+`)
+	YearRE    = regexp.MustCompile(`^\d{4,}$`)
+	VowelRE   = regexp.MustCompile(`[aeiou]`)
+)
+
+func DefaultConvertExtensions() []string {
+	return append([]string(nil), defaultConvertExtensions...)
+}
+
+type Metadata struct {
+	Title     string
+	Artist    string
+	Album     string
+	Date      string
+	Rights    string
+	Notes     string
+	Comment   string
+	Copyright string
+	License   string
+}
+
+type CreditsEntry struct {
+	Artist       string `json:"artist"`
+	Title        string `json:"title"`
+	OutputFile   string `json:"output_file"`
+	OriginalFile string `json:"original_file"`
+	Album        string `json:"album"`
+	Date         string `json:"date"`
+	Rights       string `json:"rights"`
+	Notes        string `json:"notes"`
+}
+
+type CreditsSource struct {
+	Path    string         `json:"path"`
+	Kind    string         `json:"kind"`
+	Entries []CreditsEntry `json:"entries"`
+}
+
+type CreditsInventory struct {
+	Sources []CreditsSource `json:"sources"`
+}
+
+type CreditsOverlay struct {
+	ByOriginal map[string]CreditsEntry
+	ByOutput   map[string]CreditsEntry
+	Entries    []CreditsEntry
+}
+
+func ResolveFFmpeg() (string, error) {
+	return ResolveFFmpegWithConfig("")
+}
+
+func ResolveFFmpegWithConfig(configuredPath string) (string, error) {
+	if configured := strings.TrimSpace(configuredPath); configured != "" {
+		return configured, nil
+	}
+	if configured := strings.TrimSpace(os.Getenv("SOW_FFMPEG")); configured != "" {
+		return configured, nil
+	}
+	path, err := exec.LookPath("ffmpeg")
+	if err != nil {
+		return "", err
+	}
+	return path, nil
+}
+
+func ResolveFFprobe() (string, error) {
+	return ResolveFFprobeWithConfig("")
+}
+
+func ResolveFFprobeWithConfig(configuredPath string) (string, error) {
+	if configured := strings.TrimSpace(configuredPath); configured != "" {
+		return configured, nil
+	}
+	if configured := strings.TrimSpace(os.Getenv("SOW_FFPROBE")); configured != "" {
+		return configured, nil
+	}
+	path, err := exec.LookPath("ffprobe")
+	if err != nil {
+		return "", err
+	}
+	return path, nil
+}
+
+func IsMusicAssetPath(rel string) bool {
+	rel = filepath.ToSlash(strings.TrimSpace(rel))
+	return rel == "envi/music" || strings.HasPrefix(rel, "envi/music/")
+}
+
+func PathIsUnder(root, rel string) bool {
+	root = TrimSlashes(filepath.ToSlash(root))
+	rel = TrimSlashes(filepath.ToSlash(rel))
+	if root == "" || rel == "" {
+		return false
+	}
+	return rel == root || strings.HasPrefix(rel, root+"/")
+}
+
+func PrefixForDir(prefixes map[string]string, dir string) string {
+	dir = TrimSlashes(filepath.ToSlash(dir))
+	bestPrefix := ""
+	bestLen := -1
+	keys := make([]string, 0, len(prefixes))
+	for key := range prefixes {
+		keys = append(keys, key)
+	}
+	sort.Strings(keys)
+	for _, key := range keys {
+		value := prefixes[key]
+		normalizedKey := TrimSlashes(filepath.ToSlash(key))
+		if normalizedKey == "" {
+			continue
+		}
+		if dir != normalizedKey && !strings.HasPrefix(dir, normalizedKey+"/") {
+			continue
+		}
+		if len(normalizedKey) > bestLen {
+			bestLen = len(normalizedKey)
+			bestPrefix = SanitizePrefix(value)
+		}
+	}
+	return bestPrefix
+}
+
+func SanitizePrefix(prefix string) string {
+	prefix = strings.ToLower(strings.TrimSpace(prefix))
+	var builder strings.Builder
+	for _, r := range prefix {
+		switch {
+		case r >= 'a' && r <= 'z':
+			builder.WriteRune(r)
+		case r >= '0' && r <= '9':
+			builder.WriteRune(r)
+		case r == '_':
+			builder.WriteRune(r)
+		}
+	}
+	result := builder.String()
+	if strings.TrimSpace(prefix) != "" && strings.HasSuffix(strings.TrimSpace(prefix), "_") && !strings.HasSuffix(result, "_") {
+		result += "_"
+	}
+	return result
+}
+
+func TrimSlashes(value string) string {
+	return strings.Trim(strings.TrimSpace(value), "/")
+}
+
+func Min(a, b int) int {
+	if a < b {
+		return a
+	}
+	return b
+}
+
+func Max(a, b int) int {
+	if a > b {
+		return a
+	}
+	return b
+}
+
+func FirstNonEmpty(values ...string) string {
+	for _, value := range values {
+		if strings.TrimSpace(value) != "" {
+			return strings.TrimSpace(value)
+		}
+	}
+	return ""
+}
+
+func JoinNonEmpty(sep string, values ...string) string {
+	parts := make([]string, 0, len(values))
+	for _, value := range values {
+		value = strings.TrimSpace(value)
+		if value != "" {
+			parts = append(parts, value)
+		}
+	}
+	return strings.Join(parts, sep)
+}

internal/music/music_test.go

@@ -0,0 +1,404 @@
+package music
+
+import (
+	"os"
+	"path/filepath"
+	"strings"
+	"testing"
+)
+
+func TestUniqueNameHandlesShortStemCollisions(t *testing.T) {
+	used := map[string]struct{}{
+		"mus_wg_mystc": {},
+	}
+
+	got := UniqueName("mus_wg_mystc", used)
+	if got != "mus_wg_mystc_1" {
+		t.Fatalf("unexpected collision result: got %q want %q", got, "mus_wg_mystc_1")
+	}
+}
+
+func TestUniqueNameNoCollision(t *testing.T) {
+	used := map[string]struct{}{}
+	got := UniqueName("new_stem", used)
+	if got != "new_stem" {
+		t.Fatalf("expected 'new_stem', got %q", got)
+	}
+}
+
+func TestUniqueNameMultipleCollisions(t *testing.T) {
+	used := map[string]struct{}{
+		"stem":   {},
+		"stem_1": {},
+		"stem_2": {},
+	}
+	got := UniqueName("stem", used)
+	if got != "stem_3" {
+		t.Fatalf("expected 'stem_3', got %q", got)
+	}
+}
+
+func TestGenerateStem(t *testing.T) {
+	used := map[string]struct{}{
+		"existing": {},
+	}
+	stem, err := GenerateStem("My Cool Track.mp3", "mus_", used)
+	if err != nil {
+		t.Fatalf("unexpected error: %v", err)
+	}
+	if stem == "" {
+		t.Fatal("expected non-empty stem")
+	}
+	if len(stem) > MaxStemLen {
+		t.Fatalf("stem %q exceeds max length %d", stem, MaxStemLen)
+	}
+	if _, exists := used[stem]; !exists {
+		t.Fatal("expected stem to be registered in used set")
+	}
+}
+
+func TestGenerateStemPrefixTooLong(t *testing.T) {
+	_, err := GenerateStem("test.mp3", "this_prefix_is_way_too_long_for_stem_", nil)
+	if err == nil {
+		t.Fatal("expected error for too-long prefix")
+	}
+}
+
+func TestSlugWords(t *testing.T) {
+	words := SlugWords("My Cool Music Track (Official) [HD]")
+	if len(words) == 0 {
+		t.Fatal("expected non-empty words")
+	}
+}
+
+func TestSlugWordsStripsBrackets(t *testing.T) {
+	words := SlugWords("Song [Explicit] (Remix)")
+	for _, w := range words {
+		if w == "explicit" || w == "remix" {
+			t.Fatalf("bracket content should be removed, got word %q", w)
+		}
+	}
+}
+
+func TestSanitizePrefix(t *testing.T) {
+	if got := SanitizePrefix("Mus_WG_"); got != "mus_wg_" {
+		t.Fatalf("unexpected prefix: %q", got)
+	}
+	if got := SanitizePrefix("  Test  "); got != "test" {
+		t.Fatalf("unexpected prefix: %q", got)
+	}
+	if got := SanitizePrefix(""); got != "" {
+		t.Fatalf("expected empty prefix, got %q", got)
+	}
+}
+
+func TestReserveName(t *testing.T) {
+	used := map[string]struct{}{}
+	if err := ReserveName("testname", used); err != nil {
+		t.Fatalf("unexpected error: %v", err)
+	}
+	if _, exists := used["testname"]; !exists {
+		t.Fatal("expected name to be reserved")
+	}
+	if err := ReserveName("testname", used); err == nil {
+		t.Fatal("expected error for duplicate name")
+	}
+}
+
+func TestValidateManualOutputFile(t *testing.T) {
+	if err := ValidateManualOutputFile("test.bmu"); err != nil {
+		t.Fatalf("unexpected error: %v", err)
+	}
+	if err := ValidateManualOutputFile("test.txt"); err == nil {
+		t.Fatal("expected error for non-bmu extension")
+	}
+	if err := ValidateManualOutputFile(".bmu"); err == nil {
+		t.Fatal("expected error for empty stem")
+	}
+}
+
+func TestParseCreditsMarkdownMissingFile(t *testing.T) {
+	entries, err := ParseCreditsMarkdown("nonexistent.md")
+	if err != nil {
+		t.Fatalf("unexpected error for missing file: %v", err)
+	}
+	if entries != nil {
+		t.Fatalf("expected nil entries for missing file, got %v", entries)
+	}
+}
+
+func TestParseCreditsMarkdownRealContent(t *testing.T) {
+	content := "# Header\n\n| Artist | Title | Output File | Original File | Album | Date | License / Copyright | Notes |\n|---|---|---|---|---|---|---|---|\n| Test Artist | Test Title | `output.bmu` | `original.mp3` | Test Album | 2024 | MIT | test |\n"
+	path := filepath.Join(t.TempDir(), "CREDITS.md")
+	if err := os.WriteFile(path, []byte(content), 0o644); err != nil {
+		t.Fatalf("write test credits: %v", err)
+	}
+
+	entries, err := ParseCreditsMarkdown(path)
+	if err != nil {
+		t.Fatalf("unexpected error: %v", err)
+	}
+	if len(entries) != 1 {
+		t.Fatalf("expected 1 entry, got %d", len(entries))
+	}
+	if entries[0].Artist != "Test Artist" {
+		t.Fatalf("expected 'Test Artist', got %q", entries[0].Artist)
+	}
+	if entries[0].Title != "Test Title" {
+		t.Fatalf("expected 'Test Title', got %q", entries[0].Title)
+	}
+	if entries[0].OutputFile != "output.bmu" {
+		t.Fatalf("expected 'output.bmu', got %q", entries[0].OutputFile)
+	}
+}
+
+func TestRenderCreditsMarkdown(t *testing.T) {
+	entries := []CreditsEntry{
+		{
+			Artist:       "Test Artist",
+			Title:        "Test Title",
+			OutputFile:   "output.bmu",
+			OriginalFile: "original.mp3",
+			Album:        "Test Album",
+			Date:         "2024",
+			Rights:       "MIT",
+			Notes:        "test note",
+		},
+	}
+
+	result := RenderCreditsMarkdown(entries)
+	if !strings.Contains(result, "Test Artist") {
+		t.Fatal("expected rendered output to contain artist")
+	}
+	if !strings.Contains(result, "output.bmu") {
+		t.Fatal("expected rendered output to contain output file")
+	}
+}
+
+func TestParseCreditsOverlay(t *testing.T) {
+	content := "# Header\n\n| Artist | Title | Output File | Original File |\n|---|---|---|---|\n| Overlay Artist | Ovr Title | `override.bmu` | `original.mp3` |\n"
+	path := filepath.Join(t.TempDir(), "CREDITS.md")
+	if err := os.WriteFile(path, []byte(content), 0o644); err != nil {
+		t.Fatalf("write test overlay: %v", err)
+	}
+
+	overlay, err := ParseCreditsOverlay(path)
+	if err != nil {
+		t.Fatalf("unexpected error: %v", err)
+	}
+	if len(overlay.Entries) != 1 {
+		t.Fatalf("expected 1 overlay entry, got %d", len(overlay.Entries))
+	}
+
+	matched := overlay.Match("original.mp3", "")
+	if matched == nil {
+		t.Fatal("expected overlay match by original file")
+	}
+	if matched.Artist != "Overlay Artist" {
+		t.Fatalf("expected 'Overlay Artist', got %q", matched.Artist)
+	}
+}
+
+func TestApplyCreditsOverlay(t *testing.T) {
+	entry := &CreditsEntry{
+		Artist: "Original Artist",
+		Title:  "Original Title",
+	}
+	overlay := &CreditsEntry{
+		Artist: "Overlay Artist",
+	}
+
+	ApplyCreditsOverlay(entry, overlay)
+	if entry.Artist != "Overlay Artist" {
+		t.Fatalf("expected 'Overlay Artist', got %q", entry.Artist)
+	}
+	if entry.Title != "Original Title" {
+		t.Fatalf("expected 'Original Title' preserved, got %q", entry.Title)
+	}
+
+	ApplyCreditsOverlay(entry, nil)
+	if entry.Artist != "Overlay Artist" {
+		t.Fatalf("expected nil overlay to be no-op, got %q", entry.Artist)
+	}
+}
+
+func TestValidateOverlayEntries(t *testing.T) {
+	generated := []CreditsEntry{
+		{OriginalFile: "track1.mp3", OutputFile: "track1.bmu"},
+	}
+	overlay := CreditsOverlay{
+		Entries: []CreditsEntry{
+			{OriginalFile: "track1.mp3"},
+		},
+	}
+	if err := ValidateOverlayEntries("test", overlay, generated); err != nil {
+		t.Fatalf("unexpected validation error: %v", err)
+	}
+
+	badOverlay := CreditsOverlay{
+		Entries: []CreditsEntry{
+			{OriginalFile: "nonexistent.mp3"},
+		},
+	}
+	if err := ValidateOverlayEntries("test", badOverlay, generated); err == nil {
+		t.Fatal("expected validation error for unknown original file")
+	}
+}
+
+func TestIsMusicAssetPath(t *testing.T) {
+	if !IsMusicAssetPath("envi/music") {
+		t.Fatal("expected 'envi/music' to be music path")
+	}
+	if !IsMusicAssetPath("envi/music/westgate") {
+		t.Fatal("expected 'envi/music/westgate' to be music path")
+	}
+	if IsMusicAssetPath("envi/textures") {
+		t.Fatal("expected 'envi/textures' not to be music path")
+	}
+	if IsMusicAssetPath("") {
+		t.Fatal("expected empty string not to be music path")
+	}
+}
+
+func TestFirstNonEmpty(t *testing.T) {
+	if got := FirstNonEmpty("", "hello", "world"); got != "hello" {
+		t.Fatalf("expected 'hello', got %q", got)
+	}
+	if got := FirstNonEmpty("", "", ""); got != "" {
+		t.Fatalf("expected empty, got %q", got)
+	}
+	if got := FirstNonEmpty("first"); got != "first" {
+		t.Fatalf("expected 'first', got %q", got)
+	}
+}
+
+func TestJoinNonEmpty(t *testing.T) {
+	if got := JoinNonEmpty(", ", "a", "", "b"); got != "a, b" {
+		t.Fatalf("expected 'a, b', got %q", got)
+	}
+	if got := JoinNonEmpty(", "); got != "" {
+		t.Fatalf("expected empty, got %q", got)
+	}
+}
+
+func TestTrimSlashes(t *testing.T) {
+	if got := TrimSlashes("/foo/bar/"); got != "foo/bar" {
+		t.Fatalf("expected 'foo/bar', got %q", got)
+	}
+	if got := TrimSlashes("///"); got != "" {
+		t.Fatalf("expected empty, got %q", got)
+	}
+}
+
+func TestEscapeMarkdownCell(t *testing.T) {
+	result := EscapeMarkdownCell("a|b\nc")
+	if !strings.Contains(result, `\|`) {
+		t.Fatal("expected pipe to be escaped")
+	}
+	if !strings.Contains(result, "<br>") {
+		t.Fatal("expected newline to be replaced with <br>")
+	}
+}
+
+func TestWrapCodeCell(t *testing.T) {
+	if got := WrapCodeCell("test.bmu"); got != "`test.bmu`" {
+		t.Fatalf("expected '`test.bmu`', got %q", got)
+	}
+	if got := WrapCodeCell(""); got != "" {
+		t.Fatalf("expected empty string, got %q", got)
+	}
+}
+
+func TestSyncGeneratedCreditsArtifactsWritesNewFiles(t *testing.T) {
+	root := t.TempDir()
+	desired := map[string][]byte{
+		filepath.Join(root, "test.md"): []byte("content"),
+	}
+	changed, err := SyncGeneratedCreditsArtifacts(root, desired)
+	if err != nil {
+		t.Fatalf("unexpected error: %v", err)
+	}
+	if changed != 1 {
+		t.Fatalf("expected 1 changed file, got %d", changed)
+	}
+}
+
+func TestSyncGeneratedCreditsArtifactsNoChanges(t *testing.T) {
+	root := t.TempDir()
+	path := filepath.Join(root, "test.md")
+	if err := os.WriteFile(path, []byte("content"), 0o644); err != nil {
+		t.Fatalf("write file: %v", err)
+	}
+	desired := map[string][]byte{
+		path: []byte("content"),
+	}
+	changed, err := SyncGeneratedCreditsArtifacts(root, desired)
+	if err != nil {
+		t.Fatalf("unexpected error: %v", err)
+	}
+	if changed != 0 {
+		t.Fatalf("expected 0 changed files, got %d", changed)
+	}
+}
+
+func TestSyncGeneratedCreditsArtifactsRemovesStale(t *testing.T) {
+	root := t.TempDir()
+	stalePath := filepath.Join(root, "stale.md")
+	if err := os.WriteFile(stalePath, []byte("stale"), 0o644); err != nil {
+		t.Fatalf("write stale file: %v", err)
+	}
+	desired := map[string][]byte{}
+	changed, err := SyncGeneratedCreditsArtifacts(root, desired)
+	if err != nil {
+		t.Fatalf("unexpected error: %v", err)
+	}
+	if changed != 1 {
+		t.Fatalf("expected 1 removed file, got %d", changed)
+	}
+}
+
+func TestPrefixForDir(t *testing.T) {
+	prefixes := map[string]string{
+		"envi/music/westgate": "mus_wg_",
+	}
+	if got := PrefixForDir(prefixes, "envi/music/westgate"); got != "mus_wg_" {
+		t.Fatalf("expected 'mus_wg_', got %q", got)
+	}
+	if got := PrefixForDir(prefixes, "envi/music/other"); got != "" {
+		t.Fatalf("expected empty prefix, got %q", got)
+	}
+	if got := PrefixForDir(nil, "envi/music"); got != "" {
+		t.Fatalf("expected empty prefix for nil map, got %q", got)
+	}
+}
+
+func TestMaxMin(t *testing.T) {
+	if got := Min(3, 5); got != 3 {
+		t.Fatalf("Min(3,5) = %d, want 3", got)
+	}
+	if got := Max(3, 5); got != 5 {
+		t.Fatalf("Max(3,5) = %d, want 5", got)
+	}
+}
+
+func TestResolveFFmpegEnv(t *testing.T) {
+	t.Setenv("SOW_FFMPEG", "/custom/ffmpeg")
+	path, err := ResolveFFmpeg()
+	if err != nil {
+		t.Fatalf("unexpected error: %v", err)
+	}
+	if path != "/custom/ffmpeg" {
+		t.Fatalf("expected '/custom/ffmpeg', got %q", path)
+	}
+}
+
+func TestResolveFFprobeEnv(t *testing.T) {
+	t.Setenv("SOW_FFPROBE", "/custom/ffprobe")
+	path, err := ResolveFFprobe()
+	if err != nil {
+		t.Fatalf("unexpected error: %v", err)
+	}
+	if path != "/custom/ffprobe" {
+		t.Fatalf("expected '/custom/ffprobe', got %q", path)
+	}
+}

internal/music/naming.go

@@ -0,0 +1,197 @@
+package music
+
+import (
+	"crypto/sha1"
+	"fmt"
+	"path/filepath"
+	"strings"
+)
+
+func GenerateStem(fileName, prefix string, used map[string]struct{}) (string, error) {
+	return GenerateStemWithMax(fileName, prefix, MaxStemLen, used)
+}
+
+func GenerateStemWithMax(fileName, prefix string, maxStemLen int, used map[string]struct{}) (string, error) {
+	if maxStemLen <= 0 {
+		maxStemLen = MaxStemLen
+	}
+	maxCore := maxStemLen - len(prefix)
+	if maxCore < 3 {
+		return "", fmt.Errorf("prefix too long: %q", prefix)
+	}
+	base := strings.TrimSuffix(fileName, filepath.Ext(fileName))
+	words := SlugWords(base)
+	core := MakeMusicCore(words, maxCore)
+	if core == "" {
+		sum := sha1.Sum([]byte(fileName))
+		core = fmt.Sprintf("%x", sum[:])[:maxCore]
+	}
+	stem := strings.TrimRight((prefix + core)[:Min(maxStemLen, len(prefix)+len(core))], "_")
+	if stem == "" {
+		return "", fmt.Errorf("could not derive music stem for %s", fileName)
+	}
+	return UniqueNameWithMax(stem, maxStemLen, used), nil
+}
+
+func SlugWords(value string) []string {
+	value = BracketRE.ReplaceAllString(value, " ")
+	value = strings.ReplaceAll(value, "&", " and ")
+	value = strings.ReplaceAll(value, "'", "")
+	value = strings.ReplaceAll(value, "\u2019", "")
+	value = strings.ToLower(value)
+	var ascii strings.Builder
+	for _, r := range value {
+		switch {
+		case r >= 'a' && r <= 'z':
+			ascii.WriteRune(r)
+		case r >= '0' && r <= '9':
+			ascii.WriteRune(r)
+		default:
+			ascii.WriteRune(' ')
+		}
+	}
+	rawWords := SplitRE.Split(ascii.String(), -1)
+	words := make([]string, 0, len(rawWords))
+	for _, word := range rawWords {
+		if word == "" {
+			continue
+		}
+		if _, drop := DropWords[word]; drop {
+			continue
+		}
+		if YearRE.MatchString(word) {
+			continue
+		}
+		words = append(words, word)
+	}
+	return words
+}
+
+func MakeMusicCore(words []string, maxCore int) string {
+	core := ""
+	for _, word := range words {
+		candidate := word
+		if core != "" {
+			candidate = core + "_" + word
+		}
+		if len(candidate) <= maxCore {
+			core = candidate
+			continue
+		}
+		break
+	}
+	if core != "" {
+		return core
+	}
+	abbrev := make([]string, 0, Min(4, len(words)))
+	for _, word := range words {
+		if len(abbrev) == 4 {
+			break
+		}
+		abbrev = append(abbrev, AbbreviateWord(word, 5))
+	}
+	for _, word := range abbrev {
+		candidate := word
+		if core != "" {
+			candidate = core + "_" + word
+		}
+		if len(candidate) <= maxCore {
+			core = candidate
+			continue
+		}
+		break
+	}
+	if core != "" {
+		return core
+	}
+	if len(words) > 0 {
+		return strings.Trim(words[0][:Min(maxCore, len(words[0]))], "_")
+	}
+	return ""
+}
+
+func AbbreviateWord(word string, maxLen int) string {
+	if len(word) <= maxLen {
+		return word
+	}
+	consonants := VowelRE.ReplaceAllString(word, "")
+	candidate := word[:1]
+	if len(consonants) > 1 {
+		candidate += consonants[1:]
+	}
+	if len(candidate) >= 3 {
+		return candidate[:Min(maxLen, len(candidate))]
+	}
+	return word[:Min(maxLen, len(word))]
+}
+
+func UniqueName(stem string, used map[string]struct{}) string {
+	return UniqueNameWithMax(stem, MaxStemLen, used)
+}
+
+func UniqueNameWithMax(stem string, maxStemLen int, used map[string]struct{}) string {
+	if maxStemLen <= 0 {
+		maxStemLen = MaxStemLen
+	}
+	if _, exists := used[stem]; !exists {
+		used[stem] = struct{}{}
+		return stem
+	}
+	for index := 1; ; index++ {
+		suffix := fmt.Sprintf("_%d", index)
+		prefixLen := Min(len(stem), Max(0, maxStemLen-len(suffix)))
+		candidate := strings.TrimRight(stem[:prefixLen], "_") + suffix
+		if _, exists := used[candidate]; exists {
+			continue
+		}
+		used[candidate] = struct{}{}
+		return candidate
+	}
+}
+
+func ReserveName(stem string, used map[string]struct{}) error {
+	stem = strings.ToLower(strings.TrimSpace(stem))
+	if stem == "" {
+		return fmt.Errorf("output filename is empty")
+	}
+	if _, exists := used[stem]; exists {
+		return fmt.Errorf("output filename %s collides with an existing asset", stem)
+	}
+	used[stem] = struct{}{}
+	return nil
+}
+
+func ValidateManualOutputFile(name string) error {
+	return ValidateManualOutputFileWithRules(name, ".bmu", MaxStemLen)
+}
+
+func ValidateManualOutputFileWithRules(name, extension string, maxStemLen int) error {
+	name = strings.ToLower(strings.TrimSpace(name))
+	extension = strings.ToLower(strings.TrimSpace(extension))
+	if extension == "" {
+		extension = ".bmu"
+	}
+	if maxStemLen <= 0 {
+		maxStemLen = MaxStemLen
+	}
+	if !strings.HasSuffix(name, extension) {
+		return fmt.Errorf("output file must end with %s", extension)
+	}
+	stem := strings.TrimSuffix(name, extension)
+	if stem == "" {
+		return fmt.Errorf("output file stem is empty")
+	}
+	if len(stem) > maxStemLen {
+		return fmt.Errorf("output file stem %q exceeds %d characters", stem, maxStemLen)
+	}
+	for _, r := range stem {
+		switch {
+		case r >= 'a' && r <= 'z':
+		case r >= '0' && r <= '9':
+		case r == '_':
+		default:
+			return fmt.Errorf("output file stem %q contains unsupported character %q", stem, string(r))
+		}
+	}
+	return nil
+}

internal/pipeline/apply_manifest.go

@@ -0,0 +1,70 @@
+package pipeline
+
+import (
+	"encoding/json"
+	"fmt"
+	"os"
+	"path/filepath"
+	"strings"
+
+	"git.westgate.pw/ShadowsOverWestgate/sow-tools/internal/gff"
+	"git.westgate.pw/ShadowsOverWestgate/sow-tools/internal/project"
+)
+
+type ApplyManifestResult struct {
+	ManifestPath string
+	ModuleSource string
+	HAKCount     int
+}
+
+func ApplyHAKManifest(p *project.Project, manifestPath string) (ApplyManifestResult, error) {
+	if manifestPath == "" {
+		manifestPath = p.HAKManifestPath()
+	}
+	if strings.TrimSpace(p.EffectiveConfig().Paths.Source) == "" {
+		return ApplyManifestResult{}, fmt.Errorf("cannot apply hak manifest: paths.source is not configured")
+	}
+	sourceRoot := filepath.Clean(p.SourceDir())
+	if sourceRoot == "." || sourceRoot == string(filepath.Separator) || sourceRoot == filepath.Clean(p.Root) {
+		return ApplyManifestResult{}, fmt.Errorf("cannot apply hak manifest: paths.source resolves to unsafe source root %s", sourceRoot)
+	}
+
+	raw, err := os.ReadFile(manifestPath)
+	if err != nil {
+		return ApplyManifestResult{}, fmt.Errorf("read hak manifest: %w", err)
+	}
+
+	var manifest BuildManifest
+	if err := json.Unmarshal(raw, &manifest); err != nil {
+		return ApplyManifestResult{}, fmt.Errorf("parse hak manifest: %w", err)
+	}
+
+	moduleSource := filepath.Join(sourceRoot, "module", "module.ifo.json")
+	sourceRaw, err := os.ReadFile(moduleSource)
+	if err != nil {
+		return ApplyManifestResult{}, fmt.Errorf("read module ifo source: %w", err)
+	}
+
+	var document gff.Document
+	if err := json.Unmarshal(sourceRaw, &document); err != nil {
+		return ApplyManifestResult{}, fmt.Errorf("parse module ifo source: %w", err)
+	}
+
+	setModuleHAKList(&document, manifest.ModuleHAKs)
+
+	formatted, err := json.MarshalIndent(document, "", "  ")
+	if err != nil {
+		return ApplyManifestResult{}, fmt.Errorf("marshal updated module ifo: %w", err)
+	}
+	formatted = append(formatted, '\n')
+
+	if err := os.WriteFile(moduleSource, formatted, 0o644); err != nil {
+		return ApplyManifestResult{}, fmt.Errorf("write module ifo source: %w", err)
+	}
+
+	return ApplyManifestResult{
+		ManifestPath: manifestPath,
+		ModuleSource: moduleSource,
+		HAKCount:     len(manifest.ModuleHAKs),
+	}, nil
+}

internal/pipeline/chunk_validation.go

@@ -0,0 +1,27 @@
+package pipeline
+
+import (
+	"fmt"
+
+	"git.westgate.pw/ShadowsOverWestgate/sow-tools/internal/erf"
+)
+
+func ensureUniqueChunkResources(chunk hakChunk) error {
+	seen := map[string]string{}
+	for _, asset := range chunk.Assets {
+		key := fmt.Sprintf("%s:%04x", asset.Resource.Name, asset.Resource.Type)
+		if previous, exists := seen[key]; exists {
+			return fmt.Errorf("resource %s from %s conflicts with %s inside generated hak %s", chunkResourceLabel(asset), asset.Rel, previous, chunk.Name)
+		}
+		seen[key] = asset.Rel
+	}
+	return nil
+}
+
+func chunkResourceLabel(asset assetResource) string {
+	extension, ok := erf.ExtensionForResourceType(asset.Resource.Type)
+	if !ok {
+		return asset.Resource.Name
+	}
+	return asset.Resource.Name + "." + extension
+}

internal/pipeline/compare.go

@@ -0,0 +1,359 @@
+package pipeline
+
+import (
+	"bytes"
+	"encoding/json"
+	"errors"
+	"fmt"
+	"io/fs"
+	"os"
+	"path/filepath"
+	"slices"
+	"strings"
+	"time"
+
+	"git.westgate.pw/ShadowsOverWestgate/sow-tools/internal/erf"
+	"git.westgate.pw/ShadowsOverWestgate/sow-tools/internal/gff"
+	"git.westgate.pw/ShadowsOverWestgate/sow-tools/internal/project"
+)
+
+type CompareResult struct {
+	ModulePath string
+	HAKPaths   []string
+	Checked    int
+}
+
+type resourceExpectation struct {
+	Key   string
+	Kind  string
+	Bytes []byte
+}
+
+func Compare(p *project.Project) (CompareResult, error) {
+	modulePath := p.ModuleArchivePath()
+
+	var hakPaths []string
+	if len(p.Inventory.AssetFiles) > 0 {
+		var err error
+		hakPaths, err = manifestHAKPaths(p)
+		if err != nil {
+			return CompareResult{}, err
+		}
+	}
+	if err := ensureBuildIsCurrent(p, modulePath, hakPaths); err != nil {
+		return CompareResult{ModulePath: modulePath, HAKPaths: hakPaths}, err
+	}
+
+	moduleArchive, err := readArchive(modulePath)
+	if err != nil {
+		return CompareResult{}, err
+	}
+
+	expected, err := expectedResources(p)
+	if err != nil {
+		return CompareResult{}, err
+	}
+
+	actual := archiveIndex(moduleArchive)
+	for _, hakPath := range hakPaths {
+		hakArchive, err := readArchive(hakPath)
+		if err != nil {
+			return CompareResult{}, err
+		}
+		for key, value := range archiveIndex(hakArchive) {
+			actual[key] = value
+		}
+	}
+
+	var diagnostics []error
+	checked := 0
+
+	expectedKeys := make([]string, 0, len(expected))
+	for key := range expected {
+		expectedKeys = append(expectedKeys, key)
+	}
+	slices.Sort(expectedKeys)
+
+	for _, key := range expectedKeys {
+		want := expected[key]
+		got, ok := actual[key]
+		if !ok {
+			diagnostics = append(diagnostics, fmt.Errorf("missing resource in built archives: %s", key))
+			continue
+		}
+		if !bytes.Equal(want.Bytes, got.Data) {
+			diagnostics = append(diagnostics, fmt.Errorf("resource content mismatch: %s", key))
+			continue
+		}
+		checked++
+		delete(actual, key)
+	}
+
+	if len(actual) > 0 {
+		extraKeys := make([]string, 0, len(actual))
+		for key := range actual {
+			extraKeys = append(extraKeys, key)
+		}
+		slices.Sort(extraKeys)
+		for _, key := range extraKeys {
+			diagnostics = append(diagnostics, fmt.Errorf("unexpected extra resource in built archives: %s", key))
+		}
+	}
+
+	result := CompareResult{
+		ModulePath: modulePath,
+		HAKPaths:   hakPaths,
+		Checked:    checked,
+	}
+	if len(diagnostics) > 0 {
+		return result, errors.Join(diagnostics...)
+	}
+	return result, nil
+}
+
+func expectedResources(p *project.Project) (map[string]resourceExpectation, error) {
+	out := map[string]resourceExpectation{}
+	moduleHakOrder, err := plannedModuleHAKOrder(p)
+	if err != nil {
+		return nil, err
+	}
+
+	for _, rel := range p.Inventory.SourceFiles {
+		abs := filepath.Join(p.SourceDir(), filepath.FromSlash(rel))
+		name, extension, err := splitSourceName(abs)
+		if err != nil {
+			return nil, err
+		}
+
+		raw, err := os.ReadFile(abs)
+		if err != nil {
+			return nil, fmt.Errorf("read %s: %w", abs, err)
+		}
+		var document gff.Document
+		if err := json.Unmarshal(raw, &document); err != nil {
+			return nil, fmt.Errorf("parse gff json %s: %w", abs, err)
+		}
+		if extension == ".ifo" && strings.EqualFold(name, "module") && len(moduleHakOrder) > 0 {
+			setModuleHAKList(&document, moduleHakOrder)
+		}
+
+		canonical, err := json.Marshal(document)
+		if err != nil {
+			return nil, fmt.Errorf("marshal canonical gff json %s: %w", abs, err)
+		}
+
+		out[resourceKey(name, extension)] = resourceExpectation{
+			Key:   resourceKey(name, extension),
+			Kind:  "gff-json",
+			Bytes: canonical,
+		}
+	}
+
+	for _, rel := range p.Inventory.ScriptFiles {
+		abs := filepath.Join(p.SourceDir(), filepath.FromSlash(rel))
+		data, err := os.ReadFile(abs)
+		if err != nil {
+			return nil, fmt.Errorf("read %s: %w", abs, err)
+		}
+		name := strings.ToLower(strings.TrimSuffix(filepath.Base(abs), filepath.Ext(abs)))
+		extension := strings.TrimPrefix(strings.ToLower(filepath.Ext(abs)), ".")
+		out[resourceKey(name, extension)] = resourceExpectation{
+			Key:   resourceKey(name, extension),
+			Kind:  "raw",
+			Bytes: data,
+		}
+	}
+
+	for _, rel := range p.Inventory.AssetFiles {
+		abs := filepath.Join(p.AssetsDir(), filepath.FromSlash(rel))
+		data, err := os.ReadFile(abs)
+		if err != nil {
+			return nil, fmt.Errorf("read %s: %w", abs, err)
+		}
+		name := strings.ToLower(strings.TrimSuffix(filepath.Base(abs), filepath.Ext(abs)))
+		extension := strings.TrimPrefix(strings.ToLower(filepath.Ext(abs)), ".")
+		out[resourceKey(name, extension)] = resourceExpectation{
+			Key:   resourceKey(name, extension),
+			Kind:  "raw",
+			Bytes: data,
+		}
+	}
+
+	return out, nil
+}
+
+func ensureBuildIsCurrent(p *project.Project, modulePath string, hakPaths []string) error {
+	archivePaths := make([]string, 0, 1+len(hakPaths))
+	archivePaths = append(archivePaths, modulePath)
+	archivePaths = append(archivePaths, hakPaths...)
+
+	oldestArchive := time.Time{}
+	for _, path := range archivePaths {
+		info, err := os.Stat(path)
+		if err != nil {
+			return fmt.Errorf("stat archive %s: %w", path, err)
+		}
+		if oldestArchive.IsZero() || info.ModTime().Before(oldestArchive) {
+			oldestArchive = info.ModTime()
+		}
+	}
+
+	sourcePaths := make([]string, 0, len(p.Inventory.SourceFiles)+len(p.Inventory.ScriptFiles)+len(p.Inventory.AssetFiles))
+	for _, rel := range p.Inventory.SourceFiles {
+		sourcePaths = append(sourcePaths, filepath.Join(p.SourceDir(), filepath.FromSlash(rel)))
+	}
+	for _, rel := range p.Inventory.ScriptFiles {
+		sourcePaths = append(sourcePaths, filepath.Join(p.SourceDir(), filepath.FromSlash(rel)))
+	}
+	for _, rel := range p.Inventory.AssetFiles {
+		sourcePaths = append(sourcePaths, filepath.Join(p.AssetsDir(), filepath.FromSlash(rel)))
+	}
+
+	newestSource := time.Time{}
+	newestPath := ""
+	for _, path := range sourcePaths {
+		info, err := os.Stat(path)
+		if err != nil {
+			return fmt.Errorf("stat source %s: %w", path, err)
+		}
+		if newestSource.IsZero() || info.ModTime().After(newestSource) {
+			newestSource = info.ModTime()
+			newestPath = path
+		}
+	}
+
+	if !newestSource.IsZero() && newestSource.After(oldestArchive) {
+		return fmt.Errorf("built archives are older than source file %s; run build-module or build-haks before compare", newestPath)
+	}
+	return nil
+}
+
+func readArchive(path string) (erf.Archive, error) {
+	input, err := os.Open(path)
+	if err != nil {
+		return erf.Archive{}, fmt.Errorf("open archive %s: %w", path, err)
+	}
+	defer input.Close()
+
+	archive, err := erf.Read(input)
+	if err != nil {
+		return erf.Archive{}, fmt.Errorf("read archive %s: %w", path, err)
+	}
+	return archive, nil
+}
+
+func archiveIndex(archive erf.Archive) map[string]erf.Resource {
+	out := map[string]erf.Resource{}
+	for _, resource := range archive.Resources {
+		extension, ok := erf.ExtensionForResourceType(resource.Type)
+		if !ok {
+			continue
+		}
+
+		key := resourceKey(strings.ToLower(resource.Name), extension)
+		switch extension {
+		case "utc", "utd", "ute", "uti", "utm", "utp", "uts", "utt", "utw",
+			"are", "dlg", "fac", "gic", "git", "ifo", "itp", "jrl":
+			document, err := gff.Read(bytes.NewReader(resource.Data))
+			if err != nil {
+				out[key] = resource
+				continue
+			}
+			canonical, err := json.Marshal(document)
+			if err != nil {
+				out[key] = resource
+				continue
+			}
+			resource.Data = canonical
+		}
+
+		out[key] = resource
+	}
+	return out
+}
+
+func resourceKey(name, extension string) string {
+	return strings.ToLower(name) + "." + strings.TrimPrefix(strings.ToLower(extension), ".")
+}
+
+func manifestHAKPaths(p *project.Project) ([]string, error) {
+	manifestPath := p.HAKManifestPath()
+	raw, err := os.ReadFile(manifestPath)
+	if err == nil {
+		var manifest BuildManifest
+		if err := json.Unmarshal(raw, &manifest); err != nil {
+			return nil, fmt.Errorf("parse hak manifest: %w", err)
+		}
+		paths := make([]string, 0, len(manifest.HAKs))
+		for _, hak := range manifest.HAKs {
+			paths = append(paths, p.HAKArchivePath(hak.Name))
+		}
+		return paths, nil
+	}
+	if !errors.Is(err, os.ErrNotExist) {
+		return nil, fmt.Errorf("read hak manifest: %w", err)
+	}
+
+	legacy := filepath.Join(p.BuildDir(), "*.hak")
+	paths, err := filepath.Glob(legacy)
+	if err != nil {
+		return nil, fmt.Errorf("scan hak archives: %w", err)
+	}
+	filtered := make([]string, 0, len(paths))
+	for _, path := range paths {
+		if filepath.Base(path) == p.TopDataPackageHAKName() {
+			continue
+		}
+		filtered = append(filtered, path)
+	}
+	slices.Sort(filtered)
+	return filtered, nil
+}
+
+func topPackageSourcePaths(p *project.Project) ([]string, error) {
+	sourceDir := p.TopDataSourceDir()
+	generated2DADir := filepath.Join(sourceDir, "assets", "2da")
+	paths := make([]string, 0)
+	for _, candidate := range []string{
+		filepath.Join(sourceDir, ".tlk_state.json"),
+		filepath.Join(sourceDir, "base_dialog.json"),
+	} {
+		if _, err := os.Stat(candidate); err == nil {
+			paths = append(paths, candidate)
+		} else if err != nil && !errors.Is(err, os.ErrNotExist) {
+			return nil, fmt.Errorf("stat topdata source %s: %w", candidate, err)
+		}
+	}
+	for _, dir := range []string{
+		filepath.Join(sourceDir, "data"),
+		filepath.Join(sourceDir, "assets"),
+	} {
+		info, err := os.Stat(dir)
+		if err != nil {
+			if errors.Is(err, os.ErrNotExist) {
+				continue
+			}
+			return nil, fmt.Errorf("stat topdata path %s: %w", dir, err)
+		}
+		if !info.IsDir() {
+			continue
+		}
+		err = filepath.WalkDir(dir, func(path string, d fs.DirEntry, walkErr error) error {
+			if walkErr != nil {
+				return walkErr
+			}
+			if d.IsDir() && path == generated2DADir {
+				return filepath.SkipDir
+			}
+			if d.IsDir() {
+				return nil
+			}
+			paths = append(paths, path)
+			return nil
+		})
+		if err != nil {
+			return nil, fmt.Errorf("scan topdata path %s: %w", dir, err)
+		}
+	}
+	return paths, nil
+}

internal/pipeline/extract.go

@@ -0,0 +1,621 @@
+package pipeline
+
+import (
+	"bytes"
+	"encoding/json"
+	"errors"
+	"fmt"
+	"os"
+	"path/filepath"
+	"slices"
+	"strings"
+
+	"git.westgate.pw/ShadowsOverWestgate/sow-tools/internal/erf"
+	"git.westgate.pw/ShadowsOverWestgate/sow-tools/internal/gff"
+	"git.westgate.pw/ShadowsOverWestgate/sow-tools/internal/project"
+)
+
+type ExtractResult struct {
+	ModulePath          string
+	HAKPaths            []string
+	DeletedArchivePaths []string
+	Written             int
+	Overwritten         int
+	Removed             int
+	Skipped             int
+}
+
+type extractionArchive struct {
+	Path      string
+	Rel       string
+	Extension string
+}
+
+type extractionScope struct {
+	Source bool
+	Assets bool
+}
+
+func Extract(p *project.Project, files ...string) (ExtractResult, error) {
+	var result ExtractResult
+	var failures []error
+	desired := map[string]struct{}{}
+	scope := extractionScope{}
+
+	archives, err := resolveExtractionArchives(p, files)
+	if err != nil {
+		return result, err
+	}
+	for _, archivePath := range archives {
+		input, err := os.Open(archivePath.Path)
+		if err != nil {
+			failures = append(failures, fmt.Errorf("open archive %s: %w", archivePath.Path, err))
+			continue
+		}
+		archive, err := erf.Read(input)
+		input.Close()
+		if err != nil {
+			failures = append(failures, fmt.Errorf("read archive %s: %w", archivePath.Path, err))
+			continue
+		}
+
+		switch archivePath.Extension {
+		case ".mod":
+			if result.ModulePath == "" {
+				result.ModulePath = archivePath.Path
+			}
+		case ".hak":
+			result.HAKPaths = append(result.HAKPaths, archivePath.Path)
+		}
+
+		written, overwritten, skipped, extractedScope, errs := extractArchiveResources(p, archive, desired)
+		result.Written += written
+		result.Overwritten += overwritten
+		result.Skipped += skipped
+		scope.Source = scope.Source || extractedScope.Source
+		scope.Assets = scope.Assets || extractedScope.Assets
+		failures = append(failures, errs...)
+	}
+
+	if p.EffectiveConfig().Extract.CleanupStale == nil || *p.EffectiveConfig().Extract.CleanupStale {
+		removed, errs := cleanupStaleFiles(p, desired, scope)
+		result.Removed = removed
+		failures = append(failures, errs...)
+	}
+
+	if len(failures) > 0 {
+		return result, errors.Join(failures...)
+	}
+	consumeAll, consumeModules := extractionConsumePolicy(p)
+	if consumeAll || consumeModules {
+		for _, archivePath := range archives {
+			if !consumeAll && archivePath.Extension != ".mod" {
+				continue
+			}
+			if err := os.Remove(archivePath.Path); err != nil {
+				return result, fmt.Errorf("delete consumed archive %s: %w", archivePath.Path, err)
+			}
+			result.DeletedArchivePaths = append(result.DeletedArchivePaths, archivePath.Path)
+		}
+	}
+	return result, nil
+}
+
+func extractArchiveResources(p *project.Project, archive erf.Archive, desired map[string]struct{}) (int, int, int, extractionScope, []error) {
+	var failures []error
+	writtenCount := 0
+	overwrittenCount := 0
+	skippedCount := 0
+	scope := extractionScope{}
+
+	ignored := make(map[string]bool)
+	for _, ext := range p.Config.Extract.IgnoreExtensions {
+		ext := strings.TrimPrefix(ext, ".")
+		ignored[ext] = true
+	}
+
+	for _, resource := range archive.Resources {
+		ext, ok := erf.ExtensionForResourceType(resource.Type)
+		if !ok {
+			failures = append(failures, fmt.Errorf("unsupported resource type 0x%04X for %s", resource.Type, resource.Name))
+			continue
+		}
+		if ignored[ext] {
+			skippedCount++
+			continue
+		}
+
+		target, data, err := extractedFile(p, resource, ext)
+		if err != nil {
+			failures = append(failures, err)
+			continue
+		}
+		desired[target] = struct{}{}
+		scope.Source = scope.Source || pathWithinRoot(target, p.SourceDir())
+		scope.Assets = scope.Assets || pathWithinRoot(target, p.AssetsDir())
+
+		state, err := writeManagedFile(target, data)
+		if err != nil {
+			failures = append(failures, err)
+			continue
+		}
+		switch state {
+		case writeNew:
+			writtenCount++
+		case writeOverwritten:
+			overwrittenCount++
+		case writeSkipped:
+			skippedCount++
+		}
+	}
+
+	return writtenCount, overwrittenCount, skippedCount, scope, failures
+}
+
+func resolveExtractionArchives(p *project.Project, overrides []string) ([]extractionArchive, error) {
+	patterns := p.EffectiveConfig().Extract.Archives
+	if len(overrides) > 0 {
+		patterns = overrides
+	}
+	if len(patterns) == 0 {
+		return nil, fmt.Errorf("extract.archives must contain at least one build-relative archive pattern")
+	}
+
+	buildRoot := filepath.Clean(p.BuildDir())
+	byPath := map[string]extractionArchive{}
+	for _, rawPattern := range patterns {
+		pattern, err := normalizeExtractionArchivePattern(p, rawPattern)
+		if err != nil {
+			return nil, err
+		}
+		matched, err := matchExtractionArchives(buildRoot, pattern)
+		if err != nil {
+			return nil, err
+		}
+		if len(matched) == 0 {
+			return nil, fmt.Errorf("extract archive pattern %q matched no files under %s", rawPattern, buildRoot)
+		}
+		for _, archive := range matched {
+			byPath[archive.Path] = archive
+		}
+	}
+
+	archives := make([]extractionArchive, 0, len(byPath))
+	for _, archive := range byPath {
+		archives = append(archives, archive)
+	}
+	slices.SortFunc(archives, func(a, b extractionArchive) int {
+		return strings.Compare(a.Rel, b.Rel)
+	})
+	return archives, nil
+}
+
+func normalizeExtractionArchivePattern(p *project.Project, raw string) (string, error) {
+	pattern := strings.TrimSpace(raw)
+	if pattern == "" {
+		return "", fmt.Errorf("extract archive pattern must not be empty")
+	}
+	pattern = strings.NewReplacer(
+		"{module.resref}", strings.TrimSpace(p.Config.Module.ResRef),
+	).Replace(pattern)
+	if strings.Contains(pattern, "\x00") {
+		return "", fmt.Errorf("extract archive pattern %q must not contain NUL bytes", raw)
+	}
+	if filepath.IsAbs(pattern) {
+		return "", fmt.Errorf("extract archive pattern %q must be relative to paths.build", raw)
+	}
+	clean := filepath.Clean(filepath.FromSlash(pattern))
+	if clean == "." || clean == ".." || strings.HasPrefix(clean, ".."+string(filepath.Separator)) {
+		return "", fmt.Errorf("extract archive pattern %q must not escape paths.build", raw)
+	}
+	return filepath.ToSlash(clean), nil
+}
+
+func matchExtractionArchives(buildRoot, pattern string) ([]extractionArchive, error) {
+	var archives []extractionArchive
+	err := filepath.WalkDir(buildRoot, func(path string, entry os.DirEntry, err error) error {
+		if err != nil {
+			return err
+		}
+		if entry.IsDir() {
+			return nil
+		}
+		rel, err := filepath.Rel(buildRoot, path)
+		if err != nil {
+			return err
+		}
+		rel = filepath.ToSlash(rel)
+		if !matchPathPattern(rel, pattern) {
+			return nil
+		}
+		archive, err := validateExtractionArchive(buildRoot, rel)
+		if err != nil {
+			return err
+		}
+		archives = append(archives, archive)
+		return nil
+	})
+	if err != nil {
+		if errors.Is(err, os.ErrNotExist) {
+			return nil, fmt.Errorf("scan build directory %s: %w", buildRoot, err)
+		}
+		return nil, err
+	}
+	return archives, nil
+}
+
+func validateExtractionArchive(buildRoot, rel string) (extractionArchive, error) {
+	cleanRel := filepath.Clean(filepath.FromSlash(rel))
+	if cleanRel == "." || cleanRel == ".." || strings.HasPrefix(cleanRel, ".."+string(filepath.Separator)) {
+		return extractionArchive{}, fmt.Errorf("extract archive %q must not escape paths.build", rel)
+	}
+	path := filepath.Join(buildRoot, cleanRel)
+	if !pathWithinRoot(path, buildRoot) {
+		return extractionArchive{}, fmt.Errorf("extract archive %q resolves outside paths.build", rel)
+	}
+	extension := strings.ToLower(filepath.Ext(cleanRel))
+	switch extension {
+	case ".mod", ".hak":
+		return extractionArchive{Path: path, Rel: filepath.ToSlash(cleanRel), Extension: extension}, nil
+	case ".erf":
+		return extractionArchive{}, fmt.Errorf("extract archive %q is an .erf; ERF extraction is unsafe because ERFs lack module.ifo context", rel)
+	default:
+		return extractionArchive{}, fmt.Errorf("extract archive %q uses unsupported extension %q", rel, extension)
+	}
+}
+
+func extractionConsumePolicy(p *project.Project) (consumeAll bool, consumeModules bool) {
+	if configured := p.Config.Extract.ConsumeArchives; configured != nil {
+		return *configured, false
+	}
+	return false, p.Config.Extract.DeleteModuleArchiveAfterSuccess
+}
+
+func pathWithinRoot(path, root string) bool {
+	root = filepath.Clean(root)
+	if root == "" || root == "." {
+		return false
+	}
+	rel, err := filepath.Rel(root, path)
+	if err != nil {
+		return false
+	}
+	return rel != ".." && !strings.HasPrefix(rel, ".."+string(filepath.Separator)) && !filepath.IsAbs(rel)
+}
+
+func extractedFile(p *project.Project, resource erf.Resource, extension string) (string, []byte, error) {
+	resref := strings.ToLower(resource.Name)
+	effective := p.EffectiveConfig()
+
+	switch extension {
+	case "nss":
+		target, err := extractionTarget(p, "paths.source", effective.Paths.Source, p.SourceDir(), "scripts", resref+".nss")
+		if err != nil {
+			return "", nil, err
+		}
+		return target, resource.Data, nil
+	case "utc", "utd", "ute", "uti", "utm", "utp", "uts", "utt", "utw",
+		"are", "dlg", "fac", "gic", "git", "ifo", "itp", "jrl":
+		document, err := gff.Read(bytes.NewReader(resource.Data))
+		if err != nil {
+			return "", nil, fmt.Errorf("decode gff %s.%s: %w", resource.Name, extension, err)
+		}
+		target, err := extractionTarget(p, "paths.source", effective.Paths.Source, p.SourceDir(), sourceSubdir(extension), resref+"."+extension+".json")
+		if err != nil {
+			return "", nil, err
+		}
+		if err := mergeExtractedGFFJSON(p, target, &document); err != nil {
+			return "", nil, fmt.Errorf("merge extracted gff json %s.%s: %w", resource.Name, extension, err)
+		}
+		formatted, err := json.MarshalIndent(document, "", "  ")
+		if err != nil {
+			return "", nil, fmt.Errorf("marshal json %s.%s: %w", resource.Name, extension, err)
+		}
+		formatted = append(formatted, '\n')
+		return target, formatted, nil
+	default:
+		target, err := extractionTarget(p, "paths.assets", effective.Paths.Assets, p.AssetsDir(), extension, resref+"."+extension)
+		if err != nil {
+			return "", nil, err
+		}
+		return target, resource.Data, nil
+	}
+}
+
+func extractionTarget(p *project.Project, field, configured, root string, parts ...string) (string, error) {
+	if strings.TrimSpace(configured) == "" {
+		return "", fmt.Errorf("cannot extract resource: %s is not configured", field)
+	}
+
+	cleanRoot := filepath.Clean(root)
+	if cleanRoot == "." || cleanRoot == string(filepath.Separator) || cleanRoot == filepath.Clean(p.Root) {
+		return "", fmt.Errorf("cannot extract resource: %s resolves to unsafe extraction root %s", field, cleanRoot)
+	}
+
+	target := filepath.Join(append([]string{cleanRoot}, parts...)...)
+	rel, err := filepath.Rel(cleanRoot, target)
+	if err != nil {
+		return "", fmt.Errorf("resolve extraction target %s: %w", target, err)
+	}
+	if rel == ".." || strings.HasPrefix(rel, ".."+string(filepath.Separator)) || filepath.IsAbs(rel) {
+		return "", fmt.Errorf("refusing to extract resource outside %s: %s", cleanRoot, target)
+	}
+	return target, nil
+}
+
+func mergeExtractedGFFJSON(p *project.Project, target string, extracted *gff.Document) error {
+	rule, ok, err := extractGFFJSONMergeRule(p, target)
+	if err != nil || !ok {
+		return err
+	}
+
+	raw, err := os.ReadFile(target)
+	if err != nil {
+		if errors.Is(err, os.ErrNotExist) {
+			return nil
+		}
+		return fmt.Errorf("read existing source %s: %w", target, err)
+	}
+	var existing gff.Document
+	if err := json.Unmarshal(raw, &existing); err != nil {
+		return fmt.Errorf("parse existing source %s: %w", target, err)
+	}
+
+	for _, label := range rule.PreserveFields {
+		if field, ok := gffField(existing.Root, label); ok {
+			setGFFField(&extracted.Root, field)
+		}
+	}
+	for _, listRule := range rule.MergeLists {
+		if err := mergeGFFListByKey(&extracted.Root, existing.Root, listRule); err != nil {
+			return err
+		}
+	}
+	return nil
+}
+
+func extractGFFJSONMergeRule(p *project.Project, target string) (project.ExtractGFFJSONMergeRule, bool, error) {
+	rel, err := filepath.Rel(filepath.Clean(p.SourceDir()), filepath.Clean(target))
+	if err != nil {
+		return project.ExtractGFFJSONMergeRule{}, false, fmt.Errorf("resolve source-relative target %s: %w", target, err)
+	}
+	rel = filepath.ToSlash(rel)
+	if rel == ".." || strings.HasPrefix(rel, "../") {
+		return project.ExtractGFFJSONMergeRule{}, false, nil
+	}
+	for _, rule := range p.EffectiveConfig().Extract.Merge.GFFJSON {
+		if rule.Target == rel {
+			return rule, true, nil
+		}
+	}
+	return project.ExtractGFFJSONMergeRule{}, false, nil
+}
+
+func gffField(s gff.Struct, label string) (gff.Field, bool) {
+	for _, field := range s.Fields {
+		if field.Label == label {
+			return field, true
+		}
+	}
+	return gff.Field{}, false
+}
+
+func setGFFField(s *gff.Struct, replacement gff.Field) {
+	for index, field := range s.Fields {
+		if field.Label == replacement.Label {
+			s.Fields[index] = replacement
+			return
+		}
+	}
+	s.Fields = append(s.Fields, replacement)
+}
+
+func mergeGFFListByKey(extracted *gff.Struct, existing gff.Struct, rule project.ExtractListMergeRule) error {
+	extractedField, ok := gffField(*extracted, rule.Field)
+	if !ok {
+		return fmt.Errorf("extracted field %q not found", rule.Field)
+	}
+	extractedList, ok := extractedField.Value.(gff.ListValue)
+	if !ok {
+		return fmt.Errorf("extracted field %q is %s, not List", rule.Field, extractedField.Type)
+	}
+	existingField, ok := gffField(existing, rule.Field)
+	if !ok {
+		return nil
+	}
+	existingList, ok := existingField.Value.(gff.ListValue)
+	if !ok {
+		return fmt.Errorf("existing field %q is %s, not List", rule.Field, existingField.Type)
+	}
+
+	extractedByKey := map[string]gff.Struct{}
+	extractedOrder := make([]string, 0, len(extractedList))
+	for _, item := range extractedList {
+		key, err := gffStructKey(item, rule.KeyField)
+		if err != nil {
+			return fmt.Errorf("extracted field %q: %w", rule.Field, err)
+		}
+		if _, exists := extractedByKey[key]; exists {
+			return fmt.Errorf("extracted field %q has duplicate %s key %q", rule.Field, rule.KeyField, key)
+		}
+		extractedByKey[key] = item
+		extractedOrder = append(extractedOrder, key)
+	}
+
+	merged := make(gff.ListValue, 0, len(extractedList))
+	seen := map[string]struct{}{}
+	for _, item := range existingList {
+		key, err := gffStructKey(item, rule.KeyField)
+		if err != nil {
+			return fmt.Errorf("existing field %q: %w", rule.Field, err)
+		}
+		if _, duplicate := seen[key]; duplicate {
+			return fmt.Errorf("existing field %q has duplicate %s key %q", rule.Field, rule.KeyField, key)
+		}
+		if extractedItem, exists := extractedByKey[key]; exists {
+			merged = append(merged, extractedItem)
+			seen[key] = struct{}{}
+		}
+	}
+	for _, key := range extractedOrder {
+		if _, exists := seen[key]; exists {
+			continue
+		}
+		merged = append(merged, extractedByKey[key])
+	}
+
+	setGFFField(extracted, gff.NewField(rule.Field, merged))
+	return nil
+}
+
+func gffStructKey(s gff.Struct, keyField string) (string, error) {
+	field, ok := gffField(s, keyField)
+	if !ok {
+		return "", fmt.Errorf("key field %q not found", keyField)
+	}
+	switch value := field.Value.(type) {
+	case gff.ResRefValue:
+		return string(value), nil
+	case gff.StringValue:
+		return string(value), nil
+	default:
+		return "", fmt.Errorf("key field %q is %s, not ResRef or CExoString", keyField, field.Type)
+	}
+}
+
+type writeState int
+
+const (
+	writeSkipped writeState = iota
+	writeNew
+	writeOverwritten
+)
+
+func writeManagedFile(path string, data []byte) (writeState, error) {
+	if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil {
+		return writeSkipped, fmt.Errorf("create parent directory for %s: %w", path, err)
+	}
+
+	existing, err := os.ReadFile(path)
+	if err == nil {
+		if bytes.Equal(existing, data) {
+			return writeSkipped, nil
+		}
+		if err := os.WriteFile(path, data, 0o644); err != nil {
+			return writeSkipped, fmt.Errorf("overwrite %s: %w", path, err)
+		}
+		return writeOverwritten, nil
+	}
+	if !errors.Is(err, os.ErrNotExist) {
+		return writeSkipped, fmt.Errorf("check existing file %s: %w", path, err)
+	}
+
+	if err := os.WriteFile(path, data, 0o644); err != nil {
+		return writeSkipped, fmt.Errorf("write %s: %w", path, err)
+	}
+	return writeNew, nil
+}
+
+func cleanupStaleFiles(p *project.Project, desired map[string]struct{}, scope extractionScope) (int, []error) {
+	candidates := make([]string, 0, len(p.Inventory.SourceFiles)+len(p.Inventory.ScriptFiles)+len(p.Inventory.AssetFiles))
+	if scope.Source && safeCleanupRoot(p.SourceDir(), p.Root) {
+		for _, rel := range p.Inventory.SourceFiles {
+			candidates = append(candidates, filepath.Join(p.SourceDir(), filepath.FromSlash(rel)))
+		}
+		for _, rel := range p.Inventory.ScriptFiles {
+			candidates = append(candidates, filepath.Join(p.SourceDir(), filepath.FromSlash(rel)))
+		}
+	}
+	if scope.Assets && safeCleanupRoot(p.AssetsDir(), p.Root) {
+		for _, rel := range p.Inventory.AssetFiles {
+			candidates = append(candidates, filepath.Join(p.AssetsDir(), filepath.FromSlash(rel)))
+		}
+	}
+
+	removed := 0
+	var failures []error
+	for _, path := range candidates {
+		if _, keep := desired[path]; keep {
+			continue
+		}
+		if err := os.Remove(path); err != nil {
+			if errors.Is(err, os.ErrNotExist) {
+				continue
+			}
+			failures = append(failures, fmt.Errorf("remove stale file %s: %w", path, err))
+			continue
+		}
+		removed++
+		cleanupEmptyParents(filepath.Dir(path), p.SourceDir(), p.AssetsDir())
+	}
+
+	return removed, failures
+}
+
+func safeCleanupRoot(root, projectRoot string) bool {
+	cleanRoot := filepath.Clean(root)
+	return cleanRoot != "." && cleanRoot != string(filepath.Separator) && cleanRoot != filepath.Clean(projectRoot)
+}
+
+func cleanupEmptyParents(dir string, roots ...string) {
+	for {
+		if dir == "." || dir == string(filepath.Separator) {
+			return
+		}
+		stop := false
+		for _, root := range roots {
+			if dir == root {
+				stop = true
+				break
+			}
+		}
+		if stop {
+			return
+		}
+		if err := os.Remove(dir); err != nil {
+			return
+		}
+		dir = filepath.Dir(dir)
+	}
+}
+
+func sourceSubdir(extension string) string {
+	switch strings.ToLower(extension) {
+	case "are":
+		return "areas"
+	case "dlg":
+		return "dialogs"
+	case "fac":
+		return "factions"
+	case "gic":
+		return "instance"
+	case "git":
+		return "instance"
+	case "ifo":
+		return "module"
+	case "itp":
+		return "palettes"
+	case "jrl":
+		return "journal"
+	case "utc":
+		return filepath.Join("blueprints", "creatures")
+	case "utd":
+		return filepath.Join("blueprints", "doors")
+	case "ute":
+		return filepath.Join("blueprints", "encounters")
+	case "uti":
+		return filepath.Join("blueprints", "items")
+	case "utm":
+		return filepath.Join("blueprints", "merchants")
+	case "utp":
+		return filepath.Join("blueprints", "placeables")
+	case "uts":
+		return filepath.Join("blueprints", "sounds")
+	case "utt":
+		return filepath.Join("blueprints", "triggers")
+	case "utw":
+		return filepath.Join("blueprints", "waypoints")
+	default:
+		return extension
+	}
+}

internal/pipeline/music.go

@@ -0,0 +1,524 @@
+package pipeline
+
+import (
+	"encoding/json"
+	"fmt"
+	"os"
+	"path/filepath"
+	"sort"
+	"strings"
+
+	"git.westgate.pw/ShadowsOverWestgate/sow-tools/internal/erf"
+	"git.westgate.pw/ShadowsOverWestgate/sow-tools/internal/music"
+	"git.westgate.pw/ShadowsOverWestgate/sow-tools/internal/project"
+)
+
+type preparedMusicAssets struct {
+	Generated     []assetResource
+	SkipSourceRel map[string]struct{}
+	Artifacts     []string
+	TempRoots     []string
+	Summary       CreditsRefreshSummary
+}
+
+type MusicBuildOptions struct {
+	DatasetIDs []string
+	Write      bool
+}
+
+type musicPrepareOptions struct {
+	datasetIDs []string
+	write      bool
+}
+
+type musicSourceGroup struct {
+	DatasetID string
+	Dataset   project.EffectiveMusicDataset
+	SourceDir string
+	OutputDir string
+	Sources   []string
+}
+
+type musicCreditGroup struct {
+	CreditsRoot string
+	SourceDir   string
+	Entries     []music.CreditsEntry
+}
+
+func musicRootPath(p *project.Project, path string) string {
+	if strings.TrimSpace(path) == "" {
+		return p.Root
+	}
+	if filepath.IsAbs(path) {
+		return path
+	}
+	return filepath.Join(p.Root, filepath.FromSlash(path))
+}
+
+func effectiveMusicDatasets(p *project.Project, onlyIDs []string) (map[string]project.EffectiveMusicDataset, error) {
+	effective := p.EffectiveConfig()
+	wanted := map[string]struct{}{}
+	for _, id := range onlyIDs {
+		id = strings.TrimSpace(id)
+		if id != "" {
+			wanted[id] = struct{}{}
+		}
+	}
+	datasets := map[string]project.EffectiveMusicDataset{}
+	for id, dataset := range effective.Music.Datasets {
+		if len(wanted) > 0 {
+			if _, ok := wanted[id]; !ok {
+				continue
+			}
+			delete(wanted, id)
+		}
+		datasets[id] = dataset
+	}
+	if len(wanted) > 0 {
+		unknown := make([]string, 0, len(wanted))
+		for id := range wanted {
+			unknown = append(unknown, id)
+		}
+		sort.Strings(unknown)
+		return nil, fmt.Errorf("unknown music dataset(s): %s", strings.Join(unknown, ", "))
+	}
+	if len(datasets) == 0 && len(wanted) == 0 {
+		for _, rel := range p.Inventory.AssetFiles {
+			ext := strings.ToLower(filepath.Ext(rel))
+			if !music.PathIsUnder("envi/music", rel) {
+				continue
+			}
+			if _, ok := extensionSet(effective.Music.ConvertExtensions)[ext]; ok {
+				datasets["legacy_envi_music"] = project.EffectiveMusicDataset{
+					Source:            "envi/music",
+					Output:            "envi/music",
+					StageRoot:         effective.Music.StageRoot,
+					CreditsRoot:       effective.Music.CreditsRoot,
+					NamingScheme:      effective.Music.NamingScheme,
+					OutputExtension:   effective.Music.OutputExtension,
+					MaxStemLength:     effective.Music.MaxStemLength,
+					PackageMode:       "hak_asset",
+					ConvertExtensions: effective.Music.ConvertExtensions,
+				}
+				break
+			}
+		}
+	}
+	return datasets, nil
+}
+
+func extensionSet(values []string) map[string]struct{} {
+	out := make(map[string]struct{}, len(values))
+	for _, value := range values {
+		value = strings.ToLower(strings.TrimSpace(value))
+		if value != "" {
+			out[value] = struct{}{}
+		}
+	}
+	return out
+}
+
+func datasetForMusicSource(datasets map[string]project.EffectiveMusicDataset, rel, ext string) (string, project.EffectiveMusicDataset, bool) {
+	bestID := ""
+	var best project.EffectiveMusicDataset
+	bestLen := -1
+	for id, dataset := range datasets {
+		if _, ok := extensionSet(dataset.ConvertExtensions)[ext]; !ok {
+			continue
+		}
+		if !music.PathIsUnder(dataset.Source, rel) {
+			continue
+		}
+		if len(dataset.Source) > bestLen {
+			bestID = id
+			best = dataset
+			bestLen = len(dataset.Source)
+		}
+	}
+	return bestID, best, bestID != ""
+}
+
+func plannedMusicAsset(outputRel, sourceRel string) (assetResource, error) {
+	extension := strings.TrimPrefix(strings.ToLower(filepath.Ext(outputRel)), ".")
+	resourceType, ok := erf.HAKResourceTypeForExtension(extension)
+	if !ok {
+		return assetResource{}, fmt.Errorf("unsupported generated music resource extension %q", filepath.Ext(outputRel))
+	}
+	name := strings.ToLower(strings.TrimSuffix(filepath.Base(outputRel), filepath.Ext(outputRel)))
+	return assetResource{
+		Rel: outputRel,
+		Resource: erf.Resource{
+			Name: name,
+			Type: resourceType,
+		},
+		ContentID: "music-plan:" + filepath.ToSlash(sourceRel),
+	}, nil
+}
+
+func BuildMusic(p *project.Project, opts MusicBuildOptions) (BuildResult, error) {
+	prepared, err := prepareMusicAssetsWithOptions(p, musicPrepareOptions{datasetIDs: opts.DatasetIDs, write: opts.Write})
+	if err != nil {
+		return BuildResult{}, err
+	}
+	if err := cleanupPreparedMusicAssets(prepared); err != nil {
+		return BuildResult{}, err
+	}
+	return BuildResult{
+		CreditsArtifactPaths: prepared.Artifacts,
+		CreditsSummary:       prepared.Summary,
+		HAKAssets:            len(prepared.Generated),
+	}, nil
+}
+
+func prepareMusicAssets(p *project.Project) (*preparedMusicAssets, error) {
+	return prepareMusicAssetsWithOptions(p, musicPrepareOptions{write: true})
+}
+
+func prepareMusicAssetsWithOptions(p *project.Project, opts musicPrepareOptions) (*preparedMusicAssets, error) {
+	usedNames := make(map[string]struct{})
+	groupsByKey := make(map[string]*musicSourceGroup)
+	datasets, err := effectiveMusicDatasets(p, opts.datasetIDs)
+	if err != nil {
+		return nil, err
+	}
+	for _, rel := range p.Inventory.AssetFiles {
+		rel = filepath.ToSlash(rel)
+		ext := strings.ToLower(filepath.Ext(rel))
+		name := strings.ToLower(strings.TrimSuffix(filepath.Base(rel), filepath.Ext(rel)))
+		if datasetID, dataset, ok := datasetForMusicSource(datasets, rel, ext); ok {
+			sourceDir := filepath.ToSlash(filepath.Dir(rel))
+			relDir, err := filepath.Rel(filepath.FromSlash(dataset.Source), filepath.FromSlash(sourceDir))
+			if err != nil {
+				return nil, fmt.Errorf("resolve music source directory %s: %w", sourceDir, err)
+			}
+			relDir = filepath.ToSlash(relDir)
+			outputDir := dataset.Output
+			if relDir != "." && relDir != "" {
+				outputDir = filepath.ToSlash(filepath.Join(outputDir, filepath.FromSlash(relDir)))
+			}
+			key := datasetID + "\x00" + sourceDir
+			group := groupsByKey[key]
+			if group == nil {
+				group = &musicSourceGroup{
+					DatasetID: datasetID,
+					Dataset:   dataset,
+					SourceDir: sourceDir,
+					OutputDir: outputDir,
+				}
+				groupsByKey[key] = group
+			}
+			group.Sources = append(group.Sources, rel)
+			continue
+		}
+		if name != "" {
+			usedNames[name] = struct{}{}
+		}
+	}
+
+	result := &preparedMusicAssets{
+		SkipSourceRel: make(map[string]struct{}),
+	}
+	if len(groupsByKey) == 0 {
+		if !opts.write {
+			return result, nil
+		}
+		writeResult, err := writeCreditsArtifacts(p, nil)
+		if err != nil {
+			return nil, err
+		}
+		result.Artifacts = writeResult.Artifacts
+		result.Summary = writeResult.Summary
+		return result, nil
+	}
+
+	groups := make([]*musicSourceGroup, 0, len(groupsByKey))
+	for _, group := range groupsByKey {
+		sort.Strings(group.Sources)
+		groups = append(groups, group)
+	}
+	sort.Slice(groups, func(i, j int) bool {
+		if groups[i].SourceDir != groups[j].SourceDir {
+			return groups[i].SourceDir < groups[j].SourceDir
+		}
+		return groups[i].DatasetID < groups[j].DatasetID
+	})
+
+	creditGroups := make([]musicCreditGroup, 0, len(groups))
+	for _, group := range groups {
+		result.Summary.ScannedDirs = append(result.Summary.ScannedDirs, group.SourceDir)
+		result.Summary.TrackCount += len(group.Sources)
+	}
+	ffmpegPath := ""
+	ffprobePath := ""
+	if opts.write {
+		var err error
+		ffmpegPath, err = music.ResolveFFmpegWithConfig(p.MusicFFmpegPath())
+		if err != nil {
+			return nil, err
+		}
+		ffprobePath, err = music.ResolveFFprobeWithConfig(p.MusicFFprobePath())
+		if err != nil {
+			return nil, err
+		}
+	}
+
+	for _, group := range groups {
+		stageRoot := musicRootPath(p, group.Dataset.StageRoot)
+		result.TempRoots = append(result.TempRoots, stageRoot)
+		prefix := music.SanitizePrefix(group.Dataset.Prefix)
+		overlayPath := filepath.Join(p.AssetsDir(), filepath.FromSlash(group.SourceDir), p.EffectiveConfig().Music.CreditsOverlay)
+		overlay, err := music.ParseCreditsOverlay(overlayPath)
+		if err != nil {
+			return nil, fmt.Errorf("parse credits overlay %s: %w", overlayPath, err)
+		}
+
+		entries := make([]music.CreditsEntry, 0, len(group.Sources))
+		for _, rel := range group.Sources {
+			base := filepath.Base(rel)
+			manual := overlay.Match(base, "")
+			outputFile := ""
+			switch {
+			case manual != nil && strings.TrimSpace(manual.OutputFile) != "":
+				outputFile = strings.ToLower(strings.TrimSpace(manual.OutputFile))
+				if err := music.ValidateManualOutputFileWithRules(outputFile, group.Dataset.OutputExtension, group.Dataset.MaxStemLength); err != nil {
+					return nil, fmt.Errorf("manual credits output for %s: %w", rel, err)
+				}
+				if err := music.ReserveName(strings.TrimSuffix(outputFile, filepath.Ext(outputFile)), usedNames); err != nil {
+					return nil, fmt.Errorf("manual credits output for %s: %w", rel, err)
+				}
+			default:
+				stem, err := music.GenerateStemWithMax(base, prefix, group.Dataset.MaxStemLength, usedNames)
+				if err != nil {
+					return nil, fmt.Errorf("generate music name for %s: %w", rel, err)
+				}
+				outputFile = stem + group.Dataset.OutputExtension
+			}
+			outputRel := filepath.ToSlash(filepath.Join(group.OutputDir, outputFile))
+			result.Summary.Mappings = append(result.Summary.Mappings, MusicFileMapping{
+				Source: rel,
+				Output: outputRel,
+			})
+			result.SkipSourceRel[rel] = struct{}{}
+
+			if !opts.write {
+				if group.Dataset.PackageMode == "hak_asset" {
+					asset, err := plannedMusicAsset(outputRel, rel)
+					if err != nil {
+						return nil, err
+					}
+					result.Generated = append(result.Generated, asset)
+				}
+				continue
+			}
+
+			metadata, err := music.ProbeMetadata(ffprobePath, filepath.Join(p.AssetsDir(), filepath.FromSlash(rel)))
+			if err != nil {
+				return nil, fmt.Errorf("probe music metadata for %s: %w", rel, err)
+			}
+			entry := music.CreditsEntry{
+				Artist:       metadata.Artist,
+				Title:        music.FirstNonEmpty(metadata.Title, strings.TrimSuffix(base, filepath.Ext(base))),
+				OutputFile:   outputFile,
+				OriginalFile: base,
+				Album:        metadata.Album,
+				Date:         metadata.Date,
+				Rights:       music.FirstNonEmpty(metadata.Rights, music.JoinNonEmpty("; ", metadata.License, metadata.Copyright)),
+				Notes:        music.FirstNonEmpty(metadata.Notes, metadata.Comment),
+			}
+			music.ApplyCreditsOverlay(&entry, manual)
+
+			stagePath := filepath.Join(stageRoot, filepath.FromSlash(outputRel))
+			if err := os.MkdirAll(filepath.Dir(stagePath), 0o755); err != nil {
+				return nil, fmt.Errorf("create music stage dir: %w", err)
+			}
+			if err := music.ConvertSource(ffmpegPath, filepath.Join(p.AssetsDir(), filepath.FromSlash(rel)), stagePath); err != nil {
+				return nil, fmt.Errorf("convert music %s: %w", rel, err)
+			}
+			if group.Dataset.PackageMode == "hak_asset" {
+				resourceInfo, err := assetResourceFromPath(stagePath, outputRel, lfsAssetInfo{}, false)
+				if err != nil {
+					return nil, fmt.Errorf("load converted music %s: %w", outputRel, err)
+				}
+				info, err := os.Stat(stagePath)
+				if err != nil {
+					return nil, fmt.Errorf("stat converted music %s: %w", stagePath, err)
+				}
+				result.Generated = append(result.Generated, assetResource{
+					Rel:       outputRel,
+					Resource:  resourceInfo.Resource,
+					Size:      erf.ArchiveSize([]erf.Resource{resourceInfo.Resource}),
+					CreatedAt: info.ModTime(),
+					ContentID: resourceInfo.ContentID,
+				})
+			}
+			entries = append(entries, entry)
+		}
+		if opts.write {
+			if err := music.ValidateOverlayEntries(group.SourceDir, overlay, entries); err != nil {
+				return nil, err
+			}
+			creditGroups = append(creditGroups, musicCreditGroup{
+				CreditsRoot: group.Dataset.CreditsRoot,
+				SourceDir:   group.SourceDir,
+				Entries:     entries,
+			})
+		}
+	}
+
+	if !opts.write {
+		sort.Slice(result.Generated, func(i, j int) bool {
+			return compareResourceKeys(result.Generated[i].Resource, result.Generated[j].Resource) < 0
+		})
+		return result, nil
+	}
+
+	writeResult, err := writeCreditsArtifacts(p, creditGroups)
+	if err != nil {
+		return nil, err
+	}
+	writeResult.Summary.ScannedDirs = append(writeResult.Summary.ScannedDirs, result.Summary.ScannedDirs...)
+	writeResult.Summary.TrackCount = result.Summary.TrackCount
+	result.Artifacts = writeResult.Artifacts
+	result.Summary = writeResult.Summary
+	sort.Slice(result.Generated, func(i, j int) bool {
+		return compareResourceKeys(result.Generated[i].Resource, result.Generated[j].Resource) < 0
+	})
+	return result, nil
+}
+
+type creditsArtifactWriteResult struct {
+	Artifacts []string
+	Summary   CreditsRefreshSummary
+}
+
+func writeCreditsArtifacts(p *project.Project, generated []musicCreditGroup) (creditsArtifactWriteResult, error) {
+	defaultCreditsRoot := p.CreditsDir()
+	sources := make([]music.CreditsSource, 0)
+	desiredByRoot := map[string]map[string][]byte{}
+	summary := CreditsRefreshSummary{}
+	if generated != nil {
+		sort.Slice(generated, func(i, j int) bool {
+			if generated[i].SourceDir != generated[j].SourceDir {
+				return generated[i].SourceDir < generated[j].SourceDir
+			}
+			return generated[i].CreditsRoot < generated[j].CreditsRoot
+		})
+		for _, group := range generated {
+			creditsRoot := musicRootPath(p, group.CreditsRoot)
+			if desiredByRoot[creditsRoot] == nil {
+				desiredByRoot[creditsRoot] = map[string][]byte{}
+			}
+			entries := append([]music.CreditsEntry(nil), group.Entries...)
+			sort.Slice(entries, func(i, j int) bool {
+				if strings.ToLower(entries[i].Artist) != strings.ToLower(entries[j].Artist) {
+					return strings.ToLower(entries[i].Artist) < strings.ToLower(entries[j].Artist)
+				}
+				if strings.ToLower(entries[i].Title) != strings.ToLower(entries[j].Title) {
+					return strings.ToLower(entries[i].Title) < strings.ToLower(entries[j].Title)
+				}
+				return strings.ToLower(entries[i].OutputFile) < strings.ToLower(entries[j].OutputFile)
+			})
+			outputPath := filepath.Join(creditsRoot, filepath.FromSlash(group.SourceDir), p.EffectiveConfig().Music.CreditsOverlay)
+			desiredByRoot[creditsRoot][outputPath] = []byte(music.RenderCreditsMarkdown(entries))
+			summary.GeneratedPaths = append(summary.GeneratedPaths, outputPath)
+			for _, entry := range entries {
+				summary.Mappings = append(summary.Mappings, MusicFileMapping{
+					Source: entry.OriginalFile,
+					Output: entry.OutputFile,
+				})
+			}
+			sources = append(sources, music.CreditsSource{
+				Path:    filepath.ToSlash(outputPath),
+				Kind:    "generated_music",
+				Entries: entries,
+			})
+		}
+	}
+
+	err := filepath.WalkDir(p.AssetsDir(), func(path string, d os.DirEntry, walkErr error) error {
+		if walkErr != nil {
+			return walkErr
+		}
+		if d.IsDir() {
+			return nil
+		}
+		if strings.ToLower(d.Name()) != "credits.md" {
+			return nil
+		}
+		entries, err := music.ParseCreditsMarkdown(path)
+		if err != nil {
+			return fmt.Errorf("parse credits %s: %w", path, err)
+		}
+		rel, err := filepath.Rel(p.Root, path)
+		if err != nil {
+			return err
+		}
+		sources = append(sources, music.CreditsSource{
+			Path:    filepath.ToSlash(rel),
+			Kind:    "authored",
+			Entries: entries,
+		})
+		return nil
+	})
+	if err != nil {
+		return creditsArtifactWriteResult{}, err
+	}
+
+	sort.Slice(sources, func(i, j int) bool { return sources[i].Path < sources[j].Path })
+	payload, err := json.MarshalIndent(music.CreditsInventory{Sources: sources}, "", "  ")
+	if err != nil {
+		return creditsArtifactWriteResult{}, fmt.Errorf("marshal credits inventory: %w", err)
+	}
+	payload = append(payload, '\n')
+	if len(desiredByRoot) == 0 {
+		desiredByRoot[defaultCreditsRoot] = map[string][]byte{}
+	}
+	artifacts := make([]string, 0)
+	changedFiles := 0
+	roots := make([]string, 0, len(desiredByRoot))
+	for root := range desiredByRoot {
+		roots = append(roots, root)
+	}
+	sort.Strings(roots)
+	for _, creditsRoot := range roots {
+		desiredFiles := desiredByRoot[creditsRoot]
+		inventoryPath := filepath.Join(creditsRoot, "credits.json")
+		desiredFiles[inventoryPath] = payload
+		if err := os.MkdirAll(creditsRoot, 0o755); err != nil {
+			return creditsArtifactWriteResult{}, fmt.Errorf("create credits dir: %w", err)
+		}
+		changed, err := music.SyncGeneratedCreditsArtifacts(creditsRoot, desiredFiles)
+		if err != nil {
+			return creditsArtifactWriteResult{}, err
+		}
+		changedFiles += changed
+		for path := range desiredFiles {
+			artifacts = append(artifacts, path)
+		}
+		if summary.InventoryPath == "" {
+			summary.InventoryPath = inventoryPath
+		}
+	}
+	sort.Strings(artifacts)
+	summary.ArtifactPaths = append(summary.ArtifactPaths, artifacts...)
+	summary.ChangedFiles = changedFiles
+	return creditsArtifactWriteResult{
+		Artifacts: artifacts,
+		Summary:   summary,
+	}, nil
+}
+
+func cleanupPreparedMusicAssets(prepared *preparedMusicAssets) error {
+	if prepared == nil {
+		return nil
+	}
+	for _, root := range prepared.TempRoots {
+		if strings.TrimSpace(root) == "" {
+			continue
+		}
+		if err := os.RemoveAll(root); err != nil {
+			return fmt.Errorf("remove temporary music artifact root %s: %w", root, err)
+		}
+	}
+	return nil
+}

internal/project/effective.go

@@ -0,0 +1,738 @@
+package project
+
+import (
+	"encoding/json"
+	"fmt"
+	"os"
+	"path/filepath"
+	"slices"
+	"strings"
+	"time"
+
+	"git.westgate.pw/ShadowsOverWestgate/sow-tools/internal/music"
+)
+
+const (
+	DefaultBuildPath                       = "build"
+	DefaultCachePath                       = ".cache"
+	DefaultToolsPath                       = "tools"
+	DefaultModuleArchiveTemplate           = "{module.resref}.mod"
+	DefaultHAKManifest                     = "haks.json"
+	DefaultHAKArchiveTemplate              = "{hak.name}.hak"
+	DefaultSourceJSONPattern               = "{resref}.{extension}.json"
+	DefaultValidationProfile               = "nwn_module"
+	DefaultMusicStageRoot                  = "{paths.cache}/music"
+	DefaultCreditsRoot                     = "{paths.cache}/credits"
+	DefaultCreditsOverlayFile              = "CREDITS.md"
+	DefaultMusicNamingScheme               = "nwn_bmu"
+	DefaultMusicOutputExtension            = ".bmu"
+	DefaultTopDataBuild                    = "{paths.build}/topdata"
+	DefaultTopDataCompiled2DADir           = "2da"
+	DefaultTopDataCompiledTLK              = "sow_tlk.tlk"
+	DefaultTopDataWikiOutputRoot           = "wiki"
+	DefaultTopDataWikiPagesDir             = "pages"
+	DefaultTopDataWikiStateFile            = "state.json"
+	DefaultTopDataWikiSource               = "topdata/wiki"
+	DefaultTopDataWikiRenderer             = "nodebb_tiptap_html"
+	DefaultTopDataWikiLinkStrategy         = "preserve_westgate_wiki_links"
+	DefaultTopDataWikiNamespacesFile       = "namespaces.yaml"
+	DefaultTopDataWikiTablesFile           = "tables.yaml"
+	DefaultTopDataWikiVisibilityFile       = "visibility.yaml"
+	DefaultTopDataWikiDataFile             = "data.yaml"
+	DefaultTopDataWikiPagePathsFile        = "wiki.yaml"
+	DefaultTopDataWikiTemplatesDir         = "templates"
+	DefaultTopDataWikiPageTemplatesDir     = "templates/pages"
+	DefaultTopDataWikiManualSectionsDir    = "manual-sections"
+	DefaultTopDataWikiManualSectionsFile   = "manual-sections/default.yaml"
+	DefaultTopDataWikiPageIndexFile        = "page-index.json"
+	DefaultTopDataWikiStaleDefault         = "report"
+	DefaultTopDataWikiStaleLiveCleanup     = "archive"
+	DefaultTopDataWikiMarkerFormat         = "html_comments"
+	DefaultTopDataWikiPageMarkerPrefix     = "sow-topdata-wiki"
+	DefaultTopDataWikiTitlePrefixMinLength = 3
+	DefaultTopDataWikiStatusPages          = true
+	DefaultTopDataWikiStatusListingScope   = "all"
+	DefaultTopDataWikiAlignmentLinkFormat  = "wiki_markup"
+	DefaultWikiDeployManifest              = ".wiki_deploy_manifest.json"
+	DefaultWikiDeployEditSummary           = "Auto-generated from native builder"
+	DefaultTopDataPackageHAK               = "sow_top.hak"
+	DefaultTopDataPackageTLK               = "sow_tlk.tlk"
+	DefaultAutogenCacheRoot                = "{paths.cache}"
+	DefaultAutogenCacheMaxAge              = time.Hour
+	DefaultAutogenRefreshEnv               = "SOW_AUTOGEN_MANIFEST_REFRESH"
+	DefaultPartsManifestRefreshEnv         = "SOW_PARTS_MANIFEST_REFRESH"
+	DefaultAutogenReleaseProvider          = "gitea"
+	DefaultAutogenServerURLEnv             = "SOW_ASSETS_SERVER_URL"
+	DefaultAutogenRepoEnv                  = "SOW_ASSETS_REPO"
+	DefaultExtractLayout                   = "nwn_canonical_json"
+	DefaultExtractHAKDiscovery             = "build_glob"
+)
+
+type ConfigProvenance map[string]ConfigValueProvenance
+
+type ConfigValueProvenance struct {
+	Source     string `json:"source" yaml:"source"`
+	Detail     string `json:"detail,omitempty" yaml:"detail,omitempty"`
+	Deprecated bool   `json:"deprecated,omitempty" yaml:"deprecated,omitempty"`
+}
+
+type EffectiveConfig struct {
+	ConfigSource ConfigSource           `json:"config_source" yaml:"config_source"`
+	Module       ModuleConfig           `json:"module" yaml:"module"`
+	Paths        EffectivePathConfig    `json:"paths" yaml:"paths"`
+	Outputs      EffectiveOutputConfig  `json:"outputs" yaml:"outputs"`
+	Build        BuildConfig            `json:"build" yaml:"build"`
+	Generated    GeneratedConfig        `json:"generated_assets" yaml:"generated_assets"`
+	Inventory    InventoryConfig        `json:"inventory" yaml:"inventory"`
+	Validation   ValidationConfig       `json:"validation" yaml:"validation"`
+	Music        EffectiveMusicConfig   `json:"music" yaml:"music"`
+	TopData      EffectiveTopDataConfig `json:"topdata" yaml:"topdata"`
+	Extract      ExtractConfig          `json:"extract" yaml:"extract"`
+	Autogen      EffectiveAutogenConfig `json:"autogen" yaml:"autogen"`
+	HAKs         []HAKConfig            `json:"haks" yaml:"haks"`
+	Provenance   ConfigProvenance       `json:"provenance" yaml:"provenance"`
+	Overrides    []ConfigOverride       `json:"overrides,omitempty" yaml:"overrides,omitempty"`
+}
+
+type EffectivePathConfig struct {
+	Source string `json:"source" yaml:"source"`
+	Assets string `json:"assets" yaml:"assets"`
+	Build  string `json:"build" yaml:"build"`
+	Cache  string `json:"cache" yaml:"cache"`
+	Tools  string `json:"tools" yaml:"tools"`
+}
+
+type EffectiveOutputConfig struct {
+	ModuleArchive string `json:"module_archive" yaml:"module_archive"`
+	HAKManifest   string `json:"hak_manifest" yaml:"hak_manifest"`
+	HAKArchive    string `json:"hak_archive" yaml:"hak_archive"`
+}
+
+type EffectiveMusicConfig struct {
+	Prefixes          map[string]string                `json:"prefixes" yaml:"prefixes"`
+	Tools             MusicToolsConfig                 `json:"tools" yaml:"tools"`
+	StageRoot         string                           `json:"stage_root" yaml:"stage_root"`
+	CreditsRoot       string                           `json:"credits_root" yaml:"credits_root"`
+	CreditsOverlay    string                           `json:"credits_overlay" yaml:"credits_overlay"`
+	MaxStemLength     int                              `json:"max_stem_length" yaml:"max_stem_length"`
+	NamingScheme      string                           `json:"naming_scheme" yaml:"naming_scheme"`
+	OutputExtension   string                           `json:"output_extension" yaml:"output_extension"`
+	ConvertExtensions []string                         `json:"convert_extensions" yaml:"convert_extensions"`
+	Datasets          map[string]EffectiveMusicDataset `json:"datasets,omitempty" yaml:"datasets,omitempty"`
+}
+
+type EffectiveMusicDataset struct {
+	Source            string   `json:"source" yaml:"source"`
+	Output            string   `json:"output" yaml:"output"`
+	Prefix            string   `json:"prefix,omitempty" yaml:"prefix,omitempty"`
+	StageRoot         string   `json:"stage_root,omitempty" yaml:"stage_root,omitempty"`
+	CreditsRoot       string   `json:"credits_root,omitempty" yaml:"credits_root,omitempty"`
+	NamingScheme      string   `json:"naming_scheme" yaml:"naming_scheme"`
+	OutputExtension   string   `json:"output_extension" yaml:"output_extension"`
+	MaxStemLength     int      `json:"max_stem_length" yaml:"max_stem_length"`
+	PackageMode       string   `json:"package_mode" yaml:"package_mode"`
+	ConvertExtensions []string `json:"convert_extensions" yaml:"convert_extensions"`
+}
+
+type EffectiveTopDataConfig struct {
+	Source              string                          `json:"source" yaml:"source"`
+	Build               string                          `json:"build" yaml:"build"`
+	ReferenceBuilder    string                          `json:"reference_builder" yaml:"reference_builder"`
+	Assets              string                          `json:"assets" yaml:"assets"`
+	Compiled2DADir      string                          `json:"compiled_2da_dir" yaml:"compiled_2da_dir"`
+	CompiledTLK         string                          `json:"compiled_tlk" yaml:"compiled_tlk"`
+	PackageHAK          string                          `json:"package_hak" yaml:"package_hak"`
+	PackageTLK          string                          `json:"package_tlk" yaml:"package_tlk"`
+	ValueEncodings      []TopDataValueEncodingConfig    `json:"value_encodings" yaml:"value_encodings"`
+	ValueDefaults       []TopDataValueDefaultConfig     `json:"value_defaults" yaml:"value_defaults"`
+	RowGeneration       []TopDataRowGenerationConfig    `json:"row_generation" yaml:"row_generation"`
+	RowExtensions       []TopDataRowExtensionConfig     `json:"row_extensions" yaml:"row_extensions"`
+	ClassFeatInjections TopDataClassFeatInjectionConfig `json:"class_feat_injections" yaml:"class_feat_injections"`
+	Wiki                TopDataWikiConfig               `json:"wiki" yaml:"wiki"`
+}
+
+type EffectiveAutogenConfig struct {
+	Producers     []AutogenProducerConfig             `json:"producers" yaml:"producers"`
+	Consumers     []AutogenConsumerConfig             `json:"consumers" yaml:"consumers"`
+	Cache         EffectiveAutogenCacheConfig         `json:"cache" yaml:"cache"`
+	ReleaseSource EffectiveAutogenReleaseSourceConfig `json:"release_source" yaml:"release_source"`
+}
+
+type EffectiveAutogenCacheConfig struct {
+	Root                    string `json:"root" yaml:"root"`
+	MaxAge                  string `json:"max_age" yaml:"max_age"`
+	RefreshEnv              string `json:"refresh_env" yaml:"refresh_env"`
+	PartsManifestRefreshEnv string `json:"parts_manifest_refresh_env" yaml:"parts_manifest_refresh_env"`
+}
+
+type EffectiveAutogenReleaseSourceConfig struct {
+	Provider     string `json:"provider" yaml:"provider"`
+	ServerURLEnv string `json:"server_url_env" yaml:"server_url_env"`
+	RepoEnv      string `json:"repo_env" yaml:"repo_env"`
+}
+
+type ConfigOverride struct {
+	Key    string `json:"key" yaml:"key"`
+	Value  string `json:"value" yaml:"value"`
+	Source string `json:"source" yaml:"source"`
+}
+
+func (p *Project) EffectiveConfig() EffectiveConfig {
+	provenance := cloneProvenance(p.Provenance)
+	markMissingDefaults(provenance)
+
+	paths := EffectivePathConfig{
+		Source: strings.TrimSpace(p.Config.Paths.Source),
+		Assets: strings.TrimSpace(p.Config.Paths.Assets),
+		Build:  defaultString(p.Config.Paths.Build, DefaultBuildPath),
+		Cache:  defaultString(p.Config.Paths.Cache, DefaultCachePath),
+		Tools:  defaultString(p.Config.Paths.Tools, DefaultToolsPath),
+	}
+	outputs := EffectiveOutputConfig{
+		ModuleArchive: defaultString(p.Config.Outputs.ModuleArchive, DefaultModuleArchiveTemplate),
+		HAKManifest:   defaultString(p.Config.Outputs.HAKManifest, DefaultHAKManifest),
+		HAKArchive:    defaultString(p.Config.Outputs.HAKArchive, DefaultHAKArchiveTemplate),
+	}
+	inventory := InventoryConfig{
+		SourceExtensions:  defaultStringSlice(p.Config.Inventory.SourceExtensions, SourceExtensions),
+		AssetExtensions:   defaultStringSlice(p.Config.Inventory.AssetExtensions, AssetExtensions),
+		SourceJSONPattern: defaultString(p.Config.Inventory.SourceJSONPattern, DefaultSourceJSONPattern),
+	}
+	validation := ValidationConfig{
+		Profile:               defaultString(p.Config.Validation.Profile, DefaultValidationProfile),
+		BuiltinScriptPrefixes: normalizeLowerStringSlice(defaultStringSlice(p.Config.Validation.BuiltinScriptPrefixes, DefaultBuiltinScriptPrefixes())),
+		RequiredFields:        defaultRequiredFields(p.Config.Validation.RequiredFields),
+	}
+	topBuild := expandPathTemplate(defaultString(p.Config.TopData.Build, DefaultTopDataBuild), paths)
+	top := EffectiveTopDataConfig{
+		Source:              strings.TrimSpace(p.Config.TopData.Source),
+		Build:               topBuild,
+		ReferenceBuilder:    strings.TrimSpace(p.Config.TopData.ReferenceBuilder),
+		Assets:              strings.TrimSpace(p.Config.TopData.Assets),
+		Compiled2DADir:      defaultString(p.Config.TopData.Compiled2DADir, DefaultTopDataCompiled2DADir),
+		CompiledTLK:         defaultString(p.Config.TopData.CompiledTLK, DefaultTopDataCompiledTLK),
+		PackageHAK:          defaultString(p.Config.TopData.PackageHAK, DefaultTopDataPackageHAK),
+		PackageTLK:          defaultString(p.Config.TopData.PackageTLK, DefaultTopDataPackageTLK),
+		ValueEncodings:      cloneTopDataValueEncodings(p.Config.TopData.ValueEncodings),
+		ValueDefaults:       cloneTopDataValueDefaults(p.Config.TopData.ValueDefaults),
+		RowGeneration:       cloneTopDataRowGeneration(p.Config.TopData.RowGeneration),
+		RowExtensions:       cloneTopDataRowExtensions(p.Config.TopData.RowExtensions),
+		ClassFeatInjections: cloneTopDataClassFeatInjections(p.Config.TopData.ClassFeatInjections),
+		Wiki: TopDataWikiConfig{
+			OutputRoot:         defaultString(p.Config.TopData.Wiki.OutputRoot, DefaultTopDataWikiOutputRoot),
+			PagesDir:           defaultString(p.Config.TopData.Wiki.PagesDir, DefaultTopDataWikiPagesDir),
+			StateFile:          defaultString(p.Config.TopData.Wiki.StateFile, DefaultTopDataWikiStateFile),
+			ManagedNamespaces:  defaultStringSlice(p.Config.TopData.Wiki.ManagedNamespaces, []string{"classes", "feat", "itemtypes", "races", "skills", "spells", "meta"}),
+			DeployManifest:     defaultString(p.Config.TopData.Wiki.DeployManifest, DefaultWikiDeployManifest),
+			DeployEditSummary:  defaultString(p.Config.TopData.Wiki.DeployEditSummary, DefaultWikiDeployEditSummary),
+			Source:             defaultString(p.Config.TopData.Wiki.Source, DefaultTopDataWikiSource),
+			Renderer:           defaultString(p.Config.TopData.Wiki.Renderer, DefaultTopDataWikiRenderer),
+			LinkStrategy:       defaultString(p.Config.TopData.Wiki.LinkStrategy, DefaultTopDataWikiLinkStrategy),
+			NamespacesFile:     defaultString(p.Config.TopData.Wiki.NamespacesFile, DefaultTopDataWikiNamespacesFile),
+			TablesFile:         defaultString(p.Config.TopData.Wiki.TablesFile, DefaultTopDataWikiTablesFile),
+			VisibilityFile:     defaultString(p.Config.TopData.Wiki.VisibilityFile, DefaultTopDataWikiVisibilityFile),
+			DataFile:           defaultString(p.Config.TopData.Wiki.DataFile, DefaultTopDataWikiDataFile),
+			PagePathsFile:      defaultString(p.Config.TopData.Wiki.PagePathsFile, DefaultTopDataWikiPagePathsFile),
+			TemplatesDir:       defaultString(p.Config.TopData.Wiki.TemplatesDir, DefaultTopDataWikiTemplatesDir),
+			PageTemplatesDir:   defaultWikiPageTemplatesDir(p.Config.TopData.Wiki),
+			ManualSectionsDir:  defaultString(p.Config.TopData.Wiki.ManualSectionsDir, DefaultTopDataWikiManualSectionsDir),
+			ManualSectionsFile: defaultWikiManualSectionsFile(p.Config.TopData.Wiki),
+			PageIndexFile:      defaultString(p.Config.TopData.Wiki.PageIndexFile, DefaultTopDataWikiPageIndexFile),
+			AlignmentLinks: TopDataWikiAlignmentLinks{
+				TargetPattern: strings.TrimSpace(p.Config.TopData.Wiki.AlignmentLinks.TargetPattern),
+				LinkFormat:    defaultString(p.Config.TopData.Wiki.AlignmentLinks.LinkFormat, DefaultTopDataWikiAlignmentLinkFormat),
+			},
+			TitlePrefixMinLength: defaultInt(p.Config.TopData.Wiki.TitlePrefixMinLength, DefaultTopDataWikiTitlePrefixMinLength),
+			StatusPages:          defaultBoolPointer(p.Config.TopData.Wiki.StatusPages, DefaultTopDataWikiStatusPages),
+			StatusListingScope:   defaultString(p.Config.TopData.Wiki.StatusListingScope, DefaultTopDataWikiStatusListingScope),
+			StalePages: TopDataWikiStalePagesConfig{
+				Default:     defaultString(p.Config.TopData.Wiki.StalePages.Default, DefaultTopDataWikiStaleDefault),
+				LiveCleanup: defaultString(p.Config.TopData.Wiki.StalePages.LiveCleanup, DefaultTopDataWikiStaleLiveCleanup),
+			},
+			ManagedRegion: TopDataWikiManagedRegionConfig{
+				MarkerFormat:     defaultString(p.Config.TopData.Wiki.ManagedRegion.MarkerFormat, DefaultTopDataWikiMarkerFormat),
+				PageMarkerPrefix: defaultString(p.Config.TopData.Wiki.ManagedRegion.PageMarkerPrefix, DefaultTopDataWikiPageMarkerPrefix),
+			},
+		},
+	}
+	generated := p.Config.Generated
+	for index := range generated.TopData2DA {
+		generated.TopData2DA[index].Output = expandPathTemplate(defaultString(generated.TopData2DA[index].Output, "{paths.cache}/generated-assets/"+generated.TopData2DA[index].ID), paths)
+	}
+
+	extract := p.Config.Extract
+	extract.Layout = defaultString(extract.Layout, DefaultExtractLayout)
+	extract.HAKDiscovery = defaultString(extract.HAKDiscovery, DefaultExtractHAKDiscovery)
+	if len(extract.Archives) == 0 {
+		extract.Archives = []string{outputs.ModuleArchiveName(p.Config.Module)}
+	}
+	if extract.CleanupStale == nil {
+		defaultCleanup := true
+		extract.CleanupStale = &defaultCleanup
+	}
+	if extract.ConsumeArchives == nil {
+		defaultConsume := false
+		extract.ConsumeArchives = &defaultConsume
+	}
+
+	musicStageRoot := expandPathTemplate(DefaultMusicStageRoot, paths)
+	musicCreditsRoot := expandPathTemplate(DefaultCreditsRoot, paths)
+	musicCreditsOverlay := DefaultCreditsOverlayFile
+	musicMaxStemLength := 16
+	musicNamingScheme := DefaultMusicNamingScheme
+	musicOutputExtension := DefaultMusicOutputExtension
+	musicConvertExtensions := music.DefaultConvertExtensions()
+	if d := p.Config.Music.Defaults; d != nil {
+		if d.StageRoot != "" {
+			musicStageRoot = expandPathTemplate(d.StageRoot, paths)
+		}
+		if d.CreditsRoot != "" {
+			musicCreditsRoot = expandPathTemplate(d.CreditsRoot, paths)
+		}
+		if d.CreditsOverlay != "" {
+			musicCreditsOverlay = d.CreditsOverlay
+		}
+		if d.MaxStemLength != nil {
+			musicMaxStemLength = *d.MaxStemLength
+		}
+		if d.NamingScheme != "" {
+			musicNamingScheme = d.NamingScheme
+		}
+		if d.OutputExtension != "" {
+			musicOutputExtension = normalizeExtension(d.OutputExtension)
+		}
+		if len(d.ConvertExtensions) > 0 {
+			musicConvertExtensions = normalizeExtensionSlice(d.ConvertExtensions)
+		}
+	}
+	musicPrefixes := cloneStringMap(p.Config.Music.Prefixes)
+	if len(musicPrefixes) == 0 {
+		musicPrefixes = make(map[string]string, len(p.Config.Music.Datasets))
+		for _, ds := range p.Config.Music.Datasets {
+			if ds.Source != "" && ds.Prefix != "" {
+				musicPrefixes[ds.Source] = ds.Prefix
+			}
+		}
+	}
+	effectiveDatasets := make(map[string]EffectiveMusicDataset, len(p.Config.Music.Datasets))
+	for id, ds := range p.Config.Music.Datasets {
+		dsStageRoot := ds.StageRoot
+		if dsStageRoot == "" {
+			dsStageRoot = musicStageRoot
+		}
+		dsCreditsRoot := ds.CreditsRoot
+		if dsCreditsRoot == "" {
+			dsCreditsRoot = musicCreditsRoot
+		}
+		dsOutput := strings.TrimSpace(ds.Output)
+		if dsOutput == "" {
+			dsOutput = ds.Source
+		}
+		dsNamingScheme := defaultString(ds.NamingScheme, musicNamingScheme)
+		dsOutputExtension := musicOutputExtension
+		if ds.OutputExtension != "" {
+			dsOutputExtension = normalizeExtension(ds.OutputExtension)
+		}
+		dsConvertExtensions := musicConvertExtensions
+		if len(ds.ConvertExtensions) > 0 {
+			dsConvertExtensions = normalizeExtensionSlice(ds.ConvertExtensions)
+		}
+		effectiveDatasets[id] = EffectiveMusicDataset{
+			Source:            ds.Source,
+			Output:            dsOutput,
+			Prefix:            ds.Prefix,
+			StageRoot:         dsStageRoot,
+			CreditsRoot:       dsCreditsRoot,
+			NamingScheme:      dsNamingScheme,
+			OutputExtension:   dsOutputExtension,
+			MaxStemLength:     musicMaxStemLength,
+			PackageMode:       defaultString(ds.PackageMode, "hak_asset"),
+			ConvertExtensions: dsConvertExtensions,
+		}
+	}
+	effective := EffectiveConfig{
+		ConfigSource: p.ConfigSource,
+		Module:       p.Config.Module,
+		Paths:        paths,
+		Outputs:      outputs,
+		Build:        p.Config.Build,
+		Generated:    generated,
+		Inventory:    inventory,
+		Validation:   validation,
+		Music: EffectiveMusicConfig{
+			Prefixes:          musicPrefixes,
+			Tools:             p.Config.Music.Tools,
+			StageRoot:         musicStageRoot,
+			CreditsRoot:       musicCreditsRoot,
+			CreditsOverlay:    musicCreditsOverlay,
+			MaxStemLength:     musicMaxStemLength,
+			NamingScheme:      musicNamingScheme,
+			OutputExtension:   musicOutputExtension,
+			ConvertExtensions: musicConvertExtensions,
+			Datasets:          effectiveDatasets,
+		},
+		TopData: top,
+		Extract: extract,
+		Autogen: EffectiveAutogenConfig{
+			Producers: slices.Clone(p.Config.Autogen.Producers),
+			Consumers: slices.Clone(p.Config.Autogen.Consumers),
+			Cache: EffectiveAutogenCacheConfig{
+				Root:                    expandPathTemplate(defaultString(p.Config.Autogen.Cache.Root, DefaultAutogenCacheRoot), paths),
+				MaxAge:                  defaultString(p.Config.Autogen.Cache.MaxAge, DefaultAutogenCacheMaxAge.String()),
+				RefreshEnv:              defaultString(p.Config.Autogen.Cache.RefreshEnv, DefaultAutogenRefreshEnv),
+				PartsManifestRefreshEnv: DefaultPartsManifestRefreshEnv,
+			},
+			ReleaseSource: EffectiveAutogenReleaseSourceConfig{
+				Provider:     defaultString(p.Config.Autogen.ReleaseSource.Provider, DefaultAutogenReleaseProvider),
+				ServerURLEnv: defaultString(p.Config.Autogen.ReleaseSource.ServerURLEnv, DefaultAutogenServerURLEnv),
+				RepoEnv:      defaultString(p.Config.Autogen.ReleaseSource.RepoEnv, DefaultAutogenRepoEnv),
+			},
+		},
+		HAKs:       slices.Clone(p.Config.HAKs),
+		Provenance: provenance,
+	}
+	effective.Overrides = activeOverrides(effective)
+	return effective
+}
+
+func cloneTopDataValueEncodings(values []TopDataValueEncodingConfig) []TopDataValueEncodingConfig {
+	if len(values) == 0 {
+		return nil
+	}
+	out := make([]TopDataValueEncodingConfig, len(values))
+	copy(out, values)
+	return out
+}
+
+func cloneTopDataValueDefaults(values []TopDataValueDefaultConfig) []TopDataValueDefaultConfig {
+	if len(values) == 0 {
+		return nil
+	}
+	out := make([]TopDataValueDefaultConfig, len(values))
+	copy(out, values)
+	return out
+}
+
+func cloneTopDataRowGeneration(values []TopDataRowGenerationConfig) []TopDataRowGenerationConfig {
+	if len(values) == 0 {
+		return nil
+	}
+	out := make([]TopDataRowGenerationConfig, len(values))
+	copy(out, values)
+	for index := range out {
+		if strings.TrimSpace(out[index].Dataset) == "" {
+			out[index].Dataset = strings.TrimSpace(out[index].Namespace)
+		}
+		if strings.TrimSpace(out[index].Namespace) == "" {
+			out[index].Namespace = strings.TrimSpace(out[index].Dataset)
+		}
+		if strings.TrimSpace(out[index].Mode) == "" {
+			out[index].Mode = "after_base"
+		}
+		out[index].Dataset = filepath.ToSlash(strings.TrimSpace(out[index].Dataset))
+		out[index].Namespace = filepath.ToSlash(strings.TrimSpace(out[index].Namespace))
+		out[index].Mode = strings.TrimSpace(out[index].Mode)
+	}
+	return out
+}
+
+func cloneTopDataRowExtensions(values []TopDataRowExtensionConfig) []TopDataRowExtensionConfig {
+	if len(values) == 0 {
+		return nil
+	}
+	out := make([]TopDataRowExtensionConfig, len(values))
+	copy(out, values)
+	for index := range out {
+		out[index].Dataset = filepath.ToSlash(strings.TrimSpace(out[index].Dataset))
+		out[index].Mode = strings.TrimSpace(out[index].Mode)
+		out[index].LevelColumn = strings.TrimSpace(out[index].LevelColumn)
+	}
+	return out
+}
+
+func cloneTopDataClassFeatInjections(value TopDataClassFeatInjectionConfig) TopDataClassFeatInjectionConfig {
+	out := TopDataClassFeatInjectionConfig{
+		GlobalFeats:           slices.Clone(value.GlobalFeats),
+		ClassSkillMasterfeats: slices.Clone(value.ClassSkillMasterfeats),
+	}
+	for index := range out.GlobalFeats {
+		out.GlobalFeats[index].RequirePresent = slices.Clone(value.GlobalFeats[index].RequirePresent)
+		out.GlobalFeats[index].UnlessPresent = slices.Clone(value.GlobalFeats[index].UnlessPresent)
+	}
+	return out
+}
+
+func (p *Project) EffectiveConfigJSON() ([]byte, error) {
+	return json.MarshalIndent(p.EffectiveConfig(), "", "  ")
+}
+
+func (p *Project) ExplainConfig(key string) (any, ConfigValueProvenance, bool) {
+	effective := p.EffectiveConfig()
+	value, ok := lookupEffectiveValue(effective, key)
+	if !ok {
+		return nil, ConfigValueProvenance{}, false
+	}
+	prov := effective.Provenance[key]
+	if prov.Source == "" {
+		prov = ConfigValueProvenance{Source: "yaml", Detail: p.ConfigSource.Name}
+	}
+	return value, prov, true
+}
+
+func lookupEffectiveValue(effective EffectiveConfig, key string) (any, bool) {
+	raw, err := json.Marshal(effective)
+	if err != nil {
+		return nil, false
+	}
+	var data any
+	if err := json.Unmarshal(raw, &data); err != nil {
+		return nil, false
+	}
+	current := data
+	for _, part := range strings.Split(key, ".") {
+		object, ok := current.(map[string]any)
+		if !ok {
+			return nil, false
+		}
+		current, ok = object[part]
+		if !ok {
+			return nil, false
+		}
+	}
+	return current, true
+}
+
+func markMissingDefaults(provenance ConfigProvenance) {
+	defaults := map[string]string{
+		"paths.build":                                    "build",
+		"paths.cache":                                    ".cache",
+		"paths.tools":                                    "tools",
+		"outputs.module_archive":                         DefaultModuleArchiveTemplate,
+		"outputs.hak_manifest":                           DefaultHAKManifest,
+		"outputs.hak_archive":                            DefaultHAKArchiveTemplate,
+		"generated_assets.topdata_2da":                   "no generated topdata 2DA assets",
+		"inventory.source_extensions":                    "NWN source resource extensions",
+		"inventory.asset_extensions":                     "NWN asset resource extensions",
+		"inventory.source_json_pattern":                  DefaultSourceJSONPattern,
+		"validation.profile":                             DefaultValidationProfile,
+		"validation.builtin_script_prefixes":             "NWN built-in script prefixes",
+		"validation.required_fields":                     "NWN required GFF fields",
+		"music.stage_root":                               DefaultMusicStageRoot,
+		"music.credits_root":                             DefaultCreditsRoot,
+		"music.credits_overlay":                          DefaultCreditsOverlayFile,
+		"music.max_stem_length":                          "16",
+		"music.naming_scheme":                            DefaultMusicNamingScheme,
+		"music.output_extension":                         DefaultMusicOutputExtension,
+		"music.convert_extensions":                       strings.Join(music.DefaultConvertExtensions(), ", "),
+		"topdata.build":                                  DefaultTopDataBuild,
+		"topdata.compiled_2da_dir":                       DefaultTopDataCompiled2DADir,
+		"topdata.compiled_tlk":                           DefaultTopDataCompiledTLK,
+		"topdata.package_hak":                            DefaultTopDataPackageHAK,
+		"topdata.package_tlk":                            DefaultTopDataPackageTLK,
+		"topdata.wiki.output_root":                       DefaultTopDataWikiOutputRoot,
+		"topdata.wiki.pages_dir":                         DefaultTopDataWikiPagesDir,
+		"topdata.wiki.state_file":                        DefaultTopDataWikiStateFile,
+		"topdata.wiki.managed_namespaces":                "NWN topdata wiki namespaces",
+		"topdata.wiki.deploy_manifest":                   DefaultWikiDeployManifest,
+		"topdata.wiki.deploy_edit_summary":               DefaultWikiDeployEditSummary,
+		"topdata.wiki.source":                            DefaultTopDataWikiSource,
+		"topdata.wiki.renderer":                          DefaultTopDataWikiRenderer,
+		"topdata.wiki.link_strategy":                     DefaultTopDataWikiLinkStrategy,
+		"topdata.wiki.namespaces_file":                   DefaultTopDataWikiNamespacesFile,
+		"topdata.wiki.tables_file":                       DefaultTopDataWikiTablesFile,
+		"topdata.wiki.visibility_file":                   DefaultTopDataWikiVisibilityFile,
+		"topdata.wiki.data_file":                         DefaultTopDataWikiDataFile,
+		"topdata.wiki.page_paths_file":                   DefaultTopDataWikiPagePathsFile,
+		"topdata.wiki.templates_dir":                     DefaultTopDataWikiTemplatesDir,
+		"topdata.wiki.page_templates_dir":                DefaultTopDataWikiPageTemplatesDir,
+		"topdata.wiki.manual_sections_dir":               DefaultTopDataWikiManualSectionsDir,
+		"topdata.wiki.manual_sections_file":              DefaultTopDataWikiManualSectionsFile,
+		"topdata.wiki.page_index_file":                   DefaultTopDataWikiPageIndexFile,
+		"topdata.wiki.title_prefix_min_length":           "3",
+		"topdata.wiki.status_pages":                      "true",
+		"topdata.wiki.status_listing_scope":              DefaultTopDataWikiStatusListingScope,
+		"topdata.wiki.alignment_links.link_format":       DefaultTopDataWikiAlignmentLinkFormat,
+		"topdata.wiki.stale_pages.default":               DefaultTopDataWikiStaleDefault,
+		"topdata.wiki.stale_pages.live_cleanup":          DefaultTopDataWikiStaleLiveCleanup,
+		"topdata.wiki.managed_region.marker_format":      DefaultTopDataWikiMarkerFormat,
+		"topdata.wiki.managed_region.page_marker_prefix": DefaultTopDataWikiPageMarkerPrefix,
+		"extract.layout":                                 DefaultExtractLayout,
+		"extract.hak_discovery":                          DefaultExtractHAKDiscovery,
+		"extract.archives":                               DefaultModuleArchiveTemplate,
+		"extract.cleanup_stale":                          "true",
+		"extract.consume_archives":                       "false",
+		"autogen.cache.root":                             DefaultAutogenCacheRoot,
+		"autogen.cache.max_age":                          DefaultAutogenCacheMaxAge.String(),
+		"autogen.cache.refresh_env":                      DefaultAutogenRefreshEnv,
+		"autogen.cache.parts_manifest_refresh_env":       DefaultPartsManifestRefreshEnv,
+		"autogen.release_source.provider":                DefaultAutogenReleaseProvider,
+		"autogen.release_source.server_url_env":          DefaultAutogenServerURLEnv,
+		"autogen.release_source.repo_env":                DefaultAutogenRepoEnv,
+	}
+	for key, detail := range defaults {
+		if _, ok := provenance[key]; !ok {
+			provenance[key] = ConfigValueProvenance{Source: "toolkit default", Detail: detail}
+		}
+	}
+}
+
+func cloneProvenance(input ConfigProvenance) ConfigProvenance {
+	out := ConfigProvenance{}
+	for key, value := range input {
+		out[key] = value
+	}
+	return out
+}
+
+func defaultString(value, fallback string) string {
+	trimmed := strings.TrimSpace(value)
+	if trimmed == "" {
+		return fallback
+	}
+	return trimmed
+}
+
+func defaultInt(value, fallback int) int {
+	if value == 0 {
+		return fallback
+	}
+	return value
+}
+
+func defaultBoolPointer(value *bool, fallback bool) *bool {
+	out := fallback
+	if value != nil {
+		out = *value
+	}
+	return &out
+}
+
+func defaultStringSlice(value, fallback []string) []string {
+	if len(value) == 0 {
+		return slices.Clone(fallback)
+	}
+	return slices.Clone(value)
+}
+
+func defaultWikiPageTemplatesDir(cfg TopDataWikiConfig) string {
+	if strings.TrimSpace(cfg.PageTemplatesDir) != "" {
+		return strings.TrimSpace(cfg.PageTemplatesDir)
+	}
+	if strings.TrimSpace(cfg.TemplatesDir) != "" {
+		return filepath.ToSlash(filepath.Join(filepath.FromSlash(strings.TrimSpace(cfg.TemplatesDir)), "pages"))
+	}
+	return DefaultTopDataWikiPageTemplatesDir
+}
+
+func defaultWikiManualSectionsFile(cfg TopDataWikiConfig) string {
+	if strings.TrimSpace(cfg.ManualSectionsFile) != "" {
+		return strings.TrimSpace(cfg.ManualSectionsFile)
+	}
+	if strings.TrimSpace(cfg.ManualSectionsDir) != "" {
+		return filepath.ToSlash(filepath.Join(filepath.FromSlash(strings.TrimSpace(cfg.ManualSectionsDir)), "default.yaml"))
+	}
+	return DefaultTopDataWikiManualSectionsFile
+}
+
+func defaultRequiredFields(configured map[string][]string) map[string][]string {
+	if len(configured) > 0 {
+		return cloneStringSliceMap(normalizeRequiredFields(configured))
+	}
+	return DefaultRequiredFields()
+}
+
+func cloneStringSliceMap(input map[string][]string) map[string][]string {
+	out := make(map[string][]string, len(input))
+	for key, values := range input {
+		out[key] = slices.Clone(values)
+	}
+	return out
+}
+
+func cloneStringMap(input map[string]string) map[string]string {
+	if len(input) == 0 {
+		return nil
+	}
+	out := make(map[string]string, len(input))
+	for key, value := range input {
+		out[key] = value
+	}
+	return out
+}
+
+func expandPathTemplate(template string, paths EffectivePathConfig) string {
+	replacer := strings.NewReplacer(
+		"{paths.source}", paths.Source,
+		"{paths.assets}", paths.Assets,
+		"{paths.build}", paths.Build,
+		"{paths.cache}", paths.Cache,
+		"{paths.tools}", paths.Tools,
+	)
+	return filepath.ToSlash(replacer.Replace(template))
+}
+
+func expandPathTemplates(templates []string, paths EffectivePathConfig) []string {
+	out := make([]string, 0, len(templates))
+	for _, template := range templates {
+		out = append(out, expandPathTemplate(template, paths))
+	}
+	return out
+}
+
+func (p *Project) ActiveOverrides() []ConfigOverride {
+	return activeOverrides(p.EffectiveConfig())
+}
+
+func activeOverrides(effective EffectiveConfig) []ConfigOverride {
+	envs := map[string]string{
+		"build.keep_existing_haks":             effectiveBuildKeepExistingEnv,
+		"autogen.cache.refresh":                effective.Autogen.Cache.RefreshEnv,
+		"autogen.cache.parts_manifest_refresh": effective.Autogen.Cache.PartsManifestRefreshEnv,
+		"autogen.release_source.server_url":    effective.Autogen.ReleaseSource.ServerURLEnv,
+		"autogen.release_source.repo":          effective.Autogen.ReleaseSource.RepoEnv,
+		"music.ffmpeg":                         "SOW_FFMPEG",
+		"music.ffprobe":                        "SOW_FFPROBE",
+		"wiki.endpoint":                        "NODEBB_API_ENDPOINT",
+		"wiki.token":                           "NODEBB_API_TOKEN",
+		"wiki.categories":                      "NODEBB_WIKI_CATEGORIES",
+		"release.version":                      "GITHUB_REF_NAME",
+	}
+	keys := make([]string, 0, len(envs))
+	for key := range envs {
+		keys = append(keys, key)
+	}
+	slices.Sort(keys)
+	overrides := make([]ConfigOverride, 0)
+	for _, key := range keys {
+		env := envs[key]
+		if value := strings.TrimSpace(os.Getenv(env)); value != "" {
+			if isSensitiveOverrideKey(key) || isSensitiveOverrideKey(env) {
+				value = "<set>"
+			}
+			overrides = append(overrides, ConfigOverride{Key: key, Value: value, Source: "env:" + env})
+		}
+	}
+	return overrides
+}
+
+const effectiveBuildKeepExistingEnv = "SOW_BUILD_HAKS_KEEP_EXISTING"
+
+func isSensitiveOverrideKey(key string) bool {
+	lower := strings.ToLower(key)
+	return strings.Contains(lower, "token") || strings.Contains(lower, "password")
+}
+
+func (o EffectiveOutputConfig) ModuleArchiveName(module ModuleConfig) string {
+	return strings.NewReplacer("{module.resref}", strings.TrimSpace(module.ResRef)).Replace(o.ModuleArchive)
+}
+
+func (o EffectiveOutputConfig) HAKArchiveName(name string) string {
+	return strings.NewReplacer("{hak.name}", strings.TrimSpace(name)).Replace(o.HAKArchive)
+}
+
+func (p ConfigValueProvenance) String() string {
+	if p.Detail == "" {
+		return p.Source
+	}
+	return fmt.Sprintf("%s (%s)", p.Source, p.Detail)
+}

internal/topdata/AUTO-INCLUDE_EXISTING_PART_MODELS_IN_2DA_GENERATION_CONTRACT.md

@@ -0,0 +1,155 @@
+# AI Agent Contract: Auto-Include Existing Part Models in 2DA Generation
+
+## Status Snapshot
+
+Current state as of 2026-05-17:
+
+- Superseded for current Shadows Over Westgate builds.
+- The module no longer consumes a released parts manifest. Parts data is owned by
+  `assets/`, generated as HAK-side 2DA assets from `assets/topdata/data/parts`
+  plus a direct scan of `assets/content/part/**/*.mdl`, and packaged in
+  `sow_part`.
+- This file remains as historical context for the legacy manifest-backed
+  consumer path.
+
+## Objective
+
+The native 2DA generation pipeline must automatically include existing part models already present in the **`sow-assets` repository** when building final parts tables.
+
+This behavior is required for parity with the current asset set and must not depend on manually authored entries for models that already exist in repository assets.
+
+---
+
+## Source of Truth
+
+The superseded manifest-backed native builder obtained existing model
+information from a released artifact published by the **`sow-assets` repository**.
+
+Resolution rules:
+
+- if `topdata.assets` is configured to a local path, use that exact path
+- otherwise derive the **`sow-assets` repository** from the module repo's Gitea origin
+  and fetch only the configured released manifest
+
+Normal builds must not clone or scan the `sow-assets` repo tree. They may cache the
+downloaded manifest locally for reuse, but the source of truth remains the released
+manifest on Gitea.
+
+Manifest-backed discovery is derived from:
+
+```text
+sow-assets/assets/part/**/*.mdl
+```
+
+The manifest generator may also accept the repo's legacy `assets/parts/` layout when
+resolving the physical paths, but the native builder itself must consume the manifest,
+not the repo tree.
+
+---
+
+## Required Scope
+
+Process existing `.mdl` files for these parts table categories only:
+
+- `belt`
+- `bicep`
+- `chest`
+- `foot`
+- `forearm`
+- `leg`
+- `neck`
+- `pelvis`
+- `robe`
+- `shin`
+- `shoulder`
+
+Dataset mapping rules:
+
+- `parts/legs` maps to asset category `leg`
+- `parts/hand` remains unsupported and is not auto-generated
+
+The scan must recurse through subdirectories under each applicable category path.
+
+---
+
+## Required Behavior
+
+### 1. Manifest-backed discovery
+
+For each supported part category, read the released manifest and discover all existing
+numeric model IDs listed for that category.
+
+### 2. Trailing numeric suffix extraction
+
+The released manifest must be generated using trailing-numeric-suffix extraction from
+the source `.mdl` filenames. Native topdata consumes the resulting numeric IDs only.
+
+### 3. Row generation
+
+If a trailing numeric suffix is present:
+
+- use that numeric suffix as the output row ID
+- emit a row for that part model in the final generated 2DA
+
+### 4. Default emitted values
+
+For rows generated from discovered existing models, set:
+
+- `COSTMODIFIER = 0`
+- `ACBONUS = 0.00`
+
+Rows without discovered existing models may be intentionally emitted as dense
+2DA null rows when configured. In that mode, missing rows must have
+`COSTMODIFIER = ****` and `ACBONUS = ****` in the final generated 2DA.
+
+### 5. Override precedence
+
+If file-based overrides exist, they must take precedence over discovered defaults.
+
+That means:
+
+- manifest discovery provides the baseline row presence and default values
+- override data may replace or augment those values
+- override data wins in any conflict
+
+---
+
+## Non-Negotiable Requirements
+
+- The implementation must read model presence from the **released parts manifest**,
+  not from assumption, hardcoded lists, or legacy output.
+- Existing repository models with trailing numeric suffixes must be reflected in generated output even if they are not otherwise explicitly authored.
+- Override application is mandatory and must win over auto-discovered defaults.
+- This is replacement behavior for the native pipeline, not an optional enhancement.
+
+---
+
+## Acceptance Criteria
+
+The superseded implementation was correct only if all of the following were true:
+
+1. The builder fetches the configured released manifest artifact for part models.
+2. Only the supported part categories are considered.
+3. Manifest IDs derived from trailing numeric suffixes generate corresponding 2DA rows.
+4. The numeric suffix is used as the row ID.
+5. Auto-generated rows default to:
+   - `COSTMODIFIER = 0`
+   - `ACBONUS = 0.00`
+
+6. Authored base rows remain exposed. Authored `ACBONUS = ****` rows remain
+   null unless model discovery activates the row.
+7. Dense gaps without authored rows or discovered models are emitted as null
+   rows with `COSTMODIFIER = ****` and `ACBONUS = ****`.
+8. File overrides are applied after discovery and take precedence.
+9. The final output includes all eligible existing models present in the manifest.
+10. Discovery does not require a sibling local `sow-assets` checkout or a Git clone.
+
+---
+
+## Implementation Directive
+
+When generating native parts 2DAs, first scan the project-resolved assets repo for
+supported part models, collect existing supported part models by category, extract
+trailing numeric suffixes from filenames, emit rows using those suffixes as row IDs
+with default `COSTMODIFIER = 0` and `ACBONUS = 0.00`, then apply overrides with
+override values taking precedence.

internal/topdata/CLASS_FEAT_GLOBAL_INJECTS_CONTRACT.md

@@ -0,0 +1,76 @@
+# Class Feat Global Injection Contract
+
+## Status Snapshot
+
+Current state as of 2026-06-05:
+
+- Preferred class feat injection policy is authored in
+  `topdata/data/classes/feats/global.json`.
+- `global.json` rows flow through the ordinary class feat expansion pipeline,
+  including successor expansion, masterfeat expansion, class-skill filtering,
+  reference resolution, deduplication, and final 2DA emission.
+- Legacy `topdata.class_feat_injections` YAML still loads only as a
+  compatibility path when no applicable class feat `global.json` exists.
+- Toolkit hardcoded defaults are compatibility-only and must not be the normal
+  source of project policy.
+
+## Objective
+
+Native class feat generation must produce deterministic, legacy-equivalent
+effective output while keeping project policy in topdata authoring files.
+
+## Requirements
+
+- For each class skill, inject the configured skill-focus masterfeat rows from
+  `classes/feats/global.json` using `FeatIndex.filter = "classskills"`.
+- Inject `feat:literate` only when the class table does not already include
+  `feat:illiterate`.
+- Inject the shared combat/menu feat rows declared in the manifest unless a
+  class already authors the equivalent `FeatIndex`.
+- Explicit class feat rows take precedence over injected rows.
+- Injections must be deterministic for identical inputs.
+
+## Manifest Shape
+
+```json
+{
+  "position": "prepend",
+  "injections": [
+    {
+      "row": {
+        "FeatIndex": { "id": "feat:literate" },
+        "GrantedOnLevel": 1,
+        "List": 3,
+        "OnMenu": 0
+      },
+      "unless_present": [
+        { "field": "FeatIndex", "id": "feat:illiterate" }
+      ]
+    },
+    {
+      "row": {
+        "FeatIndex": {
+          "id": "masterfeats:skill_focus",
+          "filter": "classskills"
+        },
+        "GrantedOnLevel": -1,
+        "List": 1,
+        "OnMenu": 0
+      }
+    }
+  ]
+}
+```
+
+`position` defaults to `append`. The module uses `prepend` for class feat
+injections to keep generated policy rows before per-class authored rows.
+
+## Acceptance Criteria
+
+- Class feat output contains the same effective shared feats as the prior YAML
+  policy.
+- Literacy injection is skipped for illiterate classes.
+- Skill-focus masterfeat rows expand only for class skills.
+- YAML compatibility continues to work for consumers that have not migrated.
+- A project with `classes/feats/global.json` does not also receive YAML or
+  hardcoded default class feat injections.

internal/topdata/FAMILY_EXPANSION_CONTRACT.md

@@ -0,0 +1,119 @@
+# Generic Family Expansion Contract
+
+## Status Snapshot
+
+Current state as of 2026-05-13:
+
+- Implemented and actively used by native feat generation.
+- `family_expansion.go`, `native.go`, and `topdata_test.go` cover the current
+  underscore-family interpretation, generated family metadata, and reapplication
+  behavior.
+- Remaining work is mostly consumer expansion: use the same primitive in more
+  datasets only when the authored data actually benefits from it.
+
+Topdata now treats underscore expansion as a structural rule:
+
+- `parent_child` means `child` is an expansion of `parent`
+- underscores are for family structure, not simulated spaces
+- existing canonical keys still win when lock or TLK state already established them
+
+This is a global interpretation rule, not a wiki-only convention.
+
+## Identity
+
+- `weaponspecialization_club`
+  - parent: `weaponspecialization`
+  - child: `club`
+- `gnome_rock`
+  - parent: `gnome`
+  - child: `rock`
+- `toughness_10`
+  - parent: `toughness`
+  - child: `10`
+
+Standalone keys without an underscore are treated as a parent identity with no child.
+
+## Generated Family Files
+
+Canonical generated families declare:
+
+```json
+{
+  "family": "weapon_focus",
+  "family_key": "weaponfocus",
+  "template": "masterfeats:weaponfocus",
+  "name_prefix": "Weapon Focus",
+  "label_prefix": "FEAT_WEAPON_FOCUS",
+  "constant_prefix": "FEAT_WEAPON_FOCUS",
+  "legacy_family_keys": ["epic_weapon_focus"],
+  "default_fields": {
+    "DESCRIPTION": {
+      "tlk": {
+        "key": "masterfeats:weaponfocus.description",
+        "text": "..."
+      }
+    },
+    "MINATTACKBONUS": "1",
+    "TOOLSCATEGORIES": "1"
+  },
+  "apply_after_modules": true,
+  "identity_source": "child_source_value",
+  "auto_prereq_fields": {
+    "PREREQFEAT1": "weaponfocus"
+  },
+  "child_source": {
+    "dataset": "baseitems",
+    "column": "WeaponFocusFeat"
+  }
+}
+```
+
+Rules:
+
+- `family` is an authored label and is not a hardcoded switch key
+- `family_key` is the structural parent identity used in generated child keys
+- `template` is optional for the primitive in general, but required by current
+  masterfeat-backed `feat` families
+- `name_prefix`, `label_prefix`, and `constant_prefix` define how new child rows are
+  named when they are not already authored
+- `template_fields` lists template-backed fields copied from the referenced template row
+  when authored; it is optional if the family owns its shared values directly
+- `default_fields` applies authored shared/default values to every generated family row
+  and can be used to move family-wide shared values out of the template row
+- `apply_after_modules: true` re-applies the generated family after normal module files,
+  so authored family-shared values remain authoritative even for legacy explicitly
+  authored child rows
+- `child_ref_field` binds the generated child row back to the source row key, e.g.
+  `REQSKILL`
+- `identity_source: "child_source_value"` tells the builder to reuse IDs/keys from the
+  source column when that column already carries feat identity
+- `legacy_family_keys` declares previous generated-family prefixes that should donate
+  existing locked row IDs to the current family when a family is renamed
+- `auto_prereq_fields` binds prerequisite fields to other authored families by
+  `family_key`, without code changes
+- `allow_existing_only` limits expansion to already-authored child rows when a family is
+  intentionally partial
+- `child_source.dataset` identifies the canonical dataset driving expansion
+- `child_source.column` is used when expansion is gated by a non-null source field
+- `child_source.predicate` is used when expansion depends on a named rule such as
+  accessibility
+
+## Row Metadata
+
+Generated rows can carry:
+
+```json
+{
+  "meta": {
+    "family": {
+      "parent": "weaponfocus",
+      "child": "club",
+      "source": "baseitems:club",
+      "template": "masterfeats:weaponfocus"
+    }
+  }
+}
+```
+
+This metadata is builder-owned and does not affect emitted 2DA columns. It preserves
+family structure for later phases without enabling wiki generation yet.

internal/topdata/FEAT_GENERATED_FAMILIES_CONTRACT.md

@@ -0,0 +1,174 @@
+# `feat` Generated Families Contract
+
+## Status Snapshot
+
+Current state as of 2026-05-13:
+
+- Implemented for the current native `feat` pipeline.
+- Native tests cover generated-family expansion, family-owned shared defaults,
+  masterfeat-backed inheritance, and lock/TLK identity preservation.
+- The one explicitly deferred item in this contract is still current:
+  `feat.2da` remains native-built but excluded from `compare-topdata`
+  reference-catalog coverage through `compare_reference: false`.
+
+Canonical authored paths:
+
+- `topdata/data/feat/base.json`
+- `topdata/data/feat/lock.json`
+- `topdata/data/feat/generated/skill_focus.json`
+- `topdata/data/feat/generated/greater_skill_focus.json`
+- `topdata/data/feat/generated/weapon_focus.json`
+- `topdata/data/feat/generated/weapon_specialization.json`
+- `topdata/data/feat/generated/greater_weapon_focus.json`
+- `topdata/data/feat/generated/greater_weapon_specialization.json`
+- `topdata/data/feat/generated/improved_critical.json`
+- `topdata/data/feat/generated/overwhelming_critical.json`
+- `topdata/data/feat/modules/activecombat/core.json`
+- `topdata/data/feat/modules/activecombat/specialattacks.json`
+- `topdata/data/feat/modules/class/core.json`
+- `topdata/data/feat/modules/combat/core.json`
+- `topdata/data/feat/modules/defensive/core.json`
+- `topdata/data/feat/modules/magical/core.json`
+- `topdata/data/feat/modules/other/core.json`
+- `topdata/data/feat/modules/proficiency/core.json`
+- `topdata/data/feat/modules/racial/core.json`
+- `topdata/data/feat/modules/removedandhidden/core.json`
+- `topdata/data/feat/modules/skillfeat/affinity.json`
+- `topdata/data/feat/modules/skillfeat/focus.json`
+- `topdata/data/feat/modules/skillfeat/greaterfocus.json`
+- `topdata/data/feat/modules/skillfeat/other.json`
+
+`skill_affinity` is no longer authored as a generated feat file. It is synthesized from
+canonical `racialtypes` race grants.
+
+Compact family-expansion shape:
+
+```json
+{
+  "family": "skill_focus",
+  "family_key": "skillfocus",
+  "template": "masterfeats:skillfocus",
+  "name_prefix": "Skill Focus",
+  "label_prefix": "FEAT_SKILL_FOCUS",
+  "constant_prefix": "FEAT_SKILL_FOCUS",
+  "default_fields": {
+    "DESCRIPTION": {
+      "tlk": {
+        "key": "masterfeats:skillfocus.description",
+        "text": "..."
+      }
+    },
+    "CRValue": "0.5",
+    "ReqSkillMinRanks": "1",
+    "TOOLSCATEGORIES": "6"
+  },
+  "apply_after_modules": true,
+  "child_ref_field": "REQSKILL",
+  "child_source": {
+    "dataset": "skills",
+    "predicate": "accessible"
+  },
+  "title_style": {
+    "child_case": "lower",
+    "child_parenthetical": "comma"
+  },
+  "overrides": {
+    "skills:craft_alchemy": {
+      "ICON": "ife_foc_alchm"
+    }
+  }
+}
+```
+
+Weapon family shape:
+
+```json
+{
+  "family": "weapon_focus",
+  "family_key": "weaponfocus",
+  "template": "masterfeats:weaponfocus",
+  "name_prefix": "Weapon Focus",
+  "label_prefix": "FEAT_WEAPON_FOCUS",
+  "constant_prefix": "FEAT_WEAPON_FOCUS",
+  "legacy_family_keys": ["epic_weapon_focus"],
+  "default_fields": {
+    "DESCRIPTION": {
+      "tlk": {
+        "key": "masterfeats:weaponfocus.description",
+        "text": "..."
+      }
+    },
+    "MINATTACKBONUS": "1",
+    "TOOLSCATEGORIES": "1"
+  },
+  "apply_after_modules": true,
+  "identity_source": "child_source_value",
+  "child_source": {
+    "dataset": "baseitems",
+    "column": "WeaponFocusFeat"
+  },
+  "title_style": {
+    "child_case": "lower",
+    "child_parenthetical": "preserve"
+  },
+  "overrides": {
+    "baseitems:heavymace": {
+      "ICON": "ife_wepfoc_Lma"
+    }
+  }
+}
+```
+
+Rules:
+
+- any generated feat file with family-expansion fields is treated as an authored
+  family-expansion definition
+- `family` is descriptive only; the builder does not hardcode family names
+- family behavior is driven by authored fields such as `template_fields`,
+  `default_fields`, `apply_after_modules`, `identity_source`, `child_ref_field`,
+  `auto_prereq_fields`, and optional `title_style`
+- `default_fields` is a family-wide shared layer; it applies to existing and newly
+  generated child rows before per-child overrides are merged
+- `apply_after_modules: true` makes the generated family the final authoritative shared
+  layer for legacy explicitly authored child rows, so shared baselines can live in the
+  generated family file instead of `masterfeats` overrides
+- `legacy_family_keys` declares old generated-family prefixes whose existing row IDs
+  should be inherited by the current family; this is how renamed families such as
+  `greater_skill_focus` retaining vanilla `epic_skill_focus` rows preserve hardcoded
+  engine behavior without hardcoding the rename in toolkit logic
+- family-expansion files must declare `template`, `family_key`, and `child_source.dataset`
+- `child_source.column` is used when expansion is gated by a non-null source field
+- `child_source.predicate: "accessible"` currently means `HideFromLevelUp != 1`
+- `title_style` only changes generated child display text while composing titles;
+  generated feat keys, TLK keys, row IDs, family metadata, and source references
+  remain tied to generated family identity
+- generated family title text is authoritative for family-managed existing rows
+  as well as newly allocated rows, so in-game TLK names and generated wiki titles
+  are derived from the same normalized `FEAT` text
+- `title_style.child_case` defaults to `preserve`; `lower` lowercases the resolved
+  child display text after parenthetical formatting
+- `title_style.child_parenthetical` defaults to `preserve`; `comma` flattens one
+  child parenthetical suffix such as `Knowledge (local)` into
+  `Knowledge, local` before the child text is wrapped by the family title
+- unsupported `title_style` values fail generated-family validation instead of
+  silently changing title behavior
+- skill focus families intentionally do not set `allow_existing_only`, so any new
+  accessible skill automatically receives matching focus rows
+- compact `overrides` are keyed by source dataset key:
+  - `skills:*` for skill families
+  - `baseitems:*` for weapon families
+- generated rows carry `meta.family` so the underscore family rule is preserved even
+  before wiki generation exists
+- emitted feat keys must prefer existing canonical lock/TLK identities over newly
+  derived child spellings
+- manual and irregular feat content continues to live under `feat/modules/`
+- generated families and module-authored feats are merged into the same final `feat.2da`
+  pipeline and share inheritance, override, TLK, and family-metadata behavior
+
+Current rollout policy:
+
+- `feat.2da` still builds natively
+- `feat.2da` is still excluded from `compare-topdata` output-catalog self-check coverage
+  via `compare_reference: false`
+- manual and irregular feat families remain authored directly in modules rather than
+  compact generated-family files

internal/topdata/INHERITANCE_CONTRACT.md

@@ -0,0 +1,78 @@
+# Topdata Inheritance Contract
+
+## Status Snapshot
+
+Current state as of 2026-05-13:
+
+- Implemented.
+- `native.go` supports both row-sugar inheritance and generic object
+  inheritance, including cycle detection and missing-target failures.
+- Validation and build regression coverage live in `topdata_test.go`.
+
+Native topdata now supports two inheritance forms:
+
+## 1. Row Sugar Compatibility
+
+This is the existing narrow row helper:
+
+```json
+{
+  "PARENT": { "id": "custom:parent" },
+  "inherit": {
+    "from": "PARENT",
+    "fields": ["VALUE", "ICON"]
+  }
+}
+```
+
+It remains supported for row-local field copying.
+
+## 2. Generic Object Inheritance
+
+This is the new reusable primitive:
+
+```json
+{
+  "inherit": {
+    "ref": "masterfeats:skillfocus"
+  },
+  "LABEL": "FEAT_SKILL_FOCUS_ATHLETICS"
+}
+```
+
+Or for a nested object:
+
+```json
+{
+  "DETAILS": {
+    "inherit": {
+      "ref": "custom:parent",
+      "field": "DETAILS"
+    },
+    "meta": {
+      "cost": "9"
+    }
+  }
+}
+```
+
+## Rules
+
+- `inherit.ref` must use stable `dataset:key` identity.
+- `inherit.field` is optional and selects an object-valued field on the target row.
+- Generic inheritance can appear on any authored object, not only top-level rows.
+- Merge precedence is `local overrides inherited`.
+- Scalars replace inherited values.
+- Arrays replace inherited values.
+- Objects merge recursively unless the local object is an atomic topdata value object such as:
+  - TLK payloads
+  - row refs
+  - table refs
+- Cycles fail the build.
+- Missing targets fail the build.
+
+## Current Intended Consumer
+
+`feat` is the first dataset expected to use this primitive heavily, especially for
+borrowing shared properties from `masterfeats` while keeping `feat`'s own dataset
+contract and row model.

internal/topdata/MASTERFEATS_CONTRACT.md

@@ -0,0 +1,82 @@
+# `masterfeats` Canonical Contract
+
+## Status Snapshot
+
+Current state as of 2026-05-13:
+
+- Implemented as a first-class native dataset.
+- The repository contains `topdata/data/masterfeats/`, migration/import support,
+  validator checks, and native output coverage for `masterfeats.2da`.
+- Remaining work is only downstream adoption and regression maintenance as other
+  native datasets continue to consume `masterfeats` more heavily.
+
+`masterfeats` is a stable native canonical family consumed directly by `feat`
+generation, class-feat expansion, and inheritance/ref resolution.
+
+Current canonical scope:
+
+- dataset root: `topdata/data/masterfeats/`
+- required files:
+  - `base.json`
+  - `lock.json`
+- generated output:
+  - `masterfeats.2da`
+
+## `base.json`
+
+Canonical shape:
+
+```json
+{
+  "output": "masterfeats.2da",
+  "columns": ["LABEL", "STRREF", "DESCRIPTION", "..."],
+  "rows": [
+    {
+      "id": 0,
+      "key": "masterfeats:weaponfocus",
+      "LABEL": "WeaponFocus",
+      "STRREF": "6490",
+      "DESCRIPTION": "436"
+    }
+  ]
+}
+```
+
+Rules:
+
+- `output` may be omitted; native dataset discovery deterministically derives
+  `masterfeats.2da` from `topdata/data/masterfeats/base.json`
+- if `output` is present, it must be exactly `masterfeats.2da`
+- `columns` must include `LABEL`, `STRREF`, and `DESCRIPTION`
+- every canonical row must have a non-empty `key`
+- row keys must start with `masterfeats:`
+- inline TLK payloads are allowed in `STRREF` and `DESCRIPTION`
+- numeric/text references are both allowed where the native TLK compiler already supports them
+
+## `lock.json`
+
+Canonical shape:
+
+```json
+{
+  "masterfeats:weaponfocus": 0
+}
+```
+
+Rules:
+
+- every key must start with `masterfeats:`
+- every value must be numeric
+- the lockfile remains the canonical stable id source for authored keys
+
+## Module contributions
+
+This family continues to support the shared canonical module shapes already used by native
+datasets:
+
+- `columns`
+- `entries`
+- `overrides`
+
+Those shapes are validated generically in `topdata.go`; this contract only adds the
+family-specific guarantees for `masterfeats`.

internal/topdata/appearance_migrate.go

@@ -0,0 +1,96 @@
+package topdata
+
+import (
+	"encoding/json"
+	"os"
+	"path/filepath"
+)
+
+func importLegacyAppearance(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
+	_ = legacyTLK
+
+	legacyDir := filepath.Join(referenceBuilderDir, "data", "appearance")
+	if _, err := os.Stat(legacyDir); err != nil {
+		if os.IsNotExist(err) {
+			return 0, nil
+		}
+		return 0, err
+	}
+
+	targetDir := filepath.Join(dataDir, "appearance")
+	targetBasePath := filepath.Join(targetDir, "base.json")
+	targetLockPath := filepath.Join(targetDir, "lock.json")
+	targetModulesDir := filepath.Join(targetDir, "modules")
+	moduleNames := []string{
+		"cotblreaver.json",
+		"crawlingclaw.json",
+		"halfogre.json",
+		"zombieknight.json",
+	}
+	targetModulePaths := make([]string, 0, len(moduleNames))
+	for _, name := range moduleNames {
+		targetModulePaths = append(targetModulePaths, filepath.Join(targetModulesDir, name))
+	}
+	if fileExists(targetBasePath) && fileExists(targetLockPath) {
+		allModulesPresent := true
+		for _, path := range targetModulePaths {
+			if !fileExists(path) {
+				allModulesPresent = false
+				break
+			}
+		}
+		if allModulesPresent {
+			return 0, nil
+		}
+	}
+
+	baseObj, err := loadJSONObject(filepath.Join(legacyDir, "base.json"))
+	if err != nil {
+		return 0, err
+	}
+	baseObj["output"] = "appearance.2da"
+
+	lockObj, err := loadJSONObject(filepath.Join(legacyDir, "lock.json"))
+	if err != nil {
+		return 0, err
+	}
+
+	moduleObjs := make([]map[string]any, 0, len(moduleNames))
+	for _, name := range moduleNames {
+		obj, err := loadJSONObject(filepath.Join(legacyDir, "modules", name))
+		if err != nil {
+			return 0, err
+		}
+		moduleObjs = append(moduleObjs, obj)
+	}
+
+	if err := os.MkdirAll(targetModulesDir, 0o755); err != nil {
+		return 0, err
+	}
+
+	writes := []struct {
+		path string
+		obj  map[string]any
+	}{
+		{path: targetBasePath, obj: baseObj},
+		{path: targetLockPath, obj: lockObj},
+	}
+	for i, path := range targetModulePaths {
+		writes = append(writes, struct {
+			path string
+			obj  map[string]any
+		}{path: path, obj: moduleObjs[i]})
+	}
+
+	for _, write := range writes {
+		raw, err := json.MarshalIndent(write.obj, "", "  ")
+		if err != nil {
+			return 0, err
+		}
+		raw = append(raw, '\n')
+		if err := os.WriteFile(write.path, raw, 0o644); err != nil {
+			return 0, err
+		}
+	}
+	return len(writes), nil
+}

internal/topdata/armor_migrate.go

@@ -0,0 +1,160 @@
+package topdata
+
+import (
+	"encoding/json"
+	"os"
+	"path/filepath"
+	"sort"
+)
+
+func importLegacyArmor(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
+	_ = legacyTLK
+
+	legacyDir := filepath.Join(referenceBuilderDir, "data", "armor")
+	if _, err := os.Stat(legacyDir); err != nil {
+		if os.IsNotExist(err) {
+			return 0, nil
+		}
+		return 0, err
+	}
+
+	targetDir := filepath.Join(dataDir, "armor")
+	targetPath := filepath.Join(targetDir, "armor.json")
+	if _, err := os.Stat(targetPath); err == nil {
+		obj, err := loadJSONObject(targetPath)
+		if err == nil && countLegacyTLKRefsInValue(obj) == 0 {
+			return 0, nil
+		}
+	}
+
+	baseObj, err := loadJSONObject(filepath.Join(legacyDir, "base.json"))
+	if err != nil {
+		return 0, err
+	}
+	rows, err := mergeArmorRows(baseObj, filepath.Join(legacyDir, "modules"))
+	if err != nil {
+		return 0, err
+	}
+
+	if err := os.MkdirAll(targetDir, 0o755); err != nil {
+		return 0, err
+	}
+	if err := removePathIfExists(filepath.Join(targetDir, "base.json")); err != nil {
+		return 0, err
+	}
+	if err := removePathIfExists(filepath.Join(targetDir, "lock.json")); err != nil {
+		return 0, err
+	}
+	if err := removePathIfExists(filepath.Join(targetDir, "modules")); err != nil {
+		return 0, err
+	}
+
+	out := map[string]any{
+		"output":  "armor.2da",
+		"columns": baseObj["columns"],
+		"rows":    rows,
+	}
+	raw, err := json.MarshalIndent(out, "", "  ")
+	if err != nil {
+		return 0, err
+	}
+	raw = append(raw, '\n')
+	if err := os.WriteFile(targetPath, raw, 0o644); err != nil {
+		return 0, err
+	}
+	return 1, nil
+}
+
+func mergeArmorRows(baseObj map[string]any, modulesDir string) ([]any, error) {
+	rawRows, ok := baseObj["rows"].([]any)
+	if !ok {
+		return nil, nil
+	}
+
+	rows := make([]map[string]any, 0, len(rawRows))
+	byID := map[int]map[string]any{}
+	for _, raw := range rawRows {
+		row, ok := deepCopyValue(raw).(map[string]any)
+		if !ok {
+			continue
+		}
+		delete(row, "key")
+		rawID, ok := row["id"]
+		if !ok {
+			continue
+		}
+		id, err := asInt(rawID)
+		if err != nil {
+			return nil, err
+		}
+		row["id"] = id
+		rows = append(rows, row)
+		byID[id] = row
+	}
+
+	modulePaths, err := collectModulePaths(modulesDir)
+	if err != nil {
+		return nil, err
+	}
+	for _, path := range modulePaths {
+		obj, err := loadJSONObject(path)
+		if err != nil {
+			return nil, err
+		}
+		if entries, ok := obj["entries"].(map[string]any); ok {
+			for _, raw := range entries {
+				row, ok := deepCopyValue(raw).(map[string]any)
+				if !ok {
+					continue
+				}
+				delete(row, "key")
+				rawID, ok := row["id"]
+				if !ok {
+					continue
+				}
+				id, err := asInt(rawID)
+				if err != nil {
+					return nil, err
+				}
+				row["id"] = id
+				rows = append(rows, row)
+				byID[id] = row
+			}
+		}
+		if overrides, ok := obj["overrides"].([]any); ok {
+			for _, raw := range overrides {
+				override, ok := raw.(map[string]any)
+				if !ok {
+					continue
+				}
+				rawID, ok := override["id"]
+				if !ok {
+					continue
+				}
+				id, err := asInt(rawID)
+				if err != nil {
+					return nil, err
+				}
+				row := byID[id]
+				if row == nil {
+					continue
+				}
+				for key, value := range override {
+					if key == "id" || key == "key" || key == "_tlk" || isMetadataField(key) {
+						continue
+					}
+					row[key] = deepCopyValue(value)
+				}
+			}
+		}
+	}
+
+	sort.Slice(rows, func(i, j int) bool {
+		return rows[i]["id"].(int) < rows[j]["id"].(int)
+	})
+	out := make([]any, 0, len(rows))
+	for _, row := range rows {
+		out = append(out, row)
+	}
+	return out, nil
+}

internal/topdata/assets_repo.go

@@ -0,0 +1,46 @@
+package topdata
+
+import (
+	"fmt"
+	"os"
+	"os/exec"
+	"strings"
+
+	"git.westgate.pw/ShadowsOverWestgate/sow-tools/internal/project"
+)
+
+func resolvePartsAssetsOverrideDir(p *project.Project) (string, error) {
+	spec := strings.TrimSpace(p.Config.TopData.Assets)
+	if spec == "" {
+		return "", nil
+	}
+	if looksLikeGitRepoSpec(spec) {
+		return "", fmt.Errorf("topdata.assets no longer supports git repo specs for parts discovery; use a local path override or the default released manifest")
+	}
+	path := p.TopDataAssetsDir()
+	if path == "" {
+		return "", nil
+	}
+	if _, err := os.Stat(path); err != nil {
+		return "", fmt.Errorf("configured topdata.assets path %s is not accessible: %w", path, err)
+	}
+	return path, nil
+}
+
+func looksLikeGitRepoSpec(value string) bool {
+	return strings.Contains(value, "://") ||
+		strings.HasPrefix(value, "git@") ||
+		strings.HasSuffix(value, ".git")
+}
+
+func gitOutput(dir string, args ...string) (string, error) {
+	cmd := exec.Command("git", args...)
+	if dir != "" {
+		cmd.Dir = dir
+	}
+	output, err := cmd.CombinedOutput()
+	if err != nil {
+		return "", fmt.Errorf("%v: %s", args, strings.TrimSpace(string(output)))
+	}
+	return strings.TrimSpace(string(output)), nil
+}

internal/topdata/base_dialog_migrate.go

@@ -0,0 +1,41 @@
+package topdata
+
+import (
+	"encoding/json"
+	"os"
+	"path/filepath"
+)
+
+func importLegacyBaseDialog(referenceBuilderDir, sourceDir string) (int, error) {
+	legacyPath := filepath.Join(referenceBuilderDir, "tlk", "base.json")
+	if _, err := os.Stat(legacyPath); err != nil {
+		if os.IsNotExist(err) {
+			return 0, nil
+		}
+		return 0, err
+	}
+
+	targetPath := filepath.Join(sourceDir, "base_dialog.json")
+	if fileExists(targetPath) {
+		current, err := loadJSONObject(targetPath)
+		if err == nil {
+			if entries, ok := current["entries"].(map[string]any); ok && len(entries) > 0 {
+				return 0, nil
+			}
+		}
+	}
+
+	obj, err := loadJSONObject(legacyPath)
+	if err != nil {
+		return 0, err
+	}
+	raw, err := json.MarshalIndent(obj, "", "  ")
+	if err != nil {
+		return 0, err
+	}
+	raw = append(raw, '\n')
+	if err := os.WriteFile(targetPath, raw, 0o644); err != nil {
+		return 0, err
+	}
+	return 1, nil
+}

internal/topdata/baseitems_migrate.go

@@ -0,0 +1,105 @@
+package topdata
+
+import (
+	"encoding/json"
+	"os"
+	"path/filepath"
+)
+
+func importLegacyBaseitems(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
+	_ = legacyTLK
+
+	legacyDir := filepath.Join(referenceBuilderDir, "data", "baseitems")
+	if _, err := os.Stat(legacyDir); err != nil {
+		if os.IsNotExist(err) {
+			return 0, nil
+		}
+		return 0, err
+	}
+
+	targetDir := filepath.Join(dataDir, "baseitems")
+	targetBasePath := filepath.Join(targetDir, "base.json")
+	targetLockPath := filepath.Join(targetDir, "lock.json")
+	targetModulesDir := filepath.Join(targetDir, "modules")
+	moduleNames := []string{
+		"blunderbuss.json",
+		"coin.json",
+		"estoc.json",
+		"heavymace.json",
+		"maulandfalchion.json",
+		"ovr_baseitems.json",
+		"ovr_cloak255.json",
+		"ovr_helmet255.json",
+		"shortspear.json",
+	}
+	targetModulePaths := make([]string, 0, len(moduleNames))
+	for _, name := range moduleNames {
+		targetModulePaths = append(targetModulePaths, filepath.Join(targetModulesDir, name))
+	}
+	if fileExists(targetBasePath) && fileExists(targetLockPath) {
+		allModulesPresent := true
+		for _, path := range targetModulePaths {
+			if !fileExists(path) {
+				allModulesPresent = false
+				break
+			}
+		}
+		if allModulesPresent {
+			return 0, nil
+		}
+	}
+
+	baseObj, err := loadJSONObject(filepath.Join(legacyDir, "base.json"))
+	if err != nil {
+		return 0, err
+	}
+	baseObj["output"] = "baseitems.2da"
+	canonicalizeWikiMetadataDocument(baseObj)
+
+	lockObj, err := loadJSONObject(filepath.Join(legacyDir, "lock.json"))
+	if err != nil {
+		return 0, err
+	}
+
+	moduleObjs := make([]map[string]any, 0, len(moduleNames))
+	for _, name := range moduleNames {
+		obj, err := loadJSONObject(filepath.Join(legacyDir, "modules", name))
+		if err != nil {
+			return 0, err
+		}
+		canonicalizeWikiMetadataDocument(obj)
+		moduleObjs = append(moduleObjs, obj)
+	}
+
+	if err := os.MkdirAll(targetModulesDir, 0o755); err != nil {
+		return 0, err
+	}
+
+	writes := []struct {
+		path string
+		obj  map[string]any
+	}{
+		{path: targetBasePath, obj: baseObj},
+		{path: targetLockPath, obj: lockObj},
+	}
+	for index, path := range targetModulePaths {
+		writes = append(writes, struct {
+			path string
+			obj  map[string]any
+		}{
+			path: path,
+			obj:  moduleObjs[index],
+		})
+	}
+	for _, write := range writes {
+		raw, err := json.MarshalIndent(write.obj, "", "  ")
+		if err != nil {
+			return 0, err
+		}
+		raw = append(raw, '\n')
+		if err := os.WriteFile(write.path, raw, 0o644); err != nil {
+			return 0, err
+		}
+	}
+	return len(writes), nil
+}

internal/topdata/class_spellbooks.go

@@ -0,0 +1,373 @@
+package topdata
+
+import (
+	"fmt"
+	"path/filepath"
+	"slices"
+	"strconv"
+	"strings"
+)
+
+type classSpellbookSpec struct {
+	Path   string
+	Key    string
+	Class  string
+	Levels map[int][]string
+}
+
+type classSpellbookPlan struct {
+	Spec        classSpellbookSpec
+	Column      string
+	SpellLevels map[string]int
+}
+
+func isClassSpellbookPath(path string) bool {
+	slashed := filepath.ToSlash(path)
+	return strings.Contains(slashed, "/data/classes/spellbooks/") && strings.HasSuffix(strings.ToLower(slashed), ".json")
+}
+
+func validateClassSpellbookShape(path string, obj map[string]any, report *ValidationReport) {
+	if _, ok := obj["class"]; !ok {
+		report.Diagnostics = append(report.Diagnostics, Diagnostic{Severity: SeverityError, Path: path, Message: "class spellbook requires class"})
+	}
+	if _, ok := obj["levels"]; !ok {
+		report.Diagnostics = append(report.Diagnostics, Diagnostic{Severity: SeverityError, Path: path, Message: "class spellbook requires levels"})
+	}
+}
+
+func validateClassSpellbooks(dataDir string, report *ValidationReport) {
+	collected, err := collectSpellbookValidationDatasets(dataDir)
+	if err != nil {
+		report.Diagnostics = append(report.Diagnostics, Diagnostic{
+			Severity: SeverityError,
+			Path:     dataDir,
+			Message:  fmt.Sprintf("validate class spellbooks: %v", err),
+		})
+		return
+	}
+	_, diagnostics := classSpellbookPlans(dataDir, collected)
+	report.Diagnostics = append(report.Diagnostics, diagnostics...)
+	if len(diagnostics) == 0 {
+		warnSpellModulesForManagedSpellbookColumns(dataDir, collected, report)
+	}
+}
+
+func collectSpellbookValidationDatasets(dataDir string) ([]nativeCollectedDataset, error) {
+	datasets, err := discoverNativeDatasets(dataDir)
+	if err != nil {
+		return nil, err
+	}
+	out := make([]nativeCollectedDataset, 0, 2)
+	for _, dataset := range datasets {
+		if dataset.Name != "classes/core" && dataset.Name != "spells" {
+			continue
+		}
+		collected, err := collectNativeDataset(dataset)
+		if err != nil {
+			return nil, err
+		}
+		out = append(out, collected)
+	}
+	return out, nil
+}
+
+func applyClassSpellbooks(sourceDir string, collected []nativeCollectedDataset) ([]nativeCollectedDataset, error) {
+	dataDir := filepath.Join(sourceDir, "data")
+	plans, diagnostics := classSpellbookPlans(dataDir, collected)
+	if len(diagnostics) > 0 {
+		return nil, fmt.Errorf("class spellbook validation failed: %s", diagnosticsTextForError(diagnostics))
+	}
+	if len(plans) == 0 {
+		return collected, nil
+	}
+
+	out := make([]nativeCollectedDataset, len(collected))
+	copy(out, collected)
+	spellsIndex := -1
+	for index := range out {
+		if out[index].Dataset.Name == "spells" {
+			spellsIndex = index
+			break
+		}
+	}
+	if spellsIndex == -1 {
+		return nil, fmt.Errorf("class spellbooks require canonical dataset %q", "spells")
+	}
+
+	spells := out[spellsIndex]
+	columns := append([]string(nil), spells.Columns...)
+	rows := make([]map[string]any, 0, len(spells.Rows))
+	for _, row := range spells.Rows {
+		rows = append(rows, cloneRowMap(row))
+	}
+	rowByKey := rowsByKey(rows)
+
+	for _, plan := range plans {
+		column := plan.Column
+		if _, ok := canonicalColumn(columns, column); !ok {
+			columns = append(columns, column)
+			ensureRowsExposeColumns(rows, columns)
+		}
+		for _, row := range rows {
+			row[column] = nullValue
+		}
+		for spellKey, level := range plan.SpellLevels {
+			row := rowByKey[spellKey]
+			if row == nil {
+				return nil, fmt.Errorf("%s: unknown spell %q", plan.Spec.Path, spellKey)
+			}
+			row[column] = level
+		}
+	}
+
+	spells.Columns = columns
+	spells.Rows = rows
+	out[spellsIndex] = spells
+	return out, nil
+}
+
+func classSpellbookPlans(dataDir string, collected []nativeCollectedDataset) ([]classSpellbookPlan, []Diagnostic) {
+	specs, diagnostics := loadClassSpellbookSpecs(filepath.Join(dataDir, "classes", "spellbooks"))
+	if len(specs) == 0 {
+		return nil, diagnostics
+	}
+	classes := collectedDatasetByName(collected, "classes/core")
+	spells := collectedDatasetByName(collected, "spells")
+	if classes == nil {
+		diagnostics = append(diagnostics, Diagnostic{Severity: SeverityError, Path: dataDir, Message: `class spellbooks require canonical dataset "classes/core"`})
+		return nil, diagnostics
+	}
+	if spells == nil {
+		diagnostics = append(diagnostics, Diagnostic{Severity: SeverityError, Path: dataDir, Message: `class spellbooks require canonical dataset "spells"`})
+		return nil, diagnostics
+	}
+
+	classRows := rowsByKey(classes.Rows)
+	spellRows := rowsByKey(spells.Rows)
+	seenColumns := map[string]string{}
+	plans := make([]classSpellbookPlan, 0, len(specs))
+	for _, spec := range specs {
+		classRow := classRows[spec.Class]
+		if classRow == nil {
+			diagnostics = append(diagnostics, Diagnostic{Severity: SeverityError, Path: spec.Path, Message: fmt.Sprintf("unknown class %q", spec.Class)})
+		}
+		column := ""
+		if classRow != nil {
+			column, _ = classRow["SpellTableColumn"].(string)
+			column = strings.TrimSpace(column)
+			if column == "" || column == nullValue {
+				diagnostics = append(diagnostics, Diagnostic{Severity: SeverityError, Path: spec.Path, Message: fmt.Sprintf("class %q has no SpellTableColumn", spec.Class)})
+			}
+		}
+		if column != "" && column != nullValue {
+			folded := strings.ToLower(column)
+			if previous, ok := seenColumns[folded]; ok {
+				diagnostics = append(diagnostics, Diagnostic{Severity: SeverityError, Path: spec.Path, Message: fmt.Sprintf("SpellTableColumn %q is already managed by %s", column, previous)})
+			} else {
+				seenColumns[folded] = spec.Path
+			}
+		}
+
+		spellLevels := map[string]int{}
+		for _, level := range sortedIntKeys(spec.Levels) {
+			for _, spellKey := range spec.Levels[level] {
+				if spellRows[spellKey] == nil {
+					diagnostics = append(diagnostics, Diagnostic{Severity: SeverityError, Path: spec.Path, Message: fmt.Sprintf("unknown spell %q", spellKey)})
+					continue
+				}
+				if previous, ok := spellLevels[spellKey]; ok {
+					diagnostics = append(diagnostics, Diagnostic{Severity: SeverityError, Path: spec.Path, Message: fmt.Sprintf("spell %q appears at both level %d and level %d", spellKey, previous, level)})
+					continue
+				}
+				spellLevels[spellKey] = level
+			}
+		}
+
+		if column != "" && column != nullValue {
+			plans = append(plans, classSpellbookPlan{Spec: spec, Column: column, SpellLevels: spellLevels})
+		}
+	}
+	return plans, diagnostics
+}
+
+func warnSpellModulesForManagedSpellbookColumns(dataDir string, collected []nativeCollectedDataset, report *ValidationReport) {
+	plans, diagnostics := classSpellbookPlans(dataDir, collected)
+	if len(diagnostics) > 0 || len(plans) == 0 {
+		return
+	}
+	managed := map[string]classSpellbookPlan{}
+	for _, plan := range plans {
+		managed[strings.ToLower(plan.Column)] = plan
+	}
+	paths, err := collectModulePaths(filepath.Join(dataDir, "spells", "modules"))
+	if err != nil {
+		report.Diagnostics = append(report.Diagnostics, Diagnostic{Severity: SeverityError, Path: dataDir, Message: fmt.Sprintf("collect spell modules for spellbook warnings: %v", err)})
+		return
+	}
+	for _, path := range paths {
+		obj, err := loadJSONObject(path)
+		if err != nil {
+			continue
+		}
+		warnSpellbookManagedColumnsInValue(dataDir, path, obj["entries"], managed, report)
+		warnSpellbookManagedColumnsInValue(dataDir, path, obj["overrides"], managed, report)
+		warnSpellbookManagedColumnsInValue(dataDir, path, obj["rows"], managed, report)
+	}
+}
+
+func warnSpellbookManagedColumnsInValue(dataDir, path string, value any, managed map[string]classSpellbookPlan, report *ValidationReport) {
+	switch typed := value.(type) {
+	case map[string]any:
+		for _, key := range sortedKeys(typed) {
+			row, ok := typed[key].(map[string]any)
+			if !ok {
+				continue
+			}
+			warnSpellbookManagedColumnsInRow(dataDir, path, row, managed, report)
+		}
+	case []any:
+		for _, raw := range typed {
+			row, ok := raw.(map[string]any)
+			if !ok {
+				continue
+			}
+			warnSpellbookManagedColumnsInRow(dataDir, path, row, managed, report)
+		}
+	}
+}
+
+func warnSpellbookManagedColumnsInRow(dataDir, path string, row map[string]any, managed map[string]classSpellbookPlan, report *ValidationReport) {
+	for field, value := range row {
+		plan, ok := managed[strings.ToLower(field)]
+		if !ok || isNullLikeValue(value) {
+			continue
+		}
+		spellbookPath := displayTopdataPath(dataDir, plan.Spec.Path)
+		report.Diagnostics = append(report.Diagnostics, Diagnostic{
+			Severity: SeverityWarning,
+			Path:     path,
+			Message: fmt.Sprintf(
+				"spell module authors class spell column %q managed by class spellbook %s; use %s instead",
+				plan.Column,
+				spellbookPath,
+				spellbookPath,
+			),
+		})
+	}
+}
+
+func displayTopdataPath(dataDir, path string) string {
+	topdataDir := filepath.Dir(dataDir)
+	rel, err := filepath.Rel(filepath.Dir(topdataDir), path)
+	if err != nil {
+		return filepath.ToSlash(path)
+	}
+	return filepath.ToSlash(rel)
+}
+
+func loadClassSpellbookSpecs(dir string) ([]classSpellbookSpec, []Diagnostic) {
+	paths, err := collectModulePaths(dir)
+	if err != nil {
+		return nil, []Diagnostic{{Severity: SeverityError, Path: dir, Message: err.Error()}}
+	}
+	specs := make([]classSpellbookSpec, 0, len(paths))
+	diagnostics := []Diagnostic{}
+	for _, path := range paths {
+		obj, err := loadJSONObject(path)
+		if err != nil {
+			diagnostics = append(diagnostics, Diagnostic{Severity: SeverityError, Path: path, Message: err.Error()})
+			continue
+		}
+		spec, specDiagnostics := parseClassSpellbookSpec(path, obj)
+		diagnostics = append(diagnostics, specDiagnostics...)
+		if spec.Class != "" && spec.Levels != nil {
+			specs = append(specs, spec)
+		}
+	}
+	return specs, diagnostics
+}
+
+func parseClassSpellbookSpec(path string, obj map[string]any) (classSpellbookSpec, []Diagnostic) {
+	diagnostics := []Diagnostic{}
+	spec := classSpellbookSpec{Path: path, Levels: map[int][]string{}}
+	if key, ok := obj["key"].(string); ok {
+		spec.Key = strings.TrimSpace(key)
+	}
+	classKey, ok := obj["class"].(string)
+	if !ok || strings.TrimSpace(classKey) == "" {
+		diagnostics = append(diagnostics, Diagnostic{Severity: SeverityError, Path: path, Message: "class spellbook requires class"})
+	} else {
+		spec.Class = strings.TrimSpace(classKey)
+		if !strings.HasPrefix(spec.Class, "classes:") {
+			diagnostics = append(diagnostics, Diagnostic{Severity: SeverityError, Path: path, Message: "class must use a classes: reference"})
+		}
+	}
+	rawLevels, ok := obj["levels"].(map[string]any)
+	if !ok {
+		diagnostics = append(diagnostics, Diagnostic{Severity: SeverityError, Path: path, Message: "class spellbook levels must be an object"})
+		return spec, diagnostics
+	}
+	for _, rawLevel := range sortedKeys(rawLevels) {
+		level, err := strconv.Atoi(rawLevel)
+		if err != nil || level < 0 {
+			diagnostics = append(diagnostics, Diagnostic{Severity: SeverityError, Path: path, Message: fmt.Sprintf("level key %q must be a nonnegative integer", rawLevel)})
+			continue
+		}
+		rawList, ok := rawLevels[rawLevel].([]any)
+		if !ok {
+			diagnostics = append(diagnostics, Diagnostic{Severity: SeverityError, Path: path, Message: fmt.Sprintf("level %d must be an array", level)})
+			continue
+		}
+		for index, rawSpell := range rawList {
+			spellKey, ok := rawSpell.(string)
+			if !ok || strings.TrimSpace(spellKey) == "" {
+				diagnostics = append(diagnostics, Diagnostic{Severity: SeverityError, Path: path, Message: fmt.Sprintf("level %d spell %d must be a spells: reference", level, index)})
+				continue
+			}
+			spellKey = strings.TrimSpace(spellKey)
+			if !strings.HasPrefix(spellKey, "spells:") {
+				diagnostics = append(diagnostics, Diagnostic{Severity: SeverityError, Path: path, Message: fmt.Sprintf("level %d spell %d must use a spells: reference", level, index)})
+				continue
+			}
+			spec.Levels[level] = append(spec.Levels[level], spellKey)
+		}
+	}
+	return spec, diagnostics
+}
+
+func collectedDatasetByName(collected []nativeCollectedDataset, name string) *nativeCollectedDataset {
+	for index := range collected {
+		if collected[index].Dataset.Name == name {
+			return &collected[index]
+		}
+	}
+	return nil
+}
+
+func rowsByKey(rows []map[string]any) map[string]map[string]any {
+	out := make(map[string]map[string]any, len(rows))
+	for _, row := range rows {
+		key, _ := row["key"].(string)
+		if key != "" {
+			out[key] = row
+		}
+	}
+	return out
+}
+
+func sortedIntKeys(values map[int][]string) []int {
+	keys := make([]int, 0, len(values))
+	for key := range values {
+		keys = append(keys, key)
+	}
+	slices.Sort(keys)
+	return keys
+}
+
+func diagnosticsTextForError(diags []Diagnostic) string {
+	messages := make([]string, 0, len(diags))
+	for _, diag := range diags {
+		messages = append(messages, diag.Message)
+	}
+	return strings.Join(messages, "; ")
+}

internal/topdata/classes_migrate.go

@@ -0,0 +1,226 @@
+package topdata
+
+import (
+	"encoding/json"
+	"os"
+	"path/filepath"
+	"slices"
+	"strings"
+)
+
+var legacyClassesPlainFamilies = []string{"feats", "skills", "savthr", "bfeat", "pres"}
+
+func importLegacyClasses(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
+	legacyRoot := filepath.Join(referenceBuilderDir, "data", "classes")
+	if _, err := os.Stat(legacyRoot); err != nil {
+		if os.IsNotExist(err) {
+			return 0, nil
+		}
+		return 0, err
+	}
+
+	targetRoot := filepath.Join(dataDir, "classes")
+	if canonicalClassesPresent(targetRoot) {
+		return 0, nil
+	}
+
+	coreCollected, plainCollected, err := collectLegacyClassesDatasets(legacyRoot)
+	if err != nil {
+		return 0, err
+	}
+
+	tableKeyByOutput := map[string]string{}
+	for _, dataset := range plainCollected {
+		if dataset.TableKey == "" {
+			continue
+		}
+		registerLegacyClassesTableKey(tableKeyByOutput, dataset.TableKey, dataset.Dataset.OutputName)
+	}
+
+	coreRows := make([]map[string]any, 0, len(coreCollected.Rows))
+	for _, rawRow := range coreCollected.Rows {
+		row, ok := deepCopyValue(rawRow).(map[string]any)
+		if !ok {
+			continue
+		}
+		if legacyTLK != nil {
+			if _, _, err := inlineLegacyTLKValue(row, legacyTLK); err != nil {
+				return 0, err
+			}
+		}
+		rewriteLegacyClassesCoreRow(row, tableKeyByOutput)
+		coreRows = append(coreRows, row)
+	}
+
+	if err := removePathIfExists(targetRoot); err != nil {
+		return 0, err
+	}
+	if err := os.MkdirAll(filepath.Join(targetRoot, "core"), 0o755); err != nil {
+		return 0, err
+	}
+
+	writes := []struct {
+		path string
+		obj  map[string]any
+	}{
+		{
+			path: filepath.Join(targetRoot, "core", "base.json"),
+			obj: map[string]any{
+				"output":  coreCollected.Dataset.OutputName,
+				"columns": stringSliceToAny(coreCollected.Columns),
+				"rows":    rowsToAny(coreRows),
+			},
+		},
+		{
+			path: filepath.Join(targetRoot, "core", "lock.json"),
+			obj:  anyMapInt(coreCollected.LockData),
+		},
+	}
+
+	for _, collected := range plainCollected {
+		obj := map[string]any{
+			"output":  collected.Dataset.OutputName,
+			"columns": stringSliceToAny(collected.Columns),
+			"rows":    rowsToAny(collected.Rows),
+		}
+		if strings.TrimSpace(collected.TableKey) != "" {
+			obj["key"] = collected.TableKey
+		}
+		writes = append(writes, struct {
+			path string
+			obj  map[string]any
+		}{
+			path: filepath.Join(dataDir, collected.Dataset.Name+".json"),
+			obj:  obj,
+		})
+	}
+
+	for _, write := range writes {
+		if err := os.MkdirAll(filepath.Dir(write.path), 0o755); err != nil {
+			return 0, err
+		}
+		raw, err := json.MarshalIndent(write.obj, "", "  ")
+		if err != nil {
+			return 0, err
+		}
+		raw = append(raw, '\n')
+		if err := os.WriteFile(write.path, raw, 0o644); err != nil {
+			return 0, err
+		}
+	}
+
+	return len(writes), nil
+}
+
+func canonicalClassesPresent(targetRoot string) bool {
+	if !fileExists(filepath.Join(targetRoot, "core", "base.json")) || !fileExists(filepath.Join(targetRoot, "core", "lock.json")) {
+		return false
+	}
+	for _, family := range legacyClassesPlainFamilies {
+		ok, err := hasJSONFiles(filepath.Join(targetRoot, family))
+		if err != nil || !ok {
+			return false
+		}
+	}
+	return true
+}
+
+func collectLegacyClassesDatasets(legacyRoot string) (nativeCollectedDataset, []nativeCollectedDataset, error) {
+	coreCollected, err := collectBaseDataset(nativeDataset{
+		Name:       "classes/core",
+		BasePath:   filepath.Join(legacyRoot, "core", "base.json"),
+		LockPath:   filepath.Join(legacyRoot, "core", "lock.json"),
+		ModulesDir: filepath.Join(legacyRoot, "core", "modules"),
+		OutputName: "classes.2da",
+		Spec:       specForDataset("classes"),
+	})
+	if err != nil {
+		return nativeCollectedDataset{}, nil, err
+	}
+	coreCollected.Dataset.OutputName = "classes.2da"
+
+	plainCollected := make([]nativeCollectedDataset, 0)
+	for _, family := range legacyClassesPlainFamilies {
+		familyDir := filepath.Join(legacyRoot, family)
+		entries, err := os.ReadDir(familyDir)
+		if err != nil {
+			if os.IsNotExist(err) {
+				continue
+			}
+			return nativeCollectedDataset{}, nil, err
+		}
+		for _, entry := range entries {
+			if entry.IsDir() || filepath.Ext(entry.Name()) != ".json" || strings.HasPrefix(entry.Name(), ".") || entry.Name() == "lock.json" {
+				continue
+			}
+			filePath := filepath.Join(familyDir, entry.Name())
+			tableData, err := loadJSONObject(filePath)
+			if err != nil {
+				return nativeCollectedDataset{}, nil, err
+			}
+			outputName, _ := tableData["output"].(string)
+			if strings.TrimSpace(outputName) == "" {
+				outputName = strings.TrimSuffix(entry.Name(), filepath.Ext(entry.Name())) + ".2da"
+			}
+			collected, err := collectPlainDataset(nativeDataset{
+				Name:       filepath.ToSlash(filepath.Join("classes", family, strings.TrimSuffix(entry.Name(), filepath.Ext(entry.Name())))),
+				BasePath:   filePath,
+				LockPath:   filepath.Join(familyDir, "lock.json"),
+				OutputName: outputName,
+				Spec:       specForDataset(filepath.ToSlash(filepath.Join("classes", family, strings.TrimSuffix(entry.Name(), filepath.Ext(entry.Name()))))),
+			})
+			if err != nil {
+				return nativeCollectedDataset{}, nil, err
+			}
+			plainCollected = append(plainCollected, collected)
+		}
+	}
+
+	slices.SortFunc(plainCollected, func(a, b nativeCollectedDataset) int {
+		return strings.Compare(a.Dataset.Name, b.Dataset.Name)
+	})
+	return coreCollected, plainCollected, nil
+}
+
+func registerLegacyClassesTableKey(tableKeyByOutput map[string]string, tableKey, outputName string) {
+	trimmedOutput := strings.TrimSpace(outputName)
+	if trimmedOutput != "" {
+		tableKeyByOutput[trimmedOutput] = tableKey
+	}
+	trimmedStem := strings.TrimSpace(outputStem(outputName))
+	if trimmedStem != "" {
+		tableKeyByOutput[trimmedStem] = tableKey
+	}
+}
+
+func rewriteLegacyClassesCoreRow(row map[string]any, tableKeyByOutput map[string]string) {
+	for _, field := range []string{"FeatsTable", "SavingThrowTable", "SkillsTable", "BonusFeatsTable", "PreReqTable"} {
+		tableKey := legacyClassesTableKeyForValue(row[field], tableKeyByOutput)
+		if tableKey == "" {
+			continue
+		}
+		row[field] = map[string]any{"table": tableKey}
+	}
+}
+
+func legacyClassesTableKeyForValue(value any, tableKeyByOutput map[string]string) string {
+	switch typed := value.(type) {
+	case map[string]any:
+		tableKey, _ := typed["table"].(string)
+		return strings.TrimSpace(tableKey)
+	case string:
+		trimmed := strings.TrimSpace(typed)
+		if trimmed == "" || trimmed == nullValue {
+			return ""
+		}
+		if trimmed != strings.ToLower(trimmed) {
+			return ""
+		}
+		if tableKey, ok := tableKeyByOutput[trimmed]; ok {
+			return tableKey
+		}
+		return ""
+	default:
+		return ""
+	}
+}

internal/topdata/cloakmodel_migrate.go

@@ -0,0 +1,6 @@
+package topdata
+
+func importLegacyCloakmodel(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
+	_ = legacyTLK
+	return importLegacyDatasetMirror(referenceBuilderDir, dataDir, "cloakmodel", "cloakmodel.2da", nil)
+}

internal/topdata/convert.go

@@ -0,0 +1,888 @@
+package topdata
+
+import (
+	"encoding/json"
+	"errors"
+	"fmt"
+	"io"
+	"os"
+	"path/filepath"
+	"regexp"
+	"sort"
+	"strconv"
+	"strings"
+
+	"git.westgate.pw/ShadowsOverWestgate/sow-tools/internal/project"
+	"gopkg.in/yaml.v3"
+)
+
+type convertOptions struct {
+	Namespace        string
+	KeyFields        []string
+	CollisionMode    string
+	BaseDialog       string
+	Type             string
+	Name             string
+	FilenamePrefixes map[string]string
+}
+
+func RunConvertCommand(args []string, stdout io.Writer) error {
+	if len(args) == 0 || args[0] == "--help" || args[0] == "-h" {
+		printConvertUsage(stdout)
+		return nil
+	}
+	switch args[0] {
+	case "2da-to-json":
+		if len(args) > 1 && (args[1] == "--help" || args[1] == "-h") {
+			_, _ = fmt.Fprintln(stdout, "usage: convert-topdata 2da-to-json [flags] <input.2da> <output.json>")
+			return nil
+		}
+		opts, input, output, err := parseConvertArgs(args[1:], false, "suffix")
+		if err != nil {
+			return err
+		}
+		data, err := parse2DAFile(input)
+		if err != nil {
+			return err
+		}
+		result, err := convert2DAToJSON(data, output, opts)
+		if err != nil {
+			return err
+		}
+		return writeJSONFile(output, result)
+	case "2da-to-module":
+		if len(args) > 1 && (args[1] == "--help" || args[1] == "-h") {
+			_, _ = fmt.Fprintln(stdout, "usage: convert-topdata 2da-to-module [--namespace <dataset>] [--name <module-name>] [--type entries|override] [flags] <input.2da> [output.json]")
+			return nil
+		}
+		opts, input, output, err := parseConvertArgs(args[1:], true, "error")
+		if err != nil {
+			return err
+		}
+		data, err := parse2DAFile(input)
+		if err != nil {
+			return err
+		}
+		if opts.Type == "override" {
+			result := map[string]any{"overrides": toOverridesRows(data.Rows)}
+			if err := writeJSONFile(output, result); err != nil {
+				return err
+			}
+			_, _ = fmt.Fprintf(stdout, "converted overrides: %d\n", len(result["overrides"].([]map[string]any)))
+			return nil
+		}
+		result, err := convert2DAToModule(data, output, opts)
+		if err != nil {
+			return err
+		}
+		if err := writeJSONFile(output, result); err != nil {
+			return err
+		}
+		_, _ = fmt.Fprintf(stdout, "converted entries: %d\n", len(result["entries"].(map[string]any)))
+		return nil
+	case "json-to-2da":
+		if len(args) > 1 && (args[1] == "--help" || args[1] == "-h") {
+			_, _ = fmt.Fprintln(stdout, "usage: convert-topdata json-to-2da <input.json> <output.2da>")
+			return nil
+		}
+		if len(args) != 3 {
+			return errors.New("usage: convert-topdata json-to-2da <input.json> <output.2da>")
+		}
+		data, err := readCanonicalJSON(args[1])
+		if err != nil {
+			return err
+		}
+		return write2DAFile(data, args[2])
+	default:
+		return fmt.Errorf("unknown convert-topdata subcommand %q", args[0])
+	}
+}
+
+func printConvertUsage(stdout io.Writer) {
+	_, _ = fmt.Fprintln(stdout, "usage: convert-topdata <2da-to-json|2da-to-module|json-to-2da> ...")
+	_, _ = fmt.Fprintln(stdout, "")
+	_, _ = fmt.Fprintln(stdout, "subcommands:")
+	_, _ = fmt.Fprintln(stdout, "  2da-to-json   Convert a 2DA file into canonical JSON rows")
+	_, _ = fmt.Fprintln(stdout, "  2da-to-module Convert a 2DA file into module entries or override JSON")
+	_, _ = fmt.Fprintln(stdout, "  json-to-2da   Convert canonical JSON rows into a 2DA file")
+	_, _ = fmt.Fprintln(stdout, "")
+	_, _ = fmt.Fprintln(stdout, "2da-to-module notes:")
+	_, _ = fmt.Fprintln(stdout, "  - --namespace is optional when topdata/templates/config.yaml declares it for the input table")
+	_, _ = fmt.Fprintln(stdout, "  - --name controls inferred output naming: <prefix>_<namespace>_<name>.json")
+	_, _ = fmt.Fprintln(stdout, "  - topdata/templates/config.yaml module_output.filename_prefixes can set entries/override prefixes")
+	_, _ = fmt.Fprintln(stdout, "  - flags accept both --flag value and --flag=value forms")
+	_, _ = fmt.Fprintln(stdout, "  - unkeyed rows now fail conversion instead of being skipped")
+}
+
+type parsed2DA struct {
+	Columns []string         `json:"columns"`
+	Rows    []map[string]any `json:"rows"`
+}
+
+func parseConvertArgs(args []string, allowFormat bool, defaultCollision string) (convertOptions, string, string, error) {
+	opts := convertOptions{
+		CollisionMode: defaultCollision,
+		Type:          "entries",
+	}
+	positional := make([]string, 0, 2)
+	for index := 0; index < len(args); index++ {
+		arg := args[index]
+		switch arg {
+		case "--namespace":
+			index++
+			if index >= len(args) {
+				return opts, "", "", errors.New("--namespace requires a value")
+			}
+			opts.Namespace = args[index]
+		case "--key-field":
+			index++
+			if index >= len(args) {
+				return opts, "", "", errors.New("--key-field requires a value")
+			}
+			opts.KeyFields = append(opts.KeyFields, args[index])
+		case "--name":
+			index++
+			if index >= len(args) {
+				return opts, "", "", errors.New("--name requires a value")
+			}
+			opts.Name = args[index]
+		case "--collision":
+			index++
+			if index >= len(args) {
+				return opts, "", "", errors.New("--collision requires a value")
+			}
+			opts.CollisionMode = args[index]
+		case "--base-dialog":
+			index++
+			if index >= len(args) {
+				return opts, "", "", errors.New("--base-dialog requires a value")
+			}
+			opts.BaseDialog = args[index]
+		case "--format", "--type":
+			if !allowFormat {
+				return opts, "", "", fmt.Errorf("%s does not accept %s", positionalSafe(args), arg)
+			}
+			index++
+			if index >= len(args) {
+				return opts, "", "", fmt.Errorf("%s requires a value", arg)
+			}
+			opts.Type = args[index]
+		default:
+			if value, ok := parseInlineFlagValue(arg, "--namespace"); ok {
+				opts.Namespace = value
+				continue
+			}
+			if value, ok := parseInlineFlagValue(arg, "--key-field"); ok {
+				opts.KeyFields = append(opts.KeyFields, value)
+				continue
+			}
+			if value, ok := parseInlineFlagValue(arg, "--name"); ok {
+				opts.Name = value
+				continue
+			}
+			if value, ok := parseInlineFlagValue(arg, "--collision"); ok {
+				opts.CollisionMode = value
+				continue
+			}
+			if value, ok := parseInlineFlagValue(arg, "--base-dialog"); ok {
+				opts.BaseDialog = value
+				continue
+			}
+			if value, ok := parseInlineFlagValue(arg, "--format"); ok {
+				if !allowFormat {
+					return opts, "", "", fmt.Errorf("%s does not accept %s", positionalSafe(args), "--format")
+				}
+				opts.Type = value
+				continue
+			}
+			if value, ok := parseInlineFlagValue(arg, "--type"); ok {
+				if !allowFormat {
+					return opts, "", "", fmt.Errorf("%s does not accept %s", positionalSafe(args), "--type")
+				}
+				opts.Type = value
+				continue
+			}
+			if strings.HasPrefix(arg, "--") {
+				return opts, "", "", fmt.Errorf("unknown flag %q", arg)
+			}
+			positional = append(positional, arg)
+		}
+	}
+	if allowFormat {
+		if len(positional) < 1 || len(positional) > 2 {
+			return opts, "", "", errors.New("usage: convert-topdata 2da-to-module [--namespace <dataset>] [--name <module-name>] [--type entries|override] [flags] <input.2da> [output.json]")
+		}
+	} else if len(positional) != 2 {
+		return opts, "", "", errors.New("usage: convert-topdata 2da-to-json [flags] <input.2da> <output.json>")
+	}
+	if opts.CollisionMode != "suffix" && opts.CollisionMode != "error" {
+		return opts, "", "", fmt.Errorf("unsupported collision mode %q", opts.CollisionMode)
+	}
+	if allowFormat {
+		output := ""
+		if len(positional) == 2 {
+			output = positional[1]
+		}
+		ctx, err := resolveConvertContext(positional[0])
+		if err != nil {
+			return opts, "", "", err
+		}
+		if ctx.Project != nil {
+			if outputCtx, err := resolveConvertOutputContext(ctx.Project, output); err != nil {
+				return opts, "", "", err
+			} else if ctx.Template.Namespace == "" && len(ctx.Template.KeyFields) == 0 {
+				ctx.Template = outputCtx
+			}
+		}
+		if strings.TrimSpace(opts.Namespace) == "" {
+			opts.Namespace = ctx.Template.Namespace
+		}
+		if len(opts.KeyFields) == 0 && len(ctx.Template.KeyFields) > 0 {
+			opts.KeyFields = append(opts.KeyFields, ctx.Template.KeyFields...)
+		}
+		if len(opts.FilenamePrefixes) == 0 && len(ctx.Config.ModuleOutput.FilenamePrefixes) > 0 {
+			opts.FilenamePrefixes = cloneStringMap(ctx.Config.ModuleOutput.FilenamePrefixes)
+		}
+		if strings.TrimSpace(opts.Namespace) == "" {
+			return opts, "", "", errors.New("2da-to-module requires --namespace or a topdata/templates/config.yaml entry for the input table")
+		}
+		switch opts.Type {
+		case "entry":
+			opts.Type = "entries"
+		case "overrides":
+			opts.Type = "override"
+		}
+		if opts.Type != "entries" && opts.Type != "override" {
+			return opts, "", "", fmt.Errorf("unsupported module type %q", opts.Type)
+		}
+		if output == "" {
+			resolved, err := defaultModuleOutputPath(positional[0], opts, ctx.Project)
+			if err != nil {
+				return opts, "", "", err
+			}
+			output = resolved
+		}
+		return opts, positional[0], output, nil
+	}
+	ctx, err := resolveConvertContext(positional[0])
+	if err != nil {
+		return opts, "", "", err
+	}
+	if ctx.Project != nil {
+		if outputCtx, err := resolveConvertOutputContext(ctx.Project, positional[1]); err != nil {
+			return opts, "", "", err
+		} else if ctx.Template.Namespace == "" && len(ctx.Template.KeyFields) == 0 {
+			ctx.Template = outputCtx
+		}
+	}
+	if strings.TrimSpace(opts.Namespace) == "" {
+		opts.Namespace = ctx.Template.Namespace
+	}
+	if len(opts.KeyFields) == 0 && len(ctx.Template.KeyFields) > 0 {
+		opts.KeyFields = append(opts.KeyFields, ctx.Template.KeyFields...)
+	}
+	return opts, positional[0], positional[1], nil
+}
+
+func parseInlineFlagValue(arg, name string) (string, bool) {
+	prefix := name + "="
+	if !strings.HasPrefix(arg, prefix) {
+		return "", false
+	}
+	return strings.TrimSpace(arg[len(prefix):]), true
+}
+
+func positionalSafe(args []string) string {
+	if len(args) == 0 {
+		return "command"
+	}
+	return args[0]
+}
+
+func parse2DAFile(path string) (parsed2DA, error) {
+	raw, err := os.ReadFile(path)
+	if err != nil {
+		return parsed2DA{}, fmt.Errorf("read %s: %w", path, err)
+	}
+	lines := strings.Split(strings.ReplaceAll(string(raw), "\r\n", "\n"), "\n")
+	trimmed := make([]string, 0, len(lines))
+	for _, line := range lines {
+		line = strings.TrimSpace(line)
+		if line != "" {
+			trimmed = append(trimmed, line)
+		}
+	}
+	if len(trimmed) < 2 || !strings.HasPrefix(trimmed[0], "2DA") {
+		return parsed2DA{}, fmt.Errorf("%s: invalid 2DA file", path)
+	}
+	columns := split2DALine(trimmed[1])
+	if len(columns) == 0 {
+		return parsed2DA{}, fmt.Errorf("%s: missing 2DA columns", path)
+	}
+	rows := make([]map[string]any, 0, max(0, len(trimmed)-2))
+	seenIDs := map[int]struct{}{}
+	for _, line := range trimmed[2:] {
+		fields := split2DALine(line)
+		if len(fields) == 0 {
+			continue
+		}
+		rowID, err := strconv.Atoi(fields[0])
+		if err != nil {
+			return parsed2DA{}, fmt.Errorf("%s: invalid 2DA row id %q", path, fields[0])
+		}
+		if len(fields)-1 > len(columns) {
+			return parsed2DA{}, fmt.Errorf("%s: row %d has %d values but only %d columns are declared", path, rowID, len(fields)-1, len(columns))
+		}
+		if _, ok := seenIDs[rowID]; ok {
+			return parsed2DA{}, fmt.Errorf("%s: duplicate 2DA row id %d", path, rowID)
+		}
+		seenIDs[rowID] = struct{}{}
+		row := map[string]any{"id": rowID}
+		for index, column := range columns {
+			if index+1 < len(fields) {
+				row[column] = smartConvertScalar(fields[index+1])
+				continue
+			}
+			row[column] = nullValue
+		}
+		rows = append(rows, row)
+	}
+	return parsed2DA{Columns: columns, Rows: rows}, nil
+}
+
+func split2DALine(line string) []string {
+	fields := make([]string, 0)
+	var current strings.Builder
+	inQuotes := false
+	for _, ch := range line {
+		switch {
+		case ch == '"':
+			inQuotes = !inQuotes
+		case !inQuotes && (ch == '\t' || ch == ' '):
+			if current.Len() > 0 {
+				fields = append(fields, current.String())
+				current.Reset()
+			}
+		default:
+			current.WriteRune(ch)
+		}
+	}
+	if current.Len() > 0 {
+		fields = append(fields, current.String())
+	}
+	return fields
+}
+
+func smartConvertScalar(value string) any {
+	if value == "" {
+		return ""
+	}
+	if parsed, err := strconv.Atoi(value); err == nil {
+		return parsed
+	}
+	return value
+}
+
+func convert2DAToJSON(data parsed2DA, outputPath string, opts convertOptions) (map[string]any, error) {
+	rows, err := assignConvertedRowKeys(data.Rows, outputPath, opts)
+	if err != nil {
+		return nil, err
+	}
+	outRows := make([]map[string]any, 0, len(rows))
+	for _, row := range rows {
+		ordered := map[string]any{"id": row["id"]}
+		if key, ok := row["key"].(string); ok && key != "" {
+			ordered["key"] = key
+		}
+		for _, column := range data.Columns {
+			ordered[column] = row[column]
+		}
+		outRows = append(outRows, ordered)
+	}
+	return map[string]any{"columns": data.Columns, "rows": outRows}, nil
+}
+
+func convert2DAToModule(data parsed2DA, outputPath string, opts convertOptions) (map[string]any, error) {
+	rows, err := assignConvertedRowKeys(data.Rows, outputPath, opts)
+	if err != nil {
+		return nil, err
+	}
+	entries := map[string]any{}
+	missing := []string{}
+	for _, row := range rows {
+		key, _ := row["key"].(string)
+		if key == "" {
+			missing = append(missing, strconv.Itoa(row["id"].(int)))
+			continue
+		}
+		entry := map[string]any{}
+		for _, column := range data.Columns {
+			value := row[column]
+			if format2DAValue(value) == nullValue {
+				continue
+			}
+			entry[column] = value
+		}
+		entries[key] = entry
+	}
+	if len(missing) > 0 {
+		return nil, fmt.Errorf("unable to generate keys for row ids %s; declare --key-field or configure key_fields in topdata/templates/config.yaml", strings.Join(missing, ", "))
+	}
+	return map[string]any{"entries": entries}, nil
+}
+
+func toOverridesRows(rows []map[string]any) []map[string]any {
+	out := make([]map[string]any, 0, len(rows))
+	for _, row := range rows {
+		override := map[string]any{"id": row["id"]}
+		keys := make([]string, 0, len(row))
+		for key := range row {
+			if key == "id" || key == "key" {
+				continue
+			}
+			keys = append(keys, key)
+		}
+		sort.Strings(keys)
+		for _, key := range keys {
+			if format2DAValue(row[key]) == nullValue {
+				continue
+			}
+			override[key] = row[key]
+		}
+		out = append(out, override)
+	}
+	return out
+}
+
+func assignConvertedRowKeys(rows []map[string]any, outputPath string, opts convertOptions) ([]map[string]any, error) {
+	namespace := strings.TrimSpace(opts.Namespace)
+	out := make([]map[string]any, 0, len(rows))
+	used := map[string]struct{}{}
+	rowIDsByKey := map[string]int{}
+	for _, row := range rows {
+		cloned := cloneRowMap(row)
+		if namespace == "" {
+			out = append(out, cloned)
+			continue
+		}
+		candidates := convertedRowKeyCandidates(cloned, opts.KeyFields)
+		if len(candidates) == 0 {
+			out = append(out, cloned)
+			continue
+		}
+		key := ""
+		for _, candidate := range candidates {
+			candidateKey := namespace + ":" + candidate
+			if _, ok := used[candidateKey]; ok {
+				continue
+			}
+			key = candidateKey
+			break
+		}
+		if key == "" {
+			key = namespace + ":" + candidates[0]
+			if opts.CollisionMode == "error" {
+				return nil, fmt.Errorf("duplicate generated key %q for row %v (already used by row %d)", key, cloned["id"], rowIDsByKey[key])
+			}
+			key = fmt.Sprintf("%s_%v", key, cloned["id"])
+		}
+		used[key] = struct{}{}
+		rowID, _ := asInt(cloned["id"])
+		rowIDsByKey[key] = rowID
+		cloned["key"] = key
+		out = append(out, cloned)
+	}
+	return out, nil
+}
+
+var convertKeyWhitespace = regexp.MustCompile(`\s+`)
+var convertKeyCleaner = regexp.MustCompile(`[^a-z0-9_]`)
+
+func convertedRowKeyCandidates(row map[string]any, preferred []string) []string {
+	fields := preferred
+	if len(fields) == 0 {
+		fields = []string{"LABEL", "Label", "Name"}
+	} else {
+		for _, field := range fields {
+			value, ok := row[field]
+			if !ok {
+				return nil
+			}
+			text := strings.TrimSpace(format2DAValue(value))
+			if text == "" || text == nullValue {
+				return nil
+			}
+		}
+	}
+	values := make([]string, 0, len(fields))
+	seen := map[string]struct{}{}
+	candidates := make([]string, 0, len(fields))
+	for _, field := range fields {
+		value, ok := row[field]
+		if !ok {
+			continue
+		}
+		text := strings.TrimSpace(format2DAValue(value))
+		if text == "" || text == nullValue {
+			continue
+		}
+		normalized := normalizeConvertedKeyText(text)
+		if normalized == "" {
+			continue
+		}
+		values = append(values, normalized)
+		joined := strings.Join(values, "_")
+		if _, ok := seen[joined]; ok {
+			continue
+		}
+		seen[joined] = struct{}{}
+		candidates = append(candidates, joined)
+	}
+	return candidates
+}
+
+func normalizeConvertedKeyText(text string) string {
+	text = strings.TrimSpace(strings.ToLower(text))
+	text = convertKeyWhitespace.ReplaceAllString(text, "_")
+	text = convertKeyCleaner.ReplaceAllString(text, "")
+	text = strings.Trim(text, "_")
+	return text
+}
+
+type convertTemplateTable struct {
+	Namespace string   `json:"namespace" yaml:"namespace"`
+	KeyFields []string `json:"key_fields" yaml:"key_fields"`
+}
+
+type convertTemplateConfig struct {
+	ModuleOutput convertModuleOutputConfig       `json:"module_output" yaml:"module_output"`
+	Tables       map[string]convertTemplateTable `json:"tables" yaml:"tables"`
+}
+
+type convertModuleOutputConfig struct {
+	FilenamePrefixes map[string]string `json:"filename_prefixes" yaml:"filename_prefixes"`
+}
+
+type convertContext struct {
+	Project  *project.Project
+	Config   convertTemplateConfig
+	Template convertTemplateTable
+}
+
+func resolveConvertContext(input string) (convertContext, error) {
+	cwd, err := os.Getwd()
+	if err != nil {
+		return convertContext{}, err
+	}
+	root, err := project.FindRoot(cwd)
+	if err != nil {
+		return convertContext{}, nil
+	}
+	p, err := project.Load(root)
+	if err != nil {
+		return convertContext{}, err
+	}
+	if !p.HasTopData() {
+		return convertContext{Project: p}, nil
+	}
+	config, err := loadConvertTemplateConfig(p)
+	if err != nil {
+		return convertContext{}, err
+	}
+	table, err := templateTableConfig(config, p, input)
+	if err != nil {
+		return convertContext{}, err
+	}
+	return convertContext{
+		Project:  p,
+		Config:   config,
+		Template: table,
+	}, nil
+}
+
+func loadTemplateTableConfig(p *project.Project, input string) (convertTemplateTable, error) {
+	config, err := loadConvertTemplateConfig(p)
+	if err != nil {
+		return convertTemplateTable{}, err
+	}
+	return templateTableConfig(config, p, input)
+}
+
+func templateTableConfig(config convertTemplateConfig, p *project.Project, input string) (convertTemplateTable, error) {
+	if len(config.Tables) == 0 {
+		return convertTemplateTable{}, nil
+	}
+	templatesDir := filepath.Join(p.TopDataSourceDir(), "templates")
+	inputAbs, err := filepath.Abs(input)
+	if err != nil {
+		return convertTemplateTable{}, err
+	}
+	templatesAbs, err := filepath.Abs(templatesDir)
+	if err != nil {
+		return convertTemplateTable{}, err
+	}
+	rel, err := filepath.Rel(templatesAbs, inputAbs)
+	if err != nil || strings.HasPrefix(rel, "..") || rel == "." {
+		return convertTemplateTable{}, nil
+	}
+
+	rel = filepath.ToSlash(rel)
+	base := filepath.Base(rel)
+	stem := strings.TrimSuffix(base, filepath.Ext(base))
+	for _, key := range []string{rel, base, stem} {
+		if table, ok := config.Tables[key]; ok {
+			return table, nil
+		}
+	}
+	return convertTemplateTable{}, nil
+}
+
+func loadConvertTemplateConfig(p *project.Project) (convertTemplateConfig, error) {
+	templatesDir := filepath.Join(p.TopDataSourceDir(), "templates")
+	for _, candidate := range []string{"config.yaml", "config.yml", "config.json"} {
+		configPath := filepath.Join(templatesDir, candidate)
+		raw, err := os.ReadFile(configPath)
+		if err != nil {
+			if errors.Is(err, os.ErrNotExist) {
+				continue
+			}
+			return convertTemplateConfig{}, fmt.Errorf("read %s: %w", configPath, err)
+		}
+
+		var config convertTemplateConfig
+		switch filepath.Ext(configPath) {
+		case ".yaml", ".yml":
+			if err := yaml.Unmarshal(raw, &config); err != nil {
+				return convertTemplateConfig{}, fmt.Errorf("parse %s: %w", configPath, err)
+			}
+		default:
+			if err := json.Unmarshal(raw, &config); err != nil {
+				return convertTemplateConfig{}, fmt.Errorf("parse %s: %w", configPath, err)
+			}
+		}
+		if err := validateConvertTemplateConfig(config); err != nil {
+			return convertTemplateConfig{}, fmt.Errorf("validate %s: %w", configPath, err)
+		}
+		return config, nil
+	}
+	return convertTemplateConfig{}, nil
+}
+
+func validateConvertTemplateConfig(config convertTemplateConfig) error {
+	for moduleType, prefix := range config.ModuleOutput.FilenamePrefixes {
+		switch moduleType {
+		case "entries", "override":
+		default:
+			return fmt.Errorf("unsupported module_output.filename_prefixes key %q; expected entries or override", moduleType)
+		}
+		if normalizeModuleFilenamePrefix(prefix) == "" {
+			return fmt.Errorf("module_output.filename_prefixes.%s must contain at least one filename-safe character", moduleType)
+		}
+	}
+	return nil
+}
+
+func resolveConvertOutputContext(p *project.Project, output string) (convertTemplateTable, error) {
+	if p == nil || !p.HasTopData() || strings.TrimSpace(output) == "" {
+		return convertTemplateTable{}, nil
+	}
+	config, err := loadConvertTemplateConfig(p)
+	if err != nil {
+		return convertTemplateTable{}, err
+	}
+	if len(config.Tables) == 0 {
+		return convertTemplateTable{}, nil
+	}
+	dataDir := filepath.Join(p.TopDataSourceDir(), "data")
+	dataAbs, err := filepath.Abs(dataDir)
+	if err != nil {
+		return convertTemplateTable{}, err
+	}
+	outputAbs, err := filepath.Abs(output)
+	if err != nil {
+		return convertTemplateTable{}, err
+	}
+	rel, err := filepath.Rel(dataAbs, outputAbs)
+	if err != nil || rel == "." || strings.HasPrefix(rel, "..") {
+		return convertTemplateTable{}, nil
+	}
+	rel = filepath.ToSlash(rel)
+
+	bestMatch := ""
+	bestTable := convertTemplateTable{}
+	for _, table := range config.Tables {
+		namespacePath := namespaceToPath(table.Namespace)
+		if namespacePath == "" {
+			continue
+		}
+		if rel != namespacePath && !strings.HasPrefix(rel, namespacePath+"/") {
+			continue
+		}
+		if len(namespacePath) <= len(bestMatch) {
+			continue
+		}
+		bestMatch = namespacePath
+		bestTable = table
+	}
+	return bestTable, nil
+}
+
+func namespaceToPath(namespace string) string {
+	namespace = strings.TrimSpace(strings.ToLower(namespace))
+	if namespace == "" {
+		return ""
+	}
+	namespace = strings.ReplaceAll(namespace, "\\", "/")
+	namespace = strings.ReplaceAll(namespace, ":", "/")
+	namespace = convertKeyWhitespace.ReplaceAllString(namespace, "_")
+	namespace = strings.Trim(namespace, "/")
+	return namespace
+}
+
+func defaultModuleOutputPath(input string, opts convertOptions, p *project.Project) (string, error) {
+	if p == nil {
+		return "", errors.New("2da-to-module requires an explicit output path when not run inside a project root")
+	}
+	if !p.HasTopData() {
+		return "", errors.New("2da-to-module requires topdata to be configured when inferring an output path")
+	}
+	filename, err := defaultModuleFileName(input, opts, p)
+	if err != nil {
+		return "", err
+	}
+	modulesDir := filepath.Join(p.TopDataSourceDir(), "data", filepath.FromSlash(opts.Namespace), "modules")
+	return filepath.Join(modulesDir, filename), nil
+}
+
+func defaultModuleFileName(input string, opts convertOptions, p *project.Project) (string, error) {
+	if strings.TrimSpace(opts.Name) != "" {
+		name := normalizeConvertedKeyText(opts.Name)
+		if name == "" {
+			return "", fmt.Errorf("invalid --name %q", opts.Name)
+		}
+		namespace := normalizeModuleFilenameNamespace(opts.Namespace)
+		prefix := moduleFilenamePrefix(opts)
+		return fmt.Sprintf("%s_%s_%s.json", prefix, namespace, name), nil
+	}
+	if isTemplateInput(p, input) {
+		return "", errors.New("2da-to-module requires --name when converting from topdata/templates without an explicit output path")
+	}
+	return strings.TrimSuffix(filepath.Base(input), filepath.Ext(input)) + ".json", nil
+}
+
+func moduleFilenamePrefix(opts convertOptions) string {
+	moduleType := strings.TrimSpace(opts.Type)
+	if moduleType == "entry" {
+		moduleType = "entries"
+	}
+	if moduleType == "overrides" {
+		moduleType = "override"
+	}
+	if configured := strings.TrimSpace(opts.FilenamePrefixes[moduleType]); configured != "" {
+		if prefix := normalizeModuleFilenamePrefix(configured); prefix != "" {
+			return prefix
+		}
+	}
+	return "add"
+}
+
+func normalizeModuleFilenamePrefix(prefix string) string {
+	prefix = strings.TrimSpace(strings.ToLower(prefix))
+	prefix = strings.ReplaceAll(prefix, "/", "_")
+	prefix = strings.ReplaceAll(prefix, "\\", "_")
+	prefix = convertKeyWhitespace.ReplaceAllString(prefix, "_")
+	prefix = convertKeyCleaner.ReplaceAllString(prefix, "_")
+	prefix = strings.Trim(prefix, "_")
+	return prefix
+}
+
+func normalizeModuleFilenameNamespace(namespace string) string {
+	namespace = strings.TrimSpace(strings.ToLower(namespace))
+	namespace = strings.ReplaceAll(namespace, "/", "_")
+	namespace = strings.ReplaceAll(namespace, "\\", "_")
+	namespace = convertKeyWhitespace.ReplaceAllString(namespace, "_")
+	namespace = convertKeyCleaner.ReplaceAllString(namespace, "_")
+	namespace = strings.Trim(namespace, "_")
+	if namespace == "" {
+		return "module"
+	}
+	return namespace
+}
+
+func isTemplateInput(p *project.Project, input string) bool {
+	if p == nil || !p.HasTopData() {
+		return false
+	}
+	templatesAbs, err := filepath.Abs(filepath.Join(p.TopDataSourceDir(), "templates"))
+	if err != nil {
+		return false
+	}
+	inputAbs, err := filepath.Abs(input)
+	if err != nil {
+		return false
+	}
+	rel, err := filepath.Rel(templatesAbs, inputAbs)
+	if err != nil {
+		return false
+	}
+	return rel != "." && !strings.HasPrefix(rel, "..")
+}
+
+func readCanonicalJSON(path string) (parsed2DA, error) {
+	raw, err := os.ReadFile(path)
+	if err != nil {
+		return parsed2DA{}, err
+	}
+	var payload struct {
+		Columns []string         `json:"columns"`
+		Rows    []map[string]any `json:"rows"`
+	}
+	if err := json.Unmarshal(raw, &payload); err != nil {
+		return parsed2DA{}, err
+	}
+	return parsed2DA{Columns: payload.Columns, Rows: payload.Rows}, nil
+}
+
+func writeJSONFile(path string, payload any) error {
+	if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil && filepath.Dir(path) != "." {
+		return err
+	}
+	raw, err := json.MarshalIndent(payload, "", "  ")
+	if err != nil {
+		return err
+	}
+	raw = append(raw, '\n')
+	return os.WriteFile(path, raw, 0o644)
+}
+
+func write2DAFile(data parsed2DA, path string) error {
+	rows := make([]map[string]any, 0, len(data.Rows))
+	for _, row := range data.Rows {
+		cloned := cloneRowMap(row)
+		if rowID, err := asInt(cloned["id"]); err == nil {
+			cloned["id"] = rowID
+		}
+		rows = append(rows, cloned)
+	}
+	sort.Slice(rows, func(i, j int) bool {
+		left, _ := asInt(rows[i]["id"])
+		right, _ := asInt(rows[j]["id"])
+		return left < right
+	})
+	table := map[string]any{
+		"columns": data.Columns,
+		"rows":    rows,
+	}
+	if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil && filepath.Dir(path) != "." {
+		return err
+	}
+	return write2DA(table, path, false, -1)
+}
+
+func cloneStringMap(in map[string]string) map[string]string {
+	out := make(map[string]string, len(in))
+	for key, value := range in {
+		out[key] = value
+	}
+	return out
+}

internal/topdata/convert_test.go

@@ -0,0 +1,611 @@
+package topdata
+
+import (
+	"bytes"
+	"encoding/json"
+	"os"
+	"path/filepath"
+	"strings"
+	"testing"
+)
+
+func TestRunConvertCommandInfersTemplateNamespaceAndOutput(t *testing.T) {
+	root := testProjectRoot(t)
+	writeFile(t, filepath.Join(root, "nwn-tool.yaml"), `
+module:
+  name: Test
+  resref: test
+paths:
+  source: src
+  assets: assets
+  build: build
+topdata:
+  source: topdata
+  build: build/topdata
+`)
+	mkdirAll(t, filepath.Join(root, "topdata", "templates"))
+	mkdirAll(t, filepath.Join(root, "topdata", "data", "appearance", "modules"))
+	writeFile(t, filepath.Join(root, "topdata", "templates", "config.yaml"), `
+tables:
+  template_appearance:
+    namespace: appearance
+    key_fields:
+      - LABEL
+`)
+	writeFile(t, filepath.Join(root, "topdata", "templates", "template_appearance.2da"), "2DA V2.0\n\nLABEL STRING_REF NAME\n960 \"Zombie, Ash, Hot\" **** Zombie_Ash_Hot\n961 \"Zombie, Ash, Done\" **** Zombie_Ash_Done\n")
+
+	oldCwd, err := os.Getwd()
+	if err != nil {
+		t.Fatalf("getwd: %v", err)
+	}
+	t.Cleanup(func() {
+		_ = os.Chdir(oldCwd)
+	})
+	if err := os.Chdir(root); err != nil {
+		t.Fatalf("chdir: %v", err)
+	}
+
+	var stdout bytes.Buffer
+	if err := RunConvertCommand([]string{
+		"2da-to-module",
+		"--name", "ashzombies",
+		"topdata/templates/template_appearance.2da",
+	}, &stdout); err != nil {
+		t.Fatalf("RunConvertCommand failed: %v", err)
+	}
+
+	outputPath := filepath.Join(root, "topdata", "data", "appearance", "modules", "add_appearance_ashzombies.json")
+	raw, err := os.ReadFile(outputPath)
+	if err != nil {
+		t.Fatalf("read output: %v", err)
+	}
+	text := string(raw)
+	if !strings.Contains(text, `"appearance:zombie_ash_hot"`) || !strings.Contains(text, `"appearance:zombie_ash_done"`) {
+		t.Fatalf("unexpected output:\n%s", text)
+	}
+	if !strings.Contains(stdout.String(), "converted entries: 2") {
+		t.Fatalf("unexpected stdout: %s", stdout.String())
+	}
+}
+
+func TestRunConvertCommandInfersOverrideOutputPrefixFromTemplateConfig(t *testing.T) {
+	root := testProjectRoot(t)
+	writeFile(t, filepath.Join(root, "nwn-tool.yaml"), `
+module:
+  name: Test
+  resref: test
+paths:
+  source: src
+  assets: assets
+  build: build
+topdata:
+  source: topdata
+  build: build/topdata
+`)
+	mkdirAll(t, filepath.Join(root, "topdata", "templates"))
+	writeFile(t, filepath.Join(root, "topdata", "templates", "config.yaml"), `
+module_output:
+  filename_prefixes:
+    entries: add
+    override: ovr
+tables:
+  template_appearance:
+    namespace: appearance
+    key_fields:
+      - LABEL
+`)
+	writeFile(t, filepath.Join(root, "topdata", "templates", "template_appearance.2da"), "2DA V2.0\n\nLABEL STRING_REF NAME\n960 \"Zombie, Ash, Hot\" **** Zombie_Ash_Hot\n")
+
+	oldCwd, err := os.Getwd()
+	if err != nil {
+		t.Fatalf("getwd: %v", err)
+	}
+	t.Cleanup(func() {
+		_ = os.Chdir(oldCwd)
+	})
+	if err := os.Chdir(root); err != nil {
+		t.Fatalf("chdir: %v", err)
+	}
+
+	var stdout bytes.Buffer
+	if err := RunConvertCommand([]string{
+		"2da-to-module",
+		"--type", "override",
+		"--name", "ashzombies",
+		"topdata/templates/template_appearance.2da",
+	}, &stdout); err != nil {
+		t.Fatalf("RunConvertCommand failed: %v", err)
+	}
+
+	outputPath := filepath.Join(root, "topdata", "data", "appearance", "modules", "ovr_appearance_ashzombies.json")
+	raw, err := os.ReadFile(outputPath)
+	if err != nil {
+		t.Fatalf("read output: %v", err)
+	}
+	if !strings.Contains(string(raw), `"overrides"`) {
+		t.Fatalf("expected override payload, got:\n%s", string(raw))
+	}
+	if _, err := os.Stat(filepath.Join(root, "topdata", "data", "appearance", "modules", "add_appearance_ashzombies.json")); !os.IsNotExist(err) {
+		t.Fatalf("expected add-prefixed output to be absent, stat err: %v", err)
+	}
+	if !strings.Contains(stdout.String(), "converted overrides: 1") {
+		t.Fatalf("unexpected stdout: %s", stdout.String())
+	}
+}
+
+func TestRunConvertCommand2DAToJSONInfersTemplateKeys(t *testing.T) {
+	root := testProjectRoot(t)
+	writeFile(t, filepath.Join(root, "nwn-tool.yaml"), `
+module:
+  name: Test
+  resref: test
+paths:
+  source: src
+  assets: assets
+  build: build
+topdata:
+  source: topdata
+  build: build/topdata
+`)
+	mkdirAll(t, filepath.Join(root, "topdata", "templates"))
+	writeFile(t, filepath.Join(root, "topdata", "templates", "config.yaml"), `
+tables:
+  template_appearance:
+    namespace: appearance
+    key_fields:
+      - LABEL
+`)
+	inputPath := filepath.Join(root, "topdata", "templates", "template_appearance.2da")
+	writeFile(t, inputPath, "2DA V2.0\n\nLABEL STRING_REF NAME\n960 \"Zombie, Ash, Hot\" **** Zombie_Ash_Hot\n961 \"Zombie, Ash, Done\" **** Zombie_Ash_Done\n")
+
+	oldCwd, err := os.Getwd()
+	if err != nil {
+		t.Fatalf("getwd: %v", err)
+	}
+	t.Cleanup(func() {
+		_ = os.Chdir(oldCwd)
+	})
+	if err := os.Chdir(root); err != nil {
+		t.Fatalf("chdir: %v", err)
+	}
+
+	outputPath := filepath.Join(root, "out", "appearance.json")
+	if err := RunConvertCommand([]string{
+		"2da-to-json",
+		"topdata/templates/template_appearance.2da",
+		outputPath,
+	}, &bytes.Buffer{}); err != nil {
+		t.Fatalf("RunConvertCommand failed: %v", err)
+	}
+
+	raw, err := os.ReadFile(outputPath)
+	if err != nil {
+		t.Fatalf("read output: %v", err)
+	}
+	var payload struct {
+		Rows []map[string]any `json:"rows"`
+	}
+	if err := json.Unmarshal(raw, &payload); err != nil {
+		t.Fatalf("unmarshal output: %v", err)
+	}
+	if got := payload.Rows[0]["key"]; got != "appearance:zombie_ash_hot" {
+		t.Fatalf("expected first key to be inferred from template config, got %#v", got)
+	}
+	if got := payload.Rows[1]["key"]; got != "appearance:zombie_ash_done" {
+		t.Fatalf("expected second key to be inferred from template config, got %#v", got)
+	}
+}
+
+func TestRunConvertCommand2DAToJSONInfersOutputDatasetKeysFromConfig(t *testing.T) {
+	root := testProjectRoot(t)
+	writeFile(t, filepath.Join(root, "nwn-tool.yaml"), `
+module:
+  name: Test
+  resref: test
+paths:
+  source: src
+  assets: assets
+  build: build
+topdata:
+  source: topdata
+  build: build/topdata
+`)
+	mkdirAll(t, filepath.Join(root, "topdata", "templates"))
+	mkdirAll(t, filepath.Join(root, "topdata", "data", "soundset"))
+	writeFile(t, filepath.Join(root, "topdata", "templates", "config.yaml"), `
+tables:
+  template_soundset:
+    namespace: soundset
+    key_fields:
+      - RESREF
+      - LABEL
+`)
+	inputPath := filepath.Join(root, "scratch_soundset.2da")
+	writeFile(t, inputPath, "2DA V2.0\n\nLABEL RESREF STRREF GENDER TYPE\n0 \"Bandit Male\" nw_bandit 100 1 4\n1 \"Bandit Female\" nw_bandit 101 2 4\n")
+
+	oldCwd, err := os.Getwd()
+	if err != nil {
+		t.Fatalf("getwd: %v", err)
+	}
+	t.Cleanup(func() {
+		_ = os.Chdir(oldCwd)
+	})
+	if err := os.Chdir(root); err != nil {
+		t.Fatalf("chdir: %v", err)
+	}
+
+	outputPath := filepath.Join(root, "topdata", "data", "soundset", "base.json")
+	if err := RunConvertCommand([]string{
+		"2da-to-json",
+		"scratch_soundset.2da",
+		outputPath,
+	}, &bytes.Buffer{}); err != nil {
+		t.Fatalf("RunConvertCommand failed: %v", err)
+	}
+
+	raw, err := os.ReadFile(outputPath)
+	if err != nil {
+		t.Fatalf("read output: %v", err)
+	}
+	var payload struct {
+		Rows []map[string]any `json:"rows"`
+	}
+	if err := json.Unmarshal(raw, &payload); err != nil {
+		t.Fatalf("unmarshal output: %v", err)
+	}
+	if got := payload.Rows[0]["key"]; got != "soundset:nw_bandit" {
+		t.Fatalf("expected first soundset key, got %#v", got)
+	}
+	if got := payload.Rows[1]["key"]; got != "soundset:nw_bandit_bandit_female" {
+		t.Fatalf("expected second soundset key to use config disambiguation, got %#v", got)
+	}
+}
+
+func TestRunConvertCommand2DAToJSONInfersAmbientTemplateKeysFromResource(t *testing.T) {
+	root := testProjectRoot(t)
+	writeFile(t, filepath.Join(root, "nwn-tool.yaml"), `
+module:
+  name: Test
+  resref: test
+paths:
+  source: src
+  assets: assets
+  build: build
+topdata:
+  source: topdata
+  build: build/topdata
+`)
+	mkdirAll(t, filepath.Join(root, "topdata", "templates"))
+	writeFile(t, filepath.Join(root, "topdata", "templates", "config.yaml"), `
+tables:
+  template_ambientmusic:
+    namespace: ambientmusic
+    key_fields:
+      - Resource
+`)
+	inputPath := filepath.Join(root, "topdata", "templates", "template_ambientmusic.2da")
+	writeFile(t, inputPath, "2DA V2.0\n\nDescription DisplayName Resource Stinger1 Stinger2 Stinger3\n0 61901 **** **** **** **** ****\n1 61842 **** mus_ruralday1 **** **** ****\n2 61843 **** mus_ruralday2 **** **** ****\n")
+
+	oldCwd, err := os.Getwd()
+	if err != nil {
+		t.Fatalf("getwd: %v", err)
+	}
+	t.Cleanup(func() {
+		_ = os.Chdir(oldCwd)
+	})
+	if err := os.Chdir(root); err != nil {
+		t.Fatalf("chdir: %v", err)
+	}
+
+	outputPath := filepath.Join(root, "out", "ambientmusic.json")
+	if err := RunConvertCommand([]string{
+		"2da-to-json",
+		"topdata/templates/template_ambientmusic.2da",
+		outputPath,
+	}, &bytes.Buffer{}); err != nil {
+		t.Fatalf("RunConvertCommand failed: %v", err)
+	}
+
+	raw, err := os.ReadFile(outputPath)
+	if err != nil {
+		t.Fatalf("read output: %v", err)
+	}
+	var payload struct {
+		Rows []map[string]any `json:"rows"`
+	}
+	if err := json.Unmarshal(raw, &payload); err != nil {
+		t.Fatalf("unmarshal output: %v", err)
+	}
+	if _, ok := payload.Rows[0]["key"]; ok {
+		t.Fatalf("expected null Resource row to remain unkeyed, got %#v", payload.Rows[0]["key"])
+	}
+	if got := payload.Rows[1]["key"]; got != "ambientmusic:mus_ruralday1" {
+		t.Fatalf("expected first populated resource row key, got %#v", got)
+	}
+	if got := payload.Rows[2]["key"]; got != "ambientmusic:mus_ruralday2" {
+		t.Fatalf("expected second populated resource row key, got %#v", got)
+	}
+}
+
+func TestRunConvertCommand2DAToJSONSuffixesDuplicateGeneratedKeys(t *testing.T) {
+	root := testProjectRoot(t)
+	writeFile(t, filepath.Join(root, "nwn-tool.yaml"), `
+module:
+  name: Test
+  resref: test
+paths:
+  source: src
+  assets: assets
+  build: build
+topdata:
+  source: topdata
+  build: build/topdata
+`)
+	inputPath := filepath.Join(root, "dup.2da")
+	writeFile(t, inputPath, "2DA V2.0\n\nResource\n1 cmp_reserved\n2 cmp_reserved\n")
+
+	oldCwd, err := os.Getwd()
+	if err != nil {
+		t.Fatalf("getwd: %v", err)
+	}
+	t.Cleanup(func() {
+		_ = os.Chdir(oldCwd)
+	})
+	if err := os.Chdir(root); err != nil {
+		t.Fatalf("chdir: %v", err)
+	}
+
+	outputPath := filepath.Join(root, "out", "ambientmusic.json")
+	if err := RunConvertCommand([]string{
+		"2da-to-json",
+		"--namespace", "ambientmusic",
+		"--key-field", "Resource",
+		inputPath,
+		outputPath,
+	}, &bytes.Buffer{}); err != nil {
+		t.Fatalf("RunConvertCommand failed: %v", err)
+	}
+
+	raw, err := os.ReadFile(outputPath)
+	if err != nil {
+		t.Fatalf("read output: %v", err)
+	}
+	var payload struct {
+		Rows []map[string]any `json:"rows"`
+	}
+	if err := json.Unmarshal(raw, &payload); err != nil {
+		t.Fatalf("unmarshal output: %v", err)
+	}
+	if got := payload.Rows[0]["key"]; got != "ambientmusic:cmp_reserved" {
+		t.Fatalf("expected first duplicate key to remain unsuffixed, got %#v", got)
+	}
+	if got := payload.Rows[1]["key"]; got != "ambientmusic:cmp_reserved_2" {
+		t.Fatalf("expected second duplicate key to gain row-id suffix, got %#v", got)
+	}
+}
+
+func TestParseConvertArgsSupportsEqualsSyntax(t *testing.T) {
+	opts, input, output, err := parseConvertArgs([]string{
+		"--namespace=environment",
+		"--name=projectq",
+		"--type=entries",
+		"--collision=suffix",
+		"--key-field=Label",
+		"input.2da",
+		"output.json",
+	}, true, "error")
+	if err != nil {
+		t.Fatalf("parseConvertArgs failed: %v", err)
+	}
+	if opts.Namespace != "environment" {
+		t.Fatalf("expected namespace from equals syntax, got %q", opts.Namespace)
+	}
+	if opts.Name != "projectq" {
+		t.Fatalf("expected name from equals syntax, got %q", opts.Name)
+	}
+	if opts.Type != "entries" {
+		t.Fatalf("expected type from equals syntax, got %q", opts.Type)
+	}
+	if opts.CollisionMode != "suffix" {
+		t.Fatalf("expected collision from equals syntax, got %q", opts.CollisionMode)
+	}
+	if len(opts.KeyFields) != 1 || opts.KeyFields[0] != "Label" {
+		t.Fatalf("expected key field from equals syntax, got %#v", opts.KeyFields)
+	}
+	if input != "input.2da" || output != "output.json" {
+		t.Fatalf("unexpected positional parse result: input=%q output=%q", input, output)
+	}
+}
+
+func TestConvert2DAToJSONSkipsConfiguredKeysWhenAnyKeyFieldIsNull(t *testing.T) {
+	result, err := convert2DAToJSON(parsed2DA{
+		Columns: []string{"LABEL", "RESREF", "STRREF"},
+		Rows: []map[string]any{
+			{"id": 309, "LABEL": "Unused", "RESREF": "unused", "STRREF": 1},
+			{"id": 310, "LABEL": nullValue, "RESREF": "unused", "STRREF": 2},
+		},
+	}, "topdata/data/soundset/base.json", convertOptions{
+		Namespace:     "soundset",
+		KeyFields:     []string{"RESREF", "LABEL"},
+		CollisionMode: "error",
+	})
+	if err != nil {
+		t.Fatalf("convert2DAToJSON failed: %v", err)
+	}
+
+	rows := result["rows"].([]map[string]any)
+	if got := rows[0]["key"]; got != "soundset:unused" {
+		t.Fatalf("expected first row key to use the primary configured key field, got %#v", got)
+	}
+	if _, ok := rows[1]["key"]; ok {
+		t.Fatalf("expected second row to remain unkeyed when one configured key field is null, got %#v", rows[1]["key"])
+	}
+}
+
+func TestRunConvertCommandFailsOnUnkeyedRows(t *testing.T) {
+	root := testProjectRoot(t)
+	writeFile(t, filepath.Join(root, "nwn-tool.json"), `{
+  "module": {"name": "Test", "resref": "test"},
+  "paths": {"source": "src", "assets": "assets", "build": "build"},
+  "topdata": {"source": "topdata", "build": "build/topdata"}
+}`+"\n")
+	mkdirAll(t, filepath.Join(root, "topdata", "templates"))
+	writeFile(t, filepath.Join(root, "topdata", "templates", "config.yaml"), `
+tables:
+  template_appearance:
+    namespace: appearance
+    key_fields:
+      - STRING_REF
+`)
+	writeFile(t, filepath.Join(root, "topdata", "templates", "template_appearance.2da"), "2DA V2.0\n\nLABEL STRING_REF NAME\n960 \"Zombie, Ash, Hot\" **** Zombie_Ash_Hot\n")
+
+	oldCwd, err := os.Getwd()
+	if err != nil {
+		t.Fatalf("getwd: %v", err)
+	}
+	t.Cleanup(func() {
+		_ = os.Chdir(oldCwd)
+	})
+	if err := os.Chdir(root); err != nil {
+		t.Fatalf("chdir: %v", err)
+	}
+
+	err = RunConvertCommand([]string{
+		"2da-to-module",
+		"--name", "ashzombies",
+		"topdata/templates/template_appearance.2da",
+	}, &bytes.Buffer{})
+	if err == nil || !strings.Contains(err.Error(), "unable to generate keys for row ids 960") {
+		t.Fatalf("expected unkeyed row failure, got %v", err)
+	}
+}
+
+func TestRunConvertCommandFallsBackToLegacyTemplateJSONConfig(t *testing.T) {
+	root := testProjectRoot(t)
+	writeFile(t, filepath.Join(root, "nwn-tool.yaml"), `
+module:
+  name: Test
+  resref: test
+paths:
+  source: src
+  assets: assets
+  build: build
+topdata:
+  source: topdata
+  build: build/topdata
+`)
+	mkdirAll(t, filepath.Join(root, "topdata", "templates"))
+	writeFile(t, filepath.Join(root, "topdata", "templates", "config.json"), `{
+  "tables": {
+    "template_appearance": {
+      "namespace": "appearance",
+      "key_fields": ["LABEL"]
+    }
+  }
+}`+"\n")
+	inputPath := filepath.Join(root, "topdata", "templates", "template_appearance.2da")
+	writeFile(t, inputPath, "2DA V2.0\n\nLABEL\n960 Zombie_Ash_Hot\n")
+
+	oldCwd, err := os.Getwd()
+	if err != nil {
+		t.Fatalf("getwd: %v", err)
+	}
+	t.Cleanup(func() {
+		_ = os.Chdir(oldCwd)
+	})
+	if err := os.Chdir(root); err != nil {
+		t.Fatalf("chdir: %v", err)
+	}
+
+	outputPath := filepath.Join(root, "out", "appearance.json")
+	if err := RunConvertCommand([]string{
+		"2da-to-json",
+		"topdata/templates/template_appearance.2da",
+		outputPath,
+	}, &bytes.Buffer{}); err != nil {
+		t.Fatalf("RunConvertCommand failed: %v", err)
+	}
+
+	raw, err := os.ReadFile(outputPath)
+	if err != nil {
+		t.Fatalf("read output: %v", err)
+	}
+	if !strings.Contains(string(raw), `"appearance:zombie_ash_hot"`) {
+		t.Fatalf("expected legacy JSON template config fallback to be honored, got %s", string(raw))
+	}
+}
+
+func TestRunConvertCommandRejectsUnknownTemplateFilenamePrefixType(t *testing.T) {
+	root := testProjectRoot(t)
+	writeFile(t, filepath.Join(root, "nwn-tool.yaml"), `
+module:
+  name: Test
+  resref: test
+paths:
+  source: src
+  assets: assets
+  build: build
+topdata:
+  source: topdata
+  build: build/topdata
+`)
+	mkdirAll(t, filepath.Join(root, "topdata", "templates"))
+	writeFile(t, filepath.Join(root, "topdata", "templates", "config.yaml"), `
+module_output:
+  filename_prefixes:
+    additions: add
+tables:
+  template_appearance:
+    namespace: appearance
+    key_fields:
+      - LABEL
+`)
+	writeFile(t, filepath.Join(root, "topdata", "templates", "template_appearance.2da"), "2DA V2.0\n\nLABEL\n960 Zombie_Ash_Hot\n")
+
+	oldCwd, err := os.Getwd()
+	if err != nil {
+		t.Fatalf("getwd: %v", err)
+	}
+	t.Cleanup(func() {
+		_ = os.Chdir(oldCwd)
+	})
+	if err := os.Chdir(root); err != nil {
+		t.Fatalf("chdir: %v", err)
+	}
+
+	err = RunConvertCommand([]string{
+		"2da-to-module",
+		"--name", "ashzombies",
+		"topdata/templates/template_appearance.2da",
+	}, &bytes.Buffer{})
+	if err == nil || !strings.Contains(err.Error(), `unsupported module_output.filename_prefixes key "additions"`) {
+		t.Fatalf("expected invalid filename prefix type error, got %v", err)
+	}
+}
+
+func TestConvert2DAToModuleFailsOnDuplicateGeneratedKeysByDefault(t *testing.T) {
+	_, err := convert2DAToModule(parsed2DA{
+		Columns: []string{"LABEL"},
+		Rows: []map[string]any{
+			{"id": 1, "LABEL": "Zombie"},
+			{"id": 2, "LABEL": "Zombie"},
+		},
+	}, "topdata/data/appearance/modules/add_appearance_ashzombies.json", convertOptions{
+		Namespace:     "appearance",
+		CollisionMode: "error",
+	})
+	if err == nil || !strings.Contains(err.Error(), `duplicate generated key "appearance:zombie"`) {
+		t.Fatalf("expected duplicate generated key error, got %v", err)
+	}
+}
+
+func TestParse2DAFileRejectsRowsWithTooManyFields(t *testing.T) {
+	root := t.TempDir()
+	inputPath := filepath.Join(root, "bad.2da")
+	writeFile(t, inputPath, "2DA V2.0\n\nLABEL NAME\n0 one two three\n")
+
+	_, err := parse2DAFile(inputPath)
+	if err == nil || !strings.Contains(err.Error(), "has 3 values but only 2 columns are declared") {
+		t.Fatalf("expected row width error, got %v", err)
+	}
+}

internal/topdata/creaturespeed_migrate.go

@@ -0,0 +1,145 @@
+package topdata
+
+import (
+	"encoding/json"
+	"os"
+	"path/filepath"
+)
+
+func importLegacyCreaturespeed(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
+	_ = legacyTLK
+
+	legacyDir := filepath.Join(referenceBuilderDir, "data", "creaturespeed")
+	if _, err := os.Stat(legacyDir); err != nil {
+		if os.IsNotExist(err) {
+			return 0, nil
+		}
+		return 0, err
+	}
+
+	targetDir := filepath.Join(dataDir, "creaturespeed")
+	targetPath := filepath.Join(targetDir, "creaturespeed.json")
+	if _, err := os.Stat(targetPath); err == nil {
+		obj, err := loadJSONObject(targetPath)
+		if err == nil && countLegacyTLKRefsInValue(obj) == 0 {
+			return 0, nil
+		}
+	}
+
+	baseObj, err := loadJSONObject(filepath.Join(legacyDir, "base.json"))
+	if err != nil {
+		return 0, err
+	}
+	rows, err := mergeCreaturespeedRows(baseObj, filepath.Join(legacyDir, "modules"))
+	if err != nil {
+		return 0, err
+	}
+	if err := os.MkdirAll(targetDir, 0o755); err != nil {
+		return 0, err
+	}
+
+	if err := removePathIfExists(filepath.Join(targetDir, "base.json")); err != nil {
+		return 0, err
+	}
+	if err := removePathIfExists(filepath.Join(targetDir, "lock.json")); err != nil {
+		return 0, err
+	}
+	if err := removePathIfExists(filepath.Join(targetDir, "modules")); err != nil {
+		return 0, err
+	}
+
+	out := map[string]any{
+		"output":  "creaturespeed.2da",
+		"columns": baseObj["columns"],
+		"rows":    rows,
+	}
+
+	raw, err := json.MarshalIndent(out, "", "  ")
+	if err != nil {
+		return 0, err
+	}
+	raw = append(raw, '\n')
+	if err := os.WriteFile(targetPath, raw, 0o644); err != nil {
+		return 0, err
+	}
+	return 1, nil
+}
+
+func mergeCreaturespeedRows(baseObj map[string]any, modulesDir string) ([]any, error) {
+	rawRows, ok := baseObj["rows"].([]any)
+	if !ok {
+		return nil, nil
+	}
+	rows := make([]map[string]any, 0, len(rawRows))
+	byID := map[int]map[string]any{}
+	for _, raw := range rawRows {
+		row, ok := deepCopyValue(raw).(map[string]any)
+		if !ok {
+			continue
+		}
+		delete(row, "key")
+		rows = append(rows, row)
+		if rawID, ok := row["id"]; ok {
+			id, err := asInt(rawID)
+			if err != nil {
+				return nil, err
+			}
+			byID[id] = row
+		}
+	}
+
+	modulePaths, err := collectModulePaths(modulesDir)
+	if err != nil {
+		return nil, err
+	}
+	for _, path := range modulePaths {
+		obj, err := loadJSONObject(path)
+		if err != nil {
+			return nil, err
+		}
+		overrideList, ok := obj["overrides"].([]any)
+		if !ok {
+			continue
+		}
+		for _, raw := range overrideList {
+			override, ok := raw.(map[string]any)
+			if !ok {
+				continue
+			}
+			rawID, ok := override["id"]
+			if !ok {
+				continue
+			}
+			id, err := asInt(rawID)
+			if err != nil {
+				return nil, err
+			}
+			row := byID[id]
+			if row == nil {
+				continue
+			}
+			for key, value := range override {
+				if key == "id" || key == "key" || key == "_tlk" {
+					continue
+				}
+				row[key] = deepCopyValue(value)
+			}
+		}
+	}
+
+	out := make([]any, 0, len(rows))
+	for _, row := range rows {
+		out = append(out, row)
+	}
+	return out, nil
+}
+
+func removePathIfExists(path string) error {
+	if _, err := os.Stat(path); err != nil {
+		if os.IsNotExist(err) {
+			return nil
+		}
+		return err
+	}
+	return os.RemoveAll(path)
+}

internal/topdata/damagetypes_migrate.go

@@ -0,0 +1,288 @@
+package topdata
+
+import (
+	"encoding/json"
+	"fmt"
+	"os"
+	"path/filepath"
+	"sort"
+	"strings"
+)
+
+func importLegacyDamagetypes(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
+	_ = legacyTLK
+
+	legacyRoot := filepath.Join(referenceBuilderDir, "data", "damagetypes")
+	if _, err := os.Stat(legacyRoot); err != nil {
+		if os.IsNotExist(err) {
+			return 0, nil
+		}
+		return 0, err
+	}
+
+	targetDir := filepath.Join(dataDir, "damagetypes", "registry")
+	targetPath := filepath.Join(targetDir, "types.json")
+	if _, err := os.Stat(targetPath); err == nil {
+		obj, err := loadJSONObject(targetPath)
+		if err == nil && countLegacyTLKRefsInValue(obj) == 0 {
+			return 0, nil
+		}
+	}
+
+	coreBase, err := loadJSONObject(filepath.Join(legacyRoot, "core", "base.json"))
+	if err != nil {
+		return 0, err
+	}
+	groupBase, err := loadJSONObject(filepath.Join(legacyRoot, "groups", "base.json"))
+	if err != nil {
+		return 0, err
+	}
+	hitvisualBase, err := loadJSONObject(filepath.Join(legacyRoot, "hitvisual", "base.json"))
+	if err != nil {
+		return 0, err
+	}
+	coreModule, err := loadJSONObject(filepath.Join(legacyRoot, "core", "modules", "add_eos.json"))
+	if err != nil {
+		return 0, err
+	}
+	coreLock, err := loadLockfile(filepath.Join(legacyRoot, "core", "lock.json"))
+	if err != nil {
+		return 0, err
+	}
+	groupModule, err := loadJSONObject(filepath.Join(legacyRoot, "groups", "modules", "add_eos.json"))
+	if err != nil {
+		return 0, err
+	}
+	groupLock, err := loadLockfile(filepath.Join(legacyRoot, "groups", "lock.json"))
+	if err != nil {
+		return 0, err
+	}
+	hitvisualModule, err := loadJSONObject(filepath.Join(legacyRoot, "hitvisual", "modules", "add_eos.json"))
+	if err != nil {
+		return 0, err
+	}
+	hitvisualLock, err := loadLockfile(filepath.Join(legacyRoot, "hitvisual", "lock.json"))
+	if err != nil {
+		return 0, err
+	}
+
+	rows, lockData, err := mergeLegacyDamagetypes(coreBase, groupBase, hitvisualBase, coreModule, coreLock, groupModule, groupLock, hitvisualModule, hitvisualLock)
+	if err != nil {
+		return 0, err
+	}
+
+	if err := os.MkdirAll(targetDir, 0o755); err != nil {
+		return 0, err
+	}
+	if err := removePathIfExists(filepath.Join(dataDir, "damagetypes", "core")); err != nil {
+		return 0, err
+	}
+	if err := removePathIfExists(filepath.Join(dataDir, "damagetypes", "groups")); err != nil {
+		return 0, err
+	}
+	if err := removePathIfExists(filepath.Join(dataDir, "damagetypes", "hitvisual")); err != nil {
+		return 0, err
+	}
+
+	typesOut := map[string]any{"rows": rows}
+	raw, err := json.MarshalIndent(typesOut, "", "  ")
+	if err != nil {
+		return 0, err
+	}
+	raw = append(raw, '\n')
+	if err := os.WriteFile(targetPath, raw, 0o644); err != nil {
+		return 0, err
+	}
+	if err := saveLockfile(filepath.Join(targetDir, damagetypesRegistryLock), lockData); err != nil {
+		return 0, err
+	}
+	return 2, nil
+}
+
+func mergeLegacyDamagetypes(coreBase, groupBase, hitvisualBase, coreModule map[string]any, coreLock map[string]int, groupModule map[string]any, groupLock map[string]int, hitvisualModule map[string]any, hitvisualLock map[string]int) ([]map[string]any, map[string]int, error) {
+	coreRows, err := coerceRowMapSlice(coreBase["rows"])
+	if err != nil {
+		return nil, nil, err
+	}
+	groupRows, err := coerceRowMapSlice(groupBase["rows"])
+	if err != nil {
+		return nil, nil, err
+	}
+	hitRows, err := coerceRowMapSlice(hitvisualBase["rows"])
+	if err != nil {
+		return nil, nil, err
+	}
+	groupByID := map[int]map[string]any{}
+	for _, row := range groupRows {
+		id, err := asInt(row["id"])
+		if err != nil {
+			return nil, nil, err
+		}
+		groupByID[id] = row
+	}
+	hitByID := map[int]map[string]any{}
+	for _, row := range hitRows {
+		id, err := asInt(row["id"])
+		if err != nil {
+			return nil, nil, err
+		}
+		hitByID[id] = row
+	}
+
+	outRows := make([]map[string]any, 0, len(coreRows))
+	lockData := map[string]int{}
+	for _, core := range coreRows {
+		id, err := asInt(core["id"])
+		if err != nil {
+			return nil, nil, err
+		}
+		groupID, err := asInt(core["DamageTypeGroup"])
+		if err != nil {
+			return nil, nil, err
+		}
+		group := groupByID[groupID]
+		if group == nil {
+			return nil, nil, fmt.Errorf("core id %d references missing group %d", id, groupID)
+		}
+		hit := hitByID[id]
+		if hit == nil {
+			return nil, nil, fmt.Errorf("core id %d is missing hitvisual row", id)
+		}
+		key := "damagetype:" + normalizeDamagetypeKey(core["Label"])
+		lockData[key] = id
+		outRows = append(outRows, map[string]any{
+			"key":                      key,
+			"id":                       id,
+			"label":                    deepCopyValue(core["Label"]),
+			"charsheet_strref":         deepCopyValue(core["CharsheetStrref"]),
+			"damage_type_group":        strconvString(groupID),
+			"damage_ranged_projectile": deepCopyValue(core["DamageRangedProjectile"]),
+			"group_label":              deepCopyValue(group["Label"]),
+			"feedback_strref":          deepCopyValue(group["FeedbackStrref"]),
+			"color_r":                  deepCopyValue(group["ColorR"]),
+			"color_g":                  deepCopyValue(group["ColorG"]),
+			"color_b":                  deepCopyValue(group["ColorB"]),
+			"visual_effect_id":         deepCopyValue(hit["VisualEffectID"]),
+			"ranged_effect_id":         deepCopyValue(hit["RangedEffectID"]),
+		})
+	}
+
+	coreEntries, err := coerceEntriesMap(coreModule["entries"])
+	if err != nil {
+		return nil, nil, err
+	}
+	groupEntries, err := coerceEntriesMap(groupModule["entries"])
+	if err != nil {
+		return nil, nil, err
+	}
+	hitEntries, err := coerceEntriesMap(hitvisualModule["entries"])
+	if err != nil {
+		return nil, nil, err
+	}
+
+	keys := make([]string, 0, len(coreEntries))
+	for key := range coreEntries {
+		keys = append(keys, key)
+	}
+	sort.Slice(keys, func(i, j int) bool {
+		a := strings.TrimPrefix(keys[i], "damagetypes:")
+		b := strings.TrimPrefix(keys[j], "damagetypes:")
+		return a < b
+	})
+	for _, coreKey := range keys {
+		core := coreEntries[coreKey]
+		suffix := strings.TrimPrefix(coreKey, "damagetypes:")
+		group := groupEntries["damagetypegroups:"+suffix]
+		if group == nil {
+			return nil, nil, fmt.Errorf("%s: missing matching group entry", coreKey)
+		}
+		hit := hitEntries["damagehitvisual:"+suffix]
+		if hit == nil {
+			return nil, nil, fmt.Errorf("%s: missing matching hitvisual entry", coreKey)
+		}
+		id, ok := coreLock[coreKey]
+		if !ok {
+			return nil, nil, fmt.Errorf("%s: missing core lock id", coreKey)
+		}
+		groupKey := "damagetypegroups:" + suffix
+		groupID, ok := groupLock[groupKey]
+		if !ok {
+			return nil, nil, fmt.Errorf("%s: missing group lock id", groupKey)
+		}
+		hitKey := "damagehitvisual:" + suffix
+		hitID, ok := hitvisualLock[hitKey]
+		if !ok {
+			return nil, nil, fmt.Errorf("%s: missing hitvisual lock id", hitKey)
+		}
+		if hitID != id {
+			return nil, nil, fmt.Errorf("%s: core id %d does not match hitvisual id %d", suffix, id, hitID)
+		}
+		key := "damagetype:" + suffix
+		lockData[key] = id
+		outRows = append(outRows, map[string]any{
+			"key":                      key,
+			"id":                       id,
+			"label":                    deepCopyValue(core["Label"]),
+			"charsheet_strref":         deepCopyValue(core["CharsheetStrref"]),
+			"damage_type_group":        strconvString(groupID),
+			"damage_ranged_projectile": deepCopyValue(core["DamageRangedProjectile"]),
+			"group_label":              deepCopyValue(group["Label"]),
+			"feedback_strref":          deepCopyValue(group["FeedbackStrref"]),
+			"color_r":                  deepCopyValue(group["ColorR"]),
+			"color_g":                  deepCopyValue(group["ColorG"]),
+			"color_b":                  deepCopyValue(group["ColorB"]),
+			"visual_effect_id":         deepCopyValue(hit["VisualEffectID"]),
+			"ranged_effect_id":         deepCopyValue(hit["RangedEffectID"]),
+		})
+	}
+
+	sort.Slice(outRows, func(i, j int) bool {
+		return outRows[i]["id"].(int) < outRows[j]["id"].(int)
+	})
+	return outRows, lockData, nil
+}
+
+func coerceRowMapSlice(value any) ([]map[string]any, error) {
+	rawRows, ok := value.([]any)
+	if !ok {
+		return nil, fmt.Errorf("rows must be an array")
+	}
+	rows := make([]map[string]any, 0, len(rawRows))
+	for _, raw := range rawRows {
+		row, ok := raw.(map[string]any)
+		if !ok {
+			return nil, fmt.Errorf("row must be an object")
+		}
+		rows = append(rows, row)
+	}
+	return rows, nil
+}
+
+func coerceEntriesMap(value any) (map[string]map[string]any, error) {
+	rawEntries, ok := value.(map[string]any)
+	if !ok {
+		return nil, fmt.Errorf("entries must be an object")
+	}
+	entries := make(map[string]map[string]any, len(rawEntries))
+	for key, raw := range rawEntries {
+		row, ok := raw.(map[string]any)
+		if !ok {
+			return nil, fmt.Errorf("entry %s must be an object", key)
+		}
+		entries[key] = row
+	}
+	return entries, nil
+}
+
+func normalizeDamagetypeKey(value any) string {
+	text := strings.TrimSpace(strings.ToLower(format2DAValue(value)))
+	text = strings.TrimSuffix(text, "damage")
+	text = strings.ReplaceAll(text, " ", "")
+	text = strings.ReplaceAll(text, "_", "")
+	text = strings.ReplaceAll(text, "-", "")
+	return text
+}
+
+func strconvString(v int) string {
+	return fmt.Sprintf("%d", v)
+}

internal/topdata/damagetypes_registry.go

@@ -0,0 +1,243 @@
+package topdata
+
+import (
+	"fmt"
+	"os"
+	"path/filepath"
+	"slices"
+	"strconv"
+	"strings"
+)
+
+const (
+	damagetypesRegistryDirName = "damagetypes/registry"
+	damagetypesRegistryLock    = "lock.json"
+)
+
+type damagetypesRegistry struct {
+	RootPath string
+	LockPath string
+	LockData map[string]int
+	Types    []map[string]any
+}
+
+func collectGeneratedRegistryDatasets(dataDir string) ([]nativeCollectedDataset, error) {
+	itempropsDatasets, err := collectItempropsRegistryDatasets(dataDir)
+	if err != nil {
+		return nil, err
+	}
+	damagetypeDatasets, err := collectDamagetypesRegistryDatasets(dataDir)
+	if err != nil {
+		return nil, err
+	}
+	racialtypesDatasets, err := collectRacialtypesRegistryDatasets(dataDir)
+	if err != nil {
+		return nil, err
+	}
+	out := make([]nativeCollectedDataset, 0, len(itempropsDatasets)+len(damagetypeDatasets)+len(racialtypesDatasets))
+	out = append(out, itempropsDatasets...)
+	out = append(out, damagetypeDatasets...)
+	out = append(out, racialtypesDatasets...)
+	return out, nil
+}
+
+func loadDamagetypesRegistry(dataDir string) (*damagetypesRegistry, error) {
+	root := filepath.Join(dataDir, filepath.FromSlash(damagetypesRegistryDirName))
+	info, err := os.Stat(root)
+	if err != nil {
+		if os.IsNotExist(err) {
+			return nil, nil
+		}
+		return nil, err
+	}
+	if !info.IsDir() {
+		return nil, fmt.Errorf("damagetypes registry root must be a directory: %s", root)
+	}
+
+	lockData, err := loadLockfile(filepath.Join(root, damagetypesRegistryLock))
+	if err != nil {
+		return nil, err
+	}
+	types, err := loadRegistryRows(filepath.Join(root, "types.json"))
+	if err != nil {
+		return nil, err
+	}
+	return &damagetypesRegistry{
+		RootPath: root,
+		LockPath: filepath.Join(root, damagetypesRegistryLock),
+		LockData: lockData,
+		Types:    types,
+	}, nil
+}
+
+func collectDamagetypesRegistryDatasets(dataDir string) ([]nativeCollectedDataset, error) {
+	registry, err := loadDamagetypesRegistry(dataDir)
+	if err != nil {
+		return nil, err
+	}
+	if registry == nil {
+		return nil, nil
+	}
+
+	typeIDs, lockModified, err := assignDamagetypeRegistryIDs(registry.Types, registry.LockData)
+	if err != nil {
+		return nil, err
+	}
+	if lockModified {
+		if err := saveLockfile(registry.LockPath, registry.LockData); err != nil {
+			return nil, err
+		}
+	}
+
+	typeRowsSorted := slices.Clone(registry.Types)
+	slices.SortFunc(typeRowsSorted, func(a, b map[string]any) int {
+		return typeIDs[stringField(a, "key")] - typeIDs[stringField(b, "key")]
+	})
+
+	coreRows := make([]map[string]any, 0, len(typeRowsSorted))
+	hitvisualRows := make([]map[string]any, 0, len(typeRowsSorted))
+	groupRowsByID := map[int]map[string]any{}
+	groupOrder := make([]int, 0)
+	for _, row := range typeRowsSorted {
+		key := stringField(row, "key")
+		rowID := typeIDs[key]
+
+		groupID, err := registryIntField(row, "damage_type_group")
+		if err != nil {
+			return nil, fmt.Errorf("%s: %w", key, err)
+		}
+		coreRows = append(coreRows, map[string]any{
+			"id":                     rowID,
+			"key":                    key,
+			"Label":                  registryStringField(row, "label"),
+			"CharsheetStrref":        deepCopyValue(row["charsheet_strref"]),
+			"DamageTypeGroup":        strconv.Itoa(groupID),
+			"DamageRangedProjectile": registryNullableField(row, "damage_ranged_projectile"),
+		})
+		hitvisualRows = append(hitvisualRows, map[string]any{
+			"id":             rowID,
+			"key":            strings.Replace(key, "damagetype:", "damagehitvisual:", 1),
+			"Label":          registryStringField(row, "label"),
+			"VisualEffectID": registryNullableField(row, "visual_effect_id"),
+			"RangedEffectID": registryNullableField(row, "ranged_effect_id"),
+		})
+
+		if _, ok := groupRowsByID[groupID]; !ok {
+			groupRowsByID[groupID] = map[string]any{
+				"id":             groupID,
+				"key":            strings.Replace(key, "damagetype:", "damagetypegroup:", 1),
+				"Label":          registryStringField(row, "group_label"),
+				"FeedbackStrref": deepCopyValue(row["feedback_strref"]),
+				"ColorR":         registryNullableField(row, "color_r"),
+				"ColorG":         registryNullableField(row, "color_g"),
+				"ColorB":         registryNullableField(row, "color_b"),
+			}
+			groupOrder = append(groupOrder, groupID)
+			continue
+		}
+		existing := groupRowsByID[groupID]
+		for field, candidate := range map[string]any{
+			"Label":          registryStringField(row, "group_label"),
+			"FeedbackStrref": deepCopyValue(row["feedback_strref"]),
+			"ColorR":         registryNullableField(row, "color_r"),
+			"ColorG":         registryNullableField(row, "color_g"),
+			"ColorB":         registryNullableField(row, "color_b"),
+		} {
+			if format2DAValue(existing[field]) != format2DAValue(candidate) {
+				return nil, fmt.Errorf("%s: conflicting group projection for DamageTypeGroup %d field %s", key, groupID, field)
+			}
+		}
+	}
+	slices.Sort(groupOrder)
+	groupRows := make([]map[string]any, 0, len(groupOrder))
+	for _, id := range groupOrder {
+		groupRows = append(groupRows, groupRowsByID[id])
+	}
+
+	return []nativeCollectedDataset{
+		newGeneratedDataset("damagetypes/registry/core", "damagetypes.2da", []string{"Label", "CharsheetStrref", "DamageTypeGroup", "DamageRangedProjectile"}, coreRows),
+		newGeneratedDataset("damagetypes/registry/groups", "damagetypegroups.2da", []string{"Label", "FeedbackStrref", "ColorR", "ColorG", "ColorB"}, groupRows),
+		newGeneratedDataset("damagetypes/registry/hitvisual", "damagehitvisual.2da", []string{"Label", "VisualEffectID", "RangedEffectID"}, hitvisualRows),
+	}, nil
+}
+
+func assignDamagetypeRegistryIDs(rows []map[string]any, lockData map[string]int) (map[string]int, bool, error) {
+	ids := map[string]int{}
+	used := map[int]struct{}{}
+	for key, id := range lockData {
+		if strings.HasPrefix(key, "damagetype:") {
+			used[id] = struct{}{}
+		}
+	}
+	modified := false
+	for _, row := range rows {
+		key := stringField(row, "key")
+		if key == "" {
+			return nil, false, fmt.Errorf("registry row is missing key")
+		}
+		if !strings.HasPrefix(key, "damagetype:") {
+			return nil, false, fmt.Errorf("registry key %q must use prefix %q", key, "damagetype:")
+		}
+		if rawID, ok := row["id"]; ok {
+			id, err := asInt(rawID)
+			if err != nil {
+				return nil, false, fmt.Errorf("%s: invalid id: %w", key, err)
+			}
+			if existing, ok := lockData[key]; ok && existing != id {
+				return nil, false, fmt.Errorf("%s: explicit id %d does not match lock id %d", key, id, existing)
+			}
+			lockData[key] = id
+			ids[key] = id
+			used[id] = struct{}{}
+			modified = true
+			continue
+		}
+		if id, ok := lockData[key]; ok {
+			ids[key] = id
+			used[id] = struct{}{}
+		}
+	}
+	nextID := nextAvailableID(used)
+	for _, row := range rows {
+		key := stringField(row, "key")
+		if _, ok := ids[key]; ok {
+			continue
+		}
+		lockData[key] = nextID
+		ids[key] = nextID
+		used[nextID] = struct{}{}
+		nextID = nextAvailableID(used)
+		modified = true
+	}
+	return ids, modified, nil
+}
+
+func registryStringField(row map[string]any, field string) string {
+	text := stringField(row, field)
+	if text == "" {
+		return nullValue
+	}
+	return text
+}
+
+func registryNullableField(row map[string]any, field string) any {
+	value, ok := row[field]
+	if !ok {
+		return nullValue
+	}
+	switch typed := value.(type) {
+	case string:
+		if strings.TrimSpace(typed) == "" {
+			return nullValue
+		}
+	}
+	return deepCopyValue(value)
+}
+
+func registryIntField(row map[string]any, field string) (int, error) {
+	value, ok := row[field]
+	if !ok {
+		return 0, fmt.Errorf("missing %s", field)
+	}
+	return asInt(value)
+}

internal/topdata/doortypes_migrate.go

@@ -0,0 +1,99 @@
+package topdata
+
+import (
+	"encoding/json"
+	"os"
+	"path/filepath"
+)
+
+func importLegacyDoortypes(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
+	_ = legacyTLK
+
+	legacyDir := filepath.Join(referenceBuilderDir, "data", "doortypes")
+	if _, err := os.Stat(legacyDir); err != nil {
+		if os.IsNotExist(err) {
+			return 0, nil
+		}
+		return 0, err
+	}
+
+	targetDir := filepath.Join(dataDir, "doortypes")
+	targetBasePath := filepath.Join(targetDir, "base.json")
+	targetLockPath := filepath.Join(targetDir, "lock.json")
+	targetModulesDir := filepath.Join(targetDir, "modules")
+	moduleNames := []string{
+		"add_pld01.json",
+		"add_sic11.json",
+		"add_tapr.json",
+		"add_tdm01.json",
+		"add_tdx01.json",
+		"add_tei01.json",
+		"add_tfm01.json",
+	}
+	targetModulePaths := make([]string, 0, len(moduleNames))
+	for _, name := range moduleNames {
+		targetModulePaths = append(targetModulePaths, filepath.Join(targetModulesDir, name))
+	}
+	if fileExists(targetBasePath) && fileExists(targetLockPath) {
+		allModulesPresent := true
+		for _, path := range targetModulePaths {
+			if !fileExists(path) {
+				allModulesPresent = false
+				break
+			}
+		}
+		if allModulesPresent {
+			return 0, nil
+		}
+	}
+
+	baseObj, err := loadJSONObject(filepath.Join(legacyDir, "base.json"))
+	if err != nil {
+		return 0, err
+	}
+	baseObj["output"] = "doortypes.2da"
+
+	lockObj, err := loadJSONObject(filepath.Join(legacyDir, "lock.json"))
+	if err != nil {
+		return 0, err
+	}
+
+	moduleObjs := make([]map[string]any, 0, len(moduleNames))
+	for _, name := range moduleNames {
+		obj, err := loadJSONObject(filepath.Join(legacyDir, "modules", name))
+		if err != nil {
+			return 0, err
+		}
+		moduleObjs = append(moduleObjs, obj)
+	}
+
+	if err := os.MkdirAll(targetModulesDir, 0o755); err != nil {
+		return 0, err
+	}
+
+	writes := []struct {
+		path string
+		obj  map[string]any
+	}{
+		{path: targetBasePath, obj: baseObj},
+		{path: targetLockPath, obj: lockObj},
+	}
+	for i, path := range targetModulePaths {
+		writes = append(writes, struct {
+			path string
+			obj  map[string]any
+		}{path: path, obj: moduleObjs[i]})
+	}
+
+	for _, write := range writes {
+		raw, err := json.MarshalIndent(write.obj, "", "  ")
+		if err != nil {
+			return 0, err
+		}
+		raw = append(raw, '\n')
+		if err := os.WriteFile(write.path, raw, 0o644); err != nil {
+			return 0, err
+		}
+	}
+	return len(writes), nil
+}

internal/topdata/editorconfig_format.go

@@ -0,0 +1,246 @@
+package topdata
+
+import (
+	"bufio"
+	"bytes"
+	"fmt"
+	"os"
+	"path/filepath"
+	"regexp"
+	"strconv"
+	"strings"
+)
+
+type jsonFileFormatting struct {
+	Indent       string
+	LineEnding   string
+	FinalNewline bool
+}
+
+type editorConfigDocument struct {
+	Dir      string
+	Root     bool
+	Sections []editorConfigSection
+}
+
+type editorConfigSection struct {
+	Pattern    string
+	Properties map[string]string
+}
+
+func defaultJSONFileFormatting() jsonFileFormatting {
+	return jsonFileFormatting{
+		Indent:       "  ",
+		LineEnding:   "\n",
+		FinalNewline: true,
+	}
+}
+
+func resolveJSONFileFormatting(path string) (jsonFileFormatting, error) {
+	formatting := defaultJSONFileFormatting()
+	documents, err := editorConfigDocumentsForPath(path)
+	if err != nil {
+		return jsonFileFormatting{}, err
+	}
+	for _, document := range documents {
+		relative, err := filepath.Rel(document.Dir, path)
+		if err != nil {
+			return jsonFileFormatting{}, fmt.Errorf("resolve editorconfig path for %s relative to %s: %w", path, document.Dir, err)
+		}
+		relative = filepath.ToSlash(relative)
+		if strings.HasPrefix(relative, "../") || relative == ".." {
+			continue
+		}
+		for _, section := range document.Sections {
+			matches, err := editorConfigPatternMatches(section.Pattern, relative)
+			if err != nil {
+				return jsonFileFormatting{}, fmt.Errorf("%s/.editorconfig section [%s]: %w", document.Dir, section.Pattern, err)
+			}
+			if !matches {
+				continue
+			}
+			next, err := applyEditorConfigProperties(formatting, section.Properties)
+			if err != nil {
+				return jsonFileFormatting{}, fmt.Errorf("%s/.editorconfig section [%s]: %w", document.Dir, section.Pattern, err)
+			}
+			formatting = next
+		}
+	}
+	return formatting, nil
+}
+
+func editorConfigDocumentsForPath(path string) ([]editorConfigDocument, error) {
+	dir := filepath.Dir(path)
+	documents := []editorConfigDocument{}
+	for {
+		configPath := filepath.Join(dir, ".editorconfig")
+		raw, err := os.ReadFile(configPath)
+		if err == nil {
+			document, err := parseEditorConfig(configPath, raw)
+			if err != nil {
+				return nil, err
+			}
+			document.Dir = dir
+			documents = append(documents, document)
+			if document.Root {
+				break
+			}
+		} else if !os.IsNotExist(err) {
+			return nil, fmt.Errorf("read %s: %w", configPath, err)
+		}
+
+		parent := filepath.Dir(dir)
+		if parent == dir {
+			break
+		}
+		dir = parent
+	}
+	for left, right := 0, len(documents)-1; left < right; left, right = left+1, right-1 {
+		documents[left], documents[right] = documents[right], documents[left]
+	}
+	return documents, nil
+}
+
+func parseEditorConfig(path string, raw []byte) (editorConfigDocument, error) {
+	document := editorConfigDocument{}
+	scanner := bufio.NewScanner(bytes.NewReader(raw))
+	var current *editorConfigSection
+	for lineNumber := 1; scanner.Scan(); lineNumber++ {
+		line := strings.TrimSpace(scanner.Text())
+		if line == "" || strings.HasPrefix(line, "#") || strings.HasPrefix(line, ";") {
+			continue
+		}
+		if strings.HasPrefix(line, "[") {
+			end := strings.LastIndex(line, "]")
+			if end < 0 {
+				return editorConfigDocument{}, fmt.Errorf("%s:%d: malformed section header", path, lineNumber)
+			}
+			section := editorConfigSection{
+				Pattern:    strings.TrimSpace(line[1:end]),
+				Properties: map[string]string{},
+			}
+			document.Sections = append(document.Sections, section)
+			current = &document.Sections[len(document.Sections)-1]
+			continue
+		}
+
+		index := strings.IndexAny(line, "=:")
+		if index < 0 {
+			return editorConfigDocument{}, fmt.Errorf("%s:%d: expected key/value property", path, lineNumber)
+		}
+		key := strings.ToLower(strings.TrimSpace(line[:index]))
+		value := strings.TrimSpace(line[index+1:])
+		if current == nil {
+			if key == "root" {
+				document.Root = strings.EqualFold(value, "true")
+			}
+			continue
+		}
+		current.Properties[key] = value
+	}
+	if err := scanner.Err(); err != nil {
+		return editorConfigDocument{}, fmt.Errorf("scan %s: %w", path, err)
+	}
+	return document, nil
+}
+
+func applyEditorConfigProperties(formatting jsonFileFormatting, properties map[string]string) (jsonFileFormatting, error) {
+	style := strings.ToLower(strings.TrimSpace(properties["indent_style"]))
+	sizeText := strings.ToLower(strings.TrimSpace(properties["indent_size"]))
+	if style == "tab" {
+		formatting.Indent = "\t"
+	} else if style != "" && style != "space" && style != "unset" {
+		return jsonFileFormatting{}, fmt.Errorf("unsupported indent_style %q", properties["indent_style"])
+	}
+	if sizeText != "" && sizeText != "unset" && sizeText != "tab" {
+		size, err := strconv.Atoi(sizeText)
+		if err != nil || size < 1 {
+			return jsonFileFormatting{}, fmt.Errorf("indent_size must be a positive integer, got %q", properties["indent_size"])
+		}
+		if style != "tab" {
+			formatting.Indent = strings.Repeat(" ", size)
+		}
+	}
+
+	switch value := strings.ToLower(strings.TrimSpace(properties["end_of_line"])); value {
+	case "", "unset":
+	case "lf":
+		formatting.LineEnding = "\n"
+	case "crlf":
+		formatting.LineEnding = "\r\n"
+	case "cr":
+		formatting.LineEnding = "\r"
+	default:
+		return jsonFileFormatting{}, fmt.Errorf("unsupported end_of_line %q", properties["end_of_line"])
+	}
+
+	switch value := strings.ToLower(strings.TrimSpace(properties["insert_final_newline"])); value {
+	case "", "unset":
+	case "true":
+		formatting.FinalNewline = true
+	case "false":
+		formatting.FinalNewline = false
+	default:
+		return jsonFileFormatting{}, fmt.Errorf("insert_final_newline must be true or false, got %q", properties["insert_final_newline"])
+	}
+	return formatting, nil
+}
+
+func editorConfigPatternMatches(pattern, relativePath string) (bool, error) {
+	pattern = filepath.ToSlash(strings.TrimSpace(pattern))
+	pattern = strings.TrimPrefix(pattern, "/")
+	if pattern == "" {
+		return false, nil
+	}
+	if !strings.Contains(pattern, "/") {
+		relativePath = pathBase(relativePath)
+	}
+	expression, err := editorConfigGlobRegexp(pattern)
+	if err != nil {
+		return false, err
+	}
+	return regexp.MatchString(expression, relativePath)
+}
+
+func editorConfigGlobRegexp(pattern string) (string, error) {
+	var builder strings.Builder
+	builder.WriteByte('^')
+	for index := 0; index < len(pattern); {
+		char := pattern[index]
+		switch char {
+		case '*':
+			if index+1 < len(pattern) && pattern[index+1] == '*' {
+				if index+2 < len(pattern) && pattern[index+2] == '/' {
+					builder.WriteString("(?:.*/)?")
+					index += 3
+				} else {
+					builder.WriteString(".*")
+					index += 2
+				}
+				continue
+			}
+			builder.WriteString("[^/]*")
+		case '?':
+			builder.WriteString("[^/]")
+		case '.', '+', '(', ')', '|', '^', '$', '{', '}', '[', ']', '\\':
+			builder.WriteByte('\\')
+			builder.WriteByte(char)
+		default:
+			builder.WriteByte(char)
+		}
+		index++
+	}
+	builder.WriteByte('$')
+	expression := builder.String()
+	if _, err := regexp.Compile(expression); err != nil {
+		return "", err
+	}
+	return expression, nil
+}
+
+func pathBase(path string) string {
+	if index := strings.LastIndex(path, "/"); index >= 0 {
+		return path[index+1:]
+	}
+	return path
+}

internal/topdata/expansion_native.go

@@ -0,0 +1,284 @@
+package topdata
+
+import (
+	"fmt"
+	"slices"
+	"strings"
+)
+
+func normalizeMetadataKey(key string) string {
+	replacer := strings.NewReplacer("-", "_", " ", "_")
+	return strings.ToLower(replacer.Replace(strings.TrimSpace(key)))
+}
+
+func parseRowMetadata(raw any) (map[string]any, error) {
+	if raw == nil {
+		return nil, nil
+	}
+	obj, ok := raw.(map[string]any)
+	if !ok {
+		return nil, fmt.Errorf("meta must be an object")
+	}
+	if len(obj) == 0 {
+		return map[string]any{}, nil
+	}
+
+	meta := make(map[string]any, len(obj))
+	for key, value := range obj {
+		switch normalizeMetadataKey(key) {
+		case "family":
+			family, err := parseFamilyMetadata(value)
+			if err != nil {
+				return nil, fmt.Errorf("meta.family: %w", err)
+			}
+			meta["family"] = family
+		case "wiki":
+			wiki, err := parseWikiMetadata(value)
+			if err != nil {
+				return nil, fmt.Errorf("meta.wiki: %w", err)
+			}
+			if len(wiki) > 0 {
+				meta["wiki"] = wiki
+			}
+		default:
+			return nil, fmt.Errorf("unknown metadata key %q", key)
+		}
+	}
+	return meta, nil
+}
+
+func mergeExpansionData(collected []nativeCollectedDataset) ([]nativeCollectedDataset, error) {
+	datasetIndexMap := make(map[string]int)
+	for i, ds := range collected {
+		datasetIndexMap[ds.Dataset.Name] = i
+		datasetIndexMap[ds.Dataset.OutputName] = i
+	}
+
+	rowsWithExpansion := []struct {
+		datasetName string
+		row         map[string]any
+	}{}
+
+	for _, ds := range collected {
+		for _, row := range ds.Rows {
+			if hasExpansionData(row) {
+				rowsWithExpansion = append(rowsWithExpansion, struct {
+					datasetName string
+					row         map[string]any
+				}{ds.Dataset.Name, row})
+			}
+		}
+	}
+
+	if len(rowsWithExpansion) == 0 {
+		return collected, nil
+	}
+
+	modifiedTargets := map[int]struct{}{}
+	for _, src := range rowsWithExpansion {
+		expansion, _ := extractExpansionData(src.row)
+		for targetDatasetName, targetRows := range expansion.Data {
+			targetIndex, ok := datasetIndexMap[targetDatasetName]
+			if !ok {
+				outputName := targetDatasetName
+				if !strings.HasSuffix(outputName, ".2da") {
+					outputName = outputName + ".2da"
+				}
+				targetIndex, ok = datasetIndexMap[outputName]
+			}
+			if !ok {
+				return nil, fmt.Errorf("expansion targets unknown dataset %q", targetDatasetName)
+			}
+			targetDS := collected[targetIndex]
+			originalLockData, _ := loadLockfile(targetDS.Dataset.LockPath)
+			if originalLockData == nil {
+				originalLockData = map[string]int{}
+			}
+			lockModified := false
+			usedIDs := map[int]struct{}{}
+			usedKeys := map[string]struct{}{}
+			for _, rowID := range targetDS.LockData {
+				usedIDs[rowID] = struct{}{}
+			}
+			for _, row := range targetDS.Rows {
+				if id, ok := row["id"].(int); ok {
+					usedIDs[id] = struct{}{}
+				}
+				if key, ok := row["key"].(string); ok {
+					usedKeys[key] = struct{}{}
+				}
+			}
+			nextID := nextAvailableIDAtLeast(usedIDs, targetDS.Dataset.RowGenerationMinRow)
+
+			for _, targetRow := range targetRows {
+				var rowID int
+				var hasID bool
+				if rawID, ok := targetRow["id"]; ok {
+					switch typed := rawID.(type) {
+					case int:
+						rowID = typed
+						hasID = true
+					case float64:
+						rowID = int(typed)
+						hasID = true
+					case string:
+						parsed, err := asInt(typed)
+						if err == nil {
+							rowID = parsed
+							hasID = true
+						}
+					}
+				}
+
+				key, _ := targetRow["key"].(string)
+				if strings.TrimSpace(key) == "" {
+					return nil, fmt.Errorf("expansion into %s: injected rows must specify key", targetDatasetName)
+				}
+
+				if key != "" {
+					if existingID, exists := targetDS.LockData[key]; exists {
+						rowID = existingID
+						hasID = true
+					} else if existingID, exists := originalLockData[key]; exists && targetDS.Dataset.RowGeneration != "first_null_row" {
+						rowID = existingID
+						hasID = true
+						targetDS.LockData[key] = existingID
+						lockModified = true
+						targetDS.LockModified = true
+					}
+					if _, seen := usedKeys[key]; seen {
+						continue
+					}
+					usedKeys[key] = struct{}{}
+				}
+
+				if !hasID {
+					rowID = nextID
+					usedIDs[rowID] = struct{}{}
+					nextID = nextAvailableIDAtLeast(usedIDs, targetDS.Dataset.RowGenerationMinRow)
+				} else {
+					if _, exists := usedIDs[rowID]; exists && key == "" {
+						return nil, fmt.Errorf("expansion into %s: row id %d already exists", targetDatasetName, rowID)
+					}
+					usedIDs[rowID] = struct{}{}
+				}
+
+				if key != "" {
+					if _, exists := targetDS.LockData[key]; !exists {
+						targetDS.LockData[key] = rowID
+						lockModified = true
+						targetDS.LockAdded++
+						targetDS.LockModified = true
+					}
+				}
+
+				newRow := map[string]any{
+					"id": rowID,
+				}
+				for k, v := range targetRow {
+					if k != "id" {
+						newRow[k] = v
+					}
+				}
+				if key != "" {
+					newRow["key"] = key
+				}
+				targetDS.Rows = append(targetDS.Rows, newRow)
+			}
+
+			if lockModified {
+				modifiedTargets[targetIndex] = struct{}{}
+			}
+
+			slices.SortFunc(targetDS.Rows, func(a, b map[string]any) int {
+				return a["id"].(int) - b["id"].(int)
+			})
+			collected[targetIndex] = targetDS
+		}
+
+		if err := applyExpansionValueToRow(src.row, expansion.Value); err != nil {
+			return nil, err
+		}
+	}
+
+	for targetIndex := range modifiedTargets {
+		targetDS := collected[targetIndex]
+		referencedKeys, err := collectReferencedLockKeys(nativeDatasetSourceDir(targetDS.Dataset))
+		if err != nil {
+			return nil, fmt.Errorf("dataset %s: collect referenced keys: %w", targetDS.Dataset.Name, err)
+		}
+		if pruned, updated := pruneLockDataToActiveRows(targetDS.LockData, targetDS.Rows, referencedKeys, targetDS.RetiredKeys); pruned > 0 || updated > 0 {
+			targetDS.LockPruned += pruned
+			targetDS.LockModified = true
+			collected[targetIndex] = targetDS
+		}
+	}
+
+	return collected, nil
+}
+
+type expansionSpec struct {
+	Value string
+	Data  map[string][]map[string]any
+}
+
+func hasExpansionData(row map[string]any) bool {
+	for _, value := range row {
+		if isExpansionValue(value) {
+			return true
+		}
+	}
+	return false
+}
+
+func isExpansionValue(value any) bool {
+	obj, ok := value.(map[string]any)
+	if !ok {
+		return false
+	}
+	_, hasValue := obj["value"]
+	_, hasData := obj["data"]
+	return hasValue && hasData
+}
+
+func extractExpansionData(row map[string]any) (expansionSpec, bool) {
+	for _, value := range row {
+		if isExpansionValue(value) {
+			obj := value.(map[string]any)
+			valueStr, _ := obj["value"].(string)
+			dataMap, _ := obj["data"].(map[string]any)
+
+			result := expansionSpec{
+				Value: valueStr,
+				Data:  make(map[string][]map[string]any),
+			}
+
+			for datasetName, rawRows := range dataMap {
+				switch typed := rawRows.(type) {
+				case []any:
+					rows := make([]map[string]any, 0, len(typed))
+					for _, r := range typed {
+						if rowObj, ok := r.(map[string]any); ok {
+							rows = append(rows, rowObj)
+						}
+					}
+					result.Data[datasetName] = rows
+				case map[string]any:
+					result.Data[datasetName] = []map[string]any{typed}
+				}
+			}
+
+			return result, true
+		}
+	}
+	return expansionSpec{}, false
+}
+
+func applyExpansionValueToRow(row map[string]any, value string) error {
+	for field, oldValue := range row {
+		if isExpansionValue(oldValue) {
+			row[field] = value
+		}
+	}
+	return nil
+}

internal/topdata/family_expansion.go

@@ -0,0 +1,345 @@
+package topdata
+
+import (
+	"fmt"
+	"strings"
+)
+
+type familyIdentity struct {
+	Parent string
+	Child  string
+}
+
+type familyExpansionSource struct {
+	Dataset   string
+	Column    string
+	Predicate string
+}
+
+type familyExpansionTitleStyle struct {
+	ChildCase          string
+	ChildParenthetical string
+}
+
+type familyExpansionSpec struct {
+	Family            string
+	FamilyKey         string
+	Template          string
+	ChildSource       familyExpansionSource
+	NamePrefix        string
+	LabelPrefix       string
+	ConstantPrefix    string
+	TemplateFields    []string
+	DefaultFields     map[string]any
+	ApplyAfterModules bool
+	ChildRefField     string
+	IdentitySource    string
+	AllowExistingOnly bool
+	AutoPrereqFields  map[string]string
+	LegacyFamilyKeys  []string
+	TitleStyle        familyExpansionTitleStyle
+}
+
+func splitFamilyExpansionIdentity(text string) familyIdentity {
+	text = strings.TrimSpace(text)
+	if text == "" {
+		return familyIdentity{}
+	}
+	if idx := strings.Index(text, "_"); idx > 0 && idx < len(text)-1 {
+		return familyIdentity{
+			Parent: text[:idx],
+			Child:  text[idx+1:],
+		}
+	}
+	return familyIdentity{Parent: text}
+}
+
+func parseFamilyExpansionSource(raw any) (familyExpansionSource, error) {
+	if raw == nil {
+		return familyExpansionSource{}, fmt.Errorf("child_source is required")
+	}
+	obj, ok := raw.(map[string]any)
+	if !ok {
+		return familyExpansionSource{}, fmt.Errorf("child_source must be an object")
+	}
+	dataset, ok := obj["dataset"].(string)
+	if !ok || strings.TrimSpace(dataset) == "" {
+		return familyExpansionSource{}, fmt.Errorf("child_source.dataset must be a non-empty string")
+	}
+	source := familyExpansionSource{
+		Dataset: strings.TrimSpace(dataset),
+	}
+	if column, ok := obj["column"].(string); ok && strings.TrimSpace(column) != "" {
+		source.Column = strings.TrimSpace(column)
+	}
+	if predicate, ok := obj["predicate"].(string); ok && strings.TrimSpace(predicate) != "" {
+		source.Predicate = strings.TrimSpace(predicate)
+	}
+	return source, nil
+}
+
+func parseFamilyExpansionSpec(path string, obj map[string]any) (familyExpansionSpec, error) {
+	family, ok := obj["family"].(string)
+	if !ok || strings.TrimSpace(family) == "" {
+		return familyExpansionSpec{}, fmt.Errorf("generated file %s: family must be a non-empty string", path)
+	}
+	familyKey, ok := obj["family_key"].(string)
+	if !ok || strings.TrimSpace(familyKey) == "" {
+		return familyExpansionSpec{}, fmt.Errorf("generated file %s: family_key must be a non-empty string", path)
+	}
+	template, ok := obj["template"].(string)
+	if !ok || strings.TrimSpace(template) == "" {
+		return familyExpansionSpec{}, fmt.Errorf("generated file %s: template must be a non-empty string", path)
+	}
+	source, err := parseFamilyExpansionSource(obj["child_source"])
+	if err != nil {
+		return familyExpansionSpec{}, fmt.Errorf("generated file %s: %w", path, err)
+	}
+	namePrefix, _ := optionalTrimmedString(obj, "name_prefix")
+	labelPrefix, _ := optionalTrimmedString(obj, "label_prefix")
+	constantPrefix, _ := optionalTrimmedString(obj, "constant_prefix")
+	templateFields, err := parseOptionalStringArray(obj["template_fields"], "template_fields")
+	if err != nil {
+		return familyExpansionSpec{}, fmt.Errorf("generated file %s: %w", path, err)
+	}
+	defaultFields, err := parseOptionalObject(obj["default_fields"], "default_fields")
+	if err != nil {
+		return familyExpansionSpec{}, fmt.Errorf("generated file %s: %w", path, err)
+	}
+	applyAfterModules, err := parseOptionalBoolField(obj["apply_after_modules"], "apply_after_modules")
+	if err != nil {
+		return familyExpansionSpec{}, fmt.Errorf("generated file %s: %w", path, err)
+	}
+	childRefField, _ := optionalTrimmedString(obj, "child_ref_field")
+	identitySource, _ := optionalTrimmedString(obj, "identity_source")
+	switch identitySource {
+	case "", "child_source_value":
+	default:
+		return familyExpansionSpec{}, fmt.Errorf("generated file %s: identity_source must be empty or child_source_value", path)
+	}
+	autoPrereqFields, err := parseOptionalStringMap(obj["auto_prereq_fields"], "auto_prereq_fields")
+	if err != nil {
+		return familyExpansionSpec{}, fmt.Errorf("generated file %s: %w", path, err)
+	}
+	legacyFamilyKeys, err := parseOptionalStringArray(obj["legacy_family_keys"], "legacy_family_keys")
+	if err != nil {
+		return familyExpansionSpec{}, fmt.Errorf("generated file %s: %w", path, err)
+	}
+	if err := validateLegacyFamilyKeys(familyKey, legacyFamilyKeys); err != nil {
+		return familyExpansionSpec{}, fmt.Errorf("generated file %s: %w", path, err)
+	}
+	titleStyle, err := parseFamilyExpansionTitleStyle(obj["title_style"])
+	if err != nil {
+		return familyExpansionSpec{}, fmt.Errorf("generated file %s: %w", path, err)
+	}
+	allowExistingOnly, err := parseOptionalBoolField(obj["allow_existing_only"], "allow_existing_only")
+	if err != nil {
+		return familyExpansionSpec{}, fmt.Errorf("generated file %s: %w", path, err)
+	}
+	return familyExpansionSpec{
+		Family:            strings.TrimSpace(family),
+		FamilyKey:         strings.TrimSpace(familyKey),
+		Template:          strings.TrimSpace(template),
+		ChildSource:       source,
+		NamePrefix:        namePrefix,
+		LabelPrefix:       labelPrefix,
+		ConstantPrefix:    constantPrefix,
+		TemplateFields:    templateFields,
+		DefaultFields:     defaultFields,
+		ApplyAfterModules: applyAfterModules,
+		ChildRefField:     childRefField,
+		IdentitySource:    identitySource,
+		AllowExistingOnly: allowExistingOnly,
+		AutoPrereqFields:  autoPrereqFields,
+		LegacyFamilyKeys:  legacyFamilyKeys,
+		TitleStyle:        titleStyle,
+	}, nil
+}
+
+func parseFamilyExpansionTitleStyle(raw any) (familyExpansionTitleStyle, error) {
+	style := familyExpansionTitleStyle{
+		ChildCase:          "preserve",
+		ChildParenthetical: "preserve",
+	}
+	if raw == nil {
+		return style, nil
+	}
+	obj, ok := raw.(map[string]any)
+	if !ok {
+		return familyExpansionTitleStyle{}, fmt.Errorf("title_style must be an object")
+	}
+	if childCase, ok := obj["child_case"].(string); ok && strings.TrimSpace(childCase) != "" {
+		style.ChildCase = strings.TrimSpace(childCase)
+	}
+	switch style.ChildCase {
+	case "preserve", "lower":
+	default:
+		return familyExpansionTitleStyle{}, fmt.Errorf("title_style.child_case must be preserve or lower")
+	}
+	if childParenthetical, ok := obj["child_parenthetical"].(string); ok && strings.TrimSpace(childParenthetical) != "" {
+		style.ChildParenthetical = strings.TrimSpace(childParenthetical)
+	}
+	switch style.ChildParenthetical {
+	case "preserve", "comma":
+	default:
+		return familyExpansionTitleStyle{}, fmt.Errorf("title_style.child_parenthetical must be preserve or comma")
+	}
+	return style, nil
+}
+
+func validateLegacyFamilyKeys(familyKey string, legacyFamilyKeys []string) error {
+	seen := map[string]string{}
+	normalizedFamilyKey := normalizeKeyIdentity(familyKey)
+	for _, legacyKey := range legacyFamilyKeys {
+		normalizedLegacyKey := normalizeKeyIdentity(legacyKey)
+		if normalizedLegacyKey == normalizedFamilyKey {
+			return fmt.Errorf("legacy_family_keys must not include family_key %q", familyKey)
+		}
+		if previous, ok := seen[normalizedLegacyKey]; ok {
+			return fmt.Errorf("legacy_family_keys contains duplicate-equivalent keys %q and %q", previous, legacyKey)
+		}
+		seen[normalizedLegacyKey] = legacyKey
+	}
+	return nil
+}
+
+func isFamilyExpansionObject(obj map[string]any) bool {
+	_, hasFamilyKey := obj["family_key"]
+	_, hasTemplate := obj["template"]
+	_, hasChildSource := obj["child_source"]
+	_, hasNamePrefix := obj["name_prefix"]
+	_, hasLabelPrefix := obj["label_prefix"]
+	_, hasConstantPrefix := obj["constant_prefix"]
+	_, hasTemplateFields := obj["template_fields"]
+	_, hasDefaultFields := obj["default_fields"]
+	_, hasApplyAfterModules := obj["apply_after_modules"]
+	_, hasChildRefField := obj["child_ref_field"]
+	_, hasIdentitySource := obj["identity_source"]
+	_, hasAllowExistingOnly := obj["allow_existing_only"]
+	_, hasAutoPrereqFields := obj["auto_prereq_fields"]
+	_, hasLegacyFamilyKeys := obj["legacy_family_keys"]
+	_, hasTitleStyle := obj["title_style"]
+	return hasFamilyKey || hasTemplate || hasChildSource || hasNamePrefix || hasLabelPrefix ||
+		hasConstantPrefix || hasTemplateFields || hasDefaultFields || hasApplyAfterModules || hasChildRefField ||
+		hasIdentitySource || hasAllowExistingOnly || hasAutoPrereqFields || hasLegacyFamilyKeys || hasTitleStyle
+}
+
+func optionalTrimmedString(obj map[string]any, field string) (string, bool) {
+	text, ok := obj[field].(string)
+	if !ok || strings.TrimSpace(text) == "" {
+		return "", false
+	}
+	return strings.TrimSpace(text), true
+}
+
+func parseOptionalStringArray(raw any, field string) ([]string, error) {
+	if raw == nil {
+		return nil, nil
+	}
+	items, ok := raw.([]any)
+	if !ok {
+		return nil, fmt.Errorf("%s must be an array of strings", field)
+	}
+	out := make([]string, 0, len(items))
+	for _, item := range items {
+		text, ok := item.(string)
+		if !ok || strings.TrimSpace(text) == "" {
+			return nil, fmt.Errorf("%s must contain only non-empty strings", field)
+		}
+		out = append(out, strings.TrimSpace(text))
+	}
+	return out, nil
+}
+
+func parseOptionalObject(raw any, field string) (map[string]any, error) {
+	if raw == nil {
+		return nil, nil
+	}
+	obj, ok := raw.(map[string]any)
+	if !ok {
+		return nil, fmt.Errorf("%s must be an object", field)
+	}
+	return obj, nil
+}
+
+func parseOptionalStringMap(raw any, field string) (map[string]string, error) {
+	if raw == nil {
+		return nil, nil
+	}
+	obj, ok := raw.(map[string]any)
+	if !ok {
+		return nil, fmt.Errorf("%s must be an object of strings", field)
+	}
+	out := make(map[string]string, len(obj))
+	for key, value := range obj {
+		text, ok := value.(string)
+		if !ok || strings.TrimSpace(text) == "" {
+			return nil, fmt.Errorf("%s must contain only non-empty string values", field)
+		}
+		out[key] = strings.TrimSpace(text)
+	}
+	return out, nil
+}
+
+func parseOptionalBoolField(raw any, field string) (bool, error) {
+	if raw == nil {
+		return false, nil
+	}
+	value, ok := raw.(bool)
+	if !ok {
+		return false, fmt.Errorf("%s must be a boolean", field)
+	}
+	return value, nil
+}
+
+func familyMetadata(parent, child, source, template string) map[string]any {
+	meta := map[string]any{
+		"parent": strings.TrimSpace(parent),
+	}
+	if strings.TrimSpace(child) != "" {
+		meta["child"] = strings.TrimSpace(child)
+	}
+	if strings.TrimSpace(source) != "" {
+		meta["source"] = strings.TrimSpace(source)
+	}
+	if strings.TrimSpace(template) != "" {
+		meta["template"] = strings.TrimSpace(template)
+	}
+	return map[string]any{"family": meta}
+}
+
+func parseFamilyMetadata(raw any) (map[string]any, error) {
+	obj, ok := raw.(map[string]any)
+	if !ok {
+		return nil, fmt.Errorf("must be an object")
+	}
+	parent, ok := obj["parent"].(string)
+	if !ok || strings.TrimSpace(parent) == "" {
+		return nil, fmt.Errorf("parent must be a non-empty string")
+	}
+	meta := map[string]any{
+		"parent": strings.TrimSpace(parent),
+	}
+	if child, ok := obj["child"].(string); ok && strings.TrimSpace(child) != "" {
+		meta["child"] = strings.TrimSpace(child)
+	}
+	if source, ok := obj["source"].(string); ok && strings.TrimSpace(source) != "" {
+		meta["source"] = strings.TrimSpace(source)
+	}
+	if template, ok := obj["template"].(string); ok && strings.TrimSpace(template) != "" {
+		meta["template"] = strings.TrimSpace(template)
+	}
+	return meta, nil
+}
+
+func childTokenFromExpandedKey(key, parent string) string {
+	key = strings.TrimSpace(key)
+	key = strings.TrimPrefix(key, "feat:")
+	if strings.HasPrefix(key, parent+"_") {
+		return strings.TrimPrefix(key, parent+"_")
+	}
+	if strings.HasPrefix(key, parent) {
+		return strings.TrimPrefix(key, parent)
+	}
+	return ""
+}

internal/topdata/feat_migrate.go

@@ -0,0 +1,64 @@
+package topdata
+
+import (
+	"encoding/json"
+	"os"
+	"path/filepath"
+)
+
+func importLegacyFeat(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
+	_ = legacyTLK
+
+	legacyDir := filepath.Join(referenceBuilderDir, "data", "feat")
+	if _, err := os.Stat(legacyDir); err != nil {
+		if os.IsNotExist(err) {
+			return 0, nil
+		}
+		return 0, err
+	}
+	legacyBasePath := filepath.Join(legacyDir, "base.json")
+	if _, err := os.Stat(legacyBasePath); err != nil {
+		if os.IsNotExist(err) {
+			return 0, nil
+		}
+		return 0, err
+	}
+
+	targetDir := filepath.Join(dataDir, "feat")
+	targetBasePath := filepath.Join(targetDir, "base.json")
+	var targetObj map[string]any
+	if _, err := os.Stat(targetBasePath); err == nil {
+		targetObj, err = loadJSONObject(targetBasePath)
+		if err != nil {
+			return 0, err
+		}
+	} else if !os.IsNotExist(err) {
+		return 0, err
+	}
+	if targetObj != nil {
+		if rows, ok := targetObj["rows"].([]any); ok && len(rows) > 0 {
+			return 0, nil
+		}
+	}
+
+	baseObj, err := loadJSONObject(legacyBasePath)
+	if err != nil {
+		return 0, err
+	}
+	baseObj["output"] = "feat.2da"
+	baseObj["compare_reference"] = false
+	canonicalizeWikiMetadataDocument(baseObj)
+
+	if err := os.MkdirAll(targetDir, 0o755); err != nil {
+		return 0, err
+	}
+	raw, err := json.MarshalIndent(baseObj, "", "  ")
+	if err != nil {
+		return 0, err
+	}
+	raw = append(raw, '\n')
+	if err := os.WriteFile(targetBasePath, raw, 0o644); err != nil {
+		return 0, err
+	}
+	return 1, nil
+}

internal/topdata/generated_assets.go

@@ -0,0 +1,205 @@
+package topdata
+
+import (
+	"fmt"
+	"os"
+	"path/filepath"
+	"strings"
+
+	"git.westgate.pw/ShadowsOverWestgate/sow-tools/internal/project"
+)
+
+type Generated2DAAsset struct {
+	Rel        string
+	SourcePath string
+}
+
+func BuildGenerated2DAAssets(p *project.Project, progress func(string)) ([]Generated2DAAsset, error) {
+	if progress == nil {
+		progress = func(string) {}
+	}
+	configs := p.EffectiveConfig().Generated.TopData2DA
+	if len(configs) == 0 {
+		return nil, nil
+	}
+
+	results := make([]Generated2DAAsset, 0)
+	for _, cfg := range configs {
+		progress(fmt.Sprintf("Building generated 2DA assets for %s...", cfg.ID))
+		generated, err := buildGenerated2DAAssetGroup(p, cfg, progress)
+		if err != nil {
+			return nil, err
+		}
+		results = append(results, generated...)
+	}
+	return results, nil
+}
+
+func buildGenerated2DAAssetGroup(p *project.Project, cfg project.GeneratedTopData2DAConfig, progress func(string)) ([]Generated2DAAsset, error) {
+	sourceDir := filepath.Join(p.Root, filepath.FromSlash(strings.TrimSpace(cfg.Source)))
+	dataDir := filepath.Join(sourceDir, "data")
+	outputDir := filepath.Join(p.Root, filepath.FromSlash(strings.TrimSpace(cfg.Output)))
+
+	datasets, err := discoverNativeDatasets(dataDir)
+	if err != nil {
+		return nil, fmt.Errorf("discover generated topdata 2DA datasets %s: %w", cfg.ID, err)
+	}
+	selected := make([]nativeDataset, 0, len(datasets))
+	for _, dataset := range datasets {
+		if generatedDatasetIncluded(dataset.Name, cfg.IncludeDatasets) {
+			selected = append(selected, dataset)
+		}
+	}
+	if len(selected) == 0 {
+		return nil, fmt.Errorf("generated topdata 2DA config %s matched no datasets under %s", cfg.ID, dataDir)
+	}
+
+	collected := make([]nativeCollectedDataset, 0, len(selected))
+	for _, dataset := range selected {
+		collectedDataset, err := collectNativeDataset(dataset)
+		if err != nil {
+			return nil, err
+		}
+		collected = append(collected, collectedDataset)
+	}
+
+	consumer := cfg.Autogen
+	consumer.LocalOverrideRoot = p.AssetsDir()
+	collected, err = applyAutogenConsumersForGeneratedAssets(collected, consumer, progress)
+	if err != nil {
+		return nil, err
+	}
+	collected, err = normalizePartsRowsACBonus(collected, consumer.PartsRows)
+	if err != nil {
+		return nil, err
+	}
+	collected, err = applyPartOverridesWithConfig(sourceDir, collected, consumer.PartsRows)
+	if err != nil {
+		return nil, err
+	}
+	if err := rejectGenerated2DATLKValues(cfg.ID, collected); err != nil {
+		return nil, err
+	}
+	if _, _, err := saveNativeLockfiles(collected); err != nil {
+		return nil, fmt.Errorf("save generated topdata 2DA lockfiles %s: %w", cfg.ID, err)
+	}
+
+	tableRegistry, err := newResolvedTableRegistry(collected)
+	if err != nil {
+		return nil, err
+	}
+	keyToID, rowByKey := generated2DAGlobalRows(collected)
+
+	if err := os.RemoveAll(outputDir); err != nil {
+		return nil, fmt.Errorf("clean generated topdata 2DA output %s: %w", outputDir, err)
+	}
+	if err := os.MkdirAll(outputDir, 0o755); err != nil {
+		return nil, fmt.Errorf("create generated topdata 2DA output %s: %w", outputDir, err)
+	}
+
+	results := make([]Generated2DAAsset, 0, len(collected))
+	for _, dataset := range collected {
+		compiled, err := resolveNativeDataset(dataset, keyToID, rowByKey, tableRegistry, nil, project.TopDataClassFeatInjectionConfig{}, nil)
+		if err != nil {
+			return nil, err
+		}
+		outputPath := filepath.Join(outputDir, dataset.Dataset.OutputName)
+		denseRows := dataset.Dataset.Kind == nativeDatasetBase ||
+			(consumer.Mode == "parts_rows" && isPartsDataset(dataset.Dataset.Name))
+		if err := write2DA(compiled, outputPath, denseRows, -1); err != nil {
+			return nil, err
+		}
+		results = append(results, Generated2DAAsset{
+			Rel:        filepath.ToSlash(filepath.Join(cfg.PackageRoot, dataset.Dataset.OutputName)),
+			SourcePath: outputPath,
+		})
+	}
+	return results, nil
+}
+
+func applyAutogenConsumersForGeneratedAssets(collected []nativeCollectedDataset, consumer project.AutogenConsumerConfig, progress func(string)) ([]nativeCollectedDataset, error) {
+	if strings.TrimSpace(consumer.Mode) == "" {
+		return collected, nil
+	}
+	if progress == nil {
+		progress = func(string) {}
+	}
+	if !autogenConsumerTargetsCollectedDataset(collected, consumer) {
+		return collected, nil
+	}
+	if progress != nil {
+		progress(fmt.Sprintf("Scanning local generated 2DA autogen input for %s from %s...", consumer.ID, consumer.LocalOverrideRoot))
+	}
+	entries, err := scanLocalAutogenEntries(consumer.LocalOverrideRoot, consumer)
+	if err != nil {
+		return nil, err
+	}
+	switch consumer.Mode {
+	case "parts_rows":
+		return augmentWithAutogeneratedPartsWithConfig(collected, autogenPartsInventory(entries), consumer.PartsRows), nil
+	case "cachedmodels_rows":
+		return augmentWithAutogeneratedCachedModels(collected, entries)
+	default:
+		return nil, fmt.Errorf("unsupported generated topdata 2DA autogen mode %q", consumer.Mode)
+	}
+}
+
+func generatedDatasetIncluded(name string, patterns []string) bool {
+	name = filepath.ToSlash(strings.TrimSpace(name))
+	for _, pattern := range patterns {
+		pattern = filepath.ToSlash(strings.TrimSpace(pattern))
+		if pattern == "" {
+			continue
+		}
+		if pattern == name {
+			return true
+		}
+		if strings.HasSuffix(pattern, "/**") {
+			prefix := strings.TrimSuffix(pattern, "/**")
+			if name == prefix || strings.HasPrefix(name, prefix+"/") {
+				return true
+			}
+		}
+	}
+	return false
+}
+
+func generated2DAGlobalRows(collected []nativeCollectedDataset) (map[string]int, map[string]map[string]any) {
+	keyToID := map[string]int{}
+	rowByKey := map[string]map[string]any{}
+	for _, dataset := range collected {
+		for key, rowID := range dataset.LockData {
+			keyToID[key] = rowID
+		}
+		for _, row := range dataset.Rows {
+			key, _ := row["key"].(string)
+			if key == "" {
+				continue
+			}
+			if rowID, ok := row["id"].(int); ok {
+				keyToID[key] = rowID
+				rowByKey[key] = row
+			}
+		}
+	}
+	return keyToID, rowByKey
+}
+
+func rejectGenerated2DATLKValues(configID string, collected []nativeCollectedDataset) error {
+	for _, dataset := range collected {
+		for _, row := range dataset.Rows {
+			for field, value := range row {
+				if field == "id" || field == "key" {
+					continue
+				}
+				allowBare := columnMatchesSpec(dataset.Dataset.Spec, field)
+				if _, ok, err := parseTLKPayload(value, allowBare); err != nil {
+					return fmt.Errorf("generated topdata 2DA config %s dataset %s field %s has invalid TLK payload: %w", configID, dataset.Dataset.Name, field, err)
+				} else if ok {
+					return fmt.Errorf("generated topdata 2DA config %s dataset %s field %s uses TLK-backed text; 2DA-only generated asset builds do not generate TLK output", configID, dataset.Dataset.Name, field)
+				}
+			}
+		}
+	}
+	return nil
+}

internal/topdata/genericdoors_migrate.go

@@ -0,0 +1,6 @@
+package topdata
+
+func importLegacyGenericdoors(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
+	_ = legacyTLK
+	return importLegacyDatasetMirror(referenceBuilderDir, dataDir, "genericdoors", "genericdoors.2da", nil)
+}

internal/topdata/legacy_dataset_migrate.go

@@ -0,0 +1,176 @@
+package topdata
+
+import (
+	"encoding/json"
+	"io/fs"
+	"os"
+	"path/filepath"
+	"sort"
+	"strings"
+)
+
+type legacyDatasetTransform func(relativePath string, obj map[string]any)
+
+func importLegacyDatasetMirror(referenceBuilderDir, dataDir, datasetName, outputName string, transform legacyDatasetTransform) (int, error) {
+	legacyDir := filepath.Join(referenceBuilderDir, "data", datasetName)
+	if _, err := os.Stat(legacyDir); err != nil {
+		if os.IsNotExist(err) {
+			return 0, nil
+		}
+		return 0, err
+	}
+
+	targetDir := filepath.Join(dataDir, datasetName)
+	baseObj, err := loadJSONObject(filepath.Join(legacyDir, "base.json"))
+	if err != nil {
+		return 0, err
+	}
+	if outputName != "" {
+		baseObj["output"] = outputName
+	}
+	if transform != nil {
+		transform("base.json", baseObj)
+	}
+
+	lockObj, err := loadJSONObject(filepath.Join(legacyDir, "lock.json"))
+	if err != nil {
+		return 0, err
+	}
+	if transform != nil {
+		transform("lock.json", lockObj)
+	}
+
+	moduleRelPaths, err := collectJSONRelativePaths(filepath.Join(legacyDir, "modules"))
+	if err != nil {
+		return 0, err
+	}
+
+	moduleObjs := make(map[string]map[string]any, len(moduleRelPaths))
+	for _, relPath := range moduleRelPaths {
+		obj, err := loadJSONObject(filepath.Join(legacyDir, "modules", relPath))
+		if err != nil {
+			return 0, err
+		}
+		normalizedRel := filepath.ToSlash(relPath)
+		if transform != nil {
+			transform(filepath.ToSlash(filepath.Join("modules", normalizedRel)), obj)
+		}
+		moduleObjs[normalizedRel] = obj
+	}
+
+	if err := os.MkdirAll(targetDir, 0o755); err != nil {
+		return 0, err
+	}
+
+	updated := 0
+	for _, write := range []struct {
+		path string
+		obj  map[string]any
+	}{
+		{path: filepath.Join(targetDir, "base.json"), obj: baseObj},
+		{path: filepath.Join(targetDir, "lock.json"), obj: lockObj},
+	} {
+		changed, err := writeJSONObjectIfChanged(write.path, write.obj)
+		if err != nil {
+			return 0, err
+		}
+		if changed {
+			updated++
+		}
+	}
+
+	targetModulesDir := filepath.Join(targetDir, "modules")
+	if len(moduleObjs) > 0 {
+		if err := os.MkdirAll(targetModulesDir, 0o755); err != nil {
+			return 0, err
+		}
+	}
+	for relPath, obj := range moduleObjs {
+		targetPath := filepath.Join(targetModulesDir, filepath.FromSlash(relPath))
+		if err := os.MkdirAll(filepath.Dir(targetPath), 0o755); err != nil {
+			return 0, err
+		}
+		changed, err := writeJSONObjectIfChanged(targetPath, obj)
+		if err != nil {
+			return 0, err
+		}
+		if changed {
+			updated++
+		}
+	}
+
+	staleModules, err := collectJSONRelativePaths(targetModulesDir)
+	if err != nil {
+		return 0, err
+	}
+	for _, relPath := range staleModules {
+		normalizedRel := filepath.ToSlash(relPath)
+		if _, ok := moduleObjs[normalizedRel]; ok {
+			continue
+		}
+		if err := os.Remove(filepath.Join(targetModulesDir, filepath.FromSlash(normalizedRel))); err != nil && !os.IsNotExist(err) {
+			return 0, err
+		}
+		updated++
+	}
+
+	return updated, nil
+}
+
+func collectJSONRelativePaths(root string) ([]string, error) {
+	if _, err := os.Stat(root); err != nil {
+		if os.IsNotExist(err) {
+			return nil, nil
+		}
+		return nil, err
+	}
+
+	var relPaths []string
+	err := filepath.WalkDir(root, func(path string, d fs.DirEntry, err error) error {
+		if err != nil {
+			return err
+		}
+		if d.IsDir() {
+			return nil
+		}
+		if !strings.EqualFold(filepath.Ext(d.Name()), ".json") {
+			return nil
+		}
+		relPath, err := filepath.Rel(root, path)
+		if err != nil {
+			return err
+		}
+		relPaths = append(relPaths, filepath.ToSlash(relPath))
+		return nil
+	})
+	if err != nil {
+		return nil, err
+	}
+	sort.Strings(relPaths)
+	return relPaths, nil
+}
+
+func writeJSONObjectIfChanged(path string, obj map[string]any) (bool, error) {
+	raw, err := json.MarshalIndent(obj, "", "  ")
+	if err != nil {
+		return false, err
+	}
+	raw = append(raw, '\n')
+
+	current, err := os.ReadFile(path)
+	if err == nil && string(current) == string(raw) {
+		return false, nil
+	}
+	if err != nil && !os.IsNotExist(err) {
+		return false, err
+	}
+	if err := os.WriteFile(path, raw, 0o644); err != nil {
+		return false, err
+	}
+	return true, nil
+}
+
+func fileExists(path string) bool {
+	_, err := os.Stat(path)
+	return err == nil
+}

internal/topdata/loadscreens_migrate.go

@@ -0,0 +1,6 @@
+package topdata
+
+func importLegacyLoadscreens(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
+	_ = legacyTLK
+	return importLegacyDatasetMirror(referenceBuilderDir, dataDir, "loadscreens", "loadscreens.2da", nil)
+}

internal/topdata/masterfeats_migrate.go

@@ -0,0 +1,237 @@
+package topdata
+
+import (
+	"encoding/json"
+	"os"
+	"path/filepath"
+	"strconv"
+)
+
+func importLegacyMasterfeats(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
+	legacyDir := filepath.Join(referenceBuilderDir, "data", "feat", "masterfeats")
+	if _, err := os.Stat(legacyDir); err != nil {
+		if os.IsNotExist(err) {
+			return 0, nil
+		}
+		return 0, err
+	}
+
+	targetDir := filepath.Join(dataDir, "masterfeats")
+	targetPath := filepath.Join(targetDir, "base.json")
+	legacyPlainPath := filepath.Join(targetDir, "masterfeats.json")
+	if _, err := os.Stat(targetPath); err == nil {
+		obj, err := loadJSONObject(targetPath)
+		if err == nil && countLegacyTLKRefsInValue(obj) == 0 {
+			return 0, nil
+		}
+	}
+
+	collected, err := importLegacyMasterfeatsDataset(legacyDir)
+	if err != nil {
+		return 0, err
+	}
+	referenceIDs, err := collectReferenceLockIDs(filepath.Join(referenceBuilderDir, "data"))
+	if err != nil {
+		return 0, err
+	}
+
+	rows := make([]any, 0, len(collected.Rows))
+	for _, row := range collected.Rows {
+		copyRow, ok := deepCopyValue(row).(map[string]any)
+		if !ok {
+			continue
+		}
+		if legacyTLK != nil {
+			if _, _, err := inlineLegacyTLKValue(copyRow, legacyTLK); err != nil {
+				return 0, err
+			}
+		}
+		copyRow = rewriteImportedIDRefs(copyRow, referenceIDs)
+		rows = append(rows, copyRow)
+	}
+
+	if err := os.MkdirAll(targetDir, 0o755); err != nil {
+		return 0, err
+	}
+	writes := []struct {
+		path string
+		obj  map[string]any
+	}{
+		{
+			path: targetPath,
+			obj: map[string]any{
+				"output":  collected.Dataset.OutputName,
+				"columns": stringSliceToAny(collected.Columns),
+				"rows":    rows,
+			},
+		},
+		{
+			path: filepath.Join(targetDir, "lock.json"),
+			obj:  anyMapInt(collected.LockData),
+		},
+	}
+	for _, write := range writes {
+		raw, err := json.MarshalIndent(write.obj, "", "  ")
+		if err != nil {
+			return 0, err
+		}
+		raw = append(raw, '\n')
+		if err := os.WriteFile(write.path, raw, 0o644); err != nil {
+			return 0, err
+		}
+	}
+	updatedFiles := len(writes)
+	if err := os.Remove(legacyPlainPath); err == nil {
+		updatedFiles++
+	} else if err != nil && !os.IsNotExist(err) {
+		return 0, err
+	}
+	return updatedFiles, nil
+}
+
+func importLegacyMasterfeatsDataset(legacyDir string) (nativeCollectedDataset, error) {
+	dataset := nativeDataset{
+		Name:     "masterfeats",
+		BasePath: filepath.Join(legacyDir, "masterfeats.json"),
+		LockPath: filepath.Join(legacyDir, "lock.json"),
+		Spec:     specForDataset("masterfeats"),
+	}
+	tableData, err := loadJSONObject(dataset.BasePath)
+	if err != nil {
+		return nativeCollectedDataset{}, err
+	}
+	columns, err := parseColumns(tableData, dataset.Name)
+	if err != nil {
+		return nativeCollectedDataset{}, err
+	}
+	rawRows, ok := tableData["rows"].([]any)
+	if !ok {
+		return nativeCollectedDataset{}, nil
+	}
+	lockData, err := loadLockfile(dataset.LockPath)
+	if err != nil {
+		return nativeCollectedDataset{}, err
+	}
+	usedIDs := map[int]struct{}{}
+	for _, rowID := range lockData {
+		usedIDs[rowID] = struct{}{}
+	}
+
+	rows := make([]map[string]any, 0, len(rawRows))
+	explicitID := make([]bool, 0, len(rawRows))
+	for index, raw := range rawRows {
+		rowObj, ok := raw.(map[string]any)
+		if !ok {
+			continue
+		}
+		rowID := index
+		if value, ok := rowObj["id"]; ok {
+			parsed, err := asInt(value)
+			if err != nil {
+				return nativeCollectedDataset{}, err
+			}
+			rowID = parsed
+		}
+		row := map[string]any{"id": rowID}
+		for _, column := range columns {
+			row[column] = nullValue
+		}
+		for key, value := range rowObj {
+			switch key {
+			case "id":
+			case "key":
+				if keyText, ok := value.(string); ok && keyText != "" {
+					row["key"] = keyText
+				}
+			default:
+				columnName, ok := canonicalColumn(columns, key)
+				if !ok {
+					continue
+				}
+				row[columnName] = value
+			}
+		}
+		rows = append(rows, row)
+		_, hasExplicitID := rowObj["id"]
+		explicitID = append(explicitID, hasExplicitID)
+		if hasExplicitID {
+			usedIDs[rowID] = struct{}{}
+		}
+	}
+
+	nextID := nextAvailableID(usedIDs)
+	lockModified := false
+	for index, row := range rows {
+		key, hasKey := row["key"].(string)
+		if hasKey && key != "" {
+			if lockedID, ok := lockData[key]; ok {
+				row["id"] = lockedID
+				continue
+			}
+			if explicitID[index] {
+				lockData[key] = row["id"].(int)
+			} else {
+				row["id"] = nextID
+				lockData[key] = nextID
+				usedIDs[nextID] = struct{}{}
+				nextID = nextAvailableID(usedIDs)
+			}
+			lockModified = true
+			continue
+		}
+		if !explicitID[index] {
+			row["id"] = index
+		}
+	}
+	if lockModified {
+		_ = lockModified
+	}
+
+	return nativeCollectedDataset{
+		Dataset: nativeDataset{
+			Name:       dataset.Name,
+			OutputName: "masterfeats.2da",
+			Columns:    columns,
+			Spec:       dataset.Spec,
+		},
+		Columns:  columns,
+		Rows:     rows,
+		LockData: lockData,
+	}, nil
+}
+
+func rewriteImportedIDRefs(value any, ids map[string]int) map[string]any {
+	row, ok := rewriteImportedIDRefsValue(value, ids).(map[string]any)
+	if !ok {
+		return map[string]any{}
+	}
+	return row
+}
+
+func rewriteImportedIDRefsValue(value any, ids map[string]int) any {
+	switch typed := value.(type) {
+	case map[string]any:
+		if len(typed) == 1 {
+			if rawID, ok := typed["id"]; ok {
+				if key, ok := rawID.(string); ok {
+					if resolved, ok := ids[key]; ok {
+						return strconv.Itoa(resolved)
+					}
+				}
+			}
+		}
+		out := make(map[string]any, len(typed))
+		for key, child := range typed {
+			out[key] = rewriteImportedIDRefsValue(child, ids)
+		}
+		return out
+	case []any:
+		out := make([]any, 0, len(typed))
+		for _, child := range typed {
+			out = append(out, rewriteImportedIDRefsValue(child, ids))
+		}
+		return out
+	default:
+		return value
+	}
+}

internal/topdata/metadata_wiki.go

@@ -0,0 +1,190 @@
+package topdata
+
+import (
+	"fmt"
+	"strings"
+)
+
+func isNullLike(value any) bool {
+	if value == nil {
+		return true
+	}
+	text, ok := value.(string)
+	if !ok {
+		return false
+	}
+	return strings.TrimSpace(text) == nullValue
+}
+
+func normalizeWikiMetadataKey(key string) string {
+	switch normalizeMetadataKey(key) {
+	case "reqfeats", "req_feats":
+		return "reqfeats"
+	case "generate", "wikigenerate":
+		return "generate"
+	case "canonical", "wikicanonical":
+		return "canonical"
+	case "status", "wikistatus":
+		return "status"
+	case "progressionnotes", "progression_notes", "classprogressionnotes", "class_progression_notes":
+		return "progression_notes"
+	default:
+		return ""
+	}
+}
+
+func legacyWikiMetadataField(key string) string {
+	switch normalizeMetadataKey(key) {
+	case "wikireqfeats":
+		return "reqfeats"
+	case "wikigenerate":
+		return "generate"
+	case "wikicanonical":
+		return "canonical"
+	case "wikistatus":
+		return "status"
+	default:
+		return ""
+	}
+}
+
+func parseWikiMetadata(raw any) (map[string]any, error) {
+	obj, ok := raw.(map[string]any)
+	if !ok {
+		return nil, fmt.Errorf("must be an object")
+	}
+	wiki := map[string]any{}
+	for key, value := range obj {
+		normalized := normalizeWikiMetadataKey(key)
+		if normalized == "" {
+			return nil, fmt.Errorf("unknown field %q", key)
+		}
+		if isNullLike(value) {
+			continue
+		}
+		wiki[normalized] = cloneAuthoringValue(value)
+	}
+	return wiki, nil
+}
+
+func mergeMetadataMaps(base, overlay map[string]any) map[string]any {
+	if len(base) == 0 && len(overlay) == 0 {
+		return nil
+	}
+	out := map[string]any{}
+	for key, value := range base {
+		out[key] = cloneAuthoringValue(value)
+	}
+	for key, value := range overlay {
+		baseValue, hasBase := out[key]
+		baseMap, baseIsMap := baseValue.(map[string]any)
+		valueMap, valueIsMap := value.(map[string]any)
+		if hasBase && baseIsMap && valueIsMap {
+			out[key] = mergeMetadataMaps(baseMap, valueMap)
+			continue
+		}
+		out[key] = cloneAuthoringValue(value)
+	}
+	if len(out) == 0 {
+		return nil
+	}
+	return out
+}
+
+func parseExistingMetadata(row map[string]any) (map[string]any, error) {
+	rawMeta, ok := lookupField(row, "meta")
+	if !ok {
+		return nil, nil
+	}
+	return parseRowMetadata(rawMeta)
+}
+
+func setLegacyWikiMetadata(row map[string]any, field string, value any) error {
+	wikiKey := legacyWikiMetadataField(field)
+	if wikiKey == "" {
+		return nil
+	}
+
+	meta, err := parseExistingMetadata(row)
+	if err != nil {
+		return err
+	}
+	if meta == nil {
+		meta = map[string]any{}
+	}
+
+	wiki, _ := meta["wiki"].(map[string]any)
+	if wiki == nil {
+		wiki = map[string]any{}
+	}
+
+	if isNullLike(value) {
+		delete(wiki, wikiKey)
+	} else {
+		wiki[wikiKey] = cloneAuthoringValue(value)
+	}
+
+	if len(wiki) == 0 {
+		delete(meta, "wiki")
+	} else {
+		meta["wiki"] = wiki
+	}
+	if len(meta) == 0 {
+		delete(row, "meta")
+	} else {
+		row["meta"] = meta
+	}
+	return nil
+}
+
+func canonicalizeWikiMetadataDocument(obj map[string]any) {
+	if rawColumns, ok := obj["columns"].([]any); ok {
+		filtered := make([]any, 0, len(rawColumns))
+		for _, raw := range rawColumns {
+			column, ok := raw.(string)
+			if ok && isAuthoringOnlyField(column) {
+				continue
+			}
+			filtered = append(filtered, raw)
+		}
+		obj["columns"] = filtered
+	}
+	if rows, ok := obj["rows"].([]any); ok {
+		for _, raw := range rows {
+			row, ok := raw.(map[string]any)
+			if !ok {
+				continue
+			}
+			canonicalizeWikiMetadataRow(row)
+		}
+	}
+	if entries, ok := obj["entries"].(map[string]any); ok {
+		for _, raw := range entries {
+			row, ok := raw.(map[string]any)
+			if !ok {
+				continue
+			}
+			canonicalizeWikiMetadataRow(row)
+		}
+	}
+	if overrides, ok := obj["overrides"].([]any); ok {
+		for _, raw := range overrides {
+			row, ok := raw.(map[string]any)
+			if !ok {
+				continue
+			}
+			canonicalizeWikiMetadataRow(row)
+		}
+	}
+}
+
+func canonicalizeWikiMetadataRow(row map[string]any) {
+	for _, field := range []string{"WIKIREQFEATS", "WIKIGENERATE", "WIKICANONICAL", "WIKISTATUS"} {
+		value, ok := row[field]
+		if !ok {
+			continue
+		}
+		_ = setLegacyWikiMetadata(row, field, value)
+		delete(row, field)
+	}
+}

internal/topdata/migrate.go

@@ -0,0 +1,415 @@
+package topdata
+
+import (
+	"encoding/json"
+	"fmt"
+	"os"
+	"path/filepath"
+	"strings"
+
+	"git.westgate.pw/ShadowsOverWestgate/sow-tools/internal/project"
+)
+
+type NormalizeResult struct {
+	StatePath           string
+	ScannedFiles        int
+	UpdatedFiles        int
+	InlineValues        int
+	RemainingFiles      int
+	RemainingLegacyRefs int
+}
+
+func NormalizeProject(p *project.Project) (NormalizeResult, error) {
+	if !p.HasTopData() {
+		return NormalizeResult{}, fmt.Errorf("topdata is not configured for this project")
+	}
+
+	sourceDir := p.TopDataSourceDir()
+	dataDir := filepath.Join(sourceDir, "data")
+	migrationRoot := resolveMigrationInputRoot(sourceDir, p.TopDataReferenceBuilderDir())
+	legacyTLK, err := loadLegacyTLK(filepath.Join(sourceDir, "tlk"))
+	if err != nil {
+		return NormalizeResult{}, err
+	}
+	if migrationRoot != "" {
+		migrationTLK, err := loadLegacyTLK(filepath.Join(migrationRoot, "tlk"))
+		if err != nil {
+			return NormalizeResult{}, err
+		}
+		legacyTLK = mergeLegacyTLK(migrationTLK, legacyTLK)
+	}
+	baseDialogImported, err := importLegacyBaseDialog(migrationRoot, sourceDir)
+	if err != nil {
+		return NormalizeResult{}, err
+	}
+
+	itempropsImported, err := importLegacyItempropsRegistry(migrationRoot, dataDir, legacyTLK)
+	if err != nil {
+		return NormalizeResult{}, err
+	}
+	masterfeatsImported, err := importLegacyMasterfeats(migrationRoot, dataDir, legacyTLK)
+	if err != nil {
+		return NormalizeResult{}, err
+	}
+	featImported, err := importLegacyFeat(migrationRoot, dataDir, legacyTLK)
+	if err != nil {
+		return NormalizeResult{}, err
+	}
+	creaturespeedImported, err := importLegacyCreaturespeed(migrationRoot, dataDir, legacyTLK)
+	if err != nil {
+		return NormalizeResult{}, err
+	}
+	armorImported, err := importLegacyArmor(migrationRoot, dataDir, legacyTLK)
+	if err != nil {
+		return NormalizeResult{}, err
+	}
+	baseitemsImported, err := importLegacyBaseitems(migrationRoot, dataDir, legacyTLK)
+	if err != nil {
+		return NormalizeResult{}, err
+	}
+	damagetypesImported, err := importLegacyDamagetypes(migrationRoot, dataDir, legacyTLK)
+	if err != nil {
+		return NormalizeResult{}, err
+	}
+	skyboxesImported, err := importLegacySkyboxes(migrationRoot, dataDir, legacyTLK)
+	if err != nil {
+		return NormalizeResult{}, err
+	}
+	vfxPersistentImported, err := importLegacyVFXPersistent(migrationRoot, dataDir, legacyTLK)
+	if err != nil {
+		return NormalizeResult{}, err
+	}
+	loadscreensImported, err := importLegacyLoadscreens(migrationRoot, dataDir, legacyTLK)
+	if err != nil {
+		return NormalizeResult{}, err
+	}
+	progfxImported, err := importLegacyProgfx(migrationRoot, dataDir, legacyTLK)
+	if err != nil {
+		return NormalizeResult{}, err
+	}
+	genericdoorsImported, err := importLegacyGenericdoors(migrationRoot, dataDir, legacyTLK)
+	if err != nil {
+		return NormalizeResult{}, err
+	}
+	cloakmodelImported, err := importLegacyCloakmodel(migrationRoot, dataDir, legacyTLK)
+	if err != nil {
+		return NormalizeResult{}, err
+	}
+	repadjustImported, err := importLegacyRepadjust(migrationRoot, dataDir, legacyTLK)
+	if err != nil {
+		return NormalizeResult{}, err
+	}
+	wingmodelImported, err := importLegacyWingmodel(migrationRoot, dataDir, legacyTLK)
+	if err != nil {
+		return NormalizeResult{}, err
+	}
+	tailmodelImported, err := importLegacyTailmodel(migrationRoot, dataDir, legacyTLK)
+	if err != nil {
+		return NormalizeResult{}, err
+	}
+	doortypesImported, err := importLegacyDoortypes(migrationRoot, dataDir, legacyTLK)
+	if err != nil {
+		return NormalizeResult{}, err
+	}
+	visualeffectsImported, err := importLegacyVisualeffects(migrationRoot, dataDir, legacyTLK)
+	if err != nil {
+		return NormalizeResult{}, err
+	}
+	appearanceImported, err := importLegacyAppearance(migrationRoot, dataDir, legacyTLK)
+	if err != nil {
+		return NormalizeResult{}, err
+	}
+	placeablesImported, err := importLegacyPlaceables(migrationRoot, dataDir, legacyTLK)
+	if err != nil {
+		return NormalizeResult{}, err
+	}
+	rulesetImported, err := importLegacyRuleset(migrationRoot, dataDir, legacyTLK)
+	if err != nil {
+		return NormalizeResult{}, err
+	}
+	skillsImported, err := importLegacySkills(migrationRoot, dataDir, legacyTLK)
+	if err != nil {
+		return NormalizeResult{}, err
+	}
+	spellsImported, err := importLegacySpells(migrationRoot, dataDir, legacyTLK)
+	if err != nil {
+		return NormalizeResult{}, err
+	}
+	racialtypesImported, err := importLegacyRacialtypes(migrationRoot, dataDir, legacyTLK)
+	if err != nil {
+		return NormalizeResult{}, err
+	}
+	classesImported, err := importLegacyClasses(migrationRoot, dataDir, legacyTLK)
+	if err != nil {
+		return NormalizeResult{}, err
+	}
+	portraitsImported, err := importLegacyPortraits(migrationRoot, dataDir, legacyTLK)
+	if err != nil {
+		return NormalizeResult{}, err
+	}
+
+	paths, err := collectDataJSONPaths(dataDir)
+	if err != nil {
+		return NormalizeResult{}, err
+	}
+
+	result := NormalizeResult{
+		StatePath:    filepath.Join(sourceDir, tlkStateFile),
+		ScannedFiles: len(paths),
+		UpdatedFiles: baseDialogImported + itempropsImported + masterfeatsImported + featImported + creaturespeedImported + armorImported + baseitemsImported + damagetypesImported + skyboxesImported + vfxPersistentImported + loadscreensImported + progfxImported + genericdoorsImported + cloakmodelImported + repadjustImported + wingmodelImported + tailmodelImported + doortypesImported + visualeffectsImported + appearanceImported + placeablesImported + rulesetImported + skillsImported + spellsImported + racialtypesImported + classesImported + portraitsImported,
+	}
+	state, err := loadTLKState(result.StatePath)
+	if err != nil {
+		return result, err
+	}
+	if state.Entries == nil {
+		state.Entries = map[string]tlkStateMapping{}
+	}
+	if legacyTLK != nil && state.Language == "" {
+		state.Language = legacyTLK.Language
+	}
+	if legacyTLK != nil {
+		for key, id := range legacyTLK.Lock {
+			if _, ok := state.Entries[key]; !ok {
+				state.Entries[key] = tlkStateMapping{ID: id}
+			}
+		}
+	}
+
+	if legacyTLK != nil {
+		for _, path := range paths {
+			updated, count, err := inlineLegacyTLKFile(path, legacyTLK)
+			if err != nil {
+				return result, err
+			}
+			if updated {
+				result.UpdatedFiles++
+				result.InlineValues += count
+			}
+		}
+	}
+
+	if err := saveTLKState(result.StatePath, state); err != nil {
+		return result, err
+	}
+	if err := removeLegacyTLKAuthoredInput(filepath.Join(sourceDir, "tlk")); err != nil {
+		return result, err
+	}
+	remainingFiles, remainingRefs, err := countLegacyTLKRefs(paths)
+	if err != nil {
+		return result, err
+	}
+	result.RemainingFiles = remainingFiles
+	result.RemainingLegacyRefs = remainingRefs
+	return result, nil
+}
+
+func mergeLegacyTLK(base, override *legacyTLKData) *legacyTLKData {
+	if base == nil && override == nil {
+		return nil
+	}
+	merged := &legacyTLKData{
+		Language: "en",
+		Entries:  map[string]tlkEntryData{},
+		Lock:     map[string]int{},
+	}
+	if base != nil {
+		if base.Language != "" {
+			merged.Language = base.Language
+		}
+		for key, entry := range base.Entries {
+			merged.Entries[key] = entry
+		}
+		for key, id := range base.Lock {
+			merged.Lock[key] = id
+		}
+	}
+	if override != nil {
+		if override.Language != "" {
+			merged.Language = override.Language
+		}
+		for key, entry := range override.Entries {
+			merged.Entries[key] = entry
+		}
+		for key, id := range override.Lock {
+			merged.Lock[key] = id
+		}
+	}
+	return merged
+}
+
+func removeLegacyTLKAuthoredInput(tlkDir string) error {
+	modulesDir := filepath.Join(tlkDir, "modules")
+	if err := os.RemoveAll(modulesDir); err != nil {
+		return fmt.Errorf("remove %s: %w", modulesDir, err)
+	}
+	lockPath := filepath.Join(tlkDir, "lock.json")
+	if err := os.Remove(lockPath); err != nil && !os.IsNotExist(err) {
+		return fmt.Errorf("remove %s: %w", lockPath, err)
+	}
+	gitkeepPath := filepath.Join(tlkDir, ".gitkeep")
+	if err := os.Remove(gitkeepPath); err != nil && !os.IsNotExist(err) {
+		return fmt.Errorf("remove %s: %w", gitkeepPath, err)
+	}
+	if err := os.Remove(tlkDir); err != nil && !os.IsNotExist(err) {
+		if !strings.Contains(err.Error(), "directory not empty") {
+			return fmt.Errorf("remove %s: %w", tlkDir, err)
+		}
+	}
+	return nil
+}
+
+func resolveMigrationInputRoot(sourceDir, referenceBuilderDir string) string {
+	if strings.TrimSpace(referenceBuilderDir) != "" {
+		return referenceBuilderDir
+	}
+	snapshotRoot := filepath.Join(sourceDir, "migration_snapshot")
+	if info, err := os.Stat(snapshotRoot); err == nil && info.IsDir() {
+		return snapshotRoot
+	}
+	return ""
+}
+
+func inlineLegacyTLKFile(path string, legacy *legacyTLKData) (bool, int, error) {
+	obj, err := loadJSONObject(path)
+	if err != nil {
+		return false, 0, err
+	}
+	updated, count, err := inlineLegacyTLKValue(obj, legacy)
+	if err != nil {
+		return false, 0, err
+	}
+	if !updated {
+		return false, 0, nil
+	}
+	raw, err := json.MarshalIndent(obj, "", "  ")
+	if err != nil {
+		return false, 0, err
+	}
+	raw = append(raw, '\n')
+	if err := os.WriteFile(path, raw, 0o644); err != nil {
+		return false, 0, err
+	}
+	return true, count, nil
+}
+
+func inlineLegacyTLKValue(value any, legacy *legacyTLKData) (bool, int, error) {
+	switch typed := value.(type) {
+	case map[string]any:
+		if rawTLK, ok := typed["tlk"]; ok {
+			key, ok := rawTLK.(string)
+			if ok {
+				entry, ok := legacy.Entries[key]
+				if !ok {
+					return false, 0, nil
+				}
+				payload := map[string]any{
+					"key":  key,
+					"text": entry.Text,
+				}
+				if entry.SoundResRef != "" {
+					payload["sound_resref"] = entry.SoundResRef
+				}
+				if entry.SoundLength != 0 {
+					payload["sound_length"] = entry.SoundLength
+				}
+				typed["tlk"] = payload
+				return true, 1, nil
+			}
+		}
+
+		updated := false
+		total := 0
+		for key, child := range typed {
+			childUpdated, childCount, err := inlineLegacyTLKValue(child, legacy)
+			if err != nil {
+				return false, 0, err
+			}
+			if childUpdated {
+				typed[key] = child
+				updated = true
+				total += childCount
+			}
+		}
+		return updated, total, nil
+	case []any:
+		updated := false
+		total := 0
+		for index, child := range typed {
+			childUpdated, childCount, err := inlineLegacyTLKValue(child, legacy)
+			if err != nil {
+				return false, 0, err
+			}
+			if childUpdated {
+				typed[index] = child
+				updated = true
+				total += childCount
+			}
+		}
+		return updated, total, nil
+	default:
+		return false, 0, nil
+	}
+}
+
+func collectDataJSONPaths(dataDir string) ([]string, error) {
+	paths := make([]string, 0)
+	err := filepath.WalkDir(dataDir, func(path string, d os.DirEntry, err error) error {
+		if err != nil {
+			return err
+		}
+		if d.IsDir() {
+			return nil
+		}
+		if strings.HasSuffix(strings.ToLower(d.Name()), ".json") && d.Name() != "lock.json" {
+			paths = append(paths, path)
+		}
+		return nil
+	})
+	if err != nil {
+		return nil, err
+	}
+	return paths, nil
+}
+
+func countLegacyTLKRefs(paths []string) (int, int, error) {
+	files := 0
+	refs := 0
+	for _, path := range paths {
+		obj, err := loadJSONObject(path)
+		if err != nil {
+			return 0, 0, err
+		}
+		count := countLegacyTLKRefsInValue(obj)
+		if count == 0 {
+			continue
+		}
+		files++
+		refs += count
+	}
+	return files, refs, nil
+}
+
+func countLegacyTLKRefsInValue(value any) int {
+	switch typed := value.(type) {
+	case map[string]any:
+		total := 0
+		if rawTLK, ok := typed["tlk"]; ok {
+			if _, ok := rawTLK.(string); ok {
+				total++
+			}
+		}
+		for _, child := range typed {
+			total += countLegacyTLKRefsInValue(child)
+		}
+		return total
+	case []any:
+		total := 0
+		for _, child := range typed {
+			total += countLegacyTLKRefsInValue(child)
+		}
+		return total
+	default:
+		return 0
+	}
+}

internal/topdata/parts_discovery.go

@@ -0,0 +1,692 @@
+package topdata
+
+import (
+	"encoding/json"
+	"fmt"
+	"os"
+	"path/filepath"
+	"regexp"
+	"slices"
+	"strconv"
+	"strings"
+
+	"git.westgate.pw/ShadowsOverWestgate/sow-tools/internal/project"
+)
+
+// supportedPartCategories defines the part types that should be auto-discovered
+// from the sow-assets repository.
+var supportedPartCategories = []string{
+	"belt",
+	"bicep",
+	"chest",
+	"foot",
+	"forearm",
+	"leg",
+	"neck",
+	"pelvis",
+	"robe",
+	"shin",
+	"shoulder",
+}
+
+var partDatasetToAssetCategory = map[string]string{
+	"belt":     "belt",
+	"bicep":    "bicep",
+	"chest":    "chest",
+	"foot":     "foot",
+	"forearm":  "forearm",
+	"legs":     "leg",
+	"neck":     "neck",
+	"pelvis":   "pelvis",
+	"robe":     "robe",
+	"shin":     "shin",
+	"shoulder": "shoulder",
+}
+
+// trailingNumberRegex matches trailing digits at the end of a string
+var trailingNumberRegex = regexp.MustCompile(`(\d+)$`)
+
+// isPartsDataset checks if a dataset name corresponds to a parts dataset
+func isPartsDataset(datasetName string) bool {
+	// Check if the dataset is under parts/ directory
+	if strings.HasPrefix(datasetName, "parts/") {
+		return true
+	}
+	// Check if it's exactly a parts file (e.g., "parts/belt")
+	parts := strings.Split(datasetName, "/")
+	if len(parts) == 2 && parts[0] == "parts" {
+		return true
+	}
+	return false
+}
+
+func isAutogenEligiblePartsDataset(dataset nativeCollectedDataset) bool {
+	if !isPartsDataset(dataset.Dataset.Name) {
+		return false
+	}
+	for _, row := range dataset.Rows {
+		key, ok := row["key"].(string)
+		if ok && strings.TrimSpace(key) != "" {
+			return false
+		}
+	}
+	return true
+}
+
+// extractPartCategory extracts the part category from a dataset name
+// e.g., "parts/belt" -> "belt", "parts/bicep" -> "bicep"
+func extractPartCategory(datasetName string) string {
+	parts := strings.Split(datasetName, "/")
+	if len(parts) >= 2 {
+		return parts[len(parts)-1]
+	}
+	return ""
+}
+
+func partAssetCategoryForDataset(datasetName string) string {
+	category := extractPartCategory(datasetName)
+	if mapped, ok := partDatasetToAssetCategory[category]; ok {
+		return mapped
+	}
+	return category
+}
+
+// isSupportedPartCategory checks if the given category is in the supported list
+func isSupportedPartCategory(category string) bool {
+	for _, c := range supportedPartCategories {
+		if c == category {
+			return true
+		}
+	}
+	return false
+}
+
+// extractTrailingNumericSuffix extracts the trailing numeric suffix from a filename stem
+// e.g., "pfa0_belt018" -> 18, "pfa0_belt171" -> 171
+// Returns 0 and false if no trailing numeric suffix is found
+func extractTrailingNumericSuffix(filenameStem string) (int, bool) {
+	matches := trailingNumberRegex.FindStringSubmatch(filenameStem)
+	if len(matches) < 2 {
+		return 0, false
+	}
+	num, err := strconv.Atoi(matches[1])
+	if err != nil {
+		return 0, false
+	}
+	return num, true
+}
+
+// DiscoveredPartModel represents a discovered model file with its row ID
+type DiscoveredPartModel struct {
+	RowID    int
+	Filename string
+}
+
+// DiscoverPartModels scans the assets/part directory for .mdl files
+// and returns a map of rowID -> default row data for the given category.
+// The assetsDir should be the root assets directory (containing the part/ subdirectory).
+func DiscoverPartModels(assetsDir string, category string) (map[int]*DiscoveredPartModel, error) {
+	result := make(map[int]*DiscoveredPartModel)
+
+	if !isSupportedPartCategory(category) {
+		return result, nil
+	}
+
+	partRoot, err := resolvePartsRoot(assetsDir)
+	if err != nil {
+		return nil, err
+	}
+	if partRoot == "" {
+		return result, nil
+	}
+
+	// Construct the path to the part category directory
+	partDir := filepath.Join(partRoot, category)
+
+	// Check if the directory exists
+	info, err := os.Stat(partDir)
+	if err != nil {
+		if os.IsNotExist(err) {
+			return result, nil
+		}
+		return nil, fmt.Errorf("failed to access part directory %s: %w", partDir, err)
+	}
+	if !info.IsDir() {
+		return nil, fmt.Errorf("part directory %s is not a directory", partDir)
+	}
+
+	// Walk the directory and find .mdl files
+	err = filepath.Walk(partDir, func(path string, info os.FileInfo, err error) error {
+		if err != nil {
+			return err
+		}
+		if info.IsDir() {
+			return nil
+		}
+
+		// Only process .mdl files
+		if !strings.HasSuffix(strings.ToLower(info.Name()), ".mdl") {
+			return nil
+		}
+
+		// Extract the filename stem (without extension)
+		filenameStem := strings.TrimSuffix(info.Name(), filepath.Ext(info.Name()))
+
+		// Extract trailing numeric suffix
+		rowID, hasSuffix := extractTrailingNumericSuffix(filenameStem)
+		if !hasSuffix {
+			return nil
+		}
+
+		// Store the discovered model
+		if existing, ok := result[rowID]; ok {
+			// If we already have this row ID, keep the first one found (alphabetically)
+			if path < existing.Filename {
+				result[rowID] = &DiscoveredPartModel{
+					RowID:    rowID,
+					Filename: path,
+				}
+			}
+		} else {
+			result[rowID] = &DiscoveredPartModel{
+				RowID:    rowID,
+				Filename: path,
+			}
+		}
+
+		return nil
+	})
+
+	if err != nil {
+		return nil, fmt.Errorf("failed to scan part directory %s: %w", partDir, err)
+	}
+
+	return result, nil
+}
+
+// CreateDefaultPartRow creates a default row for a discovered part model.
+// Base defaults are COSTMODIFIER=0 and ACBONUS=0.00. Any robe HIDE* columns
+// present in the dataset columns default to 0 as well.
+func CreateDefaultPartRow(rowID int) map[string]any {
+	return CreateDefaultPartRowForColumns(rowID, nil)
+}
+
+func CreateDefaultPartRowForColumns(rowID int, columns []string) map[string]any {
+	return createDefaultPartRowForDataset(rowID, "", columns, project.PartsRowsConfig{})
+}
+
+func createDefaultPartRowForDataset(rowID int, datasetName string, columns []string, cfg project.PartsRowsConfig) map[string]any {
+	row := map[string]any{
+		"id":           rowID,
+		"COSTMODIFIER": "0",
+		"ACBONUS":      "0.00",
+	}
+	for field, value := range cfg.RowDefaults {
+		if strings.TrimSpace(field) == "" {
+			continue
+		}
+		row[field] = value
+	}
+	if value, ok := configuredPartsRowsACBonusValue(rowID, datasetName, cfg); ok {
+		row["ACBONUS"] = value
+	}
+	for _, column := range columns {
+		if strings.HasPrefix(column, "HIDE") {
+			row[column] = "0"
+		}
+	}
+	return row
+}
+
+func isUnsetPartValue(value any) bool {
+	switch typed := value.(type) {
+	case nil:
+		return true
+	case string:
+		return typed == "" || typed == "****"
+	}
+	return false
+}
+
+func applyDiscoveredPartDefaults(row map[string]any, columns []string) {
+	applyDiscoveredPartDefaultsWithConfig(row, "", columns, project.PartsRowsConfig{})
+}
+
+func applyDiscoveredPartDefaultsWithConfig(row map[string]any, datasetName string, columns []string, cfg project.PartsRowsConfig) {
+	if isUnsetPartValue(row["COSTMODIFIER"]) {
+		row["COSTMODIFIER"] = "0"
+	}
+	for field, value := range cfg.RowDefaults {
+		if strings.TrimSpace(field) == "" {
+			continue
+		}
+		if isUnsetPartValue(row[field]) {
+			row[field] = value
+		}
+	}
+	if isUnsetPartValue(row["ACBONUS"]) {
+		row["ACBONUS"] = "0.00"
+	}
+	if value, ok := configuredPartsRowsACBonusValue(rowIDFromPartRow(row), datasetName, cfg); ok {
+		row["ACBONUS"] = value
+	}
+	for _, column := range columns {
+		if !strings.HasPrefix(column, "HIDE") {
+			continue
+		}
+		if isUnsetPartValue(row[column]) {
+			row[column] = "0"
+		}
+	}
+}
+
+func rowIDFromPartRow(row map[string]any) int {
+	rowID, _ := row["id"].(int)
+	return rowID
+}
+
+func resolvePartsRoot(root string) (string, error) {
+	if strings.TrimSpace(root) == "" {
+		return "", nil
+	}
+	candidates := []string{
+		filepath.Join(root, "assets", "parts"),
+		filepath.Join(root, "assets", "part"),
+		filepath.Join(root, "parts"),
+		filepath.Join(root, "part"),
+	}
+	for _, candidate := range candidates {
+		info, err := os.Stat(candidate)
+		if err == nil && info.IsDir() {
+			return candidate, nil
+		}
+		if err != nil && !os.IsNotExist(err) {
+			return "", fmt.Errorf("failed to access parts root %s: %w", candidate, err)
+		}
+	}
+	return "", nil
+}
+
+// scanAutogeneratedParts scans the assets/part directory for all supported part categories
+// and returns a map of category -> set of row IDs that should be auto-generated.
+// The assetsDir parameter should be the path to the assets directory that contains the part/ subdirectory.
+func scanAutogeneratedParts(assetsDir string) (map[string]map[int]struct{}, error) {
+	if assetsDir == "" {
+		return nil, nil
+	}
+
+	result := make(map[string]map[int]struct{})
+
+	for _, category := range supportedPartCategories {
+		discovered, err := DiscoverPartModels(assetsDir, category)
+		if err != nil {
+			return nil, err
+		}
+		// Always include the category, even if no IDs were discovered
+		ids := make(map[int]struct{}, len(discovered))
+		for rowID := range discovered {
+			ids[rowID] = struct{}{}
+		}
+		result[category] = ids
+	}
+
+	return result, nil
+}
+
+func resolveAutogeneratedPartsInventory(p *project.Project, progress func(string)) (map[string]map[int]struct{}, error) {
+	overrideDir, err := resolvePartsAssetsOverrideDir(p)
+	if err != nil {
+		return nil, err
+	}
+	if overrideDir != "" {
+		if progress != nil {
+			progress(fmt.Sprintf("Scanning part models from local topdata.assets override %s...", overrideDir))
+		}
+		return scanAutogeneratedParts(overrideDir)
+	}
+	return resolveReleasedPartsManifest(p, progress)
+}
+
+func hasCollectedPartsDatasets(collected []nativeCollectedDataset) bool {
+	for _, dataset := range collected {
+		if isPartsDataset(dataset.Dataset.Name) {
+			return true
+		}
+	}
+	return false
+}
+
+func loadPartOverrides(path string) ([]map[string]any, error) {
+	if _, err := os.Stat(path); err != nil {
+		if os.IsNotExist(err) {
+			return nil, nil
+		}
+		return nil, err
+	}
+	obj, err := loadJSONObject(path)
+	if err != nil {
+		return nil, err
+	}
+	rawOverrides, ok := obj["overrides"]
+	if !ok {
+		return nil, nil
+	}
+	overrideList, ok := rawOverrides.([]any)
+	if !ok {
+		return nil, fmt.Errorf("%s: overrides must be an array", path)
+	}
+	overrides := make([]map[string]any, 0, len(overrideList))
+	for index, rawOverride := range overrideList {
+		override, ok := rawOverride.(map[string]any)
+		if !ok {
+			return nil, fmt.Errorf("%s: override %d must be an object", path, index)
+		}
+		overrides = append(overrides, override)
+	}
+	return overrides, nil
+}
+
+func applyPartOverrides(sourceDir string, collected []nativeCollectedDataset) ([]nativeCollectedDataset, error) {
+	return applyPartOverridesWithConfig(sourceDir, collected, project.PartsRowsConfig{})
+}
+
+func applyPartOverridesWithConfig(sourceDir string, collected []nativeCollectedDataset, cfg project.PartsRowsConfig) ([]nativeCollectedDataset, error) {
+	result := make([]nativeCollectedDataset, len(collected))
+	copy(result, collected)
+	for i, dataset := range result {
+		if !strings.HasPrefix(dataset.Dataset.Name, "parts/") {
+			continue
+		}
+		category := extractPartCategory(dataset.Dataset.Name)
+		if category == "" {
+			continue
+		}
+		overrides, err := loadPartOverridesForCategory(sourceDir, category)
+		if err != nil {
+			return nil, err
+		}
+		if len(overrides) == 0 {
+			continue
+		}
+
+		rows := make([]map[string]any, len(dataset.Rows))
+		rowByID := make(map[int]map[string]any, len(dataset.Rows))
+		for index, row := range dataset.Rows {
+			cloned := cloneRowMap(row)
+			rows[index] = cloned
+			if rowID, ok := cloned["id"].(int); ok {
+				rowByID[rowID] = cloned
+			}
+		}
+
+		for index, override := range overrides {
+			rawID, ok := override["id"]
+			if !ok {
+				return nil, fmt.Errorf("parts/%s override %d is missing id", category, index)
+			}
+			rowID, err := asInt(rawID)
+			if err != nil {
+				return nil, fmt.Errorf("parts/%s override %d id is not numeric", category, index)
+			}
+			row, ok := rowByID[rowID]
+			if !ok {
+				row = createDefaultPartRowForDataset(rowID, dataset.Dataset.Name, dataset.Columns, cfg)
+				rows = append(rows, row)
+				rowByID[rowID] = row
+			}
+			for field, value := range override {
+				if field == "id" {
+					continue
+				}
+				row[field] = normalizePartOverrideValue(value)
+			}
+		}
+
+		slices.SortFunc(rows, func(a, b map[string]any) int {
+			idA := a["id"].(int)
+			idB := b["id"].(int)
+			return idA - idB
+		})
+		result[i].Rows = rows
+	}
+	return result, nil
+}
+
+func normalizePartsRowsACBonus(collected []nativeCollectedDataset, cfg project.PartsRowsConfig) ([]nativeCollectedDataset, error) {
+	if !partsRowsACBonusConfigured(cfg) && len(cfg.RowDefaults) == 0 {
+		return collected, nil
+	}
+	result := make([]nativeCollectedDataset, len(collected))
+	copy(result, collected)
+	for i, dataset := range result {
+		if !strings.HasPrefix(dataset.Dataset.Name, "parts/") {
+			continue
+		}
+		rows := make([]map[string]any, len(dataset.Rows))
+		for index, row := range dataset.Rows {
+			cloned := cloneRowMap(row)
+			for field, value := range cfg.RowDefaults {
+				if strings.TrimSpace(field) == "" {
+					continue
+				}
+				if isUnsetPartValue(cloned[field]) {
+					cloned[field] = value
+				}
+			}
+			rowID, ok := cloned["id"].(int)
+			if isUnsetPartValue(cloned["ACBONUS"]) {
+				cloned["COSTMODIFIER"] = nullValue
+			} else if ok {
+				if value, configured := configuredPartsRowsACBonusValue(rowID, dataset.Dataset.Name, cfg); configured {
+					cloned["ACBONUS"] = value
+				}
+			}
+			rows[index] = cloned
+		}
+		result[i].Rows = rows
+	}
+	return result, nil
+}
+
+func partsRowsACBonusConfigured(cfg project.PartsRowsConfig) bool {
+	if strings.TrimSpace(cfg.ACBonus.Default.Strategy) != "" {
+		return true
+	}
+	if len(cfg.ACBonus.Datasets) > 0 {
+		return true
+	}
+	for _, dataset := range cfg.Datasets {
+		if strings.TrimSpace(dataset.ACBonus.Strategy) != "" {
+			return true
+		}
+	}
+	return false
+}
+
+func configuredPartsRowsACBonusValue(rowID int, datasetName string, cfg project.PartsRowsConfig) (string, bool) {
+	policy, ok := partsRowsACBonusPolicyForDataset(datasetName, cfg)
+	if !ok {
+		return "", false
+	}
+	switch strings.TrimSpace(policy.Strategy) {
+	case "ascending_row_id_sort_key":
+		format := strings.TrimSpace(policy.Format)
+		if format == "" {
+			format = "%.2f"
+		}
+		return fmt.Sprintf(format, float64(rowID)/float64(policy.Divisor)), true
+	case "descending_row_id_sort_key":
+		format := strings.TrimSpace(policy.Format)
+		if format == "" {
+			format = "%.2f"
+		}
+		return fmt.Sprintf(format, float64(policy.MaxRowID-rowID)/float64(policy.Divisor)), true
+	case "fixed":
+		return policy.Value, true
+	default:
+		return "", false
+	}
+}
+
+func partsRowsACBonusPolicyForDataset(datasetName string, cfg project.PartsRowsConfig) (project.PartsRowsACBonusPolicy, bool) {
+	category := extractPartCategory(datasetName)
+	if policy, ok := cfg.ACBonus.Datasets[category]; ok && strings.TrimSpace(policy.Strategy) != "" {
+		return policy, true
+	}
+	if dataset, ok := cfg.Datasets[category]; ok && strings.TrimSpace(dataset.ACBonus.Strategy) != "" {
+		return dataset.ACBonus, true
+	}
+	if strings.TrimSpace(cfg.ACBonus.Default.Strategy) != "" {
+		return cfg.ACBonus.Default, true
+	}
+	return project.PartsRowsACBonusPolicy{}, false
+}
+
+func loadPartOverridesForCategory(sourceDir, category string) ([]map[string]any, error) {
+	overrides := []map[string]any{}
+
+	legacyPath := filepath.Join(sourceDir, "data", "parts", "overrides", category+".json")
+	legacyOverrides, err := loadPartOverrides(legacyPath)
+	if err != nil {
+		return nil, err
+	}
+	overrides = append(overrides, legacyOverrides...)
+
+	moduleDir := filepath.Join(sourceDir, "data", "parts", "modules")
+	modulePaths, err := partModuleOverridePaths(moduleDir, category)
+	if err != nil {
+		return nil, err
+	}
+	for _, path := range modulePaths {
+		moduleOverrides, err := loadPartOverrides(path)
+		if err != nil {
+			return nil, err
+		}
+		overrides = append(overrides, moduleOverrides...)
+	}
+	return overrides, nil
+}
+
+func partModuleOverridePaths(moduleDir, category string) ([]string, error) {
+	info, err := os.Stat(moduleDir)
+	if err != nil {
+		if os.IsNotExist(err) {
+			return nil, nil
+		}
+		return nil, err
+	}
+	if !info.IsDir() {
+		return nil, fmt.Errorf("parts module override path %s is not a directory", moduleDir)
+	}
+
+	var paths []string
+	err = filepath.WalkDir(moduleDir, func(path string, d os.DirEntry, err error) error {
+		if err != nil {
+			return err
+		}
+		if d.IsDir() {
+			return nil
+		}
+		if strings.ToLower(filepath.Ext(path)) != ".json" {
+			return nil
+		}
+		if partModuleOverrideMatchesCategory(path, category) {
+			paths = append(paths, path)
+		}
+		return nil
+	})
+	if err != nil {
+		return nil, err
+	}
+	slices.Sort(paths)
+	return paths, nil
+}
+
+func partModuleOverrideMatchesCategory(path, category string) bool {
+	stem := strings.TrimSuffix(strings.ToLower(filepath.Base(path)), strings.ToLower(filepath.Ext(path)))
+	category = strings.ToLower(strings.TrimSpace(category))
+	return stem == category ||
+		strings.HasPrefix(stem, category+"_") ||
+		strings.HasSuffix(stem, "_"+category) ||
+		strings.Contains(stem, "_"+category+"_")
+}
+
+func normalizePartOverrideValue(value any) any {
+	switch typed := value.(type) {
+	case float64:
+		if typed == float64(int(typed)) {
+			return strconv.Itoa(int(typed))
+		}
+	case int:
+		return strconv.Itoa(typed)
+	case json.Number:
+		return typed.String()
+	}
+	return value
+}
+
+// augmentWithAutogeneratedParts applies repository-backed discovery defaults.
+// Missing discovered IDs are added with default values. Existing discovered rows
+// retain authored values unless they still use placeholder values, in which case
+// discovery activates them with engine-visible defaults.
+func augmentWithAutogeneratedParts(collected []nativeCollectedDataset, autogenerated map[string]map[int]struct{}) []nativeCollectedDataset {
+	return augmentWithAutogeneratedPartsWithConfig(collected, autogenerated, project.PartsRowsConfig{})
+}
+
+func augmentWithAutogeneratedPartsWithConfig(collected []nativeCollectedDataset, autogenerated map[string]map[int]struct{}, cfg project.PartsRowsConfig) []nativeCollectedDataset {
+	if len(autogenerated) == 0 {
+		return collected
+	}
+
+	result := make([]nativeCollectedDataset, len(collected))
+	copy(result, collected)
+
+	for i, dataset := range collected {
+		if !isAutogenEligiblePartsDataset(dataset) {
+			continue
+		}
+		category := partAssetCategoryForDataset(dataset.Dataset.Name)
+		if !isSupportedPartCategory(category) {
+			continue
+		}
+
+		ids, ok := autogenerated[category]
+		if !ok || len(ids) == 0 {
+			continue
+		}
+
+		// Build a map of existing row IDs.
+		existingRows := make(map[int]map[string]any, len(dataset.Rows))
+		for _, row := range dataset.Rows {
+			if id, ok := row["id"].(int); ok {
+				existingRows[id] = row
+			}
+		}
+
+		// Add rows for discovered IDs that don't already exist.
+		newRows := make([]map[string]any, 0, len(dataset.Rows))
+		newRows = append(newRows, dataset.Rows...)
+
+		for rowID := range ids {
+			if row, exists := existingRows[rowID]; exists {
+				applyDiscoveredPartDefaultsWithConfig(row, dataset.Dataset.Name, dataset.Columns, cfg)
+				continue
+			}
+			if _, exists := existingRows[rowID]; !exists {
+				newRow := createDefaultPartRowForDataset(rowID, dataset.Dataset.Name, dataset.Columns, cfg)
+				newRows = append(newRows, newRow)
+			}
+		}
+
+		// Sort rows by ID
+		slices.SortFunc(newRows, func(a, b map[string]any) int {
+			idA := a["id"].(int)
+			idB := b["id"].(int)
+			return idA - idB
+		})
+
+		result[i].Rows = newRows
+	}
+
+	return result
+}

internal/topdata/parts_manifest.go

@@ -0,0 +1,287 @@
+package topdata
+
+import (
+	"bytes"
+	"encoding/json"
+	"fmt"
+	"io"
+	"net/http"
+	"net/url"
+	"os"
+	"path/filepath"
+	"strings"
+	"time"
+
+	"git.westgate.pw/ShadowsOverWestgate/sow-tools/internal/project"
+)
+
+const (
+	defaultSowAssetsRepoOwner   = "ShadowsOverWestgate"
+	defaultSowAssetsRepoName    = "sow-assets"
+	defaultGiteaBaseURL         = "https://gitea.westgate.pw"
+	partsManifestReleaseTag     = "parts-manifest-current"
+	partsManifestAssetName      = "sow-parts-manifest.json"
+	partsManifestCacheFileName  = "sow-parts-manifest.json"
+	partsManifestCacheMaxAge    = time.Hour
+	partsManifestRequestTimeout = 30 * time.Second
+)
+
+type partsManifest struct {
+	Repo        string           `json:"repo"`
+	Ref         string           `json:"ref"`
+	GeneratedAt string           `json:"generated_at"`
+	Categories  map[string][]int `json:"categories"`
+}
+
+type giteaRepoSpec struct {
+	BaseURL string
+	Owner   string
+	Repo    string
+}
+
+var partsManifestHTTPClient = &http.Client{Timeout: partsManifestRequestTimeout}
+
+func resolveReleasedPartsManifest(p *project.Project, progress func(string)) (map[string]map[int]struct{}, error) {
+	if progress == nil {
+		progress = func(string) {}
+	}
+	cachePath := p.AutogenCachePath(partsManifestCacheFileName)
+	if strings.TrimSpace(os.Getenv(p.PartsManifestRefreshEnv())) == "" {
+		manifest, fresh, err := readFreshPartsManifestCache(cachePath, partsManifestCacheMaxAge)
+		if err != nil {
+			progress(fmt.Sprintf("Ignoring cached parts manifest: %v", err))
+		} else if fresh {
+			progress(fmt.Sprintf("Using cached released parts manifest from %s...", cachePath))
+			return inventoryFromPartsManifest(manifest), nil
+		}
+	}
+	spec, err := deriveSowAssetsRepoSpec(p)
+	if err != nil {
+		return nil, err
+	}
+	manifestURL, err := resolvePartsManifestAssetURL(spec)
+	if err != nil {
+		return nil, err
+	}
+	progress(fmt.Sprintf("Fetching released parts manifest from %s...", manifestURL))
+	manifest, err := fetchPartsManifest(manifestURL)
+	if err != nil {
+		return nil, err
+	}
+	if err := writePartsManifestCache(cachePath, manifest); err != nil {
+		return nil, err
+	}
+	return inventoryFromPartsManifest(manifest), nil
+}
+
+func deriveSowAssetsRepoSpec(p *project.Project) (giteaRepoSpec, error) {
+	serverOverride := strings.TrimSpace(os.Getenv(p.AssetsServerURLEnv()))
+	repoOverride := strings.TrimSpace(os.Getenv(p.AssetsRepoEnv()))
+	remote, err := gitOutput(p.Root, "remote", "get-url", "origin")
+	spec := giteaRepoSpec{BaseURL: defaultGiteaBaseURL, Owner: defaultSowAssetsRepoOwner, Repo: defaultSowAssetsRepoName}
+	if err == nil && strings.TrimSpace(remote) != "" {
+		if derived, ok := parseRepoSpec(strings.TrimSpace(remote)); ok {
+			derived.Repo = defaultSowAssetsRepoName
+			spec = derived
+		}
+	}
+
+	if serverOverride != "" {
+		spec.BaseURL = strings.TrimRight(serverOverride, "/")
+	}
+	if repoOverride != "" {
+		owner, repo, ok := parseOwnerRepo(repoOverride)
+		if !ok {
+			return giteaRepoSpec{}, fmt.Errorf("invalid SOW_ASSETS_REPO %q; expected owner/repo", repoOverride)
+		}
+		spec.Owner = owner
+		spec.Repo = repo
+	}
+	return spec, nil
+}
+
+func parseOwnerRepo(raw string) (string, string, bool) {
+	parts := strings.Split(strings.Trim(strings.TrimSuffix(raw, ".git"), "/"), "/")
+	if len(parts) < 2 {
+		return "", "", false
+	}
+	owner := strings.TrimSpace(parts[len(parts)-2])
+	repo := strings.TrimSpace(parts[len(parts)-1])
+	if owner == "" || repo == "" {
+		return "", "", false
+	}
+	return owner, repo, true
+}
+
+func parseRepoSpec(raw string) (giteaRepoSpec, bool) {
+	if strings.HasPrefix(raw, "ssh://") || strings.HasPrefix(raw, "http://") || strings.HasPrefix(raw, "https://") {
+		u, err := url.Parse(raw)
+		if err != nil || u.Host == "" {
+			return giteaRepoSpec{}, false
+		}
+		parts := strings.Split(strings.Trim(strings.TrimSuffix(u.Path, ".git"), "/"), "/")
+		if len(parts) < 2 {
+			return giteaRepoSpec{}, false
+		}
+		base := (&url.URL{Scheme: "https", Host: normalizeGiteaHTTPHost(u.Hostname())}).String()
+		if u.Scheme == "http" || u.Scheme == "https" {
+			base = (&url.URL{Scheme: u.Scheme, Host: u.Host}).String()
+		}
+		return giteaRepoSpec{BaseURL: strings.TrimRight(base, "/"), Owner: parts[len(parts)-2], Repo: parts[len(parts)-1]}, true
+	}
+	if strings.HasPrefix(raw, "git@") {
+		withoutUser := strings.TrimPrefix(raw, "git@")
+		hostAndPath := strings.SplitN(withoutUser, ":", 2)
+		if len(hostAndPath) != 2 {
+			return giteaRepoSpec{}, false
+		}
+		parts := strings.Split(strings.Trim(strings.TrimSuffix(hostAndPath[1], ".git"), "/"), "/")
+		if len(parts) < 2 {
+			return giteaRepoSpec{}, false
+		}
+		base := (&url.URL{Scheme: "https", Host: normalizeGiteaHTTPHost(hostAndPath[0])}).String()
+		return giteaRepoSpec{BaseURL: strings.TrimRight(base, "/"), Owner: parts[len(parts)-2], Repo: parts[len(parts)-1]}, true
+	}
+	return giteaRepoSpec{}, false
+}
+
+func normalizeGiteaHTTPHost(host string) string {
+	host = strings.TrimSpace(host)
+	if strings.HasPrefix(host, "git-ssh.") {
+		return "gitea." + strings.TrimPrefix(host, "git-ssh.")
+	}
+	return host
+}
+
+func resolvePartsManifestAssetURL(spec giteaRepoSpec) (string, error) {
+	apiURL := fmt.Sprintf("%s/api/v1/repos/%s/%s/releases/tags/%s", strings.TrimRight(spec.BaseURL, "/"), url.PathEscape(spec.Owner), url.PathEscape(spec.Repo), url.PathEscape(partsManifestReleaseTag))
+	var release struct {
+		Assets []struct {
+			Name               string `json:"name"`
+			BrowserDownloadURL string `json:"browser_download_url"`
+			DownloadURL        string `json:"download_url"`
+		} `json:"assets"`
+	}
+	if err := fetchJSON(apiURL, &release); err != nil {
+		return "", fmt.Errorf("fetch parts manifest release metadata: %w", err)
+	}
+	for _, asset := range release.Assets {
+		if asset.Name != partsManifestAssetName {
+			continue
+		}
+		if asset.DownloadURL != "" {
+			return asset.DownloadURL, nil
+		}
+		if asset.BrowserDownloadURL != "" {
+			return asset.BrowserDownloadURL, nil
+		}
+	}
+	return "", fmt.Errorf("parts manifest asset %s not found in release %s/%s:%s", partsManifestAssetName, spec.Owner, spec.Repo, partsManifestReleaseTag)
+}
+
+func fetchPartsManifest(manifestURL string) (*partsManifest, error) {
+	var manifest partsManifest
+	if err := fetchJSON(manifestURL, &manifest); err != nil {
+		return nil, fmt.Errorf("download parts manifest: %w", err)
+	}
+	if len(manifest.Categories) == 0 {
+		return nil, fmt.Errorf("download parts manifest: categories is empty")
+	}
+	return &manifest, nil
+}
+
+func readFreshPartsManifestCache(path string, maxAge time.Duration) (*partsManifest, bool, error) {
+	info, err := os.Stat(path)
+	if err != nil {
+		if os.IsNotExist(err) {
+			return nil, false, nil
+		}
+		return nil, false, err
+	}
+	if maxAge > 0 && time.Since(info.ModTime()) > maxAge {
+		return nil, false, nil
+	}
+	raw, err := os.ReadFile(path)
+	if err != nil {
+		return nil, false, err
+	}
+	var manifest partsManifest
+	if err := json.Unmarshal(raw, &manifest); err != nil {
+		return nil, false, err
+	}
+	if len(manifest.Categories) == 0 {
+		return nil, false, fmt.Errorf("categories is empty")
+	}
+	return &manifest, true, nil
+}
+
+func fetchJSON(target string, out any) error {
+	req, err := http.NewRequest(http.MethodGet, target, nil)
+	if err != nil {
+		return err
+	}
+	if token := strings.TrimSpace(os.Getenv("SOW_ASSETS_TOKEN")); token != "" {
+		req.Header.Set("Authorization", "token "+token)
+	} else if token := strings.TrimSpace(os.Getenv("GITEA_TOKEN")); token != "" {
+		req.Header.Set("Authorization", "token "+token)
+	}
+	resp, err := partsManifestHTTPClient.Do(req)
+	if err != nil {
+		return err
+	}
+	defer resp.Body.Close()
+	if resp.StatusCode < 200 || resp.StatusCode >= 300 {
+		body, _ := io.ReadAll(io.LimitReader(resp.Body, 4096))
+		if resp.StatusCode == http.StatusUnauthorized || resp.StatusCode == http.StatusForbidden {
+			return fmt.Errorf("HTTP %d from %s; set SOW_ASSETS_TOKEN or GITEA_TOKEN if the repo is private", resp.StatusCode, target)
+		}
+		return fmt.Errorf("HTTP %d from %s: %s", resp.StatusCode, target, strings.TrimSpace(string(body)))
+	}
+	decoder := json.NewDecoder(resp.Body)
+	if err := decoder.Decode(out); err != nil {
+		return err
+	}
+	return nil
+}
+
+func writePartsManifestCache(path string, manifest *partsManifest) error {
+	if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil {
+		return fmt.Errorf("create parts manifest cache dir: %w", err)
+	}
+	payload, err := json.MarshalIndent(manifest, "", "  ")
+	if err != nil {
+		return fmt.Errorf("marshal parts manifest cache: %w", err)
+	}
+	payload = append(payload, '\n')
+	current, err := os.ReadFile(path)
+	if err == nil && bytes.Equal(current, payload) {
+		now := time.Now()
+		if err := os.Chtimes(path, now, now); err != nil {
+			return fmt.Errorf("refresh parts manifest cache timestamp: %w", err)
+		}
+		return nil
+	}
+	if err != nil && !os.IsNotExist(err) {
+		return fmt.Errorf("read parts manifest cache: %w", err)
+	}
+	tmpPath := path + ".tmp"
+	if err := os.WriteFile(tmpPath, payload, 0o644); err != nil {
+		return fmt.Errorf("write parts manifest cache: %w", err)
+	}
+	if err := os.Rename(tmpPath, path); err != nil {
+		return fmt.Errorf("replace parts manifest cache: %w", err)
+	}
+	return nil
+}
+
+func inventoryFromPartsManifest(manifest *partsManifest) map[string]map[int]struct{} {
+	result := make(map[string]map[int]struct{}, len(supportedPartCategories))
+	for _, category := range supportedPartCategories {
+		ids := make(map[int]struct{})
+		for _, rowID := range manifest.Categories[category] {
+			ids[rowID] = struct{}{}
+		}
+		result[category] = ids
+	}
+	return result
+}

internal/topdata/parts_manifest_test.go

@@ -0,0 +1,26 @@
+package topdata
+
+import "testing"
+
+func TestParseRepoSpecNormalizesWestgateSSHHostToPublicGiteaHost(t *testing.T) {
+	spec, ok := parseRepoSpec("ssh://git@git-ssh.westgate.pw:2222/ShadowsOverWestgate/sow-module.git")
+	if !ok {
+		t.Fatal("expected SSH remote to parse")
+	}
+	if spec.BaseURL != "https://gitea.westgate.pw" {
+		t.Fatalf("expected public Gitea base URL, got %q", spec.BaseURL)
+	}
+	if spec.Owner != "ShadowsOverWestgate" || spec.Repo != "sow-module" {
+		t.Fatalf("unexpected repo spec: %#v", spec)
+	}
+}
+
+func TestParseRepoSpecKeepsHTTPRemoteHost(t *testing.T) {
+	spec, ok := parseRepoSpec("https://example.invalid/owner/repo.git")
+	if !ok {
+		t.Fatal("expected HTTPS remote to parse")
+	}
+	if spec.BaseURL != "https://example.invalid" {
+		t.Fatalf("expected HTTPS base URL to be preserved, got %q", spec.BaseURL)
+	}
+}

internal/topdata/placeables_migrate.go

@@ -0,0 +1,6 @@
+package topdata
+
+func importLegacyPlaceables(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
+	_ = legacyTLK
+	return importLegacyDatasetMirror(referenceBuilderDir, dataDir, "placeables", "placeables.2da", nil)
+}

internal/topdata/portraits_migrate.go

@@ -0,0 +1,8 @@
+package topdata
+
+func importLegacyPortraits(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
+	if legacyTLK == nil {
+		return 0, nil
+	}
+	return importLegacyDatasetMirror(referenceBuilderDir, dataDir, "portraits", "portraits.2da", nil)
+}

internal/topdata/progfx_migrate.go

@@ -0,0 +1,6 @@
+package topdata
+
+func importLegacyProgfx(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
+	_ = legacyTLK
+	return importLegacyDatasetMirror(referenceBuilderDir, dataDir, "progfx", "progfx.2da", nil)
+}

internal/topdata/racialtypes_migrate.go

@@ -0,0 +1,282 @@
+package topdata
+
+import (
+	"encoding/json"
+	"fmt"
+	"os"
+	"path/filepath"
+	"slices"
+	"strings"
+)
+
+type legacyRacialtypesFeatTable struct {
+	Output string
+	Rows   []map[string]any
+}
+
+func importLegacyRacialtypes(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
+	if strings.TrimSpace(referenceBuilderDir) == "" {
+		return 0, nil
+	}
+
+	legacyCoreDir := filepath.Join(referenceBuilderDir, "data", "racialtypes", "core")
+	legacyFeatsDir := filepath.Join(referenceBuilderDir, "data", "racialtypes", "feats")
+	if _, err := os.Stat(legacyCoreDir); err != nil {
+		if os.IsNotExist(err) {
+			return 0, nil
+		}
+		return 0, err
+	}
+	if _, err := os.Stat(legacyFeatsDir); err != nil {
+		if os.IsNotExist(err) {
+			return 0, nil
+		}
+		return 0, err
+	}
+
+	targetRoot := filepath.Join(dataDir, "racialtypes", "registry")
+	targetBasePath := filepath.Join(targetRoot, "base.json")
+	targetLockPath := filepath.Join(targetRoot, "lock.json")
+	targetRacesDir := filepath.Join(targetRoot, "races")
+	if fileExists(targetBasePath) && fileExists(targetLockPath) {
+		entries, err := os.ReadDir(targetRacesDir)
+		if err == nil {
+			raceFiles := 0
+			for _, entry := range entries {
+				if !entry.IsDir() && strings.HasSuffix(strings.ToLower(entry.Name()), ".json") {
+					raceFiles++
+				}
+			}
+			if raceFiles == 16 {
+				return 0, nil
+			}
+		}
+	}
+
+	baseColumns, baseRows, lockData, raceRows, err := collectLegacyRacialtypesRegistry(legacyCoreDir, legacyFeatsDir, legacyTLK)
+	if err != nil {
+		return 0, err
+	}
+
+	if err := removePathIfExists(filepath.Join(dataDir, "racialtypes", "core")); err != nil {
+		return 0, err
+	}
+	if err := removePathIfExists(filepath.Join(dataDir, "racialtypes", "feats")); err != nil {
+		return 0, err
+	}
+	if err := removePathIfExists(filepath.Join(targetRoot, "base_rows.json")); err != nil {
+		return 0, err
+	}
+	if err := removePathIfExists(targetRacesDir); err != nil {
+		return 0, err
+	}
+	if err := os.MkdirAll(targetRacesDir, 0o755); err != nil {
+		return 0, err
+	}
+
+	writes := []struct {
+		path string
+		obj  any
+	}{
+		{
+			path: targetBasePath,
+			obj: map[string]any{
+				"columns": stringSliceToAny(baseColumns),
+				"rows":    rowsToAny(baseRows),
+			},
+		},
+		{
+			path: targetLockPath,
+			obj:  anyMapInt(lockData),
+		},
+	}
+
+	raceKeys := make([]string, 0, len(raceRows))
+	for key := range raceRows {
+		raceKeys = append(raceKeys, key)
+	}
+	slices.Sort(raceKeys)
+	for _, key := range raceKeys {
+		fileName := strings.TrimPrefix(key, "racialtypes:") + ".json"
+		writes = append(writes, struct {
+			path string
+			obj  any
+		}{
+			path: filepath.Join(targetRacesDir, fileName),
+			obj:  raceRows[key],
+		})
+	}
+
+	for _, write := range writes {
+		raw, err := json.MarshalIndent(write.obj, "", "  ")
+		if err != nil {
+			return 0, err
+		}
+		raw = append(raw, '\n')
+		if err := os.WriteFile(write.path, raw, 0o644); err != nil {
+			return 0, err
+		}
+	}
+	return len(writes), nil
+}
+
+func collectLegacyRacialtypesRegistry(coreDir, featsDir string, legacyTLK *legacyTLKData) ([]string, []map[string]any, map[string]int, map[string]map[string]any, error) {
+	coreCollected, err := collectBaseDataset(nativeDataset{
+		Name:       "racialtypes",
+		BasePath:   filepath.Join(coreDir, "base.json"),
+		LockPath:   filepath.Join(coreDir, "lock.json"),
+		ModulesDir: filepath.Join(coreDir, "modules"),
+		Spec:       specForDataset("racialtypes"),
+	})
+	if err != nil {
+		return nil, nil, nil, nil, err
+	}
+
+	featTables, err := loadLegacyRacialtypesFeatTables(featsDir)
+	if err != nil {
+		return nil, nil, nil, nil, err
+	}
+
+	baseRows := make([]map[string]any, 0)
+	raceRows := map[string]map[string]any{}
+	lockData := map[string]int{}
+	for key, id := range coreCollected.LockData {
+		if strings.HasPrefix(key, "racialtypes:") {
+			lockData[key] = id
+		}
+	}
+
+	for _, collectedRow := range coreCollected.Rows {
+		row, ok := deepCopyValue(collectedRow).(map[string]any)
+		if !ok {
+			continue
+		}
+		if legacyTLK != nil {
+			if _, _, err := inlineLegacyTLKValue(row, legacyTLK); err != nil {
+				return nil, nil, nil, nil, err
+			}
+		}
+		key, _ := row["key"].(string)
+		if !strings.HasPrefix(key, "racialtypes:") {
+			assignRacialtypesBaseRowKey(row)
+			if key, _ := row["key"].(string); key != "" {
+				rowID, err := asInt(row["id"])
+				if err != nil {
+					return nil, nil, nil, nil, err
+				}
+				lockData[key] = rowID
+			}
+			baseRows = append(baseRows, row)
+			continue
+		}
+
+		delete(row, "id")
+		delete(row, "key")
+
+		raceFile := map[string]any{
+			"key":  key,
+			"core": row,
+		}
+		if tableKey := extractRacialtypesFeatTableKey(row["FeatsTable"]); tableKey != "" {
+			table, ok := featTables[tableKey]
+			if !ok {
+				return nil, nil, nil, nil, fmt.Errorf("%s: unknown feat table %s", key, tableKey)
+			}
+			delete(row, "FeatsTable")
+			raceFile["feat_output"] = table.Output
+			raceFile["feats"] = rowsToAny(table.Rows)
+		}
+		raceRows[key] = raceFile
+	}
+
+	slices.SortFunc(baseRows, func(a, b map[string]any) int {
+		left, _ := asInt(a["id"])
+		right, _ := asInt(b["id"])
+		return left - right
+	})
+	return coreCollected.Columns, baseRows, lockData, raceRows, nil
+}
+
+func loadLegacyRacialtypesFeatTables(featsDir string) (map[string]legacyRacialtypesFeatTable, error) {
+	entries, err := os.ReadDir(featsDir)
+	if err != nil {
+		return nil, err
+	}
+
+	tables := map[string]legacyRacialtypesFeatTable{}
+	for _, entry := range entries {
+		if entry.IsDir() || filepath.Ext(entry.Name()) != ".json" {
+			continue
+		}
+		obj, err := loadJSONObject(filepath.Join(featsDir, entry.Name()))
+		if err != nil {
+			return nil, err
+		}
+		tableKey, _ := obj["key"].(string)
+		outputName, _ := obj["output"].(string)
+		rawRows, ok := obj["rows"].([]any)
+		if tableKey == "" || outputName == "" || !ok {
+			return nil, fmt.Errorf("%s: racialtypes feat table must define key, output, and rows", entry.Name())
+		}
+		rows := make([]map[string]any, 0, len(rawRows))
+		for index, raw := range rawRows {
+			row, ok := raw.(map[string]any)
+			if !ok {
+				return nil, fmt.Errorf("%s: row %d must be an object", entry.Name(), index)
+			}
+			rows = append(rows, row)
+		}
+		tables[tableKey] = legacyRacialtypesFeatTable{
+			Output: outputName,
+			Rows:   rows,
+		}
+	}
+	return tables, nil
+}
+
+func extractRacialtypesFeatTableKey(value any) string {
+	obj, ok := value.(map[string]any)
+	if !ok {
+		return ""
+	}
+	tableKey, _ := obj["table"].(string)
+	return tableKey
+}
+
+func assignRacialtypesBaseRowKey(row map[string]any) {
+	name, _ := row["Name"].(string)
+	if strings.TrimSpace(name) == "" || name == nullValue {
+		delete(row, "key")
+		return
+	}
+	label, _ := row["Label"].(string)
+	keySuffix := normalizeRacialtypesKeySuffix(label)
+	if keySuffix == "" {
+		delete(row, "key")
+		return
+	}
+	row["key"] = "racialtypes:" + keySuffix
+}
+
+func normalizeRacialtypesKeySuffix(label string) string {
+	normalized := strings.ToLower(strings.TrimSpace(label))
+	if normalized == "" {
+		return ""
+	}
+	var b strings.Builder
+	lastUnderscore := false
+	for _, ch := range normalized {
+		isAlphaNum := (ch >= 'a' && ch <= 'z') || (ch >= '0' && ch <= '9')
+		if isAlphaNum {
+			b.WriteRune(ch)
+			lastUnderscore = false
+			continue
+		}
+		if lastUnderscore {
+			continue
+		}
+		b.WriteByte('_')
+		lastUnderscore = true
+	}
+	return strings.Trim(b.String(), "_")
+}

internal/topdata/racialtypes_registry.go

@@ -0,0 +1,270 @@
+package topdata
+
+import (
+	"fmt"
+	"io/fs"
+	"os"
+	"path/filepath"
+	"slices"
+	"strings"
+)
+
+const (
+	racialtypesRegistryDirName = "racialtypes/registry"
+	racialtypesRegistryLock    = "lock.json"
+)
+
+type racialtypesRegistry struct {
+	RootPath    string
+	LockPath    string
+	LockData    map[string]int
+	BaseColumns []string
+	BaseRows    []map[string]any
+	Races       []map[string]any
+}
+
+func loadRacialtypesRegistry(dataDir string) (*racialtypesRegistry, error) {
+	root := filepath.Join(dataDir, filepath.FromSlash(racialtypesRegistryDirName))
+	info, err := os.Stat(root)
+	if err != nil {
+		if os.IsNotExist(err) {
+			return nil, nil
+		}
+		return nil, err
+	}
+	if !info.IsDir() {
+		return nil, fmt.Errorf("racialtypes registry root must be a directory: %s", root)
+	}
+
+	lockPath := filepath.Join(root, racialtypesRegistryLock)
+	lockData, err := loadLockfile(lockPath)
+	if err != nil {
+		return nil, err
+	}
+
+	basePath := filepath.Join(root, "base.json")
+	baseObj, err := loadJSONObject(basePath)
+	if err != nil {
+		return nil, err
+	}
+	baseColumns, err := parseColumns(baseObj, "racialtypes/registry/core")
+	if err != nil {
+		return nil, err
+	}
+	rawBaseRows, ok := baseObj["rows"].([]any)
+	if !ok {
+		return nil, fmt.Errorf("%s: rows must be an array", basePath)
+	}
+	baseRows := make([]map[string]any, 0, len(rawBaseRows))
+	for index, raw := range rawBaseRows {
+		row, ok := raw.(map[string]any)
+		if !ok {
+			return nil, fmt.Errorf("%s: row %d must be an object", basePath, index)
+		}
+		baseRows = append(baseRows, row)
+	}
+
+	raceRows, err := loadRacialtypesRegistryRaceRows(filepath.Join(root, "races"))
+	if err != nil {
+		return nil, err
+	}
+
+	return &racialtypesRegistry{
+		RootPath:    root,
+		LockPath:    lockPath,
+		LockData:    lockData,
+		BaseColumns: baseColumns,
+		BaseRows:    baseRows,
+		Races:       raceRows,
+	}, nil
+}
+
+func loadRacialtypesRegistryRaceRows(dir string) ([]map[string]any, error) {
+	info, err := os.Stat(dir)
+	if err != nil {
+		if os.IsNotExist(err) {
+			return nil, nil
+		}
+		return nil, err
+	}
+	if !info.IsDir() {
+		return nil, fmt.Errorf("racialtypes registry races path must be a directory: %s", dir)
+	}
+
+	paths := make([]string, 0)
+	err = filepath.WalkDir(dir, func(path string, d fs.DirEntry, err error) error {
+		if err != nil {
+			return err
+		}
+		if d.IsDir() {
+			return nil
+		}
+		if strings.HasSuffix(strings.ToLower(d.Name()), ".json") {
+			paths = append(paths, path)
+		}
+		return nil
+	})
+	if err != nil {
+		return nil, err
+	}
+	slices.Sort(paths)
+
+	rows := make([]map[string]any, 0, len(paths))
+	for _, path := range paths {
+		obj, err := loadJSONObject(path)
+		if err != nil {
+			return nil, err
+		}
+		rows = append(rows, obj)
+	}
+	return rows, nil
+}
+
+func collectRacialtypesRegistryDatasets(dataDir string) ([]nativeCollectedDataset, error) {
+	registry, err := loadRacialtypesRegistry(dataDir)
+	if err != nil {
+		return nil, err
+	}
+	if registry == nil {
+		return nil, nil
+	}
+
+	lockModified, err := syncRacialtypesBaseLockData(registry.BaseRows, registry.LockData)
+	if err != nil {
+		return nil, err
+	}
+	raceIDs, raceLockModified, err := assignRegistryIDs(registry.Races, registry.LockData, "racialtypes:")
+	if err != nil {
+		return nil, err
+	}
+	lockModified = lockModified || raceLockModified
+	if lockModified {
+		if err := saveLockfile(registry.LockPath, registry.LockData); err != nil {
+			return nil, err
+		}
+	}
+
+	coreDataset := nativeDataset{
+		Kind:       nativeDatasetBase,
+		Name:       "racialtypes/registry/core",
+		OutputName: "racialtypes.2da",
+		Spec:       specForDataset("racialtypes"),
+	}
+	featDataset := nativeDataset{
+		Kind: nativeDatasetPlain,
+		Name: "racialtypes/registry/feats",
+		Spec: specForDataset("racialtypes"),
+	}
+
+	coreRows := make([]map[string]any, 0, len(registry.BaseRows)+len(registry.Races))
+	rowIDSeen := map[int]string{}
+	for index, raw := range registry.BaseRows {
+		row, err := canonicalizeBaseRow(coreDataset, registry.BaseColumns, raw, index)
+		if err != nil {
+			return nil, fmt.Errorf("racialtypes registry base row %d: %w", index, err)
+		}
+		rowID := row["id"].(int)
+		if existing, ok := rowIDSeen[rowID]; ok {
+			return nil, fmt.Errorf("racialtypes registry base row id %d conflicts with %s", rowID, existing)
+		}
+		rowIDSeen[rowID] = "base"
+		coreRows = append(coreRows, row)
+	}
+
+	datasets := make([]nativeCollectedDataset, 0, 16)
+	for _, race := range registry.Races {
+		key := stringField(race, "key")
+		if key == "" {
+			return nil, fmt.Errorf("racialtypes registry row is missing key")
+		}
+		rowID, ok := raceIDs[key]
+		if !ok {
+			return nil, fmt.Errorf("racialtypes registry row %s is missing lock id", key)
+		}
+		if existing, ok := rowIDSeen[rowID]; ok {
+			return nil, fmt.Errorf("racialtypes registry row id %d for %s conflicts with %s", rowID, key, existing)
+		}
+		rowIDSeen[rowID] = key
+
+		coreRaw, ok := race["core"].(map[string]any)
+		if !ok {
+			return nil, fmt.Errorf("%s: core must be an object", key)
+		}
+		row, err := canonicalizeEntry(coreDataset, registry.BaseColumns, rowID, key, coreRaw)
+		if err != nil {
+			return nil, fmt.Errorf("%s: %w", key, err)
+		}
+
+		featOutput, _ := race["feat_output"].(string)
+		if featOutput != "" {
+			row["FeatsTable"] = outputStem(featOutput)
+		}
+		coreRows = append(coreRows, row)
+
+		if featOutput == "" {
+			continue
+		}
+		rawFeats, ok := race["feats"].([]any)
+		if !ok {
+			return nil, fmt.Errorf("%s: feats must be an array when feat_output is set", key)
+		}
+		featRows := make([]map[string]any, 0, len(rawFeats))
+		for index, rawFeat := range rawFeats {
+			featRow, ok := rawFeat.(map[string]any)
+			if !ok {
+				return nil, fmt.Errorf("%s: feat row %d must be an object", key, index)
+			}
+			row, err := canonicalizeBaseRow(featDataset, []string{"FeatLabel", "FeatIndex"}, featRow, index)
+			if err != nil {
+				return nil, fmt.Errorf("%s: feat row %d: %w", key, index, err)
+			}
+			featRows = append(featRows, row)
+		}
+		datasets = append(datasets, newGeneratedDataset(
+			"racialtypes/registry/"+strings.TrimPrefix(key, "racialtypes:"),
+			featOutput,
+			[]string{"FeatLabel", "FeatIndex"},
+			featRows,
+		))
+	}
+
+	slices.SortFunc(coreRows, func(a, b map[string]any) int {
+		return a["id"].(int) - b["id"].(int)
+	})
+	slices.SortFunc(datasets, func(a, b nativeCollectedDataset) int {
+		return strings.Compare(a.Dataset.OutputName, b.Dataset.OutputName)
+	})
+
+	coreCollected := newGeneratedDataset("racialtypes/registry/core", "racialtypes.2da", registry.BaseColumns, coreRows)
+	out := make([]nativeCollectedDataset, 0, len(datasets)+1)
+	out = append(out, coreCollected)
+	out = append(out, datasets...)
+	return out, nil
+}
+
+func syncRacialtypesBaseLockData(rows []map[string]any, lockData map[string]int) (bool, error) {
+	modified := false
+	for _, raw := range rows {
+		key := stringField(raw, "key")
+		if key == "" {
+			continue
+		}
+		rowID, ok := raw["id"]
+		if !ok {
+			return false, fmt.Errorf("racialtypes registry base row %s is missing id", key)
+		}
+		id, err := asInt(rowID)
+		if err != nil {
+			return false, fmt.Errorf("racialtypes registry base row %s has invalid id: %w", key, err)
+		}
+		if existing, ok := lockData[key]; ok {
+			if existing != id {
+				return false, fmt.Errorf("racialtypes registry base row %s id %d does not match lock id %d", key, id, existing)
+			}
+			continue
+		}
+		lockData[key] = id
+		modified = true
+	}
+	return modified, nil
+}

internal/topdata/repadjust_migrate.go

@@ -0,0 +1,63 @@
+package topdata
+
+import (
+	"encoding/json"
+	"os"
+	"path/filepath"
+)
+
+func importLegacyRepadjust(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
+	_ = legacyTLK
+
+	legacyDir := filepath.Join(referenceBuilderDir, "data", "repadjust")
+	if _, err := os.Stat(legacyDir); err != nil {
+		if os.IsNotExist(err) {
+			return 0, nil
+		}
+		return 0, err
+	}
+
+	targetDir := filepath.Join(dataDir, "repadjust")
+	targetBasePath := filepath.Join(targetDir, "base.json")
+	targetLockPath := filepath.Join(targetDir, "lock.json")
+	if fileExists(targetBasePath) && fileExists(targetLockPath) {
+		obj, err := loadJSONObject(targetBasePath)
+		if err == nil && countLegacyTLKRefsInValue(obj) == 0 {
+			return 0, nil
+		}
+	}
+
+	baseObj, err := loadJSONObject(filepath.Join(legacyDir, "base.json"))
+	if err != nil {
+		return 0, err
+	}
+	baseObj["output"] = "repadjust.2da"
+
+	lockObj, err := loadJSONObject(filepath.Join(legacyDir, "lock.json"))
+	if err != nil {
+		return 0, err
+	}
+
+	if err := os.MkdirAll(targetDir, 0o755); err != nil {
+		return 0, err
+	}
+
+	writes := []struct {
+		path string
+		obj  map[string]any
+	}{
+		{path: targetBasePath, obj: baseObj},
+		{path: targetLockPath, obj: lockObj},
+	}
+	for _, write := range writes {
+		raw, err := json.MarshalIndent(write.obj, "", "  ")
+		if err != nil {
+			return 0, err
+		}
+		raw = append(raw, '\n')
+		if err := os.WriteFile(write.path, raw, 0o644); err != nil {
+			return 0, err
+		}
+	}
+	return len(writes), nil
+}

internal/topdata/ruleset_migrate.go

@@ -0,0 +1,171 @@
+package topdata
+
+import (
+	"encoding/json"
+	"fmt"
+	"os"
+	"path/filepath"
+	"sort"
+)
+
+func importLegacyRuleset(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
+	_ = legacyTLK
+
+	legacyDir := filepath.Join(referenceBuilderDir, "data", "ruleset")
+	if _, err := os.Stat(legacyDir); err != nil {
+		if os.IsNotExist(err) {
+			return 0, nil
+		}
+		return 0, err
+	}
+
+	targetDir := filepath.Join(dataDir, "ruleset")
+	targetPath := filepath.Join(targetDir, "ruleset.json")
+	if _, err := os.Stat(targetPath); err == nil {
+		obj, err := loadJSONObject(targetPath)
+		if err == nil && countLegacyTLKRefsInValue(obj) == 0 {
+			return 0, nil
+		}
+	}
+
+	baseObj, err := loadJSONObject(filepath.Join(legacyDir, "base.json"))
+	if err != nil {
+		return 0, err
+	}
+	rows, err := mergeRulesetRows(baseObj, filepath.Join(legacyDir, "modules"))
+	if err != nil {
+		return 0, err
+	}
+
+	if err := os.MkdirAll(targetDir, 0o755); err != nil {
+		return 0, err
+	}
+	if err := removePathIfExists(filepath.Join(targetDir, "base.json")); err != nil {
+		return 0, err
+	}
+	if err := removePathIfExists(filepath.Join(targetDir, "lock.json")); err != nil {
+		return 0, err
+	}
+	if err := removePathIfExists(filepath.Join(targetDir, "modules")); err != nil {
+		return 0, err
+	}
+
+	out := map[string]any{
+		"output":  "ruleset.2da",
+		"columns": baseObj["columns"],
+		"rows":    rows,
+	}
+	raw, err := json.MarshalIndent(out, "", "  ")
+	if err != nil {
+		return 0, err
+	}
+	raw = append(raw, '\n')
+	if err := os.WriteFile(targetPath, raw, 0o644); err != nil {
+		return 0, err
+	}
+	return 1, nil
+}
+
+func mergeRulesetRows(baseObj map[string]any, modulesDir string) ([]any, error) {
+	rawRows, ok := baseObj["rows"].([]any)
+	if !ok {
+		return nil, nil
+	}
+
+	rows := make([]map[string]any, 0, len(rawRows))
+	byLabel := map[string]map[string]any{}
+	for _, raw := range rawRows {
+		row, ok := deepCopyValue(raw).(map[string]any)
+		if !ok {
+			continue
+		}
+		delete(row, "key")
+		if rawID, ok := row["id"]; ok {
+			id, err := asInt(rawID)
+			if err != nil {
+				return nil, err
+			}
+			row["id"] = id
+		}
+		if label, ok := row["Label"].(string); ok && label != "" && label != "****" {
+			if _, exists := byLabel[label]; exists {
+				return nil, fmt.Errorf("duplicate ruleset label %q", label)
+			}
+			byLabel[label] = row
+		}
+		rows = append(rows, row)
+	}
+
+	modulePaths, err := collectModulePaths(modulesDir)
+	if err != nil {
+		return nil, err
+	}
+	for _, path := range modulePaths {
+		obj, err := loadJSONObject(path)
+		if err != nil {
+			return nil, err
+		}
+		overrideList, ok := obj["overrides"].([]any)
+		if !ok {
+			continue
+		}
+		for _, raw := range overrideList {
+			override, ok := raw.(map[string]any)
+			if !ok {
+				continue
+			}
+			matchObj, ok := override["match"].(map[string]any)
+			if !ok {
+				return nil, fmt.Errorf("%s: ruleset override is missing match object", path)
+			}
+			rawLabel, ok := matchObj["Label"]
+			if !ok {
+				return nil, fmt.Errorf("%s: ruleset override match is missing Label", path)
+			}
+			labels, err := normalizeRulesetMatchLabels(rawLabel)
+			if err != nil {
+				return nil, fmt.Errorf("%s: %w", path, err)
+			}
+			for _, label := range labels {
+				row, ok := byLabel[label]
+				if !ok {
+					return nil, fmt.Errorf("%s: ruleset override target label %q was not found", path, label)
+				}
+				for key, value := range override {
+					if key == "id" || key == "key" || key == "_tlk" || isMetadataField(key) || key == "match" {
+						continue
+					}
+					row[key] = deepCopyValue(value)
+				}
+			}
+		}
+	}
+
+	sort.Slice(rows, func(i, j int) bool {
+		return rows[i]["id"].(int) < rows[j]["id"].(int)
+	})
+	out := make([]any, 0, len(rows))
+	for _, row := range rows {
+		out = append(out, row)
+	}
+	return out, nil
+}
+
+func normalizeRulesetMatchLabels(raw any) ([]string, error) {
+	switch typed := raw.(type) {
+	case string:
+		return []string{typed}, nil
+	case []any:
+		out := make([]string, 0, len(typed))
+		for _, item := range typed {
+			label, ok := item.(string)
+			if !ok {
+				return nil, fmt.Errorf("ruleset match.Label entries must be strings")
+			}
+			out = append(out, label)
+		}
+		return out, nil
+	default:
+		return nil, fmt.Errorf("ruleset match.Label must be a string or string array")
+	}
+}

internal/topdata/skills_migrate.go

@@ -0,0 +1,140 @@
+package topdata
+
+import (
+	"encoding/json"
+	"os"
+	"path/filepath"
+)
+
+func importLegacySkills(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
+	_ = legacyTLK
+
+	legacyDir := filepath.Join(referenceBuilderDir, "data", "skills")
+	if _, err := os.Stat(legacyDir); err != nil {
+		if os.IsNotExist(err) {
+			return 0, nil
+		}
+		return 0, err
+	}
+
+	targetDir := filepath.Join(dataDir, "skills")
+	targetBasePath := filepath.Join(targetDir, "base.json")
+	targetLockPath := filepath.Join(targetDir, "lock.json")
+	targetModulesDir := filepath.Join(targetDir, "modules")
+	moduleNames := []string{
+		"add_athletics.json",
+		"add_disguise.json",
+		"add_linguistics.json",
+		"add_perception.json",
+		"add_scriptcraft.json",
+		"add_sensemotive.json",
+		"add_stealth.json",
+		"add_survival.json",
+		"add_userope.json",
+		filepath.Join("craft", "add_craftalchemy.json"),
+		filepath.Join("craft", "add_craftcooking.json"),
+		filepath.Join("craft", "add_craftjewelry.json"),
+		filepath.Join("craft", "add_craftleatherworking.json"),
+		filepath.Join("craft", "add_craftstonework.json"),
+		filepath.Join("craft", "add_crafttextiles.json"),
+		filepath.Join("craft", "ovr_craftarmorsmithing.json"),
+		filepath.Join("craft", "ovr_crafttinkering.json"),
+		filepath.Join("craft", "ovr_craftweaponsmithing.json"),
+		filepath.Join("craft", "ovr_craftwoodworking.json"),
+		filepath.Join("knowledge", "add_knowledgearcana.json"),
+		filepath.Join("knowledge", "add_knowledgearchitecture.json"),
+		filepath.Join("knowledge", "add_knowledgedungeoneering.json"),
+		filepath.Join("knowledge", "add_knowledgegeography.json"),
+		filepath.Join("knowledge", "add_knowledgehistory.json"),
+		filepath.Join("knowledge", "add_knowledgelocal.json"),
+		filepath.Join("knowledge", "add_knowledgenature.json"),
+		filepath.Join("knowledge", "add_knowledgenobility.json"),
+		filepath.Join("knowledge", "add_knowledgeplanar.json"),
+		filepath.Join("knowledge", "add_knowledgereligion.json"),
+		"ovr_acrobatics.json",
+		"ovr_animalhandling.json",
+		"ovr_appraise.json",
+		"ovr_concentration.json",
+		"ovr_disabledevice.json",
+		"ovr_heal.json",
+		"ovr_hiddenskills.json",
+		"ovr_influence.json",
+		"ovr_openlock.json",
+		"ovr_parry.json",
+		"ovr_searchtowis.json",
+		"ovr_sleightofhand.json",
+		"ovr_taunttointimidate.json",
+		"rmv_removedskills.json",
+	}
+	targetModulePaths := make([]string, 0, len(moduleNames))
+	for _, name := range moduleNames {
+		targetModulePaths = append(targetModulePaths, filepath.Join(targetModulesDir, name))
+	}
+	if fileExists(targetBasePath) && fileExists(targetLockPath) {
+		allModulesPresent := true
+		for _, path := range targetModulePaths {
+			if !fileExists(path) {
+				allModulesPresent = false
+				break
+			}
+		}
+		if allModulesPresent {
+			return 0, nil
+		}
+	}
+
+	baseObj, err := loadJSONObject(filepath.Join(legacyDir, "base.json"))
+	if err != nil {
+		return 0, err
+	}
+	baseObj["output"] = "skills.2da"
+	canonicalizeWikiMetadataDocument(baseObj)
+
+	lockObj, err := loadJSONObject(filepath.Join(legacyDir, "lock.json"))
+	if err != nil {
+		return 0, err
+	}
+
+	moduleObjs := make([]map[string]any, 0, len(moduleNames))
+	for _, name := range moduleNames {
+		obj, err := loadJSONObject(filepath.Join(legacyDir, "modules", name))
+		if err != nil {
+			return 0, err
+		}
+		canonicalizeWikiMetadataDocument(obj)
+		moduleObjs = append(moduleObjs, obj)
+	}
+
+	if err := os.MkdirAll(targetModulesDir, 0o755); err != nil {
+		return 0, err
+	}
+
+	writes := []struct {
+		path string
+		obj  map[string]any
+	}{
+		{path: targetBasePath, obj: baseObj},
+		{path: targetLockPath, obj: lockObj},
+	}
+	for i, path := range targetModulePaths {
+		if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil {
+			return 0, err
+		}
+		writes = append(writes, struct {
+			path string
+			obj  map[string]any
+		}{path: path, obj: moduleObjs[i]})
+	}
+
+	for _, write := range writes {
+		raw, err := json.MarshalIndent(write.obj, "", "  ")
+		if err != nil {
+			return 0, err
+		}
+		raw = append(raw, '\n')
+		if err := os.WriteFile(write.path, raw, 0o644); err != nil {
+			return 0, err
+		}
+	}
+	return len(writes), nil
+}

internal/topdata/skyboxes_migrate.go

@@ -0,0 +1,6 @@
+package topdata
+
+func importLegacySkyboxes(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
+	_ = legacyTLK
+	return importLegacyDatasetMirror(referenceBuilderDir, dataDir, "skyboxes", "skyboxes.2da", nil)
+}

internal/topdata/spells_migrate.go

@@ -0,0 +1,103 @@
+package topdata
+
+import (
+	"encoding/json"
+	"os"
+	"path/filepath"
+)
+
+func importLegacySpells(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
+	_ = legacyTLK
+
+	legacyDir := filepath.Join(referenceBuilderDir, "data", "spells")
+	if _, err := os.Stat(legacyDir); err != nil {
+		if os.IsNotExist(err) {
+			return 0, nil
+		}
+		return 0, err
+	}
+
+	targetDir := filepath.Join(dataDir, "spells")
+	targetBasePath := filepath.Join(targetDir, "base.json")
+	targetLockPath := filepath.Join(targetDir, "lock.json")
+	targetModulesDir := filepath.Join(targetDir, "modules")
+	moduleNames := []string{
+		"specialattacks.json",
+		"ovr_fixduplicates.json",
+		filepath.Join("bardicmusic", "fascinate.json"),
+		filepath.Join("bardicmusic", "inspirecompetence.json"),
+		filepath.Join("bardicmusic", "inspirecourage.json"),
+		filepath.Join("bardicmusic", "inspiregreatness.json"),
+		filepath.Join("bardicmusic", "inspireheroics.json"),
+		filepath.Join("bardicmusic", "songoffreedom.json"),
+	}
+	targetModulePaths := make([]string, 0, len(moduleNames))
+	for _, name := range moduleNames {
+		targetModulePaths = append(targetModulePaths, filepath.Join(targetModulesDir, name))
+	}
+	if fileExists(targetBasePath) && fileExists(targetLockPath) {
+		allModulesPresent := true
+		for _, path := range targetModulePaths {
+			if !fileExists(path) {
+				allModulesPresent = false
+				break
+			}
+		}
+		if allModulesPresent {
+			return 0, nil
+		}
+	}
+
+	baseObj, err := loadJSONObject(filepath.Join(legacyDir, "base.json"))
+	if err != nil {
+		return 0, err
+	}
+	baseObj["output"] = "spells.2da"
+
+	lockObj, err := loadJSONObject(filepath.Join(legacyDir, "lock.json"))
+	if err != nil {
+		return 0, err
+	}
+
+	moduleObjs := make([]map[string]any, 0, len(moduleNames))
+	for _, name := range moduleNames {
+		obj, err := loadJSONObject(filepath.Join(legacyDir, "modules", name))
+		if err != nil {
+			return 0, err
+		}
+		moduleObjs = append(moduleObjs, obj)
+	}
+
+	if err := os.MkdirAll(targetModulesDir, 0o755); err != nil {
+		return 0, err
+	}
+
+	writes := []struct {
+		path string
+		obj  map[string]any
+	}{
+		{path: targetBasePath, obj: baseObj},
+		{path: targetLockPath, obj: lockObj},
+	}
+	for i, path := range targetModulePaths {
+		if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil {
+			return 0, err
+		}
+		writes = append(writes, struct {
+			path string
+			obj  map[string]any
+		}{path: path, obj: moduleObjs[i]})
+	}
+
+	for _, write := range writes {
+		raw, err := json.MarshalIndent(write.obj, "", "  ")
+		if err != nil {
+			return 0, err
+		}
+		raw = append(raw, '\n')
+		if err := os.WriteFile(write.path, raw, 0o644); err != nil {
+			return 0, err
+		}
+	}
+	return len(writes), nil
+}

internal/topdata/tailmodel_migrate.go

@@ -0,0 +1,86 @@
+package topdata
+
+import (
+	"encoding/json"
+	"os"
+	"path/filepath"
+)
+
+func importLegacyTailmodel(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
+	_ = legacyTLK
+
+	legacyDir := filepath.Join(referenceBuilderDir, "data", "tailmodel")
+	if _, err := os.Stat(legacyDir); err != nil {
+		if os.IsNotExist(err) {
+			return 0, nil
+		}
+		return 0, err
+	}
+
+	targetDir := filepath.Join(dataDir, "tailmodel")
+	targetBasePath := filepath.Join(targetDir, "base.json")
+	targetLockPath := filepath.Join(targetDir, "lock.json")
+	targetModulesDir := filepath.Join(targetDir, "modules")
+	targetModulePaths := []string{
+		filepath.Join(targetModulesDir, "add.json"),
+		filepath.Join(targetModulesDir, "ovr_tailprefixes.json"),
+	}
+	if fileExists(targetBasePath) && fileExists(targetLockPath) {
+		allModulesPresent := true
+		for _, path := range targetModulePaths {
+			if !fileExists(path) {
+				allModulesPresent = false
+				break
+			}
+		}
+		if allModulesPresent {
+			return 0, nil
+		}
+	}
+
+	baseObj, err := loadJSONObject(filepath.Join(legacyDir, "base.json"))
+	if err != nil {
+		return 0, err
+	}
+	baseObj["output"] = "tailmodel.2da"
+
+	lockObj, err := loadJSONObject(filepath.Join(legacyDir, "lock.json"))
+	if err != nil {
+		return 0, err
+	}
+
+	moduleNames := []string{"add.json", "ovr_tailprefixes.json"}
+	moduleObjs := make([]map[string]any, 0, len(moduleNames))
+	for _, name := range moduleNames {
+		obj, err := loadJSONObject(filepath.Join(legacyDir, "modules", name))
+		if err != nil {
+			return 0, err
+		}
+		moduleObjs = append(moduleObjs, obj)
+	}
+
+	if err := os.MkdirAll(targetModulesDir, 0o755); err != nil {
+		return 0, err
+	}
+
+	writes := []struct {
+		path string
+		obj  map[string]any
+	}{
+		{path: targetBasePath, obj: baseObj},
+		{path: targetLockPath, obj: lockObj},
+		{path: targetModulePaths[0], obj: moduleObjs[0]},
+		{path: targetModulePaths[1], obj: moduleObjs[1]},
+	}
+	for _, write := range writes {
+		raw, err := json.MarshalIndent(write.obj, "", "  ")
+		if err != nil {
+			return 0, err
+		}
+		raw = append(raw, '\n')
+		if err := os.WriteFile(write.path, raw, 0o644); err != nil {
+			return 0, err
+		}
+	}
+	return len(writes), nil
+}

internal/topdata/tlk_native.go

@@ -0,0 +1,729 @@
+package topdata
+
+import (
+	"bytes"
+	"encoding/binary"
+	"encoding/json"
+	"fmt"
+	"math"
+	"os"
+	"path/filepath"
+	"slices"
+	"strconv"
+	"strings"
+
+	"golang.org/x/text/encoding/charmap"
+)
+
+const (
+	defaultTLKName = "sow_tlk.tlk"
+	tlkStateFile   = ".tlk_state.json"
+	customTLKBase  = 0x01000000
+)
+
+type textRole string
+
+const (
+	textRoleName        textRole = "name"
+	textRoleDescription textRole = "description"
+)
+
+type datasetTextSpec struct {
+	NameFields        []string
+	DescriptionFields []string
+}
+
+var datasetTextSpecs = map[string]datasetTextSpec{
+	"baseitems": {
+		NameFields:        []string{"Name"},
+		DescriptionFields: []string{"Description"},
+	},
+	"classes": {
+		NameFields:        []string{"Short", "Name", "Plural", "Lower", "CLASS"},
+		DescriptionFields: []string{"Description"},
+	},
+	"feat": {
+		NameFields:        []string{"FEAT"},
+		DescriptionFields: []string{"DESCRIPTION"},
+	},
+	"masterfeats": {
+		NameFields:        []string{"STRREF"},
+		DescriptionFields: []string{"DESCRIPTION"},
+	},
+	"itemprops": {
+		NameFields:        []string{"Name", "StringRef"},
+		DescriptionFields: []string{"Description", "GameStrRef"},
+	},
+	"racialtypes": {
+		NameFields:        []string{"Name", "RACIALTYPE"},
+		DescriptionFields: []string{"Description"},
+	},
+	"skills": {
+		NameFields:        []string{"Name", "SKILL"},
+		DescriptionFields: []string{"Description"},
+	},
+	"spells": {
+		NameFields:        []string{"Name", "SPELL"},
+		DescriptionFields: []string{"SpellDesc", "DESCRIPTION"},
+	},
+}
+
+type tlkEntryData struct {
+	Text        string
+	SoundResRef string
+	SoundLength float32
+}
+
+type tlkStateDocument struct {
+	Version  int                        `json:"version"`
+	Language string                     `json:"language"`
+	Entries  map[string]tlkStateMapping `json:"entries"`
+}
+
+type tlkStateMapping struct {
+	ID      int  `json:"id"`
+	Retired bool `json:"retired,omitempty"`
+}
+
+type tlkCompiledValue struct {
+	Value  any
+	TLKKey string
+}
+
+type tlkCompiler struct {
+	language     string
+	statePath    string
+	state        tlkStateDocument
+	legacy       map[string]tlkEntryData
+	active       map[string]tlkEntryData
+	activeKeys   map[string]struct{}
+	reservedByID map[int]string
+	nextID       int
+}
+
+func newTLKCompiler(sourceDir string, legacy *legacyTLKData) (*tlkCompiler, error) {
+	statePath := filepath.Join(sourceDir, tlkStateFile)
+	state, err := loadTLKState(statePath)
+	if err != nil {
+		return nil, err
+	}
+	baseDialog, err := loadBaseDialogData(filepath.Join(sourceDir, "base_dialog.json"))
+	if err != nil {
+		return nil, err
+	}
+	legacy = mergeLegacyTLK(baseDialog, legacy)
+	language := state.Language
+	if language == "" {
+		language = "en"
+	}
+	if legacy != nil && legacy.Language != "" {
+		if state.Language == "" {
+			language = legacy.Language
+		}
+		for key, id := range legacy.Lock {
+			if _, ok := state.Entries[key]; !ok {
+				state.Entries[key] = tlkStateMapping{ID: id}
+			}
+		}
+	}
+
+	reserved := map[int]string{}
+	nextID := 0
+	for key, entry := range state.Entries {
+		if owner, ok := reserved[entry.ID]; ok && owner != key {
+			return nil, fmt.Errorf("tlk state id collision between %q and %q", owner, key)
+		}
+		reserved[entry.ID] = key
+		if entry.ID >= nextID {
+			nextID = entry.ID + 1
+		}
+	}
+
+	compiler := &tlkCompiler{
+		language:     language,
+		statePath:    statePath,
+		state:        state,
+		legacy:       map[string]tlkEntryData{},
+		active:       map[string]tlkEntryData{},
+		activeKeys:   map[string]struct{}{},
+		reservedByID: reserved,
+		nextID:       nextID,
+	}
+	if legacy != nil {
+		for key, entry := range legacy.Entries {
+			compiler.legacy[key] = entry
+		}
+	}
+	return compiler, nil
+}
+
+func loadBaseDialogData(path string) (*legacyTLKData, error) {
+	if _, err := os.Stat(path); err != nil {
+		if os.IsNotExist(err) {
+			return nil, nil
+		}
+		return nil, fmt.Errorf("stat %s: %w", path, err)
+	}
+	obj, err := loadJSONObject(path)
+	if err != nil {
+		return nil, err
+	}
+	result := &legacyTLKData{
+		Language: "en",
+		Entries:  map[string]tlkEntryData{},
+		Lock:     map[string]int{},
+	}
+	if language, ok := obj["language"].(string); ok && strings.TrimSpace(language) != "" {
+		result.Language = language
+	}
+	rawEntries, ok := obj["entries"]
+	if !ok {
+		return result, nil
+	}
+	entries, err := normalizeBaseDialogEntries(rawEntries)
+	if err != nil {
+		return nil, err
+	}
+	normalized, err := normalizeLegacyTLKEntries(entries)
+	if err != nil {
+		return nil, err
+	}
+	for key, entry := range normalized {
+		result.Entries[key] = entry
+	}
+	return result, nil
+}
+
+func normalizeBaseDialogEntries(raw any) (map[string]any, error) {
+	switch typed := raw.(type) {
+	case map[string]any:
+		return typed, nil
+	case []any:
+		entries := make(map[string]any, len(typed))
+		for index, item := range typed {
+			obj, ok := item.(map[string]any)
+			if !ok {
+				return nil, fmt.Errorf("base_dialog.json entry %d must be an object", index)
+			}
+			rawID, ok := obj["id"]
+			if !ok {
+				return nil, fmt.Errorf("base_dialog.json entry %d is missing id", index)
+			}
+			key, err := legacyDialogIDKey(rawID)
+			if err != nil {
+				return nil, fmt.Errorf("base_dialog.json entry %d: %w", index, err)
+			}
+			entry := map[string]any{}
+			if text, ok := obj["text"]; ok {
+				entry["text"] = text
+			}
+			if sound, ok := obj["sound_resref"]; ok {
+				entry["sound_resref"] = sound
+			}
+			if length, ok := obj["sound_length"]; ok {
+				entry["sound_length"] = length
+			}
+			entries[key] = entry
+		}
+		return entries, nil
+	default:
+		return nil, fmt.Errorf("base_dialog.json entries must be an object or array")
+	}
+}
+
+func legacyDialogIDKey(raw any) (string, error) {
+	switch typed := raw.(type) {
+	case string:
+		if strings.TrimSpace(typed) == "" {
+			return "", fmt.Errorf("id must not be empty")
+		}
+		return typed, nil
+	case float64:
+		return strconv.Itoa(int(typed)), nil
+	case int:
+		return strconv.Itoa(typed), nil
+	default:
+		return "", fmt.Errorf("id must be a string or number")
+	}
+}
+
+func loadTLKState(path string) (tlkStateDocument, error) {
+	doc := tlkStateDocument{
+		Version: 1,
+		Entries: map[string]tlkStateMapping{},
+	}
+	raw, err := os.ReadFile(path)
+	if err != nil {
+		if os.IsNotExist(err) {
+			return doc, nil
+		}
+		return doc, fmt.Errorf("read %s: %w", path, err)
+	}
+	if err := json.Unmarshal(raw, &doc); err != nil {
+		return doc, fmt.Errorf("parse %s: %w", path, err)
+	}
+	if doc.Version == 0 {
+		doc.Version = 1
+	}
+	if doc.Entries == nil {
+		doc.Entries = map[string]tlkStateMapping{}
+	}
+	return doc, nil
+}
+
+func saveTLKState(path string, doc tlkStateDocument) error {
+	doc.Version = 1
+	if doc.Language == "" {
+		doc.Language = "en"
+	}
+	if doc.Entries == nil {
+		doc.Entries = map[string]tlkStateMapping{}
+	}
+	raw, err := json.MarshalIndent(doc, "", "  ")
+	if err != nil {
+		return fmt.Errorf("marshal tlk state: %w", err)
+	}
+	raw = append(raw, '\n')
+	current, err := os.ReadFile(path)
+	if err == nil && bytes.Equal(current, raw) {
+		return nil
+	}
+	if err != nil && !os.IsNotExist(err) {
+		return fmt.Errorf("read %s: %w", path, err)
+	}
+	if err := os.WriteFile(path, raw, 0o644); err != nil {
+		return fmt.Errorf("write %s: %w", path, err)
+	}
+	return nil
+}
+
+func (c *tlkCompiler) resolveLegacyKey(key string) (tlkCompiledValue, error) {
+	if _, ok := c.legacy[key]; !ok {
+		if active, ok := c.active[key]; ok {
+			return tlkCompiledValue{
+				Value:  c.customStrrefForKey(key),
+				TLKKey: key,
+			}, c.markActive(key, active)
+		}
+		if len(c.legacy) == 0 {
+			return tlkCompiledValue{}, fmt.Errorf("unknown TLK reference: %s (inline TLK text is required for native builds)", key)
+		}
+		return tlkCompiledValue{}, fmt.Errorf("unknown TLK reference: %s", key)
+	}
+	if err := c.markActive(key, c.legacy[key]); err != nil {
+		return tlkCompiledValue{}, err
+	}
+	return tlkCompiledValue{
+		Value:  c.customStrrefForKey(key),
+		TLKKey: key,
+	}, nil
+}
+
+func (c *tlkCompiler) registerInline(key string, entry tlkEntryData) (tlkCompiledValue, error) {
+	if err := c.markActive(key, entry); err != nil {
+		return tlkCompiledValue{}, err
+	}
+	return tlkCompiledValue{
+		Value:  c.customStrrefForKey(key),
+		TLKKey: key,
+	}, nil
+}
+
+func (c *tlkCompiler) customStrrefForKey(key string) int {
+	return customTLKBase + c.state.Entries[key].ID
+}
+
+func (c *tlkCompiler) markActive(key string, entry tlkEntryData) error {
+	mapping, ok := c.state.Entries[key]
+	if !ok {
+		mapping = tlkStateMapping{ID: c.allocateID(), Retired: false}
+		c.state.Entries[key] = mapping
+	}
+	existing, ok := c.active[key]
+	if ok && existing != entry {
+		return fmt.Errorf("conflicting TLK content for key %q", key)
+	}
+	c.active[key] = entry
+	c.activeKeys[key] = struct{}{}
+	mapping.Retired = false
+	c.state.Entries[key] = mapping
+	return nil
+}
+
+func (c *tlkCompiler) allocateID() int {
+	id := c.nextID
+	c.nextID++
+	return id
+}
+
+func (c *tlkCompiler) finish(outputDir, tlkName string) (int, error) {
+	if err := os.MkdirAll(outputDir, 0o755); err != nil {
+		return 0, fmt.Errorf("create tlk output dir: %w", err)
+	}
+
+	for key, mapping := range c.state.Entries {
+		if _, ok := c.activeKeys[key]; ok {
+			mapping.Retired = false
+		} else {
+			mapping.Retired = true
+		}
+		c.state.Entries[key] = mapping
+	}
+	pruneRetiredGeneratedFeatTLKEntries(filepath.Dir(c.statePath), c.state.Entries)
+
+	if err := saveTLKState(c.statePath, c.state); err != nil {
+		return 0, err
+	}
+
+	maxID := -1
+	for key := range c.activeKeys {
+		id := c.state.Entries[key].ID
+		if id > maxID {
+			maxID = id
+		}
+	}
+	entries := make([]tlkEntryData, maxID+1)
+	for key := range c.activeKeys {
+		id := c.state.Entries[key].ID
+		entries[id] = c.active[key]
+	}
+
+	if strings.TrimSpace(tlkName) == "" {
+		tlkName = defaultTLKName
+	}
+	if err := writeTLKBinary(filepath.Join(outputDir, tlkName), entries, c.language); err != nil {
+		return 0, err
+	}
+	if maxID < 0 {
+		return 1, nil
+	}
+	return 1, nil
+}
+
+func pruneRetiredGeneratedFeatTLKEntries(sourceDir string, entries map[string]tlkStateMapping) {
+	prefixes := collectGeneratedFeatTLKPrefixes(sourceDir)
+	for key, mapping := range entries {
+		if !mapping.Retired || !isRetiredGeneratedFeatTLKKey(key, prefixes) {
+			continue
+		}
+		delete(entries, key)
+	}
+}
+
+func isRetiredGeneratedFeatTLKKey(key string, prefixes []string) bool {
+	if !strings.HasPrefix(key, "feat:") {
+		return false
+	}
+	for _, prefix := range prefixes {
+		if strings.HasPrefix(key, prefix) {
+			return true
+		}
+	}
+	return false
+}
+
+func collectGeneratedFeatTLKPrefixes(sourceDir string) []string {
+	featGeneratedDir := filepath.Join(sourceDir, "data", "feat", "generated")
+	paths, err := collectModulePaths(featGeneratedDir)
+	if err != nil {
+		return nil
+	}
+	prefixes := make([]string, 0, len(paths))
+	seen := map[string]struct{}{}
+	for _, path := range paths {
+		obj, err := loadJSONObject(path)
+		if err != nil || !isFamilyExpansionObject(obj) {
+			continue
+		}
+		spec, err := parseFamilyExpansionSpec(path, obj)
+		if err != nil || spec.FamilyKey == "" {
+			continue
+		}
+		prefix := "feat:" + spec.FamilyKey + "_"
+		if _, ok := seen[prefix]; ok {
+			continue
+		}
+		seen[prefix] = struct{}{}
+		prefixes = append(prefixes, prefix)
+	}
+	slices.Sort(prefixes)
+	return prefixes
+}
+
+func writeTLKBinary(path string, entries []tlkEntryData, language string) error {
+	if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil {
+		return fmt.Errorf("create tlk output parent: %w", err)
+	}
+	file, err := os.Create(path)
+	if err != nil {
+		return fmt.Errorf("create tlk output: %w", err)
+	}
+	defer file.Close()
+
+	langID, err := languageID(language)
+	if err != nil {
+		return err
+	}
+
+	if _, err := file.Write([]byte("TLK V3.0")); err != nil {
+		return err
+	}
+	if err := binary.Write(file, binary.LittleEndian, uint32(langID)); err != nil {
+		return err
+	}
+	if err := binary.Write(file, binary.LittleEndian, uint32(len(entries))); err != nil {
+		return err
+	}
+	if err := binary.Write(file, binary.LittleEndian, uint32(20+len(entries)*40)); err != nil {
+		return err
+	}
+
+	stringData := make([][]byte, len(entries))
+	offset := 0
+	for index, entry := range entries {
+		flags := uint32(0)
+		if entry.Text != "" {
+			flags |= 0x1
+		}
+		if entry.SoundResRef != "" {
+			flags |= 0x2
+		}
+		if entry.SoundLength != 0 {
+			flags |= 0x4
+		}
+		encodedText, err := charmap.Windows1252.NewEncoder().Bytes([]byte(entry.Text))
+		if err != nil {
+			return fmt.Errorf("encode tlk text at row %d: %w", index, err)
+		}
+		stringData[index] = encodedText
+
+		soundBytes := make([]byte, 16)
+		if len(entry.SoundResRef) > 16 {
+			return fmt.Errorf("sound resref at row %d exceeds 16 characters", index)
+		}
+		copy(soundBytes, []byte(entry.SoundResRef))
+
+		if err := binary.Write(file, binary.LittleEndian, flags); err != nil {
+			return err
+		}
+		if _, err := file.Write(soundBytes); err != nil {
+			return err
+		}
+		if err := binary.Write(file, binary.LittleEndian, uint32(0)); err != nil {
+			return err
+		}
+		if err := binary.Write(file, binary.LittleEndian, uint32(0)); err != nil {
+			return err
+		}
+		if err := binary.Write(file, binary.LittleEndian, uint32(offset)); err != nil {
+			return err
+		}
+		if err := binary.Write(file, binary.LittleEndian, uint32(len(encodedText))); err != nil {
+			return err
+		}
+		if err := binary.Write(file, binary.LittleEndian, entry.SoundLength); err != nil {
+			return err
+		}
+		offset += len(encodedText)
+	}
+
+	for _, encoded := range stringData {
+		if _, err := file.Write(encoded); err != nil {
+			return err
+		}
+	}
+	return nil
+}
+
+func languageID(code string) (int, error) {
+	switch strings.ToLower(strings.TrimSpace(code)) {
+	case "", "en":
+		return 0, nil
+	case "fr":
+		return 1, nil
+	case "de":
+		return 2, nil
+	case "it":
+		return 3, nil
+	case "es":
+		return 4, nil
+	case "pl":
+		return 5, nil
+	default:
+		return 0, fmt.Errorf("unknown TLK language code %q", code)
+	}
+}
+
+func specForDataset(name string) datasetTextSpec {
+	best := datasetTextSpecs[name]
+	bestLength := 0
+	for prefix, spec := range datasetTextSpecs {
+		if name != prefix && !strings.HasPrefix(name, prefix+"/") {
+			continue
+		}
+		if len(prefix) <= bestLength {
+			continue
+		}
+		best = spec
+		bestLength = len(prefix)
+	}
+	return best
+}
+
+func roleForField(spec datasetTextSpec, field string) textRole {
+	for _, candidate := range spec.NameFields {
+		if strings.EqualFold(candidate, field) {
+			return textRoleName
+		}
+	}
+	for _, candidate := range spec.DescriptionFields {
+		if strings.EqualFold(candidate, field) {
+			return textRoleDescription
+		}
+	}
+	return textRole(strings.ToLower(field))
+}
+
+func columnForRole(columns []string, spec datasetTextSpec, role textRole) string {
+	var candidates []string
+	switch role {
+	case textRoleName:
+		candidates = spec.NameFields
+	case textRoleDescription:
+		candidates = spec.DescriptionFields
+	default:
+		return ""
+	}
+	for _, candidate := range candidates {
+		if column, ok := canonicalColumn(columns, candidate); ok {
+			return column
+		}
+	}
+	return ""
+}
+
+type parsedTLKPayload struct {
+	Key         string
+	Text        string
+	Ref         string
+	RefField    string
+	SoundResRef string
+	SoundLength float32
+}
+
+func parseTLKPayload(value any, allowBare bool) (parsedTLKPayload, bool, error) {
+	switch typed := value.(type) {
+	case map[string]any:
+		if rawTLK, ok := typed["tlk"]; ok {
+			if key, ok := rawTLK.(string); ok {
+				return parsedTLKPayload{Key: key}, true, nil
+			}
+			return parseTLKPayload(rawTLK, true)
+		}
+		if allowBare || hasNonRefTLKPayloadKeys(typed) {
+			if _, ok := typed["text"]; ok {
+				return buildParsedTLKPayload(typed)
+			}
+			if _, ok := typed["ref"]; ok {
+				return buildParsedTLKPayload(typed)
+			}
+			if _, ok := typed["key"]; ok {
+				return buildParsedTLKPayload(typed)
+			}
+		}
+		return parsedTLKPayload{}, false, nil
+	case string:
+		return parsedTLKPayload{}, false, nil
+	default:
+		return parsedTLKPayload{}, false, nil
+	}
+}
+
+func hasNonRefTLKPayloadKeys(obj map[string]any) bool {
+	for _, key := range []string{"text", "key", "sound_resref", "sound_length"} {
+		if _, ok := obj[key]; ok {
+			return true
+		}
+	}
+	return false
+}
+
+func buildParsedTLKPayload(obj map[string]any) (parsedTLKPayload, bool, error) {
+	payload := parsedTLKPayload{}
+	if rawKey, ok := obj["key"]; ok {
+		key, ok := rawKey.(string)
+		if !ok {
+			return payload, false, fmt.Errorf("tlk key must be a string")
+		}
+		payload.Key = key
+	}
+	if rawText, ok := obj["text"]; ok {
+		text, ok := rawText.(string)
+		if !ok {
+			return payload, false, fmt.Errorf("tlk text must be a string")
+		}
+		payload.Text = text
+	}
+	if rawRef, ok := obj["ref"]; ok {
+		ref, ok := rawRef.(string)
+		if !ok {
+			return payload, false, fmt.Errorf("tlk ref must be a string")
+		}
+		payload.Ref = ref
+	}
+	if rawField, ok := obj["field"]; ok {
+		field, ok := rawField.(string)
+		if !ok {
+			return payload, false, fmt.Errorf("tlk field must be a string")
+		}
+		payload.RefField = field
+	}
+	if rawSound, ok := obj["sound_resref"]; ok {
+		sound, ok := rawSound.(string)
+		if !ok {
+			return payload, false, fmt.Errorf("tlk sound_resref must be a string")
+		}
+		payload.SoundResRef = sound
+	}
+	if rawLength, ok := obj["sound_length"]; ok {
+		switch typed := rawLength.(type) {
+		case float64:
+			payload.SoundLength = float32(typed)
+		case int:
+			payload.SoundLength = float32(typed)
+		default:
+			return payload, false, fmt.Errorf("tlk sound_length must be numeric")
+		}
+	}
+	if payload.Ref != "" && payload.RefField == "" {
+		return payload, false, fmt.Errorf("tlk ref requires field")
+	}
+	if payload.Ref != "" && payload.Text != "" {
+		return payload, false, fmt.Errorf("tlk payload cannot contain both text and ref")
+	}
+	if payload.Ref == "" && payload.Text == "" && payload.Key == "" {
+		return payload, false, fmt.Errorf("tlk payload must contain text, ref, or key")
+	}
+	return payload, true, nil
+}
+
+func deriveAutoTLKKey(rowIdentity string, field string) string {
+	fieldText := strings.ToLower(strings.TrimSpace(field))
+	if fieldText == "" {
+		fieldText = "text"
+	}
+	return rowIdentity + "." + fieldText
+}
+
+func sortedKeys[M ~map[string]V, V any](input M) []string {
+	keys := make([]string, 0, len(input))
+	for key := range input {
+		keys = append(keys, key)
+	}
+	slices.Sort(keys)
+	return keys
+}
+
+func almostEqualFloat32(a, b float32) bool {
+	return math.Abs(float64(a-b)) < 0.0001
+}

internal/topdata/top_package.go

@@ -0,0 +1,713 @@
+package topdata
+
+import (
+	"errors"
+	"fmt"
+	"io"
+	"io/fs"
+	"os"
+	"path/filepath"
+	"slices"
+	"strings"
+	"time"
+
+	"git.westgate.pw/ShadowsOverWestgate/sow-tools/internal/erf"
+	"git.westgate.pw/ShadowsOverWestgate/sow-tools/internal/project"
+)
+
+const (
+	PackageHAKResRef   = "sow_top"
+	PackageHAKFileName = PackageHAKResRef + ".hak"
+	PackageTLKFileName = defaultTLKName
+)
+
+func compiled2DAOutputDir(p *project.Project) string {
+	return p.TopDataCompiled2DADir()
+}
+
+func topDataBuildUsesRepoCache(p *project.Project) bool {
+	return p.TopDataUsesRepoCache()
+}
+
+func compiledTLKOutputPath(p *project.Project) string {
+	return p.TopDataCompiledTLKPath()
+}
+
+func compiledTLKOutputDir(p *project.Project) string {
+	return p.TopDataCompiledTLKDir()
+}
+
+func wikiOutputRootDir(p *project.Project) string {
+	return p.TopDataWikiRootDir()
+}
+
+func wikiOutputPagesDir(p *project.Project) string {
+	return p.TopDataWikiPagesDir()
+}
+
+func wikiOutputStatePath(p *project.Project) string {
+	return p.TopDataWikiStatePath()
+}
+
+func legacyCompiled2DAOutputDir(p *project.Project) string {
+	return filepath.Join(p.TopDataSourceDir(), "assets", "2da")
+}
+
+func legacyWikiOutputRootDir(p *project.Project) string {
+	return filepath.Join(p.TopDataSourceDir(), legacyWikiRootDirName)
+}
+
+func BuildPackage(p *project.Project, progress func(string)) (PackageResult, error) {
+	if progress == nil {
+		progress = func(string) {}
+	}
+	nativeResult, err := packagedBuildResult(p)
+	if err != nil {
+		return PackageResult{}, err
+	}
+	return packageBuiltTopData(p, nativeResult, progress)
+}
+
+func BuildAndPackage(p *project.Project, progress func(string)) (PackageResult, error) {
+	return BuildAndPackageWithOptions(p, BuildAndPackageOptions{}, progress)
+}
+
+type BuildAndPackageOptions struct {
+	Force     bool
+	BuildWiki bool
+}
+
+func BuildAndPackageWithOptions(p *project.Project, opts BuildAndPackageOptions, progress func(string)) (PackageResult, error) {
+	if progress == nil {
+		progress = func(string) {}
+	}
+	if !opts.Force {
+		incrementalResult, ok, err := currentTopPackageResult(p)
+		if err != nil {
+			return PackageResult{}, err
+		}
+		if ok {
+			progress("Topdata outputs are current; skipping native build and top package refresh.")
+			return incrementalResult, nil
+		}
+
+		nativeResult, ok, err := currentCompiledBuildResult(p)
+		if err != nil {
+			return PackageResult{}, err
+		}
+		if ok {
+			progress(fmt.Sprintf("Compiled topdata outputs are current; refreshing %s and %s only.", p.TopDataPackageHAKName(), p.TopDataPackageTLKName()))
+			return packageBuiltTopData(p, nativeResult, progress)
+		}
+	}
+
+	return buildAndPackageFull(p, opts, progress)
+}
+
+func buildAndPackageFull(p *project.Project, opts BuildAndPackageOptions, progress func(string)) (PackageResult, error) {
+	nativeResult, err := BuildNativeWithOptions(p, NativeBuildOptions{BuildWiki: opts.BuildWiki, Force: opts.Force}, progress)
+	if err != nil {
+		return PackageResult{}, err
+	}
+	return packageBuiltTopData(p, nativeResult, progress)
+}
+
+func BuildPackageFromBuild(p *project.Project, nativeResult BuildResult, progress func(string)) (PackageResult, error) {
+	if progress == nil {
+		progress = func(string) {}
+	}
+	return packageBuiltTopData(p, nativeResult, progress)
+}
+
+func packageBuiltTopData(p *project.Project, nativeResult BuildResult, progress func(string)) (PackageResult, error) {
+	if progress == nil {
+		progress = func(string) {}
+	}
+	outputHAK := p.TopDataPackageHAKPath()
+	outputTLK := p.TopDataPackageTLKPath()
+
+	progress(fmt.Sprintf("Packaging compiled topdata resources into %s...", filepath.Base(outputHAK)))
+	resources, assetFiles, err := collectTopPackageResources(p, nativeResult.Output2DADir)
+	if err != nil {
+		return PackageResult{}, err
+	}
+	if err := writeHAK(outputHAK, resources); err != nil {
+		return PackageResult{}, err
+	}
+
+	progress(fmt.Sprintf("Preparing compiled TLK release output %s...", filepath.Base(outputTLK)))
+	sourceTLK := compiledTLKOutputPath(p)
+	if nativeResult.OutputTLKDir != "" {
+		sourceTLK, err = resolveCompiledTLKPath(nativeResult.OutputTLKDir)
+		if err != nil {
+			return PackageResult{}, err
+		}
+	}
+	if err := copyFile(outputTLK, sourceTLK); err != nil {
+		return PackageResult{}, err
+	}
+
+	return PackageResult{
+		Mode:          nativeResult.Mode,
+		Output2DADir:  nativeResult.Output2DADir,
+		OutputTLKDir:  nativeResult.OutputTLKDir,
+		OutputHAKPath: outputHAK,
+		OutputTLKPath: outputTLK,
+		Files2DA:      nativeResult.Files2DA,
+		FilesTLK:      nativeResult.FilesTLK,
+		HAKResources:  len(resources),
+		AssetFiles:    assetFiles,
+		WikiOutputDir: nativeResult.WikiOutputDir,
+		WikiPages:     nativeResult.WikiPages,
+		WikiStatus:    nativeResult.WikiStatus,
+	}, nil
+}
+
+func collectTopPackageResources(p *project.Project, compiled2DADir string) ([]erf.Resource, int, error) {
+	resourceByKey := map[string]erf.Resource{}
+	resourceSourceByKey := map[string]string{}
+
+	entries, err := os.ReadDir(compiled2DADir)
+	if err != nil {
+		return nil, 0, fmt.Errorf("read compiled 2da output dir: %w", err)
+	}
+	for _, entry := range entries {
+		if entry.IsDir() || !strings.EqualFold(filepath.Ext(entry.Name()), ".2da") {
+			continue
+		}
+		path := filepath.Join(compiled2DADir, entry.Name())
+		resource, err := topPackageResourceFromPath(path)
+		if err != nil {
+			return nil, 0, err
+		}
+		key := topPackageResourceKey(resource)
+		resourceByKey[key] = resource
+		resourceSourceByKey[key] = path
+	}
+
+	assetFiles := 0
+	assetsDir := filepath.Join(p.TopDataSourceDir(), "assets")
+	skipDirs := map[string]struct{}{
+		filepath.Clean(compiled2DADir):                {},
+		filepath.Clean(legacyCompiled2DAOutputDir(p)): {},
+	}
+	info, err := os.Stat(assetsDir)
+	if err == nil && info.IsDir() {
+		err = filepath.WalkDir(assetsDir, func(path string, d os.DirEntry, walkErr error) error {
+			if walkErr != nil {
+				return walkErr
+			}
+			if d.IsDir() {
+				if _, ok := skipDirs[filepath.Clean(path)]; ok {
+					return filepath.SkipDir
+				}
+			}
+			if d.IsDir() {
+				return nil
+			}
+			if strings.HasPrefix(filepath.Base(path), ".") {
+				return nil
+			}
+			resource, err := topPackageResourceFromPath(path)
+			if err != nil {
+				return err
+			}
+			key := topPackageResourceKey(resource)
+			if existing, ok := resourceSourceByKey[key]; ok {
+				return fmt.Errorf("topdata asset %s collides with %s for top package resource %s", path, existing, key)
+			}
+			resourceByKey[key] = resource
+			resourceSourceByKey[key] = path
+			assetFiles++
+			return nil
+		})
+		if err != nil {
+			return nil, 0, err
+		}
+	} else if err != nil && !os.IsNotExist(err) {
+		return nil, 0, fmt.Errorf("stat topdata assets dir: %w", err)
+	}
+
+	keys := make([]string, 0, len(resourceByKey))
+	for key := range resourceByKey {
+		keys = append(keys, key)
+	}
+	slices.Sort(keys)
+
+	resources := make([]erf.Resource, 0, len(keys))
+	for _, key := range keys {
+		resources = append(resources, resourceByKey[key])
+	}
+	return resources, assetFiles, nil
+}
+
+func shouldSkipTopDataSourceDir(path string, skipDirs map[string]struct{}) bool {
+	_, ok := skipDirs[filepath.Clean(path)]
+	return ok
+}
+
+func topDataSourceSkipDirs(p *project.Project) map[string]struct{} {
+	return map[string]struct{}{
+		filepath.Clean(filepath.Join(p.TopDataSourceDir(), "templates")): {},
+		filepath.Clean(legacyWikiOutputRootDir(p)):                       {},
+		filepath.Clean(legacyCompiled2DAOutputDir(p)):                    {},
+		filepath.Clean(wikiOutputRootDir(p)):                             {},
+	}
+}
+
+func newestTopDataSource(p *project.Project) (time.Time, string, error) {
+	newest := time.Time{}
+	newestPath := ""
+	skipDirs := topDataSourceSkipDirs(p)
+	sourceDir := p.TopDataSourceDir()
+	err := filepath.WalkDir(sourceDir, func(path string, d fs.DirEntry, walkErr error) error {
+		if walkErr != nil {
+			return walkErr
+		}
+		if d.IsDir() && shouldSkipTopDataSourceDir(path, skipDirs) {
+			return filepath.SkipDir
+		}
+		if path == sourceDir {
+			return nil
+		}
+		info, err := d.Info()
+		if err != nil {
+			return err
+		}
+		if newest.IsZero() || info.ModTime().After(newest) {
+			newest = info.ModTime()
+			newestPath = path
+		}
+		return nil
+	})
+	if err != nil {
+		return time.Time{}, "", fmt.Errorf("scan topdata sources: %w", err)
+	}
+	return newest, newestPath, nil
+}
+
+func newestTopDataInput(p *project.Project, now time.Time) (time.Time, string, error) {
+	newest, newestPath, err := newestTopDataSource(p)
+	if err != nil {
+		return time.Time{}, "", err
+	}
+	autogenTime, autogenPath, err := newestAutogenInput(p, now)
+	if err != nil {
+		return time.Time{}, "", err
+	}
+	if autogenTime.After(newest) {
+		return autogenTime, autogenPath, nil
+	}
+	return newest, newestPath, nil
+}
+
+func newestAutogenInput(p *project.Project, now time.Time) (time.Time, string, error) {
+	newest := time.Time{}
+	newestPath := ""
+	for _, consumer := range p.Config.Autogen.Consumers {
+		overrideRoot, err := resolveAutogenLocalOverrideRoot(p, consumer)
+		if err != nil {
+			return time.Time{}, "", err
+		}
+		if overrideRoot != "" {
+			candidateRoot := filepath.Join(overrideRoot, consumer.Root)
+			candidateTime, candidatePath, err := newestMatchingAutogenOverrideInput(candidateRoot, consumer.Include)
+			if err != nil {
+				return time.Time{}, "", err
+			}
+			if candidateTime.After(newest) {
+				newest = candidateTime
+				newestPath = candidatePath
+			}
+			continue
+		}
+
+		cachePath := p.AutogenCachePath(consumer.Manifest.CacheName)
+		candidateTime, candidatePath, err := newestReleasedAutogenManifestInput(p, consumer, now, cachePath)
+		if err != nil {
+			return time.Time{}, "", err
+		}
+		if candidateTime.After(newest) {
+			newest = candidateTime
+			newestPath = candidatePath
+		}
+	}
+	return newest, newestPath, nil
+}
+
+func newestMatchingAutogenOverrideInput(scanRoot string, include []string) (time.Time, string, error) {
+	info, err := os.Stat(scanRoot)
+	if err != nil {
+		if os.IsNotExist(err) {
+			return time.Time{}, "", nil
+		}
+		return time.Time{}, "", fmt.Errorf("stat autogen override root %s: %w", scanRoot, err)
+	}
+	if !info.IsDir() {
+		return time.Time{}, "", fmt.Errorf("autogen override root %s is not a directory", scanRoot)
+	}
+
+	newest := time.Time{}
+	newestPath := ""
+	err = filepath.WalkDir(scanRoot, func(path string, d fs.DirEntry, walkErr error) error {
+		if walkErr != nil {
+			return walkErr
+		}
+		if d.IsDir() {
+			return nil
+		}
+		rel, err := filepath.Rel(scanRoot, path)
+		if err != nil {
+			return err
+		}
+		if !matchesAutogenInclude(filepath.ToSlash(rel), include) {
+			return nil
+		}
+		info, err := d.Info()
+		if err != nil {
+			return err
+		}
+		if info.ModTime().After(newest) {
+			newest = info.ModTime()
+			newestPath = path
+		}
+		return nil
+	})
+	if err != nil {
+		return time.Time{}, "", fmt.Errorf("scan autogen override %s: %w", scanRoot, err)
+	}
+	return newest, newestPath, nil
+}
+
+func topPackageResourceFromPath(path string) (erf.Resource, error) {
+	extension := strings.TrimPrefix(strings.ToLower(filepath.Ext(path)), ".")
+	resourceType, ok := erf.HAKResourceTypeForExtension(extension)
+	if !ok {
+		return erf.Resource{}, fmt.Errorf("unsupported top package HAK resource extension %q", filepath.Ext(path))
+	}
+	info, err := os.Stat(path)
+	if err != nil {
+		return erf.Resource{}, fmt.Errorf("stat %s: %w", path, err)
+	}
+	return erf.Resource{
+		Name:       strings.ToLower(strings.TrimSuffix(filepath.Base(path), filepath.Ext(path))),
+		Type:       resourceType,
+		SourcePath: path,
+		Size:       info.Size(),
+	}, nil
+}
+
+func topPackageResourceKey(resource erf.Resource) string {
+	extension, ok := erf.ExtensionForResourceType(resource.Type)
+	if !ok {
+		return strings.ToLower(resource.Name)
+	}
+	return strings.ToLower(resource.Name) + "." + strings.TrimPrefix(strings.ToLower(extension), ".")
+}
+
+func writeHAK(path string, resources []erf.Resource) error {
+	output, err := os.Create(path)
+	if err != nil {
+		return fmt.Errorf("create top package hak: %w", err)
+	}
+	defer output.Close()
+	if err := erf.Write(output, erf.New("HAK ", resources)); err != nil {
+		return fmt.Errorf("write top package hak: %w", err)
+	}
+	return nil
+}
+
+func resolveCompiledTLKPath(outputDir string) (string, error) {
+	entries, err := os.ReadDir(outputDir)
+	if err != nil {
+		return "", fmt.Errorf("read compiled tlk output dir: %w", err)
+	}
+	var tlkFiles []string
+	for _, entry := range entries {
+		if entry.IsDir() || !strings.EqualFold(filepath.Ext(entry.Name()), ".tlk") {
+			continue
+		}
+		tlkFiles = append(tlkFiles, filepath.Join(outputDir, entry.Name()))
+	}
+	if len(tlkFiles) != 1 {
+		return "", fmt.Errorf("expected exactly 1 compiled tlk output, found %d", len(tlkFiles))
+	}
+	return tlkFiles[0], nil
+}
+
+func copyFile(dst, src string) error {
+	if samePath(dst, src) {
+		return nil
+	}
+	input, err := os.Open(src)
+	if err != nil {
+		return fmt.Errorf("open %s: %w", src, err)
+	}
+	defer input.Close()
+
+	output, err := os.Create(dst)
+	if err != nil {
+		return fmt.Errorf("create %s: %w", dst, err)
+	}
+	defer output.Close()
+
+	if _, err := io.Copy(output, input); err != nil {
+		return fmt.Errorf("copy %s to %s: %w", src, dst, err)
+	}
+	return nil
+}
+
+func samePath(left, right string) bool {
+	leftAbs, leftErr := filepath.Abs(left)
+	rightAbs, rightErr := filepath.Abs(right)
+	if leftErr != nil || rightErr != nil {
+		return left == right
+	}
+	return leftAbs == rightAbs
+}
+
+func packagedBuildResult(p *project.Project) (BuildResult, error) {
+	if !p.HasTopData() {
+		return BuildResult{}, errors.New("topdata is not configured for this project")
+	}
+
+	output2DA := compiled2DAOutputDir(p)
+	outputTLK := compiledTLKOutputDir(p)
+	if _, err := os.Stat(output2DA); err != nil {
+		return BuildResult{}, fmt.Errorf("topdata build output missing: run build-topdata first")
+	}
+	if _, err := os.Stat(compiledTLKOutputPath(p)); err != nil {
+		return BuildResult{}, fmt.Errorf("topdata tlk output missing: run build-topdata first")
+	}
+
+	files2DA, oldest2DA, err := countCompiledFiles(output2DA, ".2da")
+	if err != nil {
+		return BuildResult{}, err
+	}
+	filesTLK := 1
+	tlkInfo, err := os.Stat(compiledTLKOutputPath(p))
+	if err != nil {
+		return BuildResult{}, fmt.Errorf("stat compiled tlk output %s: %w", compiledTLKOutputPath(p), err)
+	}
+	oldestTLK := tlkInfo.ModTime()
+	if files2DA == 0 {
+		return BuildResult{}, fmt.Errorf("topdata build output missing: run build-topdata first")
+	}
+	if err := validateCompiled2DAOutputCatalog(p, output2DA); err != nil {
+		return BuildResult{}, err
+	}
+
+	newestSource, newestPath, err := newestTopDataInput(p, time.Now())
+	if err != nil {
+		return BuildResult{}, err
+	}
+	oldestBuild := oldest2DA
+	if oldestBuild.IsZero() || (!oldestTLK.IsZero() && oldestTLK.Before(oldestBuild)) {
+		oldestBuild = oldestTLK
+	}
+	if !newestSource.IsZero() && !oldestBuild.IsZero() && newestSource.After(oldestBuild) {
+		return BuildResult{}, fmt.Errorf("topdata build output is older than source file %s; run build-topdata first", newestPath)
+	}
+
+	return BuildResult{
+		Mode:         "native",
+		Output2DADir: output2DA,
+		OutputTLKDir: outputTLK,
+		Files2DA:     files2DA,
+		FilesTLK:     filesTLK,
+	}, nil
+}
+
+func validateCompiled2DAOutputCatalog(p *project.Project, output2DA string) error {
+	expected, err := expectedCompiled2DAOutputs(p)
+	if err != nil {
+		return err
+	}
+	actual, err := compiled2DAOutputNames(output2DA)
+	if err != nil {
+		return err
+	}
+
+	missing := make([]string, 0)
+	for outputName := range expected {
+		if _, ok := actual[outputName]; !ok {
+			missing = append(missing, outputName)
+		}
+	}
+	stale := make([]string, 0)
+	for outputName := range actual {
+		if _, ok := expected[outputName]; !ok {
+			stale = append(stale, outputName)
+		}
+	}
+	slices.Sort(missing)
+	slices.Sort(stale)
+	if len(missing) > 0 {
+		return fmt.Errorf("topdata build output is missing compiled 2da %s; run build-topdata first", strings.Join(missing, ", "))
+	}
+	if len(stale) > 0 {
+		return fmt.Errorf("topdata build output contains stale compiled 2da %s; run build-topdata first", strings.Join(stale, ", "))
+	}
+	return nil
+}
+
+func expectedCompiled2DAOutputs(p *project.Project) (map[string]struct{}, error) {
+	dataDir := filepath.Join(p.TopDataSourceDir(), "data")
+	datasets, err := discoverNativeDatasets(dataDir)
+	if err != nil {
+		return nil, err
+	}
+	registryDatasets, err := collectGeneratedRegistryDatasets(dataDir)
+	if err != nil {
+		return nil, err
+	}
+	outputs := make(map[string]struct{}, len(datasets)+len(registryDatasets))
+	for _, dataset := range datasets {
+		outputs[dataset.OutputName] = struct{}{}
+	}
+	for _, dataset := range registryDatasets {
+		outputs[dataset.Dataset.OutputName] = struct{}{}
+	}
+	return outputs, nil
+}
+
+func compiled2DAOutputNames(output2DA string) (map[string]struct{}, error) {
+	entries, err := os.ReadDir(output2DA)
+	if err != nil {
+		return nil, fmt.Errorf("read compiled 2da output dir %s: %w", output2DA, err)
+	}
+	outputs := make(map[string]struct{})
+	for _, entry := range entries {
+		if entry.IsDir() || !strings.EqualFold(filepath.Ext(entry.Name()), ".2da") {
+			continue
+		}
+		outputs[entry.Name()] = struct{}{}
+	}
+	return outputs, nil
+}
+
+func currentCompiledBuildResult(p *project.Project) (BuildResult, bool, error) {
+	nativeResult, err := packagedBuildResult(p)
+	if err == nil {
+		return nativeResult, true, nil
+	}
+	if strings.Contains(err.Error(), "run build-topdata first") {
+		return BuildResult{}, false, nil
+	}
+	return BuildResult{}, false, err
+}
+
+func currentTopPackageResult(p *project.Project) (PackageResult, bool, error) {
+	nativeResult, ok, err := currentCompiledBuildResult(p)
+	if err != nil || !ok {
+		return PackageResult{}, ok, err
+	}
+
+	outputHAK := p.TopDataPackageHAKPath()
+	outputTLK := p.TopDataPackageTLKPath()
+	hakInfo, err := os.Stat(outputHAK)
+	if err != nil {
+		if os.IsNotExist(err) {
+			return PackageResult{}, false, nil
+		}
+		return PackageResult{}, false, fmt.Errorf("stat top package hak %s: %w", outputHAK, err)
+	}
+	tlkInfo, err := os.Stat(outputTLK)
+	if err != nil {
+		if os.IsNotExist(err) {
+			return PackageResult{}, false, nil
+		}
+		return PackageResult{}, false, fmt.Errorf("stat top package tlk %s: %w", outputTLK, err)
+	}
+
+	newestSource, _, err := newestTopDataInput(p, time.Now())
+	if err != nil {
+		return PackageResult{}, false, err
+	}
+	newestCompiled, err := newestFileModTime(nativeResult.Output2DADir, ".2da")
+	if err != nil {
+		return PackageResult{}, false, err
+	}
+	compiledTLKInfo, err := os.Stat(compiledTLKOutputPath(p))
+	if err != nil {
+		return PackageResult{}, false, fmt.Errorf("stat compiled tlk output %s: %w", compiledTLKOutputPath(p), err)
+	}
+	if compiledTLKInfo.ModTime().After(newestCompiled) {
+		newestCompiled = compiledTLKInfo.ModTime()
+	}
+	newestInput := newestSource
+	if newestCompiled.After(newestInput) {
+		newestInput = newestCompiled
+	}
+
+	oldestPackage := hakInfo.ModTime()
+	if tlkInfo.ModTime().Before(oldestPackage) {
+		oldestPackage = tlkInfo.ModTime()
+	}
+	if !newestInput.IsZero() && newestInput.After(oldestPackage) {
+		return PackageResult{}, false, nil
+	}
+
+	resources, assetFiles, err := collectTopPackageResources(p, nativeResult.Output2DADir)
+	if err != nil {
+		return PackageResult{}, false, err
+	}
+	return PackageResult{
+		Mode:          "incremental-noop",
+		Output2DADir:  nativeResult.Output2DADir,
+		OutputTLKDir:  nativeResult.OutputTLKDir,
+		OutputHAKPath: outputHAK,
+		OutputTLKPath: outputTLK,
+		Files2DA:      nativeResult.Files2DA,
+		FilesTLK:      nativeResult.FilesTLK,
+		HAKResources:  len(resources),
+		AssetFiles:    assetFiles,
+		WikiOutputDir: nativeResult.WikiOutputDir,
+		WikiPages:     nativeResult.WikiPages,
+		WikiStatus:    nativeResult.WikiStatus,
+	}, true, nil
+}
+
+func newestFileModTime(dir, extension string) (time.Time, error) {
+	entries, err := os.ReadDir(dir)
+	if err != nil {
+		return time.Time{}, fmt.Errorf("read output dir %s: %w", dir, err)
+	}
+	newest := time.Time{}
+	for _, entry := range entries {
+		if entry.IsDir() || !strings.EqualFold(filepath.Ext(entry.Name()), extension) {
+			continue
+		}
+		info, err := entry.Info()
+		if err != nil {
+			return time.Time{}, fmt.Errorf("stat output %s: %w", filepath.Join(dir, entry.Name()), err)
+		}
+		if newest.IsZero() || info.ModTime().After(newest) {
+			newest = info.ModTime()
+		}
+	}
+	return newest, nil
+}
+
+func countCompiledFiles(dir, extension string) (int, time.Time, error) {
+	entries, err := os.ReadDir(dir)
+	if err != nil {
+		return 0, time.Time{}, fmt.Errorf("read compiled output dir %s: %w", dir, err)
+	}
+	count := 0
+	oldest := time.Time{}
+	for _, entry := range entries {
+		if entry.IsDir() || !strings.EqualFold(filepath.Ext(entry.Name()), extension) {
+			continue
+		}
+		info, err := entry.Info()
+		if err != nil {
+			return 0, time.Time{}, fmt.Errorf("stat compiled output %s: %w", filepath.Join(dir, entry.Name()), err)
+		}
+		count++
+		if oldest.IsZero() || info.ModTime().Before(oldest) {
+			oldest = info.ModTime()
+		}
+	}
+	return count, oldest, nil
+}

internal/topdata/vfx_persistent_migrate.go

@@ -0,0 +1,198 @@
+package topdata
+
+import (
+	"os"
+	"path/filepath"
+	"strings"
+)
+
+func importLegacyVFXPersistent(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
+	_ = legacyTLK
+	if strings.TrimSpace(referenceBuilderDir) == "" {
+		return 0, nil
+	}
+
+	legacyDir := filepath.Join(referenceBuilderDir, "data", "vfx_persistent")
+	if _, err := os.Stat(legacyDir); err != nil {
+		if os.IsNotExist(err) {
+			return 0, nil
+		}
+		return 0, err
+	}
+
+	targetDir := filepath.Join(dataDir, "vfx_persistent")
+	targetBasePath := filepath.Join(targetDir, "base.json")
+	targetLockPath := filepath.Join(targetDir, "lock.json")
+	targetModulesDir := filepath.Join(targetDir, "modules")
+	targetModulePath := filepath.Join(targetModulesDir, "freeformaoes.json")
+	if fileExists(targetBasePath) && fileExists(targetLockPath) && fileExists(targetModulePath) {
+		baseObj, baseErr := loadJSONObject(targetBasePath)
+		lockObj, lockErr := loadJSONObject(targetLockPath)
+		if baseErr == nil && lockErr == nil && vfxPersistentImportComplete(baseObj, lockObj) {
+			legacyModulePaths, err := collectJSONRelativePaths(filepath.Join(legacyDir, "modules"))
+			if err == nil {
+				targetModulePaths, err := collectJSONRelativePaths(targetModulesDir)
+				if err == nil && equalStringSlices(legacyModulePaths, targetModulePaths) {
+					return 0, nil
+				}
+			}
+		}
+	}
+
+	baseObj, err := loadJSONObject(filepath.Join(legacyDir, "base.json"))
+	if err != nil {
+		return 0, err
+	}
+	lockObj, err := synthesizeVFXPersistentKeys(baseObj)
+	if err != nil {
+		return 0, err
+	}
+	legacyLockObj, err := loadJSONObject(filepath.Join(legacyDir, "lock.json"))
+	if err != nil {
+		return 0, err
+	}
+	for key, value := range legacyLockObj {
+		lockObj[key] = value
+	}
+	if err := os.MkdirAll(targetModulesDir, 0o755); err != nil {
+		return 0, err
+	}
+
+	updated := 0
+	for _, write := range []struct {
+		path string
+		obj  map[string]any
+	}{
+		{path: targetBasePath, obj: baseObj},
+		{path: targetLockPath, obj: lockObj},
+	} {
+		changed, err := writeJSONObjectIfChanged(write.path, write.obj)
+		if err != nil {
+			return 0, err
+		}
+		if changed {
+			updated++
+		}
+	}
+
+	moduleRelPaths, err := collectJSONRelativePaths(filepath.Join(legacyDir, "modules"))
+	if err != nil {
+		return 0, err
+	}
+	for _, relPath := range moduleRelPaths {
+		moduleObj, err := loadJSONObject(filepath.Join(legacyDir, "modules", filepath.FromSlash(relPath)))
+		if err != nil {
+			return 0, err
+		}
+		targetPath := filepath.Join(targetModulesDir, filepath.FromSlash(relPath))
+		if err := os.MkdirAll(filepath.Dir(targetPath), 0o755); err != nil {
+			return 0, err
+		}
+		changed, err := writeJSONObjectIfChanged(targetPath, moduleObj)
+		if err != nil {
+			return 0, err
+		}
+		if changed {
+			updated++
+		}
+	}
+
+	targetModulePaths, err := collectJSONRelativePaths(targetModulesDir)
+	if err != nil {
+		return 0, err
+	}
+	legacyModules := make(map[string]struct{}, len(moduleRelPaths))
+	for _, relPath := range moduleRelPaths {
+		legacyModules[relPath] = struct{}{}
+	}
+	for _, relPath := range targetModulePaths {
+		if _, ok := legacyModules[relPath]; ok {
+			continue
+		}
+		if err := os.Remove(filepath.Join(targetModulesDir, filepath.FromSlash(relPath))); err != nil && !os.IsNotExist(err) {
+			return 0, err
+		}
+		updated++
+	}
+
+	return updated, nil
+}
+
+func vfxPersistentImportComplete(baseObj, lockObj map[string]any) bool {
+	if !vfxPersistentRowsHaveKeys(baseObj) {
+		return false
+	}
+	expected := map[string]int{
+		"vfx_persistent:blankradius5":  47,
+		"vfx_persistent:blankradius10": 48,
+		"vfx_persistent:blankradius15": 49,
+		"vfx_persistent:blankradius20": 50,
+		"vfx_persistent:blankradius25": 51,
+		"vfx_persistent:blankradius30": 52,
+	}
+	for key, want := range expected {
+		raw, ok := lockObj[key]
+		if !ok {
+			return false
+		}
+		got, err := asInt(raw)
+		if err != nil || got != want {
+			return false
+		}
+	}
+	return true
+}
+
+func synthesizeVFXPersistentKeys(baseObj map[string]any) (map[string]any, error) {
+	rawRows, ok := baseObj["rows"].([]any)
+	if !ok {
+		return map[string]any{}, nil
+	}
+	lockObj := map[string]any{}
+	for _, raw := range rawRows {
+		row, ok := raw.(map[string]any)
+		if !ok {
+			continue
+		}
+		label, _ := row["LABEL"].(string)
+		if label == "" {
+			continue
+		}
+		key := "vfx_persistent:" + strings.ToLower(label)
+		row["key"] = key
+		if rawID, ok := row["id"]; ok {
+			lockObj[key] = rawID
+		}
+	}
+	return lockObj, nil
+}
+
+func vfxPersistentRowsHaveKeys(baseObj map[string]any) bool {
+	rawRows, ok := baseObj["rows"].([]any)
+	if !ok {
+		return false
+	}
+	for _, raw := range rawRows {
+		row, ok := raw.(map[string]any)
+		if !ok {
+			return false
+		}
+		key, _ := row["key"].(string)
+		if key == "" {
+			return false
+		}
+	}
+	return true
+}
+
+func equalStringSlices(left, right []string) bool {
+	if len(left) != len(right) {
+		return false
+	}
+	for i := range left {
+		if left[i] != right[i] {
+			return false
+		}
+	}
+	return true
+}

internal/topdata/visualeffects_migrate.go

@@ -0,0 +1,6 @@
+package topdata
+
+func importLegacyVisualeffects(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
+	_ = legacyTLK
+	return importLegacyDatasetMirror(referenceBuilderDir, dataDir, "visualeffects", "visualeffects.2da", nil)
+}

internal/topdata/wiki_data_providers.go

@@ -0,0 +1,518 @@
+package topdata
+
+import (
+	"fmt"
+	"os"
+	"strconv"
+	"strings"
+
+	"gopkg.in/yaml.v3"
+)
+
+type wikiDataProvidersDocument struct {
+	Providers map[string]wikiDataProviderDefinition `json:"providers" yaml:"providers"`
+}
+
+type wikiDataProviderDefinition struct {
+	Source         string                          `json:"source" yaml:"source"`
+	Levels         string                          `json:"levels" yaml:"levels"`
+	ClassFeats     wikiDataProviderClassFeatConfig `json:"class_feats" yaml:"class_feats"`
+	Attacks        wikiDataProviderAttacksConfig   `json:"attacks" yaml:"attacks"`
+	Facts          wikiDataProviderFactsConfig     `json:"facts" yaml:"facts"`
+	Provider       string                          `json:"provider" yaml:"provider"`
+	Columns        int                             `json:"columns" yaml:"columns"`
+	ItemsPerColumn int                             `json:"items_per_column" yaml:"items_per_column"`
+	ForceColumns   bool                            `json:"force_columns" yaml:"force_columns"`
+}
+
+type wikiDataProviderClassFeatConfig struct {
+	Fields map[string]wikiDataProviderProjectionField `json:"fields" yaml:"fields"`
+}
+
+type wikiDataProviderProjectionField struct {
+	When string `json:"when" yaml:"when"`
+}
+
+type wikiDataProviderFactsConfig struct {
+	Rows []wikiDataProviderFactRow `json:"rows" yaml:"rows"`
+}
+
+type wikiDataProviderFactRow struct {
+	Key   string `json:"key" yaml:"key"`
+	Label string `json:"label" yaml:"label"`
+	Value string `json:"value" yaml:"value"`
+	When  string `json:"when" yaml:"when"`
+}
+
+type wikiDataProviderAttacksConfig struct {
+	Field      string                               `json:"field" yaml:"field"`
+	CountField string                               `json:"count_field" yaml:"count_field"`
+	Mode       string                               `json:"mode" yaml:"mode"`
+	Step       int                                  `json:"step" yaml:"step"`
+	Attacks    int                                  `json:"attacks" yaml:"attacks"`
+	Thresholds []wikiDataProviderAttackThreshold    `json:"thresholds" yaml:"thresholds"`
+	Overrides  []wikiDataProviderAttackOverrideRule `json:"overrides" yaml:"overrides"`
+}
+
+type wikiDataProviderAttackThreshold struct {
+	MinBAB  int `json:"min_bab" yaml:"min_bab"`
+	Attacks int `json:"attacks" yaml:"attacks"`
+}
+
+type wikiDataProviderAttackOverrideRule struct {
+	Classes []string `json:"classes" yaml:"classes"`
+	Levels  any      `json:"levels" yaml:"levels"`
+	Attacks int      `json:"attacks" yaml:"attacks"`
+	Add     int      `json:"add" yaml:"add"`
+}
+
+func loadWikiDataProviders(path string) (map[string]wikiDataProviderDefinition, error) {
+	raw, err := os.ReadFile(path)
+	if err != nil {
+		if os.IsNotExist(err) {
+			return map[string]wikiDataProviderDefinition{}, nil
+		}
+		return nil, fmt.Errorf("read wiki data providers %s: %w", path, err)
+	}
+	var doc wikiDataProvidersDocument
+	if err := yaml.Unmarshal(raw, &doc); err != nil {
+		return nil, fmt.Errorf("parse wiki data providers %s: %w", path, err)
+	}
+	for name, def := range doc.Providers {
+		if strings.TrimSpace(name) == "" {
+			return nil, fmt.Errorf("wiki data providers %s: provider name must not be empty", path)
+		}
+		if strings.TrimSpace(def.Source) == "" {
+			return nil, fmt.Errorf("wiki data providers %s: provider %s missing source", path, name)
+		}
+		source := strings.TrimSpace(def.Source)
+		switch source {
+		case "class_progression":
+			for field, projection := range def.ClassFeats.Fields {
+				if strings.TrimSpace(field) == "" {
+					return nil, fmt.Errorf("wiki data providers %s: provider %s has class feat projection without field name", path, name)
+				}
+				if strings.TrimSpace(projection.When) == "" {
+					return nil, fmt.Errorf("wiki data providers %s: provider %s class feat projection %s missing when", path, name, field)
+				}
+			}
+			if err := validateWikiDataProviderAttacks(path, name, def.Attacks); err != nil {
+				return nil, err
+			}
+		case "columns":
+			if len(def.ClassFeats.Fields) > 0 {
+				return nil, fmt.Errorf("wiki data providers %s: provider %s class feat projections require source class_progression", path, name)
+			}
+			if strings.TrimSpace(def.Attacks.Mode) != "" {
+				return nil, fmt.Errorf("wiki data providers %s: provider %s attacks config requires source class_progression", path, name)
+			}
+			if strings.TrimSpace(def.Provider) == "" {
+				return nil, fmt.Errorf("wiki data providers %s: provider %s columns provider missing provider", path, name)
+			}
+			if strings.TrimSpace(def.Provider) == name {
+				return nil, fmt.Errorf("wiki data providers %s: provider %s columns provider must not reference itself", path, name)
+			}
+			if def.Columns <= 0 {
+				return nil, fmt.Errorf("wiki data providers %s: provider %s columns must be a positive integer", path, name)
+			}
+			if def.ItemsPerColumn <= 0 {
+				return nil, fmt.Errorf("wiki data providers %s: provider %s items_per_column must be a positive integer", path, name)
+			}
+		case "facts":
+			if len(def.Facts.Rows) == 0 {
+				return nil, fmt.Errorf("wiki data providers %s: provider %s facts source requires rows", path, name)
+			}
+			for index, row := range def.Facts.Rows {
+				if strings.TrimSpace(row.Label) == "" {
+					return nil, fmt.Errorf("wiki data providers %s: provider %s facts row %d missing label", path, name, index+1)
+				}
+				if strings.TrimSpace(row.Value) == "" {
+					return nil, fmt.Errorf("wiki data providers %s: provider %s facts row %d missing value", path, name, index+1)
+				}
+			}
+		case "class_feats", "class_skills", "class_summary", "class_spellbook", "race_feats", "race_name_forms":
+			if len(def.ClassFeats.Fields) > 0 {
+				return nil, fmt.Errorf("wiki data providers %s: provider %s class feat projections require source class_progression", path, name)
+			}
+			if strings.TrimSpace(def.Attacks.Mode) != "" {
+				return nil, fmt.Errorf("wiki data providers %s: provider %s attacks config requires source class_progression", path, name)
+			}
+		default:
+			return nil, fmt.Errorf("wiki data providers %s: provider %s has unknown source %q", path, name, def.Source)
+		}
+	}
+	for name, def := range doc.Providers {
+		if strings.TrimSpace(def.Source) != "columns" {
+			continue
+		}
+		if _, ok := doc.Providers[strings.TrimSpace(def.Provider)]; !ok {
+			return nil, fmt.Errorf("wiki data providers %s: provider %s columns provider references missing provider %q", path, name, def.Provider)
+		}
+	}
+	if doc.Providers == nil {
+		return map[string]wikiDataProviderDefinition{}, nil
+	}
+	return doc.Providers, nil
+}
+
+func (ctx *wikiContext) wikiDataProviderRows(name string, page wikiTemplatePage) ([]map[string]any, error) {
+	def, ok := ctx.wikiDataProviders[name]
+	if !ok {
+		return nil, fmt.Errorf("missing wiki data provider %q in %s", name, ctx.wikiDataPath)
+	}
+	switch strings.TrimSpace(def.Source) {
+	case "columns":
+		sourceRows, err := ctx.wikiDataProviderRows(strings.TrimSpace(def.Provider), page)
+		if err != nil {
+			return nil, err
+		}
+		return wikiColumnProviderRows(sourceRows, def.Columns, def.ItemsPerColumn, def.ForceColumns), nil
+	case "class_progression":
+		if page.Category != "classes" {
+			return nil, nil
+		}
+		return ctx.classProgressionRows(def.Levels, def.ClassFeats.Fields, def.Attacks, page.Key, page.Row)
+	case "facts":
+		return ctx.wikiFactRows(name, def.Facts, page)
+	case "class_feats":
+		if page.Category != "classes" {
+			return nil, nil
+		}
+		return ctx.classFeatRows(page.Row), nil
+	case "class_skills":
+		if page.Category != "classes" {
+			return nil, nil
+		}
+		return ctx.classSkillRows(page.Row), nil
+	case "class_summary":
+		if page.Category != "classes" {
+			return nil, nil
+		}
+		return ctx.classSummaryRows(page.Row), nil
+	case "class_spellbook":
+		if page.Category != "classes" {
+			return nil, nil
+		}
+		return ctx.classSpellbookRows(page.Key, page.Row), nil
+	case "race_feats":
+		if page.Category != "racialtypes" {
+			return nil, nil
+		}
+		return ctx.raceFeatRows(page.Row), nil
+	case "race_name_forms":
+		if page.Category != "racialtypes" {
+			return nil, nil
+		}
+		return ctx.raceNameFormRows(page.Row), nil
+	default:
+		return nil, fmt.Errorf("unknown wiki data provider source %q", def.Source)
+	}
+}
+
+func (ctx *wikiContext) wikiFactRows(providerName string, cfg wikiDataProviderFactsConfig, page wikiTemplatePage) ([]map[string]any, error) {
+	out := []map[string]any{}
+	renderCtx := wikiTableRenderContext{Page: page, TableName: providerName, Path: ctx.wikiDataPath}
+	for index, spec := range cfg.Rows {
+		if strings.TrimSpace(spec.When) != "" {
+			value, err := ctx.evalWikiDataProviderTemplate(spec.When, page.Row, renderCtx)
+			if err != nil {
+				return nil, fmt.Errorf("facts provider %s row %d condition %q: %w", providerName, index+1, spec.When, err)
+			}
+			if !truthyWikiTableValue(value) {
+				continue
+			}
+		}
+		value, err := ctx.evalWikiDataProviderTemplate(spec.Value, page.Row, renderCtx)
+		if err != nil {
+			return nil, fmt.Errorf("facts provider %s row %d value %q: %w", providerName, index+1, spec.Value, err)
+		}
+		text := stringifyWikiTableValue(value)
+		if strings.TrimSpace(text) == "" || strings.TrimSpace(text) == nullValue {
+			continue
+		}
+		key := strings.TrimSpace(spec.Key)
+		if key == "" {
+			key = wikiFactKeyFromLabel(spec.Label)
+		}
+		out = append(out, map[string]any{
+			"Key":   key,
+			"Label": spec.Label,
+			"Value": text,
+		})
+	}
+	return out, nil
+}
+
+func (ctx *wikiContext) evalWikiDataProviderTemplate(template string, row map[string]any, renderCtx wikiTableRenderContext) (any, error) {
+	template = strings.TrimSpace(template)
+	if strings.HasPrefix(template, "{{") && strings.HasSuffix(template, "}}") {
+		expr := strings.TrimSpace(strings.TrimSuffix(strings.TrimPrefix(template, "{{"), "}}"))
+		parser := wikiExprParser{tokens: tokenizeWikiExpr(expr), row: row, ctx: ctx, renderCtx: renderCtx, allowMissingFields: true}
+		value, err := parser.parseComparison()
+		if err != nil {
+			return nil, err
+		}
+		if parser.peek().kind != wikiExprEOF {
+			return nil, fmt.Errorf("unexpected token %q", parser.peek().text)
+		}
+		return value, nil
+	}
+	if strings.Contains(template, "{{") {
+		return nil, fmt.Errorf("mixed literal/expression templates are not supported")
+	}
+	return template, nil
+}
+
+func wikiFactKeyFromLabel(label string) string {
+	key := strings.ToLower(strings.TrimSpace(label))
+	key = strings.Map(func(r rune) rune {
+		switch {
+		case r >= 'a' && r <= 'z':
+			return r
+		case r >= '0' && r <= '9':
+			return r
+		case r == ' ' || r == '-' || r == '_':
+			return '_'
+		default:
+			return -1
+		}
+	}, key)
+	for strings.Contains(key, "__") {
+		key = strings.ReplaceAll(key, "__", "_")
+	}
+	return strings.Trim(key, "_")
+}
+
+func validateWikiDataProviderAttacks(path, provider string, cfg wikiDataProviderAttacksConfig) error {
+	if strings.TrimSpace(cfg.Mode) == "" {
+		return nil
+	}
+	switch strings.TrimSpace(cfg.Mode) {
+	case "fixed":
+		if cfg.Attacks < 1 {
+			return fmt.Errorf("wiki data providers %s: provider %s attacks fixed mode requires positive attacks", path, provider)
+		}
+	case "bab_thresholds":
+		if len(cfg.Thresholds) == 0 {
+			return fmt.Errorf("wiki data providers %s: provider %s attacks bab_thresholds mode requires thresholds", path, provider)
+		}
+		for i, threshold := range cfg.Thresholds {
+			if threshold.MinBAB < 0 {
+				return fmt.Errorf("wiki data providers %s: provider %s attacks threshold %d min_bab must be non-negative", path, provider, i+1)
+			}
+			if threshold.Attacks < 1 {
+				return fmt.Errorf("wiki data providers %s: provider %s attacks threshold %d attacks must be positive", path, provider, i+1)
+			}
+		}
+	default:
+		return fmt.Errorf("wiki data providers %s: provider %s attacks has unknown mode %q", path, provider, cfg.Mode)
+	}
+	for i, override := range cfg.Overrides {
+		if len(override.Classes) == 0 {
+			return fmt.Errorf("wiki data providers %s: provider %s attacks override %d missing classes", path, provider, i+1)
+		}
+		for _, class := range override.Classes {
+			if strings.TrimSpace(class) == "" {
+				return fmt.Errorf("wiki data providers %s: provider %s attacks override %d has empty class", path, provider, i+1)
+			}
+		}
+		if _, err := parseWikiAttackLevels(override.Levels); err != nil {
+			return fmt.Errorf("wiki data providers %s: provider %s attacks override %d bad levels: %w", path, provider, i+1, err)
+		}
+		if override.Attacks != 0 && override.Add != 0 {
+			return fmt.Errorf("wiki data providers %s: provider %s attacks override %d must not set both attacks and add", path, provider, i+1)
+		}
+		if override.Attacks == 0 && override.Add == 0 {
+			return fmt.Errorf("wiki data providers %s: provider %s attacks override %d must set attacks or add", path, provider, i+1)
+		}
+		if override.Attacks < 0 {
+			return fmt.Errorf("wiki data providers %s: provider %s attacks override %d attacks must be positive", path, provider, i+1)
+		}
+	}
+	return nil
+}
+
+func parseWikiAttackLevels(raw any) (map[int]struct{}, error) {
+	out := map[int]struct{}{}
+	addLevel := func(value any) error {
+		switch typed := value.(type) {
+		case int:
+			if typed < 1 {
+				return fmt.Errorf("level must be positive")
+			}
+			out[typed] = struct{}{}
+			return nil
+		case int64:
+			if typed < 1 {
+				return fmt.Errorf("level must be positive")
+			}
+			out[int(typed)] = struct{}{}
+			return nil
+		case float64:
+			level := int(typed)
+			if typed != float64(level) || level < 1 {
+				return fmt.Errorf("level must be a positive integer")
+			}
+			out[level] = struct{}{}
+			return nil
+		case string:
+			text := strings.TrimSpace(typed)
+			if text == "" {
+				return fmt.Errorf("level must not be empty")
+			}
+			if strings.Contains(text, "-") {
+				start, end, err := parseLevelRange(text)
+				if err != nil {
+					return err
+				}
+				for level := start; level <= end; level++ {
+					out[level] = struct{}{}
+				}
+				return nil
+			}
+			level, err := strconv.Atoi(text)
+			if err != nil || level < 1 {
+				return fmt.Errorf("level must be a positive integer")
+			}
+			out[level] = struct{}{}
+			return nil
+		default:
+			return fmt.Errorf("unsupported level value")
+		}
+	}
+	switch typed := raw.(type) {
+	case nil:
+		return nil, fmt.Errorf("levels is required")
+	case []any:
+		if len(typed) == 0 {
+			return nil, fmt.Errorf("levels must not be empty")
+		}
+		for _, value := range typed {
+			if err := addLevel(value); err != nil {
+				return nil, err
+			}
+		}
+	default:
+		if err := addLevel(raw); err != nil {
+			return nil, err
+		}
+	}
+	return out, nil
+}
+
+func wikiColumnProviderRows(sourceRows []map[string]any, configuredColumns, itemsPerColumn int, forceColumns bool) []map[string]any {
+	if configuredColumns <= 0 || itemsPerColumn <= 0 {
+		return nil
+	}
+	needed := 0
+	if len(sourceRows) > 0 {
+		needed = (len(sourceRows) + itemsPerColumn - 1) / itemsPerColumn
+	}
+	total := needed
+	if forceColumns && configuredColumns > total {
+		total = configuredColumns
+	}
+	if total == 0 {
+		return nil
+	}
+	rows := make([]map[string]any, 0, total)
+	for i := 0; i < total; i++ {
+		start := i * itemsPerColumn
+		end := start + itemsPerColumn
+		if start > len(sourceRows) {
+			start = len(sourceRows)
+		}
+		if end > len(sourceRows) {
+			end = len(sourceRows)
+		}
+		items := make([]map[string]any, 0, end-start)
+		for _, source := range sourceRows[start:end] {
+			items = append(items, cloneRowMap(source))
+		}
+		rows = append(rows, map[string]any{
+			"Index": i + 1,
+			"Count": len(items),
+			"Items": items,
+		})
+	}
+	return rows
+}
+
+func (ctx *wikiContext) classSkillRows(classRow map[string]any) []map[string]any {
+	table := ctx.tableForValue(fieldValue(classRow, "SkillsTable"), ctx.classSkillTables)
+	if table == nil {
+		return nil
+	}
+	rows := []map[string]any{}
+	for _, source := range table.Rows {
+		if stringValue(source, "ClassSkill") != "1" {
+			continue
+		}
+		key := resolveReferenceKey(fieldValue(source, "SkillIndex"), ctx.skillIDToKey)
+		if key == "" {
+			key = resolveReferenceKey(fieldValue(source, "SkillIndex"), nil)
+		}
+		if key == "" || !ctx.canReferenceWikiKey(key) {
+			continue
+		}
+		row := cloneRowMap(source)
+		row["SkillKey"] = key
+		row["SkillName"] = ctx.resolveSkillName(key)
+		if skillRow := ctx.skillRows[key]; skillRow != nil {
+			row["SkillAbility"] = stringValue(skillRow, "KeyAbility")
+		}
+		rows = append(rows, row)
+	}
+	return rows
+}
+
+func (ctx *wikiContext) raceFeatRows(raceRow map[string]any) []map[string]any {
+	values := []any{}
+	switch typed := fieldValue(raceRow, "FeatsTable").(type) {
+	case []any:
+		values = append(values, typed...)
+	case []string:
+		for _, value := range typed {
+			values = append(values, value)
+		}
+	default:
+		if table := ctx.tableForValue(typed, ctx.raceFeatTables); table != nil {
+			for _, row := range table.Rows {
+				values = append(values, fieldValue(row, "FeatIndex"))
+			}
+		}
+	}
+	rows := []map[string]any{}
+	for _, value := range values {
+		key := resolveReferenceKey(value, ctx.featIDToKey)
+		if key == "" {
+			key = resolveReferenceKey(value, nil)
+		}
+		if key == "" || !ctx.canReferenceWikiKey(key) {
+			continue
+		}
+		rows = append(rows, map[string]any{
+			"FeatKey":  key,
+			"FeatName": ctx.resolveFeatName(key),
+		})
+	}
+	return rows
+}
+
+func (ctx *wikiContext) raceNameFormRows(raceRow map[string]any) []map[string]any {
+	candidates := []struct {
+		label string
+		field string
+	}{
+		{"Plural", "NamePlural"},
+		{"Converted Name", "ConverName"},
+		{"Lower Name", "ConverNameLower"},
+	}
+	rows := []map[string]any{}
+	for _, candidate := range candidates {
+		if value := ctx.resolveTextValue(fieldValue(raceRow, candidate.field), nil); value != "" {
+			rows = append(rows, map[string]any{"Label": candidate.label, "Value": value})
+		}
+	}
+	return rows
+}

internal/topdata/wiki_formatters.go

@@ -0,0 +1,286 @@
+package topdata
+
+import (
+	"fmt"
+	"html"
+	"strings"
+)
+
+func (ctx *wikiContext) renderWikiFormatter(spec string, page wikiTemplatePage) (string, error) {
+	name := strings.Fields(spec)
+	if len(name) == 0 {
+		return "", fmt.Errorf("render wiki template for %s: empty formatter", page.PageID)
+	}
+	switch name[0] {
+	case "Description":
+		return renderWikiTemplatePlainHTML(ctx.resolveRowDescription(page.Category, page.Row)), nil
+	case "FactsTable":
+		return ctx.renderFactsHTML(page.Category, page.Key, page.Row), nil
+	case "Prerequisites":
+		return renderWikiFactTableHTML([]wikiHTMLFact{{Label: "Prerequisites", Value: ctx.renderFeatPrerequisites(page.Row)}}), nil
+	case "SkillList":
+		if page.Category != "classes" {
+			return "", nil
+		}
+		return ctx.renderClassSkillListHTML(page.Row), nil
+	case "ClassFeatureTable":
+		if page.Category != "classes" {
+			return "", nil
+		}
+		return ctx.renderClassFeatureTableHTML(page.Row), nil
+	case "FeatProgression":
+		if page.Category != "classes" {
+			return "", nil
+		}
+		return ctx.renderClassFeatProgressionHTML(page.Row), nil
+	case "SpellTable", "SavesProgression", "BABProgression":
+		return "", nil
+	case "RaceFeatList":
+		return renderWikiFactTableHTML([]wikiHTMLFact{{Label: "Racial Feats", Value: ctx.renderRaceFeatList(page.Row)}}), nil
+	case "StatusCallout":
+		return buildWikiStatusBlock(page.Status, page.Category), nil
+	case "RaceNameForms":
+		return ctx.renderRaceNameFormsHTML(page.Row), nil
+	case "UserBottomFallback":
+		return "", nil
+	default:
+		return "", fmt.Errorf("render wiki template for %s: unknown formatter %q", page.PageID, name[0])
+	}
+}
+
+type wikiHTMLFact struct {
+	Label string
+	Value string
+}
+
+func (ctx *wikiContext) renderFactsHTML(category, key string, row map[string]any) string {
+	facts := []wikiHTMLFact{}
+	add := func(label, value string) {
+		if strings.TrimSpace(value) != "" {
+			facts = append(facts, wikiHTMLFact{Label: label, Value: value})
+		}
+	}
+	switch category {
+	case "feat":
+		for _, fact := range ctx.renderFeatFactsHTML(row) {
+			add(fact.Label, fact.Value)
+		}
+	case "skills":
+		add("Key Ability", stringValue(row, "KeyAbility"))
+		add("Untrained", yesNoValue(stringValue(row, "Untrained")))
+		add("Armor Check Penalty", yesNoValue(stringValue(row, "ArmorCheckPenalty")))
+	case "spells":
+		add("Constant", stringValue(row, "Constant"))
+	case "baseitems":
+		add("Item Class", stringValue(row, "ItemClass"))
+		add("Weapon Type", stringValue(row, "WeaponType"))
+		add("Required Feats", ctx.renderReferenceList(ctx.wikiReqFeats(row)))
+		add("Base Item Stats", ctx.resolveTextValue(fieldValue(row, "BaseItemStatRef"), nil))
+	case "classes":
+		add("Hit Die", "d"+stringValue(row, "HitDie"))
+		add("Skill Points", stringValue(row, "SkillPointBase"))
+		add("Primary Ability", stringValue(row, "PrimaryAbil"))
+	case "racialtypes":
+		add("Ability Adjustments", formatAbilityAdjustments(row))
+		add("Favored Class", ctx.renderReference(fieldValue(row, "Favored"), ctx.classIDToKey, ctx.resolveClassName))
+		add("Favored Enemy", ctx.renderReference(fieldValue(row, "FavoredEnemyFeat"), ctx.featIDToKey, ctx.resolveFeatName))
+		add("Racial Feats", ctx.renderRaceFeatList(row))
+	}
+	_ = key
+	return renderWikiFactTableHTML(facts)
+}
+
+func (ctx *wikiContext) renderFeatFactsHTML(row map[string]any) []wikiHTMLFact {
+	return []wikiHTMLFact{
+		{Label: "Prerequisites", Value: ctx.renderFeatPrerequisites(row)},
+		{Label: "Minimum Attack Bonus", Value: stringValue(row, "MINATTACKBONUS")},
+		{Label: "Minimum Spell Level", Value: stringValue(row, "MINSPELLLVL")},
+		{Label: "Minimum Fortitude Save", Value: stringValue(row, "MinFortSave")},
+	}
+}
+
+func renderWikiFactTableHTML(facts []wikiHTMLFact) string {
+	rows := []string{}
+	for _, fact := range facts {
+		if strings.TrimSpace(fact.Value) == "" {
+			continue
+		}
+		rows = append(rows, `<tr><th scope="row">`+html.EscapeString(fact.Label)+`</th><td>`+renderMultilineInlineHTML(fact.Value)+`</td></tr>`)
+	}
+	if len(rows) == 0 {
+		return ""
+	}
+	return `<table class="wiki-facts"><tbody>` + strings.Join(rows, "\n") + `</tbody></table>`
+}
+
+func renderMultilineInlineHTML(text string) string {
+	lines := []string{}
+	for _, line := range strings.Split(strings.ReplaceAll(strings.TrimSpace(text), "\r\n", "\n"), "\n") {
+		line = strings.TrimSpace(line)
+		if line != "" {
+			lines = append(lines, renderInlineHTML(line))
+		}
+	}
+	return strings.Join(lines, "<br>")
+}
+
+func renderWikiListHTML(class string, items []string) string {
+	rendered := []string{}
+	for _, item := range items {
+		if strings.TrimSpace(item) != "" {
+			rendered = append(rendered, "<li>"+renderInlineHTML(item)+"</li>")
+		}
+	}
+	if len(rendered) == 0 {
+		return ""
+	}
+	classAttr := ""
+	if class != "" {
+		classAttr = ` class="` + html.EscapeString(class) + `"`
+	}
+	return "<ul" + classAttr + ">" + strings.Join(rendered, "\n") + "</ul>"
+}
+
+func (ctx *wikiContext) renderClassSkillList(row map[string]any) string {
+	table := ctx.tableForValue(fieldValue(row, "SkillsTable"), ctx.classSkillTables)
+	if table == nil {
+		return ""
+	}
+	skills := []string{}
+	for _, skillRow := range table.Rows {
+		if stringValue(skillRow, "ClassSkill") != "1" {
+			continue
+		}
+		name := ctx.renderReference(fieldValue(skillRow, "SkillIndex"), ctx.skillIDToKey, ctx.resolveSkillName)
+		if name != "" {
+			skills = append(skills, name)
+		}
+	}
+	if len(skills) == 0 {
+		return ""
+	}
+	return "==== Class Skills ====\n\n  * " + strings.Join(skills, ", ")
+}
+
+func (ctx *wikiContext) renderClassSkillListHTML(row map[string]any) string {
+	table := ctx.tableForValue(fieldValue(row, "SkillsTable"), ctx.classSkillTables)
+	if table == nil {
+		return ""
+	}
+	skills := []string{}
+	for _, skillRow := range table.Rows {
+		if stringValue(skillRow, "ClassSkill") != "1" {
+			continue
+		}
+		name := ctx.renderReference(fieldValue(skillRow, "SkillIndex"), ctx.skillIDToKey, ctx.resolveSkillName)
+		if name != "" {
+			skills = append(skills, name)
+		}
+	}
+	if len(skills) == 0 {
+		return ""
+	}
+	return `<h4>Class Skills</h4>` + renderWikiListHTML("wiki-list wiki-class-skills", skills)
+}
+
+func (ctx *wikiContext) renderClassFeatProgression(row map[string]any) string {
+	table := ctx.tableForValue(fieldValue(row, "FeatsTable"), ctx.classFeatTables)
+	if table == nil {
+		return ""
+	}
+	lines := []string{}
+	byLevel := map[string][]string{}
+	selectable := []string{}
+	for _, featRow := range table.Rows {
+		name := ctx.renderReference(fieldValue(featRow, "FeatIndex"), ctx.featIDToKey, ctx.resolveFeatName)
+		if name == "" {
+			name = ctx.renderReference(fieldValue(featRow, "FeatIndex"), nil, func(string) string { return "" })
+		}
+		if name == "" {
+			continue
+		}
+		level := stringValue(featRow, "GrantedOnLevel")
+		if level == "" || strings.HasPrefix(level, "-") {
+			selectable = append(selectable, name)
+			continue
+		}
+		byLevel[level] = append(byLevel[level], name)
+	}
+	if len(byLevel) > 0 {
+		lines = append(lines, "==== Granted Feats ====", "")
+		for _, level := range sortedKeys(byLevel) {
+			lines = append(lines, "  * Level "+level+": "+strings.Join(byLevel[level], ", "))
+		}
+		lines = append(lines, "")
+	}
+	if len(selectable) > 0 {
+		lines = append(lines, "==== Selectable Feats ====", "", "  * "+strings.Join(selectable, ", "), "")
+	}
+	return strings.TrimSpace(strings.Join(lines, "\n"))
+}
+
+func (ctx *wikiContext) renderClassFeatProgressionHTML(row map[string]any) string {
+	table := ctx.tableForValue(fieldValue(row, "FeatsTable"), ctx.classFeatTables)
+	if table == nil {
+		return ""
+	}
+	byLevel := map[string][]string{}
+	selectable := []string{}
+	for _, featRow := range table.Rows {
+		name := ctx.renderReference(fieldValue(featRow, "FeatIndex"), ctx.featIDToKey, ctx.resolveFeatName)
+		if name == "" {
+			name = ctx.renderReference(fieldValue(featRow, "FeatIndex"), nil, func(string) string { return "" })
+		}
+		if name == "" {
+			continue
+		}
+		level := stringValue(featRow, "GrantedOnLevel")
+		if level == "" || strings.HasPrefix(level, "-") {
+			selectable = append(selectable, name)
+			continue
+		}
+		byLevel[level] = append(byLevel[level], name)
+	}
+	parts := []string{}
+	if len(byLevel) > 0 {
+		items := []string{}
+		for _, level := range sortedKeys(byLevel) {
+			items = append(items, "Level "+level+": "+strings.Join(byLevel[level], ", "))
+		}
+		parts = append(parts, `<h4>Granted Feats</h4>`+renderWikiListHTML("wiki-list wiki-class-granted-feats", items))
+	}
+	if len(selectable) > 0 {
+		parts = append(parts, `<h4>Selectable Feats</h4>`+renderWikiListHTML("wiki-list wiki-class-selectable-feats", []string{strings.Join(selectable, ", ")}))
+	}
+	return strings.Join(parts, "\n")
+}
+
+func (ctx *wikiContext) renderClassFeatureTable(row map[string]any) string {
+	table := ctx.tableForValue(fieldValue(row, "BonusFeatsTable"), ctx.classBonusTables)
+	if table == nil || len(table.Rows) == 0 {
+		return ""
+	}
+	return "==== Bonus Feats ====\n\n  * Bonus feat table: " + table.OutputStem
+}
+
+func (ctx *wikiContext) renderClassFeatureTableHTML(row map[string]any) string {
+	table := ctx.tableForValue(fieldValue(row, "BonusFeatsTable"), ctx.classBonusTables)
+	if table == nil || len(table.Rows) == 0 {
+		return ""
+	}
+	return `<h4>Bonus Feats</h4>` + renderWikiListHTML("wiki-list wiki-class-bonus-feats", []string{"Bonus feat table: " + table.OutputStem})
+}
+
+func (ctx *wikiContext) renderRaceNameFormsHTML(row map[string]any) string {
+	facts := []wikiHTMLFact{}
+	if plural := ctx.resolveTextValue(fieldValue(row, "NamePlural"), nil); plural != "" {
+		facts = append(facts, wikiHTMLFact{Label: "Plural", Value: plural})
+	}
+	if converted := ctx.resolveTextValue(fieldValue(row, "ConverName"), nil); converted != "" {
+		facts = append(facts, wikiHTMLFact{Label: "Converted Name", Value: converted})
+	}
+	if lower := ctx.resolveTextValue(fieldValue(row, "ConverNameLower"), nil); lower != "" {
+		facts = append(facts, wikiHTMLFact{Label: "Lower Name", Value: lower})
+	}
+	return renderWikiFactTableHTML(facts)
+}

internal/topdata/wiki_page_paths.go

@@ -0,0 +1,77 @@
+package topdata
+
+import (
+	"fmt"
+	"os"
+	"strings"
+
+	"gopkg.in/yaml.v3"
+)
+
+type wikiPagePathDocument struct {
+	PagePaths wikiPagePaths `json:"page_paths" yaml:"page_paths"`
+}
+
+type wikiPagePaths struct {
+	SlugOverrides []wikiPageSlugOverride `json:"slug_overrides" yaml:"slug_overrides"`
+}
+
+type wikiPageSlugOverride struct {
+	PageID string `json:"page_id" yaml:"page_id"`
+	Slug   string `json:"slug" yaml:"slug"`
+}
+
+func loadWikiPagePathDeclarations(path string) (map[string]string, error) {
+	raw, err := os.ReadFile(path)
+	if os.IsNotExist(err) {
+		return map[string]string{}, nil
+	}
+	if err != nil {
+		return nil, err
+	}
+	return parseWikiPagePathDeclarations(raw, path)
+}
+
+func parseWikiPagePathDeclarations(raw []byte, path string) (map[string]string, error) {
+	var doc wikiPagePathDocument
+	if err := yaml.Unmarshal(raw, &doc); err != nil {
+		return nil, fmt.Errorf("parse wiki page paths %s: %w", path, err)
+	}
+	if len(doc.PagePaths.SlugOverrides) > 0 {
+		return nil, fmt.Errorf("parse wiki page paths %s: page_paths.slug_overrides is retired; generated public wiki paths are title-derived", path)
+	}
+	overrides := map[string]string{}
+	for _, override := range doc.PagePaths.SlugOverrides {
+		pageID := strings.TrimSpace(override.PageID)
+		slug := strings.TrimSpace(override.Slug)
+		if pageID == "" {
+			return nil, fmt.Errorf("parse wiki page paths %s: slug override has empty page_id", path)
+		}
+		if !isCanonicalWikiSlug(slug) {
+			return nil, fmt.Errorf("parse wiki page paths %s: page_id %q has invalid slug %q", path, pageID, override.Slug)
+		}
+		if _, ok := overrides[pageID]; ok {
+			return nil, fmt.Errorf("parse wiki page paths %s: duplicate page_id %q", path, pageID)
+		}
+		overrides[pageID] = slug
+	}
+	return overrides, nil
+}
+
+func isCanonicalWikiSlug(value string) bool {
+	if value == "" || value != strings.ToLower(value) {
+		return false
+	}
+	parts := strings.Split(value, "-")
+	for _, part := range parts {
+		if part == "" {
+			return false
+		}
+		for _, r := range part {
+			if !((r >= 'a' && r <= 'z') || (r >= '0' && r <= '9')) {
+				return false
+			}
+		}
+	}
+	return true
+}

internal/topdata/wiki_preserve.go

@@ -0,0 +1,54 @@
+package topdata
+
+import "strings"
+
+func normalizePreservedSectionBodies(content string) string {
+	var out strings.Builder
+	offset := 0
+	for {
+		startRel := strings.Index(content[offset:], manualStartMarkerPrefix)
+		if startRel < 0 {
+			out.WriteString(content[offset:])
+			break
+		}
+		start := offset + startRel
+		startEndRel := strings.Index(content[start:], "-->")
+		if startEndRel < 0 {
+			out.WriteString(content[offset:])
+			break
+		}
+		startMarker := content[start : start+startEndRel+len("-->")]
+		id := markerID(startMarker)
+		if id == "" {
+			out.WriteString(content[offset : start+len(startMarker)])
+			offset = start + len(startMarker)
+			continue
+		}
+		endPrefix := manualEndMarkerPrefix + " id=\"" + id + "\""
+		endRel := strings.Index(content[start+len(startMarker):], endPrefix)
+		if endRel < 0 {
+			out.WriteString(content[offset:])
+			break
+		}
+		end := start + len(startMarker) + endRel
+		endEndRel := strings.Index(content[end:], "-->")
+		if endEndRel < 0 {
+			out.WriteString(content[offset:])
+			break
+		}
+		endMarker := content[end : end+endEndRel+len("-->")]
+		out.WriteString(content[offset:start])
+		out.WriteString(startMarker)
+		out.WriteString("\n<!-- preserved-section-body -->\n")
+		out.WriteString(endMarker)
+		offset = end + len(endMarker)
+	}
+	return out.String()
+}
+
+func defaultWikiManualSections() []wikiManualSection {
+	return []wikiManualSection{
+		{ID: "user_top", Alias: "UserTopSection", Heading: "", InitialHTML: "<p></p>"},
+		{ID: "user_bottom", Alias: "UserBottomSection", Heading: "", InitialHTML: "<p></p>"},
+	}
+}

internal/topdata/wiki_slug.go

@@ -0,0 +1,210 @@
+package topdata
+
+import (
+	"html"
+	"strings"
+	"unicode"
+
+	"golang.org/x/text/unicode/norm"
+)
+
+var reservedCanonicalWikiFirstSegments = map[string]struct{}{
+	"admin":     {},
+	"api":       {},
+	"category":  {},
+	"compose":   {},
+	"edit":      {},
+	"namespace": {},
+	"search":    {},
+}
+
+func canonicalWikiSegment(value string) string {
+	value = html.UnescapeString(strings.TrimSpace(value))
+	var b strings.Builder
+	lastSeparator := false
+	for _, r := range norm.NFKD.String(value) {
+		transliterated := transliterateWikiSlugRune(unicode.ToLower(r))
+		switch {
+		case (r >= 'a' && r <= 'z') || (r >= 'A' && r <= 'Z') || (r >= '0' && r <= '9'):
+			b.WriteRune(r)
+			lastSeparator = false
+		case r == '_':
+			if b.Len() > 0 && !lastSeparator {
+				b.WriteRune(r)
+				lastSeparator = true
+			}
+		case unicode.Is(unicode.Mn, r):
+			continue
+		case isWikiSlugJoinerPunctuation(r):
+			continue
+		case transliterated != "":
+			b.WriteString(matchWikiTransliterationCase(r, transliterated))
+			lastSeparator = false
+		default:
+			if b.Len() > 0 && !lastSeparator {
+				b.WriteByte('_')
+				lastSeparator = true
+			}
+		}
+	}
+	return strings.Trim(b.String(), "_")
+}
+
+func canonicalWikiSegmentFoldedKey(value string) string {
+	return strings.Join(strings.Fields(strings.ToLower(strings.ReplaceAll(canonicalWikiSegment(value), "_", " "))), " ")
+}
+
+func canonicalWikiSlashPath(value string) string {
+	segments := []string{}
+	for _, part := range strings.Split(strings.Trim(strings.TrimSpace(value), "/"), "/") {
+		canonical := canonicalWikiSegment(part)
+		if canonical != "" {
+			segments = append(segments, canonical)
+		}
+	}
+	return strings.Join(segments, "/")
+}
+
+func canonicalWikiSlashPathFoldedKey(value string) string {
+	segments := []string{}
+	for _, part := range strings.Split(strings.Trim(strings.TrimSpace(value), "/"), "/") {
+		folded := canonicalWikiSegmentFoldedKey(part)
+		if folded != "" {
+			segments = append(segments, folded)
+		}
+	}
+	return strings.Join(segments, "/")
+}
+
+func canonicalWikiSlashPathSegments(value string) []string {
+	segments := []string{}
+	for _, part := range strings.Split(strings.Trim(strings.TrimSpace(value), "/"), "/") {
+		canonical := canonicalWikiSegment(part)
+		if canonical != "" {
+			segments = append(segments, canonical)
+		}
+	}
+	return segments
+}
+
+func isReservedCanonicalWikiFirstSegment(path string) bool {
+	segments := canonicalWikiSlashPathSegments(path)
+	if len(segments) == 0 {
+		return false
+	}
+	first := strings.ToLower(segments[0])
+	if isCanonicalWikiNumericSegment(first) {
+		return true
+	}
+	_, reserved := reservedCanonicalWikiFirstSegments[first]
+	return reserved
+}
+
+func isCanonicalWikiNumericSegment(segment string) bool {
+	if segment == "" {
+		return false
+	}
+	for _, r := range segment {
+		if r < '0' || r > '9' {
+			return false
+		}
+	}
+	return true
+}
+
+func isNestedCanonicalWikiPath(path string) bool {
+	return len(canonicalWikiSlashPathSegments(path)) > 2
+}
+
+func matchWikiTransliterationCase(r rune, value string) string {
+	if unicode.IsUpper(r) {
+		if len(value) == 1 {
+			return strings.ToUpper(value)
+		}
+		return strings.ToUpper(value[:1]) + value[1:]
+	}
+	return value
+}
+
+func canonicalWikiPath(segments ...string) string {
+	out := []string{}
+	for _, segment := range segments {
+		canonical := canonicalWikiSegment(segment)
+		if canonical != "" {
+			out = append(out, canonical)
+		}
+	}
+	return strings.Join(out, "/")
+}
+
+func slugifyWikiText(value string, fallback string) string {
+	value = html.UnescapeString(strings.TrimSpace(value))
+	var b strings.Builder
+	lastDash := false
+	for _, r := range norm.NFKD.String(strings.ToLower(value)) {
+		transliterated := transliterateWikiSlugRune(r)
+		switch {
+		case r >= 'a' && r <= 'z':
+			b.WriteRune(r)
+			lastDash = false
+		case r >= '0' && r <= '9':
+			b.WriteRune(r)
+			lastDash = false
+		case unicode.Is(unicode.Mn, r):
+			continue
+		case isWikiSlugJoinerPunctuation(r):
+			continue
+		case transliterated != "":
+			b.WriteString(transliterated)
+			lastDash = false
+		default:
+			if b.Len() > 0 && !lastDash {
+				b.WriteByte('-')
+				lastDash = true
+			}
+		}
+	}
+	slug := strings.Trim(b.String(), "-")
+	if slug == "" {
+		return fallback
+	}
+	return slug
+}
+
+func isWikiSlugJoinerPunctuation(r rune) bool {
+	switch r {
+	case '\'', '"', '`', '´', '‘', '’', '‚', '‛', '“', '”', '„', '‟', '‹', '›', '«', '»', '′', '″', '＇', '＂':
+		return true
+	default:
+		return false
+	}
+}
+
+func transliterateWikiSlugRune(r rune) string {
+	switch r {
+	case 'æ':
+		return "ae"
+	case 'œ':
+		return "oe"
+	case 'ø':
+		return "o"
+	case 'ß':
+		return "ss"
+	case 'þ':
+		return "th"
+	case 'ð', 'đ':
+		return "d"
+	case 'ł':
+		return "l"
+	case 'ħ':
+		return "h"
+	case 'ı':
+		return "i"
+	case 'ŋ':
+		return "n"
+	case 'ŧ':
+		return "t"
+	default:
+		return ""
+	}
+}

internal/topdata/wiki_template.go

@@ -0,0 +1,433 @@
+package topdata
+
+import (
+	"fmt"
+	"html"
+	"os"
+	"path/filepath"
+	"strings"
+)
+
+type wikiTemplatePage struct {
+	PageID         string
+	Category       string
+	Key            string
+	Title          string
+	Status         string
+	Row            map[string]any
+	ManualSections []wikiManualSection
+}
+
+func (ctx *wikiContext) renderWikiPageTemplate(category string, page wikiTemplatePage) (string, error) {
+	name := wikiTemplateName(category)
+	source, path := ctx.loadWikiTemplateSource(name)
+	if source == "" && name != "default.html" {
+		source, path = ctx.loadWikiTemplateSource("default.html")
+	}
+	if source == "" {
+		source = builtinWikiTemplate(category)
+		path = "builtin:" + name
+	}
+	return ctx.renderWikiTemplateString(path, source, page)
+}
+
+func wikiTemplateName(category string) string {
+	switch category {
+	case "classes", "feat", "skills", "spells", "racialtypes", "baseitems":
+		return category + ".html"
+	default:
+		return "default.html"
+	}
+}
+
+func (ctx *wikiContext) loadWikiTemplateSource(name string) (string, string) {
+	if strings.TrimSpace(ctx.templateDir) == "" {
+		return "", ""
+	}
+	path := filepath.Join(ctx.templateDir, filepath.FromSlash(name))
+	raw, err := os.ReadFile(path)
+	if err != nil {
+		return "", path
+	}
+	return string(raw), path
+}
+
+func builtinWikiTemplate(category string) string {
+	switch category {
+	case "classes":
+		return `<h1>{{title}}</h1>
+{{UserTopSection}}
+{{format:StatusCallout}}
+{{format:Description}}
+{{format:FactsTable}}
+{{format:SkillList}}
+{{format:FeatProgression}}
+{{format:UserBottomFallback}}`
+	case "racialtypes":
+		return `<h1>{{title}}</h1>
+{{UserTopSection}}
+{{format:StatusCallout}}
+{{format:Description}}
+{{format:FactsTable}}
+{{format:RaceNameForms}}
+{{format:UserBottomFallback}}`
+	default:
+		return `<h1>{{title}}</h1>
+{{UserTopSection}}
+{{format:StatusCallout}}
+{{format:Description}}
+{{format:FactsTable}}
+{{format:UserBottomFallback}}`
+	}
+}
+
+func (ctx *wikiContext) renderWikiTemplateString(path, source string, page wikiTemplatePage) (string, error) {
+	state := &wikiTemplateRenderState{
+		path:         path,
+		page:         page,
+		seenSections: map[string]struct{}{},
+		active:       true,
+	}
+	rendered, offset, closed, err := ctx.renderWikiTemplateRange(source, 0, "", state)
+	if err != nil {
+		return "", err
+	}
+	if closed {
+		return "", fmt.Errorf("render wiki template %s for %s: unexpected closing block", path, page.PageID)
+	}
+	if offset != len(source) {
+		return "", fmt.Errorf("render wiki template %s for %s: template parse stopped early", path, page.PageID)
+	}
+	return strings.TrimSpace(rendered), nil
+}
+
+type wikiTemplateRenderState struct {
+	path         string
+	page         wikiTemplatePage
+	row          map[string]any
+	dot          any
+	seenSections map[string]struct{}
+	active       bool
+}
+
+func (ctx *wikiContext) renderWikiTemplateRange(source string, offset int, stopBlock string, state *wikiTemplateRenderState) (string, int, bool, error) {
+	var out strings.Builder
+	for {
+		start := strings.Index(source[offset:], "{{")
+		if start < 0 {
+			if stopBlock != "" {
+				return "", offset, false, fmt.Errorf("render wiki template %s for %s: missing {{/%s}}", state.path, state.page.PageID, stopBlock)
+			}
+			if state.active {
+				out.WriteString(source[offset:])
+			}
+			return out.String(), len(source), false, nil
+		}
+		start += offset
+		end := strings.Index(source[start+2:], "}}")
+		if end < 0 {
+			return "", offset, false, fmt.Errorf("render wiki template %s for %s: unclosed placeholder", state.path, state.page.PageID)
+		}
+		end += start + 2
+		if state.active {
+			out.WriteString(source[offset:start])
+		}
+		token := strings.TrimSpace(source[start+2 : end])
+		if strings.HasPrefix(token, "/") {
+			name := strings.TrimSpace(strings.TrimPrefix(token, "/"))
+			if stopBlock == "" || name != stopBlock {
+				return "", offset, false, fmt.Errorf("render wiki template %s for %s: unexpected {{/%s}}", state.path, state.page.PageID, name)
+			}
+			return out.String(), end + 2, true, nil
+		}
+		if strings.HasPrefix(token, "#each ") {
+			providerName := strings.TrimSpace(strings.TrimPrefix(token, "#each "))
+			body, nextOffset, err := extractWikiTemplateBlock(source, end+2, "each", state.path, state.page.PageID)
+			if err != nil {
+				return "", offset, false, err
+			}
+			if state.active {
+				items, err := ctx.resolveWikiTemplateEachItems(providerName, state)
+				if err != nil {
+					return "", offset, false, fmt.Errorf("render wiki template %s for %s: %w", state.path, state.page.PageID, err)
+				}
+				for _, item := range items {
+					child := *state
+					child.row = item.row
+					child.dot = item.dot
+					rendered, childOffset, closed, err := ctx.renderWikiTemplateRange(body, 0, "", &child)
+					if err != nil {
+						return "", offset, false, err
+					}
+					if closed || childOffset != len(body) {
+						return "", offset, false, fmt.Errorf("render wiki template %s for %s: malformed each body", state.path, state.page.PageID)
+					}
+					out.WriteString(rendered)
+				}
+			}
+			offset = nextOffset
+			continue
+		}
+		if strings.HasPrefix(token, "#if ") {
+			expr := strings.TrimSpace(strings.TrimPrefix(token, "#if "))
+			body, nextOffset, err := extractWikiTemplateBlock(source, end+2, "if", state.path, state.page.PageID)
+			if err != nil {
+				return "", offset, false, err
+			}
+			if state.active {
+				value, err := ctx.evalWikiTemplateExpression(expr, state)
+				if err != nil {
+					return "", offset, false, err
+				}
+				if truthyWikiTableValue(value) {
+					child := *state
+					rendered, childOffset, closed, err := ctx.renderWikiTemplateRange(body, 0, "", &child)
+					if err != nil {
+						return "", offset, false, err
+					}
+					if closed || childOffset != len(body) {
+						return "", offset, false, fmt.Errorf("render wiki template %s for %s: malformed if body", state.path, state.page.PageID)
+					}
+					out.WriteString(rendered)
+				}
+			}
+			offset = nextOffset
+			continue
+		}
+		if strings.HasPrefix(token, "#with ") {
+			expr := strings.TrimSpace(strings.TrimPrefix(token, "#with "))
+			body, nextOffset, err := extractWikiTemplateBlock(source, end+2, "with", state.path, state.page.PageID)
+			if err != nil {
+				return "", offset, false, err
+			}
+			if state.active {
+				value, err := ctx.evalWikiTemplateExpression(expr, state)
+				if err != nil {
+					return "", offset, false, err
+				}
+				if truthyWikiTableValue(value) {
+					child := *state
+					child.dot = value
+					rendered, childOffset, closed, err := ctx.renderWikiTemplateRange(body, 0, "", &child)
+					if err != nil {
+						return "", offset, false, err
+					}
+					if closed || childOffset != len(body) {
+						return "", offset, false, fmt.Errorf("render wiki template %s for %s: malformed with body", state.path, state.page.PageID)
+					}
+					out.WriteString(rendered)
+				}
+			}
+			offset = nextOffset
+			continue
+		}
+		if !state.active {
+			offset = end + 2
+			continue
+		}
+		rendered, sectionID, err := ctx.renderWikiTemplateTokenWithState(token, state)
+		if err != nil {
+			return "", offset, false, err
+		}
+		if sectionID != "" {
+			if _, exists := state.seenSections[sectionID]; exists {
+				return "", offset, false, fmt.Errorf("render wiki template %s for %s: duplicate preserved section %q", state.path, state.page.PageID, sectionID)
+			}
+			state.seenSections[sectionID] = struct{}{}
+		}
+		out.WriteString(rendered)
+		offset = end + 2
+	}
+}
+
+type wikiTemplateEachItem struct {
+	row map[string]any
+	dot any
+}
+
+func (ctx *wikiContext) resolveWikiTemplateEachItems(name string, state *wikiTemplateRenderState) ([]wikiTemplateEachItem, error) {
+	if value, ok := ctx.resolveWikiTemplateScopedValue(name, state); ok {
+		return wikiTemplateEachItemsFromValue(value), nil
+	}
+	rows, err := ctx.wikiDataProviderRows(name, state.page)
+	if err != nil {
+		return nil, err
+	}
+	items := make([]wikiTemplateEachItem, 0, len(rows))
+	for _, row := range rows {
+		items = append(items, wikiTemplateEachItem{row: row, dot: row})
+	}
+	return items, nil
+}
+
+func wikiTemplateEachItemsFromValue(value any) []wikiTemplateEachItem {
+	switch typed := value.(type) {
+	case []map[string]any:
+		items := make([]wikiTemplateEachItem, 0, len(typed))
+		for _, row := range typed {
+			items = append(items, wikiTemplateEachItem{row: row, dot: row})
+		}
+		return items
+	case []any:
+		items := make([]wikiTemplateEachItem, 0, len(typed))
+		for _, value := range typed {
+			if row, ok := value.(map[string]any); ok {
+				items = append(items, wikiTemplateEachItem{row: row, dot: row})
+				continue
+			}
+			items = append(items, wikiTemplateEachItem{dot: value})
+		}
+		return items
+	case []string:
+		items := make([]wikiTemplateEachItem, 0, len(typed))
+		for _, value := range typed {
+			items = append(items, wikiTemplateEachItem{dot: value})
+		}
+		return items
+	default:
+		return nil
+	}
+}
+
+func (ctx *wikiContext) renderWikiTemplateTokenWithState(token string, state *wikiTemplateRenderState) (string, string, error) {
+	if strings.HasPrefix(token, "format:") || strings.HasPrefix(token, "table:") || strings.HasPrefix(token, "field:") || strings.HasPrefix(token, "section:") {
+		rendered, sectionID, err := ctx.renderWikiTemplateToken(state.path, token, state.page)
+		return rendered, sectionID, err
+	}
+	if strings.Contains(token, "|") || strings.Contains(token, "(") || token == "." || strings.HasPrefix(token, "page.") {
+		value, err := ctx.evalWikiTemplateExpression(token, state)
+		if err != nil {
+			return "", "", err
+		}
+		return stringifyWikiTemplateOutput(value), "", nil
+	}
+	if state.row != nil {
+		if _, ok := ctx.resolveWikiTemplateScopedValue(token, state); ok {
+			value, err := ctx.evalWikiTemplateExpression(token, state)
+			if err != nil {
+				return "", "", err
+			}
+			return stringifyWikiTemplateOutput(value), "", nil
+		}
+	}
+	rendered, sectionID, err := ctx.renderWikiTemplateToken(state.path, token, state.page)
+	return rendered, sectionID, err
+}
+
+func (ctx *wikiContext) renderWikiTemplateToken(path, token string, page wikiTemplatePage) (string, string, error) {
+	switch token {
+	case "title":
+		return html.EscapeString(page.Title), "", nil
+	case "page_id":
+		return html.EscapeString(page.PageID), "", nil
+	case "status":
+		return html.EscapeString(page.Status), "", nil
+	}
+	if section, ok := wikiManualSectionForAlias(page.ManualSections, token); ok {
+		return renderWikiManualSection(section), section.ID, nil
+	}
+	if strings.HasPrefix(token, "section:") {
+		sectionName := strings.TrimSpace(strings.TrimPrefix(token, "section:"))
+		if section, ok := wikiManualSectionForAlias(page.ManualSections, sectionName); ok {
+			return renderWikiManualSection(section), section.ID, nil
+		}
+		return "", "", nil
+	}
+	if strings.HasPrefix(token, "format:") {
+		rendered, err := ctx.renderWikiFormatter(strings.TrimSpace(strings.TrimPrefix(token, "format:")), page)
+		return rendered, "", err
+	}
+	if strings.HasPrefix(token, "table:") {
+		rendered, err := ctx.renderConfiguredWikiTable(path, strings.TrimSpace(strings.TrimPrefix(token, "table:")), page)
+		return rendered, "", err
+	}
+	if strings.HasPrefix(token, "field:") {
+		rendered, err := ctx.renderExplicitWikiField(path, strings.TrimSpace(strings.TrimPrefix(token, "field:")), page)
+		return rendered, "", err
+	}
+	rendered, err := ctx.renderRequiredWikiField(path, token, page)
+	return rendered, "", err
+}
+
+func wikiManualSectionForAlias(sections []wikiManualSection, alias string) (wikiManualSection, bool) {
+	for _, section := range sections {
+		if section.Alias == alias {
+			return section, true
+		}
+	}
+	switch alias {
+	case "UserTopSection":
+		for _, section := range sections {
+			if section.ID == "user_top" {
+				return section, true
+			}
+		}
+	case "UserBottomSection":
+		for _, section := range sections {
+			if section.ID == "user_bottom" {
+				return section, true
+			}
+		}
+	}
+	return wikiManualSection{}, false
+}
+
+func renderWikiManualSection(section wikiManualSection) string {
+	return strings.Join([]string{
+		"<!-- sow-topdata-wiki:manual:start id=\"" + html.EscapeString(section.ID) + "\" -->",
+		strings.TrimSpace(section.InitialHTML),
+		"<!-- sow-topdata-wiki:manual:end id=\"" + html.EscapeString(section.ID) + "\" -->",
+	}, "\n")
+}
+
+func (ctx *wikiContext) renderExplicitWikiField(path, spec string, page wikiTemplatePage) (string, error) {
+	fieldSpec, defaultValue, hasDefault := strings.Cut(spec, "|default=")
+	fieldSpec = strings.TrimSpace(fieldSpec)
+	value, ok := ctx.resolveWikiTemplateField(fieldSpec, page)
+	if !ok {
+		if hasDefault {
+			return html.EscapeString(defaultValue), nil
+		}
+		return "", fmt.Errorf("render wiki template %s for %s: missing field %q", path, page.PageID, fieldSpec)
+	}
+	return html.EscapeString(value), nil
+}
+
+func (ctx *wikiContext) renderRequiredWikiField(path, fieldSpec string, page wikiTemplatePage) (string, error) {
+	value, ok := ctx.resolveWikiTemplateField(fieldSpec, page)
+	if !ok {
+		return "", fmt.Errorf("render wiki template %s for %s: missing field %q", path, page.PageID, fieldSpec)
+	}
+	return html.EscapeString(value), nil
+}
+
+func (ctx *wikiContext) resolveWikiTemplateField(fieldSpec string, page wikiTemplatePage) (string, bool) {
+	fieldSpec = strings.TrimSpace(fieldSpec)
+	if strings.HasSuffix(fieldSpec, ".text") {
+		base := strings.TrimSuffix(fieldSpec, ".text")
+		value, ok := lookupField(page.Row, base)
+		if !ok {
+			return "", false
+		}
+		text := ctx.resolveTextValue(value, nil)
+		return text, text != ""
+	}
+	value, ok := lookupField(page.Row, fieldSpec)
+	if !ok || value == nil {
+		return "", false
+	}
+	switch typed := value.(type) {
+	case string:
+		if typed == "" || typed == nullValue {
+			return "", false
+		}
+		return typed, true
+	case int:
+		return fmt.Sprintf("%d", typed), true
+	case float64:
+		return fmt.Sprintf("%d", int(typed)), true
+	default:
+		text := ctx.resolveTextValue(typed, nil)
+		return text, text != ""
+	}
+}