Keep model compilation headless (#49)

## Summary

- always wrap NWN model compilation with `xvfb-run`, even when the caller has `DISPLAY`
- fail closed with an actionable error when `xvfb-run` is unavailable instead of opening the client UI
- update the compiler contract and regression coverage

## Verification

- focused red/green regression tests
- `nix develop -c make check`
- real NWN compile with `DISPLAY=:0` under transient Xvfb; binary MDL output verified

Generated with Claude CodeReviewed-on: #49

Co-authored-by: vickydotbat <vickydotbat@tutamail.com>

.agents/skills/data-driven-design/SKILL.md

@@ -0,0 +1,55 @@
+---
+name: data-driven-design
+description: Use when adding a mode, switch case, config entry, special-cased name, or per-column/per-dataset handler to a tool that processes authored data — or when a tool "knows" a project's layout, dataset names, key prefixes, or format quirks and could not be open sourced as-is.
+---
+
+# Data-Driven Design
+
+The tool implements **general mechanisms**; the data declares **all specific knowledge**. Every time code names a thing that lives in data — a dataset, a column, a format quirk — the tool stops being a tool and becomes a private extension of one project's layout.
+
+**The test:** could this repo be open sourced and build a *different* project's data without code changes? Every hardcoded name, mode, and special case is a "no".
+
+## The two smells
+
+**1. The mode zoo.** A new data shape arrives and you add a named mode plus a switch case:
+
+```yaml
+- {column: AvailableHeadsMale, mode: external_hex_list, hex_width: 3}
+- {column: PreferredAlignments, mode: alignment_set_mask, hex_width: 3}
+- {column: ProficiencyFeats,   mode: id_hex_list,        hex_width: 4}
+```
+
+Three "modes" are one concept — *a set of ids with an engine-side encoding* — wearing three names. Each mode is code the data could have been. The fix is a smaller vocabulary of orthogonal primitives declared where the column is defined:
+
+```json
+"EquipSlots": {"shape": "id_set", "encode": "bitmask", "max": 31}
+```
+
+One parser for `id_set`, one encoder per encoding. Adding the next column is a data edit, zero code. The mode zoo *shrinks*: N special cases collapse into shape × encoding.
+
+**2. Names in code.** `datasetRows("skills")`, `case "skills":`, `TrimPrefix(key, "skills:")` — the tool assumes the project's layout. When data moves (`skills` → `skills/core`), the tool breaks even though the data is self-consistent. Names must flow *from* the data: the spec that references a dataset names it; a key's namespace is whatever precedes its own colon. If code needs a name, some piece of data already knows it — read it from there.
+
+## Deciding
+
+Ask of every constant, case, and config knob: **whose knowledge is this?**
+
+- *The mechanism's* (how to parse JSON, allocate rows, write a table) → code.
+- *The data's* (which columns exist, how the engine encodes them, what things are called) → data, even if code is faster to write today.
+
+Generalize only when it deletes: a primitive that replaces N modes is lazy; a framework "for future shapes" is not. One new shape may take its case *if* the case is spelled as a reusable primitive the next shape can declare.
+
+## Rationalizations
+
+| Excuse | Reality |
+|---|---|
+| "Just follow the existing pattern, it's proven" | The pattern *is* the defect. Each repetition raises the cost of the real fix. |
+| "N cases isn't a crisis yet" | The crisis is per-case: every one couples the tool to one project forever. |
+| "Release is tonight, generalize later" | Later never comes; tonight's mode is tomorrow's baseline. Declaring shape in data is usually the *same* hour of work. |
+| "This name never changes" | Today's rename broke exactly such a name. Data moves; mechanisms shouldn't care. |
+
+## Red flags — stop and re-shape
+
+- A config entry that names both a dataset and a column to select behavior
+- A new value in a `mode`/`kind`/`type` string enum with its own switch arm
+- A literal in code that also appears in the data tree (`"skills"`, `"skills:"`)
+- Per-project constants (row ranges, hex widths, id ceilings) living in Go instead of the dataset that owns them

.gitea/workflows/build-binaries.yml

@@ -1,25 +1,22 @@
-# Cross-platform Crucible binaries (D7 trigger standard):
-#   PR / push main -> cross-build ALL targets to prove they compile (no publish)
-#   tag v*         -> build all targets + SHA256SUMS, upload to the Gitea release
+# Cross-platform Crucible release binaries: a v* tag builds all targets,
+# then uploads them, SHA256SUMS, and the canonical wrappers to its Gitea release.
 # Crucible is pure Go (CGO_ENABLED=0), so cross-compiling is a fast loop.
 name: build-binaries
 
 on:
-  pull_request:
   push:
-    branches: [main]
     tags: ['v*']
-    paths-ignore:
-      - "docs/**"
-      - "README.md"
-      - "AGENTS.md"
-      - "LICENSE"
+
+permissions:
+  code: read
+  releases: write
 
 jobs:
   build-binaries:
     runs-on: nix-docker
+    timeout-minutes: 30
     steps:
-      - uses: actions/checkout@v4
+      - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
         with: { fetch-depth: 0 }
 
       - name: Cross-build all targets
@@ -45,9 +42,8 @@ jobs:
           '
 
       - name: Upload to Gitea release
-        if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags/v')
         env:
-          TOKEN: ${{ secrets.GITHUB_TOKEN }}
+          TOKEN: ${{ secrets.GITEA_TOKEN }}
           TAG: ${{ github.ref_name }}
           REPO: ${{ github.repository }}
           SERVER: ${{ github.server_url }}

.gitea/workflows/ci.yml

@@ -0,0 +1,47 @@
+# Pull-request validation for Crucible. Releases and wrapper synchronization
+# have their own narrow workflows because they need tag/main events.
+name: ci
+
+on:
+  pull_request:
+
+permissions: read-all
+
+jobs:
+  ci:
+    runs-on: nix-docker
+    timeout-minutes: 60
+    steps:
+      - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
+        with:
+          fetch-depth: 0
+
+      - name: vet + test + lint
+        run: |
+          nix develop --command bash -c '
+            set -euo pipefail
+            go vet ./...
+            go test ./...
+            shellcheck scripts/*.sh
+            yamllint .gitea
+          '
+
+      - name: binary smoke (fail-closed contract)
+        run: nix develop --command make smoke
+
+      - name: Cross-build all targets
+        run: |
+          nix develop --command bash -c '
+            set -euo pipefail
+            sha="$(git rev-parse --short=12 HEAD)"
+            ldflags="-s -w -X git.westgate.pw/ShadowsOverWestgate/sow-tools/internal/buildinfo.Version=${sha}"
+            rm -rf dist && mkdir -p dist
+            export CGO_ENABLED=0
+            for target in linux/amd64 linux/arm64 darwin/amd64 darwin/arm64 windows/amd64 windows/arm64; do
+              os="${target%/*}"; arch="${target#*/}"
+              ext=""; [ "$os" = windows ] && ext=".exe"
+              echo "building crucible-${os}-${arch}${ext}"
+              GOOS="$os" GOARCH="$arch" go build -trimpath -ldflags "$ldflags" \
+                -o "dist/crucible-${os}-${arch}${ext}" ./cmd/crucible
+            done
+          '

.gitea/workflows/sync-wrappers.yml

@@ -14,11 +14,14 @@ on:
       - 'wrappers/crucible.sh'
       - 'wrappers/crucible.ps1'
 
+permissions: read-all
+
 jobs:
   sync:
     runs-on: nix-docker
+    timeout-minutes: 30
     steps:
-      - uses: actions/checkout@v4
+      - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
 
       - name: Open sync PRs to consumers
         env:

.gitea/workflows/test.yml

@@ -1,30 +0,0 @@
-# Lint + unit tests for the Crucible suite. PR-first: PRs + push to main (D7).
-name: test
-
-on:
-  push:
-    branches: [main]
-    paths-ignore:
-      - "docs/**"
-      - "README.md"
-      - "AGENTS.md"
-      - "LICENSE"
-  pull_request:
-
-jobs:
-  test:
-    runs-on: nix-docker
-    steps:
-      - uses: actions/checkout@v4
-        with: { fetch-depth: 0 }
-      - name: vet + test + lint
-        run: |
-          nix develop --command bash -c '
-            set -euo pipefail
-            go vet ./...
-            go test ./...
-            shellcheck scripts/*.sh
-            yamllint .gitea
-          '
-      - name: binary smoke (fail-closed contract)
-        run: nix develop --command make smoke

Makefile

@@ -6,6 +6,7 @@ SHELL := bash
 check: vet test
 	shellcheck scripts/*.sh
 	yamllint .gitea
+	bash tests/workflow-contract.sh
 
 vet:
 	go vet ./...

README.md

@@ -78,13 +78,14 @@ This retires the old habit of checking in `nwn-tool` / `sow-toolkit`.
 
 ## CI
 
-PR-first (D7): checks run on pull requests and on push to `main`; the only
-publish event is a `v*` tag (see `runbooks/ci-trigger-standard.md` in sow-docs,
+PR-first (D7): checks run once on pull requests; the only publish event is a
+`v*` tag (see `runbooks/ci-trigger-standard.md` in sow-docs,
 https://git.westgate.pw/ShadowsOverWestgate/sow-docs).
 
-- `test.yml` — vet, test, shellcheck, yamllint, binary smoke (PR + main).
-- `build-binaries.yml` — cross-build all targets (PR + main); on a `v*` tag, upload
-  the binaries, `SHA256SUMS`, and the wrappers to the Gitea release.
+- `ci.yml` — vet, test, shellcheck, yamllint, binary smoke, and cross-build all
+  targets once per pull request.
+- `build-binaries.yml` — on a `v*` tag, cross-build and upload the binaries,
+  `SHA256SUMS`, and the wrappers to the Gitea release.
 - `sync-wrappers.yml` — on a `main` push that touches `wrappers/`, auto-PR the
   canonical wrappers to the consumer repos in `wrappers/consumers.txt`.
   Consumer drift checks run after those PRs merge to `main`, not on the PRs

docs/superpowers/specs/2026-07-12-crucible-assets-builder-design.md

@@ -126,8 +126,9 @@ exactly as the reference does:
   - Beamdog install dirs.
   - `--nwn <install>` overrides (path to the install root or directly to the
     binary).
-- **Headless.** The engine is a GUI binary. If there is no `DISPLAY` and
-  `xvfb-run` is present, wrap the call:
+- **Headless.** The engine is a GUI binary. Require `xvfb-run` and wrap the call
+  regardless of the caller's `DISPLAY` so compilation never opens the client
+  UI; fail before invoking the engine when `xvfb-run` is unavailable:
   `xvfb-run -a --server-args=-screen 0 1024x768x24 nwmain-linux compilemodel <stem>`.
 - **Per model (one at a time — the engine's `development/` and `modelcompiler/`
   folders are flat and single-slot):**

internal/assets/compile.go

@@ -40,13 +40,13 @@ func runCompile(args []string, stdout, stderr io.Writer, getenv func(string) str
 	}
 	binDir := filepath.Dir(nwmain)
 
-	// Headless wrap: no DISPLAY + xvfb-run present -> run under a virtual X.
-	var wrap []string
-	if getenv("DISPLAY") == "" {
-		if xvfb := look("xvfb-run"); xvfb != "" {
-			wrap = []string{xvfb, "-a", "--server-args=-screen 0 1024x768x24"}
-		}
+	// Always use a virtual X so compilation never opens the client UI.
+	xvfb := look("xvfb-run")
+	if xvfb == "" {
+		fmt.Fprintln(stderr, "assets compile: xvfb-run not found — install it to run the NWN model compiler headlessly")
+		return exitTool
 	}
+	wrap := []string{xvfb, "-a", "--server-args=-screen 0 1024x768x24"}
 
 	files, err := walk(dirs, mdlExt, !*nonRecursive)
 	if err != nil {

internal/assets/compile_test.go

@@ -4,6 +4,7 @@ import (
 	"bytes"
 	"os"
 	"path/filepath"
+	"strings"
 	"testing"
 )
 
@@ -23,13 +24,19 @@ func TestCompileDrivesEngineAndReplacesInPlace(t *testing.T) {
 	if err := os.WriteFile(nwmain, []byte("#!/bin/sh\n"), 0o755); err != nil {
 		t.Fatal(err)
 	}
+	xvfbDir := t.TempDir()
+	xvfb := filepath.Join(xvfbDir, "xvfb-run")
+	if err := os.WriteFile(xvfb, []byte("#!/bin/sh\n"), 0o755); err != nil {
+		t.Fatal(err)
+	}
+	t.Setenv("PATH", xvfbDir+string(os.PathListSeparator)+os.Getenv("PATH"))
 
 	getenv := func(k string) string {
 		switch k {
 		case "HOME":
 			return home
 		case "DISPLAY":
-			return ":0" // pretend a display exists so no xvfb wrap is needed
+			return ":0" // a desktop display must not make compilation interactive
 		}
 		return ""
 	}
@@ -38,7 +45,11 @@ func TestCompileDrivesEngineAndReplacesInPlace(t *testing.T) {
 	orig := runner
 	defer func() { runner = orig }()
 	runner = func(dir string, env []string, name string, args ...string) ([]byte, error) {
-		// args: compilemodel <stem>
+		if name != xvfb || len(args) != 5 || args[0] != "-a" ||
+			args[1] != "--server-args=-screen 0 1024x768x24" || args[2] != nwmain ||
+			args[3] != "compilemodel" {
+			t.Fatalf("engine command = %q %q, want xvfb-run wrapping nwmain", name, args)
+		}
 		stem := args[len(args)-1]
 		compiled := filepath.Join(mc, stem+".mdl")
 		return nil, os.WriteFile(compiled, []byte("\x00\x00compiled"), 0o644)
@@ -77,6 +88,11 @@ func TestCompileAbortsOnNameMismatch(t *testing.T) {
 	if err := os.WriteFile(nwmain, []byte("#!/bin/sh\n"), 0o755); err != nil {
 		t.Fatal(err)
 	}
+	xvfbDir := t.TempDir()
+	if err := os.WriteFile(filepath.Join(xvfbDir, "xvfb-run"), []byte("#!/bin/sh\n"), 0o755); err != nil {
+		t.Fatal(err)
+	}
+	t.Setenv("PATH", xvfbDir+string(os.PathListSeparator)+os.Getenv("PATH"))
 	getenv := func(k string) string {
 		if k == "HOME" {
 			return home
@@ -108,3 +124,48 @@ func TestCompileAbortsOnNameMismatch(t *testing.T) {
 		t.Fatal("engine should not run for a name-mismatched model")
 	}
 }
+
+func TestCompileFailsClosedWithoutXvfb(t *testing.T) {
+	home := t.TempDir()
+	userData := filepath.Join(home, ".local", "share", "Neverwinter Nights")
+	for _, d := range []string{filepath.Join(userData, "development"), filepath.Join(userData, "modelcompiler")} {
+		if err := os.MkdirAll(d, 0o755); err != nil {
+			t.Fatal(err)
+		}
+	}
+	nwmain := filepath.Join(t.TempDir(), "nwmain-linux")
+	if err := os.WriteFile(nwmain, []byte("#!/bin/sh\n"), 0o755); err != nil {
+		t.Fatal(err)
+	}
+	t.Setenv("PATH", t.TempDir())
+
+	getenv := func(k string) string {
+		if k == "HOME" {
+			return home
+		}
+		return ""
+	}
+	orig := runner
+	defer func() { runner = orig }()
+	engineCalled := false
+	runner = func(string, []string, string, ...string) ([]byte, error) {
+		engineCalled = true
+		return nil, nil
+	}
+
+	srcDir := t.TempDir()
+	if err := os.WriteFile(filepath.Join(srcDir, "foo.mdl"),
+		[]byte("newmodel foo\nbeginmodelgeom foo\n  node dummy foo\n    parent null\n  endnode\nendmodelgeom foo\ndonemodel foo\n"), 0o644); err != nil {
+		t.Fatal(err)
+	}
+	var stdout, stderr bytes.Buffer
+	if code := runCompile([]string{"--nwn", nwmain, srcDir}, &stdout, &stderr, getenv); code != exitTool {
+		t.Fatalf("compile exit = %d, want %d\n%s", code, exitTool, stderr.String())
+	}
+	if engineCalled {
+		t.Fatal("engine must not run without xvfb-run")
+	}
+	if !strings.Contains(stderr.String(), "xvfb-run") {
+		t.Fatalf("missing actionable xvfb-run error: %s", stderr.String())
+	}
+}

internal/topdata/appearance_migrate.go

@@ -1,96 +0,0 @@
-package topdata
-
-import (
-	"encoding/json"
-	"os"
-	"path/filepath"
-)
-
-func importLegacyAppearance(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
-	_ = legacyTLK
-
-	legacyDir := filepath.Join(referenceBuilderDir, "data", "appearance")
-	if _, err := os.Stat(legacyDir); err != nil {
-		if os.IsNotExist(err) {
-			return 0, nil
-		}
-		return 0, err
-	}
-
-	targetDir := filepath.Join(dataDir, "appearance")
-	targetBasePath := filepath.Join(targetDir, "base.json")
-	targetLockPath := filepath.Join(targetDir, "lock.json")
-	targetModulesDir := filepath.Join(targetDir, "modules")
-	moduleNames := []string{
-		"cotblreaver.json",
-		"crawlingclaw.json",
-		"halfogre.json",
-		"zombieknight.json",
-	}
-	targetModulePaths := make([]string, 0, len(moduleNames))
-	for _, name := range moduleNames {
-		targetModulePaths = append(targetModulePaths, filepath.Join(targetModulesDir, name))
-	}
-	if fileExists(targetBasePath) && fileExists(targetLockPath) {
-		allModulesPresent := true
-		for _, path := range targetModulePaths {
-			if !fileExists(path) {
-				allModulesPresent = false
-				break
-			}
-		}
-		if allModulesPresent {
-			return 0, nil
-		}
-	}
-
-	baseObj, err := loadJSONObject(filepath.Join(legacyDir, "base.json"))
-	if err != nil {
-		return 0, err
-	}
-	baseObj["output"] = "appearance.2da"
-
-	lockObj, err := loadJSONObject(filepath.Join(legacyDir, "lock.json"))
-	if err != nil {
-		return 0, err
-	}
-
-	moduleObjs := make([]map[string]any, 0, len(moduleNames))
-	for _, name := range moduleNames {
-		obj, err := loadJSONObject(filepath.Join(legacyDir, "modules", name))
-		if err != nil {
-			return 0, err
-		}
-		moduleObjs = append(moduleObjs, obj)
-	}
-
-	if err := os.MkdirAll(targetModulesDir, 0o755); err != nil {
-		return 0, err
-	}
-
-	writes := []struct {
-		path string
-		obj  map[string]any
-	}{
-		{path: targetBasePath, obj: baseObj},
-		{path: targetLockPath, obj: lockObj},
-	}
-	for i, path := range targetModulePaths {
-		writes = append(writes, struct {
-			path string
-			obj  map[string]any
-		}{path: path, obj: moduleObjs[i]})
-	}
-
-	for _, write := range writes {
-		raw, err := json.MarshalIndent(write.obj, "", "  ")
-		if err != nil {
-			return 0, err
-		}
-		raw = append(raw, '\n')
-		if err := os.WriteFile(write.path, raw, 0o644); err != nil {
-			return 0, err
-		}
-	}
-	return len(writes), nil
-}

internal/topdata/armor_migrate.go

@@ -1,160 +0,0 @@
-package topdata
-
-import (
-	"encoding/json"
-	"os"
-	"path/filepath"
-	"sort"
-)
-
-func importLegacyArmor(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
-	_ = legacyTLK
-
-	legacyDir := filepath.Join(referenceBuilderDir, "data", "armor")
-	if _, err := os.Stat(legacyDir); err != nil {
-		if os.IsNotExist(err) {
-			return 0, nil
-		}
-		return 0, err
-	}
-
-	targetDir := filepath.Join(dataDir, "armor")
-	targetPath := filepath.Join(targetDir, "armor.json")
-	if _, err := os.Stat(targetPath); err == nil {
-		obj, err := loadJSONObject(targetPath)
-		if err == nil && countLegacyTLKRefsInValue(obj) == 0 {
-			return 0, nil
-		}
-	}
-
-	baseObj, err := loadJSONObject(filepath.Join(legacyDir, "base.json"))
-	if err != nil {
-		return 0, err
-	}
-	rows, err := mergeArmorRows(baseObj, filepath.Join(legacyDir, "modules"))
-	if err != nil {
-		return 0, err
-	}
-
-	if err := os.MkdirAll(targetDir, 0o755); err != nil {
-		return 0, err
-	}
-	if err := removePathIfExists(filepath.Join(targetDir, "base.json")); err != nil {
-		return 0, err
-	}
-	if err := removePathIfExists(filepath.Join(targetDir, "lock.json")); err != nil {
-		return 0, err
-	}
-	if err := removePathIfExists(filepath.Join(targetDir, "modules")); err != nil {
-		return 0, err
-	}
-
-	out := map[string]any{
-		"output":  "armor.2da",
-		"columns": baseObj["columns"],
-		"rows":    rows,
-	}
-	raw, err := json.MarshalIndent(out, "", "  ")
-	if err != nil {
-		return 0, err
-	}
-	raw = append(raw, '\n')
-	if err := os.WriteFile(targetPath, raw, 0o644); err != nil {
-		return 0, err
-	}
-	return 1, nil
-}
-
-func mergeArmorRows(baseObj map[string]any, modulesDir string) ([]any, error) {
-	rawRows, ok := baseObj["rows"].([]any)
-	if !ok {
-		return nil, nil
-	}
-
-	rows := make([]map[string]any, 0, len(rawRows))
-	byID := map[int]map[string]any{}
-	for _, raw := range rawRows {
-		row, ok := deepCopyValue(raw).(map[string]any)
-		if !ok {
-			continue
-		}
-		delete(row, "key")
-		rawID, ok := row["id"]
-		if !ok {
-			continue
-		}
-		id, err := asInt(rawID)
-		if err != nil {
-			return nil, err
-		}
-		row["id"] = id
-		rows = append(rows, row)
-		byID[id] = row
-	}
-
-	modulePaths, err := collectModulePaths(modulesDir)
-	if err != nil {
-		return nil, err
-	}
-	for _, path := range modulePaths {
-		obj, err := loadJSONObject(path)
-		if err != nil {
-			return nil, err
-		}
-		if entries, ok := obj["entries"].(map[string]any); ok {
-			for _, raw := range entries {
-				row, ok := deepCopyValue(raw).(map[string]any)
-				if !ok {
-					continue
-				}
-				delete(row, "key")
-				rawID, ok := row["id"]
-				if !ok {
-					continue
-				}
-				id, err := asInt(rawID)
-				if err != nil {
-					return nil, err
-				}
-				row["id"] = id
-				rows = append(rows, row)
-				byID[id] = row
-			}
-		}
-		if overrides, ok := obj["overrides"].([]any); ok {
-			for _, raw := range overrides {
-				override, ok := raw.(map[string]any)
-				if !ok {
-					continue
-				}
-				rawID, ok := override["id"]
-				if !ok {
-					continue
-				}
-				id, err := asInt(rawID)
-				if err != nil {
-					return nil, err
-				}
-				row := byID[id]
-				if row == nil {
-					continue
-				}
-				for key, value := range override {
-					if key == "id" || key == "key" || key == "_tlk" || isMetadataField(key) {
-						continue
-					}
-					row[key] = deepCopyValue(value)
-				}
-			}
-		}
-	}
-
-	sort.Slice(rows, func(i, j int) bool {
-		return rows[i]["id"].(int) < rows[j]["id"].(int)
-	})
-	out := make([]any, 0, len(rows))
-	for _, row := range rows {
-		out = append(out, row)
-	}
-	return out, nil
-}

internal/topdata/base_dialog_migrate.go

@@ -1,41 +0,0 @@
-package topdata
-
-import (
-	"encoding/json"
-	"os"
-	"path/filepath"
-)
-
-func importLegacyBaseDialog(referenceBuilderDir, sourceDir string) (int, error) {
-	legacyPath := filepath.Join(referenceBuilderDir, "tlk", "base.json")
-	if _, err := os.Stat(legacyPath); err != nil {
-		if os.IsNotExist(err) {
-			return 0, nil
-		}
-		return 0, err
-	}
-
-	targetPath := filepath.Join(sourceDir, "base_dialog.json")
-	if fileExists(targetPath) {
-		current, err := loadJSONObject(targetPath)
-		if err == nil {
-			if entries, ok := current["entries"].(map[string]any); ok && len(entries) > 0 {
-				return 0, nil
-			}
-		}
-	}
-
-	obj, err := loadJSONObject(legacyPath)
-	if err != nil {
-		return 0, err
-	}
-	raw, err := json.MarshalIndent(obj, "", "  ")
-	if err != nil {
-		return 0, err
-	}
-	raw = append(raw, '\n')
-	if err := os.WriteFile(targetPath, raw, 0o644); err != nil {
-		return 0, err
-	}
-	return 1, nil
-}

internal/topdata/baseitems_migrate.go

@@ -1,105 +0,0 @@
-package topdata
-
-import (
-	"encoding/json"
-	"os"
-	"path/filepath"
-)
-
-func importLegacyBaseitems(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
-	_ = legacyTLK
-
-	legacyDir := filepath.Join(referenceBuilderDir, "data", "baseitems")
-	if _, err := os.Stat(legacyDir); err != nil {
-		if os.IsNotExist(err) {
-			return 0, nil
-		}
-		return 0, err
-	}
-
-	targetDir := filepath.Join(dataDir, "baseitems")
-	targetBasePath := filepath.Join(targetDir, "base.json")
-	targetLockPath := filepath.Join(targetDir, "lock.json")
-	targetModulesDir := filepath.Join(targetDir, "modules")
-	moduleNames := []string{
-		"blunderbuss.json",
-		"coin.json",
-		"estoc.json",
-		"heavymace.json",
-		"maulandfalchion.json",
-		"ovr_baseitems.json",
-		"ovr_cloak255.json",
-		"ovr_helmet255.json",
-		"shortspear.json",
-	}
-	targetModulePaths := make([]string, 0, len(moduleNames))
-	for _, name := range moduleNames {
-		targetModulePaths = append(targetModulePaths, filepath.Join(targetModulesDir, name))
-	}
-	if fileExists(targetBasePath) && fileExists(targetLockPath) {
-		allModulesPresent := true
-		for _, path := range targetModulePaths {
-			if !fileExists(path) {
-				allModulesPresent = false
-				break
-			}
-		}
-		if allModulesPresent {
-			return 0, nil
-		}
-	}
-
-	baseObj, err := loadJSONObject(filepath.Join(legacyDir, "base.json"))
-	if err != nil {
-		return 0, err
-	}
-	baseObj["output"] = "baseitems.2da"
-	canonicalizeWikiMetadataDocument(baseObj)
-
-	lockObj, err := loadJSONObject(filepath.Join(legacyDir, "lock.json"))
-	if err != nil {
-		return 0, err
-	}
-
-	moduleObjs := make([]map[string]any, 0, len(moduleNames))
-	for _, name := range moduleNames {
-		obj, err := loadJSONObject(filepath.Join(legacyDir, "modules", name))
-		if err != nil {
-			return 0, err
-		}
-		canonicalizeWikiMetadataDocument(obj)
-		moduleObjs = append(moduleObjs, obj)
-	}
-
-	if err := os.MkdirAll(targetModulesDir, 0o755); err != nil {
-		return 0, err
-	}
-
-	writes := []struct {
-		path string
-		obj  map[string]any
-	}{
-		{path: targetBasePath, obj: baseObj},
-		{path: targetLockPath, obj: lockObj},
-	}
-	for index, path := range targetModulePaths {
-		writes = append(writes, struct {
-			path string
-			obj  map[string]any
-		}{
-			path: path,
-			obj:  moduleObjs[index],
-		})
-	}
-	for _, write := range writes {
-		raw, err := json.MarshalIndent(write.obj, "", "  ")
-		if err != nil {
-			return 0, err
-		}
-		raw = append(raw, '\n')
-		if err := os.WriteFile(write.path, raw, 0o644); err != nil {
-			return 0, err
-		}
-	}
-	return len(writes), nil
-}

internal/topdata/classes_migrate.go

@@ -1,226 +0,0 @@
-package topdata
-
-import (
-	"encoding/json"
-	"os"
-	"path/filepath"
-	"slices"
-	"strings"
-)
-
-var legacyClassesPlainFamilies = []string{"feats", "skills", "savthr", "bfeat", "pres"}
-
-func importLegacyClasses(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
-	legacyRoot := filepath.Join(referenceBuilderDir, "data", "classes")
-	if _, err := os.Stat(legacyRoot); err != nil {
-		if os.IsNotExist(err) {
-			return 0, nil
-		}
-		return 0, err
-	}
-
-	targetRoot := filepath.Join(dataDir, "classes")
-	if canonicalClassesPresent(targetRoot) {
-		return 0, nil
-	}
-
-	coreCollected, plainCollected, err := collectLegacyClassesDatasets(legacyRoot)
-	if err != nil {
-		return 0, err
-	}
-
-	tableKeyByOutput := map[string]string{}
-	for _, dataset := range plainCollected {
-		if dataset.TableKey == "" {
-			continue
-		}
-		registerLegacyClassesTableKey(tableKeyByOutput, dataset.TableKey, dataset.Dataset.OutputName)
-	}
-
-	coreRows := make([]map[string]any, 0, len(coreCollected.Rows))
-	for _, rawRow := range coreCollected.Rows {
-		row, ok := deepCopyValue(rawRow).(map[string]any)
-		if !ok {
-			continue
-		}
-		if legacyTLK != nil {
-			if _, _, err := inlineLegacyTLKValue(row, legacyTLK); err != nil {
-				return 0, err
-			}
-		}
-		rewriteLegacyClassesCoreRow(row, tableKeyByOutput)
-		coreRows = append(coreRows, row)
-	}
-
-	if err := removePathIfExists(targetRoot); err != nil {
-		return 0, err
-	}
-	if err := os.MkdirAll(filepath.Join(targetRoot, "core"), 0o755); err != nil {
-		return 0, err
-	}
-
-	writes := []struct {
-		path string
-		obj  map[string]any
-	}{
-		{
-			path: filepath.Join(targetRoot, "core", "base.json"),
-			obj: map[string]any{
-				"output":  coreCollected.Dataset.OutputName,
-				"columns": stringSliceToAny(coreCollected.Columns),
-				"rows":    rowsToAny(coreRows),
-			},
-		},
-		{
-			path: filepath.Join(targetRoot, "core", "lock.json"),
-			obj:  anyMapInt(coreCollected.LockData),
-		},
-	}
-
-	for _, collected := range plainCollected {
-		obj := map[string]any{
-			"output":  collected.Dataset.OutputName,
-			"columns": stringSliceToAny(collected.Columns),
-			"rows":    rowsToAny(collected.Rows),
-		}
-		if strings.TrimSpace(collected.TableKey) != "" {
-			obj["key"] = collected.TableKey
-		}
-		writes = append(writes, struct {
-			path string
-			obj  map[string]any
-		}{
-			path: filepath.Join(dataDir, collected.Dataset.Name+".json"),
-			obj:  obj,
-		})
-	}
-
-	for _, write := range writes {
-		if err := os.MkdirAll(filepath.Dir(write.path), 0o755); err != nil {
-			return 0, err
-		}
-		raw, err := json.MarshalIndent(write.obj, "", "  ")
-		if err != nil {
-			return 0, err
-		}
-		raw = append(raw, '\n')
-		if err := os.WriteFile(write.path, raw, 0o644); err != nil {
-			return 0, err
-		}
-	}
-
-	return len(writes), nil
-}
-
-func canonicalClassesPresent(targetRoot string) bool {
-	if !fileExists(filepath.Join(targetRoot, "core", "base.json")) || !fileExists(filepath.Join(targetRoot, "core", "lock.json")) {
-		return false
-	}
-	for _, family := range legacyClassesPlainFamilies {
-		ok, err := hasJSONFiles(filepath.Join(targetRoot, family))
-		if err != nil || !ok {
-			return false
-		}
-	}
-	return true
-}
-
-func collectLegacyClassesDatasets(legacyRoot string) (nativeCollectedDataset, []nativeCollectedDataset, error) {
-	coreCollected, err := collectBaseDataset(nativeDataset{
-		Name:       "classes/core",
-		BasePath:   filepath.Join(legacyRoot, "core", "base.json"),
-		LockPath:   filepath.Join(legacyRoot, "core", "lock.json"),
-		ModulesDir: filepath.Join(legacyRoot, "core", "modules"),
-		OutputName: "classes.2da",
-		Spec:       specForDataset("classes"),
-	})
-	if err != nil {
-		return nativeCollectedDataset{}, nil, err
-	}
-	coreCollected.Dataset.OutputName = "classes.2da"
-
-	plainCollected := make([]nativeCollectedDataset, 0)
-	for _, family := range legacyClassesPlainFamilies {
-		familyDir := filepath.Join(legacyRoot, family)
-		entries, err := os.ReadDir(familyDir)
-		if err != nil {
-			if os.IsNotExist(err) {
-				continue
-			}
-			return nativeCollectedDataset{}, nil, err
-		}
-		for _, entry := range entries {
-			if entry.IsDir() || filepath.Ext(entry.Name()) != ".json" || strings.HasPrefix(entry.Name(), ".") || entry.Name() == "lock.json" {
-				continue
-			}
-			filePath := filepath.Join(familyDir, entry.Name())
-			tableData, err := loadJSONObject(filePath)
-			if err != nil {
-				return nativeCollectedDataset{}, nil, err
-			}
-			outputName, _ := tableData["output"].(string)
-			if strings.TrimSpace(outputName) == "" {
-				outputName = strings.TrimSuffix(entry.Name(), filepath.Ext(entry.Name())) + ".2da"
-			}
-			collected, err := collectPlainDataset(nativeDataset{
-				Name:       filepath.ToSlash(filepath.Join("classes", family, strings.TrimSuffix(entry.Name(), filepath.Ext(entry.Name())))),
-				BasePath:   filePath,
-				LockPath:   filepath.Join(familyDir, "lock.json"),
-				OutputName: outputName,
-				Spec:       specForDataset(filepath.ToSlash(filepath.Join("classes", family, strings.TrimSuffix(entry.Name(), filepath.Ext(entry.Name()))))),
-			})
-			if err != nil {
-				return nativeCollectedDataset{}, nil, err
-			}
-			plainCollected = append(plainCollected, collected)
-		}
-	}
-
-	slices.SortFunc(plainCollected, func(a, b nativeCollectedDataset) int {
-		return strings.Compare(a.Dataset.Name, b.Dataset.Name)
-	})
-	return coreCollected, plainCollected, nil
-}
-
-func registerLegacyClassesTableKey(tableKeyByOutput map[string]string, tableKey, outputName string) {
-	trimmedOutput := strings.TrimSpace(outputName)
-	if trimmedOutput != "" {
-		tableKeyByOutput[trimmedOutput] = tableKey
-	}
-	trimmedStem := strings.TrimSpace(outputStem(outputName))
-	if trimmedStem != "" {
-		tableKeyByOutput[trimmedStem] = tableKey
-	}
-}
-
-func rewriteLegacyClassesCoreRow(row map[string]any, tableKeyByOutput map[string]string) {
-	for _, field := range []string{"FeatsTable", "SavingThrowTable", "SkillsTable", "BonusFeatsTable", "PreReqTable"} {
-		tableKey := legacyClassesTableKeyForValue(row[field], tableKeyByOutput)
-		if tableKey == "" {
-			continue
-		}
-		row[field] = map[string]any{"table": tableKey}
-	}
-}
-
-func legacyClassesTableKeyForValue(value any, tableKeyByOutput map[string]string) string {
-	switch typed := value.(type) {
-	case map[string]any:
-		tableKey, _ := typed["table"].(string)
-		return strings.TrimSpace(tableKey)
-	case string:
-		trimmed := strings.TrimSpace(typed)
-		if trimmed == "" || trimmed == nullValue {
-			return ""
-		}
-		if trimmed != strings.ToLower(trimmed) {
-			return ""
-		}
-		if tableKey, ok := tableKeyByOutput[trimmed]; ok {
-			return tableKey
-		}
-		return ""
-	default:
-		return ""
-	}
-}

internal/topdata/cloakmodel_migrate.go

@@ -1,6 +0,0 @@
-package topdata
-
-func importLegacyCloakmodel(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
-	_ = legacyTLK
-	return importLegacyDatasetMirror(referenceBuilderDir, dataDir, "cloakmodel", "cloakmodel.2da", nil)
-}

internal/topdata/creaturespeed_migrate.go

@@ -1,145 +0,0 @@
-package topdata
-
-import (
-	"encoding/json"
-	"os"
-	"path/filepath"
-)
-
-func importLegacyCreaturespeed(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
-	_ = legacyTLK
-
-	legacyDir := filepath.Join(referenceBuilderDir, "data", "creaturespeed")
-	if _, err := os.Stat(legacyDir); err != nil {
-		if os.IsNotExist(err) {
-			return 0, nil
-		}
-		return 0, err
-	}
-
-	targetDir := filepath.Join(dataDir, "creaturespeed")
-	targetPath := filepath.Join(targetDir, "creaturespeed.json")
-	if _, err := os.Stat(targetPath); err == nil {
-		obj, err := loadJSONObject(targetPath)
-		if err == nil && countLegacyTLKRefsInValue(obj) == 0 {
-			return 0, nil
-		}
-	}
-
-	baseObj, err := loadJSONObject(filepath.Join(legacyDir, "base.json"))
-	if err != nil {
-		return 0, err
-	}
-	rows, err := mergeCreaturespeedRows(baseObj, filepath.Join(legacyDir, "modules"))
-	if err != nil {
-		return 0, err
-	}
-	if err := os.MkdirAll(targetDir, 0o755); err != nil {
-		return 0, err
-	}
-
-	if err := removePathIfExists(filepath.Join(targetDir, "base.json")); err != nil {
-		return 0, err
-	}
-	if err := removePathIfExists(filepath.Join(targetDir, "lock.json")); err != nil {
-		return 0, err
-	}
-	if err := removePathIfExists(filepath.Join(targetDir, "modules")); err != nil {
-		return 0, err
-	}
-
-	out := map[string]any{
-		"output":  "creaturespeed.2da",
-		"columns": baseObj["columns"],
-		"rows":    rows,
-	}
-
-	raw, err := json.MarshalIndent(out, "", "  ")
-	if err != nil {
-		return 0, err
-	}
-	raw = append(raw, '\n')
-	if err := os.WriteFile(targetPath, raw, 0o644); err != nil {
-		return 0, err
-	}
-	return 1, nil
-}
-
-func mergeCreaturespeedRows(baseObj map[string]any, modulesDir string) ([]any, error) {
-	rawRows, ok := baseObj["rows"].([]any)
-	if !ok {
-		return nil, nil
-	}
-	rows := make([]map[string]any, 0, len(rawRows))
-	byID := map[int]map[string]any{}
-	for _, raw := range rawRows {
-		row, ok := deepCopyValue(raw).(map[string]any)
-		if !ok {
-			continue
-		}
-		delete(row, "key")
-		rows = append(rows, row)
-		if rawID, ok := row["id"]; ok {
-			id, err := asInt(rawID)
-			if err != nil {
-				return nil, err
-			}
-			byID[id] = row
-		}
-	}
-
-	modulePaths, err := collectModulePaths(modulesDir)
-	if err != nil {
-		return nil, err
-	}
-	for _, path := range modulePaths {
-		obj, err := loadJSONObject(path)
-		if err != nil {
-			return nil, err
-		}
-		overrideList, ok := obj["overrides"].([]any)
-		if !ok {
-			continue
-		}
-		for _, raw := range overrideList {
-			override, ok := raw.(map[string]any)
-			if !ok {
-				continue
-			}
-			rawID, ok := override["id"]
-			if !ok {
-				continue
-			}
-			id, err := asInt(rawID)
-			if err != nil {
-				return nil, err
-			}
-			row := byID[id]
-			if row == nil {
-				continue
-			}
-			for key, value := range override {
-				if key == "id" || key == "key" || key == "_tlk" {
-					continue
-				}
-				row[key] = deepCopyValue(value)
-			}
-		}
-	}
-
-	out := make([]any, 0, len(rows))
-	for _, row := range rows {
-		out = append(out, row)
-	}
-	return out, nil
-}
-
-func removePathIfExists(path string) error {
-	if _, err := os.Stat(path); err != nil {
-		if os.IsNotExist(err) {
-			return nil
-		}
-		return err
-	}
-	return os.RemoveAll(path)
-}

internal/topdata/damagetypes_migrate.go

@@ -1,288 +0,0 @@
-package topdata
-
-import (
-	"encoding/json"
-	"fmt"
-	"os"
-	"path/filepath"
-	"sort"
-	"strings"
-)
-
-func importLegacyDamagetypes(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
-	_ = legacyTLK
-
-	legacyRoot := filepath.Join(referenceBuilderDir, "data", "damagetypes")
-	if _, err := os.Stat(legacyRoot); err != nil {
-		if os.IsNotExist(err) {
-			return 0, nil
-		}
-		return 0, err
-	}
-
-	targetDir := filepath.Join(dataDir, "damagetypes", "registry")
-	targetPath := filepath.Join(targetDir, "types.json")
-	if _, err := os.Stat(targetPath); err == nil {
-		obj, err := loadJSONObject(targetPath)
-		if err == nil && countLegacyTLKRefsInValue(obj) == 0 {
-			return 0, nil
-		}
-	}
-
-	coreBase, err := loadJSONObject(filepath.Join(legacyRoot, "core", "base.json"))
-	if err != nil {
-		return 0, err
-	}
-	groupBase, err := loadJSONObject(filepath.Join(legacyRoot, "groups", "base.json"))
-	if err != nil {
-		return 0, err
-	}
-	hitvisualBase, err := loadJSONObject(filepath.Join(legacyRoot, "hitvisual", "base.json"))
-	if err != nil {
-		return 0, err
-	}
-	coreModule, err := loadJSONObject(filepath.Join(legacyRoot, "core", "modules", "add_eos.json"))
-	if err != nil {
-		return 0, err
-	}
-	coreLock, err := loadLockfile(filepath.Join(legacyRoot, "core", "lock.json"))
-	if err != nil {
-		return 0, err
-	}
-	groupModule, err := loadJSONObject(filepath.Join(legacyRoot, "groups", "modules", "add_eos.json"))
-	if err != nil {
-		return 0, err
-	}
-	groupLock, err := loadLockfile(filepath.Join(legacyRoot, "groups", "lock.json"))
-	if err != nil {
-		return 0, err
-	}
-	hitvisualModule, err := loadJSONObject(filepath.Join(legacyRoot, "hitvisual", "modules", "add_eos.json"))
-	if err != nil {
-		return 0, err
-	}
-	hitvisualLock, err := loadLockfile(filepath.Join(legacyRoot, "hitvisual", "lock.json"))
-	if err != nil {
-		return 0, err
-	}
-
-	rows, lockData, err := mergeLegacyDamagetypes(coreBase, groupBase, hitvisualBase, coreModule, coreLock, groupModule, groupLock, hitvisualModule, hitvisualLock)
-	if err != nil {
-		return 0, err
-	}
-
-	if err := os.MkdirAll(targetDir, 0o755); err != nil {
-		return 0, err
-	}
-	if err := removePathIfExists(filepath.Join(dataDir, "damagetypes", "core")); err != nil {
-		return 0, err
-	}
-	if err := removePathIfExists(filepath.Join(dataDir, "damagetypes", "groups")); err != nil {
-		return 0, err
-	}
-	if err := removePathIfExists(filepath.Join(dataDir, "damagetypes", "hitvisual")); err != nil {
-		return 0, err
-	}
-
-	typesOut := map[string]any{"rows": rows}
-	raw, err := json.MarshalIndent(typesOut, "", "  ")
-	if err != nil {
-		return 0, err
-	}
-	raw = append(raw, '\n')
-	if err := os.WriteFile(targetPath, raw, 0o644); err != nil {
-		return 0, err
-	}
-	if err := saveLockfile(filepath.Join(targetDir, damagetypesRegistryLock), lockData); err != nil {
-		return 0, err
-	}
-	return 2, nil
-}
-
-func mergeLegacyDamagetypes(coreBase, groupBase, hitvisualBase, coreModule map[string]any, coreLock map[string]int, groupModule map[string]any, groupLock map[string]int, hitvisualModule map[string]any, hitvisualLock map[string]int) ([]map[string]any, map[string]int, error) {
-	coreRows, err := coerceRowMapSlice(coreBase["rows"])
-	if err != nil {
-		return nil, nil, err
-	}
-	groupRows, err := coerceRowMapSlice(groupBase["rows"])
-	if err != nil {
-		return nil, nil, err
-	}
-	hitRows, err := coerceRowMapSlice(hitvisualBase["rows"])
-	if err != nil {
-		return nil, nil, err
-	}
-	groupByID := map[int]map[string]any{}
-	for _, row := range groupRows {
-		id, err := asInt(row["id"])
-		if err != nil {
-			return nil, nil, err
-		}
-		groupByID[id] = row
-	}
-	hitByID := map[int]map[string]any{}
-	for _, row := range hitRows {
-		id, err := asInt(row["id"])
-		if err != nil {
-			return nil, nil, err
-		}
-		hitByID[id] = row
-	}
-
-	outRows := make([]map[string]any, 0, len(coreRows))
-	lockData := map[string]int{}
-	for _, core := range coreRows {
-		id, err := asInt(core["id"])
-		if err != nil {
-			return nil, nil, err
-		}
-		groupID, err := asInt(core["DamageTypeGroup"])
-		if err != nil {
-			return nil, nil, err
-		}
-		group := groupByID[groupID]
-		if group == nil {
-			return nil, nil, fmt.Errorf("core id %d references missing group %d", id, groupID)
-		}
-		hit := hitByID[id]
-		if hit == nil {
-			return nil, nil, fmt.Errorf("core id %d is missing hitvisual row", id)
-		}
-		key := "damagetype:" + normalizeDamagetypeKey(core["Label"])
-		lockData[key] = id
-		outRows = append(outRows, map[string]any{
-			"key":                      key,
-			"id":                       id,
-			"label":                    deepCopyValue(core["Label"]),
-			"charsheet_strref":         deepCopyValue(core["CharsheetStrref"]),
-			"damage_type_group":        strconvString(groupID),
-			"damage_ranged_projectile": deepCopyValue(core["DamageRangedProjectile"]),
-			"group_label":              deepCopyValue(group["Label"]),
-			"feedback_strref":          deepCopyValue(group["FeedbackStrref"]),
-			"color_r":                  deepCopyValue(group["ColorR"]),
-			"color_g":                  deepCopyValue(group["ColorG"]),
-			"color_b":                  deepCopyValue(group["ColorB"]),
-			"visual_effect_id":         deepCopyValue(hit["VisualEffectID"]),
-			"ranged_effect_id":         deepCopyValue(hit["RangedEffectID"]),
-		})
-	}
-
-	coreEntries, err := coerceEntriesMap(coreModule["entries"])
-	if err != nil {
-		return nil, nil, err
-	}
-	groupEntries, err := coerceEntriesMap(groupModule["entries"])
-	if err != nil {
-		return nil, nil, err
-	}
-	hitEntries, err := coerceEntriesMap(hitvisualModule["entries"])
-	if err != nil {
-		return nil, nil, err
-	}
-
-	keys := make([]string, 0, len(coreEntries))
-	for key := range coreEntries {
-		keys = append(keys, key)
-	}
-	sort.Slice(keys, func(i, j int) bool {
-		a := strings.TrimPrefix(keys[i], "damagetypes:")
-		b := strings.TrimPrefix(keys[j], "damagetypes:")
-		return a < b
-	})
-	for _, coreKey := range keys {
-		core := coreEntries[coreKey]
-		suffix := strings.TrimPrefix(coreKey, "damagetypes:")
-		group := groupEntries["damagetypegroups:"+suffix]
-		if group == nil {
-			return nil, nil, fmt.Errorf("%s: missing matching group entry", coreKey)
-		}
-		hit := hitEntries["damagehitvisual:"+suffix]
-		if hit == nil {
-			return nil, nil, fmt.Errorf("%s: missing matching hitvisual entry", coreKey)
-		}
-		id, ok := coreLock[coreKey]
-		if !ok {
-			return nil, nil, fmt.Errorf("%s: missing core lock id", coreKey)
-		}
-		groupKey := "damagetypegroups:" + suffix
-		groupID, ok := groupLock[groupKey]
-		if !ok {
-			return nil, nil, fmt.Errorf("%s: missing group lock id", groupKey)
-		}
-		hitKey := "damagehitvisual:" + suffix
-		hitID, ok := hitvisualLock[hitKey]
-		if !ok {
-			return nil, nil, fmt.Errorf("%s: missing hitvisual lock id", hitKey)
-		}
-		if hitID != id {
-			return nil, nil, fmt.Errorf("%s: core id %d does not match hitvisual id %d", suffix, id, hitID)
-		}
-		key := "damagetype:" + suffix
-		lockData[key] = id
-		outRows = append(outRows, map[string]any{
-			"key":                      key,
-			"id":                       id,
-			"label":                    deepCopyValue(core["Label"]),
-			"charsheet_strref":         deepCopyValue(core["CharsheetStrref"]),
-			"damage_type_group":        strconvString(groupID),
-			"damage_ranged_projectile": deepCopyValue(core["DamageRangedProjectile"]),
-			"group_label":              deepCopyValue(group["Label"]),
-			"feedback_strref":          deepCopyValue(group["FeedbackStrref"]),
-			"color_r":                  deepCopyValue(group["ColorR"]),
-			"color_g":                  deepCopyValue(group["ColorG"]),
-			"color_b":                  deepCopyValue(group["ColorB"]),
-			"visual_effect_id":         deepCopyValue(hit["VisualEffectID"]),
-			"ranged_effect_id":         deepCopyValue(hit["RangedEffectID"]),
-		})
-	}
-
-	sort.Slice(outRows, func(i, j int) bool {
-		return outRows[i]["id"].(int) < outRows[j]["id"].(int)
-	})
-	return outRows, lockData, nil
-}
-
-func coerceRowMapSlice(value any) ([]map[string]any, error) {
-	rawRows, ok := value.([]any)
-	if !ok {
-		return nil, fmt.Errorf("rows must be an array")
-	}
-	rows := make([]map[string]any, 0, len(rawRows))
-	for _, raw := range rawRows {
-		row, ok := raw.(map[string]any)
-		if !ok {
-			return nil, fmt.Errorf("row must be an object")
-		}
-		rows = append(rows, row)
-	}
-	return rows, nil
-}
-
-func coerceEntriesMap(value any) (map[string]map[string]any, error) {
-	rawEntries, ok := value.(map[string]any)
-	if !ok {
-		return nil, fmt.Errorf("entries must be an object")
-	}
-	entries := make(map[string]map[string]any, len(rawEntries))
-	for key, raw := range rawEntries {
-		row, ok := raw.(map[string]any)
-		if !ok {
-			return nil, fmt.Errorf("entry %s must be an object", key)
-		}
-		entries[key] = row
-	}
-	return entries, nil
-}
-
-func normalizeDamagetypeKey(value any) string {
-	text := strings.TrimSpace(strings.ToLower(format2DAValue(value)))
-	text = strings.TrimSuffix(text, "damage")
-	text = strings.ReplaceAll(text, " ", "")
-	text = strings.ReplaceAll(text, "_", "")
-	text = strings.ReplaceAll(text, "-", "")
-	return text
-}
-
-func strconvString(v int) string {
-	return fmt.Sprintf("%d", v)
-}

internal/topdata/doortypes_migrate.go

@@ -1,99 +0,0 @@
-package topdata
-
-import (
-	"encoding/json"
-	"os"
-	"path/filepath"
-)
-
-func importLegacyDoortypes(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
-	_ = legacyTLK
-
-	legacyDir := filepath.Join(referenceBuilderDir, "data", "doortypes")
-	if _, err := os.Stat(legacyDir); err != nil {
-		if os.IsNotExist(err) {
-			return 0, nil
-		}
-		return 0, err
-	}
-
-	targetDir := filepath.Join(dataDir, "doortypes")
-	targetBasePath := filepath.Join(targetDir, "base.json")
-	targetLockPath := filepath.Join(targetDir, "lock.json")
-	targetModulesDir := filepath.Join(targetDir, "modules")
-	moduleNames := []string{
-		"add_pld01.json",
-		"add_sic11.json",
-		"add_tapr.json",
-		"add_tdm01.json",
-		"add_tdx01.json",
-		"add_tei01.json",
-		"add_tfm01.json",
-	}
-	targetModulePaths := make([]string, 0, len(moduleNames))
-	for _, name := range moduleNames {
-		targetModulePaths = append(targetModulePaths, filepath.Join(targetModulesDir, name))
-	}
-	if fileExists(targetBasePath) && fileExists(targetLockPath) {
-		allModulesPresent := true
-		for _, path := range targetModulePaths {
-			if !fileExists(path) {
-				allModulesPresent = false
-				break
-			}
-		}
-		if allModulesPresent {
-			return 0, nil
-		}
-	}
-
-	baseObj, err := loadJSONObject(filepath.Join(legacyDir, "base.json"))
-	if err != nil {
-		return 0, err
-	}
-	baseObj["output"] = "doortypes.2da"
-
-	lockObj, err := loadJSONObject(filepath.Join(legacyDir, "lock.json"))
-	if err != nil {
-		return 0, err
-	}
-
-	moduleObjs := make([]map[string]any, 0, len(moduleNames))
-	for _, name := range moduleNames {
-		obj, err := loadJSONObject(filepath.Join(legacyDir, "modules", name))
-		if err != nil {
-			return 0, err
-		}
-		moduleObjs = append(moduleObjs, obj)
-	}
-
-	if err := os.MkdirAll(targetModulesDir, 0o755); err != nil {
-		return 0, err
-	}
-
-	writes := []struct {
-		path string
-		obj  map[string]any
-	}{
-		{path: targetBasePath, obj: baseObj},
-		{path: targetLockPath, obj: lockObj},
-	}
-	for i, path := range targetModulePaths {
-		writes = append(writes, struct {
-			path string
-			obj  map[string]any
-		}{path: path, obj: moduleObjs[i]})
-	}
-
-	for _, write := range writes {
-		raw, err := json.MarshalIndent(write.obj, "", "  ")
-		if err != nil {
-			return 0, err
-		}
-		raw = append(raw, '\n')
-		if err := os.WriteFile(write.path, raw, 0o644); err != nil {
-			return 0, err
-		}
-	}
-	return len(writes), nil
-}

internal/topdata/feat_migrate.go

@@ -1,64 +0,0 @@
-package topdata
-
-import (
-	"encoding/json"
-	"os"
-	"path/filepath"
-)
-
-func importLegacyFeat(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
-	_ = legacyTLK
-
-	legacyDir := filepath.Join(referenceBuilderDir, "data", "feat")
-	if _, err := os.Stat(legacyDir); err != nil {
-		if os.IsNotExist(err) {
-			return 0, nil
-		}
-		return 0, err
-	}
-	legacyBasePath := filepath.Join(legacyDir, "base.json")
-	if _, err := os.Stat(legacyBasePath); err != nil {
-		if os.IsNotExist(err) {
-			return 0, nil
-		}
-		return 0, err
-	}
-
-	targetDir := filepath.Join(dataDir, "feat")
-	targetBasePath := filepath.Join(targetDir, "base.json")
-	var targetObj map[string]any
-	if _, err := os.Stat(targetBasePath); err == nil {
-		targetObj, err = loadJSONObject(targetBasePath)
-		if err != nil {
-			return 0, err
-		}
-	} else if !os.IsNotExist(err) {
-		return 0, err
-	}
-	if targetObj != nil {
-		if rows, ok := targetObj["rows"].([]any); ok && len(rows) > 0 {
-			return 0, nil
-		}
-	}
-
-	baseObj, err := loadJSONObject(legacyBasePath)
-	if err != nil {
-		return 0, err
-	}
-	baseObj["output"] = "feat.2da"
-	baseObj["compare_reference"] = false
-	canonicalizeWikiMetadataDocument(baseObj)
-
-	if err := os.MkdirAll(targetDir, 0o755); err != nil {
-		return 0, err
-	}
-	raw, err := json.MarshalIndent(baseObj, "", "  ")
-	if err != nil {
-		return 0, err
-	}
-	raw = append(raw, '\n')
-	if err := os.WriteFile(targetBasePath, raw, 0o644); err != nil {
-		return 0, err
-	}
-	return 1, nil
-}

internal/topdata/genericdoors_migrate.go

@@ -1,6 +0,0 @@
-package topdata
-
-func importLegacyGenericdoors(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
-	_ = legacyTLK
-	return importLegacyDatasetMirror(referenceBuilderDir, dataDir, "genericdoors", "genericdoors.2da", nil)
-}

internal/topdata/itemprops_registry.go

@@ -601,397 +601,6 @@ func deepCopyValue(value any) any {
 	return out
 }
 
-func importLegacyItempropsRegistry(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
-	legacyDataDir := filepath.Join(referenceBuilderDir, "data")
-	registryDir := filepath.Join(dataDir, filepath.FromSlash(itempropsRegistryDirName))
-	if _, err := os.Stat(filepath.Join(registryDir, "properties.json")); err == nil {
-		if refs, err := countLegacyTLKRefsInRegistry(registryDir); err == nil && refs == 0 {
-			return 0, nil
-		}
-	}
-	if _, err := os.Stat(filepath.Join(legacyDataDir, "itemprops")); err != nil {
-		if os.IsNotExist(err) {
-			return 0, nil
-		}
-		return 0, err
-	}
-
-	defs, err := collectBaseDataset(nativeDataset{
-		Name:       "itemprops/defs",
-		BasePath:   filepath.Join(legacyDataDir, "itemprops", "defs", "base.json"),
-		LockPath:   filepath.Join(legacyDataDir, "itemprops", "defs", "lock.json"),
-		ModulesDir: filepath.Join(legacyDataDir, "itemprops", "defs", "modules"),
-		Spec:       specForDataset("itemprops"),
-	})
-	if err != nil {
-		return 0, err
-	}
-	sets, err := collectBaseDataset(nativeDataset{
-		Name:       "itemprops/sets",
-		BasePath:   filepath.Join(legacyDataDir, "itemprops", "sets", "base.json"),
-		LockPath:   filepath.Join(legacyDataDir, "itemprops", "sets", "lock.json"),
-		ModulesDir: filepath.Join(legacyDataDir, "itemprops", "sets", "modules"),
-		Spec:       specForDataset("itemprops"),
-	})
-	if err != nil {
-		return 0, err
-	}
-	costIndex, err := collectBaseDataset(nativeDataset{
-		Name:       "itemprops/costtables/index",
-		BasePath:   filepath.Join(legacyDataDir, "itemprops", "costtables", "index", "base.json"),
-		LockPath:   filepath.Join(legacyDataDir, "itemprops", "costtables", "index", "lock.json"),
-		ModulesDir: filepath.Join(legacyDataDir, "itemprops", "costtables", "index", "modules"),
-		Spec:       specForDataset("itemprops"),
-	})
-	if err != nil {
-		return 0, err
-	}
-	paramIndex, err := collectBaseDataset(nativeDataset{
-		Name:       "itemprops/paramtables/index",
-		BasePath:   filepath.Join(legacyDataDir, "itemprops", "paramtables", "index", "base.json"),
-		LockPath:   filepath.Join(legacyDataDir, "itemprops", "paramtables", "index", "lock.json"),
-		ModulesDir: filepath.Join(legacyDataDir, "itemprops", "paramtables", "index", "modules"),
-		Spec:       specForDataset("itemprops"),
-	})
-	if err != nil {
-		return 0, err
-	}
-
-	ownedCostTables, err := collectLegacyOwnedItempropTables(filepath.Join(legacyDataDir, "itemprops", "costtables"), "index")
-	if err != nil {
-		return 0, err
-	}
-	ownedParamTables, err := collectLegacyOwnedItempropTables(filepath.Join(legacyDataDir, "itemprops", "paramtables"), "index")
-	if err != nil {
-		return 0, err
-	}
-	ownedSubtypeTables, err := collectLegacyOwnedItempropTables(filepath.Join(legacyDataDir, "itemprops", "subtypes"), "")
-	if err != nil && !os.IsNotExist(err) {
-		return 0, err
-	}
-
-	setsByID := map[int]map[string]any{}
-	for _, row := range sets.Rows {
-		setsByID[row["id"].(int)] = row
-	}
-	costByID := map[int]map[string]any{}
-	for _, row := range costIndex.Rows {
-		costByID[row["id"].(int)] = row
-	}
-	paramByID := map[int]map[string]any{}
-	for _, row := range paramIndex.Rows {
-		paramByID[row["id"].(int)] = row
-	}
-
-	registryLock := map[string]int{}
-	legacyKeyByID := map[string]string{}
-	if legacyTLK != nil {
-		for key, id := range legacyTLK.Lock {
-			legacyKeyByID[strconv.Itoa(id)] = key
-		}
-	}
-	properties := make([]map[string]any, 0, len(defs.Rows))
-	subtypeRegistryByKey := map[string]map[string]any{}
-	for _, row := range defs.Rows {
-		rowID := row["id"].(int)
-		key := canonicalItempropPropertyKey(row)
-		registryLock[key] = rowID
-
-		property := map[string]any{
-			"key":           key,
-			"label":         row["Label"],
-			"name":          deepCopyValue(row["Name"]),
-			"property_text": deepCopyValue(row["GameStrRef"]),
-		}
-		if description := nullableRegistryField(row, "Description"); description != nullValue {
-			property["description"] = description
-		}
-		if setRow, ok := setsByID[rowID]; ok {
-			property["availability"] = importLegacyAvailability(setRow)
-			if setName := nullableRegistryField(setRow, "StringRef"); setName != nullValue && setName != stringField(row, "Name") {
-				property["set_name"] = setName
-			}
-			if setLabel := stringField(setRow, "Label"); setLabel != "" && setLabel != stringField(row, "Label") {
-				property["set_label"] = setLabel
-			}
-		} else {
-			property["availability"] = []any{}
-		}
-
-		if subtypeResRef := stringField(row, "SubTypeResRef"); subtypeResRef != "" && subtypeResRef != nullValue {
-			subtypeKey := canonicalModelKey("subtype_models", subtypeResRef)
-			property["subtype"] = map[string]any{"ref": subtypeKey}
-			if _, ok := subtypeRegistryByKey[subtypeKey]; !ok {
-				subtypeRegistryByKey[subtypeKey] = buildLegacySubtypeModel(subtypeKey, subtypeResRef, ownedSubtypeTables)
-			}
-		}
-		costValue := stringField(row, "Cost")
-		costID := stringField(row, "CostTableResRef")
-		if costValue != "" && costValue != nullValue || costID != "" && costID != nullValue {
-			cost := map[string]any{}
-			if costValue != "" && costValue != nullValue {
-				cost["value"] = costValue
-			}
-			if costID != "" && costID != nullValue {
-				id, err := strconv.Atoi(costID)
-				if err != nil {
-					return 0, err
-				}
-				costRow, ok := costByID[id]
-				if !ok {
-					return 0, fmt.Errorf("%s references missing cost table index %s", key, costID)
-				}
-				cost["ref"] = canonicalModelKey("cost_models", stringField(costRow, "Name"))
-			}
-			property["cost"] = cost
-		}
-		if paramID := stringField(row, "Param1ResRef"); paramID != "" && paramID != nullValue {
-			id, err := strconv.Atoi(paramID)
-			if err != nil {
-				return 0, err
-			}
-			paramRow, ok := paramByID[id]
-			if !ok {
-				return 0, fmt.Errorf("%s references missing param table index %s", key, paramID)
-			}
-			property["param"] = map[string]any{"ref": canonicalModelKey("param_models", stringField(paramRow, "TableResRef"))}
-		}
-		if legacyTLK != nil {
-			property["name"] = importLegacyStrrefValue(property["name"], key+".name", legacyTLK, legacyKeyByID)
-			property["property_text"] = importLegacyStrrefValue(property["property_text"], key+".property_text", legacyTLK, legacyKeyByID)
-			if description, ok := property["description"]; ok {
-				property["description"] = importLegacyStrrefValue(description, key+".description", legacyTLK, legacyKeyByID)
-			}
-			if updated, _, err := inlineLegacyTLKValue(property, legacyTLK); err != nil {
-				return 0, err
-			} else if updated {
-				// property updated in place
-			}
-		}
-		properties = append(properties, property)
-	}
-
-	costModels := make([]map[string]any, 0, len(costIndex.Rows))
-	for _, row := range costIndex.Rows {
-		key := canonicalModelKey("cost_models", stringField(row, "Name"))
-		registryLock[key] = row["id"].(int)
-		model := map[string]any{
-			"key":         key,
-			"index_name":  row["Name"],
-			"label":       row["Label"],
-			"client_load": row["ClientLoad"],
-			"table":       buildLegacyOwnedModelTable(stringField(row, "Name"), ownedCostTables),
-		}
-		costModels = append(costModels, model)
-	}
-
-	paramModels := make([]map[string]any, 0, len(paramIndex.Rows))
-	for _, row := range paramIndex.Rows {
-		key := canonicalModelKey("param_models", stringField(row, "TableResRef"))
-		registryLock[key] = row["id"].(int)
-		model := map[string]any{
-			"key":   key,
-			"name":  deepCopyValue(row["Name"]),
-			"label": row["Lable"],
-			"table": buildLegacyOwnedModelTable(stringField(row, "TableResRef"), ownedParamTables),
-		}
-		paramModels = append(paramModels, model)
-	}
-
-	subtypeModels := make([]map[string]any, 0, len(subtypeRegistryByKey))
-	for _, key := range sortedKeysAny(subtypeRegistryByKey) {
-		subtypeModels = append(subtypeModels, subtypeRegistryByKey[key])
-	}
-
-	writes := []struct {
-		path string
-		obj  map[string]any
-	}{
-		{filepath.Join(registryDir, "properties.json"), map[string]any{"rows": properties}},
-		{filepath.Join(registryDir, "cost_models.json"), map[string]any{"rows": costModels}},
-		{filepath.Join(registryDir, "param_models.json"), map[string]any{"rows": paramModels}},
-		{filepath.Join(registryDir, "subtype_models.json"), map[string]any{"rows": subtypeModels}},
-		{filepath.Join(registryDir, itempropsRegistryLock), anyMapInt(registryLock)},
-	}
-	if err := os.MkdirAll(registryDir, 0o755); err != nil {
-		return 0, err
-	}
-	for _, write := range writes {
-		raw, err := json.MarshalIndent(write.obj, "", "  ")
-		if err != nil {
-			return 0, err
-		}
-		raw = append(raw, '\n')
-		if err := os.WriteFile(write.path, raw, 0o644); err != nil {
-			return 0, err
-		}
-	}
-	return len(writes), nil
-}
-
-func countLegacyTLKRefsInRegistry(registryDir string) (int, error) {
-	paths, err := collectDataJSONPaths(registryDir)
-	if err != nil {
-		return 0, err
-	}
-	refs := 0
-	for _, path := range paths {
-		obj, err := loadJSONObject(path)
-		if err != nil {
-			return 0, err
-		}
-		refs += countLegacyTLKRefsInValue(obj)
-	}
-	return refs, nil
-}
-
-func collectLegacyOwnedItempropTables(root, skipDir string) (map[string]map[string]any, error) {
-	result := map[string]map[string]any{}
-	entries, err := os.ReadDir(root)
-	if err != nil {
-		return nil, err
-	}
-	for _, entry := range entries {
-		if !entry.IsDir() {
-			continue
-		}
-		if skipDir != "" && entry.Name() == skipDir {
-			continue
-		}
-		dir := filepath.Join(root, entry.Name())
-		basePath := filepath.Join(dir, "base.json")
-		if _, err := os.Stat(basePath); err != nil {
-			continue
-		}
-		collected, err := collectBaseDataset(nativeDataset{
-			Name:       filepath.ToSlash(filepath.Join("itemprops", filepath.Base(root), entry.Name())),
-			BasePath:   basePath,
-			LockPath:   filepath.Join(dir, "lock.json"),
-			ModulesDir: filepath.Join(dir, "modules"),
-			Spec:       specForDataset("itemprops"),
-		})
-		if err != nil {
-			return nil, err
-		}
-		result[strings.ToLower(outputStem(collected.Dataset.OutputName))] = map[string]any{
-			"ownership": "owned",
-			"resref":    outputStem(collected.Dataset.OutputName),
-			"output":    collected.Dataset.OutputName,
-			"columns":   stringSliceToAny(collected.Columns),
-			"rows":      rowsToAny(collected.Rows),
-		}
-	}
-	return result, nil
-}
-
-func buildLegacySubtypeModel(key, resref string, owned map[string]map[string]any) map[string]any {
-	return map[string]any{
-		"key":          key,
-		"table_resref": resref,
-		"table":        buildLegacyOwnedModelTable(resref, owned),
-	}
-}
-
-func buildLegacyOwnedModelTable(resref string, owned map[string]map[string]any) map[string]any {
-	if table, ok := owned[strings.ToLower(resref)]; ok {
-		return table
-	}
-	return map[string]any{
-		"ownership": "external",
-		"resref":    resref,
-	}
-}
-
-func canonicalItempropPropertyKey(row map[string]any) string {
-	if key := stringField(row, "key"); key != "" {
-		return "itemprop:" + strings.TrimPrefix(key, "itempropdef:")
-	}
-	return canonicalModelKey("itemprop", stringField(row, "Label"))
-}
-
-func canonicalModelKey(prefix, source string) string {
-	normalized := strings.ToLower(strings.TrimSpace(source))
-	normalized = strings.ReplaceAll(normalized, " ", "")
-	normalized = strings.ReplaceAll(normalized, "-", "")
-	normalized = strings.ReplaceAll(normalized, "_", "")
-	return prefix + ":" + normalized
-}
-
-func importLegacyStrrefValue(value any, fallbackKey string, legacy *legacyTLKData, keyByID map[string]string) any {
-	if legacy == nil {
-		return value
-	}
-	if _, ok, _ := parseTLKPayload(value, true); ok {
-		return value
-	}
-	_ = keyByID
-	key := fallbackKey
-	entry, ok := legacy.Entries[key]
-	if !ok {
-		return value
-	}
-	payload := map[string]any{
-		"key":  key,
-		"text": entry.Text,
-	}
-	if entry.SoundResRef != "" {
-		payload["sound_resref"] = entry.SoundResRef
-	}
-	if entry.SoundLength != 0 {
-		payload["sound_length"] = entry.SoundLength
-	}
-	return map[string]any{"tlk": payload}
-}
-
-func importLegacyAvailability(row map[string]any) []any {
-	out := make([]any, 0)
-	for _, column := range itempropsAvailabilityColumns {
-		value, ok := row[column]
-		if !ok {
-			continue
-		}
-		text := formatRegistryScalar(value)
-		if text != "1" {
-			continue
-		}
-		out = append(out, normalizeAvailabilityName(column))
-	}
-	return out
-}
-
-func anyMapInt(values map[string]int) map[string]any {
-	out := make(map[string]any, len(values))
-	for key, value := range values {
-		out[key] = value
-	}
-	return out
-}
-
-func stringSliceToAny(values []string) []any {
-	out := make([]any, 0, len(values))
-	for _, value := range values {
-		out = append(out, value)
-	}
-	return out
-}
-
-func rowsToAny(rows []map[string]any) []any {
-	out := make([]any, 0, len(rows))
-	for _, row := range rows {
-		out = append(out, deepCopyValue(row))
-	}
-	return out
-}
-
-func sortedKeysAny(values map[string]map[string]any) []string {
-	out := make([]string, 0, len(values))
-	for key := range values {
-		out = append(out, key)
-	}
-	slices.Sort(out)
-	return out
-}
-
 func validateItempropsRegistryGraph(dataDir string, report *ValidationReport) {
 	registry, err := loadItempropsRegistry(dataDir)
 	if err != nil {

internal/topdata/legacy_dataset_migrate.go

@@ -1,176 +0,0 @@
-package topdata
-
-import (
-	"encoding/json"
-	"io/fs"
-	"os"
-	"path/filepath"
-	"sort"
-	"strings"
-)
-
-type legacyDatasetTransform func(relativePath string, obj map[string]any)
-
-func importLegacyDatasetMirror(referenceBuilderDir, dataDir, datasetName, outputName string, transform legacyDatasetTransform) (int, error) {
-	legacyDir := filepath.Join(referenceBuilderDir, "data", datasetName)
-	if _, err := os.Stat(legacyDir); err != nil {
-		if os.IsNotExist(err) {
-			return 0, nil
-		}
-		return 0, err
-	}
-
-	targetDir := filepath.Join(dataDir, datasetName)
-	baseObj, err := loadJSONObject(filepath.Join(legacyDir, "base.json"))
-	if err != nil {
-		return 0, err
-	}
-	if outputName != "" {
-		baseObj["output"] = outputName
-	}
-	if transform != nil {
-		transform("base.json", baseObj)
-	}
-
-	lockObj, err := loadJSONObject(filepath.Join(legacyDir, "lock.json"))
-	if err != nil {
-		return 0, err
-	}
-	if transform != nil {
-		transform("lock.json", lockObj)
-	}
-
-	moduleRelPaths, err := collectJSONRelativePaths(filepath.Join(legacyDir, "modules"))
-	if err != nil {
-		return 0, err
-	}
-
-	moduleObjs := make(map[string]map[string]any, len(moduleRelPaths))
-	for _, relPath := range moduleRelPaths {
-		obj, err := loadJSONObject(filepath.Join(legacyDir, "modules", relPath))
-		if err != nil {
-			return 0, err
-		}
-		normalizedRel := filepath.ToSlash(relPath)
-		if transform != nil {
-			transform(filepath.ToSlash(filepath.Join("modules", normalizedRel)), obj)
-		}
-		moduleObjs[normalizedRel] = obj
-	}
-
-	if err := os.MkdirAll(targetDir, 0o755); err != nil {
-		return 0, err
-	}
-
-	updated := 0
-	for _, write := range []struct {
-		path string
-		obj  map[string]any
-	}{
-		{path: filepath.Join(targetDir, "base.json"), obj: baseObj},
-		{path: filepath.Join(targetDir, "lock.json"), obj: lockObj},
-	} {
-		changed, err := writeJSONObjectIfChanged(write.path, write.obj)
-		if err != nil {
-			return 0, err
-		}
-		if changed {
-			updated++
-		}
-	}
-
-	targetModulesDir := filepath.Join(targetDir, "modules")
-	if len(moduleObjs) > 0 {
-		if err := os.MkdirAll(targetModulesDir, 0o755); err != nil {
-			return 0, err
-		}
-	}
-	for relPath, obj := range moduleObjs {
-		targetPath := filepath.Join(targetModulesDir, filepath.FromSlash(relPath))
-		if err := os.MkdirAll(filepath.Dir(targetPath), 0o755); err != nil {
-			return 0, err
-		}
-		changed, err := writeJSONObjectIfChanged(targetPath, obj)
-		if err != nil {
-			return 0, err
-		}
-		if changed {
-			updated++
-		}
-	}
-
-	staleModules, err := collectJSONRelativePaths(targetModulesDir)
-	if err != nil {
-		return 0, err
-	}
-	for _, relPath := range staleModules {
-		normalizedRel := filepath.ToSlash(relPath)
-		if _, ok := moduleObjs[normalizedRel]; ok {
-			continue
-		}
-		if err := os.Remove(filepath.Join(targetModulesDir, filepath.FromSlash(normalizedRel))); err != nil && !os.IsNotExist(err) {
-			return 0, err
-		}
-		updated++
-	}
-
-	return updated, nil
-}
-
-func collectJSONRelativePaths(root string) ([]string, error) {
-	if _, err := os.Stat(root); err != nil {
-		if os.IsNotExist(err) {
-			return nil, nil
-		}
-		return nil, err
-	}
-
-	var relPaths []string
-	err := filepath.WalkDir(root, func(path string, d fs.DirEntry, err error) error {
-		if err != nil {
-			return err
-		}
-		if d.IsDir() {
-			return nil
-		}
-		if !strings.EqualFold(filepath.Ext(d.Name()), ".json") {
-			return nil
-		}
-		relPath, err := filepath.Rel(root, path)
-		if err != nil {
-			return err
-		}
-		relPaths = append(relPaths, filepath.ToSlash(relPath))
-		return nil
-	})
-	if err != nil {
-		return nil, err
-	}
-	sort.Strings(relPaths)
-	return relPaths, nil
-}
-
-func writeJSONObjectIfChanged(path string, obj map[string]any) (bool, error) {
-	raw, err := json.MarshalIndent(obj, "", "  ")
-	if err != nil {
-		return false, err
-	}
-	raw = append(raw, '\n')
-
-	current, err := os.ReadFile(path)
-	if err == nil && string(current) == string(raw) {
-		return false, nil
-	}
-	if err != nil && !os.IsNotExist(err) {
-		return false, err
-	}
-	if err := os.WriteFile(path, raw, 0o644); err != nil {
-		return false, err
-	}
-	return true, nil
-}
-
-func fileExists(path string) bool {
-	_, err := os.Stat(path)
-	return err == nil
-}

internal/topdata/loadscreens_migrate.go

@@ -1,6 +0,0 @@
-package topdata
-
-func importLegacyLoadscreens(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
-	_ = legacyTLK
-	return importLegacyDatasetMirror(referenceBuilderDir, dataDir, "loadscreens", "loadscreens.2da", nil)
-}

internal/topdata/masterfeats_migrate.go

@@ -1,237 +0,0 @@
-package topdata
-
-import (
-	"encoding/json"
-	"os"
-	"path/filepath"
-	"strconv"
-)
-
-func importLegacyMasterfeats(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
-	legacyDir := filepath.Join(referenceBuilderDir, "data", "feat", "masterfeats")
-	if _, err := os.Stat(legacyDir); err != nil {
-		if os.IsNotExist(err) {
-			return 0, nil
-		}
-		return 0, err
-	}
-
-	targetDir := filepath.Join(dataDir, "masterfeats")
-	targetPath := filepath.Join(targetDir, "base.json")
-	legacyPlainPath := filepath.Join(targetDir, "masterfeats.json")
-	if _, err := os.Stat(targetPath); err == nil {
-		obj, err := loadJSONObject(targetPath)
-		if err == nil && countLegacyTLKRefsInValue(obj) == 0 {
-			return 0, nil
-		}
-	}
-
-	collected, err := importLegacyMasterfeatsDataset(legacyDir)
-	if err != nil {
-		return 0, err
-	}
-	referenceIDs, err := collectReferenceLockIDs(filepath.Join(referenceBuilderDir, "data"))
-	if err != nil {
-		return 0, err
-	}
-
-	rows := make([]any, 0, len(collected.Rows))
-	for _, row := range collected.Rows {
-		copyRow, ok := deepCopyValue(row).(map[string]any)
-		if !ok {
-			continue
-		}
-		if legacyTLK != nil {
-			if _, _, err := inlineLegacyTLKValue(copyRow, legacyTLK); err != nil {
-				return 0, err
-			}
-		}
-		copyRow = rewriteImportedIDRefs(copyRow, referenceIDs)
-		rows = append(rows, copyRow)
-	}
-
-	if err := os.MkdirAll(targetDir, 0o755); err != nil {
-		return 0, err
-	}
-	writes := []struct {
-		path string
-		obj  map[string]any
-	}{
-		{
-			path: targetPath,
-			obj: map[string]any{
-				"output":  collected.Dataset.OutputName,
-				"columns": stringSliceToAny(collected.Columns),
-				"rows":    rows,
-			},
-		},
-		{
-			path: filepath.Join(targetDir, "lock.json"),
-			obj:  anyMapInt(collected.LockData),
-		},
-	}
-	for _, write := range writes {
-		raw, err := json.MarshalIndent(write.obj, "", "  ")
-		if err != nil {
-			return 0, err
-		}
-		raw = append(raw, '\n')
-		if err := os.WriteFile(write.path, raw, 0o644); err != nil {
-			return 0, err
-		}
-	}
-	updatedFiles := len(writes)
-	if err := os.Remove(legacyPlainPath); err == nil {
-		updatedFiles++
-	} else if err != nil && !os.IsNotExist(err) {
-		return 0, err
-	}
-	return updatedFiles, nil
-}
-
-func importLegacyMasterfeatsDataset(legacyDir string) (nativeCollectedDataset, error) {
-	dataset := nativeDataset{
-		Name:     "masterfeats",
-		BasePath: filepath.Join(legacyDir, "masterfeats.json"),
-		LockPath: filepath.Join(legacyDir, "lock.json"),
-		Spec:     specForDataset("masterfeats"),
-	}
-	tableData, err := loadJSONObject(dataset.BasePath)
-	if err != nil {
-		return nativeCollectedDataset{}, err
-	}
-	columns, err := parseColumns(tableData, dataset.Name)
-	if err != nil {
-		return nativeCollectedDataset{}, err
-	}
-	rawRows, ok := tableData["rows"].([]any)
-	if !ok {
-		return nativeCollectedDataset{}, nil
-	}
-	lockData, err := loadLockfile(dataset.LockPath)
-	if err != nil {
-		return nativeCollectedDataset{}, err
-	}
-	usedIDs := map[int]struct{}{}
-	for _, rowID := range lockData {
-		usedIDs[rowID] = struct{}{}
-	}
-
-	rows := make([]map[string]any, 0, len(rawRows))
-	explicitID := make([]bool, 0, len(rawRows))
-	for index, raw := range rawRows {
-		rowObj, ok := raw.(map[string]any)
-		if !ok {
-			continue
-		}
-		rowID := index
-		if value, ok := rowObj["id"]; ok {
-			parsed, err := asInt(value)
-			if err != nil {
-				return nativeCollectedDataset{}, err
-			}
-			rowID = parsed
-		}
-		row := map[string]any{"id": rowID}
-		for _, column := range columns {
-			row[column] = nullValue
-		}
-		for key, value := range rowObj {
-			switch key {
-			case "id":
-			case "key":
-				if keyText, ok := value.(string); ok && keyText != "" {
-					row["key"] = keyText
-				}
-			default:
-				columnName, ok := canonicalColumn(columns, key)
-				if !ok {
-					continue
-				}
-				row[columnName] = value
-			}
-		}
-		rows = append(rows, row)
-		_, hasExplicitID := rowObj["id"]
-		explicitID = append(explicitID, hasExplicitID)
-		if hasExplicitID {
-			usedIDs[rowID] = struct{}{}
-		}
-	}
-
-	nextID := nextAvailableID(usedIDs)
-	lockModified := false
-	for index, row := range rows {
-		key, hasKey := row["key"].(string)
-		if hasKey && key != "" {
-			if lockedID, ok := lockData[key]; ok {
-				row["id"] = lockedID
-				continue
-			}
-			if explicitID[index] {
-				lockData[key] = row["id"].(int)
-			} else {
-				row["id"] = nextID
-				lockData[key] = nextID
-				usedIDs[nextID] = struct{}{}
-				nextID = nextAvailableID(usedIDs)
-			}
-			lockModified = true
-			continue
-		}
-		if !explicitID[index] {
-			row["id"] = index
-		}
-	}
-	if lockModified {
-		_ = lockModified
-	}
-
-	return nativeCollectedDataset{
-		Dataset: nativeDataset{
-			Name:       dataset.Name,
-			OutputName: "masterfeats.2da",
-			Columns:    columns,
-			Spec:       dataset.Spec,
-		},
-		Columns:  columns,
-		Rows:     rows,
-		LockData: lockData,
-	}, nil
-}
-
-func rewriteImportedIDRefs(value any, ids map[string]int) map[string]any {
-	row, ok := rewriteImportedIDRefsValue(value, ids).(map[string]any)
-	if !ok {
-		return map[string]any{}
-	}
-	return row
-}
-
-func rewriteImportedIDRefsValue(value any, ids map[string]int) any {
-	switch typed := value.(type) {
-	case map[string]any:
-		if len(typed) == 1 {
-			if rawID, ok := typed["id"]; ok {
-				if key, ok := rawID.(string); ok {
-					if resolved, ok := ids[key]; ok {
-						return strconv.Itoa(resolved)
-					}
-				}
-			}
-		}
-		out := make(map[string]any, len(typed))
-		for key, child := range typed {
-			out[key] = rewriteImportedIDRefsValue(child, ids)
-		}
-		return out
-	case []any:
-		out := make([]any, 0, len(typed))
-		for _, child := range typed {
-			out = append(out, rewriteImportedIDRefsValue(child, ids))
-		}
-		return out
-	default:
-		return value
-	}
-}

internal/topdata/migrate.go

@@ -1,415 +0,0 @@
-package topdata
-
-import (
-	"encoding/json"
-	"fmt"
-	"os"
-	"path/filepath"
-	"strings"
-
-	"git.westgate.pw/ShadowsOverWestgate/sow-tools/internal/project"
-)
-
-type NormalizeResult struct {
-	StatePath           string
-	ScannedFiles        int
-	UpdatedFiles        int
-	InlineValues        int
-	RemainingFiles      int
-	RemainingLegacyRefs int
-}
-
-func NormalizeProject(p *project.Project) (NormalizeResult, error) {
-	if !p.HasTopData() {
-		return NormalizeResult{}, fmt.Errorf("topdata is not configured for this project")
-	}
-
-	sourceDir := p.TopDataSourceDir()
-	dataDir := filepath.Join(sourceDir, "data")
-	migrationRoot := resolveMigrationInputRoot(sourceDir, p.TopDataReferenceBuilderDir())
-	legacyTLK, err := loadLegacyTLK(filepath.Join(sourceDir, "tlk"))
-	if err != nil {
-		return NormalizeResult{}, err
-	}
-	if migrationRoot != "" {
-		migrationTLK, err := loadLegacyTLK(filepath.Join(migrationRoot, "tlk"))
-		if err != nil {
-			return NormalizeResult{}, err
-		}
-		legacyTLK = mergeLegacyTLK(migrationTLK, legacyTLK)
-	}
-	baseDialogImported, err := importLegacyBaseDialog(migrationRoot, sourceDir)
-	if err != nil {
-		return NormalizeResult{}, err
-	}
-
-	itempropsImported, err := importLegacyItempropsRegistry(migrationRoot, dataDir, legacyTLK)
-	if err != nil {
-		return NormalizeResult{}, err
-	}
-	masterfeatsImported, err := importLegacyMasterfeats(migrationRoot, dataDir, legacyTLK)
-	if err != nil {
-		return NormalizeResult{}, err
-	}
-	featImported, err := importLegacyFeat(migrationRoot, dataDir, legacyTLK)
-	if err != nil {
-		return NormalizeResult{}, err
-	}
-	creaturespeedImported, err := importLegacyCreaturespeed(migrationRoot, dataDir, legacyTLK)
-	if err != nil {
-		return NormalizeResult{}, err
-	}
-	armorImported, err := importLegacyArmor(migrationRoot, dataDir, legacyTLK)
-	if err != nil {
-		return NormalizeResult{}, err
-	}
-	baseitemsImported, err := importLegacyBaseitems(migrationRoot, dataDir, legacyTLK)
-	if err != nil {
-		return NormalizeResult{}, err
-	}
-	damagetypesImported, err := importLegacyDamagetypes(migrationRoot, dataDir, legacyTLK)
-	if err != nil {
-		return NormalizeResult{}, err
-	}
-	skyboxesImported, err := importLegacySkyboxes(migrationRoot, dataDir, legacyTLK)
-	if err != nil {
-		return NormalizeResult{}, err
-	}
-	vfxPersistentImported, err := importLegacyVFXPersistent(migrationRoot, dataDir, legacyTLK)
-	if err != nil {
-		return NormalizeResult{}, err
-	}
-	loadscreensImported, err := importLegacyLoadscreens(migrationRoot, dataDir, legacyTLK)
-	if err != nil {
-		return NormalizeResult{}, err
-	}
-	progfxImported, err := importLegacyProgfx(migrationRoot, dataDir, legacyTLK)
-	if err != nil {
-		return NormalizeResult{}, err
-	}
-	genericdoorsImported, err := importLegacyGenericdoors(migrationRoot, dataDir, legacyTLK)
-	if err != nil {
-		return NormalizeResult{}, err
-	}
-	cloakmodelImported, err := importLegacyCloakmodel(migrationRoot, dataDir, legacyTLK)
-	if err != nil {
-		return NormalizeResult{}, err
-	}
-	repadjustImported, err := importLegacyRepadjust(migrationRoot, dataDir, legacyTLK)
-	if err != nil {
-		return NormalizeResult{}, err
-	}
-	wingmodelImported, err := importLegacyWingmodel(migrationRoot, dataDir, legacyTLK)
-	if err != nil {
-		return NormalizeResult{}, err
-	}
-	tailmodelImported, err := importLegacyTailmodel(migrationRoot, dataDir, legacyTLK)
-	if err != nil {
-		return NormalizeResult{}, err
-	}
-	doortypesImported, err := importLegacyDoortypes(migrationRoot, dataDir, legacyTLK)
-	if err != nil {
-		return NormalizeResult{}, err
-	}
-	visualeffectsImported, err := importLegacyVisualeffects(migrationRoot, dataDir, legacyTLK)
-	if err != nil {
-		return NormalizeResult{}, err
-	}
-	appearanceImported, err := importLegacyAppearance(migrationRoot, dataDir, legacyTLK)
-	if err != nil {
-		return NormalizeResult{}, err
-	}
-	placeablesImported, err := importLegacyPlaceables(migrationRoot, dataDir, legacyTLK)
-	if err != nil {
-		return NormalizeResult{}, err
-	}
-	rulesetImported, err := importLegacyRuleset(migrationRoot, dataDir, legacyTLK)
-	if err != nil {
-		return NormalizeResult{}, err
-	}
-	skillsImported, err := importLegacySkills(migrationRoot, dataDir, legacyTLK)
-	if err != nil {
-		return NormalizeResult{}, err
-	}
-	spellsImported, err := importLegacySpells(migrationRoot, dataDir, legacyTLK)
-	if err != nil {
-		return NormalizeResult{}, err
-	}
-	racialtypesImported, err := importLegacyRacialtypes(migrationRoot, dataDir, legacyTLK)
-	if err != nil {
-		return NormalizeResult{}, err
-	}
-	classesImported, err := importLegacyClasses(migrationRoot, dataDir, legacyTLK)
-	if err != nil {
-		return NormalizeResult{}, err
-	}
-	portraitsImported, err := importLegacyPortraits(migrationRoot, dataDir, legacyTLK)
-	if err != nil {
-		return NormalizeResult{}, err
-	}
-
-	paths, err := collectDataJSONPaths(dataDir)
-	if err != nil {
-		return NormalizeResult{}, err
-	}
-
-	result := NormalizeResult{
-		StatePath:    filepath.Join(sourceDir, tlkStateFile),
-		ScannedFiles: len(paths),
-		UpdatedFiles: baseDialogImported + itempropsImported + masterfeatsImported + featImported + creaturespeedImported + armorImported + baseitemsImported + damagetypesImported + skyboxesImported + vfxPersistentImported + loadscreensImported + progfxImported + genericdoorsImported + cloakmodelImported + repadjustImported + wingmodelImported + tailmodelImported + doortypesImported + visualeffectsImported + appearanceImported + placeablesImported + rulesetImported + skillsImported + spellsImported + racialtypesImported + classesImported + portraitsImported,
-	}
-	state, err := loadTLKState(result.StatePath)
-	if err != nil {
-		return result, err
-	}
-	if state.Entries == nil {
-		state.Entries = map[string]tlkStateMapping{}
-	}
-	if legacyTLK != nil && state.Language == "" {
-		state.Language = legacyTLK.Language
-	}
-	if legacyTLK != nil {
-		for key, id := range legacyTLK.Lock {
-			if _, ok := state.Entries[key]; !ok {
-				state.Entries[key] = tlkStateMapping{ID: id}
-			}
-		}
-	}
-
-	if legacyTLK != nil {
-		for _, path := range paths {
-			updated, count, err := inlineLegacyTLKFile(path, legacyTLK)
-			if err != nil {
-				return result, err
-			}
-			if updated {
-				result.UpdatedFiles++
-				result.InlineValues += count
-			}
-		}
-	}
-
-	if err := saveTLKState(result.StatePath, state); err != nil {
-		return result, err
-	}
-	if err := removeLegacyTLKAuthoredInput(filepath.Join(sourceDir, "tlk")); err != nil {
-		return result, err
-	}
-	remainingFiles, remainingRefs, err := countLegacyTLKRefs(paths)
-	if err != nil {
-		return result, err
-	}
-	result.RemainingFiles = remainingFiles
-	result.RemainingLegacyRefs = remainingRefs
-	return result, nil
-}
-
-func mergeLegacyTLK(base, override *legacyTLKData) *legacyTLKData {
-	if base == nil && override == nil {
-		return nil
-	}
-	merged := &legacyTLKData{
-		Language: "en",
-		Entries:  map[string]tlkEntryData{},
-		Lock:     map[string]int{},
-	}
-	if base != nil {
-		if base.Language != "" {
-			merged.Language = base.Language
-		}
-		for key, entry := range base.Entries {
-			merged.Entries[key] = entry
-		}
-		for key, id := range base.Lock {
-			merged.Lock[key] = id
-		}
-	}
-	if override != nil {
-		if override.Language != "" {
-			merged.Language = override.Language
-		}
-		for key, entry := range override.Entries {
-			merged.Entries[key] = entry
-		}
-		for key, id := range override.Lock {
-			merged.Lock[key] = id
-		}
-	}
-	return merged
-}
-
-func removeLegacyTLKAuthoredInput(tlkDir string) error {
-	modulesDir := filepath.Join(tlkDir, "modules")
-	if err := os.RemoveAll(modulesDir); err != nil {
-		return fmt.Errorf("remove %s: %w", modulesDir, err)
-	}
-	lockPath := filepath.Join(tlkDir, "lock.json")
-	if err := os.Remove(lockPath); err != nil && !os.IsNotExist(err) {
-		return fmt.Errorf("remove %s: %w", lockPath, err)
-	}
-	gitkeepPath := filepath.Join(tlkDir, ".gitkeep")
-	if err := os.Remove(gitkeepPath); err != nil && !os.IsNotExist(err) {
-		return fmt.Errorf("remove %s: %w", gitkeepPath, err)
-	}
-	if err := os.Remove(tlkDir); err != nil && !os.IsNotExist(err) {
-		if !strings.Contains(err.Error(), "directory not empty") {
-			return fmt.Errorf("remove %s: %w", tlkDir, err)
-		}
-	}
-	return nil
-}
-
-func resolveMigrationInputRoot(sourceDir, referenceBuilderDir string) string {
-	if strings.TrimSpace(referenceBuilderDir) != "" {
-		return referenceBuilderDir
-	}
-	snapshotRoot := filepath.Join(sourceDir, "migration_snapshot")
-	if info, err := os.Stat(snapshotRoot); err == nil && info.IsDir() {
-		return snapshotRoot
-	}
-	return ""
-}
-
-func inlineLegacyTLKFile(path string, legacy *legacyTLKData) (bool, int, error) {
-	obj, err := loadJSONObject(path)
-	if err != nil {
-		return false, 0, err
-	}
-	updated, count, err := inlineLegacyTLKValue(obj, legacy)
-	if err != nil {
-		return false, 0, err
-	}
-	if !updated {
-		return false, 0, nil
-	}
-	raw, err := json.MarshalIndent(obj, "", "  ")
-	if err != nil {
-		return false, 0, err
-	}
-	raw = append(raw, '\n')
-	if err := os.WriteFile(path, raw, 0o644); err != nil {
-		return false, 0, err
-	}
-	return true, count, nil
-}
-
-func inlineLegacyTLKValue(value any, legacy *legacyTLKData) (bool, int, error) {
-	switch typed := value.(type) {
-	case map[string]any:
-		if rawTLK, ok := typed["tlk"]; ok {
-			key, ok := rawTLK.(string)
-			if ok {
-				entry, ok := legacy.Entries[key]
-				if !ok {
-					return false, 0, nil
-				}
-				payload := map[string]any{
-					"key":  key,
-					"text": entry.Text,
-				}
-				if entry.SoundResRef != "" {
-					payload["sound_resref"] = entry.SoundResRef
-				}
-				if entry.SoundLength != 0 {
-					payload["sound_length"] = entry.SoundLength
-				}
-				typed["tlk"] = payload
-				return true, 1, nil
-			}
-		}
-
-		updated := false
-		total := 0
-		for key, child := range typed {
-			childUpdated, childCount, err := inlineLegacyTLKValue(child, legacy)
-			if err != nil {
-				return false, 0, err
-			}
-			if childUpdated {
-				typed[key] = child
-				updated = true
-				total += childCount
-			}
-		}
-		return updated, total, nil
-	case []any:
-		updated := false
-		total := 0
-		for index, child := range typed {
-			childUpdated, childCount, err := inlineLegacyTLKValue(child, legacy)
-			if err != nil {
-				return false, 0, err
-			}
-			if childUpdated {
-				typed[index] = child
-				updated = true
-				total += childCount
-			}
-		}
-		return updated, total, nil
-	default:
-		return false, 0, nil
-	}
-}
-
-func collectDataJSONPaths(dataDir string) ([]string, error) {
-	paths := make([]string, 0)
-	err := filepath.WalkDir(dataDir, func(path string, d os.DirEntry, err error) error {
-		if err != nil {
-			return err
-		}
-		if d.IsDir() {
-			return nil
-		}
-		if strings.HasSuffix(strings.ToLower(d.Name()), ".json") && d.Name() != "lock.json" {
-			paths = append(paths, path)
-		}
-		return nil
-	})
-	if err != nil {
-		return nil, err
-	}
-	return paths, nil
-}
-
-func countLegacyTLKRefs(paths []string) (int, int, error) {
-	files := 0
-	refs := 0
-	for _, path := range paths {
-		obj, err := loadJSONObject(path)
-		if err != nil {
-			return 0, 0, err
-		}
-		count := countLegacyTLKRefsInValue(obj)
-		if count == 0 {
-			continue
-		}
-		files++
-		refs += count
-	}
-	return files, refs, nil
-}
-
-func countLegacyTLKRefsInValue(value any) int {
-	switch typed := value.(type) {
-	case map[string]any:
-		total := 0
-		if rawTLK, ok := typed["tlk"]; ok {
-			if _, ok := rawTLK.(string); ok {
-				total++
-			}
-		}
-		for _, child := range typed {
-			total += countLegacyTLKRefsInValue(child)
-		}
-		return total
-	case []any:
-		total := 0
-		for _, child := range typed {
-			total += countLegacyTLKRefsInValue(child)
-		}
-		return total
-	default:
-		return 0
-	}
-}

internal/topdata/native.go

@@ -410,6 +410,9 @@ func buildNativeUnchecked(p *project.Project, opts NativeBuildOptions, progress
 	if err != nil {
 		return BuildResult{}, err
 	}
+	if err := loadStandaloneTLKStrings(sourceDir, compiler); err != nil {
+		return BuildResult{}, err
+	}
 
 	output2DA := compiled2DAOutputDir(p)
 	outputTLK := compiledTLKOutputDir(p)
@@ -6642,81 +6645,6 @@ func marshalOrderedLockfile(keys []string, lockData map[string]int, formatting j
 	return buffer.Bytes(), nil
 }
 
-func loadLegacyTLK(root string) (*legacyTLKData, error) {
-	info, err := os.Stat(root)
-	if err != nil {
-		if os.IsNotExist(err) {
-			return nil, nil
-		}
-		return nil, err
-	}
-	if !info.IsDir() {
-		return nil, fmt.Errorf("legacy tlk root must be a directory: %s", root)
-	}
-
-	result := &legacyTLKData{
-		Language: "en",
-		Entries:  map[string]tlkEntryData{},
-		Lock:     map[string]int{},
-	}
-
-	lockPath := filepath.Join(root, "lock.json")
-	lockData, err := loadLockfile(lockPath)
-	if err != nil && !os.IsNotExist(err) {
-		return nil, err
-	}
-	for key, id := range lockData {
-		result.Lock[key] = id
-	}
-
-	basePath := filepath.Join(root, "base.json")
-	if _, err := os.Stat(basePath); err == nil {
-		base, err := loadJSONObject(basePath)
-		if err != nil {
-			return nil, err
-		}
-		if language, ok := base["language"].(string); ok && language != "" {
-			result.Language = language
-		}
-		if entries, ok := base["entries"].(map[string]any); ok {
-			normalized, err := normalizeLegacyTLKEntries(entries)
-			if err != nil {
-				return nil, err
-			}
-			for key, entry := range normalized {
-				result.Entries[key] = entry
-			}
-		}
-	}
-
-	modulesDir := filepath.Join(root, "modules")
-	modulePaths, err := collectModulePaths(modulesDir)
-	if err != nil {
-		if os.IsNotExist(err) {
-			return result, nil
-		}
-		return nil, err
-	}
-	for _, path := range modulePaths {
-		obj, err := loadJSONObject(path)
-		if err != nil {
-			return nil, err
-		}
-		entries, ok := obj["entries"].(map[string]any)
-		if !ok {
-			continue
-		}
-		normalized, err := normalizeLegacyTLKEntries(entries)
-		if err != nil {
-			return nil, fmt.Errorf("%s: %w", path, err)
-		}
-		for key, entry := range normalized {
-			result.Entries[key] = entry
-		}
-	}
-	return result, nil
-}
-
 func normalizeLegacyTLKEntries(entries map[string]any) (map[string]tlkEntryData, error) {
 	normalized := map[string]tlkEntryData{}
 	for _, key := range sortedKeys(entries) {

internal/topdata/placeables_migrate.go

@@ -1,6 +0,0 @@
-package topdata
-
-func importLegacyPlaceables(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
-	_ = legacyTLK
-	return importLegacyDatasetMirror(referenceBuilderDir, dataDir, "placeables", "placeables.2da", nil)
-}

internal/topdata/portraits_migrate.go

@@ -1,8 +0,0 @@
-package topdata
-
-func importLegacyPortraits(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
-	if legacyTLK == nil {
-		return 0, nil
-	}
-	return importLegacyDatasetMirror(referenceBuilderDir, dataDir, "portraits", "portraits.2da", nil)
-}

internal/topdata/progfx_migrate.go

@@ -1,6 +0,0 @@
-package topdata
-
-func importLegacyProgfx(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
-	_ = legacyTLK
-	return importLegacyDatasetMirror(referenceBuilderDir, dataDir, "progfx", "progfx.2da", nil)
-}

internal/topdata/racialtypes_migrate.go

@@ -1,282 +0,0 @@
-package topdata
-
-import (
-	"encoding/json"
-	"fmt"
-	"os"
-	"path/filepath"
-	"slices"
-	"strings"
-)
-
-type legacyRacialtypesFeatTable struct {
-	Output string
-	Rows   []map[string]any
-}
-
-func importLegacyRacialtypes(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
-	if strings.TrimSpace(referenceBuilderDir) == "" {
-		return 0, nil
-	}
-
-	legacyCoreDir := filepath.Join(referenceBuilderDir, "data", "racialtypes", "core")
-	legacyFeatsDir := filepath.Join(referenceBuilderDir, "data", "racialtypes", "feats")
-	if _, err := os.Stat(legacyCoreDir); err != nil {
-		if os.IsNotExist(err) {
-			return 0, nil
-		}
-		return 0, err
-	}
-	if _, err := os.Stat(legacyFeatsDir); err != nil {
-		if os.IsNotExist(err) {
-			return 0, nil
-		}
-		return 0, err
-	}
-
-	targetRoot := filepath.Join(dataDir, "racialtypes", "registry")
-	targetBasePath := filepath.Join(targetRoot, "base.json")
-	targetLockPath := filepath.Join(targetRoot, "lock.json")
-	targetRacesDir := filepath.Join(targetRoot, "races")
-	if fileExists(targetBasePath) && fileExists(targetLockPath) {
-		entries, err := os.ReadDir(targetRacesDir)
-		if err == nil {
-			raceFiles := 0
-			for _, entry := range entries {
-				if !entry.IsDir() && strings.HasSuffix(strings.ToLower(entry.Name()), ".json") {
-					raceFiles++
-				}
-			}
-			if raceFiles == 16 {
-				return 0, nil
-			}
-		}
-	}
-
-	baseColumns, baseRows, lockData, raceRows, err := collectLegacyRacialtypesRegistry(legacyCoreDir, legacyFeatsDir, legacyTLK)
-	if err != nil {
-		return 0, err
-	}
-
-	if err := removePathIfExists(filepath.Join(dataDir, "racialtypes", "core")); err != nil {
-		return 0, err
-	}
-	if err := removePathIfExists(filepath.Join(dataDir, "racialtypes", "feats")); err != nil {
-		return 0, err
-	}
-	if err := removePathIfExists(filepath.Join(targetRoot, "base_rows.json")); err != nil {
-		return 0, err
-	}
-	if err := removePathIfExists(targetRacesDir); err != nil {
-		return 0, err
-	}
-	if err := os.MkdirAll(targetRacesDir, 0o755); err != nil {
-		return 0, err
-	}
-
-	writes := []struct {
-		path string
-		obj  any
-	}{
-		{
-			path: targetBasePath,
-			obj: map[string]any{
-				"columns": stringSliceToAny(baseColumns),
-				"rows":    rowsToAny(baseRows),
-			},
-		},
-		{
-			path: targetLockPath,
-			obj:  anyMapInt(lockData),
-		},
-	}
-
-	raceKeys := make([]string, 0, len(raceRows))
-	for key := range raceRows {
-		raceKeys = append(raceKeys, key)
-	}
-	slices.Sort(raceKeys)
-	for _, key := range raceKeys {
-		fileName := strings.TrimPrefix(key, "racialtypes:") + ".json"
-		writes = append(writes, struct {
-			path string
-			obj  any
-		}{
-			path: filepath.Join(targetRacesDir, fileName),
-			obj:  raceRows[key],
-		})
-	}
-
-	for _, write := range writes {
-		raw, err := json.MarshalIndent(write.obj, "", "  ")
-		if err != nil {
-			return 0, err
-		}
-		raw = append(raw, '\n')
-		if err := os.WriteFile(write.path, raw, 0o644); err != nil {
-			return 0, err
-		}
-	}
-	return len(writes), nil
-}
-
-func collectLegacyRacialtypesRegistry(coreDir, featsDir string, legacyTLK *legacyTLKData) ([]string, []map[string]any, map[string]int, map[string]map[string]any, error) {
-	coreCollected, err := collectBaseDataset(nativeDataset{
-		Name:       "racialtypes",
-		BasePath:   filepath.Join(coreDir, "base.json"),
-		LockPath:   filepath.Join(coreDir, "lock.json"),
-		ModulesDir: filepath.Join(coreDir, "modules"),
-		Spec:       specForDataset("racialtypes"),
-	})
-	if err != nil {
-		return nil, nil, nil, nil, err
-	}
-
-	featTables, err := loadLegacyRacialtypesFeatTables(featsDir)
-	if err != nil {
-		return nil, nil, nil, nil, err
-	}
-
-	baseRows := make([]map[string]any, 0)
-	raceRows := map[string]map[string]any{}
-	lockData := map[string]int{}
-	for key, id := range coreCollected.LockData {
-		if strings.HasPrefix(key, "racialtypes:") {
-			lockData[key] = id
-		}
-	}
-
-	for _, collectedRow := range coreCollected.Rows {
-		row, ok := deepCopyValue(collectedRow).(map[string]any)
-		if !ok {
-			continue
-		}
-		if legacyTLK != nil {
-			if _, _, err := inlineLegacyTLKValue(row, legacyTLK); err != nil {
-				return nil, nil, nil, nil, err
-			}
-		}
-		key, _ := row["key"].(string)
-		if !strings.HasPrefix(key, "racialtypes:") {
-			assignRacialtypesBaseRowKey(row)
-			if key, _ := row["key"].(string); key != "" {
-				rowID, err := asInt(row["id"])
-				if err != nil {
-					return nil, nil, nil, nil, err
-				}
-				lockData[key] = rowID
-			}
-			baseRows = append(baseRows, row)
-			continue
-		}
-
-		delete(row, "id")
-		delete(row, "key")
-
-		raceFile := map[string]any{
-			"key":  key,
-			"core": row,
-		}
-		if tableKey := extractRacialtypesFeatTableKey(row["FeatsTable"]); tableKey != "" {
-			table, ok := featTables[tableKey]
-			if !ok {
-				return nil, nil, nil, nil, fmt.Errorf("%s: unknown feat table %s", key, tableKey)
-			}
-			delete(row, "FeatsTable")
-			raceFile["feat_output"] = table.Output
-			raceFile["feats"] = rowsToAny(table.Rows)
-		}
-		raceRows[key] = raceFile
-	}
-
-	slices.SortFunc(baseRows, func(a, b map[string]any) int {
-		left, _ := asInt(a["id"])
-		right, _ := asInt(b["id"])
-		return left - right
-	})
-	return coreCollected.Columns, baseRows, lockData, raceRows, nil
-}
-
-func loadLegacyRacialtypesFeatTables(featsDir string) (map[string]legacyRacialtypesFeatTable, error) {
-	entries, err := os.ReadDir(featsDir)
-	if err != nil {
-		return nil, err
-	}
-
-	tables := map[string]legacyRacialtypesFeatTable{}
-	for _, entry := range entries {
-		if entry.IsDir() || filepath.Ext(entry.Name()) != ".json" {
-			continue
-		}
-		obj, err := loadJSONObject(filepath.Join(featsDir, entry.Name()))
-		if err != nil {
-			return nil, err
-		}
-		tableKey, _ := obj["key"].(string)
-		outputName, _ := obj["output"].(string)
-		rawRows, ok := obj["rows"].([]any)
-		if tableKey == "" || outputName == "" || !ok {
-			return nil, fmt.Errorf("%s: racialtypes feat table must define key, output, and rows", entry.Name())
-		}
-		rows := make([]map[string]any, 0, len(rawRows))
-		for index, raw := range rawRows {
-			row, ok := raw.(map[string]any)
-			if !ok {
-				return nil, fmt.Errorf("%s: row %d must be an object", entry.Name(), index)
-			}
-			rows = append(rows, row)
-		}
-		tables[tableKey] = legacyRacialtypesFeatTable{
-			Output: outputName,
-			Rows:   rows,
-		}
-	}
-	return tables, nil
-}
-
-func extractRacialtypesFeatTableKey(value any) string {
-	obj, ok := value.(map[string]any)
-	if !ok {
-		return ""
-	}
-	tableKey, _ := obj["table"].(string)
-	return tableKey
-}
-
-func assignRacialtypesBaseRowKey(row map[string]any) {
-	name, _ := row["Name"].(string)
-	if strings.TrimSpace(name) == "" || name == nullValue {
-		delete(row, "key")
-		return
-	}
-	label, _ := row["Label"].(string)
-	keySuffix := normalizeRacialtypesKeySuffix(label)
-	if keySuffix == "" {
-		delete(row, "key")
-		return
-	}
-	row["key"] = "racialtypes:" + keySuffix
-}
-
-func normalizeRacialtypesKeySuffix(label string) string {
-	normalized := strings.ToLower(strings.TrimSpace(label))
-	if normalized == "" {
-		return ""
-	}
-	var b strings.Builder
-	lastUnderscore := false
-	for _, ch := range normalized {
-		isAlphaNum := (ch >= 'a' && ch <= 'z') || (ch >= '0' && ch <= '9')
-		if isAlphaNum {
-			b.WriteRune(ch)
-			lastUnderscore = false
-			continue
-		}
-		if lastUnderscore {
-			continue
-		}
-		b.WriteByte('_')
-		lastUnderscore = true
-	}
-	return strings.Trim(b.String(), "_")
-}

internal/topdata/repadjust_migrate.go

@@ -1,63 +0,0 @@
-package topdata
-
-import (
-	"encoding/json"
-	"os"
-	"path/filepath"
-)
-
-func importLegacyRepadjust(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
-	_ = legacyTLK
-
-	legacyDir := filepath.Join(referenceBuilderDir, "data", "repadjust")
-	if _, err := os.Stat(legacyDir); err != nil {
-		if os.IsNotExist(err) {
-			return 0, nil
-		}
-		return 0, err
-	}
-
-	targetDir := filepath.Join(dataDir, "repadjust")
-	targetBasePath := filepath.Join(targetDir, "base.json")
-	targetLockPath := filepath.Join(targetDir, "lock.json")
-	if fileExists(targetBasePath) && fileExists(targetLockPath) {
-		obj, err := loadJSONObject(targetBasePath)
-		if err == nil && countLegacyTLKRefsInValue(obj) == 0 {
-			return 0, nil
-		}
-	}
-
-	baseObj, err := loadJSONObject(filepath.Join(legacyDir, "base.json"))
-	if err != nil {
-		return 0, err
-	}
-	baseObj["output"] = "repadjust.2da"
-
-	lockObj, err := loadJSONObject(filepath.Join(legacyDir, "lock.json"))
-	if err != nil {
-		return 0, err
-	}
-
-	if err := os.MkdirAll(targetDir, 0o755); err != nil {
-		return 0, err
-	}
-
-	writes := []struct {
-		path string
-		obj  map[string]any
-	}{
-		{path: targetBasePath, obj: baseObj},
-		{path: targetLockPath, obj: lockObj},
-	}
-	for _, write := range writes {
-		raw, err := json.MarshalIndent(write.obj, "", "  ")
-		if err != nil {
-			return 0, err
-		}
-		raw = append(raw, '\n')
-		if err := os.WriteFile(write.path, raw, 0o644); err != nil {
-			return 0, err
-		}
-	}
-	return len(writes), nil
-}

internal/topdata/ruleset_migrate.go

@@ -1,171 +0,0 @@
-package topdata
-
-import (
-	"encoding/json"
-	"fmt"
-	"os"
-	"path/filepath"
-	"sort"
-)
-
-func importLegacyRuleset(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
-	_ = legacyTLK
-
-	legacyDir := filepath.Join(referenceBuilderDir, "data", "ruleset")
-	if _, err := os.Stat(legacyDir); err != nil {
-		if os.IsNotExist(err) {
-			return 0, nil
-		}
-		return 0, err
-	}
-
-	targetDir := filepath.Join(dataDir, "ruleset")
-	targetPath := filepath.Join(targetDir, "ruleset.json")
-	if _, err := os.Stat(targetPath); err == nil {
-		obj, err := loadJSONObject(targetPath)
-		if err == nil && countLegacyTLKRefsInValue(obj) == 0 {
-			return 0, nil
-		}
-	}
-
-	baseObj, err := loadJSONObject(filepath.Join(legacyDir, "base.json"))
-	if err != nil {
-		return 0, err
-	}
-	rows, err := mergeRulesetRows(baseObj, filepath.Join(legacyDir, "modules"))
-	if err != nil {
-		return 0, err
-	}
-
-	if err := os.MkdirAll(targetDir, 0o755); err != nil {
-		return 0, err
-	}
-	if err := removePathIfExists(filepath.Join(targetDir, "base.json")); err != nil {
-		return 0, err
-	}
-	if err := removePathIfExists(filepath.Join(targetDir, "lock.json")); err != nil {
-		return 0, err
-	}
-	if err := removePathIfExists(filepath.Join(targetDir, "modules")); err != nil {
-		return 0, err
-	}
-
-	out := map[string]any{
-		"output":  "ruleset.2da",
-		"columns": baseObj["columns"],
-		"rows":    rows,
-	}
-	raw, err := json.MarshalIndent(out, "", "  ")
-	if err != nil {
-		return 0, err
-	}
-	raw = append(raw, '\n')
-	if err := os.WriteFile(targetPath, raw, 0o644); err != nil {
-		return 0, err
-	}
-	return 1, nil
-}
-
-func mergeRulesetRows(baseObj map[string]any, modulesDir string) ([]any, error) {
-	rawRows, ok := baseObj["rows"].([]any)
-	if !ok {
-		return nil, nil
-	}
-
-	rows := make([]map[string]any, 0, len(rawRows))
-	byLabel := map[string]map[string]any{}
-	for _, raw := range rawRows {
-		row, ok := deepCopyValue(raw).(map[string]any)
-		if !ok {
-			continue
-		}
-		delete(row, "key")
-		if rawID, ok := row["id"]; ok {
-			id, err := asInt(rawID)
-			if err != nil {
-				return nil, err
-			}
-			row["id"] = id
-		}
-		if label, ok := row["Label"].(string); ok && label != "" && label != "****" {
-			if _, exists := byLabel[label]; exists {
-				return nil, fmt.Errorf("duplicate ruleset label %q", label)
-			}
-			byLabel[label] = row
-		}
-		rows = append(rows, row)
-	}
-
-	modulePaths, err := collectModulePaths(modulesDir)
-	if err != nil {
-		return nil, err
-	}
-	for _, path := range modulePaths {
-		obj, err := loadJSONObject(path)
-		if err != nil {
-			return nil, err
-		}
-		overrideList, ok := obj["overrides"].([]any)
-		if !ok {
-			continue
-		}
-		for _, raw := range overrideList {
-			override, ok := raw.(map[string]any)
-			if !ok {
-				continue
-			}
-			matchObj, ok := override["match"].(map[string]any)
-			if !ok {
-				return nil, fmt.Errorf("%s: ruleset override is missing match object", path)
-			}
-			rawLabel, ok := matchObj["Label"]
-			if !ok {
-				return nil, fmt.Errorf("%s: ruleset override match is missing Label", path)
-			}
-			labels, err := normalizeRulesetMatchLabels(rawLabel)
-			if err != nil {
-				return nil, fmt.Errorf("%s: %w", path, err)
-			}
-			for _, label := range labels {
-				row, ok := byLabel[label]
-				if !ok {
-					return nil, fmt.Errorf("%s: ruleset override target label %q was not found", path, label)
-				}
-				for key, value := range override {
-					if key == "id" || key == "key" || key == "_tlk" || isMetadataField(key) || key == "match" {
-						continue
-					}
-					row[key] = deepCopyValue(value)
-				}
-			}
-		}
-	}
-
-	sort.Slice(rows, func(i, j int) bool {
-		return rows[i]["id"].(int) < rows[j]["id"].(int)
-	})
-	out := make([]any, 0, len(rows))
-	for _, row := range rows {
-		out = append(out, row)
-	}
-	return out, nil
-}
-
-func normalizeRulesetMatchLabels(raw any) ([]string, error) {
-	switch typed := raw.(type) {
-	case string:
-		return []string{typed}, nil
-	case []any:
-		out := make([]string, 0, len(typed))
-		for _, item := range typed {
-			label, ok := item.(string)
-			if !ok {
-				return nil, fmt.Errorf("ruleset match.Label entries must be strings")
-			}
-			out = append(out, label)
-		}
-		return out, nil
-	default:
-		return nil, fmt.Errorf("ruleset match.Label must be a string or string array")
-	}
-}

internal/topdata/skills_migrate.go

@@ -1,140 +0,0 @@
-package topdata
-
-import (
-	"encoding/json"
-	"os"
-	"path/filepath"
-)
-
-func importLegacySkills(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
-	_ = legacyTLK
-
-	legacyDir := filepath.Join(referenceBuilderDir, "data", "skills")
-	if _, err := os.Stat(legacyDir); err != nil {
-		if os.IsNotExist(err) {
-			return 0, nil
-		}
-		return 0, err
-	}
-
-	targetDir := filepath.Join(dataDir, "skills")
-	targetBasePath := filepath.Join(targetDir, "base.json")
-	targetLockPath := filepath.Join(targetDir, "lock.json")
-	targetModulesDir := filepath.Join(targetDir, "modules")
-	moduleNames := []string{
-		"add_athletics.json",
-		"add_disguise.json",
-		"add_linguistics.json",
-		"add_perception.json",
-		"add_scriptcraft.json",
-		"add_sensemotive.json",
-		"add_stealth.json",
-		"add_survival.json",
-		"add_userope.json",
-		filepath.Join("craft", "add_craftalchemy.json"),
-		filepath.Join("craft", "add_craftcooking.json"),
-		filepath.Join("craft", "add_craftjewelry.json"),
-		filepath.Join("craft", "add_craftleatherworking.json"),
-		filepath.Join("craft", "add_craftstonework.json"),
-		filepath.Join("craft", "add_crafttextiles.json"),
-		filepath.Join("craft", "ovr_craftarmorsmithing.json"),
-		filepath.Join("craft", "ovr_crafttinkering.json"),
-		filepath.Join("craft", "ovr_craftweaponsmithing.json"),
-		filepath.Join("craft", "ovr_craftwoodworking.json"),
-		filepath.Join("knowledge", "add_knowledgearcana.json"),
-		filepath.Join("knowledge", "add_knowledgearchitecture.json"),
-		filepath.Join("knowledge", "add_knowledgedungeoneering.json"),
-		filepath.Join("knowledge", "add_knowledgegeography.json"),
-		filepath.Join("knowledge", "add_knowledgehistory.json"),
-		filepath.Join("knowledge", "add_knowledgelocal.json"),
-		filepath.Join("knowledge", "add_knowledgenature.json"),
-		filepath.Join("knowledge", "add_knowledgenobility.json"),
-		filepath.Join("knowledge", "add_knowledgeplanar.json"),
-		filepath.Join("knowledge", "add_knowledgereligion.json"),
-		"ovr_acrobatics.json",
-		"ovr_animalhandling.json",
-		"ovr_appraise.json",
-		"ovr_concentration.json",
-		"ovr_disabledevice.json",
-		"ovr_heal.json",
-		"ovr_hiddenskills.json",
-		"ovr_influence.json",
-		"ovr_openlock.json",
-		"ovr_parry.json",
-		"ovr_searchtowis.json",
-		"ovr_sleightofhand.json",
-		"ovr_taunttointimidate.json",
-		"rmv_removedskills.json",
-	}
-	targetModulePaths := make([]string, 0, len(moduleNames))
-	for _, name := range moduleNames {
-		targetModulePaths = append(targetModulePaths, filepath.Join(targetModulesDir, name))
-	}
-	if fileExists(targetBasePath) && fileExists(targetLockPath) {
-		allModulesPresent := true
-		for _, path := range targetModulePaths {
-			if !fileExists(path) {
-				allModulesPresent = false
-				break
-			}
-		}
-		if allModulesPresent {
-			return 0, nil
-		}
-	}
-
-	baseObj, err := loadJSONObject(filepath.Join(legacyDir, "base.json"))
-	if err != nil {
-		return 0, err
-	}
-	baseObj["output"] = "skills.2da"
-	canonicalizeWikiMetadataDocument(baseObj)
-
-	lockObj, err := loadJSONObject(filepath.Join(legacyDir, "lock.json"))
-	if err != nil {
-		return 0, err
-	}
-
-	moduleObjs := make([]map[string]any, 0, len(moduleNames))
-	for _, name := range moduleNames {
-		obj, err := loadJSONObject(filepath.Join(legacyDir, "modules", name))
-		if err != nil {
-			return 0, err
-		}
-		canonicalizeWikiMetadataDocument(obj)
-		moduleObjs = append(moduleObjs, obj)
-	}
-
-	if err := os.MkdirAll(targetModulesDir, 0o755); err != nil {
-		return 0, err
-	}
-
-	writes := []struct {
-		path string
-		obj  map[string]any
-	}{
-		{path: targetBasePath, obj: baseObj},
-		{path: targetLockPath, obj: lockObj},
-	}
-	for i, path := range targetModulePaths {
-		if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil {
-			return 0, err
-		}
-		writes = append(writes, struct {
-			path string
-			obj  map[string]any
-		}{path: path, obj: moduleObjs[i]})
-	}
-
-	for _, write := range writes {
-		raw, err := json.MarshalIndent(write.obj, "", "  ")
-		if err != nil {
-			return 0, err
-		}
-		raw = append(raw, '\n')
-		if err := os.WriteFile(write.path, raw, 0o644); err != nil {
-			return 0, err
-		}
-	}
-	return len(writes), nil
-}

internal/topdata/skyboxes_migrate.go

@@ -1,6 +0,0 @@
-package topdata
-
-func importLegacySkyboxes(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
-	_ = legacyTLK
-	return importLegacyDatasetMirror(referenceBuilderDir, dataDir, "skyboxes", "skyboxes.2da", nil)
-}

internal/topdata/spells_migrate.go

@@ -1,103 +0,0 @@
-package topdata
-
-import (
-	"encoding/json"
-	"os"
-	"path/filepath"
-)
-
-func importLegacySpells(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
-	_ = legacyTLK
-
-	legacyDir := filepath.Join(referenceBuilderDir, "data", "spells")
-	if _, err := os.Stat(legacyDir); err != nil {
-		if os.IsNotExist(err) {
-			return 0, nil
-		}
-		return 0, err
-	}
-
-	targetDir := filepath.Join(dataDir, "spells")
-	targetBasePath := filepath.Join(targetDir, "base.json")
-	targetLockPath := filepath.Join(targetDir, "lock.json")
-	targetModulesDir := filepath.Join(targetDir, "modules")
-	moduleNames := []string{
-		"specialattacks.json",
-		"ovr_fixduplicates.json",
-		filepath.Join("bardicmusic", "fascinate.json"),
-		filepath.Join("bardicmusic", "inspirecompetence.json"),
-		filepath.Join("bardicmusic", "inspirecourage.json"),
-		filepath.Join("bardicmusic", "inspiregreatness.json"),
-		filepath.Join("bardicmusic", "inspireheroics.json"),
-		filepath.Join("bardicmusic", "songoffreedom.json"),
-	}
-	targetModulePaths := make([]string, 0, len(moduleNames))
-	for _, name := range moduleNames {
-		targetModulePaths = append(targetModulePaths, filepath.Join(targetModulesDir, name))
-	}
-	if fileExists(targetBasePath) && fileExists(targetLockPath) {
-		allModulesPresent := true
-		for _, path := range targetModulePaths {
-			if !fileExists(path) {
-				allModulesPresent = false
-				break
-			}
-		}
-		if allModulesPresent {
-			return 0, nil
-		}
-	}
-
-	baseObj, err := loadJSONObject(filepath.Join(legacyDir, "base.json"))
-	if err != nil {
-		return 0, err
-	}
-	baseObj["output"] = "spells.2da"
-
-	lockObj, err := loadJSONObject(filepath.Join(legacyDir, "lock.json"))
-	if err != nil {
-		return 0, err
-	}
-
-	moduleObjs := make([]map[string]any, 0, len(moduleNames))
-	for _, name := range moduleNames {
-		obj, err := loadJSONObject(filepath.Join(legacyDir, "modules", name))
-		if err != nil {
-			return 0, err
-		}
-		moduleObjs = append(moduleObjs, obj)
-	}
-
-	if err := os.MkdirAll(targetModulesDir, 0o755); err != nil {
-		return 0, err
-	}
-
-	writes := []struct {
-		path string
-		obj  map[string]any
-	}{
-		{path: targetBasePath, obj: baseObj},
-		{path: targetLockPath, obj: lockObj},
-	}
-	for i, path := range targetModulePaths {
-		if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil {
-			return 0, err
-		}
-		writes = append(writes, struct {
-			path string
-			obj  map[string]any
-		}{path: path, obj: moduleObjs[i]})
-	}
-
-	for _, write := range writes {
-		raw, err := json.MarshalIndent(write.obj, "", "  ")
-		if err != nil {
-			return 0, err
-		}
-		raw = append(raw, '\n')
-		if err := os.WriteFile(write.path, raw, 0o644); err != nil {
-			return 0, err
-		}
-	}
-	return len(writes), nil
-}

internal/topdata/tailmodel_migrate.go

@@ -1,86 +0,0 @@
-package topdata
-
-import (
-	"encoding/json"
-	"os"
-	"path/filepath"
-)
-
-func importLegacyTailmodel(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
-	_ = legacyTLK
-
-	legacyDir := filepath.Join(referenceBuilderDir, "data", "tailmodel")
-	if _, err := os.Stat(legacyDir); err != nil {
-		if os.IsNotExist(err) {
-			return 0, nil
-		}
-		return 0, err
-	}
-
-	targetDir := filepath.Join(dataDir, "tailmodel")
-	targetBasePath := filepath.Join(targetDir, "base.json")
-	targetLockPath := filepath.Join(targetDir, "lock.json")
-	targetModulesDir := filepath.Join(targetDir, "modules")
-	targetModulePaths := []string{
-		filepath.Join(targetModulesDir, "add.json"),
-		filepath.Join(targetModulesDir, "ovr_tailprefixes.json"),
-	}
-	if fileExists(targetBasePath) && fileExists(targetLockPath) {
-		allModulesPresent := true
-		for _, path := range targetModulePaths {
-			if !fileExists(path) {
-				allModulesPresent = false
-				break
-			}
-		}
-		if allModulesPresent {
-			return 0, nil
-		}
-	}
-
-	baseObj, err := loadJSONObject(filepath.Join(legacyDir, "base.json"))
-	if err != nil {
-		return 0, err
-	}
-	baseObj["output"] = "tailmodel.2da"
-
-	lockObj, err := loadJSONObject(filepath.Join(legacyDir, "lock.json"))
-	if err != nil {
-		return 0, err
-	}
-
-	moduleNames := []string{"add.json", "ovr_tailprefixes.json"}
-	moduleObjs := make([]map[string]any, 0, len(moduleNames))
-	for _, name := range moduleNames {
-		obj, err := loadJSONObject(filepath.Join(legacyDir, "modules", name))
-		if err != nil {
-			return 0, err
-		}
-		moduleObjs = append(moduleObjs, obj)
-	}
-
-	if err := os.MkdirAll(targetModulesDir, 0o755); err != nil {
-		return 0, err
-	}
-
-	writes := []struct {
-		path string
-		obj  map[string]any
-	}{
-		{path: targetBasePath, obj: baseObj},
-		{path: targetLockPath, obj: lockObj},
-		{path: targetModulePaths[0], obj: moduleObjs[0]},
-		{path: targetModulePaths[1], obj: moduleObjs[1]},
-	}
-	for _, write := range writes {
-		raw, err := json.MarshalIndent(write.obj, "", "  ")
-		if err != nil {
-			return 0, err
-		}
-		raw = append(raw, '\n')
-		if err := os.WriteFile(write.path, raw, 0o644); err != nil {
-			return 0, err
-		}
-	}
-	return len(writes), nil
-}

internal/topdata/tlk_native.go

@@ -13,6 +13,7 @@ import (
 	"strings"
 
 	"golang.org/x/text/encoding/charmap"
+	"gopkg.in/yaml.v3"
 )
 
 const (
@@ -74,6 +75,16 @@ type tlkEntryData struct {
 	SoundLength float32
 }
 
+type standaloneTLKDocument struct {
+	Schema     string `yaml:"schema"`
+	BaseStrref int    `yaml:"base_strref"`
+	Strings    []struct {
+		Key  string `yaml:"key"`
+		Text string `yaml:"text"`
+		ID   *int   `yaml:"id"`
+	} `yaml:"strings"`
+}
+
 type tlkStateDocument struct {
 	Version  int                        `json:"version"`
 	Language string                     `json:"language"`
@@ -98,6 +109,7 @@ type tlkCompiler struct {
 	active       map[string]tlkEntryData
 	activeKeys   map[string]struct{}
 	reservedByID map[int]string
+	pinnedByID   map[int]string
 	nextID       int
 }
 
@@ -147,6 +159,7 @@ func newTLKCompiler(sourceDir string, legacy *legacyTLKData) (*tlkCompiler, erro
 		active:       map[string]tlkEntryData{},
 		activeKeys:   map[string]struct{}{},
 		reservedByID: reserved,
+		pinnedByID:   map[int]string{},
 		nextID:       nextID,
 	}
 	if legacy != nil {
@@ -157,6 +170,42 @@ func newTLKCompiler(sourceDir string, legacy *legacyTLKData) (*tlkCompiler, erro
 	return compiler, nil
 }
 
+func loadStandaloneTLKStrings(sourceDir string, compiler *tlkCompiler) error {
+	path := filepath.Join(sourceDir, "tlk", "custom.tlk.yml")
+	raw, err := os.ReadFile(path)
+	if err != nil {
+		if os.IsNotExist(err) {
+			return nil
+		}
+		return fmt.Errorf("read %s: %w", path, err)
+	}
+	var document standaloneTLKDocument
+	if err := yaml.Unmarshal(raw, &document); err != nil {
+		return fmt.Errorf("parse %s: %w", path, err)
+	}
+	if document.Schema != "sow-topdata/tlk/v1" {
+		return fmt.Errorf("%s: unsupported schema %q", path, document.Schema)
+	}
+	if document.BaseStrref != customTLKBase {
+		return fmt.Errorf("%s: base_strref must be %d", path, customTLKBase)
+	}
+	seen := map[string]struct{}{}
+	for index, entry := range document.Strings {
+		entry.Key = strings.TrimSpace(entry.Key)
+		if entry.Key == "" || entry.Text == "" || entry.ID == nil {
+			return fmt.Errorf("%s: strings[%d] requires key, text, and id", path, index)
+		}
+		if _, ok := seen[entry.Key]; ok {
+			return fmt.Errorf("%s: duplicate string key %q", path, entry.Key)
+		}
+		seen[entry.Key] = struct{}{}
+		if err := compiler.registerInlineAtID(entry.Key, *entry.ID, tlkEntryData{Text: entry.Text}); err != nil {
+			return err
+		}
+	}
+	return nil
+}
+
 func loadBaseDialogData(path string) (*legacyTLKData, error) {
 	if _, err := os.Stat(path); err != nil {
 		if os.IsNotExist(err) {
@@ -329,6 +378,33 @@ func (c *tlkCompiler) registerInline(key string, entry tlkEntryData) (tlkCompile
 	}, nil
 }
 
+func (c *tlkCompiler) registerInlineAtID(key string, id int, entry tlkEntryData) error {
+	if id < 0 {
+		return fmt.Errorf("TLK key %q has negative id %d", key, id)
+	}
+	// The pin is authoritative over the per-machine .tlk_state.json cache: a
+	// stale mapping that dynamically grabbed this id on an older build must
+	// yield so the pinned key can take it. Only a genuine clash between two
+	// pins in custom.tlk.yml is an author error.
+	if owner, ok := c.pinnedByID[id]; ok && owner != key {
+		return fmt.Errorf("TLK id %d is pinned by both %q and %q", id, owner, key)
+	}
+	if mapping, ok := c.state.Entries[key]; ok && mapping.ID != id {
+		// This key held a different cached id; release it so the pin wins.
+		if c.reservedByID[mapping.ID] == key {
+			delete(c.reservedByID, mapping.ID)
+		}
+	}
+	if owner, ok := c.reservedByID[id]; ok && owner != key {
+		// Evict the stale owner; it gets a fresh id when next made active.
+		delete(c.state.Entries, owner)
+	}
+	c.pinnedByID[id] = key
+	c.state.Entries[key] = tlkStateMapping{ID: id}
+	c.reservedByID[id] = key
+	return c.markActive(key, entry)
+}
+
 func (c *tlkCompiler) customStrrefForKey(key string) int {
 	return customTLKBase + c.state.Entries[key].ID
 }
@@ -338,6 +414,7 @@ func (c *tlkCompiler) markActive(key string, entry tlkEntryData) error {
 	if !ok {
 		mapping = tlkStateMapping{ID: c.allocateID(), Retired: false}
 		c.state.Entries[key] = mapping
+		c.reservedByID[mapping.ID] = key
 	}
 	existing, ok := c.active[key]
 	if ok && existing != entry {
@@ -351,9 +428,13 @@ func (c *tlkCompiler) markActive(key string, entry tlkEntryData) error {
 }
 
 func (c *tlkCompiler) allocateID() int {
+	for {
 		id := c.nextID
 		c.nextID++
+		if _, reserved := c.reservedByID[id]; !reserved {
 			return id
+		}
+	}
 }
 
 func (c *tlkCompiler) finish(outputDir, tlkName string) (int, error) {
@@ -727,3 +808,36 @@ func sortedKeys[M ~map[string]V, V any](input M) []string {
 func almostEqualFloat32(a, b float32) bool {
 	return math.Abs(float64(a-b)) < 0.0001
 }
+func mergeLegacyTLK(base, override *legacyTLKData) *legacyTLKData {
+	if base == nil && override == nil {
+		return nil
+	}
+	merged := &legacyTLKData{
+		Language: "en",
+		Entries:  map[string]tlkEntryData{},
+		Lock:     map[string]int{},
+	}
+	if base != nil {
+		if base.Language != "" {
+			merged.Language = base.Language
+		}
+		for key, entry := range base.Entries {
+			merged.Entries[key] = entry
+		}
+		for key, id := range base.Lock {
+			merged.Lock[key] = id
+		}
+	}
+	if override != nil {
+		if override.Language != "" {
+			merged.Language = override.Language
+		}
+		for key, entry := range override.Entries {
+			merged.Entries[key] = entry
+		}
+		for key, id := range override.Lock {
+			merged.Lock[key] = id
+		}
+	}
+	return merged
+}

internal/topdata/top_package.go

@@ -2,6 +2,7 @@ package topdata
 
 import (
 	"bytes"
+	"encoding/json"
 	"errors"
 	"fmt"
 	"io"
@@ -13,6 +14,7 @@ import (
 	"time"
 
 	"git.westgate.pw/ShadowsOverWestgate/sow-tools/internal/erf"
+	"git.westgate.pw/ShadowsOverWestgate/sow-tools/internal/gff"
 	"git.westgate.pw/ShadowsOverWestgate/sow-tools/internal/project"
 )
 
@@ -215,10 +217,15 @@ func collectTopPackageResources(p *project.Project, compiled2DADir string) ([]er
 			if err != nil {
 				return err
 			}
+			var resource erf.Resource
+			if strings.HasSuffix(strings.ToLower(rel), ".itp.json") {
+				resource, err = topPackageITPResourceFromJSON(path)
+			} else {
 				if skipTopPackageAsset(rel) {
 					return nil
 				}
-			resource, err := topPackageResourceFromPath(path)
+				resource, err = topPackageResourceFromPath(path)
+			}
 			if err != nil {
 				return err
 			}
@@ -251,6 +258,27 @@ func collectTopPackageResources(p *project.Project, compiled2DADir string) ([]er
 	return resources, assetFiles, nil
 }
 
+func topPackageITPResourceFromJSON(path string) (erf.Resource, error) {
+	raw, err := os.ReadFile(path)
+	if err != nil {
+		return erf.Resource{}, fmt.Errorf("read %s: %w", path, err)
+	}
+	var document gff.Document
+	if err := json.Unmarshal(raw, &document); err != nil {
+		return erf.Resource{}, fmt.Errorf("parse %s: %w", path, err)
+	}
+	if document.FileType != "ITP " {
+		return erf.Resource{}, fmt.Errorf("%s: file_type must be ITP", path)
+	}
+	var payload bytes.Buffer
+	if err := gff.Write(&payload, document); err != nil {
+		return erf.Resource{}, fmt.Errorf("compile %s: %w", path, err)
+	}
+	resourceType, _ := erf.HAKResourceTypeForExtension("itp")
+	name := strings.TrimSuffix(filepath.Base(path), ".itp.json")
+	return erf.Resource{Name: strings.ToLower(name), Type: resourceType, Data: payload.Bytes(), Size: int64(payload.Len())}, nil
+}
+
 func shouldSkipTopDataSourceDir(path string, skipDirs map[string]struct{}) bool {
 	_, ok := skipDirs[filepath.Clean(path)]
 	return ok

internal/topdata/vfx_persistent_migrate.go

@@ -1,198 +0,0 @@
-package topdata
-
-import (
-	"os"
-	"path/filepath"
-	"strings"
-)
-
-func importLegacyVFXPersistent(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
-	_ = legacyTLK
-	if strings.TrimSpace(referenceBuilderDir) == "" {
-		return 0, nil
-	}
-
-	legacyDir := filepath.Join(referenceBuilderDir, "data", "vfx_persistent")
-	if _, err := os.Stat(legacyDir); err != nil {
-		if os.IsNotExist(err) {
-			return 0, nil
-		}
-		return 0, err
-	}
-
-	targetDir := filepath.Join(dataDir, "vfx_persistent")
-	targetBasePath := filepath.Join(targetDir, "base.json")
-	targetLockPath := filepath.Join(targetDir, "lock.json")
-	targetModulesDir := filepath.Join(targetDir, "modules")
-	targetModulePath := filepath.Join(targetModulesDir, "freeformaoes.json")
-	if fileExists(targetBasePath) && fileExists(targetLockPath) && fileExists(targetModulePath) {
-		baseObj, baseErr := loadJSONObject(targetBasePath)
-		lockObj, lockErr := loadJSONObject(targetLockPath)
-		if baseErr == nil && lockErr == nil && vfxPersistentImportComplete(baseObj, lockObj) {
-			legacyModulePaths, err := collectJSONRelativePaths(filepath.Join(legacyDir, "modules"))
-			if err == nil {
-				targetModulePaths, err := collectJSONRelativePaths(targetModulesDir)
-				if err == nil && equalStringSlices(legacyModulePaths, targetModulePaths) {
-					return 0, nil
-				}
-			}
-		}
-	}
-
-	baseObj, err := loadJSONObject(filepath.Join(legacyDir, "base.json"))
-	if err != nil {
-		return 0, err
-	}
-	lockObj, err := synthesizeVFXPersistentKeys(baseObj)
-	if err != nil {
-		return 0, err
-	}
-	legacyLockObj, err := loadJSONObject(filepath.Join(legacyDir, "lock.json"))
-	if err != nil {
-		return 0, err
-	}
-	for key, value := range legacyLockObj {
-		lockObj[key] = value
-	}
-	if err := os.MkdirAll(targetModulesDir, 0o755); err != nil {
-		return 0, err
-	}
-
-	updated := 0
-	for _, write := range []struct {
-		path string
-		obj  map[string]any
-	}{
-		{path: targetBasePath, obj: baseObj},
-		{path: targetLockPath, obj: lockObj},
-	} {
-		changed, err := writeJSONObjectIfChanged(write.path, write.obj)
-		if err != nil {
-			return 0, err
-		}
-		if changed {
-			updated++
-		}
-	}
-
-	moduleRelPaths, err := collectJSONRelativePaths(filepath.Join(legacyDir, "modules"))
-	if err != nil {
-		return 0, err
-	}
-	for _, relPath := range moduleRelPaths {
-		moduleObj, err := loadJSONObject(filepath.Join(legacyDir, "modules", filepath.FromSlash(relPath)))
-		if err != nil {
-			return 0, err
-		}
-		targetPath := filepath.Join(targetModulesDir, filepath.FromSlash(relPath))
-		if err := os.MkdirAll(filepath.Dir(targetPath), 0o755); err != nil {
-			return 0, err
-		}
-		changed, err := writeJSONObjectIfChanged(targetPath, moduleObj)
-		if err != nil {
-			return 0, err
-		}
-		if changed {
-			updated++
-		}
-	}
-
-	targetModulePaths, err := collectJSONRelativePaths(targetModulesDir)
-	if err != nil {
-		return 0, err
-	}
-	legacyModules := make(map[string]struct{}, len(moduleRelPaths))
-	for _, relPath := range moduleRelPaths {
-		legacyModules[relPath] = struct{}{}
-	}
-	for _, relPath := range targetModulePaths {
-		if _, ok := legacyModules[relPath]; ok {
-			continue
-		}
-		if err := os.Remove(filepath.Join(targetModulesDir, filepath.FromSlash(relPath))); err != nil && !os.IsNotExist(err) {
-			return 0, err
-		}
-		updated++
-	}
-
-	return updated, nil
-}
-
-func vfxPersistentImportComplete(baseObj, lockObj map[string]any) bool {
-	if !vfxPersistentRowsHaveKeys(baseObj) {
-		return false
-	}
-	expected := map[string]int{
-		"vfx_persistent:blankradius5":  47,
-		"vfx_persistent:blankradius10": 48,
-		"vfx_persistent:blankradius15": 49,
-		"vfx_persistent:blankradius20": 50,
-		"vfx_persistent:blankradius25": 51,
-		"vfx_persistent:blankradius30": 52,
-	}
-	for key, want := range expected {
-		raw, ok := lockObj[key]
-		if !ok {
-			return false
-		}
-		got, err := asInt(raw)
-		if err != nil || got != want {
-			return false
-		}
-	}
-	return true
-}
-
-func synthesizeVFXPersistentKeys(baseObj map[string]any) (map[string]any, error) {
-	rawRows, ok := baseObj["rows"].([]any)
-	if !ok {
-		return map[string]any{}, nil
-	}
-	lockObj := map[string]any{}
-	for _, raw := range rawRows {
-		row, ok := raw.(map[string]any)
-		if !ok {
-			continue
-		}
-		label, _ := row["LABEL"].(string)
-		if label == "" {
-			continue
-		}
-		key := "vfx_persistent:" + strings.ToLower(label)
-		row["key"] = key
-		if rawID, ok := row["id"]; ok {
-			lockObj[key] = rawID
-		}
-	}
-	return lockObj, nil
-}
-
-func vfxPersistentRowsHaveKeys(baseObj map[string]any) bool {
-	rawRows, ok := baseObj["rows"].([]any)
-	if !ok {
-		return false
-	}
-	for _, raw := range rawRows {
-		row, ok := raw.(map[string]any)
-		if !ok {
-			return false
-		}
-		key, _ := row["key"].(string)
-		if key == "" {
-			return false
-		}
-	}
-	return true
-}
-
-func equalStringSlices(left, right []string) bool {
-	if len(left) != len(right) {
-		return false
-	}
-	for i := range left {
-		if left[i] != right[i] {
-			return false
-		}
-	}
-	return true
-}

internal/topdata/visualeffects_migrate.go

@@ -1,6 +0,0 @@
-package topdata
-
-func importLegacyVisualeffects(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
-	_ = legacyTLK
-	return importLegacyDatasetMirror(referenceBuilderDir, dataDir, "visualeffects", "visualeffects.2da", nil)
-}

internal/topdata/wingmodel_migrate.go

@@ -1,86 +0,0 @@
-package topdata
-
-import (
-	"encoding/json"
-	"os"
-	"path/filepath"
-)
-
-func importLegacyWingmodel(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
-	_ = legacyTLK
-
-	legacyDir := filepath.Join(referenceBuilderDir, "data", "wingmodel")
-	if _, err := os.Stat(legacyDir); err != nil {
-		if os.IsNotExist(err) {
-			return 0, nil
-		}
-		return 0, err
-	}
-
-	targetDir := filepath.Join(dataDir, "wingmodel")
-	targetBasePath := filepath.Join(targetDir, "base.json")
-	targetLockPath := filepath.Join(targetDir, "lock.json")
-	targetModulesDir := filepath.Join(targetDir, "modules")
-	targetModulePaths := []string{
-		filepath.Join(targetModulesDir, "add.json"),
-		filepath.Join(targetModulesDir, "ovr_wingprefixes.json"),
-	}
-	if fileExists(targetBasePath) && fileExists(targetLockPath) {
-		allModulesPresent := true
-		for _, path := range targetModulePaths {
-			if !fileExists(path) {
-				allModulesPresent = false
-				break
-			}
-		}
-		if allModulesPresent {
-			return 0, nil
-		}
-	}
-
-	baseObj, err := loadJSONObject(filepath.Join(legacyDir, "base.json"))
-	if err != nil {
-		return 0, err
-	}
-	baseObj["output"] = "wingmodel.2da"
-
-	lockObj, err := loadJSONObject(filepath.Join(legacyDir, "lock.json"))
-	if err != nil {
-		return 0, err
-	}
-
-	moduleNames := []string{"add.json", "ovr_wingprefixes.json"}
-	moduleObjs := make([]map[string]any, 0, len(moduleNames))
-	for _, name := range moduleNames {
-		obj, err := loadJSONObject(filepath.Join(legacyDir, "modules", name))
-		if err != nil {
-			return 0, err
-		}
-		moduleObjs = append(moduleObjs, obj)
-	}
-
-	if err := os.MkdirAll(targetModulesDir, 0o755); err != nil {
-		return 0, err
-	}
-
-	writes := []struct {
-		path string
-		obj  map[string]any
-	}{
-		{path: targetBasePath, obj: baseObj},
-		{path: targetLockPath, obj: lockObj},
-		{path: targetModulePaths[0], obj: moduleObjs[0]},
-		{path: targetModulePaths[1], obj: moduleObjs[1]},
-	}
-	for _, write := range writes {
-		raw, err := json.MarshalIndent(write.obj, "", "  ")
-		if err != nil {
-			return 0, err
-		}
-		raw = append(raw, '\n')
-		if err := os.WriteFile(write.path, raw, 0o644); err != nil {
-			return 0, err
-		}
-	}
-	return len(writes), nil
-}

tests/workflow-contract.sh

@@ -0,0 +1,84 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+contract_script="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/$(basename "${BASH_SOURCE[0]}")"
+default_root="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
+repo_root="${WORKFLOW_ROOT:-$default_root}"
+cd "$repo_root"
+
+ci=.gitea/workflows/ci.yml
+release=.gitea/workflows/build-binaries.yml
+sync=.gitea/workflows/sync-wrappers.yml
+checkout='actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683'
+
+assert_exact_events() {
+  local workflow="$1" expected="$2" actual
+  actual="$(awk '
+    /^on:$/ { in_on = 1 }
+    in_on && /^$/ { next }
+    in_on && $0 != "on:" && $0 !~ /^[[:space:]]/ { exit }
+    in_on { print }
+  ' "$workflow")"
+  [[ "$actual" == "$expected" ]] || {
+    echo "workflow-contract: unexpected event scope in $workflow" >&2
+    return 1
+  }
+}
+
+[[ -f "$ci" ]] || { echo "missing consolidated CI workflow" >&2; exit 1; }
+[[ ! -e .gitea/workflows/test.yml ]] || { echo "legacy test workflow still present" >&2; exit 1; }
+
+assert_exact_events "$ci" $'on:\n  pull_request:'
+grep -Fqx 'permissions: read-all' "$ci"
+grep -Fqx '    timeout-minutes: 60' "$ci"
+[[ "$(grep -Fc "$checkout" "$ci")" -eq 1 ]]
+grep -Fqx '          fetch-depth: 0' "$ci"
+grep -Fq 'go vet ./...' "$ci"
+grep -Fq 'go test ./...' "$ci"
+grep -Fq 'shellcheck scripts/*.sh' "$ci"
+grep -Fq 'yamllint .gitea' "$ci"
+grep -Fq 'make smoke' "$ci"
+grep -Fq 'for target in linux/amd64 linux/arm64 darwin/amd64 darwin/arm64 windows/amd64 windows/arm64; do' "$ci"
+
+assert_exact_events "$release" $'on:\n  push:\n    tags: [\'v*\']'
+grep -Fqx '  code: read' "$release"
+grep -Fqx '  releases: write' "$release"
+grep -Fqx '    timeout-minutes: 30' "$release"
+[[ "$(grep -Fc "$checkout" "$release")" -eq 1 ]]
+grep -Fq 'secrets.GITEA_TOKEN' "$release"
+if grep -Fq 'secrets.GITHUB_TOKEN' "$release"; then
+  echo "workflow-contract: release uses the GitHub token alias" >&2
+  exit 1
+fi
+
+grep -Fqx 'permissions: read-all' "$sync"
+grep -Fqx '    timeout-minutes: 30' "$sync"
+[[ "$(grep -Fc "$checkout" "$sync")" -eq 1 ]]
+assert_exact_events "$sync" $'on:\n  push:\n    branches: [main]\n    paths:\n      - \'wrappers/crucible.sh\'\n      - \'wrappers/crucible.ps1\''
+
+expect_mutation_rejected() {
+  local name="$1" workflow="$2" expression="$3" tmp
+  tmp="$(mktemp -d)"
+  mkdir -p "$tmp/.gitea"
+  cp -R "$default_root/.gitea/workflows" "$tmp/.gitea/workflows"
+  sed -i "$expression" "$tmp/$workflow"
+  if WORKFLOW_ROOT="$tmp" WORKFLOW_CONTRACT_MUTATION=1 bash "$contract_script" >/dev/null 2>&1; then
+    echo "workflow-contract: accepted forbidden mutation: $name" >&2
+    rm -rf "$tmp"
+    return 1
+  fi
+  rm -rf "$tmp"
+}
+
+if [[ "${WORKFLOW_CONTRACT_MUTATION:-0}" != 1 ]]; then
+  mutations_ok=0
+  expect_mutation_rejected 'CI workflow_dispatch event' "$ci" '/^  pull_request:$/a\  workflow_dispatch:' || mutations_ok=1
+  expect_mutation_rejected 'release develop branch' "$release" "/^    tags:/a\\    branches: [develop]" || mutations_ok=1
+  expect_mutation_rejected 'release schedule event' "$release" "/^    tags:/a\\  schedule:\n    - cron: '0 0 * * *'" || mutations_ok=1
+  expect_mutation_rejected 'release extra tag pattern' "$release" "s/tags: \['v\*'\]/tags: ['v*', 'release-*']/" || mutations_ok=1
+  expect_mutation_rejected 'wrapper sync pull_request event' "$sync" '/^  push:$/i\  pull_request:' || mutations_ok=1
+  expect_mutation_rejected 'wrapper sync broad path' "$sync" "/^      - 'wrappers\/crucible.ps1'$/a\\      - 'wrappers/**'" || mutations_ok=1
+  (( mutations_ok == 0 )) || exit 1
+fi
+
+echo "workflow-contract: CI is PR-only; release and wrapper sync retain their narrow triggers"