Compare commits

..
4 Commits
Author SHA1 Message Date
archvillainette ff9424c8c3 custom palette taxonomy
ci / ci (pull_request) Successful in 3m11s
2026-07-22 16:54:56 +00:00
archvillainette b058846e16 chore(ci): consolidate Gitea CI (#47)
## Summary

- consolidate vet, tests, lint, smoke, and cross-build into one pull-request-only job
- keep binary releases strictly v-tag-only with release-only write permission
- retain wrapper synchronization only for canonical wrapper changes on main
- pin actions and enforce exact trigger contracts

## Verification

- nix develop --command make check
- trigger mutation contract
- 28-workflow cross-repository actionlint audit

Generated with Claude CodeReviewed-on: #47

Co-authored-by: vickydotbat <vickydotbat@tutamail.com>
2026-07-22 16:54:26 +00:00
archvillainette e509b7a90a Remove completed legacy-migration cluster from topdata (#45)
test / test (push) Successful in 1m16s
build-binaries / build-binaries (push) Successful in 2m7s
The legacy normalize migration is done (confirmed). This deletes the dead migration path: NormalizeProject + 26 *_migrate.go files, the importLegacyItempropsRegistry cluster, loadLegacyTLK, and their tests (~7.9k lines, -33 files worth). mergeLegacyTLK moved into tlk_native.go since the live TLK compiler still uses it for base_dialog.json. make check passes.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Reviewed-on: #45
Reviewed-by: gitea-bot <gitea-bot@noreply.git.westgate.pw>
Co-authored-by: vickydotbat <vickydotbat@tutamail.com>
Co-committed-by: vickydotbat <vickydotbat@tutamail.com>
2026-07-19 08:55:24 +00:00
archvillainette 0d12674838 skills: add data-driven-design (#44)
build-binaries / build-binaries (push) Successful in 2m7s
test / test (push) Successful in 1m24s
Adds .agents/skills/data-driven-design: general mechanisms in code, all specific knowledge in data. Written for the long-term cleanup that would let sow-tools be open sourced.

Grounded in two real failures: the value_encodings mode zoo (per-column mode enums + switch arms where a shape/encoding declaration in data would do) and the skills dataset rename (code hardcoding dataset names/key prefixes that the data already knows).

Tested per TDD-for-skills: a baseline subagent given the EquipSlots scenario added a fourth mode and rationalized it ("three switch cases isn't a pattern crisis"); with the skill loaded it declared shape in data, collapsed the modes into orthogonal primitives, and deleted the switch.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Reviewed-on: #44
Co-authored-by: vickydotbat <vickydotbat@tutamail.com>
Co-committed-by: vickydotbat <vickydotbat@tutamail.com>
2026-07-19 07:46:49 +00:00
41 changed files with 450 additions and 8011 deletions
@@ -0,0 +1,55 @@
---
name: data-driven-design
description: Use when adding a mode, switch case, config entry, special-cased name, or per-column/per-dataset handler to a tool that processes authored data — or when a tool "knows" a project's layout, dataset names, key prefixes, or format quirks and could not be open sourced as-is.
---
# Data-Driven Design
The tool implements **general mechanisms**; the data declares **all specific knowledge**. Every time code names a thing that lives in data — a dataset, a column, a format quirk — the tool stops being a tool and becomes a private extension of one project's layout.
**The test:** could this repo be open sourced and build a *different* project's data without code changes? Every hardcoded name, mode, and special case is a "no".
## The two smells
**1. The mode zoo.** A new data shape arrives and you add a named mode plus a switch case:
```yaml
- {column: AvailableHeadsMale, mode: external_hex_list, hex_width: 3}
- {column: PreferredAlignments, mode: alignment_set_mask, hex_width: 3}
- {column: ProficiencyFeats, mode: id_hex_list, hex_width: 4}
```
Three "modes" are one concept — *a set of ids with an engine-side encoding* — wearing three names. Each mode is code the data could have been. The fix is a smaller vocabulary of orthogonal primitives declared where the column is defined:
```json
"EquipSlots": {"shape": "id_set", "encode": "bitmask", "max": 31}
```
One parser for `id_set`, one encoder per encoding. Adding the next column is a data edit, zero code. The mode zoo *shrinks*: N special cases collapse into shape × encoding.
**2. Names in code.** `datasetRows("skills")`, `case "skills":`, `TrimPrefix(key, "skills:")` — the tool assumes the project's layout. When data moves (`skills``skills/core`), the tool breaks even though the data is self-consistent. Names must flow *from* the data: the spec that references a dataset names it; a key's namespace is whatever precedes its own colon. If code needs a name, some piece of data already knows it — read it from there.
## Deciding
Ask of every constant, case, and config knob: **whose knowledge is this?**
- *The mechanism's* (how to parse JSON, allocate rows, write a table) → code.
- *The data's* (which columns exist, how the engine encodes them, what things are called) → data, even if code is faster to write today.
Generalize only when it deletes: a primitive that replaces N modes is lazy; a framework "for future shapes" is not. One new shape may take its case *if* the case is spelled as a reusable primitive the next shape can declare.
## Rationalizations
| Excuse | Reality |
|---|---|
| "Just follow the existing pattern, it's proven" | The pattern *is* the defect. Each repetition raises the cost of the real fix. |
| "N cases isn't a crisis yet" | The crisis is per-case: every one couples the tool to one project forever. |
| "Release is tonight, generalize later" | Later never comes; tonight's mode is tomorrow's baseline. Declaring shape in data is usually the *same* hour of work. |
| "This name never changes" | Today's rename broke exactly such a name. Data moves; mechanisms shouldn't care. |
## Red flags — stop and re-shape
- A config entry that names both a dataset and a column to select behavior
- A new value in a `mode`/`kind`/`type` string enum with its own switch arm
- A literal in code that also appears in the data tree (`"skills"`, `"skills:"`)
- Per-project constants (row ranges, hex widths, id ceilings) living in Go instead of the dataset that owns them
+9 -13
View File
@@ -1,25 +1,22 @@
# Cross-platform Crucible binaries (D7 trigger standard):
# PR / push main -> cross-build ALL targets to prove they compile (no publish)
# tag v* -> build all targets + SHA256SUMS, upload to the Gitea release
# Cross-platform Crucible release binaries: a v* tag builds all targets,
# then uploads them, SHA256SUMS, and the canonical wrappers to its Gitea release.
# Crucible is pure Go (CGO_ENABLED=0), so cross-compiling is a fast loop.
name: build-binaries
on:
pull_request:
push:
branches: [main]
tags: ['v*']
paths-ignore:
- "docs/**"
- "README.md"
- "AGENTS.md"
- "LICENSE"
permissions:
code: read
releases: write
jobs:
build-binaries:
runs-on: nix-docker
timeout-minutes: 30
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with: { fetch-depth: 0 }
- name: Cross-build all targets
@@ -45,9 +42,8 @@ jobs:
'
- name: Upload to Gitea release
if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags/v')
env:
TOKEN: ${{ secrets.GITHUB_TOKEN }}
TOKEN: ${{ secrets.GITEA_TOKEN }}
TAG: ${{ github.ref_name }}
REPO: ${{ github.repository }}
SERVER: ${{ github.server_url }}
+47
View File
@@ -0,0 +1,47 @@
# Pull-request validation for Crucible. Releases and wrapper synchronization
# have their own narrow workflows because they need tag/main events.
name: ci
on:
pull_request:
permissions: read-all
jobs:
ci:
runs-on: nix-docker
timeout-minutes: 60
steps:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
fetch-depth: 0
- name: vet + test + lint
run: |
nix develop --command bash -c '
set -euo pipefail
go vet ./...
go test ./...
shellcheck scripts/*.sh
yamllint .gitea
'
- name: binary smoke (fail-closed contract)
run: nix develop --command make smoke
- name: Cross-build all targets
run: |
nix develop --command bash -c '
set -euo pipefail
sha="$(git rev-parse --short=12 HEAD)"
ldflags="-s -w -X git.westgate.pw/ShadowsOverWestgate/sow-tools/internal/buildinfo.Version=${sha}"
rm -rf dist && mkdir -p dist
export CGO_ENABLED=0
for target in linux/amd64 linux/arm64 darwin/amd64 darwin/arm64 windows/amd64 windows/arm64; do
os="${target%/*}"; arch="${target#*/}"
ext=""; [ "$os" = windows ] && ext=".exe"
echo "building crucible-${os}-${arch}${ext}"
GOOS="$os" GOARCH="$arch" go build -trimpath -ldflags "$ldflags" \
-o "dist/crucible-${os}-${arch}${ext}" ./cmd/crucible
done
'
+4 -1
View File
@@ -14,11 +14,14 @@ on:
- 'wrappers/crucible.sh'
- 'wrappers/crucible.ps1'
permissions: read-all
jobs:
sync:
runs-on: nix-docker
timeout-minutes: 30
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- name: Open sync PRs to consumers
env:
-30
View File
@@ -1,30 +0,0 @@
# Lint + unit tests for the Crucible suite. PR-first: PRs + push to main (D7).
name: test
on:
push:
branches: [main]
paths-ignore:
- "docs/**"
- "README.md"
- "AGENTS.md"
- "LICENSE"
pull_request:
jobs:
test:
runs-on: nix-docker
steps:
- uses: actions/checkout@v4
with: { fetch-depth: 0 }
- name: vet + test + lint
run: |
nix develop --command bash -c '
set -euo pipefail
go vet ./...
go test ./...
shellcheck scripts/*.sh
yamllint .gitea
'
- name: binary smoke (fail-closed contract)
run: nix develop --command make smoke
+1
View File
@@ -6,6 +6,7 @@ SHELL := bash
check: vet test
shellcheck scripts/*.sh
yamllint .gitea
bash tests/workflow-contract.sh
vet:
go vet ./...
+6 -5
View File
@@ -78,13 +78,14 @@ This retires the old habit of checking in `nwn-tool` / `sow-toolkit`.
## CI
PR-first (D7): checks run on pull requests and on push to `main`; the only
publish event is a `v*` tag (see `runbooks/ci-trigger-standard.md` in sow-docs,
PR-first (D7): checks run once on pull requests; the only publish event is a
`v*` tag (see `runbooks/ci-trigger-standard.md` in sow-docs,
https://git.westgate.pw/ShadowsOverWestgate/sow-docs).
- `test.yml` — vet, test, shellcheck, yamllint, binary smoke (PR + main).
- `build-binaries.yml` — cross-build all targets (PR + main); on a `v*` tag, upload
the binaries, `SHA256SUMS`, and the wrappers to the Gitea release.
- `ci.yml` — vet, test, shellcheck, yamllint, binary smoke, and cross-build all
targets once per pull request.
- `build-binaries.yml` — on a `v*` tag, cross-build and upload the binaries,
`SHA256SUMS`, and the wrappers to the Gitea release.
- `sync-wrappers.yml` — on a `main` push that touches `wrappers/`, auto-PR the
canonical wrappers to the consumer repos in `wrappers/consumers.txt`.
Consumer drift checks run after those PRs merge to `main`, not on the PRs
-96
View File
@@ -1,96 +0,0 @@
package topdata
import (
"encoding/json"
"os"
"path/filepath"
)
func importLegacyAppearance(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
_ = legacyTLK
legacyDir := filepath.Join(referenceBuilderDir, "data", "appearance")
if _, err := os.Stat(legacyDir); err != nil {
if os.IsNotExist(err) {
return 0, nil
}
return 0, err
}
targetDir := filepath.Join(dataDir, "appearance")
targetBasePath := filepath.Join(targetDir, "base.json")
targetLockPath := filepath.Join(targetDir, "lock.json")
targetModulesDir := filepath.Join(targetDir, "modules")
moduleNames := []string{
"cotblreaver.json",
"crawlingclaw.json",
"halfogre.json",
"zombieknight.json",
}
targetModulePaths := make([]string, 0, len(moduleNames))
for _, name := range moduleNames {
targetModulePaths = append(targetModulePaths, filepath.Join(targetModulesDir, name))
}
if fileExists(targetBasePath) && fileExists(targetLockPath) {
allModulesPresent := true
for _, path := range targetModulePaths {
if !fileExists(path) {
allModulesPresent = false
break
}
}
if allModulesPresent {
return 0, nil
}
}
baseObj, err := loadJSONObject(filepath.Join(legacyDir, "base.json"))
if err != nil {
return 0, err
}
baseObj["output"] = "appearance.2da"
lockObj, err := loadJSONObject(filepath.Join(legacyDir, "lock.json"))
if err != nil {
return 0, err
}
moduleObjs := make([]map[string]any, 0, len(moduleNames))
for _, name := range moduleNames {
obj, err := loadJSONObject(filepath.Join(legacyDir, "modules", name))
if err != nil {
return 0, err
}
moduleObjs = append(moduleObjs, obj)
}
if err := os.MkdirAll(targetModulesDir, 0o755); err != nil {
return 0, err
}
writes := []struct {
path string
obj map[string]any
}{
{path: targetBasePath, obj: baseObj},
{path: targetLockPath, obj: lockObj},
}
for i, path := range targetModulePaths {
writes = append(writes, struct {
path string
obj map[string]any
}{path: path, obj: moduleObjs[i]})
}
for _, write := range writes {
raw, err := json.MarshalIndent(write.obj, "", " ")
if err != nil {
return 0, err
}
raw = append(raw, '\n')
if err := os.WriteFile(write.path, raw, 0o644); err != nil {
return 0, err
}
}
return len(writes), nil
}
-160
View File
@@ -1,160 +0,0 @@
package topdata
import (
"encoding/json"
"os"
"path/filepath"
"sort"
)
func importLegacyArmor(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
_ = legacyTLK
legacyDir := filepath.Join(referenceBuilderDir, "data", "armor")
if _, err := os.Stat(legacyDir); err != nil {
if os.IsNotExist(err) {
return 0, nil
}
return 0, err
}
targetDir := filepath.Join(dataDir, "armor")
targetPath := filepath.Join(targetDir, "armor.json")
if _, err := os.Stat(targetPath); err == nil {
obj, err := loadJSONObject(targetPath)
if err == nil && countLegacyTLKRefsInValue(obj) == 0 {
return 0, nil
}
}
baseObj, err := loadJSONObject(filepath.Join(legacyDir, "base.json"))
if err != nil {
return 0, err
}
rows, err := mergeArmorRows(baseObj, filepath.Join(legacyDir, "modules"))
if err != nil {
return 0, err
}
if err := os.MkdirAll(targetDir, 0o755); err != nil {
return 0, err
}
if err := removePathIfExists(filepath.Join(targetDir, "base.json")); err != nil {
return 0, err
}
if err := removePathIfExists(filepath.Join(targetDir, "lock.json")); err != nil {
return 0, err
}
if err := removePathIfExists(filepath.Join(targetDir, "modules")); err != nil {
return 0, err
}
out := map[string]any{
"output": "armor.2da",
"columns": baseObj["columns"],
"rows": rows,
}
raw, err := json.MarshalIndent(out, "", " ")
if err != nil {
return 0, err
}
raw = append(raw, '\n')
if err := os.WriteFile(targetPath, raw, 0o644); err != nil {
return 0, err
}
return 1, nil
}
func mergeArmorRows(baseObj map[string]any, modulesDir string) ([]any, error) {
rawRows, ok := baseObj["rows"].([]any)
if !ok {
return nil, nil
}
rows := make([]map[string]any, 0, len(rawRows))
byID := map[int]map[string]any{}
for _, raw := range rawRows {
row, ok := deepCopyValue(raw).(map[string]any)
if !ok {
continue
}
delete(row, "key")
rawID, ok := row["id"]
if !ok {
continue
}
id, err := asInt(rawID)
if err != nil {
return nil, err
}
row["id"] = id
rows = append(rows, row)
byID[id] = row
}
modulePaths, err := collectModulePaths(modulesDir)
if err != nil {
return nil, err
}
for _, path := range modulePaths {
obj, err := loadJSONObject(path)
if err != nil {
return nil, err
}
if entries, ok := obj["entries"].(map[string]any); ok {
for _, raw := range entries {
row, ok := deepCopyValue(raw).(map[string]any)
if !ok {
continue
}
delete(row, "key")
rawID, ok := row["id"]
if !ok {
continue
}
id, err := asInt(rawID)
if err != nil {
return nil, err
}
row["id"] = id
rows = append(rows, row)
byID[id] = row
}
}
if overrides, ok := obj["overrides"].([]any); ok {
for _, raw := range overrides {
override, ok := raw.(map[string]any)
if !ok {
continue
}
rawID, ok := override["id"]
if !ok {
continue
}
id, err := asInt(rawID)
if err != nil {
return nil, err
}
row := byID[id]
if row == nil {
continue
}
for key, value := range override {
if key == "id" || key == "key" || key == "_tlk" || isMetadataField(key) {
continue
}
row[key] = deepCopyValue(value)
}
}
}
}
sort.Slice(rows, func(i, j int) bool {
return rows[i]["id"].(int) < rows[j]["id"].(int)
})
out := make([]any, 0, len(rows))
for _, row := range rows {
out = append(out, row)
}
return out, nil
}
-41
View File
@@ -1,41 +0,0 @@
package topdata
import (
"encoding/json"
"os"
"path/filepath"
)
func importLegacyBaseDialog(referenceBuilderDir, sourceDir string) (int, error) {
legacyPath := filepath.Join(referenceBuilderDir, "tlk", "base.json")
if _, err := os.Stat(legacyPath); err != nil {
if os.IsNotExist(err) {
return 0, nil
}
return 0, err
}
targetPath := filepath.Join(sourceDir, "base_dialog.json")
if fileExists(targetPath) {
current, err := loadJSONObject(targetPath)
if err == nil {
if entries, ok := current["entries"].(map[string]any); ok && len(entries) > 0 {
return 0, nil
}
}
}
obj, err := loadJSONObject(legacyPath)
if err != nil {
return 0, err
}
raw, err := json.MarshalIndent(obj, "", " ")
if err != nil {
return 0, err
}
raw = append(raw, '\n')
if err := os.WriteFile(targetPath, raw, 0o644); err != nil {
return 0, err
}
return 1, nil
}
-105
View File
@@ -1,105 +0,0 @@
package topdata
import (
"encoding/json"
"os"
"path/filepath"
)
func importLegacyBaseitems(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
_ = legacyTLK
legacyDir := filepath.Join(referenceBuilderDir, "data", "baseitems")
if _, err := os.Stat(legacyDir); err != nil {
if os.IsNotExist(err) {
return 0, nil
}
return 0, err
}
targetDir := filepath.Join(dataDir, "baseitems")
targetBasePath := filepath.Join(targetDir, "base.json")
targetLockPath := filepath.Join(targetDir, "lock.json")
targetModulesDir := filepath.Join(targetDir, "modules")
moduleNames := []string{
"blunderbuss.json",
"coin.json",
"estoc.json",
"heavymace.json",
"maulandfalchion.json",
"ovr_baseitems.json",
"ovr_cloak255.json",
"ovr_helmet255.json",
"shortspear.json",
}
targetModulePaths := make([]string, 0, len(moduleNames))
for _, name := range moduleNames {
targetModulePaths = append(targetModulePaths, filepath.Join(targetModulesDir, name))
}
if fileExists(targetBasePath) && fileExists(targetLockPath) {
allModulesPresent := true
for _, path := range targetModulePaths {
if !fileExists(path) {
allModulesPresent = false
break
}
}
if allModulesPresent {
return 0, nil
}
}
baseObj, err := loadJSONObject(filepath.Join(legacyDir, "base.json"))
if err != nil {
return 0, err
}
baseObj["output"] = "baseitems.2da"
canonicalizeWikiMetadataDocument(baseObj)
lockObj, err := loadJSONObject(filepath.Join(legacyDir, "lock.json"))
if err != nil {
return 0, err
}
moduleObjs := make([]map[string]any, 0, len(moduleNames))
for _, name := range moduleNames {
obj, err := loadJSONObject(filepath.Join(legacyDir, "modules", name))
if err != nil {
return 0, err
}
canonicalizeWikiMetadataDocument(obj)
moduleObjs = append(moduleObjs, obj)
}
if err := os.MkdirAll(targetModulesDir, 0o755); err != nil {
return 0, err
}
writes := []struct {
path string
obj map[string]any
}{
{path: targetBasePath, obj: baseObj},
{path: targetLockPath, obj: lockObj},
}
for index, path := range targetModulePaths {
writes = append(writes, struct {
path string
obj map[string]any
}{
path: path,
obj: moduleObjs[index],
})
}
for _, write := range writes {
raw, err := json.MarshalIndent(write.obj, "", " ")
if err != nil {
return 0, err
}
raw = append(raw, '\n')
if err := os.WriteFile(write.path, raw, 0o644); err != nil {
return 0, err
}
}
return len(writes), nil
}
-226
View File
@@ -1,226 +0,0 @@
package topdata
import (
"encoding/json"
"os"
"path/filepath"
"slices"
"strings"
)
var legacyClassesPlainFamilies = []string{"feats", "skills", "savthr", "bfeat", "pres"}
func importLegacyClasses(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
legacyRoot := filepath.Join(referenceBuilderDir, "data", "classes")
if _, err := os.Stat(legacyRoot); err != nil {
if os.IsNotExist(err) {
return 0, nil
}
return 0, err
}
targetRoot := filepath.Join(dataDir, "classes")
if canonicalClassesPresent(targetRoot) {
return 0, nil
}
coreCollected, plainCollected, err := collectLegacyClassesDatasets(legacyRoot)
if err != nil {
return 0, err
}
tableKeyByOutput := map[string]string{}
for _, dataset := range plainCollected {
if dataset.TableKey == "" {
continue
}
registerLegacyClassesTableKey(tableKeyByOutput, dataset.TableKey, dataset.Dataset.OutputName)
}
coreRows := make([]map[string]any, 0, len(coreCollected.Rows))
for _, rawRow := range coreCollected.Rows {
row, ok := deepCopyValue(rawRow).(map[string]any)
if !ok {
continue
}
if legacyTLK != nil {
if _, _, err := inlineLegacyTLKValue(row, legacyTLK); err != nil {
return 0, err
}
}
rewriteLegacyClassesCoreRow(row, tableKeyByOutput)
coreRows = append(coreRows, row)
}
if err := removePathIfExists(targetRoot); err != nil {
return 0, err
}
if err := os.MkdirAll(filepath.Join(targetRoot, "core"), 0o755); err != nil {
return 0, err
}
writes := []struct {
path string
obj map[string]any
}{
{
path: filepath.Join(targetRoot, "core", "base.json"),
obj: map[string]any{
"output": coreCollected.Dataset.OutputName,
"columns": stringSliceToAny(coreCollected.Columns),
"rows": rowsToAny(coreRows),
},
},
{
path: filepath.Join(targetRoot, "core", "lock.json"),
obj: anyMapInt(coreCollected.LockData),
},
}
for _, collected := range plainCollected {
obj := map[string]any{
"output": collected.Dataset.OutputName,
"columns": stringSliceToAny(collected.Columns),
"rows": rowsToAny(collected.Rows),
}
if strings.TrimSpace(collected.TableKey) != "" {
obj["key"] = collected.TableKey
}
writes = append(writes, struct {
path string
obj map[string]any
}{
path: filepath.Join(dataDir, collected.Dataset.Name+".json"),
obj: obj,
})
}
for _, write := range writes {
if err := os.MkdirAll(filepath.Dir(write.path), 0o755); err != nil {
return 0, err
}
raw, err := json.MarshalIndent(write.obj, "", " ")
if err != nil {
return 0, err
}
raw = append(raw, '\n')
if err := os.WriteFile(write.path, raw, 0o644); err != nil {
return 0, err
}
}
return len(writes), nil
}
func canonicalClassesPresent(targetRoot string) bool {
if !fileExists(filepath.Join(targetRoot, "core", "base.json")) || !fileExists(filepath.Join(targetRoot, "core", "lock.json")) {
return false
}
for _, family := range legacyClassesPlainFamilies {
ok, err := hasJSONFiles(filepath.Join(targetRoot, family))
if err != nil || !ok {
return false
}
}
return true
}
func collectLegacyClassesDatasets(legacyRoot string) (nativeCollectedDataset, []nativeCollectedDataset, error) {
coreCollected, err := collectBaseDataset(nativeDataset{
Name: "classes/core",
BasePath: filepath.Join(legacyRoot, "core", "base.json"),
LockPath: filepath.Join(legacyRoot, "core", "lock.json"),
ModulesDir: filepath.Join(legacyRoot, "core", "modules"),
OutputName: "classes.2da",
Spec: specForDataset("classes"),
})
if err != nil {
return nativeCollectedDataset{}, nil, err
}
coreCollected.Dataset.OutputName = "classes.2da"
plainCollected := make([]nativeCollectedDataset, 0)
for _, family := range legacyClassesPlainFamilies {
familyDir := filepath.Join(legacyRoot, family)
entries, err := os.ReadDir(familyDir)
if err != nil {
if os.IsNotExist(err) {
continue
}
return nativeCollectedDataset{}, nil, err
}
for _, entry := range entries {
if entry.IsDir() || filepath.Ext(entry.Name()) != ".json" || strings.HasPrefix(entry.Name(), ".") || entry.Name() == "lock.json" {
continue
}
filePath := filepath.Join(familyDir, entry.Name())
tableData, err := loadJSONObject(filePath)
if err != nil {
return nativeCollectedDataset{}, nil, err
}
outputName, _ := tableData["output"].(string)
if strings.TrimSpace(outputName) == "" {
outputName = strings.TrimSuffix(entry.Name(), filepath.Ext(entry.Name())) + ".2da"
}
collected, err := collectPlainDataset(nativeDataset{
Name: filepath.ToSlash(filepath.Join("classes", family, strings.TrimSuffix(entry.Name(), filepath.Ext(entry.Name())))),
BasePath: filePath,
LockPath: filepath.Join(familyDir, "lock.json"),
OutputName: outputName,
Spec: specForDataset(filepath.ToSlash(filepath.Join("classes", family, strings.TrimSuffix(entry.Name(), filepath.Ext(entry.Name()))))),
})
if err != nil {
return nativeCollectedDataset{}, nil, err
}
plainCollected = append(plainCollected, collected)
}
}
slices.SortFunc(plainCollected, func(a, b nativeCollectedDataset) int {
return strings.Compare(a.Dataset.Name, b.Dataset.Name)
})
return coreCollected, plainCollected, nil
}
func registerLegacyClassesTableKey(tableKeyByOutput map[string]string, tableKey, outputName string) {
trimmedOutput := strings.TrimSpace(outputName)
if trimmedOutput != "" {
tableKeyByOutput[trimmedOutput] = tableKey
}
trimmedStem := strings.TrimSpace(outputStem(outputName))
if trimmedStem != "" {
tableKeyByOutput[trimmedStem] = tableKey
}
}
func rewriteLegacyClassesCoreRow(row map[string]any, tableKeyByOutput map[string]string) {
for _, field := range []string{"FeatsTable", "SavingThrowTable", "SkillsTable", "BonusFeatsTable", "PreReqTable"} {
tableKey := legacyClassesTableKeyForValue(row[field], tableKeyByOutput)
if tableKey == "" {
continue
}
row[field] = map[string]any{"table": tableKey}
}
}
func legacyClassesTableKeyForValue(value any, tableKeyByOutput map[string]string) string {
switch typed := value.(type) {
case map[string]any:
tableKey, _ := typed["table"].(string)
return strings.TrimSpace(tableKey)
case string:
trimmed := strings.TrimSpace(typed)
if trimmed == "" || trimmed == nullValue {
return ""
}
if trimmed != strings.ToLower(trimmed) {
return ""
}
if tableKey, ok := tableKeyByOutput[trimmed]; ok {
return tableKey
}
return ""
default:
return ""
}
}
-6
View File
@@ -1,6 +0,0 @@
package topdata
func importLegacyCloakmodel(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
_ = legacyTLK
return importLegacyDatasetMirror(referenceBuilderDir, dataDir, "cloakmodel", "cloakmodel.2da", nil)
}
-145
View File
@@ -1,145 +0,0 @@
package topdata
import (
"encoding/json"
"os"
"path/filepath"
)
func importLegacyCreaturespeed(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
_ = legacyTLK
legacyDir := filepath.Join(referenceBuilderDir, "data", "creaturespeed")
if _, err := os.Stat(legacyDir); err != nil {
if os.IsNotExist(err) {
return 0, nil
}
return 0, err
}
targetDir := filepath.Join(dataDir, "creaturespeed")
targetPath := filepath.Join(targetDir, "creaturespeed.json")
if _, err := os.Stat(targetPath); err == nil {
obj, err := loadJSONObject(targetPath)
if err == nil && countLegacyTLKRefsInValue(obj) == 0 {
return 0, nil
}
}
baseObj, err := loadJSONObject(filepath.Join(legacyDir, "base.json"))
if err != nil {
return 0, err
}
rows, err := mergeCreaturespeedRows(baseObj, filepath.Join(legacyDir, "modules"))
if err != nil {
return 0, err
}
if err := os.MkdirAll(targetDir, 0o755); err != nil {
return 0, err
}
if err := removePathIfExists(filepath.Join(targetDir, "base.json")); err != nil {
return 0, err
}
if err := removePathIfExists(filepath.Join(targetDir, "lock.json")); err != nil {
return 0, err
}
if err := removePathIfExists(filepath.Join(targetDir, "modules")); err != nil {
return 0, err
}
out := map[string]any{
"output": "creaturespeed.2da",
"columns": baseObj["columns"],
"rows": rows,
}
raw, err := json.MarshalIndent(out, "", " ")
if err != nil {
return 0, err
}
raw = append(raw, '\n')
if err := os.WriteFile(targetPath, raw, 0o644); err != nil {
return 0, err
}
return 1, nil
}
func mergeCreaturespeedRows(baseObj map[string]any, modulesDir string) ([]any, error) {
rawRows, ok := baseObj["rows"].([]any)
if !ok {
return nil, nil
}
rows := make([]map[string]any, 0, len(rawRows))
byID := map[int]map[string]any{}
for _, raw := range rawRows {
row, ok := deepCopyValue(raw).(map[string]any)
if !ok {
continue
}
delete(row, "key")
rows = append(rows, row)
if rawID, ok := row["id"]; ok {
id, err := asInt(rawID)
if err != nil {
return nil, err
}
byID[id] = row
}
}
modulePaths, err := collectModulePaths(modulesDir)
if err != nil {
return nil, err
}
for _, path := range modulePaths {
obj, err := loadJSONObject(path)
if err != nil {
return nil, err
}
overrideList, ok := obj["overrides"].([]any)
if !ok {
continue
}
for _, raw := range overrideList {
override, ok := raw.(map[string]any)
if !ok {
continue
}
rawID, ok := override["id"]
if !ok {
continue
}
id, err := asInt(rawID)
if err != nil {
return nil, err
}
row := byID[id]
if row == nil {
continue
}
for key, value := range override {
if key == "id" || key == "key" || key == "_tlk" {
continue
}
row[key] = deepCopyValue(value)
}
}
}
out := make([]any, 0, len(rows))
for _, row := range rows {
out = append(out, row)
}
return out, nil
}
func removePathIfExists(path string) error {
if _, err := os.Stat(path); err != nil {
if os.IsNotExist(err) {
return nil
}
return err
}
return os.RemoveAll(path)
}
-288
View File
@@ -1,288 +0,0 @@
package topdata
import (
"encoding/json"
"fmt"
"os"
"path/filepath"
"sort"
"strings"
)
func importLegacyDamagetypes(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
_ = legacyTLK
legacyRoot := filepath.Join(referenceBuilderDir, "data", "damagetypes")
if _, err := os.Stat(legacyRoot); err != nil {
if os.IsNotExist(err) {
return 0, nil
}
return 0, err
}
targetDir := filepath.Join(dataDir, "damagetypes", "registry")
targetPath := filepath.Join(targetDir, "types.json")
if _, err := os.Stat(targetPath); err == nil {
obj, err := loadJSONObject(targetPath)
if err == nil && countLegacyTLKRefsInValue(obj) == 0 {
return 0, nil
}
}
coreBase, err := loadJSONObject(filepath.Join(legacyRoot, "core", "base.json"))
if err != nil {
return 0, err
}
groupBase, err := loadJSONObject(filepath.Join(legacyRoot, "groups", "base.json"))
if err != nil {
return 0, err
}
hitvisualBase, err := loadJSONObject(filepath.Join(legacyRoot, "hitvisual", "base.json"))
if err != nil {
return 0, err
}
coreModule, err := loadJSONObject(filepath.Join(legacyRoot, "core", "modules", "add_eos.json"))
if err != nil {
return 0, err
}
coreLock, err := loadLockfile(filepath.Join(legacyRoot, "core", "lock.json"))
if err != nil {
return 0, err
}
groupModule, err := loadJSONObject(filepath.Join(legacyRoot, "groups", "modules", "add_eos.json"))
if err != nil {
return 0, err
}
groupLock, err := loadLockfile(filepath.Join(legacyRoot, "groups", "lock.json"))
if err != nil {
return 0, err
}
hitvisualModule, err := loadJSONObject(filepath.Join(legacyRoot, "hitvisual", "modules", "add_eos.json"))
if err != nil {
return 0, err
}
hitvisualLock, err := loadLockfile(filepath.Join(legacyRoot, "hitvisual", "lock.json"))
if err != nil {
return 0, err
}
rows, lockData, err := mergeLegacyDamagetypes(coreBase, groupBase, hitvisualBase, coreModule, coreLock, groupModule, groupLock, hitvisualModule, hitvisualLock)
if err != nil {
return 0, err
}
if err := os.MkdirAll(targetDir, 0o755); err != nil {
return 0, err
}
if err := removePathIfExists(filepath.Join(dataDir, "damagetypes", "core")); err != nil {
return 0, err
}
if err := removePathIfExists(filepath.Join(dataDir, "damagetypes", "groups")); err != nil {
return 0, err
}
if err := removePathIfExists(filepath.Join(dataDir, "damagetypes", "hitvisual")); err != nil {
return 0, err
}
typesOut := map[string]any{"rows": rows}
raw, err := json.MarshalIndent(typesOut, "", " ")
if err != nil {
return 0, err
}
raw = append(raw, '\n')
if err := os.WriteFile(targetPath, raw, 0o644); err != nil {
return 0, err
}
if err := saveLockfile(filepath.Join(targetDir, damagetypesRegistryLock), lockData); err != nil {
return 0, err
}
return 2, nil
}
func mergeLegacyDamagetypes(coreBase, groupBase, hitvisualBase, coreModule map[string]any, coreLock map[string]int, groupModule map[string]any, groupLock map[string]int, hitvisualModule map[string]any, hitvisualLock map[string]int) ([]map[string]any, map[string]int, error) {
coreRows, err := coerceRowMapSlice(coreBase["rows"])
if err != nil {
return nil, nil, err
}
groupRows, err := coerceRowMapSlice(groupBase["rows"])
if err != nil {
return nil, nil, err
}
hitRows, err := coerceRowMapSlice(hitvisualBase["rows"])
if err != nil {
return nil, nil, err
}
groupByID := map[int]map[string]any{}
for _, row := range groupRows {
id, err := asInt(row["id"])
if err != nil {
return nil, nil, err
}
groupByID[id] = row
}
hitByID := map[int]map[string]any{}
for _, row := range hitRows {
id, err := asInt(row["id"])
if err != nil {
return nil, nil, err
}
hitByID[id] = row
}
outRows := make([]map[string]any, 0, len(coreRows))
lockData := map[string]int{}
for _, core := range coreRows {
id, err := asInt(core["id"])
if err != nil {
return nil, nil, err
}
groupID, err := asInt(core["DamageTypeGroup"])
if err != nil {
return nil, nil, err
}
group := groupByID[groupID]
if group == nil {
return nil, nil, fmt.Errorf("core id %d references missing group %d", id, groupID)
}
hit := hitByID[id]
if hit == nil {
return nil, nil, fmt.Errorf("core id %d is missing hitvisual row", id)
}
key := "damagetype:" + normalizeDamagetypeKey(core["Label"])
lockData[key] = id
outRows = append(outRows, map[string]any{
"key": key,
"id": id,
"label": deepCopyValue(core["Label"]),
"charsheet_strref": deepCopyValue(core["CharsheetStrref"]),
"damage_type_group": strconvString(groupID),
"damage_ranged_projectile": deepCopyValue(core["DamageRangedProjectile"]),
"group_label": deepCopyValue(group["Label"]),
"feedback_strref": deepCopyValue(group["FeedbackStrref"]),
"color_r": deepCopyValue(group["ColorR"]),
"color_g": deepCopyValue(group["ColorG"]),
"color_b": deepCopyValue(group["ColorB"]),
"visual_effect_id": deepCopyValue(hit["VisualEffectID"]),
"ranged_effect_id": deepCopyValue(hit["RangedEffectID"]),
})
}
coreEntries, err := coerceEntriesMap(coreModule["entries"])
if err != nil {
return nil, nil, err
}
groupEntries, err := coerceEntriesMap(groupModule["entries"])
if err != nil {
return nil, nil, err
}
hitEntries, err := coerceEntriesMap(hitvisualModule["entries"])
if err != nil {
return nil, nil, err
}
keys := make([]string, 0, len(coreEntries))
for key := range coreEntries {
keys = append(keys, key)
}
sort.Slice(keys, func(i, j int) bool {
a := strings.TrimPrefix(keys[i], "damagetypes:")
b := strings.TrimPrefix(keys[j], "damagetypes:")
return a < b
})
for _, coreKey := range keys {
core := coreEntries[coreKey]
suffix := strings.TrimPrefix(coreKey, "damagetypes:")
group := groupEntries["damagetypegroups:"+suffix]
if group == nil {
return nil, nil, fmt.Errorf("%s: missing matching group entry", coreKey)
}
hit := hitEntries["damagehitvisual:"+suffix]
if hit == nil {
return nil, nil, fmt.Errorf("%s: missing matching hitvisual entry", coreKey)
}
id, ok := coreLock[coreKey]
if !ok {
return nil, nil, fmt.Errorf("%s: missing core lock id", coreKey)
}
groupKey := "damagetypegroups:" + suffix
groupID, ok := groupLock[groupKey]
if !ok {
return nil, nil, fmt.Errorf("%s: missing group lock id", groupKey)
}
hitKey := "damagehitvisual:" + suffix
hitID, ok := hitvisualLock[hitKey]
if !ok {
return nil, nil, fmt.Errorf("%s: missing hitvisual lock id", hitKey)
}
if hitID != id {
return nil, nil, fmt.Errorf("%s: core id %d does not match hitvisual id %d", suffix, id, hitID)
}
key := "damagetype:" + suffix
lockData[key] = id
outRows = append(outRows, map[string]any{
"key": key,
"id": id,
"label": deepCopyValue(core["Label"]),
"charsheet_strref": deepCopyValue(core["CharsheetStrref"]),
"damage_type_group": strconvString(groupID),
"damage_ranged_projectile": deepCopyValue(core["DamageRangedProjectile"]),
"group_label": deepCopyValue(group["Label"]),
"feedback_strref": deepCopyValue(group["FeedbackStrref"]),
"color_r": deepCopyValue(group["ColorR"]),
"color_g": deepCopyValue(group["ColorG"]),
"color_b": deepCopyValue(group["ColorB"]),
"visual_effect_id": deepCopyValue(hit["VisualEffectID"]),
"ranged_effect_id": deepCopyValue(hit["RangedEffectID"]),
})
}
sort.Slice(outRows, func(i, j int) bool {
return outRows[i]["id"].(int) < outRows[j]["id"].(int)
})
return outRows, lockData, nil
}
func coerceRowMapSlice(value any) ([]map[string]any, error) {
rawRows, ok := value.([]any)
if !ok {
return nil, fmt.Errorf("rows must be an array")
}
rows := make([]map[string]any, 0, len(rawRows))
for _, raw := range rawRows {
row, ok := raw.(map[string]any)
if !ok {
return nil, fmt.Errorf("row must be an object")
}
rows = append(rows, row)
}
return rows, nil
}
func coerceEntriesMap(value any) (map[string]map[string]any, error) {
rawEntries, ok := value.(map[string]any)
if !ok {
return nil, fmt.Errorf("entries must be an object")
}
entries := make(map[string]map[string]any, len(rawEntries))
for key, raw := range rawEntries {
row, ok := raw.(map[string]any)
if !ok {
return nil, fmt.Errorf("entry %s must be an object", key)
}
entries[key] = row
}
return entries, nil
}
func normalizeDamagetypeKey(value any) string {
text := strings.TrimSpace(strings.ToLower(format2DAValue(value)))
text = strings.TrimSuffix(text, "damage")
text = strings.ReplaceAll(text, " ", "")
text = strings.ReplaceAll(text, "_", "")
text = strings.ReplaceAll(text, "-", "")
return text
}
func strconvString(v int) string {
return fmt.Sprintf("%d", v)
}
-99
View File
@@ -1,99 +0,0 @@
package topdata
import (
"encoding/json"
"os"
"path/filepath"
)
func importLegacyDoortypes(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
_ = legacyTLK
legacyDir := filepath.Join(referenceBuilderDir, "data", "doortypes")
if _, err := os.Stat(legacyDir); err != nil {
if os.IsNotExist(err) {
return 0, nil
}
return 0, err
}
targetDir := filepath.Join(dataDir, "doortypes")
targetBasePath := filepath.Join(targetDir, "base.json")
targetLockPath := filepath.Join(targetDir, "lock.json")
targetModulesDir := filepath.Join(targetDir, "modules")
moduleNames := []string{
"add_pld01.json",
"add_sic11.json",
"add_tapr.json",
"add_tdm01.json",
"add_tdx01.json",
"add_tei01.json",
"add_tfm01.json",
}
targetModulePaths := make([]string, 0, len(moduleNames))
for _, name := range moduleNames {
targetModulePaths = append(targetModulePaths, filepath.Join(targetModulesDir, name))
}
if fileExists(targetBasePath) && fileExists(targetLockPath) {
allModulesPresent := true
for _, path := range targetModulePaths {
if !fileExists(path) {
allModulesPresent = false
break
}
}
if allModulesPresent {
return 0, nil
}
}
baseObj, err := loadJSONObject(filepath.Join(legacyDir, "base.json"))
if err != nil {
return 0, err
}
baseObj["output"] = "doortypes.2da"
lockObj, err := loadJSONObject(filepath.Join(legacyDir, "lock.json"))
if err != nil {
return 0, err
}
moduleObjs := make([]map[string]any, 0, len(moduleNames))
for _, name := range moduleNames {
obj, err := loadJSONObject(filepath.Join(legacyDir, "modules", name))
if err != nil {
return 0, err
}
moduleObjs = append(moduleObjs, obj)
}
if err := os.MkdirAll(targetModulesDir, 0o755); err != nil {
return 0, err
}
writes := []struct {
path string
obj map[string]any
}{
{path: targetBasePath, obj: baseObj},
{path: targetLockPath, obj: lockObj},
}
for i, path := range targetModulePaths {
writes = append(writes, struct {
path string
obj map[string]any
}{path: path, obj: moduleObjs[i]})
}
for _, write := range writes {
raw, err := json.MarshalIndent(write.obj, "", " ")
if err != nil {
return 0, err
}
raw = append(raw, '\n')
if err := os.WriteFile(write.path, raw, 0o644); err != nil {
return 0, err
}
}
return len(writes), nil
}
-64
View File
@@ -1,64 +0,0 @@
package topdata
import (
"encoding/json"
"os"
"path/filepath"
)
func importLegacyFeat(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
_ = legacyTLK
legacyDir := filepath.Join(referenceBuilderDir, "data", "feat")
if _, err := os.Stat(legacyDir); err != nil {
if os.IsNotExist(err) {
return 0, nil
}
return 0, err
}
legacyBasePath := filepath.Join(legacyDir, "base.json")
if _, err := os.Stat(legacyBasePath); err != nil {
if os.IsNotExist(err) {
return 0, nil
}
return 0, err
}
targetDir := filepath.Join(dataDir, "feat")
targetBasePath := filepath.Join(targetDir, "base.json")
var targetObj map[string]any
if _, err := os.Stat(targetBasePath); err == nil {
targetObj, err = loadJSONObject(targetBasePath)
if err != nil {
return 0, err
}
} else if !os.IsNotExist(err) {
return 0, err
}
if targetObj != nil {
if rows, ok := targetObj["rows"].([]any); ok && len(rows) > 0 {
return 0, nil
}
}
baseObj, err := loadJSONObject(legacyBasePath)
if err != nil {
return 0, err
}
baseObj["output"] = "feat.2da"
baseObj["compare_reference"] = false
canonicalizeWikiMetadataDocument(baseObj)
if err := os.MkdirAll(targetDir, 0o755); err != nil {
return 0, err
}
raw, err := json.MarshalIndent(baseObj, "", " ")
if err != nil {
return 0, err
}
raw = append(raw, '\n')
if err := os.WriteFile(targetBasePath, raw, 0o644); err != nil {
return 0, err
}
return 1, nil
}
-6
View File
@@ -1,6 +0,0 @@
package topdata
func importLegacyGenericdoors(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
_ = legacyTLK
return importLegacyDatasetMirror(referenceBuilderDir, dataDir, "genericdoors", "genericdoors.2da", nil)
}
-391
View File
@@ -601,397 +601,6 @@ func deepCopyValue(value any) any {
return out
}
func importLegacyItempropsRegistry(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
legacyDataDir := filepath.Join(referenceBuilderDir, "data")
registryDir := filepath.Join(dataDir, filepath.FromSlash(itempropsRegistryDirName))
if _, err := os.Stat(filepath.Join(registryDir, "properties.json")); err == nil {
if refs, err := countLegacyTLKRefsInRegistry(registryDir); err == nil && refs == 0 {
return 0, nil
}
}
if _, err := os.Stat(filepath.Join(legacyDataDir, "itemprops")); err != nil {
if os.IsNotExist(err) {
return 0, nil
}
return 0, err
}
defs, err := collectBaseDataset(nativeDataset{
Name: "itemprops/defs",
BasePath: filepath.Join(legacyDataDir, "itemprops", "defs", "base.json"),
LockPath: filepath.Join(legacyDataDir, "itemprops", "defs", "lock.json"),
ModulesDir: filepath.Join(legacyDataDir, "itemprops", "defs", "modules"),
Spec: specForDataset("itemprops"),
})
if err != nil {
return 0, err
}
sets, err := collectBaseDataset(nativeDataset{
Name: "itemprops/sets",
BasePath: filepath.Join(legacyDataDir, "itemprops", "sets", "base.json"),
LockPath: filepath.Join(legacyDataDir, "itemprops", "sets", "lock.json"),
ModulesDir: filepath.Join(legacyDataDir, "itemprops", "sets", "modules"),
Spec: specForDataset("itemprops"),
})
if err != nil {
return 0, err
}
costIndex, err := collectBaseDataset(nativeDataset{
Name: "itemprops/costtables/index",
BasePath: filepath.Join(legacyDataDir, "itemprops", "costtables", "index", "base.json"),
LockPath: filepath.Join(legacyDataDir, "itemprops", "costtables", "index", "lock.json"),
ModulesDir: filepath.Join(legacyDataDir, "itemprops", "costtables", "index", "modules"),
Spec: specForDataset("itemprops"),
})
if err != nil {
return 0, err
}
paramIndex, err := collectBaseDataset(nativeDataset{
Name: "itemprops/paramtables/index",
BasePath: filepath.Join(legacyDataDir, "itemprops", "paramtables", "index", "base.json"),
LockPath: filepath.Join(legacyDataDir, "itemprops", "paramtables", "index", "lock.json"),
ModulesDir: filepath.Join(legacyDataDir, "itemprops", "paramtables", "index", "modules"),
Spec: specForDataset("itemprops"),
})
if err != nil {
return 0, err
}
ownedCostTables, err := collectLegacyOwnedItempropTables(filepath.Join(legacyDataDir, "itemprops", "costtables"), "index")
if err != nil {
return 0, err
}
ownedParamTables, err := collectLegacyOwnedItempropTables(filepath.Join(legacyDataDir, "itemprops", "paramtables"), "index")
if err != nil {
return 0, err
}
ownedSubtypeTables, err := collectLegacyOwnedItempropTables(filepath.Join(legacyDataDir, "itemprops", "subtypes"), "")
if err != nil && !os.IsNotExist(err) {
return 0, err
}
setsByID := map[int]map[string]any{}
for _, row := range sets.Rows {
setsByID[row["id"].(int)] = row
}
costByID := map[int]map[string]any{}
for _, row := range costIndex.Rows {
costByID[row["id"].(int)] = row
}
paramByID := map[int]map[string]any{}
for _, row := range paramIndex.Rows {
paramByID[row["id"].(int)] = row
}
registryLock := map[string]int{}
legacyKeyByID := map[string]string{}
if legacyTLK != nil {
for key, id := range legacyTLK.Lock {
legacyKeyByID[strconv.Itoa(id)] = key
}
}
properties := make([]map[string]any, 0, len(defs.Rows))
subtypeRegistryByKey := map[string]map[string]any{}
for _, row := range defs.Rows {
rowID := row["id"].(int)
key := canonicalItempropPropertyKey(row)
registryLock[key] = rowID
property := map[string]any{
"key": key,
"label": row["Label"],
"name": deepCopyValue(row["Name"]),
"property_text": deepCopyValue(row["GameStrRef"]),
}
if description := nullableRegistryField(row, "Description"); description != nullValue {
property["description"] = description
}
if setRow, ok := setsByID[rowID]; ok {
property["availability"] = importLegacyAvailability(setRow)
if setName := nullableRegistryField(setRow, "StringRef"); setName != nullValue && setName != stringField(row, "Name") {
property["set_name"] = setName
}
if setLabel := stringField(setRow, "Label"); setLabel != "" && setLabel != stringField(row, "Label") {
property["set_label"] = setLabel
}
} else {
property["availability"] = []any{}
}
if subtypeResRef := stringField(row, "SubTypeResRef"); subtypeResRef != "" && subtypeResRef != nullValue {
subtypeKey := canonicalModelKey("subtype_models", subtypeResRef)
property["subtype"] = map[string]any{"ref": subtypeKey}
if _, ok := subtypeRegistryByKey[subtypeKey]; !ok {
subtypeRegistryByKey[subtypeKey] = buildLegacySubtypeModel(subtypeKey, subtypeResRef, ownedSubtypeTables)
}
}
costValue := stringField(row, "Cost")
costID := stringField(row, "CostTableResRef")
if costValue != "" && costValue != nullValue || costID != "" && costID != nullValue {
cost := map[string]any{}
if costValue != "" && costValue != nullValue {
cost["value"] = costValue
}
if costID != "" && costID != nullValue {
id, err := strconv.Atoi(costID)
if err != nil {
return 0, err
}
costRow, ok := costByID[id]
if !ok {
return 0, fmt.Errorf("%s references missing cost table index %s", key, costID)
}
cost["ref"] = canonicalModelKey("cost_models", stringField(costRow, "Name"))
}
property["cost"] = cost
}
if paramID := stringField(row, "Param1ResRef"); paramID != "" && paramID != nullValue {
id, err := strconv.Atoi(paramID)
if err != nil {
return 0, err
}
paramRow, ok := paramByID[id]
if !ok {
return 0, fmt.Errorf("%s references missing param table index %s", key, paramID)
}
property["param"] = map[string]any{"ref": canonicalModelKey("param_models", stringField(paramRow, "TableResRef"))}
}
if legacyTLK != nil {
property["name"] = importLegacyStrrefValue(property["name"], key+".name", legacyTLK, legacyKeyByID)
property["property_text"] = importLegacyStrrefValue(property["property_text"], key+".property_text", legacyTLK, legacyKeyByID)
if description, ok := property["description"]; ok {
property["description"] = importLegacyStrrefValue(description, key+".description", legacyTLK, legacyKeyByID)
}
if updated, _, err := inlineLegacyTLKValue(property, legacyTLK); err != nil {
return 0, err
} else if updated {
// property updated in place
}
}
properties = append(properties, property)
}
costModels := make([]map[string]any, 0, len(costIndex.Rows))
for _, row := range costIndex.Rows {
key := canonicalModelKey("cost_models", stringField(row, "Name"))
registryLock[key] = row["id"].(int)
model := map[string]any{
"key": key,
"index_name": row["Name"],
"label": row["Label"],
"client_load": row["ClientLoad"],
"table": buildLegacyOwnedModelTable(stringField(row, "Name"), ownedCostTables),
}
costModels = append(costModels, model)
}
paramModels := make([]map[string]any, 0, len(paramIndex.Rows))
for _, row := range paramIndex.Rows {
key := canonicalModelKey("param_models", stringField(row, "TableResRef"))
registryLock[key] = row["id"].(int)
model := map[string]any{
"key": key,
"name": deepCopyValue(row["Name"]),
"label": row["Lable"],
"table": buildLegacyOwnedModelTable(stringField(row, "TableResRef"), ownedParamTables),
}
paramModels = append(paramModels, model)
}
subtypeModels := make([]map[string]any, 0, len(subtypeRegistryByKey))
for _, key := range sortedKeysAny(subtypeRegistryByKey) {
subtypeModels = append(subtypeModels, subtypeRegistryByKey[key])
}
writes := []struct {
path string
obj map[string]any
}{
{filepath.Join(registryDir, "properties.json"), map[string]any{"rows": properties}},
{filepath.Join(registryDir, "cost_models.json"), map[string]any{"rows": costModels}},
{filepath.Join(registryDir, "param_models.json"), map[string]any{"rows": paramModels}},
{filepath.Join(registryDir, "subtype_models.json"), map[string]any{"rows": subtypeModels}},
{filepath.Join(registryDir, itempropsRegistryLock), anyMapInt(registryLock)},
}
if err := os.MkdirAll(registryDir, 0o755); err != nil {
return 0, err
}
for _, write := range writes {
raw, err := json.MarshalIndent(write.obj, "", " ")
if err != nil {
return 0, err
}
raw = append(raw, '\n')
if err := os.WriteFile(write.path, raw, 0o644); err != nil {
return 0, err
}
}
return len(writes), nil
}
func countLegacyTLKRefsInRegistry(registryDir string) (int, error) {
paths, err := collectDataJSONPaths(registryDir)
if err != nil {
return 0, err
}
refs := 0
for _, path := range paths {
obj, err := loadJSONObject(path)
if err != nil {
return 0, err
}
refs += countLegacyTLKRefsInValue(obj)
}
return refs, nil
}
func collectLegacyOwnedItempropTables(root, skipDir string) (map[string]map[string]any, error) {
result := map[string]map[string]any{}
entries, err := os.ReadDir(root)
if err != nil {
return nil, err
}
for _, entry := range entries {
if !entry.IsDir() {
continue
}
if skipDir != "" && entry.Name() == skipDir {
continue
}
dir := filepath.Join(root, entry.Name())
basePath := filepath.Join(dir, "base.json")
if _, err := os.Stat(basePath); err != nil {
continue
}
collected, err := collectBaseDataset(nativeDataset{
Name: filepath.ToSlash(filepath.Join("itemprops", filepath.Base(root), entry.Name())),
BasePath: basePath,
LockPath: filepath.Join(dir, "lock.json"),
ModulesDir: filepath.Join(dir, "modules"),
Spec: specForDataset("itemprops"),
})
if err != nil {
return nil, err
}
result[strings.ToLower(outputStem(collected.Dataset.OutputName))] = map[string]any{
"ownership": "owned",
"resref": outputStem(collected.Dataset.OutputName),
"output": collected.Dataset.OutputName,
"columns": stringSliceToAny(collected.Columns),
"rows": rowsToAny(collected.Rows),
}
}
return result, nil
}
func buildLegacySubtypeModel(key, resref string, owned map[string]map[string]any) map[string]any {
return map[string]any{
"key": key,
"table_resref": resref,
"table": buildLegacyOwnedModelTable(resref, owned),
}
}
func buildLegacyOwnedModelTable(resref string, owned map[string]map[string]any) map[string]any {
if table, ok := owned[strings.ToLower(resref)]; ok {
return table
}
return map[string]any{
"ownership": "external",
"resref": resref,
}
}
func canonicalItempropPropertyKey(row map[string]any) string {
if key := stringField(row, "key"); key != "" {
return "itemprop:" + strings.TrimPrefix(key, "itempropdef:")
}
return canonicalModelKey("itemprop", stringField(row, "Label"))
}
func canonicalModelKey(prefix, source string) string {
normalized := strings.ToLower(strings.TrimSpace(source))
normalized = strings.ReplaceAll(normalized, " ", "")
normalized = strings.ReplaceAll(normalized, "-", "")
normalized = strings.ReplaceAll(normalized, "_", "")
return prefix + ":" + normalized
}
func importLegacyStrrefValue(value any, fallbackKey string, legacy *legacyTLKData, keyByID map[string]string) any {
if legacy == nil {
return value
}
if _, ok, _ := parseTLKPayload(value, true); ok {
return value
}
_ = keyByID
key := fallbackKey
entry, ok := legacy.Entries[key]
if !ok {
return value
}
payload := map[string]any{
"key": key,
"text": entry.Text,
}
if entry.SoundResRef != "" {
payload["sound_resref"] = entry.SoundResRef
}
if entry.SoundLength != 0 {
payload["sound_length"] = entry.SoundLength
}
return map[string]any{"tlk": payload}
}
func importLegacyAvailability(row map[string]any) []any {
out := make([]any, 0)
for _, column := range itempropsAvailabilityColumns {
value, ok := row[column]
if !ok {
continue
}
text := formatRegistryScalar(value)
if text != "1" {
continue
}
out = append(out, normalizeAvailabilityName(column))
}
return out
}
func anyMapInt(values map[string]int) map[string]any {
out := make(map[string]any, len(values))
for key, value := range values {
out[key] = value
}
return out
}
func stringSliceToAny(values []string) []any {
out := make([]any, 0, len(values))
for _, value := range values {
out = append(out, value)
}
return out
}
func rowsToAny(rows []map[string]any) []any {
out := make([]any, 0, len(rows))
for _, row := range rows {
out = append(out, deepCopyValue(row))
}
return out
}
func sortedKeysAny(values map[string]map[string]any) []string {
out := make([]string, 0, len(values))
for key := range values {
out = append(out, key)
}
slices.Sort(out)
return out
}
func validateItempropsRegistryGraph(dataDir string, report *ValidationReport) {
registry, err := loadItempropsRegistry(dataDir)
if err != nil {
-176
View File
@@ -1,176 +0,0 @@
package topdata
import (
"encoding/json"
"io/fs"
"os"
"path/filepath"
"sort"
"strings"
)
type legacyDatasetTransform func(relativePath string, obj map[string]any)
func importLegacyDatasetMirror(referenceBuilderDir, dataDir, datasetName, outputName string, transform legacyDatasetTransform) (int, error) {
legacyDir := filepath.Join(referenceBuilderDir, "data", datasetName)
if _, err := os.Stat(legacyDir); err != nil {
if os.IsNotExist(err) {
return 0, nil
}
return 0, err
}
targetDir := filepath.Join(dataDir, datasetName)
baseObj, err := loadJSONObject(filepath.Join(legacyDir, "base.json"))
if err != nil {
return 0, err
}
if outputName != "" {
baseObj["output"] = outputName
}
if transform != nil {
transform("base.json", baseObj)
}
lockObj, err := loadJSONObject(filepath.Join(legacyDir, "lock.json"))
if err != nil {
return 0, err
}
if transform != nil {
transform("lock.json", lockObj)
}
moduleRelPaths, err := collectJSONRelativePaths(filepath.Join(legacyDir, "modules"))
if err != nil {
return 0, err
}
moduleObjs := make(map[string]map[string]any, len(moduleRelPaths))
for _, relPath := range moduleRelPaths {
obj, err := loadJSONObject(filepath.Join(legacyDir, "modules", relPath))
if err != nil {
return 0, err
}
normalizedRel := filepath.ToSlash(relPath)
if transform != nil {
transform(filepath.ToSlash(filepath.Join("modules", normalizedRel)), obj)
}
moduleObjs[normalizedRel] = obj
}
if err := os.MkdirAll(targetDir, 0o755); err != nil {
return 0, err
}
updated := 0
for _, write := range []struct {
path string
obj map[string]any
}{
{path: filepath.Join(targetDir, "base.json"), obj: baseObj},
{path: filepath.Join(targetDir, "lock.json"), obj: lockObj},
} {
changed, err := writeJSONObjectIfChanged(write.path, write.obj)
if err != nil {
return 0, err
}
if changed {
updated++
}
}
targetModulesDir := filepath.Join(targetDir, "modules")
if len(moduleObjs) > 0 {
if err := os.MkdirAll(targetModulesDir, 0o755); err != nil {
return 0, err
}
}
for relPath, obj := range moduleObjs {
targetPath := filepath.Join(targetModulesDir, filepath.FromSlash(relPath))
if err := os.MkdirAll(filepath.Dir(targetPath), 0o755); err != nil {
return 0, err
}
changed, err := writeJSONObjectIfChanged(targetPath, obj)
if err != nil {
return 0, err
}
if changed {
updated++
}
}
staleModules, err := collectJSONRelativePaths(targetModulesDir)
if err != nil {
return 0, err
}
for _, relPath := range staleModules {
normalizedRel := filepath.ToSlash(relPath)
if _, ok := moduleObjs[normalizedRel]; ok {
continue
}
if err := os.Remove(filepath.Join(targetModulesDir, filepath.FromSlash(normalizedRel))); err != nil && !os.IsNotExist(err) {
return 0, err
}
updated++
}
return updated, nil
}
func collectJSONRelativePaths(root string) ([]string, error) {
if _, err := os.Stat(root); err != nil {
if os.IsNotExist(err) {
return nil, nil
}
return nil, err
}
var relPaths []string
err := filepath.WalkDir(root, func(path string, d fs.DirEntry, err error) error {
if err != nil {
return err
}
if d.IsDir() {
return nil
}
if !strings.EqualFold(filepath.Ext(d.Name()), ".json") {
return nil
}
relPath, err := filepath.Rel(root, path)
if err != nil {
return err
}
relPaths = append(relPaths, filepath.ToSlash(relPath))
return nil
})
if err != nil {
return nil, err
}
sort.Strings(relPaths)
return relPaths, nil
}
func writeJSONObjectIfChanged(path string, obj map[string]any) (bool, error) {
raw, err := json.MarshalIndent(obj, "", " ")
if err != nil {
return false, err
}
raw = append(raw, '\n')
current, err := os.ReadFile(path)
if err == nil && string(current) == string(raw) {
return false, nil
}
if err != nil && !os.IsNotExist(err) {
return false, err
}
if err := os.WriteFile(path, raw, 0o644); err != nil {
return false, err
}
return true, nil
}
func fileExists(path string) bool {
_, err := os.Stat(path)
return err == nil
}
-6
View File
@@ -1,6 +0,0 @@
package topdata
func importLegacyLoadscreens(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
_ = legacyTLK
return importLegacyDatasetMirror(referenceBuilderDir, dataDir, "loadscreens", "loadscreens.2da", nil)
}
-237
View File
@@ -1,237 +0,0 @@
package topdata
import (
"encoding/json"
"os"
"path/filepath"
"strconv"
)
func importLegacyMasterfeats(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
legacyDir := filepath.Join(referenceBuilderDir, "data", "feat", "masterfeats")
if _, err := os.Stat(legacyDir); err != nil {
if os.IsNotExist(err) {
return 0, nil
}
return 0, err
}
targetDir := filepath.Join(dataDir, "masterfeats")
targetPath := filepath.Join(targetDir, "base.json")
legacyPlainPath := filepath.Join(targetDir, "masterfeats.json")
if _, err := os.Stat(targetPath); err == nil {
obj, err := loadJSONObject(targetPath)
if err == nil && countLegacyTLKRefsInValue(obj) == 0 {
return 0, nil
}
}
collected, err := importLegacyMasterfeatsDataset(legacyDir)
if err != nil {
return 0, err
}
referenceIDs, err := collectReferenceLockIDs(filepath.Join(referenceBuilderDir, "data"))
if err != nil {
return 0, err
}
rows := make([]any, 0, len(collected.Rows))
for _, row := range collected.Rows {
copyRow, ok := deepCopyValue(row).(map[string]any)
if !ok {
continue
}
if legacyTLK != nil {
if _, _, err := inlineLegacyTLKValue(copyRow, legacyTLK); err != nil {
return 0, err
}
}
copyRow = rewriteImportedIDRefs(copyRow, referenceIDs)
rows = append(rows, copyRow)
}
if err := os.MkdirAll(targetDir, 0o755); err != nil {
return 0, err
}
writes := []struct {
path string
obj map[string]any
}{
{
path: targetPath,
obj: map[string]any{
"output": collected.Dataset.OutputName,
"columns": stringSliceToAny(collected.Columns),
"rows": rows,
},
},
{
path: filepath.Join(targetDir, "lock.json"),
obj: anyMapInt(collected.LockData),
},
}
for _, write := range writes {
raw, err := json.MarshalIndent(write.obj, "", " ")
if err != nil {
return 0, err
}
raw = append(raw, '\n')
if err := os.WriteFile(write.path, raw, 0o644); err != nil {
return 0, err
}
}
updatedFiles := len(writes)
if err := os.Remove(legacyPlainPath); err == nil {
updatedFiles++
} else if err != nil && !os.IsNotExist(err) {
return 0, err
}
return updatedFiles, nil
}
func importLegacyMasterfeatsDataset(legacyDir string) (nativeCollectedDataset, error) {
dataset := nativeDataset{
Name: "masterfeats",
BasePath: filepath.Join(legacyDir, "masterfeats.json"),
LockPath: filepath.Join(legacyDir, "lock.json"),
Spec: specForDataset("masterfeats"),
}
tableData, err := loadJSONObject(dataset.BasePath)
if err != nil {
return nativeCollectedDataset{}, err
}
columns, err := parseColumns(tableData, dataset.Name)
if err != nil {
return nativeCollectedDataset{}, err
}
rawRows, ok := tableData["rows"].([]any)
if !ok {
return nativeCollectedDataset{}, nil
}
lockData, err := loadLockfile(dataset.LockPath)
if err != nil {
return nativeCollectedDataset{}, err
}
usedIDs := map[int]struct{}{}
for _, rowID := range lockData {
usedIDs[rowID] = struct{}{}
}
rows := make([]map[string]any, 0, len(rawRows))
explicitID := make([]bool, 0, len(rawRows))
for index, raw := range rawRows {
rowObj, ok := raw.(map[string]any)
if !ok {
continue
}
rowID := index
if value, ok := rowObj["id"]; ok {
parsed, err := asInt(value)
if err != nil {
return nativeCollectedDataset{}, err
}
rowID = parsed
}
row := map[string]any{"id": rowID}
for _, column := range columns {
row[column] = nullValue
}
for key, value := range rowObj {
switch key {
case "id":
case "key":
if keyText, ok := value.(string); ok && keyText != "" {
row["key"] = keyText
}
default:
columnName, ok := canonicalColumn(columns, key)
if !ok {
continue
}
row[columnName] = value
}
}
rows = append(rows, row)
_, hasExplicitID := rowObj["id"]
explicitID = append(explicitID, hasExplicitID)
if hasExplicitID {
usedIDs[rowID] = struct{}{}
}
}
nextID := nextAvailableID(usedIDs)
lockModified := false
for index, row := range rows {
key, hasKey := row["key"].(string)
if hasKey && key != "" {
if lockedID, ok := lockData[key]; ok {
row["id"] = lockedID
continue
}
if explicitID[index] {
lockData[key] = row["id"].(int)
} else {
row["id"] = nextID
lockData[key] = nextID
usedIDs[nextID] = struct{}{}
nextID = nextAvailableID(usedIDs)
}
lockModified = true
continue
}
if !explicitID[index] {
row["id"] = index
}
}
if lockModified {
_ = lockModified
}
return nativeCollectedDataset{
Dataset: nativeDataset{
Name: dataset.Name,
OutputName: "masterfeats.2da",
Columns: columns,
Spec: dataset.Spec,
},
Columns: columns,
Rows: rows,
LockData: lockData,
}, nil
}
func rewriteImportedIDRefs(value any, ids map[string]int) map[string]any {
row, ok := rewriteImportedIDRefsValue(value, ids).(map[string]any)
if !ok {
return map[string]any{}
}
return row
}
func rewriteImportedIDRefsValue(value any, ids map[string]int) any {
switch typed := value.(type) {
case map[string]any:
if len(typed) == 1 {
if rawID, ok := typed["id"]; ok {
if key, ok := rawID.(string); ok {
if resolved, ok := ids[key]; ok {
return strconv.Itoa(resolved)
}
}
}
}
out := make(map[string]any, len(typed))
for key, child := range typed {
out[key] = rewriteImportedIDRefsValue(child, ids)
}
return out
case []any:
out := make([]any, 0, len(typed))
for _, child := range typed {
out = append(out, rewriteImportedIDRefsValue(child, ids))
}
return out
default:
return value
}
}
-415
View File
@@ -1,415 +0,0 @@
package topdata
import (
"encoding/json"
"fmt"
"os"
"path/filepath"
"strings"
"git.westgate.pw/ShadowsOverWestgate/sow-tools/internal/project"
)
type NormalizeResult struct {
StatePath string
ScannedFiles int
UpdatedFiles int
InlineValues int
RemainingFiles int
RemainingLegacyRefs int
}
func NormalizeProject(p *project.Project) (NormalizeResult, error) {
if !p.HasTopData() {
return NormalizeResult{}, fmt.Errorf("topdata is not configured for this project")
}
sourceDir := p.TopDataSourceDir()
dataDir := filepath.Join(sourceDir, "data")
migrationRoot := resolveMigrationInputRoot(sourceDir, p.TopDataReferenceBuilderDir())
legacyTLK, err := loadLegacyTLK(filepath.Join(sourceDir, "tlk"))
if err != nil {
return NormalizeResult{}, err
}
if migrationRoot != "" {
migrationTLK, err := loadLegacyTLK(filepath.Join(migrationRoot, "tlk"))
if err != nil {
return NormalizeResult{}, err
}
legacyTLK = mergeLegacyTLK(migrationTLK, legacyTLK)
}
baseDialogImported, err := importLegacyBaseDialog(migrationRoot, sourceDir)
if err != nil {
return NormalizeResult{}, err
}
itempropsImported, err := importLegacyItempropsRegistry(migrationRoot, dataDir, legacyTLK)
if err != nil {
return NormalizeResult{}, err
}
masterfeatsImported, err := importLegacyMasterfeats(migrationRoot, dataDir, legacyTLK)
if err != nil {
return NormalizeResult{}, err
}
featImported, err := importLegacyFeat(migrationRoot, dataDir, legacyTLK)
if err != nil {
return NormalizeResult{}, err
}
creaturespeedImported, err := importLegacyCreaturespeed(migrationRoot, dataDir, legacyTLK)
if err != nil {
return NormalizeResult{}, err
}
armorImported, err := importLegacyArmor(migrationRoot, dataDir, legacyTLK)
if err != nil {
return NormalizeResult{}, err
}
baseitemsImported, err := importLegacyBaseitems(migrationRoot, dataDir, legacyTLK)
if err != nil {
return NormalizeResult{}, err
}
damagetypesImported, err := importLegacyDamagetypes(migrationRoot, dataDir, legacyTLK)
if err != nil {
return NormalizeResult{}, err
}
skyboxesImported, err := importLegacySkyboxes(migrationRoot, dataDir, legacyTLK)
if err != nil {
return NormalizeResult{}, err
}
vfxPersistentImported, err := importLegacyVFXPersistent(migrationRoot, dataDir, legacyTLK)
if err != nil {
return NormalizeResult{}, err
}
loadscreensImported, err := importLegacyLoadscreens(migrationRoot, dataDir, legacyTLK)
if err != nil {
return NormalizeResult{}, err
}
progfxImported, err := importLegacyProgfx(migrationRoot, dataDir, legacyTLK)
if err != nil {
return NormalizeResult{}, err
}
genericdoorsImported, err := importLegacyGenericdoors(migrationRoot, dataDir, legacyTLK)
if err != nil {
return NormalizeResult{}, err
}
cloakmodelImported, err := importLegacyCloakmodel(migrationRoot, dataDir, legacyTLK)
if err != nil {
return NormalizeResult{}, err
}
repadjustImported, err := importLegacyRepadjust(migrationRoot, dataDir, legacyTLK)
if err != nil {
return NormalizeResult{}, err
}
wingmodelImported, err := importLegacyWingmodel(migrationRoot, dataDir, legacyTLK)
if err != nil {
return NormalizeResult{}, err
}
tailmodelImported, err := importLegacyTailmodel(migrationRoot, dataDir, legacyTLK)
if err != nil {
return NormalizeResult{}, err
}
doortypesImported, err := importLegacyDoortypes(migrationRoot, dataDir, legacyTLK)
if err != nil {
return NormalizeResult{}, err
}
visualeffectsImported, err := importLegacyVisualeffects(migrationRoot, dataDir, legacyTLK)
if err != nil {
return NormalizeResult{}, err
}
appearanceImported, err := importLegacyAppearance(migrationRoot, dataDir, legacyTLK)
if err != nil {
return NormalizeResult{}, err
}
placeablesImported, err := importLegacyPlaceables(migrationRoot, dataDir, legacyTLK)
if err != nil {
return NormalizeResult{}, err
}
rulesetImported, err := importLegacyRuleset(migrationRoot, dataDir, legacyTLK)
if err != nil {
return NormalizeResult{}, err
}
skillsImported, err := importLegacySkills(migrationRoot, dataDir, legacyTLK)
if err != nil {
return NormalizeResult{}, err
}
spellsImported, err := importLegacySpells(migrationRoot, dataDir, legacyTLK)
if err != nil {
return NormalizeResult{}, err
}
racialtypesImported, err := importLegacyRacialtypes(migrationRoot, dataDir, legacyTLK)
if err != nil {
return NormalizeResult{}, err
}
classesImported, err := importLegacyClasses(migrationRoot, dataDir, legacyTLK)
if err != nil {
return NormalizeResult{}, err
}
portraitsImported, err := importLegacyPortraits(migrationRoot, dataDir, legacyTLK)
if err != nil {
return NormalizeResult{}, err
}
paths, err := collectDataJSONPaths(dataDir)
if err != nil {
return NormalizeResult{}, err
}
result := NormalizeResult{
StatePath: filepath.Join(sourceDir, tlkStateFile),
ScannedFiles: len(paths),
UpdatedFiles: baseDialogImported + itempropsImported + masterfeatsImported + featImported + creaturespeedImported + armorImported + baseitemsImported + damagetypesImported + skyboxesImported + vfxPersistentImported + loadscreensImported + progfxImported + genericdoorsImported + cloakmodelImported + repadjustImported + wingmodelImported + tailmodelImported + doortypesImported + visualeffectsImported + appearanceImported + placeablesImported + rulesetImported + skillsImported + spellsImported + racialtypesImported + classesImported + portraitsImported,
}
state, err := loadTLKState(result.StatePath)
if err != nil {
return result, err
}
if state.Entries == nil {
state.Entries = map[string]tlkStateMapping{}
}
if legacyTLK != nil && state.Language == "" {
state.Language = legacyTLK.Language
}
if legacyTLK != nil {
for key, id := range legacyTLK.Lock {
if _, ok := state.Entries[key]; !ok {
state.Entries[key] = tlkStateMapping{ID: id}
}
}
}
if legacyTLK != nil {
for _, path := range paths {
updated, count, err := inlineLegacyTLKFile(path, legacyTLK)
if err != nil {
return result, err
}
if updated {
result.UpdatedFiles++
result.InlineValues += count
}
}
}
if err := saveTLKState(result.StatePath, state); err != nil {
return result, err
}
if err := removeLegacyTLKAuthoredInput(filepath.Join(sourceDir, "tlk")); err != nil {
return result, err
}
remainingFiles, remainingRefs, err := countLegacyTLKRefs(paths)
if err != nil {
return result, err
}
result.RemainingFiles = remainingFiles
result.RemainingLegacyRefs = remainingRefs
return result, nil
}
func mergeLegacyTLK(base, override *legacyTLKData) *legacyTLKData {
if base == nil && override == nil {
return nil
}
merged := &legacyTLKData{
Language: "en",
Entries: map[string]tlkEntryData{},
Lock: map[string]int{},
}
if base != nil {
if base.Language != "" {
merged.Language = base.Language
}
for key, entry := range base.Entries {
merged.Entries[key] = entry
}
for key, id := range base.Lock {
merged.Lock[key] = id
}
}
if override != nil {
if override.Language != "" {
merged.Language = override.Language
}
for key, entry := range override.Entries {
merged.Entries[key] = entry
}
for key, id := range override.Lock {
merged.Lock[key] = id
}
}
return merged
}
func removeLegacyTLKAuthoredInput(tlkDir string) error {
modulesDir := filepath.Join(tlkDir, "modules")
if err := os.RemoveAll(modulesDir); err != nil {
return fmt.Errorf("remove %s: %w", modulesDir, err)
}
lockPath := filepath.Join(tlkDir, "lock.json")
if err := os.Remove(lockPath); err != nil && !os.IsNotExist(err) {
return fmt.Errorf("remove %s: %w", lockPath, err)
}
gitkeepPath := filepath.Join(tlkDir, ".gitkeep")
if err := os.Remove(gitkeepPath); err != nil && !os.IsNotExist(err) {
return fmt.Errorf("remove %s: %w", gitkeepPath, err)
}
if err := os.Remove(tlkDir); err != nil && !os.IsNotExist(err) {
if !strings.Contains(err.Error(), "directory not empty") {
return fmt.Errorf("remove %s: %w", tlkDir, err)
}
}
return nil
}
func resolveMigrationInputRoot(sourceDir, referenceBuilderDir string) string {
if strings.TrimSpace(referenceBuilderDir) != "" {
return referenceBuilderDir
}
snapshotRoot := filepath.Join(sourceDir, "migration_snapshot")
if info, err := os.Stat(snapshotRoot); err == nil && info.IsDir() {
return snapshotRoot
}
return ""
}
func inlineLegacyTLKFile(path string, legacy *legacyTLKData) (bool, int, error) {
obj, err := loadJSONObject(path)
if err != nil {
return false, 0, err
}
updated, count, err := inlineLegacyTLKValue(obj, legacy)
if err != nil {
return false, 0, err
}
if !updated {
return false, 0, nil
}
raw, err := json.MarshalIndent(obj, "", " ")
if err != nil {
return false, 0, err
}
raw = append(raw, '\n')
if err := os.WriteFile(path, raw, 0o644); err != nil {
return false, 0, err
}
return true, count, nil
}
func inlineLegacyTLKValue(value any, legacy *legacyTLKData) (bool, int, error) {
switch typed := value.(type) {
case map[string]any:
if rawTLK, ok := typed["tlk"]; ok {
key, ok := rawTLK.(string)
if ok {
entry, ok := legacy.Entries[key]
if !ok {
return false, 0, nil
}
payload := map[string]any{
"key": key,
"text": entry.Text,
}
if entry.SoundResRef != "" {
payload["sound_resref"] = entry.SoundResRef
}
if entry.SoundLength != 0 {
payload["sound_length"] = entry.SoundLength
}
typed["tlk"] = payload
return true, 1, nil
}
}
updated := false
total := 0
for key, child := range typed {
childUpdated, childCount, err := inlineLegacyTLKValue(child, legacy)
if err != nil {
return false, 0, err
}
if childUpdated {
typed[key] = child
updated = true
total += childCount
}
}
return updated, total, nil
case []any:
updated := false
total := 0
for index, child := range typed {
childUpdated, childCount, err := inlineLegacyTLKValue(child, legacy)
if err != nil {
return false, 0, err
}
if childUpdated {
typed[index] = child
updated = true
total += childCount
}
}
return updated, total, nil
default:
return false, 0, nil
}
}
func collectDataJSONPaths(dataDir string) ([]string, error) {
paths := make([]string, 0)
err := filepath.WalkDir(dataDir, func(path string, d os.DirEntry, err error) error {
if err != nil {
return err
}
if d.IsDir() {
return nil
}
if strings.HasSuffix(strings.ToLower(d.Name()), ".json") && d.Name() != "lock.json" {
paths = append(paths, path)
}
return nil
})
if err != nil {
return nil, err
}
return paths, nil
}
func countLegacyTLKRefs(paths []string) (int, int, error) {
files := 0
refs := 0
for _, path := range paths {
obj, err := loadJSONObject(path)
if err != nil {
return 0, 0, err
}
count := countLegacyTLKRefsInValue(obj)
if count == 0 {
continue
}
files++
refs += count
}
return files, refs, nil
}
func countLegacyTLKRefsInValue(value any) int {
switch typed := value.(type) {
case map[string]any:
total := 0
if rawTLK, ok := typed["tlk"]; ok {
if _, ok := rawTLK.(string); ok {
total++
}
}
for _, child := range typed {
total += countLegacyTLKRefsInValue(child)
}
return total
case []any:
total := 0
for _, child := range typed {
total += countLegacyTLKRefsInValue(child)
}
return total
default:
return 0
}
}
+3 -75
View File
@@ -410,6 +410,9 @@ func buildNativeUnchecked(p *project.Project, opts NativeBuildOptions, progress
if err != nil {
return BuildResult{}, err
}
if err := loadStandaloneTLKStrings(sourceDir, compiler); err != nil {
return BuildResult{}, err
}
output2DA := compiled2DAOutputDir(p)
outputTLK := compiledTLKOutputDir(p)
@@ -6642,81 +6645,6 @@ func marshalOrderedLockfile(keys []string, lockData map[string]int, formatting j
return buffer.Bytes(), nil
}
func loadLegacyTLK(root string) (*legacyTLKData, error) {
info, err := os.Stat(root)
if err != nil {
if os.IsNotExist(err) {
return nil, nil
}
return nil, err
}
if !info.IsDir() {
return nil, fmt.Errorf("legacy tlk root must be a directory: %s", root)
}
result := &legacyTLKData{
Language: "en",
Entries: map[string]tlkEntryData{},
Lock: map[string]int{},
}
lockPath := filepath.Join(root, "lock.json")
lockData, err := loadLockfile(lockPath)
if err != nil && !os.IsNotExist(err) {
return nil, err
}
for key, id := range lockData {
result.Lock[key] = id
}
basePath := filepath.Join(root, "base.json")
if _, err := os.Stat(basePath); err == nil {
base, err := loadJSONObject(basePath)
if err != nil {
return nil, err
}
if language, ok := base["language"].(string); ok && language != "" {
result.Language = language
}
if entries, ok := base["entries"].(map[string]any); ok {
normalized, err := normalizeLegacyTLKEntries(entries)
if err != nil {
return nil, err
}
for key, entry := range normalized {
result.Entries[key] = entry
}
}
}
modulesDir := filepath.Join(root, "modules")
modulePaths, err := collectModulePaths(modulesDir)
if err != nil {
if os.IsNotExist(err) {
return result, nil
}
return nil, err
}
for _, path := range modulePaths {
obj, err := loadJSONObject(path)
if err != nil {
return nil, err
}
entries, ok := obj["entries"].(map[string]any)
if !ok {
continue
}
normalized, err := normalizeLegacyTLKEntries(entries)
if err != nil {
return nil, fmt.Errorf("%s: %w", path, err)
}
for key, entry := range normalized {
result.Entries[key] = entry
}
}
return result, nil
}
func normalizeLegacyTLKEntries(entries map[string]any) (map[string]tlkEntryData, error) {
normalized := map[string]tlkEntryData{}
for _, key := range sortedKeys(entries) {
-6
View File
@@ -1,6 +0,0 @@
package topdata
func importLegacyPlaceables(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
_ = legacyTLK
return importLegacyDatasetMirror(referenceBuilderDir, dataDir, "placeables", "placeables.2da", nil)
}
-8
View File
@@ -1,8 +0,0 @@
package topdata
func importLegacyPortraits(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
if legacyTLK == nil {
return 0, nil
}
return importLegacyDatasetMirror(referenceBuilderDir, dataDir, "portraits", "portraits.2da", nil)
}
-6
View File
@@ -1,6 +0,0 @@
package topdata
func importLegacyProgfx(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
_ = legacyTLK
return importLegacyDatasetMirror(referenceBuilderDir, dataDir, "progfx", "progfx.2da", nil)
}
-282
View File
@@ -1,282 +0,0 @@
package topdata
import (
"encoding/json"
"fmt"
"os"
"path/filepath"
"slices"
"strings"
)
type legacyRacialtypesFeatTable struct {
Output string
Rows []map[string]any
}
func importLegacyRacialtypes(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
if strings.TrimSpace(referenceBuilderDir) == "" {
return 0, nil
}
legacyCoreDir := filepath.Join(referenceBuilderDir, "data", "racialtypes", "core")
legacyFeatsDir := filepath.Join(referenceBuilderDir, "data", "racialtypes", "feats")
if _, err := os.Stat(legacyCoreDir); err != nil {
if os.IsNotExist(err) {
return 0, nil
}
return 0, err
}
if _, err := os.Stat(legacyFeatsDir); err != nil {
if os.IsNotExist(err) {
return 0, nil
}
return 0, err
}
targetRoot := filepath.Join(dataDir, "racialtypes", "registry")
targetBasePath := filepath.Join(targetRoot, "base.json")
targetLockPath := filepath.Join(targetRoot, "lock.json")
targetRacesDir := filepath.Join(targetRoot, "races")
if fileExists(targetBasePath) && fileExists(targetLockPath) {
entries, err := os.ReadDir(targetRacesDir)
if err == nil {
raceFiles := 0
for _, entry := range entries {
if !entry.IsDir() && strings.HasSuffix(strings.ToLower(entry.Name()), ".json") {
raceFiles++
}
}
if raceFiles == 16 {
return 0, nil
}
}
}
baseColumns, baseRows, lockData, raceRows, err := collectLegacyRacialtypesRegistry(legacyCoreDir, legacyFeatsDir, legacyTLK)
if err != nil {
return 0, err
}
if err := removePathIfExists(filepath.Join(dataDir, "racialtypes", "core")); err != nil {
return 0, err
}
if err := removePathIfExists(filepath.Join(dataDir, "racialtypes", "feats")); err != nil {
return 0, err
}
if err := removePathIfExists(filepath.Join(targetRoot, "base_rows.json")); err != nil {
return 0, err
}
if err := removePathIfExists(targetRacesDir); err != nil {
return 0, err
}
if err := os.MkdirAll(targetRacesDir, 0o755); err != nil {
return 0, err
}
writes := []struct {
path string
obj any
}{
{
path: targetBasePath,
obj: map[string]any{
"columns": stringSliceToAny(baseColumns),
"rows": rowsToAny(baseRows),
},
},
{
path: targetLockPath,
obj: anyMapInt(lockData),
},
}
raceKeys := make([]string, 0, len(raceRows))
for key := range raceRows {
raceKeys = append(raceKeys, key)
}
slices.Sort(raceKeys)
for _, key := range raceKeys {
fileName := strings.TrimPrefix(key, "racialtypes:") + ".json"
writes = append(writes, struct {
path string
obj any
}{
path: filepath.Join(targetRacesDir, fileName),
obj: raceRows[key],
})
}
for _, write := range writes {
raw, err := json.MarshalIndent(write.obj, "", " ")
if err != nil {
return 0, err
}
raw = append(raw, '\n')
if err := os.WriteFile(write.path, raw, 0o644); err != nil {
return 0, err
}
}
return len(writes), nil
}
func collectLegacyRacialtypesRegistry(coreDir, featsDir string, legacyTLK *legacyTLKData) ([]string, []map[string]any, map[string]int, map[string]map[string]any, error) {
coreCollected, err := collectBaseDataset(nativeDataset{
Name: "racialtypes",
BasePath: filepath.Join(coreDir, "base.json"),
LockPath: filepath.Join(coreDir, "lock.json"),
ModulesDir: filepath.Join(coreDir, "modules"),
Spec: specForDataset("racialtypes"),
})
if err != nil {
return nil, nil, nil, nil, err
}
featTables, err := loadLegacyRacialtypesFeatTables(featsDir)
if err != nil {
return nil, nil, nil, nil, err
}
baseRows := make([]map[string]any, 0)
raceRows := map[string]map[string]any{}
lockData := map[string]int{}
for key, id := range coreCollected.LockData {
if strings.HasPrefix(key, "racialtypes:") {
lockData[key] = id
}
}
for _, collectedRow := range coreCollected.Rows {
row, ok := deepCopyValue(collectedRow).(map[string]any)
if !ok {
continue
}
if legacyTLK != nil {
if _, _, err := inlineLegacyTLKValue(row, legacyTLK); err != nil {
return nil, nil, nil, nil, err
}
}
key, _ := row["key"].(string)
if !strings.HasPrefix(key, "racialtypes:") {
assignRacialtypesBaseRowKey(row)
if key, _ := row["key"].(string); key != "" {
rowID, err := asInt(row["id"])
if err != nil {
return nil, nil, nil, nil, err
}
lockData[key] = rowID
}
baseRows = append(baseRows, row)
continue
}
delete(row, "id")
delete(row, "key")
raceFile := map[string]any{
"key": key,
"core": row,
}
if tableKey := extractRacialtypesFeatTableKey(row["FeatsTable"]); tableKey != "" {
table, ok := featTables[tableKey]
if !ok {
return nil, nil, nil, nil, fmt.Errorf("%s: unknown feat table %s", key, tableKey)
}
delete(row, "FeatsTable")
raceFile["feat_output"] = table.Output
raceFile["feats"] = rowsToAny(table.Rows)
}
raceRows[key] = raceFile
}
slices.SortFunc(baseRows, func(a, b map[string]any) int {
left, _ := asInt(a["id"])
right, _ := asInt(b["id"])
return left - right
})
return coreCollected.Columns, baseRows, lockData, raceRows, nil
}
func loadLegacyRacialtypesFeatTables(featsDir string) (map[string]legacyRacialtypesFeatTable, error) {
entries, err := os.ReadDir(featsDir)
if err != nil {
return nil, err
}
tables := map[string]legacyRacialtypesFeatTable{}
for _, entry := range entries {
if entry.IsDir() || filepath.Ext(entry.Name()) != ".json" {
continue
}
obj, err := loadJSONObject(filepath.Join(featsDir, entry.Name()))
if err != nil {
return nil, err
}
tableKey, _ := obj["key"].(string)
outputName, _ := obj["output"].(string)
rawRows, ok := obj["rows"].([]any)
if tableKey == "" || outputName == "" || !ok {
return nil, fmt.Errorf("%s: racialtypes feat table must define key, output, and rows", entry.Name())
}
rows := make([]map[string]any, 0, len(rawRows))
for index, raw := range rawRows {
row, ok := raw.(map[string]any)
if !ok {
return nil, fmt.Errorf("%s: row %d must be an object", entry.Name(), index)
}
rows = append(rows, row)
}
tables[tableKey] = legacyRacialtypesFeatTable{
Output: outputName,
Rows: rows,
}
}
return tables, nil
}
func extractRacialtypesFeatTableKey(value any) string {
obj, ok := value.(map[string]any)
if !ok {
return ""
}
tableKey, _ := obj["table"].(string)
return tableKey
}
func assignRacialtypesBaseRowKey(row map[string]any) {
name, _ := row["Name"].(string)
if strings.TrimSpace(name) == "" || name == nullValue {
delete(row, "key")
return
}
label, _ := row["Label"].(string)
keySuffix := normalizeRacialtypesKeySuffix(label)
if keySuffix == "" {
delete(row, "key")
return
}
row["key"] = "racialtypes:" + keySuffix
}
func normalizeRacialtypesKeySuffix(label string) string {
normalized := strings.ToLower(strings.TrimSpace(label))
if normalized == "" {
return ""
}
var b strings.Builder
lastUnderscore := false
for _, ch := range normalized {
isAlphaNum := (ch >= 'a' && ch <= 'z') || (ch >= '0' && ch <= '9')
if isAlphaNum {
b.WriteRune(ch)
lastUnderscore = false
continue
}
if lastUnderscore {
continue
}
b.WriteByte('_')
lastUnderscore = true
}
return strings.Trim(b.String(), "_")
}
-63
View File
@@ -1,63 +0,0 @@
package topdata
import (
"encoding/json"
"os"
"path/filepath"
)
func importLegacyRepadjust(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
_ = legacyTLK
legacyDir := filepath.Join(referenceBuilderDir, "data", "repadjust")
if _, err := os.Stat(legacyDir); err != nil {
if os.IsNotExist(err) {
return 0, nil
}
return 0, err
}
targetDir := filepath.Join(dataDir, "repadjust")
targetBasePath := filepath.Join(targetDir, "base.json")
targetLockPath := filepath.Join(targetDir, "lock.json")
if fileExists(targetBasePath) && fileExists(targetLockPath) {
obj, err := loadJSONObject(targetBasePath)
if err == nil && countLegacyTLKRefsInValue(obj) == 0 {
return 0, nil
}
}
baseObj, err := loadJSONObject(filepath.Join(legacyDir, "base.json"))
if err != nil {
return 0, err
}
baseObj["output"] = "repadjust.2da"
lockObj, err := loadJSONObject(filepath.Join(legacyDir, "lock.json"))
if err != nil {
return 0, err
}
if err := os.MkdirAll(targetDir, 0o755); err != nil {
return 0, err
}
writes := []struct {
path string
obj map[string]any
}{
{path: targetBasePath, obj: baseObj},
{path: targetLockPath, obj: lockObj},
}
for _, write := range writes {
raw, err := json.MarshalIndent(write.obj, "", " ")
if err != nil {
return 0, err
}
raw = append(raw, '\n')
if err := os.WriteFile(write.path, raw, 0o644); err != nil {
return 0, err
}
}
return len(writes), nil
}
-171
View File
@@ -1,171 +0,0 @@
package topdata
import (
"encoding/json"
"fmt"
"os"
"path/filepath"
"sort"
)
func importLegacyRuleset(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
_ = legacyTLK
legacyDir := filepath.Join(referenceBuilderDir, "data", "ruleset")
if _, err := os.Stat(legacyDir); err != nil {
if os.IsNotExist(err) {
return 0, nil
}
return 0, err
}
targetDir := filepath.Join(dataDir, "ruleset")
targetPath := filepath.Join(targetDir, "ruleset.json")
if _, err := os.Stat(targetPath); err == nil {
obj, err := loadJSONObject(targetPath)
if err == nil && countLegacyTLKRefsInValue(obj) == 0 {
return 0, nil
}
}
baseObj, err := loadJSONObject(filepath.Join(legacyDir, "base.json"))
if err != nil {
return 0, err
}
rows, err := mergeRulesetRows(baseObj, filepath.Join(legacyDir, "modules"))
if err != nil {
return 0, err
}
if err := os.MkdirAll(targetDir, 0o755); err != nil {
return 0, err
}
if err := removePathIfExists(filepath.Join(targetDir, "base.json")); err != nil {
return 0, err
}
if err := removePathIfExists(filepath.Join(targetDir, "lock.json")); err != nil {
return 0, err
}
if err := removePathIfExists(filepath.Join(targetDir, "modules")); err != nil {
return 0, err
}
out := map[string]any{
"output": "ruleset.2da",
"columns": baseObj["columns"],
"rows": rows,
}
raw, err := json.MarshalIndent(out, "", " ")
if err != nil {
return 0, err
}
raw = append(raw, '\n')
if err := os.WriteFile(targetPath, raw, 0o644); err != nil {
return 0, err
}
return 1, nil
}
func mergeRulesetRows(baseObj map[string]any, modulesDir string) ([]any, error) {
rawRows, ok := baseObj["rows"].([]any)
if !ok {
return nil, nil
}
rows := make([]map[string]any, 0, len(rawRows))
byLabel := map[string]map[string]any{}
for _, raw := range rawRows {
row, ok := deepCopyValue(raw).(map[string]any)
if !ok {
continue
}
delete(row, "key")
if rawID, ok := row["id"]; ok {
id, err := asInt(rawID)
if err != nil {
return nil, err
}
row["id"] = id
}
if label, ok := row["Label"].(string); ok && label != "" && label != "****" {
if _, exists := byLabel[label]; exists {
return nil, fmt.Errorf("duplicate ruleset label %q", label)
}
byLabel[label] = row
}
rows = append(rows, row)
}
modulePaths, err := collectModulePaths(modulesDir)
if err != nil {
return nil, err
}
for _, path := range modulePaths {
obj, err := loadJSONObject(path)
if err != nil {
return nil, err
}
overrideList, ok := obj["overrides"].([]any)
if !ok {
continue
}
for _, raw := range overrideList {
override, ok := raw.(map[string]any)
if !ok {
continue
}
matchObj, ok := override["match"].(map[string]any)
if !ok {
return nil, fmt.Errorf("%s: ruleset override is missing match object", path)
}
rawLabel, ok := matchObj["Label"]
if !ok {
return nil, fmt.Errorf("%s: ruleset override match is missing Label", path)
}
labels, err := normalizeRulesetMatchLabels(rawLabel)
if err != nil {
return nil, fmt.Errorf("%s: %w", path, err)
}
for _, label := range labels {
row, ok := byLabel[label]
if !ok {
return nil, fmt.Errorf("%s: ruleset override target label %q was not found", path, label)
}
for key, value := range override {
if key == "id" || key == "key" || key == "_tlk" || isMetadataField(key) || key == "match" {
continue
}
row[key] = deepCopyValue(value)
}
}
}
}
sort.Slice(rows, func(i, j int) bool {
return rows[i]["id"].(int) < rows[j]["id"].(int)
})
out := make([]any, 0, len(rows))
for _, row := range rows {
out = append(out, row)
}
return out, nil
}
func normalizeRulesetMatchLabels(raw any) ([]string, error) {
switch typed := raw.(type) {
case string:
return []string{typed}, nil
case []any:
out := make([]string, 0, len(typed))
for _, item := range typed {
label, ok := item.(string)
if !ok {
return nil, fmt.Errorf("ruleset match.Label entries must be strings")
}
out = append(out, label)
}
return out, nil
default:
return nil, fmt.Errorf("ruleset match.Label must be a string or string array")
}
}
-140
View File
@@ -1,140 +0,0 @@
package topdata
import (
"encoding/json"
"os"
"path/filepath"
)
func importLegacySkills(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
_ = legacyTLK
legacyDir := filepath.Join(referenceBuilderDir, "data", "skills")
if _, err := os.Stat(legacyDir); err != nil {
if os.IsNotExist(err) {
return 0, nil
}
return 0, err
}
targetDir := filepath.Join(dataDir, "skills")
targetBasePath := filepath.Join(targetDir, "base.json")
targetLockPath := filepath.Join(targetDir, "lock.json")
targetModulesDir := filepath.Join(targetDir, "modules")
moduleNames := []string{
"add_athletics.json",
"add_disguise.json",
"add_linguistics.json",
"add_perception.json",
"add_scriptcraft.json",
"add_sensemotive.json",
"add_stealth.json",
"add_survival.json",
"add_userope.json",
filepath.Join("craft", "add_craftalchemy.json"),
filepath.Join("craft", "add_craftcooking.json"),
filepath.Join("craft", "add_craftjewelry.json"),
filepath.Join("craft", "add_craftleatherworking.json"),
filepath.Join("craft", "add_craftstonework.json"),
filepath.Join("craft", "add_crafttextiles.json"),
filepath.Join("craft", "ovr_craftarmorsmithing.json"),
filepath.Join("craft", "ovr_crafttinkering.json"),
filepath.Join("craft", "ovr_craftweaponsmithing.json"),
filepath.Join("craft", "ovr_craftwoodworking.json"),
filepath.Join("knowledge", "add_knowledgearcana.json"),
filepath.Join("knowledge", "add_knowledgearchitecture.json"),
filepath.Join("knowledge", "add_knowledgedungeoneering.json"),
filepath.Join("knowledge", "add_knowledgegeography.json"),
filepath.Join("knowledge", "add_knowledgehistory.json"),
filepath.Join("knowledge", "add_knowledgelocal.json"),
filepath.Join("knowledge", "add_knowledgenature.json"),
filepath.Join("knowledge", "add_knowledgenobility.json"),
filepath.Join("knowledge", "add_knowledgeplanar.json"),
filepath.Join("knowledge", "add_knowledgereligion.json"),
"ovr_acrobatics.json",
"ovr_animalhandling.json",
"ovr_appraise.json",
"ovr_concentration.json",
"ovr_disabledevice.json",
"ovr_heal.json",
"ovr_hiddenskills.json",
"ovr_influence.json",
"ovr_openlock.json",
"ovr_parry.json",
"ovr_searchtowis.json",
"ovr_sleightofhand.json",
"ovr_taunttointimidate.json",
"rmv_removedskills.json",
}
targetModulePaths := make([]string, 0, len(moduleNames))
for _, name := range moduleNames {
targetModulePaths = append(targetModulePaths, filepath.Join(targetModulesDir, name))
}
if fileExists(targetBasePath) && fileExists(targetLockPath) {
allModulesPresent := true
for _, path := range targetModulePaths {
if !fileExists(path) {
allModulesPresent = false
break
}
}
if allModulesPresent {
return 0, nil
}
}
baseObj, err := loadJSONObject(filepath.Join(legacyDir, "base.json"))
if err != nil {
return 0, err
}
baseObj["output"] = "skills.2da"
canonicalizeWikiMetadataDocument(baseObj)
lockObj, err := loadJSONObject(filepath.Join(legacyDir, "lock.json"))
if err != nil {
return 0, err
}
moduleObjs := make([]map[string]any, 0, len(moduleNames))
for _, name := range moduleNames {
obj, err := loadJSONObject(filepath.Join(legacyDir, "modules", name))
if err != nil {
return 0, err
}
canonicalizeWikiMetadataDocument(obj)
moduleObjs = append(moduleObjs, obj)
}
if err := os.MkdirAll(targetModulesDir, 0o755); err != nil {
return 0, err
}
writes := []struct {
path string
obj map[string]any
}{
{path: targetBasePath, obj: baseObj},
{path: targetLockPath, obj: lockObj},
}
for i, path := range targetModulePaths {
if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil {
return 0, err
}
writes = append(writes, struct {
path string
obj map[string]any
}{path: path, obj: moduleObjs[i]})
}
for _, write := range writes {
raw, err := json.MarshalIndent(write.obj, "", " ")
if err != nil {
return 0, err
}
raw = append(raw, '\n')
if err := os.WriteFile(write.path, raw, 0o644); err != nil {
return 0, err
}
}
return len(writes), nil
}
-6
View File
@@ -1,6 +0,0 @@
package topdata
func importLegacySkyboxes(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
_ = legacyTLK
return importLegacyDatasetMirror(referenceBuilderDir, dataDir, "skyboxes", "skyboxes.2da", nil)
}
-103
View File
@@ -1,103 +0,0 @@
package topdata
import (
"encoding/json"
"os"
"path/filepath"
)
func importLegacySpells(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
_ = legacyTLK
legacyDir := filepath.Join(referenceBuilderDir, "data", "spells")
if _, err := os.Stat(legacyDir); err != nil {
if os.IsNotExist(err) {
return 0, nil
}
return 0, err
}
targetDir := filepath.Join(dataDir, "spells")
targetBasePath := filepath.Join(targetDir, "base.json")
targetLockPath := filepath.Join(targetDir, "lock.json")
targetModulesDir := filepath.Join(targetDir, "modules")
moduleNames := []string{
"specialattacks.json",
"ovr_fixduplicates.json",
filepath.Join("bardicmusic", "fascinate.json"),
filepath.Join("bardicmusic", "inspirecompetence.json"),
filepath.Join("bardicmusic", "inspirecourage.json"),
filepath.Join("bardicmusic", "inspiregreatness.json"),
filepath.Join("bardicmusic", "inspireheroics.json"),
filepath.Join("bardicmusic", "songoffreedom.json"),
}
targetModulePaths := make([]string, 0, len(moduleNames))
for _, name := range moduleNames {
targetModulePaths = append(targetModulePaths, filepath.Join(targetModulesDir, name))
}
if fileExists(targetBasePath) && fileExists(targetLockPath) {
allModulesPresent := true
for _, path := range targetModulePaths {
if !fileExists(path) {
allModulesPresent = false
break
}
}
if allModulesPresent {
return 0, nil
}
}
baseObj, err := loadJSONObject(filepath.Join(legacyDir, "base.json"))
if err != nil {
return 0, err
}
baseObj["output"] = "spells.2da"
lockObj, err := loadJSONObject(filepath.Join(legacyDir, "lock.json"))
if err != nil {
return 0, err
}
moduleObjs := make([]map[string]any, 0, len(moduleNames))
for _, name := range moduleNames {
obj, err := loadJSONObject(filepath.Join(legacyDir, "modules", name))
if err != nil {
return 0, err
}
moduleObjs = append(moduleObjs, obj)
}
if err := os.MkdirAll(targetModulesDir, 0o755); err != nil {
return 0, err
}
writes := []struct {
path string
obj map[string]any
}{
{path: targetBasePath, obj: baseObj},
{path: targetLockPath, obj: lockObj},
}
for i, path := range targetModulePaths {
if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil {
return 0, err
}
writes = append(writes, struct {
path string
obj map[string]any
}{path: path, obj: moduleObjs[i]})
}
for _, write := range writes {
raw, err := json.MarshalIndent(write.obj, "", " ")
if err != nil {
return 0, err
}
raw = append(raw, '\n')
if err := os.WriteFile(write.path, raw, 0o644); err != nil {
return 0, err
}
}
return len(writes), nil
}
-86
View File
@@ -1,86 +0,0 @@
package topdata
import (
"encoding/json"
"os"
"path/filepath"
)
func importLegacyTailmodel(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
_ = legacyTLK
legacyDir := filepath.Join(referenceBuilderDir, "data", "tailmodel")
if _, err := os.Stat(legacyDir); err != nil {
if os.IsNotExist(err) {
return 0, nil
}
return 0, err
}
targetDir := filepath.Join(dataDir, "tailmodel")
targetBasePath := filepath.Join(targetDir, "base.json")
targetLockPath := filepath.Join(targetDir, "lock.json")
targetModulesDir := filepath.Join(targetDir, "modules")
targetModulePaths := []string{
filepath.Join(targetModulesDir, "add.json"),
filepath.Join(targetModulesDir, "ovr_tailprefixes.json"),
}
if fileExists(targetBasePath) && fileExists(targetLockPath) {
allModulesPresent := true
for _, path := range targetModulePaths {
if !fileExists(path) {
allModulesPresent = false
break
}
}
if allModulesPresent {
return 0, nil
}
}
baseObj, err := loadJSONObject(filepath.Join(legacyDir, "base.json"))
if err != nil {
return 0, err
}
baseObj["output"] = "tailmodel.2da"
lockObj, err := loadJSONObject(filepath.Join(legacyDir, "lock.json"))
if err != nil {
return 0, err
}
moduleNames := []string{"add.json", "ovr_tailprefixes.json"}
moduleObjs := make([]map[string]any, 0, len(moduleNames))
for _, name := range moduleNames {
obj, err := loadJSONObject(filepath.Join(legacyDir, "modules", name))
if err != nil {
return 0, err
}
moduleObjs = append(moduleObjs, obj)
}
if err := os.MkdirAll(targetModulesDir, 0o755); err != nil {
return 0, err
}
writes := []struct {
path string
obj map[string]any
}{
{path: targetBasePath, obj: baseObj},
{path: targetLockPath, obj: lockObj},
{path: targetModulePaths[0], obj: moduleObjs[0]},
{path: targetModulePaths[1], obj: moduleObjs[1]},
}
for _, write := range writes {
raw, err := json.MarshalIndent(write.obj, "", " ")
if err != nil {
return 0, err
}
raw = append(raw, '\n')
if err := os.WriteFile(write.path, raw, 0o644); err != nil {
return 0, err
}
}
return len(writes), nil
}
+100
View File
@@ -13,6 +13,7 @@ import (
"strings"
"golang.org/x/text/encoding/charmap"
"gopkg.in/yaml.v3"
)
const (
@@ -74,6 +75,16 @@ type tlkEntryData struct {
SoundLength float32
}
type standaloneTLKDocument struct {
Schema string `yaml:"schema"`
BaseStrref int `yaml:"base_strref"`
Strings []struct {
Key string `yaml:"key"`
Text string `yaml:"text"`
ID *int `yaml:"id"`
} `yaml:"strings"`
}
type tlkStateDocument struct {
Version int `json:"version"`
Language string `json:"language"`
@@ -157,6 +168,42 @@ func newTLKCompiler(sourceDir string, legacy *legacyTLKData) (*tlkCompiler, erro
return compiler, nil
}
func loadStandaloneTLKStrings(sourceDir string, compiler *tlkCompiler) error {
path := filepath.Join(sourceDir, "tlk", "custom.tlk.yml")
raw, err := os.ReadFile(path)
if err != nil {
if os.IsNotExist(err) {
return nil
}
return fmt.Errorf("read %s: %w", path, err)
}
var document standaloneTLKDocument
if err := yaml.Unmarshal(raw, &document); err != nil {
return fmt.Errorf("parse %s: %w", path, err)
}
if document.Schema != "sow-topdata/tlk/v1" {
return fmt.Errorf("%s: unsupported schema %q", path, document.Schema)
}
if document.BaseStrref != customTLKBase {
return fmt.Errorf("%s: base_strref must be %d", path, customTLKBase)
}
seen := map[string]struct{}{}
for index, entry := range document.Strings {
entry.Key = strings.TrimSpace(entry.Key)
if entry.Key == "" || entry.Text == "" || entry.ID == nil {
return fmt.Errorf("%s: strings[%d] requires key, text, and id", path, index)
}
if _, ok := seen[entry.Key]; ok {
return fmt.Errorf("%s: duplicate string key %q", path, entry.Key)
}
seen[entry.Key] = struct{}{}
if err := compiler.registerInlineAtID(entry.Key, *entry.ID, tlkEntryData{Text: entry.Text}); err != nil {
return err
}
}
return nil
}
func loadBaseDialogData(path string) (*legacyTLKData, error) {
if _, err := os.Stat(path); err != nil {
if os.IsNotExist(err) {
@@ -329,6 +376,21 @@ func (c *tlkCompiler) registerInline(key string, entry tlkEntryData) (tlkCompile
}, nil
}
func (c *tlkCompiler) registerInlineAtID(key string, id int, entry tlkEntryData) error {
if id < 0 {
return fmt.Errorf("TLK key %q has negative id %d", key, id)
}
if mapping, ok := c.state.Entries[key]; ok && mapping.ID != id {
return fmt.Errorf("TLK key %q changed id from %d to %d", key, mapping.ID, id)
}
if owner, ok := c.reservedByID[id]; ok && owner != key {
return fmt.Errorf("TLK id %d is already reserved by %q", id, owner)
}
c.state.Entries[key] = tlkStateMapping{ID: id}
c.reservedByID[id] = key
return c.markActive(key, entry)
}
func (c *tlkCompiler) customStrrefForKey(key string) int {
return customTLKBase + c.state.Entries[key].ID
}
@@ -338,6 +400,7 @@ func (c *tlkCompiler) markActive(key string, entry tlkEntryData) error {
if !ok {
mapping = tlkStateMapping{ID: c.allocateID(), Retired: false}
c.state.Entries[key] = mapping
c.reservedByID[mapping.ID] = key
}
existing, ok := c.active[key]
if ok && existing != entry {
@@ -351,9 +414,13 @@ func (c *tlkCompiler) markActive(key string, entry tlkEntryData) error {
}
func (c *tlkCompiler) allocateID() int {
for {
id := c.nextID
c.nextID++
if _, reserved := c.reservedByID[id]; !reserved {
return id
}
}
}
func (c *tlkCompiler) finish(outputDir, tlkName string) (int, error) {
@@ -727,3 +794,36 @@ func sortedKeys[M ~map[string]V, V any](input M) []string {
func almostEqualFloat32(a, b float32) bool {
return math.Abs(float64(a-b)) < 0.0001
}
func mergeLegacyTLK(base, override *legacyTLKData) *legacyTLKData {
if base == nil && override == nil {
return nil
}
merged := &legacyTLKData{
Language: "en",
Entries: map[string]tlkEntryData{},
Lock: map[string]int{},
}
if base != nil {
if base.Language != "" {
merged.Language = base.Language
}
for key, entry := range base.Entries {
merged.Entries[key] = entry
}
for key, id := range base.Lock {
merged.Lock[key] = id
}
}
if override != nil {
if override.Language != "" {
merged.Language = override.Language
}
for key, entry := range override.Entries {
merged.Entries[key] = entry
}
for key, id := range override.Lock {
merged.Lock[key] = id
}
}
return merged
}
+29 -1
View File
@@ -2,6 +2,7 @@ package topdata
import (
"bytes"
"encoding/json"
"errors"
"fmt"
"io"
@@ -13,6 +14,7 @@ import (
"time"
"git.westgate.pw/ShadowsOverWestgate/sow-tools/internal/erf"
"git.westgate.pw/ShadowsOverWestgate/sow-tools/internal/gff"
"git.westgate.pw/ShadowsOverWestgate/sow-tools/internal/project"
)
@@ -215,10 +217,15 @@ func collectTopPackageResources(p *project.Project, compiled2DADir string) ([]er
if err != nil {
return err
}
var resource erf.Resource
if strings.HasSuffix(strings.ToLower(rel), ".itp.json") {
resource, err = topPackageITPResourceFromJSON(path)
} else {
if skipTopPackageAsset(rel) {
return nil
}
resource, err := topPackageResourceFromPath(path)
resource, err = topPackageResourceFromPath(path)
}
if err != nil {
return err
}
@@ -251,6 +258,27 @@ func collectTopPackageResources(p *project.Project, compiled2DADir string) ([]er
return resources, assetFiles, nil
}
func topPackageITPResourceFromJSON(path string) (erf.Resource, error) {
raw, err := os.ReadFile(path)
if err != nil {
return erf.Resource{}, fmt.Errorf("read %s: %w", path, err)
}
var document gff.Document
if err := json.Unmarshal(raw, &document); err != nil {
return erf.Resource{}, fmt.Errorf("parse %s: %w", path, err)
}
if document.FileType != "ITP " {
return erf.Resource{}, fmt.Errorf("%s: file_type must be ITP", path)
}
var payload bytes.Buffer
if err := gff.Write(&payload, document); err != nil {
return erf.Resource{}, fmt.Errorf("compile %s: %w", path, err)
}
resourceType, _ := erf.HAKResourceTypeForExtension("itp")
name := strings.TrimSuffix(filepath.Base(path), ".itp.json")
return erf.Resource{Name: strings.ToLower(name), Type: resourceType, Data: payload.Bytes(), Size: int64(payload.Len())}, nil
}
func shouldSkipTopDataSourceDir(path string, skipDirs map[string]struct{}) bool {
_, ok := skipDirs[filepath.Clean(path)]
return ok
File diff suppressed because it is too large Load Diff
-198
View File
@@ -1,198 +0,0 @@
package topdata
import (
"os"
"path/filepath"
"strings"
)
func importLegacyVFXPersistent(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
_ = legacyTLK
if strings.TrimSpace(referenceBuilderDir) == "" {
return 0, nil
}
legacyDir := filepath.Join(referenceBuilderDir, "data", "vfx_persistent")
if _, err := os.Stat(legacyDir); err != nil {
if os.IsNotExist(err) {
return 0, nil
}
return 0, err
}
targetDir := filepath.Join(dataDir, "vfx_persistent")
targetBasePath := filepath.Join(targetDir, "base.json")
targetLockPath := filepath.Join(targetDir, "lock.json")
targetModulesDir := filepath.Join(targetDir, "modules")
targetModulePath := filepath.Join(targetModulesDir, "freeformaoes.json")
if fileExists(targetBasePath) && fileExists(targetLockPath) && fileExists(targetModulePath) {
baseObj, baseErr := loadJSONObject(targetBasePath)
lockObj, lockErr := loadJSONObject(targetLockPath)
if baseErr == nil && lockErr == nil && vfxPersistentImportComplete(baseObj, lockObj) {
legacyModulePaths, err := collectJSONRelativePaths(filepath.Join(legacyDir, "modules"))
if err == nil {
targetModulePaths, err := collectJSONRelativePaths(targetModulesDir)
if err == nil && equalStringSlices(legacyModulePaths, targetModulePaths) {
return 0, nil
}
}
}
}
baseObj, err := loadJSONObject(filepath.Join(legacyDir, "base.json"))
if err != nil {
return 0, err
}
lockObj, err := synthesizeVFXPersistentKeys(baseObj)
if err != nil {
return 0, err
}
legacyLockObj, err := loadJSONObject(filepath.Join(legacyDir, "lock.json"))
if err != nil {
return 0, err
}
for key, value := range legacyLockObj {
lockObj[key] = value
}
if err := os.MkdirAll(targetModulesDir, 0o755); err != nil {
return 0, err
}
updated := 0
for _, write := range []struct {
path string
obj map[string]any
}{
{path: targetBasePath, obj: baseObj},
{path: targetLockPath, obj: lockObj},
} {
changed, err := writeJSONObjectIfChanged(write.path, write.obj)
if err != nil {
return 0, err
}
if changed {
updated++
}
}
moduleRelPaths, err := collectJSONRelativePaths(filepath.Join(legacyDir, "modules"))
if err != nil {
return 0, err
}
for _, relPath := range moduleRelPaths {
moduleObj, err := loadJSONObject(filepath.Join(legacyDir, "modules", filepath.FromSlash(relPath)))
if err != nil {
return 0, err
}
targetPath := filepath.Join(targetModulesDir, filepath.FromSlash(relPath))
if err := os.MkdirAll(filepath.Dir(targetPath), 0o755); err != nil {
return 0, err
}
changed, err := writeJSONObjectIfChanged(targetPath, moduleObj)
if err != nil {
return 0, err
}
if changed {
updated++
}
}
targetModulePaths, err := collectJSONRelativePaths(targetModulesDir)
if err != nil {
return 0, err
}
legacyModules := make(map[string]struct{}, len(moduleRelPaths))
for _, relPath := range moduleRelPaths {
legacyModules[relPath] = struct{}{}
}
for _, relPath := range targetModulePaths {
if _, ok := legacyModules[relPath]; ok {
continue
}
if err := os.Remove(filepath.Join(targetModulesDir, filepath.FromSlash(relPath))); err != nil && !os.IsNotExist(err) {
return 0, err
}
updated++
}
return updated, nil
}
func vfxPersistentImportComplete(baseObj, lockObj map[string]any) bool {
if !vfxPersistentRowsHaveKeys(baseObj) {
return false
}
expected := map[string]int{
"vfx_persistent:blankradius5": 47,
"vfx_persistent:blankradius10": 48,
"vfx_persistent:blankradius15": 49,
"vfx_persistent:blankradius20": 50,
"vfx_persistent:blankradius25": 51,
"vfx_persistent:blankradius30": 52,
}
for key, want := range expected {
raw, ok := lockObj[key]
if !ok {
return false
}
got, err := asInt(raw)
if err != nil || got != want {
return false
}
}
return true
}
func synthesizeVFXPersistentKeys(baseObj map[string]any) (map[string]any, error) {
rawRows, ok := baseObj["rows"].([]any)
if !ok {
return map[string]any{}, nil
}
lockObj := map[string]any{}
for _, raw := range rawRows {
row, ok := raw.(map[string]any)
if !ok {
continue
}
label, _ := row["LABEL"].(string)
if label == "" {
continue
}
key := "vfx_persistent:" + strings.ToLower(label)
row["key"] = key
if rawID, ok := row["id"]; ok {
lockObj[key] = rawID
}
}
return lockObj, nil
}
func vfxPersistentRowsHaveKeys(baseObj map[string]any) bool {
rawRows, ok := baseObj["rows"].([]any)
if !ok {
return false
}
for _, raw := range rawRows {
row, ok := raw.(map[string]any)
if !ok {
return false
}
key, _ := row["key"].(string)
if key == "" {
return false
}
}
return true
}
func equalStringSlices(left, right []string) bool {
if len(left) != len(right) {
return false
}
for i := range left {
if left[i] != right[i] {
return false
}
}
return true
}
@@ -1,6 +0,0 @@
package topdata
func importLegacyVisualeffects(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
_ = legacyTLK
return importLegacyDatasetMirror(referenceBuilderDir, dataDir, "visualeffects", "visualeffects.2da", nil)
}
-86
View File
@@ -1,86 +0,0 @@
package topdata
import (
"encoding/json"
"os"
"path/filepath"
)
func importLegacyWingmodel(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
_ = legacyTLK
legacyDir := filepath.Join(referenceBuilderDir, "data", "wingmodel")
if _, err := os.Stat(legacyDir); err != nil {
if os.IsNotExist(err) {
return 0, nil
}
return 0, err
}
targetDir := filepath.Join(dataDir, "wingmodel")
targetBasePath := filepath.Join(targetDir, "base.json")
targetLockPath := filepath.Join(targetDir, "lock.json")
targetModulesDir := filepath.Join(targetDir, "modules")
targetModulePaths := []string{
filepath.Join(targetModulesDir, "add.json"),
filepath.Join(targetModulesDir, "ovr_wingprefixes.json"),
}
if fileExists(targetBasePath) && fileExists(targetLockPath) {
allModulesPresent := true
for _, path := range targetModulePaths {
if !fileExists(path) {
allModulesPresent = false
break
}
}
if allModulesPresent {
return 0, nil
}
}
baseObj, err := loadJSONObject(filepath.Join(legacyDir, "base.json"))
if err != nil {
return 0, err
}
baseObj["output"] = "wingmodel.2da"
lockObj, err := loadJSONObject(filepath.Join(legacyDir, "lock.json"))
if err != nil {
return 0, err
}
moduleNames := []string{"add.json", "ovr_wingprefixes.json"}
moduleObjs := make([]map[string]any, 0, len(moduleNames))
for _, name := range moduleNames {
obj, err := loadJSONObject(filepath.Join(legacyDir, "modules", name))
if err != nil {
return 0, err
}
moduleObjs = append(moduleObjs, obj)
}
if err := os.MkdirAll(targetModulesDir, 0o755); err != nil {
return 0, err
}
writes := []struct {
path string
obj map[string]any
}{
{path: targetBasePath, obj: baseObj},
{path: targetLockPath, obj: lockObj},
{path: targetModulePaths[0], obj: moduleObjs[0]},
{path: targetModulePaths[1], obj: moduleObjs[1]},
}
for _, write := range writes {
raw, err := json.MarshalIndent(write.obj, "", " ")
if err != nil {
return 0, err
}
raw = append(raw, '\n')
if err := os.WriteFile(write.path, raw, 0o644); err != nil {
return 0, err
}
}
return len(writes), nil
}
+84
View File
@@ -0,0 +1,84 @@
#!/usr/bin/env bash
set -euo pipefail
contract_script="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/$(basename "${BASH_SOURCE[0]}")"
default_root="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
repo_root="${WORKFLOW_ROOT:-$default_root}"
cd "$repo_root"
ci=.gitea/workflows/ci.yml
release=.gitea/workflows/build-binaries.yml
sync=.gitea/workflows/sync-wrappers.yml
checkout='actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683'
assert_exact_events() {
local workflow="$1" expected="$2" actual
actual="$(awk '
/^on:$/ { in_on = 1 }
in_on && /^$/ { next }
in_on && $0 != "on:" && $0 !~ /^[[:space:]]/ { exit }
in_on { print }
' "$workflow")"
[[ "$actual" == "$expected" ]] || {
echo "workflow-contract: unexpected event scope in $workflow" >&2
return 1
}
}
[[ -f "$ci" ]] || { echo "missing consolidated CI workflow" >&2; exit 1; }
[[ ! -e .gitea/workflows/test.yml ]] || { echo "legacy test workflow still present" >&2; exit 1; }
assert_exact_events "$ci" $'on:\n pull_request:'
grep -Fqx 'permissions: read-all' "$ci"
grep -Fqx ' timeout-minutes: 60' "$ci"
[[ "$(grep -Fc "$checkout" "$ci")" -eq 1 ]]
grep -Fqx ' fetch-depth: 0' "$ci"
grep -Fq 'go vet ./...' "$ci"
grep -Fq 'go test ./...' "$ci"
grep -Fq 'shellcheck scripts/*.sh' "$ci"
grep -Fq 'yamllint .gitea' "$ci"
grep -Fq 'make smoke' "$ci"
grep -Fq 'for target in linux/amd64 linux/arm64 darwin/amd64 darwin/arm64 windows/amd64 windows/arm64; do' "$ci"
assert_exact_events "$release" $'on:\n push:\n tags: [\'v*\']'
grep -Fqx ' code: read' "$release"
grep -Fqx ' releases: write' "$release"
grep -Fqx ' timeout-minutes: 30' "$release"
[[ "$(grep -Fc "$checkout" "$release")" -eq 1 ]]
grep -Fq 'secrets.GITEA_TOKEN' "$release"
if grep -Fq 'secrets.GITHUB_TOKEN' "$release"; then
echo "workflow-contract: release uses the GitHub token alias" >&2
exit 1
fi
grep -Fqx 'permissions: read-all' "$sync"
grep -Fqx ' timeout-minutes: 30' "$sync"
[[ "$(grep -Fc "$checkout" "$sync")" -eq 1 ]]
assert_exact_events "$sync" $'on:\n push:\n branches: [main]\n paths:\n - \'wrappers/crucible.sh\'\n - \'wrappers/crucible.ps1\''
expect_mutation_rejected() {
local name="$1" workflow="$2" expression="$3" tmp
tmp="$(mktemp -d)"
mkdir -p "$tmp/.gitea"
cp -R "$default_root/.gitea/workflows" "$tmp/.gitea/workflows"
sed -i "$expression" "$tmp/$workflow"
if WORKFLOW_ROOT="$tmp" WORKFLOW_CONTRACT_MUTATION=1 bash "$contract_script" >/dev/null 2>&1; then
echo "workflow-contract: accepted forbidden mutation: $name" >&2
rm -rf "$tmp"
return 1
fi
rm -rf "$tmp"
}
if [[ "${WORKFLOW_CONTRACT_MUTATION:-0}" != 1 ]]; then
mutations_ok=0
expect_mutation_rejected 'CI workflow_dispatch event' "$ci" '/^ pull_request:$/a\ workflow_dispatch:' || mutations_ok=1
expect_mutation_rejected 'release develop branch' "$release" "/^ tags:/a\\ branches: [develop]" || mutations_ok=1
expect_mutation_rejected 'release schedule event' "$release" "/^ tags:/a\\ schedule:\n - cron: '0 0 * * *'" || mutations_ok=1
expect_mutation_rejected 'release extra tag pattern' "$release" "s/tags: \['v\*'\]/tags: ['v*', 'release-*']/" || mutations_ok=1
expect_mutation_rejected 'wrapper sync pull_request event' "$sync" '/^ push:$/i\ pull_request:' || mutations_ok=1
expect_mutation_rejected 'wrapper sync broad path' "$sync" "/^ - 'wrappers\/crucible.ps1'$/a\\ - 'wrappers/**'" || mutations_ok=1
(( mutations_ok == 0 )) || exit 1
fi
echo "workflow-contract: CI is PR-only; release and wrapper sync retain their narrow triggers"