sow-tools/AGENTS.md

description, alwaysApply

description	alwaysApply
	true

Toolkit Agent Guide

toolkit/ owns the shared sow-toolkit Go binary consumed by module/ and assets/. It does not own authored game content — it owns the build, extract, validate, compare, topdata, wiki, music, and changelog machinery that those repos invoke through wrapper scripts.

1. Project Overview

Repository name: sow-tools Go module: gitea.westgate.pw/ShadowsOverWestgate/sow-tools Language: Go 1.26+ Entry point: cmd/nwn-tool/ Built binary: tools/sow-toolkit (Linux) / tools/sow-toolkit.exe (Windows)

What this repo owns:

• the sow-toolkit CLI implementation
• all build, extract, compare, and validate pipeline logic
• native topdata validation, build, compare, conversion, and packaging
• wiki rendering and NodeBB deployment
• music scanning, BMU generation, credits, and normalization
• changelog generation from Gitea API
• config loading and effective-config inspection
• HAK archive reading and writing (ERF format)
• GFF JSON conversion

What this repo does not own:

• authored module source (src/, topdata/) — that is module/
• authored binary asset source (content/) — that is assets/
• user-facing wrapper commands — those live in the consumer repos

2. Repository Layout

cmd/nwn-tool/      CLI entrypoint (flag parsing, command dispatch)
internal/app/      command wiring, console UX, verbosity model
internal/changelog/ changelog rendering from Gitea API
internal/erf/      ERF/MOD/HAK reading and writing
internal/gff/      canonical GFF JSON and binary conversion
internal/music/    music scanning, BMU build, credits, normalization
internal/pipeline/ build, extract, compare, and manifest workflows
internal/project/  config loading, effective config, repo scanning
internal/topdata/  native topdata, wiki, and autogen workflows
internal/validator/ validation and diagnostics
tools/             built development binary output
docs/              additional implementation-level documentation

Key topdata-adjacent contracts in internal/topdata/ (read before editing topdata logic):

• internal/topdata/FEAT_GENERATED_FAMILIES_CONTRACT.md
• internal/topdata/FAMILY_EXPANSION_CONTRACT.md
• internal/topdata/MASTERFEATS_CONTRACT.md
• internal/topdata/INHERITANCE_CONTRACT.md
• internal/topdata/CLASS_FEAT_GLOBAL_INJECTS_CONTRACT.md
• internal/topdata/AUTO-INCLUDE_EXISTING_PART_MODELS_IN_2DA_GENERATION_CONTRACT.md

3. Relevant Links And Documentation

Read these before making significant changes:

• README.md — full command surface, consumer resolution, release automation
• CONFIGURATION_HARDENING_CONTRACT.md — YAML authority, config refactor progress, outstanding hardening work
• CONFIGURATION_REFACTOR_CONTRACT.md — config normalization scope
• LOG_REFACTOR_CONTRACT.md — logging verbosity model, phase/summary presentation, remaining command gaps
• TOPDATA_HARDENING_CONTRACT.md — active topdata hardening priorities and findings
• WIKI_DEPLOYMENT_CONTRACT.md — wiki build/deploy phase, current state, immediate goals
• CHANGELOG_AUTOMATION_CONTRACT.md — changelog generator contract and consumer ownership
• MUSIC_REFACTOR_CONTRACT.md — music pipeline refactor status and dataset model
• MODEL_COMPILATION_ON_BUILDTIME_CONTRACT.md — outstanding model compilation build-time contract
• SCRIPT_WRAPPER_CONTRACT.md — wrapper contract status (largely complete)
• module/topdata/wiki/TEMPLATE_AUTHORITY_CONTRACT.md in sow-module — wiki template/YAML ownership rules; mandatory before editing wiki rendering

4. Full Command Surface

sow-toolkit build                    # build module ERF from src/
sow-toolkit build-module             # alias; builds module with HAK manifest refresh
sow-toolkit build-haks               # build HAKs and haks.json manifest
  [--hak <name>]                     # build one HAK by name
  [--archive <name>]                 # build one archive
  [--source-manifest <path>]         # seed split state from prior manifest
  [--plan-only]                      # show build plan without writing
  [--skip-music]                     # skip music conversion for this run
  [--music-dataset <id>]             # process a single music dataset
sow-toolkit extract [<archive>...]   # extract built archive back into src/
sow-toolkit validate                 # validate project structure and config
sow-toolkit compare                  # compare built output against extracted source
sow-toolkit apply-hak-manifest       # apply haks.json to module HAK list
  [<manifest-path>]

sow-toolkit config validate          # validate nwn-tool.yaml against schema
sow-toolkit config effective         # dump effective merged config
sow-toolkit config inspect <key>     # inspect one config value
sow-toolkit config explain <key>     # explain a config key's meaning and source
sow-toolkit config sources           # show config file discovery order

sow-toolkit music list-datasets      # list configured music datasets
sow-toolkit music scan               # scan and report music sources
sow-toolkit music build              # convert and package music BMUs
sow-toolkit music credits            # generate music credits
sow-toolkit music validate           # validate music dataset config
sow-toolkit music manifest           # emit music manifest
sow-toolkit music normalize          # normalize music metadata

sow-toolkit validate-topdata         # validate topdata authoring
sow-toolkit build-topdata            # compile topdata + optional wiki
  [--force]                          # force rebuild even if cache is fresh
  [--wiki]                           # also run wiki build
sow-toolkit build-top-package        # package sow_top.hak + sow_tlk.tlk only
  [--force]
sow-toolkit compare-topdata          # native self-check for topdata pipeline
sow-toolkit convert-topdata          # 2DA/JSON/module conversion utilities
  <2da-to-json|2da-to-module|json-to-2da> ...

sow-toolkit build-wiki               # render topdata wiki into .cache/wiki/
  [--force]
sow-toolkit deploy-wiki              # deploy rendered wiki to NodeBB
  [--source-dir <path>]
  [--endpoint <url>]
  [--token <token>]
  [--version <ver>]
  [--namespace <ns>]
  [--category <namespace=cid>]
  [--manifest <path>]
  [--stale-policy <report|archive|purge>]
  [--dry-run]
  [--create]
  [--force]
  [--reset-managed-namespaces]

sow-toolkit build-changelog          # render release changelog from Gitea API
  [--config <path>]                  # defaults to scripts/changelog.json
  [--output <path>]                  # defaults to stdout
  [--current-tag <tag>]
  [--previous-tag <tag>]
  [--api-base-url <url>]
  [--token <token>]                  # or env: GITEA_API_TOKEN, GITEA_TOKEN, SOW_TOOLS_TOKEN

Global flags (pass after the command name): --quiet, --verbose, --debug

5. Config Discovery Order

1. nwn-tool.yaml
2. nwn-tool.yml
3. nwn-tool.json (legacy, prints migration warning)

YAML is canonical. Inspect effective config with:

./scripts/run-nwn-tool.sh config effective --json
./scripts/run-nwn-tool.sh config explain topdata.package_hak
./scripts/run-nwn-tool.sh config explain autogen.cache.root
./scripts/run-nwn-tool.sh config explain topdata.wiki.page_templates_dir

6. Building The Toolkit

Normal build (produces tools/sow-toolkit):

./build-tool.sh          # Linux/macOS
.\build-tool.ps1         # Windows

Both wrappers use .cache/go-build and skip rebuild when sources are older than the existing binary.

Or run directly without building:

GOCACHE="$PWD/.cache/go-build" go run ./cmd/nwn-tool --help

After building, test it from a consumer repo:

cd ../module
SOW_TOOLS_DEV_BINARY=../toolkit/tools/sow-toolkit ./validate-topdata.sh

7. Key Tests

Run the full suite:

go test ./...

Determinism test — mandatory when changing topdata ID allocation, lockfile persistence, generated families, or null-and-relocate behavior:

go test ./internal/topdata -run TestBuildNativeKeepsGeneratedIDsStableAcrossRepeatedBuilds

App-level tests (CLI behavior, compact vs. verbose output):

go test ./internal/app/...

Changelog tests:

go test ./internal/changelog/...

8. Consumer Resolution

Consumer repos resolve the toolkit binary in this order:

1. Pinned local binary: tools/sow-toolkit / tools/sow-toolkit.exe
2. Dev override via SOW_TOOLS_DEV_BINARY=<path>
3. Auto-install from latest sow-tools Gitea release

Use SOW_TOOLS_DEV_BINARY to override for one invocation without replacing the installed binary:

cd ../module
SOW_TOOLS_DEV_BINARY=../toolkit/tools/sow-toolkit ./build-topdata.sh

9. Release Artifacts

This repo publishes:

• sow-toolkit-linux-amd64.tar.gz
• sow-toolkit-windows-amd64.exe

Consumer repos install them with scripts/install-tool.sh or scripts/install-tool.ps1.

10. Wiki Ownership Rule

When a task touches internal/topdata/wiki*:

1. Read module/topdata/wiki/TEMPLATE_AUTHORITY_CONTRACT.md in sow-module first.
2. Page structure, displayed values, headings, and wording belong in module/topdata/wiki/templates/ and module/topdata/wiki/data.yaml / tables.yaml.
3. Toolkit code owns only generic rendering mechanics, expression helpers, validation, deterministic ordering, and compatibility fallbacks.
4. Do not mention Shadows Over Westgate-specific categories, row keys, labels, or wording in toolkit Go code unless they are compatibility shims.

11. Repository Working Rules

Git

• Never create new branches or commit anything.
• Allow the user to do it manually.
• Suggest a single commit message, but do not commit yourself.
• Never reset, discard, or overwrite user changes unless explicitly asked.

Implementation Discipline

• YAML is the authoritative config format. JSON is for generated artifacts only.
• All repository-shaped behavior must flow through nwn-tool.yaml declarations, not through toolkit-internal hardcoded assumptions.
• New config fields require validation; invalid config must fail early and clearly.
• Outputs must be deterministic across runs given the same inputs.
• Run go test ./... before claiming a change is complete.
• Check config effective in a consumer repo when changing config-loading logic.