ShadowsOverWestgate/sow-tools
7
0
Fork 0
Code Issues 2 Pull Requests Actions Packages Projects Releases 38 Activity

Phase 5: scaffold Crucible (sow-tools) — dispatcher + builder shims, container, PR-first CI, fail-closed (D11, D19).

Browse Source
This commit is contained in:
archvillainette
2026-06-11 20:36:13 +02:00
parent ff264f0d7c
commit 97f8427c4b
154 changed files with 948 additions and 76902 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+53 -680
View File
@@ -1,695 +1,68 @@
# sow-tools
# sow-tools — Crucible
Shared NWN build tooling for Shadows Over Westgate.
This repository owns the Go-based `sow-toolkit` binary used by the separate module and assets repositories.
## Build
Build the development binary into `tools/sow-toolkit`:
```bash
./build-tool.sh
```
On Windows:
```powershell
.\build-tool.ps1
```
Both wrappers use `.cache/go-build` for the Go build cache and skip rebuilding
when the sources are older than the existing development binary.
Or run directly with Go:
```bash
GOCACHE="$PWD/.cache/go-build" go run ./cmd/nwn-tool --help
```
When changing native topdata id allocation, lockfile persistence, generated
families, or null-and-relocate behavior, run the repeat-build determinism test:
```bash
go test ./internal/topdata -run TestBuildNativeKeepsGeneratedIDsStableAcrossRepeatedBuilds
```
That test builds the same topdata project ten times in one process and asserts
that relocated/generated ids stay pinned across every pass.
## Repository Layout
Crucible is the Shadows Over Westgate build/conversion/sync toolchain: one Go
module producing several small binaries plus a `crucible` dispatcher (D11). It is
the **only** repo that owns builder logic; the artifact repos (`sow-module`,
`sow-topdata`, `sow-assets-manifest`) invoke Crucible through wrapper scripts and
never embed a toolkit.
```text
cmd/nwn-tool/ CLI entrypoint
internal/app/ command wiring and console UX
internal/erf/ ERF/MOD/HAK reading and writing
internal/gff/ canonical GFF JSON and binary conversion
internal/pipeline/ build, extract, compare, and manifest workflows
internal/project/ config loading and repo scanning
internal/topdata/ native topdata, wiki, and autogen workflows
internal/validator/validation and diagnostics
tools/ built development binary output
Repos produce artifacts. sow-platform deploys artifacts.
Crucible is how the artifact repos turn source into artifacts.
```
## Commands
## Binaries
The shared CLI exposes these project utilities:
| Binary | Dispatcher form | Owns |
| ------------------ | ------------------ | ----------------------------------------------- |
| `crucible` | — | dispatcher: `crucible <builder> [args]` |
| `crucible-depot` | `crucible depot` | content-addressed depot blob verify/move |
| `crucible-hak` | `crucible hak` | ERF/HAK pack/unpack + hak manifests |
| `crucible-module` | `crucible module` | build/extract/validate/compare the `.mod` |
| `crucible-topdata` | `crucible topdata` | compile 2da/tlk topdata + packages |
| `crucible-wiki` | `crucible wiki` | render + deploy mechanical wiki pages |
The dispatcher and the standalone shims share one registry
(`internal/dispatch`); the shims exist so consumer wrapper scripts can resolve a
single-token command. The full legacy `nwn-tool` command surface and where each
command lands is mapped in [`docs/command-surface.md`](docs/command-surface.md).
## Status (Phase 5 scaffold)
The suite **builds, vets, tests, and runs**, but every builder is **unwired**:
running one fails closed with exit `70` and never fakes an artifact. The internal
pipeline/topdata/erf/wiki/music packages from `gitea/sow-tools` are migrated by
the operator at cutover — the workspace hard rules forbid transplanting that
source automatically. See [`docs/migration-from-nwn-tool.md`](docs/migration-from-nwn-tool.md).
This matches the downstream skeletons: `sow-module` / `sow-topdata` package
scripts already fail closed until they can resolve a Crucible binary.
## Develop
Self-contained (D8) — a host with only Nix can run everything:
```bash
sow-toolkit build
sow-toolkit build-module
sow-toolkit build-haks [--hak <name>] [--archive <name>] [--source-manifest <path>] [--plan-only] [--skip-music] [--music-dataset <id>]
sow-toolkit extract [<build-relative-archive-or-glob> ...]
sow-toolkit validate
sow-toolkit compare
sow-toolkit apply-hak-manifest [<manifest-path>]
sow-toolkit config <validate|effective|inspect|explain|sources>
sow-toolkit music <list-datasets|scan|build|credits|validate|manifest|normalize>
sow-toolkit validate-topdata
sow-toolkit build-topdata [--force] [--wiki]
sow-toolkit build-top-package [--force]
sow-toolkit compare-topdata
sow-toolkit convert-topdata <2da-to-json|2da-to-module|json-to-2da> ...
sow-toolkit build-wiki [--force]
sow-toolkit deploy-wiki [--source-dir <path>] [--endpoint <url>] [--token <token>] [--version <ver>] [--namespace <ns>] [--category <namespace=cid>] [--manifest <path>] [--stale-policy <report|archive|purge>] [--dry-run] [--create] [--force] [--reset-managed-namespaces]
sow-toolkit build-changelog [--config <path>] [--output <path>] [--current-tag <tag>] [--previous-tag <tag>] [--api-base-url <url>] [--token <token>]
nix develop # Go + ffmpeg + shellcheck + yamllint + make
make check # go vet + go test + shellcheck + yamllint
make build # build every cmd/* into ./bin (gitignored)
make smoke # build + assert the fail-closed contract
make image # docker build -> crucible:<sha>
```
Global logging flags `--quiet`, `--verbose`, and `--debug` may be passed after
the command name. Long options that take values accept either `--flag value` or
`--flag=value`; empty inline values such as `--dataset=` are rejected.
Binaries are **never committed** — they are CI artifacts / image layers (D19).
This retires the old habit of checking in `nwn-tool` / `sow-toolkit`.
## Repository Configuration
## CI
Human-authored consumer repository configuration is YAML. The toolkit discovers
configuration in this order:
PR-first (D7): every check runs on pull requests and on push to `main`.
1. `nwn-tool.yaml`
2. `nwn-tool.yml`
3. legacy `nwn-tool.json`
- `test.yml` — vet, test, shellcheck, yamllint, binary smoke.
- `build-image.yml` — build `registry.westgate.pw/sow/crucible:<sha>`; publish
only on `main`. PRs build but never push. No mutable tags.
- `release.yml` — tag-gated binary bundles; image re-tag is wired in Phase 6.
YAML is canonical. If both YAML and legacy JSON files are present, the YAML file
wins. Legacy `nwn-tool.json` still loads for migration, but commands print a
warning and should be updated to `nwn-tool.yaml`.
## Consumers
Example:
```yaml
module:
name: Shadows Over Westgate
resref: sow_module
description: Shadows Over Westgate asset pipeline.
hak_order:
- sow_top
- group:sow_over
- group:sow_core
paths:
assets: content
build: build
music:
# Legacy shorthand — automatically normalized into a dataset.
prefixes:
envi/music/westgate: mus_wg_
# Explicit dataset config (replaces prefixes above).
tools:
ffmpeg: tools/ffmpeg
ffprobe: tools/ffprobe
defaults:
stage_root: "{paths.cache}/music"
credits_root: "{paths.cache}/credits"
credits_overlay: CREDITS.md
max_stem_length: 16
naming_scheme: nwn_bmu
output_extension: .bmu
convert_extensions: [.flac, .m4a, .mp3, .ogg, .wav]
datasets:
westgate:
source: envi/music/westgate
output: envi/music/westgate
prefix: mus_wg_
package_mode: hak_asset
```
Resolved configuration can be inspected without scanning or building:
```bash
sow-toolkit config validate
sow-toolkit config effective --json
sow-toolkit config inspect paths.build
sow-toolkit config explain topdata.package_hak
sow-toolkit config sources
```
`config effective` prints the normalized values consumed by commands, including
toolkit defaults, YAML values, legacy status, active environment overrides, and
provenance. Sensitive override values such as tokens are reported as set without
printing the secret.
Topdata value encodings under `topdata.value_encodings` are matched by dataset
and column. Set `dataset: "*"` to apply an encoding to the same column name in
any native topdata dataset; an exact dataset rule takes precedence over the
wildcard rule for that column. Supported compact encodings include packed hex
lists, external hex-list sidecar tables, alignment set masks, and ID hex lists.
Explicit `{"alignments": [...]}` and `{"ids": [...]}` objects are portable in
any native topdata dataset even without a column-specific rule; configured
rules still override when a column needs custom bounds, widths, or alignment
encoding mode. Plain `{"list": [...]}` objects remain config-driven because
numeric lists need table-specific output semantics.
Class spell availability can be authored as JSON spellbooks under
`topdata/data/classes/spellbooks/`. Each file names a `classes:*` row, and the
native builder reads that row's `SpellTableColumn` to generate the matching
`spells.2da` column. Spellbook files are discovered by convention, so ordinary
per-class spellbook authoring does not require YAML registration.
Common configurable defaults:
```yaml
paths:
source: src
assets: assets
build: build
cache: .cache
tools: tools
outputs:
module_archive: "{module.resref}.mod"
hak_manifest: haks.json
hak_archive: "{hak.name}.hak"
build:
keep_existing_haks: false
generated_assets:
topdata_2da:
- id: parts
source: topdata
output: "{paths.cache}/generated-assets/parts-2da"
include_datasets: [parts/**]
package_root: part
autogen:
id: parts
mode: parts_rows
root: part
include: ["**/*.mdl"]
derive:
kind: trailing_numeric_suffix
group_from: first_path_segment
parts_rows:
row_defaults:
COSTMODIFIER: "0"
acbonus:
default:
strategy: ascending_row_id_sort_key
divisor: 100
format: "%.2f"
datasets:
chest:
strategy: fixed
value: "0.00"
inventory:
source_extensions: [] # empty means built-in NWN source extensions
asset_extensions: [] # empty means built-in NWN asset extensions
source_json_pattern: "{resref}.{extension}.json"
validation:
profile: nwn_module
builtin_script_prefixes: [nw_, x0_, x1_, x2_, x3_, ga_, gc_, gen_, gui_, nwg_, ta_]
required_fields:
ifo: [Mod_Name]
scripts:
source_dir: scripts
cache: "{paths.cache}/ncs"
compiler:
path: ""
env:
path: SOW_NWN_SCRIPT_COMPILER
nwn_root: [SOW_NWN_ROOT, NWN_ROOT]
nwn_user_directory: [SOW_NWN_USER_DIRECTORY, NWN_HOME, NWN_USER_DIRECTORY]
search:
- "{paths.tools}/script-compiler/nwn_script_comp"
- "{paths.tools}/script-compiler/nwn_script_comp.exe"
- "{paths.tools}/nwn_script_comp"
- "{paths.tools}/nwn_script_comp.exe"
- PATH:nwn_script_comp
extract:
layout: nwn_canonical_json
archives:
- "{module.resref}.mod"
cleanup_stale: true
consume_archives: false
ignore_extensions: []
delete_module_archive_after_success: false # deprecated compatibility alias
topdata:
build: "{paths.build}/topdata"
compiled_2da_dir: 2da
compiled_tlk: sow_tlk.tlk
package_hak: sow_top.hak
package_tlk: sow_tlk.tlk
value_encodings:
- dataset: racialtypes/core
column: AvailableSkinColors
mode: packed_hex_list
min: 0
max: 175
hex_width: 2
row_generation:
- namespace: portraits
mode: first_null_row
minimum_row: 15000
wiki:
source: topdata/wiki
output_root: wiki
pages_dir: pages
state_file: state.json
page_index_file: page-index.json
managed_namespaces: [classes, feat, itemtypes, races, skills, spells, meta]
deploy_manifest: .wiki_deploy_manifest.json
deploy_edit_summary: Auto-generated from native builder
namespaces_file: namespaces.yaml
data_file: data.yaml
page_paths_file: wiki.yaml
tables_file: tables.yaml
visibility_file: visibility.yaml
templates_dir: templates
page_templates_dir: templates/pages
manual_sections_dir: manual-sections
manual_sections_file: manual-sections/default.yaml
title_prefix_min_length: 3
status_listing_scope: all
autogen:
cache:
root: "{paths.cache}"
max_age: 1h
refresh_env: SOW_AUTOGEN_MANIFEST_REFRESH
release_source:
provider: gitea
server_url_env: SOW_ASSETS_SERVER_URL
repo_env: SOW_ASSETS_REPO
```
`topdata.value_encodings` enables configured authoring sugar for compact 2DA
cell values. The `packed_hex_list` mode accepts authored values such as
`{"list": [0, {"range": [10, 12]}]}` and emits a single `0x...` string with
fixed-width hex chunks. The `alignment_hex_list` mode accepts two-letter
alignment codes such as `{"list": ["le", "CG"]}` and emits the corresponding
fixed-width hex chunks.
`topdata.value_defaults` applies configured fallback values to missing or
null-like cells before 2DA emission:
```yaml
topdata:
value_defaults:
- dataset: racialtypes/core
column: ECL
value: 0
```
`topdata.row_generation` controls where newly generated rows are allocated for
specific datasets. The default `after_base` mode preserves existing behavior by
allocating after the final imported base row. `first_null_row` fills base rows
whose canonical key is empty and whose emitted 2DA columns are all null-like
(`null`, empty string, or `****`) before extending the table. `minimum_row`
optionally prevents generated allocation below a configured row while preserving
the selected allocation mode at and above that row.
`topdata.row_extensions` can extend plain native tables at build time without
expanding the authored JSON. The `repeat_last` mode matches configured dataset
glob patterns, finds the highest authored value in `level_column`, and appends
generated rows through `target_level` by copying the last authored row's emitted
columns while incrementing the level and row id. Generated extension rows are
build output only and do not update source JSON or lockfiles.
```yaml
topdata:
row_extensions:
- dataset: classes/spellsgained/*
mode: repeat_last
level_column: Level
target_level: 60
```
Topdata dataset directories may include an optional `global.json`. For
base/module datasets, it supports normal `entries`, `overrides`, and `columns`
and applies after ordinary modules; global overrides may use `{"match":"all"}`
to update every final row. For recursive plain datasets, ancestor manifests
apply to descendant tables and may define `injections` with optional
`require_present` and `unless_present` conditions. This is the preferred
authoring surface for class feat and class skill global rows. The legacy
`topdata.class_feat_injections` YAML block still loads for compatibility when no
class feat `global.json` applies.
`topdata.wiki.alignment_links.target_pattern` controls whether formatted
alignment names become internal wiki links. The pattern may use `{alignment}`
for the title-shaped alignment segment with spaces converted to underscores, and
`{label}` for the display label. `alignment_links.link_format` defaults to
`wiki_markup`, which emits `[[target|label]]`. Use `html_anchor` when the
configured target should be emitted as a root URL anchor instead of being
resolved as namespace-relative wiki markup:
```yaml
topdata:
wiki:
alignment_links:
target_pattern: /wiki/Guides/Alignment/{alignment}
link_format: html_anchor
```
Generated wiki input and output paths are YAML-configurable. `source` points to
the wiki authoring root, `data_file`, `page_paths_file`, `tables_file`,
`visibility_file`, `page_templates_dir`, and `manual_sections_file` are resolved
under that source root, and `page_index_file`, `pages_dir`, and `state_file` are
resolved under the configured wiki output root. `templates_dir` and
`manual_sections_dir` remain as compatibility defaults for deriving
`page_templates_dir` and `manual_sections_file` when the newer explicit paths
are omitted.
Generated wiki page structure should be owned by repository templates and YAML
where possible. `topdata/wiki/data.yaml` can expose named providers for page
templates, including `source: facts` providers whose `facts.rows` declare
ordered `label` and `value` expressions. Empty and null-like fact values are
omitted, so templates can render category fact tables with `{{#each
ProviderName}}` while YAML controls which 2DA-backed values are shown.
`paths.source` and `paths.assets` must name real repository subtrees when they
are configured. They may be omitted for repositories that do not own that class
of source, but they must not resolve to the repository root (`.`). Output and
cache paths must stay relative to the repository root unless a specific setting
documents otherwise.
`generated_assets.topdata_2da` is for HAK repositories that need a small
native-topdata subset packaged as HAK resources. It builds only `.2da` output,
injects those generated files into `build-haks`, and rejects TLK-backed text
because this path does not produce TLK packages. Static generated groups can
omit `autogen.mode`; in that case the selected datasets are compiled directly,
and `package_root` plus the repository's HAK include patterns determine where
the resources are packaged.
For `cachedmodels_rows` generated assets, the configured autogen scan appends
discovered model stems from the configured include globs to the selected
`cachedmodels` dataset without `.mdl` extensions and skips models already
present in authored rows.
For `accessory_visualeffects` autogen, producers can publish the group-to-row policy
in `autogen.producers[].accessory_visualeffects`, and consumers can override or
extend it in `autogen.consumers[].accessory_visualeffects`. The toolkit uses
compatibility defaults for older configs and manifests, but YAML can own the
group token source, category source, delimiter, case policy, legacy group
mapping, model stem prefixes to strip, key/label formats, model column, and
default row values. `group_token_source: folder_name` uses the manifest folder
group directly instead of a prefix mapping, and `category_from:
immediate_parent` can expose the model's category folder in keys and labels.
Individual groups can override the model column with `model_column` or write
the discovered model stem to several columns with `model_columns`, which is
useful for root-node visualeffects. Supported format tokens are `{dataset}`,
`{group}`, `{group_raw}`, `{prefix}`, `{category}`, `{category_upper}`,
`{category_segment}`, `{category_segment_upper}`, `{subgroup}`, `{stem}`,
`{stem_upper}`, `{model_stem}`, and `{delimiter}`.
For `parts_rows` generated assets, `parts_rows.acbonus` can normalize final
ACBONUS values before module overrides are applied. The
`ascending_row_id_sort_key` strategy writes toolset ordering values from row
IDs so higher part rows can sort before lower rows in the toolset; the legacy
`descending_row_id_sort_key` strategy remains available for repositories that
already depend on it. Per-dataset policies such as `chest: fixed 0.00` can
override the default. Authored source ACBONUS values are not authoritative when
this policy is configured, except that authored `****` values remain null
unless model discovery activates the row. Dense gaps without authored rows or
discovered part models are emitted as `****` null rows. Files under
`parts/modules` remain the final override layer.
Extraction is deliberately guarded because it writes source files from binary
archives. `extract.archives` is a list of glob patterns relative to
`paths.build`; the default is only the configured module `.mod`. HAK extraction
is opt-in, for example:
```yaml
extract:
archives:
- "{module.resref}.mod"
- "sow_over*.hak"
```
Module resources require a configured `paths.source`; HAK assets require a
configured `paths.assets`; both roots must resolve to safe subtrees. If a root
is unset or resolves to the repository root, extraction fails before writing
instead of creating extension directories like `mdl/` or `png/` at the repo top
level. `.erf` files are rejected by extraction because they do not contain
`module.ifo`, so they cannot safely update module-level area membership.
`consume_archives: true` deletes the selected build archive files only after the
entire extraction succeeds. The older `delete_module_archive_after_success`
setting remains as a module-only compatibility alias when `consume_archives` is
not set. Stale cleanup is scoped to roots actually touched by extracted
resources, so the default module-only extraction does not prune assets.
`topdata.wiki.status_listing_scope` controls the generated wiki-status detail
pages. The default `all` writes both per-namespace listings and aggregate
`meta:wikistatus:<status>` listings. Use `namespace` when aggregate status
lists would be too large for the target NodeBB post limit. Set
`topdata.wiki.status_pages: false` to omit status pages entirely; existing
tracked status pages then become stale deploy-manifest entries and can be
removed with `deploy-wiki --stale-policy purge`.
GFF JSON extraction can merge selected fields and lists from the existing source
file instead of replacing the whole extracted document. Rules are matched by
`target` relative to `paths.source`; files without a matching rule keep the
default overwrite behavior. For example:
```yaml
extract:
merge:
gff_json:
- target: module/module.ifo.json
preserve_fields:
- Mod_Entry_Area
- Mod_Entry_X
- Mod_Entry_Y
- Mod_Entry_Z
- Mod_Entry_Dir_X
- Mod_Entry_Dir_Y
merge_lists:
- field: Mod_Area_list
key_field: Area_Name
strategy: preserve_existing_order_append_new
```
`preserve_fields` copies matching root fields from the existing source document
into the extracted document before writing. The
`preserve_existing_order_append_new` list strategy keeps existing keyed list
order for entries still present in the extracted archive, uses extracted entries
as the value source, appends archive-only entries in archive order, and removes
existing entries that are no longer present in the archive.
Music defaults (used when not explicitly configured):
```yaml
music:
defaults:
stage_root: "{paths.cache}/music"
credits_root: "{paths.cache}/credits"
credits_overlay: CREDITS.md
max_stem_length: 16
naming_scheme: nwn_bmu
output_extension: .bmu
convert_extensions: [.flac, .m4a, .mp3, .ogg, .wav]
```
Music datasets define individual music collections with independent sources and
naming prefixes. `source` is the authored input root under `paths.assets`;
`output` is the generated HAK asset root matched by HAK include globs. If
`output` is omitted, it defaults to `source`. `package_mode: hak_asset` converts
configured source extensions into generated HAK resources; `package_mode: none`
scans and writes credits without adding generated HAK resources. The legacy
`music.prefixes` shorthand is automatically normalized into datasets.
`convert_extensions` is configuration-driven; the built-in default set is
`.flac`, `.m4a`, `.mp3`, `.ogg`, and `.wav`.
Those extensions are discovered from YAML within configured music dataset source
roots, so setting them in `music.defaults.convert_extensions` or a dataset
override is sufficient; you do not need a parallel
`inventory.asset_extensions` override just to make music scanning see them.
Music can also be run directly:
```bash
sow-toolkit music list-datasets
sow-toolkit music scan --dataset westgate
sow-toolkit music build --dataset westgate --dry-run
sow-toolkit music build --dataset westgate
sow-toolkit music credits --dataset westgate --check
sow-toolkit music validate --dataset westgate --json
sow-toolkit music manifest --dataset westgate
sow-toolkit music normalize --dataset westgate
```
`build-haks` uses the same music pipeline. Use `--plan-only` or music
`--dry-run` to compute mappings without transcoding. Use `build-haks
--skip-music` to package authored assets only, or `build-haks --music-dataset
westgate` to limit conversion to a configured dataset.
Generated and machine-readable artifacts remain JSON, including HAK manifests,
topdata datasets, caches, credits inventories, build reports, and canonical GFF
documents. Nested metadata such as `topdata/templates/config.yaml` remains
valid when it is local workflow configuration rather than root repository
configuration.
## Wiki And Changelog Utilities
`build-topdata --wiki` or `build-wiki` renders wiki pages from the current
native topdata state as Tiptap-compatible NodeBB HTML. Page templates are HTML
fragments; they do not need to include a visible title heading because deploy
uses generated `page-index.json` metadata for page titles and public paths.
`deploy-wiki` publishes those pages to NodeBB. It reads
`NODEBB_API_ENDPOINT`, `NODEBB_API_TOKEN`, `GITHUB_REF_NAME`, and
`NODEBB_WIKI_CATEGORIES` when equivalent flags are omitted. Category mappings
use `namespace=cid`, for example:
```bash
sow-toolkit deploy-wiki --dry-run --namespace spells --category spells=42
```
Namespace declarations in `topdata/wiki/namespaces.yaml` may set
`canonical_path` when a generated namespace lives below a fuller NodeBB category
path, for example `canonical_path: Wiki/Feats`. The `title` remains the display
label, while `canonical_path` drives generated `public_path` metadata, wiki
links, page-index entries, and deploy adoption matching.
When a deploy manifest is missing but matching wiki pages already exist in
NodeBB, `deploy-wiki` checks the Westgate Wiki namespace directory API and
adopts those topic/post mappings before planning updates. It does the same when
a cached manifest mapping points to a post NodeBB no longer has. Canonical
`wikiPath`/`canonicalPath` data is preferred during adoption. Nested generated
paths require that exact canonical path data; leaf/title compatibility matching
is only used where it cannot cross a canonical parent path. `--create` is still
required for genuinely new pages that have no existing remote topic. If NodeBB
rejects a create because the clean wiki URL already exists, `deploy-wiki`
re-queries the namespace by page leaf, adopts the matching topic, and updates it
instead. Pass `--debug` when deploying large wiki changes to emit planning and
live mutation progress. Updates and stale-page archives acquire a Westgate Wiki
edit lock before saving, then send the returned
`wikiEditLockToken` with the NodeBB post edit request. Creates, updates, and
archives write the same managed HTML to both NodeBB `content` and
`sourceContent`; the deploy manifest records `source_content_synced` so older
manifests receive a one-time body repair even when the generated hash is
unchanged. Stale generated pages are reported by default.
`--stale-policy archive` rewrites stale generated pages in place as archived
managed content. `--stale-policy purge` permanently removes only stale generated
topics already tracked in the wiki deploy manifest. Dry-run that policy before
the live run; it never discovers manual wiki pages from NodeBB for purge.
`--reset-managed-namespaces` is a stronger destructive mode for slug or path
standard migrations: it permanently removes every topic currently listed by the
configured managed namespace categories, ignores the previous deploy manifest,
and recreates the current generated pages. It requires `--create` when local
generated pages are present.
For newly created NodeBB topics, `topdata.wiki.title_prefix_min_length`
controls when the deployer prefixes the namespace in the topic title. The
default is `3`, so one- and two-character page titles are created as
`Namespace: Title`, while titles with three or more characters keep the page
title alone. Existing bot-managed pages with older prefixed titles are renamed
on later deploy passes when their current title no longer matches this policy.
`build-changelog` renders a release changelog from tagged pushes. Pull-request
style commits keep their PR links and authors from the Gitea API; direct pushes
are rendered as commit links using the commit author. It defaults to
`scripts/changelog.json`, reads Gitea tokens from `GITEA_API_TOKEN`,
`GITEA_TOKEN`, or `SOW_TOOLS_TOKEN`, and writes to stdout unless `--output` is
provided.
## Intended Consumers
- `sow-module` uses `sow-toolkit` for `build`, `build-module`, `extract`, `validate`, `compare`, and `apply-hak-manifest`
- `sow-assets` uses `sow-toolkit` for `build-haks`, music processing, changelog generation, and validation
- validation now fails early if duplicate asset resources would collide inside the same generated HAK
- duplicate resources split across different HAK groups remain warnings because later HAK order can still override earlier content at runtime
- `sow-module` now also uses `sow-toolkit` for topdata:
- `validate-topdata`
- `build-topdata`
- `build-top-package`
- `compare-topdata`
- `convert-topdata`
- `build-wiki`
- `deploy-wiki`
During development, sibling repositories can point at `../sow-tools/tools/sow-toolkit`
or `../sow-tools/tools/sow-toolkit.exe`. For pinned releases, each consumer repo can
instead carry its own copy in its local `tools/` directory.
## Cross-Repository Behavior
The current ownership split is:
- `sow-tools`
- owns the shared CLI and build logic
- owns module build, extract, compare, and HAK manifest application logic
- owns topdata validation, build, compare, conversion, and packaging logic
- `sow-module`
- owns canonical module source under `src/`
- owns canonical topdata authored data under `topdata/`
- owns the user-facing module and topdata wrapper commands
- `sow-assets`
- owns canonical binary asset source under `assets/`
- owns generated HAK manifests and HAK-side parts 2DA assets
- owns the user-facing asset wrapper commands
Consumer wrapper resolution is intentionally aligned:
1. Prefer a pinned local binary in `tools/`
2. Else prefer a sibling `../sow-tools` checkout for local source development
3. Else install the latest published `sow-tools` release artifact
NWScript compilation behavior is also aligned and configurable under
`scripts.compiler`:
1. Prefer `SOW_NWN_SCRIPT_COMPILER` when explicitly set
2. Else prefer a local compiler in `tools/script-compiler/` or `tools/`
3. Else use a system-installed `nwn_script_comp` from `PATH`
When the compiler runs, the toolkit prepares `NWN_HOME` / `NWN_USER_DIRECTORY`
automatically and tries to detect `NWN_ROOT` from common Steam locations, including
custom Steam library folders, before an explicit override is required.
## Release Automation
This repo publishes:
- `sow-toolkit-linux-amd64.tar.gz`
- `sow-toolkit-windows-amd64.exe`
Consumer repos can fetch those binaries into their local `tools/` directory with:
- `install-tool.sh` on Linux/macOS
- `install-tool.ps1` on Windows
## TopData
The native topdata system is implemented in this repo and authored in `sow-module`.
Generated native topdata lockfiles use matching `.editorconfig` JSON formatting
settings when a consumer repository provides them.
High-level ownership split:
- `sow-tools`
- owns the `sow-toolkit` implementation
- owns topdata validation, build, compare, conversion, and dataset-resolution logic
- `sow-module`
- owns canonical topdata authored data under `topdata/data/`
- owns the user-facing topdata workflow scripts
For a human-readable front-to-back explanation of how the native topdata builder works
and how to use it, see:
- [sow-module/topdata/README.md](../sow-module/topdata/README.md)
Important topdata contracts still live here under `internal/topdata/` because they are
code-adjacent implementation contracts, not user workflow docs.
How the artifact repos resolve a Crucible binary (and the `NWN_ROOT` rule) is
documented in [`docs/consumer-contract.md`](docs/consumer-contract.md).