Wrapper tool hardening and cleanup

This commit is contained in:
2026-05-13 19:06:24 +02:00
parent 53cb1e6996
commit 7d286551b3
19 changed files with 1376 additions and 149 deletions
+19
View File
@@ -1,5 +1,24 @@
# Changelog Automation Contract # Changelog Automation Contract
## Status Snapshot
Current state as of 2026-05-13:
- The shared implementation is present in `internal/changelog/` and is exposed
as `sow-toolkit build-changelog`.
- Consumer configs exist in both `module/scripts/changelog.json` and
`assets/scripts/changelog.json`.
- CLI parsing and generator behavior are covered by
`internal/app/app_test.go` and `internal/changelog/changelog_test.go`.
- `module/scripts/release-all.sh` already stages `build/CHANGELOG.md`.
Remaining work:
- confirm and document the `assets/` release workflow uses the shared command in
the same way as `module/`
- build the deferred cross-repo NodeBB aggregation bot if combined public release
posts are still desired
## Scope ## Scope
This contract governs release changelog generation across: This contract governs release changelog generation across:
+22
View File
@@ -1,5 +1,27 @@
Create a long-term multi-repository hardening plan focused on making YAML configuration the authoritative source of truth. Create a long-term multi-repository hardening plan focused on making YAML configuration the authoritative source of truth.
## Status Snapshot
Current state as of 2026-05-13:
- This hardening effort is active and partially complete.
- YAML authority, strict config loading, and effective-config inspection are
implemented.
- Major repository behavior now flows through normalized config, including HAK
layout, topdata packaging, extract behavior, autogen manifests, scripts, wiki,
and music datasets.
- The remaining gaps are mostly about reducing legacy compatibility surface,
moving more tests to YAML fixtures, and deciding which residual toolkit
defaults stay intrinsic versus become explicit schema.
Primary outstanding work:
- reduce reliance on `nwn-tool.json` in tests and migration fixtures
- keep pruning repository-shaped defaults that still live in code when they
should be surfaced as schema-backed settings
- maintain docs/examples for `config effective` and the normalized key space as
the schema evolves
# Goal # Goal
Reduce hardcoded behavior across all repositories and toolkit code. Reduce hardcoded behavior across all repositories and toolkit code.
+77
View File
@@ -1,5 +1,82 @@
Refactor toolkit configuration from JSON to YAML only. Refactor toolkit configuration from JSON to YAML only.
## Status Snapshot
Current state as of 2026-05-13:
- This refactor is largely implemented.
- Root config discovery now prefers `nwn-tool.yaml`, then `nwn-tool.yml`, then
legacy `nwn-tool.json`.
- YAML is canonical in the consumer repos, and the toolkit exposes
`config validate|effective|inspect|explain|sources`.
- Legacy JSON compatibility still exists by design and is still used heavily in
test fixtures.
Remaining work:
- continue migrating representative tests from `nwn-tool.json` fixtures to YAML
while keeping only targeted legacy coverage
- decide when to deprecate or remove legacy shorthand such as `music.prefixes`
after consumer configs are fully dataset-based
- keep documentation/examples synchronized with the normalized YAML schema as it
expands
## Active Scope
This contract remains active and is one of the current priorities.
The implementation gap is no longer "load YAML at all". The real gap is
configuration authority drift:
- some repository behavior is still encoded as toolkit defaults when it should
become explicit schema
- some tests still model old JSON-era expectations
- some topdata behavior is still driven by explicit namespace/path checks rather
than generic dataset rules
## Current Findings
The most important remaining configuration work is topdata-related:
- topdata dataset discovery already derives default output names when `output` is
absent, but validation still hard-fails for
`topdata/data/masterfeats/base.json` when `output` is missing
- `internal/topdata/topdata.go` dispatches validation through explicit path
checks for `masterfeats`, `feat`, parts overrides, and racialtypes registry
- topdata defaults in `internal/project` still mix generic toolkit defaults with
behavior that may need clearer schema ownership
This means the config-refactor track now overlaps directly with topdata
hardening. The core question is no longer file format migration; it is whether a
behavior belongs in:
- root YAML
- dataset-authored JSON
- or a documented intrinsic toolkit invariant
## Deferred Scope
Defer unrelated schema expansion for now:
- model compilation config
- broader asset-maintenance policy schema
- release-announcement config
## Immediate Plan
1. Continue shrinking legacy `nwn-tool.json` coverage down to deliberate
compatibility tests.
2. Audit hardcoded topdata behaviors and classify each as:
- root YAML concern
- dataset JSON concern
- toolkit invariant
3. Remove outdated namespace-specific validation assumptions where generic
dataset logic already exists.
4. Keep generated artifacts in JSON, but stop using explicit path-based logic
when the same rule can be inferred from generic dataset shape.
5. Document every remaining intentional toolkit default in `config effective`
terms.
# Context # Context
Repositories currently use `config.json` for human-authored toolkit configuration. Repositories currently use `config.json` for human-authored toolkit configuration.
+66
View File
@@ -1,5 +1,71 @@
You are improving CLI/build logs for a NWN asset toolchain. You are improving CLI/build logs for a NWN asset toolchain.
## Status Snapshot
Current state as of 2026-05-13:
- This work is largely implemented for the current `build-haks` path.
- The app layer now exposes `--quiet`, `--verbose`, and `--debug`, and normal
mode prints compact music summaries instead of raw per-file mappings.
- Detailed mapping output is available behind verbose logging, and app tests
cover the compact versus detailed presentation split.
Remaining work:
- continue applying the same presentation discipline to other command families
where raw pipeline output still leaks through
- keep summary wording and phase naming aligned as new generated-asset stages
are added
## Active Scope
Logging remains an active priority, but no longer for `build-haks` alone.
The next logging phase should target:
- `build-topdata`
- `validate-topdata`
- `compare-topdata`
- `build-wiki`
- `deploy-wiki`
- release wrapper flows that currently interleave raw shell/tool output
## Current Findings
The asset-side log refactor proved the presentation model. The remaining gap is
uneven command coverage:
- topdata and wiki commands do not yet have the same polished phase/summarized
output contract as `build-haks`
- release scripts still emit long shell-oriented logs that are useful for CI
diagnosis but not shaped into the same structured presentation model
- wrapper-side status lines and toolkit-side progress lines are not yet fully
harmonized
## Deferred Scope
Defer cosmetic-only logging work that does not improve operator clarity.
For now, prioritize:
- structured summaries
- deterministic phase labels
- stderr/stdout hygiene
- parity between topdata/wiki flows and the existing build-haks presentation
## Immediate Plan
1. Reuse the existing verbosity model for topdata and wiki command families.
2. Define concise normal-mode summaries for topdata build/validate/compare.
3. Add deploy/build wiki progress summaries that surface:
- local pages scanned
- create/update/skip/drift counts
- manifest path
4. Keep debug mode capable of exposing raw internal progress when diagnosis is
required.
5. Extend app-layer tests to cover topdata/wiki presentation once those
commands adopt the structured output contract.
Goal: Make the logs cleaner, prettier, less repetitive, and easier to understand, without hiding important diagnostics. Goal: Make the logs cleaner, prettier, less repetitive, and easier to understand, without hiding important diagnostics.
Current issues: Current issues:
@@ -1,5 +1,22 @@
# Model Compilation Build-Time Contract # Model Compilation Build-Time Contract
## Status Snapshot
Current state as of 2026-05-13:
- This contract is still outstanding.
- `build-haks` does not currently compile authored ASCII `.mdl` files into
binary build artifacts.
- Toolkit validation and extraction understand `.mdl`, but there is no
config-backed build-time model compiler pipeline in `internal/pipeline/`.
Open work remains exactly in the areas this contract describes:
- schema and validation for `models.compile.*`
- compiler/tool resolution and staging
- generated-asset overlay integration in HAK builds
- cache/reuse semantics and regression coverage
## Scope ## Scope
Expand the toolkit so authored ASCII Neverwinter Nights `.mdl` assets can be compiled into binary `.mdl` build artifacts during `sow-toolkit build-haks` without changing the authored source tree. Expand the toolkit so authored ASCII Neverwinter Nights `.mdl` assets can be compiled into binary `.mdl` build artifacts during `sow-toolkit build-haks` without changing the authored source tree.
+19
View File
@@ -1,5 +1,24 @@
Refactor the music generation / editing / credits pipeline so it exists entirely inside the shared toolkit and is no longer hardcoded or repository-specific. Refactor the music generation / editing / credits pipeline so it exists entirely inside the shared toolkit and is no longer hardcoded or repository-specific.
## Status Snapshot
Current state as of 2026-05-13:
- This refactor is largely implemented.
- Music logic now lives under `internal/music/` with toolkit-owned CLI entry
points and shared use from `build-haks`.
- Dataset-based config is supported and documented; the toolkit exposes
`music list-datasets|scan|build|credits|validate|manifest|normalize`.
- Incremental credits artifact handling and compact/verbose log presentation are
present in the current implementation.
Remaining work:
- finish retiring old `music.prefixes`-only assumptions in docs/tests after all
consumer configs are fully dataset-native
- continue separating any remaining music-specific presentation rules from
broader `build-haks` output policy as the build graph evolves
Goal: Turn the current Shadows Over Westgate-specific music pipeline into a reusable, repository-agnostic toolkit subsystem that can operate across multiple projects and datasets with configuration-driven behavior. Goal: Turn the current Shadows Over Westgate-specific music pipeline into a reusable, repository-agnostic toolkit subsystem that can operate across multiple projects and datasets with configuration-driven behavior.
# Requirements # Requirements
+70
View File
@@ -1,5 +1,75 @@
Refactor and harden all shell wrapper scripts across the module and assets repositories. Refactor and harden all shell wrapper scripts across the module and assets repositories.
## Status Snapshot
Current state as of 2026-05-13:
- Argument passthrough is largely solved for the active root wrappers.
- `module/` and `assets/` now use repo-root launchers plus
`scripts/run-nwn-tool.*` passthrough shims.
- Wrapper regression coverage already exists in `module/tests/` and
`assets/tests/`.
- The previously stale JSON-config assumption shown in this contract is no
longer current; active repos use `nwn-tool.yaml`.
## Active Scope
This contract is still active, but the priority has changed.
The old priority was:
- make `$@` / `@args` forwarding work
The current priority is:
- remove wrapper-side business logic that still duplicates toolkit behavior
- keep wrappers thin across shell, PowerShell, and release-entrypoint paths
- align wrapper logs and error handling with the current logging contract
- finish wrapper/tooling parity before moving deeper into wiki deployment
## Current Findings
The main active wrapper gap is no longer argument forwarding. The bigger gaps are
wrapper thickness and duplicated orchestration:
- `module/build-module.sh` still shells out to `config effective --json`, parses
config locally, probes script sources, resolves manifest paths, and performs
pre-build manifest refresh.
- `module/build-module.ps1` mirrors the same logic in a second implementation.
- `module/scripts/release-all.sh` and `assets/scripts/release-haks.sh` both
parse effective config themselves and still own substantial orchestration that
is not just "launch the toolkit".
That means the wrapper contract should now optimize for:
- a single source of truth for preflight behavior
- thinner root wrappers
- shared repo plumbing only where toolkit ownership is not appropriate
- no duplicated config decoding logic between `.sh` and `.ps1`
## Deferred Scope
Defer unrelated wrapper-adjacent work for now:
- model-compilation wrapper concerns
- release-announcement publication scripting
- broader asset-processing script consolidation outside wrapper ownership
## Immediate Plan
1. Audit every wrapper that does more than `cd` plus `exec`.
2. Classify wrapper logic into:
- toolkit-owned behavior
- shared repo plumbing
- release-only orchestration
3. Move duplicated config reads and command decisions out of paired `.sh` and
`.ps1` wrappers where possible.
4. Keep root wrappers as thin launchers with transparent argument forwarding.
5. Expand regression coverage from passthrough correctness into:
- wrapper-side preflight behavior
- stderr/stdout separation for JSON-producing subcommands
- shell/PowerShell parity for any remaining unavoidable wrapper logic
# Problem # Problem
The current `.sh` and `.ps1` wrapper scripts do not properly forward arbitrary CLI arguments to the underlying toolkit commands. The current `.sh` and `.ps1` wrapper scripts do not properly forward arbitrary CLI arguments to the underlying toolkit commands.
+197
View File
@@ -0,0 +1,197 @@
# Native Topdata Hardening Contract
## Scope
This contract governs the current topdata hardening and bugfixing phase across:
- `toolkit/` as implementation owner
- `module/` as the primary authored topdata consumer
This is an active-priority contract.
## Current Priority
Focus now on:
1. removing outdated explicit namespace/path hardcoding
2. fixing validation/build mismatches
3. making native topdata behavior generic where the data model is already generic
4. improving diagnostics and operator-facing output for topdata commands
Defer for now:
- model compilation
- release patch-note automation
- unrelated asset-processing expansion
- non-topdata feature work that does not unblock wiki deployment
Wiki deployment itself is a follow-on phase after this hardening pass, because
`build-wiki` and `deploy-wiki` depend on topdata outputs and metadata being
stable.
## Problem Statement
Native topdata has reached the point where most major functionality exists, but
too much behavior is still implemented as explicit namespace knowledge instead of
generic dataset rules.
One concrete example already observed:
- dataset discovery in `internal/topdata/native.go` already derives a default
`output` name when `base.json` omits it
- validation in `internal/topdata/topdata.go` still hard-fails specifically for
`topdata/data/masterfeats/base.json` if `output` is missing
That mismatch proves the current methodology is stale:
- the runtime/build side treats `masterfeats` like a generic dataset
- the validator still treats it like a special namespace with explicit required
metadata
This same category of problem appears in several other places.
## Findings From Current Audit
### 1. Validation dispatch is path-special-cased
`validateDataObject()` currently routes behavior through explicit path checks for:
- `masterfeats/base.json`
- `masterfeats/lock.json`
- `feat/base.json`
- `feat/lock.json`
- `feat/generated/*`
- `parts/overrides/*`
- `racialtypes/registry/races/*`
Some of these are legitimate dataset-shape differences. Others are stale
special-cases that should be generalized.
### 2. `masterfeats` still carries outdated explicit validation rules
`validateMasterfeatsBaseFile()` currently requires:
- `output == masterfeats.2da`
- specific required columns
- `masterfeats:` row keys
The key-prefix and column requirements may still be valid dataset invariants.
The `output` requirement is not clearly justified anymore when generic dataset
discovery already computes a default output name from location.
### 3. Migration still seeds explicit dataset exceptions
`feat_migrate.go` explicitly writes:
- `output = "feat.2da"`
- `compare_reference = false`
This may still be operationally correct, but it means migration continues to
materialize special-case control fields instead of relying on a clearer generic
contract for dataset reference comparison.
### 4. Project-level topdata defaults are still mixed
`internal/project` still owns several topdata defaults directly:
- build directory derivation
- compiled 2DA subdirectory
- compiled TLK name
- wiki root/pages/state defaults
- wiki deploy manifest and edit summary defaults
- package HAK/TLK fallback names
Some of these are valid toolkit defaults. Some should be reviewed as explicit
schema-backed config.
### 5. Wiki generation and deploy depend on stable dataset semantics
`build-wiki` and `deploy-wiki` are downstream consumers of native topdata state.
If dataset rules remain partly explicit and partly generic, wiki generation will
keep inheriting brittle assumptions.
## Hardening Rules
1. If a rule can be inferred generically from dataset shape, do not hardcode it
to a namespace path.
2. If a rule is truly dataset-specific, document why it is intrinsic rather than
configuration-driven.
3. Validation and build behavior must agree. Discovery defaults and validation
requirements cannot contradict each other.
4. Dataset-authored JSON should remain the source of truth for dataset-level
semantics; root YAML should not absorb dataset internals just to avoid code
cleanup.
5. Error messages must point to the actual invariant being violated, not an
outdated historical implementation detail.
## Immediate Work Items
### Work Item 1: Remove outdated `masterfeats.output` hard requirement
Investigate and likely change `validateMasterfeatsBaseFile()` so:
- missing `output` is accepted when generic dataset discovery can derive the same
output name deterministically
- explicit `output` remains allowed
- wrong explicit `output` still fails clearly if the dataset truly requires a
fixed emitted file name
This should be test-covered.
### Work Item 2: Audit path-based validator routing
Classify every explicit route in `validateDataObject()` as one of:
- generic canonical shape handling
- dataset-family behavior that should become generic
- deliberate intrinsic special-case with justification
Reduce the explicit route set where possible.
### Work Item 3: Define topdata invariant boundaries
For each current explicit topdata behavior, decide whether it belongs in:
- root YAML config
- dataset JSON
- toolkit invariant
Do not move dataset-authored semantics into YAML unless the behavior truly varies
at repository level rather than dataset level.
### Work Item 4: Revisit `compare_reference` ownership
`feat.2da` is still excluded via `compare_reference = false`. Confirm whether:
- this remains a temporary rollout flag
- or compare coverage needs a more explicit generic contract for generated-family
datasets
### Work Item 5: Bring topdata logging up to current CLI standards
Coordinate with `LOG_REFACTOR_CONTRACT.md` so topdata commands emit:
- deterministic phase summaries
- concise validation/build/compare totals
- clear file/namespace diagnostics
## Testing Requirements
Add or update tests for:
- `masterfeats/base.json` without explicit `output` when the derived output would
still be `masterfeats.2da`
- explicit wrong `output` for `masterfeats`
- validator/build parity for dataset output naming
- any reduction in path-special-case validation routing
- topdata command output summaries if presentation changes land in the same phase
## Acceptance Criteria
- topdata validation no longer rejects data solely because of stale explicit
namespace assumptions
- build and validation agree on dataset output-name behavior
- path-special-casing is reduced or explicitly justified
- remaining dataset-specific invariants are documented as such
- topdata hardening leaves wiki generation/deployment with a cleaner and more
stable contract
+108
View File
@@ -0,0 +1,108 @@
# Wiki Deployment Contract
## Scope
This contract governs the next focused phase after wrappers, config refactor,
logging, and topdata hardening:
- `toolkit/` owns `build-wiki` and `deploy-wiki`
- `module/` owns the consuming release workflow and operational configuration
This work is intentionally sequenced after topdata hardening because the wiki
pipeline consumes native topdata outputs and metadata.
## Current State
As of 2026-05-13:
- `build-wiki` exists
- `deploy-wiki` exists
- deployment targets NodeBB core `/api/v3`
- local deploy manifests and NodeBB topic mapping exist
- release automation in `module/scripts/release-all.sh` already runs wiki build
and deploy steps after release publication
- toolkit tests already cover dry-run behavior, topic creation, and manifest
writes
This means wiki deployment is no longer a greenfield feature. The next phase is
hardening, integration clarity, and operator experience.
## Deferred Until Current Priority Work Completes
Do not prioritize the following until wrappers/config/logging/topdata work is in
better shape:
- new NodeBB release-announcement posting flows
- broad wiki feature expansion
- unrelated formatting changes to generated page content
## Immediate Goals For The Wiki Phase
1. align wiki build/deploy logs with the structured logging contract
2. tighten config ownership for wiki defaults and category mappings
3. reduce residual hardcoded behavior that belongs in normalized config
4. improve release-flow clarity around dry-run, drift, create, and force modes
5. verify that topdata hardening does not silently change generated page identity
## Current Findings
- `deploy-wiki` already reads managed namespaces from effective config when the
caller does not specify them explicitly
- deploy manifest naming and edit-summary defaults still come from toolkit
defaults in `internal/project/effective.go`
- release orchestration already guards publication order correctly, but the
operator-facing logs are still more shell-oriented than the newer structured
`build-haks` presentation
- wiki deployment should be treated as a first-class operational command family,
not just a release-script tail step
## Immediate Plan
### Phase 1: Logging and UX parity
- add concise summaries for `build-wiki` and `deploy-wiki`
- surface counts for:
- pages collected
- creates
- updates
- skips
- drifted pages
- keep raw debug output available when diagnosis is needed
### Phase 2: Config clarity
- audit which wiki defaults should remain toolkit defaults versus explicit repo
config
- keep namespace/category ownership visible in effective config
- ensure deploy-manifest path and edit summary remain discoverable and easy to
override
### Phase 3: Release-flow hardening
- verify `module/scripts/release-all.sh` remains the authoritative release entry
point
- keep dry-run before live deploy
- ensure failure and drift messages are crisp and actionable
- make sure cache restore/save behavior is clearly documented and tested
### Phase 4: Regression verification after topdata hardening
- confirm topdata de-hardcoding does not break page IDs or manifest mapping
- confirm generated-page identity remains stable across unchanged inputs
## Testing Requirements
Add or expand tests for:
- app-layer output for `build-wiki` and `deploy-wiki`
- release-flow sequencing assumptions where wiki deploy must happen only after
release assets are published
- stable page identity and manifest mapping across repeated runs
- config-driven namespace selection and manifest path overrides
## Acceptance Criteria
- wiki build/deploy logs match the newer structured CLI standard
- wiki defaults and overrides are clearly represented in effective config
- release orchestration remains deterministic and easy to reason about
- topdata hardening does not regress page identity or deploy manifest behavior
+416 -90
View File
@@ -6,7 +6,9 @@ import (
"fmt" "fmt"
"io" "io"
"os" "os"
"os/exec"
"path/filepath" "path/filepath"
"runtime"
"sort" "sort"
"strconv" "strconv"
"strings" "strings"
@@ -294,6 +296,9 @@ func runBuild(ctx context) error {
if err != nil { if err != nil {
return err return err
} }
if err := runBuildModulePreflight(ctx, p, nil); err != nil {
return err
}
result, err := pipeline.Build(p) result, err := pipeline.Build(p)
if err != nil { if err != nil {
@@ -321,12 +326,19 @@ func runBuildModule(ctx context) error {
return err return err
} }
result, err := pipeline.BuildModuleWithProgress(p, func(message string) { progress := func(message string) {
if ctx.logLevel == logLevelQuiet { if ctx.logLevel == logLevelQuiet {
return return
} }
spin.linebreak() spin.linebreak()
fmt.Fprintf(ctx.stdout, "[build-module] %s\n", message) fmt.Fprintf(ctx.stdout, "[build-module] %s\n", message)
}
if err := runBuildModulePreflight(ctx, p, progress); err != nil {
return err
}
result, err := pipeline.BuildModuleWithProgress(p, func(message string) {
progress(message)
}) })
if err != nil { if err != nil {
return err return err
@@ -338,6 +350,222 @@ func runBuildModule(ctx context) error {
return nil return nil
} }
func runBuildModulePreflight(ctx context, p *project.Project, progress func(string)) error {
if progress == nil {
progress = func(string) {}
}
if err := refreshBuildModuleManifest(ctx, p, progress); err != nil {
return err
}
if err := prepareBuildModuleCompiler(ctx, p, progress); err != nil {
return err
}
return nil
}
func refreshBuildModuleManifest(ctx context, p *project.Project, progress func(string)) error {
manifestPath := p.HAKManifestPath()
if override := strings.TrimSpace(os.Getenv("SOW_HAK_MANIFEST_PATH")); override != "" {
manifestPath = override
if !filepath.IsAbs(manifestPath) {
manifestPath = filepath.Join(p.Root, manifestPath)
}
}
if envEnabled("SOW_MODULE_SKIP_HAK_MANIFEST_REFRESH") {
if _, err := os.Stat(manifestPath); err != nil {
return fmt.Errorf("SOW_MODULE_SKIP_HAK_MANIFEST_REFRESH is set, but %s does not exist", manifestPath)
}
progress(fmt.Sprintf("Using existing hak manifest at %s; skipping published sow-assets refresh.", relPathFromRoot(p.Root, manifestPath)))
return nil
}
progress("Refreshing hak list from the latest published sow-assets manifest...")
if err := runProjectScript(ctx, p, []string{"scripts", "fetch-hak-manifest"}, manifestPath); err != nil {
return err
}
if _, err := pipeline.ApplyHAKManifest(p, manifestPath); err != nil {
return err
}
return nil
}
func prepareBuildModuleCompiler(ctx context, p *project.Project, progress func(string)) error {
if envEnabled("SOW_MODULE_SKIP_NSS_COMPILATION") {
progress("SOW_MODULE_SKIP_NSS_COMPILATION is set; skipping NWScript compiler resolution and NSS compilation for this run.")
return nil
}
if !projectHasScriptSources(p) {
return nil
}
compilerPathEnv := p.ScriptCompilerPathEnv()
explicitCompiler := strings.TrimSpace(os.Getenv(compilerPathEnv))
if explicitCompiler != "" {
if scriptCompilerRunnable(explicitCompiler) {
return nil
}
if fileExists(explicitCompiler) {
return disableBuildModuleCompiler(compilerPathEnv, progress, fmt.Sprintf("Configured NWScript compiler is present but cannot start: %s", explicitCompiler))
}
}
if _, ok := findUsableBuildModuleCompiler(p); ok {
return nil
}
installScript := projectScriptPath(p.Root, "scripts", "install-script-compiler")
if !fileExists(installScript) {
return disableBuildModuleCompiler(compilerPathEnv, progress, "NWScript compiler not found locally and automatic installer script is unavailable.")
}
progress("NWScript compiler not found locally. Installing it now...")
if err := runProjectScript(ctx, p, []string{"scripts", "install-script-compiler"}); err != nil {
return disableBuildModuleCompiler(compilerPathEnv, progress, "Automatic NWScript compiler install failed.")
}
if _, ok := findUsableBuildModuleCompiler(p); ok {
return nil
}
return disableBuildModuleCompiler(compilerPathEnv, progress, "Installed NWScript compiler but it still cannot start.")
}
func disableBuildModuleCompiler(compilerPathEnv string, progress func(string), reason string) error {
if err := os.Setenv("SOW_MODULE_SKIP_NSS_COMPILATION", "1"); err != nil {
return fmt.Errorf("set SOW_MODULE_SKIP_NSS_COMPILATION: %w", err)
}
progress("Skipping NWScript compilation for this build. " + reason)
if compilerPathEnv != "" {
_ = os.Unsetenv(compilerPathEnv)
}
return nil
}
func projectHasScriptSources(p *project.Project) bool {
for _, rel := range p.Inventory.ScriptFiles {
if strings.EqualFold(filepath.Ext(rel), ".nss") {
return true
}
}
return false
}
func findUsableBuildModuleCompiler(p *project.Project) (string, bool) {
candidates := make([]string, 0, 2+len(p.ScriptCompilerSearchPaths()))
if configured := p.ScriptCompilerPath(); configured != "" {
candidates = append(candidates, configured)
}
if configured := strings.TrimSpace(os.Getenv(p.ScriptCompilerPathEnv())); configured != "" {
candidates = append(candidates, configured)
}
candidates = append(candidates, p.ScriptCompilerSearchPaths()...)
for _, candidate := range candidates {
if compiler, ok := resolveRunnableCompilerCandidate(candidate); ok {
return compiler, true
}
}
return "", false
}
func resolveRunnableCompilerCandidate(candidate string) (string, bool) {
if candidate == "" {
return "", false
}
if binary, ok := strings.CutPrefix(candidate, "PATH:"); ok {
compiler, err := exec.LookPath(binary)
if err != nil {
return "", false
}
if scriptCompilerRunnable(compiler) {
return compiler, true
}
return "", false
}
if !fileExists(candidate) {
return "", false
}
if scriptCompilerRunnable(candidate) {
return candidate, true
}
return "", false
}
func scriptCompilerRunnable(path string) bool {
if !fileExists(path) {
return false
}
cmd := exec.Command(path, "--help")
cmd.Stdout = io.Discard
cmd.Stderr = io.Discard
return cmd.Run() == nil
}
func runProjectScript(ctx context, p *project.Project, scriptParts []string, args ...string) error {
if len(scriptParts) == 0 {
return errors.New("project script path is required")
}
scriptPath := projectScriptPath(p.Root, scriptParts...)
if !fileExists(scriptPath) {
return fmt.Errorf("required project script is missing: %s", scriptPath)
}
var cmd *exec.Cmd
switch filepath.Ext(scriptPath) {
case ".ps1":
powershell := "powershell"
if runtime.GOOS != "windows" {
powershell = "pwsh"
}
commandArgs := []string{"-NoProfile", "-File", scriptPath}
commandArgs = append(commandArgs, args...)
cmd = exec.Command(powershell, commandArgs...)
default:
cmd = exec.Command(scriptPath, args...)
}
cmd.Dir = p.Root
cmd.Stdout = ctx.stderr
cmd.Stderr = ctx.stderr
return cmd.Run()
}
func projectScriptPath(root string, pathParts ...string) string {
parts := append([]string{root}, pathParts...)
base := filepath.Join(parts...)
if runtime.GOOS == "windows" {
if strings.HasSuffix(base, ".ps1") || strings.HasSuffix(base, ".sh") {
return base
}
return base + ".ps1"
}
if strings.HasSuffix(base, ".ps1") || strings.HasSuffix(base, ".sh") {
return base
}
return base + ".sh"
}
func envEnabled(name string) bool {
switch strings.ToLower(strings.TrimSpace(os.Getenv(name))) {
case "1", "true", "yes", "on":
return true
default:
return false
}
}
func fileExists(path string) bool {
info, err := os.Stat(path)
return err == nil && !info.IsDir()
}
func relPathFromRoot(root, path string) string {
rel, err := filepath.Rel(root, path)
if err != nil {
return filepath.ToSlash(path)
}
return filepath.ToSlash(rel)
}
type buildHAKOptions struct { type buildHAKOptions struct {
filteredHAKs []string filteredHAKs []string
filteredArchives []string filteredArchives []string
@@ -530,6 +758,161 @@ func (c *buildHAKConsole) relPath(path string) string {
return filepath.ToSlash(rel) return filepath.ToSlash(rel)
} }
type topdataConsole struct {
stdout io.Writer
projectRoot string
projectName string
commandName string
commandLabel string
level logLevel
spinnerEnabled bool
}
func newTopdataConsole(ctx context, p *project.Project, commandName string) *topdataConsole {
return &topdataConsole{
stdout: ctx.stdout,
projectRoot: p.Root,
projectName: p.Config.Module.Name,
commandName: commandName,
commandLabel: topdataCommandLabel(commandName),
level: ctx.logLevel,
spinnerEnabled: isInteractiveTTY(ctx.stderr) && strings.TrimSpace(os.Getenv("CI")) == "" && ctx.logLevel == logLevelNormal,
}
}
func topdataCommandLabel(commandName string) string {
switch commandName {
case "validate-topdata":
return "Validate Topdata"
case "build-topdata":
return "Build Topdata"
case "build-top-package":
return "Build Top Package"
case "compare-topdata":
return "Compare Topdata"
case "build-wiki":
return "Build Wiki"
case "deploy-wiki":
return "Deploy Wiki"
default:
return commandName
}
}
func (c *topdataConsole) progress(message string) {
if phase := c.phaseLabel(message); phase != "" {
spin.update(c.commandLabel + ": " + phase)
}
if c.level != logLevelDebug {
return
}
spin.linebreak()
fmt.Fprintf(c.stdout, "[debug] %s\n", message)
}
func (c *topdataConsole) phaseLabel(message string) string {
switch {
case strings.HasPrefix(message, "Building native topdata outputs"):
return "building native outputs"
case strings.HasPrefix(message, "Topdata outputs are current"):
return "checking cached outputs"
case strings.HasPrefix(message, "Compiled topdata outputs are current"):
return "refreshing top package"
case strings.HasPrefix(message, "Packaging compiled topdata resources into "):
return "packaging HAK"
case strings.HasPrefix(message, "Preparing compiled TLK release output "):
return "packaging TLK"
case strings.HasPrefix(message, "Building native wiki pages"):
return "building wiki pages"
case strings.HasPrefix(message, "NodeBB wiki plan: "):
return "planning wiki deploy"
case strings.HasPrefix(message, "Collecting "):
return "collecting pages"
default:
return ""
}
}
func (c *topdataConsole) emitValidationResult(report topdata.ValidationReport, topdataRoot string) {
spin.linebreak()
fmt.Fprintln(c.stdout, "Validate Topdata ----------")
fmt.Fprintf(c.stdout, "project: %s\n", c.projectName)
fmt.Fprintf(c.stdout, "topdata root: %s\n", c.relPath(topdataRoot))
fmt.Fprintf(c.stdout, "topdata files: %d\n", report.Files)
fmt.Fprintf(c.stdout, "data files: %d\n", report.DataFiles)
fmt.Fprintf(c.stdout, "tlk files: %d\n", report.TLKFiles)
if warnings := report.WarningCount(); warnings > 0 {
fmt.Fprintf(c.stdout, "warnings: %d\n", warnings)
}
fmt.Fprintln(c.stdout, "topdata validation: ok")
}
func (c *topdataConsole) emitPackageResult(result topdata.PackageResult) {
spin.linebreak()
fmt.Fprintf(c.stdout, "%s ----------\n", c.commandLabel)
fmt.Fprintf(c.stdout, "project: %s\n", c.projectName)
fmt.Fprintf(c.stdout, "mode: %s\n", result.Mode)
fmt.Fprintf(c.stdout, "topdata 2da output: %s\n", c.relPath(result.Output2DADir))
fmt.Fprintf(c.stdout, "topdata tlk output: %s\n", c.relPath(result.OutputTLKDir))
fmt.Fprintf(c.stdout, "top package hak: %s\n", c.relPath(result.OutputHAKPath))
fmt.Fprintf(c.stdout, "top package tlk: %s\n", c.relPath(result.OutputTLKPath))
fmt.Fprintf(c.stdout, "2da files: %d\n", result.Files2DA)
fmt.Fprintf(c.stdout, "tlk files: %d\n", result.FilesTLK)
fmt.Fprintf(c.stdout, "top package resources: %d\n", result.HAKResources)
fmt.Fprintf(c.stdout, "top package assets: %d\n", result.AssetFiles)
if result.WikiOutputDir != "" {
fmt.Fprintf(c.stdout, "wiki output: %s\n", c.relPath(result.WikiOutputDir))
fmt.Fprintf(c.stdout, "wiki pages: %d\n", result.WikiPages)
fmt.Fprintf(c.stdout, "wiki status: %s\n", result.WikiStatus)
}
}
func (c *topdataConsole) emitCompareResult(result topdata.CompareResult) {
spin.linebreak()
fmt.Fprintln(c.stdout, "Compare Topdata ----------")
fmt.Fprintf(c.stdout, "project: %s\n", c.projectName)
fmt.Fprintf(c.stdout, "mode: %s\n", result.Mode)
fmt.Fprintf(c.stdout, "checked 2da files: %d\n", result.Compared2DA)
fmt.Fprintf(c.stdout, "checked tlk files: %d\n", result.ComparedTLK)
fmt.Fprintf(c.stdout, "native-pass: %d\n", result.NativePass)
fmt.Fprintf(c.stdout, "not-yet-canonical: %d\n", result.NotYetCanonical)
fmt.Fprintf(c.stdout, "native-mismatch: %d\n", result.NativeMismatch)
fmt.Fprintln(c.stdout, "topdata compare: ok")
}
func (c *topdataConsole) emitWikiBuildResult(outputDir string, pageCount int, status string) {
spin.linebreak()
fmt.Fprintln(c.stdout, "Build Wiki ----------")
fmt.Fprintf(c.stdout, "project: %s\n", c.projectName)
fmt.Fprintf(c.stdout, "wiki output: %s\n", c.relPath(outputDir))
fmt.Fprintf(c.stdout, "wiki pages: %d\n", pageCount)
fmt.Fprintf(c.stdout, "wiki status: %s\n", status)
}
func (c *topdataConsole) emitWikiDeployResult(localPages, created, updated, skipped, stale, drifted int, manifest string) {
spin.linebreak()
fmt.Fprintln(c.stdout, "Deploy Wiki ----------")
fmt.Fprintf(c.stdout, "project: %s\n", c.projectName)
fmt.Fprintf(c.stdout, "local pages: %d\n", localPages)
fmt.Fprintf(c.stdout, "created: %d\n", created)
fmt.Fprintf(c.stdout, "updated: %d\n", updated)
fmt.Fprintf(c.stdout, "skipped: %d\n", skipped)
fmt.Fprintf(c.stdout, "stale: %d\n", stale)
fmt.Fprintf(c.stdout, "drifted: %d\n", drifted)
fmt.Fprintf(c.stdout, "manifest: %s\n", c.relPath(manifest))
}
func (c *topdataConsole) relPath(path string) string {
if path == "" {
return path
}
rel, err := filepath.Rel(c.projectRoot, path)
if err != nil {
return filepath.ToSlash(path)
}
return filepath.ToSlash(rel)
}
func formatCount(value int) string { func formatCount(value int) string {
raw := strconv.Itoa(value) raw := strconv.Itoa(value)
if value < 1000 && value > -1000 { if value < 1000 && value > -1000 {
@@ -1145,21 +1528,14 @@ func runValidateTopData(ctx context) error {
return err return err
} }
console := newTopdataConsole(ctx, p, "validate-topdata")
report := topdata.ValidateProject(p) report := topdata.ValidateProject(p)
emitTopdataValidationReport(ctx.stderr, report) emitTopdataValidationReport(ctx.stderr, report)
if report.HasErrors() { if report.HasErrors() {
return fmt.Errorf("topdata validation failed with %d error(s)", report.ErrorCount()) return fmt.Errorf("topdata validation failed with %d error(s)", report.ErrorCount())
} }
fmt.Fprintf(ctx.stdout, "project: %s\n", p.Config.Module.Name) console.emitValidationResult(report, p.TopDataSourceDir())
fmt.Fprintf(ctx.stdout, "topdata root: %s\n", p.TopDataSourceDir())
fmt.Fprintf(ctx.stdout, "topdata files: %d\n", report.Files)
fmt.Fprintf(ctx.stdout, "data files: %d\n", report.DataFiles)
fmt.Fprintf(ctx.stdout, "tlk files: %d\n", report.TLKFiles)
if warnings := report.WarningCount(); warnings > 0 {
fmt.Fprintf(ctx.stdout, "warnings: %d\n", warnings)
}
fmt.Fprintf(ctx.stdout, "topdata validation: ok\n")
return nil return nil
} }
@@ -1174,31 +1550,16 @@ func runBuildTopData(ctx context) error {
return err return err
} }
result, err := topdata.BuildAndPackageWithOptions(p, opts, func(message string) { console := newTopdataConsole(ctx, p, "build-topdata")
if ctx.logLevel == logLevelQuiet { spin.configure(ctx.stderr, console.spinnerEnabled)
return spin.start("Build Topdata: starting")
} defer spin.stop()
fmt.Fprintf(ctx.stdout, "[build-topdata] %s\n", message) result, err := topdata.BuildAndPackageWithOptions(p, opts, console.progress)
})
if err != nil { if err != nil {
return err return err
} }
fmt.Fprintf(ctx.stdout, "project: %s\n", p.Config.Module.Name) console.emitPackageResult(result)
fmt.Fprintf(ctx.stdout, "mode: %s\n", result.Mode)
fmt.Fprintf(ctx.stdout, "topdata 2da output: %s\n", result.Output2DADir)
fmt.Fprintf(ctx.stdout, "topdata tlk output: %s\n", result.OutputTLKDir)
fmt.Fprintf(ctx.stdout, "top package hak: %s\n", result.OutputHAKPath)
fmt.Fprintf(ctx.stdout, "top package tlk: %s\n", result.OutputTLKPath)
fmt.Fprintf(ctx.stdout, "2da files: %d\n", result.Files2DA)
fmt.Fprintf(ctx.stdout, "tlk files: %d\n", result.FilesTLK)
fmt.Fprintf(ctx.stdout, "top package resources: %d\n", result.HAKResources)
fmt.Fprintf(ctx.stdout, "top package assets: %d\n", result.AssetFiles)
if result.WikiOutputDir != "" {
fmt.Fprintf(ctx.stdout, "wiki output: %s\n", result.WikiOutputDir)
fmt.Fprintf(ctx.stdout, "wiki pages: %d\n", result.WikiPages)
fmt.Fprintf(ctx.stdout, "wiki status: %s\n", result.WikiStatus)
}
return nil return nil
} }
@@ -1217,31 +1578,16 @@ func runBuildTopPackage(ctx context) error {
return fmt.Errorf("--wiki is not supported with build-top-package; use build-topdata when wiki generation is required") return fmt.Errorf("--wiki is not supported with build-top-package; use build-topdata when wiki generation is required")
} }
result, err := topdata.BuildPackage(p, func(message string) { console := newTopdataConsole(ctx, p, "build-top-package")
if ctx.logLevel == logLevelQuiet { spin.configure(ctx.stderr, console.spinnerEnabled)
return spin.start("Build Top Package: starting")
} defer spin.stop()
fmt.Fprintf(ctx.stdout, "[build-top-package] %s\n", message) result, err := topdata.BuildPackage(p, console.progress)
})
if err != nil { if err != nil {
return err return err
} }
fmt.Fprintf(ctx.stdout, "project: %s\n", p.Config.Module.Name) console.emitPackageResult(result)
fmt.Fprintf(ctx.stdout, "mode: %s\n", result.Mode)
fmt.Fprintf(ctx.stdout, "topdata 2da output: %s\n", result.Output2DADir)
fmt.Fprintf(ctx.stdout, "topdata tlk output: %s\n", result.OutputTLKDir)
fmt.Fprintf(ctx.stdout, "top package hak: %s\n", result.OutputHAKPath)
fmt.Fprintf(ctx.stdout, "top package tlk: %s\n", result.OutputTLKPath)
fmt.Fprintf(ctx.stdout, "2da files: %d\n", result.Files2DA)
fmt.Fprintf(ctx.stdout, "tlk files: %d\n", result.FilesTLK)
fmt.Fprintf(ctx.stdout, "top package resources: %d\n", result.HAKResources)
fmt.Fprintf(ctx.stdout, "top package assets: %d\n", result.AssetFiles)
if result.WikiOutputDir != "" {
fmt.Fprintf(ctx.stdout, "wiki output: %s\n", result.WikiOutputDir)
fmt.Fprintf(ctx.stdout, "wiki pages: %d\n", result.WikiPages)
fmt.Fprintf(ctx.stdout, "wiki status: %s\n", result.WikiStatus)
}
return nil return nil
} }
@@ -1268,24 +1614,16 @@ func runCompareTopData(ctx context) error {
return err return err
} }
result, err := topdata.Compare(p, func(message string) { console := newTopdataConsole(ctx, p, "compare-topdata")
if ctx.logLevel == logLevelQuiet { spin.configure(ctx.stderr, console.spinnerEnabled)
return spin.start("Compare Topdata: starting")
} defer spin.stop()
fmt.Fprintf(ctx.stdout, "[compare-topdata] %s\n", message) result, err := topdata.Compare(p, console.progress)
})
if err != nil { if err != nil {
return err return err
} }
fmt.Fprintf(ctx.stdout, "project: %s\n", p.Config.Module.Name) console.emitCompareResult(result)
fmt.Fprintf(ctx.stdout, "mode: %s\n", result.Mode)
fmt.Fprintf(ctx.stdout, "checked 2da files: %d\n", result.Compared2DA)
fmt.Fprintf(ctx.stdout, "checked tlk files: %d\n", result.ComparedTLK)
fmt.Fprintf(ctx.stdout, "native-pass: %d\n", result.NativePass)
fmt.Fprintf(ctx.stdout, "not-yet-canonical: %d\n", result.NotYetCanonical)
fmt.Fprintf(ctx.stdout, "native-mismatch: %d\n", result.NativeMismatch)
fmt.Fprintf(ctx.stdout, "topdata compare: ok\n")
return nil return nil
} }
@@ -1307,20 +1645,16 @@ func runBuildWiki(ctx context) error {
return err return err
} }
result, err := topdata.BuildWikiWithOptions(p, opts, func(message string) { console := newTopdataConsole(ctx, p, "build-wiki")
if ctx.logLevel == logLevelQuiet { spin.configure(ctx.stderr, console.spinnerEnabled)
return spin.start("Build Wiki: starting")
} defer spin.stop()
fmt.Fprintf(ctx.stdout, "[build-wiki] %s\n", message) result, err := topdata.BuildWikiWithOptions(p, opts, console.progress)
})
if err != nil { if err != nil {
return err return err
} }
fmt.Fprintf(ctx.stdout, "project: %s\n", p.Config.Module.Name) console.emitWikiBuildResult(result.OutputDir, result.PageCount, result.Status)
fmt.Fprintf(ctx.stdout, "wiki output: %s\n", result.OutputDir)
fmt.Fprintf(ctx.stdout, "wiki pages: %d\n", result.PageCount)
fmt.Fprintf(ctx.stdout, "wiki status: %s\n", result.Status)
return nil return nil
} }
@@ -1350,24 +1684,16 @@ func runDeployWiki(ctx context) error {
return err return err
} }
result, err := topdata.DeployWikiWithOptions(p, opts, func(message string) { console := newTopdataConsole(ctx, p, "deploy-wiki")
if ctx.logLevel == logLevelQuiet { spin.configure(ctx.stderr, console.spinnerEnabled)
return spin.start("Deploy Wiki: starting")
} defer spin.stop()
fmt.Fprintf(ctx.stdout, "[deploy-wiki] %s\n", message) result, err := topdata.DeployWikiWithOptions(p, opts, console.progress)
})
if err != nil { if err != nil {
return err return err
} }
fmt.Fprintf(ctx.stdout, "project: %s\n", p.Config.Module.Name) console.emitWikiDeployResult(result.LocalPages, result.Created, result.Updated, result.Skipped, result.Stale, result.Drifted, result.Manifest)
fmt.Fprintf(ctx.stdout, "local pages: %d\n", result.LocalPages)
fmt.Fprintf(ctx.stdout, "created: %d\n", result.Created)
fmt.Fprintf(ctx.stdout, "updated: %d\n", result.Updated)
fmt.Fprintf(ctx.stdout, "skipped: %d\n", result.Skipped)
fmt.Fprintf(ctx.stdout, "stale: %d\n", result.Stale)
fmt.Fprintf(ctx.stdout, "drifted: %d\n", result.Drifted)
fmt.Fprintf(ctx.stdout, "manifest: %s\n", result.Manifest)
return nil return nil
} }
+255 -54
View File
@@ -16,18 +16,16 @@ func TestRunBuildTopPackageUsesCachedCompiledOutputs(t *testing.T) {
mkdirAll(t, filepath.Join(root, "topdata", "assets", "gui")) mkdirAll(t, filepath.Join(root, "topdata", "assets", "gui"))
mkdirAll(t, filepath.Join(root, "topdata", "data", "repadjust")) mkdirAll(t, filepath.Join(root, "topdata", "data", "repadjust"))
writeFile(t, filepath.Join(root, "nwn-tool.json"), `{ writeFile(t, filepath.Join(root, "nwn-tool.yaml"), `
"module": { module:
"name": "Test Module", name: Test Module
"resref": "test_module" resref: test_module
}, topdata:
"topdata": { source: topdata
"source": "topdata", build: .cache
"build": ".cache", package_hak: sow_top.hak
"package_hak": "sow_top.hak", package_tlk: sow_tlk.tlk
"package_tlk": "sow_tlk.tlk" `)
}
}`+"\n")
writeFile(t, filepath.Join(root, ".cache", "2da", "repadjust.2da"), "2DA V2.0\n\n Label\n0 TEST_LABEL\n") writeFile(t, filepath.Join(root, ".cache", "2da", "repadjust.2da"), "2DA V2.0\n\n Label\n0 TEST_LABEL\n")
writeFile(t, filepath.Join(root, "build", "sow_tlk.tlk"), "compiled tlk") writeFile(t, filepath.Join(root, "build", "sow_tlk.tlk"), "compiled tlk")
writeFile(t, filepath.Join(root, "topdata", "assets", "gui", "testicon.png"), "icon-data") writeFile(t, filepath.Join(root, "topdata", "assets", "gui", "testicon.png"), "icon-data")
@@ -51,8 +49,12 @@ func TestRunBuildTopPackageUsesCachedCompiledOutputs(t *testing.T) {
if err := runBuildTopPackage(ctx); err != nil { if err := runBuildTopPackage(ctx); err != nil {
t.Fatalf("runBuildTopPackage failed: %v", err) t.Fatalf("runBuildTopPackage failed: %v", err)
} }
if !strings.Contains(stdout.String(), "top package hak:") { output := stdout.String()
t.Fatalf("expected build-top-package output, got %q", stdout.String()) if !strings.Contains(output, "top package hak: build/sow_top.hak") {
t.Fatalf("expected build-top-package output, got %q", output)
}
if strings.Contains(output, "[build-top-package]") {
t.Fatalf("did not expect raw progress lines in normal output, got %q", output)
} }
if _, err := os.Stat(filepath.Join(root, "build", "sow_top.hak")); err != nil { if _, err := os.Stat(filepath.Join(root, "build", "sow_top.hak")); err != nil {
t.Fatalf("expected packaged hak output: %v", err) t.Fatalf("expected packaged hak output: %v", err)
@@ -65,26 +67,22 @@ func TestRunBuildHAKsEmitsCompactSummary(t *testing.T) {
mkdirAll(t, filepath.Join(root, "build")) mkdirAll(t, filepath.Join(root, "build"))
mkdirAll(t, filepath.Join(root, "src")) mkdirAll(t, filepath.Join(root, "src"))
writeFile(t, filepath.Join(root, "nwn-tool.json"), `{ writeFile(t, filepath.Join(root, "nwn-tool.yaml"), `
"module": { module:
"name": "Test Module", name: Test Module
"resref": "testmod" resref: testmod
}, paths:
"paths": { source: src
"source": "src", assets: assets
"assets": "assets", build: build
"build": "build" haks:
}, - name: envi
"haks": [ priority: 1
{ max_bytes: 1048576
"name": "envi", split: false
"priority": 1, include:
"max_bytes": 1048576, - envi/**
"split": false, `)
"include": ["envi/**"]
}
]
}`+"\n")
writeFile(t, filepath.Join(root, "assets", "envi", "music", "westgate", "AleandAnecdotes.mp3"), "source-mp3") writeFile(t, filepath.Join(root, "assets", "envi", "music", "westgate", "AleandAnecdotes.mp3"), "source-mp3")
writeFile(t, filepath.Join(root, "assets", "envi", "music", "westgate", "CREDITS.md"), "# NWN Music Pack Credits\n\n| Artist | Title | Output File | Original File | Album | Date | License / Copyright | Notes |\n|---|---|---|---|---|---|---|---|\n| Darren Curtis | Ale and Anecdotes | `mus_wg_andnc.bmu` | `AleandAnecdotes.mp3` | Darren's Commercially Free Music | | Darren Curtis Music | |\n") writeFile(t, filepath.Join(root, "assets", "envi", "music", "westgate", "CREDITS.md"), "# NWN Music Pack Credits\n\n| Artist | Title | Output File | Original File | Album | Date | License / Copyright | Notes |\n|---|---|---|---|---|---|---|---|\n| Darren Curtis | Ale and Anecdotes | `mus_wg_andnc.bmu` | `AleandAnecdotes.mp3` | Darren's Commercially Free Music | | Darren Curtis Music | |\n")
@@ -136,26 +134,22 @@ func TestRunBuildHAKsVerboseListsMappings(t *testing.T) {
mkdirAll(t, filepath.Join(root, "build")) mkdirAll(t, filepath.Join(root, "build"))
mkdirAll(t, filepath.Join(root, "src")) mkdirAll(t, filepath.Join(root, "src"))
writeFile(t, filepath.Join(root, "nwn-tool.json"), `{ writeFile(t, filepath.Join(root, "nwn-tool.yaml"), `
"module": { module:
"name": "Test Module", name: Test Module
"resref": "testmod" resref: testmod
}, paths:
"paths": { source: src
"source": "src", assets: assets
"assets": "assets", build: build
"build": "build" haks:
}, - name: envi
"haks": [ priority: 1
{ max_bytes: 1048576
"name": "envi", split: false
"priority": 1, include:
"max_bytes": 1048576, - envi/**
"split": false, `)
"include": ["envi/**"]
}
]
}`+"\n")
writeFile(t, filepath.Join(root, "assets", "envi", "music", "westgate", "AleandAnecdotes.mp3"), "source-mp3") writeFile(t, filepath.Join(root, "assets", "envi", "music", "westgate", "AleandAnecdotes.mp3"), "source-mp3")
writeFile(t, filepath.Join(root, "assets", "envi", "music", "westgate", "CREDITS.md"), "# NWN Music Pack Credits\n\n| Artist | Title | Output File | Original File | Album | Date | License / Copyright | Notes |\n|---|---|---|---|---|---|---|---|\n| Darren Curtis | Ale and Anecdotes | `mus_wg_andnc.bmu` | `AleandAnecdotes.mp3` | Darren's Commercially Free Music | | Darren Curtis Music | |\n") writeFile(t, filepath.Join(root, "assets", "envi", "music", "westgate", "CREDITS.md"), "# NWN Music Pack Credits\n\n| Artist | Title | Output File | Original File | Album | Date | License / Copyright | Notes |\n|---|---|---|---|---|---|---|---|\n| Darren Curtis | Ale and Anecdotes | `mus_wg_andnc.bmu` | `AleandAnecdotes.mp3` | Darren's Commercially Free Music | | Darren Curtis Music | |\n")
@@ -198,6 +192,204 @@ func TestRunBuildHAKsVerboseListsMappings(t *testing.T) {
} }
} }
func TestTopdataConsoleSuppressesProgressInNormalMode(t *testing.T) {
var stdout bytes.Buffer
console := &topdataConsole{
stdout: &stdout,
projectRoot: "/workspace/project",
projectName: "Test Module",
commandName: "build-topdata",
commandLabel: "Build Topdata",
level: logLevelNormal,
}
console.progress("Packaging compiled topdata resources into sow_top.hak...")
if stdout.String() != "" {
t.Fatalf("expected no normal-mode progress output, got %q", stdout.String())
}
}
func TestTopdataConsoleDebugProgressAndRelativePaths(t *testing.T) {
var stdout bytes.Buffer
console := &topdataConsole{
stdout: &stdout,
projectRoot: "/workspace/project",
projectName: "Test Module",
commandName: "deploy-wiki",
commandLabel: "Deploy Wiki",
level: logLevelDebug,
}
console.progress("NodeBB wiki plan: create 1, update 2, skip 3, drift 0")
console.emitWikiDeployResult(10, 1, 2, 3, 4, 0, "/workspace/project/build/wiki/deploy-manifest.json")
output := stdout.String()
if !strings.Contains(output, "[debug] NodeBB wiki plan: create 1, update 2, skip 3, drift 0") {
t.Fatalf("expected debug progress line, got %q", output)
}
if !strings.Contains(output, "manifest: build/wiki/deploy-manifest.json") {
t.Fatalf("expected relative deploy manifest path, got %q", output)
}
}
func TestRunBuildModuleToolkitPreflightRefreshesManifestAndSkipsBrokenCompiler(t *testing.T) {
root := t.TempDir()
mkdirAll(t, filepath.Join(root, "src", "module"))
mkdirAll(t, filepath.Join(root, "src", "placeables"))
mkdirAll(t, filepath.Join(root, "src", "scripts"))
mkdirAll(t, filepath.Join(root, "build"))
mkdirAll(t, filepath.Join(root, "scripts"))
writeFile(t, filepath.Join(root, "nwn-tool.yaml"), `
module:
name: Test Module
resref: testmod
paths:
source: src
build: build
scripts:
compiler:
search:
- tools/script-compiler/nwn_script_comp
`)
writeFile(t, filepath.Join(root, "src", "module", "module.ifo.json"), `{
"file_type": "IFO ",
"file_version": "V3.2",
"root": {
"struct_type": 0,
"fields": [
{"label": "Mod_Name", "type": "CExoString", "value": "Test Module"}
]
}
}
`)
writeFile(t, filepath.Join(root, "src", "placeables", "thing.utp.json"), `{
"file_type": "UTP ",
"file_version": "V3.2",
"root": {
"struct_type": 0,
"fields": [
{"label": "OnUsed", "type": "ResRef", "value": "use_thing"}
]
}
}
`)
writeFile(t, filepath.Join(root, "src", "scripts", "use_thing.nss"), "void main() {}\n")
writeFile(t, filepath.Join(root, "scripts", "fetch-hak-manifest.sh"), "#!/usr/bin/env sh\nset -eu\nmkdir -p \"$(dirname \"$1\")\"\nprintf '{\"module_haks\":[\"sow_core\"],\"haks\":[]}\n' >\"$1\"\n")
if err := os.Chmod(filepath.Join(root, "scripts", "fetch-hak-manifest.sh"), 0o755); err != nil {
t.Fatalf("chmod fetch-hak-manifest.sh: %v", err)
}
badCompiler := filepath.Join(root, "bad-compiler.sh")
writeFile(t, badCompiler, "#!/usr/bin/env sh\nexit 1\n")
if err := os.Chmod(badCompiler, 0o755); err != nil {
t.Fatalf("chmod bad compiler: %v", err)
}
t.Setenv("SOW_NWN_SCRIPT_COMPILER", badCompiler)
t.Setenv("SOW_MODULE_SKIP_NSS_COMPILATION", "")
var stdout bytes.Buffer
var stderr bytes.Buffer
ctx := context{
stdout: &stdout,
stderr: &stderr,
cwd: root,
args: []string{"build-module"},
}
if err := runBuildModule(ctx); err != nil {
t.Fatalf("runBuildModule failed: %v", err)
}
moduleSource := mustReadFile(t, filepath.Join(root, "src", "module", "module.ifo.json"))
if !strings.Contains(moduleSource, "\"Mod_HakList\"") || !strings.Contains(moduleSource, "\"sow_core\"") {
t.Fatalf("expected manifest-applied module source, got %q", moduleSource)
}
if _, err := os.Stat(filepath.Join(root, ".cache", "ncs", "use_thing.ncs")); !os.IsNotExist(err) {
t.Fatalf("expected broken explicit compiler to skip compilation, got err=%v", err)
}
if !strings.Contains(stdout.String(), "Skipping NWScript compilation for this build.") {
t.Fatalf("expected skip-compilation progress in output, got %q", stdout.String())
}
}
func TestRunBuildModuleToolkitPreflightInstallsCompilerWhenNeeded(t *testing.T) {
root := t.TempDir()
mkdirAll(t, filepath.Join(root, "src", "module"))
mkdirAll(t, filepath.Join(root, "src", "placeables"))
mkdirAll(t, filepath.Join(root, "src", "scripts"))
mkdirAll(t, filepath.Join(root, "build"))
mkdirAll(t, filepath.Join(root, "scripts"))
writeFile(t, filepath.Join(root, "nwn-tool.yaml"), `
module:
name: Test Module
resref: testmod
paths:
source: src
build: build
scripts:
compiler:
search:
- tools/script-compiler/nwn_script_comp
`)
writeFile(t, filepath.Join(root, "src", "module", "module.ifo.json"), `{
"file_type": "IFO ",
"file_version": "V3.2",
"root": {
"struct_type": 0,
"fields": [
{"label": "Mod_Name", "type": "CExoString", "value": "Test Module"}
]
}
}
`)
writeFile(t, filepath.Join(root, "src", "placeables", "thing.utp.json"), `{
"file_type": "UTP ",
"file_version": "V3.2",
"root": {
"struct_type": 0,
"fields": [
{"label": "OnUsed", "type": "ResRef", "value": "use_thing"}
]
}
}
`)
writeFile(t, filepath.Join(root, "src", "scripts", "use_thing.nss"), "void main() {}\n")
writeFile(t, filepath.Join(root, "scripts", "fetch-hak-manifest.sh"), "#!/usr/bin/env sh\nset -eu\nmkdir -p \"$(dirname \"$1\")\"\nprintf '{\"module_haks\":[\"sow_core\"],\"haks\":[]}\n' >\"$1\"\n")
writeFile(t, filepath.Join(root, "scripts", "install-script-compiler.sh"), "#!/usr/bin/env sh\nset -eu\ninstall_dir=\"${SOW_SCRIPT_COMPILER_INSTALL_DIR:-./tools/script-compiler}\"\nmkdir -p \"$install_dir\"\ncat >\"$install_dir/nwn_script_comp\" <<'SH'\n#!/usr/bin/env sh\nset -eu\nif [ \"${1:-}\" = \"--help\" ]; then\n exit 0\nfi\nout=\"\"\nsrc=\"\"\nwhile [ \"$#\" -gt 0 ]; do\n case \"$1\" in\n -o)\n out=\"$2\"\n shift 2\n ;;\n --dirs)\n shift 2\n ;;\n *)\n src=\"$1\"\n shift\n ;;\n esac\ndone\nprintf 'compiled:%s\\n' \"$(basename \"$src\")\" >\"$out\"\nSH\nchmod +x \"$install_dir/nwn_script_comp\"\nprintf 'installed: %s\\n' \"$install_dir\"\n")
if err := os.Chmod(filepath.Join(root, "scripts", "fetch-hak-manifest.sh"), 0o755); err != nil {
t.Fatalf("chmod fetch-hak-manifest.sh: %v", err)
}
if err := os.Chmod(filepath.Join(root, "scripts", "install-script-compiler.sh"), 0o755); err != nil {
t.Fatalf("chmod install-script-compiler.sh: %v", err)
}
t.Setenv("SOW_NWN_SCRIPT_COMPILER", "")
t.Setenv("SOW_MODULE_SKIP_NSS_COMPILATION", "")
var stdout bytes.Buffer
var stderr bytes.Buffer
ctx := context{
stdout: &stdout,
stderr: &stderr,
cwd: root,
args: []string{"build-module"},
}
if err := runBuildModule(ctx); err != nil {
t.Fatalf("runBuildModule failed: %v\nstdout:\n%s\nstderr:\n%s", err, stdout.String(), stderr.String())
}
compiledScript := filepath.Join(root, ".cache", "ncs", "use_thing.ncs")
if _, err := os.Stat(compiledScript); err != nil {
t.Fatalf("expected installed compiler to compile script: %v", err)
}
if !strings.Contains(stdout.String(), "NWScript compiler not found locally. Installing it now...") {
t.Fatalf("expected install progress in output, got %q", stdout.String())
}
}
func TestRunMusicScanUsesConfiguredDatasetsWithoutWriting(t *testing.T) { func TestRunMusicScanUsesConfiguredDatasetsWithoutWriting(t *testing.T) {
root := t.TempDir() root := t.TempDir()
mkdirAll(t, filepath.Join(root, "assets", "audio", "westgate")) mkdirAll(t, filepath.Join(root, "assets", "audio", "westgate"))
@@ -448,3 +640,12 @@ func setFileTime(t *testing.T, path string, modTime time.Time) {
t.Fatalf("set file time %s: %v", path, err) t.Fatalf("set file time %s: %v", path, err)
} }
} }
func mustReadFile(t *testing.T, path string) string {
t.Helper()
raw, err := os.ReadFile(path)
if err != nil {
t.Fatalf("read %s: %v", path, err)
}
return string(raw)
}
+21 -5
View File
@@ -6,6 +6,7 @@ import (
"encoding/json" "encoding/json"
"errors" "errors"
"fmt" "fmt"
"io"
"os" "os"
"os/exec" "os/exec"
"path/filepath" "path/filepath"
@@ -655,14 +656,15 @@ func referencedScriptResrefs(p *project.Project) ([]string, error) {
} }
func resolveScriptCompiler(p *project.Project) (string, error) { func resolveScriptCompiler(p *project.Project) (string, error) {
candidates := make([]string, 0, 2+len(p.ScriptCompilerSearchPaths()))
if configured := p.ScriptCompilerPath(); configured != "" { if configured := p.ScriptCompilerPath(); configured != "" {
return configured, nil candidates = append(candidates, configured)
} }
if configured := strings.TrimSpace(os.Getenv(p.ScriptCompilerPathEnv())); configured != "" { if configured := strings.TrimSpace(os.Getenv(p.ScriptCompilerPathEnv())); configured != "" {
return configured, nil candidates = append(candidates, configured)
} }
candidates = append(candidates, p.ScriptCompilerSearchPaths()...)
candidates := p.ScriptCompilerSearchPaths()
if runtime.GOOS == "windows" { if runtime.GOOS == "windows" {
slices.SortStableFunc(candidates, func(a, b string) int { slices.SortStableFunc(candidates, func(a, b string) int {
aExe := strings.HasSuffix(strings.ToLower(a), ".exe") aExe := strings.HasSuffix(strings.ToLower(a), ".exe")
@@ -678,12 +680,12 @@ func resolveScriptCompiler(p *project.Project) (string, error) {
} }
for _, candidate := range candidates { for _, candidate := range candidates {
if binary, ok := strings.CutPrefix(candidate, "PATH:"); ok { if binary, ok := strings.CutPrefix(candidate, "PATH:"); ok {
if compiler, err := exec.LookPath(binary); err == nil { if compiler, err := exec.LookPath(binary); err == nil && scriptCompilerRunnable(compiler) {
return compiler, nil return compiler, nil
} }
continue continue
} }
if info, err := os.Stat(candidate); err == nil && !info.IsDir() { if info, err := os.Stat(candidate); err == nil && !info.IsDir() && scriptCompilerRunnable(candidate) {
return candidate, nil return candidate, nil
} }
} }
@@ -691,6 +693,20 @@ func resolveScriptCompiler(p *project.Project) (string, error) {
return "", fmt.Errorf("script compiler not found; configure scripts.compiler.path or set %s", p.ScriptCompilerPathEnv()) return "", fmt.Errorf("script compiler not found; configure scripts.compiler.path or set %s", p.ScriptCompilerPathEnv())
} }
func scriptCompilerRunnable(path string) bool {
if strings.TrimSpace(path) == "" {
return false
}
info, err := os.Stat(path)
if err != nil || info.IsDir() {
return false
}
cmd := exec.Command(path, "--help")
cmd.Stdout = io.Discard
cmd.Stderr = io.Discard
return cmd.Run() == nil
}
func validatorLoadDocument(path string) (gff.Document, string, string, error) { func validatorLoadDocument(path string) (gff.Document, string, string, error) {
raw, err := os.ReadFile(path) raw, err := os.ReadFile(path)
if err != nil { if err != nil {
+3
View File
@@ -1031,6 +1031,9 @@ func TestBuildModuleCompilesReferencedScripts(t *testing.T) {
compiler := filepath.Join(root, "fake-compiler.sh") compiler := filepath.Join(root, "fake-compiler.sh")
mustWriteFile(t, compiler, `#!/bin/sh mustWriteFile(t, compiler, `#!/bin/sh
set -eu set -eu
if [ "${1:-}" = "--help" ]; then
exit 0
fi
out="" out=""
src="" src=""
while [ "$#" -gt 0 ]; do while [ "$#" -gt 0 ]; do
@@ -1,5 +1,17 @@
# AI Agent Contract: Auto-Include Existing Part Models in 2DA Generation # AI Agent Contract: Auto-Include Existing Part Models in 2DA Generation
## Status Snapshot
Current state as of 2026-05-13:
- Implemented.
- The toolkit resolves released parts manifests through the configured autogen
release source and covers this behavior with `parts_manifest_test.go` plus
broader native topdata tests.
- Remaining work is operational hardening only: keep manifest contracts stable
across `assets/` releases and maintain regression coverage as autogen config
evolves.
## Objective ## Objective
The native 2DA generation pipeline must automatically include existing part models already present in the **`sow-assets` repository** when building final parts tables. The native 2DA generation pipeline must automatically include existing part models already present in the **`sow-assets` repository** when building final parts tables.
@@ -1,5 +1,22 @@
# Global Feat Injection — Specification # Global Feat Injection — Specification
## Status Snapshot
Current state as of 2026-05-13:
- This document is partially stale relative to the implementation in
`internal/topdata/native.go`.
- Current code injects:
- always-on menu feats: `feat:specialattacks`, `feat:throw`,
`feat:grapple`, `feat:offensivefighting`, `feat:defensivefighting`,
`feat:horsemenu`
- conditional combat feats: `feat:powerattack`, `feat:combatexpertise`
- class-skill shorthand rows for `masterfeats:skillfocus` at level `-1` and
`masterfeats:greaterskillfocus` at level `12`
- This contract should be treated as a parity note, not as an exact
implementation transcript, until it is fully rewritten around the current
behavior.
## Objective ## Objective
Ensure the native class feat generation pipeline produces **functionally identical output** to the legacy system by applying deterministic global feat injections. Ensure the native class feat generation pipeline produces **functionally identical output** to the legacy system by applying deterministic global feat injections.
@@ -24,6 +41,9 @@ With properties:
"OnMenu": "0" "OnMenu": "0"
} }
For the current implementation, `masterfeats:greaterskillfocus` is injected with
`GrantedOnLevel = 12`, not `-1`.
--- ---
### R2 — Spellcasting master feats ### R2 — Spellcasting master feats
@@ -37,6 +57,16 @@ Then inject:
- `masterfeats:spellfocus` - `masterfeats:spellfocus`
- `masterfeats:greaterspellfocus` - `masterfeats:greaterspellfocus`
Historical note:
- This rule is not currently implemented as an automatic global injection in
`native.go`.
- Spell-focus expansion is supported when the class feat data explicitly
references `masterfeats:spellfocus`.
If automatic spellcaster-driven injection is still required, it remains
outstanding work.
With properties: With properties:
```json ```json
@@ -75,6 +105,9 @@ With properties:
Always inject: Always inject:
- `feat:specialattacks`
- `feat:throw`
- `feat:grapple`
- `feat:offensivefighting` - `feat:offensivefighting`
- `feat:defensivefighting` - `feat:defensivefighting`
- `feat:horsemenu` - `feat:horsemenu`
@@ -1,5 +1,16 @@
# Generic Family Expansion Contract # Generic Family Expansion Contract
## Status Snapshot
Current state as of 2026-05-13:
- Implemented and actively used by native feat generation.
- `family_expansion.go`, `native.go`, and `topdata_test.go` cover the current
underscore-family interpretation, generated family metadata, and reapplication
behavior.
- Remaining work is mostly consumer expansion: use the same primitive in more
datasets only when the authored data actually benefits from it.
Topdata now treats underscore expansion as a structural rule: Topdata now treats underscore expansion as a structural rule:
- `parent_child` means `child` is an expansion of `parent` - `parent_child` means `child` is an expansion of `parent`
@@ -1,5 +1,16 @@
# `feat` Generated Families Contract # `feat` Generated Families Contract
## Status Snapshot
Current state as of 2026-05-13:
- Implemented for the current native `feat` pipeline.
- Native tests cover generated-family expansion, family-owned shared defaults,
masterfeat-backed inheritance, and lock/TLK identity preservation.
- The one explicitly deferred item in this contract is still current:
`feat.2da` remains native-built but excluded from `compare-topdata`
reference-catalog coverage through `compare_reference: false`.
Canonical authored paths: Canonical authored paths:
- `topdata/data/feat/base.json` - `topdata/data/feat/base.json`
+9
View File
@@ -1,5 +1,14 @@
# Topdata Inheritance Contract # Topdata Inheritance Contract
## Status Snapshot
Current state as of 2026-05-13:
- Implemented.
- `native.go` supports both row-sugar inheritance and generic object
inheritance, including cycle detection and missing-target failures.
- Validation and build regression coverage live in `topdata_test.go`.
Native topdata now supports two inheritance forms: Native topdata now supports two inheritance forms:
## 1. Row Sugar Compatibility ## 1. Row Sugar Compatibility
+10
View File
@@ -1,5 +1,15 @@
# `masterfeats` Canonical Contract # `masterfeats` Canonical Contract
## Status Snapshot
Current state as of 2026-05-13:
- Implemented as a first-class native dataset.
- The repository contains `topdata/data/masterfeats/`, migration/import support,
validator checks, and native output coverage for `masterfeats.2da`.
- Remaining work is only downstream adoption and regression maintenance as other
native datasets continue to consume `masterfeats` more heavily.
`masterfeats` is a stable native canonical family consumed directly by `feat` `masterfeats` is a stable native canonical family consumed directly by `feat`
generation, class-feat expansion, and inheritance/ref resolution. generation, class-feat expansion, and inheritance/ref resolution.