805 lines
27 KiB
Markdown
805 lines
27 KiB
Markdown
Create a long-term multi-repository hardening plan focused on making YAML configuration the authoritative source of truth.
|
|
|
|
## Status Snapshot
|
|
|
|
Current state as of 2026-05-13:
|
|
|
|
- This hardening effort is active and partially complete.
|
|
- YAML authority, strict config loading, and effective-config inspection are
|
|
implemented.
|
|
- Major repository behavior now flows through normalized config, including HAK
|
|
layout, topdata packaging, extract behavior, autogen manifests, scripts, wiki,
|
|
and music datasets.
|
|
- The remaining gaps are mostly about reducing legacy compatibility surface,
|
|
moving more tests to YAML fixtures, and deciding which residual toolkit
|
|
defaults stay intrinsic versus become explicit schema.
|
|
|
|
Primary outstanding work:
|
|
|
|
- reduce reliance on `nwn-tool.json` in tests and migration fixtures
|
|
- keep pruning repository-shaped defaults that still live in code when they
|
|
should be surfaced as schema-backed settings
|
|
- maintain docs/examples for `config effective` and the normalized key space as
|
|
the schema evolves
|
|
|
|
# Goal
|
|
|
|
Reduce hardcoded behavior across all repositories and toolkit code.
|
|
|
|
All behavior that touches repository-specific configuration must be driven by YAML. If a behavior currently exists only as an explicit rule in application code, move it into configuration or create a new configuration option for it.
|
|
|
|
# Core principle
|
|
|
|
Configuration is fact.
|
|
|
|
Code should interpret and enforce configuration, not silently override it.
|
|
|
|
# Requirements
|
|
|
|
- YAML is the only human-authored repository configuration format.
|
|
- JSON remains acceptable only for generated artifacts, datasets, manifests, caches, and machine-readable outputs.
|
|
- Any code path that touches configurable behavior must lock to YAML-defined values.
|
|
- If YAML defines a value, code must not substitute another value unless explicitly configured as a fallback.
|
|
- If behavior varies by repository, dataset, build target, asset type, or command, it belongs in YAML.
|
|
- If no configuration option exists for a behavior, create one.
|
|
- Remove implicit repository assumptions from code.
|
|
- Remove hidden defaults unless they are documented toolkit-level defaults.
|
|
- Make all defaults visible, documented, and overrideable.
|
|
|
|
# Audit scope
|
|
|
|
Audit all repositories and toolkit code for hardcoded or implicit behavior related to:
|
|
|
|
- paths
|
|
- module metadata
|
|
- resrefs
|
|
- HAK names
|
|
- HAK order
|
|
- HAK grouping
|
|
- HAK priority
|
|
- HAK splitting
|
|
- HAK size limits
|
|
- optional HAKs
|
|
- include/exclude globs
|
|
- generated manifest names
|
|
- cache names
|
|
- release tags
|
|
- dataset names
|
|
- autogen producers
|
|
- autogen consumers
|
|
- derive rules
|
|
- music prefixes
|
|
- credits behavior
|
|
- extract behavior
|
|
- topdata behavior
|
|
- package names
|
|
- TLK names
|
|
- build directories
|
|
- source directories
|
|
- validation rules
|
|
- cleanup rules
|
|
- normalization rules
|
|
- naming schemes
|
|
- file extension rules
|
|
- ignore rules
|
|
- command-specific behavior
|
|
- logging verbosity
|
|
- deterministic ordering rules
|
|
|
|
# Hardening rules
|
|
|
|
1. YAML wins. If YAML declares a value, the application must use it exactly.
|
|
|
|
2. No silent fallbacks. Fallbacks must be explicit, documented, and visible in logs.
|
|
|
|
3. No hidden repository assumptions. Repository-specific behavior must not live in toolkit code.
|
|
|
|
4. No magic names. Names like `sow_top`, `sow_core`, `content`, `build`, `.cache`, or specific manifest names must come from YAML unless they are generated from a configured rule.
|
|
|
|
5. No implicit behavior drift. Existing behavior should remain stable unless the YAML explicitly changes it.
|
|
|
|
6. Deterministic by default. Ordering, output paths, generated names, cache keys, manifests, and logs must be stable.
|
|
|
|
7. Validation before execution. Invalid or incomplete configuration should fail early with clear errors.
|
|
|
|
8. Configuration gaps become schema work. If code needs a behavior that YAML cannot express, extend the schema.
|
|
|
|
# Implementation direction
|
|
|
|
- Build a central configuration model.
|
|
- Route all repository behavior through that model.
|
|
- Add typed config objects or schema validation.
|
|
- Normalize config once at startup.
|
|
- Pass normalized config into commands instead of reading globals or hardcoded constants.
|
|
- Make command behavior depend on normalized config only.
|
|
- Add explicit toolkit defaults in one place.
|
|
- Generate an effective configuration view for debugging.
|
|
- Add a command such as:
|
|
|
|
```text
|
|
toolkit config inspect
|
|
toolkit config validate
|
|
toolkit config explain <key>
|
|
toolkit config effective
|
|
```
|
|
|
|
# Effective config
|
|
|
|
Add support for printing the final resolved configuration after defaults, overrides, and dataset-specific settings have been applied.
|
|
|
|
This should help verify that:
|
|
|
|
- YAML was loaded.
|
|
- defaults were applied intentionally.
|
|
- no hidden behavior was used.
|
|
- paths resolved as expected.
|
|
- command behavior is explainable.
|
|
|
|
# Repository hardening process
|
|
|
|
For each repository:
|
|
|
|
1. Inventory existing YAML config.
|
|
2. Inventory generated JSON artifacts separately.
|
|
3. Search code for hardcoded repository behavior.
|
|
4. Map each hardcoded behavior to an existing YAML entry.
|
|
5. If no YAML entry exists, add one.
|
|
6. Update schema and validation.
|
|
7. Update code to consume normalized config.
|
|
8. Add tests proving YAML controls the behavior.
|
|
9. Add regression tests for existing outputs.
|
|
10. Document the config option.
|
|
|
|
# Testing requirements
|
|
|
|
Add tests for:
|
|
|
|
- YAML config loading
|
|
- config validation
|
|
- effective config generation
|
|
- deterministic ordering
|
|
- missing required config
|
|
- duplicate config entries
|
|
- invalid paths
|
|
- invalid HAK definitions
|
|
- invalid autogen definitions
|
|
- invalid dataset definitions
|
|
- command behavior matching YAML
|
|
- no hardcoded repository names
|
|
- JSON artifacts remaining generated data only
|
|
|
|
Add regression tests where possible:
|
|
|
|
- same YAML input produces same build plan
|
|
- same YAML input produces same HAK order
|
|
- same YAML input produces same manifest names
|
|
- same YAML input produces same cache paths
|
|
- same YAML input produces same autogen behavior
|
|
|
|
# Code audit targets
|
|
|
|
Look specifically for:
|
|
|
|
- string literals that look like paths, names, prefixes, manifests, or cache files
|
|
- constants that duplicate config values
|
|
- default values inside command implementations
|
|
- repository-specific branch logic
|
|
- command-specific assumptions
|
|
- filename conventions not declared in config
|
|
- implicit glob patterns
|
|
- implicit output names
|
|
- implicit ordering rules
|
|
- duplicated config parsing
|
|
- separate code paths that bypass central config
|
|
|
|
# Deliverables
|
|
|
|
1. Long-term hardening plan
|
|
2. Config authority rules
|
|
3. Audit checklist
|
|
4. Proposed YAML schema expansions
|
|
5. Effective config design
|
|
6. Migration strategy per repository
|
|
7. Testing strategy
|
|
8. Documentation strategy
|
|
9. List of hardcoded assumptions found
|
|
10. Refactor tickets grouped by risk and priority
|
|
|
|
# Acceptance criteria
|
|
|
|
- YAML is the only human-authored repository config.
|
|
- Code does not override YAML silently.
|
|
- Repository-specific behavior is not hardcoded.
|
|
- Any behavior touching config is lock-set to the normalized YAML value.
|
|
- Missing config options are added instead of encoded in application logic.
|
|
- Generated JSON artifacts remain unchanged unless intentionally modified.
|
|
- Builds remain deterministic.
|
|
- Existing repositories continue to build.
|
|
- Developers can inspect the effective config and understand why each value was used.
|
|
|
|
# Current project-needs analysis
|
|
|
|
## Current state
|
|
|
|
- Root repository configuration is already discovered as:
|
|
1. `nwn-tool.yaml`
|
|
2. `nwn-tool.yml`
|
|
3. legacy `nwn-tool.json`
|
|
- YAML is decoded with `KnownFields(true)`.
|
|
- Legacy JSON is decoded with `DisallowUnknownFields()`.
|
|
- `internal/app.loadProject` prints the loaded config filename and warns when legacy JSON is used.
|
|
- The root config model currently lives in `internal/project.Config`.
|
|
- `internal/project.defaultConfig` currently only defaults `paths.build` to `build`.
|
|
- Generated and machine-readable JSON remains widespread and should stay JSON:
|
|
- HAK manifest: `build/haks.json`
|
|
- canonical GFF JSON documents
|
|
- topdata authored/generated datasets
|
|
- topdata wiki state
|
|
- topdata deploy manifest
|
|
- autogen manifests and release caches
|
|
- credits inventory
|
|
- Some behavior is already YAML-driven:
|
|
- module name/resref/description
|
|
- module HAK order
|
|
- source/assets/build paths
|
|
- HAK names, priorities, split behavior, max bytes, optional flag, include globs
|
|
- music prefix mapping
|
|
- topdata source/build/reference/assets/package names
|
|
- extract ignore extensions and post-extract module deletion
|
|
- autogen producer/consumer ids, roots, includes, modes, derive rules, release tags, asset names, cache names
|
|
- Some behavior is still implicit in code and should be made explicit or documented as toolkit defaults.
|
|
|
|
## Important naming clarification
|
|
|
|
The current repository root config file is `nwn-tool.yaml`, not `config.yaml`.
|
|
|
|
Hardening assumption:
|
|
|
|
- Keep `nwn-tool.yaml` as the canonical root config filename.
|
|
- Keep `nwn-tool.yml` as an accepted alternate YAML filename.
|
|
- Treat `nwn-tool.json` as temporary legacy compatibility only.
|
|
- Do not rename generated or nested JSON files.
|
|
- Do not treat topdata dataset JSON as root repository configuration.
|
|
|
|
If a future decision changes the canonical name to `config.yaml`, update `project.ConfigFile`, docs, consumer wrappers, tests, and migration scripts together.
|
|
|
|
# Current hardcoded assumptions found
|
|
|
|
## Project config and path defaults
|
|
|
|
- `paths.build` defaults to `build` in `internal/project.defaultConfig`.
|
|
- `HAKManifestPath` always returns `<build>/haks.json`.
|
|
- `ModuleArchivePath` always uses `<build>/<module.resref>.mod`.
|
|
- `HAKArchivePath` always uses `<build>/<hak.name>.hak`.
|
|
- `SourceDir`, `AssetsDir`, and `BuildDir` directly join configured paths with the repo root and do not yet expose a richer path policy.
|
|
- `TopDataBuildDir` falls back to `<paths.build>/topdata` when `topdata.build` is empty.
|
|
- `TopDataUsesRepoCache` special-cases `topdata.build == .cache`.
|
|
- `TopDataCompiled2DADir` always appends `2da`.
|
|
- `TopDataCompiledTLKPath` uses `sow_tlk.tlk` either under `build` or topdata build output.
|
|
- `TopDataWikiRootDir` uses `.cache/wiki` for repo-cache mode and `<topdata.build>/wiki` otherwise.
|
|
- `TopDataWikiStatePath` always uses `state.json`.
|
|
- `TopDataPackageHAKName` defaults to `sow_top.hak`.
|
|
- `TopDataPackageTLKName` defaults to `sow_tlk.tlk`.
|
|
|
|
Hardening direction:
|
|
|
|
- Split these into documented toolkit defaults and repository-overridable fields.
|
|
- Keep existing values as defaults only for compatibility.
|
|
- Show all derived values in effective config output.
|
|
|
|
## Inventory and extension rules
|
|
|
|
- Source extensions are hardcoded in `project.SourceExtensions`.
|
|
- Asset extensions are hardcoded in `project.AssetExtensions`.
|
|
- `Project.Scan` treats `.json` and `.nss` as source inputs.
|
|
- Source JSON extension detection assumes resource filenames like `<resref>.<ext>.json`.
|
|
- Validator required fields are hardcoded, currently including `ifo -> Mod_Name`.
|
|
- Built-in script prefixes are hardcoded in `internal/validator`.
|
|
- Validator reference checks hardcode specific field labels and expected resource extensions:
|
|
- `Conversation -> .dlg`
|
|
- `Mod_Entry_Area -> .are`
|
|
- `Model -> .mdl`
|
|
- `Sound -> .wav`
|
|
|
|
Hardening direction:
|
|
|
|
- Add a `resources` or `inventory` config section for extension policy and source/asset classification.
|
|
- Keep NWN defaults built in, but make them inspectable and overrideable.
|
|
- Consider allowing validators to be named profiles, for example `nwn_module`, instead of requiring every repository to spell out base NWN rules.
|
|
|
|
## Build and HAK behavior
|
|
|
|
- `build-haks` reads/writes the hardcoded manifest name from `Project.HAKManifestPath`.
|
|
- HAK files are always emitted under `paths.build`.
|
|
- HAK archive filenames are always derived from `haks[].name`.
|
|
- `extract` scans all `*.hak` in the build directory rather than only configured/generated HAKs.
|
|
- `build-haks` has command flags that mutate behavior independently of config:
|
|
- `--hak`
|
|
- `--archive`
|
|
- `--source-manifest`
|
|
- `--plan-only`
|
|
- `--quiet`
|
|
- `--verbose`
|
|
- `--debug`
|
|
- `SOW_BUILD_HAKS_KEEP_EXISTING` controls HAK cleanup behavior outside YAML.
|
|
|
|
Hardening direction:
|
|
|
|
- Add configurable manifest names and cleanup policy.
|
|
- Define command-line flags as explicit per-run overrides in the effective config model.
|
|
- Log all active environment overrides.
|
|
- Consider config-controlled archive discovery for `extract`.
|
|
|
|
## Script compiler behavior
|
|
|
|
- Compiled scripts are cached under `.cache/ncs`.
|
|
- Compiler discovery is hardcoded to:
|
|
- `SOW_NWN_SCRIPT_COMPILER`
|
|
- `tools/script-compiler/nwn_script_comp`
|
|
- `tools/script-compiler/nwn_script_comp.exe`
|
|
- `tools/nwn_script_comp`
|
|
- `tools/nwn_script_comp.exe`
|
|
- `PATH`
|
|
- Script source directory is always `<paths.source>/scripts`.
|
|
- Compiler environment probing uses `SOW_NWN_USER_DIRECTORY`, `NWN_HOME`, `NWN_USER_DIRECTORY`, `SOW_NWN_ROOT`, and `NWN_ROOT`.
|
|
|
|
Hardening direction:
|
|
|
|
- Add `tools.script_compiler` and `script.cache` config.
|
|
- Keep environment variables as explicit override sources.
|
|
- Include resolved compiler path and cache path in effective config and debug logs.
|
|
|
|
## Music behavior
|
|
|
|
- Music conversion/scanning is hardcoded around `envi/music`.
|
|
- Music staging is hardcoded to `.cache/music`.
|
|
- Credits output is hardcoded to `.cache/credits`.
|
|
- Credits overlay filename is hardcoded to `CREDITS.md`.
|
|
- FFmpeg/ffprobe discovery is environment/PATH based.
|
|
- BMU output naming rules and 16-character NWN stem limits are hardcoded.
|
|
|
|
Hardening direction:
|
|
|
|
- Apply `MUSIC_REFACTOR_CONTRACT.md`.
|
|
- Convert music behavior to dataset configuration.
|
|
- Keep old `music.prefixes` only as compatibility shorthand until repositories migrate.
|
|
|
|
## Topdata behavior
|
|
|
|
- Topdata source layout is partly convention-based:
|
|
- `data`
|
|
- `tlk`
|
|
- `base_dialog.json`
|
|
- `lock.json`
|
|
- `.tlk_state.json`
|
|
- `templates`
|
|
- `assets`
|
|
- Topdata compiled output subdirectories are hardcoded:
|
|
- `2da`
|
|
- `wiki`
|
|
- `wiki/pages`
|
|
- `wiki/state.json`
|
|
- Wiki generation has hardcoded managed namespaces and status labels.
|
|
- Wiki deploy defaults are hardcoded:
|
|
- `.wiki_deploy_manifest.json`
|
|
- `Auto-generated from native builder`
|
|
- managed namespaces including `classes`, `feat`, `itemtypes`, `races`, `skills`, `spells`, `meta`
|
|
- Topdata package fallback names are `sow_top.hak` and `sow_tlk.tlk`.
|
|
- Autogen released manifest cache path is `.cache/<cache_name>`.
|
|
- Autogen released manifest max age is one hour.
|
|
- `SOW_AUTOGEN_MANIFEST_REFRESH` forces refresh outside YAML.
|
|
- Released autogen manifest discovery derives a Shadows Over Westgate assets repo spec from git remote and environment overrides.
|
|
- Supported autogen modes are hardcoded:
|
|
- `parts_rows`
|
|
- `accessory_visualeffects`
|
|
- Supported derive kinds are hardcoded:
|
|
- `trailing_numeric_suffix`
|
|
- `model_stem`
|
|
- Supported derive group source is hardcoded:
|
|
- `first_path_segment`
|
|
|
|
Hardening direction:
|
|
|
|
- Separate topdata toolkit defaults from repository-specific defaults.
|
|
- Add explicit config for output subpaths, wiki generation/deploy defaults, autogen cache policy, and release source.
|
|
- Keep dataset JSON shape unchanged.
|
|
- Treat topdata dataset internals as authored data, not root YAML config, unless the behavior genuinely varies by repository.
|
|
|
|
## Extract behavior
|
|
|
|
- Extract output layout is hardcoded:
|
|
- `.nss` -> `<paths.source>/scripts/<resref>.nss`
|
|
- GFF resources -> `<paths.source>/<sourceSubdir(ext)>/<resref>.<ext>.json`
|
|
- other resources -> `<paths.assets>/<extension>/<resref>.<extension>`
|
|
- Stale cleanup uses the current project inventory.
|
|
- Ignore behavior is configurable only by extension.
|
|
|
|
Hardening direction:
|
|
|
|
- Add extract output layout settings or named extraction profiles.
|
|
- Keep current layout as `nwn_canonical_json`.
|
|
- Make cleanup scope explicit and visible.
|
|
|
|
## App and logging behavior
|
|
|
|
- Command descriptions mention hardcoded paths like `src/`, `assets/`, `build/`, `.cache/2da`, `.cache/wiki`, `sow_top.hak`, and `sow_tlk.tlk`.
|
|
- Verbosity flags are command-specific, especially for `build-haks`.
|
|
- Spinner and phase labels are hardcoded in `internal/app/app.go`.
|
|
- Config load diagnostics are not yet a full effective-config explanation.
|
|
|
|
Hardening direction:
|
|
|
|
- Update descriptions to avoid implying fixed repository layouts.
|
|
- Introduce shared global verbosity flags or a documented command-level policy.
|
|
- Expose effective config and active overrides through `config` commands.
|
|
|
|
# Proposed schema expansions
|
|
|
|
These are target concepts, not a requirement to implement every field in one pass.
|
|
|
|
```yaml
|
|
toolkit:
|
|
defaults_profile: nwn
|
|
generated_artifacts:
|
|
format: json
|
|
logging:
|
|
default_verbosity: normal
|
|
paths: relative
|
|
|
|
paths:
|
|
source: src
|
|
assets: assets
|
|
build: build
|
|
cache: .cache
|
|
tools: tools
|
|
|
|
outputs:
|
|
module_archive: "{module.resref}.mod"
|
|
hak_manifest: haks.json
|
|
hak_archive: "{hak.name}.hak"
|
|
|
|
inventory:
|
|
source_extensions:
|
|
- .are
|
|
- .dlg
|
|
- .fac
|
|
- .git
|
|
- .ifo
|
|
- .jrl
|
|
- .utc
|
|
- .utd
|
|
- .ute
|
|
- .uti
|
|
- .utm
|
|
- .utp
|
|
- .uts
|
|
- .utt
|
|
- .utw
|
|
asset_extensions:
|
|
- .2da
|
|
- .bmu
|
|
- .dds
|
|
- .mdl
|
|
- .wav
|
|
source_json_pattern: "{resref}.{extension}.json"
|
|
|
|
validation:
|
|
profile: nwn_module
|
|
builtin_script_prefixes:
|
|
- nw_
|
|
- x0_
|
|
- x1_
|
|
- x2_
|
|
- x3_
|
|
- ga_
|
|
- gc_
|
|
- gen_
|
|
- gui_
|
|
- nwg_
|
|
- ta_
|
|
|
|
scripts:
|
|
source_dir: scripts
|
|
cache: "{paths.cache}/ncs"
|
|
compiler:
|
|
path: ""
|
|
env:
|
|
path: SOW_NWN_SCRIPT_COMPILER
|
|
nwn_root: SOW_NWN_ROOT
|
|
nwn_user_directory: SOW_NWN_USER_DIRECTORY
|
|
search:
|
|
- "{paths.tools}/script-compiler/nwn_script_comp"
|
|
- "{paths.tools}/script-compiler/nwn_script_comp.exe"
|
|
- "{paths.tools}/nwn_script_comp"
|
|
- "{paths.tools}/nwn_script_comp.exe"
|
|
- PATH:nwn_script_comp
|
|
|
|
extract:
|
|
layout: nwn_canonical_json
|
|
ignore_extensions: []
|
|
cleanup_stale: true
|
|
delete_module_archive_after_success: false
|
|
|
|
topdata:
|
|
source: topdata
|
|
build: "{paths.build}/topdata"
|
|
compiled_2da_dir: 2da
|
|
compiled_tlk: sow_tlk.tlk
|
|
package_hak: sow_top.hak
|
|
package_tlk: sow_tlk.tlk
|
|
wiki:
|
|
enabled_by_default: false
|
|
output_root: wiki
|
|
pages_dir: pages
|
|
state_file: state.json
|
|
managed_namespaces:
|
|
- classes
|
|
- feat
|
|
- itemtypes
|
|
- races
|
|
- skills
|
|
- spells
|
|
deploy_manifest: .wiki_deploy_manifest.json
|
|
deploy_edit_summary: Auto-generated from native builder
|
|
|
|
autogen:
|
|
cache:
|
|
root: "{paths.cache}"
|
|
max_age: 1h
|
|
refresh_env: SOW_AUTOGEN_MANIFEST_REFRESH
|
|
release_source:
|
|
provider: gitea
|
|
server_url_env: SOW_ASSETS_SERVER_URL
|
|
repo_env: SOW_ASSETS_REPO
|
|
```
|
|
|
|
Implementation does not need to preserve exactly this naming, but each hardcoded behavior above should either become:
|
|
|
|
- a field in schema,
|
|
- a documented toolkit default visible in effective config, or
|
|
- a deliberate non-configurable engine invariant with a clear rationale.
|
|
|
|
# Effective config design
|
|
|
|
## Commands
|
|
|
|
Add config inspection commands:
|
|
|
|
```text
|
|
sow-toolkit config validate
|
|
sow-toolkit config effective [--json|--yaml]
|
|
sow-toolkit config inspect [<key>] [--json]
|
|
sow-toolkit config explain <key>
|
|
sow-toolkit config sources
|
|
```
|
|
|
|
`config validate` should load, normalize, and validate without scanning/building.
|
|
|
|
`config effective` should print the complete normalized config after defaults and compatibility migrations.
|
|
|
|
`config explain <key>` should show:
|
|
|
|
- final value
|
|
- source of the value:
|
|
- toolkit default
|
|
- root YAML
|
|
- compatibility migration
|
|
- environment override
|
|
- command-line override
|
|
- whether the value is deprecated
|
|
- validation rules that apply
|
|
|
|
`config sources` should print:
|
|
|
|
- loaded config path and format
|
|
- legacy status
|
|
- accepted root config candidates
|
|
- active environment overrides
|
|
- command-line overrides used for the current invocation
|
|
|
|
## Internal model
|
|
|
|
Introduce a normalized/effective config layer separate from raw YAML structs.
|
|
|
|
Suggested distinction:
|
|
|
|
- `RawConfig`: direct YAML representation with pointer fields where inheritance or "unset" matters.
|
|
- `EffectiveConfig`: fully resolved values consumed by commands.
|
|
- `ConfigProvenance`: source metadata for inspect/explain commands.
|
|
- `ConfigDiagnostics`: warnings/errors/deprecations produced during normalization and validation.
|
|
|
|
Commands should consume `EffectiveConfig` or methods derived from it, not raw YAML fields.
|
|
|
|
## Override rules
|
|
|
|
Order of authority:
|
|
|
|
1. toolkit built-in defaults
|
|
2. YAML root config
|
|
3. compatibility migrations from deprecated YAML fields
|
|
4. environment overrides
|
|
5. command-line overrides
|
|
|
|
For each override:
|
|
|
|
- document whether it is allowed
|
|
- record provenance
|
|
- show it in debug/effective output
|
|
- avoid silent behavior changes
|
|
|
|
Environment variables should not be removed immediately, but they should be treated as visible overrides, not hidden behavior.
|
|
|
|
# Hardening phases
|
|
|
|
## Phase 1: Baseline audit and effective config shell
|
|
|
|
- Add a config audit document generated from code search.
|
|
- Implement `config validate`.
|
|
- Implement `config effective --json` using current config and current defaults.
|
|
- Add provenance for:
|
|
- config filename
|
|
- legacy JSON use
|
|
- `paths.build` default
|
|
- topdata default package names
|
|
- HAK manifest default name
|
|
- Do not change build behavior in this phase.
|
|
|
|
## Phase 2: Centralize derived paths and names
|
|
|
|
- Move path/name derivation out of scattered methods and into effective config.
|
|
- Cover:
|
|
- module archive path
|
|
- HAK archive path template
|
|
- HAK manifest path
|
|
- cache root
|
|
- script cache
|
|
- topdata compiled dirs
|
|
- topdata package names
|
|
- wiki output paths
|
|
- autogen cache root
|
|
- Keep old methods as wrappers around effective config during migration.
|
|
- Add tests proving old and new derivations match.
|
|
|
|
## Phase 3: Schema expansion for current hardcoded behavior
|
|
|
|
- Add schema fields for outputs, cache roots, inventory extensions, script compiler settings, extract layout, topdata output subpaths, wiki deploy defaults, and autogen cache policy.
|
|
- Preserve current defaults.
|
|
- Add strict validation for new fields.
|
|
- Update README examples.
|
|
|
|
## Phase 4: Command integration
|
|
|
|
- Update build, extract, validate, compare, topdata, autogen, music, and wiki commands to consume effective config.
|
|
- Convert environment variables into explicit override records.
|
|
- Update logs to report configured names/paths instead of hardcoded labels.
|
|
- Add `--debug` output for effective values used by each command.
|
|
|
|
## Phase 5: Repository migrations
|
|
|
|
- Update consumer repository YAML to explicitly declare any behavior that is repository-specific.
|
|
- Remove compatibility shims only after all active repositories have migrated.
|
|
- Keep generated JSON artifacts stable unless a separate contract says otherwise.
|
|
- Add regression checks comparing pre/post build plans and generated manifests.
|
|
|
|
## Phase 6: Enforcement
|
|
|
|
- Add tests or static checks that reject new repository-specific literals in core command paths.
|
|
- Remove legacy JSON loading after the agreed migration window.
|
|
- Turn deprecated YAML shorthand warnings into validation errors where appropriate.
|
|
|
|
# Repository-by-repository checklist
|
|
|
|
For each consumer repository:
|
|
|
|
1. Identify root config file and confirm it is `nwn-tool.yaml` or `nwn-tool.yml`.
|
|
2. Confirm no root `nwn-tool.json` remains except as deliberate migration fixture.
|
|
3. List all wrapper scripts and environment variables they set.
|
|
4. Record local tool paths and whether they need YAML fields.
|
|
5. List generated artifact paths and confirm they are not treated as config.
|
|
6. Run current build and save:
|
|
- HAK manifest
|
|
- module HAK order
|
|
- topdata package paths
|
|
- autogen manifest paths
|
|
- credits inventory paths
|
|
7. Convert implicit behavior to explicit YAML.
|
|
8. Run `config effective` and verify values.
|
|
9. Run build/validate workflows and compare generated outputs.
|
|
10. Document intentional diffs.
|
|
|
|
# Test strategy expansion
|
|
|
|
Add tests for:
|
|
|
|
- `config validate` succeeds without scanning missing optional trees.
|
|
- `config effective --json` is deterministic.
|
|
- `config explain paths.build` identifies toolkit default when omitted.
|
|
- `config explain topdata.package_hak` identifies YAML source when configured.
|
|
- legacy `nwn-tool.json` emits warning and provenance.
|
|
- YAML wins over legacy JSON when both exist.
|
|
- unknown YAML fields fail before any scan/build side effects.
|
|
- environment overrides appear in effective config.
|
|
- command-line overrides appear in effective config for the command invocation.
|
|
- HAK manifest name can be configured without changing generated JSON schema.
|
|
- topdata package names can be configured and all command output follows them.
|
|
- script compiler cache path can be configured.
|
|
- extract layout profile preserves current output paths.
|
|
- autogen cache root and max age are configurable.
|
|
- wiki deploy manifest name is configurable.
|
|
- source/asset extension defaults match current scanning behavior.
|
|
- invalid path traversal in output/cache config fails early.
|
|
- absolute paths are rejected unless the field explicitly allows them.
|
|
- duplicate or case-colliding generated output paths fail validation.
|
|
- config-generated effective values are sorted deterministically.
|
|
|
|
Regression tests:
|
|
|
|
- current minimal YAML still builds to the same default paths.
|
|
- current Shadows Over Westgate-style topdata package output remains `sow_top.hak` / `sow_tlk.tlk` unless YAML changes it.
|
|
- current HAK manifest remains `haks.json` unless YAML changes it.
|
|
- current `.cache` outputs remain stable unless YAML changes them.
|
|
- old `music.prefixes` compatibility still maps to current behavior until removed.
|
|
|
|
# Priority refactor tickets
|
|
|
|
## P0: Authority and safety
|
|
|
|
- Add effective config command output.
|
|
- Record config provenance and legacy warnings.
|
|
- Centralize output/cache path derivation.
|
|
- Validate path traversal and unsafe output roots.
|
|
- Make environment overrides visible.
|
|
|
|
## P1: Repository-specific names and paths
|
|
|
|
- Configure HAK manifest filename.
|
|
- Configure cache root.
|
|
- Configure script compiler path/search/cache.
|
|
- Configure topdata compiled output subpaths.
|
|
- Configure wiki deploy manifest/edit summary/namespaces.
|
|
- Configure autogen cache root/max age/release source.
|
|
|
|
## P2: Inventory and validation profiles
|
|
|
|
- Configure source and asset extension lists.
|
|
- Configure or profile validator reference rules.
|
|
- Configure built-in script prefixes.
|
|
- Configure extract output layout.
|
|
|
|
## P3: Migration cleanup
|
|
|
|
- Deprecate and remove legacy JSON root config loading.
|
|
- Deprecate music prefix shorthand after dataset migration.
|
|
- Replace command descriptions that mention hardcoded defaults.
|
|
- Add static/lint checks for new repository-specific literals.
|
|
|
|
# Documentation strategy
|
|
|
|
- Document `nwn-tool.yaml` as the canonical root config file.
|
|
- Maintain a generated effective-config reference in docs.
|
|
- Document every toolkit default separately from repository examples.
|
|
- For every config field, document:
|
|
- type
|
|
- default
|
|
- whether relative paths are repo-relative or output-root-relative
|
|
- whether absolute paths are allowed
|
|
- validation rules
|
|
- generated artifacts affected
|
|
- migration notes
|
|
- Keep a clear line between:
|
|
- human-authored root YAML config
|
|
- human-authored topdata dataset JSON
|
|
- generated JSON artifacts
|
|
- compatibility/legacy files
|
|
|
|
# Expanded acceptance criteria
|
|
|
|
- `sow-toolkit config effective --json` shows all values that commands use.
|
|
- Every effective value can be traced to YAML, toolkit default, environment override, compatibility migration, or command override.
|
|
- Existing hardcoded defaults are either configurable or documented as toolkit defaults.
|
|
- Command implementations do not invent repository-specific paths after config normalization.
|
|
- Topdata, HAK, music, extract, autogen, wiki, and script compiler paths are inspectable.
|
|
- Legacy JSON root config use is visible and eventually removable.
|
|
- Generated JSON schemas remain stable unless intentionally changed.
|
|
- Existing consumer repositories build before and after migration.
|