Hardening audit

Key hardening changes:

  - Config validation now rejects paths.source: . and paths.assets: ., so source/asset roots cannot
    resolve to the repository root.
  - apply-hak-manifest now refuses to run when paths.source is unset or unsafe, instead of potentially
    writing under a root-level module/.
  - Inline flags across utility parsers now reject empty values consistently, e.g. --dataset=, --hak=,
    --endpoint=, --output=.
  - build-changelog now supports --flag=value inline syntax like the other utilities.
This commit is contained in:
2026-05-08 00:31:17 +02:00
parent 9f16aa8872
commit fa4c9116ae
8 changed files with 314 additions and 28 deletions
+75 -2
View File
@@ -12,6 +12,15 @@ Build the development binary into `tools/sow-toolkit`:
./build-tool.sh
```
On Windows:
```powershell
.\build-tool.ps1
```
Both wrappers use `.cache/go-build` for the Go build cache and skip rebuilding
when the sources are older than the existing development binary.
Or run directly with Go:
```bash
@@ -27,10 +36,39 @@ internal/erf/ ERF/MOD/HAK reading and writing
internal/gff/ canonical GFF JSON and binary conversion
internal/pipeline/ build, extract, compare, and manifest workflows
internal/project/ config loading and repo scanning
internal/topdata/ native topdata, wiki, and autogen workflows
internal/validator/validation and diagnostics
tools/ built development binary output
```
## Commands
The shared CLI exposes these project utilities:
```bash
sow-toolkit build
sow-toolkit build-module
sow-toolkit build-haks [--hak <name>] [--archive <name>] [--source-manifest <path>] [--plan-only] [--skip-music] [--music-dataset <id>]
sow-toolkit extract [<module-or-hak-file> ...]
sow-toolkit validate
sow-toolkit compare
sow-toolkit apply-hak-manifest [<manifest-path>]
sow-toolkit config <validate|effective|inspect|explain|sources>
sow-toolkit music <list-datasets|scan|build|credits|validate|manifest|normalize>
sow-toolkit validate-topdata
sow-toolkit build-topdata [--force] [--wiki]
sow-toolkit build-top-package [--force]
sow-toolkit compare-topdata
sow-toolkit convert-topdata <2da-to-json|2da-to-module|json-to-2da> ...
sow-toolkit build-wiki [--force]
sow-toolkit deploy-wiki [--source-dir <path>] [--endpoint <url>] [--token <token>] [--version <ver>] [--namespace <ns>] [--category <namespace=cid>] [--manifest <path>] [--dry-run] [--create] [--force]
sow-toolkit build-changelog [--config <path>] [--output <path>] [--current-tag <tag>] [--previous-tag <tag>] [--api-base-url <url>] [--token <token>]
```
Global logging flags `--quiet`, `--verbose`, and `--debug` may be passed after
the command name. Long options that take values accept either `--flag value` or
`--flag=value`; empty inline values such as `--dataset=` are rejected.
## Repository Configuration
Human-authored consumer repository configuration is YAML. The toolkit discovers
@@ -177,6 +215,19 @@ autogen:
repo_env: SOW_ASSETS_REPO
```
`paths.source` and `paths.assets` must name real repository subtrees when they
are configured. They may be omitted for repositories that do not own that class
of source, but they must not resolve to the repository root (`.`). Output and
cache paths must stay relative to the repository root unless a specific setting
documents otherwise.
Extraction is deliberately guarded because it writes source files from binary
archives. Module resources require a configured `paths.source`; HAK assets
require a configured `paths.assets`; both roots must resolve to safe subtrees.
If a root is unset or resolves to the repository root, extraction fails before
writing instead of creating extension directories like `mdl/` or `png/` at the
repo top level. Stale cleanup uses the same safe-root rule.
Music defaults (used when not explicitly configured):
```yaml
@@ -208,6 +259,8 @@ sow-toolkit music build --dataset westgate --dry-run
sow-toolkit music build --dataset westgate
sow-toolkit music credits --dataset westgate --check
sow-toolkit music validate --dataset westgate --json
sow-toolkit music manifest --dataset westgate
sow-toolkit music normalize --dataset westgate
```
`build-haks` uses the same music pipeline. Use `--plan-only` or music
@@ -220,17 +273,37 @@ topdata datasets, caches, credits inventories, build reports, and canonical GFF
documents. Nested metadata such as `topdata/templates/config.json` is not root
repository configuration and is intentionally unchanged.
## Wiki And Changelog Utilities
`build-topdata --wiki` or `build-wiki` renders wiki pages from the current
native topdata state. `deploy-wiki` publishes those pages to NodeBB. It reads
`NODEBB_API_ENDPOINT`, `NODEBB_API_TOKEN`, `GITHUB_REF_NAME`, and
`NODEBB_WIKI_CATEGORIES` when equivalent flags are omitted. Category mappings
use `namespace=cid`, for example:
```bash
sow-toolkit deploy-wiki --dry-run --namespace spells --category spells=42
```
`build-changelog` renders a release changelog from tagged pull requests. It
defaults to `scripts/changelog.json`, reads Gitea tokens from `GITEA_API_TOKEN`,
`GITEA_TOKEN`, or `SOW_TOOLS_TOKEN`, and writes to stdout unless `--output` is
provided.
## Intended Consumers
- `sow-module` uses `sow-toolkit` for `build-module`, `extract`, `validate`, `compare`, and `apply-hak-manifest`
- `sow-assets` uses `sow-toolkit` for `build-haks` and validation
- `sow-module` uses `sow-toolkit` for `build`, `build-module`, `extract`, `validate`, `compare`, and `apply-hak-manifest`
- `sow-assets` uses `sow-toolkit` for `build-haks`, music processing, changelog generation, and validation
- validation now fails early if duplicate asset resources would collide inside the same generated HAK
- duplicate resources split across different HAK groups remain warnings because later HAK order can still override earlier content at runtime
- `sow-module` now also uses `sow-toolkit` for topdata:
- `validate-topdata`
- `build-topdata`
- `build-top-package`
- `compare-topdata`
- `convert-topdata`
- `build-wiki`
- `deploy-wiki`
During development, sibling repositories can point at `../sow-tools/tools/sow-toolkit`
or `../sow-tools/tools/sow-toolkit.exe`. For pinned releases, each consumer repo can
+108 -23
View File
@@ -645,19 +645,31 @@ func parseBuildHAKArgs(args []string) (buildHAKOptions, error) {
}
opts.sourceManifest = args[index]
default:
if value, ok := parseInlineFlagValue(arg, "--hak"); ok {
if value, ok, err := requireInlineFlagValue(arg, "--hak"); ok || err != nil {
if err != nil {
return opts, err
}
opts.filteredHAKs = append(opts.filteredHAKs, value)
continue
}
if value, ok := parseInlineFlagValue(arg, "--archive"); ok {
if value, ok, err := requireInlineFlagValue(arg, "--archive"); ok || err != nil {
if err != nil {
return opts, err
}
opts.filteredArchives = append(opts.filteredArchives, value)
continue
}
if value, ok := parseInlineFlagValue(arg, "--source-manifest"); ok {
if value, ok, err := requireInlineFlagValue(arg, "--source-manifest"); ok || err != nil {
if err != nil {
return opts, err
}
opts.sourceManifest = value
continue
}
if value, ok := parseInlineFlagValue(arg, "--music-dataset"); ok {
if value, ok, err := requireInlineFlagValue(arg, "--music-dataset"); ok || err != nil {
if err != nil {
return opts, err
}
opts.musicDatasets = append(opts.musicDatasets, value)
continue
}
@@ -679,6 +691,17 @@ func parseInlineFlagValue(arg, name string) (string, bool) {
return strings.TrimSpace(strings.TrimPrefix(arg, prefix)), true
}
func requireInlineFlagValue(arg, name string) (string, bool, error) {
value, ok := parseInlineFlagValue(arg, name)
if !ok {
return "", false, nil
}
if value == "" {
return "", true, fmt.Errorf("%s requires a value", name)
}
return value, true, nil
}
type musicCommandOptions struct {
datasets []string
json bool
@@ -765,7 +788,10 @@ func parseMusicCommandArgs(args []string) (musicCommandOptions, error) {
case "--force":
opts.force = true
default:
if value, ok := parseInlineFlagValue(arg, "--dataset"); ok {
if value, ok, err := requireInlineFlagValue(arg, "--dataset"); ok || err != nil {
if err != nil {
return opts, err
}
opts.datasets = append(opts.datasets, value)
continue
}
@@ -1417,29 +1443,56 @@ func parseDeployWikiArgs(commandName string, args []string) (topdata.DeployWikiO
case "-h", "--help":
return opts, fmt.Errorf("usage: %s [--source-dir <path>] [--endpoint <url>] [--token <token>] [--version <ver>] [--namespace <ns> ...] [--category <namespace=cid> ...] [--manifest <path>] [--dry-run] [--create] [--force]", commandName)
default:
if strings.HasPrefix(arg, "--source-dir=") {
opts.SourceDir = strings.TrimPrefix(arg, "--source-dir=")
} else if strings.HasPrefix(arg, "--endpoint=") {
opts.Endpoint = strings.TrimPrefix(arg, "--endpoint=")
} else if strings.HasPrefix(arg, "--username=") {
opts.Username = strings.TrimPrefix(arg, "--username=")
} else if strings.HasPrefix(arg, "--password=") {
opts.Password = strings.TrimPrefix(arg, "--password=")
} else if strings.HasPrefix(arg, "--token=") {
opts.Token = strings.TrimPrefix(arg, "--token=")
} else if strings.HasPrefix(arg, "--version=") {
opts.Version = strings.TrimPrefix(arg, "--version=")
} else if strings.HasPrefix(arg, "--namespace=") {
opts.Namespaces = append(opts.Namespaces, strings.TrimPrefix(arg, "--namespace="))
} else if strings.HasPrefix(arg, "--category=") {
if value, ok, err := requireInlineFlagValue(arg, "--source-dir"); ok || err != nil {
if err != nil {
return opts, err
}
opts.SourceDir = value
} else if value, ok, err := requireInlineFlagValue(arg, "--endpoint"); ok || err != nil {
if err != nil {
return opts, err
}
opts.Endpoint = value
} else if value, ok, err := requireInlineFlagValue(arg, "--username"); ok || err != nil {
if err != nil {
return opts, err
}
opts.Username = value
} else if value, ok, err := requireInlineFlagValue(arg, "--password"); ok || err != nil {
if err != nil {
return opts, err
}
opts.Password = value
} else if value, ok, err := requireInlineFlagValue(arg, "--token"); ok || err != nil {
if err != nil {
return opts, err
}
opts.Token = value
} else if value, ok, err := requireInlineFlagValue(arg, "--version"); ok || err != nil {
if err != nil {
return opts, err
}
opts.Version = value
} else if value, ok, err := requireInlineFlagValue(arg, "--namespace"); ok || err != nil {
if err != nil {
return opts, err
}
opts.Namespaces = append(opts.Namespaces, value)
} else if value, ok, err := requireInlineFlagValue(arg, "--category"); ok || err != nil {
if err != nil {
return opts, err
}
if opts.CategoryIDs == nil {
opts.CategoryIDs = map[string]int{}
}
if err := parseWikiCategoryArg(strings.TrimPrefix(arg, "--category="), opts.CategoryIDs); err != nil {
if err := parseWikiCategoryArg(value, opts.CategoryIDs); err != nil {
return opts, err
}
} else if strings.HasPrefix(arg, "--manifest=") {
opts.ManifestPath = strings.TrimPrefix(arg, "--manifest=")
} else if value, ok, err := requireInlineFlagValue(arg, "--manifest"); ok || err != nil {
if err != nil {
return opts, err
}
opts.ManifestPath = value
} else if arg == "--dry-run" {
opts.DryRun = true
} else if arg == "--force" {
@@ -1546,9 +1599,41 @@ func parseBuildChangelogArgs(commandName string, args []string) (buildChangelogO
case "-h", "--help":
return opts, usage
default:
if value, ok, err := requireInlineFlagValue(arg, "--config"); ok || err != nil {
if err != nil {
return opts, err
}
opts.configPath = value
} else if value, ok, err := requireInlineFlagValue(arg, "--output"); ok || err != nil {
if err != nil {
return opts, err
}
opts.outputPath = value
} else if value, ok, err := requireInlineFlagValue(arg, "--current-tag"); ok || err != nil {
if err != nil {
return opts, err
}
opts.currentTag = value
} else if value, ok, err := requireInlineFlagValue(arg, "--previous-tag"); ok || err != nil {
if err != nil {
return opts, err
}
opts.previousTag = value
} else if value, ok, err := requireInlineFlagValue(arg, "--api-base-url"); ok || err != nil {
if err != nil {
return opts, err
}
opts.apiBaseURL = value
} else if value, ok, err := requireInlineFlagValue(arg, "--token"); ok || err != nil {
if err != nil {
return opts, err
}
opts.token = value
} else {
return opts, fmt.Errorf("unknown %s argument %q", commandName, arg)
}
}
}
return opts, nil
}
+37
View File
@@ -391,6 +391,43 @@ paths:
}
}
func TestInlineFlagParsersRejectEmptyValues(t *testing.T) {
if _, err := parseBuildHAKArgs([]string{"--hak="}); err == nil || !strings.Contains(err.Error(), "--hak requires a value") {
t.Fatalf("expected empty --hak inline value error, got %v", err)
}
if _, err := parseMusicCommandArgs([]string{"--dataset="}); err == nil || !strings.Contains(err.Error(), "--dataset requires a value") {
t.Fatalf("expected empty --dataset inline value error, got %v", err)
}
if _, err := parseDeployWikiArgs("deploy-wiki", []string{"--endpoint="}); err == nil || !strings.Contains(err.Error(), "--endpoint requires a value") {
t.Fatalf("expected empty --endpoint inline value error, got %v", err)
}
if _, err := parseBuildChangelogArgs("build-changelog", []string{"--output="}); err == nil || !strings.Contains(err.Error(), "--output requires a value") {
t.Fatalf("expected empty --output inline value error, got %v", err)
}
}
func TestParseBuildChangelogArgsAcceptsInlineValues(t *testing.T) {
opts, err := parseBuildChangelogArgs("build-changelog", []string{
"--config=scripts/changelog.json",
"--output=CHANGELOG.md",
"--current-tag=v2",
"--previous-tag=v1",
"--api-base-url=https://gitea.example/api/v1",
"--token=secret",
})
if err != nil {
t.Fatalf("parseBuildChangelogArgs failed: %v", err)
}
if opts.configPath != "scripts/changelog.json" ||
opts.outputPath != "CHANGELOG.md" ||
opts.currentTag != "v2" ||
opts.previousTag != "v1" ||
opts.apiBaseURL != "https://gitea.example/api/v1" ||
opts.token != "secret" {
t.Fatalf("unexpected changelog opts: %#v", opts)
}
}
func mkdirAll(t *testing.T, path string) {
t.Helper()
if err := os.MkdirAll(path, 0o755); err != nil {
+9 -1
View File
@@ -5,6 +5,7 @@ import (
"fmt"
"os"
"path/filepath"
"strings"
"gitea.westgate.pw/ShadowsOverWestgate/sow-tools/internal/gff"
"gitea.westgate.pw/ShadowsOverWestgate/sow-tools/internal/project"
@@ -20,6 +21,13 @@ func ApplyHAKManifest(p *project.Project, manifestPath string) (ApplyManifestRes
if manifestPath == "" {
manifestPath = p.HAKManifestPath()
}
if strings.TrimSpace(p.EffectiveConfig().Paths.Source) == "" {
return ApplyManifestResult{}, fmt.Errorf("cannot apply hak manifest: paths.source is not configured")
}
sourceRoot := filepath.Clean(p.SourceDir())
if sourceRoot == "." || sourceRoot == string(filepath.Separator) || sourceRoot == filepath.Clean(p.Root) {
return ApplyManifestResult{}, fmt.Errorf("cannot apply hak manifest: paths.source resolves to unsafe source root %s", sourceRoot)
}
raw, err := os.ReadFile(manifestPath)
if err != nil {
@@ -31,7 +39,7 @@ func ApplyHAKManifest(p *project.Project, manifestPath string) (ApplyManifestRes
return ApplyManifestResult{}, fmt.Errorf("parse hak manifest: %w", err)
}
moduleSource := filepath.Join(p.SourceDir(), "module", "module.ifo.json")
moduleSource := filepath.Join(sourceRoot, "module", "module.ifo.json")
sourceRaw, err := os.ReadFile(moduleSource)
if err != nil {
return ApplyManifestResult{}, fmt.Errorf("read module ifo source: %w", err)
+37
View File
@@ -548,6 +548,43 @@ func TestExtractKeepsConsumedModuleArchiveWhenExtractionFails(t *testing.T) {
}
}
func TestApplyHAKManifestRefusesUnsetSourcePath(t *testing.T) {
root := t.TempDir()
mustMkdir(t, filepath.Join(root, "assets"))
mustMkdir(t, filepath.Join(root, "build"))
mustMkdir(t, filepath.Join(root, "module"))
mustWriteFile(t, filepath.Join(root, "nwn-tool.yaml"), `
module:
name: Test Module
resref: testmod
paths:
assets: assets
build: build
`)
mustWriteFile(t, filepath.Join(root, "build", "haks.json"), `{"module_haks":["core"],"haks":[]}`)
mustWriteFile(t, filepath.Join(root, "module", "module.ifo.json"), "{}\n")
p, err := project.Load(root)
if err != nil {
t.Fatalf("load project: %v", err)
}
_, err = ApplyHAKManifest(p, filepath.Join(root, "build", "haks.json"))
if err == nil {
t.Fatal("expected apply manifest to fail")
}
if !strings.Contains(err.Error(), "paths.source is not configured") {
t.Fatalf("expected missing source path error, got %v", err)
}
raw, readErr := os.ReadFile(filepath.Join(root, "module", "module.ifo.json"))
if readErr != nil {
t.Fatalf("read root module file: %v", readErr)
}
if string(raw) != "{}\n" {
t.Fatalf("expected root module file to remain unchanged, got %q", string(raw))
}
}
func TestBuildSplitsConfiguredHAKs(t *testing.T) {
root := t.TempDir()
mustMkdir(t, filepath.Join(root, "src"))
+14
View File
@@ -470,6 +470,8 @@ func (p *Project) ValidateLayout() error {
failures = append(failures, validateValidationConfig(p.Config.Validation)...)
effective := p.EffectiveConfig()
failures = append(failures, validateRelativePath("paths.cache", effective.Paths.Cache)...)
failures = append(failures, validateTreeRootPath("paths.source", effective.Paths.Source)...)
failures = append(failures, validateTreeRootPath("paths.assets", effective.Paths.Assets)...)
failures = append(failures, validateRelativePath("outputs.module_archive", effective.Outputs.ModuleArchive)...)
failures = append(failures, validateRelativePath("outputs.hak_manifest", effective.Outputs.HAKManifest)...)
failures = append(failures, validateRelativePath("outputs.hak_archive", effective.Outputs.HAKArchive)...)
@@ -1406,6 +1408,18 @@ func validateRelativePath(field, path string) []error {
return failures
}
func validateTreeRootPath(field, path string) []error {
failures := validateRelativePath(field, path)
trimmed := strings.TrimSpace(path)
if trimmed == "" {
return failures
}
if filepath.Clean(filepath.FromSlash(trimmed)) == "." {
failures = append(failures, fmt.Errorf("%s must not resolve to the repository root: %q", field, path))
}
return failures
}
func sameCleanPath(a, b string) bool {
return filepath.Clean(a) == filepath.Clean(b)
}
+24
View File
@@ -648,6 +648,30 @@ func TestValidateLayoutRejectsUnsafeGeneratedOutputPaths(t *testing.T) {
}
}
func TestValidateLayoutRejectsRootSourceAndAssetPaths(t *testing.T) {
root := t.TempDir()
mkdirAll(t, filepath.Join(root, "build"))
proj := &Project{
Root: root,
Config: Config{
Module: ModuleConfig{Name: "Test", ResRef: "test"},
Paths: PathConfig{Source: ".", Assets: ".", Build: "build"},
},
}
err := proj.ValidateLayout()
if err == nil {
t.Fatal("expected root source and asset path validation error")
}
if !strings.Contains(err.Error(), "paths.source") {
t.Fatalf("expected source path validation error, got %v", err)
}
if !strings.Contains(err.Error(), "paths.assets") {
t.Fatalf("expected assets path validation error, got %v", err)
}
}
func TestValidateLayoutRejectsInvalidAutogenConsumerConfig(t *testing.T) {
root := t.TempDir()
mkdirAll(t, filepath.Join(root, "src"))
+9 -1
View File
@@ -1 +1,9 @@
Place the built `sow-toolkit` binary here for sibling repositories to consume during development or for release packaging.
Built development binaries live here:
- `sow-toolkit` on Linux/macOS
- `sow-toolkit.exe` on Windows
Use `../sow-tools/build-tool.sh` or `../sow-tools/build-tool.ps1` from this
repository root to refresh the binary. Sibling consumer repositories may point
their wrappers at this directory during local development; release workflows can
copy a pinned binary into each consumer repo's own `tools/` directory.