Compare commits

..
4 Commits
Author SHA1 Message Date
archvillainette 8e7cead5c0 Keep model compilation headless (#49)
build-binaries / build-binaries (push) Successful in 2m14s
## Summary

- always wrap NWN model compilation with `xvfb-run`, even when the caller has `DISPLAY`
- fail closed with an actionable error when `xvfb-run` is unavailable instead of opening the client UI
- update the compiler contract and regression coverage

## Verification

- focused red/green regression tests
- `nix develop -c make check`
- real NWN compile with `DISPLAY=:0` under transient Xvfb; binary MDL output verified

Generated with Claude CodeReviewed-on: #49

Co-authored-by: vickydotbat <vickydotbat@tutamail.com>
2026-07-23 07:27:37 +00:00
archvillainette 4d38967078 fix(topdata): let pinned TLK ids evict stale state owners (#48)
build-binaries / build-binaries (push) Successful in 2m14s
## Problem

The custom palette taxonomy (#46) pins display strings to fixed TLK ids in `tlk/custom.tlk.yml`, and `registerInlineAtID` demanded each pinned id be free. But `.tlk_state.json` is **gitignored and per-machine** — each dev grows their own copy, and before #46 every ref got its id dynamically (first-come-first-served).

On any machine whose state predates #46, a ref could have already parked on a now-pinned id. That fails the build:

```
topdata validation failed with 1 error(s): error: native topdata buildability check failed:
TLK id 2689 is already reserved by "feat:yuanti/alternate_form.feat"
```

It only passes on machines whose state was regenerated after #46 (pinned window already clean). The `allocateID` reserved-skip that #46 added protects a *fresh* build, but does nothing about a *stale cache*.

## Fix

Make the pin authoritative over the per-machine cache. A stale cached owner sitting on a pinned id is evicted and reallocated a fresh id when next made active; only two pins fighting over the same id in `custom.tlk.yml` is now an error. This self-heals on the next build — no manual `.tlk_state.json` deletion needed.

Caveat: the evicted ref's strref shifts on affected machines. That's unavoidable (something must move off the pinned id), and dynamic strrefs were never stable across machines anyway.

## Tests

Added `TestBuildStandaloneTLKPinEvictsStaleStateOwner` (seeds a pre-#46 state with a feat parked on the pinned id, asserts the build succeeds, the pin owns it, and the feat is reallocated). Full `internal/topdata` suite + `go vet` pass.

🤖 Generated with [Claude Code](https://claude.com/claude-code)Reviewed-on: #48
Reviewed-by: xtul <mpiasecki720@protonmail.com>
Co-authored-by: vickydotbat <vickydotbat@tutamail.com>
2026-07-22 18:17:04 +00:00
archvillainette 7b481ca0c0 custom palette taxonomy (#46)
build-binaries / build-binaries (push) Successful in 2m12s
Implements custom palette taxonomyReviewed-on: #46

Co-authored-by: vickydotbat <vickydotbat@tutamail.com>
2026-07-22 17:03:58 +00:00
archvillainette b058846e16 chore(ci): consolidate Gitea CI (#47)
## Summary

- consolidate vet, tests, lint, smoke, and cross-build into one pull-request-only job
- keep binary releases strictly v-tag-only with release-only write permission
- retain wrapper synchronization only for canonical wrapper changes on main
- pin actions and enforce exact trigger contracts

## Verification

- nix develop --command make check
- trigger mutation contract
- 28-workflow cross-repository actionlint audit

Generated with Claude CodeReviewed-on: #47

Co-authored-by: vickydotbat <vickydotbat@tutamail.com>
2026-07-22 16:54:26 +00:00
12 changed files with 288 additions and 61 deletions
+9 -13
View File
@@ -1,25 +1,22 @@
# Cross-platform Crucible binaries (D7 trigger standard):
# PR / push main -> cross-build ALL targets to prove they compile (no publish)
# tag v* -> build all targets + SHA256SUMS, upload to the Gitea release
# Cross-platform Crucible release binaries: a v* tag builds all targets,
# then uploads them, SHA256SUMS, and the canonical wrappers to its Gitea release.
# Crucible is pure Go (CGO_ENABLED=0), so cross-compiling is a fast loop.
name: build-binaries
on:
pull_request:
push:
branches: [main]
tags: ['v*']
paths-ignore:
- "docs/**"
- "README.md"
- "AGENTS.md"
- "LICENSE"
permissions:
code: read
releases: write
jobs:
build-binaries:
runs-on: nix-docker
timeout-minutes: 30
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with: { fetch-depth: 0 }
- name: Cross-build all targets
@@ -45,9 +42,8 @@ jobs:
'
- name: Upload to Gitea release
if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags/v')
env:
TOKEN: ${{ secrets.GITHUB_TOKEN }}
TOKEN: ${{ secrets.GITEA_TOKEN }}
TAG: ${{ github.ref_name }}
REPO: ${{ github.repository }}
SERVER: ${{ github.server_url }}
+47
View File
@@ -0,0 +1,47 @@
# Pull-request validation for Crucible. Releases and wrapper synchronization
# have their own narrow workflows because they need tag/main events.
name: ci
on:
pull_request:
permissions: read-all
jobs:
ci:
runs-on: nix-docker
timeout-minutes: 60
steps:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
fetch-depth: 0
- name: vet + test + lint
run: |
nix develop --command bash -c '
set -euo pipefail
go vet ./...
go test ./...
shellcheck scripts/*.sh
yamllint .gitea
'
- name: binary smoke (fail-closed contract)
run: nix develop --command make smoke
- name: Cross-build all targets
run: |
nix develop --command bash -c '
set -euo pipefail
sha="$(git rev-parse --short=12 HEAD)"
ldflags="-s -w -X git.westgate.pw/ShadowsOverWestgate/sow-tools/internal/buildinfo.Version=${sha}"
rm -rf dist && mkdir -p dist
export CGO_ENABLED=0
for target in linux/amd64 linux/arm64 darwin/amd64 darwin/arm64 windows/amd64 windows/arm64; do
os="${target%/*}"; arch="${target#*/}"
ext=""; [ "$os" = windows ] && ext=".exe"
echo "building crucible-${os}-${arch}${ext}"
GOOS="$os" GOARCH="$arch" go build -trimpath -ldflags "$ldflags" \
-o "dist/crucible-${os}-${arch}${ext}" ./cmd/crucible
done
'
+4 -1
View File
@@ -14,11 +14,14 @@ on:
- 'wrappers/crucible.sh'
- 'wrappers/crucible.ps1'
permissions: read-all
jobs:
sync:
runs-on: nix-docker
timeout-minutes: 30
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- name: Open sync PRs to consumers
env:
-30
View File
@@ -1,30 +0,0 @@
# Lint + unit tests for the Crucible suite. PR-first: PRs + push to main (D7).
name: test
on:
push:
branches: [main]
paths-ignore:
- "docs/**"
- "README.md"
- "AGENTS.md"
- "LICENSE"
pull_request:
jobs:
test:
runs-on: nix-docker
steps:
- uses: actions/checkout@v4
with: { fetch-depth: 0 }
- name: vet + test + lint
run: |
nix develop --command bash -c '
set -euo pipefail
go vet ./...
go test ./...
shellcheck scripts/*.sh
yamllint .gitea
'
- name: binary smoke (fail-closed contract)
run: nix develop --command make smoke
+1
View File
@@ -6,6 +6,7 @@ SHELL := bash
check: vet test
shellcheck scripts/*.sh
yamllint .gitea
bash tests/workflow-contract.sh
vet:
go vet ./...
+6 -5
View File
@@ -78,13 +78,14 @@ This retires the old habit of checking in `nwn-tool` / `sow-toolkit`.
## CI
PR-first (D7): checks run on pull requests and on push to `main`; the only
publish event is a `v*` tag (see `runbooks/ci-trigger-standard.md` in sow-docs,
PR-first (D7): checks run once on pull requests; the only publish event is a
`v*` tag (see `runbooks/ci-trigger-standard.md` in sow-docs,
https://git.westgate.pw/ShadowsOverWestgate/sow-docs).
- `test.yml` — vet, test, shellcheck, yamllint, binary smoke (PR + main).
- `build-binaries.yml` — cross-build all targets (PR + main); on a `v*` tag, upload
the binaries, `SHA256SUMS`, and the wrappers to the Gitea release.
- `ci.yml` — vet, test, shellcheck, yamllint, binary smoke, and cross-build all
targets once per pull request.
- `build-binaries.yml` — on a `v*` tag, cross-build and upload the binaries,
`SHA256SUMS`, and the wrappers to the Gitea release.
- `sync-wrappers.yml` — on a `main` push that touches `wrappers/`, auto-PR the
canonical wrappers to the consumer repos in `wrappers/consumers.txt`.
Consumer drift checks run after those PRs merge to `main`, not on the PRs
@@ -126,8 +126,9 @@ exactly as the reference does:
- Beamdog install dirs.
- `--nwn <install>` overrides (path to the install root or directly to the
binary).
- **Headless.** The engine is a GUI binary. If there is no `DISPLAY` and
`xvfb-run` is present, wrap the call:
- **Headless.** The engine is a GUI binary. Require `xvfb-run` and wrap the call
regardless of the caller's `DISPLAY` so compilation never opens the client
UI; fail before invoking the engine when `xvfb-run` is unavailable:
`xvfb-run -a --server-args=-screen 0 1024x768x24 nwmain-linux compilemodel <stem>`.
- **Per model (one at a time — the engine's `development/` and `modelcompiler/`
folders are flat and single-slot):**
+6 -6
View File
@@ -40,13 +40,13 @@ func runCompile(args []string, stdout, stderr io.Writer, getenv func(string) str
}
binDir := filepath.Dir(nwmain)
// Headless wrap: no DISPLAY + xvfb-run present -> run under a virtual X.
var wrap []string
if getenv("DISPLAY") == "" {
if xvfb := look("xvfb-run"); xvfb != "" {
wrap = []string{xvfb, "-a", "--server-args=-screen 0 1024x768x24"}
}
// Always use a virtual X so compilation never opens the client UI.
xvfb := look("xvfb-run")
if xvfb == "" {
fmt.Fprintln(stderr, "assets compile: xvfb-run not found — install it to run the NWN model compiler headlessly")
return exitTool
}
wrap := []string{xvfb, "-a", "--server-args=-screen 0 1024x768x24"}
files, err := walk(dirs, mdlExt, !*nonRecursive)
if err != nil {
+63 -2
View File
@@ -4,6 +4,7 @@ import (
"bytes"
"os"
"path/filepath"
"strings"
"testing"
)
@@ -23,13 +24,19 @@ func TestCompileDrivesEngineAndReplacesInPlace(t *testing.T) {
if err := os.WriteFile(nwmain, []byte("#!/bin/sh\n"), 0o755); err != nil {
t.Fatal(err)
}
xvfbDir := t.TempDir()
xvfb := filepath.Join(xvfbDir, "xvfb-run")
if err := os.WriteFile(xvfb, []byte("#!/bin/sh\n"), 0o755); err != nil {
t.Fatal(err)
}
t.Setenv("PATH", xvfbDir+string(os.PathListSeparator)+os.Getenv("PATH"))
getenv := func(k string) string {
switch k {
case "HOME":
return home
case "DISPLAY":
return ":0" // pretend a display exists so no xvfb wrap is needed
return ":0" // a desktop display must not make compilation interactive
}
return ""
}
@@ -38,7 +45,11 @@ func TestCompileDrivesEngineAndReplacesInPlace(t *testing.T) {
orig := runner
defer func() { runner = orig }()
runner = func(dir string, env []string, name string, args ...string) ([]byte, error) {
// args: compilemodel <stem>
if name != xvfb || len(args) != 5 || args[0] != "-a" ||
args[1] != "--server-args=-screen 0 1024x768x24" || args[2] != nwmain ||
args[3] != "compilemodel" {
t.Fatalf("engine command = %q %q, want xvfb-run wrapping nwmain", name, args)
}
stem := args[len(args)-1]
compiled := filepath.Join(mc, stem+".mdl")
return nil, os.WriteFile(compiled, []byte("\x00\x00compiled"), 0o644)
@@ -77,6 +88,11 @@ func TestCompileAbortsOnNameMismatch(t *testing.T) {
if err := os.WriteFile(nwmain, []byte("#!/bin/sh\n"), 0o755); err != nil {
t.Fatal(err)
}
xvfbDir := t.TempDir()
if err := os.WriteFile(filepath.Join(xvfbDir, "xvfb-run"), []byte("#!/bin/sh\n"), 0o755); err != nil {
t.Fatal(err)
}
t.Setenv("PATH", xvfbDir+string(os.PathListSeparator)+os.Getenv("PATH"))
getenv := func(k string) string {
if k == "HOME" {
return home
@@ -108,3 +124,48 @@ func TestCompileAbortsOnNameMismatch(t *testing.T) {
t.Fatal("engine should not run for a name-mismatched model")
}
}
func TestCompileFailsClosedWithoutXvfb(t *testing.T) {
home := t.TempDir()
userData := filepath.Join(home, ".local", "share", "Neverwinter Nights")
for _, d := range []string{filepath.Join(userData, "development"), filepath.Join(userData, "modelcompiler")} {
if err := os.MkdirAll(d, 0o755); err != nil {
t.Fatal(err)
}
}
nwmain := filepath.Join(t.TempDir(), "nwmain-linux")
if err := os.WriteFile(nwmain, []byte("#!/bin/sh\n"), 0o755); err != nil {
t.Fatal(err)
}
t.Setenv("PATH", t.TempDir())
getenv := func(k string) string {
if k == "HOME" {
return home
}
return ""
}
orig := runner
defer func() { runner = orig }()
engineCalled := false
runner = func(string, []string, string, ...string) ([]byte, error) {
engineCalled = true
return nil, nil
}
srcDir := t.TempDir()
if err := os.WriteFile(filepath.Join(srcDir, "foo.mdl"),
[]byte("newmodel foo\nbeginmodelgeom foo\n node dummy foo\n parent null\n endnode\nendmodelgeom foo\ndonemodel foo\n"), 0o644); err != nil {
t.Fatal(err)
}
var stdout, stderr bytes.Buffer
if code := runCompile([]string{"--nwn", nwmain, srcDir}, &stdout, &stderr, getenv); code != exitTool {
t.Fatalf("compile exit = %d, want %d\n%s", code, exitTool, stderr.String())
}
if engineCalled {
t.Fatal("engine must not run without xvfb-run")
}
if !strings.Contains(stderr.String(), "xvfb-run") {
t.Fatalf("missing actionable xvfb-run error: %s", stderr.String())
}
}
+16 -2
View File
@@ -109,6 +109,7 @@ type tlkCompiler struct {
active map[string]tlkEntryData
activeKeys map[string]struct{}
reservedByID map[int]string
pinnedByID map[int]string
nextID int
}
@@ -158,6 +159,7 @@ func newTLKCompiler(sourceDir string, legacy *legacyTLKData) (*tlkCompiler, erro
active: map[string]tlkEntryData{},
activeKeys: map[string]struct{}{},
reservedByID: reserved,
pinnedByID: map[int]string{},
nextID: nextID,
}
if legacy != nil {
@@ -380,12 +382,24 @@ func (c *tlkCompiler) registerInlineAtID(key string, id int, entry tlkEntryData)
if id < 0 {
return fmt.Errorf("TLK key %q has negative id %d", key, id)
}
// The pin is authoritative over the per-machine .tlk_state.json cache: a
// stale mapping that dynamically grabbed this id on an older build must
// yield so the pinned key can take it. Only a genuine clash between two
// pins in custom.tlk.yml is an author error.
if owner, ok := c.pinnedByID[id]; ok && owner != key {
return fmt.Errorf("TLK id %d is pinned by both %q and %q", id, owner, key)
}
if mapping, ok := c.state.Entries[key]; ok && mapping.ID != id {
return fmt.Errorf("TLK key %q changed id from %d to %d", key, mapping.ID, id)
// This key held a different cached id; release it so the pin wins.
if c.reservedByID[mapping.ID] == key {
delete(c.reservedByID, mapping.ID)
}
}
if owner, ok := c.reservedByID[id]; ok && owner != key {
return fmt.Errorf("TLK id %d is already reserved by %q", id, owner)
// Evict the stale owner; it gets a fresh id when next made active.
delete(c.state.Entries, owner)
}
c.pinnedByID[id] = key
c.state.Entries[key] = tlkStateMapping{ID: id}
c.reservedByID[id] = key
return c.markActive(key, entry)
+49
View File
@@ -2449,6 +2449,55 @@ strings:
}
}
func TestBuildStandaloneTLKPinEvictsStaleStateOwner(t *testing.T) {
root := testProjectRoot(t)
mkdirAll(t, filepath.Join(root, "topdata", "data", "feat"))
mkdirAll(t, filepath.Join(root, "topdata", "tlk"))
writeFile(t, filepath.Join(root, "topdata", "base_dialog.json"), "{}\n")
writeFile(t, filepath.Join(root, "topdata", "data", "feat", "base.json"), `{
"output": "feat.2da",
"columns": ["LABEL", "FEAT", "DESCRIPTION"],
"rows": [{
"id": 0,
"key": "feat:test",
"LABEL": "TEST_LABEL",
"FEAT": {"tlk": {"key": "feat:test.name", "text": "Test Feat"}},
"DESCRIPTION": "****"
}]
}`+"\n")
writeFile(t, filepath.Join(root, "topdata", "tlk", "custom.tlk.yml"), `schema: sow-topdata/tlk/v1
base_strref: 16777216
strings:
- key: sow.module.name
text: Shadows Over Westgate
id: 50
`)
// Simulate a per-machine state that predates the pinned taxonomy: the feat
// ref dynamically grabbed id 50 on an older build, exactly where the pin
// now lives. The build must self-heal instead of failing.
writeFile(t, filepath.Join(root, "topdata", tlkStateFile),
`{"version":1,"language":"en","entries":{"feat:test.name":{"id":50}}}`+"\n")
if _, err := BuildNative(testProject(root), nil); err != nil {
t.Fatalf("BuildNative failed on stale pin collision: %v", err)
}
stateRaw, err := os.ReadFile(filepath.Join(root, "topdata", tlkStateFile))
if err != nil {
t.Fatalf("read tlk state: %v", err)
}
var state tlkStateDocument
if err := json.Unmarshal(stateRaw, &state); err != nil {
t.Fatalf("parse tlk state: %v", err)
}
if state.Entries["sow.module.name"].ID != 50 {
t.Fatalf("pin must own id 50, got %#v", state.Entries["sow.module.name"])
}
if got := state.Entries["feat:test.name"].ID; got == 50 {
t.Fatalf("stale feat ref should have been reallocated off id 50, got %d", got)
}
}
func TestBuildPreservesTLKStateAcrossTextChanges(t *testing.T) {
root := testProjectRoot(t)
mkdirAll(t, filepath.Join(root, "topdata", "data", "skills"))
+84
View File
@@ -0,0 +1,84 @@
#!/usr/bin/env bash
set -euo pipefail
contract_script="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/$(basename "${BASH_SOURCE[0]}")"
default_root="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
repo_root="${WORKFLOW_ROOT:-$default_root}"
cd "$repo_root"
ci=.gitea/workflows/ci.yml
release=.gitea/workflows/build-binaries.yml
sync=.gitea/workflows/sync-wrappers.yml
checkout='actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683'
assert_exact_events() {
local workflow="$1" expected="$2" actual
actual="$(awk '
/^on:$/ { in_on = 1 }
in_on && /^$/ { next }
in_on && $0 != "on:" && $0 !~ /^[[:space:]]/ { exit }
in_on { print }
' "$workflow")"
[[ "$actual" == "$expected" ]] || {
echo "workflow-contract: unexpected event scope in $workflow" >&2
return 1
}
}
[[ -f "$ci" ]] || { echo "missing consolidated CI workflow" >&2; exit 1; }
[[ ! -e .gitea/workflows/test.yml ]] || { echo "legacy test workflow still present" >&2; exit 1; }
assert_exact_events "$ci" $'on:\n pull_request:'
grep -Fqx 'permissions: read-all' "$ci"
grep -Fqx ' timeout-minutes: 60' "$ci"
[[ "$(grep -Fc "$checkout" "$ci")" -eq 1 ]]
grep -Fqx ' fetch-depth: 0' "$ci"
grep -Fq 'go vet ./...' "$ci"
grep -Fq 'go test ./...' "$ci"
grep -Fq 'shellcheck scripts/*.sh' "$ci"
grep -Fq 'yamllint .gitea' "$ci"
grep -Fq 'make smoke' "$ci"
grep -Fq 'for target in linux/amd64 linux/arm64 darwin/amd64 darwin/arm64 windows/amd64 windows/arm64; do' "$ci"
assert_exact_events "$release" $'on:\n push:\n tags: [\'v*\']'
grep -Fqx ' code: read' "$release"
grep -Fqx ' releases: write' "$release"
grep -Fqx ' timeout-minutes: 30' "$release"
[[ "$(grep -Fc "$checkout" "$release")" -eq 1 ]]
grep -Fq 'secrets.GITEA_TOKEN' "$release"
if grep -Fq 'secrets.GITHUB_TOKEN' "$release"; then
echo "workflow-contract: release uses the GitHub token alias" >&2
exit 1
fi
grep -Fqx 'permissions: read-all' "$sync"
grep -Fqx ' timeout-minutes: 30' "$sync"
[[ "$(grep -Fc "$checkout" "$sync")" -eq 1 ]]
assert_exact_events "$sync" $'on:\n push:\n branches: [main]\n paths:\n - \'wrappers/crucible.sh\'\n - \'wrappers/crucible.ps1\''
expect_mutation_rejected() {
local name="$1" workflow="$2" expression="$3" tmp
tmp="$(mktemp -d)"
mkdir -p "$tmp/.gitea"
cp -R "$default_root/.gitea/workflows" "$tmp/.gitea/workflows"
sed -i "$expression" "$tmp/$workflow"
if WORKFLOW_ROOT="$tmp" WORKFLOW_CONTRACT_MUTATION=1 bash "$contract_script" >/dev/null 2>&1; then
echo "workflow-contract: accepted forbidden mutation: $name" >&2
rm -rf "$tmp"
return 1
fi
rm -rf "$tmp"
}
if [[ "${WORKFLOW_CONTRACT_MUTATION:-0}" != 1 ]]; then
mutations_ok=0
expect_mutation_rejected 'CI workflow_dispatch event' "$ci" '/^ pull_request:$/a\ workflow_dispatch:' || mutations_ok=1
expect_mutation_rejected 'release develop branch' "$release" "/^ tags:/a\\ branches: [develop]" || mutations_ok=1
expect_mutation_rejected 'release schedule event' "$release" "/^ tags:/a\\ schedule:\n - cron: '0 0 * * *'" || mutations_ok=1
expect_mutation_rejected 'release extra tag pattern' "$release" "s/tags: \['v\*'\]/tags: ['v*', 'release-*']/" || mutations_ok=1
expect_mutation_rejected 'wrapper sync pull_request event' "$sync" '/^ push:$/i\ pull_request:' || mutations_ok=1
expect_mutation_rejected 'wrapper sync broad path' "$sync" "/^ - 'wrappers\/crucible.ps1'$/a\\ - 'wrappers/**'" || mutations_ok=1
(( mutations_ok == 0 )) || exit 1
fi
echo "workflow-contract: CI is PR-only; release and wrapper sync retain their narrow triggers"