Compare commits

...
6 Commits
Author SHA1 Message Date
archvillainette 16d5586587 Crucible: Fix Formatting Churn (#5)
test-image / build-image (push) Successful in 43s
test / test (push) Successful in 1m21s
build-binaries / build-binaries (push) Successful in 2m2s
build-image / publish (push) Successful in 13s
RC#1 — build wrote 2-space instead of .editorconfig's 4-space
- editorconfig_format.go: the glob translator escaped {/} as literals, so [*.{json,jsonc}]→indent_size=4 never matched any file and formatting fell back to [*]→2. Implemented real editorconfig brace expansion — {a,b,c} alternation and {n..m} numeric ranges. Nothing is hardcoded; saveLockfile already read .editorconfig, it just got the wrong section.

RC#2 — validate wrote a lockfile (it must be read-only)
- The three registry collectors persisted lockfiles as a side effect of collection; ValidateProject calls collection outside its snapshot guard, so the write leaked. Threaded a persistLocks bool through collectGeneratedRegistryDatasets and the damagetypes/itemprops/racialtypes collectors. Only buildNativeUnchecked (the real build allocator) passes true; validate, discovery, packaging queries, and wiki discovery pass false.

Tests added: editorconfig_glob_test.go (brace match + 4-space via brace glob), registry_readonly_test.go (read-only writes nothing; persist writes allocations).

Reviewed-on: #5
Co-authored-by: vickydotbat <vickydotbat@tutamail.com>
Co-committed-by: vickydotbat <vickydotbat@tutamail.com>
2026-06-17 07:57:54 +00:00
archvillainette b7c0f43064 Fix autogen to fail-open on missing asset manifests (#4)
test-image / build-image (push) Successful in 45s
test / test (push) Successful in 1m22s
build-binaries / build-binaries (push) Successful in 2m2s
build-image / publish (push) Successful in 13s
Reviewed-on: #4
Co-authored-by: vickydotbat <vickydotbat@tutamail.com>
Co-committed-by: vickydotbat <vickydotbat@tutamail.com>
2026-06-16 21:50:45 +00:00
archvillainette d45bd84bc6 feat: crucible wrapper sync (#3)
test / test (push) Successful in 1m21s
test-image / build-image (push) Successful in 44s
build-binaries / build-binaries (push) Successful in 2m3s
build-image / publish (push) Successful in 12s
Reviewed-on: #3
Co-authored-by: vickydotbat <vickydotbat@tutamail.com>
Co-committed-by: vickydotbat <vickydotbat@tutamail.com>
2026-06-16 15:38:58 +00:00
archvillainette 93433e3f53 Cross-Platform Crucible (#2)
test / test (push) Successful in 1m20s
build-binaries / build-binaries (push) Successful in 2m0s
test-image / build-image (push) Successful in 44s
build-image / publish (push) Successful in 12s
Crucible groundwork for cross-platform compatibility.

Reviewed-on: #2
Reviewed-by: xtul <mpiasecki720@protonmail.com>
Co-authored-by: vickydotbat <vickydotbat@tutamail.com>
Co-committed-by: vickydotbat <vickydotbat@tutamail.com>
2026-06-16 13:52:19 +00:00
archvillainette e9d82816b7 Release workflow revision (#1)
test-image / build-image (push) Successful in 46s
test / test (push) Successful in 1m20s
Adds a manual release workflow and ensures images are only published on v* tags

Reviewed-on: #1
Co-authored-by: vickydotbat <vickydotbat@tutamail.com>
Co-committed-by: vickydotbat <vickydotbat@tutamail.com>
2026-06-16 13:08:10 +00:00
archvillainette f3211f0d45 docs: up to date
build-image / build-image (push) Successful in 50s
test / test (push) Successful in 1m18s
2026-06-14 08:44:29 +02:00
30 changed files with 1152 additions and 110 deletions
+66
View File
@@ -0,0 +1,66 @@
# Cross-platform Crucible binaries (D7 trigger standard):
# PR / push main -> cross-build ALL targets to prove they compile (no publish)
# tag v* -> build all targets + SHA256SUMS, upload to the Gitea release
# Crucible is pure Go (CGO_ENABLED=0), so cross-compiling is a fast loop.
name: build-binaries
on:
pull_request:
push:
branches: [main]
tags: ['v*']
jobs:
build-binaries:
runs-on: nix-docker
steps:
- uses: actions/checkout@v4
with: { fetch-depth: 0 }
- name: Cross-build all targets
run: |
nix develop --command bash -c '
set -euo pipefail
sha="$(git rev-parse --short=12 HEAD)"
ldflags="-s -w -X git.westgate.pw/ShadowsOverWestgate/sow-tools/internal/buildinfo.Version=${sha}"
rm -rf dist && mkdir -p dist
export CGO_ENABLED=0
for target in linux/amd64 linux/arm64 darwin/amd64 darwin/arm64 windows/amd64 windows/arm64; do
os="${target%/*}"; arch="${target#*/}"
ext=""; [ "$os" = windows ] && ext=".exe"
echo "building crucible-${os}-${arch}${ext}"
GOOS="$os" GOARCH="$arch" go build -trimpath -ldflags "$ldflags" \
-o "dist/crucible-${os}-${arch}${ext}" ./cmd/crucible
done
# Ship the canonical wrappers as release assets too: consumer
# drift-check fetches them from the latest release to compare.
cp wrappers/crucible.sh wrappers/crucible.ps1 dist/
( cd dist && sha256sum crucible-* > SHA256SUMS )
cat dist/SHA256SUMS
'
- name: Upload to Gitea release
if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags/v')
env:
TOKEN: ${{ secrets.GITHUB_TOKEN }}
TAG: ${{ github.ref_name }}
REPO: ${{ github.repository }}
SERVER: ${{ github.server_url }}
run: |
nix develop --command bash -c '
set -euo pipefail
api="${SERVER}/api/v1/repos/${REPO}"
auth="Authorization: token ${TOKEN}"
# Create the release (ignore "already exists"), then read its id.
curl -fsS -X POST -H "$auth" -H "Content-Type: application/json" \
"${api}/releases" \
-d "{\"tag_name\":\"${TAG}\",\"name\":\"${TAG}\"}" || true
id="$(curl -fsS -H "$auth" "${api}/releases/tags/${TAG}" \
| grep -o "\"id\":[0-9]*" | head -1 | cut -d: -f2)"
for f in dist/*; do
echo "uploading $(basename "$f")"
curl -fsS -X POST -H "$auth" \
-F "attachment=@${f}" \
"${api}/releases/${id}/assets?name=$(basename "$f")"
done
'
+6 -7
View File
@@ -1,5 +1,6 @@
# Build the Crucible image. PR-first (D7): PRs build only; push to main also # Publish the immutable crucible:<sha> image on version tags.
# publishes registry.westgate.pw/deployment/crucible:<git-sha>. Never a mutable tag. # PR/main test builds live in test-image.yml and never publish, so registry
# packages are only generated for releases we intend to use (not on every merge).
# #
# Daemonless: the host-mode runner has no container runtime, so the image is # Daemonless: the host-mode runner has no container runtime, so the image is
# built by Nix (`nix build .#image`, see flake.nix) and pushed with skopeo # built by Nix (`nix build .#image`, see flake.nix) and pushed with skopeo
@@ -8,15 +9,14 @@ name: build-image
on: on:
push: push:
branches: [main] tags: ['v*']
pull_request:
env: env:
REGISTRY: registry.westgate.pw REGISTRY: registry.westgate.pw
IMAGE: deployment/crucible IMAGE: deployment/crucible
jobs: jobs:
build-image: publish:
runs-on: nix-docker runs-on: nix-docker
steps: steps:
- uses: actions/checkout@v4 - uses: actions/checkout@v4
@@ -29,9 +29,8 @@ jobs:
- name: Build OCI image (daemonless) - name: Build OCI image (daemonless)
run: nix build .#image run: nix build .#image
# Publish only on main (post-merge). PRs verify the build but never push. # This workflow only runs on v* tags, so every run is a release publish.
- name: Publish image - name: Publish image
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
env: env:
REGISTRY_USER: ${{ secrets.REGISTRY_USER }} REGISTRY_USER: ${{ secrets.REGISTRY_USER }}
REGISTRY_PASSWORD: ${{ secrets.REGISTRY_PASSWORD }} REGISTRY_PASSWORD: ${{ secrets.REGISTRY_PASSWORD }}
+49
View File
@@ -0,0 +1,49 @@
# Manual compatibility publisher for the Crucible image.
# Normal release publishing happens in build-image.yml on v* tags; this is the
# break-glass / re-publish path, triggered by hand.
#
# Daemonless: built by Nix (`nix build .#image`) and pushed with skopeo from the
# OCI tarball. No `docker build`/`docker login` involved.
name: publish-image
on:
workflow_dispatch:
env:
REGISTRY: registry.westgate.pw
IMAGE: deployment/crucible
jobs:
publish:
runs-on: nix-docker
steps:
- uses: actions/checkout@v4
with: { fetch-depth: 0 }
- name: Resolve tag
id: tag
run: echo "sha=$(git rev-parse --short=12 HEAD)" >> "$GITHUB_OUTPUT"
- name: Build OCI image (daemonless)
run: nix build .#image
- name: Publish image
env:
REGISTRY_USER: ${{ secrets.REGISTRY_USER }}
REGISTRY_PASSWORD: ${{ secrets.REGISTRY_PASSWORD }}
run: |
nix shell nixpkgs#skopeo -c skopeo copy \
--dest-creds "${REGISTRY_USER}:${REGISTRY_PASSWORD}" \
docker-archive:result \
"docker://${REGISTRY}/${IMAGE}:${{ steps.tag.outputs.sha }}"
- name: Emit release fragment
run: |
FRAG_REPO=sow-tools \
FRAG_SHA=${{ steps.tag.outputs.sha }} \
FRAG_ARTIFACT="${REGISTRY}/${IMAGE}:${{ steps.tag.outputs.sha }}" \
FRAG_URL="${REGISTRY}/${IMAGE}" \
FRAG_RUN_ID=${{ gitea.run_id }} \
bash scripts/emit-release-fragment.sh
- uses: actions/upload-artifact@v3
with: { name: release-fragment, path: release-fragment.json }
-38
View File
@@ -1,38 +0,0 @@
# Tag a Crucible release: re-tag the already-built immutable image and attach
# per-platform binary bundles. Tag-gated; never PR-triggered (deploy/release is
# not a PR concern, D7). The :<sha> image from build-image is the source of truth.
name: release
on:
push:
tags:
- "v*"
env:
REGISTRY: registry.westgate.pw
IMAGE: deployment/crucible
jobs:
release:
runs-on: nix-docker
steps:
- uses: actions/checkout@v4
with: { fetch-depth: 0 }
- name: Build release binaries
run: |
nix develop --command bash -c '
set -euo pipefail
GOOS=linux GOARCH=amd64 bash scripts/build-binaries.sh
mkdir -p dist
tar -C bin -czf "dist/crucible-linux-amd64-${GITHUB_REF_NAME}.tar.gz" .
'
- name: Upload release artifacts
uses: actions/upload-artifact@v4
with:
name: crucible-${{ github.ref_name }}
path: dist/*
# TODO(phase-6): re-tag registry.westgate.pw/deployment/crucible:<sha> as :<tag>
# and pin it from sow-platform releases/*.yml once the registry is live.
+50
View File
@@ -0,0 +1,50 @@
# Auto-PR canonical wrapper updates to consumer repos when wrappers/ changes on
# main. Maintenance automation (not artifact publishing), so it is allowed on a
# main push under the D7 trigger standard. Requires WRAPPER_SYNC_TOKEN: a bot
# user's token with content+PR write to the consumer repos (never committed).
name: sync-wrappers
on:
push:
branches: [main]
paths:
- 'wrappers/crucible.sh'
- 'wrappers/crucible.ps1'
jobs:
sync:
runs-on: nix-docker
steps:
- uses: actions/checkout@v4
- name: Open sync PRs to consumers
env:
TOKEN: ${{ secrets.WRAPPER_SYNC_TOKEN }}
SERVER: ${{ github.server_url }}
SRC_SHA: ${{ github.sha }}
run: |
nix develop --command bash -c '
set -euo pipefail
host="$(echo "$SERVER" | sed -E "s#https?://##")"
branch="chore/sync-wrappers-$(echo "$SRC_SHA" | cut -c1-12)"
grep -vE "^\s*#|^\s*$" wrappers/consumers.txt | while read -r target; do
echo "== syncing $target =="
work="$(mktemp -d)"
git clone "https://oauth2:${TOKEN}@${host}/${target}.git" "$work"
cp wrappers/crucible.sh wrappers/crucible.ps1 "$work"/
( cd "$work"
git config user.name "crucible-sync-bot"
git config user.email "bot@westgate.pw"
if git diff --quiet; then echo "no changes for $target"; exit 0; fi
git checkout -b "$branch"
git add crucible.sh crucible.ps1
git commit -m "chore: sync crucible wrappers from sow-tools@${SRC_SHA}"
git push -f origin "$branch"
curl -fsS -X POST \
-H "Authorization: token ${TOKEN}" -H "Content-Type: application/json" \
"${SERVER}/api/v1/repos/${target}/pulls" \
-d "{\"head\":\"${branch}\",\"base\":\"main\",\"title\":\"chore: sync crucible wrappers from sow-tools\"}" || true
)
rm -rf "$work"
done
'
+19
View File
@@ -0,0 +1,19 @@
# Test-build the Crucible image on PRs and main. Proves `nix build .#image`
# still works (daemonless, Nix-built OCI tarball) but does NOT publish —
# release publishing happens in build-image.yml on v* tags.
name: test-image
on:
push:
branches: [main]
pull_request:
jobs:
build-image:
runs-on: nix-docker
steps:
- uses: actions/checkout@v4
with: { fetch-depth: 0 }
- name: Build OCI image (daemonless, no publish)
run: nix build .#image
+1 -1
View File
@@ -6,7 +6,7 @@ alwaysApply: true
# sow-tools / Crucible Agent Guide # sow-tools / Crucible Agent Guide
This repo owns the **builder logic** for the migration: one Go module This repo owns the **builder logic** for the migration: one Go module
(`gitea.westgate.pw/ShadowsOverWestgate/sow-tools`) producing the `crucible` (`git.westgate.pw/ShadowsOverWestgate/sow-tools`) producing the `crucible`
dispatcher and the `crucible-<name>` binaries (D11). Read dispatcher and the `crucible-<name>` binaries (D11). Read
`../AGENTS.md` (migration hub) and `../../KICKOFF_PROMPT.md` first. `../AGENTS.md` (migration hub) and `../../KICKOFF_PROMPT.md` first.
+35 -5
View File
@@ -41,6 +41,30 @@ See [`docs/migration-from-nwn-tool.md`](docs/migration-from-nwn-tool.md) for wha
was done and what remains (the consumer `--manifest/--source/--out` flag contract was done and what remains (the consumer `--manifest/--source/--out` flag contract
is the open Phase-6 item). is the open Phase-6 item).
## Quick start (no Nix)
Teammates without Nix don't build anything — they run the bootstrap wrapper,
which downloads the latest released `crucible` for your OS and runs it:
```bash
./crucible # interactive menu (pick a command)
./crucible module build
./crucible topdata validate-topdata
```
Windows (PowerShell):
```powershell
.\crucible.ps1 module build
```
The binary is cached under `~/.cache/crucible/<version>/` (`%LOCALAPPDATA%\crucible`
on Windows); `--repo-local` caches inside the repo instead. Private releases:
set `CRUCIBLE_TOKEN` or write the token to `~/.config/crucible/token`.
Only the **music** builder needs `ffmpeg`; everything else has zero dependencies.
If you run a music command without it, Crucible prints a per-OS install hint.
## Develop ## Develop
Self-contained (D8) — a host with only Nix can run everything: Self-contained (D8) — a host with only Nix can run everything:
@@ -58,12 +82,18 @@ This retires the old habit of checking in `nwn-tool` / `sow-toolkit`.
## CI ## CI
PR-first (D7): every check runs on pull requests and on push to `main`. PR-first (D7): checks run on pull requests and on push to `main`; the only
publish event is a `v*` tag (see `sow-docs/runbooks/ci-trigger-standard.md`).
- `test.yml` — vet, test, shellcheck, yamllint, binary smoke. - `test.yml` — vet, test, shellcheck, yamllint, binary smoke (PR + main).
- `build-image.yml` — build `registry.westgate.pw/deployment/crucible:<sha>`; publish - `test-image.yml` — build the OCI image to prove it compiles (PR + main, no push).
only on `main`. PRs build but never push. No mutable tags. - `build-binaries.yml` — cross-build all targets (PR + main); on a `v*` tag, upload
- `release.yml` — tag-gated binary bundles; image re-tag is wired in Phase 6. the binaries, `SHA256SUMS`, and the wrappers to the Gitea release.
- `build-image.yml` — on a `v*` tag, build and publish
`registry.westgate.pw/deployment/crucible:<sha>`.
- `publish-image.yml` — manual `workflow_dispatch` break-glass republish.
- `sync-wrappers.yml` — on a `main` push that touches `wrappers/`, auto-PR the
canonical wrappers to the consumer repos in `wrappers/consumers.txt`.
## Consumers ## Consumers
+5 -2
View File
@@ -24,14 +24,17 @@ quoting stays correct.
## In CI (preferred) ## In CI (preferred)
Run the consumer job inside the pinned image and the binaries are on `PATH`: **OPERATOR NOTE: Stop. Do not do it this way.**
**Use the pinned `prod.yml` version. That's what it's there for.**
~~Run the consumer job inside the pinned image and the binaries are on `PATH`:~~
```yaml ```yaml
container: container:
image: registry.westgate.pw/deployment/crucible:<sha> # pinned, immutable image: registry.westgate.pw/deployment/crucible:<sha> # pinned, immutable
``` ```
No host install, no `$HOME` layout, no developer machine assumptions. ~~No host install, no `$HOME` layout, no developer machine assumptions.~~
## `NWN_ROOT` rule ## `NWN_ROOT` rule
+51 -4
View File
@@ -17,6 +17,7 @@ import (
"git.westgate.pw/ShadowsOverWestgate/sow-tools/internal/app" "git.westgate.pw/ShadowsOverWestgate/sow-tools/internal/app"
"git.westgate.pw/ShadowsOverWestgate/sow-tools/internal/buildinfo" "git.westgate.pw/ShadowsOverWestgate/sow-tools/internal/buildinfo"
"git.westgate.pw/ShadowsOverWestgate/sow-tools/internal/menu"
) )
// Exit codes follow the sysexits(3) convention so CI can distinguish // Exit codes follow the sysexits(3) convention so CI can distinguish
@@ -103,8 +104,52 @@ func find(name string) (Builder, bool) {
return Builder{}, false return Builder{}, false
} }
// Main is the `crucible` dispatcher entrypoint. // Main is the `crucible` dispatcher entrypoint. With no arguments on an
func Main(args []string) int { return run(args, os.Stdout, os.Stderr) } // interactive terminal it launches the menu; otherwise (pipes, CI) it prints
// usage, preserving scriptable behavior.
func Main(args []string) int {
if len(args) == 0 && interactive(os.Stdout) {
return runMenu(os.Stdout, os.Stderr, os.Stdin)
}
return run(args, os.Stdout, os.Stderr)
}
func interactive(f *os.File) bool {
fi, err := f.Stat()
if err != nil {
return false
}
return fi.Mode()&os.ModeCharDevice != 0
}
func menuItems() []menu.Item {
var items []menu.Item
for _, b := range Registry {
if !b.Wired {
continue
}
for _, sub := range b.subcommands() {
items = append(items, menu.Item{
Label: b.Name + " " + sub,
Desc: b.Summary,
Args: []string{b.Name, sub},
})
}
}
return items
}
func runMenu(out, errw io.Writer, in io.Reader) int {
chosen, err := menu.Select(out, in, menuItems())
if err != nil {
fmt.Fprintln(errw, "crucible:", err)
return exitUsage
}
if chosen == nil {
return exitOK
}
return run(chosen, out, errw)
}
// RunBuilder is the standalone `crucible-<name>` entrypoint. // RunBuilder is the standalone `crucible-<name>` entrypoint.
func RunBuilder(name string, args []string) int { func RunBuilder(name string, args []string) int {
@@ -204,8 +249,10 @@ func usage(w io.Writer) {
fmt.Fprintf(w, " list list builders (one per line: name<TAB>bin<TAB>summary)\n") fmt.Fprintf(w, " list list builders (one per line: name<TAB>bin<TAB>summary)\n")
fmt.Fprintf(w, " config [args] inspect/validate effective configuration\n") fmt.Fprintf(w, " config [args] inspect/validate effective configuration\n")
fmt.Fprintf(w, " changelog [args] generate the release changelog\n") fmt.Fprintf(w, " changelog [args] generate the release changelog\n")
fmt.Fprintf(w, "\nNWN_ROOT must be passed explicitly (env or flag); crucible never guesses\n") fmt.Fprintf(w, "\nBuilders read all inputs from the project tree (nwn-tool.yaml + data/); no\n")
fmt.Fprintf(w, "from $HOME. See docs/consumer-contract.md.\n") fmt.Fprintf(w, "NWN install is required. If a builder ever needs an NWN root, it must be\n")
fmt.Fprintf(w, "passed explicitly via NWN_ROOT; crucible never guesses from $HOME. See\n")
fmt.Fprintf(w, "docs/consumer-contract.md.\n")
} }
func list(w io.Writer) { func list(w io.Writer) {
+19
View File
@@ -135,3 +135,22 @@ func TestLegacyCommandsHaveExactlyOneHome(t *testing.T) {
} }
} }
} }
func TestMenuItemsSkipsUnwiredIncludesWired(t *testing.T) {
items := menuItems()
if len(items) == 0 {
t.Fatal("expected menu items")
}
sawModuleBuild := false
for _, it := range items {
if it.Args[0] == "depot" {
t.Errorf("unwired builder %q must not appear in the menu", it.Args[0])
}
if len(it.Args) == 2 && it.Args[0] == "module" && it.Args[1] == "build" {
sawModuleBuild = true
}
}
if !sawModuleBuild {
t.Error("expected 'module build' in the menu")
}
}
+60
View File
@@ -0,0 +1,60 @@
// Package menu renders Crucible's interactive launcher: a numbered list of
// commands for users who don't want to memorize subcommands. It holds no
// builder logic — it returns the chosen dispatcher argument slice for the
// caller to run. This is the seed of a future GUI front-end over the same core.
package menu
import (
"bufio"
"fmt"
"io"
"strconv"
"strings"
)
// Item is one selectable command.
type Item struct {
Label string // e.g. "module build"
Desc string // e.g. "build/extract/validate/compare the .mod"
Args []string // dispatcher args, e.g. {"module", "build"}
}
// ANSI styling; terminals that don't support it simply ignore the codes.
const (
cReset = "\033[0m"
cBold = "\033[1m"
cCyan = "\033[36m"
cDim = "\033[2m"
)
// Select prints the numbered menu to out, reads a selection plus optional extra
// arguments from in, and returns the full dispatcher argument slice. It returns
// (nil, nil) when the user quits or enters an empty choice.
func Select(out io.Writer, in io.Reader, items []Item) ([]string, error) {
if len(items) == 0 {
return nil, fmt.Errorf("no commands available")
}
fmt.Fprintf(out, "%s%sCrucible%s — pick a command:\n\n", cBold, cCyan, cReset)
for i, it := range items {
fmt.Fprintf(out, " %s%2d%s %s%-24s%s %s%s%s\n",
cCyan, i+1, cReset, cBold, it.Label, cReset, cDim, it.Desc, cReset)
}
fmt.Fprintf(out, " %sq%s quit\n\nselection: ", cCyan, cReset)
r := bufio.NewReader(in)
line, _ := r.ReadString('\n')
switch choice := strings.TrimSpace(line); choice {
case "", "q", "Q":
return nil, nil
default:
n, err := strconv.Atoi(choice)
if err != nil || n < 1 || n > len(items) {
return nil, fmt.Errorf("invalid selection %q", choice)
}
it := items[n-1]
fmt.Fprintf(out, "extra args for %q (optional): ", it.Label)
argLine, _ := r.ReadString('\n')
extra := strings.Fields(strings.TrimSpace(argLine))
return append(append([]string{}, it.Args...), extra...), nil
}
}
+47
View File
@@ -0,0 +1,47 @@
package menu
import (
"io"
"reflect"
"strings"
"testing"
)
var sample = []Item{
{Label: "module build", Desc: "build the .mod", Args: []string{"module", "build"}},
{Label: "topdata validate-topdata", Desc: "validate topdata", Args: []string{"topdata", "validate-topdata"}},
}
func TestSelectReturnsChosenArgs(t *testing.T) {
got, err := Select(io.Discard, strings.NewReader("1\n\n"), sample)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if want := []string{"module", "build"}; !reflect.DeepEqual(got, want) {
t.Fatalf("got %v, want %v", got, want)
}
}
func TestSelectAppendsExtraArgs(t *testing.T) {
got, err := Select(io.Discard, strings.NewReader("2\n--dry-run foo\n"), sample)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
want := []string{"topdata", "validate-topdata", "--dry-run", "foo"}
if !reflect.DeepEqual(got, want) {
t.Fatalf("got %v, want %v", got, want)
}
}
func TestSelectQuitReturnsNil(t *testing.T) {
got, err := Select(io.Discard, strings.NewReader("q\n"), sample)
if err != nil || got != nil {
t.Fatalf("quit should return (nil,nil); got (%v,%v)", got, err)
}
}
func TestSelectInvalidErrors(t *testing.T) {
if _, err := Select(io.Discard, strings.NewReader("99\n"), sample); err == nil {
t.Fatal("out-of-range selection should error")
}
}
+15 -2
View File
@@ -1,6 +1,7 @@
package music package music
import ( import (
"fmt"
"os" "os"
"os/exec" "os/exec"
"path/filepath" "path/filepath"
@@ -104,7 +105,7 @@ func ResolveFFmpegWithConfig(configuredPath string) (string, error) {
} }
path, err := exec.LookPath("ffmpeg") path, err := exec.LookPath("ffmpeg")
if err != nil { if err != nil {
return "", err return "", ffmpegMissingErr("ffmpeg")
} }
return path, nil return path, nil
} }
@@ -122,11 +123,23 @@ func ResolveFFprobeWithConfig(configuredPath string) (string, error) {
} }
path, err := exec.LookPath("ffprobe") path, err := exec.LookPath("ffprobe")
if err != nil { if err != nil {
return "", err return "", ffmpegMissingErr("ffprobe")
} }
return path, nil return path, nil
} }
// ffmpegMissingErr renders an actionable, cross-platform "tool not found"
// message. Only the music builder needs ffmpeg/ffprobe; every other builder
// runs with zero external dependencies.
func ffmpegMissingErr(tool string) error {
return fmt.Errorf(`%[1]s not found on PATH.
Windows: winget install Gyan.FFmpeg
macOS: brew install ffmpeg
Linux: apt install ffmpeg (or your distro's package manager)
Only the music builder needs %[1]s; set SOW_%[2]s to a full path to override.`,
tool, strings.ToUpper(tool))
}
func IsMusicAssetPath(rel string) bool { func IsMusicAssetPath(rel string) bool {
rel = filepath.ToSlash(strings.TrimSpace(rel)) rel = filepath.ToSlash(strings.TrimSpace(rel))
return rel == "envi/music" || strings.HasPrefix(rel, "envi/music/") return rel == "envi/music" || strings.HasPrefix(rel, "envi/music/")
+13
View File
@@ -402,3 +402,16 @@ func TestResolveFFprobeEnv(t *testing.T) {
t.Fatalf("expected '/custom/ffprobe', got %q", path) t.Fatalf("expected '/custom/ffprobe', got %q", path)
} }
} }
func TestFFmpegMissingErrHasInstallHints(t *testing.T) {
msg := ffmpegMissingErr("ffmpeg").Error()
for _, want := range []string{"ffmpeg", "winget", "brew", "apt", "SOW_FFMPEG"} {
if !strings.Contains(msg, want) {
t.Errorf("ffmpeg error missing %q; got:\n%s", want, msg)
}
}
probe := ffmpegMissingErr("ffprobe").Error()
if !strings.Contains(probe, "SOW_FFPROBE") {
t.Errorf("ffprobe error should mention SOW_FFPROBE; got:\n%s", probe)
}
}
+131 -25
View File
@@ -3,6 +3,7 @@ package topdata
import ( import (
"bytes" "bytes"
"encoding/json" "encoding/json"
"errors"
"fmt" "fmt"
"os" "os"
"path/filepath" "path/filepath"
@@ -16,6 +17,19 @@ import (
const autogenManifestCacheMaxAge = time.Hour const autogenManifestCacheMaxAge = time.Hour
// errAutogenManifestUnavailable marks a released autogen manifest that could not
// be located or fetched (network failure, missing release/asset, empty payload,
// or an undeterminable sow-assets repo). It is deliberately distinct from an
// asset that was found but reports a row removed: for an optional consumer this
// sentinel triggers a fail-open path that *preserves* the consumer's existing
// pinned lock entries instead of pruning them, so StrRef/row IDs are not
// reshuffled while the asset source is merely temporarily out of reach.
var errAutogenManifestUnavailable = errors.New("autogen manifest unavailable")
func unavailableAutogenManifest(err error) error {
return fmt.Errorf("%w: %w", errAutogenManifestUnavailable, err)
}
type autogenManifest struct { type autogenManifest struct {
ID string `json:"id"` ID string `json:"id"`
Repo string `json:"repo"` Repo string `json:"repo"`
@@ -61,9 +75,18 @@ func applyAutogenConsumers(p *project.Project, collected []nativeCollectedDatase
} }
manifest, err := resolveAutogenConsumerManifest(p, consumer, progress) manifest, err := resolveAutogenConsumerManifest(p, consumer, progress)
if err != nil { if err != nil {
if consumer.Optional && isIgnorableOptionalAutogenError(err) { if consumer.Optional && errors.Is(err, errAutogenManifestUnavailable) {
// Fail open: the asset manifest is merely unreachable, not
// authoritatively empty. Build without its rows, but keep the
// consumer's already-pinned lock entries so their IDs are not
// freed and reshuffled on a later build once the asset returns.
preserved, perr := preserveAutogenConsumerLockEntries(result, consumer)
if perr != nil {
return nil, perr
}
result = preserved
if progress != nil { if progress != nil {
progress(fmt.Sprintf("Skipping optional autogen consumer %s: %v", consumer.ID, err)) progress(fmt.Sprintf("Autogen consumer %s: released manifest unavailable (%v); preserving existing lock entries, generating no new rows", consumer.ID, err))
} }
continue continue
} }
@@ -90,32 +113,115 @@ func applyAutogenConsumers(p *project.Project, collected []nativeCollectedDatase
func autogenConsumerTargetsCollectedDataset(collected []nativeCollectedDataset, consumer project.AutogenConsumerConfig) bool { func autogenConsumerTargetsCollectedDataset(collected []nativeCollectedDataset, consumer project.AutogenConsumerConfig) bool {
for _, dataset := range collected { for _, dataset := range collected {
switch consumer.Mode { if autogenConsumerTargetsDataset(dataset, consumer) {
case "parts_rows":
if isAutogenEligiblePartsDataset(dataset) {
return true return true
} }
case "accessory_visualeffects":
if dataset.Dataset.Name == "visualeffects" {
return true
}
case "cachedmodels_rows":
if dataset.Dataset.Name == "cachedmodels" {
return true
}
}
} }
return false return false
} }
func isIgnorableOptionalAutogenError(err error) bool { // preserveAutogenConsumerLockEntries restores the lock keys that an autogen
if err == nil { // consumer owns from the on-disk lockfile back into the in-memory dataset,
// reusing this build's already-collected (and pruned) state. It is called only
// when the consumer's released manifest is unavailable, so the rows themselves
// are not regenerated; we just keep the key->ID pins alive. Keys are matched
// against the consumer's own naming policy so unrelated (authored) rows that were
// legitimately removed are not resurrected.
func preserveAutogenConsumerLockEntries(collected []nativeCollectedDataset, consumer project.AutogenConsumerConfig) ([]nativeCollectedDataset, error) {
matches := autogenConsumerManagedLockKeyMatcher(consumer)
if matches == nil {
// No managed-key matcher for this mode: nothing safe to preserve.
return collected, nil
}
result := append([]nativeCollectedDataset(nil), collected...)
for i, dataset := range result {
if !autogenConsumerTargetsDataset(dataset, consumer) {
continue
}
if strings.TrimSpace(dataset.Dataset.LockPath) == "" {
continue
}
onDisk, err := loadLockfile(dataset.Dataset.LockPath)
if err != nil {
// No readable lockfile means there is nothing pinned to preserve.
continue
}
lockData := dataset.LockData
if lockData == nil {
lockData = map[string]int{}
}
restored := 0
for key, rowID := range onDisk {
if _, present := lockData[key]; present {
continue
}
if !matches(key) {
continue
}
lockData[key] = rowID
restored++
}
if restored == 0 {
continue
}
result[i].LockData = lockData
result[i].LockModified = !lockDataEqual(onDisk, lockData)
}
return result, nil
}
// autogenConsumerManagedLockKeyMatcher returns a predicate identifying the lock
// keys an autogen consumer is responsible for, or nil for modes whose keys we
// cannot scope precisely (in which case preservation is skipped rather than
// risk resurrecting unrelated keys).
func autogenConsumerManagedLockKeyMatcher(consumer project.AutogenConsumerConfig) func(string) bool {
switch consumer.Mode {
case "accessory_visualeffects":
policy := resolveHeadVisualeffectsPolicy(consumer, nil)
delimiter := policy.Delimiter
if delimiter == "" {
delimiter = "/"
}
prefix := "visualeffects:"
groups := make(map[string]struct{}, len(policy.Groups))
for group, groupPolicy := range policy.Groups {
token := applyHeadVisualeffectCase(headVisualeffectGroupToken(group, groupPolicy, policy), policy)
if strings.TrimSpace(token) != "" {
groups[token] = struct{}{}
}
}
if len(groups) == 0 {
return nil
}
return func(key string) bool {
if !strings.HasPrefix(key, prefix) {
return false
}
rest := key[len(prefix):]
idx := strings.Index(rest, delimiter)
if idx <= 0 {
return false
}
_, ok := groups[rest[:idx]]
return ok
}
default:
return nil
}
}
func autogenConsumerTargetsDataset(dataset nativeCollectedDataset, consumer project.AutogenConsumerConfig) bool {
switch consumer.Mode {
case "parts_rows":
return isAutogenEligiblePartsDataset(dataset)
case "accessory_visualeffects":
return dataset.Dataset.Name == "visualeffects"
case "cachedmodels_rows":
return dataset.Dataset.Name == "cachedmodels"
default:
return false return false
} }
message := err.Error()
return strings.Contains(message, "HTTP 404") ||
strings.Contains(message, "not found") ||
strings.Contains(message, "entries is empty")
} }
func resolveAutogenConsumerManifest(p *project.Project, consumer project.AutogenConsumerConfig, progress func(string)) (*autogenManifest, error) { func resolveAutogenConsumerManifest(p *project.Project, consumer project.AutogenConsumerConfig, progress func(string)) (*autogenManifest, error) {
@@ -366,14 +472,14 @@ func resolveReleasedAutogenManifest(p *project.Project, consumer project.Autogen
spec, err := deriveSowAssetsRepoSpec(p) spec, err := deriveSowAssetsRepoSpec(p)
if err != nil { if err != nil {
return nil, err return nil, unavailableAutogenManifest(err)
} }
manifestURL, err := resolveAutogenManifestAssetURL(spec, consumer.Manifest.ReleaseTag, consumer.Manifest.AssetName) manifestURL, err := resolveAutogenManifestAssetURL(spec, consumer.Manifest.ReleaseTag, consumer.Manifest.AssetName)
if err != nil { if err != nil {
if consumer.Mode == "parts_rows" { if consumer.Mode == "parts_rows" {
return nil, fmt.Errorf("%s", strings.Replace(err.Error(), "autogen manifest", "parts manifest", 1)) return nil, unavailableAutogenManifest(fmt.Errorf("%s", strings.Replace(err.Error(), "autogen manifest", "parts manifest", 1)))
} }
return nil, err return nil, unavailableAutogenManifest(err)
} }
if progress != nil { if progress != nil {
progress(fmt.Sprintf("Fetching released autogen manifest from %s...", manifestURL)) progress(fmt.Sprintf("Fetching released autogen manifest from %s...", manifestURL))
@@ -381,9 +487,9 @@ func resolveReleasedAutogenManifest(p *project.Project, consumer project.Autogen
manifest, err := fetchAutogenManifest(manifestURL) manifest, err := fetchAutogenManifest(manifestURL)
if err != nil { if err != nil {
if consumer.Mode == "parts_rows" { if consumer.Mode == "parts_rows" {
return nil, fmt.Errorf("%s", strings.Replace(err.Error(), "autogen manifest", "parts manifest", 1)) return nil, unavailableAutogenManifest(fmt.Errorf("%s", strings.Replace(err.Error(), "autogen manifest", "parts manifest", 1)))
} }
return nil, err return nil, unavailableAutogenManifest(err)
} }
if manifest.ID != "" && manifest.ID != consumer.Producer { if manifest.ID != "" && manifest.ID != consumer.Producer {
return nil, fmt.Errorf("autogen manifest %s declared id %q, expected %q", consumer.Manifest.AssetName, manifest.ID, consumer.Producer) return nil, fmt.Errorf("autogen manifest %s declared id %q, expected %q", consumer.Manifest.AssetName, manifest.ID, consumer.Producer)
+109
View File
@@ -0,0 +1,109 @@
package topdata
import (
"net/http"
"net/http/httptest"
"path/filepath"
"testing"
"git.westgate.pw/ShadowsOverWestgate/sow-tools/internal/project"
)
// When an optional autogen consumer's released manifest cannot be fetched, the
// build must fail open: it keeps the consumer's already-pinned lock entries
// (so their IDs are not freed and reshuffled later) and generates no new rows.
func TestApplyAutogenConsumersPreservesLockEntriesWhenManifestUnavailable(t *testing.T) {
root := testProjectRoot(t)
// 404 for every request → released manifest is unavailable (not empty).
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
http.NotFound(w, r)
}))
t.Cleanup(srv.Close)
t.Setenv("SOW_ASSETS_SERVER_URL", srv.URL)
t.Setenv("SOW_ASSETS_REPO", "ShadowsOverWestgate/sow-assets")
p := testProject(root)
p.Config.Autogen.Consumers = []project.AutogenConsumerConfig{
{
ID: "accessory_visualeffects",
Producer: "accessory_visualeffects",
Dataset: "visualeffects",
Mode: "accessory_visualeffects",
Optional: true,
Root: "vfxs",
Include: []string{"head_accessories/**/*.mdl"},
Derive: project.AutogenDeriveConfig{Kind: "model_stem", GroupFrom: "first_path_segment"},
Manifest: project.AutogenManifestConfig{ReleaseTag: "head-vfx-manifest-current", AssetName: "sow-accessory-vfx-manifest.json", CacheName: "sow-accessory-vfx-manifest.json"},
},
}
// On-disk lock pins an autogen-owned accessory key plus an authored key.
lockPath := filepath.Join(root, "visualeffects-lock.json")
writeFile(t, lockPath, `{"visualeffects:existing":17,"visualeffects:head_accessories/hat/bandana":10101}`+"\n")
// Simulate post-prune collected state: the accessory key has already been
// dropped from in-memory LockData (as pruneLockDataToActiveRows would do).
collected := []nativeCollectedDataset{
{
Dataset: nativeDataset{Name: "visualeffects", Kind: nativeDatasetBase, LockPath: lockPath},
Columns: []string{"Label"},
Rows: []map[string]any{{"id": 17, "key": "visualeffects:existing"}},
LockData: map[string]int{"visualeffects:existing": 17},
},
}
got, err := applyAutogenConsumers(p, collected, nil)
if err != nil {
t.Fatalf("expected fail-open build, got error: %v", err)
}
if len(got) != 1 {
t.Fatalf("expected one dataset, got %d", len(got))
}
// The pinned accessory id must survive with its exact value (no jumble).
if id, ok := got[0].LockData["visualeffects:head_accessories/hat/bandana"]; !ok || id != 10101 {
t.Fatalf("expected preserved accessory lock id 10101, got %v (present=%v) lock=%#v", id, ok, got[0].LockData)
}
// The authored key is untouched.
if id, ok := got[0].LockData["visualeffects:existing"]; !ok || id != 17 {
t.Fatalf("expected authored key retained at 17, got %#v", got[0].LockData)
}
// No row is generated for the unavailable accessory entry.
for _, row := range got[0].Rows {
if key, _ := row["key"].(string); key == "visualeffects:head_accessories/hat/bandana" {
t.Fatalf("expected no generated row while manifest unavailable, got %#v", row)
}
}
}
// A key that is NOT owned by the consumer (an authored row legitimately removed)
// must not be resurrected by the preservation path.
func TestPreserveAutogenConsumerLockEntriesIgnoresUnownedKeys(t *testing.T) {
root := testProjectRoot(t)
lockPath := filepath.Join(root, "visualeffects-lock.json")
writeFile(t, lockPath, `{"visualeffects:retired_authored":3,"visualeffects:head_accessories/hat/bandana":10101}`+"\n")
consumer := project.AutogenConsumerConfig{
ID: "accessory_visualeffects",
Dataset: "visualeffects",
Mode: "accessory_visualeffects",
}
collected := []nativeCollectedDataset{
{
Dataset: nativeDataset{Name: "visualeffects", Kind: nativeDatasetBase, LockPath: lockPath},
LockData: map[string]int{},
},
}
got, err := preserveAutogenConsumerLockEntries(collected, consumer)
if err != nil {
t.Fatalf("preserveAutogenConsumerLockEntries failed: %v", err)
}
if _, ok := got[0].LockData["visualeffects:head_accessories/hat/bandana"]; !ok {
t.Fatalf("expected owned accessory key preserved, got %#v", got[0].LockData)
}
if _, ok := got[0].LockData["visualeffects:retired_authored"]; ok {
t.Fatalf("expected unowned authored key NOT resurrected, got %#v", got[0].LockData)
}
}
+10 -6
View File
@@ -21,16 +21,20 @@ type damagetypesRegistry struct {
Types []map[string]any Types []map[string]any
} }
func collectGeneratedRegistryDatasets(dataDir string) ([]nativeCollectedDataset, error) { // collectGeneratedRegistryDatasets projects the registry datasets. When
itempropsDatasets, err := collectItempropsRegistryDatasets(dataDir) // persistLocks is true (the build pipeline) newly allocated ids are written
// back to the source lockfiles; when false (validation, discovery, queries)
// collection is read-only and never touches the source tree.
func collectGeneratedRegistryDatasets(dataDir string, persistLocks bool) ([]nativeCollectedDataset, error) {
itempropsDatasets, err := collectItempropsRegistryDatasets(dataDir, persistLocks)
if err != nil { if err != nil {
return nil, err return nil, err
} }
damagetypeDatasets, err := collectDamagetypesRegistryDatasets(dataDir) damagetypeDatasets, err := collectDamagetypesRegistryDatasets(dataDir, persistLocks)
if err != nil { if err != nil {
return nil, err return nil, err
} }
racialtypesDatasets, err := collectRacialtypesRegistryDatasets(dataDir) racialtypesDatasets, err := collectRacialtypesRegistryDatasets(dataDir, persistLocks)
if err != nil { if err != nil {
return nil, err return nil, err
} }
@@ -70,7 +74,7 @@ func loadDamagetypesRegistry(dataDir string) (*damagetypesRegistry, error) {
}, nil }, nil
} }
func collectDamagetypesRegistryDatasets(dataDir string) ([]nativeCollectedDataset, error) { func collectDamagetypesRegistryDatasets(dataDir string, persistLocks bool) ([]nativeCollectedDataset, error) {
registry, err := loadDamagetypesRegistry(dataDir) registry, err := loadDamagetypesRegistry(dataDir)
if err != nil { if err != nil {
return nil, err return nil, err
@@ -83,7 +87,7 @@ func collectDamagetypesRegistryDatasets(dataDir string) ([]nativeCollectedDatase
if err != nil { if err != nil {
return nil, err return nil, err
} }
if lockModified { if lockModified && persistLocks {
if err := saveLockfile(registry.LockPath, registry.LockData); err != nil { if err := saveLockfile(registry.LockPath, registry.LockData); err != nil {
return nil, err return nil, err
} }
+136 -11
View File
@@ -203,8 +203,23 @@ func editorConfigPatternMatches(pattern, relativePath string) (bool, error) {
} }
func editorConfigGlobRegexp(pattern string) (string, error) { func editorConfigGlobRegexp(pattern string) (string, error) {
body, err := translateEditorConfigGlobSegment(pattern)
if err != nil {
return "", err
}
expression := "^" + body + "$"
if _, err := regexp.Compile(expression); err != nil {
return "", err
}
return expression, nil
}
// translateEditorConfigGlobSegment converts an editorconfig glob into a regular
// expression body. In addition to *, **, and ?, it implements brace expansion:
// comma lists ({a,b,c}) become alternations and numeric ranges ({n..m}) expand
// to the integers in the range, matching the editorconfig specification.
func translateEditorConfigGlobSegment(pattern string) (string, error) {
var builder strings.Builder var builder strings.Builder
builder.WriteByte('^')
for index := 0; index < len(pattern); { for index := 0; index < len(pattern); {
char := pattern[index] char := pattern[index]
switch char { switch char {
@@ -220,22 +235,132 @@ func editorConfigGlobRegexp(pattern string) (string, error) {
continue continue
} }
builder.WriteString("[^/]*") builder.WriteString("[^/]*")
index++
case '?': case '?':
builder.WriteString("[^/]") builder.WriteString("[^/]")
case '.', '+', '(', ')', '|', '^', '$', '{', '}', '[', ']', '\\':
builder.WriteByte('\\')
builder.WriteByte(char)
default:
builder.WriteByte(char)
}
index++ index++
case '{':
end := matchingEditorConfigBrace(pattern, index)
if end < 0 {
// An unbalanced brace is matched literally.
builder.WriteString("\\{")
index++
continue
} }
builder.WriteByte('$') expansion, err := expandEditorConfigBrace(pattern[index+1 : end])
expression := builder.String() if err != nil {
if _, err := regexp.Compile(expression); err != nil {
return "", err return "", err
} }
return expression, nil builder.WriteString(expansion)
index = end + 1
case '.', '+', '(', ')', '|', '^', '$', '}', '[', ']', '\\':
builder.WriteByte('\\')
builder.WriteByte(char)
index++
default:
builder.WriteByte(char)
index++
}
}
return builder.String(), nil
}
// matchingEditorConfigBrace returns the index of the '}' that closes the '{' at
// open, accounting for nesting, or -1 when the braces are unbalanced.
func matchingEditorConfigBrace(pattern string, open int) int {
depth := 0
for index := open; index < len(pattern); index++ {
switch pattern[index] {
case '{':
depth++
case '}':
depth--
if depth == 0 {
return index
}
}
}
return -1
}
// expandEditorConfigBrace turns the body of a brace expression into a regex
// fragment. A comma list becomes an alternation; a lone "n..m" body becomes a
// numeric range. A body with neither comma nor range is treated as literal
// braces, matching the editorconfig reference implementation.
func expandEditorConfigBrace(inner string) (string, error) {
options := splitTopLevelBraceCommas(inner)
if len(options) == 1 {
if rangeExpr, ok, err := editorConfigNumericRange(inner); err != nil {
return "", err
} else if ok {
return rangeExpr, nil
}
segment, err := translateEditorConfigGlobSegment(inner)
if err != nil {
return "", err
}
return "\\{" + segment + "\\}", nil
}
parts := make([]string, 0, len(options))
for _, option := range options {
segment, err := translateEditorConfigGlobSegment(option)
if err != nil {
return "", err
}
parts = append(parts, segment)
}
return "(?:" + strings.Join(parts, "|") + ")", nil
}
// splitTopLevelBraceCommas splits a brace body on commas that are not nested
// inside an inner brace group.
func splitTopLevelBraceCommas(inner string) []string {
parts := []string{}
depth := 0
start := 0
for index := 0; index < len(inner); index++ {
switch inner[index] {
case '{':
depth++
case '}':
depth--
case ',':
if depth == 0 {
parts = append(parts, inner[start:index])
start = index + 1
}
}
}
return append(parts, inner[start:])
}
// editorConfigNumericRange expands "n..m" into an alternation of the integers
// in [n, m]. It returns ok=false when the body is not a numeric range.
func editorConfigNumericRange(inner string) (string, bool, error) {
separator := strings.Index(inner, "..")
if separator < 0 {
return "", false, nil
}
low, err := strconv.Atoi(strings.TrimSpace(inner[:separator]))
if err != nil {
return "", false, nil
}
high, err := strconv.Atoi(strings.TrimSpace(inner[separator+2:]))
if err != nil {
return "", false, nil
}
if low > high {
low, high = high, low
}
const maxRangeSize = 65536
if high-low >= maxRangeSize {
return "", false, fmt.Errorf("editorconfig numeric range {%s} is too large", inner)
}
parts := make([]string, 0, high-low+1)
for value := low; value <= high; value++ {
parts = append(parts, strconv.Itoa(value))
}
return "(?:" + strings.Join(parts, "|") + ")", true, nil
} }
func pathBase(path string) string { func pathBase(path string) string {
@@ -0,0 +1,56 @@
package topdata
import (
"os"
"path/filepath"
"testing"
)
func TestEditorConfigPatternMatchesBraceExpansion(t *testing.T) {
cases := []struct {
pattern string
path string
want bool
}{
{"*.{json,jsonc}", "lock.json", true},
{"*.{json,jsonc}", "types.jsonc", true},
{"*.{json,jsonc}", "notes.md", false},
{"*.{js,ts,tsx}", "main.tsx", true},
{"{foo,bar}.txt", "bar.txt", true},
{"{foo,bar}.txt", "baz.txt", false},
{"page{1..3}.json", "page2.json", true},
{"page{1..3}.json", "page4.json", false},
// No comma and not a range: editorconfig treats braces literally.
{"{single}.txt", "{single}.txt", true},
{"{single}.txt", "single.txt", false},
}
for _, c := range cases {
got, err := editorConfigPatternMatches(c.pattern, c.path)
if err != nil {
t.Fatalf("pattern %q path %q: %v", c.pattern, c.path, err)
}
if got != c.want {
t.Errorf("editorConfigPatternMatches(%q, %q) = %v, want %v", c.pattern, c.path, got, c.want)
}
}
}
func TestSaveLockfileHonorsBraceGlobIndentSize(t *testing.T) {
root := t.TempDir()
writeFile(t, filepath.Join(root, ".editorconfig"), "root = true\n\n[*]\nindent_style = space\nindent_size = 2\n\n[*.{json,jsonc}]\nindent_size = 4\n")
lockPath := filepath.Join(root, "data", "damagetypes", "registry", "lock.json")
mkdirAll(t, filepath.Dir(lockPath))
writeFile(t, lockPath, "{\n \"damagetype:acid\": 4\n}\n")
if err := saveLockfile(lockPath, map[string]int{"damagetype:acid": 4}); err != nil {
t.Fatalf("saveLockfile: %v", err)
}
got, err := os.ReadFile(lockPath)
if err != nil {
t.Fatalf("read lockfile: %v", err)
}
want := "{\n \"damagetype:acid\": 4\n}\n"
if string(got) != want {
t.Fatalf("expected 4-space indent resolved from [*.{json,jsonc}]; got:\n%q", string(got))
}
}
+2 -2
View File
@@ -166,7 +166,7 @@ func loadRegistryRows(path string) ([]map[string]any, error) {
return rows, nil return rows, nil
} }
func collectItempropsRegistryDatasets(dataDir string) ([]nativeCollectedDataset, error) { func collectItempropsRegistryDatasets(dataDir string, persistLocks bool) ([]nativeCollectedDataset, error) {
registry, err := loadItempropsRegistry(dataDir) registry, err := loadItempropsRegistry(dataDir)
if err != nil { if err != nil {
return nil, err return nil, err
@@ -188,7 +188,7 @@ func collectItempropsRegistryDatasets(dataDir string) ([]nativeCollectedDataset,
return nil, err return nil, err
} }
lockModified = lockModified || costModified || paramModified lockModified = lockModified || costModified || paramModified
if lockModified { if lockModified && persistLocks {
if err := saveLockfile(registry.LockPath, registry.LockData); err != nil { if err := saveLockfile(registry.LockPath, registry.LockData); err != nil {
return nil, err return nil, err
} }
+2 -2
View File
@@ -398,7 +398,7 @@ func buildNativeUnchecked(p *project.Project, opts NativeBuildOptions, progress
datasets = applyTopDataValueEncodings(datasets, p.Config.TopData.ValueEncodings) datasets = applyTopDataValueEncodings(datasets, p.Config.TopData.ValueEncodings)
datasets = applyTopDataValueDefaults(datasets, p.Config.TopData.ValueDefaults) datasets = applyTopDataValueDefaults(datasets, p.Config.TopData.ValueDefaults)
datasets = applyTopDataRowGeneration(datasets, p.Config.TopData.RowGeneration) datasets = applyTopDataRowGeneration(datasets, p.Config.TopData.RowGeneration)
registryDatasets, err := collectGeneratedRegistryDatasets(dataDir) registryDatasets, err := collectGeneratedRegistryDatasets(dataDir, true)
if err != nil { if err != nil {
return BuildResult{}, err return BuildResult{}, err
} }
@@ -1073,7 +1073,7 @@ func discoverNativeOutputCatalog(dataDir string) (map[string]string, error) {
if err != nil { if err != nil {
return nil, err return nil, err
} }
registryDatasets, err := collectGeneratedRegistryDatasets(dataDir) registryDatasets, err := collectGeneratedRegistryDatasets(dataDir, false)
if err != nil { if err != nil {
return nil, err return nil, err
} }
+2 -2
View File
@@ -120,7 +120,7 @@ func loadRacialtypesRegistryRaceRows(dir string) ([]map[string]any, error) {
return rows, nil return rows, nil
} }
func collectRacialtypesRegistryDatasets(dataDir string) ([]nativeCollectedDataset, error) { func collectRacialtypesRegistryDatasets(dataDir string, persistLocks bool) ([]nativeCollectedDataset, error) {
registry, err := loadRacialtypesRegistry(dataDir) registry, err := loadRacialtypesRegistry(dataDir)
if err != nil { if err != nil {
return nil, err return nil, err
@@ -138,7 +138,7 @@ func collectRacialtypesRegistryDatasets(dataDir string) ([]nativeCollectedDatase
return nil, err return nil, err
} }
lockModified = lockModified || raceLockModified lockModified = lockModified || raceLockModified
if lockModified { if lockModified && persistLocks {
if err := saveLockfile(registry.LockPath, registry.LockData); err != nil { if err := saveLockfile(registry.LockPath, registry.LockData); err != nil {
return nil, err return nil, err
} }
@@ -0,0 +1,64 @@
package topdata
import (
"os"
"path/filepath"
"testing"
)
const damagetypesRegistryTestTypes = `{
"rows": [
{"id": 0, "key": "damagetype:bludgeoning", "label": "Bludgeoning", "group_label": "Physical", "damage_type_group": 0},
{"id": 1, "key": "damagetype:piercing", "label": "Piercing", "group_label": "Physical", "damage_type_group": 0}
]
}
`
// validate-topdata must never modify source lockfiles. Collection in read-only
// mode must leave the registry lock byte-identical even though the rows carry
// explicit ids that would otherwise mark the lock "modified".
func TestCollectGeneratedRegistryDatasetsReadOnlyDoesNotWriteLock(t *testing.T) {
dataDir := t.TempDir()
regDir := filepath.Join(dataDir, "damagetypes", "registry")
mkdirAll(t, regDir)
writeFile(t, filepath.Join(regDir, "types.json"), damagetypesRegistryTestTypes)
lockPath := filepath.Join(regDir, "lock.json")
original := "{\n \"damagetype:bludgeoning\": 0,\n \"damagetype:piercing\": 1\n}\n"
writeFile(t, lockPath, original)
if _, err := collectGeneratedRegistryDatasets(dataDir, false); err != nil {
t.Fatalf("read-only collection failed: %v", err)
}
got, err := os.ReadFile(lockPath)
if err != nil {
t.Fatalf("read lockfile: %v", err)
}
if string(got) != original {
t.Fatalf("read-only collection must not modify lockfile.\nwant: %q\ngot: %q", original, string(got))
}
}
// build-topdata still persists allocated ids to source lockfiles.
func TestCollectGeneratedRegistryDatasetsPersistWritesLock(t *testing.T) {
dataDir := t.TempDir()
regDir := filepath.Join(dataDir, "damagetypes", "registry")
mkdirAll(t, regDir)
writeFile(t, filepath.Join(regDir, "types.json"), damagetypesRegistryTestTypes)
lockPath := filepath.Join(regDir, "lock.json")
writeFile(t, lockPath, "{\n \"damagetype:bludgeoning\": 0\n}\n")
if _, err := collectGeneratedRegistryDatasets(dataDir, true); err != nil {
t.Fatalf("persisting collection failed: %v", err)
}
lockData, err := loadLockfile(lockPath)
if err != nil {
t.Fatalf("load lockfile: %v", err)
}
if lockData["damagetype:piercing"] != 1 {
t.Fatalf("persisting collection should record allocated id; got %#v", lockData)
}
}
+1 -1
View File
@@ -557,7 +557,7 @@ func expectedCompiled2DAOutputs(p *project.Project) (map[string]struct{}, error)
if err != nil { if err != nil {
return nil, err return nil, err
} }
registryDatasets, err := collectGeneratedRegistryDatasets(dataDir) registryDatasets, err := collectGeneratedRegistryDatasets(dataDir, false)
if err != nil { if err != nil {
return nil, err return nil, err
} }
+2 -2
View File
@@ -1320,7 +1320,7 @@ func validateNativeOutputCatalog(dataDir string, report *ValidationReport) {
}) })
return return
} }
registryDatasets, err := collectGeneratedRegistryDatasets(dataDir) registryDatasets, err := collectGeneratedRegistryDatasets(dataDir, false)
if err != nil { if err != nil {
report.Diagnostics = append(report.Diagnostics, Diagnostic{ report.Diagnostics = append(report.Diagnostics, Diagnostic{
Severity: SeverityError, Severity: SeverityError,
@@ -2307,7 +2307,7 @@ func validateNativeBuildability(p *project.Project, report *ValidationReport) {
}) })
return return
} }
registryDatasets, err := collectGeneratedRegistryDatasets(dataDir) registryDatasets, err := collectGeneratedRegistryDatasets(dataDir, false)
if err != nil { if err != nil {
report.Diagnostics = append(report.Diagnostics, Diagnostic{ report.Diagnostics = append(report.Diagnostics, Diagnostic{
Severity: SeverityError, Severity: SeverityError,
+1 -1
View File
@@ -748,7 +748,7 @@ func collectWikiRacialtypesDatasets(dataDir string, datasets []nativeDataset) ([
if len(out) > 0 { if len(out) > 0 {
return out, nil return out, nil
} }
return collectRacialtypesRegistryDatasets(dataDir) return collectRacialtypesRegistryDatasets(dataDir, false)
} }
func collectWikiRacialtypeStatuses(dataDir string) map[string]string { func collectWikiRacialtypeStatuses(dataDir string) map[string]string {
+5
View File
@@ -0,0 +1,5 @@
# Repos that carry copies of the canonical Crucible wrappers.
# One "owner/repo" per line. sync-wrappers.yml opens an update PR to each when
# wrappers/ changes. Add a repo here AND grant the sync bot write access to it.
ShadowsOverWestgate/sow-module
ShadowsOverWestgate/sow-topdata
+89
View File
@@ -0,0 +1,89 @@
#!/usr/bin/env pwsh
# crucible - bootstrap wrapper (PowerShell). Prefers a crucible already on PATH;
# otherwise downloads the latest released binary from Gitea into a per-version
# cache and execs it.
#
# CANONICAL SOURCE: sow-tools/wrappers/crucible.ps1. Do not hand-edit copies in
# consumer repos - kept in sync by sow-tools CI (see drift-check).
#
# Env: CRUCIBLE_HOME, CRUCIBLE_TOKEN, CRUCIBLE_GITEA, CRUCIBLE_REPO.
$ErrorActionPreference = 'Stop'
# 0. Prefer an existing crucible.
$existing = Get-Command crucible -ErrorAction SilentlyContinue
if ($existing) { & $existing.Source @args; exit $LASTEXITCODE }
$gitea = if ($env:CRUCIBLE_GITEA) { $env:CRUCIBLE_GITEA } else { 'https://git.westgate.pw' }
$repo = if ($env:CRUCIBLE_REPO) { $env:CRUCIBLE_REPO } else { 'ShadowsOverWestgate/sow-tools' }
# 1. OS/arch -> asset name (.exe on Windows).
$os = if ($IsWindows) { 'windows' } elseif ($IsMacOS) { 'darwin' } else { 'linux' }
$arch = switch ($env:PROCESSOR_ARCHITECTURE) {
'AMD64' { 'amd64' } 'ARM64' { 'arm64' } default { 'amd64' }
}
$ext = if ($os -eq 'windows') { '.exe' } else { '' }
$asset = "crucible-$os-$arch$ext"
# 2. Auth header (anon first, token fallback).
$token = $env:CRUCIBLE_TOKEN
$tokenFile = Join-Path $HOME '.config/crucible/token'
if (-not $token -and (Test-Path $tokenFile)) { $token = (Get-Content $tokenFile -Raw).Trim() }
function Invoke-Fetch($url, $outFile) {
try { Invoke-WebRequest -Uri $url -OutFile $outFile -UseBasicParsing; return $true } catch {}
if ($token) {
Invoke-WebRequest -Uri $url -OutFile $outFile -UseBasicParsing -Headers @{ Authorization = "token $token" }
return $true
}
return $false
}
# 3. Resolve latest tag.
$tag = $null
try {
$hdr = if ($token) { @{ Authorization = "token $token" } } else { @{} }
$rel = Invoke-RestMethod -Uri "$gitea/api/v1/repos/$repo/releases/latest" -Headers $hdr
$tag = $rel.tag_name
} catch {}
# 4. Cache dir.
$cacheRoot = if ($env:CRUCIBLE_HOME) { $env:CRUCIBLE_HOME }
elseif ($env:LOCALAPPDATA) { Join-Path $env:LOCALAPPDATA 'crucible' }
else { Join-Path $HOME '.cache/crucible' }
# 5. Offline fallback.
if (-not $tag) {
if (Test-Path $cacheRoot) {
$cached = Get-ChildItem $cacheRoot -Directory | Sort-Object Name | Select-Object -Last 1
$cachedBin = if ($cached) { Join-Path $cached.FullName "crucible$ext" } else { $null }
if ($cachedBin -and (Test-Path $cachedBin)) {
Write-Error "crucible: offline - using cached $($cached.Name)" -ErrorAction Continue
& $cachedBin @args; exit $LASTEXITCODE
}
}
Write-Error "crucible: cannot reach $gitea and no cached binary found (set CRUCIBLE_TOKEN if private)."
exit 1
}
$destDir = Join-Path $cacheRoot $tag
$bin = Join-Path $destDir "crucible$ext"
# 6. Download + verify on cache miss.
if (-not (Test-Path $bin)) {
New-Item -ItemType Directory -Force -Path $destDir | Out-Null
$base = "$gitea/$repo/releases/download/$tag"
$tmp = New-Item -ItemType Directory -Path (Join-Path ([System.IO.Path]::GetTempPath()) ([guid]::NewGuid()))
$tmpBin = Join-Path $tmp "crucible$ext"
Write-Host "crucible: fetching $asset $tag..."
if (-not (Invoke-Fetch "$base/$asset" $tmpBin)) { Write-Error "crucible: download failed for $asset"; exit 1 }
$sums = Join-Path $tmp 'SHA256SUMS'
if (Invoke-Fetch "$base/SHA256SUMS" $sums) {
$want = (Select-String -Path $sums -Pattern ([regex]::Escape($asset) + '$') | Select-Object -First 1).Line -split '\s+' | Select-Object -First 1
$got = (Get-FileHash $tmpBin -Algorithm SHA256).Hash.ToLower()
if ($want -and ($want.ToLower() -ne $got)) { Write-Error "crucible: checksum mismatch for $asset"; exit 1 }
}
Move-Item -Force $tmpBin $bin
Remove-Item -Recurse -Force $tmp
}
& $bin @args
exit $LASTEXITCODE
+107
View File
@@ -0,0 +1,107 @@
#!/usr/bin/env bash
# crucible — bootstrap wrapper (POSIX/bash). Prefers a crucible already on PATH
# (Nix devshell / installed); otherwise downloads the latest released binary
# from Gitea into a per-version cache and execs it.
#
# CANONICAL SOURCE: sow-tools/wrappers/crucible.sh. Do not hand-edit copies in
# consumer repos — they are kept in sync by sow-tools CI (see drift-check).
#
# Requires: bash, curl, and sha256sum (Linux) or shasum (macOS).
# Env: CRUCIBLE_HOME (cache dir), CRUCIBLE_TOKEN (private releases),
# CRUCIBLE_GITEA (default https://git.westgate.pw),
# CRUCIBLE_REPO (default ShadowsOverWestgate/sow-tools).
set -euo pipefail
# 0. Prefer an existing crucible (Nix users, or already installed).
if command -v crucible >/dev/null 2>&1; then
exec crucible "$@"
fi
gitea="${CRUCIBLE_GITEA:-https://git.westgate.pw}"
repo="${CRUCIBLE_REPO:-ShadowsOverWestgate/sow-tools}"
# 1. --repo-local materializes into <repo>/.crucible instead of the user cache.
repo_local=0
if [ "${1:-}" = "--repo-local" ]; then repo_local=1; shift; fi
# 2. OS/arch -> asset name.
os="$(uname -s)"; arch="$(uname -m)"
case "$os" in
Linux) os=linux ;;
Darwin) os=darwin ;;
*) echo "crucible: unsupported OS '$os' — install ffmpeg-free crucible manually" >&2; exit 1 ;;
esac
case "$arch" in
x86_64|amd64) arch=amd64 ;;
aarch64|arm64) arch=arm64 ;;
*) echo "crucible: unsupported arch '$arch'" >&2; exit 1 ;;
esac
asset="crucible-${os}-${arch}"
# 3. Auth: anonymous first, token fallback.
token="${CRUCIBLE_TOKEN:-}"
if [ -z "$token" ] && [ -f "$HOME/.config/crucible/token" ]; then
token="$(cat "$HOME/.config/crucible/token")"
fi
fetch() { # fetch URL DEST ; tries anon then token
if curl -fsSL "$1" -o "$2" 2>/dev/null; then return 0; fi
if [ -n "$token" ]; then curl -fsSL -H "Authorization: token $token" "$1" -o "$2"; return $?; fi
return 1
}
fetch_stdout() { # prints body; anon then token
if curl -fsSL "$1" 2>/dev/null; then return 0; fi
if [ -n "$token" ]; then curl -fsSL -H "Authorization: token $token" "$1"; return $?; fi
return 1
}
# 4. Resolve the latest release tag (no jq dependency).
api="${gitea}/api/v1/repos/${repo}/releases/latest"
tag="$(fetch_stdout "$api" 2>/dev/null | grep -o '"tag_name":"[^"]*"' | head -1 | cut -d'"' -f4 || true)"
# 5. Cache dir.
if [ "$repo_local" = 1 ]; then
cache_root="$(cd "$(dirname "$0")" && pwd)/.crucible"
else
cache_root="${CRUCIBLE_HOME:-${XDG_CACHE_HOME:-$HOME/.cache}/crucible}"
fi
# 6. Offline fallback: if tag lookup failed, use the newest cached version.
if [ -z "$tag" ]; then
if [ -d "$cache_root" ]; then
cached="$(ls -1 "$cache_root" 2>/dev/null | sort -V | tail -1 || true)"
if [ -n "$cached" ] && [ -x "$cache_root/$cached/crucible" ]; then
echo "crucible: offline — using cached $cached" >&2
exec "$cache_root/$cached/crucible" "$@"
fi
fi
echo "crucible: cannot reach $gitea and no cached binary found." >&2
echo " set CRUCIBLE_TOKEN if releases are private, or check the network." >&2
exit 1
fi
dest_dir="$cache_root/$tag"
bin="$dest_dir/crucible"
# 7. Download + verify on cache miss.
if [ ! -x "$bin" ]; then
mkdir -p "$dest_dir"
base="${gitea}/${repo}/releases/download/${tag}"
tmp="$(mktemp -d)"; trap 'rm -rf "$tmp"' EXIT
echo "crucible: fetching ${asset} ${tag}..." >&2
fetch "${base}/${asset}" "$tmp/crucible" || { echo "crucible: download failed for ${asset}" >&2; exit 1; }
if fetch "${base}/SHA256SUMS" "$tmp/SHA256SUMS"; then
want="$(grep " ${asset}\$" "$tmp/SHA256SUMS" | awk '{print $1}')"
if command -v sha256sum >/dev/null 2>&1; then
got="$(sha256sum "$tmp/crucible" | awk '{print $1}')"
else
got="$(shasum -a 256 "$tmp/crucible" | awk '{print $1}')"
fi
if [ -n "$want" ] && [ "$want" != "$got" ]; then
echo "crucible: checksum mismatch for ${asset} (want $want got $got)" >&2; exit 1
fi
fi
chmod +x "$tmp/crucible"
mv "$tmp/crucible" "$bin"
fi
exec "$bin" "$@"