Compare commits

...
3 Commits
Author SHA1 Message Date
archvillainette 16d5586587 Crucible: Fix Formatting Churn (#5)
test-image / build-image (push) Successful in 43s
test / test (push) Successful in 1m21s
build-binaries / build-binaries (push) Successful in 2m2s
build-image / publish (push) Successful in 13s
RC#1 — build wrote 2-space instead of .editorconfig's 4-space
- editorconfig_format.go: the glob translator escaped {/} as literals, so [*.{json,jsonc}]→indent_size=4 never matched any file and formatting fell back to [*]→2. Implemented real editorconfig brace expansion — {a,b,c} alternation and {n..m} numeric ranges. Nothing is hardcoded; saveLockfile already read .editorconfig, it just got the wrong section.

RC#2 — validate wrote a lockfile (it must be read-only)
- The three registry collectors persisted lockfiles as a side effect of collection; ValidateProject calls collection outside its snapshot guard, so the write leaked. Threaded a persistLocks bool through collectGeneratedRegistryDatasets and the damagetypes/itemprops/racialtypes collectors. Only buildNativeUnchecked (the real build allocator) passes true; validate, discovery, packaging queries, and wiki discovery pass false.

Tests added: editorconfig_glob_test.go (brace match + 4-space via brace glob), registry_readonly_test.go (read-only writes nothing; persist writes allocations).

Reviewed-on: #5
Co-authored-by: vickydotbat <vickydotbat@tutamail.com>
Co-committed-by: vickydotbat <vickydotbat@tutamail.com>
2026-06-17 07:57:54 +00:00
archvillainette b7c0f43064 Fix autogen to fail-open on missing asset manifests (#4)
test-image / build-image (push) Successful in 45s
test / test (push) Successful in 1m22s
build-binaries / build-binaries (push) Successful in 2m2s
build-image / publish (push) Successful in 13s
Reviewed-on: #4
Co-authored-by: vickydotbat <vickydotbat@tutamail.com>
Co-committed-by: vickydotbat <vickydotbat@tutamail.com>
2026-06-16 21:50:45 +00:00
archvillainette d45bd84bc6 feat: crucible wrapper sync (#3)
test / test (push) Successful in 1m21s
test-image / build-image (push) Successful in 44s
build-binaries / build-binaries (push) Successful in 2m3s
build-image / publish (push) Successful in 12s
Reviewed-on: #3
Co-authored-by: vickydotbat <vickydotbat@tutamail.com>
Co-committed-by: vickydotbat <vickydotbat@tutamail.com>
2026-06-16 15:38:58 +00:00
17 changed files with 614 additions and 60 deletions
+50
View File
@@ -0,0 +1,50 @@
# Auto-PR canonical wrapper updates to consumer repos when wrappers/ changes on
# main. Maintenance automation (not artifact publishing), so it is allowed on a
# main push under the D7 trigger standard. Requires WRAPPER_SYNC_TOKEN: a bot
# user's token with content+PR write to the consumer repos (never committed).
name: sync-wrappers
on:
push:
branches: [main]
paths:
- 'wrappers/crucible.sh'
- 'wrappers/crucible.ps1'
jobs:
sync:
runs-on: nix-docker
steps:
- uses: actions/checkout@v4
- name: Open sync PRs to consumers
env:
TOKEN: ${{ secrets.WRAPPER_SYNC_TOKEN }}
SERVER: ${{ github.server_url }}
SRC_SHA: ${{ github.sha }}
run: |
nix develop --command bash -c '
set -euo pipefail
host="$(echo "$SERVER" | sed -E "s#https?://##")"
branch="chore/sync-wrappers-$(echo "$SRC_SHA" | cut -c1-12)"
grep -vE "^\s*#|^\s*$" wrappers/consumers.txt | while read -r target; do
echo "== syncing $target =="
work="$(mktemp -d)"
git clone "https://oauth2:${TOKEN}@${host}/${target}.git" "$work"
cp wrappers/crucible.sh wrappers/crucible.ps1 "$work"/
( cd "$work"
git config user.name "crucible-sync-bot"
git config user.email "bot@westgate.pw"
if git diff --quiet; then echo "no changes for $target"; exit 0; fi
git checkout -b "$branch"
git add crucible.sh crucible.ps1
git commit -m "chore: sync crucible wrappers from sow-tools@${SRC_SHA}"
git push -f origin "$branch"
curl -fsS -X POST \
-H "Authorization: token ${TOKEN}" -H "Content-Type: application/json" \
"${SERVER}/api/v1/repos/${target}/pulls" \
-d "{\"head\":\"${branch}\",\"base\":\"main\",\"title\":\"chore: sync crucible wrappers from sow-tools\"}" || true
)
rm -rf "$work"
done
'
+35 -5
View File
@@ -41,6 +41,30 @@ See [`docs/migration-from-nwn-tool.md`](docs/migration-from-nwn-tool.md) for wha
was done and what remains (the consumer `--manifest/--source/--out` flag contract
is the open Phase-6 item).
## Quick start (no Nix)
Teammates without Nix don't build anything — they run the bootstrap wrapper,
which downloads the latest released `crucible` for your OS and runs it:
```bash
./crucible # interactive menu (pick a command)
./crucible module build
./crucible topdata validate-topdata
```
Windows (PowerShell):
```powershell
.\crucible.ps1 module build
```
The binary is cached under `~/.cache/crucible/<version>/` (`%LOCALAPPDATA%\crucible`
on Windows); `--repo-local` caches inside the repo instead. Private releases:
set `CRUCIBLE_TOKEN` or write the token to `~/.config/crucible/token`.
Only the **music** builder needs `ffmpeg`; everything else has zero dependencies.
If you run a music command without it, Crucible prints a per-OS install hint.
## Develop
Self-contained (D8) — a host with only Nix can run everything:
@@ -58,12 +82,18 @@ This retires the old habit of checking in `nwn-tool` / `sow-toolkit`.
## CI
PR-first (D7): every check runs on pull requests and on push to `main`.
PR-first (D7): checks run on pull requests and on push to `main`; the only
publish event is a `v*` tag (see `sow-docs/runbooks/ci-trigger-standard.md`).
- `test.yml` — vet, test, shellcheck, yamllint, binary smoke.
- `build-image.yml` — build `registry.westgate.pw/deployment/crucible:<sha>`; publish
only on `main`. PRs build but never push. No mutable tags.
- `release.yml` — tag-gated binary bundles; image re-tag is wired in Phase 6.
- `test.yml` — vet, test, shellcheck, yamllint, binary smoke (PR + main).
- `test-image.yml` — build the OCI image to prove it compiles (PR + main, no push).
- `build-binaries.yml` — cross-build all targets (PR + main); on a `v*` tag, upload
the binaries, `SHA256SUMS`, and the wrappers to the Gitea release.
- `build-image.yml` — on a `v*` tag, build and publish
`registry.westgate.pw/deployment/crucible:<sha>`.
- `publish-image.yml` — manual `workflow_dispatch` break-glass republish.
- `sync-wrappers.yml` — on a `main` push that touches `wrappers/`, auto-PR the
canonical wrappers to the consumer repos in `wrappers/consumers.txt`.
## Consumers
+6 -3
View File
@@ -24,14 +24,17 @@ quoting stays correct.
## In CI (preferred)
Run the consumer job inside the pinned image and the binaries are on `PATH`:
**OPERATOR NOTE: Stop. Do not do it this way.**
**Use the pinned `prod.yml` version. That's what it's there for.**
~~Run the consumer job inside the pinned image and the binaries are on `PATH`:~~
```yaml
container:
image: registry.westgate.pw/deployment/crucible:<sha> # pinned, immutable
image: registry.westgate.pw/deployment/crucible:<sha> # pinned, immutable
```
No host install, no `$HOME` layout, no developer machine assumptions.
~~No host install, no `$HOME` layout, no developer machine assumptions.~~
## `NWN_ROOT` rule
+4 -2
View File
@@ -249,8 +249,10 @@ func usage(w io.Writer) {
fmt.Fprintf(w, " list list builders (one per line: name<TAB>bin<TAB>summary)\n")
fmt.Fprintf(w, " config [args] inspect/validate effective configuration\n")
fmt.Fprintf(w, " changelog [args] generate the release changelog\n")
fmt.Fprintf(w, "\nNWN_ROOT must be passed explicitly (env or flag); crucible never guesses\n")
fmt.Fprintf(w, "from $HOME. See docs/consumer-contract.md.\n")
fmt.Fprintf(w, "\nBuilders read all inputs from the project tree (nwn-tool.yaml + data/); no\n")
fmt.Fprintf(w, "NWN install is required. If a builder ever needs an NWN root, it must be\n")
fmt.Fprintf(w, "passed explicitly via NWN_ROOT; crucible never guesses from $HOME. See\n")
fmt.Fprintf(w, "docs/consumer-contract.md.\n")
}
func list(w io.Writer) {
+132 -26
View File
@@ -3,6 +3,7 @@ package topdata
import (
"bytes"
"encoding/json"
"errors"
"fmt"
"os"
"path/filepath"
@@ -16,6 +17,19 @@ import (
const autogenManifestCacheMaxAge = time.Hour
// errAutogenManifestUnavailable marks a released autogen manifest that could not
// be located or fetched (network failure, missing release/asset, empty payload,
// or an undeterminable sow-assets repo). It is deliberately distinct from an
// asset that was found but reports a row removed: for an optional consumer this
// sentinel triggers a fail-open path that *preserves* the consumer's existing
// pinned lock entries instead of pruning them, so StrRef/row IDs are not
// reshuffled while the asset source is merely temporarily out of reach.
var errAutogenManifestUnavailable = errors.New("autogen manifest unavailable")
func unavailableAutogenManifest(err error) error {
return fmt.Errorf("%w: %w", errAutogenManifestUnavailable, err)
}
type autogenManifest struct {
ID string `json:"id"`
Repo string `json:"repo"`
@@ -61,9 +75,18 @@ func applyAutogenConsumers(p *project.Project, collected []nativeCollectedDatase
}
manifest, err := resolveAutogenConsumerManifest(p, consumer, progress)
if err != nil {
if consumer.Optional && isIgnorableOptionalAutogenError(err) {
if consumer.Optional && errors.Is(err, errAutogenManifestUnavailable) {
// Fail open: the asset manifest is merely unreachable, not
// authoritatively empty. Build without its rows, but keep the
// consumer's already-pinned lock entries so their IDs are not
// freed and reshuffled on a later build once the asset returns.
preserved, perr := preserveAutogenConsumerLockEntries(result, consumer)
if perr != nil {
return nil, perr
}
result = preserved
if progress != nil {
progress(fmt.Sprintf("Skipping optional autogen consumer %s: %v", consumer.ID, err))
progress(fmt.Sprintf("Autogen consumer %s: released manifest unavailable (%v); preserving existing lock entries, generating no new rows", consumer.ID, err))
}
continue
}
@@ -90,32 +113,115 @@ func applyAutogenConsumers(p *project.Project, collected []nativeCollectedDatase
func autogenConsumerTargetsCollectedDataset(collected []nativeCollectedDataset, consumer project.AutogenConsumerConfig) bool {
for _, dataset := range collected {
switch consumer.Mode {
case "parts_rows":
if isAutogenEligiblePartsDataset(dataset) {
return true
}
case "accessory_visualeffects":
if dataset.Dataset.Name == "visualeffects" {
return true
}
case "cachedmodels_rows":
if dataset.Dataset.Name == "cachedmodels" {
return true
}
if autogenConsumerTargetsDataset(dataset, consumer) {
return true
}
}
return false
}
func isIgnorableOptionalAutogenError(err error) bool {
if err == nil {
// preserveAutogenConsumerLockEntries restores the lock keys that an autogen
// consumer owns from the on-disk lockfile back into the in-memory dataset,
// reusing this build's already-collected (and pruned) state. It is called only
// when the consumer's released manifest is unavailable, so the rows themselves
// are not regenerated; we just keep the key->ID pins alive. Keys are matched
// against the consumer's own naming policy so unrelated (authored) rows that were
// legitimately removed are not resurrected.
func preserveAutogenConsumerLockEntries(collected []nativeCollectedDataset, consumer project.AutogenConsumerConfig) ([]nativeCollectedDataset, error) {
matches := autogenConsumerManagedLockKeyMatcher(consumer)
if matches == nil {
// No managed-key matcher for this mode: nothing safe to preserve.
return collected, nil
}
result := append([]nativeCollectedDataset(nil), collected...)
for i, dataset := range result {
if !autogenConsumerTargetsDataset(dataset, consumer) {
continue
}
if strings.TrimSpace(dataset.Dataset.LockPath) == "" {
continue
}
onDisk, err := loadLockfile(dataset.Dataset.LockPath)
if err != nil {
// No readable lockfile means there is nothing pinned to preserve.
continue
}
lockData := dataset.LockData
if lockData == nil {
lockData = map[string]int{}
}
restored := 0
for key, rowID := range onDisk {
if _, present := lockData[key]; present {
continue
}
if !matches(key) {
continue
}
lockData[key] = rowID
restored++
}
if restored == 0 {
continue
}
result[i].LockData = lockData
result[i].LockModified = !lockDataEqual(onDisk, lockData)
}
return result, nil
}
// autogenConsumerManagedLockKeyMatcher returns a predicate identifying the lock
// keys an autogen consumer is responsible for, or nil for modes whose keys we
// cannot scope precisely (in which case preservation is skipped rather than
// risk resurrecting unrelated keys).
func autogenConsumerManagedLockKeyMatcher(consumer project.AutogenConsumerConfig) func(string) bool {
switch consumer.Mode {
case "accessory_visualeffects":
policy := resolveHeadVisualeffectsPolicy(consumer, nil)
delimiter := policy.Delimiter
if delimiter == "" {
delimiter = "/"
}
prefix := "visualeffects:"
groups := make(map[string]struct{}, len(policy.Groups))
for group, groupPolicy := range policy.Groups {
token := applyHeadVisualeffectCase(headVisualeffectGroupToken(group, groupPolicy, policy), policy)
if strings.TrimSpace(token) != "" {
groups[token] = struct{}{}
}
}
if len(groups) == 0 {
return nil
}
return func(key string) bool {
if !strings.HasPrefix(key, prefix) {
return false
}
rest := key[len(prefix):]
idx := strings.Index(rest, delimiter)
if idx <= 0 {
return false
}
_, ok := groups[rest[:idx]]
return ok
}
default:
return nil
}
}
func autogenConsumerTargetsDataset(dataset nativeCollectedDataset, consumer project.AutogenConsumerConfig) bool {
switch consumer.Mode {
case "parts_rows":
return isAutogenEligiblePartsDataset(dataset)
case "accessory_visualeffects":
return dataset.Dataset.Name == "visualeffects"
case "cachedmodels_rows":
return dataset.Dataset.Name == "cachedmodels"
default:
return false
}
message := err.Error()
return strings.Contains(message, "HTTP 404") ||
strings.Contains(message, "not found") ||
strings.Contains(message, "entries is empty")
}
func resolveAutogenConsumerManifest(p *project.Project, consumer project.AutogenConsumerConfig, progress func(string)) (*autogenManifest, error) {
@@ -366,14 +472,14 @@ func resolveReleasedAutogenManifest(p *project.Project, consumer project.Autogen
spec, err := deriveSowAssetsRepoSpec(p)
if err != nil {
return nil, err
return nil, unavailableAutogenManifest(err)
}
manifestURL, err := resolveAutogenManifestAssetURL(spec, consumer.Manifest.ReleaseTag, consumer.Manifest.AssetName)
if err != nil {
if consumer.Mode == "parts_rows" {
return nil, fmt.Errorf("%s", strings.Replace(err.Error(), "autogen manifest", "parts manifest", 1))
return nil, unavailableAutogenManifest(fmt.Errorf("%s", strings.Replace(err.Error(), "autogen manifest", "parts manifest", 1)))
}
return nil, err
return nil, unavailableAutogenManifest(err)
}
if progress != nil {
progress(fmt.Sprintf("Fetching released autogen manifest from %s...", manifestURL))
@@ -381,9 +487,9 @@ func resolveReleasedAutogenManifest(p *project.Project, consumer project.Autogen
manifest, err := fetchAutogenManifest(manifestURL)
if err != nil {
if consumer.Mode == "parts_rows" {
return nil, fmt.Errorf("%s", strings.Replace(err.Error(), "autogen manifest", "parts manifest", 1))
return nil, unavailableAutogenManifest(fmt.Errorf("%s", strings.Replace(err.Error(), "autogen manifest", "parts manifest", 1)))
}
return nil, err
return nil, unavailableAutogenManifest(err)
}
if manifest.ID != "" && manifest.ID != consumer.Producer {
return nil, fmt.Errorf("autogen manifest %s declared id %q, expected %q", consumer.Manifest.AssetName, manifest.ID, consumer.Producer)
+109
View File
@@ -0,0 +1,109 @@
package topdata
import (
"net/http"
"net/http/httptest"
"path/filepath"
"testing"
"git.westgate.pw/ShadowsOverWestgate/sow-tools/internal/project"
)
// When an optional autogen consumer's released manifest cannot be fetched, the
// build must fail open: it keeps the consumer's already-pinned lock entries
// (so their IDs are not freed and reshuffled later) and generates no new rows.
func TestApplyAutogenConsumersPreservesLockEntriesWhenManifestUnavailable(t *testing.T) {
root := testProjectRoot(t)
// 404 for every request → released manifest is unavailable (not empty).
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
http.NotFound(w, r)
}))
t.Cleanup(srv.Close)
t.Setenv("SOW_ASSETS_SERVER_URL", srv.URL)
t.Setenv("SOW_ASSETS_REPO", "ShadowsOverWestgate/sow-assets")
p := testProject(root)
p.Config.Autogen.Consumers = []project.AutogenConsumerConfig{
{
ID: "accessory_visualeffects",
Producer: "accessory_visualeffects",
Dataset: "visualeffects",
Mode: "accessory_visualeffects",
Optional: true,
Root: "vfxs",
Include: []string{"head_accessories/**/*.mdl"},
Derive: project.AutogenDeriveConfig{Kind: "model_stem", GroupFrom: "first_path_segment"},
Manifest: project.AutogenManifestConfig{ReleaseTag: "head-vfx-manifest-current", AssetName: "sow-accessory-vfx-manifest.json", CacheName: "sow-accessory-vfx-manifest.json"},
},
}
// On-disk lock pins an autogen-owned accessory key plus an authored key.
lockPath := filepath.Join(root, "visualeffects-lock.json")
writeFile(t, lockPath, `{"visualeffects:existing":17,"visualeffects:head_accessories/hat/bandana":10101}`+"\n")
// Simulate post-prune collected state: the accessory key has already been
// dropped from in-memory LockData (as pruneLockDataToActiveRows would do).
collected := []nativeCollectedDataset{
{
Dataset: nativeDataset{Name: "visualeffects", Kind: nativeDatasetBase, LockPath: lockPath},
Columns: []string{"Label"},
Rows: []map[string]any{{"id": 17, "key": "visualeffects:existing"}},
LockData: map[string]int{"visualeffects:existing": 17},
},
}
got, err := applyAutogenConsumers(p, collected, nil)
if err != nil {
t.Fatalf("expected fail-open build, got error: %v", err)
}
if len(got) != 1 {
t.Fatalf("expected one dataset, got %d", len(got))
}
// The pinned accessory id must survive with its exact value (no jumble).
if id, ok := got[0].LockData["visualeffects:head_accessories/hat/bandana"]; !ok || id != 10101 {
t.Fatalf("expected preserved accessory lock id 10101, got %v (present=%v) lock=%#v", id, ok, got[0].LockData)
}
// The authored key is untouched.
if id, ok := got[0].LockData["visualeffects:existing"]; !ok || id != 17 {
t.Fatalf("expected authored key retained at 17, got %#v", got[0].LockData)
}
// No row is generated for the unavailable accessory entry.
for _, row := range got[0].Rows {
if key, _ := row["key"].(string); key == "visualeffects:head_accessories/hat/bandana" {
t.Fatalf("expected no generated row while manifest unavailable, got %#v", row)
}
}
}
// A key that is NOT owned by the consumer (an authored row legitimately removed)
// must not be resurrected by the preservation path.
func TestPreserveAutogenConsumerLockEntriesIgnoresUnownedKeys(t *testing.T) {
root := testProjectRoot(t)
lockPath := filepath.Join(root, "visualeffects-lock.json")
writeFile(t, lockPath, `{"visualeffects:retired_authored":3,"visualeffects:head_accessories/hat/bandana":10101}`+"\n")
consumer := project.AutogenConsumerConfig{
ID: "accessory_visualeffects",
Dataset: "visualeffects",
Mode: "accessory_visualeffects",
}
collected := []nativeCollectedDataset{
{
Dataset: nativeDataset{Name: "visualeffects", Kind: nativeDatasetBase, LockPath: lockPath},
LockData: map[string]int{},
},
}
got, err := preserveAutogenConsumerLockEntries(collected, consumer)
if err != nil {
t.Fatalf("preserveAutogenConsumerLockEntries failed: %v", err)
}
if _, ok := got[0].LockData["visualeffects:head_accessories/hat/bandana"]; !ok {
t.Fatalf("expected owned accessory key preserved, got %#v", got[0].LockData)
}
if _, ok := got[0].LockData["visualeffects:retired_authored"]; ok {
t.Fatalf("expected unowned authored key NOT resurrected, got %#v", got[0].LockData)
}
}
+10 -6
View File
@@ -21,16 +21,20 @@ type damagetypesRegistry struct {
Types []map[string]any
}
func collectGeneratedRegistryDatasets(dataDir string) ([]nativeCollectedDataset, error) {
itempropsDatasets, err := collectItempropsRegistryDatasets(dataDir)
// collectGeneratedRegistryDatasets projects the registry datasets. When
// persistLocks is true (the build pipeline) newly allocated ids are written
// back to the source lockfiles; when false (validation, discovery, queries)
// collection is read-only and never touches the source tree.
func collectGeneratedRegistryDatasets(dataDir string, persistLocks bool) ([]nativeCollectedDataset, error) {
itempropsDatasets, err := collectItempropsRegistryDatasets(dataDir, persistLocks)
if err != nil {
return nil, err
}
damagetypeDatasets, err := collectDamagetypesRegistryDatasets(dataDir)
damagetypeDatasets, err := collectDamagetypesRegistryDatasets(dataDir, persistLocks)
if err != nil {
return nil, err
}
racialtypesDatasets, err := collectRacialtypesRegistryDatasets(dataDir)
racialtypesDatasets, err := collectRacialtypesRegistryDatasets(dataDir, persistLocks)
if err != nil {
return nil, err
}
@@ -70,7 +74,7 @@ func loadDamagetypesRegistry(dataDir string) (*damagetypesRegistry, error) {
}, nil
}
func collectDamagetypesRegistryDatasets(dataDir string) ([]nativeCollectedDataset, error) {
func collectDamagetypesRegistryDatasets(dataDir string, persistLocks bool) ([]nativeCollectedDataset, error) {
registry, err := loadDamagetypesRegistry(dataDir)
if err != nil {
return nil, err
@@ -83,7 +87,7 @@ func collectDamagetypesRegistryDatasets(dataDir string) ([]nativeCollectedDatase
if err != nil {
return nil, err
}
if lockModified {
if lockModified && persistLocks {
if err := saveLockfile(registry.LockPath, registry.LockData); err != nil {
return nil, err
}
+133 -8
View File
@@ -203,8 +203,23 @@ func editorConfigPatternMatches(pattern, relativePath string) (bool, error) {
}
func editorConfigGlobRegexp(pattern string) (string, error) {
body, err := translateEditorConfigGlobSegment(pattern)
if err != nil {
return "", err
}
expression := "^" + body + "$"
if _, err := regexp.Compile(expression); err != nil {
return "", err
}
return expression, nil
}
// translateEditorConfigGlobSegment converts an editorconfig glob into a regular
// expression body. In addition to *, **, and ?, it implements brace expansion:
// comma lists ({a,b,c}) become alternations and numeric ranges ({n..m}) expand
// to the integers in the range, matching the editorconfig specification.
func translateEditorConfigGlobSegment(pattern string) (string, error) {
var builder strings.Builder
builder.WriteByte('^')
for index := 0; index < len(pattern); {
char := pattern[index]
switch char {
@@ -220,22 +235,132 @@ func editorConfigGlobRegexp(pattern string) (string, error) {
continue
}
builder.WriteString("[^/]*")
index++
case '?':
builder.WriteString("[^/]")
case '.', '+', '(', ')', '|', '^', '$', '{', '}', '[', ']', '\\':
index++
case '{':
end := matchingEditorConfigBrace(pattern, index)
if end < 0 {
// An unbalanced brace is matched literally.
builder.WriteString("\\{")
index++
continue
}
expansion, err := expandEditorConfigBrace(pattern[index+1 : end])
if err != nil {
return "", err
}
builder.WriteString(expansion)
index = end + 1
case '.', '+', '(', ')', '|', '^', '$', '}', '[', ']', '\\':
builder.WriteByte('\\')
builder.WriteByte(char)
index++
default:
builder.WriteByte(char)
index++
}
index++
}
builder.WriteByte('$')
expression := builder.String()
if _, err := regexp.Compile(expression); err != nil {
return "", err
return builder.String(), nil
}
// matchingEditorConfigBrace returns the index of the '}' that closes the '{' at
// open, accounting for nesting, or -1 when the braces are unbalanced.
func matchingEditorConfigBrace(pattern string, open int) int {
depth := 0
for index := open; index < len(pattern); index++ {
switch pattern[index] {
case '{':
depth++
case '}':
depth--
if depth == 0 {
return index
}
}
}
return expression, nil
return -1
}
// expandEditorConfigBrace turns the body of a brace expression into a regex
// fragment. A comma list becomes an alternation; a lone "n..m" body becomes a
// numeric range. A body with neither comma nor range is treated as literal
// braces, matching the editorconfig reference implementation.
func expandEditorConfigBrace(inner string) (string, error) {
options := splitTopLevelBraceCommas(inner)
if len(options) == 1 {
if rangeExpr, ok, err := editorConfigNumericRange(inner); err != nil {
return "", err
} else if ok {
return rangeExpr, nil
}
segment, err := translateEditorConfigGlobSegment(inner)
if err != nil {
return "", err
}
return "\\{" + segment + "\\}", nil
}
parts := make([]string, 0, len(options))
for _, option := range options {
segment, err := translateEditorConfigGlobSegment(option)
if err != nil {
return "", err
}
parts = append(parts, segment)
}
return "(?:" + strings.Join(parts, "|") + ")", nil
}
// splitTopLevelBraceCommas splits a brace body on commas that are not nested
// inside an inner brace group.
func splitTopLevelBraceCommas(inner string) []string {
parts := []string{}
depth := 0
start := 0
for index := 0; index < len(inner); index++ {
switch inner[index] {
case '{':
depth++
case '}':
depth--
case ',':
if depth == 0 {
parts = append(parts, inner[start:index])
start = index + 1
}
}
}
return append(parts, inner[start:])
}
// editorConfigNumericRange expands "n..m" into an alternation of the integers
// in [n, m]. It returns ok=false when the body is not a numeric range.
func editorConfigNumericRange(inner string) (string, bool, error) {
separator := strings.Index(inner, "..")
if separator < 0 {
return "", false, nil
}
low, err := strconv.Atoi(strings.TrimSpace(inner[:separator]))
if err != nil {
return "", false, nil
}
high, err := strconv.Atoi(strings.TrimSpace(inner[separator+2:]))
if err != nil {
return "", false, nil
}
if low > high {
low, high = high, low
}
const maxRangeSize = 65536
if high-low >= maxRangeSize {
return "", false, fmt.Errorf("editorconfig numeric range {%s} is too large", inner)
}
parts := make([]string, 0, high-low+1)
for value := low; value <= high; value++ {
parts = append(parts, strconv.Itoa(value))
}
return "(?:" + strings.Join(parts, "|") + ")", true, nil
}
func pathBase(path string) string {
@@ -0,0 +1,56 @@
package topdata
import (
"os"
"path/filepath"
"testing"
)
func TestEditorConfigPatternMatchesBraceExpansion(t *testing.T) {
cases := []struct {
pattern string
path string
want bool
}{
{"*.{json,jsonc}", "lock.json", true},
{"*.{json,jsonc}", "types.jsonc", true},
{"*.{json,jsonc}", "notes.md", false},
{"*.{js,ts,tsx}", "main.tsx", true},
{"{foo,bar}.txt", "bar.txt", true},
{"{foo,bar}.txt", "baz.txt", false},
{"page{1..3}.json", "page2.json", true},
{"page{1..3}.json", "page4.json", false},
// No comma and not a range: editorconfig treats braces literally.
{"{single}.txt", "{single}.txt", true},
{"{single}.txt", "single.txt", false},
}
for _, c := range cases {
got, err := editorConfigPatternMatches(c.pattern, c.path)
if err != nil {
t.Fatalf("pattern %q path %q: %v", c.pattern, c.path, err)
}
if got != c.want {
t.Errorf("editorConfigPatternMatches(%q, %q) = %v, want %v", c.pattern, c.path, got, c.want)
}
}
}
func TestSaveLockfileHonorsBraceGlobIndentSize(t *testing.T) {
root := t.TempDir()
writeFile(t, filepath.Join(root, ".editorconfig"), "root = true\n\n[*]\nindent_style = space\nindent_size = 2\n\n[*.{json,jsonc}]\nindent_size = 4\n")
lockPath := filepath.Join(root, "data", "damagetypes", "registry", "lock.json")
mkdirAll(t, filepath.Dir(lockPath))
writeFile(t, lockPath, "{\n \"damagetype:acid\": 4\n}\n")
if err := saveLockfile(lockPath, map[string]int{"damagetype:acid": 4}); err != nil {
t.Fatalf("saveLockfile: %v", err)
}
got, err := os.ReadFile(lockPath)
if err != nil {
t.Fatalf("read lockfile: %v", err)
}
want := "{\n \"damagetype:acid\": 4\n}\n"
if string(got) != want {
t.Fatalf("expected 4-space indent resolved from [*.{json,jsonc}]; got:\n%q", string(got))
}
}
+2 -2
View File
@@ -166,7 +166,7 @@ func loadRegistryRows(path string) ([]map[string]any, error) {
return rows, nil
}
func collectItempropsRegistryDatasets(dataDir string) ([]nativeCollectedDataset, error) {
func collectItempropsRegistryDatasets(dataDir string, persistLocks bool) ([]nativeCollectedDataset, error) {
registry, err := loadItempropsRegistry(dataDir)
if err != nil {
return nil, err
@@ -188,7 +188,7 @@ func collectItempropsRegistryDatasets(dataDir string) ([]nativeCollectedDataset,
return nil, err
}
lockModified = lockModified || costModified || paramModified
if lockModified {
if lockModified && persistLocks {
if err := saveLockfile(registry.LockPath, registry.LockData); err != nil {
return nil, err
}
+2 -2
View File
@@ -398,7 +398,7 @@ func buildNativeUnchecked(p *project.Project, opts NativeBuildOptions, progress
datasets = applyTopDataValueEncodings(datasets, p.Config.TopData.ValueEncodings)
datasets = applyTopDataValueDefaults(datasets, p.Config.TopData.ValueDefaults)
datasets = applyTopDataRowGeneration(datasets, p.Config.TopData.RowGeneration)
registryDatasets, err := collectGeneratedRegistryDatasets(dataDir)
registryDatasets, err := collectGeneratedRegistryDatasets(dataDir, true)
if err != nil {
return BuildResult{}, err
}
@@ -1073,7 +1073,7 @@ func discoverNativeOutputCatalog(dataDir string) (map[string]string, error) {
if err != nil {
return nil, err
}
registryDatasets, err := collectGeneratedRegistryDatasets(dataDir)
registryDatasets, err := collectGeneratedRegistryDatasets(dataDir, false)
if err != nil {
return nil, err
}
+2 -2
View File
@@ -120,7 +120,7 @@ func loadRacialtypesRegistryRaceRows(dir string) ([]map[string]any, error) {
return rows, nil
}
func collectRacialtypesRegistryDatasets(dataDir string) ([]nativeCollectedDataset, error) {
func collectRacialtypesRegistryDatasets(dataDir string, persistLocks bool) ([]nativeCollectedDataset, error) {
registry, err := loadRacialtypesRegistry(dataDir)
if err != nil {
return nil, err
@@ -138,7 +138,7 @@ func collectRacialtypesRegistryDatasets(dataDir string) ([]nativeCollectedDatase
return nil, err
}
lockModified = lockModified || raceLockModified
if lockModified {
if lockModified && persistLocks {
if err := saveLockfile(registry.LockPath, registry.LockData); err != nil {
return nil, err
}
@@ -0,0 +1,64 @@
package topdata
import (
"os"
"path/filepath"
"testing"
)
const damagetypesRegistryTestTypes = `{
"rows": [
{"id": 0, "key": "damagetype:bludgeoning", "label": "Bludgeoning", "group_label": "Physical", "damage_type_group": 0},
{"id": 1, "key": "damagetype:piercing", "label": "Piercing", "group_label": "Physical", "damage_type_group": 0}
]
}
`
// validate-topdata must never modify source lockfiles. Collection in read-only
// mode must leave the registry lock byte-identical even though the rows carry
// explicit ids that would otherwise mark the lock "modified".
func TestCollectGeneratedRegistryDatasetsReadOnlyDoesNotWriteLock(t *testing.T) {
dataDir := t.TempDir()
regDir := filepath.Join(dataDir, "damagetypes", "registry")
mkdirAll(t, regDir)
writeFile(t, filepath.Join(regDir, "types.json"), damagetypesRegistryTestTypes)
lockPath := filepath.Join(regDir, "lock.json")
original := "{\n \"damagetype:bludgeoning\": 0,\n \"damagetype:piercing\": 1\n}\n"
writeFile(t, lockPath, original)
if _, err := collectGeneratedRegistryDatasets(dataDir, false); err != nil {
t.Fatalf("read-only collection failed: %v", err)
}
got, err := os.ReadFile(lockPath)
if err != nil {
t.Fatalf("read lockfile: %v", err)
}
if string(got) != original {
t.Fatalf("read-only collection must not modify lockfile.\nwant: %q\ngot: %q", original, string(got))
}
}
// build-topdata still persists allocated ids to source lockfiles.
func TestCollectGeneratedRegistryDatasetsPersistWritesLock(t *testing.T) {
dataDir := t.TempDir()
regDir := filepath.Join(dataDir, "damagetypes", "registry")
mkdirAll(t, regDir)
writeFile(t, filepath.Join(regDir, "types.json"), damagetypesRegistryTestTypes)
lockPath := filepath.Join(regDir, "lock.json")
writeFile(t, lockPath, "{\n \"damagetype:bludgeoning\": 0\n}\n")
if _, err := collectGeneratedRegistryDatasets(dataDir, true); err != nil {
t.Fatalf("persisting collection failed: %v", err)
}
lockData, err := loadLockfile(lockPath)
if err != nil {
t.Fatalf("load lockfile: %v", err)
}
if lockData["damagetype:piercing"] != 1 {
t.Fatalf("persisting collection should record allocated id; got %#v", lockData)
}
}
+1 -1
View File
@@ -557,7 +557,7 @@ func expectedCompiled2DAOutputs(p *project.Project) (map[string]struct{}, error)
if err != nil {
return nil, err
}
registryDatasets, err := collectGeneratedRegistryDatasets(dataDir)
registryDatasets, err := collectGeneratedRegistryDatasets(dataDir, false)
if err != nil {
return nil, err
}
+2 -2
View File
@@ -1320,7 +1320,7 @@ func validateNativeOutputCatalog(dataDir string, report *ValidationReport) {
})
return
}
registryDatasets, err := collectGeneratedRegistryDatasets(dataDir)
registryDatasets, err := collectGeneratedRegistryDatasets(dataDir, false)
if err != nil {
report.Diagnostics = append(report.Diagnostics, Diagnostic{
Severity: SeverityError,
@@ -2307,7 +2307,7 @@ func validateNativeBuildability(p *project.Project, report *ValidationReport) {
})
return
}
registryDatasets, err := collectGeneratedRegistryDatasets(dataDir)
registryDatasets, err := collectGeneratedRegistryDatasets(dataDir, false)
if err != nil {
report.Diagnostics = append(report.Diagnostics, Diagnostic{
Severity: SeverityError,
+1 -1
View File
@@ -748,7 +748,7 @@ func collectWikiRacialtypesDatasets(dataDir string, datasets []nativeDataset) ([
if len(out) > 0 {
return out, nil
}
return collectRacialtypesRegistryDatasets(dataDir)
return collectRacialtypesRegistryDatasets(dataDir, false)
}
func collectWikiRacialtypeStatuses(dataDir string) map[string]string {
+5
View File
@@ -0,0 +1,5 @@
# Repos that carry copies of the canonical Crucible wrappers.
# One "owner/repo" per line. sync-wrappers.yml opens an update PR to each when
# wrappers/ changes. Add a repo here AND grant the sync bot write access to it.
ShadowsOverWestgate/sow-module
ShadowsOverWestgate/sow-topdata