Closes#83. Part of #54; implements #63 Decision 5 ("Coordinated rename flip").
## What changed
One string literal in `internal/app/app.go:388`:
```go
[]string{"scripts", "fetch-hak-manifest"} -> []string{"scripts", "fetch-upstream-manifests"}
```
sow-module renamed its half in ShadowsOverWestgate/sow-module#57
(`scripts/fetch-hak-manifest.sh` -> `scripts/fetch-upstream-manifests.sh`,
extended to resolve the topdata channel as well). The script name is a
hardcoded path convention shared by the two repos, not config, so both sides
only work when they carry the same name.
No doc in this repo named the old script (`grep` over the tree found the one
call site only), so nothing else needed touching.
## No fallback, by decision
Per #63 Decision 5: no transitional symlink, no Go-side fallback. Both sides
flip and the short broken window is accepted, because the failure is loud and
unmistakable (`required project script is missing`). A fallback would keep
both names alive forever.
## Merge order matters
`sow-module/.gitea/workflows/release.yml` runs `nix flake update sow-tools`, so
it always builds against the latest crucible, unpinned. Every crucible module
build in sow-module fails between sow-module#57 merging and a crucible release
carrying this flip. **Merge sow-module#57 and cut a crucible release back to
back** to keep that gap short.
## Checks
- `go build ./...`, `go vet ./internal/app/`, and the full `go test ./...` suite pass.
- No test covers this call path, and none was added: it is a single hardcoded
constant that has to match the other repo, so a test here could only assert
the literal against itself. The real check is the sow-module build.
## Still open on the issue's "done when"
- [x] app.go runs `scripts/fetch-upstream-manifests`
- [x] no doc in this repo names `fetch-hak-manifest`
- [ ] a crucible release is cut, and the crucible module build succeeds in a
sow-module checkout at #57's head — needs a release after this merges
🤖 Generated with [Claude Code](https://claude.com/claude-code)Reviewed-on: #84
Reviewed-by: xtul <mpiasecki720@protonmail.com>
Co-authored-by: vickydotbat <vickydotbat@tutamail.com>
Fixes#81.
`crucible nwsync emit` and `assemble` printed their summary line to **stdout**. Any caller that captures a script's stdout as a value gets the summary glued onto it — `pack-haks.sh` does `release_dir="$(...)"`, so all 11 emit summaries landed in `$release_dir` and `publish-release.sh` died with "release dir not found".
Both lines move to stderr, where `lib.sh`'s own `nwsync: emitted $key` log already goes. Neither line is a machine-readable contract: sow-topdata's contract tests grep an `EMIT_LOG` their own fake-crucible stub writes, not real stdout, so nothing parses these.
`runEmit`/`runAssemble` no longer take the stdout writer — a leak in those two functions is now impossible to write by accident. `Run` still passes stdout to `printRunUsage` for explicit `-h`/`help`, which is correct.
Adds `TestRunKeepsSummariesOffStdout`: runs both verbs end to end against a local tree and asserts stdout stays empty while the summary reaches stderr. It asserts emptiness, not wording, so the summary text stays free to change.
Full `go test ./...` green.
Follow-up, outside this repo: sow-assets-manifest needs a `flake.lock` bump, then a re-run of the v0.2.1-rc1 tag.
🤖 Generated with [Claude Code](https://claude.com/claude-code)Reviewed-on: #82
Reviewed-by: xtul <mpiasecki720@protonmail.com>
Co-authored-by: vickydotbat <vickydotbat@tutamail.com>
Closes#79.
Emit is latency-bound, not CPU-bound. Every blob costs two serial round-trips to the zone — a `ProbeKey` HEAD, then a `PutReader` PUT — so a hak with a few thousand resources pays a few thousand serialised latencies. Measured on the live sow-assets-manifest backfill: 26 s of CPU across 9.5 minutes of wall clock, on a 4-core host with 5 GB free and peak RSS of 51 MB.
`emit` now hashes, compresses and stores `--jobs N` resources at once, default 16 — matching `DEPOT_JOBS` and the transport's `MaxIdleConnsPerHost`, so a worker per connection needs no fresh TLS handshake. `--jobs 1` is exactly the old behaviour.
### Three properties had to survive
Each has a test in `internal/nwsync/jobs_test.go`:
- **Deterministic manifest bytes.** `emitterVersion` promises a manifest is a function of its artifact, so `entries` is index-addressed rather than appended to — a worker owns `entries[i]` alone and the slice comes back in artifact order whatever order uploads finish in. `TestEmitProducesTheSameIndexAtEveryJobCount` diffs the `.nsym` and its sidecar between `-jobs 1` and `-jobs 16`.
- **Index still lands last.** Any worker's failure aborts before a manifest is written. `TestEmitLeavesNoIndexWhenAParallelUploadFails` fails every blob PUT with 16 workers in flight and asserts no `.nsym` appears. Under `-race` it also covers the shared counters.
- **Identical content still shares one blob.** This one bit during development and is the reason to read the diff carefully: serially, the sink's existence check absorbed two resrefs with identical bytes. In parallel both workers probe, both miss, and both upload — `TestEmitWritesBlobsAndManifest` caught it as "wrote 2 blobs, want 1". Claiming the sha1 in-process restores the dedupe and skips a probe round-trip as well.
### Memory
Peak now tracks the resources in flight rather than one resource. The ceiling is `N` × the 15 MB `fileSizeLimit` plus its compressed copy — bounded by a constant this package enforces itself, and still not tracking the archive. `TestEmitPeakMemoryIsBoundedByJobCount` re-runs the #76 regression check at `-jobs 8`: a hak 8× bigger still costs the same.
This is only cheap because of #78. Before streaming emit, N workers would have meant N whole archives resident.
### Not done
Skipping the `ProbeKey` HEAD on a first-time emit would halve round-trips, but doubles uploaded bytes on a re-run — which is exactly what a backfill is. Noted in #79 so it is not rediscovered; parallelism is the better lever and this PR takes it.
Worker compression still serialises on `blobEncoder`, which is `WithEncoderConcurrency(1)` for the memory reason in `compressedbuf.go`. At 26 s of CPU per hak that is not worth trading memory for, but it is where to look if the numbers ever say otherwise.
🤖 Generated with [Claude Code](https://claude.com/claude-code)Reviewed-on: #80
Co-authored-by: vickydotbat <vickydotbat@tutamail.com>
Closes#76. Part of #54.
`nwsync emit` held roughly 5× the artifact size in RAM, so ovh-main (7 GB, no swap) OOM-killed it on any hak over ~1.4 GB. That blocked the backfill in sow-assets-manifest and would have killed the next release rebuilding a large hak.
## What it does
- `erf.ReadIndex` / `erf.ReadPayload` — parse the header and resource table only, read one payload on demand. `erf.Read` keeps its shape but returns payloads as subslices of the buffer instead of fresh copies, which removes one full copy for the `pipeline` callers too. Callers must not mutate `Resource.Data`; the doc comment says so and no caller does.
- `nwsync.Emit` opens the artifact, hashes it by streaming for the key check (through a section reader, so the file offset stays put), then hashes, compresses and stores one resource at a time. The archive is never resident. Shadowed duplicate resrefs are now never read at all.
- Bounds checks in `ReadIndex` moved to `int64`, so key/resource-list offsets can no longer overflow.
## Not in the issue, but memory-motivated
The zstd blob encoder ran at the default concurrency, which is one encoder per CPU, each holding a window-sized history — about 200 MB of live heap doing nothing on a 24-core runner. `EncodeAll` is single-threaded per call, so concurrency 1 costs nothing. `TestSingleThreadedEncoderMatchesDefault` pins the claim that blobs come out byte-identical.
## Measured
Peak heap during emit, sampled 1 ms:
| hak | before | after |
|-----|--------|-------|
| 8 MB | 155 MB | 21 MB |
| 64 MB | 289 MB | 22 MB |
Flat, as the acceptance asks. `TestEmitPeakMemoryDoesNotScaleWithArtifactSize` fails if the 64 MB fixture costs more than the 8 MB one plus 24 MB of slack.
## Gaps
- The regression check measures Go heap, not RSS, and its largest fixture is 64 MB — a multi-GB run was not done here. A 2.15 GB hak now needs about the same ~22 MB the 64 MB one does, so the 5 GB budget is not close, but that is inference from the flat curve, not a measurement.
- mmap was suggested in the issue and skipped. Payload buffers are still anonymous, but they are one resource each (≤15 MiB), so making them file-backed buys nothing now.
🤖 Generated with [Claude Code](https://claude.com/claude-code)Reviewed-on: #78
Reviewed-by: xtul <mpiasecki720@protonmail.com>
Co-authored-by: vickydotbat <vickydotbat@tutamail.com>
Documents what `nwsync emit --out DIR` actually produces, so a zone can be checked by hand.
The trap this removes: a blob's filename is the SHA-1 of the resource's *original* bytes, but the file on disk is NWCompressedBuffer framing — a 24-byte `NSYC` header then a zstd frame. `sha1sum <blob>` therefore never matches the name it is sitting under. The doc records the recipe that does match:
```
tail -c +25 <blob> | zstd -dc | sha1sum
```
Also notes the tree layout, that the uncompressed length is a little-endian `uint32` at offset 12, and the rough 4:1 compression ratio on hak content.
Verified two ways: against a real emit of a 250 MB hak (2296 blobs, 59 MB on disk against 249 MB of resources), and against `internal/nwsync/compressedbuf.go`, where the header is written as `[]uint32{blobMagic, blobVersion, algorithmZstd, uint32(len(data)), zstdHeaderVer, zstdDictionary}` — confirming field 3 at offset 12.
Docs only, no behaviour change. Falls out of the investigation in #76; that fix is not in this PR.
🤖 Generated with [Claude Code](https://claude.com/claude-code)Reviewed-on: #77
Reviewed-by: xtul <mpiasecki720@protonmail.com>
Co-authored-by: vickydotbat <vickydotbat@tutamail.com>
Builds the code half of #60, and closes the CLI gap the #53 sweep found: PR #71 shipped #53's original surface rather than the one #56, #62 and #65 settled.
## What lands
**`depot.KeyStore`** — `ProbeKey` / `PutReader` / `GetKey`, addressing the zone by object key rather than by depot sha. NWSync cannot use the sha-addressed path: a blob is named after the sha1 of its *uncompressed* bytes while the body uploaded is the compressed form, and Bunny's `Checksum` header is sha256 of the body. Per #55 this reuses `internal/depot`'s `httpBackend` — same IPv4-pinned transport, same retry, same tri-state probe — and the sha-addressed `Backend` is now rewritten on top of it. No second HTTP client, no per-instance hash-function fields: the caller passes the key and the checksum, which turned out simpler than #55 expected.
**A sink in `internal/nwsync`** — the zone by default, a local tree under `--out DIR` as the conformance path. Blobs upload as they are produced and the index lands last, so the presence of an index is the publication marker. A blob already in the zone is skipped via #55's probe *without* paying for compression (the body is a thunk) — which matters for the backfill, where compression is the expensive part.
**The settled CLI**
```
nwsync emit [--as NAME] [--out DIR] <artifact-key> <file>
nwsync assemble --group-id N [--tlk-key KEY] [--out DIR] <artifact-key>...
```
Artifact keys are depot keys; an index lives beside its artifact with the extension replaced (#62), derived in exactly one place so `emit` and `assemble` cannot disagree. Flags may now follow positionals — Go's `flag` stops at the first non-flag argument, which cost a run during #59.
**Fail-closed in two places** — an artifact key whose embedded digest does not match the file is refused (publishing an index under the wrong key silently pairs a manifest with the wrong artifact), and `assemble` refuses an artifact with no index rather than publishing a manifest missing a hak.
## Checks
`make check` green. Six new tests run against a Bunny-shaped `httptest` zone that verifies the `Checksum` header the way Bunny does: blobs-then-index ordering, skip-if-present, no index after a failed upload, key/file mismatch, and assemble reading indexes back out of the zone.
Conformance re-run through the new CLI against upstream `nwn_nwsync_write` 2.1.2 over `sow_vfxs_01.hak` (#59's oracle): the manifest is still **byte-identical**.
## Not in this PR
- **Live upload against the real zone.** The nwsync zone and its credential are #61, still open. Everything here is proven against a fake zone only.
- **Consumer wiring** — #65, in the three producer repos.
- **The mid-hak failure *policy*.** The mechanism is here (fail closed, orphan blobs left, re-run resumes); whether a module release may proceed when an emit failed is a human call, still open on #60.
🤖 Generated with [Claude Code](https://claude.com/claude-code)Reviewed-on: #73
Co-authored-by: vickydotbat <vickydotbat@tutamail.com>
Builds sow-tools#53. Format spec followed is the resolution comment of sow-platform#94, checked line by line against niv/neverwinter.nim at HEAD (`nwsync.nim`, `compressedbuf.nim`, `nwsync/private/libupdate.nim`).
## What lands
`crucible nwsync emit <artifact> --out DIR` — explodes one `.hak`/`.erf`, or one loose file such as the TLK, into NWSync blobs plus a NSYM v3 manifest covering only that artifact, with the same `.json` sidecar upstream writes. Blob path `data/sha1/<h0h1>/<h2h3>/<sha1>`, body in NWCompressedBuffer framing (magic `NSYC`, version 3, algorithm 2, uncompressed size, zstd header version 1, dictionary 0, raw zstd frame). The sha1 that names a blob is over the uncompressed bytes.
`crucible nwsync assemble --order NAMES --entries DIR --out DIR [--group-id N]` — merges the per-artifact manifests into one, reading no bulk data at all. Merge rule is resref shadowing, not concatenation: a resref in more than one artifact resolves to the earliest artifact in `--order`, which is how the game resolves it. `--group-id` stays caller-supplied (1 current, 2 testing; 0 is absent, matching upstream omitting a zero integer meta field).
Rules taken from upstream and not re-invented: `nss`/`ndb`/`gic` always skipped; an unresolvable restype is a hard error, not a skip; a resource over 15 MB fails closed; no `latest` file and no `.origin` file, ever. A `.mod` is refused outright — a persistent world publishes no module contents, so the module contributes no bytes.
## Two deliberate departures
- **Emitter version is its own field, not the build revision.** `emitter_version` is a constant bumped only when emitted bytes change. Keying the refuse-to-merge check on `created_with` would invalidate every published index on every unrelated crucible commit and force a re-emit of the whole 15 GB corpus — the opposite of "nothing downstream ever needs the hak again".
- **`SOURCE_DATE_EPOCH` pins the sidecar timestamp.** The manifest itself was already deterministic; the sidecar's `created` was not, against the determinism rule in `docs/consumer-contract.md`.
## Not in this PR, and why
- **Direct upload.** Only the local `--out` sink exists, which is the conformance path. The upload sink and the mid-hak-failure question are sow-tools#60, and the consumer wiring is #65.
- **The conformance run against upstream.** sow-tools#59 owns getting `nwn_nwsync_write` running and capturing reference output. The format here was read from upstream source rather than from its output, so the byte-for-byte manifest comparison and the after-decompression blob comparison still have to happen — that is what #59 is for. The tests in this PR check the layout against the spec, so a shared misreading would pass them.
- **The `artifacts/haks/sha256/<a>/<b>/<sha256>.nsym` location.** emit writes `<out>/<name>.nsym`; where a publisher puts it is the publisher's business (#65).
- **The acceptance gate** — a real client syncing from an assembled manifest — is unchanged and still open.
## Checks
`make check` green (vet, unit tests, shellcheck, yamllint, workflow contract), `make smoke` green with the new builder, `nix build .#crucible` produces `crucible-nwsync`.
🤖 Generated with [Claude Code](https://claude.com/claude-code)Reviewed-on: #71
Co-authored-by: vickydotbat <vickydotbat@tutamail.com>
Closes#52.
Adds `scripts/prune-release-assets.sh` and a best-effort step at the end of `build-binaries.yml`.
- Lists every release page, sorts by creation date, skips the newest two published releases, deletes the rest's attachments via `DELETE /releases/{id}/assets/{asset_id}`.
- Drafts never consume a keep slot, so a draft cannot strip the newest real release.
- Releases, tags, source archives and release notes are never touched.
- Nothing is excluded: SHA256SUMS and the wrappers go with the binaries. Checksums are only meaningful next to the files they cover, and the consumer drift-check reads the wrappers from the latest release, which stays complete.
- Failures warn and the step has `continue-on-error: true` — a stale asset is cheaper than a blocked publish.
`tests/prune-release-assets.sh` drives the script with a stub `curl`: keep window, out-of-order responses, drafts, no non-asset DELETE, short-page pagination stop, and both failure paths. Wired into `make check`.
v0.2.0 still needs its 9 assets cleared by hand in the web UI, as the issue notes.
🤖 Generated with [Claude Code](https://claude.com/claude-code)Reviewed-on: #70
Co-authored-by: vickydotbat <vickydotbat@tutamail.com>
Closes#64.
Five entries in `internal/erf/erf.go`'s restype table disagreed with upstream neverwinter.nim (`neverwinter/restype.nim`). `HAKResourceTypeForExtension` packs arbitrary source files, so a `.jpg` packed into a HAK today is written as restype 2076, which the game reads as `tml`.
| ext | was | now |
|-----|-----|-----|
| gff | 2039 (upstream `bte`) | 2037 |
| ltr | 2067 (upstream `bak`) | 2036 |
| jpg | 2076 (upstream `tml`) | 2081 |
| dfa | 2045, name typo | `dft`, 2045 (number was already right) |
| mdb | 2070 (upstream `xbc`) | removed |
Upstream registers no restype for `mdb`, so there is no correct number to give it, and leaving it squatting on `xbc` silently mislabels the resource. It is dropped from `AssetExtensions` too, which turns a `.mdb` source into a loud "unsupported extension" build error instead of a corrupt HAK entry. **This is the one judgment call the issue left open** — if `mdb` should instead stay as a deliberate local addition like `lyt`/`vis`/`gr2`, say so and I will move it to a free number in the 0x0BB8+ local range.
No asset in the current corpus uses any of these paths, so nothing in shipped content changes.
The six local additions upstream does not have (`lyt` 3000, `vis` 3001, `mdx` 3008, `wlk` 3020, `xml` 3021, `gr2` 4003) are left alone and now carry a comment saying they are deliberate.
## Root cause
The `init()` consistency guard only panicked when a number was missing from the reverse map. Its `if canonicalExt == ext { continue }` branch did nothing when the two maps disagreed, so a mismatch was never caught. The guard now panics on disagreement and also checks the reverse direction.
## Tests
- `TestRestypeTableMatchesUpstream` pins every number shared with upstream, and asserts the four numbers upstream reserves for other extensions (2039 `bte`, 2067 `bak`, 2070 `xbc`, 2076 `tml`) stay unclaimed.
- `TestRestypeTablesAreMutualInverses` is the check `init()` is meant to enforce.
`make check` passes.Reviewed-on: #69
Co-authored-by: vickydotbat <vickydotbat@tutamail.com>
`tea comment` reads stdin to EOF and appends whatever it finds to the comment body. In a non-interactive shell — which is every agent — stdin is an open pipe that never sends EOF, so the call hangs forever instead of posting.
Verified on tea 0.14.0: with `</dev/null` it posts instantly; with an open pipe it blocks until killed; with `echo "x" | tea comment N "y"` it posts `y` followed by `x`.
Documents the redirect in the tracker guide, and applies it to the wayfinder resolve step. Same trap exists on `tea issues create --description` and `tea pr create`.
🤖 Generated with [Claude Code](https://claude.com/claude-code)Reviewed-on: #68
Co-authored-by: vickydotbat <vickydotbat@tutamail.com>
Closes#66.
`depot status` and `depot get` chose a depot tree on disk with `--target local`, taking the root from `DEPOT_DIR`. One decision, two flags — and `--target local` with no directory anywhere was valid but meaningless.
Now they take `--out DIR`, matching the crucible nwsync surface. `--target` names remote backends only (`bunny|cdn`). Passing both exits 64.
- `--target local` stays as a deprecated alias for `--out $DEPOT_DIR`, listed in the hidden-alias table in `docs/command-surface.md`. Nothing in `wrappers/` calls depot, but the three repos in `wrappers/consumers.txt` are out of reach from here, so the alias stays.
- `status --source` was already unused; it now warns "ignored; use --out DIR" instead of accepting silently.
- Fixes a real parse bug found on the way: the documented `depot get <sha> <dest> --target cdn` form never parsed its flags, because Go's `flag` package stops at the first positional. Positionals are split off by hand now; both orders are tested.
- Registry usage strings and `docs/command-surface.md` synced — that doc still said depot was unwired.
Not converted: `push --source` and `pull --dest`. Neither is `--target local`; one names a read source, the other a remote pull's destination. Say the word if they should become `--out` too.
`go test ./...` green.
🤖 Generated with [Claude Code](https://claude.com/claude-code)Reviewed-on: #67
Co-authored-by: vickydotbat <vickydotbat@tutamail.com>
Part of the workspace-wide standards rollout. Establishes where work lives (issues vs ADRs vs standing law), scaffolds `docs/agents/` for the engineering skills, and adopts ADR-0001 locally. See `sow-platform` ADR-0021 for the `sow-docs` retirement.Reviewed-on: #51
Reviewed-by: xtul <mpiasecki720@protonmail.com>
Co-authored-by: vickydotbat <vickydotbat@tutamail.com>
*palcus.itp files are Toolset-derived snapshots (category skeleton + blueprint descriptors). This makes Crucible the generator: module builds strip stale descriptors from the committed skeletons and inject descriptors computed from the source-tree blueprints (NAME/STRREF + RESREF, plus CR/FACTION for creatures, faction names resolved via repute.fac). Compare applies the same projection; extract never writes *palcus.itp back into source, so local Toolset palette mess stays local.
Companion PR in sow-module strips the committed palcus files down to skeletons.
🤖 Generated with [Claude Code](https://claude.com/claude-code)Reviewed-on: #50
Co-authored-by: vickydotbat <vickydotbat@tutamail.com>
## Summary
- always wrap NWN model compilation with `xvfb-run`, even when the caller has `DISPLAY`
- fail closed with an actionable error when `xvfb-run` is unavailable instead of opening the client UI
- update the compiler contract and regression coverage
## Verification
- focused red/green regression tests
- `nix develop -c make check`
- real NWN compile with `DISPLAY=:0` under transient Xvfb; binary MDL output verified
Generated with Claude CodeReviewed-on: #49
Co-authored-by: vickydotbat <vickydotbat@tutamail.com>
## Problem
The custom palette taxonomy (#46) pins display strings to fixed TLK ids in `tlk/custom.tlk.yml`, and `registerInlineAtID` demanded each pinned id be free. But `.tlk_state.json` is **gitignored and per-machine** — each dev grows their own copy, and before #46 every ref got its id dynamically (first-come-first-served).
On any machine whose state predates #46, a ref could have already parked on a now-pinned id. That fails the build:
```
topdata validation failed with 1 error(s): error: native topdata buildability check failed:
TLK id 2689 is already reserved by "feat:yuanti/alternate_form.feat"
```
It only passes on machines whose state was regenerated after #46 (pinned window already clean). The `allocateID` reserved-skip that #46 added protects a *fresh* build, but does nothing about a *stale cache*.
## Fix
Make the pin authoritative over the per-machine cache. A stale cached owner sitting on a pinned id is evicted and reallocated a fresh id when next made active; only two pins fighting over the same id in `custom.tlk.yml` is now an error. This self-heals on the next build — no manual `.tlk_state.json` deletion needed.
Caveat: the evicted ref's strref shifts on affected machines. That's unavoidable (something must move off the pinned id), and dynamic strrefs were never stable across machines anyway.
## Tests
Added `TestBuildStandaloneTLKPinEvictsStaleStateOwner` (seeds a pre-#46 state with a feat parked on the pinned id, asserts the build succeeds, the pin owns it, and the feat is reallocated). Full `internal/topdata` suite + `go vet` pass.
🤖 Generated with [Claude Code](https://claude.com/claude-code)Reviewed-on: #48
Reviewed-by: xtul <mpiasecki720@protonmail.com>
Co-authored-by: vickydotbat <vickydotbat@tutamail.com>
The legacy normalize migration is done (confirmed). This deletes the dead migration path: NormalizeProject + 26 *_migrate.go files, the importLegacyItempropsRegistry cluster, loadLegacyTLK, and their tests (~7.9k lines, -33 files worth). mergeLegacyTLK moved into tlk_native.go since the live TLK compiler still uses it for base_dialog.json. make check passes.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Reviewed-on: #45
Reviewed-by: gitea-bot <gitea-bot@noreply.git.westgate.pw>
Co-authored-by: vickydotbat <vickydotbat@tutamail.com>
Co-committed-by: vickydotbat <vickydotbat@tutamail.com>
Adds .agents/skills/data-driven-design: general mechanisms in code, all specific knowledge in data. Written for the long-term cleanup that would let sow-tools be open sourced.
Grounded in two real failures: the value_encodings mode zoo (per-column mode enums + switch arms where a shape/encoding declaration in data would do) and the skills dataset rename (code hardcoding dataset names/key prefixes that the data already knows).
Tested per TDD-for-skills: a baseline subagent given the EquipSlots scenario added a fourth mode and rationalized it ("three switch cases isn't a pattern crisis"); with the skill loaded it declared shape in data, collapsed the modes into orthogonal primitives, and deleted the switch.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Reviewed-on: #44
Co-authored-by: vickydotbat <vickydotbat@tutamail.com>
Co-committed-by: vickydotbat <vickydotbat@tutamail.com>
sow-topdata moved skills into a tree (skills/core, skills/specs, skillcats, skillspecranks), which broke validate/build: the generated feat families hardcoded dataset "skills".
Resolution is now data-driven: the family spec's child_source names the dataset, child slugs come from the row key's own namespace, and the wiki loader accepts both the flat and tree layouts (topdata main still uses the flat one until the overhaul merges).
Verified against the skill-grouping-overhaul topdata branch: validate/build/wiki all clean with zero lock churn, focus feats stay on their adopted rows.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Reviewed-on: #43
Co-authored-by: vickydotbat <vickydotbat@tutamail.com>
Co-committed-by: vickydotbat <vickydotbat@tutamail.com>
Extends legacyFamilyChildAliases so skill focus / greater skill focus children adopt the stock feat rows of the skills they were renamed from: medicine<-heal, investigation<-search, diplomacy<-influence/persuade, security<-open lock, intimidate<-taunt.
Existing craft_*/influence/disabledevice aliases stay: sow-topdata main still carries those skills until the overhaul merges. Needs a Crucible release before sow-topdata CI picks this up.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Reviewed-on: #42
Reviewed-by: xtul <mpiasecki720@protonmail.com>
Co-authored-by: vickydotbat <vickydotbat@tutamail.com>
Co-committed-by: vickydotbat <vickydotbat@tutamail.com>
## Summary
The `registry.westgate.pw/deployment/crucible` image has no consumer: `prod.yml` no longer reserves a slot for it (contract settled in sow-platform PR #64) and no runtime or recovery path pulls it. Binaries, wrappers, and the Nix input remain the supported ways to run Crucible.
## What changed
- Deleted `.gitea/workflows/build-image.yml`, `publish-image.yml`, `test-image.yml`
- Deleted `docker/Dockerfile` and the `make image` target
- Updated README, AGENTS.md, and `docs/consumer-contract.md` to state the image is retired
Existing `deployment/crucible` images in the registry can be deleted at leisure; nothing references them.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Reviewed-on: #41
Co-authored-by: vickydotbat <vickydotbat@tutamail.com>
Co-committed-by: vickydotbat <vickydotbat@tutamail.com>
Adds a `set_fields` option to `extract.merge.gff_json` rules: forces existing Int fields to a fixed value on every extract, applied after preserve_fields/merge_lists so the forced value always wins. Fields absent from the document stay absent. Rule targets can now be globs (e.g. `areas/*.are.json`).
Motivation: sow-module area weather chances must stay 0 (scripted regional weather owns weather); this makes the extract pipeline enforce it instead of a post-extract script.
Covered by `TestExtractSetsConfiguredGFFJSONFields`; `make check` passes.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Reviewed-on: #40
Co-authored-by: vickydotbat <vickydotbat@tutamail.com>
Co-committed-by: vickydotbat <vickydotbat@tutamail.com>
Adds the shared agent-skill core under `.agents/skills/` (grilling, grill-me, domain-modeling, codebase-design, code-review, diagnosing-bugs, tdd, research, implement, handoff), with `.claude` symlinked to `.agents` so every agent harness picks them up.
Project-local skills are left untouched. The runtime `scheduled_tasks.lock` is deliberately not committed.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Reviewed-on: #38
Reviewed-by: xtul <mpiasecki720@protonmail.com>
Co-authored-by: vickydotbat <vickydotbat@tutamail.com>
Co-committed-by: vickydotbat <vickydotbat@tutamail.com>
Docs like AGENTS.md/CLAUDE.md under assets/ broke sow-topdata builds
with "unsupported topdata asset HAK resource extension". Skip .md in
both the validator and HAK package walker.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Reviewed-on: #32
Reviewed-by: xtul <mpiasecki720@protonmail.com>
Co-authored-by: vickydotbat <vickydotbat@tutamail.com>
Co-committed-by: vickydotbat <vickydotbat@tutamail.com>
`classFeatExpansionCanUseFeat` excludes `ALLCLASSESCANUSE=0` feats from class feat table masterfeat expansion. When such a feat also has no `MinLevelClass`, no class can ever obtain it — a data bug that previously failed silently (Spell Focus vanishing from the wizard table, Favored Enemy from ranger, proficiencies from commoner).
New `validateClassFeatMasterfeatAccessibility` warns for exactly that case; exclusions with an explicit `MinLevelClass` (deliberate class restrictions, e.g. fighter weapon specialization) and `ALLCLASSESCANUSE=1` feats stay silent. No keys are hardcoded — everything is derived from the datasets.
Verified: new test + full `go test ./internal/topdata/` pass; against live sow-topdata data it flagged 36 real drops (all fixed by the companion sow-topdata PR, after which it reports 0).
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Reviewed-on: #31
Reviewed-by: xtul <mpiasecki720@protonmail.com>
Co-authored-by: vickydotbat <vickydotbat@tutamail.com>
Co-committed-by: vickydotbat <vickydotbat@tutamail.com>
## Summary
- skip overwriting existing `.are.json` files when only the root GFF `Version` differs
- keep real extraction changes writing normally
- add regression coverage for version-only area extraction
## Test Plan
- `nix develop --command make check`
- sample extraction in a clean temp `sow-module` clone using the patched `crucible` binary
Reviewed-on: #28
Reviewed-by: xtul <mpiasecki720@protonmail.com>
Co-authored-by: vickydotbat <vickydotbat@tutamail.com>
Co-committed-by: vickydotbat <vickydotbat@tutamail.com>
Phase 2 of the parts-row autogen design — Crucible (sow-tools) side only.
Resolves parts 2DA rows from the immutable \`part.yml\` published with the asset HAK release, over the anonymous Bunny CDN channel, with the parts consumer **required** (no fail-open).
## Changes
- **Resolver (T3):** generalize \`resolveCDNChannelManifest\` — derive manifest basename + provenance label from config; drop hardcoded \`assets/vfxs.yml\` so \`part.yml\` resolves through the same path. VFX behavior unchanged.
- **Parts filter (T4):** \`filterCDNChannelEntries\` dispatches on consumer mode; new \`filterPartsCDNChannelEntries\` implements the inventory contract — \`restype: mdl\` under \`part/<supported-cat>/\`, row id from trailing digits, dedup l/r/race/gender variants by (category,rowID), sort by source, **zero accepted rows = hard fail**, reject row id 0 / non-numeric / malformed / no-assets.
- **Category (T5):** add \`hand\` + \`parts/hand\` mapping (12 datasets; \`leg→legs\` already present).
- **Pipeline (T6):** native build runs one explicit augment → normalize → override sequence under the configured \`parts_rows\` policy; overrides may update an existing/discovered row but **never synthesize** one (orphan override now errors).
- **Validation (T7):** reject more than one \`parts_rows\` consumer.
## Tests
9 new tests: parts filter contract, 12-dataset coverage, offline manifest-basename, orphan-override reject + discovered-row update, and a parity guard proving a bare CDN-only parts build needs no wrapper/token/NWN_ROOT/checkout.
Gate green: \`go test ./internal/topdata/... ./internal/project/...\` + full \`go build ./... && go test ./...\`.
## Scope / ordering
Phase 2 is inert until **Phase 1** (sow-assets-manifest publishes \`part.yml\`) ships and the operational **Gate** (release publish/promote) runs, then **Phase 3** enables the sow-topdata consumer. Do not enable the consumer against a channel whose current release predates \`part.yml\`.
Plan: \`sow-topdata/docs/superpowers/plans/2026-06-25-topdata-parts-autogen-channel.md\`
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Reviewed-on: #25
Co-authored-by: vickydotbat <vickydotbat@tutamail.com>
Co-committed-by: vickydotbat <vickydotbat@tutamail.com>
It vendors crucible.sh/.ps1 for off-Nix local dev and now carries the
drift-check workflow, so sync-wrappers should open update PRs to it too.
Requires: grant the gitea-bot org token write access to
ShadowsOverWestgate/sow-assets-manifest (per the header note in this file).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Reviewed-on: #20
Reviewed-by: xtul <mpiasecki720@protonmail.com>
Co-authored-by: vickydotbat <vickydotbat@tutamail.com>
Co-committed-by: vickydotbat <vickydotbat@tutamail.com>
Auto-loads the repo's own flake devshell on `cd` via direnv. No-op for non-Nix users (direnv simply isn't installed). `.direnv/` eval cache gitignored.
Part of dropping the redundant root migration/ flake aggregator (its non-git path: input copied the whole tree and broke on the local postgres data dir). Each repo's own git-scoped flake is the source of truth now.
Co-authored-by: archvillainette <zoelynne.victoria@gmail.com>
Reviewed-on: #19
`crucible module build` runs the full pipeline (module + HAK producer +
topdata). A module-only project consumes prebuilt HAKs and has no
`paths.assets`, so the HAK stage crashed on the unset assets dir: the
music-credits walk hit `filepath.WalkDir("")` (`lstat : no such file or
directory`) and asset collection hit `filepath.Rel(root, "")`.
Guard both on an unset `AssetsDir()`: an absent assets tree simply means
no authored credits and no asset files to collect, so the HAK stage
no-ops cleanly while topdata generated-2DA assets still build. Honors
spec Decision 3 (canonical `crucible module build`, no `build-module`).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Reviewed-on: #17
Co-authored-by: vickydotbat <vickydotbat@tutamail.com>
Co-committed-by: vickydotbat <vickydotbat@tutamail.com>
forces build parity with crucible = local builds and CI/CD builds use different tools.
Reviewed-on: #14
Co-authored-by: vickydotbat <vickydotbat@tutamail.com>
Co-committed-by: vickydotbat <vickydotbat@tutamail.com>
## What
Two correctness fixes surfaced in review of the direct-depot HAK artifact work.
### Reject resref+type collisions in `chunksFromManifest`
A manifest can name two distinct source paths that collapse to the same
`resref+type` (e.g. `creature/foo.tga` and `placeable/foo.tga` — resref is the
lowercase basename minus extension). The ERF writer keys resources on
`Name:Type`, so the second silently shadowed the first: an asset would vanish
from the packed HAK with no error.
`chunksFromManifest` now runs `ensureUniqueChunkResources` per chunk and fails
the build on a duplicate. This guards **both** build paths — the legacy
`--source-manifest` flow and the direct content-addressed flow
(`chunksFromSourceManifest` → `chunksFromManifest`).
### Document erf post-write hash coupling
`writeResourceData` streams the source into the output while hashing, so
size/SHA mismatches are only detected *after* the bytes are written. A non-nil
return therefore means the writer holds partial, unverified output and the
caller must discard it. Added a comment making that contract explicit;
`writeHAKArchive` already honours it (writes to a temp file, removes on any
Write error, never renames a bad archive into place).
## Tests
- `TestChunksFromManifestRejectsResrefCollision`: collision → error, distinct
resrefs → clean. Asserts only error presence/absence — silent asset loss is
the contract, not any specific wording.
- `go vet ./internal/erf/ ./internal/pipeline/` clean.
- `go test ./internal/erf/ ./internal/pipeline/` green.
## Follow-up
A new crucible release must be cut after this merges so the guard ships in the
binary `sow-assets-manifest` pins.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Reviewed-on: #8
Co-authored-by: vickydotbat <vickydotbat@tutamail.com>
Co-committed-by: vickydotbat <vickydotbat@tutamail.com>
RC#1 — build wrote 2-space instead of .editorconfig's 4-space
- editorconfig_format.go: the glob translator escaped {/} as literals, so [*.{json,jsonc}]→indent_size=4 never matched any file and formatting fell back to [*]→2. Implemented real editorconfig brace expansion — {a,b,c} alternation and {n..m} numeric ranges. Nothing is hardcoded; saveLockfile already read .editorconfig, it just got the wrong section.
RC#2 — validate wrote a lockfile (it must be read-only)
- The three registry collectors persisted lockfiles as a side effect of collection; ValidateProject calls collection outside its snapshot guard, so the write leaked. Threaded a persistLocks bool through collectGeneratedRegistryDatasets and the damagetypes/itemprops/racialtypes collectors. Only buildNativeUnchecked (the real build allocator) passes true; validate, discovery, packaging queries, and wiki discovery pass false.
Tests added: editorconfig_glob_test.go (brace match + 4-space via brace glob), registry_readonly_test.go (read-only writes nothing; persist writes allocations).
Reviewed-on: #5
Co-authored-by: vickydotbat <vickydotbat@tutamail.com>
Co-committed-by: vickydotbat <vickydotbat@tutamail.com>
Adds a manual release workflow and ensures images are only published on v* tags
Reviewed-on: #1
Co-authored-by: vickydotbat <vickydotbat@tutamail.com>
Co-committed-by: vickydotbat <vickydotbat@tutamail.com>
The reprovision retired gitea.westgate.pw in favour of git.westgate.pw, but
internal/topdata still defaulted to the old host and made a live API call
during the native buildability check, so once CI checkout was fixed the
parts tests failed (DNS, then HTTP 404).
- defaultGiteaBaseURL -> https://git.westgate.pw; normalize git-ssh. -> git.
- Add stubEmptyPartsManifest (stub_test.go) and call it from the 2 non-optional
parts tests so they exercise the build offline. Per-test (t.Setenv) on
purpose: a global TestMain SOW_ASSETS_SERVER_URL would override the
git-remote-derived server other tests configure. VFX tests already pass
(optional consumer ignores HTTP 404).
make check + make smoke green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
global.json uses position/injections format, not rows. The wiki
loadWikiTables WalkDir was processing it as a 2DA table file and
failing the rows-must-be-array check. global.json is handled by
the native topdata path (native.go) after modules are applied.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
archvillainette
