Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
3f500dabc8 | ||
|
|
8e7cead5c0 | ||
|
|
4d38967078 | ||
|
|
7b481ca0c0 | ||
|
|
b058846e16 | ||
|
|
e509b7a90a | ||
|
|
0d12674838 | ||
|
|
bc8fa3a6fe | ||
|
|
24e57457b0 | ||
|
|
43fe5e0373 | ||
|
|
4ee28fee4f | ||
|
|
5ebef57160 | ||
|
|
87fd3d8c04 | ||
|
|
e9860c12c6 | ||
|
|
825fff8b67 | ||
|
|
754375fb08 | ||
|
|
8a90713122 | ||
|
|
895f63a81c | ||
|
|
018b0f7686 | ||
|
|
cec3466779 |
@@ -0,0 +1,89 @@
|
|||||||
|
---
|
||||||
|
name: code-review
|
||||||
|
description: Review the changes since a fixed point (commit, branch, tag, or merge-base) along two axes — Standards (does the code follow this repo's documented coding standards?) and Spec (does the code match what the originating issue/PRD asked for?). Runs both reviews in parallel sub-agents and reports them side by side. Use when the user wants to review a branch, a PR, work-in-progress changes, or asks to "review since X".
|
||||||
|
---
|
||||||
|
|
||||||
|
Two-axis review of the diff between `HEAD` and a fixed point the user supplies:
|
||||||
|
|
||||||
|
- **Standards** — does the code conform to this repo's documented coding standards?
|
||||||
|
- **Spec** — does the code faithfully implement the originating issue / PRD / spec?
|
||||||
|
|
||||||
|
Both axes run as **parallel sub-agents** so they don't pollute each other's context, then this skill aggregates their findings.
|
||||||
|
|
||||||
|
The issue tracker should have been provided to you — run `/setup-matt-pocock-skills` if `docs/agents/issue-tracker.md` is missing.
|
||||||
|
|
||||||
|
## Process
|
||||||
|
|
||||||
|
### 1. Pin the fixed point
|
||||||
|
|
||||||
|
Whatever the user said is the fixed point — a commit SHA, branch name, tag, `main`, `HEAD~5`, etc. If they didn't specify one, ask for it.
|
||||||
|
|
||||||
|
Capture the diff command once: `git diff <fixed-point>...HEAD` (three-dot, so the comparison is against the merge-base). Also note the list of commits via `git log <fixed-point>..HEAD --oneline`.
|
||||||
|
|
||||||
|
Before going further, confirm the fixed point resolves (`git rev-parse <fixed-point>`) and the diff is non-empty. A bad ref or empty diff should fail here — not inside two parallel sub-agents.
|
||||||
|
|
||||||
|
### 2. Identify the spec source
|
||||||
|
|
||||||
|
Look for the originating spec, in this order:
|
||||||
|
|
||||||
|
1. Issue references in the commit messages (`#123`, `Closes #45`, GitLab `!67`, etc.) — fetch via the workflow in `docs/agents/issue-tracker.md`.
|
||||||
|
2. A path the user passed as an argument.
|
||||||
|
3. A PRD/spec file under `docs/`, `specs/`, or `.scratch/` matching the branch name or feature.
|
||||||
|
4. If nothing is found, ask the user where the spec is. If they say there isn't one, the **Spec** sub-agent will skip and report "no spec available".
|
||||||
|
|
||||||
|
### 3. Identify the standards sources
|
||||||
|
|
||||||
|
Anything in the repo that documents how code should be written, such as `CODING_STANDARDS.md` or `CONTRIBUTING.md`.
|
||||||
|
|
||||||
|
On top of whatever the repo documents, the Standards axis always carries the **smell baseline** below — a fixed set of Fowler code smells (_Refactoring_, ch.3) that applies even when a repo documents nothing. Two rules bind it:
|
||||||
|
|
||||||
|
- **The repo overrides.** A documented repo standard always wins; where it endorses something the baseline would flag, suppress the smell.
|
||||||
|
- **Always a judgement call.** Each smell is a labelled heuristic ("possible Feature Envy"), never a hard violation — and, like any standard here, skip anything tooling already enforces.
|
||||||
|
|
||||||
|
Each smell reads *what it is* → *how to fix*; match it against the diff:
|
||||||
|
|
||||||
|
- **Mysterious Name** — a function, variable, or type whose name doesn't reveal what it does or holds. → rename it; if no honest name comes, the design's murky.
|
||||||
|
- **Duplicated Code** — the same logic shape appears in more than one hunk or file in the change. → extract the shared shape, call it from both.
|
||||||
|
- **Feature Envy** — a method that reaches into another object's data more than its own. → move the method onto the data it envies.
|
||||||
|
- **Data Clumps** — the same few fields or params keep travelling together (a type wanting to be born). → bundle them into one type, pass that.
|
||||||
|
- **Primitive Obsession** — a primitive or string standing in for a domain concept that deserves its own type. → give the concept its own small type.
|
||||||
|
- **Repeated Switches** — the same `switch`/`if`-cascade on the same type recurs across the change. → replace with polymorphism, or one map both sites share.
|
||||||
|
- **Shotgun Surgery** — one logical change forces scattered edits across many files in the diff. → gather what changes together into one module.
|
||||||
|
- **Divergent Change** — one file or module is edited for several unrelated reasons. → split so each module changes for one reason.
|
||||||
|
- **Speculative Generality** — abstraction, parameters, or hooks added for needs the spec doesn't have. → delete it; inline back until a real need shows.
|
||||||
|
- **Message Chains** — long `a.b().c().d()` navigation the caller shouldn't depend on. → hide the walk behind one method on the first object.
|
||||||
|
- **Middle Man** — a class or function that mostly just delegates onward. → cut it, call the real target direct.
|
||||||
|
- **Refused Bequest** — a subclass or implementer that ignores or overrides most of what it inherits. → drop the inheritance, use composition.
|
||||||
|
|
||||||
|
### 4. Spawn both sub-agents in parallel
|
||||||
|
|
||||||
|
Send a single message with two `Agent` tool calls. Use the `general-purpose` subagent for both.
|
||||||
|
|
||||||
|
**Standards sub-agent prompt** — include:
|
||||||
|
|
||||||
|
- The full diff command and commit list.
|
||||||
|
- The list of standards-source files you found in step 3, **plus the smell baseline from step 3** pasted in full — the sub-agent has no other access to it.
|
||||||
|
- The brief: "Report — per file/hunk where relevant — (a) every place the diff violates a documented standard: cite the standard (file + the rule); and (b) any baseline smell you spot: name it and quote the hunk. Distinguish hard violations from judgement calls — documented-standard breaches can be hard, but baseline smells are always judgement calls, and a documented repo standard overrides the baseline. Skip anything tooling enforces. Under 400 words."
|
||||||
|
|
||||||
|
**Spec sub-agent prompt** — include:
|
||||||
|
|
||||||
|
- The diff command and commit list.
|
||||||
|
- The path or fetched contents of the spec.
|
||||||
|
- The brief: "Report: (a) requirements the spec asked for that are missing or partial; (b) behaviour in the diff that wasn't asked for (scope creep); (c) requirements that look implemented but where the implementation looks wrong. Quote the spec line for each finding. Under 400 words."
|
||||||
|
|
||||||
|
If the spec is missing, skip the Spec sub-agent and note this in the final report.
|
||||||
|
|
||||||
|
### 5. Aggregate
|
||||||
|
|
||||||
|
Present the two reports under `## Standards` and `## Spec` headings, verbatim or lightly cleaned. Do **not** merge or rerank findings — the two axes are deliberately separate (see _Why two axes_).
|
||||||
|
|
||||||
|
End with a one-line summary: total findings per axis, and the worst issue _within each axis_ (if any). Don't pick a single winner across axes — that's the reranking the separation exists to prevent.
|
||||||
|
|
||||||
|
## Why two axes
|
||||||
|
|
||||||
|
A change can pass one axis and fail the other:
|
||||||
|
|
||||||
|
- Code that follows every standard but implements the wrong thing → **Standards pass, Spec fail.**
|
||||||
|
- Code that does exactly what the issue asked but breaks the project's conventions → **Spec pass, Standards fail.**
|
||||||
|
|
||||||
|
Reporting them separately stops one axis from masking the other.
|
||||||
@@ -0,0 +1,37 @@
|
|||||||
|
# Deepening
|
||||||
|
|
||||||
|
How to deepen a cluster of shallow modules safely, given its dependencies. Assumes the vocabulary in [SKILL.md](SKILL.md) — **module**, **interface**, **seam**, **adapter**.
|
||||||
|
|
||||||
|
## Dependency categories
|
||||||
|
|
||||||
|
When assessing a candidate for deepening, classify its dependencies. The category determines how the deepened module is tested across its seam.
|
||||||
|
|
||||||
|
### 1. In-process
|
||||||
|
|
||||||
|
Pure computation, in-memory state, no I/O. Always deepenable — merge the modules and test through the new interface directly. No adapter needed.
|
||||||
|
|
||||||
|
### 2. Local-substitutable
|
||||||
|
|
||||||
|
Dependencies that have local test stand-ins (PGLite for Postgres, in-memory filesystem). Deepenable if the stand-in exists. The deepened module is tested with the stand-in running in the test suite. The seam is internal; no port at the module's external interface.
|
||||||
|
|
||||||
|
### 3. Remote but owned (Ports & Adapters)
|
||||||
|
|
||||||
|
Your own services across a network boundary (microservices, internal APIs). Define a **port** (interface) at the seam. The deep module owns the logic; the transport is injected as an **adapter**. Tests use an in-memory adapter. Production uses an HTTP/gRPC/queue adapter.
|
||||||
|
|
||||||
|
Recommendation shape: *"Define a port at the seam, implement an HTTP adapter for production and an in-memory adapter for testing, so the logic sits in one deep module even though it's deployed across a network."*
|
||||||
|
|
||||||
|
### 4. True external (Mock)
|
||||||
|
|
||||||
|
Third-party services (Stripe, Twilio, etc.) you don't control. The deepened module takes the external dependency as an injected port; tests provide a mock adapter.
|
||||||
|
|
||||||
|
## Seam discipline
|
||||||
|
|
||||||
|
- **One adapter means a hypothetical seam. Two adapters means a real one.** Don't introduce a port unless at least two adapters are justified (typically production + test). A single-adapter seam is just indirection.
|
||||||
|
- **Internal seams vs external seams.** A deep module can have internal seams (private to its implementation, used by its own tests) as well as the external seam at its interface. Don't expose internal seams through the interface just because tests use them.
|
||||||
|
|
||||||
|
## Testing strategy: replace, don't layer
|
||||||
|
|
||||||
|
- Old unit tests on shallow modules become waste once tests at the deepened module's interface exist — delete them.
|
||||||
|
- Write new tests at the deepened module's interface. The **interface is the test surface**.
|
||||||
|
- Tests assert on observable outcomes through the interface, not internal state.
|
||||||
|
- Tests should survive internal refactors — they describe behaviour, not implementation. If a test has to change when the implementation changes, it's testing past the interface.
|
||||||
@@ -0,0 +1,44 @@
|
|||||||
|
# Design It Twice
|
||||||
|
|
||||||
|
When the user wants to explore alternative interfaces for a chosen deepening candidate, use this parallel sub-agent pattern. Based on "Design It Twice" (Ousterhout) — your first idea is unlikely to be the best.
|
||||||
|
|
||||||
|
Uses the vocabulary in [SKILL.md](SKILL.md) — **module**, **interface**, **seam**, **adapter**, **leverage**.
|
||||||
|
|
||||||
|
## Process
|
||||||
|
|
||||||
|
### 1. Frame the problem space
|
||||||
|
|
||||||
|
Before spawning sub-agents, write a user-facing explanation of the problem space for the chosen candidate:
|
||||||
|
|
||||||
|
- The constraints any new interface would need to satisfy
|
||||||
|
- The dependencies it would rely on, and which category they fall into (see [DEEPENING.md](DEEPENING.md))
|
||||||
|
- A rough illustrative code sketch to ground the constraints — not a proposal, just a way to make the constraints concrete
|
||||||
|
|
||||||
|
Show this to the user, then immediately proceed to Step 2. The user reads and thinks while the sub-agents work in parallel.
|
||||||
|
|
||||||
|
### 2. Spawn sub-agents
|
||||||
|
|
||||||
|
Spawn 3+ sub-agents in parallel using the Agent tool. Each must produce a **radically different** interface for the deepened module.
|
||||||
|
|
||||||
|
Prompt each sub-agent with a separate technical brief (file paths, coupling details, dependency category from [DEEPENING.md](DEEPENING.md), what sits behind the seam). The brief is independent of the user-facing problem-space explanation in Step 1. Give each agent a different design constraint:
|
||||||
|
|
||||||
|
- Agent 1: "Minimize the interface — aim for 1–3 entry points max. Maximise leverage per entry point."
|
||||||
|
- Agent 2: "Maximise flexibility — support many use cases and extension."
|
||||||
|
- Agent 3: "Optimise for the most common caller — make the default case trivial."
|
||||||
|
- Agent 4 (if applicable): "Design around ports & adapters for cross-seam dependencies."
|
||||||
|
|
||||||
|
Include both [SKILL.md](SKILL.md) vocabulary and CONTEXT.md vocabulary in the brief so each sub-agent names things consistently with the architecture language and the project's domain language.
|
||||||
|
|
||||||
|
Each sub-agent outputs:
|
||||||
|
|
||||||
|
1. Interface (types, methods, params — plus invariants, ordering, error modes)
|
||||||
|
2. Usage example showing how callers use it
|
||||||
|
3. What the implementation hides behind the seam
|
||||||
|
4. Dependency strategy and adapters (see [DEEPENING.md](DEEPENING.md))
|
||||||
|
5. Trade-offs — where leverage is high, where it's thin
|
||||||
|
|
||||||
|
### 3. Present and compare
|
||||||
|
|
||||||
|
Present designs sequentially so the user can absorb each one, then compare them in prose. Contrast by **depth** (leverage at the interface), **locality** (where change concentrates), and **seam placement**.
|
||||||
|
|
||||||
|
After comparing, give your own recommendation: which design you think is strongest and why. If elements from different designs would combine well, propose a hybrid. Be opinionated — the user wants a strong read, not a menu.
|
||||||
@@ -0,0 +1,114 @@
|
|||||||
|
---
|
||||||
|
name: codebase-design
|
||||||
|
description: Shared vocabulary for designing deep modules. Use when the user wants to design or improve a module's interface, find deepening opportunities, decide where a seam goes, make code more testable or AI-navigable, or when another skill needs the deep-module vocabulary.
|
||||||
|
---
|
||||||
|
|
||||||
|
# Codebase Design
|
||||||
|
|
||||||
|
Design **deep modules**: a lot of behaviour behind a small interface, placed at a clean seam, testable through that interface. Use this language and these principles wherever code is being designed or restructured. The aim is leverage for callers, locality for maintainers, and testability for everyone.
|
||||||
|
|
||||||
|
## Glossary
|
||||||
|
|
||||||
|
Use these terms exactly — don't substitute "component," "service," "API," or "boundary." Consistent language is the whole point.
|
||||||
|
|
||||||
|
**Module** — anything with an interface and an implementation. Deliberately scale-agnostic: a function, class, package, or tier-spanning slice. _Avoid_: unit, component, service.
|
||||||
|
|
||||||
|
**Interface** — everything a caller must know to use the module correctly: the type signature, but also invariants, ordering constraints, error modes, required configuration, and performance characteristics. _Avoid_: API, signature (too narrow — they refer only to the type-level surface).
|
||||||
|
|
||||||
|
**Implementation** — what's inside a module, its body of code. Distinct from **Adapter**: a thing can be a small adapter with a large implementation (a Postgres repo) or a large adapter with a small implementation (an in-memory fake). Reach for "adapter" when the seam is the topic; "implementation" otherwise.
|
||||||
|
|
||||||
|
**Depth** — leverage at the interface: the amount of behaviour a caller (or test) can exercise per unit of interface they have to learn. A module is **deep** when a large amount of behaviour sits behind a small interface, **shallow** when the interface is nearly as complex as the implementation.
|
||||||
|
|
||||||
|
**Seam** _(Michael Feathers)_ — a place where you can alter behaviour without editing in that place; the *location* at which a module's interface lives. Where to put the seam is its own design decision, distinct from what goes behind it. _Avoid_: boundary (overloaded with DDD's bounded context).
|
||||||
|
|
||||||
|
**Adapter** — a concrete thing that satisfies an interface at a seam. Describes *role* (what slot it fills), not substance (what's inside).
|
||||||
|
|
||||||
|
**Leverage** — what callers get from depth: more capability per unit of interface they learn. One implementation pays back across N call sites and M tests.
|
||||||
|
|
||||||
|
**Locality** — what maintainers get from depth: change, bugs, knowledge, and verification concentrate in one place rather than spreading across callers. Fix once, fixed everywhere.
|
||||||
|
|
||||||
|
## Deep vs shallow
|
||||||
|
|
||||||
|
**Deep module** = small interface + lots of implementation:
|
||||||
|
|
||||||
|
```
|
||||||
|
┌─────────────────────┐
|
||||||
|
│ Small Interface │ ← Few methods, simple params
|
||||||
|
├─────────────────────┤
|
||||||
|
│ │
|
||||||
|
│ Deep Implementation│ ← Complex logic hidden
|
||||||
|
│ │
|
||||||
|
└─────────────────────┘
|
||||||
|
```
|
||||||
|
|
||||||
|
**Shallow module** = large interface + little implementation (avoid):
|
||||||
|
|
||||||
|
```
|
||||||
|
┌─────────────────────────────────┐
|
||||||
|
│ Large Interface │ ← Many methods, complex params
|
||||||
|
├─────────────────────────────────┤
|
||||||
|
│ Thin Implementation │ ← Just passes through
|
||||||
|
└─────────────────────────────────┘
|
||||||
|
```
|
||||||
|
|
||||||
|
When designing an interface, ask:
|
||||||
|
|
||||||
|
- Can I reduce the number of methods?
|
||||||
|
- Can I simplify the parameters?
|
||||||
|
- Can I hide more complexity inside?
|
||||||
|
|
||||||
|
## Principles
|
||||||
|
|
||||||
|
- **Depth is a property of the interface, not the implementation.** A deep module can be internally composed of small, mockable, swappable parts — they just aren't part of the interface. A module can have **internal seams** (private to its implementation, used by its own tests) as well as the **external seam** at its interface.
|
||||||
|
- **The deletion test.** Imagine deleting the module. If complexity vanishes, it was a pass-through. If complexity reappears across N callers, it was earning its keep.
|
||||||
|
- **The interface is the test surface.** Callers and tests cross the same seam. If you want to test *past* the interface, the module is probably the wrong shape.
|
||||||
|
- **One adapter means a hypothetical seam. Two adapters means a real one.** Don't introduce a seam unless something actually varies across it.
|
||||||
|
|
||||||
|
## Designing for testability
|
||||||
|
|
||||||
|
Good interfaces make testing natural:
|
||||||
|
|
||||||
|
1. **Accept dependencies, don't create them.**
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// Testable
|
||||||
|
function processOrder(order, paymentGateway) {}
|
||||||
|
|
||||||
|
// Hard to test
|
||||||
|
function processOrder(order) {
|
||||||
|
const gateway = new StripeGateway();
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
2. **Return results, don't produce side effects.**
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// Testable
|
||||||
|
function calculateDiscount(cart): Discount {}
|
||||||
|
|
||||||
|
// Hard to test
|
||||||
|
function applyDiscount(cart): void {
|
||||||
|
cart.total -= discount;
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
3. **Small surface area.** Fewer methods = fewer tests needed. Fewer params = simpler test setup.
|
||||||
|
|
||||||
|
## Relationships
|
||||||
|
|
||||||
|
- A **Module** has exactly one **Interface** (the surface it presents to callers and tests).
|
||||||
|
- **Depth** is a property of a **Module**, measured against its **Interface**.
|
||||||
|
- A **Seam** is where a **Module**'s **Interface** lives.
|
||||||
|
- An **Adapter** sits at a **Seam** and satisfies the **Interface**.
|
||||||
|
- **Depth** produces **Leverage** for callers and **Locality** for maintainers.
|
||||||
|
|
||||||
|
## Rejected framings
|
||||||
|
|
||||||
|
- **Depth as ratio of implementation-lines to interface-lines** (Ousterhout): rewards padding the implementation. We use depth-as-leverage instead.
|
||||||
|
- **"Interface" as the TypeScript `interface` keyword or a class's public methods**: too narrow — interface here includes every fact a caller must know.
|
||||||
|
- **"Boundary"**: overloaded with DDD's bounded context. Say **seam** or **interface**.
|
||||||
|
|
||||||
|
## Going deeper
|
||||||
|
|
||||||
|
- **Deepening a cluster given its dependencies** — see [DEEPENING.md](DEEPENING.md): dependency categories, seam discipline, and replace-don't-layer testing.
|
||||||
|
- **Exploring alternative interfaces** — see [DESIGN-IT-TWICE.md](DESIGN-IT-TWICE.md): spin up parallel sub-agents to design the interface several radically different ways, then compare on depth, locality, and seam placement.
|
||||||
@@ -0,0 +1,55 @@
|
|||||||
|
---
|
||||||
|
name: data-driven-design
|
||||||
|
description: Use when adding a mode, switch case, config entry, special-cased name, or per-column/per-dataset handler to a tool that processes authored data — or when a tool "knows" a project's layout, dataset names, key prefixes, or format quirks and could not be open sourced as-is.
|
||||||
|
---
|
||||||
|
|
||||||
|
# Data-Driven Design
|
||||||
|
|
||||||
|
The tool implements **general mechanisms**; the data declares **all specific knowledge**. Every time code names a thing that lives in data — a dataset, a column, a format quirk — the tool stops being a tool and becomes a private extension of one project's layout.
|
||||||
|
|
||||||
|
**The test:** could this repo be open sourced and build a *different* project's data without code changes? Every hardcoded name, mode, and special case is a "no".
|
||||||
|
|
||||||
|
## The two smells
|
||||||
|
|
||||||
|
**1. The mode zoo.** A new data shape arrives and you add a named mode plus a switch case:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
- {column: AvailableHeadsMale, mode: external_hex_list, hex_width: 3}
|
||||||
|
- {column: PreferredAlignments, mode: alignment_set_mask, hex_width: 3}
|
||||||
|
- {column: ProficiencyFeats, mode: id_hex_list, hex_width: 4}
|
||||||
|
```
|
||||||
|
|
||||||
|
Three "modes" are one concept — *a set of ids with an engine-side encoding* — wearing three names. Each mode is code the data could have been. The fix is a smaller vocabulary of orthogonal primitives declared where the column is defined:
|
||||||
|
|
||||||
|
```json
|
||||||
|
"EquipSlots": {"shape": "id_set", "encode": "bitmask", "max": 31}
|
||||||
|
```
|
||||||
|
|
||||||
|
One parser for `id_set`, one encoder per encoding. Adding the next column is a data edit, zero code. The mode zoo *shrinks*: N special cases collapse into shape × encoding.
|
||||||
|
|
||||||
|
**2. Names in code.** `datasetRows("skills")`, `case "skills":`, `TrimPrefix(key, "skills:")` — the tool assumes the project's layout. When data moves (`skills` → `skills/core`), the tool breaks even though the data is self-consistent. Names must flow *from* the data: the spec that references a dataset names it; a key's namespace is whatever precedes its own colon. If code needs a name, some piece of data already knows it — read it from there.
|
||||||
|
|
||||||
|
## Deciding
|
||||||
|
|
||||||
|
Ask of every constant, case, and config knob: **whose knowledge is this?**
|
||||||
|
|
||||||
|
- *The mechanism's* (how to parse JSON, allocate rows, write a table) → code.
|
||||||
|
- *The data's* (which columns exist, how the engine encodes them, what things are called) → data, even if code is faster to write today.
|
||||||
|
|
||||||
|
Generalize only when it deletes: a primitive that replaces N modes is lazy; a framework "for future shapes" is not. One new shape may take its case *if* the case is spelled as a reusable primitive the next shape can declare.
|
||||||
|
|
||||||
|
## Rationalizations
|
||||||
|
|
||||||
|
| Excuse | Reality |
|
||||||
|
|---|---|
|
||||||
|
| "Just follow the existing pattern, it's proven" | The pattern *is* the defect. Each repetition raises the cost of the real fix. |
|
||||||
|
| "N cases isn't a crisis yet" | The crisis is per-case: every one couples the tool to one project forever. |
|
||||||
|
| "Release is tonight, generalize later" | Later never comes; tonight's mode is tomorrow's baseline. Declaring shape in data is usually the *same* hour of work. |
|
||||||
|
| "This name never changes" | Today's rename broke exactly such a name. Data moves; mechanisms shouldn't care. |
|
||||||
|
|
||||||
|
## Red flags — stop and re-shape
|
||||||
|
|
||||||
|
- A config entry that names both a dataset and a column to select behavior
|
||||||
|
- A new value in a `mode`/`kind`/`type` string enum with its own switch arm
|
||||||
|
- A literal in code that also appears in the data tree (`"skills"`, `"skills:"`)
|
||||||
|
- Per-project constants (row ranges, hex widths, id ceilings) living in Go instead of the dataset that owns them
|
||||||
@@ -0,0 +1,134 @@
|
|||||||
|
---
|
||||||
|
name: diagnosing-bugs
|
||||||
|
description: Diagnosis loop for hard bugs and performance regressions. Use when the user says "diagnose"/"debug this", or reports something broken/throwing/failing/slow.
|
||||||
|
---
|
||||||
|
|
||||||
|
# Diagnosing Bugs
|
||||||
|
|
||||||
|
A discipline for hard bugs. Skip phases only when explicitly justified.
|
||||||
|
|
||||||
|
When exploring the codebase, read `CONTEXT.md` (if it exists) to get a clear mental model of the relevant modules, and check ADRs in the area you're touching.
|
||||||
|
|
||||||
|
## Phase 1 — Build a feedback loop
|
||||||
|
|
||||||
|
**This is the skill.** Everything else is mechanical. If you have a **tight** pass/fail signal for the bug — one that goes red on _this_ bug — you will find the cause; bisection, hypothesis-testing, and instrumentation all just consume it. If you don't have one, no amount of staring at code will save you.
|
||||||
|
|
||||||
|
Spend disproportionate effort here. **Be aggressive. Be creative. Refuse to give up.**
|
||||||
|
|
||||||
|
### Ways to construct one — try them in roughly this order
|
||||||
|
|
||||||
|
1. **Failing test** at whatever seam reaches the bug — unit, integration, e2e.
|
||||||
|
2. **Curl / HTTP script** against a running dev server.
|
||||||
|
3. **CLI invocation** with a fixture input, diffing stdout against a known-good snapshot.
|
||||||
|
4. **Headless browser script** (Playwright / Puppeteer) — drives the UI, asserts on DOM/console/network.
|
||||||
|
5. **Replay a captured trace.** Save a real network request / payload / event log to disk; replay it through the code path in isolation.
|
||||||
|
6. **Throwaway harness.** Spin up a minimal subset of the system (one service, mocked deps) that exercises the bug code path with a single function call.
|
||||||
|
7. **Property / fuzz loop.** If the bug is "sometimes wrong output", run 1000 random inputs and look for the failure mode.
|
||||||
|
8. **Bisection harness.** If the bug appeared between two known states (commit, dataset, version), automate "boot at state X, check, repeat" so you can `git bisect run` it.
|
||||||
|
9. **Differential loop.** Run the same input through old-version vs new-version (or two configs) and diff outputs.
|
||||||
|
10. **HITL bash script.** Last resort. If a human must click, drive _them_ with `scripts/hitl-loop.template.sh` so the loop is still structured. Captured output feeds back to you.
|
||||||
|
|
||||||
|
Build the right feedback loop, and the bug is 90% fixed.
|
||||||
|
|
||||||
|
### Tighten the loop
|
||||||
|
|
||||||
|
Treat the loop as a product. Once you have _a_ loop, **tighten** it:
|
||||||
|
|
||||||
|
- Can I make it faster? (Cache setup, skip unrelated init, narrow the test scope.)
|
||||||
|
- Can I make the signal sharper? (Assert on the specific symptom, not "didn't crash".)
|
||||||
|
- Can I make it more deterministic? (Pin time, seed RNG, isolate filesystem, freeze network.)
|
||||||
|
|
||||||
|
A 30-second flaky loop is barely better than no loop; a 2-second deterministic one is tight — a debugging superpower.
|
||||||
|
|
||||||
|
### Non-deterministic bugs
|
||||||
|
|
||||||
|
The goal is not a clean repro but a **higher reproduction rate**. Loop the trigger 100×, parallelise, add stress, narrow timing windows, inject sleeps. A 50%-flake bug is debuggable; 1% is not — keep raising the rate until it's debuggable.
|
||||||
|
|
||||||
|
### When you genuinely cannot build a loop
|
||||||
|
|
||||||
|
Stop and say so explicitly. List what you tried. Ask the user for: (a) access to whatever environment reproduces it, (b) a captured artifact (HAR file, log dump, core dump, screen recording with timestamps), or (c) permission to add temporary production instrumentation. Do **not** proceed to hypothesise without a loop.
|
||||||
|
|
||||||
|
### Completion criterion — a tight loop that goes red
|
||||||
|
|
||||||
|
Phase 1 is done when the loop is **tight** and **red-capable**: you can name **one command** — a script path, a test invocation, a curl — that you have **already run at least once** (paste the invocation and its output), and that is:
|
||||||
|
|
||||||
|
- [ ] **Red-capable** — it drives the actual bug code path and asserts the **user's exact symptom**, so it can go red on this bug and green once fixed. Not "runs without erroring" — it must be able to _catch this specific bug_.
|
||||||
|
- [ ] **Deterministic** — same verdict every run (flaky bugs: a pinned, high reproduction rate, per above).
|
||||||
|
- [ ] **Fast** — seconds, not minutes.
|
||||||
|
- [ ] **Agent-runnable** — you can run it unattended; a human in the loop only via `scripts/hitl-loop.template.sh`.
|
||||||
|
|
||||||
|
If you catch yourself reading code to build a theory before this command exists, **stop — jumping straight to a hypothesis is the exact failure this skill prevents.** No red-capable command, no Phase 2.
|
||||||
|
|
||||||
|
## Phase 2 — Reproduce + minimise
|
||||||
|
|
||||||
|
Run the loop. Watch it go red — the bug appears.
|
||||||
|
|
||||||
|
Confirm:
|
||||||
|
|
||||||
|
- [ ] The loop produces the failure mode the **user** described — not a different failure that happens to be nearby. Wrong bug = wrong fix.
|
||||||
|
- [ ] The failure is reproducible across multiple runs (or, for non-deterministic bugs, reproducible at a high enough rate to debug against).
|
||||||
|
- [ ] You have captured the exact symptom (error message, wrong output, slow timing) so later phases can verify the fix actually addresses it.
|
||||||
|
|
||||||
|
### Minimise
|
||||||
|
|
||||||
|
Once it's red, shrink the repro to the **smallest scenario that still goes red**. Cut inputs, callers, config, data, and steps **one at a time**, re-running the loop after each cut — keep only what's load-bearing for the failure.
|
||||||
|
|
||||||
|
Why bother: a minimal repro shrinks the hypothesis space in Phase 3 (fewer moving parts left to suspect) and becomes the clean regression test in Phase 5.
|
||||||
|
|
||||||
|
Done when **every remaining element is load-bearing** — removing any one of them makes the loop go green.
|
||||||
|
|
||||||
|
Do not proceed until you have reproduced **and** minimised.
|
||||||
|
|
||||||
|
## Phase 3 — Hypothesise
|
||||||
|
|
||||||
|
Generate **3–5 ranked hypotheses** before testing any of them. Single-hypothesis generation anchors on the first plausible idea.
|
||||||
|
|
||||||
|
Each hypothesis must be **falsifiable**: state the prediction it makes.
|
||||||
|
|
||||||
|
> Format: "If <X> is the cause, then <changing Y> will make the bug disappear / <changing Z> will make it worse."
|
||||||
|
|
||||||
|
If you cannot state the prediction, the hypothesis is a vibe — discard or sharpen it.
|
||||||
|
|
||||||
|
**Show the ranked list to the user before testing.** They often have domain knowledge that re-ranks instantly ("we just deployed a change to #3"), or know hypotheses they've already ruled out. Cheap checkpoint, big time saver. Don't block on it — proceed with your ranking if the user is AFK.
|
||||||
|
|
||||||
|
## Phase 4 — Instrument
|
||||||
|
|
||||||
|
Each probe must map to a specific prediction from Phase 3. **Change one variable at a time.**
|
||||||
|
|
||||||
|
Tool preference:
|
||||||
|
|
||||||
|
1. **Debugger / REPL inspection** if the env supports it. One breakpoint beats ten logs.
|
||||||
|
2. **Targeted logs** at the boundaries that distinguish hypotheses.
|
||||||
|
3. Never "log everything and grep".
|
||||||
|
|
||||||
|
**Tag every debug log** with a unique prefix, e.g. `[DEBUG-a4f2]`. Cleanup at the end becomes a single grep. Untagged logs survive; tagged logs die.
|
||||||
|
|
||||||
|
**Perf branch.** For performance regressions, logs are usually wrong. Instead: establish a baseline measurement (timing harness, `performance.now()`, profiler, query plan), then bisect. Measure first, fix second.
|
||||||
|
|
||||||
|
## Phase 5 — Fix + regression test
|
||||||
|
|
||||||
|
Write the regression test **before the fix** — but only if there is a **correct seam** for it.
|
||||||
|
|
||||||
|
A correct seam is one where the test exercises the **real bug pattern** as it occurs at the call site. If the only available seam is too shallow (single-caller test when the bug needs multiple callers, unit test that can't replicate the chain that triggered the bug), a regression test there gives false confidence.
|
||||||
|
|
||||||
|
**If no correct seam exists, that itself is the finding.** Note it. The codebase architecture is preventing the bug from being locked down. Flag this for the next phase.
|
||||||
|
|
||||||
|
If a correct seam exists:
|
||||||
|
|
||||||
|
1. Turn the minimised repro into a failing test at that seam.
|
||||||
|
2. Watch it fail.
|
||||||
|
3. Apply the fix.
|
||||||
|
4. Watch it pass.
|
||||||
|
5. Re-run the Phase 1 feedback loop against the original (un-minimised) scenario.
|
||||||
|
|
||||||
|
## Phase 6 — Cleanup + post-mortem
|
||||||
|
|
||||||
|
Required before declaring done:
|
||||||
|
|
||||||
|
- [ ] Original repro no longer reproduces (re-run the Phase 1 loop)
|
||||||
|
- [ ] Regression test passes (or absence of seam is documented)
|
||||||
|
- [ ] All `[DEBUG-...]` instrumentation removed (`grep` the prefix)
|
||||||
|
- [ ] Throwaway prototypes deleted (or moved to a clearly-marked debug location)
|
||||||
|
- [ ] The hypothesis that turned out correct is stated in the commit / PR message — so the next debugger learns
|
||||||
|
|
||||||
|
**Then ask: what would have prevented this bug?** If the answer involves architectural change (no good test seam, tangled callers, hidden coupling) hand off to the `/improve-codebase-architecture` skill with the specifics. Make the recommendation **after** the fix is in, not before — you have more information now than when you started.
|
||||||
@@ -0,0 +1,41 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# Human-in-the-loop reproduction loop.
|
||||||
|
# Copy this file, edit the steps below, and run it.
|
||||||
|
# The agent runs the script; the user follows prompts in their terminal.
|
||||||
|
#
|
||||||
|
# Usage:
|
||||||
|
# bash hitl-loop.template.sh
|
||||||
|
#
|
||||||
|
# Two helpers:
|
||||||
|
# step "<instruction>" → show instruction, wait for Enter
|
||||||
|
# capture VAR "<question>" → show question, read response into VAR
|
||||||
|
#
|
||||||
|
# At the end, captured values are printed as KEY=VALUE for the agent to parse.
|
||||||
|
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
step() {
|
||||||
|
printf '\n>>> %s\n' "$1"
|
||||||
|
read -r -p " [Enter when done] " _
|
||||||
|
}
|
||||||
|
|
||||||
|
capture() {
|
||||||
|
local var="$1" question="$2" answer
|
||||||
|
printf '\n>>> %s\n' "$question"
|
||||||
|
read -r -p " > " answer
|
||||||
|
printf -v "$var" '%s' "$answer"
|
||||||
|
}
|
||||||
|
|
||||||
|
# --- edit below ---------------------------------------------------------
|
||||||
|
|
||||||
|
step "Open the app at http://localhost:3000 and sign in."
|
||||||
|
|
||||||
|
capture ERRORED "Click the 'Export' button. Did it throw an error? (y/n)"
|
||||||
|
|
||||||
|
capture ERROR_MSG "Paste the error message (or 'none'):"
|
||||||
|
|
||||||
|
# --- edit above ---------------------------------------------------------
|
||||||
|
|
||||||
|
printf '\n--- Captured ---\n'
|
||||||
|
printf 'ERRORED=%s\n' "$ERRORED"
|
||||||
|
printf 'ERROR_MSG=%s\n' "$ERROR_MSG"
|
||||||
@@ -0,0 +1,47 @@
|
|||||||
|
# ADR Format
|
||||||
|
|
||||||
|
ADRs live in `docs/adr/` and use sequential numbering: `0001-slug.md`, `0002-slug.md`, etc.
|
||||||
|
|
||||||
|
Create the `docs/adr/` directory lazily — only when the first ADR is needed.
|
||||||
|
|
||||||
|
## Template
|
||||||
|
|
||||||
|
```md
|
||||||
|
# {Short title of the decision}
|
||||||
|
|
||||||
|
{1-3 sentences: what's the context, what did we decide, and why.}
|
||||||
|
```
|
||||||
|
|
||||||
|
That's it. An ADR can be a single paragraph. The value is in recording *that* a decision was made and *why* — not in filling out sections.
|
||||||
|
|
||||||
|
## Optional sections
|
||||||
|
|
||||||
|
Only include these when they add genuine value. Most ADRs won't need them.
|
||||||
|
|
||||||
|
- **Status** frontmatter (`proposed | accepted | deprecated | superseded by ADR-NNNN`) — useful when decisions are revisited
|
||||||
|
- **Considered Options** — only when the rejected alternatives are worth remembering
|
||||||
|
- **Consequences** — only when non-obvious downstream effects need to be called out
|
||||||
|
|
||||||
|
## Numbering
|
||||||
|
|
||||||
|
Scan `docs/adr/` for the highest existing number and increment by one.
|
||||||
|
|
||||||
|
## When to offer an ADR
|
||||||
|
|
||||||
|
All three of these must be true:
|
||||||
|
|
||||||
|
1. **Hard to reverse** — the cost of changing your mind later is meaningful
|
||||||
|
2. **Surprising without context** — a future reader will look at the code and wonder "why on earth did they do it this way?"
|
||||||
|
3. **The result of a real trade-off** — there were genuine alternatives and you picked one for specific reasons
|
||||||
|
|
||||||
|
If a decision is easy to reverse, skip it — you'll just reverse it. If it's not surprising, nobody will wonder why. If there was no real alternative, there's nothing to record beyond "we did the obvious thing."
|
||||||
|
|
||||||
|
### What qualifies
|
||||||
|
|
||||||
|
- **Architectural shape.** "We're using a monorepo." "The write model is event-sourced, the read model is projected into Postgres."
|
||||||
|
- **Integration patterns between contexts.** "Ordering and Billing communicate via domain events, not synchronous HTTP."
|
||||||
|
- **Technology choices that carry lock-in.** Database, message bus, auth provider, deployment target. Not every library — just the ones that would take a quarter to swap out.
|
||||||
|
- **Boundary and scope decisions.** "Customer data is owned by the Customer context; other contexts reference it by ID only." The explicit no-s are as valuable as the yes-s.
|
||||||
|
- **Deliberate deviations from the obvious path.** "We're using manual SQL instead of an ORM because X." Anything where a reasonable reader would assume the opposite. These stop the next engineer from "fixing" something that was deliberate.
|
||||||
|
- **Constraints not visible in the code.** "We can't use AWS because of compliance requirements." "Response times must be under 200ms because of the partner API contract."
|
||||||
|
- **Rejected alternatives when the rejection is non-obvious.** If you considered GraphQL and picked REST for subtle reasons, record it — otherwise someone will suggest GraphQL again in six months.
|
||||||
@@ -0,0 +1,60 @@
|
|||||||
|
# CONTEXT.md Format
|
||||||
|
|
||||||
|
## Structure
|
||||||
|
|
||||||
|
```md
|
||||||
|
# {Context Name}
|
||||||
|
|
||||||
|
{One or two sentence description of what this context is and why it exists.}
|
||||||
|
|
||||||
|
## Language
|
||||||
|
|
||||||
|
**Order**:
|
||||||
|
{A one or two sentence description of the term}
|
||||||
|
_Avoid_: Purchase, transaction
|
||||||
|
|
||||||
|
**Invoice**:
|
||||||
|
A request for payment sent to a customer after delivery.
|
||||||
|
_Avoid_: Bill, payment request
|
||||||
|
|
||||||
|
**Customer**:
|
||||||
|
A person or organization that places orders.
|
||||||
|
_Avoid_: Client, buyer, account
|
||||||
|
```
|
||||||
|
|
||||||
|
## Rules
|
||||||
|
|
||||||
|
- **Be opinionated.** When multiple words exist for the same concept, pick the best one and list the others under `_Avoid_`.
|
||||||
|
- **Keep definitions tight.** One or two sentences max. Define what it IS, not what it does.
|
||||||
|
- **Only include terms specific to this project's context.** General programming concepts (timeouts, error types, utility patterns) don't belong even if the project uses them extensively. Before adding a term, ask: is this a concept unique to this context, or a general programming concept? Only the former belongs.
|
||||||
|
- **Group terms under subheadings** when natural clusters emerge. If all terms belong to a single cohesive area, a flat list is fine.
|
||||||
|
|
||||||
|
## Single vs multi-context repos
|
||||||
|
|
||||||
|
**Single context (most repos):** One `CONTEXT.md` at the repo root.
|
||||||
|
|
||||||
|
**Multiple contexts:** A `CONTEXT-MAP.md` at the repo root lists the contexts, where they live, and how they relate to each other:
|
||||||
|
|
||||||
|
```md
|
||||||
|
# Context Map
|
||||||
|
|
||||||
|
## Contexts
|
||||||
|
|
||||||
|
- [Ordering](./src/ordering/CONTEXT.md) — receives and tracks customer orders
|
||||||
|
- [Billing](./src/billing/CONTEXT.md) — generates invoices and processes payments
|
||||||
|
- [Fulfillment](./src/fulfillment/CONTEXT.md) — manages warehouse picking and shipping
|
||||||
|
|
||||||
|
## Relationships
|
||||||
|
|
||||||
|
- **Ordering → Fulfillment**: Ordering emits `OrderPlaced` events; Fulfillment consumes them to start picking
|
||||||
|
- **Fulfillment → Billing**: Fulfillment emits `ShipmentDispatched` events; Billing consumes them to generate invoices
|
||||||
|
- **Ordering ↔ Billing**: Shared types for `CustomerId` and `Money`
|
||||||
|
```
|
||||||
|
|
||||||
|
The skill infers which structure applies:
|
||||||
|
|
||||||
|
- If `CONTEXT-MAP.md` exists, read it to find contexts
|
||||||
|
- If only a root `CONTEXT.md` exists, single context
|
||||||
|
- If neither exists, create a root `CONTEXT.md` lazily when the first term is resolved
|
||||||
|
|
||||||
|
When multiple contexts exist, infer which one the current topic relates to. If unclear, ask.
|
||||||
@@ -0,0 +1,74 @@
|
|||||||
|
---
|
||||||
|
name: domain-modeling
|
||||||
|
description: Build and sharpen a project's domain model. Use when the user wants to pin down domain terminology or a ubiquitous language, record an architectural decision, or when another skill needs to maintain the domain model.
|
||||||
|
---
|
||||||
|
|
||||||
|
# Domain Modeling
|
||||||
|
|
||||||
|
Actively build and sharpen the project's domain model as you design. This is the *active* discipline — challenging terms, inventing edge-case scenarios, and writing the glossary and decisions down the moment they crystallise. (Merely *reading* `CONTEXT.md` for vocabulary is not this skill — that's a one-line habit any skill can do. This skill is for when you're changing the model, not just consuming it.)
|
||||||
|
|
||||||
|
## File structure
|
||||||
|
|
||||||
|
Most repos have a single context:
|
||||||
|
|
||||||
|
```
|
||||||
|
/
|
||||||
|
├── CONTEXT.md
|
||||||
|
├── docs/
|
||||||
|
│ └── adr/
|
||||||
|
│ ├── 0001-event-sourced-orders.md
|
||||||
|
│ └── 0002-postgres-for-write-model.md
|
||||||
|
└── src/
|
||||||
|
```
|
||||||
|
|
||||||
|
If a `CONTEXT-MAP.md` exists at the root, the repo has multiple contexts. The map points to where each one lives:
|
||||||
|
|
||||||
|
```
|
||||||
|
/
|
||||||
|
├── CONTEXT-MAP.md
|
||||||
|
├── docs/
|
||||||
|
│ └── adr/ ← system-wide decisions
|
||||||
|
├── src/
|
||||||
|
│ ├── ordering/
|
||||||
|
│ │ ├── CONTEXT.md
|
||||||
|
│ │ └── docs/adr/ ← context-specific decisions
|
||||||
|
│ └── billing/
|
||||||
|
│ ├── CONTEXT.md
|
||||||
|
│ └── docs/adr/
|
||||||
|
```
|
||||||
|
|
||||||
|
Create files lazily — only when you have something to write. If no `CONTEXT.md` exists, create one when the first term is resolved. If no `docs/adr/` exists, create it when the first ADR is needed.
|
||||||
|
|
||||||
|
## During the session
|
||||||
|
|
||||||
|
### Challenge against the glossary
|
||||||
|
|
||||||
|
When the user uses a term that conflicts with the existing language in `CONTEXT.md`, call it out immediately. "Your glossary defines 'cancellation' as X, but you seem to mean Y — which is it?"
|
||||||
|
|
||||||
|
### Sharpen fuzzy language
|
||||||
|
|
||||||
|
When the user uses vague or overloaded terms, propose a precise canonical term. "You're saying 'account' — do you mean the Customer or the User? Those are different things."
|
||||||
|
|
||||||
|
### Discuss concrete scenarios
|
||||||
|
|
||||||
|
When domain relationships are being discussed, stress-test them with specific scenarios. Invent scenarios that probe edge cases and force the user to be precise about the boundaries between concepts.
|
||||||
|
|
||||||
|
### Cross-reference with code
|
||||||
|
|
||||||
|
When the user states how something works, check whether the code agrees. If you find a contradiction, surface it: "Your code cancels entire Orders, but you just said partial cancellation is possible — which is right?"
|
||||||
|
|
||||||
|
### Update CONTEXT.md inline
|
||||||
|
|
||||||
|
When a term is resolved, update `CONTEXT.md` right there. Don't batch these up — capture them as they happen. Use the format in [CONTEXT-FORMAT.md](./CONTEXT-FORMAT.md).
|
||||||
|
|
||||||
|
`CONTEXT.md` should be totally devoid of implementation details. Do not treat `CONTEXT.md` as a spec, a scratch pad, or a repository for implementation decisions. It is a glossary and nothing else.
|
||||||
|
|
||||||
|
### Offer ADRs sparingly
|
||||||
|
|
||||||
|
Only offer to create an ADR when all three are true:
|
||||||
|
|
||||||
|
1. **Hard to reverse** — the cost of changing your mind later is meaningful
|
||||||
|
2. **Surprising without context** — a future reader will wonder "why did they do it this way?"
|
||||||
|
3. **The result of a real trade-off** — there were genuine alternatives and you picked one for specific reasons
|
||||||
|
|
||||||
|
If any of the three is missing, skip the ADR. Use the format in [ADR-FORMAT.md](./ADR-FORMAT.md).
|
||||||
@@ -0,0 +1,7 @@
|
|||||||
|
---
|
||||||
|
name: grill-me
|
||||||
|
description: A relentless interview to sharpen a plan or design.
|
||||||
|
disable-model-invocation: true
|
||||||
|
---
|
||||||
|
|
||||||
|
Run a `/grilling` session.
|
||||||
@@ -0,0 +1,12 @@
|
|||||||
|
---
|
||||||
|
name: grilling
|
||||||
|
description: Grill the user relentlessly about a plan or design. Use when the user wants to stress-test a plan before building, or uses any 'grill' trigger phrases.
|
||||||
|
---
|
||||||
|
|
||||||
|
Interview me relentlessly about every aspect of this plan until we reach a shared understanding. Walk down each branch of the design tree, resolving dependencies between decisions one-by-one. For each question, provide your recommended answer.
|
||||||
|
|
||||||
|
Ask the questions one at a time, waiting for feedback on each question before continuing. Asking multiple questions at once is bewildering.
|
||||||
|
|
||||||
|
If a question can be answered by exploring the codebase, explore the codebase instead.
|
||||||
|
|
||||||
|
Do not enact the plan until I confirm we have reached a shared understanding.
|
||||||
@@ -0,0 +1,16 @@
|
|||||||
|
---
|
||||||
|
name: handoff
|
||||||
|
description: Compact the current conversation into a handoff document for another agent to pick up.
|
||||||
|
argument-hint: "What will the next session be used for?"
|
||||||
|
disable-model-invocation: true
|
||||||
|
---
|
||||||
|
|
||||||
|
Write a handoff document summarising the current conversation so a fresh agent can continue the work. Save to the temporary directory of the user's OS - not the current workspace.
|
||||||
|
|
||||||
|
Include a "suggested skills" section in the document, which suggests skills that the agent should invoke.
|
||||||
|
|
||||||
|
Do not duplicate content already captured in other artifacts (PRDs, plans, ADRs, issues, commits, diffs). Reference them by path or URL instead.
|
||||||
|
|
||||||
|
Redact any sensitive information, such as API keys, passwords, or personally identifiable information.
|
||||||
|
|
||||||
|
If the user passed arguments, treat them as a description of what the next session will focus on and tailor the doc accordingly.
|
||||||
@@ -0,0 +1,15 @@
|
|||||||
|
---
|
||||||
|
name: implement
|
||||||
|
description: "Implement a piece of work based on a PRD or set of issues."
|
||||||
|
disable-model-invocation: true
|
||||||
|
---
|
||||||
|
|
||||||
|
Implement the work described by the user in the PRD or issues.
|
||||||
|
|
||||||
|
Use /tdd where possible, at pre-agreed seams.
|
||||||
|
|
||||||
|
Run typechecking regularly, single test files regularly, and the full test suite once at the end.
|
||||||
|
|
||||||
|
Once done, use /code-review to review the work.
|
||||||
|
|
||||||
|
Commit your work to the current branch.
|
||||||
@@ -0,0 +1,12 @@
|
|||||||
|
---
|
||||||
|
name: research
|
||||||
|
description: Investigate a question against high-trust primary sources and capture the findings as a Markdown file in the repo. Use when the user wants a topic researched, docs or API facts gathered, or reading legwork delegated to a background agent.
|
||||||
|
---
|
||||||
|
|
||||||
|
Spin up a **background agent** to do the research, so you keep working while it reads.
|
||||||
|
|
||||||
|
Its job:
|
||||||
|
|
||||||
|
1. Investigate the question against **primary sources** — official docs, source code, specs, first-party APIs — not a secondary write-up of them. Follow every claim back to the source that owns it.
|
||||||
|
2. Write the findings to a single Markdown file, citing each claim's source.
|
||||||
|
3. Save it where the repo already keeps such notes; match the existing convention, and if there is none, put it somewhere sensible and say where.
|
||||||
@@ -0,0 +1,36 @@
|
|||||||
|
---
|
||||||
|
name: tdd
|
||||||
|
description: Test-driven development. Use when the user wants to build features or fix bugs test-first, mentions "red-green-refactor", or wants integration tests.
|
||||||
|
---
|
||||||
|
|
||||||
|
# Test-Driven Development
|
||||||
|
|
||||||
|
TDD is the red → green loop. This skill is the reference that makes that loop produce tests worth keeping: what a good test is, where tests go, the anti-patterns, and the rules of the loop. Every section applies on every cycle — consult them before and during the loop, not after.
|
||||||
|
|
||||||
|
When exploring the codebase, read `CONTEXT.md` (if it exists) so test names and interface vocabulary match the project's domain language, and respect ADRs in the area you're touching.
|
||||||
|
|
||||||
|
## What a good test is
|
||||||
|
|
||||||
|
Tests verify behavior through public interfaces, not implementation details. Code can change entirely; tests shouldn't. A good test reads like a specification — "user can checkout with valid cart" tells you exactly what capability exists — and survives refactors because it doesn't care about internal structure.
|
||||||
|
|
||||||
|
See [tests.md](tests.md) for examples and [mocking.md](mocking.md) for mocking guidelines.
|
||||||
|
|
||||||
|
## Seams — where tests go
|
||||||
|
|
||||||
|
A **seam** is the public boundary you test at: the interface where you observe behavior without reaching inside. Tests live at seams, never against internals.
|
||||||
|
|
||||||
|
**Test only at pre-agreed seams.** Before writing any test, write down the seams under test and confirm them with the user. No test is written at an unconfirmed seam. You can't test everything — agreeing the seams up front is how testing effort lands on the critical paths and complex logic instead of every edge case.
|
||||||
|
|
||||||
|
Ask: "What's the public interface, and which seams should we test?"
|
||||||
|
|
||||||
|
## Anti-patterns
|
||||||
|
|
||||||
|
- **Implementation-coupled** — mocks internal collaborators, tests private methods, or verifies through a side channel (querying the database instead of using the interface). The tell: the test breaks when you refactor but behavior hasn't changed.
|
||||||
|
- **Tautological** — the assertion recomputes the expected value the way the code does (`expect(add(a, b)).toBe(a + b)`, a snapshot derived by hand the same way, a constant asserted equal to itself), so it passes by construction and can never disagree with the code. Expected values must come from an independent source of truth — a known-good literal, a worked example, the spec.
|
||||||
|
- **Horizontal slicing** — writing all tests first, then all implementation. Bulk tests verify _imagined_ behavior: you test the _shape_ of things rather than user-facing behavior, the tests go insensitive to real changes, and you commit to test structure before understanding the implementation. Work in **vertical slices** instead — one test → one implementation → repeat, each test a **tracer bullet** that responds to what the last cycle taught you.
|
||||||
|
|
||||||
|
## Rules of the loop
|
||||||
|
|
||||||
|
- **Red before green.** Write the failing test first, then only enough code to pass it. Don't anticipate future tests or add speculative features.
|
||||||
|
- **One slice at a time.** One seam, one test, one minimal implementation per cycle.
|
||||||
|
- **Refactoring is not part of the loop.** It belongs to the review stage (see the `code-review` skill), not the red → green implementation cycle.
|
||||||
@@ -0,0 +1,59 @@
|
|||||||
|
# When to Mock
|
||||||
|
|
||||||
|
Mock at **system boundaries** only:
|
||||||
|
|
||||||
|
- External APIs (payment, email, etc.)
|
||||||
|
- Databases (sometimes - prefer test DB)
|
||||||
|
- Time/randomness
|
||||||
|
- File system (sometimes)
|
||||||
|
|
||||||
|
Don't mock:
|
||||||
|
|
||||||
|
- Your own classes/modules
|
||||||
|
- Internal collaborators
|
||||||
|
- Anything you control
|
||||||
|
|
||||||
|
## Designing for Mockability
|
||||||
|
|
||||||
|
At system boundaries, design interfaces that are easy to mock:
|
||||||
|
|
||||||
|
**1. Use dependency injection**
|
||||||
|
|
||||||
|
Pass external dependencies in rather than creating them internally:
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// Easy to mock
|
||||||
|
function processPayment(order, paymentClient) {
|
||||||
|
return paymentClient.charge(order.total);
|
||||||
|
}
|
||||||
|
|
||||||
|
// Hard to mock
|
||||||
|
function processPayment(order) {
|
||||||
|
const client = new StripeClient(process.env.STRIPE_KEY);
|
||||||
|
return client.charge(order.total);
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**2. Prefer SDK-style interfaces over generic fetchers**
|
||||||
|
|
||||||
|
Create specific functions for each external operation instead of one generic function with conditional logic:
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// GOOD: Each function is independently mockable
|
||||||
|
const api = {
|
||||||
|
getUser: (id) => fetch(`/users/${id}`),
|
||||||
|
getOrders: (userId) => fetch(`/users/${userId}/orders`),
|
||||||
|
createOrder: (data) => fetch('/orders', { method: 'POST', body: data }),
|
||||||
|
};
|
||||||
|
|
||||||
|
// BAD: Mocking requires conditional logic inside the mock
|
||||||
|
const api = {
|
||||||
|
fetch: (endpoint, options) => fetch(endpoint, options),
|
||||||
|
};
|
||||||
|
```
|
||||||
|
|
||||||
|
The SDK approach means:
|
||||||
|
- Each mock returns one specific shape
|
||||||
|
- No conditional logic in test setup
|
||||||
|
- Easier to see which endpoints a test exercises
|
||||||
|
- Type safety per endpoint
|
||||||
@@ -0,0 +1,77 @@
|
|||||||
|
# Good and Bad Tests
|
||||||
|
|
||||||
|
## Good Tests
|
||||||
|
|
||||||
|
**Integration-style**: Test through real interfaces, not mocks of internal parts.
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// GOOD: Tests observable behavior
|
||||||
|
test("user can checkout with valid cart", async () => {
|
||||||
|
const cart = createCart();
|
||||||
|
cart.add(product);
|
||||||
|
const result = await checkout(cart, paymentMethod);
|
||||||
|
expect(result.status).toBe("confirmed");
|
||||||
|
});
|
||||||
|
```
|
||||||
|
|
||||||
|
Characteristics:
|
||||||
|
|
||||||
|
- Tests behavior users/callers care about
|
||||||
|
- Uses public API only
|
||||||
|
- Survives internal refactors
|
||||||
|
- Describes WHAT, not HOW
|
||||||
|
- One logical assertion per test
|
||||||
|
|
||||||
|
## Bad Tests
|
||||||
|
|
||||||
|
**Implementation-detail tests**: Coupled to internal structure.
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// BAD: Tests implementation details
|
||||||
|
test("checkout calls paymentService.process", async () => {
|
||||||
|
const mockPayment = jest.mock(paymentService);
|
||||||
|
await checkout(cart, payment);
|
||||||
|
expect(mockPayment.process).toHaveBeenCalledWith(cart.total);
|
||||||
|
});
|
||||||
|
```
|
||||||
|
|
||||||
|
Red flags:
|
||||||
|
|
||||||
|
- Mocking internal collaborators
|
||||||
|
- Testing private methods
|
||||||
|
- Asserting on call counts/order
|
||||||
|
- Test breaks when refactoring without behavior change
|
||||||
|
- Test name describes HOW not WHAT
|
||||||
|
- Verifying through external means instead of interface
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// BAD: Bypasses interface to verify
|
||||||
|
test("createUser saves to database", async () => {
|
||||||
|
await createUser({ name: "Alice" });
|
||||||
|
const row = await db.query("SELECT * FROM users WHERE name = ?", ["Alice"]);
|
||||||
|
expect(row).toBeDefined();
|
||||||
|
});
|
||||||
|
|
||||||
|
// GOOD: Verifies through interface
|
||||||
|
test("createUser makes user retrievable", async () => {
|
||||||
|
const user = await createUser({ name: "Alice" });
|
||||||
|
const retrieved = await getUser(user.id);
|
||||||
|
expect(retrieved.name).toBe("Alice");
|
||||||
|
});
|
||||||
|
```
|
||||||
|
|
||||||
|
**Tautological tests**: Expected value restates the implementation, so the test passes by construction.
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// BAD: Expected value is recomputed the way the code computes it
|
||||||
|
test("calculateTotal sums line items", () => {
|
||||||
|
const items = [{ price: 10 }, { price: 5 }];
|
||||||
|
const expected = items.reduce((sum, i) => sum + i.price, 0);
|
||||||
|
expect(calculateTotal(items)).toBe(expected);
|
||||||
|
});
|
||||||
|
|
||||||
|
// GOOD: Expected value is an independent, known literal
|
||||||
|
test("calculateTotal sums line items", () => {
|
||||||
|
expect(calculateTotal([{ price: 10 }, { price: 5 }])).toBe(15);
|
||||||
|
});
|
||||||
|
```
|
||||||
@@ -1,25 +1,22 @@
|
|||||||
# Cross-platform Crucible binaries (D7 trigger standard):
|
# Cross-platform Crucible release binaries: a v* tag builds all targets,
|
||||||
# PR / push main -> cross-build ALL targets to prove they compile (no publish)
|
# then uploads them, SHA256SUMS, and the canonical wrappers to its Gitea release.
|
||||||
# tag v* -> build all targets + SHA256SUMS, upload to the Gitea release
|
|
||||||
# Crucible is pure Go (CGO_ENABLED=0), so cross-compiling is a fast loop.
|
# Crucible is pure Go (CGO_ENABLED=0), so cross-compiling is a fast loop.
|
||||||
name: build-binaries
|
name: build-binaries
|
||||||
|
|
||||||
on:
|
on:
|
||||||
pull_request:
|
|
||||||
push:
|
push:
|
||||||
branches: [main]
|
|
||||||
tags: ['v*']
|
tags: ['v*']
|
||||||
paths-ignore:
|
|
||||||
- "docs/**"
|
permissions:
|
||||||
- "README.md"
|
code: read
|
||||||
- "AGENTS.md"
|
releases: write
|
||||||
- "LICENSE"
|
|
||||||
|
|
||||||
jobs:
|
jobs:
|
||||||
build-binaries:
|
build-binaries:
|
||||||
runs-on: nix-docker
|
runs-on: nix-docker
|
||||||
|
timeout-minutes: 30
|
||||||
steps:
|
steps:
|
||||||
- uses: actions/checkout@v4
|
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
|
||||||
with: { fetch-depth: 0 }
|
with: { fetch-depth: 0 }
|
||||||
|
|
||||||
- name: Cross-build all targets
|
- name: Cross-build all targets
|
||||||
@@ -45,9 +42,8 @@ jobs:
|
|||||||
'
|
'
|
||||||
|
|
||||||
- name: Upload to Gitea release
|
- name: Upload to Gitea release
|
||||||
if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags/v')
|
|
||||||
env:
|
env:
|
||||||
TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
TOKEN: ${{ secrets.GITEA_TOKEN }}
|
||||||
TAG: ${{ github.ref_name }}
|
TAG: ${{ github.ref_name }}
|
||||||
REPO: ${{ github.repository }}
|
REPO: ${{ github.repository }}
|
||||||
SERVER: ${{ github.server_url }}
|
SERVER: ${{ github.server_url }}
|
||||||
|
|||||||
@@ -1,52 +0,0 @@
|
|||||||
# Publish the immutable crucible:<sha> image on version tags.
|
|
||||||
# PR/main test builds live in test-image.yml and never publish, so registry
|
|
||||||
# packages are only generated for releases we intend to use (not on every merge).
|
|
||||||
#
|
|
||||||
# Daemonless: the host-mode runner has no container runtime, so the image is
|
|
||||||
# built by Nix (`nix build .#image`, see flake.nix) and pushed with skopeo
|
|
||||||
# straight from the OCI tarball. No `docker build`/`docker login` involved.
|
|
||||||
name: build-image
|
|
||||||
|
|
||||||
on:
|
|
||||||
push:
|
|
||||||
tags: ['v*']
|
|
||||||
|
|
||||||
env:
|
|
||||||
REGISTRY: registry.westgate.pw
|
|
||||||
IMAGE: deployment/crucible
|
|
||||||
|
|
||||||
jobs:
|
|
||||||
publish:
|
|
||||||
runs-on: nix-docker
|
|
||||||
steps:
|
|
||||||
- uses: actions/checkout@v4
|
|
||||||
with: { fetch-depth: 0 }
|
|
||||||
|
|
||||||
- name: Resolve tag
|
|
||||||
id: tag
|
|
||||||
run: echo "sha=$(git rev-parse --short=12 HEAD)" >> "$GITHUB_OUTPUT"
|
|
||||||
|
|
||||||
- name: Build OCI image (daemonless)
|
|
||||||
run: nix build .#image
|
|
||||||
|
|
||||||
# This workflow only runs on v* tags, so every run is a release publish.
|
|
||||||
- name: Publish image
|
|
||||||
env:
|
|
||||||
REGISTRY_USER: ${{ secrets.REGISTRY_USER }}
|
|
||||||
REGISTRY_PASSWORD: ${{ secrets.REGISTRY_PASSWORD }}
|
|
||||||
run: |
|
|
||||||
nix shell nixpkgs#skopeo -c skopeo copy \
|
|
||||||
--dest-creds "${REGISTRY_USER}:${REGISTRY_PASSWORD}" \
|
|
||||||
docker-archive:result \
|
|
||||||
"docker://${REGISTRY}/${IMAGE}:${{ steps.tag.outputs.sha }}"
|
|
||||||
|
|
||||||
- name: Emit release fragment
|
|
||||||
run: |
|
|
||||||
FRAG_REPO=sow-tools \
|
|
||||||
FRAG_SHA=${{ steps.tag.outputs.sha }} \
|
|
||||||
FRAG_ARTIFACT="${REGISTRY}/${IMAGE}:${{ steps.tag.outputs.sha }}" \
|
|
||||||
FRAG_URL="${REGISTRY}/${IMAGE}" \
|
|
||||||
FRAG_RUN_ID=${{ gitea.run_id }} \
|
|
||||||
bash scripts/emit-release-fragment.sh
|
|
||||||
- uses: actions/upload-artifact@v3
|
|
||||||
with: { name: release-fragment, path: release-fragment.json }
|
|
||||||
@@ -0,0 +1,47 @@
|
|||||||
|
# Pull-request validation for Crucible. Releases and wrapper synchronization
|
||||||
|
# have their own narrow workflows because they need tag/main events.
|
||||||
|
name: ci
|
||||||
|
|
||||||
|
on:
|
||||||
|
pull_request:
|
||||||
|
|
||||||
|
permissions: read-all
|
||||||
|
|
||||||
|
jobs:
|
||||||
|
ci:
|
||||||
|
runs-on: nix-docker
|
||||||
|
timeout-minutes: 60
|
||||||
|
steps:
|
||||||
|
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
|
||||||
|
with:
|
||||||
|
fetch-depth: 0
|
||||||
|
|
||||||
|
- name: vet + test + lint
|
||||||
|
run: |
|
||||||
|
nix develop --command bash -c '
|
||||||
|
set -euo pipefail
|
||||||
|
go vet ./...
|
||||||
|
go test ./...
|
||||||
|
shellcheck scripts/*.sh
|
||||||
|
yamllint .gitea
|
||||||
|
'
|
||||||
|
|
||||||
|
- name: binary smoke (fail-closed contract)
|
||||||
|
run: nix develop --command make smoke
|
||||||
|
|
||||||
|
- name: Cross-build all targets
|
||||||
|
run: |
|
||||||
|
nix develop --command bash -c '
|
||||||
|
set -euo pipefail
|
||||||
|
sha="$(git rev-parse --short=12 HEAD)"
|
||||||
|
ldflags="-s -w -X git.westgate.pw/ShadowsOverWestgate/sow-tools/internal/buildinfo.Version=${sha}"
|
||||||
|
rm -rf dist && mkdir -p dist
|
||||||
|
export CGO_ENABLED=0
|
||||||
|
for target in linux/amd64 linux/arm64 darwin/amd64 darwin/arm64 windows/amd64 windows/arm64; do
|
||||||
|
os="${target%/*}"; arch="${target#*/}"
|
||||||
|
ext=""; [ "$os" = windows ] && ext=".exe"
|
||||||
|
echo "building crucible-${os}-${arch}${ext}"
|
||||||
|
GOOS="$os" GOARCH="$arch" go build -trimpath -ldflags "$ldflags" \
|
||||||
|
-o "dist/crucible-${os}-${arch}${ext}" ./cmd/crucible
|
||||||
|
done
|
||||||
|
'
|
||||||
@@ -1,49 +0,0 @@
|
|||||||
# Manual compatibility publisher for the Crucible image.
|
|
||||||
# Normal release publishing happens in build-image.yml on v* tags; this is the
|
|
||||||
# break-glass / re-publish path, triggered by hand.
|
|
||||||
#
|
|
||||||
# Daemonless: built by Nix (`nix build .#image`) and pushed with skopeo from the
|
|
||||||
# OCI tarball. No `docker build`/`docker login` involved.
|
|
||||||
name: publish-image
|
|
||||||
|
|
||||||
on:
|
|
||||||
workflow_dispatch:
|
|
||||||
|
|
||||||
env:
|
|
||||||
REGISTRY: registry.westgate.pw
|
|
||||||
IMAGE: deployment/crucible
|
|
||||||
|
|
||||||
jobs:
|
|
||||||
publish:
|
|
||||||
runs-on: nix-docker
|
|
||||||
steps:
|
|
||||||
- uses: actions/checkout@v4
|
|
||||||
with: { fetch-depth: 0 }
|
|
||||||
|
|
||||||
- name: Resolve tag
|
|
||||||
id: tag
|
|
||||||
run: echo "sha=$(git rev-parse --short=12 HEAD)" >> "$GITHUB_OUTPUT"
|
|
||||||
|
|
||||||
- name: Build OCI image (daemonless)
|
|
||||||
run: nix build .#image
|
|
||||||
|
|
||||||
- name: Publish image
|
|
||||||
env:
|
|
||||||
REGISTRY_USER: ${{ secrets.REGISTRY_USER }}
|
|
||||||
REGISTRY_PASSWORD: ${{ secrets.REGISTRY_PASSWORD }}
|
|
||||||
run: |
|
|
||||||
nix shell nixpkgs#skopeo -c skopeo copy \
|
|
||||||
--dest-creds "${REGISTRY_USER}:${REGISTRY_PASSWORD}" \
|
|
||||||
docker-archive:result \
|
|
||||||
"docker://${REGISTRY}/${IMAGE}:${{ steps.tag.outputs.sha }}"
|
|
||||||
|
|
||||||
- name: Emit release fragment
|
|
||||||
run: |
|
|
||||||
FRAG_REPO=sow-tools \
|
|
||||||
FRAG_SHA=${{ steps.tag.outputs.sha }} \
|
|
||||||
FRAG_ARTIFACT="${REGISTRY}/${IMAGE}:${{ steps.tag.outputs.sha }}" \
|
|
||||||
FRAG_URL="${REGISTRY}/${IMAGE}" \
|
|
||||||
FRAG_RUN_ID=${{ gitea.run_id }} \
|
|
||||||
bash scripts/emit-release-fragment.sh
|
|
||||||
- uses: actions/upload-artifact@v3
|
|
||||||
with: { name: release-fragment, path: release-fragment.json }
|
|
||||||
@@ -14,11 +14,14 @@ on:
|
|||||||
- 'wrappers/crucible.sh'
|
- 'wrappers/crucible.sh'
|
||||||
- 'wrappers/crucible.ps1'
|
- 'wrappers/crucible.ps1'
|
||||||
|
|
||||||
|
permissions: read-all
|
||||||
|
|
||||||
jobs:
|
jobs:
|
||||||
sync:
|
sync:
|
||||||
runs-on: nix-docker
|
runs-on: nix-docker
|
||||||
|
timeout-minutes: 30
|
||||||
steps:
|
steps:
|
||||||
- uses: actions/checkout@v4
|
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
|
||||||
|
|
||||||
- name: Open sync PRs to consumers
|
- name: Open sync PRs to consumers
|
||||||
env:
|
env:
|
||||||
|
|||||||
@@ -1,19 +0,0 @@
|
|||||||
# Test-build the Crucible image on PRs. Proves `nix build .#image` still works
|
|
||||||
# (daemonless, Nix-built OCI tarball) but does NOT publish — release publishing
|
|
||||||
# happens in build-image.yml on v* tags. PR-only: with up-to-date-before-merge
|
|
||||||
# protection, main == the tested PR head, so a throwaway post-merge rebuild that
|
|
||||||
# publishes nothing is pure waste.
|
|
||||||
name: test-image
|
|
||||||
|
|
||||||
on:
|
|
||||||
pull_request:
|
|
||||||
|
|
||||||
jobs:
|
|
||||||
build-image:
|
|
||||||
runs-on: nix-docker
|
|
||||||
steps:
|
|
||||||
- uses: actions/checkout@v4
|
|
||||||
with: { fetch-depth: 0 }
|
|
||||||
|
|
||||||
- name: Build OCI image (daemonless, no publish)
|
|
||||||
run: nix build .#image
|
|
||||||
@@ -1,30 +0,0 @@
|
|||||||
# Lint + unit tests for the Crucible suite. PR-first: PRs + push to main (D7).
|
|
||||||
name: test
|
|
||||||
|
|
||||||
on:
|
|
||||||
push:
|
|
||||||
branches: [main]
|
|
||||||
paths-ignore:
|
|
||||||
- "docs/**"
|
|
||||||
- "README.md"
|
|
||||||
- "AGENTS.md"
|
|
||||||
- "LICENSE"
|
|
||||||
pull_request:
|
|
||||||
|
|
||||||
jobs:
|
|
||||||
test:
|
|
||||||
runs-on: nix-docker
|
|
||||||
steps:
|
|
||||||
- uses: actions/checkout@v4
|
|
||||||
with: { fetch-depth: 0 }
|
|
||||||
- name: vet + test + lint
|
|
||||||
run: |
|
|
||||||
nix develop --command bash -c '
|
|
||||||
set -euo pipefail
|
|
||||||
go vet ./...
|
|
||||||
go test ./...
|
|
||||||
shellcheck scripts/*.sh
|
|
||||||
yamllint .gitea
|
|
||||||
'
|
|
||||||
- name: binary smoke (fail-closed contract)
|
|
||||||
run: nix develop --command make smoke
|
|
||||||
@@ -9,6 +9,48 @@ This repo owns the **builder logic:** one Go module
|
|||||||
(`git.westgate.pw/ShadowsOverWestgate/sow-tools`) producing the `crucible`
|
(`git.westgate.pw/ShadowsOverWestgate/sow-tools`) producing the `crucible`
|
||||||
dispatcher and the `crucible-<name>` binaries.
|
dispatcher and the `crucible-<name>` binaries.
|
||||||
|
|
||||||
|
## What it is / part it serves
|
||||||
|
|
||||||
|
Shadows Over Westgate (SoW) is a Neverwinter Nights: Enhanced Edition
|
||||||
|
persistent world, split into single-purpose repos. This repo is Crucible, the
|
||||||
|
Go build toolkit. The content repos hold game source (module areas, rules
|
||||||
|
data, binary assets) and call Crucible to turn that source into artifacts:
|
||||||
|
the `.mod` file, 2DA/TLK tables, HAKs, wiki pages. Crucible is the only place
|
||||||
|
builder logic lives; content repos only run it through thin wrapper scripts.
|
||||||
|
|
||||||
|
## Guidance map
|
||||||
|
|
||||||
|
| Where | What |
|
||||||
|
|-------|------|
|
||||||
|
| `cmd/crucible/`, `cmd/crucible-<name>/` | dispatcher + per-builder shims (thin `main.go` files) |
|
||||||
|
| `internal/dispatch/` | the command registry — single source of truth for the command surface |
|
||||||
|
| `internal/` (app, pipeline, project, erf, gff, topdata, changelog, validator, depot, menu, buildinfo) | the actual builder logic |
|
||||||
|
| `wrappers/` | canonical bootstrap wrappers (`crucible`, `crucible.ps1`) synced to consumer repos; `wrappers/consumers.txt` lists targets |
|
||||||
|
| `docs/command-surface.md` | every command, old `nwn-tool` name → new home |
|
||||||
|
| `docs/consumer-contract.md` | how consumer repos resolve/pin a Crucible binary |
|
||||||
|
| `docs/migration-from-nwn-tool.md` | migration status, what remains |
|
||||||
|
| `tests/`, `Makefile`, `flake.nix` | checks, targets, dev shell |
|
||||||
|
|
||||||
|
Task routing: adding/changing a command → read `docs/command-surface.md`
|
||||||
|
first, then `internal/dispatch`. Changing how consumers get binaries →
|
||||||
|
`docs/consumer-contract.md` + `wrappers/`. Release/CI questions → README "CI"
|
||||||
|
section and `.gitea/workflows/`.
|
||||||
|
|
||||||
|
## How it is used
|
||||||
|
|
||||||
|
- Dev loop: `nix develop`, then `make check` / `make build` / `make smoke`
|
||||||
|
(see Commands below).
|
||||||
|
- Release: push a `v*` tag. CI uploads cross-built binaries + wrappers to the
|
||||||
|
Gitea release. There is no container image; Crucible ships as binaries,
|
||||||
|
wrappers, and the Nix input.
|
||||||
|
- Consumers (they download released binaries via the wrapper; they never
|
||||||
|
vendor a toolkit):
|
||||||
|
- sow-module — https://git.westgate.pw/ShadowsOverWestgate/sow-module
|
||||||
|
- sow-topdata — https://git.westgate.pw/ShadowsOverWestgate/sow-topdata
|
||||||
|
- sow-assets-manifest — https://git.westgate.pw/ShadowsOverWestgate/sow-assets-manifest
|
||||||
|
- sow-platform (infra/deploy authority) —
|
||||||
|
https://git.westgate.pw/ShadowsOverWestgate/sow-platform
|
||||||
|
|
||||||
## What this repo owns / does not own
|
## What this repo owns / does not own
|
||||||
|
|
||||||
Owns: build/extract/validate/compare pipeline, ERF/HAK packing, topdata 2da/tlk
|
Owns: build/extract/validate/compare pipeline, ERF/HAK packing, topdata 2da/tlk
|
||||||
@@ -21,7 +63,7 @@ is `sow-platform`).
|
|||||||
|
|
||||||
1. **Fail closed, never fake.** A builder with no migrated logic yet (`depot`)
|
1. **Fail closed, never fake.** A builder with no migrated logic yet (`depot`)
|
||||||
exits `70`. Do not stub a builder to emit a placeholder artifact.
|
exits `70`. Do not stub a builder to emit a placeholder artifact.
|
||||||
2. **Binaries are not committed.** They are CI artifacts / image layers.
|
2. **Binaries are not committed.** They are CI artifacts.
|
||||||
`/bin/`, `*.exe`, `nwn-tool`, `sow-toolkit` are gitignored.
|
`/bin/`, `*.exe`, `nwn-tool`, `sow-toolkit` are gitignored.
|
||||||
3. **The registry is the command surface.** `internal/dispatch.Registry` is the
|
3. **The registry is the command surface.** `internal/dispatch.Registry` is the
|
||||||
single source of truth; keep it in sync with `cmd/` and
|
single source of truth; keep it in sync with `cmd/` and
|
||||||
@@ -40,7 +82,6 @@ is `sow-platform`).
|
|||||||
nix develop && make check # vet + test + shellcheck + yamllint
|
nix develop && make check # vet + test + shellcheck + yamllint
|
||||||
make build # cmd/* -> ./bin
|
make build # cmd/* -> ./bin
|
||||||
make smoke # assert fail-closed contract
|
make smoke # assert fail-closed contract
|
||||||
make image # crucible:<sha>
|
|
||||||
```
|
```
|
||||||
|
|
||||||
## Tests
|
## Tests
|
||||||
|
|||||||
@@ -1,11 +1,12 @@
|
|||||||
SHELL := bash
|
SHELL := bash
|
||||||
.ONESHELL:
|
.ONESHELL:
|
||||||
.PHONY: check test vet build smoke fmt image
|
.PHONY: check test vet build smoke fmt
|
||||||
|
|
||||||
# Lint + unit tests. Mirrors the `test` CI job; runs green inside `nix develop`.
|
# Lint + unit tests. Mirrors the `test` CI job; runs green inside `nix develop`.
|
||||||
check: vet test
|
check: vet test
|
||||||
shellcheck scripts/*.sh
|
shellcheck scripts/*.sh
|
||||||
yamllint .gitea
|
yamllint .gitea
|
||||||
|
bash tests/workflow-contract.sh
|
||||||
|
|
||||||
vet:
|
vet:
|
||||||
go vet ./...
|
go vet ./...
|
||||||
@@ -22,9 +23,3 @@ smoke: build
|
|||||||
|
|
||||||
fmt:
|
fmt:
|
||||||
gofmt -l -w cmd internal
|
gofmt -l -w cmd internal
|
||||||
|
|
||||||
# Build the Crucible image locally. Requires docker; mirrors build-image CI.
|
|
||||||
IMAGE ?= crucible
|
|
||||||
GIT_SHA ?= $(shell git rev-parse --short=12 HEAD 2>/dev/null || echo unknown)
|
|
||||||
image:
|
|
||||||
docker build --build-arg GIT_SHA=$(GIT_SHA) -f docker/Dockerfile -t $(IMAGE):$(GIT_SHA) .
|
|
||||||
|
|||||||
@@ -71,24 +71,21 @@ nix develop # Go + shellcheck + yamllint + make
|
|||||||
make check # go vet + go test + shellcheck + yamllint
|
make check # go vet + go test + shellcheck + yamllint
|
||||||
make build # build every cmd/* into ./bin (gitignored)
|
make build # build every cmd/* into ./bin (gitignored)
|
||||||
make smoke # build + assert the fail-closed contract
|
make smoke # build + assert the fail-closed contract
|
||||||
make image # docker build -> crucible:<sha>
|
|
||||||
```
|
```
|
||||||
|
|
||||||
Binaries are **never committed** — they are CI artifacts / image layers (D19).
|
Binaries are **never committed** — they are CI artifacts (D19).
|
||||||
This retires the old habit of checking in `nwn-tool` / `sow-toolkit`.
|
This retires the old habit of checking in `nwn-tool` / `sow-toolkit`.
|
||||||
|
|
||||||
## CI
|
## CI
|
||||||
|
|
||||||
PR-first (D7): checks run on pull requests and on push to `main`; the only
|
PR-first (D7): checks run once on pull requests; the only publish event is a
|
||||||
publish event is a `v*` tag (see `sow-docs/runbooks/ci-trigger-standard.md`).
|
`v*` tag (see `runbooks/ci-trigger-standard.md` in sow-docs,
|
||||||
|
https://git.westgate.pw/ShadowsOverWestgate/sow-docs).
|
||||||
|
|
||||||
- `test.yml` — vet, test, shellcheck, yamllint, binary smoke (PR + main).
|
- `ci.yml` — vet, test, shellcheck, yamllint, binary smoke, and cross-build all
|
||||||
- `test-image.yml` — build the OCI image to prove it compiles (PR + main, no push).
|
targets once per pull request.
|
||||||
- `build-binaries.yml` — cross-build all targets (PR + main); on a `v*` tag, upload
|
- `build-binaries.yml` — on a `v*` tag, cross-build and upload the binaries,
|
||||||
the binaries, `SHA256SUMS`, and the wrappers to the Gitea release.
|
`SHA256SUMS`, and the wrappers to the Gitea release.
|
||||||
- `build-image.yml` — on a `v*` tag, build and publish
|
|
||||||
`registry.westgate.pw/deployment/crucible:<sha>`.
|
|
||||||
- `publish-image.yml` — manual `workflow_dispatch` break-glass republish.
|
|
||||||
- `sync-wrappers.yml` — on a `main` push that touches `wrappers/`, auto-PR the
|
- `sync-wrappers.yml` — on a `main` push that touches `wrappers/`, auto-PR the
|
||||||
canonical wrappers to the consumer repos in `wrappers/consumers.txt`.
|
canonical wrappers to the consumer repos in `wrappers/consumers.txt`.
|
||||||
Consumer drift checks run after those PRs merge to `main`, not on the PRs
|
Consumer drift checks run after those PRs merge to `main`, not on the PRs
|
||||||
|
|||||||
@@ -0,0 +1,11 @@
|
|||||||
|
// Command crucible-assets is the standalone assets builder (equivalent to
|
||||||
|
// `crucible assets`). A single-token binary keeps consumer wrapper scripts simple.
|
||||||
|
package main
|
||||||
|
|
||||||
|
import (
|
||||||
|
"os"
|
||||||
|
|
||||||
|
"git.westgate.pw/ShadowsOverWestgate/sow-tools/internal/dispatch"
|
||||||
|
)
|
||||||
|
|
||||||
|
func main() { os.Exit(dispatch.RunBuilder("assets", os.Args[1:])) }
|
||||||
@@ -1,38 +0,0 @@
|
|||||||
# syntax=docker/dockerfile:1
|
|
||||||
#
|
|
||||||
# Crucible toolchain image: registry.westgate.pw/deployment/crucible:<git-sha>
|
|
||||||
#
|
|
||||||
# Reproducible multi-stage build, no on-VPS build. Produces every cmd/* binary
|
|
||||||
# and ships them on a static base. The `crucible` dispatcher is the entrypoint;
|
|
||||||
# consumer CI can also call the standalone crucible-<name> binaries by path.
|
|
||||||
#
|
|
||||||
FROM golang:1.26-alpine AS build
|
|
||||||
WORKDIR /src
|
|
||||||
RUN apk add --no-cache git
|
|
||||||
# go.sum is committed now that the migrated packages pull golang.org/x/text and
|
|
||||||
# gopkg.in/yaml.v3; the glob keeps the build working if it is ever absent.
|
|
||||||
COPY go.mod go.sum* ./
|
|
||||||
RUN go mod download
|
|
||||||
COPY . .
|
|
||||||
ARG GIT_SHA=unknown
|
|
||||||
ENV CGO_ENABLED=0
|
|
||||||
RUN set -eux; \
|
|
||||||
mkdir -p /out; \
|
|
||||||
for dir in ./cmd/*/; do \
|
|
||||||
name="$(basename "${dir}")"; \
|
|
||||||
go build -trimpath \
|
|
||||||
-ldflags "-s -w -X git.westgate.pw/ShadowsOverWestgate/sow-tools/internal/buildinfo.Version=${GIT_SHA}" \
|
|
||||||
-o "/out/${name}" "${dir}"; \
|
|
||||||
done
|
|
||||||
|
|
||||||
FROM debian:12-slim
|
|
||||||
# ca-certificates: builders fetch published manifests over HTTPS.
|
|
||||||
RUN set -eux; \
|
|
||||||
apt-get update; \
|
|
||||||
apt-get install -y --no-install-recommends ca-certificates; \
|
|
||||||
rm -rf /var/lib/apt/lists/*; \
|
|
||||||
useradd --system --create-home --uid 65532 nonroot
|
|
||||||
COPY --from=build /out/ /usr/local/bin/
|
|
||||||
USER nonroot
|
|
||||||
ENTRYPOINT ["/usr/local/bin/crucible"]
|
|
||||||
CMD ["help"]
|
|
||||||
@@ -22,6 +22,13 @@ aliases.
|
|||||||
| `topdata` | `convert` | Convert between 2DA and native JSON/module formats. |
|
| `topdata` | `convert` | Convert between 2DA and native JSON/module formats. |
|
||||||
| `wiki` | `build` | Render wiki page drafts from compiled topdata. |
|
| `wiki` | `build` | Render wiki page drafts from compiled topdata. |
|
||||||
| `wiki` | `deploy` | Deploy generated wiki pages to NodeBB. |
|
| `wiki` | `deploy` | Deploy generated wiki pages to NodeBB. |
|
||||||
|
| `assets` | `compile` | Compile ASCII `.mdl` models to binary in place. |
|
||||||
|
| `assets` | `convert` | Convert textures to/from NWN DDS (flips vertically). |
|
||||||
|
| `assets` | `upscale` | Upscale textures through an installed backend. |
|
||||||
|
| `assets` | `check-mdl` | Report uncompiled ASCII `.mdl` and model-name mismatches. |
|
||||||
|
| `assets` | `fix-mdl` | Lowercase `.mdl` names and rewrite model identity to match. |
|
||||||
|
| `assets` | `check-dupes` | Report runtime-name (basename) collisions across dirs. |
|
||||||
|
| `assets` | `clean-dupes` | Delete clean-tree files whose basename collides with primary. |
|
||||||
|
|
||||||
`crucible-depot` remains registered but unwired. It fails closed with exit `70`
|
`crucible-depot` remains registered but unwired. It fails closed with exit `70`
|
||||||
and never emits placeholder artifacts.
|
and never emits placeholder artifacts.
|
||||||
|
|||||||
@@ -49,10 +49,10 @@ by `flake.lock`. CI runs the _same_ `crucible <build>` as local dev, inside the
|
|||||||
nix devshell (binary-cache fast). No build container, no token, no CI-only
|
nix devshell (binary-cache fast). No build container, no token, no CI-only
|
||||||
pre-steps.
|
pre-steps.
|
||||||
|
|
||||||
The `registry.westgate.pw/deployment/crucible:<sha>` image is **deployment-only**:
|
The `registry.westgate.pw/deployment/crucible:<sha>` image is **retired**
|
||||||
`prod.yml` pins it so tools travel with the runtime host (`nwn.enable`), a
|
(2026-07-17): nothing consumed it, `prod.yml` no longer reserves a slot for
|
||||||
disabled placeholder until the NWN stack lands. It is **not** a build tool and no
|
it, and CI no longer builds or publishes it. Binaries, wrappers, and the Nix
|
||||||
build/CI job consumes it.
|
input are the only supported ways to run Crucible.
|
||||||
|
|
||||||
## Determinism
|
## Determinism
|
||||||
|
|
||||||
|
|||||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,343 @@
|
|||||||
|
# Crucible `assets` builder — design
|
||||||
|
|
||||||
|
Date: 2026-07-12
|
||||||
|
Status: approved, pre-implementation
|
||||||
|
|
||||||
|
## Goal
|
||||||
|
|
||||||
|
Fold the NWN:EE asset tools out of two standalone toolkits into Crucible as a
|
||||||
|
new self-contained builder, `crucible assets`:
|
||||||
|
|
||||||
|
- from the Python toolkit (`nwnee-asset-processing`) — model compile, texture
|
||||||
|
convert, texture upscale. Those scripts are references only; broken as-is and
|
||||||
|
carrying a heavy config/staging/report layer that we drop.
|
||||||
|
- from `sow-assets-manifest/scripts/` — the model-name and duplicate-name
|
||||||
|
integrity tools (`check-ascii-mdl.sh`, `fix-mdl-model-names.sh`,
|
||||||
|
`check-duplicate-names.sh`, `clean-duplicate-names.sh`). These are ported into
|
||||||
|
Go and the shell drivers removed; `sow-assets-manifest` calls
|
||||||
|
`./crucible.sh assets …` through its bootstrap wrapper instead.
|
||||||
|
|
||||||
|
Seven commands:
|
||||||
|
|
||||||
|
- `crucible assets compile` — compile every ASCII `.mdl` in a directory to
|
||||||
|
binary, in place.
|
||||||
|
- `crucible assets convert` — convert every texture in a directory to NWN:EE
|
||||||
|
DDS (or back to PNG/TGA), flipping vertically **every time** so a DDS is
|
||||||
|
always upside-down relative to its source.
|
||||||
|
- `crucible assets upscale` — upscale every texture in a directory through
|
||||||
|
whatever upscaling backend is installed.
|
||||||
|
- `crucible assets check-mdl` — report uncompiled ASCII `.mdl` files and
|
||||||
|
model-name mismatches (read-only).
|
||||||
|
- `crucible assets fix-mdl` — lowercase `.mdl` filenames and rewrite ASCII
|
||||||
|
model/root names to match each file's stem.
|
||||||
|
- `crucible assets check-dupes` — report runtime-name (basename) collisions
|
||||||
|
across directories.
|
||||||
|
- `crucible assets clean-dupes` — delete files from a "clean" tree whose
|
||||||
|
basename collides with anything in a "primary" tree.
|
||||||
|
|
||||||
|
## Principles
|
||||||
|
|
||||||
|
- **No configuration files. Arguments only.** No `processing/` staging dirs, no
|
||||||
|
discards, no YAML, no reports. Operates in place on the target directory.
|
||||||
|
- **Discover, don't configure.** External tools (the NWN engine, ImageMagick, an
|
||||||
|
upscaler) are found on `PATH` and at standard install locations. Each has a
|
||||||
|
single override flag when discovery is not enough.
|
||||||
|
- **Fail closed, never fake** (Crucible rule #1). A missing backend is a clear
|
||||||
|
error naming what to install and a non-zero exit — never a placeholder
|
||||||
|
artifact.
|
||||||
|
- **Simplify.** Drop the reference tools' padding / power-of-two / `nwn-safe` /
|
||||||
|
explicit-format knobs (convert auto-picks DXT1 vs DXT5) and the small-texture
|
||||||
|
staging (upscale). These can return later if wanted.
|
||||||
|
|
||||||
|
## Architecture
|
||||||
|
|
||||||
|
A new self-contained builder that follows the existing `depot` pattern exactly
|
||||||
|
(delegates straight to its own internal package, bypassing the legacy
|
||||||
|
`internal/app` surface):
|
||||||
|
|
||||||
|
- `internal/assets/run.go` — `func Run(args []string, stdout, stderr io.Writer,
|
||||||
|
getenv func(string) string) int`. Parses the subcommand
|
||||||
|
(`compile|convert|upscale|check-mdl|fix-mdl|check-dupes|clean-dupes`) and
|
||||||
|
dispatches. Returns a sysexits-style code.
|
||||||
|
- `internal/assets/` — one file per command plus small shared helpers
|
||||||
|
(recursion/file-selection, external-command runner, backend discovery).
|
||||||
|
- `cmd/crucible-assets/main.go` — one-line shim:
|
||||||
|
`func main() { os.Exit(dispatch.RunBuilder("assets", os.Args[1:])) }`.
|
||||||
|
- `internal/dispatch`:
|
||||||
|
- one `Registry` entry for `assets` (`Wired: true`) listing the seven
|
||||||
|
commands with `Usage`/`Options`;
|
||||||
|
- extend the direct-delegate branch that today reads
|
||||||
|
`if b.Name == "depot" && b.Wired` so `assets` also routes to
|
||||||
|
`assets.Run(...)` instead of the legacy `app.Run`.
|
||||||
|
- `docs/command-surface.md` — seven visible-command rows.
|
||||||
|
- `wrappers/consumers.txt` / wrappers are unchanged (no new consumer; the
|
||||||
|
consumers listed already vendor the wrapper).
|
||||||
|
|
||||||
|
### Shared helpers (`internal/assets`)
|
||||||
|
|
||||||
|
- `walk(roots, exts)` — collect files under each root (recursive by default)
|
||||||
|
whose extension is in `exts`, case-insensitively. Reject nothing fancy; these
|
||||||
|
are arguments the caller chose.
|
||||||
|
- `run(cmd, args...)` — thin `exec.Command` wrapper capturing combined output,
|
||||||
|
returning `(output, error)`. A single indirection point so tests can observe
|
||||||
|
invocations. cwd and env overrides passed as needed.
|
||||||
|
- `look(names...)` — return the first name found via `exec.LookPath`, or the
|
||||||
|
first path that exists from a supplied list of candidates. Used by every
|
||||||
|
backend discovery.
|
||||||
|
|
||||||
|
### Shared MDL name module (`internal/assets/mdl`)
|
||||||
|
|
||||||
|
A pure-Go port of `mdl-name-lib.sh` + `mdl-scan.awk`, no external tools. One
|
||||||
|
place three commands share (`compile`'s pre-check, `check-mdl`, `fix-mdl`):
|
||||||
|
|
||||||
|
- `IsASCII(path)` — NUL in the first 256 bytes → binary; else the leading
|
||||||
|
keyword must be `#`/`newmodel`/`node`/`setsupermodel`. Mirrors `mdl_is_ascii`.
|
||||||
|
- `ExpectedName(path)` — the file stem (sans `.mdl`).
|
||||||
|
- `CheckNames(path) []Mismatch` — for an ASCII model, report header-token
|
||||||
|
mismatches (`newmodel`, `setsupermodel`, `beginmodelgeom`, `endmodelgeom`,
|
||||||
|
`donemodel`, `newanim`, `doneanim` — case-insensitive vs the stem) and the
|
||||||
|
geometry **base node** mismatch (the node inside `beginmodelgeom` whose parent
|
||||||
|
is `null`; empty if none). Mirrors `mdl_check_model_name` + `mdl_base_name`.
|
||||||
|
- `FixNames(in) (out []byte, changed bool)` — rewrite only the model identity:
|
||||||
|
the header tokens above, plus the base node's declaration / any `parent` /
|
||||||
|
`animroot` pointing at the *old* base name → the expected name. Animation bone
|
||||||
|
names and supermodel animroots are left untouched so inheritance keeps working.
|
||||||
|
Mirrors `mdl_fix_model_name` byte-for-byte in intent.
|
||||||
|
|
||||||
|
Detection must stay identical to the awk classifier so behavior does not drift
|
||||||
|
while `import.sh` still runs the awk version (see Consumer migration).
|
||||||
|
|
||||||
|
## `crucible assets compile <dir>…`
|
||||||
|
|
||||||
|
Usage: `crucible assets compile [--nwn <install>] [--non-recursive] <dir>…`
|
||||||
|
|
||||||
|
Recursively find ASCII `.mdl` files under each `<dir>` and compile each to a
|
||||||
|
binary `.mdl` in place (the compiled result replaces the source).
|
||||||
|
|
||||||
|
The NWN:EE engine is the only real ASCII→binary MDL compiler, so this drives it
|
||||||
|
exactly as the reference does:
|
||||||
|
|
||||||
|
- **Discovery.** User data dir is fixed on Linux at
|
||||||
|
`~/.local/share/Neverwinter Nights` (holds the flat `development/` and
|
||||||
|
`modelcompiler/` folders the engine uses). The `nwmain` binary is found by
|
||||||
|
probing standard install roots, first hit wins:
|
||||||
|
- `~/.local/share/Steam/steamapps/common/Neverwinter Nights/bin/linux-x86/nwmain-linux`
|
||||||
|
- GOG: `~/GOG Games/Neverwinter Nights Enhanced Edition/.../nwmain-linux`
|
||||||
|
- Beamdog install dirs.
|
||||||
|
- `--nwn <install>` overrides (path to the install root or directly to the
|
||||||
|
binary).
|
||||||
|
- **Headless.** The engine is a GUI binary. Require `xvfb-run` and wrap the call
|
||||||
|
regardless of the caller's `DISPLAY` so compilation never opens the client
|
||||||
|
UI; fail before invoking the engine when `xvfb-run` is unavailable:
|
||||||
|
`xvfb-run -a --server-args=-screen 0 1024x768x24 nwmain-linux compilemodel <stem>`.
|
||||||
|
- **Per model (one at a time — the engine's `development/` and `modelcompiler/`
|
||||||
|
folders are flat and single-slot):**
|
||||||
|
1. Skip if the file is not ASCII (binary/compiled `.mdl`) — reported, not an
|
||||||
|
error.
|
||||||
|
2. Copy source into `development/<name>`.
|
||||||
|
3. Run `nwmain-linux compilemodel <stem>` with cwd = the binary dir.
|
||||||
|
4. Collect the compiled artifact from `modelcompiler/` matched
|
||||||
|
case-insensitively by stem.
|
||||||
|
5. Move it back over the source path, lowercased.
|
||||||
|
6. Clean the temp files created in `development/` and `modelcompiler/`.
|
||||||
|
- **Collisions.** A pre-existing `development/<name>` or `modelcompiler/<stem>`
|
||||||
|
aborts that model before any mutation.
|
||||||
|
|
||||||
|
Exit: non-zero if any model fails; each failure prints a short excerpt of the
|
||||||
|
engine log (`~/.local/share/Neverwinter Nights/logs/nwengineLog.txt`) as
|
||||||
|
advisory diagnostics.
|
||||||
|
|
||||||
|
Dropped vs reference: internal-name-mismatch pre-checks and the
|
||||||
|
`Model Names Differ` fail rule are kept (cheap, prevents silent wrong output),
|
||||||
|
using the shared `internal/assets/mdl` module — `compile` aborts a model whose
|
||||||
|
names differ and tells the user to run `crucible assets fix-mdl`. It does **not**
|
||||||
|
auto-fix (kept separate, by decision). The discard-manifest, dry-run/report
|
||||||
|
layer, and `processing/in|out` staging are dropped.
|
||||||
|
|
||||||
|
## `crucible assets convert <dir>…`
|
||||||
|
|
||||||
|
Usage: `crucible assets convert [--to dds|png|tga] [--backend <path>]
|
||||||
|
[--non-recursive] <dir>…`
|
||||||
|
|
||||||
|
Recursively convert textures in place. `--to` defaults to `dds`. The source set
|
||||||
|
is every texture under the roots whose format is not already the target, among
|
||||||
|
`{.dds, .png, .tga}`.
|
||||||
|
|
||||||
|
**Every conversion flips vertically exactly once.** Converting to DDS produces
|
||||||
|
an upside-down DDS (NWN's convention); converting a DDS back to PNG/TGA flips it
|
||||||
|
upright again. The flip is unconditional — it is the defining behavior, not a
|
||||||
|
policy.
|
||||||
|
|
||||||
|
Backend: ImageMagick (`magick`), one tool for decode + flip + encode across all
|
||||||
|
three formats:
|
||||||
|
|
||||||
|
- to DDS: `magick <src> -flip -define dds:mipmaps=<n> -define
|
||||||
|
dds:compression=<dxt1|dxt5> <dst>.dds`, choosing DXT1 when the source is
|
||||||
|
opaque and DXT5 when it has alpha (detected with `magick identify`).
|
||||||
|
- from DDS: `magick <src>.dds -flip <dst>.<png|tga>`.
|
||||||
|
- `--backend <path>` points at a different ImageMagick-compatible binary or a
|
||||||
|
dedicated encoder for higher-quality DXT (e.g. compressonator); default is
|
||||||
|
`magick`.
|
||||||
|
|
||||||
|
The original file is replaced in place — canonical DDS is the point. Exit
|
||||||
|
non-zero if any file fails.
|
||||||
|
|
||||||
|
Dropped vs reference: `--pad-color`, POT/legacy-safe/`nwn-safe`/`--strict`
|
||||||
|
sizing, explicit `--format`, alpha-detect toggles, discard staging. Auto DXT
|
||||||
|
selection replaces the format knobs.
|
||||||
|
|
||||||
|
## `crucible assets upscale <dir>…`
|
||||||
|
|
||||||
|
Usage: `crucible assets upscale [--scale N] [--backend <path>]
|
||||||
|
[--non-recursive] <dir>…`
|
||||||
|
|
||||||
|
Recursively upscale textures in place.
|
||||||
|
|
||||||
|
- **Backend discovery**, first found wins: `upscayl-bin`, `upscayl`,
|
||||||
|
`realesrgan-ncnn-vulkan`, `waifu2x-ncnn-vulkan`. These share a compatible
|
||||||
|
ncnn-vulkan CLI shape (`-i <in> -o <out> -s <scale>`). `--backend <path>`
|
||||||
|
overrides.
|
||||||
|
- `--scale N` default 4.
|
||||||
|
- Backends operate on PNG; real NWN textures are DDS, so for a `.dds` input the
|
||||||
|
command reuses the convert helpers: dds→png (flip), run the backend, png→dds
|
||||||
|
(flip back) — yielding a correctly-flipped upscaled DDS. PNG/TGA inputs are
|
||||||
|
upscaled directly in place.
|
||||||
|
- No backend found → fail closed with a message naming the supported backends.
|
||||||
|
|
||||||
|
Dropped vs reference: min-dimension small-texture staging, the separate
|
||||||
|
`--dds-backend` plumbing (it reuses convert), configured backend-path table.
|
||||||
|
|
||||||
|
## `crucible assets check-mdl <path>…`
|
||||||
|
|
||||||
|
Port of `check-ascii-mdl.sh`. Read-only. Each `<path>` is a file or a directory
|
||||||
|
(recursed for `*.mdl`). For every model, using `internal/assets/mdl`:
|
||||||
|
|
||||||
|
- report an **uncompiled ASCII** `.mdl` (a binary/compiled one is fine, skipped);
|
||||||
|
- report every model-name mismatch (header tokens and geometry base node) with
|
||||||
|
file, line, the offending token, and the expected stem.
|
||||||
|
|
||||||
|
Exit non-zero if any ASCII model or mismatch is found; zero and an `OK` line
|
||||||
|
otherwise. No mutation.
|
||||||
|
|
||||||
|
## `crucible assets fix-mdl [--dry-run] <path>…`
|
||||||
|
|
||||||
|
Port of `fix-mdl-model-names.sh`. For each `*.mdl` under the paths:
|
||||||
|
|
||||||
|
1. **Lowercase the filename** if it is not already lowercase (`mv`; abort that
|
||||||
|
file on a case-collision with an existing target).
|
||||||
|
2. For ASCII models with a name mismatch, rewrite the model identity in place
|
||||||
|
via `mdl.FixNames` (binary models are skipped — the engine owns those).
|
||||||
|
|
||||||
|
`--dry-run` reports every "would lowercase" / "would fix" without touching disk.
|
||||||
|
Prints a per-file log and a final `no broken mdl model names found` when clean.
|
||||||
|
|
||||||
|
Selection optimization from the reference is preserved: a file is a fix
|
||||||
|
candidate only if it needs a content fix (name mismatch) or a filename
|
||||||
|
lowercase, so the expensive per-file read/rewrite runs only on the few that
|
||||||
|
matter, not the whole tree.
|
||||||
|
|
||||||
|
## `crucible assets check-dupes <dir>…`
|
||||||
|
|
||||||
|
Port of `check-duplicate-names.sh`'s **directory mode only**. Read-only. NWN
|
||||||
|
packs hak entries by basename, not path, so files at different paths with the
|
||||||
|
same basename (case-insensitively) silently clobber on pack. Across all `<dir>`
|
||||||
|
args, report any two files whose lowercased basename matches.
|
||||||
|
|
||||||
|
Exit non-zero if any collision is found.
|
||||||
|
|
||||||
|
The script's `--manifests` mode (hak-scoped collisions read from `assets/*.yml`)
|
||||||
|
is **not** ported here — it inspects manifest structure, not files on disk, so
|
||||||
|
it belongs with the manifest/depot tooling. It moves in the follow-up spec (see
|
||||||
|
Out of scope); until then `sow-assets-manifest` keeps
|
||||||
|
`check-duplicate-names.sh` solely for that mode.
|
||||||
|
|
||||||
|
## `crucible assets clean-dupes [--dry-run] <primary> <clean>`
|
||||||
|
|
||||||
|
Port of `clean-duplicate-names.sh`. Deletes files from the `<clean>` tree whose
|
||||||
|
basename collides (case-insensitively) with any file in the `<primary>` tree.
|
||||||
|
Mutates only `<clean>`. Same rule as `check-dupes` directory mode, but resolves
|
||||||
|
the collision by removal instead of reporting.
|
||||||
|
|
||||||
|
- Refuses to run if `<primary>` and `<clean>` overlap (either contains the
|
||||||
|
other), matching the script's guard.
|
||||||
|
- `--dry-run` reports every "would delete" without removing.
|
||||||
|
|
||||||
|
Exit zero on success with a count of removed (or would-remove) files.
|
||||||
|
|
||||||
|
## Consumer migration (`sow-assets-manifest`)
|
||||||
|
|
||||||
|
`sow-assets-manifest` already ships the `crucible` bootstrap wrapper
|
||||||
|
(`crucible.sh`) and uses `crucible depot` from its Makefile, so it resolves a
|
||||||
|
released `crucible` with zero extra setup.
|
||||||
|
|
||||||
|
- **Delete** the three fully-folded driver scripts: `scripts/check-ascii-mdl.sh`,
|
||||||
|
`scripts/fix-mdl-model-names.sh`, `scripts/clean-duplicate-names.sh`.
|
||||||
|
- **Repoint** the Makefile targets to the wrapper:
|
||||||
|
- `check-mdl` → `./crucible.sh assets check-mdl …`
|
||||||
|
- `fix-mdl` → `./crucible.sh assets fix-mdl [--dry-run] …`
|
||||||
|
- `check-dupes` — its directory scan → `./crucible.sh assets check-dupes …`;
|
||||||
|
the `check-duplicate-names.sh --manifests` line stays as-is.
|
||||||
|
- `clean-dupes` → `./crucible.sh assets clean-dupes [--dry-run] <PRIMARY> <CLEAN>`
|
||||||
|
- **Keep for now:**
|
||||||
|
- `scripts/check-duplicate-names.sh` — kept **only** for its `--manifests`
|
||||||
|
mode, still called by the `check` and `haks` targets. Its directory mode is
|
||||||
|
superseded by `crucible assets check-dupes`. Ported and deleted in the
|
||||||
|
follow-up.
|
||||||
|
- `scripts/mdl-scan.awk` and `scripts/mdl-name-lib.sh` — still sourced by
|
||||||
|
`import.sh` for its batched pre-import gate (~28× faster than per-file, on a
|
||||||
|
hot path). They retire in the follow-up too. Because both the awk classifier
|
||||||
|
and the Go `mdl` module must agree, the port keeps detection byte-for-byte
|
||||||
|
identical (a shared fixture set checks this).
|
||||||
|
|
||||||
|
Nothing in `sow-assets-manifest` is committed by this project's plan except the
|
||||||
|
script deletions and Makefile edits; that repo's change lands as its own commit
|
||||||
|
once the new `crucible` release with `assets` is available.
|
||||||
|
|
||||||
|
## Out of scope — follow-up spec
|
||||||
|
|
||||||
|
Moving the **depot-interacting pipeline** into `crucible depot` is a separate
|
||||||
|
project with its own spec
|
||||||
|
(`2026-07-12-crucible-depot-pipeline-migration-design.md`). It covers
|
||||||
|
`import.sh`, `sync-assets.sh`, `export.sh`, the inline mdl gate, and the
|
||||||
|
`check-duplicate-names.sh --manifests` manifest-collision check. When that lands:
|
||||||
|
its gate uses the in-process `internal/assets/mdl` module, and
|
||||||
|
`scripts/mdl-scan.awk`, `scripts/mdl-name-lib.sh`, and
|
||||||
|
`scripts/check-duplicate-names.sh` are all deleted. This spec deliberately stops
|
||||||
|
at the standalone, file-on-disk integrity tools to keep one focused plan.
|
||||||
|
|
||||||
|
## Error handling
|
||||||
|
|
||||||
|
- Missing external tool (engine, `magick`, upscaler) → non-zero exit, message
|
||||||
|
names the tool and how to get it. Never a faked artifact. The pure-Go
|
||||||
|
integrity commands (`check-mdl`, `fix-mdl`, `check-dupes`, `clean-dupes`) need
|
||||||
|
no external tool.
|
||||||
|
- Per-file failures are collected and printed as a summary line at the end; the
|
||||||
|
command exits non-zero if any file failed but still processes the rest.
|
||||||
|
- Report commands (`check-mdl`, `check-dupes`) exit non-zero when they *find*
|
||||||
|
problems — that is their contract as CI/pre-import gates, not a tool error.
|
||||||
|
- Unknown subcommand / bad flags → usage error, exit 64.
|
||||||
|
|
||||||
|
## Testing
|
||||||
|
|
||||||
|
Go tests in `internal/assets`:
|
||||||
|
|
||||||
|
- flag/subcommand parsing and usage errors;
|
||||||
|
- backend discovery against a fabricated `PATH` (temp dir with stub
|
||||||
|
executables) — asserts first-match ordering and the `--backend` override;
|
||||||
|
- `walk` recursion and extension filtering, including `--non-recursive`;
|
||||||
|
- one real convert round-trip using `magick` (present in the dev shell):
|
||||||
|
encode a small PNG to DDS and back, asserting the pixels are vertically
|
||||||
|
flipped after a single conversion and restored after the round trip.
|
||||||
|
- `internal/assets/mdl`: table-driven fixtures — ASCII vs binary detection,
|
||||||
|
each header-token mismatch, the base-node mismatch, and `FixNames` producing
|
||||||
|
a model that then passes `CheckNames`. A parity fixture set shared in intent
|
||||||
|
with `mdl-scan.awk` so the Go port and the still-present awk agree.
|
||||||
|
- `check-dupes`/`clean-dupes`: temp trees asserting basename-collision
|
||||||
|
detection (case-insensitive), the overlap guard, and `--dry-run` mutating
|
||||||
|
nothing.
|
||||||
|
|
||||||
|
External engine/upscaler calls are exercised through the `run` indirection with
|
||||||
|
a stub in tests; they are not invoked for real in CI.
|
||||||
|
|
||||||
|
`make check` must stay green; add an `assets` expectation to `make smoke` for
|
||||||
|
the wired exit.
|
||||||
@@ -0,0 +1,90 @@
|
|||||||
|
# Crucible `depot` pipeline migration — design (charter / draft)
|
||||||
|
|
||||||
|
Date: 2026-07-12
|
||||||
|
Status: **draft — needs its own brainstorming pass before implementation.**
|
||||||
|
Depends on: `2026-07-12-crucible-assets-builder-design.md` (the `assets` builder
|
||||||
|
and `internal/assets/mdl` module) landing first.
|
||||||
|
|
||||||
|
This is a scoping charter, not an implementation-ready design. The scripts it
|
||||||
|
covers are nontrivial and lean heavily on `sow-assets-manifest/scripts/lib.sh`
|
||||||
|
(~35 KB). A full design requires studying `lib.sh` and the manifest format in
|
||||||
|
depth; that work happens in this spec's own brainstorming before any code.
|
||||||
|
|
||||||
|
## Why this exists
|
||||||
|
|
||||||
|
`sow-assets-manifest` is the last consumer still carrying real builder logic in
|
||||||
|
shell. The `assets` spec folded its standalone integrity tools into Crucible but
|
||||||
|
deliberately left the **depot-interacting pipeline** behind, because those
|
||||||
|
scripts read/write the content-addressed depot and the asset manifests — depot
|
||||||
|
domain, and a bigger lift. This charter completes the migration so
|
||||||
|
`sow-assets-manifest` becomes what the other consumers already are: a thin
|
||||||
|
caller of released `crucible` binaries with no vendored toolkit logic.
|
||||||
|
|
||||||
|
## What moves into `crucible depot`
|
||||||
|
|
||||||
|
The depot builder already owns `status/push/verify/get/pull`. This adds the
|
||||||
|
authoring side:
|
||||||
|
|
||||||
|
| Script | Proposed command | What it does today |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| `import.sh` (318 lines) | `crucible depot import <edit-dir> [prefix…]` | Import an edit tree into the depot + manifests: sticky per-path member assignment, new paths to the tail member, per-category `max_bytes` rebalance (spill overflow forward, no full repack), a local stat-cache to skip unchanged files, batched hashing + parallel `depot_put_many`. |
|
||||||
|
| `sync-assets.sh` (61 lines) | `crucible depot sync <tree> [prefix…]` | Full reconcile = import + prune manifest entries whose path no longer exists (scoped by prefix when given). Delegates to `import.sh`. |
|
||||||
|
| `export.sh` (96 lines) | `crucible depot export [glob] [--hak m] [--restype ext] --to <dir>` | Materialize a manifest-selected subset out of the depot into a clean edit tree (category paths preserved, no metadata). |
|
||||||
|
| `check-duplicate-names.sh --manifests` | `crucible depot check-dupes` (name TBD) | Hak-scoped basename collisions read from `assets/*.yml` (`.assets[].path` grouped by `.assets[].hak`). Manifest-structure check, not files on disk. |
|
||||||
|
|
||||||
|
The inline **pre-import mdl gate** in `import.sh` (batched `mdl-scan.awk` pass
|
||||||
|
that aborts on an H/B model-name mismatch before any blob is hashed/uploaded)
|
||||||
|
is reimplemented with the in-process `internal/assets/mdl` module from the
|
||||||
|
`assets` spec — same H/B contract, no subprocess.
|
||||||
|
|
||||||
|
## What retires when this lands
|
||||||
|
|
||||||
|
Deleted from `sow-assets-manifest/scripts/` once the commands above are wired
|
||||||
|
and the Makefile is repointed:
|
||||||
|
|
||||||
|
- `import.sh`, `sync-assets.sh`, `export.sh`
|
||||||
|
- `mdl-scan.awk`, `mdl-name-lib.sh` (only remaining consumer was `import.sh`)
|
||||||
|
- `check-duplicate-names.sh` (its directory mode already superseded by
|
||||||
|
`crucible assets check-dupes`; the `--manifests` mode moves here)
|
||||||
|
- whatever else in `lib.sh` becomes dead once the above are gone (assess during
|
||||||
|
design — `lib.sh` also serves `build-haks.sh`, `pack-haks.sh`, `promote.sh`,
|
||||||
|
etc., which are **not** in scope here, so `lib.sh` likely shrinks, not dies)
|
||||||
|
|
||||||
|
Makefile targets `import`, `sync`, `haks` (its sync/import/dupe steps), `export`
|
||||||
|
repoint to `./crucible.sh depot …`.
|
||||||
|
|
||||||
|
## Known hard parts (resolve in brainstorming)
|
||||||
|
|
||||||
|
- **Manifest read/write parity.** Byte-stable output, member `max_bytes`
|
||||||
|
rebalance semantics, sticky path→member assignment. Must match the current
|
||||||
|
awk/`lib.sh` behavior or diff-review of manifests becomes noise. Crucible
|
||||||
|
already parses these manifests (`internal/pipeline`, hak build) — reuse, don't
|
||||||
|
reinvent.
|
||||||
|
- **Depot backends.** `import`/`export` drive bunny/cdn/local through
|
||||||
|
`lib.sh`'s `depot_put_many`/`depot_require_write` (incl. the
|
||||||
|
`BUNNY_STORAGE_PASSWORD` prompt and cdn→bunny write normalization). Crucible's
|
||||||
|
`internal/depot` already speaks these backends for pull/push — the authoring
|
||||||
|
side should share that client, not a second implementation.
|
||||||
|
- **Stat-cache.** `import.sh`'s `.cache/import/<target>/<cat>.tsv` skip-unchanged
|
||||||
|
optimization is the dominant re-import speedup. Decide whether Crucible
|
||||||
|
reproduces it, replaces it (content-addressing already dedupes uploads), or
|
||||||
|
drops it with a measured justification.
|
||||||
|
- **Throughput.** The shell versions are already batched to avoid per-file
|
||||||
|
forks; a Go port should be at least as fast (in-process, no forks) — verify,
|
||||||
|
don't assume.
|
||||||
|
- **Scoping semantics.** Prefix-scoped import/prune (segment-boundary match) and
|
||||||
|
export glob matching (`**`/`*`/literal dots, no shell expansion) must port
|
||||||
|
exactly.
|
||||||
|
|
||||||
|
## Out of scope
|
||||||
|
|
||||||
|
The HAK build/pack/promote/publish scripts (`build-haks.sh`, `pack-haks.sh`,
|
||||||
|
`promote.sh`, `publish-release*.sh`, `hak-artifact-record.sh`) and their
|
||||||
|
`lib.sh` support. `crucible hak build` already exists; whether these thin
|
||||||
|
release wrappers also migrate is a separate question, not this charter.
|
||||||
|
|
||||||
|
## Next step
|
||||||
|
|
||||||
|
Brainstorm this charter into a full design: read `lib.sh` and the manifest
|
||||||
|
format, settle the hard parts above, then write the implementation-ready spec
|
||||||
|
and plan. Do not implement from this document.
|
||||||
@@ -0,0 +1,67 @@
|
|||||||
|
package assets
|
||||||
|
|
||||||
|
import (
|
||||||
|
"flag"
|
||||||
|
"fmt"
|
||||||
|
"io"
|
||||||
|
|
||||||
|
"git.westgate.pw/ShadowsOverWestgate/sow-tools/internal/assets/mdl"
|
||||||
|
)
|
||||||
|
|
||||||
|
// mdlExt is the file set shared by the mdl commands.
|
||||||
|
var mdlExt = map[string]bool{".mdl": true}
|
||||||
|
|
||||||
|
// runCheckMDL reports uncompiled ASCII .mdl files and model-name mismatches.
|
||||||
|
// Read-only. Exit exitFail if any problem is found.
|
||||||
|
func runCheckMDL(args []string, stdout, stderr io.Writer) int {
|
||||||
|
fs := flag.NewFlagSet("check-mdl", flag.ContinueOnError)
|
||||||
|
fs.SetOutput(stderr)
|
||||||
|
nonRecursive := fs.Bool("non-recursive", false, "do not descend into subdirectories")
|
||||||
|
if err := fs.Parse(args); err != nil {
|
||||||
|
return exitUsage
|
||||||
|
}
|
||||||
|
paths := fs.Args()
|
||||||
|
if len(paths) == 0 {
|
||||||
|
fmt.Fprintln(stderr, "assets check-mdl: usage: check-mdl <path>...")
|
||||||
|
return exitUsage
|
||||||
|
}
|
||||||
|
|
||||||
|
files, err := walk(paths, mdlExt, !*nonRecursive)
|
||||||
|
if err != nil {
|
||||||
|
fmt.Fprintln(stderr, "assets check-mdl:", err)
|
||||||
|
return exitUsage
|
||||||
|
}
|
||||||
|
|
||||||
|
fail := false
|
||||||
|
for _, f := range files {
|
||||||
|
ascii, err := mdl.IsASCII(f)
|
||||||
|
if err != nil {
|
||||||
|
fmt.Fprintln(stderr, "assets check-mdl:", err)
|
||||||
|
return exitTool
|
||||||
|
}
|
||||||
|
if ascii {
|
||||||
|
fmt.Fprintf(stderr, "uncompiled ascii mdl: %s\n", f)
|
||||||
|
fail = true
|
||||||
|
}
|
||||||
|
mismatches, err := mdl.CheckNames(f)
|
||||||
|
if err != nil {
|
||||||
|
fmt.Fprintln(stderr, "assets check-mdl:", err)
|
||||||
|
return exitTool
|
||||||
|
}
|
||||||
|
expected := mdl.ExpectedName(f)
|
||||||
|
for _, m := range mismatches {
|
||||||
|
if m.Line == 0 {
|
||||||
|
fmt.Fprintf(stderr, "%s: mdl model-name mismatch: root node is %s, expected %s\n", f, m.Got, expected)
|
||||||
|
} else {
|
||||||
|
fmt.Fprintf(stderr, "%s: mdl model-name mismatch: line %d: %s is %s, expected %s\n", f, m.Line, m.What, m.Got, expected)
|
||||||
|
}
|
||||||
|
fail = true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if fail {
|
||||||
|
fmt.Fprintln(stderr, "check-mdl: found broken or uncompiled .mdl file(s)")
|
||||||
|
return exitFail
|
||||||
|
}
|
||||||
|
fmt.Fprintln(stdout, "check-mdl: OK")
|
||||||
|
return exitOK
|
||||||
|
}
|
||||||
@@ -0,0 +1,32 @@
|
|||||||
|
package assets
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"testing"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestCheckMDL(t *testing.T) {
|
||||||
|
dir := t.TempDir()
|
||||||
|
// Uncompiled ASCII model with a name mismatch.
|
||||||
|
bad := filepath.Join(dir, "foo.mdl")
|
||||||
|
if err := os.WriteFile(bad, []byte("newmodel wrong\nbeginmodelgeom wrong\nendmodelgeom wrong\ndonemodel wrong\n"), 0o644); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
var out, errw bytes.Buffer
|
||||||
|
if code := runCheckMDL([]string{dir}, &out, &errw); code != exitFail {
|
||||||
|
t.Fatalf("bad mdl exit = %d, want %d\n%s", code, exitFail, errw.String())
|
||||||
|
}
|
||||||
|
|
||||||
|
// A binary (compiled) model is fine.
|
||||||
|
good := t.TempDir()
|
||||||
|
if err := os.WriteFile(filepath.Join(good, "bar.mdl"), []byte("\x00\x00compiled binary blob"), 0o644); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
out.Reset()
|
||||||
|
errw.Reset()
|
||||||
|
if code := runCheckMDL([]string{good}, &out, &errw); code != exitOK {
|
||||||
|
t.Fatalf("binary mdl exit = %d, want %d\n%s", code, exitOK, errw.String())
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,190 @@
|
|||||||
|
package assets
|
||||||
|
|
||||||
|
import (
|
||||||
|
"flag"
|
||||||
|
"fmt"
|
||||||
|
"io"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"strings"
|
||||||
|
|
||||||
|
"git.westgate.pw/ShadowsOverWestgate/sow-tools/internal/assets/mdl"
|
||||||
|
)
|
||||||
|
|
||||||
|
// runCompile compiles ASCII .mdl models to binary in place, one at a time (the
|
||||||
|
// engine's development/ and modelcompiler/ folders are flat, single-slot).
|
||||||
|
func runCompile(args []string, stdout, stderr io.Writer, getenv func(string) string) int {
|
||||||
|
fs := flag.NewFlagSet("compile", flag.ContinueOnError)
|
||||||
|
fs.SetOutput(stderr)
|
||||||
|
nwn := fs.String("nwn", "", "path to the NWN install root or nwmain-linux binary")
|
||||||
|
nonRecursive := fs.Bool("non-recursive", false, "do not descend into subdirectories")
|
||||||
|
if err := fs.Parse(args); err != nil {
|
||||||
|
return exitUsage
|
||||||
|
}
|
||||||
|
dirs := fs.Args()
|
||||||
|
if len(dirs) == 0 {
|
||||||
|
fmt.Fprintln(stderr, "assets compile: usage: compile [--nwn INSTALL] <dir>...")
|
||||||
|
return exitUsage
|
||||||
|
}
|
||||||
|
|
||||||
|
home := getenv("HOME")
|
||||||
|
userData := filepath.Join(home, ".local", "share", "Neverwinter Nights")
|
||||||
|
dev := filepath.Join(userData, "development")
|
||||||
|
mc := filepath.Join(userData, "modelcompiler")
|
||||||
|
|
||||||
|
nwmain := findNWMain(*nwn, home)
|
||||||
|
if nwmain == "" {
|
||||||
|
fmt.Fprintln(stderr, "assets compile: nwmain-linux not found — pass --nwn <install> "+
|
||||||
|
"(Steam/GOG/Beamdog install root or the nwmain-linux binary)")
|
||||||
|
return exitTool
|
||||||
|
}
|
||||||
|
binDir := filepath.Dir(nwmain)
|
||||||
|
|
||||||
|
// Always use a virtual X so compilation never opens the client UI.
|
||||||
|
xvfb := look("xvfb-run")
|
||||||
|
if xvfb == "" {
|
||||||
|
fmt.Fprintln(stderr, "assets compile: xvfb-run not found — install it to run the NWN model compiler headlessly")
|
||||||
|
return exitTool
|
||||||
|
}
|
||||||
|
wrap := []string{xvfb, "-a", "--server-args=-screen 0 1024x768x24"}
|
||||||
|
|
||||||
|
files, err := walk(dirs, mdlExt, !*nonRecursive)
|
||||||
|
if err != nil {
|
||||||
|
fmt.Fprintln(stderr, "assets compile:", err)
|
||||||
|
return exitUsage
|
||||||
|
}
|
||||||
|
|
||||||
|
failed := false
|
||||||
|
for _, src := range files {
|
||||||
|
if err := compileOne(src, binDir, nwmain, dev, mc, wrap, userData, stdout, stderr); err != nil {
|
||||||
|
fmt.Fprintf(stderr, "assets compile: %s: %v\n", src, err)
|
||||||
|
failed = true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if failed {
|
||||||
|
return exitFail
|
||||||
|
}
|
||||||
|
return exitOK
|
||||||
|
}
|
||||||
|
|
||||||
|
// findNWMain resolves the nwmain-linux binary from --nwn or standard roots.
|
||||||
|
func findNWMain(override, home string) string {
|
||||||
|
if override != "" {
|
||||||
|
if fi, err := os.Stat(override); err == nil && !fi.IsDir() {
|
||||||
|
return override
|
||||||
|
}
|
||||||
|
cand := filepath.Join(override, "bin", "linux-x86", "nwmain-linux")
|
||||||
|
if fi, err := os.Stat(cand); err == nil && !fi.IsDir() {
|
||||||
|
return cand
|
||||||
|
}
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
return look(
|
||||||
|
filepath.Join(home, ".local/share/Steam/steamapps/common/Neverwinter Nights/bin/linux-x86/nwmain-linux"),
|
||||||
|
filepath.Join(home, "GOG Games/Neverwinter Nights Enhanced Edition/game/bin/linux-x86/nwmain-linux"),
|
||||||
|
filepath.Join(home, ".steam/steam/steamapps/common/Neverwinter Nights/bin/linux-x86/nwmain-linux"),
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
// compileOne compiles a single model. Binary models are reported and skipped
|
||||||
|
// (not an error). A name-mismatched model is aborted with guidance.
|
||||||
|
func compileOne(src, binDir, nwmain, dev, mc string, wrap []string, userData string, stdout, stderr io.Writer) error {
|
||||||
|
ascii, err := mdl.IsASCII(src)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if !ascii {
|
||||||
|
fmt.Fprintf(stdout, "skip (already compiled): %s\n", src)
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
mismatches, err := mdl.CheckNames(src)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if len(mismatches) > 0 {
|
||||||
|
return fmt.Errorf("model names differ from the file stem; run `crucible assets fix-mdl` first")
|
||||||
|
}
|
||||||
|
|
||||||
|
stem := mdl.ExpectedNameStem(src)
|
||||||
|
name := strings.ToLower(filepath.Base(src))
|
||||||
|
devFile := filepath.Join(dev, name)
|
||||||
|
// Collision guard: a pre-existing slot aborts before any mutation.
|
||||||
|
if _, err := os.Stat(devFile); err == nil {
|
||||||
|
return fmt.Errorf("development slot already occupied: %s", devFile)
|
||||||
|
}
|
||||||
|
|
||||||
|
data, err := os.ReadFile(src)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if err := os.WriteFile(devFile, data, 0o644); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
defer os.Remove(devFile)
|
||||||
|
|
||||||
|
// Run the engine with cwd = the binary dir.
|
||||||
|
cmd := append(append([]string{}, wrap...), nwmain, "compilemodel", stem)
|
||||||
|
if out, err := runner(binDir, nil, cmd[0], cmd[1:]...); err != nil {
|
||||||
|
printEngineLog(userData, stderr)
|
||||||
|
return fmt.Errorf("engine: %v: %s", err, strings.TrimSpace(string(out)))
|
||||||
|
}
|
||||||
|
|
||||||
|
// Collect the compiled artifact from modelcompiler/ (match by stem, any case).
|
||||||
|
compiled, err := findCompiled(mc, stem)
|
||||||
|
if err != nil {
|
||||||
|
printEngineLog(userData, stderr)
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
defer os.Remove(compiled)
|
||||||
|
|
||||||
|
out, err := os.ReadFile(compiled)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
// Move it back over the source path, lowercased.
|
||||||
|
dst := filepath.Join(filepath.Dir(src), name)
|
||||||
|
if err := os.WriteFile(dst, out, 0o644); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if dst != src {
|
||||||
|
_ = os.Remove(src)
|
||||||
|
}
|
||||||
|
fmt.Fprintf(stdout, "compiled: %s\n", dst)
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// findCompiled returns the modelcompiler/ artifact whose stem matches (case-
|
||||||
|
// insensitively).
|
||||||
|
func findCompiled(mc, stem string) (string, error) {
|
||||||
|
entries, err := os.ReadDir(mc)
|
||||||
|
if err != nil {
|
||||||
|
return "", err
|
||||||
|
}
|
||||||
|
want := strings.ToLower(stem) + ".mdl"
|
||||||
|
for _, e := range entries {
|
||||||
|
if e.IsDir() {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if strings.ToLower(e.Name()) == want {
|
||||||
|
return filepath.Join(mc, e.Name()), nil
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return "", fmt.Errorf("engine produced no compiled model for %q", stem)
|
||||||
|
}
|
||||||
|
|
||||||
|
// printEngineLog appends a short tail of the engine log as advisory diagnostics.
|
||||||
|
func printEngineLog(userData string, stderr io.Writer) {
|
||||||
|
log := filepath.Join(userData, "logs", "nwengineLog.txt")
|
||||||
|
data, err := os.ReadFile(log)
|
||||||
|
if err != nil {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
lines := strings.Split(strings.TrimRight(string(data), "\n"), "\n")
|
||||||
|
if len(lines) > 8 {
|
||||||
|
lines = lines[len(lines)-8:]
|
||||||
|
}
|
||||||
|
fmt.Fprintf(stderr, " engine log tail (%s):\n", log)
|
||||||
|
for _, l := range lines {
|
||||||
|
fmt.Fprintf(stderr, " %s\n", l)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,171 @@
|
|||||||
|
package assets
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestCompileDrivesEngineAndReplacesInPlace(t *testing.T) {
|
||||||
|
home := t.TempDir()
|
||||||
|
userData := filepath.Join(home, ".local", "share", "Neverwinter Nights")
|
||||||
|
dev := filepath.Join(userData, "development")
|
||||||
|
mc := filepath.Join(userData, "modelcompiler")
|
||||||
|
for _, d := range []string{dev, mc} {
|
||||||
|
if err := os.MkdirAll(d, 0o755); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// Fake nwmain binary for discovery via --nwn.
|
||||||
|
binDir := t.TempDir()
|
||||||
|
nwmain := filepath.Join(binDir, "nwmain-linux")
|
||||||
|
if err := os.WriteFile(nwmain, []byte("#!/bin/sh\n"), 0o755); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
xvfbDir := t.TempDir()
|
||||||
|
xvfb := filepath.Join(xvfbDir, "xvfb-run")
|
||||||
|
if err := os.WriteFile(xvfb, []byte("#!/bin/sh\n"), 0o755); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
t.Setenv("PATH", xvfbDir+string(os.PathListSeparator)+os.Getenv("PATH"))
|
||||||
|
|
||||||
|
getenv := func(k string) string {
|
||||||
|
switch k {
|
||||||
|
case "HOME":
|
||||||
|
return home
|
||||||
|
case "DISPLAY":
|
||||||
|
return ":0" // a desktop display must not make compilation interactive
|
||||||
|
}
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
|
||||||
|
// Stub the engine: writes a binary compiled model into modelcompiler/.
|
||||||
|
orig := runner
|
||||||
|
defer func() { runner = orig }()
|
||||||
|
runner = func(dir string, env []string, name string, args ...string) ([]byte, error) {
|
||||||
|
if name != xvfb || len(args) != 5 || args[0] != "-a" ||
|
||||||
|
args[1] != "--server-args=-screen 0 1024x768x24" || args[2] != nwmain ||
|
||||||
|
args[3] != "compilemodel" {
|
||||||
|
t.Fatalf("engine command = %q %q, want xvfb-run wrapping nwmain", name, args)
|
||||||
|
}
|
||||||
|
stem := args[len(args)-1]
|
||||||
|
compiled := filepath.Join(mc, stem+".mdl")
|
||||||
|
return nil, os.WriteFile(compiled, []byte("\x00\x00compiled"), 0o644)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Source tree with one ASCII model whose names already match its stem.
|
||||||
|
srcDir := t.TempDir()
|
||||||
|
src := filepath.Join(srcDir, "foo.mdl")
|
||||||
|
body := "newmodel foo\nbeginmodelgeom foo\n node dummy foo\n parent null\n endnode\nendmodelgeom foo\ndonemodel foo\n"
|
||||||
|
if err := os.WriteFile(src, []byte(body), 0o644); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
|
||||||
|
var stdout, stderr bytes.Buffer
|
||||||
|
code := runCompile([]string{"--nwn", nwmain, srcDir}, &stdout, &stderr, getenv)
|
||||||
|
if code != exitOK {
|
||||||
|
t.Fatalf("compile exit = %d\n%s", code, stderr.String())
|
||||||
|
}
|
||||||
|
// The source is now binary (the compiled artifact moved back over it).
|
||||||
|
data, err := os.ReadFile(src)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("compiled model missing: %v", err)
|
||||||
|
}
|
||||||
|
if string(data) != "\x00\x00compiled" {
|
||||||
|
t.Fatalf("source not replaced by compiled binary: %q", data)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestCompileAbortsOnNameMismatch(t *testing.T) {
|
||||||
|
home := t.TempDir()
|
||||||
|
if err := os.MkdirAll(filepath.Join(home, ".local", "share", "Neverwinter Nights", "development"), 0o755); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
binDir := t.TempDir()
|
||||||
|
nwmain := filepath.Join(binDir, "nwmain-linux")
|
||||||
|
if err := os.WriteFile(nwmain, []byte("#!/bin/sh\n"), 0o755); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
xvfbDir := t.TempDir()
|
||||||
|
if err := os.WriteFile(filepath.Join(xvfbDir, "xvfb-run"), []byte("#!/bin/sh\n"), 0o755); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
t.Setenv("PATH", xvfbDir+string(os.PathListSeparator)+os.Getenv("PATH"))
|
||||||
|
getenv := func(k string) string {
|
||||||
|
if k == "HOME" {
|
||||||
|
return home
|
||||||
|
}
|
||||||
|
if k == "DISPLAY" {
|
||||||
|
return ":0"
|
||||||
|
}
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
orig := runner
|
||||||
|
defer func() { runner = orig }()
|
||||||
|
engineCalled := false
|
||||||
|
runner = func(string, []string, string, ...string) ([]byte, error) {
|
||||||
|
engineCalled = true
|
||||||
|
return nil, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
srcDir := t.TempDir()
|
||||||
|
// Internal name "wrong" != stem "foo": must abort before touching the engine.
|
||||||
|
if err := os.WriteFile(filepath.Join(srcDir, "foo.mdl"),
|
||||||
|
[]byte("newmodel wrong\ndonemodel wrong\n"), 0o644); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
var stdout, stderr bytes.Buffer
|
||||||
|
if code := runCompile([]string{"--nwn", nwmain, srcDir}, &stdout, &stderr, getenv); code != exitFail {
|
||||||
|
t.Fatalf("mismatch exit = %d, want %d", code, exitFail)
|
||||||
|
}
|
||||||
|
if engineCalled {
|
||||||
|
t.Fatal("engine should not run for a name-mismatched model")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestCompileFailsClosedWithoutXvfb(t *testing.T) {
|
||||||
|
home := t.TempDir()
|
||||||
|
userData := filepath.Join(home, ".local", "share", "Neverwinter Nights")
|
||||||
|
for _, d := range []string{filepath.Join(userData, "development"), filepath.Join(userData, "modelcompiler")} {
|
||||||
|
if err := os.MkdirAll(d, 0o755); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
nwmain := filepath.Join(t.TempDir(), "nwmain-linux")
|
||||||
|
if err := os.WriteFile(nwmain, []byte("#!/bin/sh\n"), 0o755); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
t.Setenv("PATH", t.TempDir())
|
||||||
|
|
||||||
|
getenv := func(k string) string {
|
||||||
|
if k == "HOME" {
|
||||||
|
return home
|
||||||
|
}
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
orig := runner
|
||||||
|
defer func() { runner = orig }()
|
||||||
|
engineCalled := false
|
||||||
|
runner = func(string, []string, string, ...string) ([]byte, error) {
|
||||||
|
engineCalled = true
|
||||||
|
return nil, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
srcDir := t.TempDir()
|
||||||
|
if err := os.WriteFile(filepath.Join(srcDir, "foo.mdl"),
|
||||||
|
[]byte("newmodel foo\nbeginmodelgeom foo\n node dummy foo\n parent null\n endnode\nendmodelgeom foo\ndonemodel foo\n"), 0o644); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
var stdout, stderr bytes.Buffer
|
||||||
|
if code := runCompile([]string{"--nwn", nwmain, srcDir}, &stdout, &stderr, getenv); code != exitTool {
|
||||||
|
t.Fatalf("compile exit = %d, want %d\n%s", code, exitTool, stderr.String())
|
||||||
|
}
|
||||||
|
if engineCalled {
|
||||||
|
t.Fatal("engine must not run without xvfb-run")
|
||||||
|
}
|
||||||
|
if !strings.Contains(stderr.String(), "xvfb-run") {
|
||||||
|
t.Fatalf("missing actionable xvfb-run error: %s", stderr.String())
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,116 @@
|
|||||||
|
package assets
|
||||||
|
|
||||||
|
import (
|
||||||
|
"flag"
|
||||||
|
"fmt"
|
||||||
|
"io"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"strings"
|
||||||
|
)
|
||||||
|
|
||||||
|
var textureExts = map[string]bool{".dds": true, ".png": true, ".tga": true}
|
||||||
|
|
||||||
|
// runConvert converts textures in place, flipping vertically exactly once per
|
||||||
|
// conversion. --to defaults to dds. Files already in the target format are
|
||||||
|
// skipped.
|
||||||
|
func runConvert(args []string, stdout, stderr io.Writer) int {
|
||||||
|
fs := flag.NewFlagSet("convert", flag.ContinueOnError)
|
||||||
|
fs.SetOutput(stderr)
|
||||||
|
to := fs.String("to", "dds", "target format: dds|png|tga")
|
||||||
|
backend := fs.String("backend", "magick", "ImageMagick-compatible binary")
|
||||||
|
nonRecursive := fs.Bool("non-recursive", false, "do not descend into subdirectories")
|
||||||
|
if err := fs.Parse(args); err != nil {
|
||||||
|
return exitUsage
|
||||||
|
}
|
||||||
|
target := strings.ToLower(*to)
|
||||||
|
if target != "dds" && target != "png" && target != "tga" {
|
||||||
|
fmt.Fprintln(stderr, "assets convert: --to must be dds, png, or tga")
|
||||||
|
return exitUsage
|
||||||
|
}
|
||||||
|
dirs := fs.Args()
|
||||||
|
if len(dirs) == 0 {
|
||||||
|
fmt.Fprintln(stderr, "assets convert: usage: convert [--to dds|png|tga] <dir>...")
|
||||||
|
return exitUsage
|
||||||
|
}
|
||||||
|
|
||||||
|
magick := look(*backend)
|
||||||
|
if magick == "" {
|
||||||
|
fmt.Fprintf(stderr, "assets convert: backend %q not found — install ImageMagick (magick)\n", *backend)
|
||||||
|
return exitTool
|
||||||
|
}
|
||||||
|
|
||||||
|
files, err := walk(dirs, textureExts, !*nonRecursive)
|
||||||
|
if err != nil {
|
||||||
|
fmt.Fprintln(stderr, "assets convert:", err)
|
||||||
|
return exitUsage
|
||||||
|
}
|
||||||
|
|
||||||
|
targetExt := "." + target
|
||||||
|
failed := false
|
||||||
|
for _, src := range files {
|
||||||
|
srcExt := strings.ToLower(filepath.Ext(src))
|
||||||
|
if srcExt == targetExt {
|
||||||
|
continue // already in the target format
|
||||||
|
}
|
||||||
|
dst := strings.TrimSuffix(src, filepath.Ext(src)) + targetExt
|
||||||
|
var convErr error
|
||||||
|
if target == "dds" {
|
||||||
|
convErr = pngToDDS(magick, src, dst)
|
||||||
|
} else {
|
||||||
|
convErr = ddsToPNG(magick, src, dst) // works for any decodable source
|
||||||
|
}
|
||||||
|
if convErr != nil {
|
||||||
|
fmt.Fprintf(stderr, "assets convert: %s: %v\n", src, convErr)
|
||||||
|
failed = true
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if dst != src {
|
||||||
|
if err := os.Remove(src); err != nil {
|
||||||
|
fmt.Fprintf(stderr, "assets convert: %s: %v\n", src, err)
|
||||||
|
failed = true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if failed {
|
||||||
|
return exitFail
|
||||||
|
}
|
||||||
|
return exitOK
|
||||||
|
}
|
||||||
|
|
||||||
|
// ddsToPNG decodes src to dst (PNG/TGA by dst's extension), flipping vertically
|
||||||
|
// once. The single flip is the defining NWN behavior.
|
||||||
|
func ddsToPNG(magickBin, src, dst string) error {
|
||||||
|
if out, err := runner("", nil, magickBin, src, "-flip", dst); err != nil {
|
||||||
|
return fmt.Errorf("magick: %v: %s", err, strings.TrimSpace(string(out)))
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// pngToDDS encodes src to a DDS at dst, flipping vertically once and choosing
|
||||||
|
// DXT1 for opaque images, DXT5 for images with alpha.
|
||||||
|
func pngToDDS(magickBin, src, dst string) error {
|
||||||
|
compression := "dxt1"
|
||||||
|
if hasAlpha(magickBin, src) {
|
||||||
|
compression = "dxt5"
|
||||||
|
}
|
||||||
|
// ponytail: mipmaps rely on magick's default full chain (NWN wants a
|
||||||
|
// pyramid, which magick writes by default). Add `-define dds:mipmaps=N`
|
||||||
|
// here if a specific count is ever required.
|
||||||
|
out, err := runner("", nil, magickBin, src, "-flip",
|
||||||
|
"-define", "dds:compression="+compression, dst)
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("magick: %v: %s", err, strings.TrimSpace(string(out)))
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// hasAlpha reports whether src has any non-opaque pixel, via `magick identify`.
|
||||||
|
// On any probe error it assumes alpha (DXT5), the safe/lossless-alpha default.
|
||||||
|
func hasAlpha(magickBin, src string) bool {
|
||||||
|
out, err := runner("", nil, magickBin, "identify", "-format", "%[opaque]", src)
|
||||||
|
if err != nil {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
return !strings.EqualFold(strings.TrimSpace(string(out)), "true")
|
||||||
|
}
|
||||||
@@ -0,0 +1,108 @@
|
|||||||
|
package assets
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"image"
|
||||||
|
"image/color"
|
||||||
|
"image/png"
|
||||||
|
"os"
|
||||||
|
"os/exec"
|
||||||
|
"path/filepath"
|
||||||
|
"testing"
|
||||||
|
)
|
||||||
|
|
||||||
|
// writeTestPNG writes a 16x16 image: top half red, bottom half blue. The size
|
||||||
|
// and the half-way split keep every 4x4 DXT block a single flat color, so the
|
||||||
|
// DXT1 round trip stays lossless and only the flip is under test. (A 4x4 image
|
||||||
|
// is one mixed DXT block that magick's encoder collapses to a single color.)
|
||||||
|
func writeTestPNG(t *testing.T, path string) {
|
||||||
|
t.Helper()
|
||||||
|
const n = 16
|
||||||
|
img := image.NewRGBA(image.Rect(0, 0, n, n))
|
||||||
|
for y := 0; y < n; y++ {
|
||||||
|
c := color.RGBA{255, 0, 0, 255} // red
|
||||||
|
if y >= n/2 {
|
||||||
|
c = color.RGBA{0, 0, 255, 255} // blue
|
||||||
|
}
|
||||||
|
for x := 0; x < n; x++ {
|
||||||
|
img.Set(x, y, c)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
f, err := os.Create(path)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
defer f.Close()
|
||||||
|
if err := png.Encode(f, img); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func topRowIsBlue(t *testing.T, pngPath string) bool {
|
||||||
|
t.Helper()
|
||||||
|
f, err := os.Open(pngPath)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
defer f.Close()
|
||||||
|
img, err := png.Decode(f)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
r, _, b, _ := img.At(0, 0).RGBA()
|
||||||
|
return b > r // blue dominates the top-left after a flip
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestConvertFlipsOnceAndRoundTrips(t *testing.T) {
|
||||||
|
magick := look("magick")
|
||||||
|
if magick == "" {
|
||||||
|
t.Skip("magick not on PATH")
|
||||||
|
}
|
||||||
|
dir := t.TempDir()
|
||||||
|
src := filepath.Join(dir, "tex.png")
|
||||||
|
writeTestPNG(t, src)
|
||||||
|
|
||||||
|
// Convert PNG -> DDS (in place: tex.png becomes tex.dds, original removed).
|
||||||
|
var out, errw bytes.Buffer
|
||||||
|
if code := runConvert([]string{"--to", "dds", dir}, &out, &errw); code != exitOK {
|
||||||
|
t.Fatalf("to-dds exit = %d\n%s", code, errw.String())
|
||||||
|
}
|
||||||
|
dds := filepath.Join(dir, "tex.dds")
|
||||||
|
if _, err := os.Stat(dds); err != nil {
|
||||||
|
t.Fatalf("dds not produced: %v", err)
|
||||||
|
}
|
||||||
|
if _, err := os.Stat(src); !os.IsNotExist(err) {
|
||||||
|
t.Fatal("source png was not replaced")
|
||||||
|
}
|
||||||
|
|
||||||
|
// Decode the DDS RAW (no extra flip) and confirm a single flip happened:
|
||||||
|
// the source top row was red, so the DDS top row must now be blue.
|
||||||
|
raw := filepath.Join(dir, "raw.png")
|
||||||
|
if err := exec.Command(magick, dds, raw).Run(); err != nil {
|
||||||
|
t.Fatalf("raw decode: %v", err)
|
||||||
|
}
|
||||||
|
if !topRowIsBlue(t, raw) {
|
||||||
|
t.Fatal("expected the DDS to be vertically flipped vs the source")
|
||||||
|
}
|
||||||
|
|
||||||
|
// Convert DDS -> PNG (another flip). Result should match the original.
|
||||||
|
out.Reset()
|
||||||
|
errw.Reset()
|
||||||
|
if code := runConvert([]string{"--to", "png", dir}, &out, &errw); code != exitOK {
|
||||||
|
t.Fatalf("to-png exit = %d\n%s", code, errw.String())
|
||||||
|
}
|
||||||
|
restored := filepath.Join(dir, "tex.png")
|
||||||
|
if topRowIsBlue(t, restored) {
|
||||||
|
t.Fatal("round trip did not restore the original orientation")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestConvertMissingBackendFailsClosed(t *testing.T) {
|
||||||
|
dir := t.TempDir()
|
||||||
|
writeTestPNG(t, filepath.Join(dir, "tex.png"))
|
||||||
|
var out, errw bytes.Buffer
|
||||||
|
code := runConvert([]string{"--backend", "/nonexistent/magick", dir}, &out, &errw)
|
||||||
|
if code != exitTool {
|
||||||
|
t.Fatalf("missing backend exit = %d, want %d", code, exitTool)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,151 @@
|
|||||||
|
package assets
|
||||||
|
|
||||||
|
import (
|
||||||
|
"flag"
|
||||||
|
"fmt"
|
||||||
|
"io"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"sort"
|
||||||
|
"strings"
|
||||||
|
)
|
||||||
|
|
||||||
|
// allFiles returns every regular file under root (recursive).
|
||||||
|
func allFiles(root string) ([]string, error) {
|
||||||
|
var out []string
|
||||||
|
err := filepath.WalkDir(root, func(p string, d os.DirEntry, err error) error {
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if !d.IsDir() {
|
||||||
|
out = append(out, p)
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
})
|
||||||
|
sort.Strings(out)
|
||||||
|
return out, err
|
||||||
|
}
|
||||||
|
|
||||||
|
// runCheckDupes reports files whose lower-cased basename collides across the
|
||||||
|
// given dirs. Read-only. Exit exitFail if any collision is found.
|
||||||
|
func runCheckDupes(args []string, stdout, stderr io.Writer) int {
|
||||||
|
fs := flag.NewFlagSet("check-dupes", flag.ContinueOnError)
|
||||||
|
fs.SetOutput(stderr)
|
||||||
|
if err := fs.Parse(args); err != nil {
|
||||||
|
return exitUsage
|
||||||
|
}
|
||||||
|
dirs := fs.Args()
|
||||||
|
if len(dirs) == 0 {
|
||||||
|
fmt.Fprintln(stderr, "assets check-dupes: usage: check-dupes <dir>...")
|
||||||
|
return exitUsage
|
||||||
|
}
|
||||||
|
|
||||||
|
first := map[string]string{}
|
||||||
|
fail := false
|
||||||
|
for _, dir := range dirs {
|
||||||
|
if fi, err := os.Stat(dir); err != nil || !fi.IsDir() {
|
||||||
|
fmt.Fprintf(stderr, "assets check-dupes: no such dir: %s\n", dir)
|
||||||
|
return exitUsage
|
||||||
|
}
|
||||||
|
files, err := allFiles(dir)
|
||||||
|
if err != nil {
|
||||||
|
fmt.Fprintln(stderr, "assets check-dupes:", err)
|
||||||
|
return exitTool
|
||||||
|
}
|
||||||
|
for _, f := range files {
|
||||||
|
key := strings.ToLower(filepath.Base(f))
|
||||||
|
if prev, ok := first[key]; ok {
|
||||||
|
fmt.Fprintf(stderr, "runtime-name collision: %s collides with %s\n", f, prev)
|
||||||
|
fail = true
|
||||||
|
} else {
|
||||||
|
first[key] = f
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if fail {
|
||||||
|
fmt.Fprintln(stderr, "check-dupes: found runtime-name collision(s)")
|
||||||
|
return exitFail
|
||||||
|
}
|
||||||
|
fmt.Fprintln(stdout, "check-dupes: OK")
|
||||||
|
return exitOK
|
||||||
|
}
|
||||||
|
|
||||||
|
// runCleanDupes deletes files from <clean> whose lower-cased basename collides
|
||||||
|
// with any file in <primary>. Mutates only <clean>. Refuses to run if the two
|
||||||
|
// trees overlap.
|
||||||
|
func runCleanDupes(args []string, stdout, stderr io.Writer) int {
|
||||||
|
fs := flag.NewFlagSet("clean-dupes", flag.ContinueOnError)
|
||||||
|
fs.SetOutput(stderr)
|
||||||
|
dryRun := fs.Bool("dry-run", false, "report deletions without removing")
|
||||||
|
if err := fs.Parse(args); err != nil {
|
||||||
|
return exitUsage
|
||||||
|
}
|
||||||
|
pos := fs.Args()
|
||||||
|
if len(pos) != 2 {
|
||||||
|
fmt.Fprintln(stderr, "assets clean-dupes: usage: clean-dupes [--dry-run] <primary> <clean>")
|
||||||
|
return exitUsage
|
||||||
|
}
|
||||||
|
primary, clean := pos[0], pos[1]
|
||||||
|
for _, d := range []string{primary, clean} {
|
||||||
|
if fi, err := os.Stat(d); err != nil || !fi.IsDir() {
|
||||||
|
fmt.Fprintf(stderr, "assets clean-dupes: no such dir: %s\n", d)
|
||||||
|
return exitUsage
|
||||||
|
}
|
||||||
|
}
|
||||||
|
primaryReal, err1 := filepath.EvalSymlinks(primary)
|
||||||
|
cleanReal, err2 := filepath.EvalSymlinks(clean)
|
||||||
|
if err1 != nil || err2 != nil {
|
||||||
|
fmt.Fprintln(stderr, "assets clean-dupes: cannot resolve dirs")
|
||||||
|
return exitTool
|
||||||
|
}
|
||||||
|
if overlaps(primaryReal, cleanReal) {
|
||||||
|
fmt.Fprintln(stderr, "assets clean-dupes: primary and clean dirs must not overlap")
|
||||||
|
return exitUsage
|
||||||
|
}
|
||||||
|
|
||||||
|
primaryFiles, err := allFiles(primary)
|
||||||
|
if err != nil {
|
||||||
|
fmt.Fprintln(stderr, "assets clean-dupes:", err)
|
||||||
|
return exitTool
|
||||||
|
}
|
||||||
|
names := map[string]bool{}
|
||||||
|
for _, f := range primaryFiles {
|
||||||
|
names[strings.ToLower(filepath.Base(f))] = true
|
||||||
|
}
|
||||||
|
|
||||||
|
cleanFiles, err := allFiles(clean)
|
||||||
|
if err != nil {
|
||||||
|
fmt.Fprintln(stderr, "assets clean-dupes:", err)
|
||||||
|
return exitTool
|
||||||
|
}
|
||||||
|
removed := 0
|
||||||
|
for _, f := range cleanFiles {
|
||||||
|
if !names[strings.ToLower(filepath.Base(f))] {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if *dryRun {
|
||||||
|
fmt.Fprintf(stderr, "would delete clean-tree collision: %s\n", f)
|
||||||
|
} else {
|
||||||
|
if err := os.Remove(f); err != nil {
|
||||||
|
fmt.Fprintln(stderr, "assets clean-dupes:", err)
|
||||||
|
return exitTool
|
||||||
|
}
|
||||||
|
fmt.Fprintf(stderr, "deleted clean-tree collision: %s\n", f)
|
||||||
|
}
|
||||||
|
removed++
|
||||||
|
}
|
||||||
|
verb := "removed"
|
||||||
|
if *dryRun {
|
||||||
|
verb = "would remove"
|
||||||
|
}
|
||||||
|
fmt.Fprintf(stdout, "clean-dupes: %s %d file(s)\n", verb, removed)
|
||||||
|
return exitOK
|
||||||
|
}
|
||||||
|
|
||||||
|
// overlaps reports whether either directory contains the other (or they are
|
||||||
|
// equal), using cleaned absolute-ish paths with a trailing separator.
|
||||||
|
func overlaps(a, b string) bool {
|
||||||
|
as := a + string(filepath.Separator)
|
||||||
|
bs := b + string(filepath.Separator)
|
||||||
|
return strings.HasPrefix(as, bs) || strings.HasPrefix(bs, as)
|
||||||
|
}
|
||||||
@@ -0,0 +1,80 @@
|
|||||||
|
package assets
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"testing"
|
||||||
|
)
|
||||||
|
|
||||||
|
func touch(t *testing.T, path string) {
|
||||||
|
t.Helper()
|
||||||
|
if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := os.WriteFile(path, []byte("x"), 0o644); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestCheckDupes(t *testing.T) {
|
||||||
|
root := t.TempDir()
|
||||||
|
touch(t, filepath.Join(root, "tex", "Foo.tga"))
|
||||||
|
touch(t, filepath.Join(root, "plc", "foo.tga")) // basename collision (case-insensitive)
|
||||||
|
|
||||||
|
var out, errw bytes.Buffer
|
||||||
|
if code := runCheckDupes([]string{root}, &out, &errw); code != exitFail {
|
||||||
|
t.Fatalf("collision exit = %d, want %d\n%s", code, exitFail, errw.String())
|
||||||
|
}
|
||||||
|
|
||||||
|
clean := t.TempDir()
|
||||||
|
touch(t, filepath.Join(clean, "a.tga"))
|
||||||
|
touch(t, filepath.Join(clean, "sub", "b.tga"))
|
||||||
|
out.Reset()
|
||||||
|
errw.Reset()
|
||||||
|
if code := runCheckDupes([]string{clean}, &out, &errw); code != exitOK {
|
||||||
|
t.Fatalf("no-collision exit = %d, want %d\n%s", code, exitOK, errw.String())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestCleanDupes(t *testing.T) {
|
||||||
|
primary := t.TempDir()
|
||||||
|
clean := t.TempDir()
|
||||||
|
touch(t, filepath.Join(primary, "keep.tga"))
|
||||||
|
collide := filepath.Join(clean, "sub", "Keep.tga")
|
||||||
|
survive := filepath.Join(clean, "unique.tga")
|
||||||
|
touch(t, collide)
|
||||||
|
touch(t, survive)
|
||||||
|
|
||||||
|
// dry-run removes nothing.
|
||||||
|
var out, errw bytes.Buffer
|
||||||
|
if code := runCleanDupes([]string{"--dry-run", primary, clean}, &out, &errw); code != exitOK {
|
||||||
|
t.Fatalf("dry-run exit = %d\n%s", code, errw.String())
|
||||||
|
}
|
||||||
|
if _, err := os.Stat(collide); err != nil {
|
||||||
|
t.Fatal("dry-run deleted a file")
|
||||||
|
}
|
||||||
|
|
||||||
|
// real run deletes the collision, keeps the unique file.
|
||||||
|
out.Reset()
|
||||||
|
errw.Reset()
|
||||||
|
if code := runCleanDupes([]string{primary, clean}, &out, &errw); code != exitOK {
|
||||||
|
t.Fatalf("clean exit = %d\n%s", code, errw.String())
|
||||||
|
}
|
||||||
|
if _, err := os.Stat(collide); !os.IsNotExist(err) {
|
||||||
|
t.Fatal("collision file was not deleted")
|
||||||
|
}
|
||||||
|
if _, err := os.Stat(survive); err != nil {
|
||||||
|
t.Fatal("unique file was wrongly deleted")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestCleanDupesRejectsOverlap(t *testing.T) {
|
||||||
|
root := t.TempDir()
|
||||||
|
sub := filepath.Join(root, "child")
|
||||||
|
touch(t, filepath.Join(sub, "x.tga"))
|
||||||
|
var out, errw bytes.Buffer
|
||||||
|
if code := runCleanDupes([]string{root, sub}, &out, &errw); code != exitUsage {
|
||||||
|
t.Fatalf("overlap exit = %d, want %d", code, exitUsage)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,103 @@
|
|||||||
|
package assets
|
||||||
|
|
||||||
|
import (
|
||||||
|
"flag"
|
||||||
|
"fmt"
|
||||||
|
"io"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"strings"
|
||||||
|
|
||||||
|
"git.westgate.pw/ShadowsOverWestgate/sow-tools/internal/assets/mdl"
|
||||||
|
)
|
||||||
|
|
||||||
|
// runFixMDL lowercases .mdl filenames and rewrites ASCII model identity to
|
||||||
|
// match each file's stem. Binary models are skipped (the engine owns those).
|
||||||
|
func runFixMDL(args []string, stdout, stderr io.Writer) int {
|
||||||
|
fs := flag.NewFlagSet("fix-mdl", flag.ContinueOnError)
|
||||||
|
fs.SetOutput(stderr)
|
||||||
|
dryRun := fs.Bool("dry-run", false, "report changes without touching disk")
|
||||||
|
nonRecursive := fs.Bool("non-recursive", false, "do not descend into subdirectories")
|
||||||
|
if err := fs.Parse(args); err != nil {
|
||||||
|
return exitUsage
|
||||||
|
}
|
||||||
|
paths := fs.Args()
|
||||||
|
if len(paths) == 0 {
|
||||||
|
fmt.Fprintln(stderr, "assets fix-mdl: usage: fix-mdl [--dry-run] <path>...")
|
||||||
|
return exitUsage
|
||||||
|
}
|
||||||
|
|
||||||
|
files, err := walk(paths, mdlExt, !*nonRecursive)
|
||||||
|
if err != nil {
|
||||||
|
fmt.Fprintln(stderr, "assets fix-mdl:", err)
|
||||||
|
return exitUsage
|
||||||
|
}
|
||||||
|
|
||||||
|
changed := 0
|
||||||
|
failed := false
|
||||||
|
for _, f := range files {
|
||||||
|
// 1. Lowercase the basename if needed.
|
||||||
|
//
|
||||||
|
// ponytail: 35k files are each read once here (CheckNames + the rewrite
|
||||||
|
// on candidates). The shell reference batched an awk pass to avoid
|
||||||
|
// per-file subprocess spawns; in Go a plain read is cheap, so the batch
|
||||||
|
// is not worth porting. If profiling ever shows this hot, parallelize
|
||||||
|
// the loop.
|
||||||
|
lower := f
|
||||||
|
if base := filepath.Base(f); base != strings.ToLower(base) {
|
||||||
|
lower = filepath.Join(filepath.Dir(f), strings.ToLower(base))
|
||||||
|
if _, err := os.Stat(lower); err == nil {
|
||||||
|
fmt.Fprintf(stderr, "assets fix-mdl: lowercase collision: %s -> %s\n", f, lower)
|
||||||
|
failed = true
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if *dryRun {
|
||||||
|
fmt.Fprintf(stdout, "would lowercase: %s -> %s\n", f, lower)
|
||||||
|
} else {
|
||||||
|
if err := os.Rename(f, lower); err != nil {
|
||||||
|
fmt.Fprintf(stderr, "assets fix-mdl: failed to lowercase %s: %v\n", f, err)
|
||||||
|
failed = true
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
fmt.Fprintf(stdout, "lowercased: %s -> %s\n", f, lower)
|
||||||
|
}
|
||||||
|
changed++
|
||||||
|
}
|
||||||
|
|
||||||
|
// 2. Rewrite model identity if the (lowercased) file has a mismatch.
|
||||||
|
// In dry-run the on-disk file is still the original path.
|
||||||
|
readPath := lower
|
||||||
|
if *dryRun && lower != f {
|
||||||
|
readPath = f
|
||||||
|
}
|
||||||
|
data, err := os.ReadFile(readPath)
|
||||||
|
if err != nil {
|
||||||
|
fmt.Fprintln(stderr, "assets fix-mdl:", err)
|
||||||
|
failed = true
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
out, wasChanged := mdl.FixNames(data, mdl.ExpectedName(lower))
|
||||||
|
if !wasChanged {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if *dryRun {
|
||||||
|
fmt.Fprintf(stdout, "would fix: %s\n", lower)
|
||||||
|
} else {
|
||||||
|
if err := os.WriteFile(lower, out, 0o644); err != nil {
|
||||||
|
fmt.Fprintln(stderr, "assets fix-mdl:", err)
|
||||||
|
failed = true
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
fmt.Fprintf(stdout, "fixed: %s\n", lower)
|
||||||
|
}
|
||||||
|
changed++
|
||||||
|
}
|
||||||
|
|
||||||
|
if changed == 0 {
|
||||||
|
fmt.Fprintln(stdout, "no broken mdl model names found")
|
||||||
|
}
|
||||||
|
if failed {
|
||||||
|
return exitFail
|
||||||
|
}
|
||||||
|
return exitOK
|
||||||
|
}
|
||||||
@@ -0,0 +1,48 @@
|
|||||||
|
package assets
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"git.westgate.pw/ShadowsOverWestgate/sow-tools/internal/assets/mdl"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestFixMDL(t *testing.T) {
|
||||||
|
dir := t.TempDir()
|
||||||
|
// Uppercase name + internal mismatch.
|
||||||
|
upper := filepath.Join(dir, "Foo.MDL")
|
||||||
|
body := "newmodel wrong\nbeginmodelgeom wrong\nendmodelgeom wrong\ndonemodel wrong\n"
|
||||||
|
if err := os.WriteFile(upper, []byte(body), 0o644); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// dry-run: nothing changes on disk.
|
||||||
|
var out, errw bytes.Buffer
|
||||||
|
if code := runFixMDL([]string{"--dry-run", dir}, &out, &errw); code != exitOK {
|
||||||
|
t.Fatalf("dry-run exit = %d\n%s", code, errw.String())
|
||||||
|
}
|
||||||
|
if _, err := os.Stat(upper); err != nil {
|
||||||
|
t.Fatal("dry-run renamed a file")
|
||||||
|
}
|
||||||
|
|
||||||
|
// real run: file is lowercased and its identity rewritten to match.
|
||||||
|
out.Reset()
|
||||||
|
errw.Reset()
|
||||||
|
if code := runFixMDL([]string{dir}, &out, &errw); code != exitOK {
|
||||||
|
t.Fatalf("fix exit = %d\n%s", code, errw.String())
|
||||||
|
}
|
||||||
|
lowered := filepath.Join(dir, "foo.mdl")
|
||||||
|
if _, err := os.Stat(lowered); err != nil {
|
||||||
|
t.Fatalf("file was not lowercased: %v", err)
|
||||||
|
}
|
||||||
|
mismatches, err := mdl.CheckNames(lowered)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if len(mismatches) != 0 {
|
||||||
|
data, _ := os.ReadFile(lowered)
|
||||||
|
t.Fatalf("model still has mismatches after fix: %+v\n%s", mismatches, data)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,84 @@
|
|||||||
|
// Package assets is the crucible `assets` builder: NWN:EE model/texture tools
|
||||||
|
// that operate in place on a target directory. Mirrors internal/depot's
|
||||||
|
// Run(args, stdout, stderr, getenv) shape and bypasses the legacy internal/app
|
||||||
|
// surface entirely.
|
||||||
|
package assets
|
||||||
|
|
||||||
|
import (
|
||||||
|
"os"
|
||||||
|
"os/exec"
|
||||||
|
"path/filepath"
|
||||||
|
"sort"
|
||||||
|
"strings"
|
||||||
|
)
|
||||||
|
|
||||||
|
// runner is the single external-command indirection so tests can observe and
|
||||||
|
// stub every engine / ImageMagick / upscaler invocation. dir is the working
|
||||||
|
// directory ("" = inherit); env replaces the child environment when non-nil.
|
||||||
|
var runner = func(dir string, env []string, name string, args ...string) ([]byte, error) {
|
||||||
|
cmd := exec.Command(name, args...)
|
||||||
|
cmd.Dir = dir
|
||||||
|
if env != nil {
|
||||||
|
cmd.Env = env
|
||||||
|
}
|
||||||
|
return cmd.CombinedOutput()
|
||||||
|
}
|
||||||
|
|
||||||
|
// walk collects files under each root whose lower-cased extension is in exts. A
|
||||||
|
// root that is itself a matching file is included. recursive controls descent
|
||||||
|
// into subdirectories. Results are sorted and deduplicated.
|
||||||
|
func walk(roots []string, exts map[string]bool, recursive bool) ([]string, error) {
|
||||||
|
seen := map[string]bool{}
|
||||||
|
var out []string
|
||||||
|
add := func(p string) {
|
||||||
|
if exts[strings.ToLower(filepath.Ext(p))] && !seen[p] {
|
||||||
|
seen[p] = true
|
||||||
|
out = append(out, p)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for _, root := range roots {
|
||||||
|
fi, err := os.Stat(root)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
if !fi.IsDir() {
|
||||||
|
add(root)
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
err = filepath.WalkDir(root, func(p string, d os.DirEntry, err error) error {
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if d.IsDir() {
|
||||||
|
if !recursive && p != root {
|
||||||
|
return filepath.SkipDir
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
add(p)
|
||||||
|
return nil
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
sort.Strings(out)
|
||||||
|
return out, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// look returns the first bare name found on PATH, or the first candidate that
|
||||||
|
// is an existing path. Returns "" if none resolve.
|
||||||
|
func look(candidates ...string) string {
|
||||||
|
for _, c := range candidates {
|
||||||
|
if strings.ContainsRune(c, filepath.Separator) {
|
||||||
|
if fi, err := os.Stat(c); err == nil && !fi.IsDir() {
|
||||||
|
return c
|
||||||
|
}
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if p, err := exec.LookPath(c); err == nil {
|
||||||
|
return p
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return ""
|
||||||
|
}
|
||||||
@@ -0,0 +1,63 @@
|
|||||||
|
package assets
|
||||||
|
|
||||||
|
import (
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"testing"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestWalk(t *testing.T) {
|
||||||
|
root := t.TempDir()
|
||||||
|
must := func(rel string) {
|
||||||
|
p := filepath.Join(root, rel)
|
||||||
|
if err := os.MkdirAll(filepath.Dir(p), 0o755); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := os.WriteFile(p, []byte("x"), 0o644); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
must("a.MDL")
|
||||||
|
must("sub/b.mdl")
|
||||||
|
must("sub/c.txt")
|
||||||
|
|
||||||
|
exts := map[string]bool{".mdl": true}
|
||||||
|
got, err := walk([]string{root}, exts, true)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if len(got) != 2 {
|
||||||
|
t.Fatalf("recursive walk = %v, want 2 mdl files", got)
|
||||||
|
}
|
||||||
|
|
||||||
|
got, _ = walk([]string{root}, exts, false)
|
||||||
|
if len(got) != 1 || filepath.Base(got[0]) != "a.MDL" {
|
||||||
|
t.Fatalf("non-recursive walk = %v, want only a.MDL", got)
|
||||||
|
}
|
||||||
|
|
||||||
|
// A file argument that matches is included directly.
|
||||||
|
got, _ = walk([]string{filepath.Join(root, "sub", "b.mdl")}, exts, true)
|
||||||
|
if len(got) != 1 {
|
||||||
|
t.Fatalf("file arg walk = %v, want the file itself", got)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestLook(t *testing.T) {
|
||||||
|
dir := t.TempDir()
|
||||||
|
stub := filepath.Join(dir, "mytool")
|
||||||
|
if err := os.WriteFile(stub, []byte("#!/bin/sh\n"), 0o755); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
t.Setenv("PATH", dir)
|
||||||
|
|
||||||
|
if got := look("nope-not-here", "mytool"); got == "" {
|
||||||
|
t.Fatal("look should find mytool on PATH")
|
||||||
|
}
|
||||||
|
// Explicit existing path candidate.
|
||||||
|
if got := look(stub); got != stub {
|
||||||
|
t.Fatalf("look(%q) = %q, want the path itself", stub, got)
|
||||||
|
}
|
||||||
|
if got := look("definitely-absent-binary-xyz"); got != "" {
|
||||||
|
t.Fatalf("look = %q, want empty", got)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,218 @@
|
|||||||
|
// Package mdl ports the ASCII MDL model-name checks from sow-assets-manifest's
|
||||||
|
// mdl-name-lib.sh + mdl-scan.awk to pure Go. Binary (compiled) MDLs are opaque
|
||||||
|
// here; the NWN engine is the validator for those. Detection mirrors
|
||||||
|
// mdl-scan.awk exactly so behavior does not drift while that awk still runs in
|
||||||
|
// sow-assets-manifest.
|
||||||
|
package mdl
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"strings"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Mismatch is one model-name problem. Line is 1-based; 0 means the geometry
|
||||||
|
// base-node mismatch, which the source has no single line for.
|
||||||
|
type Mismatch struct {
|
||||||
|
Line int
|
||||||
|
What string
|
||||||
|
Got string
|
||||||
|
}
|
||||||
|
|
||||||
|
// ExpectedName is the file stem with a trailing ".mdl" (any case) removed.
|
||||||
|
func ExpectedName(path string) string {
|
||||||
|
base := filepath.Base(path)
|
||||||
|
if len(base) >= 4 && strings.EqualFold(base[len(base)-4:], ".mdl") {
|
||||||
|
return base[:len(base)-4]
|
||||||
|
}
|
||||||
|
return base
|
||||||
|
}
|
||||||
|
|
||||||
|
// ExpectedNameStem is an alias of ExpectedName kept for call-site clarity where
|
||||||
|
// the value is used as the engine's model stem argument.
|
||||||
|
func ExpectedNameStem(path string) string { return ExpectedName(path) }
|
||||||
|
|
||||||
|
// IsASCII reports whether path is an uncompiled ASCII model: no NUL in the
|
||||||
|
// first 256 bytes, and a leading keyword of '#' / newmodel / node /
|
||||||
|
// setsupermodel. Mirrors mdl_is_ascii.
|
||||||
|
func IsASCII(path string) (bool, error) {
|
||||||
|
data, err := os.ReadFile(path)
|
||||||
|
if err != nil {
|
||||||
|
return false, err
|
||||||
|
}
|
||||||
|
return isASCII(data), nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func isASCII(data []byte) bool {
|
||||||
|
head := data
|
||||||
|
if len(head) > 256 {
|
||||||
|
head = head[:256]
|
||||||
|
}
|
||||||
|
if bytes.IndexByte(head, 0) >= 0 {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
line := head
|
||||||
|
if i := bytes.IndexByte(line, '\n'); i >= 0 {
|
||||||
|
line = line[:i]
|
||||||
|
}
|
||||||
|
trimmed := strings.TrimLeft(string(line), " \t\r")
|
||||||
|
if strings.HasPrefix(trimmed, "#") {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
fields := strings.Fields(trimmed)
|
||||||
|
if len(fields) == 0 {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
tok := strings.ToLower(fields[0])
|
||||||
|
return strings.HasPrefix(tok, "newmodel") ||
|
||||||
|
strings.HasPrefix(tok, "node") ||
|
||||||
|
strings.HasPrefix(tok, "setsupermodel")
|
||||||
|
}
|
||||||
|
|
||||||
|
// lineFields splits a line into whitespace-delimited tokens with the trailing
|
||||||
|
// CR removed, matching awk's default field split + \r strip.
|
||||||
|
func lineFields(line string) []string {
|
||||||
|
return strings.Fields(strings.TrimRight(line, "\r"))
|
||||||
|
}
|
||||||
|
|
||||||
|
// field returns the 1-based nth field or "" if absent.
|
||||||
|
func field(f []string, n int) string {
|
||||||
|
if n >= 1 && n <= len(f) {
|
||||||
|
return f[n-1]
|
||||||
|
}
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
|
||||||
|
// CheckNames returns every model-name mismatch in an ASCII model. Returns nil
|
||||||
|
// for a binary model or one with no newmodel line (not a real model).
|
||||||
|
func CheckNames(path string) ([]Mismatch, error) {
|
||||||
|
data, err := os.ReadFile(path)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
if !isASCII(data) {
|
||||||
|
return nil, nil
|
||||||
|
}
|
||||||
|
return checkNames(data, ExpectedName(path)), nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func checkNames(data []byte, expected string) []Mismatch {
|
||||||
|
expLower := strings.ToLower(expected)
|
||||||
|
var out []Mismatch
|
||||||
|
seenModel := false
|
||||||
|
inGeom := false
|
||||||
|
curNode := ""
|
||||||
|
base := ""
|
||||||
|
baseFound := false
|
||||||
|
|
||||||
|
// header records one header-token mismatch (field n, 1-based) if the token
|
||||||
|
// is present and differs case-insensitively from the stem.
|
||||||
|
lines := strings.Split(string(data), "\n")
|
||||||
|
for i, raw := range lines {
|
||||||
|
f := lineFields(raw)
|
||||||
|
if len(f) == 0 {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
key := strings.ToLower(f[0])
|
||||||
|
mis := func(what string, n int) {
|
||||||
|
got := field(f, n)
|
||||||
|
if got != "" && !strings.EqualFold(got, expected) {
|
||||||
|
out = append(out, Mismatch{Line: i + 1, What: what, Got: got})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
switch key {
|
||||||
|
case "newmodel":
|
||||||
|
seenModel = true
|
||||||
|
mis("newmodel", 2)
|
||||||
|
case "setsupermodel":
|
||||||
|
mis("setsupermodel model", 2)
|
||||||
|
case "beginmodelgeom":
|
||||||
|
inGeom = true
|
||||||
|
mis("beginmodelgeom", 2)
|
||||||
|
case "endmodelgeom":
|
||||||
|
inGeom = false
|
||||||
|
mis("endmodelgeom", 2)
|
||||||
|
case "donemodel":
|
||||||
|
mis("donemodel", 2)
|
||||||
|
case "newanim":
|
||||||
|
mis("newanim model", 3)
|
||||||
|
case "doneanim":
|
||||||
|
mis("doneanim model", 3)
|
||||||
|
case "node":
|
||||||
|
if inGeom {
|
||||||
|
curNode = field(f, 3)
|
||||||
|
}
|
||||||
|
case "parent":
|
||||||
|
if inGeom && !baseFound && strings.EqualFold(field(f, 2), "null") {
|
||||||
|
base = curNode
|
||||||
|
baseFound = true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if !seenModel {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
if baseFound && base != "" && !strings.EqualFold(base, expLower) {
|
||||||
|
out = append(out, Mismatch{Line: 0, What: "root node", Got: base})
|
||||||
|
}
|
||||||
|
return out
|
||||||
|
}
|
||||||
|
|
||||||
|
// FixNames rewrites only the model identity to expected: the header tokens plus
|
||||||
|
// the geometry base node's declaration / any parent / animroot pointing at the
|
||||||
|
// old base name. Animation bone names and supermodel animroots are left alone
|
||||||
|
// so inheritance keeps working. Mirrors mdl_fix_model_name in intent.
|
||||||
|
//
|
||||||
|
// ponytail: a rewritten line is rebuilt joining fields with single spaces and
|
||||||
|
// drops a trailing CR, exactly like the awk it replaces; unchanged lines are
|
||||||
|
// byte-identical. The engine ignores identity-line whitespace, and FixNames'
|
||||||
|
// contract is only "produces a model that passes CheckNames", not byte parity.
|
||||||
|
func FixNames(in []byte, expected string) (out []byte, changed bool) {
|
||||||
|
if !isASCII(in) {
|
||||||
|
return in, false
|
||||||
|
}
|
||||||
|
// oldbase: the base-node name to rename, only if it differs from expected.
|
||||||
|
oldbase := ""
|
||||||
|
for _, m := range checkNames(in, expected) {
|
||||||
|
if m.What == "root node" {
|
||||||
|
oldbase = m.Got
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
lines := strings.Split(string(in), "\n")
|
||||||
|
for idx, raw := range lines {
|
||||||
|
f := lineFields(raw)
|
||||||
|
if len(f) == 0 {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
key := strings.ToLower(f[0])
|
||||||
|
set := func(n int) bool {
|
||||||
|
if field(f, n) == "" {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
f[n-1] = expected
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
modified := false
|
||||||
|
switch key {
|
||||||
|
case "newmodel", "setsupermodel", "beginmodelgeom", "endmodelgeom", "donemodel":
|
||||||
|
modified = set(2)
|
||||||
|
case "newanim", "doneanim":
|
||||||
|
modified = set(3)
|
||||||
|
case "node":
|
||||||
|
if oldbase != "" && strings.EqualFold(field(f, 3), oldbase) {
|
||||||
|
modified = set(3)
|
||||||
|
}
|
||||||
|
case "parent", "animroot":
|
||||||
|
if oldbase != "" && strings.EqualFold(field(f, 2), oldbase) {
|
||||||
|
modified = set(2)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if modified {
|
||||||
|
lines[idx] = strings.Join(f, " ")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
out = []byte(strings.Join(lines, "\n"))
|
||||||
|
return out, !bytes.Equal(out, in)
|
||||||
|
}
|
||||||
@@ -0,0 +1,114 @@
|
|||||||
|
package mdl
|
||||||
|
|
||||||
|
import (
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"testing"
|
||||||
|
)
|
||||||
|
|
||||||
|
func writeMDL(t *testing.T, name, body string) string {
|
||||||
|
t.Helper()
|
||||||
|
p := filepath.Join(t.TempDir(), name)
|
||||||
|
if err := os.WriteFile(p, []byte(body), 0o644); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
return p
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestIsASCII(t *testing.T) {
|
||||||
|
ascii := writeMDL(t, "foo.mdl", "newmodel foo\nbeginmodelgeom foo\n")
|
||||||
|
if ok, err := IsASCII(ascii); err != nil || !ok {
|
||||||
|
t.Fatalf("ascii: ok=%v err=%v, want true nil", ok, err)
|
||||||
|
}
|
||||||
|
bin := writeMDL(t, "bar.mdl", "\x00\x01binary\x00garbage")
|
||||||
|
if ok, err := IsASCII(bin); err != nil || ok {
|
||||||
|
t.Fatalf("binary: ok=%v err=%v, want false nil", ok, err)
|
||||||
|
}
|
||||||
|
hash := writeMDL(t, "baz.mdl", "# a comment\nnewmodel baz\n")
|
||||||
|
if ok, _ := IsASCII(hash); !ok {
|
||||||
|
t.Fatal("leading # should be ascii")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestExpectedName(t *testing.T) {
|
||||||
|
for in, want := range map[string]string{
|
||||||
|
"a/b/Foo.MDL": "Foo",
|
||||||
|
"waxbt_b_091.mdl": "waxbt_b_091",
|
||||||
|
"x.mdl": "x",
|
||||||
|
} {
|
||||||
|
if got := ExpectedName(in); got != want {
|
||||||
|
t.Errorf("ExpectedName(%q)=%q want %q", in, got, want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestCheckNames(t *testing.T) {
|
||||||
|
// Header token mismatch + base-node mismatch in one file.
|
||||||
|
body := "newmodel wrong\n" +
|
||||||
|
"setsupermodel wrong a_base\n" +
|
||||||
|
"beginmodelgeom foo\n" +
|
||||||
|
" node dummy Wmgst_m_081\n" +
|
||||||
|
" parent null\n" +
|
||||||
|
" endnode\n" +
|
||||||
|
"endmodelgeom foo\n" +
|
||||||
|
"donemodel foo\n"
|
||||||
|
p := writeMDL(t, "foo.mdl", body)
|
||||||
|
got, err := CheckNames(p)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
// Expected stem "foo": newmodel(wrong), setsupermodel(wrong), base node(Wmgst_m_081).
|
||||||
|
if len(got) != 3 {
|
||||||
|
t.Fatalf("got %d mismatches, want 3: %+v", len(got), got)
|
||||||
|
}
|
||||||
|
if got[0].What != "newmodel" || got[0].Got != "wrong" || got[0].Line != 1 {
|
||||||
|
t.Errorf("first mismatch = %+v", got[0])
|
||||||
|
}
|
||||||
|
base := got[len(got)-1]
|
||||||
|
if base.What != "root node" || base.Got != "Wmgst_m_081" || base.Line != 0 {
|
||||||
|
t.Errorf("base mismatch = %+v", base)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Clean model -> no mismatches.
|
||||||
|
clean := writeMDL(t, "bar.mdl",
|
||||||
|
"newmodel bar\nbeginmodelgeom bar\n node dummy bar\n parent null\n endnode\nendmodelgeom bar\ndonemodel bar\n")
|
||||||
|
if got, _ := CheckNames(clean); len(got) != 0 {
|
||||||
|
t.Fatalf("clean model reported mismatches: %+v", got)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Binary model -> nil, no error.
|
||||||
|
bin := writeMDL(t, "b.mdl", "\x00\x00binary")
|
||||||
|
if got, _ := CheckNames(bin); got != nil {
|
||||||
|
t.Fatalf("binary reported mismatches: %+v", got)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestFixNamesRoundTrips(t *testing.T) {
|
||||||
|
body := "newmodel wrong\n" +
|
||||||
|
"setsupermodel wrong\n" +
|
||||||
|
"beginmodelgeom wrong\n" +
|
||||||
|
" node dummy Wmgst_m_081\n" +
|
||||||
|
" parent null\n" +
|
||||||
|
" endnode\n" +
|
||||||
|
"endmodelgeom wrong\n" +
|
||||||
|
"donemodel wrong\n"
|
||||||
|
out, changed := FixNames([]byte(body), "foo")
|
||||||
|
if !changed {
|
||||||
|
t.Fatal("expected changed=true")
|
||||||
|
}
|
||||||
|
p := filepath.Join(t.TempDir(), "foo.mdl")
|
||||||
|
if err := os.WriteFile(p, out, 0o644); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
got, err := CheckNames(p)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if len(got) != 0 {
|
||||||
|
t.Fatalf("fixed model still has mismatches: %+v\n%s", got, out)
|
||||||
|
}
|
||||||
|
// Idempotent: fixing a clean model changes nothing.
|
||||||
|
if _, changed := FixNames(out, "foo"); changed {
|
||||||
|
t.Fatal("second fix reported a change")
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,55 @@
|
|||||||
|
package assets
|
||||||
|
|
||||||
|
import (
|
||||||
|
"fmt"
|
||||||
|
"io"
|
||||||
|
)
|
||||||
|
|
||||||
|
const (
|
||||||
|
exitOK = 0
|
||||||
|
exitFail = 1 // problems found / per-file failures
|
||||||
|
exitUsage = 64 // bad invocation / unknown subcommand / bad flags
|
||||||
|
exitTool = 70 // missing external tool / internal error
|
||||||
|
)
|
||||||
|
|
||||||
|
// Run executes an assets subcommand. args[0] is the subcommand; returns the
|
||||||
|
// process exit code.
|
||||||
|
func Run(args []string, stdout, stderr io.Writer, getenv func(string) string) int {
|
||||||
|
if len(args) == 0 {
|
||||||
|
printUsage(stderr)
|
||||||
|
return exitUsage
|
||||||
|
}
|
||||||
|
rest := args[1:]
|
||||||
|
switch args[0] {
|
||||||
|
case "check-dupes":
|
||||||
|
return runCheckDupes(rest, stdout, stderr)
|
||||||
|
case "clean-dupes":
|
||||||
|
return runCleanDupes(rest, stdout, stderr)
|
||||||
|
case "check-mdl":
|
||||||
|
return runCheckMDL(rest, stdout, stderr)
|
||||||
|
case "fix-mdl":
|
||||||
|
return runFixMDL(rest, stdout, stderr)
|
||||||
|
case "convert":
|
||||||
|
return runConvert(rest, stdout, stderr)
|
||||||
|
case "upscale":
|
||||||
|
return runUpscale(rest, stdout, stderr)
|
||||||
|
case "compile":
|
||||||
|
return runCompile(rest, stdout, stderr, getenv)
|
||||||
|
default:
|
||||||
|
fmt.Fprintf(stderr, "assets: unknown subcommand %q\n\n", args[0])
|
||||||
|
printUsage(stderr)
|
||||||
|
return exitUsage
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func printUsage(w io.Writer) {
|
||||||
|
fmt.Fprint(w, `usage:
|
||||||
|
assets compile [--nwn INSTALL] [--non-recursive] <dir>...
|
||||||
|
assets convert [--to dds|png|tga] [--backend PATH] [--non-recursive] <dir>...
|
||||||
|
assets upscale [--scale N] [--backend PATH] [--non-recursive] <dir>...
|
||||||
|
assets check-mdl <path>...
|
||||||
|
assets fix-mdl [--dry-run] <path>...
|
||||||
|
assets check-dupes <dir>...
|
||||||
|
assets clean-dupes [--dry-run] <primary> <clean>
|
||||||
|
`)
|
||||||
|
}
|
||||||
@@ -0,0 +1,26 @@
|
|||||||
|
package assets
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
)
|
||||||
|
|
||||||
|
func env(string) string { return "" }
|
||||||
|
|
||||||
|
func TestRunNoArgsIsUsage(t *testing.T) {
|
||||||
|
var out, errw bytes.Buffer
|
||||||
|
if code := Run(nil, &out, &errw, env); code != exitUsage {
|
||||||
|
t.Fatalf("no args exit = %d, want %d", code, exitUsage)
|
||||||
|
}
|
||||||
|
if !strings.Contains(errw.String(), "usage") {
|
||||||
|
t.Fatalf("no args should print usage, got: %q", errw.String())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestRunUnknownSubcommandIsUsage(t *testing.T) {
|
||||||
|
var out, errw bytes.Buffer
|
||||||
|
if code := Run([]string{"frobnicate"}, &out, &errw, env); code != exitUsage {
|
||||||
|
t.Fatalf("unknown subcommand exit = %d, want %d", code, exitUsage)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,98 @@
|
|||||||
|
package assets
|
||||||
|
|
||||||
|
import (
|
||||||
|
"flag"
|
||||||
|
"fmt"
|
||||||
|
"io"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"strconv"
|
||||||
|
"strings"
|
||||||
|
)
|
||||||
|
|
||||||
|
var upscaleBackends = []string{"upscayl-bin", "upscayl", "realesrgan-ncnn-vulkan", "waifu2x-ncnn-vulkan"}
|
||||||
|
|
||||||
|
// runUpscale upscales textures in place through an installed ncnn-vulkan
|
||||||
|
// backend. DDS inputs are bridged through PNG so the NWN flip stays correct.
|
||||||
|
func runUpscale(args []string, stdout, stderr io.Writer) int {
|
||||||
|
fs := flag.NewFlagSet("upscale", flag.ContinueOnError)
|
||||||
|
fs.SetOutput(stderr)
|
||||||
|
scale := fs.Int("scale", 4, "upscale factor")
|
||||||
|
backend := fs.String("backend", "", "override the upscaler binary")
|
||||||
|
nonRecursive := fs.Bool("non-recursive", false, "do not descend into subdirectories")
|
||||||
|
if err := fs.Parse(args); err != nil {
|
||||||
|
return exitUsage
|
||||||
|
}
|
||||||
|
dirs := fs.Args()
|
||||||
|
if len(dirs) == 0 {
|
||||||
|
fmt.Fprintln(stderr, "assets upscale: usage: upscale [--scale N] <dir>...")
|
||||||
|
return exitUsage
|
||||||
|
}
|
||||||
|
|
||||||
|
candidates := upscaleBackends
|
||||||
|
if *backend != "" {
|
||||||
|
candidates = []string{*backend}
|
||||||
|
}
|
||||||
|
tool := look(candidates...)
|
||||||
|
if tool == "" {
|
||||||
|
fmt.Fprintf(stderr, "assets upscale: no upscaler found — install one of: %s\n", strings.Join(upscaleBackends, ", "))
|
||||||
|
return exitTool
|
||||||
|
}
|
||||||
|
|
||||||
|
files, err := walk(dirs, textureExts, !*nonRecursive)
|
||||||
|
if err != nil {
|
||||||
|
fmt.Fprintln(stderr, "assets upscale:", err)
|
||||||
|
return exitUsage
|
||||||
|
}
|
||||||
|
|
||||||
|
// magick is only needed if a .dds input is present; resolve lazily.
|
||||||
|
magick := ""
|
||||||
|
failed := false
|
||||||
|
for _, src := range files {
|
||||||
|
var upErr error
|
||||||
|
if strings.EqualFold(filepath.Ext(src), ".dds") {
|
||||||
|
if magick == "" {
|
||||||
|
if magick = look("magick"); magick == "" {
|
||||||
|
fmt.Fprintln(stderr, "assets upscale: .dds input needs ImageMagick (magick) for the png bridge")
|
||||||
|
return exitTool
|
||||||
|
}
|
||||||
|
}
|
||||||
|
upErr = upscaleDDS(tool, magick, src, *scale)
|
||||||
|
} else {
|
||||||
|
upErr = upscaleImage(tool, src, src, *scale)
|
||||||
|
}
|
||||||
|
if upErr != nil {
|
||||||
|
fmt.Fprintf(stderr, "assets upscale: %s: %v\n", src, upErr)
|
||||||
|
failed = true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if failed {
|
||||||
|
return exitFail
|
||||||
|
}
|
||||||
|
return exitOK
|
||||||
|
}
|
||||||
|
|
||||||
|
// upscaleImage runs the ncnn-vulkan backend to upscale src into dst (may be the
|
||||||
|
// same path).
|
||||||
|
func upscaleImage(tool, src, dst string, scale int) error {
|
||||||
|
tmp := dst + ".upscaled.png"
|
||||||
|
out, err := runner("", nil, tool, "-i", src, "-o", tmp, "-s", strconv.Itoa(scale))
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("%s: %v: %s", filepath.Base(tool), err, strings.TrimSpace(string(out)))
|
||||||
|
}
|
||||||
|
return os.Rename(tmp, dst)
|
||||||
|
}
|
||||||
|
|
||||||
|
// upscaleDDS bridges a DDS through PNG: dds->png (flip), upscale, png->dds
|
||||||
|
// (flip back), yielding a correctly-flipped upscaled DDS.
|
||||||
|
func upscaleDDS(tool, magick, src string, scale int) error {
|
||||||
|
tmpPNG := src + ".bridge.png"
|
||||||
|
if err := ddsToPNG(magick, src, tmpPNG); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
defer os.Remove(tmpPNG)
|
||||||
|
if err := upscaleImage(tool, tmpPNG, tmpPNG, scale); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
return pngToDDS(magick, tmpPNG, src)
|
||||||
|
}
|
||||||
@@ -0,0 +1,69 @@
|
|||||||
|
package assets
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"testing"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestUpscalePNGUsesBackend(t *testing.T) {
|
||||||
|
// A fake backend binary on PATH so look() resolves it.
|
||||||
|
binDir := t.TempDir()
|
||||||
|
fake := filepath.Join(binDir, "upscayl-bin")
|
||||||
|
if err := os.WriteFile(fake, []byte("#!/bin/sh\n"), 0o755); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
t.Setenv("PATH", binDir)
|
||||||
|
|
||||||
|
// Stub runner: emulate `-i in -o out -s scale` by copying in->out.
|
||||||
|
orig := runner
|
||||||
|
defer func() { runner = orig }()
|
||||||
|
var gotScale string
|
||||||
|
runner = func(dir string, env []string, name string, args ...string) ([]byte, error) {
|
||||||
|
var in, out string
|
||||||
|
for i := 0; i < len(args)-1; i++ {
|
||||||
|
switch args[i] {
|
||||||
|
case "-i":
|
||||||
|
in = args[i+1]
|
||||||
|
case "-o":
|
||||||
|
out = args[i+1]
|
||||||
|
case "-s":
|
||||||
|
gotScale = args[i+1]
|
||||||
|
}
|
||||||
|
}
|
||||||
|
data, err := os.ReadFile(in)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
return nil, os.WriteFile(out, data, 0o644)
|
||||||
|
}
|
||||||
|
|
||||||
|
dir := t.TempDir()
|
||||||
|
src := filepath.Join(dir, "tex.png")
|
||||||
|
if err := os.WriteFile(src, []byte("pngdata"), 0o644); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
var stdout, stderr bytes.Buffer
|
||||||
|
if code := runUpscale([]string{"--scale", "2", dir}, &stdout, &stderr); code != exitOK {
|
||||||
|
t.Fatalf("upscale exit = %d\n%s", code, stderr.String())
|
||||||
|
}
|
||||||
|
if gotScale != "2" {
|
||||||
|
t.Fatalf("scale passed to backend = %q, want 2", gotScale)
|
||||||
|
}
|
||||||
|
if data, _ := os.ReadFile(src); string(data) != "pngdata" {
|
||||||
|
t.Fatalf("upscaled file content = %q, want the backend output", data)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestUpscaleNoBackendFailsClosed(t *testing.T) {
|
||||||
|
t.Setenv("PATH", t.TempDir()) // empty PATH: no backend resolvable
|
||||||
|
dir := t.TempDir()
|
||||||
|
if err := os.WriteFile(filepath.Join(dir, "tex.png"), []byte("x"), 0o644); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
var stdout, stderr bytes.Buffer
|
||||||
|
if code := runUpscale([]string{dir}, &stdout, &stderr); code != exitTool {
|
||||||
|
t.Fatalf("no-backend exit = %d, want %d", code, exitTool)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -143,7 +143,7 @@ func runStatus(args []string, stdout, stderr io.Writer, getenv func(string) stri
|
|||||||
}
|
}
|
||||||
shas := shaKeys(shaSizes)
|
shas := shaKeys(shaSizes)
|
||||||
|
|
||||||
res, err := Sweep(context.Background(), backend, shas, cfg, io.Discard)
|
res, err := Sweep(context.Background(), backend, shas, cfg, stderr)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
fmt.Fprintln(stderr, "depot status:", err)
|
fmt.Fprintln(stderr, "depot status:", err)
|
||||||
return exitInternal
|
return exitInternal
|
||||||
@@ -185,7 +185,7 @@ func runPush(args []string, stdout, stderr io.Writer, getenv func(string) string
|
|||||||
}
|
}
|
||||||
shas := shaKeys(shaSizes)
|
shas := shaKeys(shaSizes)
|
||||||
|
|
||||||
res, err := Sweep(context.Background(), backend, shas, cfg, io.Discard)
|
res, err := Sweep(context.Background(), backend, shas, cfg, stderr)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
fmt.Fprintln(stderr, "depot push:", err)
|
fmt.Fprintln(stderr, "depot push:", err)
|
||||||
return exitInternal
|
return exitInternal
|
||||||
@@ -266,7 +266,7 @@ func runVerify(args []string, stdout, stderr io.Writer, getenv func(string) stri
|
|||||||
}
|
}
|
||||||
shas := shaKeys(shaSizes)
|
shas := shaKeys(shaSizes)
|
||||||
|
|
||||||
res, err := Sweep(context.Background(), backend, shas, cfg, io.Discard)
|
res, err := Sweep(context.Background(), backend, shas, cfg, stderr)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
fmt.Fprintln(stderr, "depot verify:", err)
|
fmt.Fprintln(stderr, "depot verify:", err)
|
||||||
return exitInternal
|
return exitInternal
|
||||||
@@ -389,7 +389,7 @@ func runPull(args []string, stdout, stderr io.Writer, getenv func(string) string
|
|||||||
|
|
||||||
// Sweep the source to distinguish "not there" (exit 1, listed) from
|
// Sweep the source to distinguish "not there" (exit 1, listed) from
|
||||||
// "there, download it".
|
// "there, download it".
|
||||||
res, err := Sweep(context.Background(), backend, toCheck, cfg, io.Discard)
|
res, err := Sweep(context.Background(), backend, toCheck, cfg, stderr)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
fmt.Fprintln(stderr, "depot pull:", err)
|
fmt.Fprintln(stderr, "depot pull:", err)
|
||||||
return exitInternal
|
return exitInternal
|
||||||
|
|||||||
@@ -17,6 +17,7 @@ import (
|
|||||||
"os"
|
"os"
|
||||||
|
|
||||||
"git.westgate.pw/ShadowsOverWestgate/sow-tools/internal/app"
|
"git.westgate.pw/ShadowsOverWestgate/sow-tools/internal/app"
|
||||||
|
"git.westgate.pw/ShadowsOverWestgate/sow-tools/internal/assets"
|
||||||
"git.westgate.pw/ShadowsOverWestgate/sow-tools/internal/buildinfo"
|
"git.westgate.pw/ShadowsOverWestgate/sow-tools/internal/buildinfo"
|
||||||
"git.westgate.pw/ShadowsOverWestgate/sow-tools/internal/depot"
|
"git.westgate.pw/ShadowsOverWestgate/sow-tools/internal/depot"
|
||||||
"git.westgate.pw/ShadowsOverWestgate/sow-tools/internal/menu"
|
"git.westgate.pw/ShadowsOverWestgate/sow-tools/internal/menu"
|
||||||
@@ -103,6 +104,21 @@ var Registry = []Builder{
|
|||||||
},
|
},
|
||||||
Wired: true,
|
Wired: true,
|
||||||
},
|
},
|
||||||
|
{
|
||||||
|
Name: "assets",
|
||||||
|
Bin: "crucible-assets",
|
||||||
|
Summary: "compile/convert/upscale NWN assets + mdl/dupe integrity",
|
||||||
|
Commands: []Command{
|
||||||
|
{Name: "compile", Summary: "compile ASCII .mdl models to binary in place", Usage: "crucible assets compile [--nwn INSTALL] [--non-recursive] <dir>..."},
|
||||||
|
{Name: "convert", Summary: "convert textures to/from NWN DDS (flips vertically)", Usage: "crucible assets convert [--to dds|png|tga] [--backend PATH] [--non-recursive] <dir>..."},
|
||||||
|
{Name: "upscale", Summary: "upscale textures through an installed backend", Usage: "crucible assets upscale [--scale N] [--backend PATH] [--non-recursive] <dir>..."},
|
||||||
|
{Name: "check-mdl", Summary: "report uncompiled ASCII .mdl and model-name mismatches", Usage: "crucible assets check-mdl <path>..."},
|
||||||
|
{Name: "fix-mdl", Summary: "lowercase .mdl names and rewrite model identity to match", Usage: "crucible assets fix-mdl [--dry-run] <path>..."},
|
||||||
|
{Name: "check-dupes", Summary: "report runtime-name (basename) collisions across dirs", Usage: "crucible assets check-dupes <dir>..."},
|
||||||
|
{Name: "clean-dupes", Summary: "delete clean-tree files whose basename collides with primary", Usage: "crucible assets clean-dupes [--dry-run] <primary> <clean>"},
|
||||||
|
},
|
||||||
|
Wired: true,
|
||||||
|
},
|
||||||
{
|
{
|
||||||
Name: "hak",
|
Name: "hak",
|
||||||
Bin: "crucible-hak",
|
Bin: "crucible-hak",
|
||||||
@@ -375,11 +391,16 @@ func runBuilder(name string, args []string, out, errw io.Writer) int {
|
|||||||
return exitOK
|
return exitOK
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
if b.Name == "depot" && b.Wired {
|
if b.Wired {
|
||||||
// depot parses its own subcommand (status/push/verify/get/pull) and
|
// depot and assets are self-contained builders: they parse their own
|
||||||
// has its own richer exit contract (0/1/2/64/70), so it bypasses the
|
// subcommands and own their exit contract, bypassing the
|
||||||
// b.Commands/delegateLegacy routing entirely.
|
// b.Commands/delegateLegacy legacy routing entirely.
|
||||||
return depot.Run(args, out, errw, os.Getenv)
|
switch b.Name {
|
||||||
|
case "depot":
|
||||||
|
return depot.Run(args, out, errw, os.Getenv)
|
||||||
|
case "assets":
|
||||||
|
return assets.Run(args, out, errw, os.Getenv)
|
||||||
|
}
|
||||||
}
|
}
|
||||||
if !b.Wired {
|
if !b.Wired {
|
||||||
// No migrated logic yet (depot): fail closed, never fake an artifact.
|
// No migrated logic yet (depot): fail closed, never fake an artifact.
|
||||||
|
|||||||
@@ -151,6 +151,7 @@ func TestBuilderHelpIsOK(t *testing.T) {
|
|||||||
func TestCanonicalCommandSurface(t *testing.T) {
|
func TestCanonicalCommandSurface(t *testing.T) {
|
||||||
want := map[string][]string{
|
want := map[string][]string{
|
||||||
"depot": {"status", "push", "verify", "get", "pull"},
|
"depot": {"status", "push", "verify", "get", "pull"},
|
||||||
|
"assets": {"compile", "convert", "upscale", "check-mdl", "fix-mdl", "check-dupes", "clean-dupes"},
|
||||||
"hak": {"build", "manifest"},
|
"hak": {"build", "manifest"},
|
||||||
"module": {"build", "extract", "validate", "compare", "manifest"},
|
"module": {"build", "extract", "validate", "compare", "manifest"},
|
||||||
"topdata": {"validate", "build", "package", "compare", "convert"},
|
"topdata": {"validate", "build", "package", "compare", "convert"},
|
||||||
@@ -172,10 +173,10 @@ func TestRegistryCommandNamesAndAliasesAreUnambiguous(t *testing.T) {
|
|||||||
for _, builder := range Registry {
|
for _, builder := range Registry {
|
||||||
seen := map[string]bool{}
|
seen := map[string]bool{}
|
||||||
for _, command := range builder.Commands {
|
for _, command := range builder.Commands {
|
||||||
// depot parses its own subcommands and bypasses AppCommand routing
|
// depot and assets parse their own subcommands and bypass AppCommand
|
||||||
// entirely (see the depot special-case in runBuilder), so its
|
// routing entirely (see the self-contained-builder branch in
|
||||||
// Commands carry no AppCommand.
|
// runBuilder), so their Commands carry no AppCommand.
|
||||||
requireAppCommand := builder.Name != "depot"
|
requireAppCommand := builder.Name != "depot" && builder.Name != "assets"
|
||||||
if command.Name == "" || command.Summary == "" || command.Usage == "" || (requireAppCommand && command.AppCommand == "") {
|
if command.Name == "" || command.Summary == "" || command.Usage == "" || (requireAppCommand && command.AppCommand == "") {
|
||||||
t.Errorf("%s has incomplete command metadata: %#v", builder.Name, command)
|
t.Errorf("%s has incomplete command metadata: %#v", builder.Name, command)
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -123,6 +123,7 @@ var extensionTypes = map[string]uint16{
|
|||||||
"mtr": 0x0818,
|
"mtr": 0x0818,
|
||||||
"jpg": 0x081C,
|
"jpg": 0x081C,
|
||||||
"lod": 0x081E,
|
"lod": 0x081E,
|
||||||
|
"gif": 0x081F,
|
||||||
"png": 0x0820,
|
"png": 0x0820,
|
||||||
"lyt": 0x0BB8,
|
"lyt": 0x0BB8,
|
||||||
"vis": 0x0BB9,
|
"vis": 0x0BB9,
|
||||||
@@ -188,6 +189,7 @@ var typeExtensions = map[uint16]string{
|
|||||||
0x0818: "mtr",
|
0x0818: "mtr",
|
||||||
0x081C: "jpg",
|
0x081C: "jpg",
|
||||||
0x081E: "lod",
|
0x081E: "lod",
|
||||||
|
0x081F: "gif",
|
||||||
0x0820: "png",
|
0x0820: "png",
|
||||||
0x0BB8: "lyt",
|
0x0BB8: "lyt",
|
||||||
0x0BB9: "vis",
|
0x0BB9: "vis",
|
||||||
|
|||||||
@@ -593,9 +593,14 @@ func envBool(name string) bool {
|
|||||||
func collectModuleResources(p *project.Project, moduleHakOrder []string) ([]erf.Resource, error) {
|
func collectModuleResources(p *project.Project, moduleHakOrder []string) ([]erf.Resource, error) {
|
||||||
var moduleResources []erf.Resource
|
var moduleResources []erf.Resource
|
||||||
|
|
||||||
|
palettes, err := collectPaletteDescriptors(p)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
|
||||||
for _, rel := range p.Inventory.SourceFiles {
|
for _, rel := range p.Inventory.SourceFiles {
|
||||||
abs := filepath.Join(p.SourceDir(), filepath.FromSlash(rel))
|
abs := filepath.Join(p.SourceDir(), filepath.FromSlash(rel))
|
||||||
resource, err := resourceFromJSON(abs, moduleHakOrder)
|
resource, err := resourceFromJSON(abs, moduleHakOrder, palettes)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
@@ -1030,7 +1035,7 @@ func compareResourceKeys(a, b erf.Resource) int {
|
|||||||
return 0
|
return 0
|
||||||
}
|
}
|
||||||
|
|
||||||
func resourceFromJSON(path string, moduleHakOrder []string) (erf.Resource, error) {
|
func resourceFromJSON(path string, moduleHakOrder []string, palettes paletteProjection) (erf.Resource, error) {
|
||||||
name, extension, err := splitSourceName(path)
|
name, extension, err := splitSourceName(path)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return erf.Resource{}, err
|
return erf.Resource{}, err
|
||||||
@@ -1059,6 +1064,9 @@ func resourceFromJSON(path string, moduleHakOrder []string) (erf.Resource, error
|
|||||||
if extension == ".ifo" && name == "module" && len(moduleHakOrder) > 0 {
|
if extension == ".ifo" && name == "module" && len(moduleHakOrder) > 0 {
|
||||||
setModuleHAKList(&document, moduleHakOrder)
|
setModuleHAKList(&document, moduleHakOrder)
|
||||||
}
|
}
|
||||||
|
if extension == ".itp" && isPaletteProjectionResref(name) {
|
||||||
|
projectPaletteDocument(&document, palettes[strings.ToLower(name)])
|
||||||
|
}
|
||||||
|
|
||||||
var buf bytes.Buffer
|
var buf bytes.Buffer
|
||||||
if err := gff.Write(&buf, document); err != nil {
|
if err := gff.Write(&buf, document); err != nil {
|
||||||
|
|||||||
@@ -117,6 +117,10 @@ func expectedResources(p *project.Project) (map[string]resourceExpectation, erro
|
|||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
|
palettes, err := collectPaletteDescriptors(p)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
|
||||||
for _, rel := range p.Inventory.SourceFiles {
|
for _, rel := range p.Inventory.SourceFiles {
|
||||||
abs := filepath.Join(p.SourceDir(), filepath.FromSlash(rel))
|
abs := filepath.Join(p.SourceDir(), filepath.FromSlash(rel))
|
||||||
@@ -136,6 +140,9 @@ func expectedResources(p *project.Project) (map[string]resourceExpectation, erro
|
|||||||
if extension == ".ifo" && strings.EqualFold(name, "module") && len(moduleHakOrder) > 0 {
|
if extension == ".ifo" && strings.EqualFold(name, "module") && len(moduleHakOrder) > 0 {
|
||||||
setModuleHAKList(&document, moduleHakOrder)
|
setModuleHAKList(&document, moduleHakOrder)
|
||||||
}
|
}
|
||||||
|
if extension == ".itp" && isPaletteProjectionResref(name) {
|
||||||
|
projectPaletteDocument(&document, palettes[strings.ToLower(name)])
|
||||||
|
}
|
||||||
|
|
||||||
canonical, err := json.Marshal(document)
|
canonical, err := json.Marshal(document)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
|
|||||||
@@ -124,6 +124,13 @@ func extractArchiveResources(p *project.Project, archive erf.Archive, desired ma
|
|||||||
skippedCount++
|
skippedCount++
|
||||||
continue
|
continue
|
||||||
}
|
}
|
||||||
|
// *palcus.itp are Toolset-generated palette projections; the module
|
||||||
|
// build regenerates them from source blueprints, so extraction never
|
||||||
|
// writes them back into source.
|
||||||
|
if ext == "itp" && isPaletteProjectionResref(resource.Name) {
|
||||||
|
skippedCount++
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
|
||||||
target, data, err := extractedFile(p, resource, ext)
|
target, data, err := extractedFile(p, resource, ext)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
@@ -350,26 +357,48 @@ func mergeExtractedGFFJSON(p *project.Project, target string, extracted *gff.Doc
|
|||||||
}
|
}
|
||||||
|
|
||||||
raw, err := os.ReadFile(target)
|
raw, err := os.ReadFile(target)
|
||||||
if err != nil {
|
if err != nil && !errors.Is(err, os.ErrNotExist) {
|
||||||
if errors.Is(err, os.ErrNotExist) {
|
|
||||||
return nil
|
|
||||||
}
|
|
||||||
return fmt.Errorf("read existing source %s: %w", target, err)
|
return fmt.Errorf("read existing source %s: %w", target, err)
|
||||||
}
|
}
|
||||||
var existing gff.Document
|
if err == nil {
|
||||||
if err := json.Unmarshal(raw, &existing); err != nil {
|
var existing gff.Document
|
||||||
return fmt.Errorf("parse existing source %s: %w", target, err)
|
if err := json.Unmarshal(raw, &existing); err != nil {
|
||||||
|
return fmt.Errorf("parse existing source %s: %w", target, err)
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, label := range rule.PreserveFields {
|
||||||
|
if field, ok := gffField(existing.Root, label); ok {
|
||||||
|
setGFFField(&extracted.Root, field)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for _, listRule := range rule.MergeLists {
|
||||||
|
if err := mergeGFFListByKey(&extracted.Root, existing.Root, listRule); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
for _, label := range rule.PreserveFields {
|
// set_fields runs last so a forced value also wins over preserve_fields.
|
||||||
if field, ok := gffField(existing.Root, label); ok {
|
for _, setRule := range rule.SetFields {
|
||||||
setGFFField(&extracted.Root, field)
|
if err := setGFFIntField(&extracted.Root, setRule); err != nil {
|
||||||
|
return fmt.Errorf("set_fields on %s: %w", target, err)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
for _, listRule := range rule.MergeLists {
|
return nil
|
||||||
if err := mergeGFFListByKey(&extracted.Root, existing.Root, listRule); err != nil {
|
}
|
||||||
return err
|
|
||||||
|
// setGFFIntField forces an existing Int field to a fixed value; a field the
|
||||||
|
// document does not have is left absent rather than invented.
|
||||||
|
func setGFFIntField(s *gff.Struct, rule project.ExtractSetFieldRule) error {
|
||||||
|
for index, field := range s.Fields {
|
||||||
|
if field.Label != rule.Field {
|
||||||
|
continue
|
||||||
}
|
}
|
||||||
|
if _, ok := field.Value.(gff.IntValue); !ok {
|
||||||
|
return fmt.Errorf("field %q is %s, not Int", rule.Field, field.Type)
|
||||||
|
}
|
||||||
|
s.Fields[index].Value = gff.IntValue(rule.Value)
|
||||||
|
return nil
|
||||||
}
|
}
|
||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
@@ -384,7 +413,7 @@ func extractGFFJSONMergeRule(p *project.Project, target string) (project.Extract
|
|||||||
return project.ExtractGFFJSONMergeRule{}, false, nil
|
return project.ExtractGFFJSONMergeRule{}, false, nil
|
||||||
}
|
}
|
||||||
for _, rule := range p.EffectiveConfig().Extract.Merge.GFFJSON {
|
for _, rule := range p.EffectiveConfig().Extract.Merge.GFFJSON {
|
||||||
if rule.Target == rel {
|
if rule.Target == rel || matchPathPattern(rel, rule.Target) {
|
||||||
return rule, true, nil
|
return rule, true, nil
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -0,0 +1,298 @@
|
|||||||
|
package pipeline
|
||||||
|
|
||||||
|
// Custom palette projections (*palcus.itp) are generated artifacts: the Toolset
|
||||||
|
// derives them from the category frameworks plus the blueprints present in the
|
||||||
|
// module. Crucible reproduces that projection deterministically at build time so
|
||||||
|
// the module source only carries descriptor-free category skeletons and the
|
||||||
|
// blueprint files themselves. Extract never writes palcus files back.
|
||||||
|
|
||||||
|
import (
|
||||||
|
"encoding/json"
|
||||||
|
"fmt"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"sort"
|
||||||
|
"strings"
|
||||||
|
|
||||||
|
"git.westgate.pw/ShadowsOverWestgate/sow-tools/internal/gff"
|
||||||
|
"git.westgate.pw/ShadowsOverWestgate/sow-tools/internal/project"
|
||||||
|
)
|
||||||
|
|
||||||
|
const noStrref = 0xFFFFFFFF
|
||||||
|
|
||||||
|
// hiddenPaletteID suppresses a blueprint from the Custom palette (engine
|
||||||
|
// convention; see docs in sow-module's palette research notes).
|
||||||
|
const hiddenPaletteID = 255
|
||||||
|
|
||||||
|
var paletteFamilyByExtension = map[string]string{
|
||||||
|
".utc": "creaturepalcus",
|
||||||
|
".utd": "doorpalcus",
|
||||||
|
".ute": "encounterpalcus",
|
||||||
|
".uti": "itempalcus",
|
||||||
|
".utm": "storepalcus",
|
||||||
|
".utp": "placeablepalcus",
|
||||||
|
".uts": "soundpalcus",
|
||||||
|
".utt": "triggerpalcus",
|
||||||
|
".utw": "waypointpalcus",
|
||||||
|
}
|
||||||
|
|
||||||
|
type paletteDescriptor struct {
|
||||||
|
name string
|
||||||
|
strref uint32
|
||||||
|
resref string
|
||||||
|
creature bool
|
||||||
|
cr float32
|
||||||
|
faction string
|
||||||
|
}
|
||||||
|
|
||||||
|
func (d paletteDescriptor) sortKey() string {
|
||||||
|
if d.strref != noStrref || d.name == "" {
|
||||||
|
return strings.ToLower(d.resref)
|
||||||
|
}
|
||||||
|
return strings.ToLower(d.name)
|
||||||
|
}
|
||||||
|
|
||||||
|
// palette resref (e.g. "itempalcus") -> terminal category ID -> descriptors.
|
||||||
|
type paletteProjection map[string]map[uint8][]paletteDescriptor
|
||||||
|
|
||||||
|
func isPaletteProjectionResref(name string) bool {
|
||||||
|
return strings.HasSuffix(strings.ToLower(name), "palcus")
|
||||||
|
}
|
||||||
|
|
||||||
|
func collectPaletteDescriptors(p *project.Project) (paletteProjection, error) {
|
||||||
|
projection := paletteProjection{}
|
||||||
|
factions, err := loadFactionNames(p)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, rel := range p.Inventory.SourceFiles {
|
||||||
|
abs := filepath.Join(p.SourceDir(), filepath.FromSlash(rel))
|
||||||
|
name, extension, err := splitSourceName(abs)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
family, ok := paletteFamilyByExtension[extension]
|
||||||
|
if !ok {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
|
||||||
|
raw, err := os.ReadFile(abs)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("read %s: %w", abs, err)
|
||||||
|
}
|
||||||
|
var document gff.Document
|
||||||
|
if err := json.Unmarshal(raw, &document); err != nil {
|
||||||
|
return nil, fmt.Errorf("parse gff json %s: %w", abs, err)
|
||||||
|
}
|
||||||
|
|
||||||
|
descriptor, paletteID, ok := blueprintDescriptor(document.Root, name, extension, factions)
|
||||||
|
if !ok {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if projection[family] == nil {
|
||||||
|
projection[family] = map[uint8][]paletteDescriptor{}
|
||||||
|
}
|
||||||
|
projection[family][paletteID] = append(projection[family][paletteID], descriptor)
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, byID := range projection {
|
||||||
|
for _, descriptors := range byID {
|
||||||
|
sort.SliceStable(descriptors, func(i, j int) bool {
|
||||||
|
a, b := descriptors[i], descriptors[j]
|
||||||
|
if a.sortKey() != b.sortKey() {
|
||||||
|
return a.sortKey() < b.sortKey()
|
||||||
|
}
|
||||||
|
return a.resref < b.resref
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return projection, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func blueprintDescriptor(root gff.Struct, fileName, extension string, factions []string) (paletteDescriptor, uint8, bool) {
|
||||||
|
descriptor := paletteDescriptor{
|
||||||
|
strref: noStrref,
|
||||||
|
resref: strings.ToLower(fileName),
|
||||||
|
creature: extension == ".utc",
|
||||||
|
}
|
||||||
|
paletteID := -1
|
||||||
|
|
||||||
|
for _, field := range root.Fields {
|
||||||
|
switch field.Label {
|
||||||
|
case "PaletteID":
|
||||||
|
if v, ok := field.Value.(gff.ByteValue); ok {
|
||||||
|
paletteID = int(v)
|
||||||
|
}
|
||||||
|
case "TemplateResRef":
|
||||||
|
if v, ok := field.Value.(gff.ResRefValue); ok && v != "" {
|
||||||
|
descriptor.resref = strings.ToLower(string(v))
|
||||||
|
}
|
||||||
|
case "LocalizedName", "LocName", "FirstName":
|
||||||
|
if v, ok := field.Value.(gff.LocString); ok {
|
||||||
|
name, strref := locStringLabel(v)
|
||||||
|
if field.Label == "FirstName" {
|
||||||
|
descriptor.name = strings.TrimSpace(descriptor.name + " " + name)
|
||||||
|
if descriptor.strref == noStrref {
|
||||||
|
descriptor.strref = strref
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
descriptor.name = name
|
||||||
|
descriptor.strref = strref
|
||||||
|
}
|
||||||
|
}
|
||||||
|
case "LastName":
|
||||||
|
if v, ok := field.Value.(gff.LocString); ok {
|
||||||
|
name, _ := locStringLabel(v)
|
||||||
|
descriptor.name = strings.TrimSpace(descriptor.name + " " + name)
|
||||||
|
}
|
||||||
|
case "ChallengeRating":
|
||||||
|
if v, ok := field.Value.(gff.FloatValue); ok {
|
||||||
|
descriptor.cr = float32(v)
|
||||||
|
}
|
||||||
|
case "FactionID":
|
||||||
|
if v, ok := field.Value.(gff.WordValue); ok && int(v) < len(factions) {
|
||||||
|
descriptor.faction = factions[v]
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if paletteID < 0 || paletteID == hiddenPaletteID {
|
||||||
|
return paletteDescriptor{}, 0, false
|
||||||
|
}
|
||||||
|
return descriptor, uint8(paletteID), true
|
||||||
|
}
|
||||||
|
|
||||||
|
func locStringLabel(value gff.LocString) (string, uint32) {
|
||||||
|
if value.StringRef != noStrref {
|
||||||
|
return "", value.StringRef
|
||||||
|
}
|
||||||
|
for _, entry := range value.Entries {
|
||||||
|
if entry.Value != "" {
|
||||||
|
return entry.Value, noStrref
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return "", noStrref
|
||||||
|
}
|
||||||
|
|
||||||
|
func loadFactionNames(p *project.Project) ([]string, error) {
|
||||||
|
for _, rel := range p.Inventory.SourceFiles {
|
||||||
|
if !strings.HasSuffix(strings.ToLower(rel), "repute.fac.json") {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
abs := filepath.Join(p.SourceDir(), filepath.FromSlash(rel))
|
||||||
|
raw, err := os.ReadFile(abs)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("read %s: %w", abs, err)
|
||||||
|
}
|
||||||
|
var document gff.Document
|
||||||
|
if err := json.Unmarshal(raw, &document); err != nil {
|
||||||
|
return nil, fmt.Errorf("parse gff json %s: %w", abs, err)
|
||||||
|
}
|
||||||
|
var names []string
|
||||||
|
for _, field := range document.Root.Fields {
|
||||||
|
if field.Label != "FactionList" {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
list, ok := field.Value.(gff.ListValue)
|
||||||
|
if !ok {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
for _, faction := range list {
|
||||||
|
name := ""
|
||||||
|
for _, f := range faction.Fields {
|
||||||
|
if f.Label == "FactionName" {
|
||||||
|
if v, ok := f.Value.(gff.StringValue); ok {
|
||||||
|
name = string(v)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
names = append(names, name)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return names, nil
|
||||||
|
}
|
||||||
|
return nil, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// projectPaletteDocument strips every blueprint descriptor from the palette
|
||||||
|
// tree and re-inserts the descriptors derived from module source. Category
|
||||||
|
// structure (branches, terminal IDs, labels) passes through untouched.
|
||||||
|
func projectPaletteDocument(document *gff.Document, byID map[uint8][]paletteDescriptor) {
|
||||||
|
for i, field := range document.Root.Fields {
|
||||||
|
if field.Label != "MAIN" {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if list, ok := field.Value.(gff.ListValue); ok {
|
||||||
|
document.Root.Fields[i] = gff.NewField("MAIN", projectPaletteNodes(list, byID))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func projectPaletteNodes(nodes gff.ListValue, byID map[uint8][]paletteDescriptor) gff.ListValue {
|
||||||
|
out := make(gff.ListValue, 0, len(nodes))
|
||||||
|
for _, node := range nodes {
|
||||||
|
if structHasField(node, "RESREF") {
|
||||||
|
continue // blueprint descriptor — regenerated below
|
||||||
|
}
|
||||||
|
|
||||||
|
terminalID := -1
|
||||||
|
fields := make([]gff.Field, 0, len(node.Fields))
|
||||||
|
for _, field := range node.Fields {
|
||||||
|
if field.Label == "LIST" {
|
||||||
|
continue // rebuilt for terminals, recursed for branches
|
||||||
|
}
|
||||||
|
if field.Label == "ID" {
|
||||||
|
if v, ok := field.Value.(gff.ByteValue); ok {
|
||||||
|
terminalID = int(v)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
fields = append(fields, field)
|
||||||
|
}
|
||||||
|
|
||||||
|
switch {
|
||||||
|
case terminalID >= 0:
|
||||||
|
if descriptors := byID[uint8(terminalID)]; len(descriptors) > 0 {
|
||||||
|
fields = append(fields, gff.NewField("LIST", descriptorList(descriptors)))
|
||||||
|
}
|
||||||
|
default:
|
||||||
|
for _, field := range node.Fields {
|
||||||
|
if field.Label == "LIST" {
|
||||||
|
if list, ok := field.Value.(gff.ListValue); ok {
|
||||||
|
fields = append(fields, gff.NewField("LIST", projectPaletteNodes(list, byID)))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
out = append(out, gff.Struct{Type: node.Type, Fields: fields})
|
||||||
|
}
|
||||||
|
return out
|
||||||
|
}
|
||||||
|
|
||||||
|
func descriptorList(descriptors []paletteDescriptor) gff.ListValue {
|
||||||
|
list := make(gff.ListValue, 0, len(descriptors))
|
||||||
|
for _, d := range descriptors {
|
||||||
|
fields := make([]gff.Field, 0, 4)
|
||||||
|
if d.strref != noStrref {
|
||||||
|
fields = append(fields, gff.NewField("STRREF", gff.DWordValue(d.strref)))
|
||||||
|
} else {
|
||||||
|
fields = append(fields, gff.NewField("NAME", gff.StringValue(d.name)))
|
||||||
|
}
|
||||||
|
fields = append(fields, gff.NewField("RESREF", gff.ResRefValue(d.resref)))
|
||||||
|
if d.creature {
|
||||||
|
fields = append(fields, gff.NewField("CR", gff.FloatValue(d.cr)))
|
||||||
|
fields = append(fields, gff.NewField("FACTION", gff.StringValue(d.faction)))
|
||||||
|
}
|
||||||
|
list = append(list, gff.Struct{Fields: fields})
|
||||||
|
}
|
||||||
|
return list
|
||||||
|
}
|
||||||
|
|
||||||
|
func structHasField(s gff.Struct, label string) bool {
|
||||||
|
for _, field := range s.Fields {
|
||||||
|
if field.Label == label {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return false
|
||||||
|
}
|
||||||
@@ -0,0 +1,191 @@
|
|||||||
|
package pipeline
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"encoding/json"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"git.westgate.pw/ShadowsOverWestgate/sow-tools/internal/erf"
|
||||||
|
"git.westgate.pw/ShadowsOverWestgate/sow-tools/internal/gff"
|
||||||
|
"git.westgate.pw/ShadowsOverWestgate/sow-tools/internal/project"
|
||||||
|
)
|
||||||
|
|
||||||
|
const paletteTestSkeleton = `{
|
||||||
|
"file_type": "ITP ",
|
||||||
|
"file_version": "V3.2",
|
||||||
|
"root": {
|
||||||
|
"struct_type": 4294967295,
|
||||||
|
"fields": [
|
||||||
|
{
|
||||||
|
"label": "MAIN",
|
||||||
|
"type": "List",
|
||||||
|
"value": [
|
||||||
|
{
|
||||||
|
"struct_type": 0,
|
||||||
|
"fields": [
|
||||||
|
{"label": "STRREF", "type": "DWord", "value": 500},
|
||||||
|
{
|
||||||
|
"label": "LIST",
|
||||||
|
"type": "List",
|
||||||
|
"value": [
|
||||||
|
{
|
||||||
|
"struct_type": 0,
|
||||||
|
"fields": [
|
||||||
|
{"label": "STRREF", "type": "DWord", "value": 6699},
|
||||||
|
{"label": "ID", "type": "Byte", "value": 23},
|
||||||
|
{
|
||||||
|
"label": "LIST",
|
||||||
|
"type": "List",
|
||||||
|
"value": [
|
||||||
|
{
|
||||||
|
"struct_type": 0,
|
||||||
|
"fields": [
|
||||||
|
{"label": "NAME", "type": "CExoString", "value": "Stale Junk"},
|
||||||
|
{"label": "RESREF", "type": "ResRef", "value": "stalejunk"}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"struct_type": 0,
|
||||||
|
"fields": [
|
||||||
|
{"label": "STRREF", "type": "DWord", "value": 6753},
|
||||||
|
{"label": "ID", "type": "Byte", "value": 24}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
}
|
||||||
|
`
|
||||||
|
|
||||||
|
func paletteTestItem(resref, name string, paletteID int) string {
|
||||||
|
return `{
|
||||||
|
"file_type": "UTI ",
|
||||||
|
"file_version": "V3.2",
|
||||||
|
"root": {
|
||||||
|
"struct_type": 4294967295,
|
||||||
|
"fields": [
|
||||||
|
{"label": "TemplateResRef", "type": "ResRef", "value": "` + resref + `"},
|
||||||
|
{
|
||||||
|
"label": "LocalizedName",
|
||||||
|
"type": "CExoLocString",
|
||||||
|
"value": {"string_ref": 4294967295, "entries": [{"id": 0, "value": "` + name + `"}]}
|
||||||
|
},
|
||||||
|
{"label": "PaletteID", "type": "Byte", "value": ` + itoa(paletteID) + `}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
}
|
||||||
|
`
|
||||||
|
}
|
||||||
|
|
||||||
|
func itoa(v int) string {
|
||||||
|
data, _ := json.Marshal(v)
|
||||||
|
return string(data)
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestBuildProjectsPaletteDescriptorsAndExtractSkipsThem(t *testing.T) {
|
||||||
|
root := t.TempDir()
|
||||||
|
mustMkdir(t, filepath.Join(root, "src", "module"))
|
||||||
|
mustMkdir(t, filepath.Join(root, "src", "palettes"))
|
||||||
|
mustMkdir(t, filepath.Join(root, "src", "blueprints", "items"))
|
||||||
|
mustMkdir(t, filepath.Join(root, "build"))
|
||||||
|
|
||||||
|
mustWriteFile(t, filepath.Join(root, "nwn-tool.json"), `{
|
||||||
|
"module": {"name": "Test Module", "resref": "testmod"},
|
||||||
|
"paths": {"source": "src", "build": "build"}
|
||||||
|
}
|
||||||
|
`)
|
||||||
|
mustWriteFile(t, filepath.Join(root, "src", "module", "module.ifo.json"), `{
|
||||||
|
"file_type": "IFO ",
|
||||||
|
"file_version": "V3.2",
|
||||||
|
"root": {"struct_type": 4294967295, "fields": [{"label": "Mod_Name", "type": "CExoString", "value": "Test Module"}]}
|
||||||
|
}
|
||||||
|
`)
|
||||||
|
mustWriteFile(t, filepath.Join(root, "src", "palettes", "itempalcus.itp.json"), paletteTestSkeleton)
|
||||||
|
mustWriteFile(t, filepath.Join(root, "src", "blueprints", "items", "i_b.uti.json"), paletteTestItem("i_b", "Bravo Item", 23))
|
||||||
|
mustWriteFile(t, filepath.Join(root, "src", "blueprints", "items", "i_a.uti.json"), paletteTestItem("i_a", "Alpha Item", 23))
|
||||||
|
mustWriteFile(t, filepath.Join(root, "src", "blueprints", "items", "i_hidden.uti.json"), paletteTestItem("i_hidden", "Hidden Item", 255))
|
||||||
|
|
||||||
|
p, err := project.Load(root)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("load project: %v", err)
|
||||||
|
}
|
||||||
|
if err := p.Scan(); err != nil {
|
||||||
|
t.Fatalf("scan: %v", err)
|
||||||
|
}
|
||||||
|
if _, err := BuildModule(p); err != nil {
|
||||||
|
t.Fatalf("build: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
archiveFile, err := os.Open(p.ModuleArchivePath())
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("open module archive: %v", err)
|
||||||
|
}
|
||||||
|
defer archiveFile.Close()
|
||||||
|
archive, err := erf.Read(archiveFile)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("read module archive: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
var palette *gff.Document
|
||||||
|
for _, resource := range archive.Resources {
|
||||||
|
if resource.Name == "itempalcus" {
|
||||||
|
document, err := gff.Read(bytes.NewReader(resource.Data))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("decode itempalcus: %v", err)
|
||||||
|
}
|
||||||
|
palette = &document
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if palette == nil {
|
||||||
|
t.Fatal("itempalcus missing from built module")
|
||||||
|
}
|
||||||
|
|
||||||
|
canonical, err := json.Marshal(palette)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("marshal palette: %v", err)
|
||||||
|
}
|
||||||
|
text := string(canonical)
|
||||||
|
if bytes.Contains(canonical, []byte("stalejunk")) {
|
||||||
|
t.Fatalf("stale descriptor survived projection: %s", text)
|
||||||
|
}
|
||||||
|
for _, resref := range []string{"i_a", "i_b"} {
|
||||||
|
if !bytes.Contains(canonical, []byte(resref)) {
|
||||||
|
t.Fatalf("descriptor %s missing from projection: %s", resref, text)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if bytes.Contains(canonical, []byte("i_hidden")) {
|
||||||
|
t.Fatalf("PaletteID 255 blueprint leaked into projection: %s", text)
|
||||||
|
}
|
||||||
|
if a, b := bytes.Index(canonical, []byte("i_a")), bytes.Index(canonical, []byte("i_b")); a > b {
|
||||||
|
t.Fatalf("descriptors not name-sorted: %s", text)
|
||||||
|
}
|
||||||
|
|
||||||
|
if _, err := Compare(p); err != nil {
|
||||||
|
t.Fatalf("compare after build: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Extraction must never write palcus files back into source.
|
||||||
|
if err := os.Remove(filepath.Join(root, "src", "palettes", "itempalcus.itp.json")); err != nil {
|
||||||
|
t.Fatalf("remove skeleton: %v", err)
|
||||||
|
}
|
||||||
|
if err := p.Scan(); err != nil {
|
||||||
|
t.Fatalf("rescan: %v", err)
|
||||||
|
}
|
||||||
|
if _, err := Extract(p); err != nil {
|
||||||
|
t.Fatalf("extract: %v", err)
|
||||||
|
}
|
||||||
|
if _, err := os.Stat(filepath.Join(root, "src", "palettes", "itempalcus.itp.json")); !os.IsNotExist(err) {
|
||||||
|
t.Fatalf("extract wrote palcus file back (stat err: %v)", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -3503,6 +3503,100 @@ extract:
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestExtractSetsConfiguredGFFJSONFields(t *testing.T) {
|
||||||
|
root := t.TempDir()
|
||||||
|
mustMkdir(t, filepath.Join(root, "src", "module"))
|
||||||
|
mustMkdir(t, filepath.Join(root, "src", "areas"))
|
||||||
|
mustMkdir(t, filepath.Join(root, "assets"))
|
||||||
|
mustMkdir(t, filepath.Join(root, "build"))
|
||||||
|
|
||||||
|
mustWriteFile(t, filepath.Join(root, "nwn-tool.yaml"), `
|
||||||
|
module:
|
||||||
|
name: Test Module
|
||||||
|
resref: testmod
|
||||||
|
paths:
|
||||||
|
source: src
|
||||||
|
assets: assets
|
||||||
|
build: build
|
||||||
|
extract:
|
||||||
|
merge:
|
||||||
|
gff_json:
|
||||||
|
- target: areas/*.are.json
|
||||||
|
set_fields:
|
||||||
|
- field: ChanceRain
|
||||||
|
value: 0
|
||||||
|
- field: ChanceSnow
|
||||||
|
value: 0
|
||||||
|
`)
|
||||||
|
|
||||||
|
mustWriteFile(t, filepath.Join(root, "src", "module", "module.ifo.json"), `{
|
||||||
|
"file_type": "IFO ",
|
||||||
|
"file_version": "V3.2",
|
||||||
|
"root": {
|
||||||
|
"struct_type": 0,
|
||||||
|
"fields": [
|
||||||
|
{
|
||||||
|
"label": "Mod_Name",
|
||||||
|
"type": "CExoString",
|
||||||
|
"value": "Test Module"
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
}
|
||||||
|
`)
|
||||||
|
mustWriteFile(t, filepath.Join(root, "src", "areas", "area_a.are.json"), `{
|
||||||
|
"file_type": "ARE ",
|
||||||
|
"file_version": "V3.2",
|
||||||
|
"root": {
|
||||||
|
"struct_type": 0,
|
||||||
|
"fields": [
|
||||||
|
{
|
||||||
|
"label": "ChanceRain",
|
||||||
|
"type": "Int",
|
||||||
|
"value": 40
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"label": "WindPower",
|
||||||
|
"type": "Int",
|
||||||
|
"value": 2
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
}
|
||||||
|
`)
|
||||||
|
|
||||||
|
p, err := project.Load(root)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("load project: %v", err)
|
||||||
|
}
|
||||||
|
if err := p.ValidateLayout(); err != nil {
|
||||||
|
t.Fatalf("validate layout: %v", err)
|
||||||
|
}
|
||||||
|
if err := p.Scan(); err != nil {
|
||||||
|
t.Fatalf("scan: %v", err)
|
||||||
|
}
|
||||||
|
if _, err := BuildModule(p); err != nil {
|
||||||
|
t.Fatalf("build module: %v", err)
|
||||||
|
}
|
||||||
|
if err := p.Scan(); err != nil {
|
||||||
|
t.Fatalf("rescan before extract: %v", err)
|
||||||
|
}
|
||||||
|
if _, err := Extract(p); err != nil {
|
||||||
|
t.Fatalf("extract: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
document := readGFFJSON(t, filepath.Join(root, "src", "areas", "area_a.are.json"))
|
||||||
|
if got, want := fieldValue(t, document.Root, "ChanceRain"), gff.IntValue(0); got != want {
|
||||||
|
t.Fatalf("expected forced ChanceRain %#v, got %#v", want, got)
|
||||||
|
}
|
||||||
|
if got, want := fieldValue(t, document.Root, "WindPower"), gff.IntValue(2); got != want {
|
||||||
|
t.Fatalf("expected untouched WindPower %#v, got %#v", want, got)
|
||||||
|
}
|
||||||
|
if _, ok := gffField(document.Root, "ChanceSnow"); ok {
|
||||||
|
t.Fatalf("expected absent ChanceSnow to stay absent")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
func TestExtractNormalizesResourceNamesToLowercase(t *testing.T) {
|
func TestExtractNormalizesResourceNamesToLowercase(t *testing.T) {
|
||||||
root := t.TempDir()
|
root := t.TempDir()
|
||||||
mustMkdir(t, filepath.Join(root, "src", "blueprints", "items"))
|
mustMkdir(t, filepath.Join(root, "src", "blueprints", "items"))
|
||||||
|
|||||||
@@ -281,6 +281,15 @@ type ExtractGFFJSONMergeRule struct {
|
|||||||
Target string `json:"target" yaml:"target"`
|
Target string `json:"target" yaml:"target"`
|
||||||
PreserveFields []string `json:"preserve_fields" yaml:"preserve_fields"`
|
PreserveFields []string `json:"preserve_fields" yaml:"preserve_fields"`
|
||||||
MergeLists []ExtractListMergeRule `json:"merge_lists" yaml:"merge_lists"`
|
MergeLists []ExtractListMergeRule `json:"merge_lists" yaml:"merge_lists"`
|
||||||
|
SetFields []ExtractSetFieldRule `json:"set_fields" yaml:"set_fields"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// ExtractSetFieldRule forces an Int field to a fixed value on every extract,
|
||||||
|
// overriding whatever the toolset saved. Fields absent from the extracted
|
||||||
|
// document are left absent.
|
||||||
|
type ExtractSetFieldRule struct {
|
||||||
|
Field string `json:"field" yaml:"field"`
|
||||||
|
Value int32 `json:"value" yaml:"value"`
|
||||||
}
|
}
|
||||||
|
|
||||||
type ExtractListMergeRule struct {
|
type ExtractListMergeRule struct {
|
||||||
@@ -980,6 +989,12 @@ func validateExtractMergeConfig(config ExtractMergeConfig) []error {
|
|||||||
seenTargets[normalized] = struct{}{}
|
seenTargets[normalized] = struct{}{}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
for setIndex, setRule := range rule.SetFields {
|
||||||
|
if strings.TrimSpace(setRule.Field) == "" {
|
||||||
|
failures = append(failures, fmt.Errorf("%s.set_fields[%d].field must not be empty", prefix, setIndex))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
for listIndex, listRule := range rule.MergeLists {
|
for listIndex, listRule := range rule.MergeLists {
|
||||||
listPrefix := fmt.Sprintf("%s.merge_lists[%d]", prefix, listIndex)
|
listPrefix := fmt.Sprintf("%s.merge_lists[%d]", prefix, listIndex)
|
||||||
if strings.TrimSpace(listRule.Field) == "" {
|
if strings.TrimSpace(listRule.Field) == "" {
|
||||||
@@ -1346,6 +1361,9 @@ func normalizeConfig(cfg *Config) {
|
|||||||
}
|
}
|
||||||
cfg.Extract.Merge.GFFJSON[i].Target = target
|
cfg.Extract.Merge.GFFJSON[i].Target = target
|
||||||
cfg.Extract.Merge.GFFJSON[i].PreserveFields = normalizeStringSlice(cfg.Extract.Merge.GFFJSON[i].PreserveFields)
|
cfg.Extract.Merge.GFFJSON[i].PreserveFields = normalizeStringSlice(cfg.Extract.Merge.GFFJSON[i].PreserveFields)
|
||||||
|
for j := range cfg.Extract.Merge.GFFJSON[i].SetFields {
|
||||||
|
cfg.Extract.Merge.GFFJSON[i].SetFields[j].Field = strings.TrimSpace(cfg.Extract.Merge.GFFJSON[i].SetFields[j].Field)
|
||||||
|
}
|
||||||
for j := range cfg.Extract.Merge.GFFJSON[i].MergeLists {
|
for j := range cfg.Extract.Merge.GFFJSON[i].MergeLists {
|
||||||
cfg.Extract.Merge.GFFJSON[i].MergeLists[j].Field = strings.TrimSpace(cfg.Extract.Merge.GFFJSON[i].MergeLists[j].Field)
|
cfg.Extract.Merge.GFFJSON[i].MergeLists[j].Field = strings.TrimSpace(cfg.Extract.Merge.GFFJSON[i].MergeLists[j].Field)
|
||||||
cfg.Extract.Merge.GFFJSON[i].MergeLists[j].KeyField = strings.TrimSpace(cfg.Extract.Merge.GFFJSON[i].MergeLists[j].KeyField)
|
cfg.Extract.Merge.GFFJSON[i].MergeLists[j].KeyField = strings.TrimSpace(cfg.Extract.Merge.GFFJSON[i].MergeLists[j].KeyField)
|
||||||
|
|||||||
@@ -1,96 +0,0 @@
|
|||||||
package topdata
|
|
||||||
|
|
||||||
import (
|
|
||||||
"encoding/json"
|
|
||||||
"os"
|
|
||||||
"path/filepath"
|
|
||||||
)
|
|
||||||
|
|
||||||
func importLegacyAppearance(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
|
|
||||||
_ = legacyTLK
|
|
||||||
|
|
||||||
legacyDir := filepath.Join(referenceBuilderDir, "data", "appearance")
|
|
||||||
if _, err := os.Stat(legacyDir); err != nil {
|
|
||||||
if os.IsNotExist(err) {
|
|
||||||
return 0, nil
|
|
||||||
}
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
|
|
||||||
targetDir := filepath.Join(dataDir, "appearance")
|
|
||||||
targetBasePath := filepath.Join(targetDir, "base.json")
|
|
||||||
targetLockPath := filepath.Join(targetDir, "lock.json")
|
|
||||||
targetModulesDir := filepath.Join(targetDir, "modules")
|
|
||||||
moduleNames := []string{
|
|
||||||
"cotblreaver.json",
|
|
||||||
"crawlingclaw.json",
|
|
||||||
"halfogre.json",
|
|
||||||
"zombieknight.json",
|
|
||||||
}
|
|
||||||
targetModulePaths := make([]string, 0, len(moduleNames))
|
|
||||||
for _, name := range moduleNames {
|
|
||||||
targetModulePaths = append(targetModulePaths, filepath.Join(targetModulesDir, name))
|
|
||||||
}
|
|
||||||
if fileExists(targetBasePath) && fileExists(targetLockPath) {
|
|
||||||
allModulesPresent := true
|
|
||||||
for _, path := range targetModulePaths {
|
|
||||||
if !fileExists(path) {
|
|
||||||
allModulesPresent = false
|
|
||||||
break
|
|
||||||
}
|
|
||||||
}
|
|
||||||
if allModulesPresent {
|
|
||||||
return 0, nil
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
baseObj, err := loadJSONObject(filepath.Join(legacyDir, "base.json"))
|
|
||||||
if err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
baseObj["output"] = "appearance.2da"
|
|
||||||
|
|
||||||
lockObj, err := loadJSONObject(filepath.Join(legacyDir, "lock.json"))
|
|
||||||
if err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
|
|
||||||
moduleObjs := make([]map[string]any, 0, len(moduleNames))
|
|
||||||
for _, name := range moduleNames {
|
|
||||||
obj, err := loadJSONObject(filepath.Join(legacyDir, "modules", name))
|
|
||||||
if err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
moduleObjs = append(moduleObjs, obj)
|
|
||||||
}
|
|
||||||
|
|
||||||
if err := os.MkdirAll(targetModulesDir, 0o755); err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
|
|
||||||
writes := []struct {
|
|
||||||
path string
|
|
||||||
obj map[string]any
|
|
||||||
}{
|
|
||||||
{path: targetBasePath, obj: baseObj},
|
|
||||||
{path: targetLockPath, obj: lockObj},
|
|
||||||
}
|
|
||||||
for i, path := range targetModulePaths {
|
|
||||||
writes = append(writes, struct {
|
|
||||||
path string
|
|
||||||
obj map[string]any
|
|
||||||
}{path: path, obj: moduleObjs[i]})
|
|
||||||
}
|
|
||||||
|
|
||||||
for _, write := range writes {
|
|
||||||
raw, err := json.MarshalIndent(write.obj, "", " ")
|
|
||||||
if err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
raw = append(raw, '\n')
|
|
||||||
if err := os.WriteFile(write.path, raw, 0o644); err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return len(writes), nil
|
|
||||||
}
|
|
||||||
@@ -1,160 +0,0 @@
|
|||||||
package topdata
|
|
||||||
|
|
||||||
import (
|
|
||||||
"encoding/json"
|
|
||||||
"os"
|
|
||||||
"path/filepath"
|
|
||||||
"sort"
|
|
||||||
)
|
|
||||||
|
|
||||||
func importLegacyArmor(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
|
|
||||||
_ = legacyTLK
|
|
||||||
|
|
||||||
legacyDir := filepath.Join(referenceBuilderDir, "data", "armor")
|
|
||||||
if _, err := os.Stat(legacyDir); err != nil {
|
|
||||||
if os.IsNotExist(err) {
|
|
||||||
return 0, nil
|
|
||||||
}
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
|
|
||||||
targetDir := filepath.Join(dataDir, "armor")
|
|
||||||
targetPath := filepath.Join(targetDir, "armor.json")
|
|
||||||
if _, err := os.Stat(targetPath); err == nil {
|
|
||||||
obj, err := loadJSONObject(targetPath)
|
|
||||||
if err == nil && countLegacyTLKRefsInValue(obj) == 0 {
|
|
||||||
return 0, nil
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
baseObj, err := loadJSONObject(filepath.Join(legacyDir, "base.json"))
|
|
||||||
if err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
rows, err := mergeArmorRows(baseObj, filepath.Join(legacyDir, "modules"))
|
|
||||||
if err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
|
|
||||||
if err := os.MkdirAll(targetDir, 0o755); err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
if err := removePathIfExists(filepath.Join(targetDir, "base.json")); err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
if err := removePathIfExists(filepath.Join(targetDir, "lock.json")); err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
if err := removePathIfExists(filepath.Join(targetDir, "modules")); err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
|
|
||||||
out := map[string]any{
|
|
||||||
"output": "armor.2da",
|
|
||||||
"columns": baseObj["columns"],
|
|
||||||
"rows": rows,
|
|
||||||
}
|
|
||||||
raw, err := json.MarshalIndent(out, "", " ")
|
|
||||||
if err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
raw = append(raw, '\n')
|
|
||||||
if err := os.WriteFile(targetPath, raw, 0o644); err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
return 1, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func mergeArmorRows(baseObj map[string]any, modulesDir string) ([]any, error) {
|
|
||||||
rawRows, ok := baseObj["rows"].([]any)
|
|
||||||
if !ok {
|
|
||||||
return nil, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
rows := make([]map[string]any, 0, len(rawRows))
|
|
||||||
byID := map[int]map[string]any{}
|
|
||||||
for _, raw := range rawRows {
|
|
||||||
row, ok := deepCopyValue(raw).(map[string]any)
|
|
||||||
if !ok {
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
delete(row, "key")
|
|
||||||
rawID, ok := row["id"]
|
|
||||||
if !ok {
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
id, err := asInt(rawID)
|
|
||||||
if err != nil {
|
|
||||||
return nil, err
|
|
||||||
}
|
|
||||||
row["id"] = id
|
|
||||||
rows = append(rows, row)
|
|
||||||
byID[id] = row
|
|
||||||
}
|
|
||||||
|
|
||||||
modulePaths, err := collectModulePaths(modulesDir)
|
|
||||||
if err != nil {
|
|
||||||
return nil, err
|
|
||||||
}
|
|
||||||
for _, path := range modulePaths {
|
|
||||||
obj, err := loadJSONObject(path)
|
|
||||||
if err != nil {
|
|
||||||
return nil, err
|
|
||||||
}
|
|
||||||
if entries, ok := obj["entries"].(map[string]any); ok {
|
|
||||||
for _, raw := range entries {
|
|
||||||
row, ok := deepCopyValue(raw).(map[string]any)
|
|
||||||
if !ok {
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
delete(row, "key")
|
|
||||||
rawID, ok := row["id"]
|
|
||||||
if !ok {
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
id, err := asInt(rawID)
|
|
||||||
if err != nil {
|
|
||||||
return nil, err
|
|
||||||
}
|
|
||||||
row["id"] = id
|
|
||||||
rows = append(rows, row)
|
|
||||||
byID[id] = row
|
|
||||||
}
|
|
||||||
}
|
|
||||||
if overrides, ok := obj["overrides"].([]any); ok {
|
|
||||||
for _, raw := range overrides {
|
|
||||||
override, ok := raw.(map[string]any)
|
|
||||||
if !ok {
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
rawID, ok := override["id"]
|
|
||||||
if !ok {
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
id, err := asInt(rawID)
|
|
||||||
if err != nil {
|
|
||||||
return nil, err
|
|
||||||
}
|
|
||||||
row := byID[id]
|
|
||||||
if row == nil {
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
for key, value := range override {
|
|
||||||
if key == "id" || key == "key" || key == "_tlk" || isMetadataField(key) {
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
row[key] = deepCopyValue(value)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
sort.Slice(rows, func(i, j int) bool {
|
|
||||||
return rows[i]["id"].(int) < rows[j]["id"].(int)
|
|
||||||
})
|
|
||||||
out := make([]any, 0, len(rows))
|
|
||||||
for _, row := range rows {
|
|
||||||
out = append(out, row)
|
|
||||||
}
|
|
||||||
return out, nil
|
|
||||||
}
|
|
||||||
@@ -1,41 +0,0 @@
|
|||||||
package topdata
|
|
||||||
|
|
||||||
import (
|
|
||||||
"encoding/json"
|
|
||||||
"os"
|
|
||||||
"path/filepath"
|
|
||||||
)
|
|
||||||
|
|
||||||
func importLegacyBaseDialog(referenceBuilderDir, sourceDir string) (int, error) {
|
|
||||||
legacyPath := filepath.Join(referenceBuilderDir, "tlk", "base.json")
|
|
||||||
if _, err := os.Stat(legacyPath); err != nil {
|
|
||||||
if os.IsNotExist(err) {
|
|
||||||
return 0, nil
|
|
||||||
}
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
|
|
||||||
targetPath := filepath.Join(sourceDir, "base_dialog.json")
|
|
||||||
if fileExists(targetPath) {
|
|
||||||
current, err := loadJSONObject(targetPath)
|
|
||||||
if err == nil {
|
|
||||||
if entries, ok := current["entries"].(map[string]any); ok && len(entries) > 0 {
|
|
||||||
return 0, nil
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
obj, err := loadJSONObject(legacyPath)
|
|
||||||
if err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
raw, err := json.MarshalIndent(obj, "", " ")
|
|
||||||
if err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
raw = append(raw, '\n')
|
|
||||||
if err := os.WriteFile(targetPath, raw, 0o644); err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
return 1, nil
|
|
||||||
}
|
|
||||||
@@ -1,105 +0,0 @@
|
|||||||
package topdata
|
|
||||||
|
|
||||||
import (
|
|
||||||
"encoding/json"
|
|
||||||
"os"
|
|
||||||
"path/filepath"
|
|
||||||
)
|
|
||||||
|
|
||||||
func importLegacyBaseitems(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
|
|
||||||
_ = legacyTLK
|
|
||||||
|
|
||||||
legacyDir := filepath.Join(referenceBuilderDir, "data", "baseitems")
|
|
||||||
if _, err := os.Stat(legacyDir); err != nil {
|
|
||||||
if os.IsNotExist(err) {
|
|
||||||
return 0, nil
|
|
||||||
}
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
|
|
||||||
targetDir := filepath.Join(dataDir, "baseitems")
|
|
||||||
targetBasePath := filepath.Join(targetDir, "base.json")
|
|
||||||
targetLockPath := filepath.Join(targetDir, "lock.json")
|
|
||||||
targetModulesDir := filepath.Join(targetDir, "modules")
|
|
||||||
moduleNames := []string{
|
|
||||||
"blunderbuss.json",
|
|
||||||
"coin.json",
|
|
||||||
"estoc.json",
|
|
||||||
"heavymace.json",
|
|
||||||
"maulandfalchion.json",
|
|
||||||
"ovr_baseitems.json",
|
|
||||||
"ovr_cloak255.json",
|
|
||||||
"ovr_helmet255.json",
|
|
||||||
"shortspear.json",
|
|
||||||
}
|
|
||||||
targetModulePaths := make([]string, 0, len(moduleNames))
|
|
||||||
for _, name := range moduleNames {
|
|
||||||
targetModulePaths = append(targetModulePaths, filepath.Join(targetModulesDir, name))
|
|
||||||
}
|
|
||||||
if fileExists(targetBasePath) && fileExists(targetLockPath) {
|
|
||||||
allModulesPresent := true
|
|
||||||
for _, path := range targetModulePaths {
|
|
||||||
if !fileExists(path) {
|
|
||||||
allModulesPresent = false
|
|
||||||
break
|
|
||||||
}
|
|
||||||
}
|
|
||||||
if allModulesPresent {
|
|
||||||
return 0, nil
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
baseObj, err := loadJSONObject(filepath.Join(legacyDir, "base.json"))
|
|
||||||
if err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
baseObj["output"] = "baseitems.2da"
|
|
||||||
canonicalizeWikiMetadataDocument(baseObj)
|
|
||||||
|
|
||||||
lockObj, err := loadJSONObject(filepath.Join(legacyDir, "lock.json"))
|
|
||||||
if err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
|
|
||||||
moduleObjs := make([]map[string]any, 0, len(moduleNames))
|
|
||||||
for _, name := range moduleNames {
|
|
||||||
obj, err := loadJSONObject(filepath.Join(legacyDir, "modules", name))
|
|
||||||
if err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
canonicalizeWikiMetadataDocument(obj)
|
|
||||||
moduleObjs = append(moduleObjs, obj)
|
|
||||||
}
|
|
||||||
|
|
||||||
if err := os.MkdirAll(targetModulesDir, 0o755); err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
|
|
||||||
writes := []struct {
|
|
||||||
path string
|
|
||||||
obj map[string]any
|
|
||||||
}{
|
|
||||||
{path: targetBasePath, obj: baseObj},
|
|
||||||
{path: targetLockPath, obj: lockObj},
|
|
||||||
}
|
|
||||||
for index, path := range targetModulePaths {
|
|
||||||
writes = append(writes, struct {
|
|
||||||
path string
|
|
||||||
obj map[string]any
|
|
||||||
}{
|
|
||||||
path: path,
|
|
||||||
obj: moduleObjs[index],
|
|
||||||
})
|
|
||||||
}
|
|
||||||
for _, write := range writes {
|
|
||||||
raw, err := json.MarshalIndent(write.obj, "", " ")
|
|
||||||
if err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
raw = append(raw, '\n')
|
|
||||||
if err := os.WriteFile(write.path, raw, 0o644); err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return len(writes), nil
|
|
||||||
}
|
|
||||||
@@ -0,0 +1,98 @@
|
|||||||
|
package topdata
|
||||||
|
|
||||||
|
import (
|
||||||
|
"fmt"
|
||||||
|
"path/filepath"
|
||||||
|
"slices"
|
||||||
|
"strings"
|
||||||
|
)
|
||||||
|
|
||||||
|
// validateClassFeatMasterfeatAccessibility warns when a class feat table
|
||||||
|
// references a masterfeat whose member feats are silently dropped from
|
||||||
|
// expansion by classFeatExpansionCanUseFeat. Only the ambiguous case is
|
||||||
|
// flagged: ALLCLASSESCANUSE=0 with no MinLevelClass, where the feat cannot
|
||||||
|
// be obtained by any class at all. Exclusions with an explicit MinLevelClass
|
||||||
|
// are deliberate cross-class restrictions and stay silent.
|
||||||
|
func validateClassFeatMasterfeatAccessibility(dataDir string, report *ValidationReport) {
|
||||||
|
datasets, err := discoverNativeDatasets(dataDir)
|
||||||
|
if err != nil {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
keyToID := map[string]int{}
|
||||||
|
var featRowByKey map[string]map[string]any
|
||||||
|
type classFeatTable struct {
|
||||||
|
path string
|
||||||
|
classKey string
|
||||||
|
rows []map[string]any
|
||||||
|
}
|
||||||
|
classTables := []classFeatTable{}
|
||||||
|
for _, dataset := range datasets {
|
||||||
|
name := filepath.ToSlash(dataset.Name)
|
||||||
|
isClassFeats := strings.HasPrefix(name, "classes/feats/")
|
||||||
|
if name != "feat" && name != "masterfeats" && name != "classes/core" && !isClassFeats {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
collected, err := collectNativeDataset(dataset)
|
||||||
|
if err != nil {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
for key, id := range collected.LockData {
|
||||||
|
keyToID[key] = id
|
||||||
|
}
|
||||||
|
switch {
|
||||||
|
case name == "feat":
|
||||||
|
featRowByKey = rowsByKey(collected.Rows)
|
||||||
|
case isClassFeats:
|
||||||
|
classTables = append(classTables, classFeatTable{
|
||||||
|
path: dataset.BasePath,
|
||||||
|
classKey: "classes:" + name[strings.LastIndex(name, "/")+1:],
|
||||||
|
rows: collected.Rows,
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if featRowByKey == nil {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
for _, table := range classTables {
|
||||||
|
for _, row := range table.rows {
|
||||||
|
featRef, ok := row["FeatIndex"].(map[string]any)
|
||||||
|
if !ok {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
refKey, _ := featRef["id"].(string)
|
||||||
|
if !strings.HasPrefix(refKey, "masterfeats:") {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
masterfeatKey := resolveCanonicalDatasetKey(refKey, keyToID)
|
||||||
|
if _, ok := keyToID[masterfeatKey]; !ok {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
featKeys := make([]string, 0, len(featRowByKey))
|
||||||
|
for featKey := range featRowByKey {
|
||||||
|
if strings.HasPrefix(featKey, "feat:") {
|
||||||
|
featKeys = append(featKeys, featKey)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
slices.Sort(featKeys)
|
||||||
|
for _, featKey := range featKeys {
|
||||||
|
featRow := featRowByKey[featKey]
|
||||||
|
if !rowReferencesMasterfeat(featRow, masterfeatKey, keyToID) {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if classFeatExpansionCanUseFeat(featRow, table.classKey, keyToID) {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if minLevelClass, ok := lookupField(featRow, "MinLevelClass"); ok && !isNullishValue(minLevelClass) {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
report.Diagnostics = append(report.Diagnostics, Diagnostic{
|
||||||
|
Severity: SeverityWarning,
|
||||||
|
Path: table.path,
|
||||||
|
Message: fmt.Sprintf(
|
||||||
|
"%s references %s but %s is excluded from expansion (ALLCLASSESCANUSE=0 with no MinLevelClass makes it unobtainable); override ALLCLASSESCANUSE or set MinLevelClass",
|
||||||
|
table.classKey, masterfeatKey, featKey),
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,82 @@
|
|||||||
|
package topdata
|
||||||
|
|
||||||
|
import (
|
||||||
|
"path/filepath"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestValidateClassFeatMasterfeatAccessibilityWarnsOnUnobtainableExpansion(t *testing.T) {
|
||||||
|
root := testProjectRoot(t)
|
||||||
|
mkdirAll(t, filepath.Join(root, "topdata", "data", "classes", "feats"))
|
||||||
|
mkdirAll(t, filepath.Join(root, "topdata", "data", "feat"))
|
||||||
|
mkdirAll(t, filepath.Join(root, "topdata", "data", "masterfeats"))
|
||||||
|
writeFile(t, filepath.Join(root, "topdata", "data", "classes", "base.json"), `{
|
||||||
|
"output": "classes.2da",
|
||||||
|
"columns": ["Label", "Name", "FeatsTable"],
|
||||||
|
"rows": [
|
||||||
|
{"id": 4, "key": "classes:fighter", "Label": "Fighter", "Name": "111", "FeatsTable": "cls_feat_fighter"},
|
||||||
|
{"id": 10, "key": "classes:wizard", "Label": "Wizard", "Name": "112", "FeatsTable": "cls_feat_wizard"}
|
||||||
|
]
|
||||||
|
}`+"\n")
|
||||||
|
writeFile(t, filepath.Join(root, "topdata", "data", "classes", "lock.json"), `{"classes:fighter":4,"classes:wizard":10}`+"\n")
|
||||||
|
writeFile(t, filepath.Join(root, "topdata", "data", "classes", "feats", "wizard.json"), `{
|
||||||
|
"key": "classes/feats:wizard",
|
||||||
|
"output": "cls_feat_wizard.2da",
|
||||||
|
"columns": ["FeatLabel", "FeatIndex", "List", "GrantedOnLevel", "OnMenu"],
|
||||||
|
"rows": [
|
||||||
|
{"FeatIndex": {"id": "masterfeats:spellfocus"}, "List": "1", "GrantedOnLevel": "-1", "OnMenu": "0"},
|
||||||
|
{"FeatIndex": {"id": "masterfeats:weaponspecialization"}, "List": "1", "GrantedOnLevel": "-1", "OnMenu": "0"}
|
||||||
|
]
|
||||||
|
}`+"\n")
|
||||||
|
writeFile(t, filepath.Join(root, "topdata", "data", "feat", "base.json"), `{
|
||||||
|
"output": "feat.2da",
|
||||||
|
"columns": ["LABEL", "FEAT", "DESCRIPTION", "MASTERFEAT", "ALLCLASSESCANUSE", "MinLevelClass"],
|
||||||
|
"rows": [
|
||||||
|
{"id": 100, "key": "feat:spellfocus_abjuration", "LABEL": "SpellFocusAbj", "FEAT": "1000", "DESCRIPTION": "1001", "MASTERFEAT": "1", "ALLCLASSESCANUSE": "0", "MinLevelClass": "****"},
|
||||||
|
{"id": 101, "key": "feat:spellfocus_conjuration", "LABEL": "SpellFocusCon", "FEAT": "1002", "DESCRIPTION": "1003", "MASTERFEAT": "1", "ALLCLASSESCANUSE": "1", "MinLevelClass": "****"},
|
||||||
|
{"id": 200, "key": "feat:weaponspecialization_longsword", "LABEL": "WeaponSpecLongsword", "FEAT": "2000", "DESCRIPTION": "2001", "MASTERFEAT": "2", "ALLCLASSESCANUSE": "0", "MinLevelClass": {"id": "classes:fighter"}}
|
||||||
|
]
|
||||||
|
}`+"\n")
|
||||||
|
writeFile(t, filepath.Join(root, "topdata", "data", "feat", "lock.json"), `{
|
||||||
|
"feat:spellfocus_abjuration": 100,
|
||||||
|
"feat:spellfocus_conjuration": 101,
|
||||||
|
"feat:weaponspecialization_longsword": 200
|
||||||
|
}`+"\n")
|
||||||
|
writeFile(t, filepath.Join(root, "topdata", "data", "masterfeats", "base.json"), `{
|
||||||
|
"output": "masterfeats.2da",
|
||||||
|
"columns": ["LABEL", "STRREF", "DESCRIPTION", "ICON"],
|
||||||
|
"rows": [
|
||||||
|
{"id": 1, "key": "masterfeats:spellfocus", "LABEL": "SpellFocus", "STRREF": "6492", "DESCRIPTION": "426", "ICON": "ife_magic"},
|
||||||
|
{"id": 2, "key": "masterfeats:weaponspecialization", "LABEL": "WeaponSpecialization", "STRREF": "6491", "DESCRIPTION": "437", "ICON": "ife_wepspec"}
|
||||||
|
]
|
||||||
|
}`+"\n")
|
||||||
|
writeFile(t, filepath.Join(root, "topdata", "data", "masterfeats", "lock.json"), `{
|
||||||
|
"masterfeats:spellfocus": 1,
|
||||||
|
"masterfeats:weaponspecialization": 2
|
||||||
|
}`+"\n")
|
||||||
|
|
||||||
|
report := ValidationReport{}
|
||||||
|
validateClassFeatMasterfeatAccessibility(filepath.Join(root, "topdata", "data"), &report)
|
||||||
|
|
||||||
|
warned := map[string]bool{}
|
||||||
|
for _, diagnostic := range report.Diagnostics {
|
||||||
|
if diagnostic.Severity != SeverityWarning {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
for _, featKey := range []string{"feat:spellfocus_abjuration", "feat:spellfocus_conjuration", "feat:weaponspecialization_longsword"} {
|
||||||
|
if strings.Contains(diagnostic.Message, featKey) {
|
||||||
|
warned[featKey] = true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if !warned["feat:spellfocus_abjuration"] {
|
||||||
|
t.Fatalf("expected warning for unobtainable ALLCLASSESCANUSE=0 feat, got: %+v", report.Diagnostics)
|
||||||
|
}
|
||||||
|
if warned["feat:spellfocus_conjuration"] {
|
||||||
|
t.Fatalf("did not expect warning for ALLCLASSESCANUSE=1 feat, got: %+v", report.Diagnostics)
|
||||||
|
}
|
||||||
|
if warned["feat:weaponspecialization_longsword"] {
|
||||||
|
t.Fatalf("did not expect warning for feat with explicit MinLevelClass, got: %+v", report.Diagnostics)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -1,226 +0,0 @@
|
|||||||
package topdata
|
|
||||||
|
|
||||||
import (
|
|
||||||
"encoding/json"
|
|
||||||
"os"
|
|
||||||
"path/filepath"
|
|
||||||
"slices"
|
|
||||||
"strings"
|
|
||||||
)
|
|
||||||
|
|
||||||
var legacyClassesPlainFamilies = []string{"feats", "skills", "savthr", "bfeat", "pres"}
|
|
||||||
|
|
||||||
func importLegacyClasses(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
|
|
||||||
legacyRoot := filepath.Join(referenceBuilderDir, "data", "classes")
|
|
||||||
if _, err := os.Stat(legacyRoot); err != nil {
|
|
||||||
if os.IsNotExist(err) {
|
|
||||||
return 0, nil
|
|
||||||
}
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
|
|
||||||
targetRoot := filepath.Join(dataDir, "classes")
|
|
||||||
if canonicalClassesPresent(targetRoot) {
|
|
||||||
return 0, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
coreCollected, plainCollected, err := collectLegacyClassesDatasets(legacyRoot)
|
|
||||||
if err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
|
|
||||||
tableKeyByOutput := map[string]string{}
|
|
||||||
for _, dataset := range plainCollected {
|
|
||||||
if dataset.TableKey == "" {
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
registerLegacyClassesTableKey(tableKeyByOutput, dataset.TableKey, dataset.Dataset.OutputName)
|
|
||||||
}
|
|
||||||
|
|
||||||
coreRows := make([]map[string]any, 0, len(coreCollected.Rows))
|
|
||||||
for _, rawRow := range coreCollected.Rows {
|
|
||||||
row, ok := deepCopyValue(rawRow).(map[string]any)
|
|
||||||
if !ok {
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
if legacyTLK != nil {
|
|
||||||
if _, _, err := inlineLegacyTLKValue(row, legacyTLK); err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
}
|
|
||||||
rewriteLegacyClassesCoreRow(row, tableKeyByOutput)
|
|
||||||
coreRows = append(coreRows, row)
|
|
||||||
}
|
|
||||||
|
|
||||||
if err := removePathIfExists(targetRoot); err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
if err := os.MkdirAll(filepath.Join(targetRoot, "core"), 0o755); err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
|
|
||||||
writes := []struct {
|
|
||||||
path string
|
|
||||||
obj map[string]any
|
|
||||||
}{
|
|
||||||
{
|
|
||||||
path: filepath.Join(targetRoot, "core", "base.json"),
|
|
||||||
obj: map[string]any{
|
|
||||||
"output": coreCollected.Dataset.OutputName,
|
|
||||||
"columns": stringSliceToAny(coreCollected.Columns),
|
|
||||||
"rows": rowsToAny(coreRows),
|
|
||||||
},
|
|
||||||
},
|
|
||||||
{
|
|
||||||
path: filepath.Join(targetRoot, "core", "lock.json"),
|
|
||||||
obj: anyMapInt(coreCollected.LockData),
|
|
||||||
},
|
|
||||||
}
|
|
||||||
|
|
||||||
for _, collected := range plainCollected {
|
|
||||||
obj := map[string]any{
|
|
||||||
"output": collected.Dataset.OutputName,
|
|
||||||
"columns": stringSliceToAny(collected.Columns),
|
|
||||||
"rows": rowsToAny(collected.Rows),
|
|
||||||
}
|
|
||||||
if strings.TrimSpace(collected.TableKey) != "" {
|
|
||||||
obj["key"] = collected.TableKey
|
|
||||||
}
|
|
||||||
writes = append(writes, struct {
|
|
||||||
path string
|
|
||||||
obj map[string]any
|
|
||||||
}{
|
|
||||||
path: filepath.Join(dataDir, collected.Dataset.Name+".json"),
|
|
||||||
obj: obj,
|
|
||||||
})
|
|
||||||
}
|
|
||||||
|
|
||||||
for _, write := range writes {
|
|
||||||
if err := os.MkdirAll(filepath.Dir(write.path), 0o755); err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
raw, err := json.MarshalIndent(write.obj, "", " ")
|
|
||||||
if err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
raw = append(raw, '\n')
|
|
||||||
if err := os.WriteFile(write.path, raw, 0o644); err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
return len(writes), nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func canonicalClassesPresent(targetRoot string) bool {
|
|
||||||
if !fileExists(filepath.Join(targetRoot, "core", "base.json")) || !fileExists(filepath.Join(targetRoot, "core", "lock.json")) {
|
|
||||||
return false
|
|
||||||
}
|
|
||||||
for _, family := range legacyClassesPlainFamilies {
|
|
||||||
ok, err := hasJSONFiles(filepath.Join(targetRoot, family))
|
|
||||||
if err != nil || !ok {
|
|
||||||
return false
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return true
|
|
||||||
}
|
|
||||||
|
|
||||||
func collectLegacyClassesDatasets(legacyRoot string) (nativeCollectedDataset, []nativeCollectedDataset, error) {
|
|
||||||
coreCollected, err := collectBaseDataset(nativeDataset{
|
|
||||||
Name: "classes/core",
|
|
||||||
BasePath: filepath.Join(legacyRoot, "core", "base.json"),
|
|
||||||
LockPath: filepath.Join(legacyRoot, "core", "lock.json"),
|
|
||||||
ModulesDir: filepath.Join(legacyRoot, "core", "modules"),
|
|
||||||
OutputName: "classes.2da",
|
|
||||||
Spec: specForDataset("classes"),
|
|
||||||
})
|
|
||||||
if err != nil {
|
|
||||||
return nativeCollectedDataset{}, nil, err
|
|
||||||
}
|
|
||||||
coreCollected.Dataset.OutputName = "classes.2da"
|
|
||||||
|
|
||||||
plainCollected := make([]nativeCollectedDataset, 0)
|
|
||||||
for _, family := range legacyClassesPlainFamilies {
|
|
||||||
familyDir := filepath.Join(legacyRoot, family)
|
|
||||||
entries, err := os.ReadDir(familyDir)
|
|
||||||
if err != nil {
|
|
||||||
if os.IsNotExist(err) {
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
return nativeCollectedDataset{}, nil, err
|
|
||||||
}
|
|
||||||
for _, entry := range entries {
|
|
||||||
if entry.IsDir() || filepath.Ext(entry.Name()) != ".json" || strings.HasPrefix(entry.Name(), ".") || entry.Name() == "lock.json" {
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
filePath := filepath.Join(familyDir, entry.Name())
|
|
||||||
tableData, err := loadJSONObject(filePath)
|
|
||||||
if err != nil {
|
|
||||||
return nativeCollectedDataset{}, nil, err
|
|
||||||
}
|
|
||||||
outputName, _ := tableData["output"].(string)
|
|
||||||
if strings.TrimSpace(outputName) == "" {
|
|
||||||
outputName = strings.TrimSuffix(entry.Name(), filepath.Ext(entry.Name())) + ".2da"
|
|
||||||
}
|
|
||||||
collected, err := collectPlainDataset(nativeDataset{
|
|
||||||
Name: filepath.ToSlash(filepath.Join("classes", family, strings.TrimSuffix(entry.Name(), filepath.Ext(entry.Name())))),
|
|
||||||
BasePath: filePath,
|
|
||||||
LockPath: filepath.Join(familyDir, "lock.json"),
|
|
||||||
OutputName: outputName,
|
|
||||||
Spec: specForDataset(filepath.ToSlash(filepath.Join("classes", family, strings.TrimSuffix(entry.Name(), filepath.Ext(entry.Name()))))),
|
|
||||||
})
|
|
||||||
if err != nil {
|
|
||||||
return nativeCollectedDataset{}, nil, err
|
|
||||||
}
|
|
||||||
plainCollected = append(plainCollected, collected)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
slices.SortFunc(plainCollected, func(a, b nativeCollectedDataset) int {
|
|
||||||
return strings.Compare(a.Dataset.Name, b.Dataset.Name)
|
|
||||||
})
|
|
||||||
return coreCollected, plainCollected, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func registerLegacyClassesTableKey(tableKeyByOutput map[string]string, tableKey, outputName string) {
|
|
||||||
trimmedOutput := strings.TrimSpace(outputName)
|
|
||||||
if trimmedOutput != "" {
|
|
||||||
tableKeyByOutput[trimmedOutput] = tableKey
|
|
||||||
}
|
|
||||||
trimmedStem := strings.TrimSpace(outputStem(outputName))
|
|
||||||
if trimmedStem != "" {
|
|
||||||
tableKeyByOutput[trimmedStem] = tableKey
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func rewriteLegacyClassesCoreRow(row map[string]any, tableKeyByOutput map[string]string) {
|
|
||||||
for _, field := range []string{"FeatsTable", "SavingThrowTable", "SkillsTable", "BonusFeatsTable", "PreReqTable"} {
|
|
||||||
tableKey := legacyClassesTableKeyForValue(row[field], tableKeyByOutput)
|
|
||||||
if tableKey == "" {
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
row[field] = map[string]any{"table": tableKey}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func legacyClassesTableKeyForValue(value any, tableKeyByOutput map[string]string) string {
|
|
||||||
switch typed := value.(type) {
|
|
||||||
case map[string]any:
|
|
||||||
tableKey, _ := typed["table"].(string)
|
|
||||||
return strings.TrimSpace(tableKey)
|
|
||||||
case string:
|
|
||||||
trimmed := strings.TrimSpace(typed)
|
|
||||||
if trimmed == "" || trimmed == nullValue {
|
|
||||||
return ""
|
|
||||||
}
|
|
||||||
if trimmed != strings.ToLower(trimmed) {
|
|
||||||
return ""
|
|
||||||
}
|
|
||||||
if tableKey, ok := tableKeyByOutput[trimmed]; ok {
|
|
||||||
return tableKey
|
|
||||||
}
|
|
||||||
return ""
|
|
||||||
default:
|
|
||||||
return ""
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -1,6 +0,0 @@
|
|||||||
package topdata
|
|
||||||
|
|
||||||
func importLegacyCloakmodel(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
|
|
||||||
_ = legacyTLK
|
|
||||||
return importLegacyDatasetMirror(referenceBuilderDir, dataDir, "cloakmodel", "cloakmodel.2da", nil)
|
|
||||||
}
|
|
||||||
@@ -1,145 +0,0 @@
|
|||||||
package topdata
|
|
||||||
|
|
||||||
import (
|
|
||||||
"encoding/json"
|
|
||||||
"os"
|
|
||||||
"path/filepath"
|
|
||||||
)
|
|
||||||
|
|
||||||
func importLegacyCreaturespeed(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
|
|
||||||
_ = legacyTLK
|
|
||||||
|
|
||||||
legacyDir := filepath.Join(referenceBuilderDir, "data", "creaturespeed")
|
|
||||||
if _, err := os.Stat(legacyDir); err != nil {
|
|
||||||
if os.IsNotExist(err) {
|
|
||||||
return 0, nil
|
|
||||||
}
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
|
|
||||||
targetDir := filepath.Join(dataDir, "creaturespeed")
|
|
||||||
targetPath := filepath.Join(targetDir, "creaturespeed.json")
|
|
||||||
if _, err := os.Stat(targetPath); err == nil {
|
|
||||||
obj, err := loadJSONObject(targetPath)
|
|
||||||
if err == nil && countLegacyTLKRefsInValue(obj) == 0 {
|
|
||||||
return 0, nil
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
baseObj, err := loadJSONObject(filepath.Join(legacyDir, "base.json"))
|
|
||||||
if err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
rows, err := mergeCreaturespeedRows(baseObj, filepath.Join(legacyDir, "modules"))
|
|
||||||
if err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
if err := os.MkdirAll(targetDir, 0o755); err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
|
|
||||||
if err := removePathIfExists(filepath.Join(targetDir, "base.json")); err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
if err := removePathIfExists(filepath.Join(targetDir, "lock.json")); err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
if err := removePathIfExists(filepath.Join(targetDir, "modules")); err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
|
|
||||||
out := map[string]any{
|
|
||||||
"output": "creaturespeed.2da",
|
|
||||||
"columns": baseObj["columns"],
|
|
||||||
"rows": rows,
|
|
||||||
}
|
|
||||||
|
|
||||||
raw, err := json.MarshalIndent(out, "", " ")
|
|
||||||
if err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
raw = append(raw, '\n')
|
|
||||||
if err := os.WriteFile(targetPath, raw, 0o644); err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
return 1, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func mergeCreaturespeedRows(baseObj map[string]any, modulesDir string) ([]any, error) {
|
|
||||||
rawRows, ok := baseObj["rows"].([]any)
|
|
||||||
if !ok {
|
|
||||||
return nil, nil
|
|
||||||
}
|
|
||||||
rows := make([]map[string]any, 0, len(rawRows))
|
|
||||||
byID := map[int]map[string]any{}
|
|
||||||
for _, raw := range rawRows {
|
|
||||||
row, ok := deepCopyValue(raw).(map[string]any)
|
|
||||||
if !ok {
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
delete(row, "key")
|
|
||||||
rows = append(rows, row)
|
|
||||||
if rawID, ok := row["id"]; ok {
|
|
||||||
id, err := asInt(rawID)
|
|
||||||
if err != nil {
|
|
||||||
return nil, err
|
|
||||||
}
|
|
||||||
byID[id] = row
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
modulePaths, err := collectModulePaths(modulesDir)
|
|
||||||
if err != nil {
|
|
||||||
return nil, err
|
|
||||||
}
|
|
||||||
for _, path := range modulePaths {
|
|
||||||
obj, err := loadJSONObject(path)
|
|
||||||
if err != nil {
|
|
||||||
return nil, err
|
|
||||||
}
|
|
||||||
overrideList, ok := obj["overrides"].([]any)
|
|
||||||
if !ok {
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
for _, raw := range overrideList {
|
|
||||||
override, ok := raw.(map[string]any)
|
|
||||||
if !ok {
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
rawID, ok := override["id"]
|
|
||||||
if !ok {
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
id, err := asInt(rawID)
|
|
||||||
if err != nil {
|
|
||||||
return nil, err
|
|
||||||
}
|
|
||||||
row := byID[id]
|
|
||||||
if row == nil {
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
for key, value := range override {
|
|
||||||
if key == "id" || key == "key" || key == "_tlk" {
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
row[key] = deepCopyValue(value)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
out := make([]any, 0, len(rows))
|
|
||||||
for _, row := range rows {
|
|
||||||
out = append(out, row)
|
|
||||||
}
|
|
||||||
return out, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func removePathIfExists(path string) error {
|
|
||||||
if _, err := os.Stat(path); err != nil {
|
|
||||||
if os.IsNotExist(err) {
|
|
||||||
return nil
|
|
||||||
}
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
return os.RemoveAll(path)
|
|
||||||
}
|
|
||||||
@@ -1,288 +0,0 @@
|
|||||||
package topdata
|
|
||||||
|
|
||||||
import (
|
|
||||||
"encoding/json"
|
|
||||||
"fmt"
|
|
||||||
"os"
|
|
||||||
"path/filepath"
|
|
||||||
"sort"
|
|
||||||
"strings"
|
|
||||||
)
|
|
||||||
|
|
||||||
func importLegacyDamagetypes(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
|
|
||||||
_ = legacyTLK
|
|
||||||
|
|
||||||
legacyRoot := filepath.Join(referenceBuilderDir, "data", "damagetypes")
|
|
||||||
if _, err := os.Stat(legacyRoot); err != nil {
|
|
||||||
if os.IsNotExist(err) {
|
|
||||||
return 0, nil
|
|
||||||
}
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
|
|
||||||
targetDir := filepath.Join(dataDir, "damagetypes", "registry")
|
|
||||||
targetPath := filepath.Join(targetDir, "types.json")
|
|
||||||
if _, err := os.Stat(targetPath); err == nil {
|
|
||||||
obj, err := loadJSONObject(targetPath)
|
|
||||||
if err == nil && countLegacyTLKRefsInValue(obj) == 0 {
|
|
||||||
return 0, nil
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
coreBase, err := loadJSONObject(filepath.Join(legacyRoot, "core", "base.json"))
|
|
||||||
if err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
groupBase, err := loadJSONObject(filepath.Join(legacyRoot, "groups", "base.json"))
|
|
||||||
if err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
hitvisualBase, err := loadJSONObject(filepath.Join(legacyRoot, "hitvisual", "base.json"))
|
|
||||||
if err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
coreModule, err := loadJSONObject(filepath.Join(legacyRoot, "core", "modules", "add_eos.json"))
|
|
||||||
if err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
coreLock, err := loadLockfile(filepath.Join(legacyRoot, "core", "lock.json"))
|
|
||||||
if err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
groupModule, err := loadJSONObject(filepath.Join(legacyRoot, "groups", "modules", "add_eos.json"))
|
|
||||||
if err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
groupLock, err := loadLockfile(filepath.Join(legacyRoot, "groups", "lock.json"))
|
|
||||||
if err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
hitvisualModule, err := loadJSONObject(filepath.Join(legacyRoot, "hitvisual", "modules", "add_eos.json"))
|
|
||||||
if err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
hitvisualLock, err := loadLockfile(filepath.Join(legacyRoot, "hitvisual", "lock.json"))
|
|
||||||
if err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
|
|
||||||
rows, lockData, err := mergeLegacyDamagetypes(coreBase, groupBase, hitvisualBase, coreModule, coreLock, groupModule, groupLock, hitvisualModule, hitvisualLock)
|
|
||||||
if err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
|
|
||||||
if err := os.MkdirAll(targetDir, 0o755); err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
if err := removePathIfExists(filepath.Join(dataDir, "damagetypes", "core")); err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
if err := removePathIfExists(filepath.Join(dataDir, "damagetypes", "groups")); err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
if err := removePathIfExists(filepath.Join(dataDir, "damagetypes", "hitvisual")); err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
|
|
||||||
typesOut := map[string]any{"rows": rows}
|
|
||||||
raw, err := json.MarshalIndent(typesOut, "", " ")
|
|
||||||
if err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
raw = append(raw, '\n')
|
|
||||||
if err := os.WriteFile(targetPath, raw, 0o644); err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
if err := saveLockfile(filepath.Join(targetDir, damagetypesRegistryLock), lockData); err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
return 2, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func mergeLegacyDamagetypes(coreBase, groupBase, hitvisualBase, coreModule map[string]any, coreLock map[string]int, groupModule map[string]any, groupLock map[string]int, hitvisualModule map[string]any, hitvisualLock map[string]int) ([]map[string]any, map[string]int, error) {
|
|
||||||
coreRows, err := coerceRowMapSlice(coreBase["rows"])
|
|
||||||
if err != nil {
|
|
||||||
return nil, nil, err
|
|
||||||
}
|
|
||||||
groupRows, err := coerceRowMapSlice(groupBase["rows"])
|
|
||||||
if err != nil {
|
|
||||||
return nil, nil, err
|
|
||||||
}
|
|
||||||
hitRows, err := coerceRowMapSlice(hitvisualBase["rows"])
|
|
||||||
if err != nil {
|
|
||||||
return nil, nil, err
|
|
||||||
}
|
|
||||||
groupByID := map[int]map[string]any{}
|
|
||||||
for _, row := range groupRows {
|
|
||||||
id, err := asInt(row["id"])
|
|
||||||
if err != nil {
|
|
||||||
return nil, nil, err
|
|
||||||
}
|
|
||||||
groupByID[id] = row
|
|
||||||
}
|
|
||||||
hitByID := map[int]map[string]any{}
|
|
||||||
for _, row := range hitRows {
|
|
||||||
id, err := asInt(row["id"])
|
|
||||||
if err != nil {
|
|
||||||
return nil, nil, err
|
|
||||||
}
|
|
||||||
hitByID[id] = row
|
|
||||||
}
|
|
||||||
|
|
||||||
outRows := make([]map[string]any, 0, len(coreRows))
|
|
||||||
lockData := map[string]int{}
|
|
||||||
for _, core := range coreRows {
|
|
||||||
id, err := asInt(core["id"])
|
|
||||||
if err != nil {
|
|
||||||
return nil, nil, err
|
|
||||||
}
|
|
||||||
groupID, err := asInt(core["DamageTypeGroup"])
|
|
||||||
if err != nil {
|
|
||||||
return nil, nil, err
|
|
||||||
}
|
|
||||||
group := groupByID[groupID]
|
|
||||||
if group == nil {
|
|
||||||
return nil, nil, fmt.Errorf("core id %d references missing group %d", id, groupID)
|
|
||||||
}
|
|
||||||
hit := hitByID[id]
|
|
||||||
if hit == nil {
|
|
||||||
return nil, nil, fmt.Errorf("core id %d is missing hitvisual row", id)
|
|
||||||
}
|
|
||||||
key := "damagetype:" + normalizeDamagetypeKey(core["Label"])
|
|
||||||
lockData[key] = id
|
|
||||||
outRows = append(outRows, map[string]any{
|
|
||||||
"key": key,
|
|
||||||
"id": id,
|
|
||||||
"label": deepCopyValue(core["Label"]),
|
|
||||||
"charsheet_strref": deepCopyValue(core["CharsheetStrref"]),
|
|
||||||
"damage_type_group": strconvString(groupID),
|
|
||||||
"damage_ranged_projectile": deepCopyValue(core["DamageRangedProjectile"]),
|
|
||||||
"group_label": deepCopyValue(group["Label"]),
|
|
||||||
"feedback_strref": deepCopyValue(group["FeedbackStrref"]),
|
|
||||||
"color_r": deepCopyValue(group["ColorR"]),
|
|
||||||
"color_g": deepCopyValue(group["ColorG"]),
|
|
||||||
"color_b": deepCopyValue(group["ColorB"]),
|
|
||||||
"visual_effect_id": deepCopyValue(hit["VisualEffectID"]),
|
|
||||||
"ranged_effect_id": deepCopyValue(hit["RangedEffectID"]),
|
|
||||||
})
|
|
||||||
}
|
|
||||||
|
|
||||||
coreEntries, err := coerceEntriesMap(coreModule["entries"])
|
|
||||||
if err != nil {
|
|
||||||
return nil, nil, err
|
|
||||||
}
|
|
||||||
groupEntries, err := coerceEntriesMap(groupModule["entries"])
|
|
||||||
if err != nil {
|
|
||||||
return nil, nil, err
|
|
||||||
}
|
|
||||||
hitEntries, err := coerceEntriesMap(hitvisualModule["entries"])
|
|
||||||
if err != nil {
|
|
||||||
return nil, nil, err
|
|
||||||
}
|
|
||||||
|
|
||||||
keys := make([]string, 0, len(coreEntries))
|
|
||||||
for key := range coreEntries {
|
|
||||||
keys = append(keys, key)
|
|
||||||
}
|
|
||||||
sort.Slice(keys, func(i, j int) bool {
|
|
||||||
a := strings.TrimPrefix(keys[i], "damagetypes:")
|
|
||||||
b := strings.TrimPrefix(keys[j], "damagetypes:")
|
|
||||||
return a < b
|
|
||||||
})
|
|
||||||
for _, coreKey := range keys {
|
|
||||||
core := coreEntries[coreKey]
|
|
||||||
suffix := strings.TrimPrefix(coreKey, "damagetypes:")
|
|
||||||
group := groupEntries["damagetypegroups:"+suffix]
|
|
||||||
if group == nil {
|
|
||||||
return nil, nil, fmt.Errorf("%s: missing matching group entry", coreKey)
|
|
||||||
}
|
|
||||||
hit := hitEntries["damagehitvisual:"+suffix]
|
|
||||||
if hit == nil {
|
|
||||||
return nil, nil, fmt.Errorf("%s: missing matching hitvisual entry", coreKey)
|
|
||||||
}
|
|
||||||
id, ok := coreLock[coreKey]
|
|
||||||
if !ok {
|
|
||||||
return nil, nil, fmt.Errorf("%s: missing core lock id", coreKey)
|
|
||||||
}
|
|
||||||
groupKey := "damagetypegroups:" + suffix
|
|
||||||
groupID, ok := groupLock[groupKey]
|
|
||||||
if !ok {
|
|
||||||
return nil, nil, fmt.Errorf("%s: missing group lock id", groupKey)
|
|
||||||
}
|
|
||||||
hitKey := "damagehitvisual:" + suffix
|
|
||||||
hitID, ok := hitvisualLock[hitKey]
|
|
||||||
if !ok {
|
|
||||||
return nil, nil, fmt.Errorf("%s: missing hitvisual lock id", hitKey)
|
|
||||||
}
|
|
||||||
if hitID != id {
|
|
||||||
return nil, nil, fmt.Errorf("%s: core id %d does not match hitvisual id %d", suffix, id, hitID)
|
|
||||||
}
|
|
||||||
key := "damagetype:" + suffix
|
|
||||||
lockData[key] = id
|
|
||||||
outRows = append(outRows, map[string]any{
|
|
||||||
"key": key,
|
|
||||||
"id": id,
|
|
||||||
"label": deepCopyValue(core["Label"]),
|
|
||||||
"charsheet_strref": deepCopyValue(core["CharsheetStrref"]),
|
|
||||||
"damage_type_group": strconvString(groupID),
|
|
||||||
"damage_ranged_projectile": deepCopyValue(core["DamageRangedProjectile"]),
|
|
||||||
"group_label": deepCopyValue(group["Label"]),
|
|
||||||
"feedback_strref": deepCopyValue(group["FeedbackStrref"]),
|
|
||||||
"color_r": deepCopyValue(group["ColorR"]),
|
|
||||||
"color_g": deepCopyValue(group["ColorG"]),
|
|
||||||
"color_b": deepCopyValue(group["ColorB"]),
|
|
||||||
"visual_effect_id": deepCopyValue(hit["VisualEffectID"]),
|
|
||||||
"ranged_effect_id": deepCopyValue(hit["RangedEffectID"]),
|
|
||||||
})
|
|
||||||
}
|
|
||||||
|
|
||||||
sort.Slice(outRows, func(i, j int) bool {
|
|
||||||
return outRows[i]["id"].(int) < outRows[j]["id"].(int)
|
|
||||||
})
|
|
||||||
return outRows, lockData, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func coerceRowMapSlice(value any) ([]map[string]any, error) {
|
|
||||||
rawRows, ok := value.([]any)
|
|
||||||
if !ok {
|
|
||||||
return nil, fmt.Errorf("rows must be an array")
|
|
||||||
}
|
|
||||||
rows := make([]map[string]any, 0, len(rawRows))
|
|
||||||
for _, raw := range rawRows {
|
|
||||||
row, ok := raw.(map[string]any)
|
|
||||||
if !ok {
|
|
||||||
return nil, fmt.Errorf("row must be an object")
|
|
||||||
}
|
|
||||||
rows = append(rows, row)
|
|
||||||
}
|
|
||||||
return rows, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func coerceEntriesMap(value any) (map[string]map[string]any, error) {
|
|
||||||
rawEntries, ok := value.(map[string]any)
|
|
||||||
if !ok {
|
|
||||||
return nil, fmt.Errorf("entries must be an object")
|
|
||||||
}
|
|
||||||
entries := make(map[string]map[string]any, len(rawEntries))
|
|
||||||
for key, raw := range rawEntries {
|
|
||||||
row, ok := raw.(map[string]any)
|
|
||||||
if !ok {
|
|
||||||
return nil, fmt.Errorf("entry %s must be an object", key)
|
|
||||||
}
|
|
||||||
entries[key] = row
|
|
||||||
}
|
|
||||||
return entries, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func normalizeDamagetypeKey(value any) string {
|
|
||||||
text := strings.TrimSpace(strings.ToLower(format2DAValue(value)))
|
|
||||||
text = strings.TrimSuffix(text, "damage")
|
|
||||||
text = strings.ReplaceAll(text, " ", "")
|
|
||||||
text = strings.ReplaceAll(text, "_", "")
|
|
||||||
text = strings.ReplaceAll(text, "-", "")
|
|
||||||
return text
|
|
||||||
}
|
|
||||||
|
|
||||||
func strconvString(v int) string {
|
|
||||||
return fmt.Sprintf("%d", v)
|
|
||||||
}
|
|
||||||
@@ -1,99 +0,0 @@
|
|||||||
package topdata
|
|
||||||
|
|
||||||
import (
|
|
||||||
"encoding/json"
|
|
||||||
"os"
|
|
||||||
"path/filepath"
|
|
||||||
)
|
|
||||||
|
|
||||||
func importLegacyDoortypes(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
|
|
||||||
_ = legacyTLK
|
|
||||||
|
|
||||||
legacyDir := filepath.Join(referenceBuilderDir, "data", "doortypes")
|
|
||||||
if _, err := os.Stat(legacyDir); err != nil {
|
|
||||||
if os.IsNotExist(err) {
|
|
||||||
return 0, nil
|
|
||||||
}
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
|
|
||||||
targetDir := filepath.Join(dataDir, "doortypes")
|
|
||||||
targetBasePath := filepath.Join(targetDir, "base.json")
|
|
||||||
targetLockPath := filepath.Join(targetDir, "lock.json")
|
|
||||||
targetModulesDir := filepath.Join(targetDir, "modules")
|
|
||||||
moduleNames := []string{
|
|
||||||
"add_pld01.json",
|
|
||||||
"add_sic11.json",
|
|
||||||
"add_tapr.json",
|
|
||||||
"add_tdm01.json",
|
|
||||||
"add_tdx01.json",
|
|
||||||
"add_tei01.json",
|
|
||||||
"add_tfm01.json",
|
|
||||||
}
|
|
||||||
targetModulePaths := make([]string, 0, len(moduleNames))
|
|
||||||
for _, name := range moduleNames {
|
|
||||||
targetModulePaths = append(targetModulePaths, filepath.Join(targetModulesDir, name))
|
|
||||||
}
|
|
||||||
if fileExists(targetBasePath) && fileExists(targetLockPath) {
|
|
||||||
allModulesPresent := true
|
|
||||||
for _, path := range targetModulePaths {
|
|
||||||
if !fileExists(path) {
|
|
||||||
allModulesPresent = false
|
|
||||||
break
|
|
||||||
}
|
|
||||||
}
|
|
||||||
if allModulesPresent {
|
|
||||||
return 0, nil
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
baseObj, err := loadJSONObject(filepath.Join(legacyDir, "base.json"))
|
|
||||||
if err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
baseObj["output"] = "doortypes.2da"
|
|
||||||
|
|
||||||
lockObj, err := loadJSONObject(filepath.Join(legacyDir, "lock.json"))
|
|
||||||
if err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
|
|
||||||
moduleObjs := make([]map[string]any, 0, len(moduleNames))
|
|
||||||
for _, name := range moduleNames {
|
|
||||||
obj, err := loadJSONObject(filepath.Join(legacyDir, "modules", name))
|
|
||||||
if err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
moduleObjs = append(moduleObjs, obj)
|
|
||||||
}
|
|
||||||
|
|
||||||
if err := os.MkdirAll(targetModulesDir, 0o755); err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
|
|
||||||
writes := []struct {
|
|
||||||
path string
|
|
||||||
obj map[string]any
|
|
||||||
}{
|
|
||||||
{path: targetBasePath, obj: baseObj},
|
|
||||||
{path: targetLockPath, obj: lockObj},
|
|
||||||
}
|
|
||||||
for i, path := range targetModulePaths {
|
|
||||||
writes = append(writes, struct {
|
|
||||||
path string
|
|
||||||
obj map[string]any
|
|
||||||
}{path: path, obj: moduleObjs[i]})
|
|
||||||
}
|
|
||||||
|
|
||||||
for _, write := range writes {
|
|
||||||
raw, err := json.MarshalIndent(write.obj, "", " ")
|
|
||||||
if err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
raw = append(raw, '\n')
|
|
||||||
if err := os.WriteFile(write.path, raw, 0o644); err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return len(writes), nil
|
|
||||||
}
|
|
||||||
@@ -1,64 +0,0 @@
|
|||||||
package topdata
|
|
||||||
|
|
||||||
import (
|
|
||||||
"encoding/json"
|
|
||||||
"os"
|
|
||||||
"path/filepath"
|
|
||||||
)
|
|
||||||
|
|
||||||
func importLegacyFeat(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
|
|
||||||
_ = legacyTLK
|
|
||||||
|
|
||||||
legacyDir := filepath.Join(referenceBuilderDir, "data", "feat")
|
|
||||||
if _, err := os.Stat(legacyDir); err != nil {
|
|
||||||
if os.IsNotExist(err) {
|
|
||||||
return 0, nil
|
|
||||||
}
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
legacyBasePath := filepath.Join(legacyDir, "base.json")
|
|
||||||
if _, err := os.Stat(legacyBasePath); err != nil {
|
|
||||||
if os.IsNotExist(err) {
|
|
||||||
return 0, nil
|
|
||||||
}
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
|
|
||||||
targetDir := filepath.Join(dataDir, "feat")
|
|
||||||
targetBasePath := filepath.Join(targetDir, "base.json")
|
|
||||||
var targetObj map[string]any
|
|
||||||
if _, err := os.Stat(targetBasePath); err == nil {
|
|
||||||
targetObj, err = loadJSONObject(targetBasePath)
|
|
||||||
if err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
} else if !os.IsNotExist(err) {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
if targetObj != nil {
|
|
||||||
if rows, ok := targetObj["rows"].([]any); ok && len(rows) > 0 {
|
|
||||||
return 0, nil
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
baseObj, err := loadJSONObject(legacyBasePath)
|
|
||||||
if err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
baseObj["output"] = "feat.2da"
|
|
||||||
baseObj["compare_reference"] = false
|
|
||||||
canonicalizeWikiMetadataDocument(baseObj)
|
|
||||||
|
|
||||||
if err := os.MkdirAll(targetDir, 0o755); err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
raw, err := json.MarshalIndent(baseObj, "", " ")
|
|
||||||
if err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
raw = append(raw, '\n')
|
|
||||||
if err := os.WriteFile(targetBasePath, raw, 0o644); err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
return 1, nil
|
|
||||||
}
|
|
||||||
@@ -1,6 +0,0 @@
|
|||||||
package topdata
|
|
||||||
|
|
||||||
func importLegacyGenericdoors(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
|
|
||||||
_ = legacyTLK
|
|
||||||
return importLegacyDatasetMirror(referenceBuilderDir, dataDir, "genericdoors", "genericdoors.2da", nil)
|
|
||||||
}
|
|
||||||
@@ -601,397 +601,6 @@ func deepCopyValue(value any) any {
|
|||||||
return out
|
return out
|
||||||
}
|
}
|
||||||
|
|
||||||
func importLegacyItempropsRegistry(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
|
|
||||||
legacyDataDir := filepath.Join(referenceBuilderDir, "data")
|
|
||||||
registryDir := filepath.Join(dataDir, filepath.FromSlash(itempropsRegistryDirName))
|
|
||||||
if _, err := os.Stat(filepath.Join(registryDir, "properties.json")); err == nil {
|
|
||||||
if refs, err := countLegacyTLKRefsInRegistry(registryDir); err == nil && refs == 0 {
|
|
||||||
return 0, nil
|
|
||||||
}
|
|
||||||
}
|
|
||||||
if _, err := os.Stat(filepath.Join(legacyDataDir, "itemprops")); err != nil {
|
|
||||||
if os.IsNotExist(err) {
|
|
||||||
return 0, nil
|
|
||||||
}
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
|
|
||||||
defs, err := collectBaseDataset(nativeDataset{
|
|
||||||
Name: "itemprops/defs",
|
|
||||||
BasePath: filepath.Join(legacyDataDir, "itemprops", "defs", "base.json"),
|
|
||||||
LockPath: filepath.Join(legacyDataDir, "itemprops", "defs", "lock.json"),
|
|
||||||
ModulesDir: filepath.Join(legacyDataDir, "itemprops", "defs", "modules"),
|
|
||||||
Spec: specForDataset("itemprops"),
|
|
||||||
})
|
|
||||||
if err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
sets, err := collectBaseDataset(nativeDataset{
|
|
||||||
Name: "itemprops/sets",
|
|
||||||
BasePath: filepath.Join(legacyDataDir, "itemprops", "sets", "base.json"),
|
|
||||||
LockPath: filepath.Join(legacyDataDir, "itemprops", "sets", "lock.json"),
|
|
||||||
ModulesDir: filepath.Join(legacyDataDir, "itemprops", "sets", "modules"),
|
|
||||||
Spec: specForDataset("itemprops"),
|
|
||||||
})
|
|
||||||
if err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
costIndex, err := collectBaseDataset(nativeDataset{
|
|
||||||
Name: "itemprops/costtables/index",
|
|
||||||
BasePath: filepath.Join(legacyDataDir, "itemprops", "costtables", "index", "base.json"),
|
|
||||||
LockPath: filepath.Join(legacyDataDir, "itemprops", "costtables", "index", "lock.json"),
|
|
||||||
ModulesDir: filepath.Join(legacyDataDir, "itemprops", "costtables", "index", "modules"),
|
|
||||||
Spec: specForDataset("itemprops"),
|
|
||||||
})
|
|
||||||
if err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
paramIndex, err := collectBaseDataset(nativeDataset{
|
|
||||||
Name: "itemprops/paramtables/index",
|
|
||||||
BasePath: filepath.Join(legacyDataDir, "itemprops", "paramtables", "index", "base.json"),
|
|
||||||
LockPath: filepath.Join(legacyDataDir, "itemprops", "paramtables", "index", "lock.json"),
|
|
||||||
ModulesDir: filepath.Join(legacyDataDir, "itemprops", "paramtables", "index", "modules"),
|
|
||||||
Spec: specForDataset("itemprops"),
|
|
||||||
})
|
|
||||||
if err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
|
|
||||||
ownedCostTables, err := collectLegacyOwnedItempropTables(filepath.Join(legacyDataDir, "itemprops", "costtables"), "index")
|
|
||||||
if err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
ownedParamTables, err := collectLegacyOwnedItempropTables(filepath.Join(legacyDataDir, "itemprops", "paramtables"), "index")
|
|
||||||
if err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
ownedSubtypeTables, err := collectLegacyOwnedItempropTables(filepath.Join(legacyDataDir, "itemprops", "subtypes"), "")
|
|
||||||
if err != nil && !os.IsNotExist(err) {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
|
|
||||||
setsByID := map[int]map[string]any{}
|
|
||||||
for _, row := range sets.Rows {
|
|
||||||
setsByID[row["id"].(int)] = row
|
|
||||||
}
|
|
||||||
costByID := map[int]map[string]any{}
|
|
||||||
for _, row := range costIndex.Rows {
|
|
||||||
costByID[row["id"].(int)] = row
|
|
||||||
}
|
|
||||||
paramByID := map[int]map[string]any{}
|
|
||||||
for _, row := range paramIndex.Rows {
|
|
||||||
paramByID[row["id"].(int)] = row
|
|
||||||
}
|
|
||||||
|
|
||||||
registryLock := map[string]int{}
|
|
||||||
legacyKeyByID := map[string]string{}
|
|
||||||
if legacyTLK != nil {
|
|
||||||
for key, id := range legacyTLK.Lock {
|
|
||||||
legacyKeyByID[strconv.Itoa(id)] = key
|
|
||||||
}
|
|
||||||
}
|
|
||||||
properties := make([]map[string]any, 0, len(defs.Rows))
|
|
||||||
subtypeRegistryByKey := map[string]map[string]any{}
|
|
||||||
for _, row := range defs.Rows {
|
|
||||||
rowID := row["id"].(int)
|
|
||||||
key := canonicalItempropPropertyKey(row)
|
|
||||||
registryLock[key] = rowID
|
|
||||||
|
|
||||||
property := map[string]any{
|
|
||||||
"key": key,
|
|
||||||
"label": row["Label"],
|
|
||||||
"name": deepCopyValue(row["Name"]),
|
|
||||||
"property_text": deepCopyValue(row["GameStrRef"]),
|
|
||||||
}
|
|
||||||
if description := nullableRegistryField(row, "Description"); description != nullValue {
|
|
||||||
property["description"] = description
|
|
||||||
}
|
|
||||||
if setRow, ok := setsByID[rowID]; ok {
|
|
||||||
property["availability"] = importLegacyAvailability(setRow)
|
|
||||||
if setName := nullableRegistryField(setRow, "StringRef"); setName != nullValue && setName != stringField(row, "Name") {
|
|
||||||
property["set_name"] = setName
|
|
||||||
}
|
|
||||||
if setLabel := stringField(setRow, "Label"); setLabel != "" && setLabel != stringField(row, "Label") {
|
|
||||||
property["set_label"] = setLabel
|
|
||||||
}
|
|
||||||
} else {
|
|
||||||
property["availability"] = []any{}
|
|
||||||
}
|
|
||||||
|
|
||||||
if subtypeResRef := stringField(row, "SubTypeResRef"); subtypeResRef != "" && subtypeResRef != nullValue {
|
|
||||||
subtypeKey := canonicalModelKey("subtype_models", subtypeResRef)
|
|
||||||
property["subtype"] = map[string]any{"ref": subtypeKey}
|
|
||||||
if _, ok := subtypeRegistryByKey[subtypeKey]; !ok {
|
|
||||||
subtypeRegistryByKey[subtypeKey] = buildLegacySubtypeModel(subtypeKey, subtypeResRef, ownedSubtypeTables)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
costValue := stringField(row, "Cost")
|
|
||||||
costID := stringField(row, "CostTableResRef")
|
|
||||||
if costValue != "" && costValue != nullValue || costID != "" && costID != nullValue {
|
|
||||||
cost := map[string]any{}
|
|
||||||
if costValue != "" && costValue != nullValue {
|
|
||||||
cost["value"] = costValue
|
|
||||||
}
|
|
||||||
if costID != "" && costID != nullValue {
|
|
||||||
id, err := strconv.Atoi(costID)
|
|
||||||
if err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
costRow, ok := costByID[id]
|
|
||||||
if !ok {
|
|
||||||
return 0, fmt.Errorf("%s references missing cost table index %s", key, costID)
|
|
||||||
}
|
|
||||||
cost["ref"] = canonicalModelKey("cost_models", stringField(costRow, "Name"))
|
|
||||||
}
|
|
||||||
property["cost"] = cost
|
|
||||||
}
|
|
||||||
if paramID := stringField(row, "Param1ResRef"); paramID != "" && paramID != nullValue {
|
|
||||||
id, err := strconv.Atoi(paramID)
|
|
||||||
if err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
paramRow, ok := paramByID[id]
|
|
||||||
if !ok {
|
|
||||||
return 0, fmt.Errorf("%s references missing param table index %s", key, paramID)
|
|
||||||
}
|
|
||||||
property["param"] = map[string]any{"ref": canonicalModelKey("param_models", stringField(paramRow, "TableResRef"))}
|
|
||||||
}
|
|
||||||
if legacyTLK != nil {
|
|
||||||
property["name"] = importLegacyStrrefValue(property["name"], key+".name", legacyTLK, legacyKeyByID)
|
|
||||||
property["property_text"] = importLegacyStrrefValue(property["property_text"], key+".property_text", legacyTLK, legacyKeyByID)
|
|
||||||
if description, ok := property["description"]; ok {
|
|
||||||
property["description"] = importLegacyStrrefValue(description, key+".description", legacyTLK, legacyKeyByID)
|
|
||||||
}
|
|
||||||
if updated, _, err := inlineLegacyTLKValue(property, legacyTLK); err != nil {
|
|
||||||
return 0, err
|
|
||||||
} else if updated {
|
|
||||||
// property updated in place
|
|
||||||
}
|
|
||||||
}
|
|
||||||
properties = append(properties, property)
|
|
||||||
}
|
|
||||||
|
|
||||||
costModels := make([]map[string]any, 0, len(costIndex.Rows))
|
|
||||||
for _, row := range costIndex.Rows {
|
|
||||||
key := canonicalModelKey("cost_models", stringField(row, "Name"))
|
|
||||||
registryLock[key] = row["id"].(int)
|
|
||||||
model := map[string]any{
|
|
||||||
"key": key,
|
|
||||||
"index_name": row["Name"],
|
|
||||||
"label": row["Label"],
|
|
||||||
"client_load": row["ClientLoad"],
|
|
||||||
"table": buildLegacyOwnedModelTable(stringField(row, "Name"), ownedCostTables),
|
|
||||||
}
|
|
||||||
costModels = append(costModels, model)
|
|
||||||
}
|
|
||||||
|
|
||||||
paramModels := make([]map[string]any, 0, len(paramIndex.Rows))
|
|
||||||
for _, row := range paramIndex.Rows {
|
|
||||||
key := canonicalModelKey("param_models", stringField(row, "TableResRef"))
|
|
||||||
registryLock[key] = row["id"].(int)
|
|
||||||
model := map[string]any{
|
|
||||||
"key": key,
|
|
||||||
"name": deepCopyValue(row["Name"]),
|
|
||||||
"label": row["Lable"],
|
|
||||||
"table": buildLegacyOwnedModelTable(stringField(row, "TableResRef"), ownedParamTables),
|
|
||||||
}
|
|
||||||
paramModels = append(paramModels, model)
|
|
||||||
}
|
|
||||||
|
|
||||||
subtypeModels := make([]map[string]any, 0, len(subtypeRegistryByKey))
|
|
||||||
for _, key := range sortedKeysAny(subtypeRegistryByKey) {
|
|
||||||
subtypeModels = append(subtypeModels, subtypeRegistryByKey[key])
|
|
||||||
}
|
|
||||||
|
|
||||||
writes := []struct {
|
|
||||||
path string
|
|
||||||
obj map[string]any
|
|
||||||
}{
|
|
||||||
{filepath.Join(registryDir, "properties.json"), map[string]any{"rows": properties}},
|
|
||||||
{filepath.Join(registryDir, "cost_models.json"), map[string]any{"rows": costModels}},
|
|
||||||
{filepath.Join(registryDir, "param_models.json"), map[string]any{"rows": paramModels}},
|
|
||||||
{filepath.Join(registryDir, "subtype_models.json"), map[string]any{"rows": subtypeModels}},
|
|
||||||
{filepath.Join(registryDir, itempropsRegistryLock), anyMapInt(registryLock)},
|
|
||||||
}
|
|
||||||
if err := os.MkdirAll(registryDir, 0o755); err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
for _, write := range writes {
|
|
||||||
raw, err := json.MarshalIndent(write.obj, "", " ")
|
|
||||||
if err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
raw = append(raw, '\n')
|
|
||||||
if err := os.WriteFile(write.path, raw, 0o644); err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return len(writes), nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func countLegacyTLKRefsInRegistry(registryDir string) (int, error) {
|
|
||||||
paths, err := collectDataJSONPaths(registryDir)
|
|
||||||
if err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
refs := 0
|
|
||||||
for _, path := range paths {
|
|
||||||
obj, err := loadJSONObject(path)
|
|
||||||
if err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
refs += countLegacyTLKRefsInValue(obj)
|
|
||||||
}
|
|
||||||
return refs, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func collectLegacyOwnedItempropTables(root, skipDir string) (map[string]map[string]any, error) {
|
|
||||||
result := map[string]map[string]any{}
|
|
||||||
entries, err := os.ReadDir(root)
|
|
||||||
if err != nil {
|
|
||||||
return nil, err
|
|
||||||
}
|
|
||||||
for _, entry := range entries {
|
|
||||||
if !entry.IsDir() {
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
if skipDir != "" && entry.Name() == skipDir {
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
dir := filepath.Join(root, entry.Name())
|
|
||||||
basePath := filepath.Join(dir, "base.json")
|
|
||||||
if _, err := os.Stat(basePath); err != nil {
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
collected, err := collectBaseDataset(nativeDataset{
|
|
||||||
Name: filepath.ToSlash(filepath.Join("itemprops", filepath.Base(root), entry.Name())),
|
|
||||||
BasePath: basePath,
|
|
||||||
LockPath: filepath.Join(dir, "lock.json"),
|
|
||||||
ModulesDir: filepath.Join(dir, "modules"),
|
|
||||||
Spec: specForDataset("itemprops"),
|
|
||||||
})
|
|
||||||
if err != nil {
|
|
||||||
return nil, err
|
|
||||||
}
|
|
||||||
result[strings.ToLower(outputStem(collected.Dataset.OutputName))] = map[string]any{
|
|
||||||
"ownership": "owned",
|
|
||||||
"resref": outputStem(collected.Dataset.OutputName),
|
|
||||||
"output": collected.Dataset.OutputName,
|
|
||||||
"columns": stringSliceToAny(collected.Columns),
|
|
||||||
"rows": rowsToAny(collected.Rows),
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return result, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func buildLegacySubtypeModel(key, resref string, owned map[string]map[string]any) map[string]any {
|
|
||||||
return map[string]any{
|
|
||||||
"key": key,
|
|
||||||
"table_resref": resref,
|
|
||||||
"table": buildLegacyOwnedModelTable(resref, owned),
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func buildLegacyOwnedModelTable(resref string, owned map[string]map[string]any) map[string]any {
|
|
||||||
if table, ok := owned[strings.ToLower(resref)]; ok {
|
|
||||||
return table
|
|
||||||
}
|
|
||||||
return map[string]any{
|
|
||||||
"ownership": "external",
|
|
||||||
"resref": resref,
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func canonicalItempropPropertyKey(row map[string]any) string {
|
|
||||||
if key := stringField(row, "key"); key != "" {
|
|
||||||
return "itemprop:" + strings.TrimPrefix(key, "itempropdef:")
|
|
||||||
}
|
|
||||||
return canonicalModelKey("itemprop", stringField(row, "Label"))
|
|
||||||
}
|
|
||||||
|
|
||||||
func canonicalModelKey(prefix, source string) string {
|
|
||||||
normalized := strings.ToLower(strings.TrimSpace(source))
|
|
||||||
normalized = strings.ReplaceAll(normalized, " ", "")
|
|
||||||
normalized = strings.ReplaceAll(normalized, "-", "")
|
|
||||||
normalized = strings.ReplaceAll(normalized, "_", "")
|
|
||||||
return prefix + ":" + normalized
|
|
||||||
}
|
|
||||||
|
|
||||||
func importLegacyStrrefValue(value any, fallbackKey string, legacy *legacyTLKData, keyByID map[string]string) any {
|
|
||||||
if legacy == nil {
|
|
||||||
return value
|
|
||||||
}
|
|
||||||
if _, ok, _ := parseTLKPayload(value, true); ok {
|
|
||||||
return value
|
|
||||||
}
|
|
||||||
_ = keyByID
|
|
||||||
key := fallbackKey
|
|
||||||
entry, ok := legacy.Entries[key]
|
|
||||||
if !ok {
|
|
||||||
return value
|
|
||||||
}
|
|
||||||
payload := map[string]any{
|
|
||||||
"key": key,
|
|
||||||
"text": entry.Text,
|
|
||||||
}
|
|
||||||
if entry.SoundResRef != "" {
|
|
||||||
payload["sound_resref"] = entry.SoundResRef
|
|
||||||
}
|
|
||||||
if entry.SoundLength != 0 {
|
|
||||||
payload["sound_length"] = entry.SoundLength
|
|
||||||
}
|
|
||||||
return map[string]any{"tlk": payload}
|
|
||||||
}
|
|
||||||
|
|
||||||
func importLegacyAvailability(row map[string]any) []any {
|
|
||||||
out := make([]any, 0)
|
|
||||||
for _, column := range itempropsAvailabilityColumns {
|
|
||||||
value, ok := row[column]
|
|
||||||
if !ok {
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
text := formatRegistryScalar(value)
|
|
||||||
if text != "1" {
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
out = append(out, normalizeAvailabilityName(column))
|
|
||||||
}
|
|
||||||
return out
|
|
||||||
}
|
|
||||||
|
|
||||||
func anyMapInt(values map[string]int) map[string]any {
|
|
||||||
out := make(map[string]any, len(values))
|
|
||||||
for key, value := range values {
|
|
||||||
out[key] = value
|
|
||||||
}
|
|
||||||
return out
|
|
||||||
}
|
|
||||||
|
|
||||||
func stringSliceToAny(values []string) []any {
|
|
||||||
out := make([]any, 0, len(values))
|
|
||||||
for _, value := range values {
|
|
||||||
out = append(out, value)
|
|
||||||
}
|
|
||||||
return out
|
|
||||||
}
|
|
||||||
|
|
||||||
func rowsToAny(rows []map[string]any) []any {
|
|
||||||
out := make([]any, 0, len(rows))
|
|
||||||
for _, row := range rows {
|
|
||||||
out = append(out, deepCopyValue(row))
|
|
||||||
}
|
|
||||||
return out
|
|
||||||
}
|
|
||||||
|
|
||||||
func sortedKeysAny(values map[string]map[string]any) []string {
|
|
||||||
out := make([]string, 0, len(values))
|
|
||||||
for key := range values {
|
|
||||||
out = append(out, key)
|
|
||||||
}
|
|
||||||
slices.Sort(out)
|
|
||||||
return out
|
|
||||||
}
|
|
||||||
|
|
||||||
func validateItempropsRegistryGraph(dataDir string, report *ValidationReport) {
|
func validateItempropsRegistryGraph(dataDir string, report *ValidationReport) {
|
||||||
registry, err := loadItempropsRegistry(dataDir)
|
registry, err := loadItempropsRegistry(dataDir)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
|
|||||||
@@ -1,176 +0,0 @@
|
|||||||
package topdata
|
|
||||||
|
|
||||||
import (
|
|
||||||
"encoding/json"
|
|
||||||
"io/fs"
|
|
||||||
"os"
|
|
||||||
"path/filepath"
|
|
||||||
"sort"
|
|
||||||
"strings"
|
|
||||||
)
|
|
||||||
|
|
||||||
type legacyDatasetTransform func(relativePath string, obj map[string]any)
|
|
||||||
|
|
||||||
func importLegacyDatasetMirror(referenceBuilderDir, dataDir, datasetName, outputName string, transform legacyDatasetTransform) (int, error) {
|
|
||||||
legacyDir := filepath.Join(referenceBuilderDir, "data", datasetName)
|
|
||||||
if _, err := os.Stat(legacyDir); err != nil {
|
|
||||||
if os.IsNotExist(err) {
|
|
||||||
return 0, nil
|
|
||||||
}
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
|
|
||||||
targetDir := filepath.Join(dataDir, datasetName)
|
|
||||||
baseObj, err := loadJSONObject(filepath.Join(legacyDir, "base.json"))
|
|
||||||
if err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
if outputName != "" {
|
|
||||||
baseObj["output"] = outputName
|
|
||||||
}
|
|
||||||
if transform != nil {
|
|
||||||
transform("base.json", baseObj)
|
|
||||||
}
|
|
||||||
|
|
||||||
lockObj, err := loadJSONObject(filepath.Join(legacyDir, "lock.json"))
|
|
||||||
if err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
if transform != nil {
|
|
||||||
transform("lock.json", lockObj)
|
|
||||||
}
|
|
||||||
|
|
||||||
moduleRelPaths, err := collectJSONRelativePaths(filepath.Join(legacyDir, "modules"))
|
|
||||||
if err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
|
|
||||||
moduleObjs := make(map[string]map[string]any, len(moduleRelPaths))
|
|
||||||
for _, relPath := range moduleRelPaths {
|
|
||||||
obj, err := loadJSONObject(filepath.Join(legacyDir, "modules", relPath))
|
|
||||||
if err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
normalizedRel := filepath.ToSlash(relPath)
|
|
||||||
if transform != nil {
|
|
||||||
transform(filepath.ToSlash(filepath.Join("modules", normalizedRel)), obj)
|
|
||||||
}
|
|
||||||
moduleObjs[normalizedRel] = obj
|
|
||||||
}
|
|
||||||
|
|
||||||
if err := os.MkdirAll(targetDir, 0o755); err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
|
|
||||||
updated := 0
|
|
||||||
for _, write := range []struct {
|
|
||||||
path string
|
|
||||||
obj map[string]any
|
|
||||||
}{
|
|
||||||
{path: filepath.Join(targetDir, "base.json"), obj: baseObj},
|
|
||||||
{path: filepath.Join(targetDir, "lock.json"), obj: lockObj},
|
|
||||||
} {
|
|
||||||
changed, err := writeJSONObjectIfChanged(write.path, write.obj)
|
|
||||||
if err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
if changed {
|
|
||||||
updated++
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
targetModulesDir := filepath.Join(targetDir, "modules")
|
|
||||||
if len(moduleObjs) > 0 {
|
|
||||||
if err := os.MkdirAll(targetModulesDir, 0o755); err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
}
|
|
||||||
for relPath, obj := range moduleObjs {
|
|
||||||
targetPath := filepath.Join(targetModulesDir, filepath.FromSlash(relPath))
|
|
||||||
if err := os.MkdirAll(filepath.Dir(targetPath), 0o755); err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
changed, err := writeJSONObjectIfChanged(targetPath, obj)
|
|
||||||
if err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
if changed {
|
|
||||||
updated++
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
staleModules, err := collectJSONRelativePaths(targetModulesDir)
|
|
||||||
if err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
for _, relPath := range staleModules {
|
|
||||||
normalizedRel := filepath.ToSlash(relPath)
|
|
||||||
if _, ok := moduleObjs[normalizedRel]; ok {
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
if err := os.Remove(filepath.Join(targetModulesDir, filepath.FromSlash(normalizedRel))); err != nil && !os.IsNotExist(err) {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
updated++
|
|
||||||
}
|
|
||||||
|
|
||||||
return updated, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func collectJSONRelativePaths(root string) ([]string, error) {
|
|
||||||
if _, err := os.Stat(root); err != nil {
|
|
||||||
if os.IsNotExist(err) {
|
|
||||||
return nil, nil
|
|
||||||
}
|
|
||||||
return nil, err
|
|
||||||
}
|
|
||||||
|
|
||||||
var relPaths []string
|
|
||||||
err := filepath.WalkDir(root, func(path string, d fs.DirEntry, err error) error {
|
|
||||||
if err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
if d.IsDir() {
|
|
||||||
return nil
|
|
||||||
}
|
|
||||||
if !strings.EqualFold(filepath.Ext(d.Name()), ".json") {
|
|
||||||
return nil
|
|
||||||
}
|
|
||||||
relPath, err := filepath.Rel(root, path)
|
|
||||||
if err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
relPaths = append(relPaths, filepath.ToSlash(relPath))
|
|
||||||
return nil
|
|
||||||
})
|
|
||||||
if err != nil {
|
|
||||||
return nil, err
|
|
||||||
}
|
|
||||||
sort.Strings(relPaths)
|
|
||||||
return relPaths, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func writeJSONObjectIfChanged(path string, obj map[string]any) (bool, error) {
|
|
||||||
raw, err := json.MarshalIndent(obj, "", " ")
|
|
||||||
if err != nil {
|
|
||||||
return false, err
|
|
||||||
}
|
|
||||||
raw = append(raw, '\n')
|
|
||||||
|
|
||||||
current, err := os.ReadFile(path)
|
|
||||||
if err == nil && string(current) == string(raw) {
|
|
||||||
return false, nil
|
|
||||||
}
|
|
||||||
if err != nil && !os.IsNotExist(err) {
|
|
||||||
return false, err
|
|
||||||
}
|
|
||||||
if err := os.WriteFile(path, raw, 0o644); err != nil {
|
|
||||||
return false, err
|
|
||||||
}
|
|
||||||
return true, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func fileExists(path string) bool {
|
|
||||||
_, err := os.Stat(path)
|
|
||||||
return err == nil
|
|
||||||
}
|
|
||||||
@@ -1,6 +0,0 @@
|
|||||||
package topdata
|
|
||||||
|
|
||||||
func importLegacyLoadscreens(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
|
|
||||||
_ = legacyTLK
|
|
||||||
return importLegacyDatasetMirror(referenceBuilderDir, dataDir, "loadscreens", "loadscreens.2da", nil)
|
|
||||||
}
|
|
||||||
@@ -1,237 +0,0 @@
|
|||||||
package topdata
|
|
||||||
|
|
||||||
import (
|
|
||||||
"encoding/json"
|
|
||||||
"os"
|
|
||||||
"path/filepath"
|
|
||||||
"strconv"
|
|
||||||
)
|
|
||||||
|
|
||||||
func importLegacyMasterfeats(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
|
|
||||||
legacyDir := filepath.Join(referenceBuilderDir, "data", "feat", "masterfeats")
|
|
||||||
if _, err := os.Stat(legacyDir); err != nil {
|
|
||||||
if os.IsNotExist(err) {
|
|
||||||
return 0, nil
|
|
||||||
}
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
|
|
||||||
targetDir := filepath.Join(dataDir, "masterfeats")
|
|
||||||
targetPath := filepath.Join(targetDir, "base.json")
|
|
||||||
legacyPlainPath := filepath.Join(targetDir, "masterfeats.json")
|
|
||||||
if _, err := os.Stat(targetPath); err == nil {
|
|
||||||
obj, err := loadJSONObject(targetPath)
|
|
||||||
if err == nil && countLegacyTLKRefsInValue(obj) == 0 {
|
|
||||||
return 0, nil
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
collected, err := importLegacyMasterfeatsDataset(legacyDir)
|
|
||||||
if err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
referenceIDs, err := collectReferenceLockIDs(filepath.Join(referenceBuilderDir, "data"))
|
|
||||||
if err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
|
|
||||||
rows := make([]any, 0, len(collected.Rows))
|
|
||||||
for _, row := range collected.Rows {
|
|
||||||
copyRow, ok := deepCopyValue(row).(map[string]any)
|
|
||||||
if !ok {
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
if legacyTLK != nil {
|
|
||||||
if _, _, err := inlineLegacyTLKValue(copyRow, legacyTLK); err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
}
|
|
||||||
copyRow = rewriteImportedIDRefs(copyRow, referenceIDs)
|
|
||||||
rows = append(rows, copyRow)
|
|
||||||
}
|
|
||||||
|
|
||||||
if err := os.MkdirAll(targetDir, 0o755); err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
writes := []struct {
|
|
||||||
path string
|
|
||||||
obj map[string]any
|
|
||||||
}{
|
|
||||||
{
|
|
||||||
path: targetPath,
|
|
||||||
obj: map[string]any{
|
|
||||||
"output": collected.Dataset.OutputName,
|
|
||||||
"columns": stringSliceToAny(collected.Columns),
|
|
||||||
"rows": rows,
|
|
||||||
},
|
|
||||||
},
|
|
||||||
{
|
|
||||||
path: filepath.Join(targetDir, "lock.json"),
|
|
||||||
obj: anyMapInt(collected.LockData),
|
|
||||||
},
|
|
||||||
}
|
|
||||||
for _, write := range writes {
|
|
||||||
raw, err := json.MarshalIndent(write.obj, "", " ")
|
|
||||||
if err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
raw = append(raw, '\n')
|
|
||||||
if err := os.WriteFile(write.path, raw, 0o644); err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
}
|
|
||||||
updatedFiles := len(writes)
|
|
||||||
if err := os.Remove(legacyPlainPath); err == nil {
|
|
||||||
updatedFiles++
|
|
||||||
} else if err != nil && !os.IsNotExist(err) {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
return updatedFiles, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func importLegacyMasterfeatsDataset(legacyDir string) (nativeCollectedDataset, error) {
|
|
||||||
dataset := nativeDataset{
|
|
||||||
Name: "masterfeats",
|
|
||||||
BasePath: filepath.Join(legacyDir, "masterfeats.json"),
|
|
||||||
LockPath: filepath.Join(legacyDir, "lock.json"),
|
|
||||||
Spec: specForDataset("masterfeats"),
|
|
||||||
}
|
|
||||||
tableData, err := loadJSONObject(dataset.BasePath)
|
|
||||||
if err != nil {
|
|
||||||
return nativeCollectedDataset{}, err
|
|
||||||
}
|
|
||||||
columns, err := parseColumns(tableData, dataset.Name)
|
|
||||||
if err != nil {
|
|
||||||
return nativeCollectedDataset{}, err
|
|
||||||
}
|
|
||||||
rawRows, ok := tableData["rows"].([]any)
|
|
||||||
if !ok {
|
|
||||||
return nativeCollectedDataset{}, nil
|
|
||||||
}
|
|
||||||
lockData, err := loadLockfile(dataset.LockPath)
|
|
||||||
if err != nil {
|
|
||||||
return nativeCollectedDataset{}, err
|
|
||||||
}
|
|
||||||
usedIDs := map[int]struct{}{}
|
|
||||||
for _, rowID := range lockData {
|
|
||||||
usedIDs[rowID] = struct{}{}
|
|
||||||
}
|
|
||||||
|
|
||||||
rows := make([]map[string]any, 0, len(rawRows))
|
|
||||||
explicitID := make([]bool, 0, len(rawRows))
|
|
||||||
for index, raw := range rawRows {
|
|
||||||
rowObj, ok := raw.(map[string]any)
|
|
||||||
if !ok {
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
rowID := index
|
|
||||||
if value, ok := rowObj["id"]; ok {
|
|
||||||
parsed, err := asInt(value)
|
|
||||||
if err != nil {
|
|
||||||
return nativeCollectedDataset{}, err
|
|
||||||
}
|
|
||||||
rowID = parsed
|
|
||||||
}
|
|
||||||
row := map[string]any{"id": rowID}
|
|
||||||
for _, column := range columns {
|
|
||||||
row[column] = nullValue
|
|
||||||
}
|
|
||||||
for key, value := range rowObj {
|
|
||||||
switch key {
|
|
||||||
case "id":
|
|
||||||
case "key":
|
|
||||||
if keyText, ok := value.(string); ok && keyText != "" {
|
|
||||||
row["key"] = keyText
|
|
||||||
}
|
|
||||||
default:
|
|
||||||
columnName, ok := canonicalColumn(columns, key)
|
|
||||||
if !ok {
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
row[columnName] = value
|
|
||||||
}
|
|
||||||
}
|
|
||||||
rows = append(rows, row)
|
|
||||||
_, hasExplicitID := rowObj["id"]
|
|
||||||
explicitID = append(explicitID, hasExplicitID)
|
|
||||||
if hasExplicitID {
|
|
||||||
usedIDs[rowID] = struct{}{}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
nextID := nextAvailableID(usedIDs)
|
|
||||||
lockModified := false
|
|
||||||
for index, row := range rows {
|
|
||||||
key, hasKey := row["key"].(string)
|
|
||||||
if hasKey && key != "" {
|
|
||||||
if lockedID, ok := lockData[key]; ok {
|
|
||||||
row["id"] = lockedID
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
if explicitID[index] {
|
|
||||||
lockData[key] = row["id"].(int)
|
|
||||||
} else {
|
|
||||||
row["id"] = nextID
|
|
||||||
lockData[key] = nextID
|
|
||||||
usedIDs[nextID] = struct{}{}
|
|
||||||
nextID = nextAvailableID(usedIDs)
|
|
||||||
}
|
|
||||||
lockModified = true
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
if !explicitID[index] {
|
|
||||||
row["id"] = index
|
|
||||||
}
|
|
||||||
}
|
|
||||||
if lockModified {
|
|
||||||
_ = lockModified
|
|
||||||
}
|
|
||||||
|
|
||||||
return nativeCollectedDataset{
|
|
||||||
Dataset: nativeDataset{
|
|
||||||
Name: dataset.Name,
|
|
||||||
OutputName: "masterfeats.2da",
|
|
||||||
Columns: columns,
|
|
||||||
Spec: dataset.Spec,
|
|
||||||
},
|
|
||||||
Columns: columns,
|
|
||||||
Rows: rows,
|
|
||||||
LockData: lockData,
|
|
||||||
}, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func rewriteImportedIDRefs(value any, ids map[string]int) map[string]any {
|
|
||||||
row, ok := rewriteImportedIDRefsValue(value, ids).(map[string]any)
|
|
||||||
if !ok {
|
|
||||||
return map[string]any{}
|
|
||||||
}
|
|
||||||
return row
|
|
||||||
}
|
|
||||||
|
|
||||||
func rewriteImportedIDRefsValue(value any, ids map[string]int) any {
|
|
||||||
switch typed := value.(type) {
|
|
||||||
case map[string]any:
|
|
||||||
if len(typed) == 1 {
|
|
||||||
if rawID, ok := typed["id"]; ok {
|
|
||||||
if key, ok := rawID.(string); ok {
|
|
||||||
if resolved, ok := ids[key]; ok {
|
|
||||||
return strconv.Itoa(resolved)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
out := make(map[string]any, len(typed))
|
|
||||||
for key, child := range typed {
|
|
||||||
out[key] = rewriteImportedIDRefsValue(child, ids)
|
|
||||||
}
|
|
||||||
return out
|
|
||||||
case []any:
|
|
||||||
out := make([]any, 0, len(typed))
|
|
||||||
for _, child := range typed {
|
|
||||||
out = append(out, rewriteImportedIDRefsValue(child, ids))
|
|
||||||
}
|
|
||||||
return out
|
|
||||||
default:
|
|
||||||
return value
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -1,415 +0,0 @@
|
|||||||
package topdata
|
|
||||||
|
|
||||||
import (
|
|
||||||
"encoding/json"
|
|
||||||
"fmt"
|
|
||||||
"os"
|
|
||||||
"path/filepath"
|
|
||||||
"strings"
|
|
||||||
|
|
||||||
"git.westgate.pw/ShadowsOverWestgate/sow-tools/internal/project"
|
|
||||||
)
|
|
||||||
|
|
||||||
type NormalizeResult struct {
|
|
||||||
StatePath string
|
|
||||||
ScannedFiles int
|
|
||||||
UpdatedFiles int
|
|
||||||
InlineValues int
|
|
||||||
RemainingFiles int
|
|
||||||
RemainingLegacyRefs int
|
|
||||||
}
|
|
||||||
|
|
||||||
func NormalizeProject(p *project.Project) (NormalizeResult, error) {
|
|
||||||
if !p.HasTopData() {
|
|
||||||
return NormalizeResult{}, fmt.Errorf("topdata is not configured for this project")
|
|
||||||
}
|
|
||||||
|
|
||||||
sourceDir := p.TopDataSourceDir()
|
|
||||||
dataDir := filepath.Join(sourceDir, "data")
|
|
||||||
migrationRoot := resolveMigrationInputRoot(sourceDir, p.TopDataReferenceBuilderDir())
|
|
||||||
legacyTLK, err := loadLegacyTLK(filepath.Join(sourceDir, "tlk"))
|
|
||||||
if err != nil {
|
|
||||||
return NormalizeResult{}, err
|
|
||||||
}
|
|
||||||
if migrationRoot != "" {
|
|
||||||
migrationTLK, err := loadLegacyTLK(filepath.Join(migrationRoot, "tlk"))
|
|
||||||
if err != nil {
|
|
||||||
return NormalizeResult{}, err
|
|
||||||
}
|
|
||||||
legacyTLK = mergeLegacyTLK(migrationTLK, legacyTLK)
|
|
||||||
}
|
|
||||||
baseDialogImported, err := importLegacyBaseDialog(migrationRoot, sourceDir)
|
|
||||||
if err != nil {
|
|
||||||
return NormalizeResult{}, err
|
|
||||||
}
|
|
||||||
|
|
||||||
itempropsImported, err := importLegacyItempropsRegistry(migrationRoot, dataDir, legacyTLK)
|
|
||||||
if err != nil {
|
|
||||||
return NormalizeResult{}, err
|
|
||||||
}
|
|
||||||
masterfeatsImported, err := importLegacyMasterfeats(migrationRoot, dataDir, legacyTLK)
|
|
||||||
if err != nil {
|
|
||||||
return NormalizeResult{}, err
|
|
||||||
}
|
|
||||||
featImported, err := importLegacyFeat(migrationRoot, dataDir, legacyTLK)
|
|
||||||
if err != nil {
|
|
||||||
return NormalizeResult{}, err
|
|
||||||
}
|
|
||||||
creaturespeedImported, err := importLegacyCreaturespeed(migrationRoot, dataDir, legacyTLK)
|
|
||||||
if err != nil {
|
|
||||||
return NormalizeResult{}, err
|
|
||||||
}
|
|
||||||
armorImported, err := importLegacyArmor(migrationRoot, dataDir, legacyTLK)
|
|
||||||
if err != nil {
|
|
||||||
return NormalizeResult{}, err
|
|
||||||
}
|
|
||||||
baseitemsImported, err := importLegacyBaseitems(migrationRoot, dataDir, legacyTLK)
|
|
||||||
if err != nil {
|
|
||||||
return NormalizeResult{}, err
|
|
||||||
}
|
|
||||||
damagetypesImported, err := importLegacyDamagetypes(migrationRoot, dataDir, legacyTLK)
|
|
||||||
if err != nil {
|
|
||||||
return NormalizeResult{}, err
|
|
||||||
}
|
|
||||||
skyboxesImported, err := importLegacySkyboxes(migrationRoot, dataDir, legacyTLK)
|
|
||||||
if err != nil {
|
|
||||||
return NormalizeResult{}, err
|
|
||||||
}
|
|
||||||
vfxPersistentImported, err := importLegacyVFXPersistent(migrationRoot, dataDir, legacyTLK)
|
|
||||||
if err != nil {
|
|
||||||
return NormalizeResult{}, err
|
|
||||||
}
|
|
||||||
loadscreensImported, err := importLegacyLoadscreens(migrationRoot, dataDir, legacyTLK)
|
|
||||||
if err != nil {
|
|
||||||
return NormalizeResult{}, err
|
|
||||||
}
|
|
||||||
progfxImported, err := importLegacyProgfx(migrationRoot, dataDir, legacyTLK)
|
|
||||||
if err != nil {
|
|
||||||
return NormalizeResult{}, err
|
|
||||||
}
|
|
||||||
genericdoorsImported, err := importLegacyGenericdoors(migrationRoot, dataDir, legacyTLK)
|
|
||||||
if err != nil {
|
|
||||||
return NormalizeResult{}, err
|
|
||||||
}
|
|
||||||
cloakmodelImported, err := importLegacyCloakmodel(migrationRoot, dataDir, legacyTLK)
|
|
||||||
if err != nil {
|
|
||||||
return NormalizeResult{}, err
|
|
||||||
}
|
|
||||||
repadjustImported, err := importLegacyRepadjust(migrationRoot, dataDir, legacyTLK)
|
|
||||||
if err != nil {
|
|
||||||
return NormalizeResult{}, err
|
|
||||||
}
|
|
||||||
wingmodelImported, err := importLegacyWingmodel(migrationRoot, dataDir, legacyTLK)
|
|
||||||
if err != nil {
|
|
||||||
return NormalizeResult{}, err
|
|
||||||
}
|
|
||||||
tailmodelImported, err := importLegacyTailmodel(migrationRoot, dataDir, legacyTLK)
|
|
||||||
if err != nil {
|
|
||||||
return NormalizeResult{}, err
|
|
||||||
}
|
|
||||||
doortypesImported, err := importLegacyDoortypes(migrationRoot, dataDir, legacyTLK)
|
|
||||||
if err != nil {
|
|
||||||
return NormalizeResult{}, err
|
|
||||||
}
|
|
||||||
visualeffectsImported, err := importLegacyVisualeffects(migrationRoot, dataDir, legacyTLK)
|
|
||||||
if err != nil {
|
|
||||||
return NormalizeResult{}, err
|
|
||||||
}
|
|
||||||
appearanceImported, err := importLegacyAppearance(migrationRoot, dataDir, legacyTLK)
|
|
||||||
if err != nil {
|
|
||||||
return NormalizeResult{}, err
|
|
||||||
}
|
|
||||||
placeablesImported, err := importLegacyPlaceables(migrationRoot, dataDir, legacyTLK)
|
|
||||||
if err != nil {
|
|
||||||
return NormalizeResult{}, err
|
|
||||||
}
|
|
||||||
rulesetImported, err := importLegacyRuleset(migrationRoot, dataDir, legacyTLK)
|
|
||||||
if err != nil {
|
|
||||||
return NormalizeResult{}, err
|
|
||||||
}
|
|
||||||
skillsImported, err := importLegacySkills(migrationRoot, dataDir, legacyTLK)
|
|
||||||
if err != nil {
|
|
||||||
return NormalizeResult{}, err
|
|
||||||
}
|
|
||||||
spellsImported, err := importLegacySpells(migrationRoot, dataDir, legacyTLK)
|
|
||||||
if err != nil {
|
|
||||||
return NormalizeResult{}, err
|
|
||||||
}
|
|
||||||
racialtypesImported, err := importLegacyRacialtypes(migrationRoot, dataDir, legacyTLK)
|
|
||||||
if err != nil {
|
|
||||||
return NormalizeResult{}, err
|
|
||||||
}
|
|
||||||
classesImported, err := importLegacyClasses(migrationRoot, dataDir, legacyTLK)
|
|
||||||
if err != nil {
|
|
||||||
return NormalizeResult{}, err
|
|
||||||
}
|
|
||||||
portraitsImported, err := importLegacyPortraits(migrationRoot, dataDir, legacyTLK)
|
|
||||||
if err != nil {
|
|
||||||
return NormalizeResult{}, err
|
|
||||||
}
|
|
||||||
|
|
||||||
paths, err := collectDataJSONPaths(dataDir)
|
|
||||||
if err != nil {
|
|
||||||
return NormalizeResult{}, err
|
|
||||||
}
|
|
||||||
|
|
||||||
result := NormalizeResult{
|
|
||||||
StatePath: filepath.Join(sourceDir, tlkStateFile),
|
|
||||||
ScannedFiles: len(paths),
|
|
||||||
UpdatedFiles: baseDialogImported + itempropsImported + masterfeatsImported + featImported + creaturespeedImported + armorImported + baseitemsImported + damagetypesImported + skyboxesImported + vfxPersistentImported + loadscreensImported + progfxImported + genericdoorsImported + cloakmodelImported + repadjustImported + wingmodelImported + tailmodelImported + doortypesImported + visualeffectsImported + appearanceImported + placeablesImported + rulesetImported + skillsImported + spellsImported + racialtypesImported + classesImported + portraitsImported,
|
|
||||||
}
|
|
||||||
state, err := loadTLKState(result.StatePath)
|
|
||||||
if err != nil {
|
|
||||||
return result, err
|
|
||||||
}
|
|
||||||
if state.Entries == nil {
|
|
||||||
state.Entries = map[string]tlkStateMapping{}
|
|
||||||
}
|
|
||||||
if legacyTLK != nil && state.Language == "" {
|
|
||||||
state.Language = legacyTLK.Language
|
|
||||||
}
|
|
||||||
if legacyTLK != nil {
|
|
||||||
for key, id := range legacyTLK.Lock {
|
|
||||||
if _, ok := state.Entries[key]; !ok {
|
|
||||||
state.Entries[key] = tlkStateMapping{ID: id}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
if legacyTLK != nil {
|
|
||||||
for _, path := range paths {
|
|
||||||
updated, count, err := inlineLegacyTLKFile(path, legacyTLK)
|
|
||||||
if err != nil {
|
|
||||||
return result, err
|
|
||||||
}
|
|
||||||
if updated {
|
|
||||||
result.UpdatedFiles++
|
|
||||||
result.InlineValues += count
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
if err := saveTLKState(result.StatePath, state); err != nil {
|
|
||||||
return result, err
|
|
||||||
}
|
|
||||||
if err := removeLegacyTLKAuthoredInput(filepath.Join(sourceDir, "tlk")); err != nil {
|
|
||||||
return result, err
|
|
||||||
}
|
|
||||||
remainingFiles, remainingRefs, err := countLegacyTLKRefs(paths)
|
|
||||||
if err != nil {
|
|
||||||
return result, err
|
|
||||||
}
|
|
||||||
result.RemainingFiles = remainingFiles
|
|
||||||
result.RemainingLegacyRefs = remainingRefs
|
|
||||||
return result, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func mergeLegacyTLK(base, override *legacyTLKData) *legacyTLKData {
|
|
||||||
if base == nil && override == nil {
|
|
||||||
return nil
|
|
||||||
}
|
|
||||||
merged := &legacyTLKData{
|
|
||||||
Language: "en",
|
|
||||||
Entries: map[string]tlkEntryData{},
|
|
||||||
Lock: map[string]int{},
|
|
||||||
}
|
|
||||||
if base != nil {
|
|
||||||
if base.Language != "" {
|
|
||||||
merged.Language = base.Language
|
|
||||||
}
|
|
||||||
for key, entry := range base.Entries {
|
|
||||||
merged.Entries[key] = entry
|
|
||||||
}
|
|
||||||
for key, id := range base.Lock {
|
|
||||||
merged.Lock[key] = id
|
|
||||||
}
|
|
||||||
}
|
|
||||||
if override != nil {
|
|
||||||
if override.Language != "" {
|
|
||||||
merged.Language = override.Language
|
|
||||||
}
|
|
||||||
for key, entry := range override.Entries {
|
|
||||||
merged.Entries[key] = entry
|
|
||||||
}
|
|
||||||
for key, id := range override.Lock {
|
|
||||||
merged.Lock[key] = id
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return merged
|
|
||||||
}
|
|
||||||
|
|
||||||
func removeLegacyTLKAuthoredInput(tlkDir string) error {
|
|
||||||
modulesDir := filepath.Join(tlkDir, "modules")
|
|
||||||
if err := os.RemoveAll(modulesDir); err != nil {
|
|
||||||
return fmt.Errorf("remove %s: %w", modulesDir, err)
|
|
||||||
}
|
|
||||||
lockPath := filepath.Join(tlkDir, "lock.json")
|
|
||||||
if err := os.Remove(lockPath); err != nil && !os.IsNotExist(err) {
|
|
||||||
return fmt.Errorf("remove %s: %w", lockPath, err)
|
|
||||||
}
|
|
||||||
gitkeepPath := filepath.Join(tlkDir, ".gitkeep")
|
|
||||||
if err := os.Remove(gitkeepPath); err != nil && !os.IsNotExist(err) {
|
|
||||||
return fmt.Errorf("remove %s: %w", gitkeepPath, err)
|
|
||||||
}
|
|
||||||
if err := os.Remove(tlkDir); err != nil && !os.IsNotExist(err) {
|
|
||||||
if !strings.Contains(err.Error(), "directory not empty") {
|
|
||||||
return fmt.Errorf("remove %s: %w", tlkDir, err)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func resolveMigrationInputRoot(sourceDir, referenceBuilderDir string) string {
|
|
||||||
if strings.TrimSpace(referenceBuilderDir) != "" {
|
|
||||||
return referenceBuilderDir
|
|
||||||
}
|
|
||||||
snapshotRoot := filepath.Join(sourceDir, "migration_snapshot")
|
|
||||||
if info, err := os.Stat(snapshotRoot); err == nil && info.IsDir() {
|
|
||||||
return snapshotRoot
|
|
||||||
}
|
|
||||||
return ""
|
|
||||||
}
|
|
||||||
|
|
||||||
func inlineLegacyTLKFile(path string, legacy *legacyTLKData) (bool, int, error) {
|
|
||||||
obj, err := loadJSONObject(path)
|
|
||||||
if err != nil {
|
|
||||||
return false, 0, err
|
|
||||||
}
|
|
||||||
updated, count, err := inlineLegacyTLKValue(obj, legacy)
|
|
||||||
if err != nil {
|
|
||||||
return false, 0, err
|
|
||||||
}
|
|
||||||
if !updated {
|
|
||||||
return false, 0, nil
|
|
||||||
}
|
|
||||||
raw, err := json.MarshalIndent(obj, "", " ")
|
|
||||||
if err != nil {
|
|
||||||
return false, 0, err
|
|
||||||
}
|
|
||||||
raw = append(raw, '\n')
|
|
||||||
if err := os.WriteFile(path, raw, 0o644); err != nil {
|
|
||||||
return false, 0, err
|
|
||||||
}
|
|
||||||
return true, count, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func inlineLegacyTLKValue(value any, legacy *legacyTLKData) (bool, int, error) {
|
|
||||||
switch typed := value.(type) {
|
|
||||||
case map[string]any:
|
|
||||||
if rawTLK, ok := typed["tlk"]; ok {
|
|
||||||
key, ok := rawTLK.(string)
|
|
||||||
if ok {
|
|
||||||
entry, ok := legacy.Entries[key]
|
|
||||||
if !ok {
|
|
||||||
return false, 0, nil
|
|
||||||
}
|
|
||||||
payload := map[string]any{
|
|
||||||
"key": key,
|
|
||||||
"text": entry.Text,
|
|
||||||
}
|
|
||||||
if entry.SoundResRef != "" {
|
|
||||||
payload["sound_resref"] = entry.SoundResRef
|
|
||||||
}
|
|
||||||
if entry.SoundLength != 0 {
|
|
||||||
payload["sound_length"] = entry.SoundLength
|
|
||||||
}
|
|
||||||
typed["tlk"] = payload
|
|
||||||
return true, 1, nil
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
updated := false
|
|
||||||
total := 0
|
|
||||||
for key, child := range typed {
|
|
||||||
childUpdated, childCount, err := inlineLegacyTLKValue(child, legacy)
|
|
||||||
if err != nil {
|
|
||||||
return false, 0, err
|
|
||||||
}
|
|
||||||
if childUpdated {
|
|
||||||
typed[key] = child
|
|
||||||
updated = true
|
|
||||||
total += childCount
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return updated, total, nil
|
|
||||||
case []any:
|
|
||||||
updated := false
|
|
||||||
total := 0
|
|
||||||
for index, child := range typed {
|
|
||||||
childUpdated, childCount, err := inlineLegacyTLKValue(child, legacy)
|
|
||||||
if err != nil {
|
|
||||||
return false, 0, err
|
|
||||||
}
|
|
||||||
if childUpdated {
|
|
||||||
typed[index] = child
|
|
||||||
updated = true
|
|
||||||
total += childCount
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return updated, total, nil
|
|
||||||
default:
|
|
||||||
return false, 0, nil
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func collectDataJSONPaths(dataDir string) ([]string, error) {
|
|
||||||
paths := make([]string, 0)
|
|
||||||
err := filepath.WalkDir(dataDir, func(path string, d os.DirEntry, err error) error {
|
|
||||||
if err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
if d.IsDir() {
|
|
||||||
return nil
|
|
||||||
}
|
|
||||||
if strings.HasSuffix(strings.ToLower(d.Name()), ".json") && d.Name() != "lock.json" {
|
|
||||||
paths = append(paths, path)
|
|
||||||
}
|
|
||||||
return nil
|
|
||||||
})
|
|
||||||
if err != nil {
|
|
||||||
return nil, err
|
|
||||||
}
|
|
||||||
return paths, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func countLegacyTLKRefs(paths []string) (int, int, error) {
|
|
||||||
files := 0
|
|
||||||
refs := 0
|
|
||||||
for _, path := range paths {
|
|
||||||
obj, err := loadJSONObject(path)
|
|
||||||
if err != nil {
|
|
||||||
return 0, 0, err
|
|
||||||
}
|
|
||||||
count := countLegacyTLKRefsInValue(obj)
|
|
||||||
if count == 0 {
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
files++
|
|
||||||
refs += count
|
|
||||||
}
|
|
||||||
return files, refs, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func countLegacyTLKRefsInValue(value any) int {
|
|
||||||
switch typed := value.(type) {
|
|
||||||
case map[string]any:
|
|
||||||
total := 0
|
|
||||||
if rawTLK, ok := typed["tlk"]; ok {
|
|
||||||
if _, ok := rawTLK.(string); ok {
|
|
||||||
total++
|
|
||||||
}
|
|
||||||
}
|
|
||||||
for _, child := range typed {
|
|
||||||
total += countLegacyTLKRefsInValue(child)
|
|
||||||
}
|
|
||||||
return total
|
|
||||||
case []any:
|
|
||||||
total := 0
|
|
||||||
for _, child := range typed {
|
|
||||||
total += countLegacyTLKRefsInValue(child)
|
|
||||||
}
|
|
||||||
return total
|
|
||||||
default:
|
|
||||||
return 0
|
|
||||||
}
|
|
||||||
}
|
|
||||||
+49
-78
@@ -410,6 +410,9 @@ func buildNativeUnchecked(p *project.Project, opts NativeBuildOptions, progress
|
|||||||
if err != nil {
|
if err != nil {
|
||||||
return BuildResult{}, err
|
return BuildResult{}, err
|
||||||
}
|
}
|
||||||
|
if err := loadStandaloneTLKStrings(sourceDir, compiler); err != nil {
|
||||||
|
return BuildResult{}, err
|
||||||
|
}
|
||||||
|
|
||||||
output2DA := compiled2DAOutputDir(p)
|
output2DA := compiled2DAOutputDir(p)
|
||||||
outputTLK := compiledTLKOutputDir(p)
|
outputTLK := compiledTLKOutputDir(p)
|
||||||
@@ -2317,6 +2320,18 @@ func (c *featGeneratedContext) familyHasExistingRows(familyKey string) bool {
|
|||||||
return false
|
return false
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// skillsDatasetName returns the dataset the skill_focus family sources its
|
||||||
|
// children from. The skills dataset location is data-driven via that spec;
|
||||||
|
// "skills" is only the fallback when no spec is loaded.
|
||||||
|
func (c *featGeneratedContext) skillsDatasetName() string {
|
||||||
|
for key, spec := range c.familySpecs {
|
||||||
|
if normalizeKeyIdentity(key) == "skillfocus" && spec.ChildSource.Dataset != "" {
|
||||||
|
return spec.ChildSource.Dataset
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return "skills"
|
||||||
|
}
|
||||||
|
|
||||||
func (c *featGeneratedContext) datasetRows(name string) (map[string]map[string]any, error) {
|
func (c *featGeneratedContext) datasetRows(name string) (map[string]map[string]any, error) {
|
||||||
if rows, ok := c.rowsByDataset[name]; ok {
|
if rows, ok := c.rowsByDataset[name]; ok {
|
||||||
return rows, nil
|
return rows, nil
|
||||||
@@ -2555,7 +2570,10 @@ func buildFamilyExpansionGeneratedModule(path string, obj map[string]any, ctx *f
|
|||||||
if !include {
|
if !include {
|
||||||
continue
|
continue
|
||||||
}
|
}
|
||||||
slug := strings.TrimPrefix(sourceKey, spec.ChildSource.Dataset+":")
|
slug := sourceKey
|
||||||
|
if idx := strings.Index(slug, ":"); idx >= 0 {
|
||||||
|
slug = slug[idx+1:]
|
||||||
|
}
|
||||||
featKey, rowID, hasID, err := ctx.resolveGeneratedFeatIdentity(spec, slug, row)
|
featKey, rowID, hasID, err := ctx.resolveGeneratedFeatIdentity(spec, slug, row)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, fmt.Errorf("generated feat file %s: %w", path, err)
|
return nil, fmt.Errorf("generated feat file %s: %w", path, err)
|
||||||
@@ -2655,7 +2673,7 @@ func buildRacialtypesSkillAffinityModule(ctx *featGeneratedContext) (map[string]
|
|||||||
if len(grants) == 0 {
|
if len(grants) == 0 {
|
||||||
return nil, nil
|
return nil, nil
|
||||||
}
|
}
|
||||||
skillRows, err := ctx.datasetRows("skills")
|
skillRows, err := ctx.datasetRows(ctx.skillsDatasetName())
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
@@ -2854,7 +2872,7 @@ func (c *featGeneratedContext) displayNameForGeneratedSource(dataset string, row
|
|||||||
return text
|
return text
|
||||||
}
|
}
|
||||||
switch dataset {
|
switch dataset {
|
||||||
case "skills":
|
case c.skillsDatasetName():
|
||||||
return displayNameForSkill(row)
|
return displayNameForSkill(row)
|
||||||
case "baseitems":
|
case "baseitems":
|
||||||
return displayNameForBaseitem(row)
|
return displayNameForBaseitem(row)
|
||||||
@@ -3457,10 +3475,20 @@ func legacyFamilyChildAliases(familyKey, sourceSlug string) []string {
|
|||||||
aliases = append(aliases, "craftweapon")
|
aliases = append(aliases, "craftweapon")
|
||||||
case "craftwoodworking":
|
case "craftwoodworking":
|
||||||
aliases = append(aliases, "crafttrap")
|
aliases = append(aliases, "crafttrap")
|
||||||
|
case "diplomacy":
|
||||||
|
aliases = append(aliases, "influence", "persuade")
|
||||||
case "disabledevice":
|
case "disabledevice":
|
||||||
aliases = append(aliases, "disabletrap")
|
aliases = append(aliases, "disabletrap")
|
||||||
case "influence":
|
case "influence":
|
||||||
aliases = append(aliases, "persuade")
|
aliases = append(aliases, "persuade")
|
||||||
|
case "intimidate":
|
||||||
|
aliases = append(aliases, "taunt")
|
||||||
|
case "investigation":
|
||||||
|
aliases = append(aliases, "search")
|
||||||
|
case "medicine":
|
||||||
|
aliases = append(aliases, "heal")
|
||||||
|
case "security":
|
||||||
|
aliases = append(aliases, "openlock")
|
||||||
case "sleightofhand":
|
case "sleightofhand":
|
||||||
aliases = append(aliases, "pickpocket")
|
aliases = append(aliases, "pickpocket")
|
||||||
}
|
}
|
||||||
@@ -4712,6 +4740,24 @@ func (r *valueResolver) resolveValue(row map[string]any, field string, value any
|
|||||||
|
|
||||||
switch typed := value.(type) {
|
switch typed := value.(type) {
|
||||||
case map[string]any:
|
case map[string]any:
|
||||||
|
if radial, ok := typed["spellradial"].(map[string]any); ok && len(typed) == 1 {
|
||||||
|
if r.dataset.Name != "spells" || field != "FeatID" {
|
||||||
|
return tlkCompiledValue{}, fmt.Errorf("spellradial reference is only valid for spells.FeatID")
|
||||||
|
}
|
||||||
|
radialKey, ok := radial["id"].(string)
|
||||||
|
if !ok || len(radial) != 1 {
|
||||||
|
return tlkCompiledValue{}, fmt.Errorf("spellradial reference requires exactly one id")
|
||||||
|
}
|
||||||
|
featID, ok := r.keyToID[radialKey]
|
||||||
|
if !ok {
|
||||||
|
return tlkCompiledValue{}, fmt.Errorf("unknown spellradial feat reference: %s", radialKey)
|
||||||
|
}
|
||||||
|
spellID, ok := row["id"].(int)
|
||||||
|
if !ok {
|
||||||
|
return tlkCompiledValue{}, fmt.Errorf("spellradial reference requires a numeric spell row id")
|
||||||
|
}
|
||||||
|
return tlkCompiledValue{Value: spellID<<16 + featID}, nil
|
||||||
|
}
|
||||||
if encoding, ok := r.valueEncodingForField(field); ok {
|
if encoding, ok := r.valueEncodingForField(field); ok {
|
||||||
mode := strings.TrimSpace(encoding.Mode)
|
mode := strings.TrimSpace(encoding.Mode)
|
||||||
_, hasList := typed["list"]
|
_, hasList := typed["list"]
|
||||||
@@ -6599,81 +6645,6 @@ func marshalOrderedLockfile(keys []string, lockData map[string]int, formatting j
|
|||||||
return buffer.Bytes(), nil
|
return buffer.Bytes(), nil
|
||||||
}
|
}
|
||||||
|
|
||||||
func loadLegacyTLK(root string) (*legacyTLKData, error) {
|
|
||||||
info, err := os.Stat(root)
|
|
||||||
if err != nil {
|
|
||||||
if os.IsNotExist(err) {
|
|
||||||
return nil, nil
|
|
||||||
}
|
|
||||||
return nil, err
|
|
||||||
}
|
|
||||||
if !info.IsDir() {
|
|
||||||
return nil, fmt.Errorf("legacy tlk root must be a directory: %s", root)
|
|
||||||
}
|
|
||||||
|
|
||||||
result := &legacyTLKData{
|
|
||||||
Language: "en",
|
|
||||||
Entries: map[string]tlkEntryData{},
|
|
||||||
Lock: map[string]int{},
|
|
||||||
}
|
|
||||||
|
|
||||||
lockPath := filepath.Join(root, "lock.json")
|
|
||||||
lockData, err := loadLockfile(lockPath)
|
|
||||||
if err != nil && !os.IsNotExist(err) {
|
|
||||||
return nil, err
|
|
||||||
}
|
|
||||||
for key, id := range lockData {
|
|
||||||
result.Lock[key] = id
|
|
||||||
}
|
|
||||||
|
|
||||||
basePath := filepath.Join(root, "base.json")
|
|
||||||
if _, err := os.Stat(basePath); err == nil {
|
|
||||||
base, err := loadJSONObject(basePath)
|
|
||||||
if err != nil {
|
|
||||||
return nil, err
|
|
||||||
}
|
|
||||||
if language, ok := base["language"].(string); ok && language != "" {
|
|
||||||
result.Language = language
|
|
||||||
}
|
|
||||||
if entries, ok := base["entries"].(map[string]any); ok {
|
|
||||||
normalized, err := normalizeLegacyTLKEntries(entries)
|
|
||||||
if err != nil {
|
|
||||||
return nil, err
|
|
||||||
}
|
|
||||||
for key, entry := range normalized {
|
|
||||||
result.Entries[key] = entry
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
modulesDir := filepath.Join(root, "modules")
|
|
||||||
modulePaths, err := collectModulePaths(modulesDir)
|
|
||||||
if err != nil {
|
|
||||||
if os.IsNotExist(err) {
|
|
||||||
return result, nil
|
|
||||||
}
|
|
||||||
return nil, err
|
|
||||||
}
|
|
||||||
for _, path := range modulePaths {
|
|
||||||
obj, err := loadJSONObject(path)
|
|
||||||
if err != nil {
|
|
||||||
return nil, err
|
|
||||||
}
|
|
||||||
entries, ok := obj["entries"].(map[string]any)
|
|
||||||
if !ok {
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
normalized, err := normalizeLegacyTLKEntries(entries)
|
|
||||||
if err != nil {
|
|
||||||
return nil, fmt.Errorf("%s: %w", path, err)
|
|
||||||
}
|
|
||||||
for key, entry := range normalized {
|
|
||||||
result.Entries[key] = entry
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return result, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func normalizeLegacyTLKEntries(entries map[string]any) (map[string]tlkEntryData, error) {
|
func normalizeLegacyTLKEntries(entries map[string]any) (map[string]tlkEntryData, error) {
|
||||||
normalized := map[string]tlkEntryData{}
|
normalized := map[string]tlkEntryData{}
|
||||||
for _, key := range sortedKeys(entries) {
|
for _, key := range sortedKeys(entries) {
|
||||||
|
|||||||
@@ -1,6 +0,0 @@
|
|||||||
package topdata
|
|
||||||
|
|
||||||
func importLegacyPlaceables(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
|
|
||||||
_ = legacyTLK
|
|
||||||
return importLegacyDatasetMirror(referenceBuilderDir, dataDir, "placeables", "placeables.2da", nil)
|
|
||||||
}
|
|
||||||
@@ -1,8 +0,0 @@
|
|||||||
package topdata
|
|
||||||
|
|
||||||
func importLegacyPortraits(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
|
|
||||||
if legacyTLK == nil {
|
|
||||||
return 0, nil
|
|
||||||
}
|
|
||||||
return importLegacyDatasetMirror(referenceBuilderDir, dataDir, "portraits", "portraits.2da", nil)
|
|
||||||
}
|
|
||||||
@@ -1,6 +0,0 @@
|
|||||||
package topdata
|
|
||||||
|
|
||||||
func importLegacyProgfx(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
|
|
||||||
_ = legacyTLK
|
|
||||||
return importLegacyDatasetMirror(referenceBuilderDir, dataDir, "progfx", "progfx.2da", nil)
|
|
||||||
}
|
|
||||||
@@ -1,282 +0,0 @@
|
|||||||
package topdata
|
|
||||||
|
|
||||||
import (
|
|
||||||
"encoding/json"
|
|
||||||
"fmt"
|
|
||||||
"os"
|
|
||||||
"path/filepath"
|
|
||||||
"slices"
|
|
||||||
"strings"
|
|
||||||
)
|
|
||||||
|
|
||||||
type legacyRacialtypesFeatTable struct {
|
|
||||||
Output string
|
|
||||||
Rows []map[string]any
|
|
||||||
}
|
|
||||||
|
|
||||||
func importLegacyRacialtypes(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
|
|
||||||
if strings.TrimSpace(referenceBuilderDir) == "" {
|
|
||||||
return 0, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
legacyCoreDir := filepath.Join(referenceBuilderDir, "data", "racialtypes", "core")
|
|
||||||
legacyFeatsDir := filepath.Join(referenceBuilderDir, "data", "racialtypes", "feats")
|
|
||||||
if _, err := os.Stat(legacyCoreDir); err != nil {
|
|
||||||
if os.IsNotExist(err) {
|
|
||||||
return 0, nil
|
|
||||||
}
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
if _, err := os.Stat(legacyFeatsDir); err != nil {
|
|
||||||
if os.IsNotExist(err) {
|
|
||||||
return 0, nil
|
|
||||||
}
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
|
|
||||||
targetRoot := filepath.Join(dataDir, "racialtypes", "registry")
|
|
||||||
targetBasePath := filepath.Join(targetRoot, "base.json")
|
|
||||||
targetLockPath := filepath.Join(targetRoot, "lock.json")
|
|
||||||
targetRacesDir := filepath.Join(targetRoot, "races")
|
|
||||||
if fileExists(targetBasePath) && fileExists(targetLockPath) {
|
|
||||||
entries, err := os.ReadDir(targetRacesDir)
|
|
||||||
if err == nil {
|
|
||||||
raceFiles := 0
|
|
||||||
for _, entry := range entries {
|
|
||||||
if !entry.IsDir() && strings.HasSuffix(strings.ToLower(entry.Name()), ".json") {
|
|
||||||
raceFiles++
|
|
||||||
}
|
|
||||||
}
|
|
||||||
if raceFiles == 16 {
|
|
||||||
return 0, nil
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
baseColumns, baseRows, lockData, raceRows, err := collectLegacyRacialtypesRegistry(legacyCoreDir, legacyFeatsDir, legacyTLK)
|
|
||||||
if err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
|
|
||||||
if err := removePathIfExists(filepath.Join(dataDir, "racialtypes", "core")); err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
if err := removePathIfExists(filepath.Join(dataDir, "racialtypes", "feats")); err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
if err := removePathIfExists(filepath.Join(targetRoot, "base_rows.json")); err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
if err := removePathIfExists(targetRacesDir); err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
if err := os.MkdirAll(targetRacesDir, 0o755); err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
|
|
||||||
writes := []struct {
|
|
||||||
path string
|
|
||||||
obj any
|
|
||||||
}{
|
|
||||||
{
|
|
||||||
path: targetBasePath,
|
|
||||||
obj: map[string]any{
|
|
||||||
"columns": stringSliceToAny(baseColumns),
|
|
||||||
"rows": rowsToAny(baseRows),
|
|
||||||
},
|
|
||||||
},
|
|
||||||
{
|
|
||||||
path: targetLockPath,
|
|
||||||
obj: anyMapInt(lockData),
|
|
||||||
},
|
|
||||||
}
|
|
||||||
|
|
||||||
raceKeys := make([]string, 0, len(raceRows))
|
|
||||||
for key := range raceRows {
|
|
||||||
raceKeys = append(raceKeys, key)
|
|
||||||
}
|
|
||||||
slices.Sort(raceKeys)
|
|
||||||
for _, key := range raceKeys {
|
|
||||||
fileName := strings.TrimPrefix(key, "racialtypes:") + ".json"
|
|
||||||
writes = append(writes, struct {
|
|
||||||
path string
|
|
||||||
obj any
|
|
||||||
}{
|
|
||||||
path: filepath.Join(targetRacesDir, fileName),
|
|
||||||
obj: raceRows[key],
|
|
||||||
})
|
|
||||||
}
|
|
||||||
|
|
||||||
for _, write := range writes {
|
|
||||||
raw, err := json.MarshalIndent(write.obj, "", " ")
|
|
||||||
if err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
raw = append(raw, '\n')
|
|
||||||
if err := os.WriteFile(write.path, raw, 0o644); err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return len(writes), nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func collectLegacyRacialtypesRegistry(coreDir, featsDir string, legacyTLK *legacyTLKData) ([]string, []map[string]any, map[string]int, map[string]map[string]any, error) {
|
|
||||||
coreCollected, err := collectBaseDataset(nativeDataset{
|
|
||||||
Name: "racialtypes",
|
|
||||||
BasePath: filepath.Join(coreDir, "base.json"),
|
|
||||||
LockPath: filepath.Join(coreDir, "lock.json"),
|
|
||||||
ModulesDir: filepath.Join(coreDir, "modules"),
|
|
||||||
Spec: specForDataset("racialtypes"),
|
|
||||||
})
|
|
||||||
if err != nil {
|
|
||||||
return nil, nil, nil, nil, err
|
|
||||||
}
|
|
||||||
|
|
||||||
featTables, err := loadLegacyRacialtypesFeatTables(featsDir)
|
|
||||||
if err != nil {
|
|
||||||
return nil, nil, nil, nil, err
|
|
||||||
}
|
|
||||||
|
|
||||||
baseRows := make([]map[string]any, 0)
|
|
||||||
raceRows := map[string]map[string]any{}
|
|
||||||
lockData := map[string]int{}
|
|
||||||
for key, id := range coreCollected.LockData {
|
|
||||||
if strings.HasPrefix(key, "racialtypes:") {
|
|
||||||
lockData[key] = id
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
for _, collectedRow := range coreCollected.Rows {
|
|
||||||
row, ok := deepCopyValue(collectedRow).(map[string]any)
|
|
||||||
if !ok {
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
if legacyTLK != nil {
|
|
||||||
if _, _, err := inlineLegacyTLKValue(row, legacyTLK); err != nil {
|
|
||||||
return nil, nil, nil, nil, err
|
|
||||||
}
|
|
||||||
}
|
|
||||||
key, _ := row["key"].(string)
|
|
||||||
if !strings.HasPrefix(key, "racialtypes:") {
|
|
||||||
assignRacialtypesBaseRowKey(row)
|
|
||||||
if key, _ := row["key"].(string); key != "" {
|
|
||||||
rowID, err := asInt(row["id"])
|
|
||||||
if err != nil {
|
|
||||||
return nil, nil, nil, nil, err
|
|
||||||
}
|
|
||||||
lockData[key] = rowID
|
|
||||||
}
|
|
||||||
baseRows = append(baseRows, row)
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
|
|
||||||
delete(row, "id")
|
|
||||||
delete(row, "key")
|
|
||||||
|
|
||||||
raceFile := map[string]any{
|
|
||||||
"key": key,
|
|
||||||
"core": row,
|
|
||||||
}
|
|
||||||
if tableKey := extractRacialtypesFeatTableKey(row["FeatsTable"]); tableKey != "" {
|
|
||||||
table, ok := featTables[tableKey]
|
|
||||||
if !ok {
|
|
||||||
return nil, nil, nil, nil, fmt.Errorf("%s: unknown feat table %s", key, tableKey)
|
|
||||||
}
|
|
||||||
delete(row, "FeatsTable")
|
|
||||||
raceFile["feat_output"] = table.Output
|
|
||||||
raceFile["feats"] = rowsToAny(table.Rows)
|
|
||||||
}
|
|
||||||
raceRows[key] = raceFile
|
|
||||||
}
|
|
||||||
|
|
||||||
slices.SortFunc(baseRows, func(a, b map[string]any) int {
|
|
||||||
left, _ := asInt(a["id"])
|
|
||||||
right, _ := asInt(b["id"])
|
|
||||||
return left - right
|
|
||||||
})
|
|
||||||
return coreCollected.Columns, baseRows, lockData, raceRows, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func loadLegacyRacialtypesFeatTables(featsDir string) (map[string]legacyRacialtypesFeatTable, error) {
|
|
||||||
entries, err := os.ReadDir(featsDir)
|
|
||||||
if err != nil {
|
|
||||||
return nil, err
|
|
||||||
}
|
|
||||||
|
|
||||||
tables := map[string]legacyRacialtypesFeatTable{}
|
|
||||||
for _, entry := range entries {
|
|
||||||
if entry.IsDir() || filepath.Ext(entry.Name()) != ".json" {
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
obj, err := loadJSONObject(filepath.Join(featsDir, entry.Name()))
|
|
||||||
if err != nil {
|
|
||||||
return nil, err
|
|
||||||
}
|
|
||||||
tableKey, _ := obj["key"].(string)
|
|
||||||
outputName, _ := obj["output"].(string)
|
|
||||||
rawRows, ok := obj["rows"].([]any)
|
|
||||||
if tableKey == "" || outputName == "" || !ok {
|
|
||||||
return nil, fmt.Errorf("%s: racialtypes feat table must define key, output, and rows", entry.Name())
|
|
||||||
}
|
|
||||||
rows := make([]map[string]any, 0, len(rawRows))
|
|
||||||
for index, raw := range rawRows {
|
|
||||||
row, ok := raw.(map[string]any)
|
|
||||||
if !ok {
|
|
||||||
return nil, fmt.Errorf("%s: row %d must be an object", entry.Name(), index)
|
|
||||||
}
|
|
||||||
rows = append(rows, row)
|
|
||||||
}
|
|
||||||
tables[tableKey] = legacyRacialtypesFeatTable{
|
|
||||||
Output: outputName,
|
|
||||||
Rows: rows,
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return tables, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func extractRacialtypesFeatTableKey(value any) string {
|
|
||||||
obj, ok := value.(map[string]any)
|
|
||||||
if !ok {
|
|
||||||
return ""
|
|
||||||
}
|
|
||||||
tableKey, _ := obj["table"].(string)
|
|
||||||
return tableKey
|
|
||||||
}
|
|
||||||
|
|
||||||
func assignRacialtypesBaseRowKey(row map[string]any) {
|
|
||||||
name, _ := row["Name"].(string)
|
|
||||||
if strings.TrimSpace(name) == "" || name == nullValue {
|
|
||||||
delete(row, "key")
|
|
||||||
return
|
|
||||||
}
|
|
||||||
label, _ := row["Label"].(string)
|
|
||||||
keySuffix := normalizeRacialtypesKeySuffix(label)
|
|
||||||
if keySuffix == "" {
|
|
||||||
delete(row, "key")
|
|
||||||
return
|
|
||||||
}
|
|
||||||
row["key"] = "racialtypes:" + keySuffix
|
|
||||||
}
|
|
||||||
|
|
||||||
func normalizeRacialtypesKeySuffix(label string) string {
|
|
||||||
normalized := strings.ToLower(strings.TrimSpace(label))
|
|
||||||
if normalized == "" {
|
|
||||||
return ""
|
|
||||||
}
|
|
||||||
var b strings.Builder
|
|
||||||
lastUnderscore := false
|
|
||||||
for _, ch := range normalized {
|
|
||||||
isAlphaNum := (ch >= 'a' && ch <= 'z') || (ch >= '0' && ch <= '9')
|
|
||||||
if isAlphaNum {
|
|
||||||
b.WriteRune(ch)
|
|
||||||
lastUnderscore = false
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
if lastUnderscore {
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
b.WriteByte('_')
|
|
||||||
lastUnderscore = true
|
|
||||||
}
|
|
||||||
return strings.Trim(b.String(), "_")
|
|
||||||
}
|
|
||||||
@@ -1,63 +0,0 @@
|
|||||||
package topdata
|
|
||||||
|
|
||||||
import (
|
|
||||||
"encoding/json"
|
|
||||||
"os"
|
|
||||||
"path/filepath"
|
|
||||||
)
|
|
||||||
|
|
||||||
func importLegacyRepadjust(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
|
|
||||||
_ = legacyTLK
|
|
||||||
|
|
||||||
legacyDir := filepath.Join(referenceBuilderDir, "data", "repadjust")
|
|
||||||
if _, err := os.Stat(legacyDir); err != nil {
|
|
||||||
if os.IsNotExist(err) {
|
|
||||||
return 0, nil
|
|
||||||
}
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
|
|
||||||
targetDir := filepath.Join(dataDir, "repadjust")
|
|
||||||
targetBasePath := filepath.Join(targetDir, "base.json")
|
|
||||||
targetLockPath := filepath.Join(targetDir, "lock.json")
|
|
||||||
if fileExists(targetBasePath) && fileExists(targetLockPath) {
|
|
||||||
obj, err := loadJSONObject(targetBasePath)
|
|
||||||
if err == nil && countLegacyTLKRefsInValue(obj) == 0 {
|
|
||||||
return 0, nil
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
baseObj, err := loadJSONObject(filepath.Join(legacyDir, "base.json"))
|
|
||||||
if err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
baseObj["output"] = "repadjust.2da"
|
|
||||||
|
|
||||||
lockObj, err := loadJSONObject(filepath.Join(legacyDir, "lock.json"))
|
|
||||||
if err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
|
|
||||||
if err := os.MkdirAll(targetDir, 0o755); err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
|
|
||||||
writes := []struct {
|
|
||||||
path string
|
|
||||||
obj map[string]any
|
|
||||||
}{
|
|
||||||
{path: targetBasePath, obj: baseObj},
|
|
||||||
{path: targetLockPath, obj: lockObj},
|
|
||||||
}
|
|
||||||
for _, write := range writes {
|
|
||||||
raw, err := json.MarshalIndent(write.obj, "", " ")
|
|
||||||
if err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
raw = append(raw, '\n')
|
|
||||||
if err := os.WriteFile(write.path, raw, 0o644); err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return len(writes), nil
|
|
||||||
}
|
|
||||||
@@ -1,171 +0,0 @@
|
|||||||
package topdata
|
|
||||||
|
|
||||||
import (
|
|
||||||
"encoding/json"
|
|
||||||
"fmt"
|
|
||||||
"os"
|
|
||||||
"path/filepath"
|
|
||||||
"sort"
|
|
||||||
)
|
|
||||||
|
|
||||||
func importLegacyRuleset(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
|
|
||||||
_ = legacyTLK
|
|
||||||
|
|
||||||
legacyDir := filepath.Join(referenceBuilderDir, "data", "ruleset")
|
|
||||||
if _, err := os.Stat(legacyDir); err != nil {
|
|
||||||
if os.IsNotExist(err) {
|
|
||||||
return 0, nil
|
|
||||||
}
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
|
|
||||||
targetDir := filepath.Join(dataDir, "ruleset")
|
|
||||||
targetPath := filepath.Join(targetDir, "ruleset.json")
|
|
||||||
if _, err := os.Stat(targetPath); err == nil {
|
|
||||||
obj, err := loadJSONObject(targetPath)
|
|
||||||
if err == nil && countLegacyTLKRefsInValue(obj) == 0 {
|
|
||||||
return 0, nil
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
baseObj, err := loadJSONObject(filepath.Join(legacyDir, "base.json"))
|
|
||||||
if err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
rows, err := mergeRulesetRows(baseObj, filepath.Join(legacyDir, "modules"))
|
|
||||||
if err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
|
|
||||||
if err := os.MkdirAll(targetDir, 0o755); err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
if err := removePathIfExists(filepath.Join(targetDir, "base.json")); err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
if err := removePathIfExists(filepath.Join(targetDir, "lock.json")); err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
if err := removePathIfExists(filepath.Join(targetDir, "modules")); err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
|
|
||||||
out := map[string]any{
|
|
||||||
"output": "ruleset.2da",
|
|
||||||
"columns": baseObj["columns"],
|
|
||||||
"rows": rows,
|
|
||||||
}
|
|
||||||
raw, err := json.MarshalIndent(out, "", " ")
|
|
||||||
if err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
raw = append(raw, '\n')
|
|
||||||
if err := os.WriteFile(targetPath, raw, 0o644); err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
return 1, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func mergeRulesetRows(baseObj map[string]any, modulesDir string) ([]any, error) {
|
|
||||||
rawRows, ok := baseObj["rows"].([]any)
|
|
||||||
if !ok {
|
|
||||||
return nil, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
rows := make([]map[string]any, 0, len(rawRows))
|
|
||||||
byLabel := map[string]map[string]any{}
|
|
||||||
for _, raw := range rawRows {
|
|
||||||
row, ok := deepCopyValue(raw).(map[string]any)
|
|
||||||
if !ok {
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
delete(row, "key")
|
|
||||||
if rawID, ok := row["id"]; ok {
|
|
||||||
id, err := asInt(rawID)
|
|
||||||
if err != nil {
|
|
||||||
return nil, err
|
|
||||||
}
|
|
||||||
row["id"] = id
|
|
||||||
}
|
|
||||||
if label, ok := row["Label"].(string); ok && label != "" && label != "****" {
|
|
||||||
if _, exists := byLabel[label]; exists {
|
|
||||||
return nil, fmt.Errorf("duplicate ruleset label %q", label)
|
|
||||||
}
|
|
||||||
byLabel[label] = row
|
|
||||||
}
|
|
||||||
rows = append(rows, row)
|
|
||||||
}
|
|
||||||
|
|
||||||
modulePaths, err := collectModulePaths(modulesDir)
|
|
||||||
if err != nil {
|
|
||||||
return nil, err
|
|
||||||
}
|
|
||||||
for _, path := range modulePaths {
|
|
||||||
obj, err := loadJSONObject(path)
|
|
||||||
if err != nil {
|
|
||||||
return nil, err
|
|
||||||
}
|
|
||||||
overrideList, ok := obj["overrides"].([]any)
|
|
||||||
if !ok {
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
for _, raw := range overrideList {
|
|
||||||
override, ok := raw.(map[string]any)
|
|
||||||
if !ok {
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
matchObj, ok := override["match"].(map[string]any)
|
|
||||||
if !ok {
|
|
||||||
return nil, fmt.Errorf("%s: ruleset override is missing match object", path)
|
|
||||||
}
|
|
||||||
rawLabel, ok := matchObj["Label"]
|
|
||||||
if !ok {
|
|
||||||
return nil, fmt.Errorf("%s: ruleset override match is missing Label", path)
|
|
||||||
}
|
|
||||||
labels, err := normalizeRulesetMatchLabels(rawLabel)
|
|
||||||
if err != nil {
|
|
||||||
return nil, fmt.Errorf("%s: %w", path, err)
|
|
||||||
}
|
|
||||||
for _, label := range labels {
|
|
||||||
row, ok := byLabel[label]
|
|
||||||
if !ok {
|
|
||||||
return nil, fmt.Errorf("%s: ruleset override target label %q was not found", path, label)
|
|
||||||
}
|
|
||||||
for key, value := range override {
|
|
||||||
if key == "id" || key == "key" || key == "_tlk" || isMetadataField(key) || key == "match" {
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
row[key] = deepCopyValue(value)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
sort.Slice(rows, func(i, j int) bool {
|
|
||||||
return rows[i]["id"].(int) < rows[j]["id"].(int)
|
|
||||||
})
|
|
||||||
out := make([]any, 0, len(rows))
|
|
||||||
for _, row := range rows {
|
|
||||||
out = append(out, row)
|
|
||||||
}
|
|
||||||
return out, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func normalizeRulesetMatchLabels(raw any) ([]string, error) {
|
|
||||||
switch typed := raw.(type) {
|
|
||||||
case string:
|
|
||||||
return []string{typed}, nil
|
|
||||||
case []any:
|
|
||||||
out := make([]string, 0, len(typed))
|
|
||||||
for _, item := range typed {
|
|
||||||
label, ok := item.(string)
|
|
||||||
if !ok {
|
|
||||||
return nil, fmt.Errorf("ruleset match.Label entries must be strings")
|
|
||||||
}
|
|
||||||
out = append(out, label)
|
|
||||||
}
|
|
||||||
return out, nil
|
|
||||||
default:
|
|
||||||
return nil, fmt.Errorf("ruleset match.Label must be a string or string array")
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -1,140 +0,0 @@
|
|||||||
package topdata
|
|
||||||
|
|
||||||
import (
|
|
||||||
"encoding/json"
|
|
||||||
"os"
|
|
||||||
"path/filepath"
|
|
||||||
)
|
|
||||||
|
|
||||||
func importLegacySkills(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
|
|
||||||
_ = legacyTLK
|
|
||||||
|
|
||||||
legacyDir := filepath.Join(referenceBuilderDir, "data", "skills")
|
|
||||||
if _, err := os.Stat(legacyDir); err != nil {
|
|
||||||
if os.IsNotExist(err) {
|
|
||||||
return 0, nil
|
|
||||||
}
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
|
|
||||||
targetDir := filepath.Join(dataDir, "skills")
|
|
||||||
targetBasePath := filepath.Join(targetDir, "base.json")
|
|
||||||
targetLockPath := filepath.Join(targetDir, "lock.json")
|
|
||||||
targetModulesDir := filepath.Join(targetDir, "modules")
|
|
||||||
moduleNames := []string{
|
|
||||||
"add_athletics.json",
|
|
||||||
"add_disguise.json",
|
|
||||||
"add_linguistics.json",
|
|
||||||
"add_perception.json",
|
|
||||||
"add_scriptcraft.json",
|
|
||||||
"add_sensemotive.json",
|
|
||||||
"add_stealth.json",
|
|
||||||
"add_survival.json",
|
|
||||||
"add_userope.json",
|
|
||||||
filepath.Join("craft", "add_craftalchemy.json"),
|
|
||||||
filepath.Join("craft", "add_craftcooking.json"),
|
|
||||||
filepath.Join("craft", "add_craftjewelry.json"),
|
|
||||||
filepath.Join("craft", "add_craftleatherworking.json"),
|
|
||||||
filepath.Join("craft", "add_craftstonework.json"),
|
|
||||||
filepath.Join("craft", "add_crafttextiles.json"),
|
|
||||||
filepath.Join("craft", "ovr_craftarmorsmithing.json"),
|
|
||||||
filepath.Join("craft", "ovr_crafttinkering.json"),
|
|
||||||
filepath.Join("craft", "ovr_craftweaponsmithing.json"),
|
|
||||||
filepath.Join("craft", "ovr_craftwoodworking.json"),
|
|
||||||
filepath.Join("knowledge", "add_knowledgearcana.json"),
|
|
||||||
filepath.Join("knowledge", "add_knowledgearchitecture.json"),
|
|
||||||
filepath.Join("knowledge", "add_knowledgedungeoneering.json"),
|
|
||||||
filepath.Join("knowledge", "add_knowledgegeography.json"),
|
|
||||||
filepath.Join("knowledge", "add_knowledgehistory.json"),
|
|
||||||
filepath.Join("knowledge", "add_knowledgelocal.json"),
|
|
||||||
filepath.Join("knowledge", "add_knowledgenature.json"),
|
|
||||||
filepath.Join("knowledge", "add_knowledgenobility.json"),
|
|
||||||
filepath.Join("knowledge", "add_knowledgeplanar.json"),
|
|
||||||
filepath.Join("knowledge", "add_knowledgereligion.json"),
|
|
||||||
"ovr_acrobatics.json",
|
|
||||||
"ovr_animalhandling.json",
|
|
||||||
"ovr_appraise.json",
|
|
||||||
"ovr_concentration.json",
|
|
||||||
"ovr_disabledevice.json",
|
|
||||||
"ovr_heal.json",
|
|
||||||
"ovr_hiddenskills.json",
|
|
||||||
"ovr_influence.json",
|
|
||||||
"ovr_openlock.json",
|
|
||||||
"ovr_parry.json",
|
|
||||||
"ovr_searchtowis.json",
|
|
||||||
"ovr_sleightofhand.json",
|
|
||||||
"ovr_taunttointimidate.json",
|
|
||||||
"rmv_removedskills.json",
|
|
||||||
}
|
|
||||||
targetModulePaths := make([]string, 0, len(moduleNames))
|
|
||||||
for _, name := range moduleNames {
|
|
||||||
targetModulePaths = append(targetModulePaths, filepath.Join(targetModulesDir, name))
|
|
||||||
}
|
|
||||||
if fileExists(targetBasePath) && fileExists(targetLockPath) {
|
|
||||||
allModulesPresent := true
|
|
||||||
for _, path := range targetModulePaths {
|
|
||||||
if !fileExists(path) {
|
|
||||||
allModulesPresent = false
|
|
||||||
break
|
|
||||||
}
|
|
||||||
}
|
|
||||||
if allModulesPresent {
|
|
||||||
return 0, nil
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
baseObj, err := loadJSONObject(filepath.Join(legacyDir, "base.json"))
|
|
||||||
if err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
baseObj["output"] = "skills.2da"
|
|
||||||
canonicalizeWikiMetadataDocument(baseObj)
|
|
||||||
|
|
||||||
lockObj, err := loadJSONObject(filepath.Join(legacyDir, "lock.json"))
|
|
||||||
if err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
|
|
||||||
moduleObjs := make([]map[string]any, 0, len(moduleNames))
|
|
||||||
for _, name := range moduleNames {
|
|
||||||
obj, err := loadJSONObject(filepath.Join(legacyDir, "modules", name))
|
|
||||||
if err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
canonicalizeWikiMetadataDocument(obj)
|
|
||||||
moduleObjs = append(moduleObjs, obj)
|
|
||||||
}
|
|
||||||
|
|
||||||
if err := os.MkdirAll(targetModulesDir, 0o755); err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
|
|
||||||
writes := []struct {
|
|
||||||
path string
|
|
||||||
obj map[string]any
|
|
||||||
}{
|
|
||||||
{path: targetBasePath, obj: baseObj},
|
|
||||||
{path: targetLockPath, obj: lockObj},
|
|
||||||
}
|
|
||||||
for i, path := range targetModulePaths {
|
|
||||||
if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
writes = append(writes, struct {
|
|
||||||
path string
|
|
||||||
obj map[string]any
|
|
||||||
}{path: path, obj: moduleObjs[i]})
|
|
||||||
}
|
|
||||||
|
|
||||||
for _, write := range writes {
|
|
||||||
raw, err := json.MarshalIndent(write.obj, "", " ")
|
|
||||||
if err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
raw = append(raw, '\n')
|
|
||||||
if err := os.WriteFile(write.path, raw, 0o644); err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return len(writes), nil
|
|
||||||
}
|
|
||||||
@@ -1,6 +0,0 @@
|
|||||||
package topdata
|
|
||||||
|
|
||||||
func importLegacySkyboxes(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
|
|
||||||
_ = legacyTLK
|
|
||||||
return importLegacyDatasetMirror(referenceBuilderDir, dataDir, "skyboxes", "skyboxes.2da", nil)
|
|
||||||
}
|
|
||||||
@@ -1,103 +0,0 @@
|
|||||||
package topdata
|
|
||||||
|
|
||||||
import (
|
|
||||||
"encoding/json"
|
|
||||||
"os"
|
|
||||||
"path/filepath"
|
|
||||||
)
|
|
||||||
|
|
||||||
func importLegacySpells(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
|
|
||||||
_ = legacyTLK
|
|
||||||
|
|
||||||
legacyDir := filepath.Join(referenceBuilderDir, "data", "spells")
|
|
||||||
if _, err := os.Stat(legacyDir); err != nil {
|
|
||||||
if os.IsNotExist(err) {
|
|
||||||
return 0, nil
|
|
||||||
}
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
|
|
||||||
targetDir := filepath.Join(dataDir, "spells")
|
|
||||||
targetBasePath := filepath.Join(targetDir, "base.json")
|
|
||||||
targetLockPath := filepath.Join(targetDir, "lock.json")
|
|
||||||
targetModulesDir := filepath.Join(targetDir, "modules")
|
|
||||||
moduleNames := []string{
|
|
||||||
"specialattacks.json",
|
|
||||||
"ovr_fixduplicates.json",
|
|
||||||
filepath.Join("bardicmusic", "fascinate.json"),
|
|
||||||
filepath.Join("bardicmusic", "inspirecompetence.json"),
|
|
||||||
filepath.Join("bardicmusic", "inspirecourage.json"),
|
|
||||||
filepath.Join("bardicmusic", "inspiregreatness.json"),
|
|
||||||
filepath.Join("bardicmusic", "inspireheroics.json"),
|
|
||||||
filepath.Join("bardicmusic", "songoffreedom.json"),
|
|
||||||
}
|
|
||||||
targetModulePaths := make([]string, 0, len(moduleNames))
|
|
||||||
for _, name := range moduleNames {
|
|
||||||
targetModulePaths = append(targetModulePaths, filepath.Join(targetModulesDir, name))
|
|
||||||
}
|
|
||||||
if fileExists(targetBasePath) && fileExists(targetLockPath) {
|
|
||||||
allModulesPresent := true
|
|
||||||
for _, path := range targetModulePaths {
|
|
||||||
if !fileExists(path) {
|
|
||||||
allModulesPresent = false
|
|
||||||
break
|
|
||||||
}
|
|
||||||
}
|
|
||||||
if allModulesPresent {
|
|
||||||
return 0, nil
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
baseObj, err := loadJSONObject(filepath.Join(legacyDir, "base.json"))
|
|
||||||
if err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
baseObj["output"] = "spells.2da"
|
|
||||||
|
|
||||||
lockObj, err := loadJSONObject(filepath.Join(legacyDir, "lock.json"))
|
|
||||||
if err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
|
|
||||||
moduleObjs := make([]map[string]any, 0, len(moduleNames))
|
|
||||||
for _, name := range moduleNames {
|
|
||||||
obj, err := loadJSONObject(filepath.Join(legacyDir, "modules", name))
|
|
||||||
if err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
moduleObjs = append(moduleObjs, obj)
|
|
||||||
}
|
|
||||||
|
|
||||||
if err := os.MkdirAll(targetModulesDir, 0o755); err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
|
|
||||||
writes := []struct {
|
|
||||||
path string
|
|
||||||
obj map[string]any
|
|
||||||
}{
|
|
||||||
{path: targetBasePath, obj: baseObj},
|
|
||||||
{path: targetLockPath, obj: lockObj},
|
|
||||||
}
|
|
||||||
for i, path := range targetModulePaths {
|
|
||||||
if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
writes = append(writes, struct {
|
|
||||||
path string
|
|
||||||
obj map[string]any
|
|
||||||
}{path: path, obj: moduleObjs[i]})
|
|
||||||
}
|
|
||||||
|
|
||||||
for _, write := range writes {
|
|
||||||
raw, err := json.MarshalIndent(write.obj, "", " ")
|
|
||||||
if err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
raw = append(raw, '\n')
|
|
||||||
if err := os.WriteFile(write.path, raw, 0o644); err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return len(writes), nil
|
|
||||||
}
|
|
||||||
@@ -1,86 +0,0 @@
|
|||||||
package topdata
|
|
||||||
|
|
||||||
import (
|
|
||||||
"encoding/json"
|
|
||||||
"os"
|
|
||||||
"path/filepath"
|
|
||||||
)
|
|
||||||
|
|
||||||
func importLegacyTailmodel(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
|
|
||||||
_ = legacyTLK
|
|
||||||
|
|
||||||
legacyDir := filepath.Join(referenceBuilderDir, "data", "tailmodel")
|
|
||||||
if _, err := os.Stat(legacyDir); err != nil {
|
|
||||||
if os.IsNotExist(err) {
|
|
||||||
return 0, nil
|
|
||||||
}
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
|
|
||||||
targetDir := filepath.Join(dataDir, "tailmodel")
|
|
||||||
targetBasePath := filepath.Join(targetDir, "base.json")
|
|
||||||
targetLockPath := filepath.Join(targetDir, "lock.json")
|
|
||||||
targetModulesDir := filepath.Join(targetDir, "modules")
|
|
||||||
targetModulePaths := []string{
|
|
||||||
filepath.Join(targetModulesDir, "add.json"),
|
|
||||||
filepath.Join(targetModulesDir, "ovr_tailprefixes.json"),
|
|
||||||
}
|
|
||||||
if fileExists(targetBasePath) && fileExists(targetLockPath) {
|
|
||||||
allModulesPresent := true
|
|
||||||
for _, path := range targetModulePaths {
|
|
||||||
if !fileExists(path) {
|
|
||||||
allModulesPresent = false
|
|
||||||
break
|
|
||||||
}
|
|
||||||
}
|
|
||||||
if allModulesPresent {
|
|
||||||
return 0, nil
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
baseObj, err := loadJSONObject(filepath.Join(legacyDir, "base.json"))
|
|
||||||
if err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
baseObj["output"] = "tailmodel.2da"
|
|
||||||
|
|
||||||
lockObj, err := loadJSONObject(filepath.Join(legacyDir, "lock.json"))
|
|
||||||
if err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
|
|
||||||
moduleNames := []string{"add.json", "ovr_tailprefixes.json"}
|
|
||||||
moduleObjs := make([]map[string]any, 0, len(moduleNames))
|
|
||||||
for _, name := range moduleNames {
|
|
||||||
obj, err := loadJSONObject(filepath.Join(legacyDir, "modules", name))
|
|
||||||
if err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
moduleObjs = append(moduleObjs, obj)
|
|
||||||
}
|
|
||||||
|
|
||||||
if err := os.MkdirAll(targetModulesDir, 0o755); err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
|
|
||||||
writes := []struct {
|
|
||||||
path string
|
|
||||||
obj map[string]any
|
|
||||||
}{
|
|
||||||
{path: targetBasePath, obj: baseObj},
|
|
||||||
{path: targetLockPath, obj: lockObj},
|
|
||||||
{path: targetModulePaths[0], obj: moduleObjs[0]},
|
|
||||||
{path: targetModulePaths[1], obj: moduleObjs[1]},
|
|
||||||
}
|
|
||||||
for _, write := range writes {
|
|
||||||
raw, err := json.MarshalIndent(write.obj, "", " ")
|
|
||||||
if err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
raw = append(raw, '\n')
|
|
||||||
if err := os.WriteFile(write.path, raw, 0o644); err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return len(writes), nil
|
|
||||||
}
|
|
||||||
@@ -13,6 +13,7 @@ import (
|
|||||||
"strings"
|
"strings"
|
||||||
|
|
||||||
"golang.org/x/text/encoding/charmap"
|
"golang.org/x/text/encoding/charmap"
|
||||||
|
"gopkg.in/yaml.v3"
|
||||||
)
|
)
|
||||||
|
|
||||||
const (
|
const (
|
||||||
@@ -74,6 +75,16 @@ type tlkEntryData struct {
|
|||||||
SoundLength float32
|
SoundLength float32
|
||||||
}
|
}
|
||||||
|
|
||||||
|
type standaloneTLKDocument struct {
|
||||||
|
Schema string `yaml:"schema"`
|
||||||
|
BaseStrref int `yaml:"base_strref"`
|
||||||
|
Strings []struct {
|
||||||
|
Key string `yaml:"key"`
|
||||||
|
Text string `yaml:"text"`
|
||||||
|
ID *int `yaml:"id"`
|
||||||
|
} `yaml:"strings"`
|
||||||
|
}
|
||||||
|
|
||||||
type tlkStateDocument struct {
|
type tlkStateDocument struct {
|
||||||
Version int `json:"version"`
|
Version int `json:"version"`
|
||||||
Language string `json:"language"`
|
Language string `json:"language"`
|
||||||
@@ -98,6 +109,7 @@ type tlkCompiler struct {
|
|||||||
active map[string]tlkEntryData
|
active map[string]tlkEntryData
|
||||||
activeKeys map[string]struct{}
|
activeKeys map[string]struct{}
|
||||||
reservedByID map[int]string
|
reservedByID map[int]string
|
||||||
|
pinnedByID map[int]string
|
||||||
nextID int
|
nextID int
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -147,6 +159,7 @@ func newTLKCompiler(sourceDir string, legacy *legacyTLKData) (*tlkCompiler, erro
|
|||||||
active: map[string]tlkEntryData{},
|
active: map[string]tlkEntryData{},
|
||||||
activeKeys: map[string]struct{}{},
|
activeKeys: map[string]struct{}{},
|
||||||
reservedByID: reserved,
|
reservedByID: reserved,
|
||||||
|
pinnedByID: map[int]string{},
|
||||||
nextID: nextID,
|
nextID: nextID,
|
||||||
}
|
}
|
||||||
if legacy != nil {
|
if legacy != nil {
|
||||||
@@ -157,6 +170,42 @@ func newTLKCompiler(sourceDir string, legacy *legacyTLKData) (*tlkCompiler, erro
|
|||||||
return compiler, nil
|
return compiler, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func loadStandaloneTLKStrings(sourceDir string, compiler *tlkCompiler) error {
|
||||||
|
path := filepath.Join(sourceDir, "tlk", "custom.tlk.yml")
|
||||||
|
raw, err := os.ReadFile(path)
|
||||||
|
if err != nil {
|
||||||
|
if os.IsNotExist(err) {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
return fmt.Errorf("read %s: %w", path, err)
|
||||||
|
}
|
||||||
|
var document standaloneTLKDocument
|
||||||
|
if err := yaml.Unmarshal(raw, &document); err != nil {
|
||||||
|
return fmt.Errorf("parse %s: %w", path, err)
|
||||||
|
}
|
||||||
|
if document.Schema != "sow-topdata/tlk/v1" {
|
||||||
|
return fmt.Errorf("%s: unsupported schema %q", path, document.Schema)
|
||||||
|
}
|
||||||
|
if document.BaseStrref != customTLKBase {
|
||||||
|
return fmt.Errorf("%s: base_strref must be %d", path, customTLKBase)
|
||||||
|
}
|
||||||
|
seen := map[string]struct{}{}
|
||||||
|
for index, entry := range document.Strings {
|
||||||
|
entry.Key = strings.TrimSpace(entry.Key)
|
||||||
|
if entry.Key == "" || entry.Text == "" || entry.ID == nil {
|
||||||
|
return fmt.Errorf("%s: strings[%d] requires key, text, and id", path, index)
|
||||||
|
}
|
||||||
|
if _, ok := seen[entry.Key]; ok {
|
||||||
|
return fmt.Errorf("%s: duplicate string key %q", path, entry.Key)
|
||||||
|
}
|
||||||
|
seen[entry.Key] = struct{}{}
|
||||||
|
if err := compiler.registerInlineAtID(entry.Key, *entry.ID, tlkEntryData{Text: entry.Text}); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
func loadBaseDialogData(path string) (*legacyTLKData, error) {
|
func loadBaseDialogData(path string) (*legacyTLKData, error) {
|
||||||
if _, err := os.Stat(path); err != nil {
|
if _, err := os.Stat(path); err != nil {
|
||||||
if os.IsNotExist(err) {
|
if os.IsNotExist(err) {
|
||||||
@@ -329,6 +378,33 @@ func (c *tlkCompiler) registerInline(key string, entry tlkEntryData) (tlkCompile
|
|||||||
}, nil
|
}, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func (c *tlkCompiler) registerInlineAtID(key string, id int, entry tlkEntryData) error {
|
||||||
|
if id < 0 {
|
||||||
|
return fmt.Errorf("TLK key %q has negative id %d", key, id)
|
||||||
|
}
|
||||||
|
// The pin is authoritative over the per-machine .tlk_state.json cache: a
|
||||||
|
// stale mapping that dynamically grabbed this id on an older build must
|
||||||
|
// yield so the pinned key can take it. Only a genuine clash between two
|
||||||
|
// pins in custom.tlk.yml is an author error.
|
||||||
|
if owner, ok := c.pinnedByID[id]; ok && owner != key {
|
||||||
|
return fmt.Errorf("TLK id %d is pinned by both %q and %q", id, owner, key)
|
||||||
|
}
|
||||||
|
if mapping, ok := c.state.Entries[key]; ok && mapping.ID != id {
|
||||||
|
// This key held a different cached id; release it so the pin wins.
|
||||||
|
if c.reservedByID[mapping.ID] == key {
|
||||||
|
delete(c.reservedByID, mapping.ID)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if owner, ok := c.reservedByID[id]; ok && owner != key {
|
||||||
|
// Evict the stale owner; it gets a fresh id when next made active.
|
||||||
|
delete(c.state.Entries, owner)
|
||||||
|
}
|
||||||
|
c.pinnedByID[id] = key
|
||||||
|
c.state.Entries[key] = tlkStateMapping{ID: id}
|
||||||
|
c.reservedByID[id] = key
|
||||||
|
return c.markActive(key, entry)
|
||||||
|
}
|
||||||
|
|
||||||
func (c *tlkCompiler) customStrrefForKey(key string) int {
|
func (c *tlkCompiler) customStrrefForKey(key string) int {
|
||||||
return customTLKBase + c.state.Entries[key].ID
|
return customTLKBase + c.state.Entries[key].ID
|
||||||
}
|
}
|
||||||
@@ -338,6 +414,7 @@ func (c *tlkCompiler) markActive(key string, entry tlkEntryData) error {
|
|||||||
if !ok {
|
if !ok {
|
||||||
mapping = tlkStateMapping{ID: c.allocateID(), Retired: false}
|
mapping = tlkStateMapping{ID: c.allocateID(), Retired: false}
|
||||||
c.state.Entries[key] = mapping
|
c.state.Entries[key] = mapping
|
||||||
|
c.reservedByID[mapping.ID] = key
|
||||||
}
|
}
|
||||||
existing, ok := c.active[key]
|
existing, ok := c.active[key]
|
||||||
if ok && existing != entry {
|
if ok && existing != entry {
|
||||||
@@ -351,9 +428,13 @@ func (c *tlkCompiler) markActive(key string, entry tlkEntryData) error {
|
|||||||
}
|
}
|
||||||
|
|
||||||
func (c *tlkCompiler) allocateID() int {
|
func (c *tlkCompiler) allocateID() int {
|
||||||
id := c.nextID
|
for {
|
||||||
c.nextID++
|
id := c.nextID
|
||||||
return id
|
c.nextID++
|
||||||
|
if _, reserved := c.reservedByID[id]; !reserved {
|
||||||
|
return id
|
||||||
|
}
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func (c *tlkCompiler) finish(outputDir, tlkName string) (int, error) {
|
func (c *tlkCompiler) finish(outputDir, tlkName string) (int, error) {
|
||||||
@@ -727,3 +808,36 @@ func sortedKeys[M ~map[string]V, V any](input M) []string {
|
|||||||
func almostEqualFloat32(a, b float32) bool {
|
func almostEqualFloat32(a, b float32) bool {
|
||||||
return math.Abs(float64(a-b)) < 0.0001
|
return math.Abs(float64(a-b)) < 0.0001
|
||||||
}
|
}
|
||||||
|
func mergeLegacyTLK(base, override *legacyTLKData) *legacyTLKData {
|
||||||
|
if base == nil && override == nil {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
merged := &legacyTLKData{
|
||||||
|
Language: "en",
|
||||||
|
Entries: map[string]tlkEntryData{},
|
||||||
|
Lock: map[string]int{},
|
||||||
|
}
|
||||||
|
if base != nil {
|
||||||
|
if base.Language != "" {
|
||||||
|
merged.Language = base.Language
|
||||||
|
}
|
||||||
|
for key, entry := range base.Entries {
|
||||||
|
merged.Entries[key] = entry
|
||||||
|
}
|
||||||
|
for key, id := range base.Lock {
|
||||||
|
merged.Lock[key] = id
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if override != nil {
|
||||||
|
if override.Language != "" {
|
||||||
|
merged.Language = override.Language
|
||||||
|
}
|
||||||
|
for key, entry := range override.Entries {
|
||||||
|
merged.Entries[key] = entry
|
||||||
|
}
|
||||||
|
for key, id := range override.Lock {
|
||||||
|
merged.Lock[key] = id
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return merged
|
||||||
|
}
|
||||||
|
|||||||
@@ -2,6 +2,7 @@ package topdata
|
|||||||
|
|
||||||
import (
|
import (
|
||||||
"bytes"
|
"bytes"
|
||||||
|
"encoding/json"
|
||||||
"errors"
|
"errors"
|
||||||
"fmt"
|
"fmt"
|
||||||
"io"
|
"io"
|
||||||
@@ -13,6 +14,7 @@ import (
|
|||||||
"time"
|
"time"
|
||||||
|
|
||||||
"git.westgate.pw/ShadowsOverWestgate/sow-tools/internal/erf"
|
"git.westgate.pw/ShadowsOverWestgate/sow-tools/internal/erf"
|
||||||
|
"git.westgate.pw/ShadowsOverWestgate/sow-tools/internal/gff"
|
||||||
"git.westgate.pw/ShadowsOverWestgate/sow-tools/internal/project"
|
"git.westgate.pw/ShadowsOverWestgate/sow-tools/internal/project"
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -211,10 +213,19 @@ func collectTopPackageResources(p *project.Project, compiled2DADir string) ([]er
|
|||||||
if d.IsDir() {
|
if d.IsDir() {
|
||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
if strings.HasPrefix(filepath.Base(path), ".") {
|
rel, err := filepath.Rel(assetsDir, path)
|
||||||
return nil
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
var resource erf.Resource
|
||||||
|
if strings.HasSuffix(strings.ToLower(rel), ".itp.json") {
|
||||||
|
resource, err = topPackageITPResourceFromJSON(path)
|
||||||
|
} else {
|
||||||
|
if skipTopPackageAsset(rel) {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
resource, err = topPackageResourceFromPath(path)
|
||||||
}
|
}
|
||||||
resource, err := topPackageResourceFromPath(path)
|
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
@@ -247,6 +258,27 @@ func collectTopPackageResources(p *project.Project, compiled2DADir string) ([]er
|
|||||||
return resources, assetFiles, nil
|
return resources, assetFiles, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func topPackageITPResourceFromJSON(path string) (erf.Resource, error) {
|
||||||
|
raw, err := os.ReadFile(path)
|
||||||
|
if err != nil {
|
||||||
|
return erf.Resource{}, fmt.Errorf("read %s: %w", path, err)
|
||||||
|
}
|
||||||
|
var document gff.Document
|
||||||
|
if err := json.Unmarshal(raw, &document); err != nil {
|
||||||
|
return erf.Resource{}, fmt.Errorf("parse %s: %w", path, err)
|
||||||
|
}
|
||||||
|
if document.FileType != "ITP " {
|
||||||
|
return erf.Resource{}, fmt.Errorf("%s: file_type must be ITP", path)
|
||||||
|
}
|
||||||
|
var payload bytes.Buffer
|
||||||
|
if err := gff.Write(&payload, document); err != nil {
|
||||||
|
return erf.Resource{}, fmt.Errorf("compile %s: %w", path, err)
|
||||||
|
}
|
||||||
|
resourceType, _ := erf.HAKResourceTypeForExtension("itp")
|
||||||
|
name := strings.TrimSuffix(filepath.Base(path), ".itp.json")
|
||||||
|
return erf.Resource{Name: strings.ToLower(name), Type: resourceType, Data: payload.Bytes(), Size: int64(payload.Len())}, nil
|
||||||
|
}
|
||||||
|
|
||||||
func shouldSkipTopDataSourceDir(path string, skipDirs map[string]struct{}) bool {
|
func shouldSkipTopDataSourceDir(path string, skipDirs map[string]struct{}) bool {
|
||||||
_, ok := skipDirs[filepath.Clean(path)]
|
_, ok := skipDirs[filepath.Clean(path)]
|
||||||
return ok
|
return ok
|
||||||
@@ -385,6 +417,25 @@ func newestMatchingAutogenOverrideInput(scanRoot string, include []string) (time
|
|||||||
return newest, newestPath, nil
|
return newest, newestPath, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// skipTopPackageAsset reports whether a file under assets/ is not a HAK
|
||||||
|
// resource and must be ignored by both validation and packing: anything in a
|
||||||
|
// hidden or underscore-prefixed directory (working dirs like _candidates),
|
||||||
|
// hidden files, docs, and any extension that is not a known NWN ResType
|
||||||
|
// (erf.extensionTypes is the whitelist). rel is the path relative to assets/.
|
||||||
|
func skipTopPackageAsset(rel string) bool {
|
||||||
|
for _, part := range strings.Split(filepath.ToSlash(rel), "/") {
|
||||||
|
if strings.HasPrefix(part, ".") || strings.HasPrefix(part, "_") {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if strings.EqualFold(filepath.Ext(rel), ".md") {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
ext := strings.TrimPrefix(strings.ToLower(filepath.Ext(rel)), ".")
|
||||||
|
_, ok := erf.HAKResourceTypeForExtension(ext)
|
||||||
|
return !ok
|
||||||
|
}
|
||||||
|
|
||||||
func topPackageResourceFromPath(path string) (erf.Resource, error) {
|
func topPackageResourceFromPath(path string) (erf.Resource, error) {
|
||||||
extension := strings.TrimPrefix(strings.ToLower(filepath.Ext(path)), ".")
|
extension := strings.TrimPrefix(strings.ToLower(filepath.Ext(path)), ".")
|
||||||
resourceType, ok := erf.HAKResourceTypeForExtension(extension)
|
resourceType, ok := erf.HAKResourceTypeForExtension(extension)
|
||||||
|
|||||||
+12
-15
@@ -12,7 +12,6 @@ import (
|
|||||||
"strconv"
|
"strconv"
|
||||||
"strings"
|
"strings"
|
||||||
|
|
||||||
"git.westgate.pw/ShadowsOverWestgate/sow-tools/internal/erf"
|
|
||||||
"git.westgate.pw/ShadowsOverWestgate/sow-tools/internal/project"
|
"git.westgate.pw/ShadowsOverWestgate/sow-tools/internal/project"
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -268,6 +267,7 @@ func ValidateProject(p *project.Project) ValidationReport {
|
|||||||
validateNativeLockAllocation(dataDir, p.EffectiveConfig().TopData.RowGeneration, &report)
|
validateNativeLockAllocation(dataDir, p.EffectiveConfig().TopData.RowGeneration, &report)
|
||||||
validateGeneratedFeatFamilies(dataDir, &report)
|
validateGeneratedFeatFamilies(dataDir, &report)
|
||||||
validateClassSpellbooks(dataDir, &report)
|
validateClassSpellbooks(dataDir, &report)
|
||||||
|
validateClassFeatMasterfeatAccessibility(dataDir, &report)
|
||||||
validateItempropsRegistryGraph(dataDir, &report)
|
validateItempropsRegistryGraph(dataDir, &report)
|
||||||
if _, err := os.Stat(statePath); err == nil {
|
if _, err := os.Stat(statePath); err == nil {
|
||||||
report.Files++
|
report.Files++
|
||||||
@@ -1867,8 +1867,10 @@ func validateGeneratedFeatFamilies(dataDir string, report *ValidationReport) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
required := []requiredFeatFamily{
|
required := []requiredFeatFamily{
|
||||||
{FamilyKey: "skillfocus", Dataset: "skills", Predicate: "accessible"},
|
// skill families: the source dataset is data-driven (family spec), only
|
||||||
{FamilyKey: "greaterskillfocus", Dataset: "skills", Predicate: "accessible"},
|
// the accessibility predicate is required
|
||||||
|
{FamilyKey: "skillfocus", Predicate: "accessible"},
|
||||||
|
{FamilyKey: "greaterskillfocus", Predicate: "accessible"},
|
||||||
{FamilyKey: "weaponfocus", Dataset: "baseitems", Column: "WeaponFocusFeat"},
|
{FamilyKey: "weaponfocus", Dataset: "baseitems", Column: "WeaponFocusFeat"},
|
||||||
{FamilyKey: "weaponspecialization", Dataset: "baseitems", Column: "WeaponSpecializationFeat"},
|
{FamilyKey: "weaponspecialization", Dataset: "baseitems", Column: "WeaponSpecializationFeat"},
|
||||||
{FamilyKey: "improvedcritical", Dataset: "baseitems", Column: "WeaponImprovedCriticalFeat"},
|
{FamilyKey: "improvedcritical", Dataset: "baseitems", Column: "WeaponImprovedCriticalFeat"},
|
||||||
@@ -1882,7 +1884,7 @@ func validateGeneratedFeatFamilies(dataDir string, report *ValidationReport) {
|
|||||||
if !ok {
|
if !ok {
|
||||||
continue
|
continue
|
||||||
}
|
}
|
||||||
if spec.ChildSource.Dataset != requirement.Dataset ||
|
if (requirement.Dataset != "" && spec.ChildSource.Dataset != requirement.Dataset) ||
|
||||||
spec.ChildSource.Column != requirement.Column ||
|
spec.ChildSource.Column != requirement.Column ||
|
||||||
spec.ChildSource.Predicate != requirement.Predicate {
|
spec.ChildSource.Predicate != requirement.Predicate {
|
||||||
report.Diagnostics = append(report.Diagnostics, Diagnostic{
|
report.Diagnostics = append(report.Diagnostics, Diagnostic{
|
||||||
@@ -2081,7 +2083,10 @@ func validateGeneratedFeatFamilyCompleteness(path string, spec familyExpansionSp
|
|||||||
}
|
}
|
||||||
if include {
|
if include {
|
||||||
if familyAllowlist {
|
if familyAllowlist {
|
||||||
slug := strings.TrimPrefix(sourceKey, spec.ChildSource.Dataset+":")
|
slug := sourceKey
|
||||||
|
if idx := strings.Index(slug, ":"); idx >= 0 {
|
||||||
|
slug = slug[idx+1:]
|
||||||
|
}
|
||||||
featKey, _, _, err := ctx.resolveGeneratedFeatIdentity(spec, slug, row)
|
featKey, _, _, err := ctx.resolveGeneratedFeatIdentity(spec, slug, row)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
report.Diagnostics = append(report.Diagnostics, Diagnostic{
|
report.Diagnostics = append(report.Diagnostics, Diagnostic{
|
||||||
@@ -2229,19 +2234,11 @@ func validateTopPackageAssets(sourceDir, dataDir string, report *ValidationRepor
|
|||||||
if err != nil {
|
if err != nil {
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
if strings.HasPrefix(filepath.Base(path), ".") {
|
if skipTopPackageAsset(rel) {
|
||||||
return nil
|
return nil // not a NWN ResType (script, doc, _work dir, ...): never packed, never checked
|
||||||
}
|
}
|
||||||
base := strings.ToLower(strings.TrimSuffix(filepath.Base(path), filepath.Ext(path)))
|
base := strings.ToLower(strings.TrimSuffix(filepath.Base(path), filepath.Ext(path)))
|
||||||
ext := strings.TrimPrefix(strings.ToLower(filepath.Ext(path)), ".")
|
ext := strings.TrimPrefix(strings.ToLower(filepath.Ext(path)), ".")
|
||||||
if _, ok := erf.HAKResourceTypeForExtension(ext); !ok {
|
|
||||||
report.Diagnostics = append(report.Diagnostics, Diagnostic{
|
|
||||||
Severity: SeverityError,
|
|
||||||
Path: path,
|
|
||||||
Message: fmt.Sprintf("unsupported topdata asset HAK resource extension %q", filepath.Ext(path)),
|
|
||||||
})
|
|
||||||
return nil
|
|
||||||
}
|
|
||||||
key := base + "." + ext
|
key := base + "." + ext
|
||||||
if previous, ok := seen[key]; ok {
|
if previous, ok := seen[key]; ok {
|
||||||
report.Diagnostics = append(report.Diagnostics, Diagnostic{
|
report.Diagnostics = append(report.Diagnostics, Diagnostic{
|
||||||
|
|||||||
+232
-4261
File diff suppressed because it is too large
Load Diff
@@ -1,198 +0,0 @@
|
|||||||
package topdata
|
|
||||||
|
|
||||||
import (
|
|
||||||
"os"
|
|
||||||
"path/filepath"
|
|
||||||
"strings"
|
|
||||||
)
|
|
||||||
|
|
||||||
func importLegacyVFXPersistent(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
|
|
||||||
_ = legacyTLK
|
|
||||||
if strings.TrimSpace(referenceBuilderDir) == "" {
|
|
||||||
return 0, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
legacyDir := filepath.Join(referenceBuilderDir, "data", "vfx_persistent")
|
|
||||||
if _, err := os.Stat(legacyDir); err != nil {
|
|
||||||
if os.IsNotExist(err) {
|
|
||||||
return 0, nil
|
|
||||||
}
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
|
|
||||||
targetDir := filepath.Join(dataDir, "vfx_persistent")
|
|
||||||
targetBasePath := filepath.Join(targetDir, "base.json")
|
|
||||||
targetLockPath := filepath.Join(targetDir, "lock.json")
|
|
||||||
targetModulesDir := filepath.Join(targetDir, "modules")
|
|
||||||
targetModulePath := filepath.Join(targetModulesDir, "freeformaoes.json")
|
|
||||||
if fileExists(targetBasePath) && fileExists(targetLockPath) && fileExists(targetModulePath) {
|
|
||||||
baseObj, baseErr := loadJSONObject(targetBasePath)
|
|
||||||
lockObj, lockErr := loadJSONObject(targetLockPath)
|
|
||||||
if baseErr == nil && lockErr == nil && vfxPersistentImportComplete(baseObj, lockObj) {
|
|
||||||
legacyModulePaths, err := collectJSONRelativePaths(filepath.Join(legacyDir, "modules"))
|
|
||||||
if err == nil {
|
|
||||||
targetModulePaths, err := collectJSONRelativePaths(targetModulesDir)
|
|
||||||
if err == nil && equalStringSlices(legacyModulePaths, targetModulePaths) {
|
|
||||||
return 0, nil
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
baseObj, err := loadJSONObject(filepath.Join(legacyDir, "base.json"))
|
|
||||||
if err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
lockObj, err := synthesizeVFXPersistentKeys(baseObj)
|
|
||||||
if err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
legacyLockObj, err := loadJSONObject(filepath.Join(legacyDir, "lock.json"))
|
|
||||||
if err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
for key, value := range legacyLockObj {
|
|
||||||
lockObj[key] = value
|
|
||||||
}
|
|
||||||
if err := os.MkdirAll(targetModulesDir, 0o755); err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
|
|
||||||
updated := 0
|
|
||||||
for _, write := range []struct {
|
|
||||||
path string
|
|
||||||
obj map[string]any
|
|
||||||
}{
|
|
||||||
{path: targetBasePath, obj: baseObj},
|
|
||||||
{path: targetLockPath, obj: lockObj},
|
|
||||||
} {
|
|
||||||
changed, err := writeJSONObjectIfChanged(write.path, write.obj)
|
|
||||||
if err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
if changed {
|
|
||||||
updated++
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
moduleRelPaths, err := collectJSONRelativePaths(filepath.Join(legacyDir, "modules"))
|
|
||||||
if err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
for _, relPath := range moduleRelPaths {
|
|
||||||
moduleObj, err := loadJSONObject(filepath.Join(legacyDir, "modules", filepath.FromSlash(relPath)))
|
|
||||||
if err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
targetPath := filepath.Join(targetModulesDir, filepath.FromSlash(relPath))
|
|
||||||
if err := os.MkdirAll(filepath.Dir(targetPath), 0o755); err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
changed, err := writeJSONObjectIfChanged(targetPath, moduleObj)
|
|
||||||
if err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
if changed {
|
|
||||||
updated++
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
targetModulePaths, err := collectJSONRelativePaths(targetModulesDir)
|
|
||||||
if err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
legacyModules := make(map[string]struct{}, len(moduleRelPaths))
|
|
||||||
for _, relPath := range moduleRelPaths {
|
|
||||||
legacyModules[relPath] = struct{}{}
|
|
||||||
}
|
|
||||||
for _, relPath := range targetModulePaths {
|
|
||||||
if _, ok := legacyModules[relPath]; ok {
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
if err := os.Remove(filepath.Join(targetModulesDir, filepath.FromSlash(relPath))); err != nil && !os.IsNotExist(err) {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
updated++
|
|
||||||
}
|
|
||||||
|
|
||||||
return updated, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func vfxPersistentImportComplete(baseObj, lockObj map[string]any) bool {
|
|
||||||
if !vfxPersistentRowsHaveKeys(baseObj) {
|
|
||||||
return false
|
|
||||||
}
|
|
||||||
expected := map[string]int{
|
|
||||||
"vfx_persistent:blankradius5": 47,
|
|
||||||
"vfx_persistent:blankradius10": 48,
|
|
||||||
"vfx_persistent:blankradius15": 49,
|
|
||||||
"vfx_persistent:blankradius20": 50,
|
|
||||||
"vfx_persistent:blankradius25": 51,
|
|
||||||
"vfx_persistent:blankradius30": 52,
|
|
||||||
}
|
|
||||||
for key, want := range expected {
|
|
||||||
raw, ok := lockObj[key]
|
|
||||||
if !ok {
|
|
||||||
return false
|
|
||||||
}
|
|
||||||
got, err := asInt(raw)
|
|
||||||
if err != nil || got != want {
|
|
||||||
return false
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return true
|
|
||||||
}
|
|
||||||
|
|
||||||
func synthesizeVFXPersistentKeys(baseObj map[string]any) (map[string]any, error) {
|
|
||||||
rawRows, ok := baseObj["rows"].([]any)
|
|
||||||
if !ok {
|
|
||||||
return map[string]any{}, nil
|
|
||||||
}
|
|
||||||
lockObj := map[string]any{}
|
|
||||||
for _, raw := range rawRows {
|
|
||||||
row, ok := raw.(map[string]any)
|
|
||||||
if !ok {
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
label, _ := row["LABEL"].(string)
|
|
||||||
if label == "" {
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
key := "vfx_persistent:" + strings.ToLower(label)
|
|
||||||
row["key"] = key
|
|
||||||
if rawID, ok := row["id"]; ok {
|
|
||||||
lockObj[key] = rawID
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return lockObj, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func vfxPersistentRowsHaveKeys(baseObj map[string]any) bool {
|
|
||||||
rawRows, ok := baseObj["rows"].([]any)
|
|
||||||
if !ok {
|
|
||||||
return false
|
|
||||||
}
|
|
||||||
for _, raw := range rawRows {
|
|
||||||
row, ok := raw.(map[string]any)
|
|
||||||
if !ok {
|
|
||||||
return false
|
|
||||||
}
|
|
||||||
key, _ := row["key"].(string)
|
|
||||||
if key == "" {
|
|
||||||
return false
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return true
|
|
||||||
}
|
|
||||||
|
|
||||||
func equalStringSlices(left, right []string) bool {
|
|
||||||
if len(left) != len(right) {
|
|
||||||
return false
|
|
||||||
}
|
|
||||||
for i := range left {
|
|
||||||
if left[i] != right[i] {
|
|
||||||
return false
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return true
|
|
||||||
}
|
|
||||||
@@ -1,6 +0,0 @@
|
|||||||
package topdata
|
|
||||||
|
|
||||||
func importLegacyVisualeffects(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
|
|
||||||
_ = legacyTLK
|
|
||||||
return importLegacyDatasetMirror(referenceBuilderDir, dataDir, "visualeffects", "visualeffects.2da", nil)
|
|
||||||
}
|
|
||||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user