Topdata Migration (#2)
Native topdata migration summary - Replaced the legacy 2dabuilder runtime path with the native topdata builder. - Native build, validate, compare, and convert flows now cover the canonical topdata pipeline. - compare-topdata is now a native self-check; legacy reference-builder runtime usage was removed. - Parts generation follows the native asset-scan contract and uses sow-assets via project asset resolution. - Generated feat families, class-feat injects, masterfeat/successor expansion, and dataset-derived feat generation were brought to parity and regression-covered. - Remaining mirrored datasets were migrated into native-owned canonical data while preserving lock IDs. - normalize-topdata bridge behavior and migration-era ownership markers were removed. - Documentation was rewritten around the native workflow and active contracts were cleaned up. - Final hardening pass removed stale validator assumptions and cleaned ignored/generated artifacts for PR readiness. Reviewed-on: https://gitea.westgate.pw/ShadowsOverWestgate/sow-tools/pulls/2 Co-authored-by: vickydotbat <vickydotbat@tutamail.com> Co-committed-by: vickydotbat <vickydotbat@tutamail.com>
This commit is contained in:
@@ -0,0 +1,127 @@
|
||||
# AI Agent Contract: Auto-Include Existing Part Models in 2DA Generation
|
||||
|
||||
## Objective
|
||||
|
||||
The native 2DA generation pipeline must automatically include existing part models already present in the **`sow-assets` repository** when building final parts tables.
|
||||
|
||||
This behavior is required for parity with the current asset set and must not depend on manually authored entries for models that already exist in repository assets.
|
||||
|
||||
---
|
||||
|
||||
## Source of Truth
|
||||
|
||||
The native builder must obtain existing model information from the project-resolved
|
||||
topdata assets root, which defaults to the sibling **`sow-assets` repository** when
|
||||
`topdata.assets` is not explicitly configured.
|
||||
|
||||
Repository-backed discovery is taken from:
|
||||
|
||||
```text
|
||||
sow-assets/assets/parts/**/*.mdl
|
||||
```
|
||||
|
||||
The implementation may also accept the sibling repo's legacy `assets/part/` layout when
|
||||
resolving the physical path, but it must not substitute non-project asset sources.
|
||||
|
||||
---
|
||||
|
||||
## Required Scope
|
||||
|
||||
Process existing `.mdl` files for these parts table categories only:
|
||||
|
||||
- `belt`
|
||||
- `bicep`
|
||||
- `chest`
|
||||
- `foot`
|
||||
- `forearm`
|
||||
- `leg`
|
||||
- `neck`
|
||||
- `pelvis`
|
||||
- `robe`
|
||||
- `shin`
|
||||
- `shoulder`
|
||||
|
||||
Dataset mapping rules:
|
||||
|
||||
- `parts/legs` maps to asset category `leg`
|
||||
- `parts/hand` remains unsupported and is not auto-generated
|
||||
|
||||
The scan must recurse through subdirectories under each applicable category path.
|
||||
|
||||
---
|
||||
|
||||
## Required Behavior
|
||||
|
||||
### 1. Repository-backed discovery
|
||||
|
||||
For each supported part category, scan the corresponding files under the resolved
|
||||
topdata assets root and discover all existing `.mdl` files.
|
||||
|
||||
### 2. Trailing numeric suffix extraction
|
||||
|
||||
For each discovered `.mdl` filename:
|
||||
|
||||
- inspect the filename stem
|
||||
- extract the trailing numeric suffix, if one exists
|
||||
- ignore files with no trailing numeric suffix for row-generation purposes
|
||||
|
||||
### 3. Row generation
|
||||
|
||||
If a trailing numeric suffix is present:
|
||||
|
||||
- use that numeric suffix as the output row ID
|
||||
- emit a row for that part model in the final generated 2DA
|
||||
|
||||
### 4. Default emitted values
|
||||
|
||||
For rows generated from discovered existing models, set:
|
||||
|
||||
- `COSTMODIFIER = 0`
|
||||
- `ACBONUS = 0.00`
|
||||
|
||||
### 5. Override precedence
|
||||
|
||||
If file-based overrides exist, they must take precedence over discovered defaults.
|
||||
|
||||
That means:
|
||||
|
||||
- repository discovery provides the baseline row presence and default values
|
||||
- override data may replace or augment those values
|
||||
- override data wins in any conflict
|
||||
|
||||
---
|
||||
|
||||
## Non-Negotiable Requirements
|
||||
|
||||
- The implementation must read model presence from the **project-resolved assets repo**,
|
||||
not from assumption, hardcoded lists, or legacy output.
|
||||
- Existing repository models with trailing numeric suffixes must be reflected in generated output even if they are not otherwise explicitly authored.
|
||||
- Override application is mandatory and must win over auto-discovered defaults.
|
||||
- This is replacement behavior for the native pipeline, not an optional enhancement.
|
||||
|
||||
---
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
Implementation is correct only if all of the following are true:
|
||||
|
||||
1. The builder scans the resolved `sow-assets` assets tree for part models.
|
||||
2. Only the supported part categories are considered.
|
||||
3. Files with trailing numeric suffixes generate corresponding 2DA rows.
|
||||
4. The numeric suffix is used as the row ID.
|
||||
5. Auto-generated rows default to:
|
||||
- `COSTMODIFIER = 0`
|
||||
- `ACBONUS = 0.00`
|
||||
|
||||
6. File overrides are applied after discovery and take precedence.
|
||||
7. The final output includes all eligible existing models present in the repository.
|
||||
|
||||
---
|
||||
|
||||
## Implementation Directive
|
||||
|
||||
When generating native parts 2DAs, first scan the project-resolved assets repo for
|
||||
supported part models, collect existing supported part models by category, extract
|
||||
trailing numeric suffixes from filenames, emit rows using those suffixes as row IDs
|
||||
with default `COSTMODIFIER = 0` and `ACBONUS = 0.00`, then apply overrides with
|
||||
override values taking precedence.
|
||||
@@ -0,0 +1,174 @@
|
||||
# Global Feat Injection — Specification
|
||||
|
||||
## Objective
|
||||
|
||||
Ensure the native class feat generation pipeline produces **functionally identical output** to the legacy system by applying deterministic global feat injections.
|
||||
|
||||
---
|
||||
|
||||
## Requirements
|
||||
|
||||
### R1 — Skill-based master feats
|
||||
|
||||
For every **class skill**, inject:
|
||||
|
||||
- `masterfeats:skillfocus`
|
||||
- `masterfeats:greaterskillfocus`
|
||||
|
||||
With properties:
|
||||
|
||||
````json
|
||||
{
|
||||
"GrantedOnLevel": "-1",
|
||||
"List": "1",
|
||||
"OnMenu": "0"
|
||||
}
|
||||
|
||||
---
|
||||
|
||||
### R2 — Spellcasting master feats
|
||||
|
||||
If class core defines:
|
||||
|
||||
- `"SpellCaster" = 1`
|
||||
|
||||
Then inject:
|
||||
|
||||
- `masterfeats:spellfocus`
|
||||
- `masterfeats:greaterspellfocus`
|
||||
|
||||
With properties:
|
||||
|
||||
```json
|
||||
{
|
||||
"GrantedOnLevel": "-1",
|
||||
"List": "1",
|
||||
"OnMenu": "0"
|
||||
}
|
||||
````
|
||||
|
||||
---
|
||||
|
||||
### R3 — Literacy rule
|
||||
|
||||
Inject:
|
||||
|
||||
- `feat:literate`
|
||||
|
||||
**Only if**:
|
||||
|
||||
- `feat:illiterate` is NOT present
|
||||
|
||||
With properties:
|
||||
|
||||
```json
|
||||
{
|
||||
"GrantedOnLevel": "1",
|
||||
"List": "3",
|
||||
"OnMenu": "0"
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### R4 — Default combat/menu feats
|
||||
|
||||
Always inject:
|
||||
|
||||
- `feat:offensivefighting`
|
||||
- `feat:defensivefighting`
|
||||
- `feat:horsemenu`
|
||||
|
||||
With properties:
|
||||
|
||||
```json
|
||||
{
|
||||
"GrantedOnLevel": "1",
|
||||
"List": "3",
|
||||
"OnMenu": "1"
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### R5 — Conditional combat feats
|
||||
|
||||
Inject:
|
||||
|
||||
- `feat:powerattack`
|
||||
- `feat:combatexpertise`
|
||||
|
||||
With properties:
|
||||
|
||||
```json
|
||||
{
|
||||
"GrantedOnLevel": "-1",
|
||||
"List": "0",
|
||||
"OnMenu": "1"
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### R6 — Override precedence
|
||||
|
||||
- Overrides MUST be applied after injection.
|
||||
- Overrides MUST take precedence over injected defaults.
|
||||
- Injection MUST NOT block or replace explicit override data.
|
||||
|
||||
---
|
||||
|
||||
## Behavioral Rules
|
||||
|
||||
### B1 — Output equivalence
|
||||
|
||||
- The resulting feat set MUST match legacy output **in substance**.
|
||||
- Exact row ordering and numeric IDs are NOT significant.
|
||||
|
||||
### B2 — Deterministic inclusion
|
||||
|
||||
- All injections MUST occur consistently for identical inputs.
|
||||
- No randomness or order-dependent behavior is allowed.
|
||||
|
||||
### B3 — Conditional correctness
|
||||
|
||||
- `feat:literate` MUST NOT be injected if `feat:illiterate` exists.
|
||||
- Spell focus feats MUST ONLY be injected when `"SpellCaster" = 1`.
|
||||
|
||||
### B4 — Integration with pipeline
|
||||
|
||||
- Injected feats MUST flow through the same processing stages as authored feats.
|
||||
- They MUST be subject to:
|
||||
- overrides
|
||||
- deduplication
|
||||
- final emission rules
|
||||
|
||||
---
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
An implementation is considered correct if:
|
||||
|
||||
1. **Parity**
|
||||
- For any given class input, native output produces the same effective feats as legacy.
|
||||
|
||||
2. **Injection coverage**
|
||||
- All required feats (R1–R5) appear when conditions are met.
|
||||
|
||||
3. **Exclusion correctness**
|
||||
- No invalid injections occur (e.g., literacy conflict).
|
||||
|
||||
4. **Override dominance**
|
||||
- Any override affecting injected feats is correctly applied.
|
||||
|
||||
5. **Stability**
|
||||
- Repeated runs with identical input produce identical output.
|
||||
|
||||
---
|
||||
|
||||
## Non-Negotiables
|
||||
|
||||
- This is **parity-critical**, not a best-effort feature.
|
||||
- Native pipeline MUST be capable of fully replacing legacy output.
|
||||
- There MUST be **zero dependency** on legacy systems at runtime.
|
||||
- Any deviation from legacy-equivalent output is a **defect**.
|
||||
@@ -0,0 +1,70 @@
|
||||
# Generic Family Expansion Contract
|
||||
|
||||
Topdata now treats underscore expansion as a structural rule:
|
||||
|
||||
- `parent_child` means `child` is an expansion of `parent`
|
||||
- underscores are for family structure, not simulated spaces
|
||||
- existing canonical keys still win when lock or TLK state already established them
|
||||
|
||||
This is a global interpretation rule, not a wiki-only convention.
|
||||
|
||||
## Identity
|
||||
|
||||
- `weaponspecialization_club`
|
||||
- parent: `weaponspecialization`
|
||||
- child: `club`
|
||||
- `gnome_rock`
|
||||
- parent: `gnome`
|
||||
- child: `rock`
|
||||
- `toughness_10`
|
||||
- parent: `toughness`
|
||||
- child: `10`
|
||||
|
||||
Standalone keys without an underscore are treated as a parent identity with no child.
|
||||
|
||||
## Generated Family Files
|
||||
|
||||
Canonical generated families declare:
|
||||
|
||||
```json
|
||||
{
|
||||
"family": "weapon_focus",
|
||||
"family_key": "weaponfocus",
|
||||
"template": "masterfeats:weaponfocus",
|
||||
"child_source": {
|
||||
"dataset": "baseitems",
|
||||
"column": "WeaponFocusFeat"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Rules:
|
||||
|
||||
- `family` is the authored family type
|
||||
- `family_key` is the structural parent identity used in generated child keys
|
||||
- `template` is optional for the primitive in general, but required by current
|
||||
masterfeat-backed `feat` families
|
||||
- `child_source.dataset` identifies the canonical dataset driving expansion
|
||||
- `child_source.column` is used when expansion is gated by a non-null source field
|
||||
- `child_source.predicate` is used when expansion depends on a named rule such as
|
||||
accessibility
|
||||
|
||||
## Row Metadata
|
||||
|
||||
Generated rows can carry:
|
||||
|
||||
```json
|
||||
{
|
||||
"meta": {
|
||||
"family": {
|
||||
"parent": "weaponfocus",
|
||||
"child": "club",
|
||||
"source": "baseitems:club",
|
||||
"template": "masterfeats:weaponfocus"
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
This metadata is builder-owned and does not affect emitted 2DA columns. It preserves
|
||||
family structure for later phases without enabling wiki generation yet.
|
||||
@@ -0,0 +1,110 @@
|
||||
# `feat` Generated Families Contract
|
||||
|
||||
Canonical authored paths:
|
||||
|
||||
- `topdata/data/feat/base.json`
|
||||
- `topdata/data/feat/lock.json`
|
||||
- `topdata/data/feat/generated/skill_focus.json`
|
||||
- `topdata/data/feat/generated/greater_skill_focus.json`
|
||||
- `topdata/data/feat/generated/weapon_focus.json`
|
||||
- `topdata/data/feat/generated/weapon_specialization.json`
|
||||
- `topdata/data/feat/generated/greater_weapon_focus.json`
|
||||
- `topdata/data/feat/generated/greater_weapon_specialization.json`
|
||||
- `topdata/data/feat/generated/improved_critical.json`
|
||||
- `topdata/data/feat/generated/overwhelming_critical.json`
|
||||
- `topdata/data/feat/generated/proficiencies.json`
|
||||
- `topdata/data/feat/modules/activecombat/core.json`
|
||||
- `topdata/data/feat/modules/activecombat/specialattacks.json`
|
||||
- `topdata/data/feat/modules/class/core.json`
|
||||
- `topdata/data/feat/modules/combat/core.json`
|
||||
- `topdata/data/feat/modules/defensive/core.json`
|
||||
- `topdata/data/feat/modules/magical/core.json`
|
||||
- `topdata/data/feat/modules/other/core.json`
|
||||
- `topdata/data/feat/modules/proficiency/core.json`
|
||||
- `topdata/data/feat/modules/racial/core.json`
|
||||
- `topdata/data/feat/modules/removedandhidden/core.json`
|
||||
- `topdata/data/feat/modules/skillfeat/affinity.json`
|
||||
- `topdata/data/feat/modules/skillfeat/focus.json`
|
||||
- `topdata/data/feat/modules/skillfeat/greaterfocus.json`
|
||||
- `topdata/data/feat/modules/skillfeat/other.json`
|
||||
|
||||
`skill_affinity` is no longer authored as a generated feat file. It is synthesized from
|
||||
canonical `racialtypes` race grants.
|
||||
|
||||
Compact family-expansion shape:
|
||||
|
||||
```json
|
||||
{
|
||||
"family": "skill_focus",
|
||||
"family_key": "skillfocus",
|
||||
"template": "masterfeats:skillfocus",
|
||||
"child_source": {
|
||||
"dataset": "skills",
|
||||
"predicate": "accessible"
|
||||
},
|
||||
"overrides": {
|
||||
"skills:craft_alchemy": {
|
||||
"ICON": "ife_foc_alchm"
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Weapon family shape:
|
||||
|
||||
```json
|
||||
{
|
||||
"family": "weapon_focus",
|
||||
"family_key": "weaponfocus",
|
||||
"template": "masterfeats:weaponfocus",
|
||||
"child_source": {
|
||||
"dataset": "baseitems",
|
||||
"column": "WeaponFocusFeat"
|
||||
},
|
||||
"overrides": {
|
||||
"baseitems:heavymace": {
|
||||
"ICON": "ife_wepfoc_Lma"
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Rules:
|
||||
|
||||
- compact generated families are:
|
||||
- `skill_focus`
|
||||
- `greater_skill_focus`
|
||||
- `weapon_focus`
|
||||
- `weapon_specialization`
|
||||
- `greater_weapon_focus`
|
||||
- `greater_weapon_specialization`
|
||||
- `improved_critical`
|
||||
- `overwhelming_critical`
|
||||
- `proficiencies`
|
||||
- compact families must declare `template`
|
||||
- compact families must also declare:
|
||||
- `family_key`
|
||||
- `child_source.dataset`
|
||||
- weapon families must also declare `child_source.column`
|
||||
- skill families use `child_source.predicate: "accessible"` to mean
|
||||
`HideFromLevelUp != 1`
|
||||
- compact `overrides` are keyed by source dataset key:
|
||||
- `skills:*` for skill families
|
||||
- `baseitems:*` for weapon families
|
||||
- generated rows carry `meta.family` so the underscore family rule is preserved even
|
||||
before wiki generation exists
|
||||
- emitted feat keys must prefer existing canonical lock/TLK identities over newly
|
||||
derived child spellings
|
||||
- `proficiencies.json` is limited to the real masterfeat-backed `weaponproficiencycommoner_*`
|
||||
family
|
||||
- manual and irregular feat content continues to live under `feat/modules/`
|
||||
- generated families and module-authored feats are merged into the same final `feat.2da`
|
||||
pipeline and share inheritance, override, TLK, and family-metadata behavior
|
||||
|
||||
Current rollout policy:
|
||||
|
||||
- `feat.2da` still builds natively
|
||||
- `feat.2da` is still excluded from `compare-topdata` output-catalog self-check coverage
|
||||
via `compare_reference: false`
|
||||
- manual and irregular feat families remain authored directly in modules rather than
|
||||
compact generated-family files
|
||||
@@ -0,0 +1,69 @@
|
||||
# Topdata Inheritance Contract
|
||||
|
||||
Native topdata now supports two inheritance forms:
|
||||
|
||||
## 1. Row Sugar Compatibility
|
||||
|
||||
This is the existing narrow row helper:
|
||||
|
||||
```json
|
||||
{
|
||||
"PARENT": { "id": "custom:parent" },
|
||||
"inherit": {
|
||||
"from": "PARENT",
|
||||
"fields": ["VALUE", "ICON"]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
It remains supported for row-local field copying.
|
||||
|
||||
## 2. Generic Object Inheritance
|
||||
|
||||
This is the new reusable primitive:
|
||||
|
||||
```json
|
||||
{
|
||||
"inherit": {
|
||||
"ref": "masterfeats:skillfocus"
|
||||
},
|
||||
"LABEL": "FEAT_SKILL_FOCUS_ATHLETICS"
|
||||
}
|
||||
```
|
||||
|
||||
Or for a nested object:
|
||||
|
||||
```json
|
||||
{
|
||||
"DETAILS": {
|
||||
"inherit": {
|
||||
"ref": "custom:parent",
|
||||
"field": "DETAILS"
|
||||
},
|
||||
"meta": {
|
||||
"cost": "9"
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Rules
|
||||
|
||||
- `inherit.ref` must use stable `dataset:key` identity.
|
||||
- `inherit.field` is optional and selects an object-valued field on the target row.
|
||||
- Generic inheritance can appear on any authored object, not only top-level rows.
|
||||
- Merge precedence is `local overrides inherited`.
|
||||
- Scalars replace inherited values.
|
||||
- Arrays replace inherited values.
|
||||
- Objects merge recursively unless the local object is an atomic topdata value object such as:
|
||||
- TLK payloads
|
||||
- row refs
|
||||
- table refs
|
||||
- Cycles fail the build.
|
||||
- Missing targets fail the build.
|
||||
|
||||
## Current Intended Consumer
|
||||
|
||||
`feat` is the first dataset expected to use this primitive heavily, especially for
|
||||
borrowing shared properties from `masterfeats` while keeping `feat`'s own dataset
|
||||
contract and row model.
|
||||
@@ -0,0 +1,70 @@
|
||||
# `masterfeats` Canonical Contract
|
||||
|
||||
`masterfeats` is a stable native canonical family consumed directly by `feat`
|
||||
generation, class-feat expansion, and inheritance/ref resolution.
|
||||
|
||||
Current canonical scope:
|
||||
|
||||
- dataset root: `topdata/data/masterfeats/`
|
||||
- required files:
|
||||
- `base.json`
|
||||
- `lock.json`
|
||||
- generated output:
|
||||
- `masterfeats.2da`
|
||||
|
||||
## `base.json`
|
||||
|
||||
Canonical shape:
|
||||
|
||||
```json
|
||||
{
|
||||
"output": "masterfeats.2da",
|
||||
"columns": ["LABEL", "STRREF", "DESCRIPTION", "..."],
|
||||
"rows": [
|
||||
{
|
||||
"id": 0,
|
||||
"key": "masterfeats:weaponfocus",
|
||||
"LABEL": "WeaponFocus",
|
||||
"STRREF": "6490",
|
||||
"DESCRIPTION": "436"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
Rules:
|
||||
|
||||
- `output` must be exactly `masterfeats.2da`
|
||||
- `columns` must include `LABEL`, `STRREF`, and `DESCRIPTION`
|
||||
- every canonical row must have a non-empty `key`
|
||||
- row keys must start with `masterfeats:`
|
||||
- inline TLK payloads are allowed in `STRREF` and `DESCRIPTION`
|
||||
- numeric/text references are both allowed where the native TLK compiler already supports them
|
||||
|
||||
## `lock.json`
|
||||
|
||||
Canonical shape:
|
||||
|
||||
```json
|
||||
{
|
||||
"masterfeats:weaponfocus": 0
|
||||
}
|
||||
```
|
||||
|
||||
Rules:
|
||||
|
||||
- every key must start with `masterfeats:`
|
||||
- every value must be numeric
|
||||
- the lockfile remains the canonical stable id source for authored keys
|
||||
|
||||
## Module contributions
|
||||
|
||||
This family continues to support the shared canonical module shapes already used by native
|
||||
datasets:
|
||||
|
||||
- `entries`
|
||||
- `overrides`
|
||||
- `rows`
|
||||
|
||||
Those shapes are validated generically in `topdata.go`; this contract only adds the
|
||||
family-specific guarantees for `masterfeats`.
|
||||
@@ -0,0 +1,125 @@
|
||||
# Portrait Metadata Contract
|
||||
|
||||
## Objective
|
||||
|
||||
Define the canonical behavior for portrait metadata embedded in topdata rows, enabling cross-dataset portrait injection into the portraits.2da table.
|
||||
|
||||
---
|
||||
|
||||
## Portrait Metadata Shape
|
||||
|
||||
Portrait metadata is embedded in row `meta.portrait`:
|
||||
|
||||
```json
|
||||
{
|
||||
"meta": {
|
||||
"portrait": {
|
||||
"resref": "nw2book1_",
|
||||
"sex": 4,
|
||||
"race": 10,
|
||||
"inanimate_type": 4,
|
||||
"plot": 0,
|
||||
"low_gore": "****"
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Required Fields
|
||||
|
||||
- `resref` — non-empty string identifying the portrait base resource reference
|
||||
|
||||
### Optional Fields
|
||||
|
||||
- `sex` — numeric or string value mapped to portraits.2da `Sex` column
|
||||
- `race` — numeric or string value mapped to portraits.2da `Race` column
|
||||
- `inanimate_type` — numeric or string value mapped to portraits.2da `InanimateType` column
|
||||
- `plot` — numeric or string value mapped to portraits.2da `Plot` column
|
||||
- `low_gore` — numeric or string value mapped to portraits.2da `LowGore` column
|
||||
|
||||
Field names are case-insensitive with hyphen/underscore normalization:
|
||||
- `inanimate_type` = `inanimate-type` = `InanimateType`
|
||||
|
||||
---
|
||||
|
||||
## Merge Behavior
|
||||
|
||||
### Identity Matching
|
||||
|
||||
Portrait entries are matched by `resref`. When any row in any dataset carries `meta.portrait.resref`:
|
||||
|
||||
1. If a portraits.2da row with matching `BaseResRef` already exists → merge into that row
|
||||
2. If no matching row exists → create a new row with auto-generated identity `portraits:auto:<resref>`
|
||||
|
||||
### Field Mapping
|
||||
|
||||
Portrait metadata fields map to portraits.2da columns:
|
||||
|
||||
| Metadata Key | 2DA Column |
|
||||
|---|---|
|
||||
| `resref` | `BaseResRef` |
|
||||
| `sex` | `Sex` |
|
||||
| `race` | `Race` |
|
||||
| `inanimate_type` | `InanimateType` |
|
||||
| `plot` | `Plot` |
|
||||
| `low_gore` | `LowGore` |
|
||||
|
||||
### Override Precedence
|
||||
|
||||
- First contributor wins for each column
|
||||
- If a column already has a non-null value, subsequent contributions are rejected if they differ
|
||||
- Identical values (after normalization) are silently accepted
|
||||
- Conflicting values produce a build error
|
||||
|
||||
### Null-Like Values
|
||||
|
||||
Missing fields and `nullValue` ("****") are treated as unset. They do not override existing values and are not considered conflicts.
|
||||
|
||||
---
|
||||
|
||||
## Lock Data
|
||||
|
||||
Auto-generated portrait rows are tracked in lock data with key pattern:
|
||||
|
||||
```
|
||||
portraits:auto:<resref>
|
||||
```
|
||||
|
||||
For example: `portraits:auto:nw2book1_`
|
||||
|
||||
---
|
||||
|
||||
## Cross-Dataset Injection
|
||||
|
||||
Any dataset may contribute portrait metadata. The build pipeline:
|
||||
|
||||
1. Collects all datasets
|
||||
2. Runs `mergePortraitMetadata()` after dataset collection
|
||||
3. Injects portrait rows into the portraits.2da dataset
|
||||
4. Updates lock data if new auto-generated rows were created
|
||||
|
||||
This enables non-portrait datasets (e.g., placeables, genericdoors) to define portrait associations without modifying the portraits dataset directly.
|
||||
|
||||
---
|
||||
|
||||
## Validation Rules
|
||||
|
||||
1. `resref` is required — missing resref produces a parse error
|
||||
2. Unknown metadata fields in `meta.portrait` produce a parse error
|
||||
3. Unknown top-level keys in `meta` produce a parse error (allowed keys: `family`, `wiki`, `portrait`)
|
||||
4. Conflicting portrait field values from different contributors produce a build error
|
||||
5. Portrait metadata is excluded from emitted 2DA output — it exists only at authoring/build time
|
||||
|
||||
---
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
Implementation is correct only if all of the following are true:
|
||||
|
||||
1. Portrait metadata is parsed from `meta.portrait` with case-insensitive field names
|
||||
2. Rows with portrait metadata inject into portraits.2da by `resref` matching
|
||||
3. Auto-generated rows receive `portraits:auto:<resref>` lock keys
|
||||
4. First-contributor-wins precedence is enforced per column
|
||||
5. Conflicting values produce build errors with clear diagnostic messages
|
||||
6. Portrait metadata does not leak into emitted 2DA column output
|
||||
7. Lock data is persisted when new auto-generated rows are created
|
||||
@@ -0,0 +1,96 @@
|
||||
package topdata
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"os"
|
||||
"path/filepath"
|
||||
)
|
||||
|
||||
func importLegacyAppearance(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
|
||||
_ = legacyTLK
|
||||
|
||||
legacyDir := filepath.Join(referenceBuilderDir, "data", "appearance")
|
||||
if _, err := os.Stat(legacyDir); err != nil {
|
||||
if os.IsNotExist(err) {
|
||||
return 0, nil
|
||||
}
|
||||
return 0, err
|
||||
}
|
||||
|
||||
targetDir := filepath.Join(dataDir, "appearance")
|
||||
targetBasePath := filepath.Join(targetDir, "base.json")
|
||||
targetLockPath := filepath.Join(targetDir, "lock.json")
|
||||
targetModulesDir := filepath.Join(targetDir, "modules")
|
||||
moduleNames := []string{
|
||||
"cotblreaver.json",
|
||||
"crawlingclaw.json",
|
||||
"halfogre.json",
|
||||
"zombieknight.json",
|
||||
}
|
||||
targetModulePaths := make([]string, 0, len(moduleNames))
|
||||
for _, name := range moduleNames {
|
||||
targetModulePaths = append(targetModulePaths, filepath.Join(targetModulesDir, name))
|
||||
}
|
||||
if fileExists(targetBasePath) && fileExists(targetLockPath) {
|
||||
allModulesPresent := true
|
||||
for _, path := range targetModulePaths {
|
||||
if !fileExists(path) {
|
||||
allModulesPresent = false
|
||||
break
|
||||
}
|
||||
}
|
||||
if allModulesPresent {
|
||||
return 0, nil
|
||||
}
|
||||
}
|
||||
|
||||
baseObj, err := loadJSONObject(filepath.Join(legacyDir, "base.json"))
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
baseObj["output"] = "appearance.2da"
|
||||
|
||||
lockObj, err := loadJSONObject(filepath.Join(legacyDir, "lock.json"))
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
|
||||
moduleObjs := make([]map[string]any, 0, len(moduleNames))
|
||||
for _, name := range moduleNames {
|
||||
obj, err := loadJSONObject(filepath.Join(legacyDir, "modules", name))
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
moduleObjs = append(moduleObjs, obj)
|
||||
}
|
||||
|
||||
if err := os.MkdirAll(targetModulesDir, 0o755); err != nil {
|
||||
return 0, err
|
||||
}
|
||||
|
||||
writes := []struct {
|
||||
path string
|
||||
obj map[string]any
|
||||
}{
|
||||
{path: targetBasePath, obj: baseObj},
|
||||
{path: targetLockPath, obj: lockObj},
|
||||
}
|
||||
for i, path := range targetModulePaths {
|
||||
writes = append(writes, struct {
|
||||
path string
|
||||
obj map[string]any
|
||||
}{path: path, obj: moduleObjs[i]})
|
||||
}
|
||||
|
||||
for _, write := range writes {
|
||||
raw, err := json.MarshalIndent(write.obj, "", " ")
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
raw = append(raw, '\n')
|
||||
if err := os.WriteFile(write.path, raw, 0o644); err != nil {
|
||||
return 0, err
|
||||
}
|
||||
}
|
||||
return len(writes), nil
|
||||
}
|
||||
@@ -0,0 +1,160 @@
|
||||
package topdata
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"sort"
|
||||
)
|
||||
|
||||
func importLegacyArmor(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
|
||||
_ = legacyTLK
|
||||
|
||||
legacyDir := filepath.Join(referenceBuilderDir, "data", "armor")
|
||||
if _, err := os.Stat(legacyDir); err != nil {
|
||||
if os.IsNotExist(err) {
|
||||
return 0, nil
|
||||
}
|
||||
return 0, err
|
||||
}
|
||||
|
||||
targetDir := filepath.Join(dataDir, "armor")
|
||||
targetPath := filepath.Join(targetDir, "armor.json")
|
||||
if _, err := os.Stat(targetPath); err == nil {
|
||||
obj, err := loadJSONObject(targetPath)
|
||||
if err == nil && countLegacyTLKRefsInValue(obj) == 0 {
|
||||
return 0, nil
|
||||
}
|
||||
}
|
||||
|
||||
baseObj, err := loadJSONObject(filepath.Join(legacyDir, "base.json"))
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
rows, err := mergeArmorRows(baseObj, filepath.Join(legacyDir, "modules"))
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
|
||||
if err := os.MkdirAll(targetDir, 0o755); err != nil {
|
||||
return 0, err
|
||||
}
|
||||
if err := removePathIfExists(filepath.Join(targetDir, "base.json")); err != nil {
|
||||
return 0, err
|
||||
}
|
||||
if err := removePathIfExists(filepath.Join(targetDir, "lock.json")); err != nil {
|
||||
return 0, err
|
||||
}
|
||||
if err := removePathIfExists(filepath.Join(targetDir, "modules")); err != nil {
|
||||
return 0, err
|
||||
}
|
||||
|
||||
out := map[string]any{
|
||||
"output": "armor.2da",
|
||||
"columns": baseObj["columns"],
|
||||
"rows": rows,
|
||||
}
|
||||
raw, err := json.MarshalIndent(out, "", " ")
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
raw = append(raw, '\n')
|
||||
if err := os.WriteFile(targetPath, raw, 0o644); err != nil {
|
||||
return 0, err
|
||||
}
|
||||
return 1, nil
|
||||
}
|
||||
|
||||
func mergeArmorRows(baseObj map[string]any, modulesDir string) ([]any, error) {
|
||||
rawRows, ok := baseObj["rows"].([]any)
|
||||
if !ok {
|
||||
return nil, nil
|
||||
}
|
||||
|
||||
rows := make([]map[string]any, 0, len(rawRows))
|
||||
byID := map[int]map[string]any{}
|
||||
for _, raw := range rawRows {
|
||||
row, ok := deepCopyValue(raw).(map[string]any)
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
delete(row, "key")
|
||||
rawID, ok := row["id"]
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
id, err := asInt(rawID)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
row["id"] = id
|
||||
rows = append(rows, row)
|
||||
byID[id] = row
|
||||
}
|
||||
|
||||
modulePaths, err := collectModulePaths(modulesDir)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
for _, path := range modulePaths {
|
||||
obj, err := loadJSONObject(path)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if entries, ok := obj["entries"].(map[string]any); ok {
|
||||
for _, raw := range entries {
|
||||
row, ok := deepCopyValue(raw).(map[string]any)
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
delete(row, "key")
|
||||
rawID, ok := row["id"]
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
id, err := asInt(rawID)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
row["id"] = id
|
||||
rows = append(rows, row)
|
||||
byID[id] = row
|
||||
}
|
||||
}
|
||||
if overrides, ok := obj["overrides"].([]any); ok {
|
||||
for _, raw := range overrides {
|
||||
override, ok := raw.(map[string]any)
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
rawID, ok := override["id"]
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
id, err := asInt(rawID)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
row := byID[id]
|
||||
if row == nil {
|
||||
continue
|
||||
}
|
||||
for key, value := range override {
|
||||
if key == "id" || key == "key" || key == "_tlk" || isMetadataField(key) {
|
||||
continue
|
||||
}
|
||||
row[key] = deepCopyValue(value)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
sort.Slice(rows, func(i, j int) bool {
|
||||
return rows[i]["id"].(int) < rows[j]["id"].(int)
|
||||
})
|
||||
out := make([]any, 0, len(rows))
|
||||
for _, row := range rows {
|
||||
out = append(out, row)
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
@@ -0,0 +1,41 @@
|
||||
package topdata
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"os"
|
||||
"path/filepath"
|
||||
)
|
||||
|
||||
func importLegacyBaseDialog(referenceBuilderDir, sourceDir string) (int, error) {
|
||||
legacyPath := filepath.Join(referenceBuilderDir, "tlk", "base.json")
|
||||
if _, err := os.Stat(legacyPath); err != nil {
|
||||
if os.IsNotExist(err) {
|
||||
return 0, nil
|
||||
}
|
||||
return 0, err
|
||||
}
|
||||
|
||||
targetPath := filepath.Join(sourceDir, "base_dialog.json")
|
||||
if fileExists(targetPath) {
|
||||
current, err := loadJSONObject(targetPath)
|
||||
if err == nil {
|
||||
if entries, ok := current["entries"].(map[string]any); ok && len(entries) > 0 {
|
||||
return 0, nil
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
obj, err := loadJSONObject(legacyPath)
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
raw, err := json.MarshalIndent(obj, "", " ")
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
raw = append(raw, '\n')
|
||||
if err := os.WriteFile(targetPath, raw, 0o644); err != nil {
|
||||
return 0, err
|
||||
}
|
||||
return 1, nil
|
||||
}
|
||||
@@ -0,0 +1,105 @@
|
||||
package topdata
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"os"
|
||||
"path/filepath"
|
||||
)
|
||||
|
||||
func importLegacyBaseitems(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
|
||||
_ = legacyTLK
|
||||
|
||||
legacyDir := filepath.Join(referenceBuilderDir, "data", "baseitems")
|
||||
if _, err := os.Stat(legacyDir); err != nil {
|
||||
if os.IsNotExist(err) {
|
||||
return 0, nil
|
||||
}
|
||||
return 0, err
|
||||
}
|
||||
|
||||
targetDir := filepath.Join(dataDir, "baseitems")
|
||||
targetBasePath := filepath.Join(targetDir, "base.json")
|
||||
targetLockPath := filepath.Join(targetDir, "lock.json")
|
||||
targetModulesDir := filepath.Join(targetDir, "modules")
|
||||
moduleNames := []string{
|
||||
"blunderbuss.json",
|
||||
"coin.json",
|
||||
"estoc.json",
|
||||
"heavymace.json",
|
||||
"maulandfalchion.json",
|
||||
"ovr_baseitems.json",
|
||||
"ovr_cloak255.json",
|
||||
"ovr_helmet255.json",
|
||||
"shortspear.json",
|
||||
}
|
||||
targetModulePaths := make([]string, 0, len(moduleNames))
|
||||
for _, name := range moduleNames {
|
||||
targetModulePaths = append(targetModulePaths, filepath.Join(targetModulesDir, name))
|
||||
}
|
||||
if fileExists(targetBasePath) && fileExists(targetLockPath) {
|
||||
allModulesPresent := true
|
||||
for _, path := range targetModulePaths {
|
||||
if !fileExists(path) {
|
||||
allModulesPresent = false
|
||||
break
|
||||
}
|
||||
}
|
||||
if allModulesPresent {
|
||||
return 0, nil
|
||||
}
|
||||
}
|
||||
|
||||
baseObj, err := loadJSONObject(filepath.Join(legacyDir, "base.json"))
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
baseObj["output"] = "baseitems.2da"
|
||||
canonicalizeWikiMetadataDocument(baseObj)
|
||||
|
||||
lockObj, err := loadJSONObject(filepath.Join(legacyDir, "lock.json"))
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
|
||||
moduleObjs := make([]map[string]any, 0, len(moduleNames))
|
||||
for _, name := range moduleNames {
|
||||
obj, err := loadJSONObject(filepath.Join(legacyDir, "modules", name))
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
canonicalizeWikiMetadataDocument(obj)
|
||||
moduleObjs = append(moduleObjs, obj)
|
||||
}
|
||||
|
||||
if err := os.MkdirAll(targetModulesDir, 0o755); err != nil {
|
||||
return 0, err
|
||||
}
|
||||
|
||||
writes := []struct {
|
||||
path string
|
||||
obj map[string]any
|
||||
}{
|
||||
{path: targetBasePath, obj: baseObj},
|
||||
{path: targetLockPath, obj: lockObj},
|
||||
}
|
||||
for index, path := range targetModulePaths {
|
||||
writes = append(writes, struct {
|
||||
path string
|
||||
obj map[string]any
|
||||
}{
|
||||
path: path,
|
||||
obj: moduleObjs[index],
|
||||
})
|
||||
}
|
||||
for _, write := range writes {
|
||||
raw, err := json.MarshalIndent(write.obj, "", " ")
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
raw = append(raw, '\n')
|
||||
if err := os.WriteFile(write.path, raw, 0o644); err != nil {
|
||||
return 0, err
|
||||
}
|
||||
}
|
||||
return len(writes), nil
|
||||
}
|
||||
@@ -0,0 +1,226 @@
|
||||
package topdata
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"slices"
|
||||
"strings"
|
||||
)
|
||||
|
||||
var legacyClassesPlainFamilies = []string{"feats", "skills", "savthr", "bfeat", "pres"}
|
||||
|
||||
func importLegacyClasses(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
|
||||
legacyRoot := filepath.Join(referenceBuilderDir, "data", "classes")
|
||||
if _, err := os.Stat(legacyRoot); err != nil {
|
||||
if os.IsNotExist(err) {
|
||||
return 0, nil
|
||||
}
|
||||
return 0, err
|
||||
}
|
||||
|
||||
targetRoot := filepath.Join(dataDir, "classes")
|
||||
if canonicalClassesPresent(targetRoot) {
|
||||
return 0, nil
|
||||
}
|
||||
|
||||
coreCollected, plainCollected, err := collectLegacyClassesDatasets(legacyRoot)
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
|
||||
tableKeyByOutput := map[string]string{}
|
||||
for _, dataset := range plainCollected {
|
||||
if dataset.TableKey == "" {
|
||||
continue
|
||||
}
|
||||
registerLegacyClassesTableKey(tableKeyByOutput, dataset.TableKey, dataset.Dataset.OutputName)
|
||||
}
|
||||
|
||||
coreRows := make([]map[string]any, 0, len(coreCollected.Rows))
|
||||
for _, rawRow := range coreCollected.Rows {
|
||||
row, ok := deepCopyValue(rawRow).(map[string]any)
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
if legacyTLK != nil {
|
||||
if _, _, err := inlineLegacyTLKValue(row, legacyTLK); err != nil {
|
||||
return 0, err
|
||||
}
|
||||
}
|
||||
rewriteLegacyClassesCoreRow(row, tableKeyByOutput)
|
||||
coreRows = append(coreRows, row)
|
||||
}
|
||||
|
||||
if err := removePathIfExists(targetRoot); err != nil {
|
||||
return 0, err
|
||||
}
|
||||
if err := os.MkdirAll(filepath.Join(targetRoot, "core"), 0o755); err != nil {
|
||||
return 0, err
|
||||
}
|
||||
|
||||
writes := []struct {
|
||||
path string
|
||||
obj map[string]any
|
||||
}{
|
||||
{
|
||||
path: filepath.Join(targetRoot, "core", "base.json"),
|
||||
obj: map[string]any{
|
||||
"output": coreCollected.Dataset.OutputName,
|
||||
"columns": stringSliceToAny(coreCollected.Columns),
|
||||
"rows": rowsToAny(coreRows),
|
||||
},
|
||||
},
|
||||
{
|
||||
path: filepath.Join(targetRoot, "core", "lock.json"),
|
||||
obj: anyMapInt(coreCollected.LockData),
|
||||
},
|
||||
}
|
||||
|
||||
for _, collected := range plainCollected {
|
||||
obj := map[string]any{
|
||||
"output": collected.Dataset.OutputName,
|
||||
"columns": stringSliceToAny(collected.Columns),
|
||||
"rows": rowsToAny(collected.Rows),
|
||||
}
|
||||
if strings.TrimSpace(collected.TableKey) != "" {
|
||||
obj["key"] = collected.TableKey
|
||||
}
|
||||
writes = append(writes, struct {
|
||||
path string
|
||||
obj map[string]any
|
||||
}{
|
||||
path: filepath.Join(dataDir, collected.Dataset.Name+".json"),
|
||||
obj: obj,
|
||||
})
|
||||
}
|
||||
|
||||
for _, write := range writes {
|
||||
if err := os.MkdirAll(filepath.Dir(write.path), 0o755); err != nil {
|
||||
return 0, err
|
||||
}
|
||||
raw, err := json.MarshalIndent(write.obj, "", " ")
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
raw = append(raw, '\n')
|
||||
if err := os.WriteFile(write.path, raw, 0o644); err != nil {
|
||||
return 0, err
|
||||
}
|
||||
}
|
||||
|
||||
return len(writes), nil
|
||||
}
|
||||
|
||||
func canonicalClassesPresent(targetRoot string) bool {
|
||||
if !fileExists(filepath.Join(targetRoot, "core", "base.json")) || !fileExists(filepath.Join(targetRoot, "core", "lock.json")) {
|
||||
return false
|
||||
}
|
||||
for _, family := range legacyClassesPlainFamilies {
|
||||
ok, err := hasJSONFiles(filepath.Join(targetRoot, family))
|
||||
if err != nil || !ok {
|
||||
return false
|
||||
}
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
func collectLegacyClassesDatasets(legacyRoot string) (nativeCollectedDataset, []nativeCollectedDataset, error) {
|
||||
coreCollected, err := collectBaseDataset(nativeDataset{
|
||||
Name: "classes/core",
|
||||
BasePath: filepath.Join(legacyRoot, "core", "base.json"),
|
||||
LockPath: filepath.Join(legacyRoot, "core", "lock.json"),
|
||||
ModulesDir: filepath.Join(legacyRoot, "core", "modules"),
|
||||
OutputName: "classes.2da",
|
||||
Spec: specForDataset("classes"),
|
||||
})
|
||||
if err != nil {
|
||||
return nativeCollectedDataset{}, nil, err
|
||||
}
|
||||
coreCollected.Dataset.OutputName = "classes.2da"
|
||||
|
||||
plainCollected := make([]nativeCollectedDataset, 0)
|
||||
for _, family := range legacyClassesPlainFamilies {
|
||||
familyDir := filepath.Join(legacyRoot, family)
|
||||
entries, err := os.ReadDir(familyDir)
|
||||
if err != nil {
|
||||
if os.IsNotExist(err) {
|
||||
continue
|
||||
}
|
||||
return nativeCollectedDataset{}, nil, err
|
||||
}
|
||||
for _, entry := range entries {
|
||||
if entry.IsDir() || filepath.Ext(entry.Name()) != ".json" || strings.HasPrefix(entry.Name(), ".") || entry.Name() == "lock.json" {
|
||||
continue
|
||||
}
|
||||
filePath := filepath.Join(familyDir, entry.Name())
|
||||
tableData, err := loadJSONObject(filePath)
|
||||
if err != nil {
|
||||
return nativeCollectedDataset{}, nil, err
|
||||
}
|
||||
outputName, _ := tableData["output"].(string)
|
||||
if strings.TrimSpace(outputName) == "" {
|
||||
outputName = strings.TrimSuffix(entry.Name(), filepath.Ext(entry.Name())) + ".2da"
|
||||
}
|
||||
collected, err := collectPlainDataset(nativeDataset{
|
||||
Name: filepath.ToSlash(filepath.Join("classes", family, strings.TrimSuffix(entry.Name(), filepath.Ext(entry.Name())))),
|
||||
BasePath: filePath,
|
||||
LockPath: filepath.Join(familyDir, "lock.json"),
|
||||
OutputName: outputName,
|
||||
Spec: specForDataset(filepath.ToSlash(filepath.Join("classes", family, strings.TrimSuffix(entry.Name(), filepath.Ext(entry.Name()))))),
|
||||
})
|
||||
if err != nil {
|
||||
return nativeCollectedDataset{}, nil, err
|
||||
}
|
||||
plainCollected = append(plainCollected, collected)
|
||||
}
|
||||
}
|
||||
|
||||
slices.SortFunc(plainCollected, func(a, b nativeCollectedDataset) int {
|
||||
return strings.Compare(a.Dataset.Name, b.Dataset.Name)
|
||||
})
|
||||
return coreCollected, plainCollected, nil
|
||||
}
|
||||
|
||||
func registerLegacyClassesTableKey(tableKeyByOutput map[string]string, tableKey, outputName string) {
|
||||
trimmedOutput := strings.TrimSpace(outputName)
|
||||
if trimmedOutput != "" {
|
||||
tableKeyByOutput[trimmedOutput] = tableKey
|
||||
}
|
||||
trimmedStem := strings.TrimSpace(outputStem(outputName))
|
||||
if trimmedStem != "" {
|
||||
tableKeyByOutput[trimmedStem] = tableKey
|
||||
}
|
||||
}
|
||||
|
||||
func rewriteLegacyClassesCoreRow(row map[string]any, tableKeyByOutput map[string]string) {
|
||||
for _, field := range []string{"FeatsTable", "SavingThrowTable", "SkillsTable", "BonusFeatsTable", "PreReqTable"} {
|
||||
tableKey := legacyClassesTableKeyForValue(row[field], tableKeyByOutput)
|
||||
if tableKey == "" {
|
||||
continue
|
||||
}
|
||||
row[field] = map[string]any{"table": tableKey}
|
||||
}
|
||||
}
|
||||
|
||||
func legacyClassesTableKeyForValue(value any, tableKeyByOutput map[string]string) string {
|
||||
switch typed := value.(type) {
|
||||
case map[string]any:
|
||||
tableKey, _ := typed["table"].(string)
|
||||
return strings.TrimSpace(tableKey)
|
||||
case string:
|
||||
trimmed := strings.TrimSpace(typed)
|
||||
if trimmed == "" || trimmed == nullValue {
|
||||
return ""
|
||||
}
|
||||
if trimmed != strings.ToLower(trimmed) {
|
||||
return ""
|
||||
}
|
||||
if tableKey, ok := tableKeyByOutput[trimmed]; ok {
|
||||
return tableKey
|
||||
}
|
||||
return ""
|
||||
default:
|
||||
return ""
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,6 @@
|
||||
package topdata
|
||||
|
||||
func importLegacyCloakmodel(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
|
||||
_ = legacyTLK
|
||||
return importLegacyDatasetMirror(referenceBuilderDir, dataDir, "cloakmodel", "cloakmodel.2da", nil)
|
||||
}
|
||||
@@ -0,0 +1,530 @@
|
||||
package topdata
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"regexp"
|
||||
"sort"
|
||||
"strconv"
|
||||
"strings"
|
||||
|
||||
"gitea.westgate.pw/ShadowsOverWestgate/sow-tools/internal/project"
|
||||
)
|
||||
|
||||
type convertOptions struct {
|
||||
Namespace string
|
||||
KeyFields []string
|
||||
CollisionMode string
|
||||
BaseDialog string
|
||||
Type string
|
||||
}
|
||||
|
||||
func RunConvertCommand(args []string, stdout io.Writer) error {
|
||||
if len(args) == 0 || args[0] == "--help" || args[0] == "-h" {
|
||||
printConvertUsage(stdout)
|
||||
return nil
|
||||
}
|
||||
switch args[0] {
|
||||
case "2da-to-json":
|
||||
if len(args) > 1 && (args[1] == "--help" || args[1] == "-h") {
|
||||
_, _ = fmt.Fprintln(stdout, "usage: convert-topdata 2da-to-json [flags] <input.2da> <output.json>")
|
||||
return nil
|
||||
}
|
||||
opts, input, output, err := parseConvertArgs(args[1:], false)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
data, err := parse2DAFile(input)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
result, err := convert2DAToJSON(data, output, opts)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return writeJSONFile(output, result)
|
||||
case "2da-to-module":
|
||||
if len(args) > 1 && (args[1] == "--help" || args[1] == "-h") {
|
||||
_, _ = fmt.Fprintln(stdout, "usage: convert-topdata 2da-to-module --namespace <dataset> [--type entries|override] [flags] <input.2da> [output.json]")
|
||||
return nil
|
||||
}
|
||||
opts, input, output, err := parseConvertArgs(args[1:], true)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
data, err := parse2DAFile(input)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if opts.Type == "override" {
|
||||
result := map[string]any{"overrides": toOverridesRows(data.Rows)}
|
||||
if err := writeJSONFile(output, result); err != nil {
|
||||
return err
|
||||
}
|
||||
_, _ = fmt.Fprintf(stdout, "converted overrides: %d\n", len(result["overrides"].([]map[string]any)))
|
||||
return nil
|
||||
}
|
||||
result, skipped, err := convert2DAToModule(data, output, opts)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if err := writeJSONFile(output, result); err != nil {
|
||||
return err
|
||||
}
|
||||
if skipped > 0 {
|
||||
_, _ = fmt.Fprintf(stdout, "converted entries: %d, skipped unkeyed: %d\n", len(result["entries"].(map[string]any)), skipped)
|
||||
return nil
|
||||
}
|
||||
_, _ = fmt.Fprintf(stdout, "converted entries: %d\n", len(result["entries"].(map[string]any)))
|
||||
return nil
|
||||
case "json-to-2da":
|
||||
if len(args) > 1 && (args[1] == "--help" || args[1] == "-h") {
|
||||
_, _ = fmt.Fprintln(stdout, "usage: convert-topdata json-to-2da <input.json> <output.2da>")
|
||||
return nil
|
||||
}
|
||||
if len(args) != 3 {
|
||||
return errors.New("usage: convert-topdata json-to-2da <input.json> <output.2da>")
|
||||
}
|
||||
data, err := readCanonicalJSON(args[1])
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return write2DAFile(data, args[2])
|
||||
default:
|
||||
return fmt.Errorf("unknown convert-topdata subcommand %q", args[0])
|
||||
}
|
||||
}
|
||||
|
||||
func printConvertUsage(stdout io.Writer) {
|
||||
_, _ = fmt.Fprintln(stdout, "usage: convert-topdata <2da-to-json|2da-to-module|json-to-2da> ...")
|
||||
_, _ = fmt.Fprintln(stdout, "")
|
||||
_, _ = fmt.Fprintln(stdout, "subcommands:")
|
||||
_, _ = fmt.Fprintln(stdout, " 2da-to-json Convert a 2DA file into canonical JSON rows")
|
||||
_, _ = fmt.Fprintln(stdout, " 2da-to-module Convert a 2DA file into module entries or override JSON")
|
||||
_, _ = fmt.Fprintln(stdout, " json-to-2da Convert canonical JSON rows into a 2DA file")
|
||||
}
|
||||
|
||||
type parsed2DA struct {
|
||||
Columns []string `json:"columns"`
|
||||
Rows []map[string]any `json:"rows"`
|
||||
}
|
||||
|
||||
func parseConvertArgs(args []string, allowFormat bool) (convertOptions, string, string, error) {
|
||||
opts := convertOptions{
|
||||
CollisionMode: "prompt",
|
||||
Type: "entries",
|
||||
}
|
||||
positional := make([]string, 0, 2)
|
||||
for index := 0; index < len(args); index++ {
|
||||
arg := args[index]
|
||||
switch arg {
|
||||
case "--namespace":
|
||||
index++
|
||||
if index >= len(args) {
|
||||
return opts, "", "", errors.New("--namespace requires a value")
|
||||
}
|
||||
opts.Namespace = args[index]
|
||||
case "--key-field":
|
||||
index++
|
||||
if index >= len(args) {
|
||||
return opts, "", "", errors.New("--key-field requires a value")
|
||||
}
|
||||
opts.KeyFields = append(opts.KeyFields, args[index])
|
||||
case "--collision":
|
||||
index++
|
||||
if index >= len(args) {
|
||||
return opts, "", "", errors.New("--collision requires a value")
|
||||
}
|
||||
opts.CollisionMode = args[index]
|
||||
case "--base-dialog":
|
||||
index++
|
||||
if index >= len(args) {
|
||||
return opts, "", "", errors.New("--base-dialog requires a value")
|
||||
}
|
||||
opts.BaseDialog = args[index]
|
||||
case "--format", "--type":
|
||||
if !allowFormat {
|
||||
return opts, "", "", fmt.Errorf("%s does not accept %s", positionalSafe(args), arg)
|
||||
}
|
||||
index++
|
||||
if index >= len(args) {
|
||||
return opts, "", "", fmt.Errorf("%s requires a value", arg)
|
||||
}
|
||||
opts.Type = args[index]
|
||||
default:
|
||||
if strings.HasPrefix(arg, "--") {
|
||||
return opts, "", "", fmt.Errorf("unknown flag %q", arg)
|
||||
}
|
||||
positional = append(positional, arg)
|
||||
}
|
||||
}
|
||||
if allowFormat {
|
||||
if strings.TrimSpace(opts.Namespace) == "" {
|
||||
return opts, "", "", errors.New("2da-to-module requires --namespace")
|
||||
}
|
||||
if len(positional) < 1 || len(positional) > 2 {
|
||||
return opts, "", "", errors.New("usage: convert-topdata 2da-to-module --namespace <dataset> [--type entries|override] [flags] <input.2da> [output.json]")
|
||||
}
|
||||
} else if len(positional) != 2 {
|
||||
return opts, "", "", errors.New("usage: convert-topdata 2da-to-json [flags] <input.2da> <output.json>")
|
||||
}
|
||||
if opts.CollisionMode == "prompt" {
|
||||
opts.CollisionMode = "suffix"
|
||||
}
|
||||
if opts.CollisionMode != "suffix" && opts.CollisionMode != "error" {
|
||||
return opts, "", "", fmt.Errorf("unsupported collision mode %q", opts.CollisionMode)
|
||||
}
|
||||
if allowFormat {
|
||||
switch opts.Type {
|
||||
case "entry":
|
||||
opts.Type = "entries"
|
||||
case "overrides":
|
||||
opts.Type = "override"
|
||||
}
|
||||
if opts.Type != "entries" && opts.Type != "override" {
|
||||
return opts, "", "", fmt.Errorf("unsupported module type %q", opts.Type)
|
||||
}
|
||||
output := ""
|
||||
if len(positional) == 2 {
|
||||
output = positional[1]
|
||||
} else {
|
||||
resolved, err := defaultModuleOutputPath(positional[0], opts)
|
||||
if err != nil {
|
||||
return opts, "", "", err
|
||||
}
|
||||
output = resolved
|
||||
}
|
||||
return opts, positional[0], output, nil
|
||||
}
|
||||
return opts, positional[0], positional[1], nil
|
||||
}
|
||||
|
||||
func positionalSafe(args []string) string {
|
||||
if len(args) == 0 {
|
||||
return "command"
|
||||
}
|
||||
return args[0]
|
||||
}
|
||||
|
||||
func parse2DAFile(path string) (parsed2DA, error) {
|
||||
raw, err := os.ReadFile(path)
|
||||
if err != nil {
|
||||
return parsed2DA{}, err
|
||||
}
|
||||
lines := strings.Split(strings.ReplaceAll(string(raw), "\r\n", "\n"), "\n")
|
||||
trimmed := make([]string, 0, len(lines))
|
||||
for _, line := range lines {
|
||||
line = strings.TrimSpace(line)
|
||||
if line != "" {
|
||||
trimmed = append(trimmed, line)
|
||||
}
|
||||
}
|
||||
if len(trimmed) < 2 || !strings.HasPrefix(trimmed[0], "2DA") {
|
||||
return parsed2DA{}, errors.New("invalid 2DA file")
|
||||
}
|
||||
columns := split2DALine(trimmed[1])
|
||||
rows := make([]map[string]any, 0, max(0, len(trimmed)-2))
|
||||
for _, line := range trimmed[2:] {
|
||||
fields := split2DALine(line)
|
||||
if len(fields) == 0 {
|
||||
continue
|
||||
}
|
||||
rowID, err := strconv.Atoi(fields[0])
|
||||
if err != nil {
|
||||
return parsed2DA{}, fmt.Errorf("invalid 2DA row id %q", fields[0])
|
||||
}
|
||||
row := map[string]any{"id": rowID}
|
||||
for index, column := range columns {
|
||||
if index+1 < len(fields) {
|
||||
row[column] = smartConvertScalar(fields[index+1])
|
||||
continue
|
||||
}
|
||||
row[column] = nullValue
|
||||
}
|
||||
rows = append(rows, row)
|
||||
}
|
||||
return parsed2DA{Columns: columns, Rows: rows}, nil
|
||||
}
|
||||
|
||||
func split2DALine(line string) []string {
|
||||
fields := make([]string, 0)
|
||||
var current strings.Builder
|
||||
inQuotes := false
|
||||
for _, ch := range line {
|
||||
switch {
|
||||
case ch == '"':
|
||||
inQuotes = !inQuotes
|
||||
case !inQuotes && (ch == '\t' || ch == ' '):
|
||||
if current.Len() > 0 {
|
||||
fields = append(fields, current.String())
|
||||
current.Reset()
|
||||
}
|
||||
default:
|
||||
current.WriteRune(ch)
|
||||
}
|
||||
}
|
||||
if current.Len() > 0 {
|
||||
fields = append(fields, current.String())
|
||||
}
|
||||
return fields
|
||||
}
|
||||
|
||||
func smartConvertScalar(value string) any {
|
||||
if value == "" {
|
||||
return ""
|
||||
}
|
||||
if parsed, err := strconv.Atoi(value); err == nil {
|
||||
return parsed
|
||||
}
|
||||
return value
|
||||
}
|
||||
|
||||
func convert2DAToJSON(data parsed2DA, outputPath string, opts convertOptions) (map[string]any, error) {
|
||||
rows, err := assignConvertedRowKeys(data.Rows, outputPath, opts)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
outRows := make([]map[string]any, 0, len(rows))
|
||||
for _, row := range rows {
|
||||
ordered := map[string]any{"id": row["id"]}
|
||||
if key, ok := row["key"].(string); ok && key != "" {
|
||||
ordered["key"] = key
|
||||
}
|
||||
for _, column := range data.Columns {
|
||||
ordered[column] = row[column]
|
||||
}
|
||||
outRows = append(outRows, ordered)
|
||||
}
|
||||
return map[string]any{"columns": data.Columns, "rows": outRows}, nil
|
||||
}
|
||||
|
||||
func convert2DAToModule(data parsed2DA, outputPath string, opts convertOptions) (map[string]any, int, error) {
|
||||
rows, err := assignConvertedRowKeys(data.Rows, outputPath, opts)
|
||||
if err != nil {
|
||||
return nil, 0, err
|
||||
}
|
||||
entries := map[string]any{}
|
||||
skipped := 0
|
||||
for _, row := range rows {
|
||||
key, _ := row["key"].(string)
|
||||
if key == "" {
|
||||
skipped++
|
||||
continue
|
||||
}
|
||||
entry := map[string]any{}
|
||||
for _, column := range data.Columns {
|
||||
value := row[column]
|
||||
if format2DAValue(value) == nullValue {
|
||||
continue
|
||||
}
|
||||
entry[column] = value
|
||||
}
|
||||
entries[key] = entry
|
||||
}
|
||||
return map[string]any{"entries": entries}, skipped, nil
|
||||
}
|
||||
|
||||
func toOverridesRows(rows []map[string]any) []map[string]any {
|
||||
out := make([]map[string]any, 0, len(rows))
|
||||
for _, row := range rows {
|
||||
override := map[string]any{"id": row["id"]}
|
||||
keys := make([]string, 0, len(row))
|
||||
for key := range row {
|
||||
if key == "id" || key == "key" {
|
||||
continue
|
||||
}
|
||||
keys = append(keys, key)
|
||||
}
|
||||
sort.Strings(keys)
|
||||
for _, key := range keys {
|
||||
if format2DAValue(row[key]) == nullValue {
|
||||
continue
|
||||
}
|
||||
override[key] = row[key]
|
||||
}
|
||||
out = append(out, override)
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func assignConvertedRowKeys(rows []map[string]any, outputPath string, opts convertOptions) ([]map[string]any, error) {
|
||||
namespace, profiles := inferConvertNamespace(outputPath, opts.Namespace)
|
||||
out := make([]map[string]any, 0, len(rows))
|
||||
used := map[string]struct{}{}
|
||||
for _, row := range rows {
|
||||
cloned := cloneRowMap(row)
|
||||
if namespace == "" {
|
||||
out = append(out, cloned)
|
||||
continue
|
||||
}
|
||||
key := convertedRowKey(cloned, namespace, profiles, opts.KeyFields)
|
||||
if key == "" {
|
||||
out = append(out, cloned)
|
||||
continue
|
||||
}
|
||||
if _, ok := used[key]; ok {
|
||||
if opts.CollisionMode == "error" {
|
||||
return nil, fmt.Errorf("duplicate generated key %q", key)
|
||||
}
|
||||
key = fmt.Sprintf("%s_%v", key, cloned["id"])
|
||||
}
|
||||
used[key] = struct{}{}
|
||||
cloned["key"] = key
|
||||
out = append(out, cloned)
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
type convertKeyProfile struct {
|
||||
Field string
|
||||
Mode string
|
||||
}
|
||||
|
||||
func inferConvertNamespace(outputPath, override string) (string, []convertKeyProfile) {
|
||||
if strings.TrimSpace(override) != "" {
|
||||
return strings.TrimSpace(override), nil
|
||||
}
|
||||
normalized := filepath.ToSlash(outputPath)
|
||||
prefixes := []struct {
|
||||
Prefix string
|
||||
Namespace string
|
||||
Profiles []convertKeyProfile
|
||||
}{
|
||||
{"data/itemprops/costtables/index/", "itemprops:costtable_index", []convertKeyProfile{{"Name", "literal"}, {"Label", "literal"}}},
|
||||
{"data/itemprops/paramtables/index/", "itemprops:paramtable_index", []convertKeyProfile{{"Name", "literal"}, {"Lable", "literal"}, {"TableResRef", "literal"}}},
|
||||
{"data/itemprops/defs/", "itempropdef", []convertKeyProfile{{"Name", "literal"}, {"Label", "literal"}}},
|
||||
{"data/itemprops/sets/", "itemprops", []convertKeyProfile{{"StringRef", "literal"}, {"Label", "literal"}}},
|
||||
{"data/appearance/", "appearance", []convertKeyProfile{{"STRING_REF", "literal"}, {"LABEL", "literal"}}},
|
||||
{"data/baseitems/", "baseitems", []convertKeyProfile{{"Name", "literal"}, {"Label", "literal"}}},
|
||||
{"data/feat/", "feat", []convertKeyProfile{{"FEAT", "literal"}, {"LABEL", "literal"}, {"Label", "literal"}}},
|
||||
{"data/spells/", "spells", []convertKeyProfile{{"Name", "literal"}, {"Label", "literal"}}},
|
||||
{"data/skills/", "skills", []convertKeyProfile{{"Name", "literal"}, {"Label", "literal"}}},
|
||||
{"data/portraits/", "portraits", []convertKeyProfile{{"BaseResRef", "literal"}, {"Label", "literal"}}},
|
||||
{"data/racialtypes/", "racialtypes", []convertKeyProfile{{"Name", "literal"}, {"Label", "literal"}}},
|
||||
{"data/damagetypes/", "damagetypes", []convertKeyProfile{{"Name", "literal"}, {"Label", "literal"}}},
|
||||
}
|
||||
for _, prefix := range prefixes {
|
||||
if strings.Contains(normalized, prefix.Prefix) {
|
||||
return prefix.Namespace, prefix.Profiles
|
||||
}
|
||||
}
|
||||
return "", nil
|
||||
}
|
||||
|
||||
var convertKeyWhitespace = regexp.MustCompile(`\s+`)
|
||||
var convertKeyCleaner = regexp.MustCompile(`[^a-z0-9_]`)
|
||||
|
||||
func convertedRowKey(row map[string]any, namespace string, profiles []convertKeyProfile, preferred []string) string {
|
||||
candidates := make([]convertKeyProfile, 0, len(preferred)+len(profiles))
|
||||
for _, field := range preferred {
|
||||
candidates = append(candidates, convertKeyProfile{Field: field, Mode: "literal"})
|
||||
}
|
||||
candidates = append(candidates, profiles...)
|
||||
if len(candidates) == 0 {
|
||||
candidates = append(candidates,
|
||||
convertKeyProfile{Field: "LABEL", Mode: "literal"},
|
||||
convertKeyProfile{Field: "Label", Mode: "literal"},
|
||||
convertKeyProfile{Field: "Name", Mode: "literal"},
|
||||
)
|
||||
}
|
||||
for _, candidate := range candidates {
|
||||
value, ok := row[candidate.Field]
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
text := strings.TrimSpace(format2DAValue(value))
|
||||
if text == "" || text == nullValue {
|
||||
continue
|
||||
}
|
||||
normalized := normalizeConvertedKeyText(text)
|
||||
if normalized == "" {
|
||||
continue
|
||||
}
|
||||
return namespace + ":" + normalized
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
func normalizeConvertedKeyText(text string) string {
|
||||
text = strings.TrimSpace(strings.ToLower(text))
|
||||
text = convertKeyWhitespace.ReplaceAllString(text, "_")
|
||||
text = convertKeyCleaner.ReplaceAllString(text, "")
|
||||
text = strings.Trim(text, "_")
|
||||
return text
|
||||
}
|
||||
|
||||
func defaultModuleOutputPath(input string, opts convertOptions) (string, error) {
|
||||
cwd, err := os.Getwd()
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
root, err := project.FindRoot(cwd)
|
||||
if err != nil {
|
||||
return "", errors.New("2da-to-module requires an explicit output path when not run inside a project root")
|
||||
}
|
||||
p, err := project.Load(root)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
if !p.HasTopData() {
|
||||
return "", errors.New("2da-to-module requires topdata to be configured when inferring an output path")
|
||||
}
|
||||
filename := strings.TrimSuffix(filepath.Base(input), filepath.Ext(input)) + ".json"
|
||||
modulesDir := filepath.Join(p.TopDataSourceDir(), "data", filepath.FromSlash(opts.Namespace), "modules")
|
||||
return filepath.Join(modulesDir, filename), nil
|
||||
}
|
||||
|
||||
func readCanonicalJSON(path string) (parsed2DA, error) {
|
||||
raw, err := os.ReadFile(path)
|
||||
if err != nil {
|
||||
return parsed2DA{}, err
|
||||
}
|
||||
var payload struct {
|
||||
Columns []string `json:"columns"`
|
||||
Rows []map[string]any `json:"rows"`
|
||||
}
|
||||
if err := json.Unmarshal(raw, &payload); err != nil {
|
||||
return parsed2DA{}, err
|
||||
}
|
||||
return parsed2DA{Columns: payload.Columns, Rows: payload.Rows}, nil
|
||||
}
|
||||
|
||||
func writeJSONFile(path string, payload any) error {
|
||||
if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil && filepath.Dir(path) != "." {
|
||||
return err
|
||||
}
|
||||
raw, err := json.MarshalIndent(payload, "", " ")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
raw = append(raw, '\n')
|
||||
return os.WriteFile(path, raw, 0o644)
|
||||
}
|
||||
|
||||
func write2DAFile(data parsed2DA, path string) error {
|
||||
rows := make([]map[string]any, 0, len(data.Rows))
|
||||
for _, row := range data.Rows {
|
||||
cloned := cloneRowMap(row)
|
||||
if rowID, err := asInt(cloned["id"]); err == nil {
|
||||
cloned["id"] = rowID
|
||||
}
|
||||
rows = append(rows, cloned)
|
||||
}
|
||||
sort.Slice(rows, func(i, j int) bool {
|
||||
left, _ := asInt(rows[i]["id"])
|
||||
right, _ := asInt(rows[j]["id"])
|
||||
return left < right
|
||||
})
|
||||
table := map[string]any{
|
||||
"columns": data.Columns,
|
||||
"rows": rows,
|
||||
}
|
||||
if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil && filepath.Dir(path) != "." {
|
||||
return err
|
||||
}
|
||||
return write2DA(table, path, false)
|
||||
}
|
||||
@@ -0,0 +1,145 @@
|
||||
package topdata
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"os"
|
||||
"path/filepath"
|
||||
)
|
||||
|
||||
func importLegacyCreaturespeed(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
|
||||
_ = legacyTLK
|
||||
|
||||
legacyDir := filepath.Join(referenceBuilderDir, "data", "creaturespeed")
|
||||
if _, err := os.Stat(legacyDir); err != nil {
|
||||
if os.IsNotExist(err) {
|
||||
return 0, nil
|
||||
}
|
||||
return 0, err
|
||||
}
|
||||
|
||||
targetDir := filepath.Join(dataDir, "creaturespeed")
|
||||
targetPath := filepath.Join(targetDir, "creaturespeed.json")
|
||||
if _, err := os.Stat(targetPath); err == nil {
|
||||
obj, err := loadJSONObject(targetPath)
|
||||
if err == nil && countLegacyTLKRefsInValue(obj) == 0 {
|
||||
return 0, nil
|
||||
}
|
||||
}
|
||||
|
||||
baseObj, err := loadJSONObject(filepath.Join(legacyDir, "base.json"))
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
rows, err := mergeCreaturespeedRows(baseObj, filepath.Join(legacyDir, "modules"))
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
if err := os.MkdirAll(targetDir, 0o755); err != nil {
|
||||
return 0, err
|
||||
}
|
||||
|
||||
if err := removePathIfExists(filepath.Join(targetDir, "base.json")); err != nil {
|
||||
return 0, err
|
||||
}
|
||||
if err := removePathIfExists(filepath.Join(targetDir, "lock.json")); err != nil {
|
||||
return 0, err
|
||||
}
|
||||
if err := removePathIfExists(filepath.Join(targetDir, "modules")); err != nil {
|
||||
return 0, err
|
||||
}
|
||||
|
||||
out := map[string]any{
|
||||
"output": "creaturespeed.2da",
|
||||
"columns": baseObj["columns"],
|
||||
"rows": rows,
|
||||
}
|
||||
|
||||
raw, err := json.MarshalIndent(out, "", " ")
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
raw = append(raw, '\n')
|
||||
if err := os.WriteFile(targetPath, raw, 0o644); err != nil {
|
||||
return 0, err
|
||||
}
|
||||
return 1, nil
|
||||
}
|
||||
|
||||
func mergeCreaturespeedRows(baseObj map[string]any, modulesDir string) ([]any, error) {
|
||||
rawRows, ok := baseObj["rows"].([]any)
|
||||
if !ok {
|
||||
return nil, nil
|
||||
}
|
||||
rows := make([]map[string]any, 0, len(rawRows))
|
||||
byID := map[int]map[string]any{}
|
||||
for _, raw := range rawRows {
|
||||
row, ok := deepCopyValue(raw).(map[string]any)
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
delete(row, "key")
|
||||
rows = append(rows, row)
|
||||
if rawID, ok := row["id"]; ok {
|
||||
id, err := asInt(rawID)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
byID[id] = row
|
||||
}
|
||||
}
|
||||
|
||||
modulePaths, err := collectModulePaths(modulesDir)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
for _, path := range modulePaths {
|
||||
obj, err := loadJSONObject(path)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
overrideList, ok := obj["overrides"].([]any)
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
for _, raw := range overrideList {
|
||||
override, ok := raw.(map[string]any)
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
rawID, ok := override["id"]
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
id, err := asInt(rawID)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
row := byID[id]
|
||||
if row == nil {
|
||||
continue
|
||||
}
|
||||
for key, value := range override {
|
||||
if key == "id" || key == "key" || key == "_tlk" {
|
||||
continue
|
||||
}
|
||||
row[key] = deepCopyValue(value)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
out := make([]any, 0, len(rows))
|
||||
for _, row := range rows {
|
||||
out = append(out, row)
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
func removePathIfExists(path string) error {
|
||||
if _, err := os.Stat(path); err != nil {
|
||||
if os.IsNotExist(err) {
|
||||
return nil
|
||||
}
|
||||
return err
|
||||
}
|
||||
return os.RemoveAll(path)
|
||||
}
|
||||
@@ -0,0 +1,288 @@
|
||||
package topdata
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"sort"
|
||||
"strings"
|
||||
)
|
||||
|
||||
func importLegacyDamagetypes(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
|
||||
_ = legacyTLK
|
||||
|
||||
legacyRoot := filepath.Join(referenceBuilderDir, "data", "damagetypes")
|
||||
if _, err := os.Stat(legacyRoot); err != nil {
|
||||
if os.IsNotExist(err) {
|
||||
return 0, nil
|
||||
}
|
||||
return 0, err
|
||||
}
|
||||
|
||||
targetDir := filepath.Join(dataDir, "damagetypes", "registry")
|
||||
targetPath := filepath.Join(targetDir, "types.json")
|
||||
if _, err := os.Stat(targetPath); err == nil {
|
||||
obj, err := loadJSONObject(targetPath)
|
||||
if err == nil && countLegacyTLKRefsInValue(obj) == 0 {
|
||||
return 0, nil
|
||||
}
|
||||
}
|
||||
|
||||
coreBase, err := loadJSONObject(filepath.Join(legacyRoot, "core", "base.json"))
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
groupBase, err := loadJSONObject(filepath.Join(legacyRoot, "groups", "base.json"))
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
hitvisualBase, err := loadJSONObject(filepath.Join(legacyRoot, "hitvisual", "base.json"))
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
coreModule, err := loadJSONObject(filepath.Join(legacyRoot, "core", "modules", "add_eos.json"))
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
coreLock, err := loadLockfile(filepath.Join(legacyRoot, "core", "lock.json"))
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
groupModule, err := loadJSONObject(filepath.Join(legacyRoot, "groups", "modules", "add_eos.json"))
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
groupLock, err := loadLockfile(filepath.Join(legacyRoot, "groups", "lock.json"))
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
hitvisualModule, err := loadJSONObject(filepath.Join(legacyRoot, "hitvisual", "modules", "add_eos.json"))
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
hitvisualLock, err := loadLockfile(filepath.Join(legacyRoot, "hitvisual", "lock.json"))
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
|
||||
rows, lockData, err := mergeLegacyDamagetypes(coreBase, groupBase, hitvisualBase, coreModule, coreLock, groupModule, groupLock, hitvisualModule, hitvisualLock)
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
|
||||
if err := os.MkdirAll(targetDir, 0o755); err != nil {
|
||||
return 0, err
|
||||
}
|
||||
if err := removePathIfExists(filepath.Join(dataDir, "damagetypes", "core")); err != nil {
|
||||
return 0, err
|
||||
}
|
||||
if err := removePathIfExists(filepath.Join(dataDir, "damagetypes", "groups")); err != nil {
|
||||
return 0, err
|
||||
}
|
||||
if err := removePathIfExists(filepath.Join(dataDir, "damagetypes", "hitvisual")); err != nil {
|
||||
return 0, err
|
||||
}
|
||||
|
||||
typesOut := map[string]any{"rows": rows}
|
||||
raw, err := json.MarshalIndent(typesOut, "", " ")
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
raw = append(raw, '\n')
|
||||
if err := os.WriteFile(targetPath, raw, 0o644); err != nil {
|
||||
return 0, err
|
||||
}
|
||||
if err := saveLockfile(filepath.Join(targetDir, damagetypesRegistryLock), lockData); err != nil {
|
||||
return 0, err
|
||||
}
|
||||
return 2, nil
|
||||
}
|
||||
|
||||
func mergeLegacyDamagetypes(coreBase, groupBase, hitvisualBase, coreModule map[string]any, coreLock map[string]int, groupModule map[string]any, groupLock map[string]int, hitvisualModule map[string]any, hitvisualLock map[string]int) ([]map[string]any, map[string]int, error) {
|
||||
coreRows, err := coerceRowMapSlice(coreBase["rows"])
|
||||
if err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
groupRows, err := coerceRowMapSlice(groupBase["rows"])
|
||||
if err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
hitRows, err := coerceRowMapSlice(hitvisualBase["rows"])
|
||||
if err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
groupByID := map[int]map[string]any{}
|
||||
for _, row := range groupRows {
|
||||
id, err := asInt(row["id"])
|
||||
if err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
groupByID[id] = row
|
||||
}
|
||||
hitByID := map[int]map[string]any{}
|
||||
for _, row := range hitRows {
|
||||
id, err := asInt(row["id"])
|
||||
if err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
hitByID[id] = row
|
||||
}
|
||||
|
||||
outRows := make([]map[string]any, 0, len(coreRows))
|
||||
lockData := map[string]int{}
|
||||
for _, core := range coreRows {
|
||||
id, err := asInt(core["id"])
|
||||
if err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
groupID, err := asInt(core["DamageTypeGroup"])
|
||||
if err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
group := groupByID[groupID]
|
||||
if group == nil {
|
||||
return nil, nil, fmt.Errorf("core id %d references missing group %d", id, groupID)
|
||||
}
|
||||
hit := hitByID[id]
|
||||
if hit == nil {
|
||||
return nil, nil, fmt.Errorf("core id %d is missing hitvisual row", id)
|
||||
}
|
||||
key := "damagetype:" + normalizeDamagetypeKey(core["Label"])
|
||||
lockData[key] = id
|
||||
outRows = append(outRows, map[string]any{
|
||||
"key": key,
|
||||
"id": id,
|
||||
"label": deepCopyValue(core["Label"]),
|
||||
"charsheet_strref": deepCopyValue(core["CharsheetStrref"]),
|
||||
"damage_type_group": strconvString(groupID),
|
||||
"damage_ranged_projectile": deepCopyValue(core["DamageRangedProjectile"]),
|
||||
"group_label": deepCopyValue(group["Label"]),
|
||||
"feedback_strref": deepCopyValue(group["FeedbackStrref"]),
|
||||
"color_r": deepCopyValue(group["ColorR"]),
|
||||
"color_g": deepCopyValue(group["ColorG"]),
|
||||
"color_b": deepCopyValue(group["ColorB"]),
|
||||
"visual_effect_id": deepCopyValue(hit["VisualEffectID"]),
|
||||
"ranged_effect_id": deepCopyValue(hit["RangedEffectID"]),
|
||||
})
|
||||
}
|
||||
|
||||
coreEntries, err := coerceEntriesMap(coreModule["entries"])
|
||||
if err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
groupEntries, err := coerceEntriesMap(groupModule["entries"])
|
||||
if err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
hitEntries, err := coerceEntriesMap(hitvisualModule["entries"])
|
||||
if err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
|
||||
keys := make([]string, 0, len(coreEntries))
|
||||
for key := range coreEntries {
|
||||
keys = append(keys, key)
|
||||
}
|
||||
sort.Slice(keys, func(i, j int) bool {
|
||||
a := strings.TrimPrefix(keys[i], "damagetypes:")
|
||||
b := strings.TrimPrefix(keys[j], "damagetypes:")
|
||||
return a < b
|
||||
})
|
||||
for _, coreKey := range keys {
|
||||
core := coreEntries[coreKey]
|
||||
suffix := strings.TrimPrefix(coreKey, "damagetypes:")
|
||||
group := groupEntries["damagetypegroups:"+suffix]
|
||||
if group == nil {
|
||||
return nil, nil, fmt.Errorf("%s: missing matching group entry", coreKey)
|
||||
}
|
||||
hit := hitEntries["damagehitvisual:"+suffix]
|
||||
if hit == nil {
|
||||
return nil, nil, fmt.Errorf("%s: missing matching hitvisual entry", coreKey)
|
||||
}
|
||||
id, ok := coreLock[coreKey]
|
||||
if !ok {
|
||||
return nil, nil, fmt.Errorf("%s: missing core lock id", coreKey)
|
||||
}
|
||||
groupKey := "damagetypegroups:" + suffix
|
||||
groupID, ok := groupLock[groupKey]
|
||||
if !ok {
|
||||
return nil, nil, fmt.Errorf("%s: missing group lock id", groupKey)
|
||||
}
|
||||
hitKey := "damagehitvisual:" + suffix
|
||||
hitID, ok := hitvisualLock[hitKey]
|
||||
if !ok {
|
||||
return nil, nil, fmt.Errorf("%s: missing hitvisual lock id", hitKey)
|
||||
}
|
||||
if hitID != id {
|
||||
return nil, nil, fmt.Errorf("%s: core id %d does not match hitvisual id %d", suffix, id, hitID)
|
||||
}
|
||||
key := "damagetype:" + suffix
|
||||
lockData[key] = id
|
||||
outRows = append(outRows, map[string]any{
|
||||
"key": key,
|
||||
"id": id,
|
||||
"label": deepCopyValue(core["Label"]),
|
||||
"charsheet_strref": deepCopyValue(core["CharsheetStrref"]),
|
||||
"damage_type_group": strconvString(groupID),
|
||||
"damage_ranged_projectile": deepCopyValue(core["DamageRangedProjectile"]),
|
||||
"group_label": deepCopyValue(group["Label"]),
|
||||
"feedback_strref": deepCopyValue(group["FeedbackStrref"]),
|
||||
"color_r": deepCopyValue(group["ColorR"]),
|
||||
"color_g": deepCopyValue(group["ColorG"]),
|
||||
"color_b": deepCopyValue(group["ColorB"]),
|
||||
"visual_effect_id": deepCopyValue(hit["VisualEffectID"]),
|
||||
"ranged_effect_id": deepCopyValue(hit["RangedEffectID"]),
|
||||
})
|
||||
}
|
||||
|
||||
sort.Slice(outRows, func(i, j int) bool {
|
||||
return outRows[i]["id"].(int) < outRows[j]["id"].(int)
|
||||
})
|
||||
return outRows, lockData, nil
|
||||
}
|
||||
|
||||
func coerceRowMapSlice(value any) ([]map[string]any, error) {
|
||||
rawRows, ok := value.([]any)
|
||||
if !ok {
|
||||
return nil, fmt.Errorf("rows must be an array")
|
||||
}
|
||||
rows := make([]map[string]any, 0, len(rawRows))
|
||||
for _, raw := range rawRows {
|
||||
row, ok := raw.(map[string]any)
|
||||
if !ok {
|
||||
return nil, fmt.Errorf("row must be an object")
|
||||
}
|
||||
rows = append(rows, row)
|
||||
}
|
||||
return rows, nil
|
||||
}
|
||||
|
||||
func coerceEntriesMap(value any) (map[string]map[string]any, error) {
|
||||
rawEntries, ok := value.(map[string]any)
|
||||
if !ok {
|
||||
return nil, fmt.Errorf("entries must be an object")
|
||||
}
|
||||
entries := make(map[string]map[string]any, len(rawEntries))
|
||||
for key, raw := range rawEntries {
|
||||
row, ok := raw.(map[string]any)
|
||||
if !ok {
|
||||
return nil, fmt.Errorf("entry %s must be an object", key)
|
||||
}
|
||||
entries[key] = row
|
||||
}
|
||||
return entries, nil
|
||||
}
|
||||
|
||||
func normalizeDamagetypeKey(value any) string {
|
||||
text := strings.TrimSpace(strings.ToLower(format2DAValue(value)))
|
||||
text = strings.TrimSuffix(text, "damage")
|
||||
text = strings.ReplaceAll(text, " ", "")
|
||||
text = strings.ReplaceAll(text, "_", "")
|
||||
text = strings.ReplaceAll(text, "-", "")
|
||||
return text
|
||||
}
|
||||
|
||||
func strconvString(v int) string {
|
||||
return fmt.Sprintf("%d", v)
|
||||
}
|
||||
@@ -0,0 +1,243 @@
|
||||
package topdata
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"slices"
|
||||
"strconv"
|
||||
"strings"
|
||||
)
|
||||
|
||||
const (
|
||||
damagetypesRegistryDirName = "damagetypes/registry"
|
||||
damagetypesRegistryLock = "lock.json"
|
||||
)
|
||||
|
||||
type damagetypesRegistry struct {
|
||||
RootPath string
|
||||
LockPath string
|
||||
LockData map[string]int
|
||||
Types []map[string]any
|
||||
}
|
||||
|
||||
func collectGeneratedRegistryDatasets(dataDir string) ([]nativeCollectedDataset, error) {
|
||||
itempropsDatasets, err := collectItempropsRegistryDatasets(dataDir)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
damagetypeDatasets, err := collectDamagetypesRegistryDatasets(dataDir)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
racialtypesDatasets, err := collectRacialtypesRegistryDatasets(dataDir)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
out := make([]nativeCollectedDataset, 0, len(itempropsDatasets)+len(damagetypeDatasets)+len(racialtypesDatasets))
|
||||
out = append(out, itempropsDatasets...)
|
||||
out = append(out, damagetypeDatasets...)
|
||||
out = append(out, racialtypesDatasets...)
|
||||
return out, nil
|
||||
}
|
||||
|
||||
func loadDamagetypesRegistry(dataDir string) (*damagetypesRegistry, error) {
|
||||
root := filepath.Join(dataDir, filepath.FromSlash(damagetypesRegistryDirName))
|
||||
info, err := os.Stat(root)
|
||||
if err != nil {
|
||||
if os.IsNotExist(err) {
|
||||
return nil, nil
|
||||
}
|
||||
return nil, err
|
||||
}
|
||||
if !info.IsDir() {
|
||||
return nil, fmt.Errorf("damagetypes registry root must be a directory: %s", root)
|
||||
}
|
||||
|
||||
lockData, err := loadLockfile(filepath.Join(root, damagetypesRegistryLock))
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
types, err := loadRegistryRows(filepath.Join(root, "types.json"))
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return &damagetypesRegistry{
|
||||
RootPath: root,
|
||||
LockPath: filepath.Join(root, damagetypesRegistryLock),
|
||||
LockData: lockData,
|
||||
Types: types,
|
||||
}, nil
|
||||
}
|
||||
|
||||
func collectDamagetypesRegistryDatasets(dataDir string) ([]nativeCollectedDataset, error) {
|
||||
registry, err := loadDamagetypesRegistry(dataDir)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if registry == nil {
|
||||
return nil, nil
|
||||
}
|
||||
|
||||
typeIDs, lockModified, err := assignDamagetypeRegistryIDs(registry.Types, registry.LockData)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if lockModified {
|
||||
if err := saveLockfile(registry.LockPath, registry.LockData); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
|
||||
typeRowsSorted := slices.Clone(registry.Types)
|
||||
slices.SortFunc(typeRowsSorted, func(a, b map[string]any) int {
|
||||
return typeIDs[stringField(a, "key")] - typeIDs[stringField(b, "key")]
|
||||
})
|
||||
|
||||
coreRows := make([]map[string]any, 0, len(typeRowsSorted))
|
||||
hitvisualRows := make([]map[string]any, 0, len(typeRowsSorted))
|
||||
groupRowsByID := map[int]map[string]any{}
|
||||
groupOrder := make([]int, 0)
|
||||
for _, row := range typeRowsSorted {
|
||||
key := stringField(row, "key")
|
||||
rowID := typeIDs[key]
|
||||
|
||||
groupID, err := registryIntField(row, "damage_type_group")
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("%s: %w", key, err)
|
||||
}
|
||||
coreRows = append(coreRows, map[string]any{
|
||||
"id": rowID,
|
||||
"key": key,
|
||||
"Label": registryStringField(row, "label"),
|
||||
"CharsheetStrref": deepCopyValue(row["charsheet_strref"]),
|
||||
"DamageTypeGroup": strconv.Itoa(groupID),
|
||||
"DamageRangedProjectile": registryNullableField(row, "damage_ranged_projectile"),
|
||||
})
|
||||
hitvisualRows = append(hitvisualRows, map[string]any{
|
||||
"id": rowID,
|
||||
"key": strings.Replace(key, "damagetype:", "damagehitvisual:", 1),
|
||||
"Label": registryStringField(row, "label"),
|
||||
"VisualEffectID": registryNullableField(row, "visual_effect_id"),
|
||||
"RangedEffectID": registryNullableField(row, "ranged_effect_id"),
|
||||
})
|
||||
|
||||
if _, ok := groupRowsByID[groupID]; !ok {
|
||||
groupRowsByID[groupID] = map[string]any{
|
||||
"id": groupID,
|
||||
"key": strings.Replace(key, "damagetype:", "damagetypegroup:", 1),
|
||||
"Label": registryStringField(row, "group_label"),
|
||||
"FeedbackStrref": deepCopyValue(row["feedback_strref"]),
|
||||
"ColorR": registryNullableField(row, "color_r"),
|
||||
"ColorG": registryNullableField(row, "color_g"),
|
||||
"ColorB": registryNullableField(row, "color_b"),
|
||||
}
|
||||
groupOrder = append(groupOrder, groupID)
|
||||
continue
|
||||
}
|
||||
existing := groupRowsByID[groupID]
|
||||
for field, candidate := range map[string]any{
|
||||
"Label": registryStringField(row, "group_label"),
|
||||
"FeedbackStrref": deepCopyValue(row["feedback_strref"]),
|
||||
"ColorR": registryNullableField(row, "color_r"),
|
||||
"ColorG": registryNullableField(row, "color_g"),
|
||||
"ColorB": registryNullableField(row, "color_b"),
|
||||
} {
|
||||
if format2DAValue(existing[field]) != format2DAValue(candidate) {
|
||||
return nil, fmt.Errorf("%s: conflicting group projection for DamageTypeGroup %d field %s", key, groupID, field)
|
||||
}
|
||||
}
|
||||
}
|
||||
slices.Sort(groupOrder)
|
||||
groupRows := make([]map[string]any, 0, len(groupOrder))
|
||||
for _, id := range groupOrder {
|
||||
groupRows = append(groupRows, groupRowsByID[id])
|
||||
}
|
||||
|
||||
return []nativeCollectedDataset{
|
||||
newGeneratedDataset("damagetypes/registry/core", "damagetypes.2da", []string{"Label", "CharsheetStrref", "DamageTypeGroup", "DamageRangedProjectile"}, coreRows),
|
||||
newGeneratedDataset("damagetypes/registry/groups", "damagetypegroups.2da", []string{"Label", "FeedbackStrref", "ColorR", "ColorG", "ColorB"}, groupRows),
|
||||
newGeneratedDataset("damagetypes/registry/hitvisual", "damagehitvisual.2da", []string{"Label", "VisualEffectID", "RangedEffectID"}, hitvisualRows),
|
||||
}, nil
|
||||
}
|
||||
|
||||
func assignDamagetypeRegistryIDs(rows []map[string]any, lockData map[string]int) (map[string]int, bool, error) {
|
||||
ids := map[string]int{}
|
||||
used := map[int]struct{}{}
|
||||
for key, id := range lockData {
|
||||
if strings.HasPrefix(key, "damagetype:") {
|
||||
used[id] = struct{}{}
|
||||
}
|
||||
}
|
||||
modified := false
|
||||
for _, row := range rows {
|
||||
key := stringField(row, "key")
|
||||
if key == "" {
|
||||
return nil, false, fmt.Errorf("registry row is missing key")
|
||||
}
|
||||
if !strings.HasPrefix(key, "damagetype:") {
|
||||
return nil, false, fmt.Errorf("registry key %q must use prefix %q", key, "damagetype:")
|
||||
}
|
||||
if rawID, ok := row["id"]; ok {
|
||||
id, err := asInt(rawID)
|
||||
if err != nil {
|
||||
return nil, false, fmt.Errorf("%s: invalid id: %w", key, err)
|
||||
}
|
||||
if existing, ok := lockData[key]; ok && existing != id {
|
||||
return nil, false, fmt.Errorf("%s: explicit id %d does not match lock id %d", key, id, existing)
|
||||
}
|
||||
lockData[key] = id
|
||||
ids[key] = id
|
||||
used[id] = struct{}{}
|
||||
modified = true
|
||||
continue
|
||||
}
|
||||
if id, ok := lockData[key]; ok {
|
||||
ids[key] = id
|
||||
used[id] = struct{}{}
|
||||
}
|
||||
}
|
||||
nextID := nextAvailableID(used)
|
||||
for _, row := range rows {
|
||||
key := stringField(row, "key")
|
||||
if _, ok := ids[key]; ok {
|
||||
continue
|
||||
}
|
||||
lockData[key] = nextID
|
||||
ids[key] = nextID
|
||||
used[nextID] = struct{}{}
|
||||
nextID = nextAvailableID(used)
|
||||
modified = true
|
||||
}
|
||||
return ids, modified, nil
|
||||
}
|
||||
|
||||
func registryStringField(row map[string]any, field string) string {
|
||||
text := stringField(row, field)
|
||||
if text == "" {
|
||||
return nullValue
|
||||
}
|
||||
return text
|
||||
}
|
||||
|
||||
func registryNullableField(row map[string]any, field string) any {
|
||||
value, ok := row[field]
|
||||
if !ok {
|
||||
return nullValue
|
||||
}
|
||||
switch typed := value.(type) {
|
||||
case string:
|
||||
if strings.TrimSpace(typed) == "" {
|
||||
return nullValue
|
||||
}
|
||||
}
|
||||
return deepCopyValue(value)
|
||||
}
|
||||
|
||||
func registryIntField(row map[string]any, field string) (int, error) {
|
||||
value, ok := row[field]
|
||||
if !ok {
|
||||
return 0, fmt.Errorf("missing %s", field)
|
||||
}
|
||||
return asInt(value)
|
||||
}
|
||||
@@ -0,0 +1,99 @@
|
||||
package topdata
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"os"
|
||||
"path/filepath"
|
||||
)
|
||||
|
||||
func importLegacyDoortypes(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
|
||||
_ = legacyTLK
|
||||
|
||||
legacyDir := filepath.Join(referenceBuilderDir, "data", "doortypes")
|
||||
if _, err := os.Stat(legacyDir); err != nil {
|
||||
if os.IsNotExist(err) {
|
||||
return 0, nil
|
||||
}
|
||||
return 0, err
|
||||
}
|
||||
|
||||
targetDir := filepath.Join(dataDir, "doortypes")
|
||||
targetBasePath := filepath.Join(targetDir, "base.json")
|
||||
targetLockPath := filepath.Join(targetDir, "lock.json")
|
||||
targetModulesDir := filepath.Join(targetDir, "modules")
|
||||
moduleNames := []string{
|
||||
"add_pld01.json",
|
||||
"add_sic11.json",
|
||||
"add_tapr.json",
|
||||
"add_tdm01.json",
|
||||
"add_tdx01.json",
|
||||
"add_tei01.json",
|
||||
"add_tfm01.json",
|
||||
}
|
||||
targetModulePaths := make([]string, 0, len(moduleNames))
|
||||
for _, name := range moduleNames {
|
||||
targetModulePaths = append(targetModulePaths, filepath.Join(targetModulesDir, name))
|
||||
}
|
||||
if fileExists(targetBasePath) && fileExists(targetLockPath) {
|
||||
allModulesPresent := true
|
||||
for _, path := range targetModulePaths {
|
||||
if !fileExists(path) {
|
||||
allModulesPresent = false
|
||||
break
|
||||
}
|
||||
}
|
||||
if allModulesPresent {
|
||||
return 0, nil
|
||||
}
|
||||
}
|
||||
|
||||
baseObj, err := loadJSONObject(filepath.Join(legacyDir, "base.json"))
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
baseObj["output"] = "doortypes.2da"
|
||||
|
||||
lockObj, err := loadJSONObject(filepath.Join(legacyDir, "lock.json"))
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
|
||||
moduleObjs := make([]map[string]any, 0, len(moduleNames))
|
||||
for _, name := range moduleNames {
|
||||
obj, err := loadJSONObject(filepath.Join(legacyDir, "modules", name))
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
moduleObjs = append(moduleObjs, obj)
|
||||
}
|
||||
|
||||
if err := os.MkdirAll(targetModulesDir, 0o755); err != nil {
|
||||
return 0, err
|
||||
}
|
||||
|
||||
writes := []struct {
|
||||
path string
|
||||
obj map[string]any
|
||||
}{
|
||||
{path: targetBasePath, obj: baseObj},
|
||||
{path: targetLockPath, obj: lockObj},
|
||||
}
|
||||
for i, path := range targetModulePaths {
|
||||
writes = append(writes, struct {
|
||||
path string
|
||||
obj map[string]any
|
||||
}{path: path, obj: moduleObjs[i]})
|
||||
}
|
||||
|
||||
for _, write := range writes {
|
||||
raw, err := json.MarshalIndent(write.obj, "", " ")
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
raw = append(raw, '\n')
|
||||
if err := os.WriteFile(write.path, raw, 0o644); err != nil {
|
||||
return 0, err
|
||||
}
|
||||
}
|
||||
return len(writes), nil
|
||||
}
|
||||
@@ -0,0 +1,139 @@
|
||||
package topdata
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"strings"
|
||||
)
|
||||
|
||||
type familyIdentity struct {
|
||||
Parent string
|
||||
Child string
|
||||
}
|
||||
|
||||
type familyExpansionSource struct {
|
||||
Dataset string
|
||||
Column string
|
||||
Predicate string
|
||||
}
|
||||
|
||||
type familyExpansionSpec struct {
|
||||
Family string
|
||||
FamilyKey string
|
||||
Template string
|
||||
ChildSource familyExpansionSource
|
||||
}
|
||||
|
||||
func splitFamilyExpansionIdentity(text string) familyIdentity {
|
||||
text = strings.TrimSpace(text)
|
||||
if text == "" {
|
||||
return familyIdentity{}
|
||||
}
|
||||
if idx := strings.Index(text, "_"); idx > 0 && idx < len(text)-1 {
|
||||
return familyIdentity{
|
||||
Parent: text[:idx],
|
||||
Child: text[idx+1:],
|
||||
}
|
||||
}
|
||||
return familyIdentity{Parent: text}
|
||||
}
|
||||
|
||||
func parseFamilyExpansionSource(raw any) (familyExpansionSource, error) {
|
||||
if raw == nil {
|
||||
return familyExpansionSource{}, fmt.Errorf("child_source is required")
|
||||
}
|
||||
obj, ok := raw.(map[string]any)
|
||||
if !ok {
|
||||
return familyExpansionSource{}, fmt.Errorf("child_source must be an object")
|
||||
}
|
||||
dataset, ok := obj["dataset"].(string)
|
||||
if !ok || strings.TrimSpace(dataset) == "" {
|
||||
return familyExpansionSource{}, fmt.Errorf("child_source.dataset must be a non-empty string")
|
||||
}
|
||||
source := familyExpansionSource{
|
||||
Dataset: strings.TrimSpace(dataset),
|
||||
}
|
||||
if column, ok := obj["column"].(string); ok && strings.TrimSpace(column) != "" {
|
||||
source.Column = strings.TrimSpace(column)
|
||||
}
|
||||
if predicate, ok := obj["predicate"].(string); ok && strings.TrimSpace(predicate) != "" {
|
||||
source.Predicate = strings.TrimSpace(predicate)
|
||||
}
|
||||
return source, nil
|
||||
}
|
||||
|
||||
func parseFamilyExpansionSpec(path string, obj map[string]any) (familyExpansionSpec, error) {
|
||||
family, ok := obj["family"].(string)
|
||||
if !ok || strings.TrimSpace(family) == "" {
|
||||
return familyExpansionSpec{}, fmt.Errorf("generated file %s: family must be a non-empty string", path)
|
||||
}
|
||||
familyKey, ok := obj["family_key"].(string)
|
||||
if !ok || strings.TrimSpace(familyKey) == "" {
|
||||
return familyExpansionSpec{}, fmt.Errorf("generated file %s: family_key must be a non-empty string", path)
|
||||
}
|
||||
template, ok := obj["template"].(string)
|
||||
if !ok || strings.TrimSpace(template) == "" {
|
||||
return familyExpansionSpec{}, fmt.Errorf("generated file %s: template must be a non-empty string", path)
|
||||
}
|
||||
source, err := parseFamilyExpansionSource(obj["child_source"])
|
||||
if err != nil {
|
||||
return familyExpansionSpec{}, fmt.Errorf("generated file %s: %w", path, err)
|
||||
}
|
||||
return familyExpansionSpec{
|
||||
Family: strings.TrimSpace(family),
|
||||
FamilyKey: strings.TrimSpace(familyKey),
|
||||
Template: strings.TrimSpace(template),
|
||||
ChildSource: source,
|
||||
}, nil
|
||||
}
|
||||
|
||||
func familyMetadata(parent, child, source, template string) map[string]any {
|
||||
meta := map[string]any{
|
||||
"parent": strings.TrimSpace(parent),
|
||||
}
|
||||
if strings.TrimSpace(child) != "" {
|
||||
meta["child"] = strings.TrimSpace(child)
|
||||
}
|
||||
if strings.TrimSpace(source) != "" {
|
||||
meta["source"] = strings.TrimSpace(source)
|
||||
}
|
||||
if strings.TrimSpace(template) != "" {
|
||||
meta["template"] = strings.TrimSpace(template)
|
||||
}
|
||||
return map[string]any{"family": meta}
|
||||
}
|
||||
|
||||
func parseFamilyMetadata(raw any) (map[string]any, error) {
|
||||
obj, ok := raw.(map[string]any)
|
||||
if !ok {
|
||||
return nil, fmt.Errorf("must be an object")
|
||||
}
|
||||
parent, ok := obj["parent"].(string)
|
||||
if !ok || strings.TrimSpace(parent) == "" {
|
||||
return nil, fmt.Errorf("parent must be a non-empty string")
|
||||
}
|
||||
meta := map[string]any{
|
||||
"parent": strings.TrimSpace(parent),
|
||||
}
|
||||
if child, ok := obj["child"].(string); ok && strings.TrimSpace(child) != "" {
|
||||
meta["child"] = strings.TrimSpace(child)
|
||||
}
|
||||
if source, ok := obj["source"].(string); ok && strings.TrimSpace(source) != "" {
|
||||
meta["source"] = strings.TrimSpace(source)
|
||||
}
|
||||
if template, ok := obj["template"].(string); ok && strings.TrimSpace(template) != "" {
|
||||
meta["template"] = strings.TrimSpace(template)
|
||||
}
|
||||
return meta, nil
|
||||
}
|
||||
|
||||
func childTokenFromExpandedKey(key, parent string) string {
|
||||
key = strings.TrimSpace(key)
|
||||
key = strings.TrimPrefix(key, "feat:")
|
||||
if strings.HasPrefix(key, parent+"_") {
|
||||
return strings.TrimPrefix(key, parent+"_")
|
||||
}
|
||||
if strings.HasPrefix(key, parent) {
|
||||
return strings.TrimPrefix(key, parent)
|
||||
}
|
||||
return ""
|
||||
}
|
||||
@@ -0,0 +1,64 @@
|
||||
package topdata
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"os"
|
||||
"path/filepath"
|
||||
)
|
||||
|
||||
func importLegacyFeat(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
|
||||
_ = legacyTLK
|
||||
|
||||
legacyDir := filepath.Join(referenceBuilderDir, "data", "feat")
|
||||
if _, err := os.Stat(legacyDir); err != nil {
|
||||
if os.IsNotExist(err) {
|
||||
return 0, nil
|
||||
}
|
||||
return 0, err
|
||||
}
|
||||
legacyBasePath := filepath.Join(legacyDir, "base.json")
|
||||
if _, err := os.Stat(legacyBasePath); err != nil {
|
||||
if os.IsNotExist(err) {
|
||||
return 0, nil
|
||||
}
|
||||
return 0, err
|
||||
}
|
||||
|
||||
targetDir := filepath.Join(dataDir, "feat")
|
||||
targetBasePath := filepath.Join(targetDir, "base.json")
|
||||
var targetObj map[string]any
|
||||
if _, err := os.Stat(targetBasePath); err == nil {
|
||||
targetObj, err = loadJSONObject(targetBasePath)
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
} else if !os.IsNotExist(err) {
|
||||
return 0, err
|
||||
}
|
||||
if targetObj != nil {
|
||||
if rows, ok := targetObj["rows"].([]any); ok && len(rows) > 0 {
|
||||
return 0, nil
|
||||
}
|
||||
}
|
||||
|
||||
baseObj, err := loadJSONObject(legacyBasePath)
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
baseObj["output"] = "feat.2da"
|
||||
baseObj["compare_reference"] = false
|
||||
canonicalizeWikiMetadataDocument(baseObj)
|
||||
|
||||
if err := os.MkdirAll(targetDir, 0o755); err != nil {
|
||||
return 0, err
|
||||
}
|
||||
raw, err := json.MarshalIndent(baseObj, "", " ")
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
raw = append(raw, '\n')
|
||||
if err := os.WriteFile(targetBasePath, raw, 0o644); err != nil {
|
||||
return 0, err
|
||||
}
|
||||
return 1, nil
|
||||
}
|
||||
@@ -0,0 +1,6 @@
|
||||
package topdata
|
||||
|
||||
func importLegacyGenericdoors(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
|
||||
_ = legacyTLK
|
||||
return importLegacyDatasetMirror(referenceBuilderDir, dataDir, "genericdoors", "genericdoors.2da", nil)
|
||||
}
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,176 @@
|
||||
package topdata
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"io/fs"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"sort"
|
||||
"strings"
|
||||
)
|
||||
|
||||
type legacyDatasetTransform func(relativePath string, obj map[string]any)
|
||||
|
||||
func importLegacyDatasetMirror(referenceBuilderDir, dataDir, datasetName, outputName string, transform legacyDatasetTransform) (int, error) {
|
||||
legacyDir := filepath.Join(referenceBuilderDir, "data", datasetName)
|
||||
if _, err := os.Stat(legacyDir); err != nil {
|
||||
if os.IsNotExist(err) {
|
||||
return 0, nil
|
||||
}
|
||||
return 0, err
|
||||
}
|
||||
|
||||
targetDir := filepath.Join(dataDir, datasetName)
|
||||
baseObj, err := loadJSONObject(filepath.Join(legacyDir, "base.json"))
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
if outputName != "" {
|
||||
baseObj["output"] = outputName
|
||||
}
|
||||
if transform != nil {
|
||||
transform("base.json", baseObj)
|
||||
}
|
||||
|
||||
lockObj, err := loadJSONObject(filepath.Join(legacyDir, "lock.json"))
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
if transform != nil {
|
||||
transform("lock.json", lockObj)
|
||||
}
|
||||
|
||||
moduleRelPaths, err := collectJSONRelativePaths(filepath.Join(legacyDir, "modules"))
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
|
||||
moduleObjs := make(map[string]map[string]any, len(moduleRelPaths))
|
||||
for _, relPath := range moduleRelPaths {
|
||||
obj, err := loadJSONObject(filepath.Join(legacyDir, "modules", relPath))
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
normalizedRel := filepath.ToSlash(relPath)
|
||||
if transform != nil {
|
||||
transform(filepath.ToSlash(filepath.Join("modules", normalizedRel)), obj)
|
||||
}
|
||||
moduleObjs[normalizedRel] = obj
|
||||
}
|
||||
|
||||
if err := os.MkdirAll(targetDir, 0o755); err != nil {
|
||||
return 0, err
|
||||
}
|
||||
|
||||
updated := 0
|
||||
for _, write := range []struct {
|
||||
path string
|
||||
obj map[string]any
|
||||
}{
|
||||
{path: filepath.Join(targetDir, "base.json"), obj: baseObj},
|
||||
{path: filepath.Join(targetDir, "lock.json"), obj: lockObj},
|
||||
} {
|
||||
changed, err := writeJSONObjectIfChanged(write.path, write.obj)
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
if changed {
|
||||
updated++
|
||||
}
|
||||
}
|
||||
|
||||
targetModulesDir := filepath.Join(targetDir, "modules")
|
||||
if len(moduleObjs) > 0 {
|
||||
if err := os.MkdirAll(targetModulesDir, 0o755); err != nil {
|
||||
return 0, err
|
||||
}
|
||||
}
|
||||
for relPath, obj := range moduleObjs {
|
||||
targetPath := filepath.Join(targetModulesDir, filepath.FromSlash(relPath))
|
||||
if err := os.MkdirAll(filepath.Dir(targetPath), 0o755); err != nil {
|
||||
return 0, err
|
||||
}
|
||||
changed, err := writeJSONObjectIfChanged(targetPath, obj)
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
if changed {
|
||||
updated++
|
||||
}
|
||||
}
|
||||
|
||||
staleModules, err := collectJSONRelativePaths(targetModulesDir)
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
for _, relPath := range staleModules {
|
||||
normalizedRel := filepath.ToSlash(relPath)
|
||||
if _, ok := moduleObjs[normalizedRel]; ok {
|
||||
continue
|
||||
}
|
||||
if err := os.Remove(filepath.Join(targetModulesDir, filepath.FromSlash(normalizedRel))); err != nil && !os.IsNotExist(err) {
|
||||
return 0, err
|
||||
}
|
||||
updated++
|
||||
}
|
||||
|
||||
return updated, nil
|
||||
}
|
||||
|
||||
func collectJSONRelativePaths(root string) ([]string, error) {
|
||||
if _, err := os.Stat(root); err != nil {
|
||||
if os.IsNotExist(err) {
|
||||
return nil, nil
|
||||
}
|
||||
return nil, err
|
||||
}
|
||||
|
||||
var relPaths []string
|
||||
err := filepath.WalkDir(root, func(path string, d fs.DirEntry, err error) error {
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if d.IsDir() {
|
||||
return nil
|
||||
}
|
||||
if !strings.EqualFold(filepath.Ext(d.Name()), ".json") {
|
||||
return nil
|
||||
}
|
||||
relPath, err := filepath.Rel(root, path)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
relPaths = append(relPaths, filepath.ToSlash(relPath))
|
||||
return nil
|
||||
})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
sort.Strings(relPaths)
|
||||
return relPaths, nil
|
||||
}
|
||||
|
||||
func writeJSONObjectIfChanged(path string, obj map[string]any) (bool, error) {
|
||||
raw, err := json.MarshalIndent(obj, "", " ")
|
||||
if err != nil {
|
||||
return false, err
|
||||
}
|
||||
raw = append(raw, '\n')
|
||||
|
||||
current, err := os.ReadFile(path)
|
||||
if err == nil && string(current) == string(raw) {
|
||||
return false, nil
|
||||
}
|
||||
if err != nil && !os.IsNotExist(err) {
|
||||
return false, err
|
||||
}
|
||||
if err := os.WriteFile(path, raw, 0o644); err != nil {
|
||||
return false, err
|
||||
}
|
||||
return true, nil
|
||||
}
|
||||
|
||||
func fileExists(path string) bool {
|
||||
_, err := os.Stat(path)
|
||||
return err == nil
|
||||
}
|
||||
@@ -0,0 +1,6 @@
|
||||
package topdata
|
||||
|
||||
func importLegacyLoadscreens(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
|
||||
_ = legacyTLK
|
||||
return importLegacyDatasetMirror(referenceBuilderDir, dataDir, "loadscreens", "loadscreens.2da", nil)
|
||||
}
|
||||
@@ -0,0 +1,237 @@
|
||||
package topdata
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strconv"
|
||||
)
|
||||
|
||||
func importLegacyMasterfeats(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
|
||||
legacyDir := filepath.Join(referenceBuilderDir, "data", "feat", "masterfeats")
|
||||
if _, err := os.Stat(legacyDir); err != nil {
|
||||
if os.IsNotExist(err) {
|
||||
return 0, nil
|
||||
}
|
||||
return 0, err
|
||||
}
|
||||
|
||||
targetDir := filepath.Join(dataDir, "masterfeats")
|
||||
targetPath := filepath.Join(targetDir, "base.json")
|
||||
legacyPlainPath := filepath.Join(targetDir, "masterfeats.json")
|
||||
if _, err := os.Stat(targetPath); err == nil {
|
||||
obj, err := loadJSONObject(targetPath)
|
||||
if err == nil && countLegacyTLKRefsInValue(obj) == 0 {
|
||||
return 0, nil
|
||||
}
|
||||
}
|
||||
|
||||
collected, err := importLegacyMasterfeatsDataset(legacyDir)
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
referenceIDs, err := collectReferenceLockIDs(filepath.Join(referenceBuilderDir, "data"))
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
|
||||
rows := make([]any, 0, len(collected.Rows))
|
||||
for _, row := range collected.Rows {
|
||||
copyRow, ok := deepCopyValue(row).(map[string]any)
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
if legacyTLK != nil {
|
||||
if _, _, err := inlineLegacyTLKValue(copyRow, legacyTLK); err != nil {
|
||||
return 0, err
|
||||
}
|
||||
}
|
||||
copyRow = rewriteImportedIDRefs(copyRow, referenceIDs)
|
||||
rows = append(rows, copyRow)
|
||||
}
|
||||
|
||||
if err := os.MkdirAll(targetDir, 0o755); err != nil {
|
||||
return 0, err
|
||||
}
|
||||
writes := []struct {
|
||||
path string
|
||||
obj map[string]any
|
||||
}{
|
||||
{
|
||||
path: targetPath,
|
||||
obj: map[string]any{
|
||||
"output": collected.Dataset.OutputName,
|
||||
"columns": stringSliceToAny(collected.Columns),
|
||||
"rows": rows,
|
||||
},
|
||||
},
|
||||
{
|
||||
path: filepath.Join(targetDir, "lock.json"),
|
||||
obj: anyMapInt(collected.LockData),
|
||||
},
|
||||
}
|
||||
for _, write := range writes {
|
||||
raw, err := json.MarshalIndent(write.obj, "", " ")
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
raw = append(raw, '\n')
|
||||
if err := os.WriteFile(write.path, raw, 0o644); err != nil {
|
||||
return 0, err
|
||||
}
|
||||
}
|
||||
updatedFiles := len(writes)
|
||||
if err := os.Remove(legacyPlainPath); err == nil {
|
||||
updatedFiles++
|
||||
} else if err != nil && !os.IsNotExist(err) {
|
||||
return 0, err
|
||||
}
|
||||
return updatedFiles, nil
|
||||
}
|
||||
|
||||
func importLegacyMasterfeatsDataset(legacyDir string) (nativeCollectedDataset, error) {
|
||||
dataset := nativeDataset{
|
||||
Name: "masterfeats",
|
||||
BasePath: filepath.Join(legacyDir, "masterfeats.json"),
|
||||
LockPath: filepath.Join(legacyDir, "lock.json"),
|
||||
Spec: specForDataset("masterfeats"),
|
||||
}
|
||||
tableData, err := loadJSONObject(dataset.BasePath)
|
||||
if err != nil {
|
||||
return nativeCollectedDataset{}, err
|
||||
}
|
||||
columns, err := parseColumns(tableData, dataset.Name)
|
||||
if err != nil {
|
||||
return nativeCollectedDataset{}, err
|
||||
}
|
||||
rawRows, ok := tableData["rows"].([]any)
|
||||
if !ok {
|
||||
return nativeCollectedDataset{}, nil
|
||||
}
|
||||
lockData, err := loadLockfile(dataset.LockPath)
|
||||
if err != nil {
|
||||
return nativeCollectedDataset{}, err
|
||||
}
|
||||
usedIDs := map[int]struct{}{}
|
||||
for _, rowID := range lockData {
|
||||
usedIDs[rowID] = struct{}{}
|
||||
}
|
||||
|
||||
rows := make([]map[string]any, 0, len(rawRows))
|
||||
explicitID := make([]bool, 0, len(rawRows))
|
||||
for index, raw := range rawRows {
|
||||
rowObj, ok := raw.(map[string]any)
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
rowID := index
|
||||
if value, ok := rowObj["id"]; ok {
|
||||
parsed, err := asInt(value)
|
||||
if err != nil {
|
||||
return nativeCollectedDataset{}, err
|
||||
}
|
||||
rowID = parsed
|
||||
}
|
||||
row := map[string]any{"id": rowID}
|
||||
for _, column := range columns {
|
||||
row[column] = nullValue
|
||||
}
|
||||
for key, value := range rowObj {
|
||||
switch key {
|
||||
case "id":
|
||||
case "key":
|
||||
if keyText, ok := value.(string); ok && keyText != "" {
|
||||
row["key"] = keyText
|
||||
}
|
||||
default:
|
||||
columnName, ok := canonicalColumn(columns, key)
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
row[columnName] = value
|
||||
}
|
||||
}
|
||||
rows = append(rows, row)
|
||||
_, hasExplicitID := rowObj["id"]
|
||||
explicitID = append(explicitID, hasExplicitID)
|
||||
if hasExplicitID {
|
||||
usedIDs[rowID] = struct{}{}
|
||||
}
|
||||
}
|
||||
|
||||
nextID := nextAvailableID(usedIDs)
|
||||
lockModified := false
|
||||
for index, row := range rows {
|
||||
key, hasKey := row["key"].(string)
|
||||
if hasKey && key != "" {
|
||||
if lockedID, ok := lockData[key]; ok {
|
||||
row["id"] = lockedID
|
||||
continue
|
||||
}
|
||||
if explicitID[index] {
|
||||
lockData[key] = row["id"].(int)
|
||||
} else {
|
||||
row["id"] = nextID
|
||||
lockData[key] = nextID
|
||||
usedIDs[nextID] = struct{}{}
|
||||
nextID = nextAvailableID(usedIDs)
|
||||
}
|
||||
lockModified = true
|
||||
continue
|
||||
}
|
||||
if !explicitID[index] {
|
||||
row["id"] = index
|
||||
}
|
||||
}
|
||||
if lockModified {
|
||||
_ = lockModified
|
||||
}
|
||||
|
||||
return nativeCollectedDataset{
|
||||
Dataset: nativeDataset{
|
||||
Name: dataset.Name,
|
||||
OutputName: "masterfeats.2da",
|
||||
Columns: columns,
|
||||
Spec: dataset.Spec,
|
||||
},
|
||||
Columns: columns,
|
||||
Rows: rows,
|
||||
LockData: lockData,
|
||||
}, nil
|
||||
}
|
||||
|
||||
func rewriteImportedIDRefs(value any, ids map[string]int) map[string]any {
|
||||
row, ok := rewriteImportedIDRefsValue(value, ids).(map[string]any)
|
||||
if !ok {
|
||||
return map[string]any{}
|
||||
}
|
||||
return row
|
||||
}
|
||||
|
||||
func rewriteImportedIDRefsValue(value any, ids map[string]int) any {
|
||||
switch typed := value.(type) {
|
||||
case map[string]any:
|
||||
if len(typed) == 1 {
|
||||
if rawID, ok := typed["id"]; ok {
|
||||
if key, ok := rawID.(string); ok {
|
||||
if resolved, ok := ids[key]; ok {
|
||||
return strconv.Itoa(resolved)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
out := make(map[string]any, len(typed))
|
||||
for key, child := range typed {
|
||||
out[key] = rewriteImportedIDRefsValue(child, ids)
|
||||
}
|
||||
return out
|
||||
case []any:
|
||||
out := make([]any, 0, len(typed))
|
||||
for _, child := range typed {
|
||||
out = append(out, rewriteImportedIDRefsValue(child, ids))
|
||||
}
|
||||
return out
|
||||
default:
|
||||
return value
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,188 @@
|
||||
package topdata
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"strings"
|
||||
)
|
||||
|
||||
func isNullLike(value any) bool {
|
||||
if value == nil {
|
||||
return true
|
||||
}
|
||||
text, ok := value.(string)
|
||||
if !ok {
|
||||
return false
|
||||
}
|
||||
return strings.TrimSpace(text) == nullValue
|
||||
}
|
||||
|
||||
func normalizeWikiMetadataKey(key string) string {
|
||||
switch normalizeMetadataKey(key) {
|
||||
case "reqfeats", "req_feats":
|
||||
return "reqfeats"
|
||||
case "generate", "wikigenerate":
|
||||
return "generate"
|
||||
case "canonical", "wikicanonical":
|
||||
return "canonical"
|
||||
case "status", "wikistatus":
|
||||
return "status"
|
||||
default:
|
||||
return ""
|
||||
}
|
||||
}
|
||||
|
||||
func legacyWikiMetadataField(key string) string {
|
||||
switch normalizeMetadataKey(key) {
|
||||
case "wikireqfeats":
|
||||
return "reqfeats"
|
||||
case "wikigenerate":
|
||||
return "generate"
|
||||
case "wikicanonical":
|
||||
return "canonical"
|
||||
case "wikistatus":
|
||||
return "status"
|
||||
default:
|
||||
return ""
|
||||
}
|
||||
}
|
||||
|
||||
func parseWikiMetadata(raw any) (map[string]any, error) {
|
||||
obj, ok := raw.(map[string]any)
|
||||
if !ok {
|
||||
return nil, fmt.Errorf("must be an object")
|
||||
}
|
||||
wiki := map[string]any{}
|
||||
for key, value := range obj {
|
||||
normalized := normalizeWikiMetadataKey(key)
|
||||
if normalized == "" {
|
||||
return nil, fmt.Errorf("unknown field %q", key)
|
||||
}
|
||||
if isNullLike(value) {
|
||||
continue
|
||||
}
|
||||
wiki[normalized] = cloneAuthoringValue(value)
|
||||
}
|
||||
return wiki, nil
|
||||
}
|
||||
|
||||
func mergeMetadataMaps(base, overlay map[string]any) map[string]any {
|
||||
if len(base) == 0 && len(overlay) == 0 {
|
||||
return nil
|
||||
}
|
||||
out := map[string]any{}
|
||||
for key, value := range base {
|
||||
out[key] = cloneAuthoringValue(value)
|
||||
}
|
||||
for key, value := range overlay {
|
||||
baseValue, hasBase := out[key]
|
||||
baseMap, baseIsMap := baseValue.(map[string]any)
|
||||
valueMap, valueIsMap := value.(map[string]any)
|
||||
if hasBase && baseIsMap && valueIsMap {
|
||||
out[key] = mergeMetadataMaps(baseMap, valueMap)
|
||||
continue
|
||||
}
|
||||
out[key] = cloneAuthoringValue(value)
|
||||
}
|
||||
if len(out) == 0 {
|
||||
return nil
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func parseExistingMetadata(row map[string]any) (map[string]any, error) {
|
||||
rawMeta, ok := lookupField(row, "meta")
|
||||
if !ok {
|
||||
return nil, nil
|
||||
}
|
||||
return parseRowMetadata(rawMeta)
|
||||
}
|
||||
|
||||
func setLegacyWikiMetadata(row map[string]any, field string, value any) error {
|
||||
wikiKey := legacyWikiMetadataField(field)
|
||||
if wikiKey == "" {
|
||||
return nil
|
||||
}
|
||||
|
||||
meta, err := parseExistingMetadata(row)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if meta == nil {
|
||||
meta = map[string]any{}
|
||||
}
|
||||
|
||||
wiki, _ := meta["wiki"].(map[string]any)
|
||||
if wiki == nil {
|
||||
wiki = map[string]any{}
|
||||
}
|
||||
|
||||
if isNullLike(value) {
|
||||
delete(wiki, wikiKey)
|
||||
} else {
|
||||
wiki[wikiKey] = cloneAuthoringValue(value)
|
||||
}
|
||||
|
||||
if len(wiki) == 0 {
|
||||
delete(meta, "wiki")
|
||||
} else {
|
||||
meta["wiki"] = wiki
|
||||
}
|
||||
if len(meta) == 0 {
|
||||
delete(row, "meta")
|
||||
} else {
|
||||
row["meta"] = meta
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func canonicalizeWikiMetadataDocument(obj map[string]any) {
|
||||
if rawColumns, ok := obj["columns"].([]any); ok {
|
||||
filtered := make([]any, 0, len(rawColumns))
|
||||
for _, raw := range rawColumns {
|
||||
column, ok := raw.(string)
|
||||
if ok && isAuthoringOnlyField(column) {
|
||||
continue
|
||||
}
|
||||
filtered = append(filtered, raw)
|
||||
}
|
||||
obj["columns"] = filtered
|
||||
}
|
||||
if rows, ok := obj["rows"].([]any); ok {
|
||||
for _, raw := range rows {
|
||||
row, ok := raw.(map[string]any)
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
canonicalizeWikiMetadataRow(row)
|
||||
}
|
||||
}
|
||||
if entries, ok := obj["entries"].(map[string]any); ok {
|
||||
for _, raw := range entries {
|
||||
row, ok := raw.(map[string]any)
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
canonicalizeWikiMetadataRow(row)
|
||||
}
|
||||
}
|
||||
if overrides, ok := obj["overrides"].([]any); ok {
|
||||
for _, raw := range overrides {
|
||||
row, ok := raw.(map[string]any)
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
canonicalizeWikiMetadataRow(row)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func canonicalizeWikiMetadataRow(row map[string]any) {
|
||||
for _, field := range []string{"WIKIREQFEATS", "WIKIGENERATE", "WIKICANONICAL", "WIKISTATUS"} {
|
||||
value, ok := row[field]
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
_ = setLegacyWikiMetadata(row, field, value)
|
||||
delete(row, field)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,415 @@
|
||||
package topdata
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
|
||||
"gitea.westgate.pw/ShadowsOverWestgate/sow-tools/internal/project"
|
||||
)
|
||||
|
||||
type NormalizeResult struct {
|
||||
StatePath string
|
||||
ScannedFiles int
|
||||
UpdatedFiles int
|
||||
InlineValues int
|
||||
RemainingFiles int
|
||||
RemainingLegacyRefs int
|
||||
}
|
||||
|
||||
func NormalizeProject(p *project.Project) (NormalizeResult, error) {
|
||||
if !p.HasTopData() {
|
||||
return NormalizeResult{}, fmt.Errorf("topdata is not configured for this project")
|
||||
}
|
||||
|
||||
sourceDir := p.TopDataSourceDir()
|
||||
dataDir := filepath.Join(sourceDir, "data")
|
||||
migrationRoot := resolveMigrationInputRoot(sourceDir, p.TopDataReferenceBuilderDir())
|
||||
legacyTLK, err := loadLegacyTLK(filepath.Join(sourceDir, "tlk"))
|
||||
if err != nil {
|
||||
return NormalizeResult{}, err
|
||||
}
|
||||
if migrationRoot != "" {
|
||||
migrationTLK, err := loadLegacyTLK(filepath.Join(migrationRoot, "tlk"))
|
||||
if err != nil {
|
||||
return NormalizeResult{}, err
|
||||
}
|
||||
legacyTLK = mergeLegacyTLK(migrationTLK, legacyTLK)
|
||||
}
|
||||
baseDialogImported, err := importLegacyBaseDialog(migrationRoot, sourceDir)
|
||||
if err != nil {
|
||||
return NormalizeResult{}, err
|
||||
}
|
||||
|
||||
itempropsImported, err := importLegacyItempropsRegistry(migrationRoot, dataDir, legacyTLK)
|
||||
if err != nil {
|
||||
return NormalizeResult{}, err
|
||||
}
|
||||
masterfeatsImported, err := importLegacyMasterfeats(migrationRoot, dataDir, legacyTLK)
|
||||
if err != nil {
|
||||
return NormalizeResult{}, err
|
||||
}
|
||||
featImported, err := importLegacyFeat(migrationRoot, dataDir, legacyTLK)
|
||||
if err != nil {
|
||||
return NormalizeResult{}, err
|
||||
}
|
||||
creaturespeedImported, err := importLegacyCreaturespeed(migrationRoot, dataDir, legacyTLK)
|
||||
if err != nil {
|
||||
return NormalizeResult{}, err
|
||||
}
|
||||
armorImported, err := importLegacyArmor(migrationRoot, dataDir, legacyTLK)
|
||||
if err != nil {
|
||||
return NormalizeResult{}, err
|
||||
}
|
||||
baseitemsImported, err := importLegacyBaseitems(migrationRoot, dataDir, legacyTLK)
|
||||
if err != nil {
|
||||
return NormalizeResult{}, err
|
||||
}
|
||||
damagetypesImported, err := importLegacyDamagetypes(migrationRoot, dataDir, legacyTLK)
|
||||
if err != nil {
|
||||
return NormalizeResult{}, err
|
||||
}
|
||||
skyboxesImported, err := importLegacySkyboxes(migrationRoot, dataDir, legacyTLK)
|
||||
if err != nil {
|
||||
return NormalizeResult{}, err
|
||||
}
|
||||
vfxPersistentImported, err := importLegacyVFXPersistent(migrationRoot, dataDir, legacyTLK)
|
||||
if err != nil {
|
||||
return NormalizeResult{}, err
|
||||
}
|
||||
loadscreensImported, err := importLegacyLoadscreens(migrationRoot, dataDir, legacyTLK)
|
||||
if err != nil {
|
||||
return NormalizeResult{}, err
|
||||
}
|
||||
progfxImported, err := importLegacyProgfx(migrationRoot, dataDir, legacyTLK)
|
||||
if err != nil {
|
||||
return NormalizeResult{}, err
|
||||
}
|
||||
genericdoorsImported, err := importLegacyGenericdoors(migrationRoot, dataDir, legacyTLK)
|
||||
if err != nil {
|
||||
return NormalizeResult{}, err
|
||||
}
|
||||
cloakmodelImported, err := importLegacyCloakmodel(migrationRoot, dataDir, legacyTLK)
|
||||
if err != nil {
|
||||
return NormalizeResult{}, err
|
||||
}
|
||||
repadjustImported, err := importLegacyRepadjust(migrationRoot, dataDir, legacyTLK)
|
||||
if err != nil {
|
||||
return NormalizeResult{}, err
|
||||
}
|
||||
wingmodelImported, err := importLegacyWingmodel(migrationRoot, dataDir, legacyTLK)
|
||||
if err != nil {
|
||||
return NormalizeResult{}, err
|
||||
}
|
||||
tailmodelImported, err := importLegacyTailmodel(migrationRoot, dataDir, legacyTLK)
|
||||
if err != nil {
|
||||
return NormalizeResult{}, err
|
||||
}
|
||||
doortypesImported, err := importLegacyDoortypes(migrationRoot, dataDir, legacyTLK)
|
||||
if err != nil {
|
||||
return NormalizeResult{}, err
|
||||
}
|
||||
visualeffectsImported, err := importLegacyVisualeffects(migrationRoot, dataDir, legacyTLK)
|
||||
if err != nil {
|
||||
return NormalizeResult{}, err
|
||||
}
|
||||
appearanceImported, err := importLegacyAppearance(migrationRoot, dataDir, legacyTLK)
|
||||
if err != nil {
|
||||
return NormalizeResult{}, err
|
||||
}
|
||||
placeablesImported, err := importLegacyPlaceables(migrationRoot, dataDir, legacyTLK)
|
||||
if err != nil {
|
||||
return NormalizeResult{}, err
|
||||
}
|
||||
rulesetImported, err := importLegacyRuleset(migrationRoot, dataDir, legacyTLK)
|
||||
if err != nil {
|
||||
return NormalizeResult{}, err
|
||||
}
|
||||
skillsImported, err := importLegacySkills(migrationRoot, dataDir, legacyTLK)
|
||||
if err != nil {
|
||||
return NormalizeResult{}, err
|
||||
}
|
||||
spellsImported, err := importLegacySpells(migrationRoot, dataDir, legacyTLK)
|
||||
if err != nil {
|
||||
return NormalizeResult{}, err
|
||||
}
|
||||
racialtypesImported, err := importLegacyRacialtypes(migrationRoot, dataDir, legacyTLK)
|
||||
if err != nil {
|
||||
return NormalizeResult{}, err
|
||||
}
|
||||
classesImported, err := importLegacyClasses(migrationRoot, dataDir, legacyTLK)
|
||||
if err != nil {
|
||||
return NormalizeResult{}, err
|
||||
}
|
||||
portraitsImported, err := importLegacyPortraits(migrationRoot, dataDir, legacyTLK)
|
||||
if err != nil {
|
||||
return NormalizeResult{}, err
|
||||
}
|
||||
|
||||
paths, err := collectDataJSONPaths(dataDir)
|
||||
if err != nil {
|
||||
return NormalizeResult{}, err
|
||||
}
|
||||
|
||||
result := NormalizeResult{
|
||||
StatePath: filepath.Join(sourceDir, tlkStateFile),
|
||||
ScannedFiles: len(paths),
|
||||
UpdatedFiles: baseDialogImported + itempropsImported + masterfeatsImported + featImported + creaturespeedImported + armorImported + baseitemsImported + damagetypesImported + skyboxesImported + vfxPersistentImported + loadscreensImported + progfxImported + genericdoorsImported + cloakmodelImported + repadjustImported + wingmodelImported + tailmodelImported + doortypesImported + visualeffectsImported + appearanceImported + placeablesImported + rulesetImported + skillsImported + spellsImported + racialtypesImported + classesImported + portraitsImported,
|
||||
}
|
||||
state, err := loadTLKState(result.StatePath)
|
||||
if err != nil {
|
||||
return result, err
|
||||
}
|
||||
if state.Entries == nil {
|
||||
state.Entries = map[string]tlkStateMapping{}
|
||||
}
|
||||
if legacyTLK != nil && state.Language == "" {
|
||||
state.Language = legacyTLK.Language
|
||||
}
|
||||
if legacyTLK != nil {
|
||||
for key, id := range legacyTLK.Lock {
|
||||
if _, ok := state.Entries[key]; !ok {
|
||||
state.Entries[key] = tlkStateMapping{ID: id}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if legacyTLK != nil {
|
||||
for _, path := range paths {
|
||||
updated, count, err := inlineLegacyTLKFile(path, legacyTLK)
|
||||
if err != nil {
|
||||
return result, err
|
||||
}
|
||||
if updated {
|
||||
result.UpdatedFiles++
|
||||
result.InlineValues += count
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if err := saveTLKState(result.StatePath, state); err != nil {
|
||||
return result, err
|
||||
}
|
||||
if err := removeLegacyTLKAuthoredInput(filepath.Join(sourceDir, "tlk")); err != nil {
|
||||
return result, err
|
||||
}
|
||||
remainingFiles, remainingRefs, err := countLegacyTLKRefs(paths)
|
||||
if err != nil {
|
||||
return result, err
|
||||
}
|
||||
result.RemainingFiles = remainingFiles
|
||||
result.RemainingLegacyRefs = remainingRefs
|
||||
return result, nil
|
||||
}
|
||||
|
||||
func mergeLegacyTLK(base, override *legacyTLKData) *legacyTLKData {
|
||||
if base == nil && override == nil {
|
||||
return nil
|
||||
}
|
||||
merged := &legacyTLKData{
|
||||
Language: "en",
|
||||
Entries: map[string]tlkEntryData{},
|
||||
Lock: map[string]int{},
|
||||
}
|
||||
if base != nil {
|
||||
if base.Language != "" {
|
||||
merged.Language = base.Language
|
||||
}
|
||||
for key, entry := range base.Entries {
|
||||
merged.Entries[key] = entry
|
||||
}
|
||||
for key, id := range base.Lock {
|
||||
merged.Lock[key] = id
|
||||
}
|
||||
}
|
||||
if override != nil {
|
||||
if override.Language != "" {
|
||||
merged.Language = override.Language
|
||||
}
|
||||
for key, entry := range override.Entries {
|
||||
merged.Entries[key] = entry
|
||||
}
|
||||
for key, id := range override.Lock {
|
||||
merged.Lock[key] = id
|
||||
}
|
||||
}
|
||||
return merged
|
||||
}
|
||||
|
||||
func removeLegacyTLKAuthoredInput(tlkDir string) error {
|
||||
modulesDir := filepath.Join(tlkDir, "modules")
|
||||
if err := os.RemoveAll(modulesDir); err != nil {
|
||||
return fmt.Errorf("remove %s: %w", modulesDir, err)
|
||||
}
|
||||
lockPath := filepath.Join(tlkDir, "lock.json")
|
||||
if err := os.Remove(lockPath); err != nil && !os.IsNotExist(err) {
|
||||
return fmt.Errorf("remove %s: %w", lockPath, err)
|
||||
}
|
||||
gitkeepPath := filepath.Join(tlkDir, ".gitkeep")
|
||||
if err := os.Remove(gitkeepPath); err != nil && !os.IsNotExist(err) {
|
||||
return fmt.Errorf("remove %s: %w", gitkeepPath, err)
|
||||
}
|
||||
if err := os.Remove(tlkDir); err != nil && !os.IsNotExist(err) {
|
||||
if !strings.Contains(err.Error(), "directory not empty") {
|
||||
return fmt.Errorf("remove %s: %w", tlkDir, err)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func resolveMigrationInputRoot(sourceDir, referenceBuilderDir string) string {
|
||||
if strings.TrimSpace(referenceBuilderDir) != "" {
|
||||
return referenceBuilderDir
|
||||
}
|
||||
snapshotRoot := filepath.Join(sourceDir, "migration_snapshot")
|
||||
if info, err := os.Stat(snapshotRoot); err == nil && info.IsDir() {
|
||||
return snapshotRoot
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
func inlineLegacyTLKFile(path string, legacy *legacyTLKData) (bool, int, error) {
|
||||
obj, err := loadJSONObject(path)
|
||||
if err != nil {
|
||||
return false, 0, err
|
||||
}
|
||||
updated, count, err := inlineLegacyTLKValue(obj, legacy)
|
||||
if err != nil {
|
||||
return false, 0, err
|
||||
}
|
||||
if !updated {
|
||||
return false, 0, nil
|
||||
}
|
||||
raw, err := json.MarshalIndent(obj, "", " ")
|
||||
if err != nil {
|
||||
return false, 0, err
|
||||
}
|
||||
raw = append(raw, '\n')
|
||||
if err := os.WriteFile(path, raw, 0o644); err != nil {
|
||||
return false, 0, err
|
||||
}
|
||||
return true, count, nil
|
||||
}
|
||||
|
||||
func inlineLegacyTLKValue(value any, legacy *legacyTLKData) (bool, int, error) {
|
||||
switch typed := value.(type) {
|
||||
case map[string]any:
|
||||
if rawTLK, ok := typed["tlk"]; ok {
|
||||
key, ok := rawTLK.(string)
|
||||
if ok {
|
||||
entry, ok := legacy.Entries[key]
|
||||
if !ok {
|
||||
return false, 0, nil
|
||||
}
|
||||
payload := map[string]any{
|
||||
"key": key,
|
||||
"text": entry.Text,
|
||||
}
|
||||
if entry.SoundResRef != "" {
|
||||
payload["sound_resref"] = entry.SoundResRef
|
||||
}
|
||||
if entry.SoundLength != 0 {
|
||||
payload["sound_length"] = entry.SoundLength
|
||||
}
|
||||
typed["tlk"] = payload
|
||||
return true, 1, nil
|
||||
}
|
||||
}
|
||||
|
||||
updated := false
|
||||
total := 0
|
||||
for key, child := range typed {
|
||||
childUpdated, childCount, err := inlineLegacyTLKValue(child, legacy)
|
||||
if err != nil {
|
||||
return false, 0, err
|
||||
}
|
||||
if childUpdated {
|
||||
typed[key] = child
|
||||
updated = true
|
||||
total += childCount
|
||||
}
|
||||
}
|
||||
return updated, total, nil
|
||||
case []any:
|
||||
updated := false
|
||||
total := 0
|
||||
for index, child := range typed {
|
||||
childUpdated, childCount, err := inlineLegacyTLKValue(child, legacy)
|
||||
if err != nil {
|
||||
return false, 0, err
|
||||
}
|
||||
if childUpdated {
|
||||
typed[index] = child
|
||||
updated = true
|
||||
total += childCount
|
||||
}
|
||||
}
|
||||
return updated, total, nil
|
||||
default:
|
||||
return false, 0, nil
|
||||
}
|
||||
}
|
||||
|
||||
func collectDataJSONPaths(dataDir string) ([]string, error) {
|
||||
paths := make([]string, 0)
|
||||
err := filepath.WalkDir(dataDir, func(path string, d os.DirEntry, err error) error {
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if d.IsDir() {
|
||||
return nil
|
||||
}
|
||||
if strings.HasSuffix(strings.ToLower(d.Name()), ".json") && d.Name() != "lock.json" {
|
||||
paths = append(paths, path)
|
||||
}
|
||||
return nil
|
||||
})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return paths, nil
|
||||
}
|
||||
|
||||
func countLegacyTLKRefs(paths []string) (int, int, error) {
|
||||
files := 0
|
||||
refs := 0
|
||||
for _, path := range paths {
|
||||
obj, err := loadJSONObject(path)
|
||||
if err != nil {
|
||||
return 0, 0, err
|
||||
}
|
||||
count := countLegacyTLKRefsInValue(obj)
|
||||
if count == 0 {
|
||||
continue
|
||||
}
|
||||
files++
|
||||
refs += count
|
||||
}
|
||||
return files, refs, nil
|
||||
}
|
||||
|
||||
func countLegacyTLKRefsInValue(value any) int {
|
||||
switch typed := value.(type) {
|
||||
case map[string]any:
|
||||
total := 0
|
||||
if rawTLK, ok := typed["tlk"]; ok {
|
||||
if _, ok := rawTLK.(string); ok {
|
||||
total++
|
||||
}
|
||||
}
|
||||
for _, child := range typed {
|
||||
total += countLegacyTLKRefsInValue(child)
|
||||
}
|
||||
return total
|
||||
case []any:
|
||||
total := 0
|
||||
for _, child := range typed {
|
||||
total += countLegacyTLKRefsInValue(child)
|
||||
}
|
||||
return total
|
||||
default:
|
||||
return 0
|
||||
}
|
||||
}
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,428 @@
|
||||
package topdata
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"regexp"
|
||||
"slices"
|
||||
"strconv"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// supportedPartCategories defines the part types that should be auto-discovered
|
||||
// from the sow-assets repository.
|
||||
var supportedPartCategories = []string{
|
||||
"belt",
|
||||
"bicep",
|
||||
"chest",
|
||||
"foot",
|
||||
"forearm",
|
||||
"leg",
|
||||
"neck",
|
||||
"pelvis",
|
||||
"robe",
|
||||
"shin",
|
||||
"shoulder",
|
||||
}
|
||||
|
||||
var partDatasetToAssetCategory = map[string]string{
|
||||
"belt": "belt",
|
||||
"bicep": "bicep",
|
||||
"chest": "chest",
|
||||
"foot": "foot",
|
||||
"forearm": "forearm",
|
||||
"legs": "leg",
|
||||
"neck": "neck",
|
||||
"pelvis": "pelvis",
|
||||
"robe": "robe",
|
||||
"shin": "shin",
|
||||
"shoulder": "shoulder",
|
||||
}
|
||||
|
||||
// trailingNumberRegex matches trailing digits at the end of a string
|
||||
var trailingNumberRegex = regexp.MustCompile(`(\d+)$`)
|
||||
|
||||
// isPartsDataset checks if a dataset name corresponds to a parts dataset
|
||||
func isPartsDataset(datasetName string) bool {
|
||||
// Check if the dataset is under parts/ directory
|
||||
if strings.HasPrefix(datasetName, "parts/") {
|
||||
return true
|
||||
}
|
||||
// Check if it's exactly a parts file (e.g., "parts/belt")
|
||||
parts := strings.Split(datasetName, "/")
|
||||
if len(parts) == 2 && parts[0] == "parts" {
|
||||
return true
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// extractPartCategory extracts the part category from a dataset name
|
||||
// e.g., "parts/belt" -> "belt", "parts/bicep" -> "bicep"
|
||||
func extractPartCategory(datasetName string) string {
|
||||
parts := strings.Split(datasetName, "/")
|
||||
if len(parts) >= 2 {
|
||||
return parts[len(parts)-1]
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
func partAssetCategoryForDataset(datasetName string) string {
|
||||
category := extractPartCategory(datasetName)
|
||||
if mapped, ok := partDatasetToAssetCategory[category]; ok {
|
||||
return mapped
|
||||
}
|
||||
return category
|
||||
}
|
||||
|
||||
// isSupportedPartCategory checks if the given category is in the supported list
|
||||
func isSupportedPartCategory(category string) bool {
|
||||
for _, c := range supportedPartCategories {
|
||||
if c == category {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// extractTrailingNumericSuffix extracts the trailing numeric suffix from a filename stem
|
||||
// e.g., "pfa0_belt018" -> 18, "pfa0_belt171" -> 171
|
||||
// Returns 0 and false if no trailing numeric suffix is found
|
||||
func extractTrailingNumericSuffix(filenameStem string) (int, bool) {
|
||||
matches := trailingNumberRegex.FindStringSubmatch(filenameStem)
|
||||
if len(matches) < 2 {
|
||||
return 0, false
|
||||
}
|
||||
num, err := strconv.Atoi(matches[1])
|
||||
if err != nil {
|
||||
return 0, false
|
||||
}
|
||||
return num, true
|
||||
}
|
||||
|
||||
// DiscoveredPartModel represents a discovered model file with its row ID
|
||||
type DiscoveredPartModel struct {
|
||||
RowID int
|
||||
Filename string
|
||||
}
|
||||
|
||||
// DiscoverPartModels scans the assets/part directory for .mdl files
|
||||
// and returns a map of rowID -> default row data for the given category.
|
||||
// The assetsDir should be the root assets directory (containing the part/ subdirectory).
|
||||
func DiscoverPartModels(assetsDir string, category string) (map[int]*DiscoveredPartModel, error) {
|
||||
result := make(map[int]*DiscoveredPartModel)
|
||||
|
||||
if !isSupportedPartCategory(category) {
|
||||
return result, nil
|
||||
}
|
||||
|
||||
partRoot, err := resolvePartsRoot(assetsDir)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if partRoot == "" {
|
||||
return result, nil
|
||||
}
|
||||
|
||||
// Construct the path to the part category directory
|
||||
partDir := filepath.Join(partRoot, category)
|
||||
|
||||
// Check if the directory exists
|
||||
info, err := os.Stat(partDir)
|
||||
if err != nil {
|
||||
if os.IsNotExist(err) {
|
||||
return result, nil
|
||||
}
|
||||
return nil, fmt.Errorf("failed to access part directory %s: %w", partDir, err)
|
||||
}
|
||||
if !info.IsDir() {
|
||||
return nil, fmt.Errorf("part directory %s is not a directory", partDir)
|
||||
}
|
||||
|
||||
// Walk the directory and find .mdl files
|
||||
err = filepath.Walk(partDir, func(path string, info os.FileInfo, err error) error {
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if info.IsDir() {
|
||||
return nil
|
||||
}
|
||||
|
||||
// Only process .mdl files
|
||||
if !strings.HasSuffix(strings.ToLower(info.Name()), ".mdl") {
|
||||
return nil
|
||||
}
|
||||
|
||||
// Extract the filename stem (without extension)
|
||||
filenameStem := strings.TrimSuffix(info.Name(), filepath.Ext(info.Name()))
|
||||
|
||||
// Extract trailing numeric suffix
|
||||
rowID, hasSuffix := extractTrailingNumericSuffix(filenameStem)
|
||||
if !hasSuffix {
|
||||
return nil
|
||||
}
|
||||
|
||||
// Store the discovered model
|
||||
if existing, ok := result[rowID]; ok {
|
||||
// If we already have this row ID, keep the first one found (alphabetically)
|
||||
if path < existing.Filename {
|
||||
result[rowID] = &DiscoveredPartModel{
|
||||
RowID: rowID,
|
||||
Filename: path,
|
||||
}
|
||||
}
|
||||
} else {
|
||||
result[rowID] = &DiscoveredPartModel{
|
||||
RowID: rowID,
|
||||
Filename: path,
|
||||
}
|
||||
}
|
||||
|
||||
return nil
|
||||
})
|
||||
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to scan part directory %s: %w", partDir, err)
|
||||
}
|
||||
|
||||
return result, nil
|
||||
}
|
||||
|
||||
// CreateDefaultPartRow creates a default row for a discovered part model
|
||||
func CreateDefaultPartRow(rowID int) map[string]any {
|
||||
return map[string]any{
|
||||
"id": rowID,
|
||||
"COSTMODIFIER": "0",
|
||||
"ACBONUS": "0.00",
|
||||
}
|
||||
}
|
||||
|
||||
func isUnsetPartValue(value any) bool {
|
||||
switch typed := value.(type) {
|
||||
case nil:
|
||||
return true
|
||||
case string:
|
||||
return typed == "" || typed == "****"
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
func resolvePartsRoot(root string) (string, error) {
|
||||
if strings.TrimSpace(root) == "" {
|
||||
return "", nil
|
||||
}
|
||||
candidates := []string{
|
||||
filepath.Join(root, "assets", "parts"),
|
||||
filepath.Join(root, "assets", "part"),
|
||||
filepath.Join(root, "parts"),
|
||||
filepath.Join(root, "part"),
|
||||
}
|
||||
for _, candidate := range candidates {
|
||||
info, err := os.Stat(candidate)
|
||||
if err == nil && info.IsDir() {
|
||||
return candidate, nil
|
||||
}
|
||||
if err != nil && !os.IsNotExist(err) {
|
||||
return "", fmt.Errorf("failed to access parts root %s: %w", candidate, err)
|
||||
}
|
||||
}
|
||||
return "", nil
|
||||
}
|
||||
|
||||
// scanAutogeneratedParts scans the assets/part directory for all supported part categories
|
||||
// and returns a map of category -> set of row IDs that should be auto-generated.
|
||||
// The assetsDir parameter should be the path to the assets directory that contains the part/ subdirectory.
|
||||
func scanAutogeneratedParts(assetsDir string) (map[string]map[int]struct{}, error) {
|
||||
if assetsDir == "" {
|
||||
return nil, nil
|
||||
}
|
||||
|
||||
result := make(map[string]map[int]struct{})
|
||||
|
||||
for _, category := range supportedPartCategories {
|
||||
discovered, err := DiscoverPartModels(assetsDir, category)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
// Always include the category, even if no IDs were discovered
|
||||
ids := make(map[int]struct{}, len(discovered))
|
||||
for rowID := range discovered {
|
||||
ids[rowID] = struct{}{}
|
||||
}
|
||||
result[category] = ids
|
||||
}
|
||||
|
||||
return result, nil
|
||||
}
|
||||
|
||||
func loadPartOverrides(path string) ([]map[string]any, error) {
|
||||
if _, err := os.Stat(path); err != nil {
|
||||
if os.IsNotExist(err) {
|
||||
return nil, nil
|
||||
}
|
||||
return nil, err
|
||||
}
|
||||
obj, err := loadJSONObject(path)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
rawOverrides, ok := obj["overrides"]
|
||||
if !ok {
|
||||
return nil, nil
|
||||
}
|
||||
overrideList, ok := rawOverrides.([]any)
|
||||
if !ok {
|
||||
return nil, fmt.Errorf("%s: overrides must be an array", path)
|
||||
}
|
||||
overrides := make([]map[string]any, 0, len(overrideList))
|
||||
for index, rawOverride := range overrideList {
|
||||
override, ok := rawOverride.(map[string]any)
|
||||
if !ok {
|
||||
return nil, fmt.Errorf("%s: override %d must be an object", path, index)
|
||||
}
|
||||
overrides = append(overrides, override)
|
||||
}
|
||||
return overrides, nil
|
||||
}
|
||||
|
||||
func applyPartOverrides(sourceDir string, collected []nativeCollectedDataset) ([]nativeCollectedDataset, error) {
|
||||
result := make([]nativeCollectedDataset, len(collected))
|
||||
copy(result, collected)
|
||||
for i, dataset := range result {
|
||||
if !strings.HasPrefix(dataset.Dataset.Name, "parts/") {
|
||||
continue
|
||||
}
|
||||
category := extractPartCategory(dataset.Dataset.Name)
|
||||
if category == "" {
|
||||
continue
|
||||
}
|
||||
overridePath := filepath.Join(sourceDir, "data", "parts", "overrides", category+".json")
|
||||
overrides, err := loadPartOverrides(overridePath)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if len(overrides) == 0 {
|
||||
continue
|
||||
}
|
||||
|
||||
rows := make([]map[string]any, len(dataset.Rows))
|
||||
rowByID := make(map[int]map[string]any, len(dataset.Rows))
|
||||
for index, row := range dataset.Rows {
|
||||
cloned := cloneRowMap(row)
|
||||
rows[index] = cloned
|
||||
if rowID, ok := cloned["id"].(int); ok {
|
||||
rowByID[rowID] = cloned
|
||||
}
|
||||
}
|
||||
|
||||
for index, override := range overrides {
|
||||
rawID, ok := override["id"]
|
||||
if !ok {
|
||||
return nil, fmt.Errorf("%s: override %d is missing id", overridePath, index)
|
||||
}
|
||||
rowID, err := asInt(rawID)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("%s: override %d id is not numeric", overridePath, index)
|
||||
}
|
||||
row, ok := rowByID[rowID]
|
||||
if !ok {
|
||||
row = CreateDefaultPartRow(rowID)
|
||||
rows = append(rows, row)
|
||||
rowByID[rowID] = row
|
||||
}
|
||||
for field, value := range override {
|
||||
if field == "id" {
|
||||
continue
|
||||
}
|
||||
row[field] = normalizePartOverrideValue(value)
|
||||
}
|
||||
}
|
||||
|
||||
slices.SortFunc(rows, func(a, b map[string]any) int {
|
||||
idA := a["id"].(int)
|
||||
idB := b["id"].(int)
|
||||
return idA - idB
|
||||
})
|
||||
result[i].Rows = rows
|
||||
}
|
||||
return result, nil
|
||||
}
|
||||
|
||||
func normalizePartOverrideValue(value any) any {
|
||||
switch typed := value.(type) {
|
||||
case float64:
|
||||
if typed == float64(int(typed)) {
|
||||
return strconv.Itoa(int(typed))
|
||||
}
|
||||
case int:
|
||||
return strconv.Itoa(typed)
|
||||
case json.Number:
|
||||
return typed.String()
|
||||
}
|
||||
return value
|
||||
}
|
||||
|
||||
// augmentWithAutogeneratedParts applies repository-backed discovery defaults.
|
||||
// Missing discovered IDs are added with default values. Existing discovered rows
|
||||
// retain authored values unless they still use placeholder values, in which case
|
||||
// discovery activates them with engine-visible defaults.
|
||||
func augmentWithAutogeneratedParts(collected []nativeCollectedDataset, autogenerated map[string]map[int]struct{}) []nativeCollectedDataset {
|
||||
if len(autogenerated) == 0 {
|
||||
return collected
|
||||
}
|
||||
|
||||
result := make([]nativeCollectedDataset, len(collected))
|
||||
copy(result, collected)
|
||||
|
||||
for i, dataset := range collected {
|
||||
category := partAssetCategoryForDataset(dataset.Dataset.Name)
|
||||
if !isSupportedPartCategory(category) {
|
||||
continue
|
||||
}
|
||||
|
||||
ids, ok := autogenerated[category]
|
||||
if !ok || len(ids) == 0 {
|
||||
continue
|
||||
}
|
||||
|
||||
// Build a map of existing row IDs.
|
||||
existingRows := make(map[int]map[string]any, len(dataset.Rows))
|
||||
for _, row := range dataset.Rows {
|
||||
if id, ok := row["id"].(int); ok {
|
||||
existingRows[id] = row
|
||||
}
|
||||
}
|
||||
|
||||
// Add rows for discovered IDs that don't already exist
|
||||
newRows := make([]map[string]any, 0, len(dataset.Rows))
|
||||
newRows = append(newRows, dataset.Rows...)
|
||||
|
||||
for rowID := range ids {
|
||||
if row, exists := existingRows[rowID]; exists {
|
||||
if isUnsetPartValue(row["COSTMODIFIER"]) {
|
||||
row["COSTMODIFIER"] = "0"
|
||||
}
|
||||
if isUnsetPartValue(row["ACBONUS"]) {
|
||||
row["ACBONUS"] = "0.00"
|
||||
}
|
||||
continue
|
||||
}
|
||||
if _, exists := existingRows[rowID]; !exists {
|
||||
newRow := CreateDefaultPartRow(rowID)
|
||||
newRows = append(newRows, newRow)
|
||||
}
|
||||
}
|
||||
|
||||
// Sort rows by ID
|
||||
slices.SortFunc(newRows, func(a, b map[string]any) int {
|
||||
idA := a["id"].(int)
|
||||
idB := b["id"].(int)
|
||||
return idA - idB
|
||||
})
|
||||
|
||||
result[i].Rows = newRows
|
||||
}
|
||||
|
||||
return result
|
||||
}
|
||||
@@ -0,0 +1,6 @@
|
||||
package topdata
|
||||
|
||||
func importLegacyPlaceables(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
|
||||
_ = legacyTLK
|
||||
return importLegacyDatasetMirror(referenceBuilderDir, dataDir, "placeables", "placeables.2da", nil)
|
||||
}
|
||||
@@ -0,0 +1,6 @@
|
||||
package topdata
|
||||
|
||||
func importLegacyPortraits(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
|
||||
_ = legacyTLK
|
||||
return importLegacyDatasetMirror(referenceBuilderDir, dataDir, "portraits", "portraits.2da", nil)
|
||||
}
|
||||
@@ -0,0 +1,223 @@
|
||||
package topdata
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"slices"
|
||||
"strings"
|
||||
)
|
||||
|
||||
var portraitColumns = []string{"BaseResRef", "Sex", "Race", "InanimateType", "Plot", "LowGore"}
|
||||
|
||||
func parseRowMetadata(raw any) (map[string]any, error) {
|
||||
if raw == nil {
|
||||
return nil, nil
|
||||
}
|
||||
obj, ok := raw.(map[string]any)
|
||||
if !ok {
|
||||
return nil, fmt.Errorf("meta must be an object")
|
||||
}
|
||||
if len(obj) == 0 {
|
||||
return map[string]any{}, nil
|
||||
}
|
||||
|
||||
meta := make(map[string]any, len(obj))
|
||||
for key, value := range obj {
|
||||
switch normalizeMetadataKey(key) {
|
||||
case "family":
|
||||
family, err := parseFamilyMetadata(value)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("meta.family: %w", err)
|
||||
}
|
||||
meta["family"] = family
|
||||
case "wiki":
|
||||
wiki, err := parseWikiMetadata(value)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("meta.wiki: %w", err)
|
||||
}
|
||||
if len(wiki) > 0 {
|
||||
meta["wiki"] = wiki
|
||||
}
|
||||
case "portrait":
|
||||
portrait, err := parsePortraitMetadata(value)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("meta.portrait: %w", err)
|
||||
}
|
||||
meta["portrait"] = portrait
|
||||
default:
|
||||
return nil, fmt.Errorf("unknown metadata key %q", key)
|
||||
}
|
||||
}
|
||||
return meta, nil
|
||||
}
|
||||
|
||||
func parsePortraitMetadata(raw any) (map[string]any, error) {
|
||||
obj, ok := raw.(map[string]any)
|
||||
if !ok {
|
||||
return nil, fmt.Errorf("must be an object")
|
||||
}
|
||||
portrait := map[string]any{}
|
||||
for key, value := range obj {
|
||||
switch normalizeMetadataKey(key) {
|
||||
case "resref":
|
||||
text, ok := value.(string)
|
||||
if !ok || strings.TrimSpace(text) == "" {
|
||||
return nil, fmt.Errorf("resref must be a non-empty string")
|
||||
}
|
||||
portrait["resref"] = text
|
||||
case "sex", "race", "inanimate_type", "plot", "low_gore":
|
||||
portrait[normalizeMetadataKey(key)] = value
|
||||
default:
|
||||
return nil, fmt.Errorf("unknown field %q", key)
|
||||
}
|
||||
}
|
||||
if _, ok := portrait["resref"]; !ok {
|
||||
return nil, fmt.Errorf("resref is required")
|
||||
}
|
||||
return portrait, nil
|
||||
}
|
||||
|
||||
func normalizeMetadataKey(key string) string {
|
||||
replacer := strings.NewReplacer("-", "_", " ", "_")
|
||||
return strings.ToLower(replacer.Replace(strings.TrimSpace(key)))
|
||||
}
|
||||
|
||||
func portraitMetaFromRow(row map[string]any) (map[string]any, bool) {
|
||||
rawMeta, ok := lookupField(row, "meta")
|
||||
if !ok {
|
||||
return nil, false
|
||||
}
|
||||
meta, ok := rawMeta.(map[string]any)
|
||||
if !ok {
|
||||
return nil, false
|
||||
}
|
||||
rawPortrait, ok := meta["portrait"]
|
||||
if !ok {
|
||||
return nil, false
|
||||
}
|
||||
portrait, ok := rawPortrait.(map[string]any)
|
||||
if !ok {
|
||||
return nil, false
|
||||
}
|
||||
return portrait, true
|
||||
}
|
||||
|
||||
func mergePortraitMetadata(collected []nativeCollectedDataset) ([]nativeCollectedDataset, error) {
|
||||
portraitIndex := -1
|
||||
for i, dataset := range collected {
|
||||
if dataset.Dataset.OutputName == "portraits.2da" || dataset.Dataset.Name == "portraits" {
|
||||
portraitIndex = i
|
||||
break
|
||||
}
|
||||
}
|
||||
if portraitIndex < 0 {
|
||||
return collected, nil
|
||||
}
|
||||
|
||||
portraits := collected[portraitIndex]
|
||||
if len(portraits.Columns) == 0 {
|
||||
portraits.Columns = append([]string(nil), portraitColumns...)
|
||||
}
|
||||
|
||||
rowByResref := map[string]map[string]any{}
|
||||
usedIDs := map[int]struct{}{}
|
||||
for _, row := range portraits.Rows {
|
||||
rowID, ok := row["id"].(int)
|
||||
if ok {
|
||||
usedIDs[rowID] = struct{}{}
|
||||
}
|
||||
resref, _ := lookupField(row, "BaseResRef")
|
||||
if text, ok := resref.(string); ok {
|
||||
rowByResref[text] = row
|
||||
}
|
||||
}
|
||||
for _, rowID := range portraits.LockData {
|
||||
usedIDs[rowID] = struct{}{}
|
||||
}
|
||||
nextID := nextAvailableID(usedIDs)
|
||||
lockModified := false
|
||||
|
||||
for _, dataset := range collected {
|
||||
for _, row := range dataset.Rows {
|
||||
portrait, ok := portraitMetaFromRow(row)
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
resref := portrait["resref"].(string)
|
||||
target, ok := rowByResref[resref]
|
||||
if !ok {
|
||||
lockKey := "portraits:auto:" + resref
|
||||
rowID, found := portraits.LockData[lockKey]
|
||||
if !found {
|
||||
rowID = nextID
|
||||
portraits.LockData[lockKey] = rowID
|
||||
lockModified = true
|
||||
usedIDs[rowID] = struct{}{}
|
||||
nextID = nextAvailableID(usedIDs)
|
||||
}
|
||||
target = map[string]any{
|
||||
"id": rowID,
|
||||
"key": lockKey,
|
||||
}
|
||||
for _, column := range portraits.Columns {
|
||||
target[column] = nullValue
|
||||
}
|
||||
target["BaseResRef"] = resref
|
||||
portraits.Rows = append(portraits.Rows, target)
|
||||
rowByResref[resref] = target
|
||||
}
|
||||
if err := applyPortraitContribution(target, portrait, rowIdentity(dataset.Dataset.Name, row)); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if lockModified && portraits.Dataset.LockPath != "" {
|
||||
if err := saveLockfile(portraits.Dataset.LockPath, portraits.LockData); err != nil {
|
||||
return nil, fmt.Errorf("dataset %s: %w", portraits.Dataset.Name, err)
|
||||
}
|
||||
}
|
||||
|
||||
slices.SortFunc(portraits.Rows, func(a, b map[string]any) int {
|
||||
return a["id"].(int) - b["id"].(int)
|
||||
})
|
||||
collected[portraitIndex] = portraits
|
||||
return collected, nil
|
||||
}
|
||||
|
||||
func applyPortraitContribution(target map[string]any, portrait map[string]any, source string) error {
|
||||
mapping := map[string]string{
|
||||
"resref": "BaseResRef",
|
||||
"sex": "Sex",
|
||||
"race": "Race",
|
||||
"inanimate_type": "InanimateType",
|
||||
"plot": "Plot",
|
||||
"low_gore": "LowGore",
|
||||
}
|
||||
for key, value := range portrait {
|
||||
column, ok := mapping[normalizeMetadataKey(key)]
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
current, _ := lookupField(target, column)
|
||||
if current == nil || current == nullValue {
|
||||
target[column] = value
|
||||
continue
|
||||
}
|
||||
if normalizeComparableValue(current) == normalizeComparableValue(value) {
|
||||
continue
|
||||
}
|
||||
return fmt.Errorf("portrait %q conflicts for column %s between existing value %v and contributor %s value %v", portrait["resref"], column, current, source, value)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func normalizeComparableValue(value any) string {
|
||||
switch typed := value.(type) {
|
||||
case nil:
|
||||
return nullValue
|
||||
case string:
|
||||
return typed
|
||||
default:
|
||||
return format2DAValue(typed)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,6 @@
|
||||
package topdata
|
||||
|
||||
func importLegacyProgfx(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
|
||||
_ = legacyTLK
|
||||
return importLegacyDatasetMirror(referenceBuilderDir, dataDir, "progfx", "progfx.2da", nil)
|
||||
}
|
||||
@@ -0,0 +1,282 @@
|
||||
package topdata
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"slices"
|
||||
"strings"
|
||||
)
|
||||
|
||||
type legacyRacialtypesFeatTable struct {
|
||||
Output string
|
||||
Rows []map[string]any
|
||||
}
|
||||
|
||||
func importLegacyRacialtypes(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
|
||||
if strings.TrimSpace(referenceBuilderDir) == "" {
|
||||
return 0, nil
|
||||
}
|
||||
|
||||
legacyCoreDir := filepath.Join(referenceBuilderDir, "data", "racialtypes", "core")
|
||||
legacyFeatsDir := filepath.Join(referenceBuilderDir, "data", "racialtypes", "feats")
|
||||
if _, err := os.Stat(legacyCoreDir); err != nil {
|
||||
if os.IsNotExist(err) {
|
||||
return 0, nil
|
||||
}
|
||||
return 0, err
|
||||
}
|
||||
if _, err := os.Stat(legacyFeatsDir); err != nil {
|
||||
if os.IsNotExist(err) {
|
||||
return 0, nil
|
||||
}
|
||||
return 0, err
|
||||
}
|
||||
|
||||
targetRoot := filepath.Join(dataDir, "racialtypes", "registry")
|
||||
targetBasePath := filepath.Join(targetRoot, "base.json")
|
||||
targetLockPath := filepath.Join(targetRoot, "lock.json")
|
||||
targetRacesDir := filepath.Join(targetRoot, "races")
|
||||
if fileExists(targetBasePath) && fileExists(targetLockPath) {
|
||||
entries, err := os.ReadDir(targetRacesDir)
|
||||
if err == nil {
|
||||
raceFiles := 0
|
||||
for _, entry := range entries {
|
||||
if !entry.IsDir() && strings.HasSuffix(strings.ToLower(entry.Name()), ".json") {
|
||||
raceFiles++
|
||||
}
|
||||
}
|
||||
if raceFiles == 16 {
|
||||
return 0, nil
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
baseColumns, baseRows, lockData, raceRows, err := collectLegacyRacialtypesRegistry(legacyCoreDir, legacyFeatsDir, legacyTLK)
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
|
||||
if err := removePathIfExists(filepath.Join(dataDir, "racialtypes", "core")); err != nil {
|
||||
return 0, err
|
||||
}
|
||||
if err := removePathIfExists(filepath.Join(dataDir, "racialtypes", "feats")); err != nil {
|
||||
return 0, err
|
||||
}
|
||||
if err := removePathIfExists(filepath.Join(targetRoot, "base_rows.json")); err != nil {
|
||||
return 0, err
|
||||
}
|
||||
if err := removePathIfExists(targetRacesDir); err != nil {
|
||||
return 0, err
|
||||
}
|
||||
if err := os.MkdirAll(targetRacesDir, 0o755); err != nil {
|
||||
return 0, err
|
||||
}
|
||||
|
||||
writes := []struct {
|
||||
path string
|
||||
obj any
|
||||
}{
|
||||
{
|
||||
path: targetBasePath,
|
||||
obj: map[string]any{
|
||||
"columns": stringSliceToAny(baseColumns),
|
||||
"rows": rowsToAny(baseRows),
|
||||
},
|
||||
},
|
||||
{
|
||||
path: targetLockPath,
|
||||
obj: anyMapInt(lockData),
|
||||
},
|
||||
}
|
||||
|
||||
raceKeys := make([]string, 0, len(raceRows))
|
||||
for key := range raceRows {
|
||||
raceKeys = append(raceKeys, key)
|
||||
}
|
||||
slices.Sort(raceKeys)
|
||||
for _, key := range raceKeys {
|
||||
fileName := strings.TrimPrefix(key, "racialtypes:") + ".json"
|
||||
writes = append(writes, struct {
|
||||
path string
|
||||
obj any
|
||||
}{
|
||||
path: filepath.Join(targetRacesDir, fileName),
|
||||
obj: raceRows[key],
|
||||
})
|
||||
}
|
||||
|
||||
for _, write := range writes {
|
||||
raw, err := json.MarshalIndent(write.obj, "", " ")
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
raw = append(raw, '\n')
|
||||
if err := os.WriteFile(write.path, raw, 0o644); err != nil {
|
||||
return 0, err
|
||||
}
|
||||
}
|
||||
return len(writes), nil
|
||||
}
|
||||
|
||||
func collectLegacyRacialtypesRegistry(coreDir, featsDir string, legacyTLK *legacyTLKData) ([]string, []map[string]any, map[string]int, map[string]map[string]any, error) {
|
||||
coreCollected, err := collectBaseDataset(nativeDataset{
|
||||
Name: "racialtypes",
|
||||
BasePath: filepath.Join(coreDir, "base.json"),
|
||||
LockPath: filepath.Join(coreDir, "lock.json"),
|
||||
ModulesDir: filepath.Join(coreDir, "modules"),
|
||||
Spec: specForDataset("racialtypes"),
|
||||
})
|
||||
if err != nil {
|
||||
return nil, nil, nil, nil, err
|
||||
}
|
||||
|
||||
featTables, err := loadLegacyRacialtypesFeatTables(featsDir)
|
||||
if err != nil {
|
||||
return nil, nil, nil, nil, err
|
||||
}
|
||||
|
||||
baseRows := make([]map[string]any, 0)
|
||||
raceRows := map[string]map[string]any{}
|
||||
lockData := map[string]int{}
|
||||
for key, id := range coreCollected.LockData {
|
||||
if strings.HasPrefix(key, "racialtypes:") {
|
||||
lockData[key] = id
|
||||
}
|
||||
}
|
||||
|
||||
for _, collectedRow := range coreCollected.Rows {
|
||||
row, ok := deepCopyValue(collectedRow).(map[string]any)
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
if legacyTLK != nil {
|
||||
if _, _, err := inlineLegacyTLKValue(row, legacyTLK); err != nil {
|
||||
return nil, nil, nil, nil, err
|
||||
}
|
||||
}
|
||||
key, _ := row["key"].(string)
|
||||
if !strings.HasPrefix(key, "racialtypes:") {
|
||||
assignRacialtypesBaseRowKey(row)
|
||||
if key, _ := row["key"].(string); key != "" {
|
||||
rowID, err := asInt(row["id"])
|
||||
if err != nil {
|
||||
return nil, nil, nil, nil, err
|
||||
}
|
||||
lockData[key] = rowID
|
||||
}
|
||||
baseRows = append(baseRows, row)
|
||||
continue
|
||||
}
|
||||
|
||||
delete(row, "id")
|
||||
delete(row, "key")
|
||||
|
||||
raceFile := map[string]any{
|
||||
"key": key,
|
||||
"core": row,
|
||||
}
|
||||
if tableKey := extractRacialtypesFeatTableKey(row["FeatsTable"]); tableKey != "" {
|
||||
table, ok := featTables[tableKey]
|
||||
if !ok {
|
||||
return nil, nil, nil, nil, fmt.Errorf("%s: unknown feat table %s", key, tableKey)
|
||||
}
|
||||
delete(row, "FeatsTable")
|
||||
raceFile["feat_output"] = table.Output
|
||||
raceFile["feats"] = rowsToAny(table.Rows)
|
||||
}
|
||||
raceRows[key] = raceFile
|
||||
}
|
||||
|
||||
slices.SortFunc(baseRows, func(a, b map[string]any) int {
|
||||
left, _ := asInt(a["id"])
|
||||
right, _ := asInt(b["id"])
|
||||
return left - right
|
||||
})
|
||||
return coreCollected.Columns, baseRows, lockData, raceRows, nil
|
||||
}
|
||||
|
||||
func loadLegacyRacialtypesFeatTables(featsDir string) (map[string]legacyRacialtypesFeatTable, error) {
|
||||
entries, err := os.ReadDir(featsDir)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
tables := map[string]legacyRacialtypesFeatTable{}
|
||||
for _, entry := range entries {
|
||||
if entry.IsDir() || filepath.Ext(entry.Name()) != ".json" {
|
||||
continue
|
||||
}
|
||||
obj, err := loadJSONObject(filepath.Join(featsDir, entry.Name()))
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
tableKey, _ := obj["key"].(string)
|
||||
outputName, _ := obj["output"].(string)
|
||||
rawRows, ok := obj["rows"].([]any)
|
||||
if tableKey == "" || outputName == "" || !ok {
|
||||
return nil, fmt.Errorf("%s: racialtypes feat table must define key, output, and rows", entry.Name())
|
||||
}
|
||||
rows := make([]map[string]any, 0, len(rawRows))
|
||||
for index, raw := range rawRows {
|
||||
row, ok := raw.(map[string]any)
|
||||
if !ok {
|
||||
return nil, fmt.Errorf("%s: row %d must be an object", entry.Name(), index)
|
||||
}
|
||||
rows = append(rows, row)
|
||||
}
|
||||
tables[tableKey] = legacyRacialtypesFeatTable{
|
||||
Output: outputName,
|
||||
Rows: rows,
|
||||
}
|
||||
}
|
||||
return tables, nil
|
||||
}
|
||||
|
||||
func extractRacialtypesFeatTableKey(value any) string {
|
||||
obj, ok := value.(map[string]any)
|
||||
if !ok {
|
||||
return ""
|
||||
}
|
||||
tableKey, _ := obj["table"].(string)
|
||||
return tableKey
|
||||
}
|
||||
|
||||
func assignRacialtypesBaseRowKey(row map[string]any) {
|
||||
name, _ := row["Name"].(string)
|
||||
if strings.TrimSpace(name) == "" || name == nullValue {
|
||||
delete(row, "key")
|
||||
return
|
||||
}
|
||||
label, _ := row["Label"].(string)
|
||||
keySuffix := normalizeRacialtypesKeySuffix(label)
|
||||
if keySuffix == "" {
|
||||
delete(row, "key")
|
||||
return
|
||||
}
|
||||
row["key"] = "racialtypes:" + keySuffix
|
||||
}
|
||||
|
||||
func normalizeRacialtypesKeySuffix(label string) string {
|
||||
normalized := strings.ToLower(strings.TrimSpace(label))
|
||||
if normalized == "" {
|
||||
return ""
|
||||
}
|
||||
var b strings.Builder
|
||||
lastUnderscore := false
|
||||
for _, ch := range normalized {
|
||||
isAlphaNum := (ch >= 'a' && ch <= 'z') || (ch >= '0' && ch <= '9')
|
||||
if isAlphaNum {
|
||||
b.WriteRune(ch)
|
||||
lastUnderscore = false
|
||||
continue
|
||||
}
|
||||
if lastUnderscore {
|
||||
continue
|
||||
}
|
||||
b.WriteByte('_')
|
||||
lastUnderscore = true
|
||||
}
|
||||
return strings.Trim(b.String(), "_")
|
||||
}
|
||||
@@ -0,0 +1,270 @@
|
||||
package topdata
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"io/fs"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"slices"
|
||||
"strings"
|
||||
)
|
||||
|
||||
const (
|
||||
racialtypesRegistryDirName = "racialtypes/registry"
|
||||
racialtypesRegistryLock = "lock.json"
|
||||
)
|
||||
|
||||
type racialtypesRegistry struct {
|
||||
RootPath string
|
||||
LockPath string
|
||||
LockData map[string]int
|
||||
BaseColumns []string
|
||||
BaseRows []map[string]any
|
||||
Races []map[string]any
|
||||
}
|
||||
|
||||
func loadRacialtypesRegistry(dataDir string) (*racialtypesRegistry, error) {
|
||||
root := filepath.Join(dataDir, filepath.FromSlash(racialtypesRegistryDirName))
|
||||
info, err := os.Stat(root)
|
||||
if err != nil {
|
||||
if os.IsNotExist(err) {
|
||||
return nil, nil
|
||||
}
|
||||
return nil, err
|
||||
}
|
||||
if !info.IsDir() {
|
||||
return nil, fmt.Errorf("racialtypes registry root must be a directory: %s", root)
|
||||
}
|
||||
|
||||
lockPath := filepath.Join(root, racialtypesRegistryLock)
|
||||
lockData, err := loadLockfile(lockPath)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
basePath := filepath.Join(root, "base.json")
|
||||
baseObj, err := loadJSONObject(basePath)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
baseColumns, err := parseColumns(baseObj, "racialtypes/registry/core")
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
rawBaseRows, ok := baseObj["rows"].([]any)
|
||||
if !ok {
|
||||
return nil, fmt.Errorf("%s: rows must be an array", basePath)
|
||||
}
|
||||
baseRows := make([]map[string]any, 0, len(rawBaseRows))
|
||||
for index, raw := range rawBaseRows {
|
||||
row, ok := raw.(map[string]any)
|
||||
if !ok {
|
||||
return nil, fmt.Errorf("%s: row %d must be an object", basePath, index)
|
||||
}
|
||||
baseRows = append(baseRows, row)
|
||||
}
|
||||
|
||||
raceRows, err := loadRacialtypesRegistryRaceRows(filepath.Join(root, "races"))
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return &racialtypesRegistry{
|
||||
RootPath: root,
|
||||
LockPath: lockPath,
|
||||
LockData: lockData,
|
||||
BaseColumns: baseColumns,
|
||||
BaseRows: baseRows,
|
||||
Races: raceRows,
|
||||
}, nil
|
||||
}
|
||||
|
||||
func loadRacialtypesRegistryRaceRows(dir string) ([]map[string]any, error) {
|
||||
info, err := os.Stat(dir)
|
||||
if err != nil {
|
||||
if os.IsNotExist(err) {
|
||||
return nil, nil
|
||||
}
|
||||
return nil, err
|
||||
}
|
||||
if !info.IsDir() {
|
||||
return nil, fmt.Errorf("racialtypes registry races path must be a directory: %s", dir)
|
||||
}
|
||||
|
||||
paths := make([]string, 0)
|
||||
err = filepath.WalkDir(dir, func(path string, d fs.DirEntry, err error) error {
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if d.IsDir() {
|
||||
return nil
|
||||
}
|
||||
if strings.HasSuffix(strings.ToLower(d.Name()), ".json") {
|
||||
paths = append(paths, path)
|
||||
}
|
||||
return nil
|
||||
})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
slices.Sort(paths)
|
||||
|
||||
rows := make([]map[string]any, 0, len(paths))
|
||||
for _, path := range paths {
|
||||
obj, err := loadJSONObject(path)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
rows = append(rows, obj)
|
||||
}
|
||||
return rows, nil
|
||||
}
|
||||
|
||||
func collectRacialtypesRegistryDatasets(dataDir string) ([]nativeCollectedDataset, error) {
|
||||
registry, err := loadRacialtypesRegistry(dataDir)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if registry == nil {
|
||||
return nil, nil
|
||||
}
|
||||
|
||||
lockModified, err := syncRacialtypesBaseLockData(registry.BaseRows, registry.LockData)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
raceIDs, raceLockModified, err := assignRegistryIDs(registry.Races, registry.LockData, "racialtypes:")
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
lockModified = lockModified || raceLockModified
|
||||
if lockModified {
|
||||
if err := saveLockfile(registry.LockPath, registry.LockData); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
|
||||
coreDataset := nativeDataset{
|
||||
Kind: nativeDatasetBase,
|
||||
Name: "racialtypes/registry/core",
|
||||
OutputName: "racialtypes.2da",
|
||||
Spec: specForDataset("racialtypes"),
|
||||
}
|
||||
featDataset := nativeDataset{
|
||||
Kind: nativeDatasetPlain,
|
||||
Name: "racialtypes/registry/feats",
|
||||
Spec: specForDataset("racialtypes"),
|
||||
}
|
||||
|
||||
coreRows := make([]map[string]any, 0, len(registry.BaseRows)+len(registry.Races))
|
||||
rowIDSeen := map[int]string{}
|
||||
for index, raw := range registry.BaseRows {
|
||||
row, err := canonicalizeBaseRow(coreDataset, registry.BaseColumns, raw, index)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("racialtypes registry base row %d: %w", index, err)
|
||||
}
|
||||
rowID := row["id"].(int)
|
||||
if existing, ok := rowIDSeen[rowID]; ok {
|
||||
return nil, fmt.Errorf("racialtypes registry base row id %d conflicts with %s", rowID, existing)
|
||||
}
|
||||
rowIDSeen[rowID] = "base"
|
||||
coreRows = append(coreRows, row)
|
||||
}
|
||||
|
||||
datasets := make([]nativeCollectedDataset, 0, 16)
|
||||
for _, race := range registry.Races {
|
||||
key := stringField(race, "key")
|
||||
if key == "" {
|
||||
return nil, fmt.Errorf("racialtypes registry row is missing key")
|
||||
}
|
||||
rowID, ok := raceIDs[key]
|
||||
if !ok {
|
||||
return nil, fmt.Errorf("racialtypes registry row %s is missing lock id", key)
|
||||
}
|
||||
if existing, ok := rowIDSeen[rowID]; ok {
|
||||
return nil, fmt.Errorf("racialtypes registry row id %d for %s conflicts with %s", rowID, key, existing)
|
||||
}
|
||||
rowIDSeen[rowID] = key
|
||||
|
||||
coreRaw, ok := race["core"].(map[string]any)
|
||||
if !ok {
|
||||
return nil, fmt.Errorf("%s: core must be an object", key)
|
||||
}
|
||||
row, err := canonicalizeEntry(coreDataset, registry.BaseColumns, rowID, key, coreRaw)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("%s: %w", key, err)
|
||||
}
|
||||
|
||||
featOutput, _ := race["feat_output"].(string)
|
||||
if featOutput != "" {
|
||||
row["FeatsTable"] = outputStem(featOutput)
|
||||
}
|
||||
coreRows = append(coreRows, row)
|
||||
|
||||
if featOutput == "" {
|
||||
continue
|
||||
}
|
||||
rawFeats, ok := race["feats"].([]any)
|
||||
if !ok {
|
||||
return nil, fmt.Errorf("%s: feats must be an array when feat_output is set", key)
|
||||
}
|
||||
featRows := make([]map[string]any, 0, len(rawFeats))
|
||||
for index, rawFeat := range rawFeats {
|
||||
featRow, ok := rawFeat.(map[string]any)
|
||||
if !ok {
|
||||
return nil, fmt.Errorf("%s: feat row %d must be an object", key, index)
|
||||
}
|
||||
row, err := canonicalizeBaseRow(featDataset, []string{"FeatLabel", "FeatIndex"}, featRow, index)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("%s: feat row %d: %w", key, index, err)
|
||||
}
|
||||
featRows = append(featRows, row)
|
||||
}
|
||||
datasets = append(datasets, newGeneratedDataset(
|
||||
"racialtypes/registry/"+strings.TrimPrefix(key, "racialtypes:"),
|
||||
featOutput,
|
||||
[]string{"FeatLabel", "FeatIndex"},
|
||||
featRows,
|
||||
))
|
||||
}
|
||||
|
||||
slices.SortFunc(coreRows, func(a, b map[string]any) int {
|
||||
return a["id"].(int) - b["id"].(int)
|
||||
})
|
||||
slices.SortFunc(datasets, func(a, b nativeCollectedDataset) int {
|
||||
return strings.Compare(a.Dataset.OutputName, b.Dataset.OutputName)
|
||||
})
|
||||
|
||||
coreCollected := newGeneratedDataset("racialtypes/registry/core", "racialtypes.2da", registry.BaseColumns, coreRows)
|
||||
out := make([]nativeCollectedDataset, 0, len(datasets)+1)
|
||||
out = append(out, coreCollected)
|
||||
out = append(out, datasets...)
|
||||
return out, nil
|
||||
}
|
||||
|
||||
func syncRacialtypesBaseLockData(rows []map[string]any, lockData map[string]int) (bool, error) {
|
||||
modified := false
|
||||
for _, raw := range rows {
|
||||
key := stringField(raw, "key")
|
||||
if key == "" {
|
||||
continue
|
||||
}
|
||||
rowID, ok := raw["id"]
|
||||
if !ok {
|
||||
return false, fmt.Errorf("racialtypes registry base row %s is missing id", key)
|
||||
}
|
||||
id, err := asInt(rowID)
|
||||
if err != nil {
|
||||
return false, fmt.Errorf("racialtypes registry base row %s has invalid id: %w", key, err)
|
||||
}
|
||||
if existing, ok := lockData[key]; ok {
|
||||
if existing != id {
|
||||
return false, fmt.Errorf("racialtypes registry base row %s id %d does not match lock id %d", key, id, existing)
|
||||
}
|
||||
continue
|
||||
}
|
||||
lockData[key] = id
|
||||
modified = true
|
||||
}
|
||||
return modified, nil
|
||||
}
|
||||
@@ -0,0 +1,63 @@
|
||||
package topdata
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"os"
|
||||
"path/filepath"
|
||||
)
|
||||
|
||||
func importLegacyRepadjust(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
|
||||
_ = legacyTLK
|
||||
|
||||
legacyDir := filepath.Join(referenceBuilderDir, "data", "repadjust")
|
||||
if _, err := os.Stat(legacyDir); err != nil {
|
||||
if os.IsNotExist(err) {
|
||||
return 0, nil
|
||||
}
|
||||
return 0, err
|
||||
}
|
||||
|
||||
targetDir := filepath.Join(dataDir, "repadjust")
|
||||
targetBasePath := filepath.Join(targetDir, "base.json")
|
||||
targetLockPath := filepath.Join(targetDir, "lock.json")
|
||||
if fileExists(targetBasePath) && fileExists(targetLockPath) {
|
||||
obj, err := loadJSONObject(targetBasePath)
|
||||
if err == nil && countLegacyTLKRefsInValue(obj) == 0 {
|
||||
return 0, nil
|
||||
}
|
||||
}
|
||||
|
||||
baseObj, err := loadJSONObject(filepath.Join(legacyDir, "base.json"))
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
baseObj["output"] = "repadjust.2da"
|
||||
|
||||
lockObj, err := loadJSONObject(filepath.Join(legacyDir, "lock.json"))
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
|
||||
if err := os.MkdirAll(targetDir, 0o755); err != nil {
|
||||
return 0, err
|
||||
}
|
||||
|
||||
writes := []struct {
|
||||
path string
|
||||
obj map[string]any
|
||||
}{
|
||||
{path: targetBasePath, obj: baseObj},
|
||||
{path: targetLockPath, obj: lockObj},
|
||||
}
|
||||
for _, write := range writes {
|
||||
raw, err := json.MarshalIndent(write.obj, "", " ")
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
raw = append(raw, '\n')
|
||||
if err := os.WriteFile(write.path, raw, 0o644); err != nil {
|
||||
return 0, err
|
||||
}
|
||||
}
|
||||
return len(writes), nil
|
||||
}
|
||||
@@ -0,0 +1,171 @@
|
||||
package topdata
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"sort"
|
||||
)
|
||||
|
||||
func importLegacyRuleset(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
|
||||
_ = legacyTLK
|
||||
|
||||
legacyDir := filepath.Join(referenceBuilderDir, "data", "ruleset")
|
||||
if _, err := os.Stat(legacyDir); err != nil {
|
||||
if os.IsNotExist(err) {
|
||||
return 0, nil
|
||||
}
|
||||
return 0, err
|
||||
}
|
||||
|
||||
targetDir := filepath.Join(dataDir, "ruleset")
|
||||
targetPath := filepath.Join(targetDir, "ruleset.json")
|
||||
if _, err := os.Stat(targetPath); err == nil {
|
||||
obj, err := loadJSONObject(targetPath)
|
||||
if err == nil && countLegacyTLKRefsInValue(obj) == 0 {
|
||||
return 0, nil
|
||||
}
|
||||
}
|
||||
|
||||
baseObj, err := loadJSONObject(filepath.Join(legacyDir, "base.json"))
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
rows, err := mergeRulesetRows(baseObj, filepath.Join(legacyDir, "modules"))
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
|
||||
if err := os.MkdirAll(targetDir, 0o755); err != nil {
|
||||
return 0, err
|
||||
}
|
||||
if err := removePathIfExists(filepath.Join(targetDir, "base.json")); err != nil {
|
||||
return 0, err
|
||||
}
|
||||
if err := removePathIfExists(filepath.Join(targetDir, "lock.json")); err != nil {
|
||||
return 0, err
|
||||
}
|
||||
if err := removePathIfExists(filepath.Join(targetDir, "modules")); err != nil {
|
||||
return 0, err
|
||||
}
|
||||
|
||||
out := map[string]any{
|
||||
"output": "ruleset.2da",
|
||||
"columns": baseObj["columns"],
|
||||
"rows": rows,
|
||||
}
|
||||
raw, err := json.MarshalIndent(out, "", " ")
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
raw = append(raw, '\n')
|
||||
if err := os.WriteFile(targetPath, raw, 0o644); err != nil {
|
||||
return 0, err
|
||||
}
|
||||
return 1, nil
|
||||
}
|
||||
|
||||
func mergeRulesetRows(baseObj map[string]any, modulesDir string) ([]any, error) {
|
||||
rawRows, ok := baseObj["rows"].([]any)
|
||||
if !ok {
|
||||
return nil, nil
|
||||
}
|
||||
|
||||
rows := make([]map[string]any, 0, len(rawRows))
|
||||
byLabel := map[string]map[string]any{}
|
||||
for _, raw := range rawRows {
|
||||
row, ok := deepCopyValue(raw).(map[string]any)
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
delete(row, "key")
|
||||
if rawID, ok := row["id"]; ok {
|
||||
id, err := asInt(rawID)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
row["id"] = id
|
||||
}
|
||||
if label, ok := row["Label"].(string); ok && label != "" && label != "****" {
|
||||
if _, exists := byLabel[label]; exists {
|
||||
return nil, fmt.Errorf("duplicate ruleset label %q", label)
|
||||
}
|
||||
byLabel[label] = row
|
||||
}
|
||||
rows = append(rows, row)
|
||||
}
|
||||
|
||||
modulePaths, err := collectModulePaths(modulesDir)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
for _, path := range modulePaths {
|
||||
obj, err := loadJSONObject(path)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
overrideList, ok := obj["overrides"].([]any)
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
for _, raw := range overrideList {
|
||||
override, ok := raw.(map[string]any)
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
matchObj, ok := override["match"].(map[string]any)
|
||||
if !ok {
|
||||
return nil, fmt.Errorf("%s: ruleset override is missing match object", path)
|
||||
}
|
||||
rawLabel, ok := matchObj["Label"]
|
||||
if !ok {
|
||||
return nil, fmt.Errorf("%s: ruleset override match is missing Label", path)
|
||||
}
|
||||
labels, err := normalizeRulesetMatchLabels(rawLabel)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("%s: %w", path, err)
|
||||
}
|
||||
for _, label := range labels {
|
||||
row, ok := byLabel[label]
|
||||
if !ok {
|
||||
return nil, fmt.Errorf("%s: ruleset override target label %q was not found", path, label)
|
||||
}
|
||||
for key, value := range override {
|
||||
if key == "id" || key == "key" || key == "_tlk" || isMetadataField(key) || key == "match" {
|
||||
continue
|
||||
}
|
||||
row[key] = deepCopyValue(value)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
sort.Slice(rows, func(i, j int) bool {
|
||||
return rows[i]["id"].(int) < rows[j]["id"].(int)
|
||||
})
|
||||
out := make([]any, 0, len(rows))
|
||||
for _, row := range rows {
|
||||
out = append(out, row)
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
func normalizeRulesetMatchLabels(raw any) ([]string, error) {
|
||||
switch typed := raw.(type) {
|
||||
case string:
|
||||
return []string{typed}, nil
|
||||
case []any:
|
||||
out := make([]string, 0, len(typed))
|
||||
for _, item := range typed {
|
||||
label, ok := item.(string)
|
||||
if !ok {
|
||||
return nil, fmt.Errorf("ruleset match.Label entries must be strings")
|
||||
}
|
||||
out = append(out, label)
|
||||
}
|
||||
return out, nil
|
||||
default:
|
||||
return nil, fmt.Errorf("ruleset match.Label must be a string or string array")
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,140 @@
|
||||
package topdata
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"os"
|
||||
"path/filepath"
|
||||
)
|
||||
|
||||
func importLegacySkills(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
|
||||
_ = legacyTLK
|
||||
|
||||
legacyDir := filepath.Join(referenceBuilderDir, "data", "skills")
|
||||
if _, err := os.Stat(legacyDir); err != nil {
|
||||
if os.IsNotExist(err) {
|
||||
return 0, nil
|
||||
}
|
||||
return 0, err
|
||||
}
|
||||
|
||||
targetDir := filepath.Join(dataDir, "skills")
|
||||
targetBasePath := filepath.Join(targetDir, "base.json")
|
||||
targetLockPath := filepath.Join(targetDir, "lock.json")
|
||||
targetModulesDir := filepath.Join(targetDir, "modules")
|
||||
moduleNames := []string{
|
||||
"add_athletics.json",
|
||||
"add_disguise.json",
|
||||
"add_linguistics.json",
|
||||
"add_perception.json",
|
||||
"add_scriptcraft.json",
|
||||
"add_sensemotive.json",
|
||||
"add_stealth.json",
|
||||
"add_survival.json",
|
||||
"add_userope.json",
|
||||
filepath.Join("craft", "add_craftalchemy.json"),
|
||||
filepath.Join("craft", "add_craftcooking.json"),
|
||||
filepath.Join("craft", "add_craftjewelry.json"),
|
||||
filepath.Join("craft", "add_craftleatherworking.json"),
|
||||
filepath.Join("craft", "add_craftstonework.json"),
|
||||
filepath.Join("craft", "add_crafttextiles.json"),
|
||||
filepath.Join("craft", "ovr_craftarmorsmithing.json"),
|
||||
filepath.Join("craft", "ovr_crafttinkering.json"),
|
||||
filepath.Join("craft", "ovr_craftweaponsmithing.json"),
|
||||
filepath.Join("craft", "ovr_craftwoodworking.json"),
|
||||
filepath.Join("knowledge", "add_knowledgearcana.json"),
|
||||
filepath.Join("knowledge", "add_knowledgearchitecture.json"),
|
||||
filepath.Join("knowledge", "add_knowledgedungeoneering.json"),
|
||||
filepath.Join("knowledge", "add_knowledgegeography.json"),
|
||||
filepath.Join("knowledge", "add_knowledgehistory.json"),
|
||||
filepath.Join("knowledge", "add_knowledgelocal.json"),
|
||||
filepath.Join("knowledge", "add_knowledgenature.json"),
|
||||
filepath.Join("knowledge", "add_knowledgenobility.json"),
|
||||
filepath.Join("knowledge", "add_knowledgeplanar.json"),
|
||||
filepath.Join("knowledge", "add_knowledgereligion.json"),
|
||||
"ovr_acrobatics.json",
|
||||
"ovr_animalhandling.json",
|
||||
"ovr_appraise.json",
|
||||
"ovr_concentration.json",
|
||||
"ovr_disabledevice.json",
|
||||
"ovr_heal.json",
|
||||
"ovr_hiddenskills.json",
|
||||
"ovr_influence.json",
|
||||
"ovr_openlock.json",
|
||||
"ovr_parry.json",
|
||||
"ovr_searchtowis.json",
|
||||
"ovr_sleightofhand.json",
|
||||
"ovr_taunttointimidate.json",
|
||||
"rmv_removedskills.json",
|
||||
}
|
||||
targetModulePaths := make([]string, 0, len(moduleNames))
|
||||
for _, name := range moduleNames {
|
||||
targetModulePaths = append(targetModulePaths, filepath.Join(targetModulesDir, name))
|
||||
}
|
||||
if fileExists(targetBasePath) && fileExists(targetLockPath) {
|
||||
allModulesPresent := true
|
||||
for _, path := range targetModulePaths {
|
||||
if !fileExists(path) {
|
||||
allModulesPresent = false
|
||||
break
|
||||
}
|
||||
}
|
||||
if allModulesPresent {
|
||||
return 0, nil
|
||||
}
|
||||
}
|
||||
|
||||
baseObj, err := loadJSONObject(filepath.Join(legacyDir, "base.json"))
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
baseObj["output"] = "skills.2da"
|
||||
canonicalizeWikiMetadataDocument(baseObj)
|
||||
|
||||
lockObj, err := loadJSONObject(filepath.Join(legacyDir, "lock.json"))
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
|
||||
moduleObjs := make([]map[string]any, 0, len(moduleNames))
|
||||
for _, name := range moduleNames {
|
||||
obj, err := loadJSONObject(filepath.Join(legacyDir, "modules", name))
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
canonicalizeWikiMetadataDocument(obj)
|
||||
moduleObjs = append(moduleObjs, obj)
|
||||
}
|
||||
|
||||
if err := os.MkdirAll(targetModulesDir, 0o755); err != nil {
|
||||
return 0, err
|
||||
}
|
||||
|
||||
writes := []struct {
|
||||
path string
|
||||
obj map[string]any
|
||||
}{
|
||||
{path: targetBasePath, obj: baseObj},
|
||||
{path: targetLockPath, obj: lockObj},
|
||||
}
|
||||
for i, path := range targetModulePaths {
|
||||
if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil {
|
||||
return 0, err
|
||||
}
|
||||
writes = append(writes, struct {
|
||||
path string
|
||||
obj map[string]any
|
||||
}{path: path, obj: moduleObjs[i]})
|
||||
}
|
||||
|
||||
for _, write := range writes {
|
||||
raw, err := json.MarshalIndent(write.obj, "", " ")
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
raw = append(raw, '\n')
|
||||
if err := os.WriteFile(write.path, raw, 0o644); err != nil {
|
||||
return 0, err
|
||||
}
|
||||
}
|
||||
return len(writes), nil
|
||||
}
|
||||
@@ -0,0 +1,6 @@
|
||||
package topdata
|
||||
|
||||
func importLegacySkyboxes(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
|
||||
_ = legacyTLK
|
||||
return importLegacyDatasetMirror(referenceBuilderDir, dataDir, "skyboxes", "skyboxes.2da", nil)
|
||||
}
|
||||
@@ -0,0 +1,103 @@
|
||||
package topdata
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"os"
|
||||
"path/filepath"
|
||||
)
|
||||
|
||||
func importLegacySpells(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
|
||||
_ = legacyTLK
|
||||
|
||||
legacyDir := filepath.Join(referenceBuilderDir, "data", "spells")
|
||||
if _, err := os.Stat(legacyDir); err != nil {
|
||||
if os.IsNotExist(err) {
|
||||
return 0, nil
|
||||
}
|
||||
return 0, err
|
||||
}
|
||||
|
||||
targetDir := filepath.Join(dataDir, "spells")
|
||||
targetBasePath := filepath.Join(targetDir, "base.json")
|
||||
targetLockPath := filepath.Join(targetDir, "lock.json")
|
||||
targetModulesDir := filepath.Join(targetDir, "modules")
|
||||
moduleNames := []string{
|
||||
"specialattacks.json",
|
||||
"ovr_fixduplicates.json",
|
||||
filepath.Join("bardicmusic", "fascinate.json"),
|
||||
filepath.Join("bardicmusic", "inspirecompetence.json"),
|
||||
filepath.Join("bardicmusic", "inspirecourage.json"),
|
||||
filepath.Join("bardicmusic", "inspiregreatness.json"),
|
||||
filepath.Join("bardicmusic", "inspireheroics.json"),
|
||||
filepath.Join("bardicmusic", "songoffreedom.json"),
|
||||
}
|
||||
targetModulePaths := make([]string, 0, len(moduleNames))
|
||||
for _, name := range moduleNames {
|
||||
targetModulePaths = append(targetModulePaths, filepath.Join(targetModulesDir, name))
|
||||
}
|
||||
if fileExists(targetBasePath) && fileExists(targetLockPath) {
|
||||
allModulesPresent := true
|
||||
for _, path := range targetModulePaths {
|
||||
if !fileExists(path) {
|
||||
allModulesPresent = false
|
||||
break
|
||||
}
|
||||
}
|
||||
if allModulesPresent {
|
||||
return 0, nil
|
||||
}
|
||||
}
|
||||
|
||||
baseObj, err := loadJSONObject(filepath.Join(legacyDir, "base.json"))
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
baseObj["output"] = "spells.2da"
|
||||
|
||||
lockObj, err := loadJSONObject(filepath.Join(legacyDir, "lock.json"))
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
|
||||
moduleObjs := make([]map[string]any, 0, len(moduleNames))
|
||||
for _, name := range moduleNames {
|
||||
obj, err := loadJSONObject(filepath.Join(legacyDir, "modules", name))
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
moduleObjs = append(moduleObjs, obj)
|
||||
}
|
||||
|
||||
if err := os.MkdirAll(targetModulesDir, 0o755); err != nil {
|
||||
return 0, err
|
||||
}
|
||||
|
||||
writes := []struct {
|
||||
path string
|
||||
obj map[string]any
|
||||
}{
|
||||
{path: targetBasePath, obj: baseObj},
|
||||
{path: targetLockPath, obj: lockObj},
|
||||
}
|
||||
for i, path := range targetModulePaths {
|
||||
if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil {
|
||||
return 0, err
|
||||
}
|
||||
writes = append(writes, struct {
|
||||
path string
|
||||
obj map[string]any
|
||||
}{path: path, obj: moduleObjs[i]})
|
||||
}
|
||||
|
||||
for _, write := range writes {
|
||||
raw, err := json.MarshalIndent(write.obj, "", " ")
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
raw = append(raw, '\n')
|
||||
if err := os.WriteFile(write.path, raw, 0o644); err != nil {
|
||||
return 0, err
|
||||
}
|
||||
}
|
||||
return len(writes), nil
|
||||
}
|
||||
@@ -0,0 +1,86 @@
|
||||
package topdata
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"os"
|
||||
"path/filepath"
|
||||
)
|
||||
|
||||
func importLegacyTailmodel(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
|
||||
_ = legacyTLK
|
||||
|
||||
legacyDir := filepath.Join(referenceBuilderDir, "data", "tailmodel")
|
||||
if _, err := os.Stat(legacyDir); err != nil {
|
||||
if os.IsNotExist(err) {
|
||||
return 0, nil
|
||||
}
|
||||
return 0, err
|
||||
}
|
||||
|
||||
targetDir := filepath.Join(dataDir, "tailmodel")
|
||||
targetBasePath := filepath.Join(targetDir, "base.json")
|
||||
targetLockPath := filepath.Join(targetDir, "lock.json")
|
||||
targetModulesDir := filepath.Join(targetDir, "modules")
|
||||
targetModulePaths := []string{
|
||||
filepath.Join(targetModulesDir, "add.json"),
|
||||
filepath.Join(targetModulesDir, "ovr_tailprefixes.json"),
|
||||
}
|
||||
if fileExists(targetBasePath) && fileExists(targetLockPath) {
|
||||
allModulesPresent := true
|
||||
for _, path := range targetModulePaths {
|
||||
if !fileExists(path) {
|
||||
allModulesPresent = false
|
||||
break
|
||||
}
|
||||
}
|
||||
if allModulesPresent {
|
||||
return 0, nil
|
||||
}
|
||||
}
|
||||
|
||||
baseObj, err := loadJSONObject(filepath.Join(legacyDir, "base.json"))
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
baseObj["output"] = "tailmodel.2da"
|
||||
|
||||
lockObj, err := loadJSONObject(filepath.Join(legacyDir, "lock.json"))
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
|
||||
moduleNames := []string{"add.json", "ovr_tailprefixes.json"}
|
||||
moduleObjs := make([]map[string]any, 0, len(moduleNames))
|
||||
for _, name := range moduleNames {
|
||||
obj, err := loadJSONObject(filepath.Join(legacyDir, "modules", name))
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
moduleObjs = append(moduleObjs, obj)
|
||||
}
|
||||
|
||||
if err := os.MkdirAll(targetModulesDir, 0o755); err != nil {
|
||||
return 0, err
|
||||
}
|
||||
|
||||
writes := []struct {
|
||||
path string
|
||||
obj map[string]any
|
||||
}{
|
||||
{path: targetBasePath, obj: baseObj},
|
||||
{path: targetLockPath, obj: lockObj},
|
||||
{path: targetModulePaths[0], obj: moduleObjs[0]},
|
||||
{path: targetModulePaths[1], obj: moduleObjs[1]},
|
||||
}
|
||||
for _, write := range writes {
|
||||
raw, err := json.MarshalIndent(write.obj, "", " ")
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
raw = append(raw, '\n')
|
||||
if err := os.WriteFile(write.path, raw, 0o644); err != nil {
|
||||
return 0, err
|
||||
}
|
||||
}
|
||||
return len(writes), nil
|
||||
}
|
||||
@@ -0,0 +1,647 @@
|
||||
package topdata
|
||||
|
||||
import (
|
||||
"encoding/binary"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"math"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"slices"
|
||||
"strings"
|
||||
|
||||
"golang.org/x/text/encoding/charmap"
|
||||
)
|
||||
|
||||
const (
|
||||
defaultTLKName = "sow_tlk.tlk"
|
||||
tlkStateFile = ".tlk_state.json"
|
||||
customTLKBase = 0x01000000
|
||||
)
|
||||
|
||||
type textRole string
|
||||
|
||||
const (
|
||||
textRoleName textRole = "name"
|
||||
textRoleDescription textRole = "description"
|
||||
)
|
||||
|
||||
type datasetTextSpec struct {
|
||||
NameFields []string
|
||||
DescriptionFields []string
|
||||
}
|
||||
|
||||
var datasetTextSpecs = map[string]datasetTextSpec{
|
||||
"baseitems": {
|
||||
NameFields: []string{"Name"},
|
||||
DescriptionFields: []string{"Description"},
|
||||
},
|
||||
"classes": {
|
||||
NameFields: []string{"Short", "Name", "Plural", "Lower", "CLASS"},
|
||||
DescriptionFields: []string{"Description"},
|
||||
},
|
||||
"feat": {
|
||||
NameFields: []string{"FEAT"},
|
||||
DescriptionFields: []string{"DESCRIPTION"},
|
||||
},
|
||||
"masterfeats": {
|
||||
NameFields: []string{"STRREF"},
|
||||
DescriptionFields: []string{"DESCRIPTION"},
|
||||
},
|
||||
"itemprops": {
|
||||
NameFields: []string{"Name", "StringRef"},
|
||||
DescriptionFields: []string{"Description", "GameStrRef"},
|
||||
},
|
||||
"racialtypes": {
|
||||
NameFields: []string{"Name", "RACIALTYPE"},
|
||||
DescriptionFields: []string{"Description"},
|
||||
},
|
||||
"skills": {
|
||||
NameFields: []string{"Name", "SKILL"},
|
||||
DescriptionFields: []string{"Description"},
|
||||
},
|
||||
"spells": {
|
||||
NameFields: []string{"Name", "SPELL"},
|
||||
DescriptionFields: []string{"SpellDesc", "DESCRIPTION"},
|
||||
},
|
||||
}
|
||||
|
||||
type tlkEntryData struct {
|
||||
Text string
|
||||
SoundResRef string
|
||||
SoundLength float32
|
||||
}
|
||||
|
||||
type tlkStateDocument struct {
|
||||
Version int `json:"version"`
|
||||
Language string `json:"language"`
|
||||
Entries map[string]tlkStateMapping `json:"entries"`
|
||||
}
|
||||
|
||||
type tlkStateMapping struct {
|
||||
ID int `json:"id"`
|
||||
Retired bool `json:"retired,omitempty"`
|
||||
}
|
||||
|
||||
type tlkCompiledValue struct {
|
||||
Value any
|
||||
TLKKey string
|
||||
}
|
||||
|
||||
type tlkCompiler struct {
|
||||
language string
|
||||
statePath string
|
||||
state tlkStateDocument
|
||||
legacy map[string]tlkEntryData
|
||||
active map[string]tlkEntryData
|
||||
activeKeys map[string]struct{}
|
||||
reservedByID map[int]string
|
||||
nextID int
|
||||
}
|
||||
|
||||
func newTLKCompiler(sourceDir string, legacy *legacyTLKData) (*tlkCompiler, error) {
|
||||
statePath := filepath.Join(sourceDir, tlkStateFile)
|
||||
state, err := loadTLKState(statePath)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
baseDialog, err := loadBaseDialogData(filepath.Join(sourceDir, "base_dialog.json"))
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
legacy = mergeLegacyTLK(baseDialog, legacy)
|
||||
language := state.Language
|
||||
if language == "" {
|
||||
language = "en"
|
||||
}
|
||||
if legacy != nil && legacy.Language != "" {
|
||||
if state.Language == "" {
|
||||
language = legacy.Language
|
||||
}
|
||||
for key, id := range legacy.Lock {
|
||||
if _, ok := state.Entries[key]; !ok {
|
||||
state.Entries[key] = tlkStateMapping{ID: id}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
reserved := map[int]string{}
|
||||
nextID := 0
|
||||
for key, entry := range state.Entries {
|
||||
if owner, ok := reserved[entry.ID]; ok && owner != key {
|
||||
return nil, fmt.Errorf("tlk state id collision between %q and %q", owner, key)
|
||||
}
|
||||
reserved[entry.ID] = key
|
||||
if entry.ID >= nextID {
|
||||
nextID = entry.ID + 1
|
||||
}
|
||||
}
|
||||
|
||||
compiler := &tlkCompiler{
|
||||
language: language,
|
||||
statePath: statePath,
|
||||
state: state,
|
||||
legacy: map[string]tlkEntryData{},
|
||||
active: map[string]tlkEntryData{},
|
||||
activeKeys: map[string]struct{}{},
|
||||
reservedByID: reserved,
|
||||
nextID: nextID,
|
||||
}
|
||||
if legacy != nil {
|
||||
for key, entry := range legacy.Entries {
|
||||
compiler.legacy[key] = entry
|
||||
}
|
||||
}
|
||||
return compiler, nil
|
||||
}
|
||||
|
||||
func loadBaseDialogData(path string) (*legacyTLKData, error) {
|
||||
if _, err := os.Stat(path); err != nil {
|
||||
if os.IsNotExist(err) {
|
||||
return nil, nil
|
||||
}
|
||||
return nil, fmt.Errorf("stat %s: %w", path, err)
|
||||
}
|
||||
obj, err := loadJSONObject(path)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
result := &legacyTLKData{
|
||||
Language: "en",
|
||||
Entries: map[string]tlkEntryData{},
|
||||
Lock: map[string]int{},
|
||||
}
|
||||
if language, ok := obj["language"].(string); ok && strings.TrimSpace(language) != "" {
|
||||
result.Language = language
|
||||
}
|
||||
rawEntries, ok := obj["entries"]
|
||||
if !ok {
|
||||
return result, nil
|
||||
}
|
||||
entries, ok := rawEntries.(map[string]any)
|
||||
if !ok {
|
||||
return nil, fmt.Errorf("base_dialog.json entries must be an object")
|
||||
}
|
||||
normalized, err := normalizeLegacyTLKEntries(entries)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
for key, entry := range normalized {
|
||||
result.Entries[key] = entry
|
||||
}
|
||||
return result, nil
|
||||
}
|
||||
|
||||
func loadTLKState(path string) (tlkStateDocument, error) {
|
||||
doc := tlkStateDocument{
|
||||
Version: 1,
|
||||
Entries: map[string]tlkStateMapping{},
|
||||
}
|
||||
raw, err := os.ReadFile(path)
|
||||
if err != nil {
|
||||
if os.IsNotExist(err) {
|
||||
return doc, nil
|
||||
}
|
||||
return doc, fmt.Errorf("read %s: %w", path, err)
|
||||
}
|
||||
if err := json.Unmarshal(raw, &doc); err != nil {
|
||||
return doc, fmt.Errorf("parse %s: %w", path, err)
|
||||
}
|
||||
if doc.Version == 0 {
|
||||
doc.Version = 1
|
||||
}
|
||||
if doc.Entries == nil {
|
||||
doc.Entries = map[string]tlkStateMapping{}
|
||||
}
|
||||
return doc, nil
|
||||
}
|
||||
|
||||
func saveTLKState(path string, doc tlkStateDocument) error {
|
||||
doc.Version = 1
|
||||
if doc.Language == "" {
|
||||
doc.Language = "en"
|
||||
}
|
||||
if doc.Entries == nil {
|
||||
doc.Entries = map[string]tlkStateMapping{}
|
||||
}
|
||||
raw, err := json.MarshalIndent(doc, "", " ")
|
||||
if err != nil {
|
||||
return fmt.Errorf("marshal tlk state: %w", err)
|
||||
}
|
||||
raw = append(raw, '\n')
|
||||
if err := os.WriteFile(path, raw, 0o644); err != nil {
|
||||
return fmt.Errorf("write %s: %w", path, err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (c *tlkCompiler) resolveLegacyKey(key string) (tlkCompiledValue, error) {
|
||||
if _, ok := c.legacy[key]; !ok {
|
||||
if active, ok := c.active[key]; ok {
|
||||
return tlkCompiledValue{
|
||||
Value: c.customStrrefForKey(key),
|
||||
TLKKey: key,
|
||||
}, c.markActive(key, active)
|
||||
}
|
||||
if len(c.legacy) == 0 {
|
||||
return tlkCompiledValue{}, fmt.Errorf("unknown TLK reference: %s (inline TLK text is required for native builds)", key)
|
||||
}
|
||||
return tlkCompiledValue{}, fmt.Errorf("unknown TLK reference: %s", key)
|
||||
}
|
||||
if err := c.markActive(key, c.legacy[key]); err != nil {
|
||||
return tlkCompiledValue{}, err
|
||||
}
|
||||
return tlkCompiledValue{
|
||||
Value: c.customStrrefForKey(key),
|
||||
TLKKey: key,
|
||||
}, nil
|
||||
}
|
||||
|
||||
func (c *tlkCompiler) registerInline(key string, entry tlkEntryData) (tlkCompiledValue, error) {
|
||||
if err := c.markActive(key, entry); err != nil {
|
||||
return tlkCompiledValue{}, err
|
||||
}
|
||||
return tlkCompiledValue{
|
||||
Value: c.customStrrefForKey(key),
|
||||
TLKKey: key,
|
||||
}, nil
|
||||
}
|
||||
|
||||
func (c *tlkCompiler) customStrrefForKey(key string) int {
|
||||
return customTLKBase + c.state.Entries[key].ID
|
||||
}
|
||||
|
||||
func (c *tlkCompiler) markActive(key string, entry tlkEntryData) error {
|
||||
mapping, ok := c.state.Entries[key]
|
||||
if !ok {
|
||||
mapping = tlkStateMapping{ID: c.allocateID(), Retired: false}
|
||||
c.state.Entries[key] = mapping
|
||||
}
|
||||
existing, ok := c.active[key]
|
||||
if ok && existing != entry {
|
||||
return fmt.Errorf("conflicting TLK content for key %q", key)
|
||||
}
|
||||
c.active[key] = entry
|
||||
c.activeKeys[key] = struct{}{}
|
||||
mapping.Retired = false
|
||||
c.state.Entries[key] = mapping
|
||||
return nil
|
||||
}
|
||||
|
||||
func (c *tlkCompiler) allocateID() int {
|
||||
id := c.nextID
|
||||
c.nextID++
|
||||
return id
|
||||
}
|
||||
|
||||
func (c *tlkCompiler) finish(outputDir string) (int, error) {
|
||||
if err := os.MkdirAll(outputDir, 0o755); err != nil {
|
||||
return 0, fmt.Errorf("create tlk output dir: %w", err)
|
||||
}
|
||||
|
||||
for key, mapping := range c.state.Entries {
|
||||
if _, ok := c.activeKeys[key]; ok {
|
||||
mapping.Retired = false
|
||||
} else {
|
||||
mapping.Retired = true
|
||||
}
|
||||
c.state.Entries[key] = mapping
|
||||
}
|
||||
pruneRetiredGeneratedFeatTLKEntries(c.state.Entries)
|
||||
|
||||
if err := saveTLKState(c.statePath, c.state); err != nil {
|
||||
return 0, err
|
||||
}
|
||||
|
||||
maxID := -1
|
||||
for key := range c.activeKeys {
|
||||
id := c.state.Entries[key].ID
|
||||
if id > maxID {
|
||||
maxID = id
|
||||
}
|
||||
}
|
||||
entries := make([]tlkEntryData, maxID+1)
|
||||
for key := range c.activeKeys {
|
||||
id := c.state.Entries[key].ID
|
||||
entries[id] = c.active[key]
|
||||
}
|
||||
|
||||
if err := writeTLKBinary(filepath.Join(outputDir, defaultTLKName), entries, c.language); err != nil {
|
||||
return 0, err
|
||||
}
|
||||
if maxID < 0 {
|
||||
return 1, nil
|
||||
}
|
||||
return 1, nil
|
||||
}
|
||||
|
||||
var generatedFeatTLKPrefixes = []string{
|
||||
"feat:skillfocus_",
|
||||
"feat:greaterskillfocus_",
|
||||
"feat:weaponfocus_",
|
||||
"feat:weaponspecialization_",
|
||||
"feat:greaterweaponfocus_",
|
||||
"feat:greaterweaponspecialization_",
|
||||
"feat:improvedcritical_",
|
||||
"feat:overwhelmingcritical_",
|
||||
"feat:weaponproficiencycommoner_",
|
||||
}
|
||||
|
||||
func pruneRetiredGeneratedFeatTLKEntries(entries map[string]tlkStateMapping) {
|
||||
for key, mapping := range entries {
|
||||
if !mapping.Retired || !isRetiredGeneratedFeatTLKKey(key) {
|
||||
continue
|
||||
}
|
||||
delete(entries, key)
|
||||
}
|
||||
}
|
||||
|
||||
func isRetiredGeneratedFeatTLKKey(key string) bool {
|
||||
if !strings.HasPrefix(key, "feat:") {
|
||||
return false
|
||||
}
|
||||
for _, prefix := range generatedFeatTLKPrefixes {
|
||||
if strings.HasPrefix(key, prefix) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
func writeTLKBinary(path string, entries []tlkEntryData, language string) error {
|
||||
if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil {
|
||||
return fmt.Errorf("create tlk output parent: %w", err)
|
||||
}
|
||||
file, err := os.Create(path)
|
||||
if err != nil {
|
||||
return fmt.Errorf("create tlk output: %w", err)
|
||||
}
|
||||
defer file.Close()
|
||||
|
||||
langID, err := languageID(language)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
if _, err := file.Write([]byte("TLK V3.0")); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := binary.Write(file, binary.LittleEndian, uint32(langID)); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := binary.Write(file, binary.LittleEndian, uint32(len(entries))); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := binary.Write(file, binary.LittleEndian, uint32(20+len(entries)*40)); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
stringData := make([][]byte, len(entries))
|
||||
offset := 0
|
||||
for index, entry := range entries {
|
||||
flags := uint32(0)
|
||||
if entry.Text != "" {
|
||||
flags |= 0x1
|
||||
}
|
||||
if entry.SoundResRef != "" {
|
||||
flags |= 0x2
|
||||
}
|
||||
if entry.SoundLength != 0 {
|
||||
flags |= 0x4
|
||||
}
|
||||
encodedText, err := charmap.Windows1252.NewEncoder().Bytes([]byte(entry.Text))
|
||||
if err != nil {
|
||||
return fmt.Errorf("encode tlk text at row %d: %w", index, err)
|
||||
}
|
||||
stringData[index] = encodedText
|
||||
|
||||
soundBytes := make([]byte, 16)
|
||||
if len(entry.SoundResRef) > 16 {
|
||||
return fmt.Errorf("sound resref at row %d exceeds 16 characters", index)
|
||||
}
|
||||
copy(soundBytes, []byte(entry.SoundResRef))
|
||||
|
||||
if err := binary.Write(file, binary.LittleEndian, flags); err != nil {
|
||||
return err
|
||||
}
|
||||
if _, err := file.Write(soundBytes); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := binary.Write(file, binary.LittleEndian, uint32(0)); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := binary.Write(file, binary.LittleEndian, uint32(0)); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := binary.Write(file, binary.LittleEndian, uint32(offset)); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := binary.Write(file, binary.LittleEndian, uint32(len(encodedText))); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := binary.Write(file, binary.LittleEndian, entry.SoundLength); err != nil {
|
||||
return err
|
||||
}
|
||||
offset += len(encodedText)
|
||||
}
|
||||
|
||||
for _, encoded := range stringData {
|
||||
if _, err := file.Write(encoded); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func languageID(code string) (int, error) {
|
||||
switch strings.ToLower(strings.TrimSpace(code)) {
|
||||
case "", "en":
|
||||
return 0, nil
|
||||
case "fr":
|
||||
return 1, nil
|
||||
case "de":
|
||||
return 2, nil
|
||||
case "it":
|
||||
return 3, nil
|
||||
case "es":
|
||||
return 4, nil
|
||||
case "pl":
|
||||
return 5, nil
|
||||
default:
|
||||
return 0, fmt.Errorf("unknown TLK language code %q", code)
|
||||
}
|
||||
}
|
||||
|
||||
func specForDataset(name string) datasetTextSpec {
|
||||
best := datasetTextSpecs[name]
|
||||
bestLength := 0
|
||||
for prefix, spec := range datasetTextSpecs {
|
||||
if name != prefix && !strings.HasPrefix(name, prefix+"/") {
|
||||
continue
|
||||
}
|
||||
if len(prefix) <= bestLength {
|
||||
continue
|
||||
}
|
||||
best = spec
|
||||
bestLength = len(prefix)
|
||||
}
|
||||
return best
|
||||
}
|
||||
|
||||
func roleForField(spec datasetTextSpec, field string) textRole {
|
||||
for _, candidate := range spec.NameFields {
|
||||
if strings.EqualFold(candidate, field) {
|
||||
return textRoleName
|
||||
}
|
||||
}
|
||||
for _, candidate := range spec.DescriptionFields {
|
||||
if strings.EqualFold(candidate, field) {
|
||||
return textRoleDescription
|
||||
}
|
||||
}
|
||||
return textRole(strings.ToLower(field))
|
||||
}
|
||||
|
||||
func columnForRole(columns []string, spec datasetTextSpec, role textRole) string {
|
||||
var candidates []string
|
||||
switch role {
|
||||
case textRoleName:
|
||||
candidates = spec.NameFields
|
||||
case textRoleDescription:
|
||||
candidates = spec.DescriptionFields
|
||||
default:
|
||||
return ""
|
||||
}
|
||||
for _, candidate := range candidates {
|
||||
if column, ok := canonicalColumn(columns, candidate); ok {
|
||||
return column
|
||||
}
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
type parsedTLKPayload struct {
|
||||
Key string
|
||||
Text string
|
||||
Ref string
|
||||
RefField string
|
||||
SoundResRef string
|
||||
SoundLength float32
|
||||
}
|
||||
|
||||
func parseTLKPayload(value any, allowBare bool) (parsedTLKPayload, bool, error) {
|
||||
switch typed := value.(type) {
|
||||
case map[string]any:
|
||||
if rawTLK, ok := typed["tlk"]; ok {
|
||||
if key, ok := rawTLK.(string); ok {
|
||||
return parsedTLKPayload{Key: key}, true, nil
|
||||
}
|
||||
return parseTLKPayload(rawTLK, true)
|
||||
}
|
||||
if allowBare || hasNonRefTLKPayloadKeys(typed) {
|
||||
if _, ok := typed["text"]; ok {
|
||||
return buildParsedTLKPayload(typed)
|
||||
}
|
||||
if _, ok := typed["ref"]; ok {
|
||||
return buildParsedTLKPayload(typed)
|
||||
}
|
||||
if _, ok := typed["key"]; ok {
|
||||
return buildParsedTLKPayload(typed)
|
||||
}
|
||||
}
|
||||
return parsedTLKPayload{}, false, nil
|
||||
case string:
|
||||
return parsedTLKPayload{}, false, nil
|
||||
default:
|
||||
return parsedTLKPayload{}, false, nil
|
||||
}
|
||||
}
|
||||
|
||||
func hasNonRefTLKPayloadKeys(obj map[string]any) bool {
|
||||
for _, key := range []string{"text", "key", "sound_resref", "sound_length"} {
|
||||
if _, ok := obj[key]; ok {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
func buildParsedTLKPayload(obj map[string]any) (parsedTLKPayload, bool, error) {
|
||||
payload := parsedTLKPayload{}
|
||||
if rawKey, ok := obj["key"]; ok {
|
||||
key, ok := rawKey.(string)
|
||||
if !ok {
|
||||
return payload, false, fmt.Errorf("tlk key must be a string")
|
||||
}
|
||||
payload.Key = key
|
||||
}
|
||||
if rawText, ok := obj["text"]; ok {
|
||||
text, ok := rawText.(string)
|
||||
if !ok {
|
||||
return payload, false, fmt.Errorf("tlk text must be a string")
|
||||
}
|
||||
payload.Text = text
|
||||
}
|
||||
if rawRef, ok := obj["ref"]; ok {
|
||||
ref, ok := rawRef.(string)
|
||||
if !ok {
|
||||
return payload, false, fmt.Errorf("tlk ref must be a string")
|
||||
}
|
||||
payload.Ref = ref
|
||||
}
|
||||
if rawField, ok := obj["field"]; ok {
|
||||
field, ok := rawField.(string)
|
||||
if !ok {
|
||||
return payload, false, fmt.Errorf("tlk field must be a string")
|
||||
}
|
||||
payload.RefField = field
|
||||
}
|
||||
if rawSound, ok := obj["sound_resref"]; ok {
|
||||
sound, ok := rawSound.(string)
|
||||
if !ok {
|
||||
return payload, false, fmt.Errorf("tlk sound_resref must be a string")
|
||||
}
|
||||
payload.SoundResRef = sound
|
||||
}
|
||||
if rawLength, ok := obj["sound_length"]; ok {
|
||||
switch typed := rawLength.(type) {
|
||||
case float64:
|
||||
payload.SoundLength = float32(typed)
|
||||
case int:
|
||||
payload.SoundLength = float32(typed)
|
||||
default:
|
||||
return payload, false, fmt.Errorf("tlk sound_length must be numeric")
|
||||
}
|
||||
}
|
||||
if payload.Ref != "" && payload.RefField == "" {
|
||||
return payload, false, fmt.Errorf("tlk ref requires field")
|
||||
}
|
||||
if payload.Ref != "" && payload.Text != "" {
|
||||
return payload, false, fmt.Errorf("tlk payload cannot contain both text and ref")
|
||||
}
|
||||
if payload.Ref == "" && payload.Text == "" && payload.Key == "" {
|
||||
return payload, false, fmt.Errorf("tlk payload must contain text, ref, or key")
|
||||
}
|
||||
return payload, true, nil
|
||||
}
|
||||
|
||||
func deriveAutoTLKKey(rowIdentity string, field string) string {
|
||||
fieldText := strings.ToLower(strings.TrimSpace(field))
|
||||
if fieldText == "" {
|
||||
fieldText = "text"
|
||||
}
|
||||
return rowIdentity + "." + fieldText
|
||||
}
|
||||
|
||||
func sortedKeys[M ~map[string]V, V any](input M) []string {
|
||||
keys := make([]string, 0, len(input))
|
||||
for key := range input {
|
||||
keys = append(keys, key)
|
||||
}
|
||||
slices.Sort(keys)
|
||||
return keys
|
||||
}
|
||||
|
||||
func almostEqualFloat32(a, b float32) bool {
|
||||
return math.Abs(float64(a-b)) < 0.0001
|
||||
}
|
||||
+1030
-222
File diff suppressed because it is too large
Load Diff
+8053
-60
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,198 @@
|
||||
package topdata
|
||||
|
||||
import (
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
)
|
||||
|
||||
func importLegacyVFXPersistent(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
|
||||
_ = legacyTLK
|
||||
if strings.TrimSpace(referenceBuilderDir) == "" {
|
||||
return 0, nil
|
||||
}
|
||||
|
||||
legacyDir := filepath.Join(referenceBuilderDir, "data", "vfx_persistent")
|
||||
if _, err := os.Stat(legacyDir); err != nil {
|
||||
if os.IsNotExist(err) {
|
||||
return 0, nil
|
||||
}
|
||||
return 0, err
|
||||
}
|
||||
|
||||
targetDir := filepath.Join(dataDir, "vfx_persistent")
|
||||
targetBasePath := filepath.Join(targetDir, "base.json")
|
||||
targetLockPath := filepath.Join(targetDir, "lock.json")
|
||||
targetModulesDir := filepath.Join(targetDir, "modules")
|
||||
targetModulePath := filepath.Join(targetModulesDir, "freeformaoes.json")
|
||||
if fileExists(targetBasePath) && fileExists(targetLockPath) && fileExists(targetModulePath) {
|
||||
baseObj, baseErr := loadJSONObject(targetBasePath)
|
||||
lockObj, lockErr := loadJSONObject(targetLockPath)
|
||||
if baseErr == nil && lockErr == nil && vfxPersistentImportComplete(baseObj, lockObj) {
|
||||
legacyModulePaths, err := collectJSONRelativePaths(filepath.Join(legacyDir, "modules"))
|
||||
if err == nil {
|
||||
targetModulePaths, err := collectJSONRelativePaths(targetModulesDir)
|
||||
if err == nil && equalStringSlices(legacyModulePaths, targetModulePaths) {
|
||||
return 0, nil
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
baseObj, err := loadJSONObject(filepath.Join(legacyDir, "base.json"))
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
lockObj, err := synthesizeVFXPersistentKeys(baseObj)
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
legacyLockObj, err := loadJSONObject(filepath.Join(legacyDir, "lock.json"))
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
for key, value := range legacyLockObj {
|
||||
lockObj[key] = value
|
||||
}
|
||||
if err := os.MkdirAll(targetModulesDir, 0o755); err != nil {
|
||||
return 0, err
|
||||
}
|
||||
|
||||
updated := 0
|
||||
for _, write := range []struct {
|
||||
path string
|
||||
obj map[string]any
|
||||
}{
|
||||
{path: targetBasePath, obj: baseObj},
|
||||
{path: targetLockPath, obj: lockObj},
|
||||
} {
|
||||
changed, err := writeJSONObjectIfChanged(write.path, write.obj)
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
if changed {
|
||||
updated++
|
||||
}
|
||||
}
|
||||
|
||||
moduleRelPaths, err := collectJSONRelativePaths(filepath.Join(legacyDir, "modules"))
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
for _, relPath := range moduleRelPaths {
|
||||
moduleObj, err := loadJSONObject(filepath.Join(legacyDir, "modules", filepath.FromSlash(relPath)))
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
targetPath := filepath.Join(targetModulesDir, filepath.FromSlash(relPath))
|
||||
if err := os.MkdirAll(filepath.Dir(targetPath), 0o755); err != nil {
|
||||
return 0, err
|
||||
}
|
||||
changed, err := writeJSONObjectIfChanged(targetPath, moduleObj)
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
if changed {
|
||||
updated++
|
||||
}
|
||||
}
|
||||
|
||||
targetModulePaths, err := collectJSONRelativePaths(targetModulesDir)
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
legacyModules := make(map[string]struct{}, len(moduleRelPaths))
|
||||
for _, relPath := range moduleRelPaths {
|
||||
legacyModules[relPath] = struct{}{}
|
||||
}
|
||||
for _, relPath := range targetModulePaths {
|
||||
if _, ok := legacyModules[relPath]; ok {
|
||||
continue
|
||||
}
|
||||
if err := os.Remove(filepath.Join(targetModulesDir, filepath.FromSlash(relPath))); err != nil && !os.IsNotExist(err) {
|
||||
return 0, err
|
||||
}
|
||||
updated++
|
||||
}
|
||||
|
||||
return updated, nil
|
||||
}
|
||||
|
||||
func vfxPersistentImportComplete(baseObj, lockObj map[string]any) bool {
|
||||
if !vfxPersistentRowsHaveKeys(baseObj) {
|
||||
return false
|
||||
}
|
||||
expected := map[string]int{
|
||||
"vfx_persistent:blankradius5": 47,
|
||||
"vfx_persistent:blankradius10": 48,
|
||||
"vfx_persistent:blankradius15": 49,
|
||||
"vfx_persistent:blankradius20": 50,
|
||||
"vfx_persistent:blankradius25": 51,
|
||||
"vfx_persistent:blankradius30": 52,
|
||||
}
|
||||
for key, want := range expected {
|
||||
raw, ok := lockObj[key]
|
||||
if !ok {
|
||||
return false
|
||||
}
|
||||
got, err := asInt(raw)
|
||||
if err != nil || got != want {
|
||||
return false
|
||||
}
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
func synthesizeVFXPersistentKeys(baseObj map[string]any) (map[string]any, error) {
|
||||
rawRows, ok := baseObj["rows"].([]any)
|
||||
if !ok {
|
||||
return map[string]any{}, nil
|
||||
}
|
||||
lockObj := map[string]any{}
|
||||
for _, raw := range rawRows {
|
||||
row, ok := raw.(map[string]any)
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
label, _ := row["LABEL"].(string)
|
||||
if label == "" {
|
||||
continue
|
||||
}
|
||||
key := "vfx_persistent:" + strings.ToLower(label)
|
||||
row["key"] = key
|
||||
if rawID, ok := row["id"]; ok {
|
||||
lockObj[key] = rawID
|
||||
}
|
||||
}
|
||||
return lockObj, nil
|
||||
}
|
||||
|
||||
func vfxPersistentRowsHaveKeys(baseObj map[string]any) bool {
|
||||
rawRows, ok := baseObj["rows"].([]any)
|
||||
if !ok {
|
||||
return false
|
||||
}
|
||||
for _, raw := range rawRows {
|
||||
row, ok := raw.(map[string]any)
|
||||
if !ok {
|
||||
return false
|
||||
}
|
||||
key, _ := row["key"].(string)
|
||||
if key == "" {
|
||||
return false
|
||||
}
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
func equalStringSlices(left, right []string) bool {
|
||||
if len(left) != len(right) {
|
||||
return false
|
||||
}
|
||||
for i := range left {
|
||||
if left[i] != right[i] {
|
||||
return false
|
||||
}
|
||||
}
|
||||
return true
|
||||
}
|
||||
@@ -0,0 +1,6 @@
|
||||
package topdata
|
||||
|
||||
func importLegacyVisualeffects(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
|
||||
_ = legacyTLK
|
||||
return importLegacyDatasetMirror(referenceBuilderDir, dataDir, "visualeffects", "visualeffects.2da", nil)
|
||||
}
|
||||
@@ -0,0 +1,86 @@
|
||||
package topdata
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"os"
|
||||
"path/filepath"
|
||||
)
|
||||
|
||||
func importLegacyWingmodel(referenceBuilderDir, dataDir string, legacyTLK *legacyTLKData) (int, error) {
|
||||
_ = legacyTLK
|
||||
|
||||
legacyDir := filepath.Join(referenceBuilderDir, "data", "wingmodel")
|
||||
if _, err := os.Stat(legacyDir); err != nil {
|
||||
if os.IsNotExist(err) {
|
||||
return 0, nil
|
||||
}
|
||||
return 0, err
|
||||
}
|
||||
|
||||
targetDir := filepath.Join(dataDir, "wingmodel")
|
||||
targetBasePath := filepath.Join(targetDir, "base.json")
|
||||
targetLockPath := filepath.Join(targetDir, "lock.json")
|
||||
targetModulesDir := filepath.Join(targetDir, "modules")
|
||||
targetModulePaths := []string{
|
||||
filepath.Join(targetModulesDir, "add.json"),
|
||||
filepath.Join(targetModulesDir, "ovr_wingprefixes.json"),
|
||||
}
|
||||
if fileExists(targetBasePath) && fileExists(targetLockPath) {
|
||||
allModulesPresent := true
|
||||
for _, path := range targetModulePaths {
|
||||
if !fileExists(path) {
|
||||
allModulesPresent = false
|
||||
break
|
||||
}
|
||||
}
|
||||
if allModulesPresent {
|
||||
return 0, nil
|
||||
}
|
||||
}
|
||||
|
||||
baseObj, err := loadJSONObject(filepath.Join(legacyDir, "base.json"))
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
baseObj["output"] = "wingmodel.2da"
|
||||
|
||||
lockObj, err := loadJSONObject(filepath.Join(legacyDir, "lock.json"))
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
|
||||
moduleNames := []string{"add.json", "ovr_wingprefixes.json"}
|
||||
moduleObjs := make([]map[string]any, 0, len(moduleNames))
|
||||
for _, name := range moduleNames {
|
||||
obj, err := loadJSONObject(filepath.Join(legacyDir, "modules", name))
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
moduleObjs = append(moduleObjs, obj)
|
||||
}
|
||||
|
||||
if err := os.MkdirAll(targetModulesDir, 0o755); err != nil {
|
||||
return 0, err
|
||||
}
|
||||
|
||||
writes := []struct {
|
||||
path string
|
||||
obj map[string]any
|
||||
}{
|
||||
{path: targetBasePath, obj: baseObj},
|
||||
{path: targetLockPath, obj: lockObj},
|
||||
{path: targetModulePaths[0], obj: moduleObjs[0]},
|
||||
{path: targetModulePaths[1], obj: moduleObjs[1]},
|
||||
}
|
||||
for _, write := range writes {
|
||||
raw, err := json.MarshalIndent(write.obj, "", " ")
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
raw = append(raw, '\n')
|
||||
if err := os.WriteFile(write.path, raw, 0o644); err != nil {
|
||||
return 0, err
|
||||
}
|
||||
}
|
||||
return len(writes), nil
|
||||
}
|
||||
Reference in New Issue
Block a user