222 lines
7.7 KiB
Markdown
222 lines
7.7 KiB
Markdown
# Native Topdata Hardening Contract
|
|
|
|
## Scope
|
|
|
|
This contract governs the current topdata hardening and bugfixing phase across:
|
|
|
|
- `toolkit/` as implementation owner
|
|
- `module/` as the primary authored topdata consumer
|
|
|
|
This is an active-priority contract.
|
|
|
|
## Current Priority
|
|
|
|
Focus now on:
|
|
|
|
1. removing outdated explicit namespace/path hardcoding
|
|
2. fixing validation/build mismatches
|
|
3. making native topdata behavior generic where the data model is already generic
|
|
4. improving diagnostics and operator-facing output for topdata commands
|
|
|
|
Defer for now:
|
|
|
|
- model compilation
|
|
- release patch-note automation
|
|
- unrelated asset-processing expansion
|
|
- non-topdata feature work that does not unblock wiki deployment
|
|
|
|
Wiki deployment itself is a follow-on phase after this hardening pass, because
|
|
`build-wiki` and `deploy-wiki` depend on topdata outputs and metadata being
|
|
stable.
|
|
|
|
## Problem Statement
|
|
|
|
Native topdata has reached the point where most major functionality exists, but
|
|
too much behavior is still implemented as explicit namespace knowledge instead of
|
|
generic dataset rules.
|
|
|
|
One concrete example already observed:
|
|
|
|
- dataset discovery in `internal/topdata/native.go` already derives a default
|
|
`output` name when `base.json` omits it
|
|
- validation in `internal/topdata/topdata.go` still hard-fails specifically for
|
|
`topdata/data/masterfeats/base.json` if `output` is missing
|
|
|
|
That mismatch proved the current methodology was stale:
|
|
|
|
- the runtime/build side treats `masterfeats` like a generic dataset
|
|
- the validator still treats it like a special namespace with explicit required
|
|
metadata
|
|
|
|
Current status: the output-name mismatch has been fixed for `masterfeats` and
|
|
the equivalent `feat` base dataset rule by sharing the derived output-name rule
|
|
between discovery and validation.
|
|
|
|
This same category of problem appears in several other places.
|
|
|
|
## Findings From Current Audit
|
|
|
|
### 1. Validation dispatch is path-special-cased
|
|
|
|
`validateDataObject()` currently routes behavior through explicit path checks for:
|
|
|
|
- `masterfeats/base.json`
|
|
- `masterfeats/lock.json`
|
|
- `feat/base.json`
|
|
- `feat/lock.json`
|
|
- `feat/generated/*`
|
|
- `parts/overrides/*`
|
|
- `racialtypes/registry/races/*`
|
|
|
|
Some of these are legitimate dataset-shape differences. Others are stale
|
|
special-cases that should be generalized.
|
|
|
|
### 2. `masterfeats` carries dataset-specific invariants
|
|
|
|
`validateMasterfeatsBaseFile()` currently enforces:
|
|
|
|
- explicit `output == masterfeats.2da` when `output` is authored
|
|
- specific required columns
|
|
- `masterfeats:` row keys
|
|
|
|
The key-prefix and column requirements may still be valid dataset invariants.
|
|
The stale requirement that `output` must be authored has been removed; validation
|
|
now shares the same derived default output-name rule as native discovery.
|
|
|
|
### 3. Migration still seeds explicit dataset exceptions
|
|
|
|
`feat_migrate.go` explicitly writes:
|
|
|
|
- `output = "feat.2da"`
|
|
- `compare_reference = false`
|
|
|
|
This may still be operationally correct, but it means migration continues to
|
|
materialize special-case control fields instead of relying on a clearer generic
|
|
contract for dataset reference comparison.
|
|
|
|
### 4. Project-level topdata defaults are still mixed
|
|
|
|
`internal/project` still owns several topdata defaults directly:
|
|
|
|
- build directory derivation
|
|
- compiled 2DA subdirectory
|
|
- compiled TLK name
|
|
- wiki root/pages/state defaults
|
|
- wiki deploy manifest and edit summary defaults
|
|
- package HAK/TLK fallback names
|
|
|
|
Some of these are valid toolkit defaults. Some should be reviewed as explicit
|
|
schema-backed config.
|
|
|
|
### 5. Wiki generation and deploy depend on stable dataset semantics
|
|
|
|
`build-wiki` and `deploy-wiki` are downstream consumers of native topdata state.
|
|
If dataset rules remain partly explicit and partly generic, wiki generation will
|
|
keep inheriting brittle assumptions.
|
|
|
|
## Hardening Rules
|
|
|
|
1. If a rule can be inferred generically from dataset shape, do not hardcode it
|
|
to a namespace path.
|
|
2. If a rule is truly dataset-specific, document why it is intrinsic rather than
|
|
configuration-driven.
|
|
3. Validation and build behavior must agree. Discovery defaults and validation
|
|
requirements cannot contradict each other.
|
|
4. Dataset-authored JSON should remain the source of truth for dataset-level
|
|
semantics; root YAML should not absorb dataset internals just to avoid code
|
|
cleanup.
|
|
5. Error messages must point to the actual invariant being violated, not an
|
|
outdated historical implementation detail.
|
|
|
|
## Immediate Work Items
|
|
|
|
### Work Item 1: Remove outdated `masterfeats.output` hard requirement
|
|
|
|
Status: implemented.
|
|
|
|
`validateMasterfeatsBaseFile()` now:
|
|
|
|
- missing `output` is accepted when generic dataset discovery can derive the same
|
|
output name deterministically
|
|
- explicit `output` remains allowed
|
|
- wrong explicit `output` still fails clearly if the dataset truly requires a
|
|
fixed emitted file name
|
|
|
|
This should be test-covered.
|
|
|
|
### Work Item 2: Audit path-based validator routing
|
|
|
|
Status: implemented for root dataset invariants; family-specific routes remain
|
|
where the authored shape is not a generic base/module/lock file.
|
|
|
|
Classify every explicit route in `validateDataObject()` as one of:
|
|
|
|
- generic canonical shape handling
|
|
- dataset-family behavior that should become generic
|
|
- deliberate intrinsic special-case with justification
|
|
|
|
Reduce the explicit route set where possible.
|
|
|
|
Current classification:
|
|
|
|
- `masterfeats/base.json`, `masterfeats/lock.json`, `feat/base.json`, and
|
|
`feat/lock.json` are now handled by a generic dataset invariant table. These
|
|
remain toolkit invariants because the row key namespaces and required columns
|
|
are intrinsic to those native datasets, not repository-level YAML choices.
|
|
- `feat/generated/*` remains an intrinsic dataset-family route. Generated feat
|
|
files support compact family expansion and generated entries/overrides, which
|
|
are not generic module shapes.
|
|
- `parts/overrides/*` remains a dataset-family route for non-module override
|
|
files outside the standard `modules/` tree.
|
|
- `racialtypes/registry/races/*` remains a registry-family route because those
|
|
files describe race registry records, optional feat-table generation, and core
|
|
row payloads rather than generic row/module documents.
|
|
|
|
### Work Item 3: Define topdata invariant boundaries
|
|
|
|
For each current explicit topdata behavior, decide whether it belongs in:
|
|
|
|
- root YAML config
|
|
- dataset JSON
|
|
- toolkit invariant
|
|
|
|
Do not move dataset-authored semantics into YAML unless the behavior truly varies
|
|
at repository level rather than dataset level.
|
|
|
|
### Work Item 4: Revisit `compare_reference` ownership
|
|
|
|
`feat.2da` is still excluded via `compare_reference = false`. Confirm whether:
|
|
|
|
- this remains a temporary rollout flag
|
|
- or compare coverage needs a more explicit generic contract for generated-family
|
|
datasets
|
|
|
|
### Work Item 5: Bring topdata logging up to current CLI standards
|
|
|
|
Coordinate with `LOG_REFACTOR_CONTRACT.md` so topdata commands emit:
|
|
|
|
- deterministic phase summaries
|
|
- concise validation/build/compare totals
|
|
- clear file/namespace diagnostics
|
|
|
|
## Testing Requirements
|
|
|
|
Add or update tests for:
|
|
|
|
- `masterfeats/base.json` without explicit `output` when the derived output would
|
|
still be `masterfeats.2da`
|
|
- explicit wrong `output` for `masterfeats`
|
|
- validator/build parity for dataset output naming
|
|
- any reduction in path-special-case validation routing
|
|
- topdata command output summaries if presentation changes land in the same phase
|
|
|
|
## Acceptance Criteria
|
|
|
|
- topdata validation no longer rejects data solely because of stale explicit
|
|
namespace assumptions
|
|
- build and validation agree on dataset output-name behavior
|
|
- path-special-casing is reduced or explicitly justified
|
|
- remaining dataset-specific invariants are documented as such
|
|
- topdata hardening leaves wiki generation/deployment with a cleaner and more
|
|
stable contract
|