Wrapper tool hardening and cleanup
This commit is contained in:
@@ -0,0 +1,197 @@
|
||||
# Native Topdata Hardening Contract
|
||||
|
||||
## Scope
|
||||
|
||||
This contract governs the current topdata hardening and bugfixing phase across:
|
||||
|
||||
- `toolkit/` as implementation owner
|
||||
- `module/` as the primary authored topdata consumer
|
||||
|
||||
This is an active-priority contract.
|
||||
|
||||
## Current Priority
|
||||
|
||||
Focus now on:
|
||||
|
||||
1. removing outdated explicit namespace/path hardcoding
|
||||
2. fixing validation/build mismatches
|
||||
3. making native topdata behavior generic where the data model is already generic
|
||||
4. improving diagnostics and operator-facing output for topdata commands
|
||||
|
||||
Defer for now:
|
||||
|
||||
- model compilation
|
||||
- release patch-note automation
|
||||
- unrelated asset-processing expansion
|
||||
- non-topdata feature work that does not unblock wiki deployment
|
||||
|
||||
Wiki deployment itself is a follow-on phase after this hardening pass, because
|
||||
`build-wiki` and `deploy-wiki` depend on topdata outputs and metadata being
|
||||
stable.
|
||||
|
||||
## Problem Statement
|
||||
|
||||
Native topdata has reached the point where most major functionality exists, but
|
||||
too much behavior is still implemented as explicit namespace knowledge instead of
|
||||
generic dataset rules.
|
||||
|
||||
One concrete example already observed:
|
||||
|
||||
- dataset discovery in `internal/topdata/native.go` already derives a default
|
||||
`output` name when `base.json` omits it
|
||||
- validation in `internal/topdata/topdata.go` still hard-fails specifically for
|
||||
`topdata/data/masterfeats/base.json` if `output` is missing
|
||||
|
||||
That mismatch proves the current methodology is stale:
|
||||
|
||||
- the runtime/build side treats `masterfeats` like a generic dataset
|
||||
- the validator still treats it like a special namespace with explicit required
|
||||
metadata
|
||||
|
||||
This same category of problem appears in several other places.
|
||||
|
||||
## Findings From Current Audit
|
||||
|
||||
### 1. Validation dispatch is path-special-cased
|
||||
|
||||
`validateDataObject()` currently routes behavior through explicit path checks for:
|
||||
|
||||
- `masterfeats/base.json`
|
||||
- `masterfeats/lock.json`
|
||||
- `feat/base.json`
|
||||
- `feat/lock.json`
|
||||
- `feat/generated/*`
|
||||
- `parts/overrides/*`
|
||||
- `racialtypes/registry/races/*`
|
||||
|
||||
Some of these are legitimate dataset-shape differences. Others are stale
|
||||
special-cases that should be generalized.
|
||||
|
||||
### 2. `masterfeats` still carries outdated explicit validation rules
|
||||
|
||||
`validateMasterfeatsBaseFile()` currently requires:
|
||||
|
||||
- `output == masterfeats.2da`
|
||||
- specific required columns
|
||||
- `masterfeats:` row keys
|
||||
|
||||
The key-prefix and column requirements may still be valid dataset invariants.
|
||||
The `output` requirement is not clearly justified anymore when generic dataset
|
||||
discovery already computes a default output name from location.
|
||||
|
||||
### 3. Migration still seeds explicit dataset exceptions
|
||||
|
||||
`feat_migrate.go` explicitly writes:
|
||||
|
||||
- `output = "feat.2da"`
|
||||
- `compare_reference = false`
|
||||
|
||||
This may still be operationally correct, but it means migration continues to
|
||||
materialize special-case control fields instead of relying on a clearer generic
|
||||
contract for dataset reference comparison.
|
||||
|
||||
### 4. Project-level topdata defaults are still mixed
|
||||
|
||||
`internal/project` still owns several topdata defaults directly:
|
||||
|
||||
- build directory derivation
|
||||
- compiled 2DA subdirectory
|
||||
- compiled TLK name
|
||||
- wiki root/pages/state defaults
|
||||
- wiki deploy manifest and edit summary defaults
|
||||
- package HAK/TLK fallback names
|
||||
|
||||
Some of these are valid toolkit defaults. Some should be reviewed as explicit
|
||||
schema-backed config.
|
||||
|
||||
### 5. Wiki generation and deploy depend on stable dataset semantics
|
||||
|
||||
`build-wiki` and `deploy-wiki` are downstream consumers of native topdata state.
|
||||
If dataset rules remain partly explicit and partly generic, wiki generation will
|
||||
keep inheriting brittle assumptions.
|
||||
|
||||
## Hardening Rules
|
||||
|
||||
1. If a rule can be inferred generically from dataset shape, do not hardcode it
|
||||
to a namespace path.
|
||||
2. If a rule is truly dataset-specific, document why it is intrinsic rather than
|
||||
configuration-driven.
|
||||
3. Validation and build behavior must agree. Discovery defaults and validation
|
||||
requirements cannot contradict each other.
|
||||
4. Dataset-authored JSON should remain the source of truth for dataset-level
|
||||
semantics; root YAML should not absorb dataset internals just to avoid code
|
||||
cleanup.
|
||||
5. Error messages must point to the actual invariant being violated, not an
|
||||
outdated historical implementation detail.
|
||||
|
||||
## Immediate Work Items
|
||||
|
||||
### Work Item 1: Remove outdated `masterfeats.output` hard requirement
|
||||
|
||||
Investigate and likely change `validateMasterfeatsBaseFile()` so:
|
||||
|
||||
- missing `output` is accepted when generic dataset discovery can derive the same
|
||||
output name deterministically
|
||||
- explicit `output` remains allowed
|
||||
- wrong explicit `output` still fails clearly if the dataset truly requires a
|
||||
fixed emitted file name
|
||||
|
||||
This should be test-covered.
|
||||
|
||||
### Work Item 2: Audit path-based validator routing
|
||||
|
||||
Classify every explicit route in `validateDataObject()` as one of:
|
||||
|
||||
- generic canonical shape handling
|
||||
- dataset-family behavior that should become generic
|
||||
- deliberate intrinsic special-case with justification
|
||||
|
||||
Reduce the explicit route set where possible.
|
||||
|
||||
### Work Item 3: Define topdata invariant boundaries
|
||||
|
||||
For each current explicit topdata behavior, decide whether it belongs in:
|
||||
|
||||
- root YAML config
|
||||
- dataset JSON
|
||||
- toolkit invariant
|
||||
|
||||
Do not move dataset-authored semantics into YAML unless the behavior truly varies
|
||||
at repository level rather than dataset level.
|
||||
|
||||
### Work Item 4: Revisit `compare_reference` ownership
|
||||
|
||||
`feat.2da` is still excluded via `compare_reference = false`. Confirm whether:
|
||||
|
||||
- this remains a temporary rollout flag
|
||||
- or compare coverage needs a more explicit generic contract for generated-family
|
||||
datasets
|
||||
|
||||
### Work Item 5: Bring topdata logging up to current CLI standards
|
||||
|
||||
Coordinate with `LOG_REFACTOR_CONTRACT.md` so topdata commands emit:
|
||||
|
||||
- deterministic phase summaries
|
||||
- concise validation/build/compare totals
|
||||
- clear file/namespace diagnostics
|
||||
|
||||
## Testing Requirements
|
||||
|
||||
Add or update tests for:
|
||||
|
||||
- `masterfeats/base.json` without explicit `output` when the derived output would
|
||||
still be `masterfeats.2da`
|
||||
- explicit wrong `output` for `masterfeats`
|
||||
- validator/build parity for dataset output naming
|
||||
- any reduction in path-special-case validation routing
|
||||
- topdata command output summaries if presentation changes land in the same phase
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- topdata validation no longer rejects data solely because of stale explicit
|
||||
namespace assumptions
|
||||
- build and validation agree on dataset output-name behavior
|
||||
- path-special-casing is reduced or explicitly justified
|
||||
- remaining dataset-specific invariants are documented as such
|
||||
- topdata hardening leaves wiki generation/deployment with a cleaner and more
|
||||
stable contract
|
||||
Reference in New Issue
Block a user