Explicit masterfeats/feats removal

This commit is contained in:
2026-05-13 19:32:41 +02:00
parent 03e2320788
commit 4b55628ac8
5 changed files with 102 additions and 31 deletions
+13 -7
View File
@@ -42,12 +42,16 @@ One concrete example already observed:
- validation in `internal/topdata/topdata.go` still hard-fails specifically for
`topdata/data/masterfeats/base.json` if `output` is missing
That mismatch proves the current methodology is stale:
That mismatch proved the current methodology was stale:
- the runtime/build side treats `masterfeats` like a generic dataset
- the validator still treats it like a special namespace with explicit required
metadata
Current status: the output-name mismatch has been fixed for `masterfeats` and
the equivalent `feat` base dataset rule by sharing the derived output-name rule
between discovery and validation.
This same category of problem appears in several other places.
## Findings From Current Audit
@@ -67,17 +71,17 @@ This same category of problem appears in several other places.
Some of these are legitimate dataset-shape differences. Others are stale
special-cases that should be generalized.
### 2. `masterfeats` still carries outdated explicit validation rules
### 2. `masterfeats` carries dataset-specific invariants
`validateMasterfeatsBaseFile()` currently requires:
`validateMasterfeatsBaseFile()` currently enforces:
- `output == masterfeats.2da`
- explicit `output == masterfeats.2da` when `output` is authored
- specific required columns
- `masterfeats:` row keys
The key-prefix and column requirements may still be valid dataset invariants.
The `output` requirement is not clearly justified anymore when generic dataset
discovery already computes a default output name from location.
The stale requirement that `output` must be authored has been removed; validation
now shares the same derived default output-name rule as native discovery.
### 3. Migration still seeds explicit dataset exceptions
@@ -128,7 +132,9 @@ keep inheriting brittle assumptions.
### Work Item 1: Remove outdated `masterfeats.output` hard requirement
Investigate and likely change `validateMasterfeatsBaseFile()` so:
Status: implemented.
`validateMasterfeatsBaseFile()` now:
- missing `output` is accepted when generic dataset discovery can derive the same
output name deterministically