diff --git a/docs/superpowers/specs/2026-06-25-topdata-json-validation-design.md b/docs/superpowers/specs/2026-06-25-topdata-json-validation-design.md new file mode 100644 index 0000000..756ea3e --- /dev/null +++ b/docs/superpowers/specs/2026-06-25-topdata-json-validation-design.md @@ -0,0 +1,269 @@ +# Topdata JSON Validation Tightening Design + +**Date:** 2026-06-25 +**Status:** Approved design; implementation pending +**Scope:** `sow-tools` + +## Problem + +Topdata JSON validation checks many required properties and value types, but it +often ignores unknown structural properties. This allows plausible typos to +pass validation and then be ignored by the builder. + +The immediate failure occurred in +`topdata/data/classes/feats/global.json`: an injection used +`when_present` even though the implemented condition is `require_present`. +Validation succeeded, the builder ignored `when_present`, and the injection was +therefore applied without the intended condition. + +This is a fail-open authoring contract. A misspelled build-control property must +not silently change generated game data. + +## Goals + +- Reject unsupported structural properties in canonical topdata JSON files. +- Add an `any_present` global injection condition. +- Keep standalone validation and direct native builds consistent. +- Preserve dataset-specific row flexibility. +- Tighten only established canonical formats; do not turn this work into a + repository-wide schema rewrite. +- Produce diagnostics that identify the file, object, and unsupported property. + +## Non-goals + +- No schema migration or formatting change to authored topdata. +- No generic JSON Schema framework or new validation dependency. +- No blanket closure of every specialized JSON dialect. +- No change to generated feat family, registry, spellbook, TLK, or other + specialized formats unless they use the canonical containers covered here. +- No restriction of row values beyond existing column, metadata, inheritance, + reference, TLK, and authoring-sugar contracts. +- No compatibility alias for `when_present`; it is invalid syntax. + +## Design + +### Validation boundary + +Structural objects use closed property sets. Dataset payload objects remain +dataset-aware. + +Closed structural objects include: + +| Object | Supported properties | +| --- | --- | +| Base or plain rows-file root | `output`, `key`, `columns`, `rows` | +| Canonical module root | `output`, `key`, `columns`, `entries`, `overrides`, `rows` | +| `global.json` root | `columns`, `entries`, `overrides`, `defaults`, `position`, `injections` | +| Global injection | `row`, `require_present`, `any_present`, `unless_present` | +| Global condition | `field`, `id` | +| Global default rule | `match`, `values` | +| Global default match object | `source` | +| Global default format object | `format` | + +The allowed root set is selected from the file's existing canonical context; +properties from unrelated formats are not accepted merely because another +topdata file type supports them. + +When a canonical root contains more than one supported container, each present +container must be validated. For example, a module containing both `entries` +and `overrides` must not stop validation after recognizing `entries`. + +The following remain payload objects rather than global allowlists: + +- objects inside `rows`; +- values inside `entries`; +- row-like objects inside `overrides`; +- an injection's `row`; +- a default rule's `values`. + +Those objects continue to accept declared dataset columns plus documented row +control and metadata fields. The builder already rejects unknown row and entry +columns when it canonicalizes a dataset. This work must reuse or align with +that knowledge where practical, but must not invent a single fixed row schema +across all datasets. + +### Moderate strictness + +This change closes only canonical containers whose grammar is already +established by the parser and current authored data. + +Before adding a closed root check, implementation must inventory active +`sow-topdata` files and existing `sow-tools` fixtures for legitimate property +combinations. A property used by an established canonical format must be added +to that format's explicit set rather than removed from authored data merely to +satisfy the validator. + +Specialized parsers remain responsible for their own object shapes. Extending +closed-property checks to those formats is separate work. + +### Global injection conditions + +Global injections support three condition groups: + +- `require_present`: every listed condition must match a current row. +- `any_present`: at least one listed condition must match a current row. +- `unless_present`: no listed condition may match a current row. + +When more than one group is present, every group must pass. Conditions continue +to use the existing shape: + +```json +{ + "field": "FeatIndex", + "id": "feat:example" +} +``` + +Example: + +```json +{ + "row": { + "FeatIndex": { + "id": "feat:example" + } + }, + "require_present": [ + { + "field": "FeatIndex", + "id": "feat:base_requirement" + } + ], + "any_present": [ + { + "field": "FeatIndex", + "id": "masterfeats:metamagic" + }, + { + "field": "FeatIndex", + "id": "masterfeats:combat" + } + ], + "unless_present": [ + { + "field": "FeatIndex", + "id": "feat:example" + } + ] +} +``` + +This injection applies only when the required feat exists, either listed +masterfeat exists, and the injected feat does not already exist. + +`any_present` must contain at least one condition. An empty list is an +authoring error because it cannot express a useful successful condition. +Existing empty-list behavior for `require_present` and `unless_present` is not +changed by this work. + +Conditions are evaluated against the same current-row view and in the same +injection order used today. This preserves the existing behavior where an +earlier injection can affect a later injection's conditions. + +### Validation and build consistency + +`ValidateProject` must report unsupported structural properties before a build. +The native builder must also fail closed when invoked directly with the same +invalid control syntax. + +The implementation should centralize: + +- deterministic unsupported-property checks; +- global condition parsing and validation; +- condition group evaluation. + +Validation may collect multiple diagnostics while building returns the first +blocking error. Both paths must accept and reject the same property names and +condition shapes. + +### Diagnostics + +Diagnostics must name the containing object and unsupported property. Property +lists must be deterministic. + +For the original error, an acceptable diagnostic is: + +```text +global injection 10 contains unsupported key "when_present"; +supported keys are row, require_present, any_present, and unless_present +``` + +Equivalent concise wording is acceptable if tests assert the stable semantic +parts rather than the entire sentence. + +## Compatibility + +Current canonical `sow-topdata` roots fit the property sets above. The +implementation must rerun the inventory immediately before tightening checks +and run validation against the active `sow-topdata` checkout. + +Unknown properties are not retained as compatibility behavior. They currently +have no defined effect, and silently accepting them is the bug being fixed. +Existing documented properties and dataset-specific row fields remain valid. + +`require_present` and `unless_present` retain their current semantics. +`any_present` is additive. + +## Testing + +Tests must follow red-green development and cover validation and direct-build +paths. + +### Unsupported properties + +- Reject an unknown `global.json` root property. +- Reject `when_present` and other unknown injection properties. +- Reject an unknown condition property. +- Reject unknown properties in default rules, match objects, and format + objects. +- Reject unknown canonical base/plain rows-file root properties. +- Reject unknown canonical module root properties. +- Continue accepting legitimate combinations such as: + - a columns-only module; + - entries-only and overrides-only modules; + - a module containing both entries and overrides; + - current base and plain rows-file roots. +- Continue accepting dataset columns and documented row control fields inside + rows, entries, overrides, and injection rows. + +### `any_present` + +- Validation accepts a non-empty `any_present` condition list. +- Validation rejects a non-array or empty `any_present`. +- A build injects when the first alternative is present. +- A build injects when a later alternative is present. +- A build skips the injection when no alternative is present. +- `any_present` combines correctly with `require_present`. +- `any_present` combines correctly with `unless_present`. +- Existing `require_present` and `unless_present` behavior remains unchanged. +- Direct building rejects unsupported condition syntax even when project + validation was not called first. + +### Integration + +- Run focused `internal/topdata` tests. +- Run `nix develop --command make check`. +- Validate the active `sow-topdata` checkout with the changed Crucible. +- Build topdata far enough to exercise global injections and confirm no + generated files are tracked. + +## Documentation + +Update the class feat global injection contract to document `require_present`, +`any_present`, and `unless_present`, including their all/any/none semantics and +how multiple groups combine. + +No broad validation guide is added unless implementation reveals an existing +operator document that would otherwise become stale. + +## Acceptance criteria + +- `when_present` in a global injection fails validation and direct building. +- `any_present` is valid only as a non-empty list of valid conditions. +- `any_present` applies an injection when one or more alternatives exist and + skips it when none exist. +- Unknown canonical structural properties fail with actionable diagnostics. +- Dataset-specific row fields remain valid when declared by the dataset. +- Existing active `sow-topdata` validates and builds after replacing invalid + syntax with the documented grammar. +- No specialized topdata format is tightened outside the stated scope.