sow-tools/docs/superpowers/specs/2026-06-25-topdata-json-validation-design.md

Topdata JSON Validation Tightening Design

Date: 2026-06-25 Status: Implemented and reviewed Scope: sow-tools

Problem

Topdata JSON validation checks many required properties and value types, but it often ignores unknown structural properties. This allows plausible typos to pass validation and then be ignored by the builder.

The immediate failure occurred in topdata/data/classes/feats/global.json: an injection used when_present even though the implemented condition is require_present. Validation succeeded, the builder ignored when_present, and the injection was therefore applied without the intended condition.

This is a fail-open authoring contract. A misspelled build-control property must not silently change generated game data.

Goals

• Reject unsupported structural properties in canonical topdata JSON files.
• Add an any_present global injection condition.
• Keep standalone validation and direct native builds consistent.
• Preserve dataset-specific row flexibility.
• Tighten only established canonical formats; do not turn this work into a repository-wide schema rewrite.
• Produce diagnostics that identify the file, object, and unsupported property.

Non-goals

• No schema migration or formatting change to authored topdata.
• No generic JSON Schema framework or new validation dependency.
• No blanket closure of every specialized JSON dialect.
• No change to generated feat family, registry, spellbook, TLK, or other specialized formats unless they use the canonical containers covered here.
• No restriction of row values beyond existing column, metadata, inheritance, reference, TLK, and authoring-sugar contracts.
• No compatibility alias for when_present; it is invalid syntax.

Design

Validation boundary

Structural objects use closed property sets. Dataset payload objects remain dataset-aware.

Closed structural objects include:

Object	Supported properties
Base or plain rows-file root	output, key, columns, rows
Canonical module root	output, key, columns, entries, overrides, rows
global.json root	columns, entries, overrides, defaults, position, injections
Global injection	row, require_present, any_present, unless_present
Global condition	field, id
Global default rule	match, values
Global default match object	source
Global default format object	format

The allowed root set is selected from the file's existing canonical context; properties from unrelated formats are not accepted merely because another topdata file type supports them.

When a canonical root contains more than one supported container, each present container must be validated. For example, a module containing both entries and overrides must not stop validation after recognizing entries.

The following remain payload objects rather than global allowlists:

• objects inside rows;
• values inside entries;
• row-like objects inside overrides;
• an injection's row;
• a default rule's values.

Those objects continue to accept declared dataset columns plus documented row control and metadata fields. The builder already rejects unknown row and entry columns when it canonicalizes a dataset. This work must reuse or align with that knowledge where practical, but must not invent a single fixed row schema across all datasets.

Moderate strictness

This change closes only canonical containers whose grammar is already established by the parser and current authored data.

Before adding a closed root check, implementation must inventory active sow-topdata files and existing sow-tools fixtures for legitimate property combinations. A property used by an established canonical format must be added to that format's explicit set rather than removed from authored data merely to satisfy the validator.

Specialized parsers remain responsible for their own object shapes. Extending closed-property checks to those formats is separate work.

Global injection conditions

Global injections support three condition groups:

• require_present: every listed condition must match a current row.
• any_present: at least one listed condition must match a current row.
• unless_present: no listed condition may match a current row.

When more than one group is present, every group must pass. Conditions continue to use the existing shape:

{
  "field": "FeatIndex",
  "id": "feat:example"
}

Example:

{
  "row": {
    "FeatIndex": {
      "id": "feat:example"
    }
  },
  "require_present": [
    {
      "field": "FeatIndex",
      "id": "feat:base_requirement"
    }
  ],
  "any_present": [
    {
      "field": "FeatIndex",
      "id": "masterfeats:metamagic"
    },
    {
      "field": "FeatIndex",
      "id": "masterfeats:combat"
    }
  ],
  "unless_present": [
    {
      "field": "FeatIndex",
      "id": "feat:example"
    }
  ]
}

This injection applies only when the required feat exists, either listed masterfeat exists, and the injected feat does not already exist.

any_present must contain at least one condition. An empty list is an authoring error because it cannot express a useful successful condition. Existing empty-list behavior for require_present and unless_present is not changed by this work.

Conditions are evaluated against the same current-row view and in the same injection order used today. This preserves the existing behavior where an earlier injection can affect a later injection's conditions.

Validation and build consistency

ValidateProject must report unsupported structural properties before a build. The native builder must also fail closed when invoked directly with the same invalid control syntax.

The implementation should centralize:

• deterministic unsupported-property checks;
• global condition parsing and validation;
• condition group evaluation.

Validation may collect multiple diagnostics while building returns the first blocking error. Both paths must accept and reject the same property names and condition shapes.

Diagnostics

Diagnostics must name the containing object and unsupported property. Property lists must be deterministic.

For the original error, an acceptable diagnostic is:

global injection 10 contains unsupported key "when_present";
supported keys are row, require_present, any_present, and unless_present

Equivalent concise wording is acceptable if tests assert the stable semantic parts rather than the entire sentence.

Compatibility

Current canonical sow-topdata roots fit the property sets above. The implementation must rerun the inventory immediately before tightening checks and run validation against the active sow-topdata checkout.

Unknown properties are not retained as compatibility behavior. They currently have no defined effect, and silently accepting them is the bug being fixed. Existing documented properties and dataset-specific row fields remain valid.

require_present and unless_present retain their current semantics. any_present is additive.

Testing

Tests must follow red-green development and cover validation and direct-build paths.

Unsupported properties

• Reject an unknown global.json root property.
• Reject when_present and other unknown injection properties.
• Reject an unknown condition property.
• Reject unknown properties in default rules, match objects, and format objects.
• Reject unknown canonical base/plain rows-file root properties.
• Reject unknown canonical module root properties.
• Continue accepting legitimate combinations such as:
  • a columns-only module;
  • entries-only and overrides-only modules;
  • a module containing both entries and overrides;
  • current base and plain rows-file roots.
• Continue accepting dataset columns and documented row control fields inside rows, entries, overrides, and injection rows.

any_present

• Validation accepts a non-empty any_present condition list.
• Validation rejects a non-array or empty any_present.
• A build injects when the first alternative is present.
• A build injects when a later alternative is present.
• A build skips the injection when no alternative is present.
• any_present combines correctly with require_present.
• any_present combines correctly with unless_present.
• Existing require_present and unless_present behavior remains unchanged.
• Direct building rejects unsupported condition syntax even when project validation was not called first.

Integration

• Run focused internal/topdata tests.
• Run nix develop --command make check.
• Validate the active sow-topdata checkout with the changed Crucible.
• Build topdata far enough to exercise global injections and confirm no generated files are tracked.

Documentation

Update the class feat global injection contract to document require_present, any_present, and unless_present, including their all/any/none semantics and how multiple groups combine.

No broad validation guide is added unless implementation reveals an existing operator document that would otherwise become stale.

Acceptance criteria

• when_present in a global injection fails validation and direct building.
• any_present is valid only as a non-empty list of valid conditions.
• any_present applies an injection when one or more alternatives exist and skips it when none exist.
• Unknown canonical structural properties fail with actionable diagnostics.
• Dataset-specific row fields remain valid when declared by the dataset.
• Existing active sow-topdata validates and builds after replacing invalid syntax with the documented grammar.
• No specialized topdata format is tightened outside the stated scope.