sow-tools/docs/superpowers/specs/2026-06-25-topdata-json-validation-design.md

# Topdata JSON Validation Tightening Design

**Date:** 2026-06-25
**Status:** Implemented and reviewed
**Scope:** `sow-tools`

## Problem

Topdata JSON validation checks many required properties and value types, but it
often ignores unknown structural properties. This allows plausible typos to
pass validation and then be ignored by the builder.

The immediate failure occurred in
`topdata/data/classes/feats/global.json`: an injection used
`when_present` even though the implemented condition is `require_present`.
Validation succeeded, the builder ignored `when_present`, and the injection was
therefore applied without the intended condition.

This is a fail-open authoring contract. A misspelled build-control property must
not silently change generated game data.

## Goals

- Reject unsupported structural properties in canonical topdata JSON files.
- Add an `any_present` global injection condition.
- Keep standalone validation and direct native builds consistent.
- Preserve dataset-specific row flexibility.
- Tighten only established canonical formats; do not turn this work into a
  repository-wide schema rewrite.
- Produce diagnostics that identify the file, object, and unsupported property.

## Non-goals

- No schema migration or formatting change to authored topdata.
- No generic JSON Schema framework or new validation dependency.
- No blanket closure of every specialized JSON dialect.
- No change to generated feat family, registry, spellbook, TLK, or other
  specialized formats unless they use the canonical containers covered here.
- No restriction of row values beyond existing column, metadata, inheritance,
  reference, TLK, and authoring-sugar contracts.
- No compatibility alias for `when_present`; it is invalid syntax.

## Design

### Validation boundary

Structural objects use closed property sets. Dataset payload objects remain
dataset-aware.

Closed structural objects include:

| Object | Supported properties |
| --- | --- |
| Base or plain rows-file root | `output`, `key`, `columns`, `rows` |
| Canonical module root | `output`, `key`, `columns`, `entries`, `overrides`, `rows` |
| `global.json` root | `columns`, `entries`, `overrides`, `defaults`, `position`, `injections` |
| Global injection | `row`, `require_present`, `any_present`, `unless_present` |
| Global condition | `field`, `id` |
| Global default rule | `match`, `values` |
| Global default match object | `source` |
| Global default format object | `format` |

The allowed root set is selected from the file's existing canonical context;
properties from unrelated formats are not accepted merely because another
topdata file type supports them.

When a canonical root contains more than one supported container, each present
container must be validated. For example, a module containing both `entries`
and `overrides` must not stop validation after recognizing `entries`.

The following remain payload objects rather than global allowlists:

- objects inside `rows`;
- values inside `entries`;
- row-like objects inside `overrides`;
- an injection's `row`;
- a default rule's `values`.

Those objects continue to accept declared dataset columns plus documented row
control and metadata fields. The builder already rejects unknown row and entry
columns when it canonicalizes a dataset. This work must reuse or align with
that knowledge where practical, but must not invent a single fixed row schema
across all datasets.

### Moderate strictness

This change closes only canonical containers whose grammar is already
established by the parser and current authored data.

Before adding a closed root check, implementation must inventory active
`sow-topdata` files and existing `sow-tools` fixtures for legitimate property
combinations. A property used by an established canonical format must be added
to that format's explicit set rather than removed from authored data merely to
satisfy the validator.

Specialized parsers remain responsible for their own object shapes. Extending
closed-property checks to those formats is separate work.

### Global injection conditions

Global injections support three condition groups:

- `require_present`: every listed condition must match a current row.
- `any_present`: at least one listed condition must match a current row.
- `unless_present`: no listed condition may match a current row.

When more than one group is present, every group must pass. Conditions continue
to use the existing shape:

```json
{
  "field": "FeatIndex",
  "id": "feat:example"
}
```

Example:

```json
{
  "row": {
    "FeatIndex": {
      "id": "feat:example"
    }
  },
  "require_present": [
    {
      "field": "FeatIndex",
      "id": "feat:base_requirement"
    }
  ],
  "any_present": [
    {
      "field": "FeatIndex",
      "id": "masterfeats:metamagic"
    },
    {
      "field": "FeatIndex",
      "id": "masterfeats:combat"
    }
  ],
  "unless_present": [
    {
      "field": "FeatIndex",
      "id": "feat:example"
    }
  ]
}
```

This injection applies only when the required feat exists, either listed
masterfeat exists, and the injected feat does not already exist.

`any_present` must contain at least one condition. An empty list is an
authoring error because it cannot express a useful successful condition.
Existing empty-list behavior for `require_present` and `unless_present` is not
changed by this work.

Conditions are evaluated against the same current-row view and in the same
injection order used today. This preserves the existing behavior where an
earlier injection can affect a later injection's conditions.

### Validation and build consistency

`ValidateProject` must report unsupported structural properties before a build.
The native builder must also fail closed when invoked directly with the same
invalid control syntax.

The implementation should centralize:

- deterministic unsupported-property checks;
- global condition parsing and validation;
- condition group evaluation.

Validation may collect multiple diagnostics while building returns the first
blocking error. Both paths must accept and reject the same property names and
condition shapes.

### Diagnostics

Diagnostics must name the containing object and unsupported property. Property
lists must be deterministic.

For the original error, an acceptable diagnostic is:

```text
global injection 10 contains unsupported key "when_present";
supported keys are row, require_present, any_present, and unless_present
```

Equivalent concise wording is acceptable if tests assert the stable semantic
parts rather than the entire sentence.

## Compatibility

Current canonical `sow-topdata` roots fit the property sets above. The
implementation must rerun the inventory immediately before tightening checks
and run validation against the active `sow-topdata` checkout.

Unknown properties are not retained as compatibility behavior. They currently
have no defined effect, and silently accepting them is the bug being fixed.
Existing documented properties and dataset-specific row fields remain valid.

`require_present` and `unless_present` retain their current semantics.
`any_present` is additive.

## Testing

Tests must follow red-green development and cover validation and direct-build
paths.

### Unsupported properties

- Reject an unknown `global.json` root property.
- Reject `when_present` and other unknown injection properties.
- Reject an unknown condition property.
- Reject unknown properties in default rules, match objects, and format
  objects.
- Reject unknown canonical base/plain rows-file root properties.
- Reject unknown canonical module root properties.
- Continue accepting legitimate combinations such as:
  - a columns-only module;
  - entries-only and overrides-only modules;
  - a module containing both entries and overrides;
  - current base and plain rows-file roots.
- Continue accepting dataset columns and documented row control fields inside
  rows, entries, overrides, and injection rows.

### `any_present`

- Validation accepts a non-empty `any_present` condition list.
- Validation rejects a non-array or empty `any_present`.
- A build injects when the first alternative is present.
- A build injects when a later alternative is present.
- A build skips the injection when no alternative is present.
- `any_present` combines correctly with `require_present`.
- `any_present` combines correctly with `unless_present`.
- Existing `require_present` and `unless_present` behavior remains unchanged.
- Direct building rejects unsupported condition syntax even when project
  validation was not called first.

### Integration

- Run focused `internal/topdata` tests.
- Run `nix develop --command make check`.
- Validate the active `sow-topdata` checkout with the changed Crucible.
- Build topdata far enough to exercise global injections and confirm no
  generated files are tracked.

## Documentation

Update the class feat global injection contract to document `require_present`,
`any_present`, and `unless_present`, including their all/any/none semantics and
how multiple groups combine.

No broad validation guide is added unless implementation reveals an existing
operator document that would otherwise become stale.

## Acceptance criteria

- `when_present` in a global injection fails validation and direct building.
- `any_present` is valid only as a non-empty list of valid conditions.
- `any_present` applies an injection when one or more alternatives exist and
  skips it when none exist.
- Unknown canonical structural properties fail with actionable diagnostics.
- Dataset-specific row fields remain valid when declared by the dataset.
- Existing active `sow-topdata` validates and builds after replacing invalid
  syntax with the documented grammar.
- No specialized topdata format is tightened outside the stated scope.