stricter topdata validation (#24)
test / test (push) Successful in 1m23s
build-binaries / build-binaries (push) Successful in 2m13s
build-image / publish (push) Successful in 39s

makes topdata json validation much stricter, currently still too permissive

Reviewed-on: #24
Reviewed-by: xtul <mpiasecki720@protonmail.com>
Co-authored-by: vickydotbat <vickydotbat@tutamail.com>
Co-committed-by: vickydotbat <vickydotbat@tutamail.com>
This commit was merged in pull request #24.
This commit is contained in:
2026-06-25 21:49:03 +00:00
committed by archvillainette
parent da6bf17f69
commit 332a24fd3e
7 changed files with 2773 additions and 123 deletions
@@ -0,0 +1,858 @@
# Topdata JSON Validation Tightening Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use
> `superpowers:subagent-driven-development` (recommended) or
> `superpowers:executing-plans` to implement this plan task-by-task. Steps use
> checkbox (`- [ ]`) syntax for tracking.
**Goal:** Reject unsupported properties in established canonical topdata JSON
containers, add the non-empty `any_present` global-injection condition, and keep
standalone validation and direct native builds fail-closed and consistent.
**Architecture:** Add one focused `internal/topdata/json_contract.go` unit that
owns deterministic closed-key checks plus parsing and evaluation of global
injection condition groups. Existing validation in `topdata.go` uses that unit
to collect actionable diagnostics, while `native.go` uses the same parsed
conditions during ordered injection evaluation. Dataset payload rows continue
through the existing column-aware canonicalization paths and are not assigned a
global schema.
**Tech Stack:** Go 1.26, standard-library `encoding/json`, Go tests, Nix flakes,
Make, authored JSON in the sibling `sow-topdata` checkout.
## Global Constraints
- Work on a fresh non-`main` branch and open all changes as a pull request.
- Never commit or push secrets, `.sops` data, generated binaries, `.cache/`,
`generated/`, HAK/TLK artifacts, or result symlinks.
- Add no JSON Schema framework and no new validation dependency.
- Close only the canonical base/plain rows root, canonical module root,
`global.json` root, global injection, global condition, global default rule,
global default match object, and global default format object.
- Preserve specialized parser-owned JSON dialects outside those canonical
containers.
- Preserve dataset-specific columns and documented row controls inside `rows`,
`entries`, `overrides`, injection `row`, and default `values`.
- Preserve current-row, ordered-injection behavior: an earlier injection may
satisfy or block a later injection.
- Preserve empty-list behavior for `require_present` and `unless_present`;
reject an empty `any_present`.
- Do not add a compatibility alias for `when_present`.
- Use deterministic property ordering in diagnostics.
- Inventory evidence requires `compare_reference` to remain valid on
base/plain rows roots: `internal/topdata/native.go` reads it,
`internal/topdata/feat_migrate.go` emits it, and existing tests exercise it.
This is an established canonical property covered by the design's moderate
strictness rule even though the initial property table omitted it.
- The active `sow-topdata` branch
`fix-always-present-metamagic` already replaces the known `when_present` with
`require_present` in commit `ed139f5`; preserve that change and use the
checkout for integration validation rather than duplicating or rewriting it.
---
### Task 1: Introduce the shared closed-object and condition contract
**Files:**
- Create: `internal/topdata/json_contract.go`
- Create: `internal/topdata/json_contract_test.go`
**Interfaces:**
- Produces:
```go
type globalCondition struct {
Field string
ID string
}
type globalConditionGroups struct {
RequirePresent []globalCondition
AnyPresent []globalCondition
UnlessPresent []globalCondition
}
func unsupportedObjectKeys(obj map[string]any, supported ...string) []string
func unsupportedObjectKeysError(label string, obj map[string]any, supported ...string) error
func parseGlobalConditionGroups(injection map[string]any) (globalConditionGroups, error)
func globalConditionGroupsMatch(groups globalConditionGroups, rows []map[string]any) bool
```
- `unsupportedObjectKeys` returns unknown keys in lexical order.
- `parseGlobalConditionGroups` accepts absent groups, permits empty
`require_present` and `unless_present`, rejects empty `any_present`, validates
condition keys as exactly `field` and `id`, and returns the first deterministic
error.
- `globalConditionGroupsMatch` implements all/any/none semantics without
reparsing JSON.
- [ ] **Step 1: Create the feature branch after checking it was not previously
merged and deleted**
```bash
git fetch origin
git branch --show-current
git ls-remote --heads origin feat/topdata-json-validation
git log --all --oneline --decorate --grep='topdata json validation'
```
Expected: current branch is not `main`. If
`feat/topdata-json-validation` has never been used, create it from the approved
design commit:
```bash
git switch -c feat/topdata-json-validation
```
If that exact branch name has prior remote history, choose a new descriptive
name and record it in the PR; do not reuse the old branch.
- [ ] **Step 2: Re-run the canonical-format inventory before writing the
allowlists**
```bash
for f in $(find ../sow-topdata/data -type f -name 'base.json' | sort); do
jq -r --arg f "${f#../sow-topdata/}" \
'[$f, (keys | sort | join(","))] | @tsv' "$f"
done
for f in $(find ../sow-topdata/data -type f -name 'global.json' | sort); do
jq -r --arg f "${f#../sow-topdata/}" \
'[$f, (keys | sort | join(","))] | @tsv' "$f"
done
find ../sow-topdata/data -type f -path '*/modules/*.json' -print0 |
while IFS= read -r -d '' f; do
jq -r --arg f "${f#../sow-topdata/}" \
'[$f, (keys | sort | join(","))] | @tsv' "$f"
done | sort
rg -n 'compare_reference' internal/topdata ../sow-topdata/data
```
Expected: active base roots use `columns`, `key`, `output`, and `rows`;
active modules use `columns`, `entries`, `overrides`, or the legitimate
`entries`+`overrides` combination; active globals use the approved global
keys. `sow-tools` additionally confirms established `compare_reference`
support.
- [ ] **Step 3: Write failing unit tests for deterministic unsupported-key
diagnostics**
Add table-driven tests equivalent to:
```go
func TestUnsupportedObjectKeysAreSorted(t *testing.T) {
obj := map[string]any{
"row": map[string]any{},
"when_present": []any{},
"aaa": true,
}
got := unsupportedObjectKeys(
obj,
"row",
"require_present",
"any_present",
"unless_present",
)
want := []string{"aaa", "when_present"}
if !slices.Equal(got, want) {
t.Fatalf("unsupportedObjectKeys() = %v, want %v", got, want)
}
}
func TestUnsupportedObjectKeysErrorNamesObjectAndContract(t *testing.T) {
err := unsupportedObjectKeysError(
"global injection 10",
map[string]any{"row": map[string]any{}, "when_present": []any{}},
"row",
"require_present",
"any_present",
"unless_present",
)
if err == nil {
t.Fatal("expected unsupported-property error")
}
for _, want := range []string{
"global injection 10",
`"when_present"`,
"row",
"require_present",
"any_present",
"unless_present",
} {
if !strings.Contains(err.Error(), want) {
t.Fatalf("error %q does not contain %q", err, want)
}
}
}
```
- [ ] **Step 4: Write failing parser tests for all condition-group contracts**
Cover these exact cases in `json_contract_test.go`:
```go
tests := []struct {
name string
injection map[string]any
want globalConditionGroups
wantError string
}{
{
name: "all groups",
injection: map[string]any{
"require_present": []any{
map[string]any{"field": "FeatIndex", "id": "feat:required"},
},
"any_present": []any{
map[string]any{"field": "FeatIndex", "id": "feat:first"},
map[string]any{"field": "FeatIndex", "id": "feat:second"},
},
"unless_present": []any{
map[string]any{"field": "FeatIndex", "id": "feat:blocked"},
},
},
want: globalConditionGroups{
RequirePresent: []globalCondition{
{Field: "FeatIndex", ID: "feat:required"},
},
AnyPresent: []globalCondition{
{Field: "FeatIndex", ID: "feat:first"},
{Field: "FeatIndex", ID: "feat:second"},
},
UnlessPresent: []globalCondition{
{Field: "FeatIndex", ID: "feat:blocked"},
},
},
},
{
name: "empty existing groups remain valid",
injection: map[string]any{
"require_present": []any{},
"unless_present": []any{},
},
},
{
name: "any present must be array",
injection: map[string]any{
"any_present": "feat:first",
},
wantError: "any_present must be an array",
},
{
name: "any present must not be empty",
injection: map[string]any{
"any_present": []any{},
},
wantError: "any_present must contain at least one condition",
},
{
name: "condition is closed",
injection: map[string]any{
"require_present": []any{
map[string]any{
"field": "FeatIndex",
"id": "feat:required",
"typo": true,
},
},
},
wantError: `require_present[0] contains unsupported key "typo"`,
},
{
name: "field is required",
injection: map[string]any{
"require_present": []any{
map[string]any{"id": "feat:required"},
},
},
wantError: "require_present[0].field is required",
},
{
name: "id is required",
injection: map[string]any{
"require_present": []any{
map[string]any{"field": "FeatIndex"},
},
},
wantError: "require_present[0].id is required",
},
}
```
- [ ] **Step 5: Write failing evaluator tests for all/any/none and group
conjunction**
Build rows with `FeatIndex` references for `feat:required`,
`feat:second`, and `feat:blocked`, then assert:
```go
if !globalConditionGroupsMatch(globalConditionGroups{
RequirePresent: []globalCondition{
{Field: "FeatIndex", ID: "feat:required"},
},
AnyPresent: []globalCondition{
{Field: "FeatIndex", ID: "feat:first"},
{Field: "FeatIndex", ID: "feat:second"},
},
}, rowsWithoutBlocked) {
t.Fatal("expected required + later any_present alternative to match")
}
if globalConditionGroupsMatch(globalConditionGroups{
AnyPresent: []globalCondition{
{Field: "FeatIndex", ID: "feat:missing"},
},
}, rowsWithoutBlocked) {
t.Fatal("expected missing any_present alternatives to reject")
}
if globalConditionGroupsMatch(globalConditionGroups{
UnlessPresent: []globalCondition{
{Field: "FeatIndex", ID: "feat:blocked"},
},
}, rowsWithBlocked) {
t.Fatal("expected unless_present match to reject")
}
```
- [ ] **Step 6: Run the focused tests and confirm they fail**
```bash
nix develop --command go test ./internal/topdata \
-run 'TestUnsupportedObjectKeys|TestParseGlobalConditionGroups|TestGlobalConditionGroupsMatch' \
-v
```
Expected: build failure because the shared contract types and functions do
not exist.
- [ ] **Step 7: Implement the minimal shared contract**
In `json_contract.go`, define fixed key sets and implement:
```go
var (
baseOrPlainRootKeys = []string{
"output", "key", "columns", "rows", "compare_reference",
}
canonicalModuleRootKeys = []string{
"output", "key", "columns", "entries", "overrides", "rows",
}
globalRootKeys = []string{
"columns", "entries", "overrides", "defaults", "position", "injections",
}
globalInjectionKeys = []string{
"row", "require_present", "any_present", "unless_present",
}
globalConditionKeys = []string{"field", "id"}
globalDefaultRuleKeys = []string{"match", "values"}
globalDefaultMatchKeys = []string{"source"}
globalDefaultFormatKeys = []string{"format"}
)
```
`unsupportedObjectKeysError` must report one deterministic error per call,
using the first lexically sorted unsupported key and a supported-key list in
declaration order. Implement condition parsing through one private helper:
```go
func parseGlobalConditionList(
name string,
raw any,
requireNonEmpty bool,
) ([]globalCondition, error)
```
It must reject non-arrays, reject non-object elements, run the closed-key
check before required-field checks, trim `field` and `id`, and enforce the
`any_present` non-empty rule.
`globalConditionGroupsMatch` must call the existing
`globalReferencePresent` helper and implement:
```text
require_present: every condition is found
any_present: at least one condition is found
unless_present: no condition is found
all groups: every present group passes
```
- [ ] **Step 8: Run the focused and package tests**
```bash
nix develop --command go test ./internal/topdata \
-run 'TestUnsupportedObjectKeys|TestParseGlobalConditionGroups|TestGlobalConditionGroupsMatch' \
-v
nix develop --command go test ./internal/topdata
```
Expected: pass.
- [ ] **Step 9: Commit the shared contract**
```bash
git add internal/topdata/json_contract.go internal/topdata/json_contract_test.go
git commit -m "feat(topdata): define canonical json contracts"
```
---
### Task 2: Close canonical structural objects during project validation
**Files:**
- Modify: `internal/topdata/topdata.go`
- Modify: `internal/topdata/json_contract_test.go`
**Interfaces:**
- Consumes the key sets and condition parser from Task 1.
- `validateDataObject` selects exactly one root contract from file context:
base/plain rows root, canonical module root, or `global.json` root.
- A module validates every present supported container; it does not return
after the first of `entries`, `overrides`, `rows`, or `columns`.
- Specialized paths continue to return to their existing validators before
canonical root closure.
- [ ] **Step 1: Add failing validation tests for unsupported canonical root
properties**
Add table-driven temporary-project cases covering:
- base root with `"typo": true`;
- loose plain rows root with `"typo": true`;
- module root with `"typo": true`;
- `global.json` root with `"typo": true`.
For every case, call `ValidateProject`, require errors, and assert only stable
semantic fragments: the file/object label, `typo`, and `unsupported`.
- [ ] **Step 2: Add failing validation tests for closed nested global objects**
Add cases for:
```json
{"injections":[{"row":{},"when_present":[]}]}
{"injections":[{"row":{},"require_present":[{"field":"FeatIndex","id":"feat:x","typo":true}]}]}
{"defaults":[{"match":"all","values":{},"typo":true}]}
{"defaults":[{"match":{"source":"entries","typo":true},"values":{}}]}
{"defaults":[{"match":"all","values":{"ImpactScript":{"format":"ss_{id}","typo":true}}}]}
```
Expected diagnostics name `global injection 0`, the condition label,
`global default 0`, `global default 0 match`, or
`global default 0 values.ImpactScript` respectively.
- [ ] **Step 3: Add failing validation tests for `any_present` shape**
Test one accepted non-empty list and rejected string/empty-list cases. The
accepted case must use a declared dataset column in the injection row so this
also protects payload flexibility.
- [ ] **Step 4: Add regression tests for canonical combinations and payload
flexibility**
Build one table-driven test that validates without errors:
- a columns-only module;
- an entries-only module;
- an overrides-only module;
- a module containing both entries and overrides;
- a rows module;
- a current base root;
- a loose plain rows root;
- a base/plain root with `compare_reference: false`;
- dataset columns plus `id`, `key`, `inherit`, `_tlk`, and `meta` in row-like
payloads where their existing contracts allow them;
- an injection row using a declared dataset column.
The combined entries+overrides case must contain an independently invalid
override value first, prove validation catches it, then fix only that value
and prove the module passes. This specifically prevents the current
first-container early return.
- [ ] **Step 5: Run the focused tests and confirm failures**
```bash
nix develop --command go test ./internal/topdata \
-run 'TestValidateProject(RejectsUnsupportedCanonicalJSON|RejectsUnsupportedGlobalJSON|ValidatesEveryCanonicalModuleContainer|AcceptsCanonicalJSONContracts|AcceptsAnyPresent|RejectsInvalidAnyPresent)' \
-v
```
Expected: unknown keys are currently ignored, `any_present` is not validated,
and the combined module does not validate all containers.
- [ ] **Step 6: Add a validation adapter for shared contract errors**
In `topdata.go`, add:
```go
func reportTopdataContractError(path string, err error, report *ValidationReport) {
if err == nil {
return
}
report.Diagnostics = append(report.Diagnostics, Diagnostic{
Severity: SeverityError,
Path: path,
Message: err.Error(),
})
}
```
Use `unsupportedObjectKeysError` for canonical roots and nested structural
objects. Keep row payload validation in existing `validateRowCollection`,
`validateEntriesFile`, and `validateOverridesFile` paths.
- [ ] **Step 7: Refactor canonical module dispatch to validate all containers**
After specialized-path checks, classify module files by directory context.
For recognized canonical modules:
1. check `canonicalModuleRootKeys`;
2. call `validateColumnsFile` when `columns` is present;
3. call `validateEntriesFile` when `entries` is present;
4. call `validateOverridesFile` when `overrides` is present;
5. call `validateRowsFile(path, obj, report, false)` when `rows` is present;
6. emit the existing warning only when none of those containers is present.
Avoid duplicate `validateColumnsFile` calls by removing the nested column
calls from this module dispatch path or by adding a caller-controlled flag;
retain current behavior for callers outside canonical module dispatch.
- [ ] **Step 8: Close global roots, injections, defaults, matches, and format
objects**
- Run the global-root key check before validating present containers.
- Run the injection key check before validating `row` and condition groups.
- Replace calls to `validateGlobalConditionList` with
`parseGlobalConditionGroups`; report its error through
`reportTopdataContractError`.
- Run default-rule and match-object key checks before existing type/value
checks.
- Treat an object containing `format` as a format object, reject every key
other than `format`, and preserve non-format object literals as payload
values.
- [ ] **Step 9: Run focused and full package tests**
```bash
nix develop --command go test ./internal/topdata \
-run 'TestValidateProject(RejectsUnsupportedCanonicalJSON|RejectsUnsupportedGlobalJSON|ValidatesEveryCanonicalModuleContainer|AcceptsCanonicalJSONContracts|AcceptsAnyPresent|RejectsInvalidAnyPresent)' \
-v
nix develop --command go test ./internal/topdata
```
Expected: pass.
- [ ] **Step 10: Commit validation tightening**
```bash
git add internal/topdata/topdata.go internal/topdata/json_contract_test.go
git commit -m "feat(topdata): reject unsupported canonical json properties"
```
---
### Task 3: Use shared condition parsing and `any_present` in native builds
**Files:**
- Modify: `internal/topdata/native.go`
- Modify: `internal/topdata/json_contract_test.go`
**Interfaces:**
- Consumes `parseGlobalConditionGroups` and
`globalConditionGroupsMatch` from Task 1.
- Removes the old boolean-mode
`globalConditionListMatches(name, raw, rows, required)` path.
- Direct `BuildNativeWithOptions` calls fail before output mutation when
canonical control syntax is invalid because the existing build entry point
runs `ValidateProject` first and native evaluation uses the same parser.
- [ ] **Step 1: Add failing native-build tests for `any_present` alternatives**
Use one compact plain-dataset fixture with:
- an authored row satisfying the first alternative;
- a second subtest satisfying only the later alternative;
- a third subtest satisfying neither alternative.
Assert the injected row exists in the first two outputs and is absent in the
third. Inspect the built 2DA content rather than internal slices.
- [ ] **Step 2: Add failing native-build tests for condition-group
conjunction**
Add subtests proving:
- `require_present` + `any_present` injects only when both pass;
- `any_present` + `unless_present` injects only when both pass;
- existing `require_present` all semantics remain unchanged;
- existing `unless_present` none semantics remain unchanged.
- [ ] **Step 3: Add a failing ordered-injection test**
Author injection 0 so it adds the row referenced by injection 1's
`any_present`. Assert injection 1 applies. This protects evaluation against
the same current-row view and ordered mutation used today.
- [ ] **Step 4: Add direct-build fail-closed tests**
Call `BuildNativeWithOptions` without a prior explicit `ValidateProject` call
for fixtures containing:
- `when_present` on an injection;
- an unknown condition key;
- an empty `any_present`;
- an unknown canonical root key.
Require an error containing the stable object/property fragments and assert
the expected output 2DA does not exist.
- [ ] **Step 5: Run focused tests and confirm failures**
```bash
nix develop --command go test ./internal/topdata \
-run 'TestBuildNative(AnyPresent|CombinesGlobalConditionGroups|UsesCurrentRowsForAnyPresent|RejectsUnsupportedGlobalControlSyntax|RejectsUnsupportedCanonicalRoot)' \
-v
```
Expected: `any_present` is ignored by current native evaluation and the new
parser/evaluator is not wired.
- [ ] **Step 6: Replace native condition evaluation with the shared contract**
Replace `globalInjectionConditionsMatch` and
`globalConditionListMatches` with:
```go
func globalInjectionConditionsMatch(
injection map[string]any,
rows []map[string]any,
) (bool, error) {
groups, err := parseGlobalConditionGroups(injection)
if err != nil {
return false, err
}
return globalConditionGroupsMatch(groups, rows), nil
}
```
Keep `applyPlainDatasetGlobalInjections` calling this function immediately
before deduplication/canonicalization, using `currentRows()` for each
injection.
- [ ] **Step 7: Run focused, package, and repository tests**
```bash
nix develop --command go test ./internal/topdata \
-run 'TestBuildNative(AnyPresent|CombinesGlobalConditionGroups|UsesCurrentRowsForAnyPresent|RejectsUnsupportedGlobalControlSyntax|RejectsUnsupportedCanonicalRoot)' \
-v
nix develop --command go test ./internal/topdata
nix develop --command go test ./...
```
Expected: pass.
- [ ] **Step 8: Commit native `any_present` support**
```bash
git add internal/topdata/native.go internal/topdata/json_contract_test.go
git commit -m "feat(topdata): add any-present injection conditions"
```
---
### Task 4: Document the global injection grammar
**Files:**
- Modify: `internal/topdata/CLASS_FEAT_GLOBAL_INJECTS_CONTRACT.md`
**Interfaces:**
- Documents the exact injection keys and all/any/none condition semantics.
- States that all present groups must pass and that conditions observe current
rows in injection order.
- States that `any_present` must be a non-empty array and `when_present` is
invalid.
- [ ] **Step 1: Add a documentation contract check**
Run this before editing:
```bash
rg -n 'any_present|when_present|all.*any.*none|require_present.*unless_present' \
internal/topdata/CLASS_FEAT_GLOBAL_INJECTS_CONTRACT.md
```
Expected: no complete description of `any_present` and group conjunction.
- [ ] **Step 2: Update the manifest shape and semantics**
Add an example containing all three groups:
```json
{
"row": {
"FeatIndex": { "id": "feat:example" }
},
"require_present": [{ "field": "FeatIndex", "id": "feat:required" }],
"any_present": [
{ "field": "FeatIndex", "id": "masterfeats:metamagic" },
{ "field": "FeatIndex", "id": "masterfeats:combat" }
],
"unless_present": [{ "field": "FeatIndex", "id": "feat:example" }]
}
```
Document:
- `require_present`: all listed references must exist;
- `any_present`: at least one listed reference must exist and the list cannot
be empty;
- `unless_present`: none of the listed references may exist;
- every present group must pass;
- earlier injections affect later conditions;
- supported injection keys are `row`, `require_present`, `any_present`, and
`unless_present`;
- `when_present` is rejected rather than aliased.
- [ ] **Step 3: Verify the documentation contains the complete contract**
```bash
rg -n 'require_present|any_present|unless_present|when_present|earlier injection|every present group' \
internal/topdata/CLASS_FEAT_GLOBAL_INJECTS_CONTRACT.md
```
Expected: each semantic point appears in the contract.
- [ ] **Step 4: Commit documentation**
```bash
git add internal/topdata/CLASS_FEAT_GLOBAL_INJECTS_CONTRACT.md
git commit -m "docs(topdata): define global injection conditions"
```
---
### Task 5: Validate the active topdata checkout and complete repository checks
**Files:**
- No authored files should change in `sow-tools`.
- Preserve existing sibling change:
`../sow-topdata/data/classes/feats/global.json`.
- Do not retain: `bin/`, `.cache/`, `generated/`, `result`, HAK/TLK files, or
temporary Crucible binaries.
**Interfaces:**
- The built Crucible validates the active `sow-topdata` checkout.
- The active checkout's single-condition correction uses `require_present`;
`any_present` remains available for future multi-alternative authoring.
- [ ] **Step 1: Format and run the full `sow-tools` checks**
```bash
nix develop --command gofmt -w \
internal/topdata/json_contract.go \
internal/topdata/json_contract_test.go \
internal/topdata/topdata.go \
internal/topdata/native.go
nix develop --command make check
```
Expected: vet, Go tests, shellcheck, and yamllint pass.
- [ ] **Step 2: Build a temporary Crucible binary without tracking it**
```bash
nix develop --command go build \
-o /tmp/crucible-topdata-json-validation \
./cmd/crucible
```
Expected: `/tmp/crucible-topdata-json-validation` exists outside the
repository.
- [ ] **Step 3: Confirm the active consumer syntax and repository state**
```bash
git -C ../sow-topdata status --short --branch
rg -n 'when_present|require_present|any_present|unless_present' \
../sow-topdata/data/classes/feats/global.json
```
Expected: the existing consumer branch contains `require_present` and no
`when_present`. Do not alter unrelated consumer work.
- [ ] **Step 4: Validate and build active `sow-topdata` with the changed
Crucible**
```bash
(
cd ../sow-topdata
nix develop --command /tmp/crucible-topdata-json-validation topdata validate
nix develop --command /tmp/crucible-topdata-json-validation topdata build
)
```
Expected: validation and build pass, global injections execute, and generated
outputs remain ignored.
- [ ] **Step 5: Verify no generated files became tracked**
```bash
git status --short
git -C ../sow-topdata status --short
git ls-files --error-unmatch bin generated .cache result 2>/dev/null
git -C ../sow-topdata ls-files --error-unmatch generated .cache result 2>/dev/null
```
Expected: the first two commands show only intentional source/doc changes;
the `git ls-files --error-unmatch` commands find no newly tracked generated
output. Remove only untracked artifacts created by this plan.
- [ ] **Step 6: Run final semantic searches and diff checks**
```bash
rg -n 'when_present' internal/topdata --glob '!docs/superpowers/**'
rg -n 'any_present' \
internal/topdata/json_contract.go \
internal/topdata/json_contract_test.go \
internal/topdata/CLASS_FEAT_GLOBAL_INJECTS_CONTRACT.md
git diff --check
git status --short --branch
git log --oneline origin/main..HEAD
```
Expected: `when_present` appears only in rejection tests/documentation;
implementation, tests, and contract document `any_present`; diff check is
clean.
- [ ] **Step 7: Remove the temporary binary**
```bash
rm -f /tmp/crucible-topdata-json-validation
```
- [ ] **Step 8: Push the feature branch and open the required pull request**
```bash
git push -u origin HEAD
tea pr create \
--title "feat: tighten topdata JSON validation" \
--description $'## Summary\n- reject unsupported canonical topdata JSON properties\n- add non-empty any_present global injection conditions\n- keep validation and direct builds on one shared contract\n\n## Verification\n- nix develop --command make check\n- active sow-topdata validate and build with the changed Crucible'
```
Expected: a PR targeting `main`; no production workflow or release tag is
triggered.
- [ ] **Step 9: Record the handoff**
Report:
- the `sow-tools` branch and PR URL;
- the existing `sow-topdata` consumer branch/commit used for integration;
- exact focused and full verification commands with outcomes;
- confirmation that no generated files were tracked;
- any diagnostics wording intentionally asserted as public semantic
contract.
@@ -0,0 +1,269 @@
# Topdata JSON Validation Tightening Design
**Date:** 2026-06-25
**Status:** Implemented and reviewed
**Scope:** `sow-tools`
## Problem
Topdata JSON validation checks many required properties and value types, but it
often ignores unknown structural properties. This allows plausible typos to
pass validation and then be ignored by the builder.
The immediate failure occurred in
`topdata/data/classes/feats/global.json`: an injection used
`when_present` even though the implemented condition is `require_present`.
Validation succeeded, the builder ignored `when_present`, and the injection was
therefore applied without the intended condition.
This is a fail-open authoring contract. A misspelled build-control property must
not silently change generated game data.
## Goals
- Reject unsupported structural properties in canonical topdata JSON files.
- Add an `any_present` global injection condition.
- Keep standalone validation and direct native builds consistent.
- Preserve dataset-specific row flexibility.
- Tighten only established canonical formats; do not turn this work into a
repository-wide schema rewrite.
- Produce diagnostics that identify the file, object, and unsupported property.
## Non-goals
- No schema migration or formatting change to authored topdata.
- No generic JSON Schema framework or new validation dependency.
- No blanket closure of every specialized JSON dialect.
- No change to generated feat family, registry, spellbook, TLK, or other
specialized formats unless they use the canonical containers covered here.
- No restriction of row values beyond existing column, metadata, inheritance,
reference, TLK, and authoring-sugar contracts.
- No compatibility alias for `when_present`; it is invalid syntax.
## Design
### Validation boundary
Structural objects use closed property sets. Dataset payload objects remain
dataset-aware.
Closed structural objects include:
| Object | Supported properties |
| --- | --- |
| Base or plain rows-file root | `output`, `key`, `columns`, `rows` |
| Canonical module root | `output`, `key`, `columns`, `entries`, `overrides`, `rows` |
| `global.json` root | `columns`, `entries`, `overrides`, `defaults`, `position`, `injections` |
| Global injection | `row`, `require_present`, `any_present`, `unless_present` |
| Global condition | `field`, `id` |
| Global default rule | `match`, `values` |
| Global default match object | `source` |
| Global default format object | `format` |
The allowed root set is selected from the file's existing canonical context;
properties from unrelated formats are not accepted merely because another
topdata file type supports them.
When a canonical root contains more than one supported container, each present
container must be validated. For example, a module containing both `entries`
and `overrides` must not stop validation after recognizing `entries`.
The following remain payload objects rather than global allowlists:
- objects inside `rows`;
- values inside `entries`;
- row-like objects inside `overrides`;
- an injection's `row`;
- a default rule's `values`.
Those objects continue to accept declared dataset columns plus documented row
control and metadata fields. The builder already rejects unknown row and entry
columns when it canonicalizes a dataset. This work must reuse or align with
that knowledge where practical, but must not invent a single fixed row schema
across all datasets.
### Moderate strictness
This change closes only canonical containers whose grammar is already
established by the parser and current authored data.
Before adding a closed root check, implementation must inventory active
`sow-topdata` files and existing `sow-tools` fixtures for legitimate property
combinations. A property used by an established canonical format must be added
to that format's explicit set rather than removed from authored data merely to
satisfy the validator.
Specialized parsers remain responsible for their own object shapes. Extending
closed-property checks to those formats is separate work.
### Global injection conditions
Global injections support three condition groups:
- `require_present`: every listed condition must match a current row.
- `any_present`: at least one listed condition must match a current row.
- `unless_present`: no listed condition may match a current row.
When more than one group is present, every group must pass. Conditions continue
to use the existing shape:
```json
{
"field": "FeatIndex",
"id": "feat:example"
}
```
Example:
```json
{
"row": {
"FeatIndex": {
"id": "feat:example"
}
},
"require_present": [
{
"field": "FeatIndex",
"id": "feat:base_requirement"
}
],
"any_present": [
{
"field": "FeatIndex",
"id": "masterfeats:metamagic"
},
{
"field": "FeatIndex",
"id": "masterfeats:combat"
}
],
"unless_present": [
{
"field": "FeatIndex",
"id": "feat:example"
}
]
}
```
This injection applies only when the required feat exists, either listed
masterfeat exists, and the injected feat does not already exist.
`any_present` must contain at least one condition. An empty list is an
authoring error because it cannot express a useful successful condition.
Existing empty-list behavior for `require_present` and `unless_present` is not
changed by this work.
Conditions are evaluated against the same current-row view and in the same
injection order used today. This preserves the existing behavior where an
earlier injection can affect a later injection's conditions.
### Validation and build consistency
`ValidateProject` must report unsupported structural properties before a build.
The native builder must also fail closed when invoked directly with the same
invalid control syntax.
The implementation should centralize:
- deterministic unsupported-property checks;
- global condition parsing and validation;
- condition group evaluation.
Validation may collect multiple diagnostics while building returns the first
blocking error. Both paths must accept and reject the same property names and
condition shapes.
### Diagnostics
Diagnostics must name the containing object and unsupported property. Property
lists must be deterministic.
For the original error, an acceptable diagnostic is:
```text
global injection 10 contains unsupported key "when_present";
supported keys are row, require_present, any_present, and unless_present
```
Equivalent concise wording is acceptable if tests assert the stable semantic
parts rather than the entire sentence.
## Compatibility
Current canonical `sow-topdata` roots fit the property sets above. The
implementation must rerun the inventory immediately before tightening checks
and run validation against the active `sow-topdata` checkout.
Unknown properties are not retained as compatibility behavior. They currently
have no defined effect, and silently accepting them is the bug being fixed.
Existing documented properties and dataset-specific row fields remain valid.
`require_present` and `unless_present` retain their current semantics.
`any_present` is additive.
## Testing
Tests must follow red-green development and cover validation and direct-build
paths.
### Unsupported properties
- Reject an unknown `global.json` root property.
- Reject `when_present` and other unknown injection properties.
- Reject an unknown condition property.
- Reject unknown properties in default rules, match objects, and format
objects.
- Reject unknown canonical base/plain rows-file root properties.
- Reject unknown canonical module root properties.
- Continue accepting legitimate combinations such as:
- a columns-only module;
- entries-only and overrides-only modules;
- a module containing both entries and overrides;
- current base and plain rows-file roots.
- Continue accepting dataset columns and documented row control fields inside
rows, entries, overrides, and injection rows.
### `any_present`
- Validation accepts a non-empty `any_present` condition list.
- Validation rejects a non-array or empty `any_present`.
- A build injects when the first alternative is present.
- A build injects when a later alternative is present.
- A build skips the injection when no alternative is present.
- `any_present` combines correctly with `require_present`.
- `any_present` combines correctly with `unless_present`.
- Existing `require_present` and `unless_present` behavior remains unchanged.
- Direct building rejects unsupported condition syntax even when project
validation was not called first.
### Integration
- Run focused `internal/topdata` tests.
- Run `nix develop --command make check`.
- Validate the active `sow-topdata` checkout with the changed Crucible.
- Build topdata far enough to exercise global injections and confirm no
generated files are tracked.
## Documentation
Update the class feat global injection contract to document `require_present`,
`any_present`, and `unless_present`, including their all/any/none semantics and
how multiple groups combine.
No broad validation guide is added unless implementation reveals an existing
operator document that would otherwise become stale.
## Acceptance criteria
- `when_present` in a global injection fails validation and direct building.
- `any_present` is valid only as a non-empty list of valid conditions.
- `any_present` applies an injection when one or more alternatives exist and
skips it when none exist.
- Unknown canonical structural properties fail with actionable diagnostics.
- Dataset-specific row fields remain valid when declared by the dataset.
- Existing active `sow-topdata` validates and builds after replacing invalid
syntax with the documented grammar.
- No specialized topdata format is tightened outside the stated scope.
@@ -65,6 +65,33 @@ effective output while keeping project policy in topdata authoring files.
`position` defaults to `append`. The module uses `prepend` for class feat `position` defaults to `append`. The module uses `prepend` for class feat
injections to keep generated policy rows before per-class authored rows. injections to keep generated policy rows before per-class authored rows.
## Global Injection Grammar
Each injection entry may use only `row`, `require_present`, `any_present`, and
`unless_present`. `when_present` is invalid and rejected; it is not an alias.
```json
{
"row": {
"FeatIndex": { "id": "feat:example" }
},
"require_present": [{ "field": "FeatIndex", "id": "feat:required" }],
"any_present": [
{ "field": "FeatIndex", "id": "masterfeats:metamagic" },
{ "field": "FeatIndex", "id": "masterfeats:combat" }
],
"unless_present": [{ "field": "FeatIndex", "id": "feat:example" }]
}
```
- `require_present` means every listed reference must exist.
- `any_present` means at least one listed reference must exist, and the array
must not be empty.
- `unless_present` means none of the listed references may exist.
- Every present group on an entry must pass.
- Conditions observe the current rows in injection order, so earlier injections
can affect later conditions.
## Acceptance Criteria ## Acceptance Criteria
- Class feat output contains the same effective shared feats as the prior YAML - Class feat output contains the same effective shared feats as the prior YAML
+195
View File
@@ -0,0 +1,195 @@
package topdata
import (
"errors"
"fmt"
"slices"
"strings"
)
type globalCondition struct {
Field string
ID string
}
type globalConditionGroups struct {
RequirePresent []globalCondition
AnyPresent []globalCondition
UnlessPresent []globalCondition
}
var (
baseOrPlainRootKeys = []string{
"output", "key", "columns", "rows", "compare_reference",
}
canonicalModuleRootKeys = []string{
"output", "key", "columns", "entries", "overrides", "rows",
}
globalRootKeys = []string{
"columns", "entries", "overrides", "defaults", "position", "injections",
}
globalInjectionKeys = []string{
"row", "require_present", "any_present", "unless_present",
}
globalConditionKeys = []string{"field", "id"}
globalDefaultRuleKeys = []string{"match", "values"}
globalDefaultMatchKeys = []string{"source"}
globalDefaultFormatKeys = []string{"format"}
)
func unsupportedObjectKeys(obj map[string]any, supported ...string) []string {
if len(obj) == 0 {
return nil
}
supportedSet := make(map[string]struct{}, len(supported))
for _, key := range supported {
supportedSet[key] = struct{}{}
}
unsupported := make([]string, 0, len(obj))
for key := range obj {
if _, ok := supportedSet[key]; ok {
continue
}
unsupported = append(unsupported, key)
}
slices.Sort(unsupported)
return unsupported
}
func unsupportedObjectKeysError(label string, obj map[string]any, supported ...string) error {
unsupported := unsupportedObjectKeys(obj, supported...)
if len(unsupported) == 0 {
return nil
}
return fmt.Errorf("%s contains unsupported key %q (supported keys: %s)", label, unsupported[0], strings.Join(supported, ", "))
}
type globalConditionParseOptions struct {
Label string
Report func(error)
}
func parseGlobalConditionGroups(injection map[string]any, options ...globalConditionParseOptions) (globalConditionGroups, error) {
if injection == nil {
return globalConditionGroups{}, nil
}
label := "global injection"
var report func(error)
if len(options) > 0 {
if strings.TrimSpace(options[0].Label) != "" {
label = options[0].Label
}
report = options[0].Report
}
var errs []error
addError := func(err error) {
if err == nil {
return
}
errs = append(errs, err)
if report != nil {
report(err)
}
}
if err := unsupportedObjectKeysError(label, injection, globalInjectionKeys...); err != nil {
addError(err)
}
var groups globalConditionGroups
if raw, ok := injection["require_present"]; ok {
parsed, err := parseGlobalConditionList(label+" require_present", raw, false)
if err != nil {
addError(err)
} else {
groups.RequirePresent = parsed
}
}
if raw, ok := injection["any_present"]; ok {
parsed, err := parseGlobalConditionList(label+" any_present", raw, true)
if err != nil {
addError(err)
} else {
groups.AnyPresent = parsed
}
}
if raw, ok := injection["unless_present"]; ok {
parsed, err := parseGlobalConditionList(label+" unless_present", raw, false)
if err != nil {
addError(err)
} else {
groups.UnlessPresent = parsed
}
}
return groups, errors.Join(errs...)
}
func parseGlobalConditionList(name string, raw any, requireNonEmpty bool) ([]globalCondition, error) {
if raw == nil {
return nil, fmt.Errorf("%s must be an array", name)
}
conditions, ok := raw.([]any)
if !ok {
return nil, fmt.Errorf("%s must be an array", name)
}
if requireNonEmpty && len(conditions) == 0 {
return nil, fmt.Errorf("%s must contain at least one condition", name)
}
parsed := make([]globalCondition, 0, len(conditions))
for index, rawCondition := range conditions {
condition, ok := rawCondition.(map[string]any)
if !ok {
return nil, fmt.Errorf("%s[%d] must be an object", name, index)
}
if err := unsupportedObjectKeysError(fmt.Sprintf("%s[%d]", name, index), condition, globalConditionKeys...); err != nil {
return nil, err
}
field, _ := condition["field"].(string)
field = strings.TrimSpace(field)
if field == "" {
return nil, fmt.Errorf("%s[%d].field is required", name, index)
}
id, _ := condition["id"].(string)
id = strings.TrimSpace(id)
if id == "" {
return nil, fmt.Errorf("%s[%d].id is required", name, index)
}
parsed = append(parsed, globalCondition{Field: field, ID: id})
}
return parsed, nil
}
func globalConditionGroupsMatch(groups globalConditionGroups, rows []map[string]any) bool {
for _, condition := range groups.RequirePresent {
if !globalReferencePresent(rows, condition.Field, condition.ID) {
return false
}
}
if groups.AnyPresent != nil {
if len(groups.AnyPresent) == 0 {
return false
}
matched := false
for _, condition := range groups.AnyPresent {
if globalReferencePresent(rows, condition.Field, condition.ID) {
matched = true
break
}
}
if !matched {
return false
}
}
for _, condition := range groups.UnlessPresent {
if globalReferencePresent(rows, condition.Field, condition.ID) {
return false
}
}
return true
}
File diff suppressed because it is too large Load Diff
+4 -41
View File
@@ -3886,48 +3886,11 @@ func globalManifestPosition(module nativeGeneratedModule) (string, error) {
} }
func globalInjectionConditionsMatch(injection map[string]any, rows []map[string]any) (bool, error) { func globalInjectionConditionsMatch(injection map[string]any, rows []map[string]any) (bool, error) {
if rawRequired, ok := injection["require_present"]; ok { groups, err := parseGlobalConditionGroups(injection)
matches, err := globalConditionListMatches("require_present", rawRequired, rows, true) if err != nil {
if err != nil || !matches { return false, err
return matches, err
}
} }
if rawBlocked, ok := injection["unless_present"]; ok { return globalConditionGroupsMatch(groups, rows), nil
matches, err := globalConditionListMatches("unless_present", rawBlocked, rows, false)
if err != nil || !matches {
return matches, err
}
}
return true, nil
}
func globalConditionListMatches(name string, raw any, rows []map[string]any, required bool) (bool, error) {
conditions, ok := raw.([]any)
if !ok {
return false, fmt.Errorf("%s must be an array", name)
}
for index, rawCondition := range conditions {
condition, ok := rawCondition.(map[string]any)
if !ok {
return false, fmt.Errorf("%s[%d] must be an object", name, index)
}
field, _ := condition["field"].(string)
id, _ := condition["id"].(string)
if strings.TrimSpace(field) == "" {
return false, fmt.Errorf("%s[%d].field is required", name, index)
}
if strings.TrimSpace(id) == "" {
return false, fmt.Errorf("%s[%d].id is required", name, index)
}
found := globalReferencePresent(rows, field, id)
if required && !found {
return false, nil
}
if !required && found {
return false, nil
}
}
return true, nil
} }
func globalRowIdentityPresent(row map[string]any, rows []map[string]any) bool { func globalRowIdentityPresent(row map[string]any, rows []map[string]any) bool {
+68 -82
View File
@@ -333,6 +333,17 @@ func validateDuplicateJSONKeys(path string, raw []byte, report *ValidationReport
} }
} }
func reportTopdataContractError(path string, err error, report *ValidationReport) {
if err == nil {
return
}
report.Diagnostics = append(report.Diagnostics, Diagnostic{
Severity: SeverityError,
Path: path,
Message: err.Error(),
})
}
func scanJSONValueForDuplicateKeys(decoder *json.Decoder, path string, report *ValidationReport) error { func scanJSONValueForDuplicateKeys(decoder *json.Decoder, path string, report *ValidationReport) error {
token, err := decoder.Token() token, err := decoder.Token()
if err != nil { if err != nil {
@@ -425,35 +436,40 @@ func validateDataObject(path string, obj map[string]any, report *ValidationRepor
return return
} }
if base == "base.json" { if base == "base.json" {
reportTopdataContractError(path, unsupportedObjectKeysError("base.json root", obj, baseOrPlainRootKeys...), report)
validateRowsFile(path, obj, report, true) validateRowsFile(path, obj, report, true)
return return
} }
if inModules { if inModules {
if _, hasEntries := obj["entries"]; hasEntries { reportTopdataContractError(path, unsupportedObjectKeysError("module root", obj, canonicalModuleRootKeys...), report)
validateEntriesFile(path, obj, report) _, hasColumns := obj["columns"]
return _, hasEntries := obj["entries"]
} _, hasOverrides := obj["overrides"]
if _, hasOverrides := obj["overrides"]; hasOverrides { _, hasRows := obj["rows"]
validateOverridesFile(path, obj, report) if hasColumns {
return
}
if _, hasRows := obj["rows"]; hasRows {
validateRowsFile(path, obj, report, false)
return
}
if _, hasColumns := obj["columns"]; hasColumns {
validateColumnsFile(path, obj, report) validateColumnsFile(path, obj, report)
return
} }
report.Diagnostics = append(report.Diagnostics, Diagnostic{ if hasEntries {
Severity: SeverityWarning, validateEntriesFileWithColumns(path, obj, report, false)
Path: path, }
Message: "module file does not use a recognized canonical shape yet; expected one of columns, entries, overrides, or rows", if hasOverrides {
}) validateOverridesFileWithColumns(path, obj, report, false)
}
if hasRows {
validateRowsFileWithColumns(path, obj, report, false, false)
}
if !hasColumns && !hasEntries && !hasOverrides && !hasRows {
report.Diagnostics = append(report.Diagnostics, Diagnostic{
Severity: SeverityWarning,
Path: path,
Message: "module file does not use a recognized canonical shape yet; expected one of columns, entries, overrides, or rows",
})
}
return return
} }
if _, hasRows := obj["rows"]; hasRows { if _, hasRows := obj["rows"]; hasRows {
reportTopdataContractError(path, unsupportedObjectKeysError("plain rows root", obj, baseOrPlainRootKeys...), report)
validateRowsFile(path, obj, report, false) validateRowsFile(path, obj, report, false)
return return
} }
@@ -516,6 +532,7 @@ func datasetInvariantForPath(dataPath topdataDataPath) (datasetValidationInvaria
} }
func validateDatasetInvariantBaseFile(path string, obj map[string]any, invariant datasetValidationInvariant, report *ValidationReport) { func validateDatasetInvariantBaseFile(path string, obj map[string]any, invariant datasetValidationInvariant, report *ValidationReport) {
reportTopdataContractError(path, unsupportedObjectKeysError("base.json root", obj, baseOrPlainRootKeys...), report)
validateRowsFile(path, obj, report, true) validateRowsFile(path, obj, report, true)
validateDerivedBaseOutput(path, obj, invariant.Name, report) validateDerivedBaseOutput(path, obj, invariant.Name, report)
validateRequiredColumns(path, obj, invariant, report) validateRequiredColumns(path, obj, invariant, report)
@@ -826,6 +843,10 @@ func validateLockObject(path string, obj map[string]any, report *ValidationRepor
} }
func validateRowsFile(path string, obj map[string]any, report *ValidationReport, requireColumns bool) { func validateRowsFile(path string, obj map[string]any, report *ValidationReport, requireColumns bool) {
validateRowsFileWithColumns(path, obj, report, requireColumns, true)
}
func validateRowsFileWithColumns(path string, obj map[string]any, report *ValidationReport, requireColumns bool, validateColumns bool) {
rows, ok := obj["rows"] rows, ok := obj["rows"]
if !ok { if !ok {
report.Diagnostics = append(report.Diagnostics, Diagnostic{ report.Diagnostics = append(report.Diagnostics, Diagnostic{
@@ -845,7 +866,7 @@ func validateRowsFile(path string, obj map[string]any, report *ValidationReport,
if rowsList, ok := rows.([]any); ok { if rowsList, ok := rows.([]any); ok {
validateRowCollection(path, obj, rowsList, report) validateRowCollection(path, obj, rowsList, report)
} }
if _, ok := obj["columns"]; ok { if _, ok := obj["columns"]; ok && validateColumns {
validateColumnsFile(path, obj, report) validateColumnsFile(path, obj, report)
} else if requireColumns { } else if requireColumns {
report.Diagnostics = append(report.Diagnostics, Diagnostic{ report.Diagnostics = append(report.Diagnostics, Diagnostic{
@@ -896,6 +917,10 @@ func validateColumnsFile(path string, obj map[string]any, report *ValidationRepo
} }
func validateEntriesFile(path string, obj map[string]any, report *ValidationReport) { func validateEntriesFile(path string, obj map[string]any, report *ValidationReport) {
validateEntriesFileWithColumns(path, obj, report, true)
}
func validateEntriesFileWithColumns(path string, obj map[string]any, report *ValidationReport, validateColumns bool) {
entries, ok := obj["entries"] entries, ok := obj["entries"]
if !ok { if !ok {
report.Diagnostics = append(report.Diagnostics, Diagnostic{ report.Diagnostics = append(report.Diagnostics, Diagnostic{
@@ -931,12 +956,16 @@ func validateEntriesFile(path string, obj map[string]any, report *ValidationRepo
validateInlineTextUsage(path, key, entry, report) validateInlineTextUsage(path, key, entry, report)
} }
} }
if _, ok := obj["columns"]; ok { if _, ok := obj["columns"]; ok && validateColumns {
validateColumnsFile(path, obj, report) validateColumnsFile(path, obj, report)
} }
} }
func validateOverridesFile(path string, obj map[string]any, report *ValidationReport) { func validateOverridesFile(path string, obj map[string]any, report *ValidationReport) {
validateOverridesFileWithColumns(path, obj, report, true)
}
func validateOverridesFileWithColumns(path string, obj map[string]any, report *ValidationReport, validateColumns bool) {
overrides, ok := obj["overrides"] overrides, ok := obj["overrides"]
if !ok { if !ok {
report.Diagnostics = append(report.Diagnostics, Diagnostic{ report.Diagnostics = append(report.Diagnostics, Diagnostic{
@@ -957,20 +986,21 @@ func validateOverridesFile(path string, obj map[string]any, report *ValidationRe
container := map[string]any{"rows": overrideList} container := map[string]any{"rows": overrideList}
validateRowCollection(path, container, overrideList, report) validateRowCollection(path, container, overrideList, report)
} }
if _, ok := obj["columns"]; ok { if _, ok := obj["columns"]; ok && validateColumns {
validateColumnsFile(path, obj, report) validateColumnsFile(path, obj, report)
} }
} }
func validateGlobalJSONFile(path string, obj map[string]any, report *ValidationReport) { func validateGlobalJSONFile(path string, obj map[string]any, report *ValidationReport) {
reportTopdataContractError(path, unsupportedObjectKeysError("global.json root", obj, globalRootKeys...), report)
if _, ok := obj["columns"]; ok { if _, ok := obj["columns"]; ok {
validateColumnsFile(path, obj, report) validateColumnsFile(path, obj, report)
} }
if _, ok := obj["entries"]; ok { if _, ok := obj["entries"]; ok {
validateEntriesFile(path, obj, report) validateEntriesFileWithColumns(path, obj, report, false)
} }
if _, ok := obj["overrides"]; ok { if _, ok := obj["overrides"]; ok {
validateOverridesFile(path, obj, report) validateOverridesFileWithColumns(path, obj, report, false)
} }
if _, ok := obj["defaults"]; ok { if _, ok := obj["defaults"]; ok {
validateGlobalDefaults(path, obj, report) validateGlobalDefaults(path, obj, report)
@@ -1029,8 +1059,12 @@ func validateGlobalJSONFile(path string, obj map[string]any, report *ValidationR
} else { } else {
rows = append(rows, row) rows = append(rows, row)
} }
validateGlobalConditionList(path, fmt.Sprintf("global injection %d require_present", index), injection["require_present"], report) _, _ = parseGlobalConditionGroups(injection, globalConditionParseOptions{
validateGlobalConditionList(path, fmt.Sprintf("global injection %d unless_present", index), injection["unless_present"], report) Label: fmt.Sprintf("global injection %d", index),
Report: func(err error) {
reportTopdataContractError(path, err, report)
},
})
} }
validateRowCollection(path, obj, rows, report) validateRowCollection(path, obj, rows, report)
} }
@@ -1099,6 +1133,7 @@ func validateGlobalDefaults(path string, obj map[string]any, report *ValidationR
}) })
continue continue
} }
reportTopdataContractError(path, unsupportedObjectKeysError(fmt.Sprintf("global default %d", index), defaultRule, globalDefaultRuleKeys...), report)
validateGlobalDefaultMatch(path, index, defaultRule["match"], report) validateGlobalDefaultMatch(path, index, defaultRule["match"], report)
rawValues, ok := defaultRule["values"] rawValues, ok := defaultRule["values"]
if !ok { if !ok {
@@ -1155,14 +1190,12 @@ func validateGlobalDefaultMatch(path string, index int, rawMatch any, report *Va
}) })
return return
} }
for key := range match { for _, key := range unsupportedObjectKeys(match, globalDefaultMatchKeys...) {
if key != "source" { report.Diagnostics = append(report.Diagnostics, Diagnostic{
report.Diagnostics = append(report.Diagnostics, Diagnostic{ Severity: SeverityError,
Severity: SeverityError, Path: path,
Path: path, Message: fmt.Sprintf("global default %d match.%s is not supported (unsupported key; supported keys: %s)", index, key, strings.Join(globalDefaultMatchKeys, ", ")),
Message: fmt.Sprintf("global default %d match.%s is not supported", index, key), })
})
}
} }
source, ok := match["source"].(string) source, ok := match["source"].(string)
if !ok || strings.TrimSpace(source) == "" { if !ok || strings.TrimSpace(source) == "" {
@@ -1193,14 +1226,7 @@ func validateGlobalDefaultValue(path string, index int, field string, value any,
if !hasFormat { if !hasFormat {
return return
} }
if len(obj) != 1 { reportTopdataContractError(path, unsupportedObjectKeysError(fmt.Sprintf("global default %d values.%s", index, field), obj, globalDefaultFormatKeys...), report)
report.Diagnostics = append(report.Diagnostics, Diagnostic{
Severity: SeverityError,
Path: path,
Message: fmt.Sprintf("global default %d values.%s format object must contain only format", index, field),
})
return
}
format, ok := rawFormat.(string) format, ok := rawFormat.(string)
if !ok { if !ok {
report.Diagnostics = append(report.Diagnostics, Diagnostic{ report.Diagnostics = append(report.Diagnostics, Diagnostic{
@@ -1220,46 +1246,6 @@ func validateGlobalDefaultValue(path string, index int, field string, value any,
} }
} }
func validateGlobalConditionList(path, label string, raw any, report *ValidationReport) {
if raw == nil {
return
}
conditions, ok := raw.([]any)
if !ok {
report.Diagnostics = append(report.Diagnostics, Diagnostic{
Severity: SeverityError,
Path: path,
Message: fmt.Sprintf("%s must be a JSON array", label),
})
return
}
for index, rawCondition := range conditions {
condition, ok := rawCondition.(map[string]any)
if !ok {
report.Diagnostics = append(report.Diagnostics, Diagnostic{
Severity: SeverityError,
Path: path,
Message: fmt.Sprintf("%s[%d] must be an object", label, index),
})
continue
}
if field, ok := condition["field"].(string); !ok || strings.TrimSpace(field) == "" {
report.Diagnostics = append(report.Diagnostics, Diagnostic{
Severity: SeverityError,
Path: path,
Message: fmt.Sprintf("%s[%d].field is required", label, index),
})
}
if id, ok := condition["id"].(string); !ok || strings.TrimSpace(id) == "" {
report.Diagnostics = append(report.Diagnostics, Diagnostic{
Severity: SeverityError,
Path: path,
Message: fmt.Sprintf("%s[%d].id is required", label, index),
})
}
}
}
func validateStateObject(path string, obj map[string]any, report *ValidationReport) { func validateStateObject(path string, obj map[string]any, report *ValidationReport) {
entries, ok := obj["entries"].(map[string]any) entries, ok := obj["entries"].(map[string]any)
if !ok { if !ok {