Wiki pipeline (#3)

Implemented the wiki pipeline to a shippable local state across `toolkit/`, `module/`, and the NodeBB wiki plugin.

**Scope Audited**
- Toolkit config/project loading, wiki renderer, deployer, app command wiring.
- Module `nwn-tool.yaml`, wiki source/docs, release script, topdata docs.
- Plugin sanitizer/API storage contract docs and sanitizer tests.

**Changes Made**
- Added topdata-owned wiki declarations under `module/topdata/wiki/`.
- Added `topdata.wiki.*` config fields, validation, effective config output.
- Switched generated wiki pages to `.html` with HTML comment managed/manual markers.
- Added `page-index.json` generation and deterministic metadata.
- Added manual-section merge preservation and stale page `report`/`archive` handling.
- Added namespace `category_env` loading from `topdata/wiki/namespaces.yaml`, while keeping `--category`/`NODEBB_WIKI_CATEGORIES` overrides.
- Updated release deployment to keep dry-run first and make stale cleanup opt-in via `SOW_MODULE_WIKI_DEPLOY_STALE_POLICY`.
- Updated plugin sanitizer to preserve only `sow-topdata-wiki` comments.

**Configuration/Schema Impact**
- `module/nwn-tool.yaml` now declares wiki source, renderer, link strategy, templates, manual sections, managed marker policy, and stale policy.
- Deploy manifest entries now support stale/archive metadata: `stale`, `last_seen_hash`, `archived_hash`, `title`, `namespace`, `tid`, `pid`, `cid`.

**Compatibility Impact**
- Legacy Markdown managed markers are still readable for migration.
- Generated output is now HTML, not Markdown.
- Existing mapped pages update by `pid`; creates still require explicit `--create`.

**Tests Added/Updated**
- Go tests for wiki config validation, HTML rendering, page index, manual merge, stale report/archive, app summary output.
- Plugin sanitizer test for generated topdata bot HTML markers/subset.

**Validation Performed**
- `go test ./internal/project ./internal/topdata ./internal/app` passed.
- `./build-tool.sh` passed.
- `SOW_TOOLS_DEV_BINARY=../toolkit/tools/sow-toolkit ./validate-topdata.sh` passed.
- `SOW_TOOLS_DEV_BINARY=../toolkit/tools/sow-toolkit ./build-wiki.sh --force` passed, generating 1345 entity pages.
- `deploy-wiki --dry-run --create` with dummy endpoint/token and namespace CID env vars passed locally: 1377 planned creates, 0 stale/drift.
- `node tests/wiki-html-sanitizer.test.js` passed.

**Remaining Risks**
- Full plugin `npm test` is blocked by an existing unrelated drawer CSS contract failure in `tests/wiki-article-drawers.test.js`.
- Controlled live migration was not performed because it requires a non-production NodeBB namespace and credentials.
- Direct `PUT /api/v3/posts/{pid}` save-filter behavior is documented as needing live confirmation against the deployed NodeBB/plugin stack.

Reviewed-on: https://gitea.westgate.pw/ShadowsOverWestgate/sow-tools/pulls/3
Co-authored-by: vickydotbat <vickydotbat@tutamail.com>
Co-committed-by: vickydotbat <vickydotbat@tutamail.com>
This commit is contained in:
2026-05-15 09:09:37 +02:00
committed by archvillainette
parent d767ec0a25
commit 9b2084344d
9 changed files with 1030 additions and 166 deletions
+81
View File
@@ -256,10 +256,19 @@ extract:
cleanup_stale: false
topdata:
wiki:
source: topdata/wiki
managed_namespaces:
- skills
renderer: nodebb_tiptap_html
link_strategy: preserve_westgate_wiki_links
namespaces_file: namespaces.yaml
templates_dir: templates
manual_sections_dir: manual-sections
deploy_manifest: wiki-manifest.json
deploy_edit_summary: Custom summary
stale_pages:
default: report
live_cleanup: archive
autogen:
cache:
root: "{paths.cache}/released"
@@ -297,6 +306,30 @@ autogen:
if got, want := effective.TopData.Wiki.DeployManifest, "wiki-manifest.json"; got != want {
t.Fatalf("expected wiki deploy manifest %q, got %q", want, got)
}
if got, want := effective.TopData.Wiki.Source, "topdata/wiki"; got != want {
t.Fatalf("expected wiki source %q, got %q", want, got)
}
if got, want := effective.TopData.Wiki.Renderer, "nodebb_tiptap_html"; got != want {
t.Fatalf("expected wiki renderer %q, got %q", want, got)
}
if got, want := effective.TopData.Wiki.LinkStrategy, "preserve_westgate_wiki_links"; got != want {
t.Fatalf("expected wiki link strategy %q, got %q", want, got)
}
if got, want := effective.TopData.Wiki.NamespacesFile, "namespaces.yaml"; got != want {
t.Fatalf("expected wiki namespaces file %q, got %q", want, got)
}
if got, want := effective.TopData.Wiki.TemplatesDir, "templates"; got != want {
t.Fatalf("expected wiki templates dir %q, got %q", want, got)
}
if got, want := effective.TopData.Wiki.ManualSectionsDir, "manual-sections"; got != want {
t.Fatalf("expected wiki manual sections dir %q, got %q", want, got)
}
if got, want := effective.TopData.Wiki.StalePages.Default, "report"; got != want {
t.Fatalf("expected wiki stale default %q, got %q", want, got)
}
if got, want := effective.TopData.Wiki.StalePages.LiveCleanup, "archive"; got != want {
t.Fatalf("expected wiki stale live cleanup %q, got %q", want, got)
}
if got, want := effective.Autogen.Cache.MaxAge, "30m"; got != want {
t.Fatalf("expected autogen cache max age %q, got %q", want, got)
}
@@ -704,6 +737,54 @@ func TestValidateLayoutRejectsUnsafeGeneratedOutputPaths(t *testing.T) {
}
}
func TestValidateLayoutRejectsInvalidTopDataWikiConfig(t *testing.T) {
root := t.TempDir()
mkdirAll(t, filepath.Join(root, "src"))
mkdirAll(t, filepath.Join(root, "build"))
proj := &Project{
Root: root,
Config: Config{
Module: ModuleConfig{Name: "Test", ResRef: "test"},
Paths: PathConfig{Source: "src", Build: "build"},
TopData: TopDataConfig{
Source: "topdata",
Wiki: TopDataWikiConfig{
Source: "../wiki",
Renderer: "dokuwiki",
LinkStrategy: "rewrite_links",
NamespacesFile: "../namespaces.yaml",
TemplatesDir: ".",
ManualSectionsDir: "../manual",
StalePages: TopDataWikiStalePagesConfig{
Default: "delete",
LiveCleanup: "delete",
},
},
},
},
}
err := proj.ValidateLayout()
if err == nil {
t.Fatal("expected invalid wiki config validation error")
}
for _, needle := range []string{
"topdata.wiki.source",
"topdata.wiki.renderer",
"topdata.wiki.link_strategy",
"topdata.wiki.namespaces_file",
"topdata.wiki.templates_dir",
"topdata.wiki.manual_sections_dir",
"topdata.wiki.stale_pages.default",
"topdata.wiki.stale_pages.live_cleanup",
} {
if !strings.Contains(err.Error(), needle) {
t.Fatalf("expected validation error to mention %s, got %v", needle, err)
}
}
}
func TestValidateLayoutRejectsRootSourceAndAssetPaths(t *testing.T) {
root := t.TempDir()
mkdirAll(t, filepath.Join(root, "build"))