758 lines
25 KiB
Markdown
758 lines
25 KiB
Markdown
Refactor the music generation / editing / credits pipeline so it exists entirely inside the shared toolkit and is no longer hardcoded or repository-specific.
|
|
|
|
## Status Snapshot
|
|
|
|
Current state as of 2026-05-13:
|
|
|
|
- This refactor is largely implemented.
|
|
- Music logic now lives under `internal/music/` with toolkit-owned CLI entry
|
|
points and shared use from `build-haks`.
|
|
- Dataset-based config is supported and documented; the toolkit exposes
|
|
`music list-datasets|scan|build|credits|validate|manifest|normalize`.
|
|
- Incremental credits artifact handling and compact/verbose log presentation are
|
|
present in the current implementation.
|
|
|
|
Remaining work:
|
|
|
|
- finish retiring old `music.prefixes`-only assumptions in docs/tests after all
|
|
consumer configs are fully dataset-native
|
|
- continue separating any remaining music-specific presentation rules from
|
|
broader `build-haks` output policy as the build graph evolves
|
|
|
|
Goal: Turn the current Shadows Over Westgate-specific music pipeline into a reusable, repository-agnostic toolkit subsystem that can operate across multiple projects and datasets with configuration-driven behavior.
|
|
|
|
# Requirements
|
|
|
|
## Architecture
|
|
|
|
- Move all music pipeline logic into the toolkit itself.
|
|
- Eliminate repository-specific assumptions, paths, naming conventions, or hardcoded behavior.
|
|
- Repositories should only provide configuration and assets.
|
|
- The toolkit should expose reusable commands/services for:
|
|
- music scanning
|
|
- metadata extraction
|
|
- credits generation
|
|
- track normalization
|
|
- BMU generation/mapping
|
|
- music editing/transcoding
|
|
- manifest generation
|
|
- cache generation
|
|
- validation
|
|
|
|
## Configuration
|
|
|
|
Support configuration at multiple levels:
|
|
|
|
1. Global toolkit defaults
|
|
2. Repository-level configuration
|
|
3. Dataset-level overrides
|
|
|
|
Example hierarchy:
|
|
|
|
- toolkit defaults
|
|
- repo config
|
|
- dataset config
|
|
- per-task overrides
|
|
|
|
The system should merge configuration predictably.
|
|
|
|
## Datasets
|
|
|
|
A "dataset" should represent an isolated music collection/pipeline target.
|
|
|
|
Examples:
|
|
|
|
- envi/music/westgate
|
|
- modules/oc/music
|
|
- premium/music/bardpack
|
|
|
|
Each dataset should support:
|
|
|
|
- independent input/output paths
|
|
- naming rules
|
|
- normalization settings
|
|
- credits settings
|
|
- encoding/transcoding options
|
|
- cache behavior
|
|
- manifest generation
|
|
- validation rules
|
|
|
|
# Desired Features
|
|
|
|
## Reusable pipeline stages
|
|
|
|
Design the pipeline as composable stages.
|
|
|
|
Example:
|
|
|
|
- discover
|
|
- scan metadata
|
|
- normalize
|
|
- transcode
|
|
- generate identifiers
|
|
- build mappings
|
|
- generate credits
|
|
- write manifests
|
|
- validate outputs
|
|
|
|
Stages should:
|
|
|
|
- be independently callable
|
|
- support dry-run
|
|
- support incremental rebuilds
|
|
- support verbose/debug logging
|
|
- expose machine-readable results
|
|
|
|
## Incremental behavior
|
|
|
|
Avoid unnecessary rebuilds.
|
|
|
|
The pipeline should:
|
|
|
|
- detect unchanged inputs
|
|
- reuse generated artifacts
|
|
- avoid duplicate rescans
|
|
- avoid rewriting unchanged outputs
|
|
- report concise summaries
|
|
|
|
## Repository independence
|
|
|
|
Do not assume:
|
|
|
|
- NWN-specific paths
|
|
- westgate naming
|
|
- BMU naming formats
|
|
- specific cache directories
|
|
- specific manifest names
|
|
|
|
All of these should be configurable.
|
|
|
|
# CLI design
|
|
|
|
Design clean toolkit commands.
|
|
|
|
Example direction:
|
|
|
|
- toolkit music scan
|
|
- toolkit music build
|
|
- toolkit music credits
|
|
- toolkit music normalize
|
|
- toolkit music validate
|
|
|
|
Or:
|
|
|
|
- toolkit dataset build <dataset>
|
|
|
|
The exact CLI structure may change if a cleaner abstraction exists.
|
|
|
|
# Configuration examples
|
|
|
|
Support something like:
|
|
|
|
```yaml
|
|
music:
|
|
defaults:
|
|
normalization:
|
|
enabled: true
|
|
|
|
datasets:
|
|
westgate:
|
|
source: envi/music/westgate
|
|
output: build/music/westgate
|
|
|
|
naming:
|
|
scheme: nwn_bmu
|
|
|
|
credits:
|
|
enabled: true
|
|
output: .cache/credits/westgate
|
|
|
|
premium_bardpack:
|
|
source: premium/music/bardpack
|
|
|
|
normalization:
|
|
loudness: -14
|
|
|
|
transcoding:
|
|
format: ogg
|
|
```
|
|
|
|
# Implementation expectations
|
|
|
|
- Separate orchestration from implementation details.
|
|
- Separate scanning, transformation, and output stages cleanly.
|
|
- Introduce typed dataset definitions/config models.
|
|
- Centralize logging and cache management.
|
|
- Prefer declarative configuration over imperative repository scripts.
|
|
- Keep generated outputs deterministic.
|
|
- Preserve compatibility where possible through adapters/shims.
|
|
- Minimize repository glue code.
|
|
|
|
# Migration
|
|
|
|
- Migrate existing Shadows Over Westgate behavior onto the new system.
|
|
- Preserve existing outputs unless intentionally improved.
|
|
- Add compatibility wrappers if needed.
|
|
- Ensure old workflows continue functioning during transition.
|
|
|
|
# Deliverables
|
|
|
|
1. Proposed architecture
|
|
2. Configuration schema
|
|
3. CLI design
|
|
4. Migration strategy
|
|
5. Refactor plan
|
|
6. Example configs
|
|
7. Before/after repository responsibilities
|
|
8. Suggested module/package layout
|
|
9. Identification of reusable abstractions
|
|
10. Example logs/output
|
|
|
|
Focus on maintainability, reuse, composability, and repository independence.
|
|
|
|
# Initial project-needs analysis
|
|
|
|
## Current state
|
|
|
|
- Music processing is currently implemented in `internal/pipeline/music.go`.
|
|
- The music pipeline is not independently callable. It is invoked as part of:
|
|
- `BuildHAKs` / `planOrBuildHAKs`
|
|
- `plannedModuleHAKOrder`
|
|
- The current behavior assumes the repository has an asset tree and that music lives under `envi/music`.
|
|
- Convertible source formats are currently hardcoded to:
|
|
- `.mp3`
|
|
- `.ogg`
|
|
- Passthrough audio/resource formats are currently hardcoded to:
|
|
- `.bmu`
|
|
- `.wav`
|
|
- Generated BMU-like resources are staged under `.cache/music`, loaded into HAK resources, and then the staging root is deleted.
|
|
- Generated credits are written under `.cache/credits`.
|
|
- Credits inventory is written to `.cache/credits/credits.json`.
|
|
- Authored credit overlays are discovered by walking the whole configured assets directory for `CREDITS.md`.
|
|
- A per-directory `CREDITS.md` can override probed metadata and manually pin the generated output filename.
|
|
- FFmpeg and ffprobe are resolved from:
|
|
- `SOW_FFMPEG`
|
|
- `SOW_FFPROBE`
|
|
- `PATH`
|
|
- The ffmpeg command currently strips metadata and writes MP3-compatible audio bytes to a file named `.bmu`.
|
|
- Output stems are constrained to 16 characters, lower-case ASCII letters, digits, and underscores.
|
|
- Generated output names are reserved against every existing asset stem, not just music files.
|
|
- Current generated credits writes are already incremental: desired files are rendered in memory, unchanged files are not rewritten, and stale generated artifacts are removed.
|
|
- App-layer output for music is currently coupled to `build-haks` summary output in `internal/app/app.go`.
|
|
|
|
## Existing behavior to preserve during migration
|
|
|
|
- `build-haks` must continue to convert configured music sources into HAK resources without requiring repository scripts.
|
|
- Existing Shadows Over Westgate outputs should remain stable by default:
|
|
- source root: `envi/music`
|
|
- westgate prefix mapping: `envi/music/westgate -> mus_wg_`
|
|
- generated output extension: `.bmu`
|
|
- generated cache inventory: `.cache/credits/credits.json`
|
|
- generated credits markdown shape and columns
|
|
- manual `CREDITS.md` overlay semantics
|
|
- duplicate output-name collision behavior
|
|
- 16-character NWN resource stem limit
|
|
- Existing concise/verbose `build-haks` log behavior should keep working.
|
|
- Existing environment variable overrides for ffmpeg/ffprobe should continue to work as compatibility aliases.
|
|
- Machine-readable generated artifacts should remain JSON unless there is a separate explicit migration contract.
|
|
|
|
## Current code areas affected
|
|
|
|
- `internal/project/project.go`
|
|
- `Config.Music` currently only supports `prefixes`.
|
|
- `ValidateLayout` calls `validateMusicConfig`.
|
|
- `AssetExtensions` currently defines which music files appear in project inventory.
|
|
- `Scan` only scans the configured assets directory, so datasets outside `paths.assets` need a deliberate design.
|
|
- `internal/pipeline/music.go`
|
|
- all scanning, naming, metadata, conversion, credits, cache, and cleanup logic is currently one private pipeline helper.
|
|
- hardcoded assumptions include `envi/music`, `.cache/music`, `.cache/credits`, `CREDITS.md`, BMU output, ffmpeg args, metadata tags, and NWN stem rules.
|
|
- `internal/pipeline/build.go`
|
|
- HAK planning/building depends on `prepareMusicAssets`.
|
|
- plan-only HAK workflows can still invoke music probing/conversion today through HAK planning paths.
|
|
- `internal/app/app.go`
|
|
- `build-haks` owns the only user-facing music summary.
|
|
- New music commands should reuse the same logging/verbosity model instead of inventing unrelated output.
|
|
- `internal/pipeline/pipeline_test.go`
|
|
- currently has regression coverage for conversion, generated credits, manual overlay, inventory inclusion, staging cleanup, and unchanged second pass.
|
|
- `internal/app/app_test.go`
|
|
- currently checks normal versus verbose mapping output.
|
|
- `README.md`
|
|
- currently documents only `music.prefixes`.
|
|
|
|
# Design constraints and decisions
|
|
|
|
## Dataset roots
|
|
|
|
Datasets should explicitly state where they read from and where generated artifacts go.
|
|
|
|
Important distinction:
|
|
|
|
- `source` is the authored input root for a dataset.
|
|
- `output` is the generated publish/staging root for transformed audio resources.
|
|
- `credits.output` is where generated human-readable credits go.
|
|
- `manifest.output` is where generated machine-readable dataset state goes.
|
|
- `cache.root` is where reusable implementation caches live.
|
|
|
|
Do not assume dataset sources are under `paths.assets`. Support both:
|
|
|
|
- asset-relative dataset roots for existing HAK packaging behavior
|
|
- project-relative dataset roots for future non-HAK or external datasets
|
|
|
|
For HAK builds, the pipeline must know whether generated outputs are meant to be injected into the project asset inventory and under which resource-relative path.
|
|
|
|
## NWN-specific behavior
|
|
|
|
The toolkit can support NWN conventions, but they must be selected by configuration instead of implicit global behavior.
|
|
|
|
Suggested built-in profiles:
|
|
|
|
- `nwn_bmu`
|
|
- output extension: `.bmu`
|
|
- max stem length: 16
|
|
- allowed stem characters: `[a-z0-9_]`
|
|
- lower-case output names
|
|
- default ffmpeg encoder compatible with current behavior
|
|
- `passthrough`
|
|
- no renaming unless configured
|
|
- no transcoding unless configured
|
|
- `audio_export`
|
|
- configurable format, stem length, and codec rules without NWN resource constraints
|
|
|
|
Do not make `nwn_bmu` the only naming implementation.
|
|
|
|
## Configuration merge semantics
|
|
|
|
Merge configuration in this order:
|
|
|
|
1. toolkit defaults
|
|
2. repository `music.defaults`
|
|
3. dataset definition
|
|
4. command-line/per-task overrides
|
|
|
|
Merge rules should be explicit:
|
|
|
|
- scalar fields replace inherited values
|
|
- maps merge by key
|
|
- lists replace by default unless a field is documented as appendable
|
|
- empty strings mean "not set", not "clear this inherited field"
|
|
- booleans that need tri-state inheritance should use pointer/nullable semantics internally
|
|
- command-line overrides should be represented in result metadata so logs can show effective behavior
|
|
|
|
Reject unknown fields via strict YAML decode, matching the rest of the project config philosophy.
|
|
|
|
## Compatibility config shape
|
|
|
|
The current config:
|
|
|
|
```yaml
|
|
music:
|
|
prefixes:
|
|
envi/music/westgate: mus_wg_
|
|
```
|
|
|
|
should be treated as a compatibility shorthand equivalent to:
|
|
|
|
```yaml
|
|
music:
|
|
datasets:
|
|
envi_music:
|
|
source: envi/music
|
|
package:
|
|
mode: hak_asset
|
|
output_root: envi/music
|
|
naming:
|
|
scheme: nwn_bmu
|
|
prefixes:
|
|
westgate: mus_wg_
|
|
```
|
|
|
|
Exact normalized dataset ids may differ, but the migration must define how old `music.prefixes` maps into the new dataset model.
|
|
|
|
## Proposed expanded schema
|
|
|
|
Example target:
|
|
|
|
```yaml
|
|
music:
|
|
tools:
|
|
ffmpeg: "" # optional; fallback to SOW_FFMPEG then PATH
|
|
ffprobe: "" # optional; fallback to SOW_FFPROBE then PATH
|
|
|
|
defaults:
|
|
discover:
|
|
include:
|
|
- "**/*.mp3"
|
|
- "**/*.ogg"
|
|
passthrough:
|
|
- "**/*.bmu"
|
|
- "**/*.wav"
|
|
exclude: []
|
|
credits_file: CREDITS.md
|
|
|
|
naming:
|
|
scheme: nwn_bmu
|
|
max_stem_length: 16
|
|
prefix: ""
|
|
prefixes: {}
|
|
drop_words: []
|
|
reserved_stems_from_assets: true
|
|
|
|
metadata:
|
|
enabled: true
|
|
prefer_overlay: true
|
|
probe_tags:
|
|
title:
|
|
- title
|
|
artist:
|
|
- artist
|
|
- album_artist
|
|
- composer
|
|
rights:
|
|
- license
|
|
- license_url
|
|
- copyright
|
|
|
|
transcoding:
|
|
enabled: true
|
|
output_extension: .bmu
|
|
codec: libmp3lame
|
|
format: mp3
|
|
sample_rate: 44100
|
|
channels: 2
|
|
bitrate: 192k
|
|
strip_metadata: true
|
|
ffmpeg_args: []
|
|
|
|
credits:
|
|
enabled: true
|
|
output: .cache/credits
|
|
inventory: .cache/credits/credits.json
|
|
format: nwn_markdown_table
|
|
|
|
cache:
|
|
root: .cache/music
|
|
mode: content_hash
|
|
keep_staging: false
|
|
|
|
manifest:
|
|
enabled: true
|
|
output: .cache/music/manifest.json
|
|
format: json
|
|
|
|
validation:
|
|
require_credits_for_generated: false
|
|
reject_unknown_overlay_rows: true
|
|
reject_output_collisions: true
|
|
|
|
datasets:
|
|
westgate:
|
|
source: envi/music/westgate
|
|
output: envi/music/westgate
|
|
package:
|
|
mode: hak_asset
|
|
naming:
|
|
prefix: mus_wg_
|
|
```
|
|
|
|
Schema names can change during implementation, but the final shape should preserve these concepts.
|
|
|
|
# Pipeline architecture
|
|
|
|
## Suggested package layout
|
|
|
|
Prefer a dedicated internal subsystem instead of expanding `internal/pipeline/music.go`.
|
|
|
|
Suggested layout:
|
|
|
|
```text
|
|
internal/music/
|
|
config.go typed config, defaults, merge, validation
|
|
dataset.go resolved dataset model and effective settings
|
|
discover.go source discovery and file classification
|
|
metadata.go ffprobe integration and metadata normalization
|
|
credits.go markdown parsing/rendering and inventory generation
|
|
naming.go naming schemes, collision detection, compatibility prefix logic
|
|
transcode.go ffmpeg integration and output writing
|
|
cache.go content hashing and incremental state
|
|
manifest.go machine-readable dataset/stage manifests
|
|
validate.go dataset validation diagnostics
|
|
pipeline.go stage orchestration
|
|
result.go machine-readable stage/build result types
|
|
```
|
|
|
|
Then keep `internal/pipeline` responsible for HAK/module orchestration and call `internal/music` through a narrow adapter.
|
|
|
|
## Stage model
|
|
|
|
Each stage should have:
|
|
|
|
- typed input
|
|
- typed output
|
|
- deterministic ordering
|
|
- dry-run support where it can avoid writes
|
|
- changed/unchanged reporting
|
|
- structured diagnostics
|
|
- stable machine-readable result data
|
|
|
|
Suggested stages:
|
|
|
|
1. Resolve datasets and effective config.
|
|
2. Discover candidate files.
|
|
3. Load authored overlays.
|
|
4. Probe source metadata.
|
|
5. Plan output names and detect collisions.
|
|
6. Plan transformations.
|
|
7. Reuse cached outputs or transcode changed files.
|
|
8. Render generated credits.
|
|
9. Write dataset manifest/inventory.
|
|
10. Return generated package resources to the HAK pipeline.
|
|
11. Validate outputs and report diagnostics.
|
|
|
|
## Result model
|
|
|
|
Music command results should be structured enough for both CLI output and tests.
|
|
|
|
Include:
|
|
|
|
- dataset id
|
|
- source root
|
|
- output root
|
|
- files discovered
|
|
- files converted
|
|
- files reused from cache
|
|
- passthrough files
|
|
- skipped files
|
|
- generated resources
|
|
- credit artifacts
|
|
- manifest paths
|
|
- changed files
|
|
- warnings/errors
|
|
- source-to-output mappings
|
|
- effective naming/transcoding profile
|
|
|
|
Do not require the app layer to parse free-form progress strings.
|
|
|
|
# Incremental and cache behavior
|
|
|
|
## Content identity
|
|
|
|
Incremental rebuilds should be based on content and effective configuration, not only mtimes.
|
|
|
|
Cache keys should include:
|
|
|
|
- source content hash
|
|
- source relative path
|
|
- effective dataset id
|
|
- effective naming settings
|
|
- effective transcoding settings
|
|
- ffmpeg command/profile version
|
|
- toolkit music pipeline version
|
|
- manual overlay data that affects output metadata/name
|
|
|
|
Changing an output prefix, codec, bitrate, overlay output filename, or metadata rendering rules must invalidate the relevant planned output.
|
|
|
|
## Output writes
|
|
|
|
- Avoid rewriting unchanged credits, manifests, and transformed outputs.
|
|
- Write through temporary files and atomic rename where practical.
|
|
- Preserve deterministic JSON indentation and key ordering.
|
|
- Remove stale generated artifacts inside owned generated roots.
|
|
- Never remove authored source files.
|
|
- Never remove files outside configured owned cache/output roots.
|
|
|
|
## Plan-only behavior
|
|
|
|
Current HAK planning can trigger conversion work. The refactor should define stricter behavior:
|
|
|
|
- `music scan` and HAK `--plan-only` should not transcode by default.
|
|
- They may probe metadata only if needed and allowed.
|
|
- They should be able to produce planned mappings without writing transformed audio.
|
|
- If actual resource size is needed for HAK chunk planning, use cached transformed size when available and report when exact planning requires a build.
|
|
|
|
# CLI design expansion
|
|
|
|
Add first-class commands while preserving `build-haks` compatibility.
|
|
|
|
Suggested commands:
|
|
|
|
```text
|
|
sow-toolkit music list-datasets
|
|
sow-toolkit music scan [--dataset <id>] [--json]
|
|
sow-toolkit music build [--dataset <id>] [--dry-run] [--force]
|
|
sow-toolkit music credits [--dataset <id>] [--check]
|
|
sow-toolkit music validate [--dataset <id>] [--json]
|
|
sow-toolkit music manifest [--dataset <id>] [--check]
|
|
```
|
|
|
|
Integration commands:
|
|
|
|
```text
|
|
sow-toolkit build-haks
|
|
sow-toolkit build-haks --skip-music
|
|
sow-toolkit build-haks --music-dataset <id>
|
|
```
|
|
|
|
Command output should follow the existing log refactor direction:
|
|
|
|
- normal mode: phase summaries and counts
|
|
- verbose mode: mappings and per-file actions
|
|
- debug mode: resolved config, tool paths, ffmpeg args, cache keys
|
|
- quiet mode: final summary only
|
|
- `--json`: machine-readable result on stdout and human diagnostics on stderr
|
|
|
|
# Validation requirements
|
|
|
|
Validate repository config:
|
|
|
|
- duplicate dataset ids
|
|
- invalid dataset source/output paths
|
|
- absolute paths unless explicitly allowed
|
|
- path traversal outside project root for owned outputs/caches
|
|
- duplicate or ambiguous prefix mappings
|
|
- prefix too long for selected naming scheme
|
|
- unsupported output extension for selected package mode
|
|
- invalid codec/profile fields
|
|
- impossible inherited config combinations
|
|
- `package.mode: hak_asset` without an asset-relative output root
|
|
- unknown naming/transcoding/credits/manifest schemes
|
|
|
|
Validate discovered music:
|
|
|
|
- source files with unsupported extensions
|
|
- duplicate case-insensitive output stems
|
|
- output name collision with existing assets and generated outputs
|
|
- manual overlay rows referencing unknown originals or outputs
|
|
- manual overlay output filenames violating selected naming scheme
|
|
- metadata probe failures
|
|
- source paths differing only by case on case-insensitive targets
|
|
- generated resource names that exceed NWN constraints
|
|
- generated output extension/resource type compatibility
|
|
|
|
Validation severity should distinguish:
|
|
|
|
- errors that block deterministic build output
|
|
- warnings for incomplete credits or missing optional metadata
|
|
- info for skipped files and unchanged caches
|
|
|
|
# Fringe considerations
|
|
|
|
## Credits and licensing
|
|
|
|
- Preserve authored `CREDITS.md` files as source overlays.
|
|
- Generated credits should clearly identify generated versus authored sources in inventory.
|
|
- Missing artist/title/license can be warning-level or error-level per dataset.
|
|
- Overlay parsing should keep accepting backtick-wrapped filenames.
|
|
- Markdown parsing should continue to support escaped pipes and `<br>` line breaks.
|
|
- Generated credits should be deterministic across platforms.
|
|
- Consider whether aggregate credits inventory should include datasets disabled for build but enabled for credits.
|
|
|
|
## Filename and path edge cases
|
|
|
|
- Normalize separators to forward slashes in config, manifests, and logs.
|
|
- Treat paths case-insensitively for resource collision checks.
|
|
- Reject NUL bytes and unsafe relative paths.
|
|
- Define behavior for spaces, Unicode, apostrophes, brackets, years, and punctuation in stems.
|
|
- Keep existing slug/drop-word behavior for `nwn_bmu` compatibility, but make drop words configurable.
|
|
- Avoid Windows-reserved names and problematic trailing dots/spaces if outputs may be checked out or built on Windows.
|
|
- Detect collisions after truncation and suffixing.
|
|
- Validate suffixing still respects max stem length.
|
|
|
|
## Audio tooling
|
|
|
|
- Keep `SOW_FFMPEG` and `SOW_FFPROBE` as compatibility fallbacks.
|
|
- Add config-level tool paths for repositories that pin local toolchain binaries.
|
|
- Expose ffmpeg/ffprobe version in debug output and manifest metadata.
|
|
- Capture ffmpeg failures with enough context to diagnose source file, dataset, and command profile.
|
|
- Do not print enormous ffmpeg logs in normal mode unless the command fails.
|
|
- Decide whether `.ogg -> .bmu` should continue transcoding or become configurable passthrough for datasets that want native OGG output.
|
|
|
|
## HAK/resource integration
|
|
|
|
- Generated music resources must enter collision detection alongside authored resources.
|
|
- Source `.mp3`/`.ogg` files that are converted must be omitted from HAK packaging unless config says otherwise.
|
|
- Passthrough `.bmu` and `.wav` handling should remain distinct from generated outputs.
|
|
- Resource sorting should stay deterministic and reuse existing HAK resource ordering rules.
|
|
- Generated music should be represented in build manifests with enough source provenance to debug.
|
|
- HAK include globs should match generated output paths, not source paths, unless explicitly configured otherwise.
|
|
|
|
## Cross-repository behavior
|
|
|
|
- A repository with no `music` config should not pay a heavy ffmpeg/ffprobe cost.
|
|
- A repository with authored `.bmu` files but no generated music config should preserve current passthrough packaging.
|
|
- Multiple datasets should be able to target different output roots without collisions.
|
|
- Multiple repositories should be able to use different naming schemes without code changes.
|
|
- Dataset config should be portable between Linux and Windows path conventions.
|
|
- The toolkit should not assume Shadows Over Westgate names, HAK names, prefixes, or source layout.
|
|
|
|
# Migration strategy
|
|
|
|
## Phase 1: Extract without behavior changes
|
|
|
|
- Move existing private music functions from `internal/pipeline/music.go` into `internal/music`.
|
|
- Keep the current `prepareMusicAssets` adapter in `internal/pipeline` to avoid broad HAK changes.
|
|
- Preserve every existing test expectation.
|
|
- Add focused unit tests around moved naming, overlay, metadata, and credits functions.
|
|
|
|
## Phase 2: Introduce dataset config
|
|
|
|
- Extend `project.MusicConfig` with typed defaults and datasets.
|
|
- Keep `music.prefixes` as deprecated compatibility shorthand.
|
|
- Implement config normalization from legacy prefixes into a default dataset.
|
|
- Add strict validation for dataset ids, paths, naming, and prefix constraints.
|
|
- Update README examples.
|
|
|
|
## Phase 3: Add first-class music commands
|
|
|
|
- Wire `music scan`, `music build`, `music credits`, and `music validate` into `internal/app`.
|
|
- Reuse existing verbosity levels and summary formatting.
|
|
- Add JSON output support if the command result model is ready.
|
|
- Ensure `build-haks` calls the same service API as the music commands.
|
|
|
|
## Phase 4: Incremental generated output cache
|
|
|
|
- Replace temporary-only `.cache/music` staging with a content-addressed cache when configured.
|
|
- Keep cleanup behavior compatible for existing builds unless `cache.keep_staging` is enabled.
|
|
- Avoid conversion during dry-run/plan-only unless exact output artifacts are explicitly requested.
|
|
- Add stale generated artifact cleanup scoped only to owned roots.
|
|
|
|
## Phase 5: Repository migration
|
|
|
|
- Convert Shadows Over Westgate config from `music.prefixes` to explicit dataset config.
|
|
- Keep compatibility warning for one release window.
|
|
- Update consumer wrapper scripts only if command names change.
|
|
- Verify existing generated HAK manifests and credits inventories remain stable or document intentional diffs.
|
|
|
|
# Test plan
|
|
|
|
Add or preserve tests for:
|
|
|
|
- legacy `music.prefixes` compatibility
|
|
- explicit dataset config equivalent to current Westgate behavior
|
|
- dataset source outside `envi/music`
|
|
- multiple datasets with independent prefixes and outputs
|
|
- no music config with authored `.bmu` passthrough assets
|
|
- manual overlay output filename validation
|
|
- manual overlay references unknown source/output
|
|
- duplicate generated output collisions
|
|
- collision with existing authored asset stem
|
|
- max stem length and suffix collision behavior
|
|
- unchanged second build does not rewrite credits or manifests
|
|
- dry-run/plan-only does not transcode
|
|
- missing ffmpeg/ffprobe failure messages
|
|
- configured ffmpeg/ffprobe paths
|
|
- normal, verbose, quiet, debug, and JSON CLI output
|
|
- Windows-style path separators in config normalize deterministically
|
|
- generated credits and inventory are deterministic
|
|
|
|
# Acceptance criteria
|
|
|
|
- Music functionality is reusable through a dedicated toolkit subsystem.
|
|
- No repository-specific path or prefix is hardcoded in music implementation.
|
|
- Existing `build-haks` behavior remains compatible for Shadows Over Westgate.
|
|
- First-class music commands can scan, build, generate credits, and validate a configured dataset.
|
|
- Dataset configuration can express current Westgate behavior and at least one non-Westgate layout.
|
|
- Generated outputs are deterministic and unchanged files are not rewritten.
|
|
- Machine-readable artifacts remain JSON.
|
|
- Logs are concise by default and detailed under verbose/debug.
|
|
- Tests cover the compatibility path and the new dataset-driven path.
|