Log Refactor

This commit is contained in:
2026-05-07 13:33:25 +02:00
parent 13a615dbd7
commit 7abcbf1a38
6 changed files with 706 additions and 53 deletions
+149
View File
@@ -0,0 +1,149 @@
You are improving CLI/build logs for a NWN asset toolchain.
Goal: Make the logs cleaner, prettier, less repetitive, and easier to understand, without hiding important diagnostics.
Current issues:
- The same credits refresh output appears twice.
- Long file-by-file mappings overwhelm the main log.
- Some messages are noisy or too implementation-focused.
- The final build summary is useful but visually cramped.
- Paths are too long for normal human reading unless needed for debugging.
- Status prefixes are inconsistent: [credits], [build-haks], run-nwn-tool.sh, [|].
Desired behavior:
1. Use clear phase headers.
2. Collapse repetitive file listings by default.
3. Show counts and summaries in normal mode.
4. Keep detailed mappings available behind a verbose/debug flag.
5. Avoid printing the same credits scan twice unless something changed.
6. Use consistent prefixes and indentation.
7. Prefer relative paths when possible.
8. Group related output together.
9. End with a compact, readable summary.
10. Preserve warnings, errors, and actionable diagnostics prominently.
Suggested output style:
Build HAKs ────────── • Refreshing credits cache
- Scanned: envi/music/westgate
- Tracks: 38
- Output: .cache/credits/envi/music/westgate/credits.json
- Aggregate cache updated
• Validating project • Collecting asset resources • Writing autogen manifests
- sow-parts-manifest.json
- sow-head-vfx-manifest.json
• Planning HAK chunks • Resolving module HAK order • Reusing unchanged HAKs
- 12 / 12 reused
- 65,320 assets total
• Cleaning stale generated HAKs • Writing HAK manifest
Summary ─────── Project: Shadows Over Westgate HAK archives: 12 HAK assets: 65,320 Manifest: assets/build/haks.json Credits artifacts: 2 Autogen manifests: 2 Status: complete
Rules:
- In normal mode, do not print every `mp3 -> bmu` mapping.
- Instead print something like: `Mapped 38 music files. Use --verbose to list mappings.`
- In verbose mode, print mappings under an indented section.
- If the credits cache is refreshed before and after the build for normalization, make the second refresh terse: `Restored normalized credits cache: unchanged.`
- If files changed, say how many changed and where.
- Do not remove any warnings or errors.
- Do not make failure logs terse; failures should include enough context to debug.
- Keep output deterministic so CI diffs remain readable.
- Use plain ASCII unless the toolchain already supports Unicode reliably.
- Do not introduce color unless color can be disabled with NO_COLOR or --no-color.
Implementation guidance:
- Add logging helpers for phases, steps, summaries, verbose details, warnings, and errors.
- Add a verbosity level, for example: quiet, normal, verbose, debug.
- Route repeated detailed output through verbose/debug logging.
- Track counts while scanning instead of printing every scanned item.
- Detect unchanged cache writes and report them as unchanged rather than rewriting noisy output.
- Normalize absolute paths to project-relative paths in normal output.
- Keep machine-readable artifacts unchanged.
Concrete implementation plan:
1. CLI presentation layer
- Keep the pipeline responsible for state and artifact generation.
- Move `build-haks` presentation rules into `internal/app/app.go`.
- Introduce `--quiet`, `--verbose`, and `--debug` for `build-haks`.
- In normal mode, print grouped sections and a compact summary.
- In verbose mode, include detailed mapping and per-archive action lines.
- In debug mode, preserve raw pipeline progress messages for diagnosis.
2. Spinner behavior
- Reuse the existing spinner infrastructure instead of adding a second animation system.
- Make the spinner phase-aware so it shows the current high-level step, for example:
- `Build HAKs: refreshing credits cache`
- `Build HAKs: collecting asset resources`
- `Build HAKs: planning HAK chunks`
- Only animate on interactive TTY stderr.
- Disable spinner animation automatically in CI and non-interactive output.
- Always keep stable human-readable logs on stdout.
3. Pipeline data surfaced to the CLI
- Extend `pipeline.BuildResult` with log-summary metadata instead of forcing the CLI to parse raw strings.
- Return credits refresh summary data:
- scanned music directories
- converted track count
- generated credits artifact paths
- per-file source/output mappings
- changed-versus-unchanged artifact count
- Return HAK archive summary data:
- total archive count
- reused archive count
- written archive count
- per-archive actions for verbose mode
4. Credits artifact handling
- Stop blindly deleting and rewriting `.cache/credits` on every run.
- Render the desired credits outputs in memory first.
- Compare desired artifacts against the current on-disk artifacts.
- Rewrite only changed files.
- Remove stale artifacts that are no longer desired.
- Report `unchanged` when the second pass produces identical output.
5. Output rules
- Normal mode:
- show phase blocks
- show counts
- show relative paths
- hide file-by-file mappings
- Verbose mode:
- show music mappings under an indented `mappings:` section
- show per-archive `reused` or `wrote` details
- Quiet mode:
- keep only the final summary
- Debug mode:
- allow raw internal progress lines in addition to the formatted summary
6. Tests
- Add app-layer tests for:
- compact normal-mode output
- verbose mapping output
- Add pipeline regression coverage proving:
- music conversion still writes the expected credits artifacts
- a second identical credits refresh reports zero changed files
Acceptance checks:
- `build-haks` emits the new grouped summary format.
- `build-haks --verbose` includes per-track mappings and per-archive details.
- The spinner reflects phase changes without polluting stdout logs.
- Re-running the same build keeps credits refresh output deterministic and reports `unchanged` when applicable.
- Existing machine-readable artifacts remain unchanged in schema and location.
+273 -27
View File
@@ -37,26 +37,50 @@ type spinner struct {
text string text string
msg []string msg []string
painted bool painted bool
writer io.Writer
enabled bool
} }
var spin = &spinner{ func newSpinner() *spinner {
return &spinner{
msg: []string{"-", "\\", "|", "/"}, msg: []string{"-", "\\", "|", "/"},
}
}
var spin = newSpinner()
func (s *spinner) configure(writer io.Writer, enabled bool) {
s.mu.Lock()
defer s.mu.Unlock()
s.writer = writer
s.enabled = enabled
} }
func (s *spinner) start(text string) { func (s *spinner) start(text string) {
s.mu.Lock() s.mu.Lock()
defer s.mu.Unlock() defer s.mu.Unlock()
if !s.enabled || s.writer == nil {
s.on = false
s.painted = false
s.text = text
return
}
s.on = true s.on = true
s.text = text s.text = text
go s.run(text) go s.run()
} }
func (s *spinner) stop() { func (s *spinner) stop() {
s.mu.Lock() s.mu.Lock()
defer s.mu.Unlock() defer s.mu.Unlock()
if !s.enabled || s.writer == nil {
s.on = false
s.painted = false
return
}
s.on = false s.on = false
if s.painted { if s.painted {
fmt.Fprint(os.Stderr, "\r\033[K\n") fmt.Fprint(s.writer, "\r\033[K")
s.painted = false s.painted = false
} }
} }
@@ -64,23 +88,28 @@ func (s *spinner) stop() {
func (s *spinner) linebreak() { func (s *spinner) linebreak() {
s.mu.Lock() s.mu.Lock()
defer s.mu.Unlock() defer s.mu.Unlock()
if s.on && s.painted { if s.enabled && s.on && s.painted {
fmt.Fprint(os.Stderr, "\r\033[K\n") fmt.Fprint(s.writer, "\r\033[K")
s.painted = false s.painted = false
} }
} }
func (s *spinner) run(text string) { func (s *spinner) update(text string) {
s.mu.Lock()
defer s.mu.Unlock()
s.text = text
}
func (s *spinner) run() {
i := 0 i := 0
for { for {
s.mu.Lock() s.mu.Lock()
if !s.on { if !s.on {
s.mu.Unlock() s.mu.Unlock()
fmt.Fprintf(os.Stderr, "\r")
return return
} }
msg := s.msg[i%len(s.msg)] msg := s.msg[i%len(s.msg)]
fmt.Fprintf(os.Stderr, "\r[%s] %s", msg, text) fmt.Fprintf(s.writer, "\r[%s] %s", msg, s.text)
s.painted = true s.painted = true
s.mu.Unlock() s.mu.Unlock()
i++ i++
@@ -88,6 +117,18 @@ func (s *spinner) run(text string) {
} }
} }
func isInteractiveTTY(w io.Writer) bool {
file, ok := w.(*os.File)
if !ok {
return false
}
info, err := file.Stat()
if err != nil {
return false
}
return (info.Mode() & os.ModeCharDevice) != 0
}
var commands = []command{ var commands = []command{
{ {
name: "build", name: "build",
@@ -184,6 +225,7 @@ func Run(args []string) (int, error) {
default: default:
for _, cmd := range commands { for _, cmd := range commands {
if cmd.name == args[0] { if cmd.name == args[0] {
spin.configure(ctx.stderr, isInteractiveTTY(ctx.stderr) && strings.TrimSpace(os.Getenv("CI")) == "")
spin.start("running " + cmd.name) spin.start("running " + cmd.name)
if err := cmd.run(ctx); err != nil { if err := cmd.run(ctx); err != nil {
spin.stop() spin.stop()
@@ -263,6 +305,212 @@ type buildHAKOptions struct {
filteredArchives []string filteredArchives []string
sourceManifest string sourceManifest string
planOnly bool planOnly bool
logLevel logLevel
}
type logLevel int
const (
logLevelNormal logLevel = iota
logLevelQuiet
logLevelVerbose
logLevelDebug
)
type buildHAKConsole struct {
stdout io.Writer
projectRoot string
projectName string
planOnly bool
level logLevel
spinnerEnabled bool
}
func newBuildHAKConsole(ctx context, p *project.Project, opts buildHAKOptions) *buildHAKConsole {
return &buildHAKConsole{
stdout: ctx.stdout,
projectRoot: p.Root,
projectName: p.Config.Module.Name,
planOnly: opts.planOnly,
level: opts.logLevel,
spinnerEnabled: isInteractiveTTY(ctx.stderr) && strings.TrimSpace(os.Getenv("CI")) == "" && opts.logLevel == logLevelNormal,
}
}
func (c *buildHAKConsole) progress(message string) {
if phase := c.phaseLabel(message); phase != "" {
spin.update("Build HAKs: " + phase)
}
if c.level != logLevelDebug {
return
}
spin.linebreak()
fmt.Fprintf(c.stdout, "[debug] %s\n", message)
}
func (c *buildHAKConsole) phaseLabel(message string) string {
switch {
case strings.HasPrefix(message, "Validating project"):
return "validating project"
case strings.HasPrefix(message, "Collecting asset resources"):
return "collecting asset resources"
case strings.HasPrefix(message, "Planning HAK chunks"):
return "planning HAK chunks"
case strings.HasPrefix(message, "Resolving module HAK order"):
return "resolving module HAK order"
case strings.HasPrefix(message, "Reusing unchanged generated HAKs"):
return "reusing unchanged HAKs"
case strings.HasPrefix(message, "Reusing HAK "), strings.HasPrefix(message, "Writing HAK "):
return "writing HAK archives"
case strings.HasPrefix(message, "Cleaning stale generated HAKs"):
return "cleaning stale generated HAKs"
case strings.HasPrefix(message, "Cleaning previous generated HAKs"):
return "cleaning previous generated HAKs"
case strings.HasPrefix(message, "Writing HAK manifest"):
return "writing HAK manifest"
case strings.HasPrefix(message, "Writing autogen manifest "):
return "writing autogen manifests"
default:
return ""
}
}
func (c *buildHAKConsole) emitResult(result pipeline.BuildResult) {
spin.linebreak()
fmt.Fprintln(c.stdout, "Build HAKs ----------")
if c.level != logLevelQuiet {
c.emitCreditsSummary(result)
c.emitCoreSteps(result)
}
c.emitFinalSummary(result)
}
func (c *buildHAKConsole) emitCreditsSummary(result pipeline.BuildResult) {
summary := result.CreditsSummary
if len(summary.ArtifactPaths) == 0 {
return
}
fmt.Fprintln(c.stdout, "Refreshing credits cache")
if len(summary.ScannedDirs) > 0 {
fmt.Fprintf(c.stdout, " scanned: %s\n", strings.Join(summary.ScannedDirs, ", "))
}
fmt.Fprintf(c.stdout, " tracks: %d\n", summary.TrackCount)
if len(summary.GeneratedPaths) > 0 {
for _, path := range summary.GeneratedPaths {
fmt.Fprintf(c.stdout, " output: %s\n", c.relPath(path))
}
}
switch {
case summary.ChangedFiles == 0:
fmt.Fprintln(c.stdout, " status: unchanged")
default:
fmt.Fprintf(c.stdout, " updated: %d artifact(s)\n", summary.ChangedFiles)
}
if summary.TrackCount > 0 {
if c.level >= logLevelVerbose {
fmt.Fprintln(c.stdout, " mappings:")
for _, mapping := range summary.Mappings {
fmt.Fprintf(c.stdout, " %s -> %s\n", mapping.Source, mapping.Output)
}
} else {
fmt.Fprintf(c.stdout, " mapped: %d music file(s); use --verbose to list mappings\n", summary.TrackCount)
}
}
fmt.Fprintln(c.stdout)
}
func (c *buildHAKConsole) emitCoreSteps(result pipeline.BuildResult) {
fmt.Fprintln(c.stdout, "Validating project")
fmt.Fprintln(c.stdout, "Collecting asset resources")
if len(result.AutogenManifestPaths) > 0 {
fmt.Fprintln(c.stdout, "Writing autogen manifests")
for _, path := range result.AutogenManifestPaths {
fmt.Fprintf(c.stdout, " %s\n", filepath.Base(path))
}
fmt.Fprintln(c.stdout)
}
fmt.Fprintln(c.stdout, "Planning HAK chunks")
fmt.Fprintln(c.stdout, "Resolving module HAK order")
if !c.planOnly {
fmt.Fprintln(c.stdout, "Reusing unchanged HAKs")
fmt.Fprintf(c.stdout, " reused: %d / %d\n", result.HAKSummary.Reused, result.HAKSummary.Total)
fmt.Fprintf(c.stdout, " written: %d / %d\n", result.HAKSummary.Written, result.HAKSummary.Total)
fmt.Fprintf(c.stdout, " assets: %s\n", formatCount(result.HAKAssets))
if c.level >= logLevelVerbose {
for _, action := range result.HAKSummary.Actions {
verb := "wrote"
if action.Reused {
verb = "reused"
}
fmt.Fprintf(c.stdout, " %s: %s (%s assets)\n", verb, action.Name, formatCount(action.AssetCount))
}
}
fmt.Fprintln(c.stdout)
fmt.Fprintln(c.stdout, "Cleaning stale generated HAKs")
} else {
fmt.Fprintf(c.stdout, "Planned HAK archives: %d\n", result.HAKSummary.Total)
}
fmt.Fprintln(c.stdout, "Writing HAK manifest")
fmt.Fprintln(c.stdout)
}
func (c *buildHAKConsole) emitFinalSummary(result pipeline.BuildResult) {
fmt.Fprintln(c.stdout, "Summary ----------")
fmt.Fprintf(c.stdout, "project: %s\n", c.projectName)
if c.planOnly {
fmt.Fprintf(c.stdout, "planned archives: %d\n", result.HAKSummary.Total)
} else {
fmt.Fprintf(c.stdout, "hak archives: %d\n", len(result.HAKPaths))
fmt.Fprintf(c.stdout, "hak assets: %s\n", formatCount(result.HAKAssets))
}
if result.Manifest != "" {
fmt.Fprintf(c.stdout, "manifest: %s\n", c.relPath(result.Manifest))
}
if len(result.CreditsArtifactPaths) > 0 {
fmt.Fprintf(c.stdout, "credits artifacts: %d\n", len(result.CreditsArtifactPaths))
}
if len(result.AutogenManifestPaths) > 0 {
fmt.Fprintf(c.stdout, "autogen manifests: %d\n", len(result.AutogenManifestPaths))
}
status := "complete"
if c.planOnly {
status = "planned"
}
fmt.Fprintf(c.stdout, "status: %s\n", status)
}
func (c *buildHAKConsole) relPath(path string) string {
if path == "" {
return path
}
rel, err := filepath.Rel(c.projectRoot, path)
if err != nil {
return filepath.ToSlash(path)
}
return filepath.ToSlash(rel)
}
func formatCount(value int) string {
raw := strconv.Itoa(value)
if value < 1000 && value > -1000 {
return raw
}
sign := ""
if strings.HasPrefix(raw, "-") {
sign = "-"
raw = strings.TrimPrefix(raw, "-")
}
parts := make([]string, 0, (len(raw)+2)/3)
for len(raw) > 3 {
parts = append(parts, raw[len(raw)-3:])
raw = raw[:len(raw)-3]
}
parts = append(parts, raw)
slices := make([]string, 0, len(parts))
for index := len(parts) - 1; index >= 0; index-- {
slices = append(slices, parts[index])
}
return sign + strings.Join(slices, ",")
} }
func runBuildHAKs(ctx context) error { func runBuildHAKs(ctx context) error {
@@ -280,16 +528,15 @@ func runBuildHAKs(ctx context) error {
return err return err
} }
progress := func(message string) { console := newBuildHAKConsole(ctx, p, opts)
spin.linebreak() spin.configure(ctx.stderr, console.spinnerEnabled)
fmt.Fprintf(ctx.stdout, "[build-haks] %s\n", message) spin.update("Build HAKs: starting")
}
var result pipeline.BuildResult var result pipeline.BuildResult
if opts.planOnly { if opts.planOnly {
result, err = pipeline.PlanHAKsWithProgress(p, progress) result, err = pipeline.PlanHAKsWithProgress(p, console.progress)
} else { } else {
result, err = pipeline.BuildHAKsWithOptions(p, pipeline.BuildHAKOptions{ result, err = pipeline.BuildHAKsWithOptions(p, pipeline.BuildHAKOptions{
Progress: progress, Progress: console.progress,
ArchiveNames: opts.filteredArchives, ArchiveNames: opts.filteredArchives,
SourceManifestPath: opts.sourceManifest, SourceManifestPath: opts.sourceManifest,
}) })
@@ -298,18 +545,7 @@ func runBuildHAKs(ctx context) error {
return err return err
} }
fmt.Fprintf(ctx.stdout, "project: %s\n", p.Config.Module.Name) console.emitResult(result)
fmt.Fprintf(ctx.stdout, "hak archives: %d\n", len(result.HAKPaths))
fmt.Fprintf(ctx.stdout, "hak assets: %d\n", result.HAKAssets)
if result.Manifest != "" {
fmt.Fprintf(ctx.stdout, "hak manifest: %s\n", result.Manifest)
}
if len(result.CreditsArtifactPaths) > 0 {
fmt.Fprintf(ctx.stdout, "credits artifacts: %d\n", len(result.CreditsArtifactPaths))
}
if len(result.AutogenManifestPaths) > 0 {
fmt.Fprintf(ctx.stdout, "autogen manifests: %d\n", len(result.AutogenManifestPaths))
}
return nil return nil
} }
@@ -323,7 +559,7 @@ func parseBuildHAKArgs(args []string) (buildHAKOptions, error) {
arg := args[index] arg := args[index]
switch arg { switch arg {
case "-h", "--help": case "-h", "--help":
return opts, errors.New("usage: build-haks [--hak <hak-name> ...] [--archive <archive-name> ...] [--source-manifest <path>] [--plan-only]") return opts, errors.New("usage: build-haks [--hak <hak-name> ...] [--archive <archive-name> ...] [--source-manifest <path>] [--plan-only] [--quiet|--verbose|--debug]")
case "--hak": case "--hak":
index++ index++
if index >= len(args) { if index >= len(args) {
@@ -338,6 +574,12 @@ func parseBuildHAKArgs(args []string) (buildHAKOptions, error) {
opts.filteredArchives = append(opts.filteredArchives, args[index]) opts.filteredArchives = append(opts.filteredArchives, args[index])
case "--plan-only": case "--plan-only":
opts.planOnly = true opts.planOnly = true
case "--quiet":
opts.logLevel = logLevelQuiet
case "--verbose":
opts.logLevel = logLevelVerbose
case "--debug":
opts.logLevel = logLevelDebug
case "--source-manifest": case "--source-manifest":
index++ index++
if index >= len(args) { if index >= len(args) {
@@ -357,6 +599,10 @@ func parseBuildHAKArgs(args []string) (buildHAKOptions, error) {
opts.sourceManifest = value opts.sourceManifest = value
continue continue
} }
if arg == "--quiet=true" {
opts.logLevel = logLevelQuiet
continue
}
return opts, fmt.Errorf("unknown build-haks argument %q", arg) return opts, fmt.Errorf("unknown build-haks argument %q", arg)
} }
} }
+139
View File
@@ -59,6 +59,145 @@ func TestRunBuildTopPackageUsesCachedCompiledOutputs(t *testing.T) {
} }
} }
func TestRunBuildHAKsEmitsCompactSummary(t *testing.T) {
root := t.TempDir()
mkdirAll(t, filepath.Join(root, "assets", "envi", "music", "westgate"))
mkdirAll(t, filepath.Join(root, "build"))
mkdirAll(t, filepath.Join(root, "src"))
writeFile(t, filepath.Join(root, "nwn-tool.json"), `{
"module": {
"name": "Test Module",
"resref": "testmod"
},
"paths": {
"source": "src",
"assets": "assets",
"build": "build"
},
"haks": [
{
"name": "envi",
"priority": 1,
"max_bytes": 1048576,
"split": false,
"include": ["envi/**"]
}
]
}`+"\n")
writeFile(t, filepath.Join(root, "assets", "envi", "music", "westgate", "AleandAnecdotes.mp3"), "source-mp3")
writeFile(t, filepath.Join(root, "assets", "envi", "music", "westgate", "CREDITS.md"), "# NWN Music Pack Credits\n\n| Artist | Title | Output File | Original File | Album | Date | License / Copyright | Notes |\n|---|---|---|---|---|---|---|---|\n| Darren Curtis | Ale and Anecdotes | `mus_wg_andnc.bmu` | `AleandAnecdotes.mp3` | Darren's Commercially Free Music | | Darren Curtis Music | |\n")
ffprobePath := filepath.Join(root, "ffprobe")
writeFile(t, ffprobePath, "#!/bin/sh\nprintf '%s\\n' '{\"format\":{\"tags\":{\"artist\":\"Unknown\",\"title\":\"Broken Metadata\"}}}'\n")
if err := os.Chmod(ffprobePath, 0o755); err != nil {
t.Fatalf("chmod ffprobe: %v", err)
}
ffmpegPath := filepath.Join(root, "ffmpeg")
writeFile(t, ffmpegPath, "#!/bin/sh\nfor last; do :; done\nprintf 'bmu-data' > \"$last\"\n")
if err := os.Chmod(ffmpegPath, 0o755); err != nil {
t.Fatalf("chmod ffmpeg: %v", err)
}
t.Setenv("SOW_FFMPEG", ffmpegPath)
t.Setenv("SOW_FFPROBE", ffprobePath)
var stdout bytes.Buffer
ctx := context{
stdout: &stdout,
stderr: &bytes.Buffer{},
cwd: root,
args: []string{"build-haks"},
}
if err := runBuildHAKs(ctx); err != nil {
t.Fatalf("runBuildHAKs failed: %v", err)
}
output := stdout.String()
if !strings.Contains(output, "Build HAKs ----------") {
t.Fatalf("expected build header, got %q", output)
}
if !strings.Contains(output, "mapped: 1 music file(s); use --verbose to list mappings") {
t.Fatalf("expected compact mapping summary, got %q", output)
}
if strings.Contains(output, "AleandAnecdotes.mp3 -> mus_wg_andnc.bmu") {
t.Fatalf("did not expect verbose mapping in normal mode, got %q", output)
}
if !strings.Contains(output, "manifest: build/haks.json") {
t.Fatalf("expected relative manifest path, got %q", output)
}
}
func TestRunBuildHAKsVerboseListsMappings(t *testing.T) {
root := t.TempDir()
mkdirAll(t, filepath.Join(root, "assets", "envi", "music", "westgate"))
mkdirAll(t, filepath.Join(root, "build"))
mkdirAll(t, filepath.Join(root, "src"))
writeFile(t, filepath.Join(root, "nwn-tool.json"), `{
"module": {
"name": "Test Module",
"resref": "testmod"
},
"paths": {
"source": "src",
"assets": "assets",
"build": "build"
},
"haks": [
{
"name": "envi",
"priority": 1,
"max_bytes": 1048576,
"split": false,
"include": ["envi/**"]
}
]
}`+"\n")
writeFile(t, filepath.Join(root, "assets", "envi", "music", "westgate", "AleandAnecdotes.mp3"), "source-mp3")
writeFile(t, filepath.Join(root, "assets", "envi", "music", "westgate", "CREDITS.md"), "# NWN Music Pack Credits\n\n| Artist | Title | Output File | Original File | Album | Date | License / Copyright | Notes |\n|---|---|---|---|---|---|---|---|\n| Darren Curtis | Ale and Anecdotes | `mus_wg_andnc.bmu` | `AleandAnecdotes.mp3` | Darren's Commercially Free Music | | Darren Curtis Music | |\n")
ffprobePath := filepath.Join(root, "ffprobe")
writeFile(t, ffprobePath, "#!/bin/sh\nprintf '%s\\n' '{\"format\":{\"tags\":{\"artist\":\"Unknown\",\"title\":\"Broken Metadata\"}}}'\n")
if err := os.Chmod(ffprobePath, 0o755); err != nil {
t.Fatalf("chmod ffprobe: %v", err)
}
ffmpegPath := filepath.Join(root, "ffmpeg")
writeFile(t, ffmpegPath, "#!/bin/sh\nfor last; do :; done\nprintf 'bmu-data' > \"$last\"\n")
if err := os.Chmod(ffmpegPath, 0o755); err != nil {
t.Fatalf("chmod ffmpeg: %v", err)
}
t.Setenv("SOW_FFMPEG", ffmpegPath)
t.Setenv("SOW_FFPROBE", ffprobePath)
var stdout bytes.Buffer
ctx := context{
stdout: &stdout,
stderr: &bytes.Buffer{},
cwd: root,
args: []string{"build-haks", "--verbose"},
}
if err := runBuildHAKs(ctx); err != nil {
t.Fatalf("runBuildHAKs failed: %v", err)
}
output := stdout.String()
if !strings.Contains(output, "mappings:") {
t.Fatalf("expected verbose mappings header, got %q", output)
}
if !strings.Contains(output, "AleandAnecdotes.mp3 -> mus_wg_andnc.bmu") {
t.Fatalf("expected verbose mapping output, got %q", output)
}
if !strings.Contains(output, "wrote: envi (1 assets)") {
t.Fatalf("expected verbose archive action, got %q", output)
}
}
func mkdirAll(t *testing.T, path string) { func mkdirAll(t *testing.T, path string) {
t.Helper() t.Helper()
if err := os.MkdirAll(path, 0o755); err != nil { if err := os.MkdirAll(path, 0o755); err != nil {
+41
View File
@@ -28,6 +28,8 @@ type BuildResult struct {
Manifest string Manifest string
AutogenManifestPaths []string AutogenManifestPaths []string
CreditsArtifactPaths []string CreditsArtifactPaths []string
CreditsSummary CreditsRefreshSummary
HAKSummary HAKArchiveSummary
Resources int Resources int
HAKAssets int HAKAssets int
TopPackageHAK string TopPackageHAK string
@@ -35,6 +37,35 @@ type BuildResult struct {
TopPackageFiles int TopPackageFiles int
} }
type CreditsRefreshSummary struct {
ScannedDirs []string
TrackCount int
ArtifactPaths []string
GeneratedPaths []string
InventoryPath string
ChangedFiles int
Mappings []MusicFileMapping
}
type MusicFileMapping struct {
Source string
Output string
}
type HAKArchiveSummary struct {
Total int
Reused int
Written int
Actions []HAKArchiveAction
}
type HAKArchiveAction struct {
Name string
AssetCount int
Reused bool
OutputPath string
}
type BuildManifest struct { type BuildManifest struct {
ModuleHAKs []string `json:"module_haks"` ModuleHAKs []string `json:"module_haks"`
HAKs []BuildManifestHAK `json:"haks"` HAKs []BuildManifestHAK `json:"haks"`
@@ -220,6 +251,7 @@ func planOrBuildHAKs(p *project.Project, progress ProgressFunc, writeArchives bo
result = BuildResult{HAKAssets: len(assetResources)} result = BuildResult{HAKAssets: len(assetResources)}
result.CreditsArtifactPaths = append(result.CreditsArtifactPaths, musicAssets.Artifacts...) result.CreditsArtifactPaths = append(result.CreditsArtifactPaths, musicAssets.Artifacts...)
result.CreditsSummary = musicAssets.Summary
if err := writeAutogenManifestOutputs(progress, autogenManifests); err != nil { if err := writeAutogenManifestOutputs(progress, autogenManifests); err != nil {
return BuildResult{}, err return BuildResult{}, err
} }
@@ -298,6 +330,7 @@ func planOrBuildHAKs(p *project.Project, progress ProgressFunc, writeArchives bo
progressf(progress, "Reusing unchanged generated HAKs where possible...") progressf(progress, "Reusing unchanged generated HAKs where possible...")
result.HAKSummary.Total = len(chunks)
result.HAKPaths = make([]string, 0, len(chunks)) result.HAKPaths = make([]string, 0, len(chunks))
for index, chunk := range chunks { for index, chunk := range chunks {
hakPath := p.HAKArchivePath(chunk.Name) hakPath := p.HAKArchivePath(chunk.Name)
@@ -307,8 +340,10 @@ func planOrBuildHAKs(p *project.Project, progress ProgressFunc, writeArchives bo
return BuildResult{}, err return BuildResult{}, err
} }
if reused { if reused {
result.HAKSummary.Reused++
progressf(progress, fmt.Sprintf("Reusing HAK %d/%d: %s (%d assets)", index+1, len(chunks), chunk.Name, len(chunk.Assets))) progressf(progress, fmt.Sprintf("Reusing HAK %d/%d: %s (%d assets)", index+1, len(chunks), chunk.Name, len(chunk.Assets)))
} else { } else {
result.HAKSummary.Written++
progressf(progress, fmt.Sprintf("Writing HAK %d/%d: %s (%d assets)", index+1, len(chunks), chunk.Name, len(chunk.Assets))) progressf(progress, fmt.Sprintf("Writing HAK %d/%d: %s (%d assets)", index+1, len(chunks), chunk.Name, len(chunk.Assets)))
hakOutput, err := os.Create(hakPath) hakOutput, err := os.Create(hakPath)
if err != nil { if err != nil {
@@ -323,6 +358,12 @@ func planOrBuildHAKs(p *project.Project, progress ProgressFunc, writeArchives bo
} }
} }
result.HAKSummary.Actions = append(result.HAKSummary.Actions, HAKArchiveAction{
Name: chunk.Name,
AssetCount: len(chunk.Assets),
Reused: reused,
OutputPath: hakPath,
})
result.HAKPaths = append(result.HAKPaths, hakPath) result.HAKPaths = append(result.HAKPaths, hakPath)
} }
progressf(progress, "Cleaning stale generated HAKs...") progressf(progress, "Cleaning stale generated HAKs...")
+92 -25
View File
@@ -49,6 +49,7 @@ type preparedMusicAssets struct {
SkipSourceRel map[string]struct{} SkipSourceRel map[string]struct{}
Artifacts []string Artifacts []string
TempRoots []string TempRoots []string
Summary CreditsRefreshSummary
} }
type musicMetadata struct { type musicMetadata struct {
@@ -111,11 +112,12 @@ func prepareMusicAssets(p *project.Project) (*preparedMusicAssets, error) {
SkipSourceRel: make(map[string]struct{}), SkipSourceRel: make(map[string]struct{}),
} }
if len(dirSources) == 0 { if len(dirSources) == 0 {
artifactPaths, err := writeCreditsArtifacts(p, nil) writeResult, err := writeCreditsArtifacts(p, nil)
if err != nil { if err != nil {
return nil, err return nil, err
} }
result.Artifacts = artifactPaths result.Artifacts = writeResult.Artifacts
result.Summary = writeResult.Summary
return result, nil return result, nil
} }
@@ -217,28 +219,30 @@ func prepareMusicAssets(p *project.Project) (*preparedMusicAssets, error) {
generatedByDir[dir] = entries generatedByDir[dir] = entries
} }
artifactPaths, err := writeCreditsArtifacts(p, generatedByDir) writeResult, err := writeCreditsArtifacts(p, generatedByDir)
if err != nil { if err != nil {
return nil, err return nil, err
} }
result.Artifacts = artifactPaths writeResult.Summary.ScannedDirs = append(writeResult.Summary.ScannedDirs, dirs...)
writeResult.Summary.TrackCount = len(result.Generated)
result.Artifacts = writeResult.Artifacts
result.Summary = writeResult.Summary
sort.Slice(result.Generated, func(i, j int) bool { sort.Slice(result.Generated, func(i, j int) bool {
return compareResourceKeys(result.Generated[i].Resource, result.Generated[j].Resource) < 0 return compareResourceKeys(result.Generated[i].Resource, result.Generated[j].Resource) < 0
}) })
return result, nil return result, nil
} }
func writeCreditsArtifacts(p *project.Project, generatedByDir map[string][]creditsEntry) ([]string, error) { type creditsArtifactWriteResult struct {
creditsRoot := filepath.Join(p.Root, ".cache", "credits") Artifacts []string
if err := os.RemoveAll(creditsRoot); err != nil { Summary CreditsRefreshSummary
return nil, fmt.Errorf("reset credits dir: %w", err) }
}
if err := os.MkdirAll(creditsRoot, 0o755); err != nil {
return nil, fmt.Errorf("create credits dir: %w", err)
}
func writeCreditsArtifacts(p *project.Project, generatedByDir map[string][]creditsEntry) (creditsArtifactWriteResult, error) {
creditsRoot := filepath.Join(p.Root, ".cache", "credits")
sources := make([]creditsSource, 0) sources := make([]creditsSource, 0)
artifacts := make([]string, 0) desiredFiles := make(map[string][]byte)
summary := CreditsRefreshSummary{}
if generatedByDir != nil { if generatedByDir != nil {
dirs := make([]string, 0, len(generatedByDir)) dirs := make([]string, 0, len(generatedByDir))
for dir := range generatedByDir { for dir := range generatedByDir {
@@ -257,13 +261,14 @@ func writeCreditsArtifacts(p *project.Project, generatedByDir map[string][]credi
return strings.ToLower(entries[i].OutputFile) < strings.ToLower(entries[j].OutputFile) return strings.ToLower(entries[i].OutputFile) < strings.ToLower(entries[j].OutputFile)
}) })
outputPath := filepath.Join(creditsRoot, filepath.FromSlash(dir), "CREDITS.md") outputPath := filepath.Join(creditsRoot, filepath.FromSlash(dir), "CREDITS.md")
if err := os.MkdirAll(filepath.Dir(outputPath), 0o755); err != nil { desiredFiles[outputPath] = []byte(renderCreditsMarkdown(entries))
return nil, fmt.Errorf("create generated credits dir: %w", err) summary.GeneratedPaths = append(summary.GeneratedPaths, outputPath)
for _, entry := range entries {
summary.Mappings = append(summary.Mappings, MusicFileMapping{
Source: entry.OriginalFile,
Output: entry.OutputFile,
})
} }
if err := os.WriteFile(outputPath, []byte(renderCreditsMarkdown(entries)), 0o644); err != nil {
return nil, fmt.Errorf("write generated credits %s: %w", outputPath, err)
}
artifacts = append(artifacts, outputPath)
sources = append(sources, creditsSource{ sources = append(sources, creditsSource{
Path: filepath.ToSlash(outputPath), Path: filepath.ToSlash(outputPath),
Kind: "generated_music", Kind: "generated_music",
@@ -298,21 +303,83 @@ func writeCreditsArtifacts(p *project.Project, generatedByDir map[string][]credi
return nil return nil
}) })
if err != nil { if err != nil {
return nil, err return creditsArtifactWriteResult{}, err
} }
sort.Slice(sources, func(i, j int) bool { return sources[i].Path < sources[j].Path }) sort.Slice(sources, func(i, j int) bool { return sources[i].Path < sources[j].Path })
inventoryPath := filepath.Join(creditsRoot, "credits.json") inventoryPath := filepath.Join(creditsRoot, "credits.json")
payload, err := json.MarshalIndent(creditsInventory{Sources: sources}, "", " ") payload, err := json.MarshalIndent(creditsInventory{Sources: sources}, "", " ")
if err != nil { if err != nil {
return nil, fmt.Errorf("marshal credits inventory: %w", err) return creditsArtifactWriteResult{}, fmt.Errorf("marshal credits inventory: %w", err)
} }
payload = append(payload, '\n') payload = append(payload, '\n')
if err := os.WriteFile(inventoryPath, payload, 0o644); err != nil { desiredFiles[inventoryPath] = payload
return nil, fmt.Errorf("write credits inventory: %w", err) if err := os.MkdirAll(creditsRoot, 0o755); err != nil {
return creditsArtifactWriteResult{}, fmt.Errorf("create credits dir: %w", err)
} }
artifacts = append(artifacts, inventoryPath) changedFiles, err := syncGeneratedCreditsArtifacts(creditsRoot, desiredFiles)
return artifacts, nil if err != nil {
return creditsArtifactWriteResult{}, err
}
artifacts := make([]string, 0, len(desiredFiles))
for path := range desiredFiles {
artifacts = append(artifacts, path)
}
sort.Strings(artifacts)
summary.ArtifactPaths = append(summary.ArtifactPaths, artifacts...)
summary.InventoryPath = inventoryPath
summary.ChangedFiles = changedFiles
return creditsArtifactWriteResult{
Artifacts: artifacts,
Summary: summary,
}, nil
}
func syncGeneratedCreditsArtifacts(root string, desiredFiles map[string][]byte) (int, error) {
changedFiles := 0
existingFiles := make(map[string]struct{})
if err := filepath.WalkDir(root, func(path string, d os.DirEntry, walkErr error) error {
if walkErr != nil {
if os.IsNotExist(walkErr) {
return nil
}
return walkErr
}
if d.IsDir() {
return nil
}
existingFiles[path] = struct{}{}
return nil
}); err != nil {
return 0, fmt.Errorf("scan credits dir: %w", err)
}
for path, content := range desiredFiles {
current, err := os.ReadFile(path)
if err != nil && !os.IsNotExist(err) {
return 0, fmt.Errorf("read credits artifact %s: %w", path, err)
}
if err == nil && string(current) == string(content) {
delete(existingFiles, path)
continue
}
if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil {
return 0, fmt.Errorf("create generated credits dir: %w", err)
}
if err := os.WriteFile(path, content, 0o644); err != nil {
return 0, fmt.Errorf("write credits artifact %s: %w", path, err)
}
changedFiles++
delete(existingFiles, path)
}
for path := range existingFiles {
if err := os.Remove(path); err != nil && !os.IsNotExist(err) {
return 0, fmt.Errorf("remove stale credits artifact %s: %w", path, err)
}
changedFiles++
}
return changedFiles, nil
} }
func cleanupPreparedMusicAssets(prepared *preparedMusicAssets) error { func cleanupPreparedMusicAssets(prepared *preparedMusicAssets) error {
+11
View File
@@ -2685,6 +2685,17 @@ func TestBuildHAKsConvertsMusicSourcesAndWritesCreditsArtifacts(t *testing.T) {
if _, err := os.Stat(filepath.Join(root, ".cache", "music")); !os.IsNotExist(err) { if _, err := os.Stat(filepath.Join(root, ".cache", "music")); !os.IsNotExist(err) {
t.Fatalf("expected temporary music staging to be cleaned, got err=%v", err) t.Fatalf("expected temporary music staging to be cleaned, got err=%v", err)
} }
secondResult, err := BuildHAKs(p)
if err != nil {
t.Fatalf("build haks second pass: %v", err)
}
if secondResult.CreditsSummary.ChangedFiles != 0 {
t.Fatalf("expected second credits refresh to be unchanged, got %d changed files", secondResult.CreditsSummary.ChangedFiles)
}
if secondResult.CreditsSummary.TrackCount != 1 {
t.Fatalf("expected one tracked music conversion, got %d", secondResult.CreditsSummary.TrackCount)
}
} }
func TestUniqueMusicNameHandlesShortStemCollisions(t *testing.T) { func TestUniqueMusicNameHandlesShortStemCollisions(t *testing.T) {