Log Refactor
This commit is contained in:
@@ -0,0 +1,149 @@
|
||||
You are improving CLI/build logs for a NWN asset toolchain.
|
||||
|
||||
Goal: Make the logs cleaner, prettier, less repetitive, and easier to understand, without hiding important diagnostics.
|
||||
|
||||
Current issues:
|
||||
|
||||
- The same credits refresh output appears twice.
|
||||
- Long file-by-file mappings overwhelm the main log.
|
||||
- Some messages are noisy or too implementation-focused.
|
||||
- The final build summary is useful but visually cramped.
|
||||
- Paths are too long for normal human reading unless needed for debugging.
|
||||
- Status prefixes are inconsistent: [credits], [build-haks], run-nwn-tool.sh, [|].
|
||||
|
||||
Desired behavior:
|
||||
|
||||
1. Use clear phase headers.
|
||||
2. Collapse repetitive file listings by default.
|
||||
3. Show counts and summaries in normal mode.
|
||||
4. Keep detailed mappings available behind a verbose/debug flag.
|
||||
5. Avoid printing the same credits scan twice unless something changed.
|
||||
6. Use consistent prefixes and indentation.
|
||||
7. Prefer relative paths when possible.
|
||||
8. Group related output together.
|
||||
9. End with a compact, readable summary.
|
||||
10. Preserve warnings, errors, and actionable diagnostics prominently.
|
||||
|
||||
Suggested output style:
|
||||
|
||||
Build HAKs ────────── • Refreshing credits cache
|
||||
|
||||
- Scanned: envi/music/westgate
|
||||
- Tracks: 38
|
||||
- Output: .cache/credits/envi/music/westgate/credits.json
|
||||
- Aggregate cache updated
|
||||
|
||||
• Validating project • Collecting asset resources • Writing autogen manifests
|
||||
|
||||
- sow-parts-manifest.json
|
||||
- sow-head-vfx-manifest.json
|
||||
|
||||
• Planning HAK chunks • Resolving module HAK order • Reusing unchanged HAKs
|
||||
|
||||
- 12 / 12 reused
|
||||
- 65,320 assets total
|
||||
|
||||
• Cleaning stale generated HAKs • Writing HAK manifest
|
||||
|
||||
Summary ─────── Project: Shadows Over Westgate HAK archives: 12 HAK assets: 65,320 Manifest: assets/build/haks.json Credits artifacts: 2 Autogen manifests: 2 Status: complete
|
||||
|
||||
Rules:
|
||||
|
||||
- In normal mode, do not print every `mp3 -> bmu` mapping.
|
||||
- Instead print something like: `Mapped 38 music files. Use --verbose to list mappings.`
|
||||
- In verbose mode, print mappings under an indented section.
|
||||
- If the credits cache is refreshed before and after the build for normalization, make the second refresh terse: `Restored normalized credits cache: unchanged.`
|
||||
- If files changed, say how many changed and where.
|
||||
- Do not remove any warnings or errors.
|
||||
- Do not make failure logs terse; failures should include enough context to debug.
|
||||
- Keep output deterministic so CI diffs remain readable.
|
||||
- Use plain ASCII unless the toolchain already supports Unicode reliably.
|
||||
- Do not introduce color unless color can be disabled with NO_COLOR or --no-color.
|
||||
|
||||
Implementation guidance:
|
||||
|
||||
- Add logging helpers for phases, steps, summaries, verbose details, warnings, and errors.
|
||||
- Add a verbosity level, for example: quiet, normal, verbose, debug.
|
||||
- Route repeated detailed output through verbose/debug logging.
|
||||
- Track counts while scanning instead of printing every scanned item.
|
||||
- Detect unchanged cache writes and report them as unchanged rather than rewriting noisy output.
|
||||
- Normalize absolute paths to project-relative paths in normal output.
|
||||
- Keep machine-readable artifacts unchanged.
|
||||
|
||||
Concrete implementation plan:
|
||||
|
||||
1. CLI presentation layer
|
||||
|
||||
- Keep the pipeline responsible for state and artifact generation.
|
||||
- Move `build-haks` presentation rules into `internal/app/app.go`.
|
||||
- Introduce `--quiet`, `--verbose`, and `--debug` for `build-haks`.
|
||||
- In normal mode, print grouped sections and a compact summary.
|
||||
- In verbose mode, include detailed mapping and per-archive action lines.
|
||||
- In debug mode, preserve raw pipeline progress messages for diagnosis.
|
||||
|
||||
2. Spinner behavior
|
||||
|
||||
- Reuse the existing spinner infrastructure instead of adding a second animation system.
|
||||
- Make the spinner phase-aware so it shows the current high-level step, for example:
|
||||
- `Build HAKs: refreshing credits cache`
|
||||
- `Build HAKs: collecting asset resources`
|
||||
- `Build HAKs: planning HAK chunks`
|
||||
- Only animate on interactive TTY stderr.
|
||||
- Disable spinner animation automatically in CI and non-interactive output.
|
||||
- Always keep stable human-readable logs on stdout.
|
||||
|
||||
3. Pipeline data surfaced to the CLI
|
||||
|
||||
- Extend `pipeline.BuildResult` with log-summary metadata instead of forcing the CLI to parse raw strings.
|
||||
- Return credits refresh summary data:
|
||||
- scanned music directories
|
||||
- converted track count
|
||||
- generated credits artifact paths
|
||||
- per-file source/output mappings
|
||||
- changed-versus-unchanged artifact count
|
||||
- Return HAK archive summary data:
|
||||
- total archive count
|
||||
- reused archive count
|
||||
- written archive count
|
||||
- per-archive actions for verbose mode
|
||||
|
||||
4. Credits artifact handling
|
||||
|
||||
- Stop blindly deleting and rewriting `.cache/credits` on every run.
|
||||
- Render the desired credits outputs in memory first.
|
||||
- Compare desired artifacts against the current on-disk artifacts.
|
||||
- Rewrite only changed files.
|
||||
- Remove stale artifacts that are no longer desired.
|
||||
- Report `unchanged` when the second pass produces identical output.
|
||||
|
||||
5. Output rules
|
||||
|
||||
- Normal mode:
|
||||
- show phase blocks
|
||||
- show counts
|
||||
- show relative paths
|
||||
- hide file-by-file mappings
|
||||
- Verbose mode:
|
||||
- show music mappings under an indented `mappings:` section
|
||||
- show per-archive `reused` or `wrote` details
|
||||
- Quiet mode:
|
||||
- keep only the final summary
|
||||
- Debug mode:
|
||||
- allow raw internal progress lines in addition to the formatted summary
|
||||
|
||||
6. Tests
|
||||
|
||||
- Add app-layer tests for:
|
||||
- compact normal-mode output
|
||||
- verbose mapping output
|
||||
- Add pipeline regression coverage proving:
|
||||
- music conversion still writes the expected credits artifacts
|
||||
- a second identical credits refresh reports zero changed files
|
||||
|
||||
Acceptance checks:
|
||||
|
||||
- `build-haks` emits the new grouped summary format.
|
||||
- `build-haks --verbose` includes per-track mappings and per-archive details.
|
||||
- The spinner reflects phase changes without polluting stdout logs.
|
||||
- Re-running the same build keeps credits refresh output deterministic and reports `unchanged` when applicable.
|
||||
- Existing machine-readable artifacts remain unchanged in schema and location.
|
||||
+274
-28
@@ -37,26 +37,50 @@ type spinner struct {
|
||||
text string
|
||||
msg []string
|
||||
painted bool
|
||||
writer io.Writer
|
||||
enabled bool
|
||||
}
|
||||
|
||||
var spin = &spinner{
|
||||
msg: []string{"-", "\\", "|", "/"},
|
||||
func newSpinner() *spinner {
|
||||
return &spinner{
|
||||
msg: []string{"-", "\\", "|", "/"},
|
||||
}
|
||||
}
|
||||
|
||||
var spin = newSpinner()
|
||||
|
||||
func (s *spinner) configure(writer io.Writer, enabled bool) {
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
s.writer = writer
|
||||
s.enabled = enabled
|
||||
}
|
||||
|
||||
func (s *spinner) start(text string) {
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
if !s.enabled || s.writer == nil {
|
||||
s.on = false
|
||||
s.painted = false
|
||||
s.text = text
|
||||
return
|
||||
}
|
||||
s.on = true
|
||||
s.text = text
|
||||
go s.run(text)
|
||||
go s.run()
|
||||
}
|
||||
|
||||
func (s *spinner) stop() {
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
if !s.enabled || s.writer == nil {
|
||||
s.on = false
|
||||
s.painted = false
|
||||
return
|
||||
}
|
||||
s.on = false
|
||||
if s.painted {
|
||||
fmt.Fprint(os.Stderr, "\r\033[K\n")
|
||||
fmt.Fprint(s.writer, "\r\033[K")
|
||||
s.painted = false
|
||||
}
|
||||
}
|
||||
@@ -64,23 +88,28 @@ func (s *spinner) stop() {
|
||||
func (s *spinner) linebreak() {
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
if s.on && s.painted {
|
||||
fmt.Fprint(os.Stderr, "\r\033[K\n")
|
||||
if s.enabled && s.on && s.painted {
|
||||
fmt.Fprint(s.writer, "\r\033[K")
|
||||
s.painted = false
|
||||
}
|
||||
}
|
||||
|
||||
func (s *spinner) run(text string) {
|
||||
func (s *spinner) update(text string) {
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
s.text = text
|
||||
}
|
||||
|
||||
func (s *spinner) run() {
|
||||
i := 0
|
||||
for {
|
||||
s.mu.Lock()
|
||||
if !s.on {
|
||||
s.mu.Unlock()
|
||||
fmt.Fprintf(os.Stderr, "\r")
|
||||
return
|
||||
}
|
||||
msg := s.msg[i%len(s.msg)]
|
||||
fmt.Fprintf(os.Stderr, "\r[%s] %s", msg, text)
|
||||
fmt.Fprintf(s.writer, "\r[%s] %s", msg, s.text)
|
||||
s.painted = true
|
||||
s.mu.Unlock()
|
||||
i++
|
||||
@@ -88,6 +117,18 @@ func (s *spinner) run(text string) {
|
||||
}
|
||||
}
|
||||
|
||||
func isInteractiveTTY(w io.Writer) bool {
|
||||
file, ok := w.(*os.File)
|
||||
if !ok {
|
||||
return false
|
||||
}
|
||||
info, err := file.Stat()
|
||||
if err != nil {
|
||||
return false
|
||||
}
|
||||
return (info.Mode() & os.ModeCharDevice) != 0
|
||||
}
|
||||
|
||||
var commands = []command{
|
||||
{
|
||||
name: "build",
|
||||
@@ -184,6 +225,7 @@ func Run(args []string) (int, error) {
|
||||
default:
|
||||
for _, cmd := range commands {
|
||||
if cmd.name == args[0] {
|
||||
spin.configure(ctx.stderr, isInteractiveTTY(ctx.stderr) && strings.TrimSpace(os.Getenv("CI")) == "")
|
||||
spin.start("running " + cmd.name)
|
||||
if err := cmd.run(ctx); err != nil {
|
||||
spin.stop()
|
||||
@@ -263,6 +305,212 @@ type buildHAKOptions struct {
|
||||
filteredArchives []string
|
||||
sourceManifest string
|
||||
planOnly bool
|
||||
logLevel logLevel
|
||||
}
|
||||
|
||||
type logLevel int
|
||||
|
||||
const (
|
||||
logLevelNormal logLevel = iota
|
||||
logLevelQuiet
|
||||
logLevelVerbose
|
||||
logLevelDebug
|
||||
)
|
||||
|
||||
type buildHAKConsole struct {
|
||||
stdout io.Writer
|
||||
projectRoot string
|
||||
projectName string
|
||||
planOnly bool
|
||||
level logLevel
|
||||
spinnerEnabled bool
|
||||
}
|
||||
|
||||
func newBuildHAKConsole(ctx context, p *project.Project, opts buildHAKOptions) *buildHAKConsole {
|
||||
return &buildHAKConsole{
|
||||
stdout: ctx.stdout,
|
||||
projectRoot: p.Root,
|
||||
projectName: p.Config.Module.Name,
|
||||
planOnly: opts.planOnly,
|
||||
level: opts.logLevel,
|
||||
spinnerEnabled: isInteractiveTTY(ctx.stderr) && strings.TrimSpace(os.Getenv("CI")) == "" && opts.logLevel == logLevelNormal,
|
||||
}
|
||||
}
|
||||
|
||||
func (c *buildHAKConsole) progress(message string) {
|
||||
if phase := c.phaseLabel(message); phase != "" {
|
||||
spin.update("Build HAKs: " + phase)
|
||||
}
|
||||
if c.level != logLevelDebug {
|
||||
return
|
||||
}
|
||||
spin.linebreak()
|
||||
fmt.Fprintf(c.stdout, "[debug] %s\n", message)
|
||||
}
|
||||
|
||||
func (c *buildHAKConsole) phaseLabel(message string) string {
|
||||
switch {
|
||||
case strings.HasPrefix(message, "Validating project"):
|
||||
return "validating project"
|
||||
case strings.HasPrefix(message, "Collecting asset resources"):
|
||||
return "collecting asset resources"
|
||||
case strings.HasPrefix(message, "Planning HAK chunks"):
|
||||
return "planning HAK chunks"
|
||||
case strings.HasPrefix(message, "Resolving module HAK order"):
|
||||
return "resolving module HAK order"
|
||||
case strings.HasPrefix(message, "Reusing unchanged generated HAKs"):
|
||||
return "reusing unchanged HAKs"
|
||||
case strings.HasPrefix(message, "Reusing HAK "), strings.HasPrefix(message, "Writing HAK "):
|
||||
return "writing HAK archives"
|
||||
case strings.HasPrefix(message, "Cleaning stale generated HAKs"):
|
||||
return "cleaning stale generated HAKs"
|
||||
case strings.HasPrefix(message, "Cleaning previous generated HAKs"):
|
||||
return "cleaning previous generated HAKs"
|
||||
case strings.HasPrefix(message, "Writing HAK manifest"):
|
||||
return "writing HAK manifest"
|
||||
case strings.HasPrefix(message, "Writing autogen manifest "):
|
||||
return "writing autogen manifests"
|
||||
default:
|
||||
return ""
|
||||
}
|
||||
}
|
||||
|
||||
func (c *buildHAKConsole) emitResult(result pipeline.BuildResult) {
|
||||
spin.linebreak()
|
||||
fmt.Fprintln(c.stdout, "Build HAKs ----------")
|
||||
if c.level != logLevelQuiet {
|
||||
c.emitCreditsSummary(result)
|
||||
c.emitCoreSteps(result)
|
||||
}
|
||||
c.emitFinalSummary(result)
|
||||
}
|
||||
|
||||
func (c *buildHAKConsole) emitCreditsSummary(result pipeline.BuildResult) {
|
||||
summary := result.CreditsSummary
|
||||
if len(summary.ArtifactPaths) == 0 {
|
||||
return
|
||||
}
|
||||
fmt.Fprintln(c.stdout, "Refreshing credits cache")
|
||||
if len(summary.ScannedDirs) > 0 {
|
||||
fmt.Fprintf(c.stdout, " scanned: %s\n", strings.Join(summary.ScannedDirs, ", "))
|
||||
}
|
||||
fmt.Fprintf(c.stdout, " tracks: %d\n", summary.TrackCount)
|
||||
if len(summary.GeneratedPaths) > 0 {
|
||||
for _, path := range summary.GeneratedPaths {
|
||||
fmt.Fprintf(c.stdout, " output: %s\n", c.relPath(path))
|
||||
}
|
||||
}
|
||||
switch {
|
||||
case summary.ChangedFiles == 0:
|
||||
fmt.Fprintln(c.stdout, " status: unchanged")
|
||||
default:
|
||||
fmt.Fprintf(c.stdout, " updated: %d artifact(s)\n", summary.ChangedFiles)
|
||||
}
|
||||
if summary.TrackCount > 0 {
|
||||
if c.level >= logLevelVerbose {
|
||||
fmt.Fprintln(c.stdout, " mappings:")
|
||||
for _, mapping := range summary.Mappings {
|
||||
fmt.Fprintf(c.stdout, " %s -> %s\n", mapping.Source, mapping.Output)
|
||||
}
|
||||
} else {
|
||||
fmt.Fprintf(c.stdout, " mapped: %d music file(s); use --verbose to list mappings\n", summary.TrackCount)
|
||||
}
|
||||
}
|
||||
fmt.Fprintln(c.stdout)
|
||||
}
|
||||
|
||||
func (c *buildHAKConsole) emitCoreSteps(result pipeline.BuildResult) {
|
||||
fmt.Fprintln(c.stdout, "Validating project")
|
||||
fmt.Fprintln(c.stdout, "Collecting asset resources")
|
||||
if len(result.AutogenManifestPaths) > 0 {
|
||||
fmt.Fprintln(c.stdout, "Writing autogen manifests")
|
||||
for _, path := range result.AutogenManifestPaths {
|
||||
fmt.Fprintf(c.stdout, " %s\n", filepath.Base(path))
|
||||
}
|
||||
fmt.Fprintln(c.stdout)
|
||||
}
|
||||
fmt.Fprintln(c.stdout, "Planning HAK chunks")
|
||||
fmt.Fprintln(c.stdout, "Resolving module HAK order")
|
||||
if !c.planOnly {
|
||||
fmt.Fprintln(c.stdout, "Reusing unchanged HAKs")
|
||||
fmt.Fprintf(c.stdout, " reused: %d / %d\n", result.HAKSummary.Reused, result.HAKSummary.Total)
|
||||
fmt.Fprintf(c.stdout, " written: %d / %d\n", result.HAKSummary.Written, result.HAKSummary.Total)
|
||||
fmt.Fprintf(c.stdout, " assets: %s\n", formatCount(result.HAKAssets))
|
||||
if c.level >= logLevelVerbose {
|
||||
for _, action := range result.HAKSummary.Actions {
|
||||
verb := "wrote"
|
||||
if action.Reused {
|
||||
verb = "reused"
|
||||
}
|
||||
fmt.Fprintf(c.stdout, " %s: %s (%s assets)\n", verb, action.Name, formatCount(action.AssetCount))
|
||||
}
|
||||
}
|
||||
fmt.Fprintln(c.stdout)
|
||||
fmt.Fprintln(c.stdout, "Cleaning stale generated HAKs")
|
||||
} else {
|
||||
fmt.Fprintf(c.stdout, "Planned HAK archives: %d\n", result.HAKSummary.Total)
|
||||
}
|
||||
fmt.Fprintln(c.stdout, "Writing HAK manifest")
|
||||
fmt.Fprintln(c.stdout)
|
||||
}
|
||||
|
||||
func (c *buildHAKConsole) emitFinalSummary(result pipeline.BuildResult) {
|
||||
fmt.Fprintln(c.stdout, "Summary ----------")
|
||||
fmt.Fprintf(c.stdout, "project: %s\n", c.projectName)
|
||||
if c.planOnly {
|
||||
fmt.Fprintf(c.stdout, "planned archives: %d\n", result.HAKSummary.Total)
|
||||
} else {
|
||||
fmt.Fprintf(c.stdout, "hak archives: %d\n", len(result.HAKPaths))
|
||||
fmt.Fprintf(c.stdout, "hak assets: %s\n", formatCount(result.HAKAssets))
|
||||
}
|
||||
if result.Manifest != "" {
|
||||
fmt.Fprintf(c.stdout, "manifest: %s\n", c.relPath(result.Manifest))
|
||||
}
|
||||
if len(result.CreditsArtifactPaths) > 0 {
|
||||
fmt.Fprintf(c.stdout, "credits artifacts: %d\n", len(result.CreditsArtifactPaths))
|
||||
}
|
||||
if len(result.AutogenManifestPaths) > 0 {
|
||||
fmt.Fprintf(c.stdout, "autogen manifests: %d\n", len(result.AutogenManifestPaths))
|
||||
}
|
||||
status := "complete"
|
||||
if c.planOnly {
|
||||
status = "planned"
|
||||
}
|
||||
fmt.Fprintf(c.stdout, "status: %s\n", status)
|
||||
}
|
||||
|
||||
func (c *buildHAKConsole) relPath(path string) string {
|
||||
if path == "" {
|
||||
return path
|
||||
}
|
||||
rel, err := filepath.Rel(c.projectRoot, path)
|
||||
if err != nil {
|
||||
return filepath.ToSlash(path)
|
||||
}
|
||||
return filepath.ToSlash(rel)
|
||||
}
|
||||
|
||||
func formatCount(value int) string {
|
||||
raw := strconv.Itoa(value)
|
||||
if value < 1000 && value > -1000 {
|
||||
return raw
|
||||
}
|
||||
sign := ""
|
||||
if strings.HasPrefix(raw, "-") {
|
||||
sign = "-"
|
||||
raw = strings.TrimPrefix(raw, "-")
|
||||
}
|
||||
parts := make([]string, 0, (len(raw)+2)/3)
|
||||
for len(raw) > 3 {
|
||||
parts = append(parts, raw[len(raw)-3:])
|
||||
raw = raw[:len(raw)-3]
|
||||
}
|
||||
parts = append(parts, raw)
|
||||
slices := make([]string, 0, len(parts))
|
||||
for index := len(parts) - 1; index >= 0; index-- {
|
||||
slices = append(slices, parts[index])
|
||||
}
|
||||
return sign + strings.Join(slices, ",")
|
||||
}
|
||||
|
||||
func runBuildHAKs(ctx context) error {
|
||||
@@ -280,16 +528,15 @@ func runBuildHAKs(ctx context) error {
|
||||
return err
|
||||
}
|
||||
|
||||
progress := func(message string) {
|
||||
spin.linebreak()
|
||||
fmt.Fprintf(ctx.stdout, "[build-haks] %s\n", message)
|
||||
}
|
||||
console := newBuildHAKConsole(ctx, p, opts)
|
||||
spin.configure(ctx.stderr, console.spinnerEnabled)
|
||||
spin.update("Build HAKs: starting")
|
||||
var result pipeline.BuildResult
|
||||
if opts.planOnly {
|
||||
result, err = pipeline.PlanHAKsWithProgress(p, progress)
|
||||
result, err = pipeline.PlanHAKsWithProgress(p, console.progress)
|
||||
} else {
|
||||
result, err = pipeline.BuildHAKsWithOptions(p, pipeline.BuildHAKOptions{
|
||||
Progress: progress,
|
||||
Progress: console.progress,
|
||||
ArchiveNames: opts.filteredArchives,
|
||||
SourceManifestPath: opts.sourceManifest,
|
||||
})
|
||||
@@ -298,18 +545,7 @@ func runBuildHAKs(ctx context) error {
|
||||
return err
|
||||
}
|
||||
|
||||
fmt.Fprintf(ctx.stdout, "project: %s\n", p.Config.Module.Name)
|
||||
fmt.Fprintf(ctx.stdout, "hak archives: %d\n", len(result.HAKPaths))
|
||||
fmt.Fprintf(ctx.stdout, "hak assets: %d\n", result.HAKAssets)
|
||||
if result.Manifest != "" {
|
||||
fmt.Fprintf(ctx.stdout, "hak manifest: %s\n", result.Manifest)
|
||||
}
|
||||
if len(result.CreditsArtifactPaths) > 0 {
|
||||
fmt.Fprintf(ctx.stdout, "credits artifacts: %d\n", len(result.CreditsArtifactPaths))
|
||||
}
|
||||
if len(result.AutogenManifestPaths) > 0 {
|
||||
fmt.Fprintf(ctx.stdout, "autogen manifests: %d\n", len(result.AutogenManifestPaths))
|
||||
}
|
||||
console.emitResult(result)
|
||||
return nil
|
||||
}
|
||||
|
||||
@@ -323,7 +559,7 @@ func parseBuildHAKArgs(args []string) (buildHAKOptions, error) {
|
||||
arg := args[index]
|
||||
switch arg {
|
||||
case "-h", "--help":
|
||||
return opts, errors.New("usage: build-haks [--hak <hak-name> ...] [--archive <archive-name> ...] [--source-manifest <path>] [--plan-only]")
|
||||
return opts, errors.New("usage: build-haks [--hak <hak-name> ...] [--archive <archive-name> ...] [--source-manifest <path>] [--plan-only] [--quiet|--verbose|--debug]")
|
||||
case "--hak":
|
||||
index++
|
||||
if index >= len(args) {
|
||||
@@ -338,6 +574,12 @@ func parseBuildHAKArgs(args []string) (buildHAKOptions, error) {
|
||||
opts.filteredArchives = append(opts.filteredArchives, args[index])
|
||||
case "--plan-only":
|
||||
opts.planOnly = true
|
||||
case "--quiet":
|
||||
opts.logLevel = logLevelQuiet
|
||||
case "--verbose":
|
||||
opts.logLevel = logLevelVerbose
|
||||
case "--debug":
|
||||
opts.logLevel = logLevelDebug
|
||||
case "--source-manifest":
|
||||
index++
|
||||
if index >= len(args) {
|
||||
@@ -357,6 +599,10 @@ func parseBuildHAKArgs(args []string) (buildHAKOptions, error) {
|
||||
opts.sourceManifest = value
|
||||
continue
|
||||
}
|
||||
if arg == "--quiet=true" {
|
||||
opts.logLevel = logLevelQuiet
|
||||
continue
|
||||
}
|
||||
return opts, fmt.Errorf("unknown build-haks argument %q", arg)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -59,6 +59,145 @@ func TestRunBuildTopPackageUsesCachedCompiledOutputs(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestRunBuildHAKsEmitsCompactSummary(t *testing.T) {
|
||||
root := t.TempDir()
|
||||
mkdirAll(t, filepath.Join(root, "assets", "envi", "music", "westgate"))
|
||||
mkdirAll(t, filepath.Join(root, "build"))
|
||||
mkdirAll(t, filepath.Join(root, "src"))
|
||||
|
||||
writeFile(t, filepath.Join(root, "nwn-tool.json"), `{
|
||||
"module": {
|
||||
"name": "Test Module",
|
||||
"resref": "testmod"
|
||||
},
|
||||
"paths": {
|
||||
"source": "src",
|
||||
"assets": "assets",
|
||||
"build": "build"
|
||||
},
|
||||
"haks": [
|
||||
{
|
||||
"name": "envi",
|
||||
"priority": 1,
|
||||
"max_bytes": 1048576,
|
||||
"split": false,
|
||||
"include": ["envi/**"]
|
||||
}
|
||||
]
|
||||
}`+"\n")
|
||||
writeFile(t, filepath.Join(root, "assets", "envi", "music", "westgate", "AleandAnecdotes.mp3"), "source-mp3")
|
||||
writeFile(t, filepath.Join(root, "assets", "envi", "music", "westgate", "CREDITS.md"), "# NWN Music Pack Credits\n\n| Artist | Title | Output File | Original File | Album | Date | License / Copyright | Notes |\n|---|---|---|---|---|---|---|---|\n| Darren Curtis | Ale and Anecdotes | `mus_wg_andnc.bmu` | `AleandAnecdotes.mp3` | Darren's Commercially Free Music | | Darren Curtis Music | |\n")
|
||||
|
||||
ffprobePath := filepath.Join(root, "ffprobe")
|
||||
writeFile(t, ffprobePath, "#!/bin/sh\nprintf '%s\\n' '{\"format\":{\"tags\":{\"artist\":\"Unknown\",\"title\":\"Broken Metadata\"}}}'\n")
|
||||
if err := os.Chmod(ffprobePath, 0o755); err != nil {
|
||||
t.Fatalf("chmod ffprobe: %v", err)
|
||||
}
|
||||
|
||||
ffmpegPath := filepath.Join(root, "ffmpeg")
|
||||
writeFile(t, ffmpegPath, "#!/bin/sh\nfor last; do :; done\nprintf 'bmu-data' > \"$last\"\n")
|
||||
if err := os.Chmod(ffmpegPath, 0o755); err != nil {
|
||||
t.Fatalf("chmod ffmpeg: %v", err)
|
||||
}
|
||||
|
||||
t.Setenv("SOW_FFMPEG", ffmpegPath)
|
||||
t.Setenv("SOW_FFPROBE", ffprobePath)
|
||||
|
||||
var stdout bytes.Buffer
|
||||
ctx := context{
|
||||
stdout: &stdout,
|
||||
stderr: &bytes.Buffer{},
|
||||
cwd: root,
|
||||
args: []string{"build-haks"},
|
||||
}
|
||||
|
||||
if err := runBuildHAKs(ctx); err != nil {
|
||||
t.Fatalf("runBuildHAKs failed: %v", err)
|
||||
}
|
||||
|
||||
output := stdout.String()
|
||||
if !strings.Contains(output, "Build HAKs ----------") {
|
||||
t.Fatalf("expected build header, got %q", output)
|
||||
}
|
||||
if !strings.Contains(output, "mapped: 1 music file(s); use --verbose to list mappings") {
|
||||
t.Fatalf("expected compact mapping summary, got %q", output)
|
||||
}
|
||||
if strings.Contains(output, "AleandAnecdotes.mp3 -> mus_wg_andnc.bmu") {
|
||||
t.Fatalf("did not expect verbose mapping in normal mode, got %q", output)
|
||||
}
|
||||
if !strings.Contains(output, "manifest: build/haks.json") {
|
||||
t.Fatalf("expected relative manifest path, got %q", output)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRunBuildHAKsVerboseListsMappings(t *testing.T) {
|
||||
root := t.TempDir()
|
||||
mkdirAll(t, filepath.Join(root, "assets", "envi", "music", "westgate"))
|
||||
mkdirAll(t, filepath.Join(root, "build"))
|
||||
mkdirAll(t, filepath.Join(root, "src"))
|
||||
|
||||
writeFile(t, filepath.Join(root, "nwn-tool.json"), `{
|
||||
"module": {
|
||||
"name": "Test Module",
|
||||
"resref": "testmod"
|
||||
},
|
||||
"paths": {
|
||||
"source": "src",
|
||||
"assets": "assets",
|
||||
"build": "build"
|
||||
},
|
||||
"haks": [
|
||||
{
|
||||
"name": "envi",
|
||||
"priority": 1,
|
||||
"max_bytes": 1048576,
|
||||
"split": false,
|
||||
"include": ["envi/**"]
|
||||
}
|
||||
]
|
||||
}`+"\n")
|
||||
writeFile(t, filepath.Join(root, "assets", "envi", "music", "westgate", "AleandAnecdotes.mp3"), "source-mp3")
|
||||
writeFile(t, filepath.Join(root, "assets", "envi", "music", "westgate", "CREDITS.md"), "# NWN Music Pack Credits\n\n| Artist | Title | Output File | Original File | Album | Date | License / Copyright | Notes |\n|---|---|---|---|---|---|---|---|\n| Darren Curtis | Ale and Anecdotes | `mus_wg_andnc.bmu` | `AleandAnecdotes.mp3` | Darren's Commercially Free Music | | Darren Curtis Music | |\n")
|
||||
|
||||
ffprobePath := filepath.Join(root, "ffprobe")
|
||||
writeFile(t, ffprobePath, "#!/bin/sh\nprintf '%s\\n' '{\"format\":{\"tags\":{\"artist\":\"Unknown\",\"title\":\"Broken Metadata\"}}}'\n")
|
||||
if err := os.Chmod(ffprobePath, 0o755); err != nil {
|
||||
t.Fatalf("chmod ffprobe: %v", err)
|
||||
}
|
||||
|
||||
ffmpegPath := filepath.Join(root, "ffmpeg")
|
||||
writeFile(t, ffmpegPath, "#!/bin/sh\nfor last; do :; done\nprintf 'bmu-data' > \"$last\"\n")
|
||||
if err := os.Chmod(ffmpegPath, 0o755); err != nil {
|
||||
t.Fatalf("chmod ffmpeg: %v", err)
|
||||
}
|
||||
|
||||
t.Setenv("SOW_FFMPEG", ffmpegPath)
|
||||
t.Setenv("SOW_FFPROBE", ffprobePath)
|
||||
|
||||
var stdout bytes.Buffer
|
||||
ctx := context{
|
||||
stdout: &stdout,
|
||||
stderr: &bytes.Buffer{},
|
||||
cwd: root,
|
||||
args: []string{"build-haks", "--verbose"},
|
||||
}
|
||||
|
||||
if err := runBuildHAKs(ctx); err != nil {
|
||||
t.Fatalf("runBuildHAKs failed: %v", err)
|
||||
}
|
||||
|
||||
output := stdout.String()
|
||||
if !strings.Contains(output, "mappings:") {
|
||||
t.Fatalf("expected verbose mappings header, got %q", output)
|
||||
}
|
||||
if !strings.Contains(output, "AleandAnecdotes.mp3 -> mus_wg_andnc.bmu") {
|
||||
t.Fatalf("expected verbose mapping output, got %q", output)
|
||||
}
|
||||
if !strings.Contains(output, "wrote: envi (1 assets)") {
|
||||
t.Fatalf("expected verbose archive action, got %q", output)
|
||||
}
|
||||
}
|
||||
|
||||
func mkdirAll(t *testing.T, path string) {
|
||||
t.Helper()
|
||||
if err := os.MkdirAll(path, 0o755); err != nil {
|
||||
|
||||
@@ -28,6 +28,8 @@ type BuildResult struct {
|
||||
Manifest string
|
||||
AutogenManifestPaths []string
|
||||
CreditsArtifactPaths []string
|
||||
CreditsSummary CreditsRefreshSummary
|
||||
HAKSummary HAKArchiveSummary
|
||||
Resources int
|
||||
HAKAssets int
|
||||
TopPackageHAK string
|
||||
@@ -35,6 +37,35 @@ type BuildResult struct {
|
||||
TopPackageFiles int
|
||||
}
|
||||
|
||||
type CreditsRefreshSummary struct {
|
||||
ScannedDirs []string
|
||||
TrackCount int
|
||||
ArtifactPaths []string
|
||||
GeneratedPaths []string
|
||||
InventoryPath string
|
||||
ChangedFiles int
|
||||
Mappings []MusicFileMapping
|
||||
}
|
||||
|
||||
type MusicFileMapping struct {
|
||||
Source string
|
||||
Output string
|
||||
}
|
||||
|
||||
type HAKArchiveSummary struct {
|
||||
Total int
|
||||
Reused int
|
||||
Written int
|
||||
Actions []HAKArchiveAction
|
||||
}
|
||||
|
||||
type HAKArchiveAction struct {
|
||||
Name string
|
||||
AssetCount int
|
||||
Reused bool
|
||||
OutputPath string
|
||||
}
|
||||
|
||||
type BuildManifest struct {
|
||||
ModuleHAKs []string `json:"module_haks"`
|
||||
HAKs []BuildManifestHAK `json:"haks"`
|
||||
@@ -220,6 +251,7 @@ func planOrBuildHAKs(p *project.Project, progress ProgressFunc, writeArchives bo
|
||||
|
||||
result = BuildResult{HAKAssets: len(assetResources)}
|
||||
result.CreditsArtifactPaths = append(result.CreditsArtifactPaths, musicAssets.Artifacts...)
|
||||
result.CreditsSummary = musicAssets.Summary
|
||||
if err := writeAutogenManifestOutputs(progress, autogenManifests); err != nil {
|
||||
return BuildResult{}, err
|
||||
}
|
||||
@@ -298,6 +330,7 @@ func planOrBuildHAKs(p *project.Project, progress ProgressFunc, writeArchives bo
|
||||
|
||||
progressf(progress, "Reusing unchanged generated HAKs where possible...")
|
||||
|
||||
result.HAKSummary.Total = len(chunks)
|
||||
result.HAKPaths = make([]string, 0, len(chunks))
|
||||
for index, chunk := range chunks {
|
||||
hakPath := p.HAKArchivePath(chunk.Name)
|
||||
@@ -307,8 +340,10 @@ func planOrBuildHAKs(p *project.Project, progress ProgressFunc, writeArchives bo
|
||||
return BuildResult{}, err
|
||||
}
|
||||
if reused {
|
||||
result.HAKSummary.Reused++
|
||||
progressf(progress, fmt.Sprintf("Reusing HAK %d/%d: %s (%d assets)", index+1, len(chunks), chunk.Name, len(chunk.Assets)))
|
||||
} else {
|
||||
result.HAKSummary.Written++
|
||||
progressf(progress, fmt.Sprintf("Writing HAK %d/%d: %s (%d assets)", index+1, len(chunks), chunk.Name, len(chunk.Assets)))
|
||||
hakOutput, err := os.Create(hakPath)
|
||||
if err != nil {
|
||||
@@ -323,6 +358,12 @@ func planOrBuildHAKs(p *project.Project, progress ProgressFunc, writeArchives bo
|
||||
}
|
||||
}
|
||||
|
||||
result.HAKSummary.Actions = append(result.HAKSummary.Actions, HAKArchiveAction{
|
||||
Name: chunk.Name,
|
||||
AssetCount: len(chunk.Assets),
|
||||
Reused: reused,
|
||||
OutputPath: hakPath,
|
||||
})
|
||||
result.HAKPaths = append(result.HAKPaths, hakPath)
|
||||
}
|
||||
progressf(progress, "Cleaning stale generated HAKs...")
|
||||
|
||||
+92
-25
@@ -49,6 +49,7 @@ type preparedMusicAssets struct {
|
||||
SkipSourceRel map[string]struct{}
|
||||
Artifacts []string
|
||||
TempRoots []string
|
||||
Summary CreditsRefreshSummary
|
||||
}
|
||||
|
||||
type musicMetadata struct {
|
||||
@@ -111,11 +112,12 @@ func prepareMusicAssets(p *project.Project) (*preparedMusicAssets, error) {
|
||||
SkipSourceRel: make(map[string]struct{}),
|
||||
}
|
||||
if len(dirSources) == 0 {
|
||||
artifactPaths, err := writeCreditsArtifacts(p, nil)
|
||||
writeResult, err := writeCreditsArtifacts(p, nil)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
result.Artifacts = artifactPaths
|
||||
result.Artifacts = writeResult.Artifacts
|
||||
result.Summary = writeResult.Summary
|
||||
return result, nil
|
||||
}
|
||||
|
||||
@@ -217,28 +219,30 @@ func prepareMusicAssets(p *project.Project) (*preparedMusicAssets, error) {
|
||||
generatedByDir[dir] = entries
|
||||
}
|
||||
|
||||
artifactPaths, err := writeCreditsArtifacts(p, generatedByDir)
|
||||
writeResult, err := writeCreditsArtifacts(p, generatedByDir)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
result.Artifacts = artifactPaths
|
||||
writeResult.Summary.ScannedDirs = append(writeResult.Summary.ScannedDirs, dirs...)
|
||||
writeResult.Summary.TrackCount = len(result.Generated)
|
||||
result.Artifacts = writeResult.Artifacts
|
||||
result.Summary = writeResult.Summary
|
||||
sort.Slice(result.Generated, func(i, j int) bool {
|
||||
return compareResourceKeys(result.Generated[i].Resource, result.Generated[j].Resource) < 0
|
||||
})
|
||||
return result, nil
|
||||
}
|
||||
|
||||
func writeCreditsArtifacts(p *project.Project, generatedByDir map[string][]creditsEntry) ([]string, error) {
|
||||
creditsRoot := filepath.Join(p.Root, ".cache", "credits")
|
||||
if err := os.RemoveAll(creditsRoot); err != nil {
|
||||
return nil, fmt.Errorf("reset credits dir: %w", err)
|
||||
}
|
||||
if err := os.MkdirAll(creditsRoot, 0o755); err != nil {
|
||||
return nil, fmt.Errorf("create credits dir: %w", err)
|
||||
}
|
||||
type creditsArtifactWriteResult struct {
|
||||
Artifacts []string
|
||||
Summary CreditsRefreshSummary
|
||||
}
|
||||
|
||||
func writeCreditsArtifacts(p *project.Project, generatedByDir map[string][]creditsEntry) (creditsArtifactWriteResult, error) {
|
||||
creditsRoot := filepath.Join(p.Root, ".cache", "credits")
|
||||
sources := make([]creditsSource, 0)
|
||||
artifacts := make([]string, 0)
|
||||
desiredFiles := make(map[string][]byte)
|
||||
summary := CreditsRefreshSummary{}
|
||||
if generatedByDir != nil {
|
||||
dirs := make([]string, 0, len(generatedByDir))
|
||||
for dir := range generatedByDir {
|
||||
@@ -257,13 +261,14 @@ func writeCreditsArtifacts(p *project.Project, generatedByDir map[string][]credi
|
||||
return strings.ToLower(entries[i].OutputFile) < strings.ToLower(entries[j].OutputFile)
|
||||
})
|
||||
outputPath := filepath.Join(creditsRoot, filepath.FromSlash(dir), "CREDITS.md")
|
||||
if err := os.MkdirAll(filepath.Dir(outputPath), 0o755); err != nil {
|
||||
return nil, fmt.Errorf("create generated credits dir: %w", err)
|
||||
desiredFiles[outputPath] = []byte(renderCreditsMarkdown(entries))
|
||||
summary.GeneratedPaths = append(summary.GeneratedPaths, outputPath)
|
||||
for _, entry := range entries {
|
||||
summary.Mappings = append(summary.Mappings, MusicFileMapping{
|
||||
Source: entry.OriginalFile,
|
||||
Output: entry.OutputFile,
|
||||
})
|
||||
}
|
||||
if err := os.WriteFile(outputPath, []byte(renderCreditsMarkdown(entries)), 0o644); err != nil {
|
||||
return nil, fmt.Errorf("write generated credits %s: %w", outputPath, err)
|
||||
}
|
||||
artifacts = append(artifacts, outputPath)
|
||||
sources = append(sources, creditsSource{
|
||||
Path: filepath.ToSlash(outputPath),
|
||||
Kind: "generated_music",
|
||||
@@ -298,21 +303,83 @@ func writeCreditsArtifacts(p *project.Project, generatedByDir map[string][]credi
|
||||
return nil
|
||||
})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
return creditsArtifactWriteResult{}, err
|
||||
}
|
||||
|
||||
sort.Slice(sources, func(i, j int) bool { return sources[i].Path < sources[j].Path })
|
||||
inventoryPath := filepath.Join(creditsRoot, "credits.json")
|
||||
payload, err := json.MarshalIndent(creditsInventory{Sources: sources}, "", " ")
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("marshal credits inventory: %w", err)
|
||||
return creditsArtifactWriteResult{}, fmt.Errorf("marshal credits inventory: %w", err)
|
||||
}
|
||||
payload = append(payload, '\n')
|
||||
if err := os.WriteFile(inventoryPath, payload, 0o644); err != nil {
|
||||
return nil, fmt.Errorf("write credits inventory: %w", err)
|
||||
desiredFiles[inventoryPath] = payload
|
||||
if err := os.MkdirAll(creditsRoot, 0o755); err != nil {
|
||||
return creditsArtifactWriteResult{}, fmt.Errorf("create credits dir: %w", err)
|
||||
}
|
||||
artifacts = append(artifacts, inventoryPath)
|
||||
return artifacts, nil
|
||||
changedFiles, err := syncGeneratedCreditsArtifacts(creditsRoot, desiredFiles)
|
||||
if err != nil {
|
||||
return creditsArtifactWriteResult{}, err
|
||||
}
|
||||
artifacts := make([]string, 0, len(desiredFiles))
|
||||
for path := range desiredFiles {
|
||||
artifacts = append(artifacts, path)
|
||||
}
|
||||
sort.Strings(artifacts)
|
||||
summary.ArtifactPaths = append(summary.ArtifactPaths, artifacts...)
|
||||
summary.InventoryPath = inventoryPath
|
||||
summary.ChangedFiles = changedFiles
|
||||
return creditsArtifactWriteResult{
|
||||
Artifacts: artifacts,
|
||||
Summary: summary,
|
||||
}, nil
|
||||
}
|
||||
|
||||
func syncGeneratedCreditsArtifacts(root string, desiredFiles map[string][]byte) (int, error) {
|
||||
changedFiles := 0
|
||||
existingFiles := make(map[string]struct{})
|
||||
if err := filepath.WalkDir(root, func(path string, d os.DirEntry, walkErr error) error {
|
||||
if walkErr != nil {
|
||||
if os.IsNotExist(walkErr) {
|
||||
return nil
|
||||
}
|
||||
return walkErr
|
||||
}
|
||||
if d.IsDir() {
|
||||
return nil
|
||||
}
|
||||
existingFiles[path] = struct{}{}
|
||||
return nil
|
||||
}); err != nil {
|
||||
return 0, fmt.Errorf("scan credits dir: %w", err)
|
||||
}
|
||||
|
||||
for path, content := range desiredFiles {
|
||||
current, err := os.ReadFile(path)
|
||||
if err != nil && !os.IsNotExist(err) {
|
||||
return 0, fmt.Errorf("read credits artifact %s: %w", path, err)
|
||||
}
|
||||
if err == nil && string(current) == string(content) {
|
||||
delete(existingFiles, path)
|
||||
continue
|
||||
}
|
||||
if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil {
|
||||
return 0, fmt.Errorf("create generated credits dir: %w", err)
|
||||
}
|
||||
if err := os.WriteFile(path, content, 0o644); err != nil {
|
||||
return 0, fmt.Errorf("write credits artifact %s: %w", path, err)
|
||||
}
|
||||
changedFiles++
|
||||
delete(existingFiles, path)
|
||||
}
|
||||
|
||||
for path := range existingFiles {
|
||||
if err := os.Remove(path); err != nil && !os.IsNotExist(err) {
|
||||
return 0, fmt.Errorf("remove stale credits artifact %s: %w", path, err)
|
||||
}
|
||||
changedFiles++
|
||||
}
|
||||
return changedFiles, nil
|
||||
}
|
||||
|
||||
func cleanupPreparedMusicAssets(prepared *preparedMusicAssets) error {
|
||||
|
||||
@@ -2685,6 +2685,17 @@ func TestBuildHAKsConvertsMusicSourcesAndWritesCreditsArtifacts(t *testing.T) {
|
||||
if _, err := os.Stat(filepath.Join(root, ".cache", "music")); !os.IsNotExist(err) {
|
||||
t.Fatalf("expected temporary music staging to be cleaned, got err=%v", err)
|
||||
}
|
||||
|
||||
secondResult, err := BuildHAKs(p)
|
||||
if err != nil {
|
||||
t.Fatalf("build haks second pass: %v", err)
|
||||
}
|
||||
if secondResult.CreditsSummary.ChangedFiles != 0 {
|
||||
t.Fatalf("expected second credits refresh to be unchanged, got %d changed files", secondResult.CreditsSummary.ChangedFiles)
|
||||
}
|
||||
if secondResult.CreditsSummary.TrackCount != 1 {
|
||||
t.Fatalf("expected one tracked music conversion, got %d", secondResult.CreditsSummary.TrackCount)
|
||||
}
|
||||
}
|
||||
|
||||
func TestUniqueMusicNameHandlesShortStemCollisions(t *testing.T) {
|
||||
|
||||
Reference in New Issue
Block a user