Skip to content

[Feature]: W1: Core pkg/log Standard Library (log/slog) Engine & zap Removal #1843

Open

Description

@skevetter

Parent Epic: #1841

Overview

Replace go.uber.org/zap inside pkg/log with Go's standard library log/slog while maintaining 100% backwards compatibility for the 1,288 existing log.*f/log.* call sites across 340 files (and the 12 external Go provider repositories that import github.com/devsy-org/devsy/pkg/log), preserving the CLI-to-Desktop JSON wire format (ts, level, msg), removing go.uber.org/zap and github.com/go-logr/zapr from go.mod, and archiving the legacy devsy-org/log repository.


Subtasks & Engineering Specifications

  • Task 1.1: Core slog.Handler Architecture, Custom Levels & Wire Compatibility (size/m)

    • Target Files:
      • pkg/log/levels.go
      • pkg/log/logger.go, pkg/log/logger_test.go
    • Implementation Details:
      1. Define slog.Level mappings for Devsy's 6 levels:
        • SlogLevelTrace = slog.Level(-8) ("trace")
        • SlogLevelDebug = slog.LevelDebug (-4, "debug")
        • SlogLevelInfo = slog.LevelInfo (0, "info")
        • SlogLevelWarn = slog.LevelWarn (4, "warn")
        • SlogLevelError = slog.LevelError (8, "error")
        • SlogLevelFatal = slog.Level(12) ("fatal")
      2. Store an atomic.Pointer[slog.Logger] in pkg/log alongside atomic.Int32 for fast lock-free level checks (TraceEnabled(), DebugEnabled(), InfoEnabled()).
      3. Configure slog.NewJSONHandler with ReplaceAttr to preserve the exact CLI/Desktop wire contract:
        • slog.TimeKey ("time") -> "ts" formatted as ISO8601 (2006-01-02T15:04:05.000Z0700).
        • slog.LevelKey ("level") -> lowercase "trace" | "debug" | "info" | "warn" | "error" | "fatal".
        • slog.MessageKey -> "msg".
      4. Preserve redactingWriter, consoleWriter (SuppressConsoleOutput()), and extraSinks (AddSink(w)).
  • Task 1.2: Port logfmt and Console (Color/Plain) Encoders to slog.Handler (size/m)

    • Target Files:
      • pkg/log/logfmt.go
      • pkg/log/console_handler.go (new)
      • pkg/log/logger_test.go
    • Implementation Details:
      1. Rewrite pkg/log/logfmt.go from zapcore.Encoder to a stdlib logfmtHandler implementing slog.Handler (Enabled, Handle, WithAttrs, WithGroup) with identical quoting/escaping semantics.
      2. Implement consoleHandler implementing slog.Handler for --log-format=text (NO_COLOR / TTY aware): ISO8601 timestamp, ANSI-colored or uppercase level (TRACE, DEBUG, INFO, WARN, ERROR, FATAL), message, and key=value attributes.
  • Task 1.3: Context-Aware Structured Logging API (WithContext, FromContext, slog Helpers) (size/s)

    • Target Files:
      • pkg/log/context.go (new)
      • pkg/log/attrs.go (new)
      • pkg/log/logger.go
    • Implementation Details:
      1. Expose structured and context-aware functions in pkg/log:
        • WithAttrs(attrs ...slog.Attr) *slog.Logger
        • WithContext(ctx context.Context, attrs ...slog.Attr) context.Context
        • FromContext(ctx context.Context) *slog.Logger
        • TraceContext, DebugContext, InfoContext, WarnContext, ErrorContext(ctx context.Context, msg string, attrs ...slog.Attr)
      2. Implement a wrapping contextHandler that automatically extracts context-attached slog.Attrs plus active status.OperationID(ctx) and status.ParentOperationID(ctx).
      3. Define canonical attribute constructors in pkg/log/attrs.go: AttrWorkspaceID, AttrMachineID, AttrProvider, AttrComponent, AttrOperation, AttrPhase, AttrDurationMs, AttrErrorCode.
  • Task 1.4: Migrate clierr, logr (klog Bridge for api/apiserver/agentapi), and custom Driver Off zap (size/s)

    • Target Files:
      • pkg/clierr/errors.go
      • pkg/log/logr.go
      • pkg/driver/custom/custom.go
    • Implementation Details:
      1. Replace MarshalLogObject(enc zapcore.ObjectEncoder) on *CLIError in pkg/clierr/errors.go with slog.LogValuer (LogValue() slog.Value returning slog.GroupValue(...)).
      2. Replace zapr.NewLogger in pkg/log/logr.go with logr.FromSlogHandler(Slog().Handler()) (supported natively by github.com/go-logr/logr v1.4.4, which also bridges k8s.io/klog/v2 logs from devsy-org/api, devsy-org/apiserver, devsy-org/agentapi, and devsy-org/admin-apis into slog).
      3. Replace log.Underlying().Core().Enabled(zapcore.DebugLevel) in pkg/driver/custom/custom.go with log.DebugEnabled().
  • Task 1.5: Migrate pkg/log/testing.go Observer, Remove go.uber.org/zap from go.mod & Archive devsy-org/log (size/m)

    • Target Files:
      • pkg/log/testing.go
      • 18 *test.go files calling log.InitTestObserved
      • go.mod, go.sum
      • External repo: devsy-org/log
    • Implementation Details:
      1. Replace zaptest/observer in pkg/log/testing.go with a native slog.Handler test recorder (ObservedLogs, LoggedEntry with Level, Message, AttrsMap(), FilterMessageSnippet, FilterLevelExact).
      2. Change InitTestObserved(t, level) signature to accept log.Level instead of zapcore.Level across all 18 test files.
      3. Run mise run cli:tidy to remove go.uber.org/zap and github.com/go-logr/zapr from go.mod.
      4. Archive the unused legacy devsy-org/log repository (logrus + zap) on GitHub with a deprecation note pointing to github.com/devsy-org/devsy/pkg/log.

Verification & Acceptance Criteria

  • mise run cli:test and mise run cli:lint:ci pass.
  • go.mod contains zero references to go.uber.org/zap or github.com/go-logr/zapr.
  • devsy-org/log is archived on GitHub.
  • JSON, logfmt, and text outputs match golden wire-format expectations (ts, level, msg, plus structured attributes) with secret redaction enforced across all sinks.

Activity

  1. skevetter commented on Nov 20, 2026

    @skevetter
    ContributorAuthor

    The shared logging module in PR #1851 passed hosted CI. The Core slog engine and caller migration are in PR #1850. The separate redactor migration is prepared and needs the foundation's merged Go module version. Greptile is still at its usage limit. The engine cutover and legacy logging repository archival remain open.

  2. skevetter commented on Nov 20, 2026

    @skevetter
    ContributorAuthor

    Foundation PR #1851 now includes the independent review follow-up, signed and verified at fe0aa820. The fix bounds retained buffers, masks unsupported structured Authorization schemes, and preserves diagnostic delimiters while handling truncated URLs. Both supported Go toolchains passed 93 race test results, with vet, lint, and pre-commit passing. Hosted CI and required reviews must cover the new head. Greptile remains at its usage limit.

  3. skevetter commented on Nov 20, 2026

    @skevetter
    ContributorAuthor

    The hosted Shared Logging job passed on foundation PR #1851 at fe0aa820. The full CI run is still underway. CodeRabbit auto-paused on this head, so the prior completed review at 9b1b1d4d does not cover the hardening commit. Greptile remains at its usage limit.

  4. skevetter commented on Nov 20, 2026

    @skevetter
    ContributorAuthor

    CodeRabbit reviewed the previous shared logging head and found that URL redaction could drop the next word after an emitted backslash. The regression reproduced the loss, and signed head 7bb30803 fixes it. Independent review cleared the change. All 94 race tests passed on each of Go 1.23.12 and 1.26.9, along with vet, lint, and repository hooks. The current head has a new hosted CI run. CodeRabbit paused on this head, and required Greptile review is still unavailable.

  5. skevetter commented on Nov 20, 2026

    @skevetter
    ContributorAuthor

    CodeRabbit finished its review of the shared logging foundation on 7bb30803, with no actionable findings. All six review threads are resolved. Hosted Shared Logging also passed. Aggregate CI and Greptile remain open.

  6. skevetter commented on Nov 20, 2026

    @skevetter
    ContributorAuthor

    Foundation PR #1851 on 7bb30803 passed all six hosted workflows. The CI Success job passed, and CodeRabbit completed on the same head with no actionable findings. All six review threads are resolved. Greptile remains required, and the foundation will need fresh checks after the lint prerequisite merges.

  7. skevetter commented on Nov 20, 2026

    @skevetter
    ContributorAuthor

    Foundation PR is now at signed head 41f2e766 after a normal merge from main. The logging module files match the previously reviewed tree. Local race tests, vet, lint, and hooks passed. Hosted CI and CodeRabbit will qualify this exact head. The owner waived Greptile for this foundation and the lint prerequisite only.

  8. skevetter commented on Nov 20, 2026

    @skevetter
    ContributorAuthor

    Foundation PR passed all hosted workflows and CI Success at signed 41f2e766 after syncing with main. Linux, macOS, and Windows unit jobs passed. One CodeRabbit review is running. CodeFactor flagged 26 ranges already on main. Greptile is waived for this PR.

  9. skevetter commented on Nov 20, 2026

    @skevetter
    ContributorAuthor

    Foundation PR passed hosted CI at signed 41f2e766. The current CodeRabbit review opened nine threads and raised one additional URL-scheme finding. Confirmed privacy gaps are being addressed. The foundation remains open and is awaiting review corrections.

  10. skevetter commented on Nov 20, 2026

    @skevetter
    ContributorAuthor

    Foundation PR is now at signed cf43b60f with the CodeRabbit corrections. Nine regression tests fail on the previous source and pass on the fix. Both Go versions passed 106 race checks each. Fresh hosted CI and CodeRabbit remain open.

  11. skevetter commented on Nov 20, 2026

    @skevetter
    ContributorAuthor

    Foundation PR passed all six hosted workflows and CI Success on signed cf43b60f. Native Linux, macOS, and Windows unit jobs passed. All nine previous review threads are resolved. The final authorized CodeRabbit review is underway.

  12. skevetter commented on Nov 20, 2026

    @skevetter
    ContributorAuthor

    The foundation passed all hosted workflows on signed cf43b60f. The final authorized review confirmed the previous threads are resolved and found two remaining privacy gaps: serialized Authorization arrays and escaped whitespace after the scheme. The head remains blocked. Another correction and exact-head review need the lead’s approval.

  13. skevetter commented on Nov 20, 2026

    @skevetter
    ContributorAuthor

    Signed tokenizer correction a7d6c927 is on the foundation PR. It masks serialized credential fields through their value boundaries, covers every split in the new fixtures, and closes the structured-compound gap. Independent adversarial review, local Go 1.23 and 1.26 race tests, lint and hooks passed. Hosted CI and the next full review are pending.

  14. 34 remaining items

  15. skevetter commented on Nov 21, 2026

    @skevetter
    ContributorAuthor
  16. skevetter commented on Nov 21, 2026

    @skevetter
    ContributorAuthor

    CodeRabbit reviewed the signed Core head. The six previous threads are resolved and four additional outputs need the same folded masking helper. Status output, persisted task state, stored machine diagnostics, and the workspace journal each failed a focused fixture before the change. All five affected packages now pass locally with race detection. Lint, hooks, and independent review are underway. Core adoption

  17. skevetter commented on Nov 21, 2026

    @skevetter
    ContributorAuthor

    Core redactor adoption at signed b713d7a applies the common folded masking helper to all ten reviewed output paths. Four additional fixtures failed first and now pass. The affected race suites, lint, hooks, and independent review passed, and all prior threads are resolved. Fresh hosted CI is pending. CodeRabbit estimated shared reviewer capacity around 11:32 UTC. The dependent branches are in local verification.

  18. skevetter commented on Nov 21, 2026

    @skevetter
    ContributorAuthor

    Core redactor adoption at b713d7a1 uses the common folded masking helper for all ten reviewed output paths. The logging engine at 489e6ad0 and every dependent branch carry the signed fix. All affected race tests, lint, hooks and independent review passed, with zero previous open threads. Core CI is active. CodeRabbit estimated shared reviewer capacity around 11:32 UTC.

  19. skevetter commented on Nov 21, 2026

    @skevetter
    ContributorAuthor

    Core redactor adoption at signed b713d7a applies the same folded masking helper across ten reviewed paths, with all eight dependent PRs updated. Local targeted races, lint, hooks, and independent review passed. Hosted CI passed builds on Linux, macOS, and Windows and passed Linux unit tests. The remaining hosted tests and the fresh CodeRabbit review are active.

  20. skevetter commented on Nov 21, 2026

    @skevetter
    ContributorAuthor

    Core redactor adoption at signed b713d7a passed local qualification and updated all eight dependent PRs. CodeRabbit reviewed this exact head and opened four findings in SSH report excerpts, agent status, Compose errors, and structured output. The reported callers still use ordinary masking and a synthetic fixture confirmed the different folded behavior. The two correction rounds are used. Hosted CI passed Linux and Windows unit tests. macOS unit tests and end-to-end checks remain open.

  21. skevetter commented on Nov 21, 2026

    @skevetter
    ContributorAuthor

    Core redactor adoption remains at signed b713d7a. Hosted CI passed native unit tests on Linux, macOS, and Windows for that head. The new local correction covers the four review findings plus audited command diagnostics. Targeted race fixtures, full logger, subprocess, and workspace log race suites, configured lint, and independent review passed. The combined change spans 139 paths against the previously observed 100-file CodeRabbit limit. Publication awaits approval for two focused prerequisites or confirmation of full review coverage.

  22. skevetter commented on Nov 21, 2026

    @skevetter
    ContributorAuthor

    Command diagnostics prerequisite is open against main at signed, verified 40ab524. Its 12 files mask displayed command errors while preserving Docker and CLI classifications. Race tests, the Windows cross-build, scoped hooks, lint, and independent review passed. Hosted CI and review remain pending. The tool and E2E prerequisites are undergoing local qualification. Core adoption retains its published head.

  23. skevetter commented on Nov 21, 2026

    @skevetter
    ContributorAuthor

    Three signed prerequisites are open. The 16-file command change masks complete JSON and bounded stderr, with both review threads resolved. The 29-file E2E and 24-file tool changes depend on the command helper. The tool branch incorporates the signed command update and passes its Docker Stop regression. Local qualification and independent review passed. Hosted checks remain pending. CodeRabbit skips the two temporary bases until they advance to main. A private 85-file Core replay passed five race suites, including the writer guard. Command · E2E · Tools · Core

  24. skevetter commented on Nov 21, 2026

    @skevetter
    ContributorAuthor

    The signed command prerequisite at 3cc34264 passed CI Success. CodeRabbit accepted the single review for this head. Both prior review threads are resolved. The E2E and tool prerequisites still use its branch as their base. A private Core replay leaves 85 files and passed the writer guard and focused race tests. Command · CI · Review · E2E · Tools

  25. skevetter commented on Nov 21, 2026

    @skevetter
    ContributorAuthor

    The signed command prerequisite at 3cc34264 passed CI. Its review raised three findings reproduced locally with synthetic fixtures. The remaining work covers quoted records with trailing text, UTF-8 boundaries in bounded diagnostic output, and the Docker tail fixture under a multiline registered value. The E2E and tool prerequisites await the command merge. Command · Review · E2E · Tools

  26. skevetter commented on Nov 21, 2026

    @skevetter
    ContributorAuthor

    The signed command correction at 83cf9f17 masks leading quoted diagnostic records, keeps bounded UTF-8 display valid, and preserves the Docker fixture tail under a multiline environment. All three fixtures failed before the fix; focused race tests, lint, scoped checks, Windows cross-compilation and independent review passed. All three prior threads are resolved. Fresh CI and review remain pending. The E2E and tool prerequisites await the command merge. Command · CI · E2E · Tools

  27. skevetter commented on Nov 21, 2026

    @skevetter
    ContributorAuthor

    The signed command correction at 83cf9f17 covers all three regression fixtures, and their review threads are resolved. Local races, lint, scoped hooks, cross-compilation and independent review passed. Hosted Pre-commit, Lint and Shared Logging passed on this head. The remaining platform jobs and the single accepted CodeRabbit review are active. The E2E and tool prerequisites await the command merge. Command · CI · Review

  28. skevetter commented on Nov 21, 2026

    @skevetter
    ContributorAuthor

    The command diagnostics prerequisite remains at signed head 83cf9f17. The single CodeRabbit review completed with two open findings covering carriage-return record boundaries and Windows taskkill output truncation. Exact-head CI is still running. The E2E and tool prerequisites await the command merge.

  29. skevetter commented on Nov 21, 2026

    @skevetter
    ContributorAuthor

    The command diagnostics prerequisite passed CI Success on signed head 83cf9f17. The completed CodeRabbit review has two open findings on carriage-return record boundaries and Windows taskkill output truncation. The E2E and tool prerequisites still await the command merge.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions