Skip to content

Conversation

@bastiandoetsch
Copy link
Contributor

Summary

This PR adds a new API log writer that integrates with the ldx-sync API for remote logging capabilities.

Changes

  • Updated ldx-sync spec to latest version (2024-10-15)

    • Regenerated Go client code with new LogMessage, LogSource, and CreateLogMessageJSONBody types
    • Updated mocks to match new API structure
  • Added APILogWriter (pkg/logging/apiLogWriter.go)

    • Buffers log messages in memory with configurable max buffer size (default: 10MB)
    • Triggers sending logs when a message at or above configured level is received (default: Error level)
    • Automatically batches large payloads to respect 1MB API limit
    • Thread-safe implementation with proper locking
    • Supports flushing remaining logs on demand
    • Includes error callback for handling send failures
  • Added comprehensive test coverage (pkg/logging/apiLogWriter_test.go)

    • 88.3% test coverage
    • Tests for buffering, trimming, trigger levels, thread safety, batching, and error handling
    • Performance benchmarks covering all critical paths
  • Code quality improvements

    • Refactored to eliminate duplication between conversion methods
    • Added helper functions for cleaner code organization

Performance Benchmarks

BenchmarkAPILogWriter_WriteLevel-12                   13575536    76.78 ns/op     263 B/op    1 allocs/op
BenchmarkAPILogWriter_WriteLevel_WithTrigger-12        6577623   205.1 ns/op     589 B/op    7 allocs/op
BenchmarkAPILogWriter_Batching-12                         7926   148869 ns/op  218348 B/op 4012 allocs/op
BenchmarkAPILogWriter_ToAPIFormat-12                    376257     3051 ns/op    7664 B/op  208 allocs/op
BenchmarkAPILogWriter_ConvertLevel-12              1000000000     1.057 ns/op       0 B/op    0 allocs/op
BenchmarkAPILogWriter_ConcurrentWrites-12              7302974   161.6 ns/op     271 B/op    1 allocs/op
BenchmarkAPILogWriter_BufferTrimming-12               19489166    61.98 ns/op     163 B/op    1 allocs/op
BenchmarkLogEntry_ToJSON-12                            8613387   139.6 ns/op     112 B/op    4 allocs/op

Key Performance Characteristics:

  • 🚀 Low latency: ~77 ns per write operation
  • 💾 Memory efficient: Minimal allocations (1 alloc per write)
  • 🔄 Concurrent: ~162 ns/op for parallel writes
  • 📦 Scalable batching: ~149 µs for 1000 entries
  • Fast conversion: Sub-nanosecond level conversion, ~3 µs for 100 entry batch formatting

Testing

✅ All tests pass
✅ Code passes go vet
✅ Code passes gofmt

go test ./pkg/logging/... -cover
# ok  	github.com/snyk/go-application-framework/pkg/logging	3.037s	coverage: 88.3%

Related

  • Issue: IDE-1327

- Update ldx-sync spec to latest version (2024-10-15)
- Add APILogWriter for buffered remote logging with configurable trigger levels
- Implement LogSource integration for tracking integration metadata
- Add comprehensive test coverage (88.3%)
- Refactor conversion logic to eliminate duplication between marshalEntry and ToAPIFormat
Add performance benchmarks covering:
- Basic WriteLevel operations (~77 ns/op)
- WriteLevel with API triggers (~205 ns/op)
- Batching logic for large payloads (~149 µs/op for 1000 entries)
- Format conversion (~3 µs/op for 100 entries)
- Level conversion (~1 ns/op)
- Concurrent/parallel writes (~162 ns/op)
- Buffer trimming under memory pressure (~62 ns/op)
- JSON serialization (~140 ns/op)

Results show excellent performance characteristics:
- Low latency for buffering operations
- Minimal memory allocations
- Good concurrent write performance
- Efficient batching for large volumes
@snyk-io
Copy link

snyk-io bot commented Oct 20, 2025

Snyk checks have passed. No issues have been found so far.

Status Scanner Critical High Medium Low Total (0)
Licenses 0 0 0 0 0 issues
Code Security 0 0 0 0 0 issues
Open Source Security 0 0 0 0 0 issues

💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse.

@snyk-io
Copy link

snyk-io bot commented Oct 20, 2025

Snyk checks have passed. No issues have been found so far.

Status Scanner Critical High Medium Low Total (0)
Licenses 0 0 0 0 0 issues
Code Security 0 0 0 0 0 issues
Open Source Security 0 0 0 0 0 issues

💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants