Skip to content

Consolidate and modernize ATS logging: unify log files, timestamps, and naming #12944

@bryancall

Description

@bryancall

Problem

ATS currently has four separate log files with inconsistent timestamp formats and overlapping purposes, while comparable proxy servers (Nginx, Envoy, HAProxy) use only one or two.

Current log files

Log File Written By Timestamp Format Purpose
squid.log Logging subsystem (LogObject) epoch.ms (e.g. 1741210395.123) Per-transaction access log
error.log Log::error() via LogObject::va_log() %Y%m%d.%Hh%Mm%Ss (e.g. 20260305.21h33m15s) Connection failures, DNS errors
diags.log Diags::print_va() [DiagTimestamp] with thread name + level Debug messages, warnings, notes, errors
traffic.out Same Diags system (stdout/stderr) Same as diags.log Duplicate of diags.log to stdout

Comparison with other proxies

Proxy Log Files
Nginx 2 — access.log + error.log
Envoy 1 — access log (errors to stderr)
HAProxy 2 — access log + admin log
ATS 4squid.log + error.log + diags.log + traffic.out

Issues

  1. Three different timestamp formats across log files make cross-log correlation difficult
  2. diags.log and traffic.out are the same Diags system writing to two destinations — redundant
  3. error.log and diags.log both contain error-level messages but use completely different code paths (Log::error() vs Diags Error level) with different formats
  4. No SM-aware error macrosSMDbg/TxnDbg prefix debug messages with [SM_ID] but there's no SMError/TxnError equivalent for Log::error(), Log::warning(), or Log::note(). This makes it impossible to correlate error.log entries back to a specific transaction without manually adding the SM ID (as seen in PR Add AuTest for connect_attempts rr_retries and max_retries #12932)
  5. "Squid log" is a legacy name — the default format hasn't been actual Squid format in a long time, but field names (client_req_timestamp_squid, cqtq) and config still reference "squid"

Proposed Changes

1. Add SM-aware error/warning/note macros

#define SMError(fmt, ...) Log::error("[%" PRId64 "] " fmt, sm_id, ##__VA_ARGS__)
#define SMWarning(fmt, ...) Log::warning("[%" PRId64 "] " fmt, sm_id, ##__VA_ARGS__)
#define SMNote(fmt, ...) Log::note("[%" PRId64 "] " fmt, sm_id, ##__VA_ARGS__)
// And TxnError/TxnWarning/TxnNote for HttpTransact context

2. Standardize timestamp format

Pick one format and use it everywhere, or at minimum make them correlatable (e.g. all ISO 8601).

3. Rename "squid log" to "access log"

Update config names, field names, documentation, and code references.

4. Consolidate log files

Long-term goal: merge error.log, diags.log, and traffic.out into a single diagnostic/error log, similar to how Nginx has one error.log for everything that isn't per-transaction access logging.

Notes

  • Items 1-3 can be done incrementally without breaking changes
  • Item 4 (log consolidation) is a larger architectural change that should be designed carefully
  • The SM-aware macros (item 1) are the most immediately useful and could unblock PR Add AuTest for connect_attempts rr_retries and max_retries #12932's approach of adding sm_id to error log messages in a standardized way

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions