[ENHANCEMENT] Add structured logging with correlation IDs for agent workflows

[harikapadia999]

Problem

Current logging makes it difficult to:

• Track requests across multiple agents
• Debug complex multi-agent workflows
• Correlate logs from different components
• Analyze agent performance

Proposed Solution

Implement structured logging with correlation IDs:

# flo_ai/logging/structured.py
import logging
import uuid
from contextvars import ContextVar
from typing import Optional, Dict, Any

# Context variable for correlation ID
correlation_id: ContextVar[Optional[str]] = ContextVar('correlation_id', default=None)

class StructuredLogger:
    def __init__(self, name: str):
        self.logger = logging.getLogger(name)
        self.name = name
    
    def _add_context(self, extra: Dict[str, Any]) -> Dict[str, Any]:
        """Add correlation ID and other context to log"""
        context = {
            'correlation_id': correlation_id.get(),
            'logger_name': self.name,
            **extra
        }
        return {k: v for k, v in context.items() if v is not None}
    
    def info(self, message: str, **kwargs):
        self.logger.info(message, extra=self._add_context(kwargs))
    
    def warning(self, message: str, **kwargs):
        self.logger.warning(message, extra=self._add_context(kwargs))
    
    def error(self, message: str, **kwargs):
        self.logger.error(message, extra=self._add_context(kwargs))

# Usage in agent
class Agent:
    def __init__(self, name: str):
        self.logger = StructuredLogger(f"agent.{name}")
    
    def run(self, input: str) -> str:
        # Generate correlation ID for this request
        corr_id = str(uuid.uuid4())
        correlation_id.set(corr_id)
        
        self.logger.info(
            "Agent started",
            agent_name=self.name,
            input_length=len(input)
        )
        
        # ... agent logic ...
        
        self.logger.info(
            "Agent completed",
            agent_name=self.name,
            duration_ms=duration
        )

Log Output Format

JSON Format (for production):

{
  "timestamp": "2025-12-16T06:45:00Z",
  "level": "INFO",
  "message": "Agent started",
  "correlation_id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
  "logger_name": "agent.customer_support",
  "agent_name": "customer_support",
  "input_length": 150
}

Human-Readable Format (for development):

2025-12-16 06:45:00 [INFO] [a1b2c3d4] agent.customer_support: Agent started (input_length=150)

Benefits

1. ✅ Easy request tracing across components
2. ✅ Better debugging of multi-agent workflows
3. ✅ Performance analysis per correlation ID
4. ✅ Integration with log aggregation tools (ELK, Datadog)
5. ✅ Compliance and audit trails

Configuration

logging:
  format: "json"  # or "human"
  level: "INFO"
  correlation_id:
    enabled: true
    header_name: "X-Correlation-ID"  # For HTTP requests
  fields:
    - agent_name
    - workflow_name
    - user_id

Implementation Checklist

• Create structured logger class
• Add correlation ID context management
• Integrate with all agents and workflows
• Add JSON formatter for production
• Add human-readable formatter for development
• Update documentation
• Add examples for log analysis

Related Tools

• ELK Stack (Elasticsearch, Logstash, Kibana)
• Datadog
• Splunk
• CloudWatch Logs Insights