Add observability architecture: tiered retention, log-sink contract, profile opt-in

[RonaldHensbergen]

Summary

Establishes the design foundation for #174 (long-term observability epic: tiered Docker log retention and structured runtime events).

This PR delivers the architectural pieces called out in #174's acceptance criteria — architecture guidance, structured event fields, module-agnostic profile opt-in, and redaction/query guidance — while explicitly respecting #174's ownership boundary against #219 (named DB connections) and #167 (event persistence).

Changes

• docs/observability.md — ADR-style guidance: two-tier retention design (short-retention raw logs vs. longer-retention structured events), the log-sink contract, profile-level opt-in pattern, redaction guidance, non-mandatory recommended pipelines, and example operator incident queries.
• schemas/structured-event.schema.json — provider-neutral JSON Schema for the structured event record: required service/profile/environment/timestamp/severity, optional correlationId/workloadId/resource/recordCount/retryCount/message.
• shared/contracts/log-sink.yaml — new shared contract (kind: log-sink) a centralized log collector module can provide, following the existing sql-database/cache-service contract pattern so nothing hardcodes a specific logging vendor.
• schemas/profile.schema.json + cli/validator.py — adds an optional spec.observability.logShipping block. A profile can enable log shipping and declare retention tiers without naming a module; sink.contractRef is only needed to pin a specific log-sink provider, reusing the existing <module-id>.<contract-name> resolution already used by spec.outputs.contracts.
• docs/architecture.md — links the new doc from the existing "logging and monitoring integration points" bullet.
• tests/test_validator.py — unit coverage for validate_observability_config: shape validation, retention bounds (structuredDays >= rawDays), and sink.contractRef resolution/kind-matching.

Validation

• make check (lint + full unit test suite, 426 tests) passes.
• cds validate local-dagster-postgres-superset still passes (no regression to existing profiles, since spec.observability is optional).
• Manually validated the new log-sink contract against schemas/contract.schema.json.

Out of scope (per #174 and this doc's own "out of scope" section)

• Mandating an observability vendor or shipping a bundled logging stack.
• Persisting raw logs indefinitely.
• A second database/credential API for event storage (owned by Provide a shared Python runtime API for named database connections #219/Make shared DB event persistence truly backend-portable #167).
• Wiring an actual log-sink provider module (e.g. Loki/Fluent Bit) — that's a natural follow-up once a concrete module is desired.

Progresses #174 (left open for follow-up implementation work, per its long-term/epic nature).