Skip to content

feat(cli): add bounded telemetry retention - #294

Open
robbiemu wants to merge 3 commits into
MapleTechLabs:mainfrom
robbiemu:codex/bounded-telemetry-retention
Open

feat(cli): add bounded telemetry retention#294
robbiemu wants to merge 3 commits into
MapleTechLabs:mainfrom
robbiemu:codex/bounded-telemetry-retention

Conversation

@robbiemu

@robbiemu robbiemu commented Jul 31, 2026

Copy link
Copy Markdown
Contributor

Summary

Add a bounded, crash-safe lifecycle for local telemetry archives without changing Maple's default retention policy:

  • maple archive retire-live <range-date> --apply retires one sealed UTC day only after all six active archive generations match the live data by count and fixed-memory content digest.
  • maple archive expire <range-date> --apply removes only a complete day already recorded in the durable retired-day authority.
  • <data-dir>.retired-days.json survives reset and checkpoint restore; startup repairs resurrected retired rows before binding.
  • The server coordinates retirement behind a quiescence gate and commits authority before exact UTC deletion.
  • Mixed OTLP batches accept current rows and return OTLP partial success for retired rows.
  • Once retirement authority exists, arbitrary /local/query writes are rejected before execution, preventing materialized-view side effects. Checkpoint backup uses a typed authenticated endpoint.
  • maple start --minimum-raw-telemetry-retention-days N is an optional deployment policy. Omission preserves upstream defaults; configured values are capped at 3650 days, monotonic, validated against the live database before persistence, and never shorten a higher table TTL.

Safety and recovery

  • Archive creation checks retired-day authority both before and inside the maintenance lock, after reconciliation and before any create intent or pointer read.
  • Retirement freezes generation identity, verifies shard SHA-256 and size, compares canonical live/Parquet digests, then re-verifies active generations immediately before the durable commit.
  • The digest uses fixed-size commutative aggregate state rather than an unbounded sorted groupArray.
  • Startup replay performs at most one discovery query and one deletion per raw table, independent of ledger length; idempotent retirement repairs only the requested day.
  • Expiration uses a strict durable journal with per-signal removal intent and canonical progress.
  • UTC session pinning, sealed-day validation, and a default 24-hour sealing lag prevent host-timezone and still-open-day errors.

Validation

  • CLI typecheck passed.
  • Formatting, oxlint, shellcheck, and git diff --check passed.
  • 401 non-listener tests passed.
  • 10/10 listener/network tests passed with host networking.
  • Native checkpoint smoke passed through create, restore, dirty recovery, reset, and repeated reopen cycles.
  • Native retention probe passed with a 500,001-row logs day spanning multiple Parquet shards, same-count/different-content refusal, SQL pre-execution refusal with zero logs_aggregates_hourly/service_usage pollution, mixed OTLP partial acceptance, re-archive refusal, and restore replay.
  • The native retention probe is included in the required local-archive-native CI matrix and shellcheck list.

Based directly on upstream main at 6578edf1.

@robbiemu
robbiemu marked this pull request as ready for review August 1, 2026 17:21
@pullfrog

pullfrog Bot commented Aug 1, 2026

Copy link
Copy Markdown
Contributor

Your Pullfrog Router balance is exhausted.

You have a payment method on file but auto-reload is disabled, so runs paused once your balance went past the overdraft buffer.

Top up balance → · Enable auto-reload →

Pullfrog  | ⚠️ this action is pinned to a commit SHA, which freezes the cleanup step — switch to @v0 or keep the SHA fresh with Dependabot | Rerun failed job ➔View workflow run | via Pullfrog | Using Claude Opus𝕏

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant