[Epic] P2 — Cluster Consensus Safety (v0.6)

[habibtalib]

Epic — v0.6 production-hardening, phase P2: Cluster Consensus Safety

Goal: a restart or partition never produces two leaders, divergent committed entries, or silent state-machine divergence; linearizable reads are actually linearizable.

Critical (filed)

• Raft HardState (current_term/voted_for) is never persisted — split-brain on restart #161 — Raft HardState never persisted (+ reply-before-durable ordering) → split-brain on restart
• Data-group InstallSnapshot advances the Raft index without transferring state-machine data — silent divergence #162 — data-group InstallSnapshot advances the index without the data → silent divergence

High

• Deposed/partitioned leader serves stale linearizable reads — no ReadIndex heartbeat, no leader lease. nodedb-cluster/src/follower_read.rs:60-61.
• BoundedStaleness freshness measures time-since-last-apply, not lag-vs-leader — a badly-lagging follower reports "fresh". closed_timestamp.rs:104-110.
• Rolling upgrade blocked on any wire bump (MIN_WIRE_FORMAT_VERSION == WIRE_FORMAT_VERSION) → version hard-partition mid-upgrade. nodedb-types/src/wire_version.rs:19-23.
• No fencing tokens anywhere in raft/cluster.

Medium / Low

• Scatter-gather has no shard-failure/timeout guard → hang or silently-partial aggregate. distributed_timeseries/coordinator.rs:109-115.
• Descriptor-lease expiry uses raw cross-node wall clock, no skew bound / fencing token. control/lease/renewal.rs:269-293.
• SWIM fast-restart rejoin can stick (restart at incarnation 0 refuted vs lingering Dead). swim/bootstrap.rs:128.
• No pre-vote (partition-heal term inflation forces healthy leader to step down); no leadership transfer (TimeoutNow).
• Snapshot GC sweep_orphans runs at startup only; decommission doesn't propagate cluster-wide ShutdownWatch.

Tracked in .planning/v1-production-readiness/AUDIT.md.