Skip to content

bug(consensus-db): --prune.certificates.before above the chain tip prunes the just-committed certificate and wedges the node on restart #497

Open

Description

@forumevi

Summary
PruningConfig::effective_certificates_min_height returns certificates_before unclamped. If an operator starts the CL with --prune.certificates.before N where N is greater than the current chain tip (a typo, or a height copied from another network), every decide calls prune_historical_certs(latest_height) with retain_height = N and deletes every certificate below N, including the one that was just committed.

Affected code (main, 6e76402)
crates/types/src/config.rs effective_certificates_min_height: returns self.certificates_before without comparing it to current_height.
crates/consensus-db/src/services/pruning.rs prune_historical_certs(latest_height): uses that value directly as the retain height.
crates/consensus-db/src/store.rs prune_height_table_batch: deletes ..retain_height.
crates/malachite-cli/src/cmd/start.rs --prune.certificates.before: no validation of the value.
--prune.certificates.distance cannot hit this because it always retains current - distance.

Consequences
While running: max_height() / min_height() return None, so /commit RPC and GetDecidedValues sync serving return nothing; the node silently stops serving sync to peers.
On restart: the handshake in crates/malachite-app/src/handlers/consensus_ready.rs reads CL height 0 while the EL is at the tip and aborts with a misleading "EL has blocks ... but CL has no committed state" error. The node stays wedged until the consensus DB is restored from a snapshot.
Reproduction
Unit test added to the tests module of crates/consensus-db/src/services/pruning.rs (uses the existing create_store and seed_all_tables_at_height helpers):

#[tokio::test]
async fn repro_certificates_before_above_tip_prunes_latest_certificate() {
let store = create_store().await;
for h in 1u64..=5 {
seed_all_tables_at_height(&store, Height::new(h)).await;
}
let config = PruningConfig {
certificates_distance: 0,
certificates_before: Height::new(1_000),
};
let service = ProdPruningService::new(&store, &config);
// Called after committing height 5, exactly as decide does.
service.prune_historical_certs(Height::new(5)).await.unwrap();
assert_eq!(
store.max_height().await.unwrap(),
Some(Height::new(5)),
"the certificate of the just-committed height must never be pruned"
);
}
cargo test -p arc-consensus-db --lib repro_certificates_before_above_tip_prunes_latest_certificate

Result on main:

test services::pruning::tests::repro_certificates_before_above_tip_prunes_latest_certificate ... FAILED
assertion left == right failed: the certificate of the just-committed height must never be pruned
left: None
right: Some(Height(5))
Expected behavior
The certificate of the latest committed height is never pruned, whatever --prune.certificates.before is set to.

Suggested fix
Clamp the retain height to the current height, e.g. in effective_certificates_min_height:

if self.certificates_before > Height::ZERO {
self.certificates_before.min(current_height)
}
Optionally also log a warning (or reject at startup) when --prune.certificates.before is above the local tip. Happy to open a PR with the fix and the regression test if this gets assigned.

Activity

  1. janmd07 commented on Oct 29, 2026

    @janmd07

    Hi Arc team — I’d like to contribute to this issue if it is still available.

    I plan to reproduce the pruning behavior using the regression test described here, inspect PruningConfig::effective_certificates_min_height and the pruning service, and propose the smallest safe fix to ensure the latest committed certificate is never pruned when certificates_before exceeds the chain tip.

    I’ll run the focused arc-consensus-db test and relevant checks before opening a PR. Could you please assign this issue to me if the scope and approach look good? I’ll wait for maintainer approval before submitting a PR.

  2. forumevi commented on Oct 29, 2026

    @forumevi
    Author

    Hi, I opened this issue and I'm planning to submit the fix myself. I have the regression test already written and a patch ready. Please assign to @forumevi.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions