Search before asking
Paimon version
version 1.2
Compute Engine
Spark 3.5.6
Minimal reproduce step
table max retained snapshot is 1
'snapshot.num-retained.max' = '1'
2026-04-25 04:37:12,116, Task 1 completed sys.compact operation and submitted snapshot 4959
2026-04-25 04:37:13,195, Task 1 completed snapshot expiration. Mark snapshot 4958 as expired and delete it
2026-04-25 04:37:37,605, Task 2 completed writing and submitted snapshot 4960
2026-04-25 04:37:38,185, Task 2 completed snapshot expiration. Mark snapshot 4959 as expired and delete it
At the same time, Task 1 started executing the remove_orphan_files, but the current snapshot 4959 was not read because 4959 had already been deleted by Task 2, resulting in usedFiles being empty, which in turn caused deleted=candidates, meaning that all files written one day ago were marked as files to be deleted and ultimately deleted.
What doesn't meet your expectations?
Incorrect deletion of historical data
Anything else?
No response
Are you willing to submit a PR?
Search before asking
Paimon version
version 1.2
Compute Engine
Spark 3.5.6
Minimal reproduce step
table max retained snapshot is 1
'snapshot.num-retained.max' = '1'
2026-04-25 04:37:12,116, Task 1 completed sys.compact operation and submitted snapshot 4959
2026-04-25 04:37:13,195, Task 1 completed snapshot expiration. Mark snapshot 4958 as expired and delete it
2026-04-25 04:37:37,605, Task 2 completed writing and submitted snapshot 4960
2026-04-25 04:37:38,185, Task 2 completed snapshot expiration. Mark snapshot 4959 as expired and delete it
At the same time, Task 1 started executing the remove_orphan_files, but the current snapshot 4959 was not read because 4959 had already been deleted by Task 2, resulting in usedFiles being empty, which in turn caused deleted=candidates, meaning that all files written one day ago were marked as files to be deleted and ultimately deleted.
What doesn't meet your expectations?
Incorrect deletion of historical data
Anything else?
No response
Are you willing to submit a PR?