[fix](build) Unbreak master: stale unity-skip entry (BE) and dropped count probe (FE) - #66831
Conversation
…_statistics.cpp apache#66052 moved storage/compaction/collection_statistics.{cpp,h} to storage/index/inverted/similarity/ (rewritten) but left behind the SKIP_UNITY_BUILD_INCLUSION entry that apache#66789 had added for the old path. The fail-loud validation from apache#66789 turns the dangling entry into a configure-time error, so BE configure on current master fails: CMake Error at CMakeLists.txt:1002 (message): unity skip entry does not exist (renamed or moved?): .../be/src/storage/compaction/collection_statistics.cpp The CI pipelines merge each PR into the latest master before building, so every PR pipeline that picked up master after apache#66052 landed is red as well (apache#66826, apache#66824, apache#66819, apache#66820 were the first hits). The old entry existed because the old test #included the .cpp into a second TU; nothing #includes the rewritten file, so the new path needs no skip entry. Remove the entry and its comment block. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ND7L1ZVTJf91TBpLwYSqct
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
|
run buildall |
|
PR approved by at least one committer and no changes requested. |
|
PR approved by anyone and no changes requested. |
|
run compile |
|
skip buildall |
…ggregates apache#66778 replaced the snapshot-summary COUNT(*) pushdown with a manifest-derived count and deleted getCountFromSnapshot, but canServeMetadataOnlyCount (added by apache#66413) still calls it. The two PRs never conflict textually, so master merged both cleanly and FE stopped compiling: IcebergScanPlanProvider.java:[505,16] cannot find symbol symbol: method getCountFromSnapshot(org.apache.iceberg.TableScan, org.apache.doris.connector.spi.ConnectorSession) Every pipeline that merges current master hits it; apache master 35fea58 is still red. Re-express the probe in apache#66778's terms instead of resurrecting the deleted method: reuse its delete gate and additionally require the data manifests to carry aggregate row counters, so the answer is proved from the manifest list alone (O(manifests), no data-file enumeration) and never from writer-provided snapshot summary fields. Manifest lists that omit those aggregates now answer false, where count planning still serves them through its bounded per-file fallback -- a capability probe that runs before planning should under-promise rather than over-promise. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
I did not find a blocking correctness, memory-safety, or concurrency issue in the current patch. I have three review suggestions:
|
|
PR approved by at least one committer and no changes requested. |
|
skip buildall |
What
Two independent breakages that each make current master fail to build — one in BE configure, one in FE compile.
They share a shape: a pair of PRs that never conflict textually, merge cleanly, and only break once combined, so each PR's own pipeline was green.
cmakeconfigure abortsfe-connector-icebergdoes not compilegetCountFromSnapshot(), #66413 added a caller for itCI merges each PR into the latest master before building, so every PR pipeline that picks up current master is red on one or both.
1. BE — stale
STORAGE_UNITY_SKIPentry for a moved fileRemove the stale
STORAGE_UNITY_SKIPentry (and its comment block) forcompaction/collection_statistics.cpp, which no longer exists.Why — master configure is currently broken
#66052 moved
storage/compaction/collection_statistics.{cpp,h}tostorage/index/inverted/similarity/(rewritten), but left behind the unity-skip entry that #66789 had added for the old path. The fail-loud validation introduced by #66789 turns a dangling skip entry into a configure-time error — which is exactly what it is designed to catch (a skip list rotting after a file move), so BE configure on current master fails immediately:#66826, #66824, #66819, #66820 were the first hits — same error on multiple independent agents.
Why deletion (not a path update) is correct
The old entry existed because the old
collection_statistics_test#included the.cppinto a second TU (unity batching would then produce a duplicate definition at link time). The rewritten file at the new location is not#included by any test (grep -rn 'collection_statistics.cpp' be/test/is empty on master), so the new path needs no skip entry.Verification
2. FE — the metadata-only COUNT(*) probe calls a deleted method
#66778 replaced the snapshot-summary COUNT(*) pushdown with a manifest-derived count and deleted
getCountFromSnapshot().canServeMetadataOnlyCount(), added by #66413, still calls it, so FE compilation fails:Why re-express the probe instead of restoring the method
Bringing
getCountFromSnapshot()back would reintroduce precisely what #66778 removed: a query result derived from optional, writer-provided snapshot summary fields.The probe is rebuilt in #66778's own terms instead. It reuses that PR's delete gate and additionally requires the data manifests to carry aggregate row counters, so the answer is proved from the manifest list alone —
O(manifests), no data-file enumeration, which is what a probe running before planning can afford.Manifest lists that omit those aggregates now answer
false, where count planning still serves them through its bounded per-file fallback. A capability probe should under-promise rather than over-promise; the planner itself is untouched.Verification
mvn packageover the full FE reactor (74 modules): all green. This also confirms no second semantic break is hiding behind the first — CI's maven stops atfe-connector-icebergand never reaches the rest.IcebergScanPlanProviderTest: 153 tests, 0 failures, including a new case pinning that the probe follows the same delete gate as count planning.Release note
None (both fixes only restore a buildable master; no user-visible behavior change).
Check List (For Author)
IcebergScanPlanProviderTest(153 tests, 0 failures)