Skip to content

branch-4.1: [fix](iceberg) Avoid BE abort on partition predicates - #66835

Open
suxiaogang223 wants to merge 1 commit into
apache:branch-4.1from
suxiaogang223:codex/fix-iceberg-branch-partition-coredump
Open

branch-4.1: [fix](iceberg) Avoid BE abort on partition predicates#66835
suxiaogang223 wants to merge 1 commit into
apache:branch-4.1from
suxiaogang223:codex/fix-iceberg-branch-partition-coredump

Conversation

@suxiaogang223

Copy link
Copy Markdown
Member

What problem does this PR solve?

Issue Number: DORIS-27947

Related PR: #66012, #66481

Problem Summary:

On branch-4.1, Iceberg identity partition columns are classified as partition keys and are therefore omitted from the Parquet file-column schema tree. A predicate on such a column still reaches the Parquet file-level predicate probe. StructNode::children_column_exists() assumed that every probed column had a schema-tree entry, so a partition predicate could trigger a DCHECK and abort the BE.

This change makes the existence probe return false when the schema tree has no entry for the column. Parquet then skips file-level predicate pushdown for that partition column while the existing partition-value fill and upper-level predicate evaluation remain in effect. The added unit test covers present, known-missing, and absent schema-tree children.

Release note

Fix a BE crash when querying Iceberg branch data with an identity partition predicate.

Validation

  • Formatted the changed files with clang-format 16.0.6.
  • git diff --check passed.
  • BE compilation and unit tests were not run locally because of their cost.

Behavior changed

Yes. A column absent from the file schema tree is now treated as unavailable for file-level predicate pushdown instead of terminating the BE.

@hello-stephen

Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@suxiaogang223

Copy link
Copy Markdown
Member Author

run buildall

@hello-stephen

Copy link
Copy Markdown
Contributor

BE UT Coverage Report

Increment line coverage 100.00% (2/2) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 58.92% (25251/42856)
Line Coverage 43.38% (256880/592188)
Region Coverage 39.39% (203394/516395)
Branch Coverage 40.89% (93773/229346)

@suxiaogang223
suxiaogang223 marked this pull request as ready for review August 18, 2026 09:14
@suxiaogang223

Copy link
Copy Markdown
Member Author

run external

@hello-stephen

Copy link
Copy Markdown
Contributor

BE Regression && UT Coverage Report

Increment line coverage 100.00% (2/2) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 73.89% (30861/41766)
Line Coverage 58.03% (341092/587738)
Region Coverage 54.87% (284042/517701)
Branch Coverage 55.76% (127743/229076)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants