[python] Refuse a read whose query auth rules cannot be bound to columns - #10018
Open
plusplusjiajia wants to merge 1 commit into
Open
plusplusjiajia wants to merge 1 commit into
plusplusjiajia wants to merge 1 commit into
Conversation
plusplusjiajia
marked this pull request as draft
September 20, 2026 12:19
plusplusjiajia
force-pushed
the
fix/pypaimon-auth-fail-closed
branch
3 times, most recently
from
September 20, 2026 17:22
6bf5939 to
a66af3e
Compare
plusplusjiajia
marked this pull request as ready for review
September 20, 2026 17:41
plusplusjiajia
force-pushed
the
fix/pypaimon-auth-fail-closed
branch
from
September 20, 2026 17:50
a66af3e to
cc8c50e
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose
pypaimon applies the row filter and the column masks that
authTableQueryreturns. When a rule could not be matched to a column by name, the masking reader skipped it and returned the column untransformed:ROWcolumn where the query projects a sub-field, sinceread_type()flattenss.ainto a field nameds_a— projecting a sub-field was enough to bypass the mask at will;Java fails closed on all three. This ports
validateAgainstSchema,validateReadableWithoutRenameandvalidateReadType, called where Java calls them, plus two things the port needed:Binding needs both schemas. Identity comes from the latest one, which is what the rules name, so a re-added column or a name moved between columns refuses rather than masking unrelated data. The type comes from the schema the split was written under, so a column widened since then still reads.
Some names cannot be bound at all.
_ROW_ID,_SEQUENCE_NUMBERand_VALUE_KINDmay each be the synthetic field or a physical column, and a rename makes the two indistinguishable, so any rule naming one is refused. A physical column carrying such a name therefore cannot carry a rule — a loud refusal rather than a silent skip.Discovery also had to match execution:
_resolve_transform_inputreads any dict input as a column, while the collector only counted one carrying bothnameandindex, so an indexless reference escaped every check. Both the scan and the reader now run the same validation, so a split reaching a reader unplanned is held to the same rules.Tests
TestReaderAppliesTheRulesdrives a realTableReadwith only storage stubbed and asserts the returned values, so a reader that is constructed but masks nothing fails. Each original situation has a case, as do the schema-history ones. Reverting any check fails the suite, including omitting the snapshot schema, swapping the two schemas, restoring the name exemption, and reverting the collector.