Skip to content

[FLINK-40368][table-planner] Fix deduplication changelog inference - #28980

Open
ltyfulan9 wants to merge 1 commit into
apache:masterfrom
ltyfulan9:FLINK-40368-dedup-non-time
Open

[FLINK-40368][table-planner] Fix deduplication changelog inference#28980
ltyfulan9 wants to merge 1 commit into
apache:masterfrom
ltyfulan9:FLINK-40368-dedup-non-time

Conversation

@ltyfulan9

Copy link
Copy Markdown

What is the purpose of the change

Streaming deduplication represented by a Top-1 Rank can be incorrectly inferred as INSERT_ONLY when its ORDER BY uses a non-time attribute or multiple columns. This can cause downstream operators to miss required updates and retractions. This change restricts INSERT_ONLY inference to the existing single-time-attribute case.

Brief change log

  • Gate deduplication INSERT_ONLY changelog inference with RankUtil.sortOnTimeAttributeOnly.
  • Preserve ALL_CHANGES for non-time-attribute and multi-column ORDER BY keys.

Verifying this change

  • Existing plan regressions cover RankTest#testDeduplicateOnNonTimeAttributeGeneratesUpdates and RankTest#testDeduplicateOnMultipleColumnsGeneratesUpdates.
  • flink-table-planner Spotless check passes.
  • Full reactor compilation was not completed locally because the configured Maven repository could not resolve flink-core:tests:2.4-SNAPSHOT before reaching Table Planner.

Does this pull request potentially affect one of the following parts:

  • Dependencies: no
  • Public API: no
  • Serializers: no
  • Runtime per-record code paths: no
  • Deployment or recovery: no
  • S3 file system connector: no

Documentation

  • New feature: no
  • Documentation: not applicable

Was generative AI tooling used to co-author this PR?
  • Yes (Codex GPT-5)

Generated-by: Codex GPT-5

@flinkbot

flinkbot commented Aug 15, 2026

Copy link
Copy Markdown
Collaborator

CI report:

Bot commands The @flinkbot bot supports the following commands:
  • @flinkbot run azure re-run the last Azure build

@ltyfulan9

Copy link
Copy Markdown
Author

@flinkbot run azure

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants