Skip to content

[FLINK-39162][checkpoints] Disable UC for CustomPartitioner#27731

Open
rkhachatryan wants to merge 1 commit intoapache:masterfrom
rkhachatryan:f39162
Open

[FLINK-39162][checkpoints] Disable UC for CustomPartitioner#27731
rkhachatryan wants to merge 1 commit intoapache:masterfrom
rkhachatryan:f39162

Conversation

@rkhachatryan
Copy link
Contributor

@rkhachatryan rkhachatryan commented Mar 3, 2026

No description provided.

@flinkbot
Copy link
Collaborator

flinkbot commented Mar 3, 2026 •
edited
Loading

CI report:

Bot commands The @flinkbot bot supports the following commands:
  • @flinkbot run azure re-run the last Azure build

@rkhachatryan rkhachatryan mentioned this pull request Mar 3, 2026
Merged
@rkhachatryan rkhachatryan marked this pull request as ready for review March 3, 2026 22:50
@rkhachatryan rkhachatryan requested a review from 1996fanrui March 3, 2026 22:51
1996fanrui
1996fanrui approved these changes Mar 4, 2026
View reviewed changes
Copy link
Member

@1996fanrui 1996fanrui left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @rkhachatryan for fix!

The original purpose of FLINK-39162 is to support UC for CustomPartitioner, because it has been supported partially and that is why downscale works.

I agree to disable it explicitly first since it has potential bugs. And it could be improved in the future if needed.

@rkhachatryan
Copy link
Contributor Author

rkhachatryan commented Mar 4, 2026

Thanks for the review!

The original purpose of FLINK-39162 is to support UC for CustomPartitioner, because it has been supported partially and that is why downscale works.

I don't think that's possible but maybe I'm missing something.

1996fanrui
1996fanrui reviewed Mar 4, 2026
View reviewed changes
...e/src/main/java/org/apache/flink/streaming/runtime/partitioner/CustomPartitionerWrapper.java
Comment on lines 64 to 67
// fully rely on filtering downstream
// note that custom partitioners are not officially supported - the user has to force
// rescaling in that case, we assume that the custom partitioner is deterministic
return SubtaskStateMapper.FULL;
Copy link
Member

@1996fanrui 1996fanrui Mar 4, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems unaligned checkpoint could be enabled for custom partitioner when setForceUnalignedCheckpoints is enabled.

disableUnalignedCheckpoints(); will disable UC thoroughly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@1996fanrui 1996fanrui 1996fanrui approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@rkhachatryan @flinkbot @1996fanrui