Skip to content

Add API to update segment ZK metadata without uploading - #19180

Open
goutamadwant wants to merge 1 commit into
apache:masterfrom
goutamadwant:update-segment-zk-metadata-api
Open

Add API to update segment ZK metadata without uploading#19180
goutamadwant wants to merge 1 commit into
apache:masterfrom
goutamadwant:update-segment-zk-metadata-api

Conversation

@goutamadwant

Copy link
Copy Markdown
Contributor

Closes #10458

What this PR does

  • Adds a dedicated PUT /segments/{tableNameWithType}/{segmentName}/metadata API for updating the segment ZK metadata custom map.
  • Validates the expected segment CRC through If-Match and uses the ZK record version for compare-and-set protection.
  • Routes unchanged single-segment conversions through the metadata-only API, avoiding segment compression and upload.
  • Falls back to the existing refresh-only upload path when a mixed-version controller does not expose the new endpoint.

Backward compatibility

The endpoint is additive. Existing upload behavior is unchanged for modified segments, and older controllers continue to work through the fallback path.

Tests

  • PinotSegmentRestletResourceTest
  • BaseSingleSegmentConversionExecutorTest
  • Pinot pre-commit checks for pinot-common, pinot-controller, and pinot-minion-builtin-tasks

@codecov-commenter

codecov-commenter commented Aug 7, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 47.69231% with 34 lines in your changes missing coverage. Please review.
✅ Project coverage is 66.64%. Comparing base (d3604a5) to head (090e039).
⚠️ Report is 1 commits behind head on master.

Files with missing lines Patch % Lines
...ot/plugin/minion/tasks/SegmentConversionUtils.java 0.00% 10 Missing ⚠️
...ler/api/resources/PinotSegmentRestletResource.java 68.96% 6 Missing and 3 partials ⚠️
...ion/tasks/BaseSingleSegmentConversionExecutor.java 57.89% 7 Missing and 1 partial ⚠️
...e/pinot/common/utils/FileUploadDownloadClient.java 0.00% 7 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##             master   #19180      +/-   ##
============================================
+ Coverage     66.62%   66.64%   +0.01%     
- Complexity     1423     1425       +2     
============================================
  Files          3443     3443              
  Lines        218577   218636      +59     
  Branches      34792    34798       +6     
============================================
+ Hits         145624   145703      +79     
+ Misses        61218    61197      -21     
- Partials      11735    11736       +1     
Flag Coverage Δ
custom-integration1 100.00% <ø> (ø)
integration 100.00% <ø> (ø)
integration1 100.00% <ø> (ø)
integration2 0.00% <ø> (ø)
java-25 66.64% <47.69%> (+0.01%) ⬆️
temurin 66.64% <47.69%> (+0.01%) ⬆️
unittests 66.63% <47.69%> (+0.01%) ⬆️
unittests1 57.14% <0.00%> (+0.01%) ⬆️
unittests2 38.94% <47.69%> (+0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@Jackie-Jiang Jackie-Jiang added enhancement Improvement to existing functionality ingestion Related to data ingestion pipeline rest-api Related to REST API endpoints labels Aug 7, 2026
@Jackie-Jiang

Copy link
Copy Markdown
Contributor

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement Improvement to existing functionality ingestion Related to data ingestion pipeline rest-api Related to REST API endpoints

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add API to update ZK metadata of a segment

3 participants