Skip to content

Implement histogram cardinality reduction#868

Merged
xiaoxichen merged 2 commits intoeBay:masterfrom
xiaoxichen:metrics-reduction
Mar 11, 2026
Merged

Implement histogram cardinality reduction#868
xiaoxichen merged 2 commits intoeBay:masterfrom
xiaoxichen:metrics-reduction

Conversation

@xiaoxichen
Copy link
Collaborator

Convert 19 internal histogram metrics to sum/count mode in production builds while preserving full histograms in debug builds, reducing Prometheus cardinality by ~79% (17,000 -> 3,695 time series).

Convert 19 internal histogram metrics to sum/count mode in production builds
while preserving full histograms in debug builds, reducing Prometheus cardinality
by ~79% (17,000 -> 3,695 time series).

Signed-off-by: Xiaoxi Chen <xiaoxchen@ebay.com>
@codecov-commenter
Copy link

codecov-commenter commented Mar 10, 2026

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

❌ Patch coverage is 0% with 18 lines in your changes missing coverage. Please review.
✅ Project coverage is 48.24%. Comparing base (1a0cef8) to head (cf27c4a).
⚠️ Report is 320 commits behind head on master.

Files with missing lines Patch % Lines
.../include/homestore/btree/detail/btree_internal.hpp 0.00% 0 Missing and 6 partials ⚠️
src/lib/replication/repl_dev/raft_repl_dev.h 0.00% 0 Missing and 5 partials ⚠️
src/lib/logstore/log_store_service.cpp 0.00% 0 Missing and 3 partials ⚠️
src/include/homestore/meta_service.hpp 0.00% 0 Missing and 1 partial ⚠️
src/lib/blkalloc/varsize_blk_allocator.h 0.00% 0 Missing and 1 partial ⚠️
src/lib/blkdata_svc/blk_read_tracker.hpp 0.00% 0 Missing and 1 partial ⚠️
src/lib/device/virtual_dev.hpp 0.00% 0 Missing and 1 partial ⚠️
❗ Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@            Coverage Diff             @@
##           master     #868      +/-   ##
==========================================
- Coverage   56.51%   48.24%   -8.27%     
==========================================
  Files         108      110       +2     
  Lines       10300    12877    +2577     
  Branches     1402     6180    +4778     
==========================================
+ Hits         5821     6213     +392     
+ Misses       3894     2567    -1327     
- Partials      585     4097    +3512     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copy link
Collaborator

@szmyd szmyd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we do something about:

#ifdef _PRERELEASE
    REGISTER_HISTOGRAM(metric_name, "Description", ...);  // Full histogram in debug
#else
    REGISTER_HISTOGRAM(metric_name, "Description", ..., _publish_as::publish_as_sum_count);  // Sum/count in production
#endif

Even if that means doing something akin to:

homestore_metrics.hpp:

...
#define REGISTER_HS_HISTOGRAM(metric_name, ...) \
  #ifdef _PRERELEASE
  ... // you get the point

JacksonYao287

This comment was marked as abuse.

JacksonYao287

This comment was marked as abuse.

JacksonYao287

This comment was marked as abuse.

Add REGISTER_HISTOGRAM_WITH_CARDINALITY_REDUCTION macro to eliminate
repetitive #ifdef _PRERELEASE blocks around histogram registrations.

Signed-off-by: Xiaoxi Chen <xiaoxchen@ebay.com>
Copy link
Collaborator

@szmyd szmyd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Image
lgtm

@xiaoxichen xiaoxichen merged commit cb25a5e into eBay:master Mar 11, 2026
21 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants