Skip to content

ci(versioning): release proposal workflow improvements#1763

Draft
iunanua wants to merge 8 commits intomainfrom
igor/versioning/use-commit-headless-and-check-deps
Draft

ci(versioning): release proposal workflow improvements#1763
iunanua wants to merge 8 commits intomainfrom
igor/versioning/use-commit-headless-and-check-deps

Conversation

@iunanua
Copy link
Contributor

@iunanua iunanua commented Mar 18, 2026

What does this PR do?

  • use commit-headless to sign commits
  • add a new workflow input to allow selecting the starting SHA commit where to start the release from
  • add the new check-breaking-deps job to check if the crates in the proposal have duplicated dependencies with different versions. It just prints the warning but it should be included in the PR's body if it is useful

@pr-commenter
Copy link

pr-commenter bot commented Mar 18, 2026 •
edited
Loading

Benchmarks

Comparison

Benchmark execution time: 2026-03-23 12:31:39

Comparing candidate commit 1c7a149 in PR branch igor/versioning/use-commit-headless-and-check-deps with baseline commit ed00b92 in branch main.

Found 1 performance improvements and 14 performance regressions! Performance is the same for 45 metrics, 1 unstable metrics.

Explanation

This is an A/B test comparing a candidate commit's performance against that of a baseline commit. Performance changes are noted in the tables below as:

  • 🟩 = significantly better candidate vs. baseline
  • 🟥 = significantly worse candidate vs. baseline

We compute a confidence interval (CI) over the relative difference of means between metrics from the candidate and baseline commits, considering the baseline as the reference.

If the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD), the change is considered significant.

Feel free to reach out to #apm-benchmarking-platform on Slack if you have any questions.

More details about the CI and significant changes

You can imagine this CI as a range of values that is likely to contain the true difference of means between the candidate and baseline commits.

CIs of the difference of means are often centered around 0%, because often changes are not that big:

---------------------------------(------|---^--------)-------------------------------->
                              -0.6%    0%  0.3%     +1.2%
                                 |          |        |
         lower bound of the CI --'          |        |
sample mean (center of the CI) -------------'        |
         upper bound of the CI ----------------------'

As described above, a change is considered significant if the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD).

For instance, for an execution time metric, this confidence interval indicates a significantly worse performance:

----------------------------------------|---------|---(---------^---------)---------->
                                       0%        1%  1.3%      2.2%      3.1%
                                                  |   |         |         |
       significant impact threshold --------------'   |         |         |
                      lower bound of CI --------------'         |         |
       sample mean (center of the CI) --------------------------'         |
                      upper bound of CI ----------------------------------'

scenario:credit_card/is_card_number/ 3782-8224-6310-005

  • 🟥 execution_time [+4.960µs; +5.173µs] or [+6.552%; +6.833%]
  • 🟥 throughput [-847821.980op/s; -811380.933op/s] or [-6.418%; -6.142%]

scenario:credit_card/is_card_number/ 378282246310005

  • 🟥 execution_time [+4.899µs; +4.997µs] or [+7.128%; +7.271%]
  • 🟥 throughput [-986061.334op/s; -967748.545op/s] or [-6.777%; -6.651%]

scenario:credit_card/is_card_number/378282246310005

  • 🟥 execution_time [+5.690µs; +5.770µs] or [+8.771%; +8.893%]
  • 🟥 throughput [-1258989.280op/s; -1242511.244op/s] or [-8.168%; -8.061%]

scenario:credit_card/is_card_number/37828224631000521389798

  • 🟥 execution_time [+7.368µs; +7.395µs] or [+16.108%; +16.166%]
  • 🟥 throughput [-3043833.922op/s; -3031400.007op/s] or [-13.923%; -13.867%]

scenario:credit_card/is_card_number_no_luhn/ 378282246310005

  • 🟥 execution_time [+4.138µs; +4.186µs] or [+7.632%; +7.720%]
  • 🟥 throughput [-1322520.114op/s; -1307308.291op/s] or [-7.170%; -7.088%]

scenario:credit_card/is_card_number_no_luhn/378282246310005

  • 🟥 execution_time [+5.043µs; +5.110µs] or [+9.993%; +10.124%]
  • 🟥 throughput [-1822595.186op/s; -1798901.975op/s] or [-9.199%; -9.079%]

scenario:credit_card/is_card_number_no_luhn/37828224631000521389798

  • 🟥 execution_time [+7.393µs; +7.420µs] or [+16.168%; +16.228%]
  • 🟥 throughput [-3055021.899op/s; -3042407.175op/s] or [-13.969%; -13.911%]

scenario:write only interface

  • 🟩 execution_time [-2.582µs; -2.184µs] or [-46.527%; -39.358%]

Candidate

Candidate benchmark details

Group 1

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 1c7a149 1774268055 igor/versioning/use-commit-headless-and-check-deps
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... execution_time 186.222µs 186.648µs ± 0.174µs 186.640µs ± 0.108µs 186.748µs 186.935µs 187.052µs 187.213µs 0.31% 0.270 -0.027 0.09% 0.012µs 1 200
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... throughput 5341510.117op/s 5357678.918op/s ± 4987.426op/s 5357900.378op/s ± 3101.082op/s 5360957.815op/s 5365437.284op/s 5367856.783op/s 5369929.042op/s 0.22% -0.265 -0.033 0.09% 352.664op/s 1 200
normalization/normalize_name/normalize_name/bad-name execution_time 17.941µs 18.018µs ± 0.038µs 18.015µs ± 0.026µs 18.040µs 18.083µs 18.094µs 18.203µs 1.04% 0.660 1.814 0.21% 0.003µs 1 200
normalization/normalize_name/normalize_name/bad-name throughput 54936126.366op/s 55500296.331op/s ± 116938.580op/s 55508197.373op/s ± 79241.605op/s 55589773.633op/s 55667536.467op/s 55735177.127op/s 55738670.881op/s 0.42% -0.639 1.728 0.21% 8268.806op/s 1 200
normalization/normalize_name/normalize_name/good execution_time 10.463µs 10.603µs ± 0.053µs 10.611µs ± 0.032µs 10.634µs 10.678µs 10.728µs 10.823µs 2.00% 0.182 1.033 0.50% 0.004µs 1 200
normalization/normalize_name/normalize_name/good throughput 92392525.445op/s 94311452.209op/s ± 470555.150op/s 94243641.233op/s ± 279904.165op/s 94614601.429op/s 95148833.169op/s 95264936.886op/s 95576612.982op/s 1.41% -0.138 0.940 0.50% 33273.274op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... execution_time [186.624µs; 186.672µs] or [-0.013%; +0.013%] None None None
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... throughput [5356987.709op/s; 5358370.128op/s] or [-0.013%; +0.013%] None None None
normalization/normalize_name/normalize_name/bad-name execution_time [18.013µs; 18.023µs] or [-0.029%; +0.029%] None None None
normalization/normalize_name/normalize_name/bad-name throughput [55484089.768op/s; 55516502.893op/s] or [-0.029%; +0.029%] None None None
normalization/normalize_name/normalize_name/good execution_time [10.596µs; 10.611µs] or [-0.069%; +0.069%] None None None
normalization/normalize_name/normalize_name/good throughput [94246237.791op/s; 94376666.628op/s] or [-0.069%; +0.069%] None None None

Group 2

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 1c7a149 1774268055 igor/versioning/use-commit-headless-and-check-deps
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
write only interface execution_time 1.246µs 3.167µs ± 1.435µs 2.991µs ± 0.020µs 3.011µs 3.311µs 13.834µs 15.163µs 407.00% 7.541 57.314 45.21% 0.101µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
write only interface execution_time [2.968µs; 3.366µs] or [-6.282%; +6.282%] None None None

Group 3

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 1c7a149 1774268055 igor/versioning/use-commit-headless-and-check-deps
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
normalization/normalize_trace/test_trace execution_time 236.335ns 246.968ns ± 12.691ns 242.548ns ± 4.561ns 248.719ns 278.035ns 287.564ns 289.784ns 19.47% 1.942 2.896 5.13% 0.897ns 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
normalization/normalize_trace/test_trace execution_time [245.209ns; 248.727ns] or [-0.712%; +0.712%] None None None

Group 4

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 1c7a149 1774268055 igor/versioning/use-commit-headless-and-check-deps
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
benching string interning on wordpress profile execution_time 159.485µs 159.974µs ± 0.408µs 159.894µs ± 0.130µs 160.031µs 160.578µs 161.868µs 163.289µs 2.12% 4.228 25.534 0.25% 0.029µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
benching string interning on wordpress profile execution_time [159.918µs; 160.031µs] or [-0.035%; +0.035%] None None None

Group 5

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 1c7a149 1774268055 igor/versioning/use-commit-headless-and-check-deps
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
tags/replace_trace_tags execution_time 2.448µs 2.472µs ± 0.017µs 2.468µs ± 0.011µs 2.480µs 2.510µs 2.517µs 2.520µs 2.09% 1.007 0.461 0.68% 0.001µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
tags/replace_trace_tags execution_time [2.470µs; 2.474µs] or [-0.095%; +0.095%] None None None

Group 6

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 1c7a149 1774268055 igor/versioning/use-commit-headless-and-check-deps
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
profile_add_sample_frames_x1000 execution_time 4.164ms 4.172ms ± 0.003ms 4.172ms ± 0.002ms 4.174ms 4.177ms 4.180ms 4.205ms 0.80% 4.424 38.418 0.08% 0.000ms 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
profile_add_sample_frames_x1000 execution_time [4.172ms; 4.173ms] or [-0.012%; +0.012%] None None None

Group 7

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 1c7a149 1774268055 igor/versioning/use-commit-headless-and-check-deps
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
concentrator/add_spans_to_concentrator execution_time 12.934ms 12.962ms ± 0.018ms 12.959ms ± 0.009ms 12.970ms 12.993ms 13.016ms 13.055ms 0.74% 1.469 4.348 0.14% 0.001ms 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
concentrator/add_spans_to_concentrator execution_time [12.959ms; 12.964ms] or [-0.019%; +0.019%] None None None

Group 8

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 1c7a149 1774268055 igor/versioning/use-commit-headless-and-check-deps
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
benching deserializing traces from msgpack to their internal representation execution_time 49.469ms 49.729ms ± 0.958ms 49.605ms ± 0.042ms 49.642ms 49.757ms 53.480ms 59.940ms 20.83% 8.816 81.959 1.92% 0.068ms 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
benching deserializing traces from msgpack to their internal representation execution_time [49.597ms; 49.862ms] or [-0.267%; +0.267%] None None None

Group 9

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 1c7a149 1774268055 igor/versioning/use-commit-headless-and-check-deps
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
sql/obfuscate_sql_string execution_time 86.478µs 86.734µs ± 0.234µs 86.703µs ± 0.050µs 86.757µs 86.924µs 87.210µs 89.412µs 3.13% 8.428 88.438 0.27% 0.017µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
sql/obfuscate_sql_string execution_time [86.702µs; 86.767µs] or [-0.037%; +0.037%] None None None

Group 10

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 1c7a149 1774268055 igor/versioning/use-commit-headless-and-check-deps
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
profile_add_sample2_frames_x1000 execution_time 727.829µs 728.695µs ± 0.460µs 728.642µs ± 0.320µs 728.950µs 729.528µs 729.980µs 730.238µs 0.22% 0.718 0.511 0.06% 0.032µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
profile_add_sample2_frames_x1000 execution_time [728.631µs; 728.758µs] or [-0.009%; +0.009%] None None None

Group 11

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 1c7a149 1774268055 igor/versioning/use-commit-headless-and-check-deps
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
sdk_test_data/rules-based execution_time 145.500µs 147.446µs ± 1.719µs 147.149µs ± 0.511µs 147.703µs 149.178µs 154.626µs 163.554µs 11.15% 5.483 41.818 1.16% 0.122µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
sdk_test_data/rules-based execution_time [147.208µs; 147.684µs] or [-0.162%; +0.162%] None None None

Group 12

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 1c7a149 1774268055 igor/versioning/use-commit-headless-and-check-deps
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
ip_address/quantize_peer_ip_address_benchmark execution_time 5.004µs 5.062µs ± 0.045µs 5.059µs ± 0.041µs 5.105µs 5.138µs 5.141µs 5.142µs 1.65% 0.368 -1.357 0.88% 0.003µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
ip_address/quantize_peer_ip_address_benchmark execution_time [5.056µs; 5.068µs] or [-0.122%; +0.122%] None None None

Group 13

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 1c7a149 1774268055 igor/versioning/use-commit-headless-and-check-deps
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
single_flag_killswitch/rules-based execution_time 190.516ns 192.814ns ± 1.985ns 192.378ns ± 1.233ns 193.734ns 197.186ns 199.134ns 200.290ns 4.11% 1.350 1.587 1.03% 0.140ns 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
single_flag_killswitch/rules-based execution_time [192.538ns; 193.089ns] or [-0.143%; +0.143%] None None None

Group 14

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 1c7a149 1774268055 igor/versioning/use-commit-headless-and-check-deps
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... execution_time 495.485µs 496.258µs ± 0.457µs 496.222µs ± 0.319µs 496.529µs 496.887µs 497.246µs 499.741µs 0.71% 2.290 15.212 0.09% 0.032µs 1 200
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... throughput 2001037.649op/s 2015081.297op/s ± 1853.387op/s 2015226.830op/s ± 1293.810op/s 2016529.888op/s 2017478.006op/s 2018173.771op/s 2018223.889op/s 0.15% -2.257 14.899 0.09% 131.054op/s 1 200
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて execution_time 369.738µs 370.322µs ± 0.458µs 370.274µs ± 0.164µs 370.415µs 370.820µs 372.918µs 373.681µs 0.92% 4.438 27.053 0.12% 0.032µs 1 200
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて throughput 2676078.396op/s 2700355.409op/s ± 3319.697op/s 2700702.723op/s ± 1192.521op/s 2701995.612op/s 2703723.795op/s 2704368.005op/s 2704620.091op/s 0.15% -4.403 26.737 0.12% 234.738op/s 1 200
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters execution_time 169.170µs 169.517µs ± 0.162µs 169.504µs ± 0.103µs 169.613µs 169.781µs 170.034µs 170.150µs 0.38% 0.630 1.162 0.10% 0.011µs 1 200
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters throughput 5877155.959op/s 5899121.061op/s ± 5645.474op/s 5899567.411op/s ± 3570.331op/s 5902855.067op/s 5908126.195op/s 5910031.649op/s 5911203.008op/s 0.20% -0.622 1.142 0.10% 399.195op/s 1 200
normalization/normalize_service/normalize_service/[empty string] execution_time 36.810µs 36.990µs ± 0.110µs 36.972µs ± 0.097µs 37.089µs 37.161µs 37.200µs 37.258µs 0.77% 0.191 -1.137 0.30% 0.008µs 1 200
normalization/normalize_service/normalize_service/[empty string] throughput 26839925.080op/s 27034718.380op/s ± 80546.607op/s 27047731.992op/s ± 70557.473op/s 27102399.698op/s 27147676.680op/s 27159544.197op/s 27166747.381op/s 0.44% -0.184 -1.143 0.30% 5695.505op/s 1 200
normalization/normalize_service/normalize_service/test_ASCII execution_time 46.197µs 46.299µs ± 0.110µs 46.284µs ± 0.026µs 46.316µs 46.361µs 46.560µs 47.613µs 2.87% 8.983 100.809 0.24% 0.008µs 1 200
normalization/normalize_service/normalize_service/test_ASCII throughput 21002700.902op/s 21598748.953op/s ± 50331.795op/s 21605611.635op/s ± 12166.544op/s 21616743.249op/s 21632695.223op/s 21639868.175op/s 21646554.586op/s 0.19% -8.826 98.133 0.23% 3558.995op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... execution_time [496.195µs; 496.322µs] or [-0.013%; +0.013%] None None None
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... throughput [2014824.435op/s; 2015338.159op/s] or [-0.013%; +0.013%] None None None
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて execution_time [370.259µs; 370.386µs] or [-0.017%; +0.017%] None None None
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて throughput [2699895.331op/s; 2700815.487op/s] or [-0.017%; +0.017%] None None None
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters execution_time [169.494µs; 169.539µs] or [-0.013%; +0.013%] None None None
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters throughput [5898338.652op/s; 5899903.469op/s] or [-0.013%; +0.013%] None None None
normalization/normalize_service/normalize_service/[empty string] execution_time [36.975µs; 37.005µs] or [-0.041%; +0.041%] None None None
normalization/normalize_service/normalize_service/[empty string] throughput [27023555.395op/s; 27045881.365op/s] or [-0.041%; +0.041%] None None None
normalization/normalize_service/normalize_service/test_ASCII execution_time [46.284µs; 46.315µs] or [-0.033%; +0.033%] None None None
normalization/normalize_service/normalize_service/test_ASCII throughput [21591773.450op/s; 21605724.456op/s] or [-0.032%; +0.032%] None None None

Group 15

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 1c7a149 1774268055 igor/versioning/use-commit-headless-and-check-deps
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
profile_add_sample_timestamped_x1000 execution_time 4.159ms 4.165ms ± 0.008ms 4.164ms ± 0.002ms 4.166ms 4.168ms 4.173ms 4.277ms 2.70% 12.092 157.782 0.20% 0.001ms 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
profile_add_sample_timestamped_x1000 execution_time [4.164ms; 4.166ms] or [-0.028%; +0.028%] None None None

Group 16

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 1c7a149 1774268055 igor/versioning/use-commit-headless-and-check-deps
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
receiver_entry_point/report/2598 execution_time 3.407ms 3.443ms ± 0.020ms 3.438ms ± 0.010ms 3.451ms 3.484ms 3.497ms 3.520ms 2.41% 1.219 1.250 0.58% 0.001ms 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
receiver_entry_point/report/2598 execution_time [3.440ms; 3.446ms] or [-0.080%; +0.080%] None None None

Group 17

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 1c7a149 1774268055 igor/versioning/use-commit-headless-and-check-deps
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
redis/obfuscate_redis_string execution_time 33.777µs 34.350µs ± 0.723µs 33.916µs ± 0.093µs 35.101µs 35.608µs 35.975µs 37.432µs 10.37% 1.254 0.697 2.10% 0.051µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
redis/obfuscate_redis_string execution_time [34.250µs; 34.450µs] or [-0.292%; +0.292%] None None None

Group 18

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 1c7a149 1774268055 igor/versioning/use-commit-headless-and-check-deps
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
two way interface execution_time 17.940µs 25.558µs ± 9.402µs 18.219µs ± 0.238µs 34.017µs 42.314µs 43.529µs 67.349µs 269.66% 0.914 0.425 36.69% 0.665µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
two way interface execution_time [24.255µs; 26.861µs] or [-5.098%; +5.098%] None None None

Group 19

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 1c7a149 1774268055 igor/versioning/use-commit-headless-and-check-deps
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
benching serializing traces from their internal representation to msgpack execution_time 14.205ms 14.252ms ± 0.029ms 14.248ms ± 0.011ms 14.258ms 14.287ms 14.348ms 14.475ms 1.59% 3.977 24.303 0.20% 0.002ms 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
benching serializing traces from their internal representation to msgpack execution_time [14.247ms; 14.256ms] or [-0.028%; +0.028%] None None None

Group 20

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 1c7a149 1774268055 igor/versioning/use-commit-headless-and-check-deps
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
credit_card/is_card_number/ execution_time 3.893µs 3.912µs ± 0.003µs 3.912µs ± 0.001µs 3.913µs 3.916µs 3.920µs 3.924µs 0.31% -0.573 10.777 0.07% 0.000µs 1 200
credit_card/is_card_number/ throughput 254853559.410op/s 255628187.537op/s ± 182652.014op/s 255635188.184op/s ± 91918.788op/s 255729162.408op/s 255847436.453op/s 255917524.304op/s 256850683.031op/s 0.48% 0.600 10.912 0.07% 12915.448op/s 1 200
credit_card/is_card_number/ 3782-8224-6310-005 execution_time 80.324µs 80.770µs ± 0.278µs 80.731µs ± 0.176µs 80.927µs 81.275µs 81.616µs 81.786µs 1.31% 0.963 1.186 0.34% 0.020µs 1 200
credit_card/is_card_number/ 3782-8224-6310-005 throughput 12226980.884op/s 12380929.063op/s ± 42419.433op/s 12386804.967op/s ± 26985.083op/s 12410787.463op/s 12438243.966op/s 12448558.429op/s 12449562.955op/s 0.51% -0.940 1.119 0.34% 2999.507op/s 1 200
credit_card/is_card_number/ 378282246310005 execution_time 73.009µs 73.676µs ± 0.344µs 73.652µs ± 0.237µs 73.903µs 74.264µs 74.572µs 74.930µs 1.74% 0.571 0.359 0.47% 0.024µs 1 200
credit_card/is_card_number/ 378282246310005 throughput 13345805.119op/s 13573276.274op/s ± 63220.357op/s 13577418.354op/s ± 43745.718op/s 13618962.726op/s 13665590.611op/s 13687467.337op/s 13696952.509op/s 0.88% -0.543 0.296 0.46% 4470.354op/s 1 200
credit_card/is_card_number/37828224631 execution_time 3.891µs 3.912µs ± 0.003µs 3.912µs ± 0.001µs 3.914µs 3.915µs 3.920µs 3.921µs 0.23% -1.424 17.043 0.07% 0.000µs 1 200
credit_card/is_card_number/37828224631 throughput 255019607.687op/s 255607488.073op/s ± 179406.591op/s 255618406.625op/s ± 92461.087op/s 255709346.180op/s 255809787.166op/s 255852117.900op/s 256981592.902op/s 0.53% 1.460 17.306 0.07% 12685.962op/s 1 200
credit_card/is_card_number/378282246310005 execution_time 70.073µs 70.606µs ± 0.258µs 70.589µs ± 0.163µs 70.764µs 71.066µs 71.266µs 71.439µs 1.20% 0.495 0.233 0.37% 0.018µs 1 200
credit_card/is_card_number/378282246310005 throughput 13997967.337op/s 14163288.250op/s ± 51749.361op/s 14166463.730op/s ± 32754.603op/s 14198181.932op/s 14238832.432op/s 14265677.897op/s 14270869.136op/s 0.74% -0.473 0.200 0.36% 3659.232op/s 1 200
credit_card/is_card_number/37828224631000521389798 execution_time 53.059µs 53.125µs ± 0.029µs 53.121µs ± 0.018µs 53.141µs 53.175µs 53.206µs 53.222µs 0.19% 0.609 0.548 0.05% 0.002µs 1 200
credit_card/is_card_number/37828224631000521389798 throughput 18789099.221op/s 18823524.606op/s ± 10194.673op/s 18824833.382op/s ± 6368.435op/s 18830412.212op/s 18838239.022op/s 18841980.427op/s 18846872.551op/s 0.12% -0.606 0.541 0.05% 720.872op/s 1 200
credit_card/is_card_number/x371413321323331 execution_time 6.430µs 6.442µs ± 0.006µs 6.441µs ± 0.004µs 6.446µs 6.452µs 6.458µs 6.461µs 0.31% 0.487 0.323 0.09% 0.000µs 1 200
credit_card/is_card_number/x371413321323331 throughput 154770221.532op/s 155230266.346op/s ± 146152.174op/s 155251739.997op/s ± 92585.719op/s 155318475.144op/s 155451107.808op/s 155513513.211op/s 155532676.885op/s 0.18% -0.481 0.313 0.09% 10334.519op/s 1 200
credit_card/is_card_number_no_luhn/ execution_time 3.895µs 3.913µs ± 0.003µs 3.913µs ± 0.002µs 3.915µs 3.918µs 3.922µs 3.936µs 0.57% 0.959 12.938 0.08% 0.000µs 1 200
credit_card/is_card_number_no_luhn/ throughput 254089529.743op/s 255528197.758op/s ± 216586.078op/s 255535435.180op/s ± 120522.492op/s 255659580.223op/s 255776143.280op/s 255851085.702op/s 256722448.496op/s 0.46% -0.923 12.848 0.08% 15314.948op/s 1 200
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 execution_time 64.753µs 65.151µs ± 0.134µs 65.123µs ± 0.070µs 65.210µs 65.387µs 65.528µs 65.804µs 1.05% 1.130 3.811 0.20% 0.009µs 1 200
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 throughput 15196631.085op/s 15348952.998op/s ± 31415.136op/s 15355450.766op/s ± 16491.388op/s 15367042.689op/s 15383105.965op/s 15418775.934op/s 15443352.579op/s 0.57% -1.103 3.720 0.20% 2221.386op/s 1 200
credit_card/is_card_number_no_luhn/ 378282246310005 execution_time 58.204µs 58.380µs ± 0.119µs 58.365µs ± 0.072µs 58.438µs 58.618µs 58.726µs 58.933µs 0.97% 1.285 2.505 0.20% 0.008µs 1 200
credit_card/is_card_number_no_luhn/ 378282246310005 throughput 16968368.199op/s 17129227.129op/s ± 34766.378op/s 17133433.207op/s ± 21164.387op/s 17154731.252op/s 17172368.543op/s 17175687.062op/s 17180810.752op/s 0.28% -1.268 2.428 0.20% 2458.354op/s 1 200
credit_card/is_card_number_no_luhn/37828224631 execution_time 3.894µs 3.913µs ± 0.003µs 3.913µs ± 0.002µs 3.914µs 3.916µs 3.919µs 3.926µs 0.35% -0.733 14.509 0.07% 0.000µs 1 200
credit_card/is_card_number_no_luhn/37828224631 throughput 254688393.832op/s 255582976.237op/s ± 171520.646op/s 255586658.377op/s ± 98267.371op/s 255687080.630op/s 255776174.795op/s 255824878.287op/s 256795898.499op/s 0.47% 0.766 14.651 0.07% 12128.341op/s 1 200
credit_card/is_card_number_no_luhn/378282246310005 execution_time 55.231µs 55.549µs ± 0.175µs 55.520µs ± 0.104µs 55.641µs 55.842µs 56.083µs 56.537µs 1.83% 1.387 4.704 0.31% 0.012µs 1 200
credit_card/is_card_number_no_luhn/378282246310005 throughput 17687376.500op/s 18002241.344op/s ± 56327.289op/s 18011495.117op/s ± 33574.726op/s 18037953.499op/s 18078590.622op/s 18094922.510op/s 18105906.147op/s 0.52% -1.343 4.444 0.31% 3982.941op/s 1 200
credit_card/is_card_number_no_luhn/37828224631000521389798 execution_time 52.904µs 53.132µs ± 0.040µs 53.129µs ± 0.020µs 53.150µs 53.193µs 53.223µs 53.316µs 0.35% -0.700 8.234 0.08% 0.003µs 1 200
credit_card/is_card_number_no_luhn/37828224631000521389798 throughput 18756242.777op/s 18821207.598op/s ± 14235.632op/s 18821980.069op/s ± 6979.209op/s 18828778.263op/s 18836710.073op/s 18864800.181op/s 18902315.534op/s 0.43% 0.722 8.297 0.08% 1006.611op/s 1 200
credit_card/is_card_number_no_luhn/x371413321323331 execution_time 6.430µs 6.438µs ± 0.005µs 6.437µs ± 0.003µs 6.441µs 6.449µs 6.457µs 6.462µs 0.39% 1.559 3.384 0.08% 0.000µs 1 200
credit_card/is_card_number_no_luhn/x371413321323331 throughput 154739001.408op/s 155317789.046op/s ± 130985.129op/s 155347205.253op/s ± 69199.080op/s 155402163.975op/s 155464253.401op/s 155504568.743op/s 155518458.410op/s 0.11% -1.552 3.351 0.08% 9262.047op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
credit_card/is_card_number/ execution_time [3.912µs; 3.912µs] or [-0.010%; +0.010%] None None None
credit_card/is_card_number/ throughput [255602873.725op/s; 255653501.350op/s] or [-0.010%; +0.010%] None None None
credit_card/is_card_number/ 3782-8224-6310-005 execution_time [80.732µs; 80.809µs] or [-0.048%; +0.048%] None None None
credit_card/is_card_number/ 3782-8224-6310-005 throughput [12375050.138op/s; 12386807.989op/s] or [-0.047%; +0.047%] None None None
credit_card/is_card_number/ 378282246310005 execution_time [73.628µs; 73.723µs] or [-0.065%; +0.065%] None None None
credit_card/is_card_number/ 378282246310005 throughput [13564514.540op/s; 13582038.007op/s] or [-0.065%; +0.065%] None None None
credit_card/is_card_number/37828224631 execution_time [3.912µs; 3.913µs] or [-0.010%; +0.010%] None None None
credit_card/is_card_number/37828224631 throughput [255582624.045op/s; 255632352.101op/s] or [-0.010%; +0.010%] None None None
credit_card/is_card_number/378282246310005 execution_time [70.570µs; 70.642µs] or [-0.051%; +0.051%] None None None
credit_card/is_card_number/378282246310005 throughput [14156116.286op/s; 14170460.214op/s] or [-0.051%; +0.051%] None None None
credit_card/is_card_number/37828224631000521389798 execution_time [53.121µs; 53.129µs] or [-0.008%; +0.008%] None None None
credit_card/is_card_number/37828224631000521389798 throughput [18822111.722op/s; 18824937.489op/s] or [-0.008%; +0.008%] None None None
credit_card/is_card_number/x371413321323331 execution_time [6.441µs; 6.443µs] or [-0.013%; +0.013%] None None None
credit_card/is_card_number/x371413321323331 throughput [155210011.060op/s; 155250521.631op/s] or [-0.013%; +0.013%] None None None
credit_card/is_card_number_no_luhn/ execution_time [3.913µs; 3.914µs] or [-0.012%; +0.012%] None None None
credit_card/is_card_number_no_luhn/ throughput [255498181.011op/s; 255558214.506op/s] or [-0.012%; +0.012%] None None None
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 execution_time [65.133µs; 65.170µs] or [-0.028%; +0.028%] None None None
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 throughput [15344599.162op/s; 15353306.834op/s] or [-0.028%; +0.028%] None None None
credit_card/is_card_number_no_luhn/ 378282246310005 execution_time [58.364µs; 58.396µs] or [-0.028%; +0.028%] None None None
credit_card/is_card_number_no_luhn/ 378282246310005 throughput [17124408.844op/s; 17134045.415op/s] or [-0.028%; +0.028%] None None None
credit_card/is_card_number_no_luhn/37828224631 execution_time [3.912µs; 3.913µs] or [-0.009%; +0.009%] None None None
credit_card/is_card_number_no_luhn/37828224631 throughput [255559205.125op/s; 255606747.349op/s] or [-0.009%; +0.009%] None None None
credit_card/is_card_number_no_luhn/378282246310005 execution_time [55.525µs; 55.573µs] or [-0.044%; +0.044%] None None None
credit_card/is_card_number_no_luhn/378282246310005 throughput [17994434.924op/s; 18010047.765op/s] or [-0.043%; +0.043%] None None None
credit_card/is_card_number_no_luhn/37828224631000521389798 execution_time [53.126µs; 53.137µs] or [-0.010%; +0.010%] None None None
credit_card/is_card_number_no_luhn/37828224631000521389798 throughput [18819234.677op/s; 18823180.520op/s] or [-0.010%; +0.010%] None None None
credit_card/is_card_number_no_luhn/x371413321323331 execution_time [6.438µs; 6.439µs] or [-0.012%; +0.012%] None None None
credit_card/is_card_number_no_luhn/x371413321323331 throughput [155299635.767op/s; 155335942.325op/s] or [-0.012%; +0.012%] None None None

Baseline

Omitted due to size.

@codecov-commenter
Copy link

codecov-commenter commented Mar 18, 2026 •
edited
Loading

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 70.33%. Comparing base (4cdb16c) to head (1c7a149).
⚠️ Report is 19 commits behind head on main.

Additional details and impacted files 
@@            Coverage Diff             @@
##             main    #1763      +/-   ##
==========================================
- Coverage   71.47%   70.33%   -1.14%     
==========================================
  Files         430      410      -20     
  Lines       64105    62123    -1982     
==========================================
- Hits        45820    43697    -2123     
- Misses      18285    18426     +141
Components Coverage Δ
libdd-crashtracker 64.77% <ø> (+0.83%) ⬆️
libdd-crashtracker-ffi 34.09% <ø> (+15.90%) ⬆️
libdd-alloc 98.77% <ø> (ø)
libdd-data-pipeline 87.96% <ø> (+0.03%) ⬆️
libdd-data-pipeline-ffi 75.43% <ø> (+0.58%) ⬆️
libdd-common 79.78% <ø> (+0.05%) ⬆️
libdd-common-ffi 73.87% <ø> (+0.46%) ⬆️
libdd-telemetry 62.48% <ø> (ø)
libdd-telemetry-ffi 16.75% <ø> (ø)
libdd-dogstatsd-client 82.64% <ø> (ø)
datadog-ipc 70.51% <ø> (-9.85%) ⬇️
libdd-profiling 81.60% <ø> (+0.11%) ⬆️
libdd-profiling-ffi 64.94% <ø> (+1.29%) ⬆️
datadog-sidecar 30.66% <ø> (-2.45%) ⬇️
datdog-sidecar-ffi 8.84% <ø> (-1.96%) ⬇️
spawn-worker 54.69% <ø> (ø)
libdd-tinybytes 93.16% <ø> (ø)
libdd-trace-normalization 81.71% <ø> (ø)
libdd-trace-obfuscation 92.26% <ø> (+0.46%) ⬆️
libdd-trace-protobuf 68.25% <ø> (ø)
libdd-trace-utils 88.95% <ø> (ø)
datadog-tracer-flare 86.88% <ø> (-3.59%) ⬇️
libdd-log 74.69% <ø> (ø)
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@dd-octo-sts
Copy link
Contributor

dd-octo-sts bot commented Mar 18, 2026 •
edited
Loading

Artifact Size Benchmark Report

aarch64-alpine-linux-musl
Artifact Baseline Commit Change
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.a 100.36 MB 100.36 MB 0% (0 B) 👌
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.so 8.70 MB 8.70 MB 0% (0 B) 👌
aarch64-unknown-linux-gnu
Artifact Baseline Commit Change
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.a 117.03 MB 117.03 MB 0% (0 B) 👌
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.so 11.28 MB 11.28 MB 0% (0 B) 👌
libdatadog-x64-windows
Artifact Baseline Commit Change
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.dll 27.18 MB 27.18 MB -0% (-1.50 KB) 👌
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.lib 77.17 KB 77.17 KB 0% (0 B) 👌
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.pdb 186.12 MB 186.51 MB +.21% (+408.00 KB) 🔍
/libdatadog-x64-windows/debug/static/datadog_profiling_ffi.lib 917.08 MB 917.60 MB +.05% (+533.76 KB) 🔍
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.dll 9.94 MB 9.94 MB --.01% (-1.50 KB) 💪
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.lib 77.17 KB 77.17 KB 0% (0 B) 👌
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.pdb 24.78 MB 24.78 MB 0% (0 B) 👌
/libdatadog-x64-windows/release/static/datadog_profiling_ffi.lib 51.47 MB 51.46 MB -0% (-1.68 KB) 👌
libdatadog-x86-windows
Artifact Baseline Commit Change
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.dll 22.98 MB 22.98 MB 0% (0 B) 👌
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.lib 78.37 KB 78.37 KB 0% (0 B) 👌
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.pdb 190.31 MB 190.31 MB 0% (0 B) 👌
/libdatadog-x86-windows/debug/static/datadog_profiling_ffi.lib 900.75 MB 900.75 MB 0% (0 B) 👌
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.dll 7.54 MB 7.54 MB 0% (0 B) 👌
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.lib 78.37 KB 78.37 KB 0% (0 B) 👌
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.pdb 26.52 MB 26.52 MB 0% (0 B) 👌
/libdatadog-x86-windows/release/static/datadog_profiling_ffi.lib 47.09 MB 47.09 MB 0% (0 B) 👌
x86_64-alpine-linux-musl
Artifact Baseline Commit Change
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.a 87.57 MB 87.57 MB 0% (0 B) 👌
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.so 10.22 MB 10.22 MB 0% (0 B) 👌
x86_64-unknown-linux-gnu
Artifact Baseline Commit Change
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.a 109.90 MB 109.90 MB 0% (0 B) 👌
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.so 11.97 MB 11.97 MB 0% (0 B) 👌

@datadog-official

This comment has been minimized.

Sign in to view
@iunanua iunanua changed the title ci(versioning): use commit headless to sign commits ci(versioning): release proposal workflow improvements Mar 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@hoolioh hoolioh hoolioh approved these changes

Assignees

No one assigned

Labels

ci-build

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@iunanua @codecov-commenter @hoolioh