Move benchmark machines to c8gd.metal-24xl - #9761
Conversation
Signed-off-by: Robert Kruszewski <github@robertk.io>
Merging this PR will degrade performance by 4.52%
|
| Mode | Benchmark | BASE |
HEAD |
Efficiency | |
|---|---|---|---|---|---|
| ❌ | Simulation | random_i8[0.5] |
66.6 µs | 90.6 µs | -26.51% |
| ❌ | Simulation | decompress[u64, (4000, 1024)] |
70.3 µs | 85.5 µs | -17.73% |
| ⚡ | WallTime | arrow_checked_add_u32_avx2[16384] |
21.4 µs | 17.6 µs | +21.19% |
| ⚡ | Simulation | allocate_drop_arrow[0] |
456.9 ns | 402.7 ns | +13.45% |
Tip
Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.
Comparing rk/benchboxes (a42e6d5) with develop (78f4d38)
Footnotes
-
206 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩
commented
Sep 3, 2026
Codecov Report✅ All modified and coverable lines are covered by tests. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Polar Signals Profiling ResultsLatest Run
Previous Runs (13)
Powered by Polar Signals Cloud |
Benchmarks: Compression 📖Commits: PR vortex / vortex-file-compressed / ns (0.736x ✅, 24↑ 0↓)
vortex / vortex-file-compressed / bytes (1.000x ➖, 0↑ 0↓)
vortex / vortex-file-compressed / ratio (0.960x ➖, 9↑ 7↓)
vortex / parquet / ns (0.783x ✅, 30↑ 0↓)
vortex / parquet / bytes (1.000x ➖, 0↑ 0↓)
vortex / arrow-ipc / ns (0.526x ✅, 30↑ 0↓)
vortex / arrow-ipc / bytes (1.000x ➖, 0↑ 0↓)
|
Benchmarks: Clickbench Sorted on NVME 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (0.703x ✅, 9↑ 0↓)
datafusion / vortex-compact / ns (0.490x ✅, 10↑ 0↓)
datafusion / parquet / ns (0.596x ✅, 10↑ 0↓)
duckdb / vortex-file-compressed / ns (0.656x ✅, 10↑ 0↓)
duckdb / vortex-compact / ns (0.616x ✅, 10↑ 0↓)
duckdb / parquet / ns (0.565x ✅, 10↑ 0↓)
File Size Changes (200 files changed, -0.0% overall, 87↑ 113↓)
Totals:
|
Benchmarks: TPC-DS SF=1 on NVME 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (0.765x ✅, 90↑ 2↓)
datafusion / vortex-compact / ns (0.696x ✅, 95↑ 0↓)
datafusion / parquet / ns (0.848x ✅, 66↑ 2↓)
duckdb / vortex-file-compressed / ns (0.587x ✅, 99↑ 0↓)
duckdb / vortex-compact / ns (0.544x ✅, 99↑ 0↓)
duckdb / parquet / ns (0.683x ✅, 99↑ 0↓)
File Size Changes (3 files changed, -0.0% overall, 1↑ 2↓)
Totals:
|
Benchmarks: String Encoding 📖Commits: PR vortex / vortex-file-compressed / ms (0.674x ✅, 8↑ 0↓)
vortex / vortex-file-compressed / % (1.000x ➖, 0↑ 0↓)
|
Benchmarks: Statistical and Population Genetics 📖Commits: PR How to read Verdict and Engines
duckdb / vortex-file-compressed / ns (0.644x ✅, 11↑ 0↓)
duckdb / vortex-compact / ns (0.659x ✅, 11↑ 0↓)
duckdb / parquet / ns (0.720x ✅, 11↑ 0↓)
File Size Changes (1 files changed, -0.0% overall, 0↑ 1↓)
Totals:
|
Benchmarks: Clickbench on NVME 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (0.498x ✅, 43↑ 0↓)
datafusion / vortex-compact / ns (0.433x ✅, 43↑ 0↓)
datafusion / parquet / ns (0.495x ✅, 41↑ 0↓)
duckdb / vortex-file-compressed / ns (0.566x ✅, 43↑ 0↓)
duckdb / vortex-compact / ns (0.564x ✅, 43↑ 0↓)
duckdb / parquet / ns (0.537x ✅, 43↑ 0↓)
File Size Changes (83 files changed, -0.0% overall, 41↑ 42↓)
Totals:
|
Benchmarks: TPC-H SF=10 on NVME 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (0.505x ✅, 22↑ 0↓)
datafusion / vortex-compact / ns (0.456x ✅, 22↑ 0↓)
datafusion / parquet / ns (0.526x ✅, 22↑ 0↓)
duckdb / vortex-file-compressed / ns (0.513x ✅, 22↑ 0↓)
duckdb / vortex-compact / ns (0.529x ✅, 22↑ 0↓)
duckdb / parquet / ns (0.660x ✅, 22↑ 0↓)
File Size Changes (5 files changed, +0.0% overall, 4↑ 1↓)
Totals:
|
Benchmarks: PolarSignals Profiling 📖Commits: PR datafusion / vortex-file-compressed / ns (0.585x ✅, 8↑ 0↓)
No file size changes detected. |
Benchmarks: TPC-H SF=1 on NVME 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (0.750x ✅, 19↑ 1↓)
datafusion / vortex-compact / ns (0.699x ✅, 21↑ 0↓)
datafusion / parquet / ns (0.746x ✅, 21↑ 0↓)
duckdb / vortex-file-compressed / ns (0.554x ✅, 22↑ 0↓)
duckdb / vortex-compact / ns (0.545x ✅, 22↑ 0↓)
duckdb / parquet / ns (0.768x ✅, 22↑ 0↓)
File Size Changes (5 files changed, +0.0% overall, 4↑ 1↓)
Totals:
|
Benchmarks: FineWeb NVMe 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (0.469x ✅, 9↑ 0↓)
datafusion / vortex-compact / ns (0.401x ✅, 9↑ 0↓)
datafusion / parquet / ns (0.418x ✅, 8↑ 1↓)
duckdb / vortex-file-compressed / ns (0.395x ✅, 9↑ 0↓)
duckdb / vortex-compact / ns (0.467x ✅, 9↑ 0↓)
duckdb / parquet / ns (0.576x ✅, 9↑ 0↓)
File Size Changes (1 files changed, +0.0% overall, 1↑ 0↓)
Totals:
|
Benchmarks: Appian on NVME 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-compact / ns (0.560x ✅, 8↑ 0↓)
datafusion / parquet / ns (0.621x ✅, 8↑ 0↓)
duckdb / vortex-compact / ns (0.479x ✅, 8↑ 0↓)
duckdb / parquet / ns (0.477x ✅, 8↑ 0↓)
No file size changes detected. |
Benchmarks: FineWeb S3 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (0.886x ➖, 2↑ 1↓)
datafusion / vortex-compact / ns (0.776x ➖, 3↑ 1↓)
datafusion / parquet / ns (0.702x ➖, 4↑ 0↓)
duckdb / vortex-file-compressed / ns (0.958x ➖, 0↑ 0↓)
duckdb / vortex-compact / ns (0.878x ➖, 0↑ 0↓)
duckdb / parquet / ns (0.153x ✅, 9↑ 0↓)
|
Benchmarks: TPC-H SF=1 on S3 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (0.989x ➖, 1↑ 2↓)
datafusion / vortex-compact / ns (0.982x ➖, 0↑ 2↓)
datafusion / parquet / ns (0.880x ➖, 1↑ 0↓)
duckdb / vortex-file-compressed / ns (0.970x ➖, 1↑ 0↓)
duckdb / vortex-compact / ns (0.983x ➖, 0↑ 0↓)
duckdb / parquet / ns (0.909x ➖, 0↑ 0↓)
|
Benchmarks: TPC-H SF=10 on S3 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-compact / ns (0.882x ➖, 5↑ 2↓)
datafusion / parquet / ns (0.922x ➖, 2↑ 1↓)
duckdb / vortex-compact / ns (1.167x ➖, 0↑ 4↓)
duckdb / parquet / ns (0.971x ➖, 0↑ 0↓)
|
Benchmarks: Random Access 📖Commits: PR How to read Verdict and Engines
vortex / arrow-ipc / ns (0.765x ✅, 13↑ 0↓)
random-access / vortex-file-compressed / ns (0.718x ✅, 17↑ 0↓)
random-access / parquet / ns (0.468x ✅, 18↑ 0↓)
random-access / lance / ns (0.851x ✅, 15↑ 0↓)
|
c7gd use graviton3 which suffers from llvm bugs, c8gd shouldn't have this
problem