Repository navigation
vortex-bench: run ClickBench on its first N files - #10936
Merged
Merged
Conversation
Merging this PR will improve performance by 32.4%2 warnings
Performance Changes
Tip Curious why performance improved? Comment Comparing 
Footnotes
|
Member
|
Does this change the naming of stuff? |
connortsui20
added
the
action/bench-sql
github-actions
Bot
removed
the
action/bench-sql
Contributor
Polar Signals Profiling ResultsLatest Run
Powered by Polar Signals Cloud |
Contributor
Benchmarks: PolarSignals Profiling 📖Commits: PR datafusion / vortex-file-compressed / ns (0.995x ➖, 0↑ 0↓)
No file size changes detected. |
Contributor
Benchmarks: FineWeb NVMe 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (1.016x ➖, 1↑ 2↓)
datafusion / parquet / ns (1.005x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed / ns (1.007x ➖, 0↑ 0↓)
duckdb / parquet / ns (0.997x ➖, 0↑ 0↓)
No file size changes detected. |
Contributor
Benchmarks: TPC-DS SF=1 on NVME 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (1.004x ➖, 0↑ 0↓)
datafusion / parquet / ns (0.998x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed / ns (1.002x ➖, 0↑ 0↓)
duckdb / parquet / ns (0.998x ➖, 1↑ 0↓)
No file size changes detected. |
Contributor
Benchmarks: TPC-H SF=1 on S3 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (1.227x ➖, 0↑ 5↓)
datafusion / parquet / ns (1.055x ➖, 0↑ 1↓)
duckdb / vortex-file-compressed / ns (1.049x ➖, 1↑ 5↓)
duckdb / parquet / ns (1.033x ➖, 0↑ 0↓)
|
Contributor
Benchmarks: TPC-H SF=10 on NVME 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (0.998x ➖, 0↑ 0↓)
datafusion / parquet / ns (0.998x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed / ns (1.000x ➖, 0↑ 0↓)
duckdb / parquet / ns (0.998x ➖, 0↑ 0↓)
No file size changes detected. |
Contributor
Benchmarks: TPC-H SF=1 on NVME 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (1.002x ➖, 0↑ 0↓)
datafusion / parquet / ns (1.000x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed / ns (0.999x ➖, 1↑ 0↓)
duckdb / parquet / ns (1.001x ➖, 0↑ 0↓)
No file size changes detected. |
Contributor
Benchmarks: Clickbench on NVME 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (1.000x ➖, 0↑ 0↓)
datafusion / parquet / ns (0.999x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed / ns (0.995x ➖, 0↑ 0↓)
duckdb / parquet / ns (1.004x ➖, 0↑ 0↓)
No file size changes detected. |
Member
|
yeah this needs some changes (see the errors) edit: oh it was just AWS once all the benchmarks pass we can probably just merge |
--opt files=N runs the partitioned dataset's first N of 100 files from a data directory of its own. The known per-query row counts hold only for the full dataset, so they are checked only at 100 files. Subset results carry the file count in their dataset identity; full runs keep theirs unchanged. Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: Martin Prammer <martin@spiraldb.com>
Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: Martin Prammer <martin@spiraldb.com>
mprammer
force-pushed
the
mp/clickbench-files
branch
from
October 28, 2026 19:41
3c2f118 to
93d7a0c
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds
--opt files=Nto the partitioned ClickBench benchmark, which runs the first N of its 100 files (about 1M rows each) from a data directory of its own, such asclickbench_partitioned_10. This gives ClickBench sizes to compare across, the way TPC-H has scale factors, and a small run for quick local checks. Running a subset today fails, because the hard-coded per-query row counts hold only for the full dataset: on one file, DataFusion panics at Q7 (expected 18 rows, got 5) and the run ends there.Changes
Row counts are checked only at 100 files, and the default stays at 100, so existing runs, their data paths and their result identities are unchanged. A subset's results carry its file count, as
fileson theclickbenchdataset in JSON output and asdataset_variantfiles=Nin v3 records, so they never compare against full runs.ClickBenchBenchmark::newandFlavor::downloadtake the file count;fileswith the single-file flavor is an error. Checked locally:data-gen clickbench --opt files=1downloads one file and converts it, anddatafusion-bench clickbench --opt files=1 --formats parquet,vortexruns all 43 queries on both formats.🤖 Generated with Claude Code