Repository navigation
bench: name the data directory when capturing file sizes - #10935
Merged
Merged
Conversation
capture-file-sizes.py took every directory whose name started with the benchmark's, so it also measured any other data that happened to sit next to it. The CI matrix now names each benchmark's data directory, defaulting to its subcommand, and the script measures exactly that one. Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: Martin Prammer <martin@spiraldb.com>
connortsui20
approved these changes
Oct 28, 2026
connortsui20
added
the
action/bench-sql
github-actions
Bot
removed
the
action/bench-sql
Contributor
Polar Signals Profiling ResultsLatest Run
Powered by Polar Signals Cloud |
Contributor
Benchmarks: PolarSignals Profiling 📖Commits: PR datafusion / vortex-file-compressed / ns (1.002x ➖, 0↑ 0↓)
No file size changes detected. |
Contributor
Benchmarks: TPC-DS SF=1 on NVME 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (1.002x ➖, 0↑ 0↓)
datafusion / parquet / ns (0.998x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed / ns (1.003x ➖, 0↑ 0↓)
duckdb / parquet / ns (0.999x ➖, 1↑ 0↓)
No file size changes detected. |
Contributor
Benchmarks: TPC-H SF=1 on NVME 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (0.998x ➖, 0↑ 0↓)
datafusion / parquet / ns (0.999x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed / ns (1.001x ➖, 0↑ 0↓)
duckdb / parquet / ns (0.996x ➖, 0↑ 0↓)
No file size changes detected. |
Contributor
Benchmarks: TPC-H SF=1 on S3 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (1.079x ➖, 0↑ 5↓)
datafusion / parquet / ns (0.988x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed / ns (0.987x ➖, 1↑ 2↓)
duckdb / parquet / ns (0.971x ➖, 0↑ 0↓)
|
Contributor
Benchmarks: Clickbench on NVME 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (0.992x ➖, 0↑ 0↓)
datafusion / parquet / ns (0.996x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed / ns (0.995x ➖, 0↑ 1↓)
duckdb / parquet / ns (0.989x ➖, 0↑ 0↓)
No file size changes detected. |
Contributor
Benchmarks: FineWeb NVMe 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (1.025x ➖, 0↑ 1↓)
datafusion / parquet / ns (1.003x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed / ns (1.010x ➖, 0↑ 0↓)
duckdb / parquet / ns (1.000x ➖, 0↑ 0↓)
No file size changes detected. |
Contributor
Benchmarks: TPC-H SF=10 on NVME 📖Commits: PR How to read Verdict and Engines
datafusion / vortex-file-compressed / ns (0.993x ➖, 0↑ 0↓)
datafusion / parquet / ns (0.999x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed / ns (0.995x ➖, 0↑ 0↓)
duckdb / parquet / ns (0.997x ➖, 0↑ 0↓)
No file size changes detected. |
joseph-isaacs
pushed a commit
that referenced
this pull request
Oct 29, 2026
## Summary `scripts/capture-file-sizes.py` measured every directory under `vortex-bench/data` whose name started with the benchmark's (`clickbench` matched `clickbench_partitioned`, and anything else named `clickbench_*`), so data sitting next to a benchmark's own could leak into its file-size report. The script now takes `--dataset-dir` and measures exactly that directory. ## Changes Each CI matrix entry carries a `dataset_dir`, which defaults to the benchmark's subcommand; ClickBench sets `clickbench_partitioned`. The workflow passes it through. For every benchmark the CI captures today, the script's output is byte-identical to before. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Signed-off-by: Martin Prammer <martin@spiraldb.com> Co-authored-by: Claude <noreply@anthropic.com> (cherry picked from commit 10d2d39) Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
scripts/capture-file-sizes.pymeasured every directory undervortex-bench/datawhose name started with the benchmark's (clickbenchmatchedclickbench_partitioned, and anything else namedclickbench_*), so data sitting next to a benchmark's own could leak into its file-size report. The script now takes--dataset-dirand measures exactly that directory.Changes
Each CI matrix entry carries a
dataset_dir, which defaults to the benchmark's subcommand; ClickBench setsclickbench_partitioned. The workflow passes it through. For every benchmark the CI captures today, the script's output is byte-identical to before.🤖 Generated with Claude Code