🌪 0.43.0

Changes

⚠️ Breaks

• Remove scan async API (#4005) @gatesn

✨ Features

• feat: duckdb:vortex s3 support (#4100) @0ax1
• feat: implement Cast Kernel everywhere (#4086) @lwwmanning
• feat: Make all built-in expressions Eq + replace dyn-eq with our trait (#4091) @AdamGS
• feat: Add serialization support for Select expression (#4077) @AdamGS
• zip compute function (#4031) @onursatici
• feat: teach SparseArray to canonicalize lists (#3992) @danking
• feature: Add From implementation to the vtable in addition to dedicated trait (#4037) @AdamGS
• Remove scan async API (#4005) @gatesn

🚀 Performance

• perf[vortex-array]: use all_valid instead of invalid_count() == 0 (#4108) @joseph-isaacs
• perf[duckdb]: add file footer caching (#4104) @joseph-isaacs
• varbinview zip kernel (#4054) @onursatici
• varbinview builder buffer deduplication (#4036) @onursatici
• perf[bench]: dont delete duckdb.db between db runs (#4017) @joseph-isaacs

🐛 Bug Fixes

16 changes

• fix[duckdb]: glob expansion canonicalize input path (#4121) @joseph-isaacs
• fix[duckdb]: use executor in ddb writer (#4117) @joseph-isaacs
• TransposedStream in struct writer to wake other potentially waiting streams (#4107) @onursatici
• fix[duckdb]: make batch number unique over all threads (#4095) @joseph-isaacs
• fix: mobile benchmark rendering (#4096) @lwwmanning
• fix: Add select-expr deserialization (#4082) @AdamGS
• Fix prefetch accounting (#4072) @gatesn
• fix: compress benchmarks opts doesnt conflict with output (#4070) @robert3005
• fix: benchmark compress hang (#4051) @0ax1
• fix tui segment display for auxiliary children (#4044) @robert3005
• fix: Correctly calculate the size of FlatLayouts array flatbuffer size (#4049) @robert3005
• fix: Forbid SequenceScheme for String and Float dictionary codes as well as Integer (#4048) @robert3005
• fix: duckdb ext symbol visibility (#4046) @0ax1
• fix: Implement compact for ListArray (#4002) @robert3005
• fix: fix sql benchmarks option (#4023) @danking
• Empty chunks don't affect arrays validity, empty Mask is always both all true and all false (#4018) @robert3005

🧰 Maintenance

57 changes

• IO Fuzzer uses single threaded reader and writer (#4119) @robert3005
• chore: Add workflow to minimize fuzz corpuses (#4120) @robert3005
• chore(deps): update plugin com.gradleup.shadow to v8.3.9 (#4115) @renovate[bot]
• fix(deps): update dependency org.immutables:value to v2.11.2 (#4112) @renovate[bot]
• chore[bench]: nightly s3 vortex tpc-h(sf=100) (#4111) @joseph-isaacs
• Revert "feat: implement Cast Kernel everywhere" (#4109) @joseph-isaacs
• chore: ArbitraryArray::arbitrary_with (#4105) @blaginin
• chore(deps): lock file maintenance (#4103) @renovate[bot]
• chore[bench-site]: more mobile fixes (#4102) @lwwmanning
• chore[bench-site]: bench site CSS cleanup (#4099) @lwwmanning
• chore[bench-website]: adjust benchmark size and usability features (#4097) @joseph-isaacs
• chore[bench-site]: add aggregates of each group to bench site (#4098) @lwwmanning
• chore: fix some typos (#4088) @XinyuZeng
• chore[bench]: update vortex bench color (#4094) @joseph-isaacs
• chore(deps): update plugin com.palantir.consistent-versions to v2.37.0 (#4093) @renovate[bot]
• chore(deps): update all patch updates (patch) (#4092) @renovate[bot]
• chore: C++ API improvement (#4050) @XinyuZeng
• chore(deps): update rust crate serde_json to v1.0.142 (#4087) @renovate[bot]
• chore: make bench site both responsive and faster (#4084) @lwwmanning
• chore: stricter yaml validation (#4078) @robert3005
• chore: Add roundtrip serialization tests to vortex-expr (#4083) @AdamGS
• chore: rewrite the benchmark website (#4079) @lwwmanning
• chore: Enable polarsignals diffs on benchmarks (#4067) @AdamGS
• chore: Make vortex pretty (#4081) @AdamGS
• chore: No longer need to mount debugfs and tracefs (#4080) @robert3005
• chore: Manually mount tracefs and debugfs until we bump ami (#4076) @robert3005
• chore(deps): update rust crate divan to v3.0.5 (#4073) @renovate[bot]
• [chore] improve binary_numeric conformance (#4069) @lwwmanning
• Clean up decompress benchmarks (#4068) @gatesn
• [chore] cross-operation consistency tests (#4055) @lwwmanning
• chore: cleanup commented-out code in vortex-expr (#4066) @AdamGS
• chore: add into_cpu_stream (#4053) @blaginin
• test: block_in_place (#4059) @0ax1
• chore(deps): update rust crate clap to v4.5.42 (#4060) @renovate[bot]
• chore(deps): update rust crate crossterm to 0.29 (#2925) @renovate[bot]
• chore: reject warnings on clippy but not on check (#4052) @blaginin
• Clean up stats propagation for slicing (#3356) @gatesn
• [chore] conformance testing for TakeFn (#4043) @lwwmanning
• [chore] use rstest for filter & mask conformance test harnesses (#4045) @lwwmanning
• chore: add missing {doctest} (#4042) @blaginin
• [chore] add filter conformance test & fill existing conformance testing gaps (#4041) @lwwmanning
• chore: cleanup COMPUTE_FN definitions (#4040) @a10y
• docs: Document all canonical array types (#4039) @a10y
• chore: Timeout data downloads in benchmarks (#4038) @AdamGS
• use arc slice for buffers in varbinview (#4035) @onursatici
• chore: bump datafusion to 49 (#4006) @AdamGS
• chore(deps): lock file maintenance (#4030) @renovate[bot]
• chore(deps): update rust crate rstest to v0.26.1 (#4029) @renovate[bot]
• chore(deps): update rust crate rstest to 0.26 (#4026) @renovate[bot]
• chore(deps): update rust crate tokio to v1.47.0 (#4025) @renovate[bot]
• chore(deps): update rust crate roaring to v0.11.2 (#4024) @renovate[bot]
• chore: benchmark individual encoding throughputs (#4014) @lwwmanning
• chore: optionally delete the duckdb database on engine creation. (#4022) @danking
• chore: Install llvm in fuzzer for slightly better logging and remove submodules from checkout (#4015) @robert3005
• Remove segment source factory (#4016) @gatesn
• Add Claude Code GitHub Workflow (#4009) @robert3005
• chore: unit tests for Arrow conversion (#4007) @a10y

🌪 0.42.1

Changes

🐛 Bug Fixes

• fix[ci]: use cargo +nightly in publish (#4011) @joseph-isaacs

🌪 0.42.0

Changes

⚠️ Breaks

• chore: Add new_expr to all built-in expressions in the name of consistency (#3847) @AdamGS

✨ Features

• feat[duckdb]: impl get_partition_data (#4008) @joseph-isaacs
• feat: teach SparseArray to canonicalize Binary arrays (#3993) @danking
• Add rayon parallel iterators for scanning (#3983) @gatesn
• feat[duckdb]: add file pruning and fix scanner (#3986) @joseph-isaacs
• Add a work-stealing ArrayIterator to drive scans (#3982) @gatesn
• feature: Add IsConstant for DecimalBytePartsArray (#3972) @robert3005
• feat: Add traversal logic for more containers (#3970) @AdamGS
• feat: Refactor traversal logic (#3926) @AdamGS
• DuckDB Table Filters (#3951) @gatesn
• feat: duckdb workstealing (#3927) @0ax1
• feat: use VarBinViewBuilder internally (#3925) @a10y
• Vortex - C++ API (#3792) @XinyuZeng
• feat: display as table (#3906) @blaginin
• propagate spans in dispatcher (#3916) @onursatici
• feat: duckdb global conversion cache (#3901) @0ax1
• feat: impl optimize for more arrays (#3909) @a10y
• feat: call optimize in compressor (#3852) @a10y
• feat: build with stable rust (#3881) @a10y
• feature: multi-file-scan thread-safe iterator (#3863) @0ax1
• Compact Compression (#3765) @mwlon
• Arrow iterator conversions (#3875) @gatesn
• feat: display (#3866) @danking
• feat[bench]: inter-query memory tracking (#3861) @joseph-isaacs
• feat: Minimal support for extension array casting (#3851) @AdamGS
• feat[compressor]: wire up sequence array into the compressor (#3844) @joseph-isaacs
• feat: teach utf8-type SparseArray to canonicalize (#3848) @danking
• feat: teach SparseArray to canonicalize Null & Decimal (#3662) @danking
• feat: Add optimize ArrayOp with VBView implementation (#3825) @a10y
• Row idx layout reader (#3819) @gatesn
• feature: Support overriding layout verification settings via environment variables (#3794) @robert3005
• Add expression transformation to replace all occurrences of a node (#3795) @gatesn
• VortexExpr VTables (#3713) @gatesn

🚀 Performance

• DuckDB Dynamic Expressions (#3966) @gatesn
• Add cardinality callback to DuckDB TableFunction (#3941) @gatesn
• perf: fine tune workstealing (#3938) @0ax1
• perf[vortex-duckdb]: set dictionary id to allow hash value once opt (#3621) @joseph-isaacs

🐛 Bug Fixes

35 changes

• fix: unify duckdb timestamps on i64 (#4004) @0ax1
• fix: Decimal256 from arrow casts the array to Decimal256Type (#4003) @robert3005
• fix: increment num_factories_constructed on error (#4001) @0ax1
• fix: Pruning expressions check NanCount where appropriate (#3973) @a10y
• fix: Make NotExpr display more explicit (#3998) @AdamGS
• fix[duckdb]: handle empty file globs (#3994) @joseph-isaacs
• fix: list array builder respects sliced list arrays (#3991) @danking
• feat[duckdb]: add file pruning and fix scanner (#3986) @joseph-isaacs
• fix: correctly expose gen_test_data cxx ffi (#3969) @XinyuZeng
• prevent double stealing (#3984) @onursatici
• fix: Prevent compressor from choosing SequenceArray for dictionary codes (#3979) @robert3005
• Fix DuckDB timestamp values (#3965) @gatesn
• Fix data_wrapper pointer (#3955) @gatesn
• fix: remove optimize Op, replace with compact_buffers function (#3931) @a10y
• fix: nbytes returns a u64 (#3924) @robert3005
• fix: duckdb multithreaded scan (#3920) @0ax1
• fix: duckdb scan cleanup (#3912) @0ax1
• fix: Fix NNF nested negation rewriting (#3905) @AdamGS
• fix: DictMaskEvaluation to respect validity (#3894) @blaginin
• fix: StructArray with empty fields is considered constant (#3895) @robert3005
• fix[layout]: row id layout reader cannot stop if inner layout registering its splits (#3867) @joseph-isaacs
• fix: Drain prefetch buffer (#3856) @a10y
• fix: teach SparseArray to account for null values when the fill is null (#3846) @danking
• fix[bench-website]: allow sf1000 and missing results (#3849) @joseph-isaacs
• fix[bench]: website render data with scale factor as string (#3838) @joseph-isaacs
• fix[bench]: correct s3 path to run benchmarks (#3837) @joseph-isaacs
• fix[bench]: nightly config that doesn't fail (#3830) @joseph-isaacs
• Fix empty take for ChunkedArray (#3823) @blaginin
• fix: Propagate executor to scan tasks (#3815) @AdamGS
• fix: Add test for using correct self._dataset variable for scanner schema (#3812) @NeejWeej
• fix: Remove redundant copy from delta compression (#3797) @robert3005
• fix: make Java test gradle self-contained (#3791) @a10y
• fix: DecimalByteParts compare with overflow and nullable elements (#3789) @a10y
• fix: STRIDE for u8/i32 AVX2 take kernel (#3771) @a10y
• fix: Correct Iterator::size_hint and ExactSizeIterator implementation for FieldNames (#3768) @AdamGS

🧰 Maintenance

110 changes

• docs: StructFields behavior (#4000) @a10y
• chore: Add some tests and docs to layouts and duckdb (#3999) @AdamGS
• chore[duckdb]: scan log info -> trace (#3989) @joseph-isaacs
• chore: RunEndArray ends must be unsigned so only enumerate unsigned ptypes for decompression (#3985) @robert3005
• chore: duckdb decimal to arrow conversion (#3980) @0ax1
• Extract work queue logic (#3978) @gatesn
• chore: check Struct Scalar dtype on creation (#3974) @blaginin
• chore: Convert cases of Buffer::extend to using extend_trusted (#3971) @robert3005
• chore(deps): update plugin com.diffplug.spotless to v7.2.1 (#3961) @renovate[bot]
• chore[bench]: assert partial taxi take result correctness (#3948) @joseph-isaacs
• chore: Implement FileSource::fmt_extra for VortexSource to display predicates in query plans (#3959) @robert3005
• Setup JDK in CI (#3967) @gatesn
• Use c-unwrap ABI for duckdb extension (#3956) @gatesn
• chore: All runs-on workflows have unique tags (#3957) @robert3005
• Improve work stealing loop (#3946) @gatesn
• chore(deps): update rust crate divan to v3.0.4 (#3950) @renovate[bot]
• chore: Propagate traces to task executor and a bunch of other places (#3947) @AdamGS
• chore(deps): update junit-framework monorepo to v5.13.4 (patch) (#3945) @renovate[bot]
• chore(deps): lock file maintenance (#3944) @renovate[bot]
• chore(deps): update rust crate rand to v0.9.2 (#3942) @renovate[bot]
• chore(deps): update plugin com.diffplug.spotless to v7.2.0 (#3943) @renovate[bot]
• chore(deps): update rust crate strum to v0.27.2 (#3939) @renovate[bot]
• fix(deps): update all patch updates to v1.0.161 (patch) (#3933) @renovate[bot]
• chore(deps): update rust crate serde_json to v1.0.141 (#3932) @renovate[bot]
• chore: fix type bound on Scalar::binary (#3930) @a10y
• chore: Fix c++ ci after it's a required check (#3928) @robert3005
• chore(deps): update all patch updates (patch) (#3910) @renovate[bot]
• chore: Enable workspace lints on vortex-scan (#3923) @robert3005
• Add BtrBlocksCompressor decision tree (#3921) @robert3005
• chore: Disable yamlint warnings (#3915) @AdamGS
• chore: Disable code coverage for arbitrary-related code (#3913) @AdamGS
• chore: Migrate NNF doctests to actual tests (#3911) @AdamGS
• chore: Fix fmt-failing duckdb code (#3914) @AdamGS
• chore: exclude arbitrary from coverage (#3908) @lwwmanning
• chore[docs]: deny missing_docs on vortex-scalar, vortex-utils, & vortex-flatbuffers (#3907) @lwwmanning
• feat[bench-vortex]: benchmark vortex compact format (#3886) @joseph-isaacs
• Use ptr::default in Rust 1.88 (#3896) @gatesn
• [chore] Update README.md (#3893) @lwwmanning
• chore[vortex-duckdb]: allow C++ 20 ext syntax (#3884) @joseph-isaacs
• duckdb build script http timeout (#3888) @onursatici
• chore[vortex-scan]: clippy default MultiScan (#3889) @joseph-isaacs
• chore(deps): update all patch updates (patch) (#3887) @renovate[bot]
• chore(config): migrate renovate config (#3883) @renovate[bot]
• chore: Don't generate compare expressions for lists when fuzzing (#3882) @robert3005
• fix(deps): update dependency org.immutables:value to v2.11.1 (#3880) @renovate[bot]
• chore: Consitent naming for display functions (#3871) @robert3005
• chore(deps): update rust crate fastlanes to 0.2.0 (#3878) @renovate[bot]
• chore[bench]: move tpcds into new benchmark format (#3870) @joseph-isaacs
• fix: Fuzzer does not generate filter on struct (#3873) @a10y
• chore(deps): update plugin com.palantir.consistent-versions to v2.36.0 (#3872) @renovate[bot]
• Fix intra-doc links (#3869) @gatesn
• chore[layout]: empty pack round trip scan (#3868) @joseph-isaacs
• chore: Replace dependency on top level arrow create with individual modules now that pyarrow is a standalone crate (#3864) @robert3005
• chore: Bump prost dependency (#3862) @AdamGS
• chore(deps): update plugin com.vanniktech.maven.publish to v0.34.0 (#3860) @renovate[bot]
• chore(deps): lock file maintenance (#3857) @renovate[bot]
• chore(deps): update rust crate memmap2 to v0.9.7 (#3855) @renovate[bot]
• Move array context inside FlatLayout (#3854) @gatesn
• chore: better naming for SendableLayoutFuture (#3843) @a10y
• chore: pull struct case of sparse to_canonical into function (#3850) @danking
• chore(deps): update rust crate divan to v3.0.3 (#3845) @renovate[bot]
• chore[fuzz]: smaller machine to save run cost m8g.2xlarge (#3842) @joseph-...

🌪 0.41.2

Changes

🧰 Maintenance

• chore: java and python publish do not block on Rust publish (#3762) @a10y
• chore: Update OpenSSF badge (#3761) @lwwmanning

🌪 0.41.1

Changes

• chore[ci]: log what data we are generating (#3759) @joseph-isaacs
• chore(deps): update rust crate target-lexicon to 0.13 (#3757) @renovate[bot]

🐛 Bug Fixes

• fix[benchmark]: nightly run input (#3756) @joseph-isaacs

🧰 Maintenance

• ci: noverify on publish (#3760) @a10y
• chore[benchmark]: attach env triple to benchmarks measurements (#3755) @joseph-isaacs
• chore[benchmark]: workflow_dispatch for nightly bench (#3754) @joseph-isaacs

🌪 0.41.0

Changes

• feat: simplify again after removing select & merge (#3683) @danking

⚠️ Breaks

• chore: remove unused worker dep from repo (#3721) @a10y
• feat: Move the StructFields arc inside (#3686) @AdamGS

✨ Features

• feat: migrate take_primitive_simd to stable AVX2 kernel (#3579) @a10y
• feat: Hide as many of FieldNames' Arcs as possible (#3747) @AdamGS
• feat: public constants for row id fields (#3735) @danking
• feat[vortex-duckdb]: add copy from function bindings and vortex impl (#3693) @joseph-isaacs
• feat: teach ScanBuilder with_row_index (#3605) @danking
• feat: teach StructFields to Default to empty (#3679) @danking
• feat: teach StructFields empty() (#3678) @danking
• Add scope variables to expression scope (#3672) @gatesn
• feat: multiversion for primitive take (#3665) @a10y
• feat: teach SparseArray to canoncialize structs (#3651) @danking

🐛 Bug Fixes

15 changes

• fix: Make all JNI types support idempotent close (#3752) @a10y
• fix: teach StructFieldExpressionSplitter the scope of its root expr (#3743) @danking
• fix: Handle DictArray::fill_null with nullable codes where fill is in the dictionary (#3740) @robert3005
• fix: column type leak (#3729) @0ax1
• fix: fix decimal byte parts compare & duckdb as_decimal (#3726) @joseph-isaacs
• fix: SparseArray handles take with null indices if the fill value is not null (#3677) @robert3005
• fix: skip tpch q6 for duckdb:vortex (#3700) @0ax1
• fix: Bump polars dependency and fix expr conversion (#3697) @AdamGS
• fix: duckdb scalar date and time conversion (#3690) @0ax1
• fix: FFI example fixed up, hooked into CI (#3684) @a10y
• fix[vortex-array]: check NaN for inferring is constant with min/max (#3663) @joseph-isaacs
• fix[fuzz]: file_io apply filter + proj to create expected array (#3658) @joseph-isaacs
• fix[vortex-layout]: correctly write struct (w/o field) length to layout (#3657) @joseph-isaacs
• fix: table func parameters leak (#3655) @0ax1
• dict layout writer to allow empty input stream (#3652) @onursatici

🧰 Maintenance

57 changes

• chore[benchmark]: add scale factor 100 (#3751) @joseph-isaacs
• chore: CastKernel for List (#3748) @blaginin
• chore[benchmark]: support scale factors other than 1 in tpch (#3746) @joseph-isaacs
• chore[benchmark]: use q15a (from tpch) WITH not VIEW (#3749) @joseph-isaacs
• chore: make flavor part of clickbench dataset (#3745) @0ax1
• chore[benchmark]: add -o option to specify where the write results (#3739) @joseph-isaacs
• chore: run duckdb:vortex on clickbench (#3741) @0ax1
• chore: update toolchain to nightly-2025-06-26 (#3742) @danking
• chore[benchmark]: enable duckdb:vortex for tpch (local) (#3738) @joseph-isaacs
• chore[vortex-duckdb]: catch error by ref (#3737) @joseph-isaacs
• chore: Remove setup c++ action (#3734) @robert3005
• chore(deps): update plugin org.gradle.toolchains.foojay-resolver to v1 (#3732) @renovate[bot]
• chore(deps): update plugin com.palantir.git-version to v4 (#3731) @renovate[bot]
• chore: drop setup-c++ step (#3733) @0ax1
• fix(deps): update protobuf monorepo to v4.31.1 (minor) (#3728) @renovate[bot]
• fix(deps): update org.apache.arrow:* to v18.3.0 (minor) (#3727) @renovate[bot]
• chore(deps): update rust crate bzip2 to 0.6.0 (#3712) @renovate[bot]
• fix(deps): update dependency org.immutables:value to v2.11.0 (#3725) @renovate[bot]
• chore: remove unused worker dep from repo (#3721) @a10y
• chore(deps): update rust crate tokio to v1.46.0 (#3722) @renovate[bot]
• chore(deps): update spiraldb/actions action to v0.14.0 (#3723) @renovate[bot]
• chore(deps): update rust crate worker to 0.6.0 (#3719) @renovate[bot]
• chore(deps): update rust crate tabled to 0.20.0 (#3718) @renovate[bot]
• chore(deps): update rust crate strum to 0.27 (#3717) @renovate[bot]
• chore(deps): update rust crate roaring to 0.11.0 (#3716) @renovate[bot]
• chore(deps): update rust crate compio to 0.15 (#3714) @renovate[bot]
• chore(deps): update rust crate bindgen to 0.72.0 (#3711) @renovate[bot]
• chore(deps): update rust crate reqwest to v0.12.22 (#3709) @renovate[bot]
• chore: rename error => error_out (#3708) @0ax1
• chore: Make sure that workspace is clean after build (#3707) @robert3005
• chore: auto-gen c header via bindgen for vortex-ffi (#3704) @0ax1
• chore(deps): update plugin net.ltgt.errorprone to v4.3.0 (#3706) @renovate[bot]
• chore(deps): update plugin com.vanniktech.maven.publish to v0.33.0 (#3705) @renovate[bot]
• chore(deps): update plugin com.palantir.git-version to v3.4.0 (#3703) @renovate[bot]
• chore(deps): update plugin com.gradleup.shadow to v8.3.8 (#3702) @renovate[bot]
• chore: polish benchmark row count verification (#3701) @0ax1
• chore(deps): lock file maintenance (#3695) @renovate[bot]
• chore(deps): update org.junit.jupiter:* to v5.13.2 (minor) (#3699) @renovate[bot]
• chore(deps): update opentelemetry-rust monorepo to 0.30.0 (minor) (#3698) @renovate[bot]
• chore(config): migrate renovate config (#3696) @renovate[bot]
• chore: Apply default label to renovate PRs (#3694) @AdamGS
• chore(deps): update octokit/request-action action to v2.4.0 (#3692) @renovate[bot]
• fix(deps): update all patch updates (patch) (#3691) @renovate[bot]
• chore: Manually bump cargo dependencies (#3689) @robert3005
• Use config:recommended instead of config:base (#3688) @robert3005
• chore: Nicer taplo config (#3687) @AdamGS
• chore: Move allowed casting from vortex-array to fuzzer (#3682) @robert3005
• chore: move tpch to duckdb vortex v2 (#3680) @0ax1
• chore: drop fn name suffix (#3675) @0ax1
• chore: drop obsolete vortex-duckdb_v1 crate (#3676) @0ax1
• chore: Remove Patches#into_parts (#3673) @robert3005
• chore: Set DuckDB submodule URL to https (#3670) @AdamGS
• chore: coveralls ignore generated/language-binding code (#3668) @lwwmanning
• chore: Update CONTRIBUTING.md (#3667) @lwwmanning
• Run clickbench file download on a local rayon pool (#3661) @robert3005
• chore: look mom, no forks (#3666) @0ax1
• chore: Always use primitive array simd take (#3647) @robert3005

🌪 0.40.0

Changes

• bug[vortex-array]: take nullability chunked array not respects validity (#3594) @joseph-isaacs
• bug[vortex-layout]: file_row_number & file_index i64-> u64 in type signature (#3587) @joseph-isaacs
• bug[decimal]: to arrow use precision not scale to decide arrow type (#3577) @joseph-isaacs
• fix: add default fields when canonicalize-ing nullable structs (#3551) @blaginin
• docs: clarify docs of the minimum and maximum compute kernels (#3507) @danking

✨ Features

• feat: teach ScopeDType to Display (#3616) @danking
• feat[vortex-expr]: CheckedAdd operator (#3603) @danking
• feat: pack & merge display their nullability (#3599) @danking
• feat[vortex-duckdb-ext]: add vx duckdb type conversion (#3588) @joseph-isaacs
• feat[vortex-duckdb-ext]: support enough scalar expr and array conversion for tpch & clickbench (#3570) @joseph-isaacs
• feat[vortex-duckdb-ext]: remove arrow-rs export and use arrow-rs fork with decimal32/64 (#3564) @joseph-isaacs
• chore: remove most nightly features usage (#3549) @a10y
• feat[vortex-dtype]: vx struct and list dtype ctors (#3518) @joseph-isaacs
• feat[vortex-duckdb-ext]: add more expression conversion from duckdb to vx (#3556) @joseph-isaacs
• feat: add StructArray::remove_column (#3540) @blaginin
• feat[vortex-array]: add list contains compute fn (#3528) @joseph-isaacs
• feat[vortex-duckdb-ext]: scan with table filter (#3534) @joseph-isaacs
• Pco encoding (#3525) @mwlon
• feat: do not swallow errors from streams being written (#3524) @danking
• feat[duckdb-vortex]: Add sequence vector exporter (#3522) @joseph-isaacs
• feat[vortex-layout]: add row_id layout reader to support row_id expr queries (#3449) @joseph-isaacs

🚀 Performance

• perf[vortex-duckdb-ext]: add column pruning/projection (#3548) @joseph-isaacs

🐛 Bug Fixes

26 changes

• fix[encodings]: decimal-byte-parts-array compare when scalar is larger than any value in the array (#3650) @joseph-isaacs
• fix: TableFunction leak (#3648) @0ax1
• fix: scalar varchar leak (#3649) @0ax1
• fix[vortex-array]: struct validity + runend validity (#3645) @joseph-isaacs
• fix: bound column name ref leak (#3646) @0ax1
• list is_constant (#3637) @onursatici
• chore: move ci to duckdb v1.3.1 (#3636) @0ax1
• bug[vortex-array]: take compare nullable bugs (#3628) @joseph-isaacs
• fix: duckdb_logical_type leak (#3630) @0ax1
• fix: duckdb varchar leak (#3631) @0ax1
• fix: don't leak selection vector (#3629) @0ax1
• fix: duckdb vx ext memory leak (#3627) @0ax1
• bug[fuzzer]: fix varbin take baseline (#3625) @joseph-isaacs
• fix[vortex-array]: varbin take nullability (#3624) @joseph-isaacs
• Fix: Handle DateTimeParts compare with nullable rhs and non nullable lhs (#3620) @robert3005
• fix: skip ddb vortex in clickbench (#3615) @0ax1
• fix: drop ddb skip build flag (#3614) @0ax1
• bug[vortex-duckdb-ext]: fix the scan termination condition (#3606) @joseph-isaacs
• fix[fuzzer/vortex-array]: use optional indicies in fuzzer take/Decimal::take (#3597) @joseph-isaacs
• fix: make dylib rpath available for all targets (#3590) @0ax1
• docs: fix DType docs (#3572) @a10y
• fix: taiki-e/install-action nextest install (#3559) @0ax1
• fix: glob expansion & termination condition (#3554) @0ax1
• chore: fix table comment to be real markdown (#3552) @a10y
• fix: vortex-flatbuffers docs has bad links (#3542) @danking
• fix: teach dict layout how to evaluate expressions with non-trivial nullability (#3535) @danking

🧰 Maintenance

52 changes

• docs: more docs around AnalysisExpr and boolean operators (#3508) @danking
• chore: Add setup-rust to fuzz workflow (#3644) @robert3005
• chore: default_value for nullable dtypes (#3643) @blaginin
• chore: Do we need gzip install? (#3642) @robert3005
• Fix: delay getting benchmarks s3 token until after binary has been built (#3638) @robert3005
• Disable auto-labeler (#3635) @gatesn
• bug[fuzzer]: fuzzer only infallible casts (for now). (#3626) @joseph-isaacs
• chore: Enable runs-on s3 magic cache (#3623) @robert3005
• bug[fuzzer]: fix varbin take baseline (#3625) @joseph-isaacs
• chore: drop unused crate deps (#3619) @0ax1
• chore: rename vortex duckdb crates (#3618) @0ax1
• chore: nuke duckdb-rs bundled build (#3617) @0ax1
• chore: add try_from_iter_with_validity (#3608) @blaginin
• chore: Rename CheckedAdd operator to Add (#3612) @robert3005
• chore: kill duckdb ci build tasks (#3602) @0ax1
• chore: rename references to python package (#3607) @AdamGS
• chore[vortex-expr]: remove let expr (#3604) @joseph-isaacs
• chore[vortex-expr]: move core expressions into their own module (#3596) @joseph-isaacs
• chore: More idiomatic display impls for expressions (#3601) @robert3005
• duckdb: marshal query results via C API (#3593) @0ax1
• chore[fuzzer]: move array and file ops into their own modules (#3595) @joseph-isaacs
• chore[fuzzer]: add projection and filter expressions to the fuzzer (#3586) @joseph-isaacs
• chore[fuzzer]: add array casting op (#3583) @joseph-isaacs
• chore: simplify rpath handling (#3592) @0ax1
• duckdb: switch to v2 extension for clickbench benchmark (#3585) @0ax1
• chore[fuzzer]: update readme (#3581) @joseph-isaacs
• chore: explicit error if struct is different from schema (#3589) @blaginin
• chore[fuzzer]: nullable take indices and compare scalar (#3582) @joseph-isaacs
• chore: add StructFields::merge (#3584) @blaginin
• bug[decimal-byte-parts]: unconvertible scalar in compare (#3578) @joseph-isaacs
• chore[fuzzer]: add u256 to implemented Int for i256 correctly (#3576) @joseph-isaacs
• chore[fuzz]: added decimal array ops (#3571) @joseph-isaacs
• Chore: Implement std::io::Write for ByteBufferMut (#3566) @robert3005
• chore: Add IntoArray for TemporalArray (#3568) @robert3005
• datafusion 48 (#3560) @onursatici
• chore: remove most nightly features usage (#3549) @a10y
• chore: Add Field::as_name and remove unused From impls (#3565) @robert3005
• chore: StructFields field access returns option instead of result (#3530) @robert3005
• chore: Cleanup some effectively unused deps (#3563) @AdamGS
• chore[vortex-duckdb-ext]: reformat files using new clang format (#3561) @joseph-isaacs
• duckdb: reuse first file opened in bind (#3558) @0ax1
• duckdb: factor out fns from init_global (#3557) @0ax1
• chore: panic on nullable top-level struct (#3550) @blaginin
• chore: fix table comment to be real markdown (#3552) @a10y
• chore: add test file generation for Java (#3543) @a10y
• duckdb: impl clone for bind data (#3541) @0ax1
• chore: update file format docs, including diagrams (#3545) @lwwmanning
• Zstd encoding improvements (#3537) @mwlon
• duckdb: file list via glob (#3532) @0ax1
• chore: followup to #3522 (#3529) @joseph-isaacs
• duckdb: fall back to arrow exporter (#3527) @0ax1
• chore: scope dyn symbol resolution to vortex-python in build.rs (#3526) @0ax1

🌪 0.39.0

Changes

✨ Features

• feat[vortex-expr]: remove pruning expr in favor of checked_pruning_expr. (#3501) @joseph-isaacs
• duckdb: temporal, list & decimal dtype conversions (#3517) @0ax1
• Zstd encoding (#3456) @mwlon
• feat: Initial limit pushdown in the scan (#3483) @AdamGS
• feat[encodings/run-end]: is constant run fn end (#3511) @joseph-isaacs

🚀 Performance

• stats calculation parallelism in layout writer (#3500) @onursatici

🐛 Bug Fixes

• fix: Expose Decimal Literal over Java (#3521) @a10y
• fix: move back linker flags to global config toml (#3523) @0ax1

🧰 Maintenance

4 changes

• duckdb: conversion docs (#3519) @0ax1
• feat[vortex-expr]: remove pruning expr in favor of checked_pruning_expr. (#3501) @joseph-isaacs
• docs: improve public API docs (part 1/N) (#3347) @a10y
• chore: update README.md (#3516) @lwwmanning

🌪 0.38.0

Changes

✨ Features

• feat: scope dynamic symbol resolution to vortex-python (#3515) @0ax1
• duckdb: basic vortex scan (#3513) @0ax1
• duckdb: extract schema in bind function (#3502) @0ax1
• feat: use tree display for df emit plan (#3510) @joseph-isaacs
• feat: link duckdb dynamically (#3506) @0ax1

🐛 Bug Fixes

• fix: row_range vs row_indices pushdown in JNI (#3514) @a10y

🧰 Maintenance

• feat: link duckdb dynamically (#3506) @0ax1

🌪 0.37.2

Changes

🐛 Bug Fixes

• fix: deserialize ident proto expressions as the new Var("") (#3509) @a10y