Skip to content

GH-50478: [C++][Compute] Support string_view/binary_view in scalar string predicate kernels#50479

Open
fangchenli wants to merge 3 commits into
apache:mainfrom
fangchenli:gh-string-view-predicates
Open

GH-50478: [C++][Compute] Support string_view/binary_view in scalar string predicate kernels#50479
fangchenli wants to merge 3 commits into
apache:mainfrom
fangchenli:gh-string-view-predicates

Conversation

@fangchenli

@fangchenli fangchenli commented Jul 11, 2026

Copy link
Copy Markdown
Contributor

Rationale for this change

The scalar string predicate/measurement kernels return NotImplemented for
string_view/binary_view input, so evaluating a string predicate (e.g. a
LIKE filter) on a view array first requires casting the whole column to
utf8/binary, a full copy. This closes that gap so the predicates run
directly on view arrays.

Part of the view-kernel effort (umbrella #44336, tracking #39634); follows
grouper keys (#50224) and cast (#50166). Take/filter (#43010) is handled
separately by #50164.

What changes are included in this PR?

Adds STRING_VIEW/BINARY_VIEW kernels for the input-view, fixed-output subset:

  • match_substring(_regex), match_like, starts_with, ends_with
  • find_substring(_regex), count_substring(_regex)
  • binary_length, utf8_length
  • string_is_ascii, ascii_is_*, utf8_is_*

Implementation notes:

  • The match-substring family and the classification predicates read every slot,
    so their view paths (an if constexpr branch in MatchSubstringImpl and in
    StringPredicateFunctor) iterate with VisitArrayValuesInline, which skips
    null slots. This matters for correctness: a null slot's view header is not
    validated (ValidateBinaryView skips nulls) and may carry a bogus
    buffer_index/offset that decoding would dereference. The base binary/string
    fast paths are unchanged.
  • The applicator-based kernels (find/count/length) already skip nulls, so
    they only needed registration entries plus a small StringOffsetType helper
    mapping the view types to Int32 output (a view element length is int32-sized).

Kernels that emit strings/lists (utf8_upper/lower/trim*,
replace_substring, split_pattern, utf8_slice_codeunits, …) are out of scope
for a follow-up, since constructing view output is the harder half.

Are these changes tested?

Yes. New TestStringViewPredicates.* cases in scalar_string_test.cc cover empty,
inlined (≤12 byte), out-of-line (>12 byte), sliced, and null values for both
binary_view and utf8_view, asserting results match the plain string/binary
semantics.

A MatchSubstringView / MatchSubstringViewCast benchmark pair quantifies
running the predicate directly on a view array vs. the pre-PR cast-to-utf8
workaround. Release build, ~1M rows of avg-16-byte ASCII, mean of 3 reps:

Benchmark Time Throughput
MatchSubstring (utf8, direct) 27.7 ms 572 MiB/s
MatchSubstringView (utf8_view, direct, this PR) 31.5 ms 503 MiB/s
MatchSubstringViewCast (view → cast to utf8 → match, pre-PR workaround) 42.2 ms 378 MiB/s

Evaluating the predicate directly on the view array is ~1.34x faster than
casting to utf8 first, by skipping the full-column copy. Direct view evaluation
is ~14% slower than native utf8 (inherent to the view layout's per-element
decode) but avoids the cast entirely.

Are there any user-facing changes?

Yes. These compute functions now accept string_view/binary_view input. The
compute.rst type-support tables are updated accordingly.

@github-actions

Copy link
Copy Markdown

⚠️ GitHub issue #50478 has been automatically assigned in GitHub to PR creator.

@fangchenli fangchenli force-pushed the gh-string-view-predicates branch 4 times, most recently from 4b94187 to 4e3a526 Compare July 12, 2026 02:44
@fangchenli fangchenli marked this pull request as ready for review July 12, 2026 15:16
@fangchenli fangchenli requested a review from pitrou as a code owner July 12, 2026 15:16
Copilot AI review requested due to automatic review settings July 12, 2026 15:16

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR extends Arrow C++ scalar compute string predicate/measurement kernels to accept string_view/binary_view inputs so predicates can run directly on view arrays (avoiding a cast-and-copy to utf8/binary), and updates docs/tests/benchmarks accordingly.

Changes:

  • Register view-type kernels for substring/like predicates, find/count, length, and classification predicates.
  • Add unit tests covering view arrays (including slicing/chunking via the shared test harness) and add benchmarks comparing direct view evaluation vs cast-to-utf8 baseline.
  • Update C++ compute documentation to reflect new input type support and output type behavior for view inputs.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
docs/source/cpp/compute.rst Documents added StringView/BinaryView support for relevant predicate/measurement kernels.
cpp/src/arrow/compute/kernels/scalar_string_utf8.cc Registers utf8_length support for utf8_view (Int32 output).
cpp/src/arrow/compute/kernels/scalar_string_internal.h Adds utf8_view registration for unary string predicates (classification/is_*) via StringPredicateFunctor.
cpp/src/arrow/compute/kernels/scalar_string_test.cc Adds TestStringViewPredicates.* coverage for view inputs across several kernels.
cpp/src/arrow/compute/kernels/scalar_string_benchmark.cc Adds benchmarks for match_substring on utf8_view and cast-to-utf8 workaround; refactors shared dataset generation.
cpp/src/arrow/compute/kernels/scalar_string_ascii.cc Implements/Registers view support for match-substring predicates and find/count/length kernels, including StringOffsetType for view outputs.

Comment thread cpp/src/arrow/compute/kernels/scalar_string_ascii.cc
Comment thread cpp/src/arrow/compute/kernels/scalar_string_internal.h
…lar string predicate kernels

Add STRING_VIEW/BINARY_VIEW support to the scalar string predicate and
measurement kernels that read a string and emit a fixed-width type:
match_substring(+regex), match_like, starts_with, ends_with,
find_substring(+regex), count_substring(+regex), binary_length,
utf8_length, string_is_ascii, and the ascii_is_*/utf8_is_* classifiers.

The matcher/predicate logic already operates on std::string_view, so the
change feeds view elements into the existing logic: MatchSubstringImpl gets
a per-element ArrayIterator branch for the view layout (the packed-offset
fast path stays for base binary/string), and the applicator-based kernels
only need registration entries plus a StringOffsetType helper mapping views
to Int32 output. Kernels that emit strings/lists are left for a follow-up.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@fangchenli fangchenli force-pushed the gh-string-view-predicates branch from 4e3a526 to 19583cc Compare July 12, 2026 16:44
Copilot AI review requested due to automatic review settings July 12, 2026 16:44

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated no new comments.

TEST(TestStringViewPredicates, MatchSubstringRegex) {
MatchSubstringOptions options{"a+"};
const auto* input = R"(["", "cat", null, "aaa banana", "concatenation"])";
for (const auto& ty : {binary_view(), utf8_view()}) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we add one utf8_view case with ignore_case=true and non-ASCII input here (for example mirroring the existing MatchSubstringIgnoreCase coverage)?

The implementation intentionally registers utf8_view as StringViewType instead of using the generic BinaryViewType dispatch, so it would be good to have a test that would fail if we accidentally lost the UTF-8/Latin1 distinction. The current view-specific tests are mostly ASCII, so they wouldn't catch that regression.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good call. Added TestStringViewPredicates.MatchSubstringIgnoreCase in 8082d31: it mirrors the existing MatchSubstringIgnoreCase coverage with pattern "aé(", ignore_case=true, and input ["abc", "aEb", "baÉ(", "aé(", "ae(", "Aé("][false, false, true, true, false, true]. The uppercase É matching lowercase é only holds because utf8_view folds over the full Unicode range; if it were dispatched through the generic BinaryViewType path (ASCII/Latin1 folding only) that slot would come back false, so the test fails on that regression.

@github-actions github-actions Bot added awaiting committer review Awaiting committer review and removed awaiting review Awaiting review labels Jul 13, 2026
fangchenli and others added 2 commits July 13, 2026 21:03
…g UTF-8/Latin1 distinction

Add a match_substring ignore_case=true case on utf8_view with non-ASCII
input (uppercase E-acute) mirroring the existing MatchSubstringIgnoreCase
coverage. utf8_view is registered as StringViewType rather than the generic
BinaryViewType dispatch, so case folding spans the full Unicode range; this
test fails if that distinction is ever lost. Requested in review by
zanmato1984.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings July 14, 2026 04:39

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated no new comments.

ASSERT_OK_AND_ASSIGN(Datum matched, CallFunction("match_substring", {arr}, &contains));
AssertDatumsEqual(Datum(expected), matched);
ASSERT_OK_AND_ASSIGN(Datum ascii, CallFunction("string_is_ascii", {arr}));
AssertDatumsEqual(Datum(expected), ascii);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Optional test coverage nit: could we also exercise one fixed-width numeric-output kernel in NullSlotWithUnvalidatedHeader, e.g. find_substring or utf8_length? The current regression covers the two boolean-output paths, while several newly registered view kernels go through the applicator path. It looks null-safe too, but this would make the null-slot header invariant explicit across both paths.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants