Skip to content

GH-4160: Apply the filter-disjunction rewrite only when the disjuncts are mutually exclusive - #4161

Open
faubulous wants to merge 1 commit into
apache:mainfrom
faubulous:fix-filter-disjunction-duplicates
Open

GH-4160: Apply the filter-disjunction rewrite only when the disjuncts are mutually exclusive#4161
faubulous wants to merge 1 commit into
apache:mainfrom
faubulous:fix-filter-disjunction-duplicates

Conversation

@faubulous

Copy link
Copy Markdown

GitHub issue resolved #4160

Pull request Description:

TransformFilterDisjunction rewrites filter(e1 || e2, P) into a disjunction that evaluates P once per disjunct, so a solution satisfying k disjuncts is returned k times where the filter returns it once. FILTER(?x = :c || ?x = :c) returns every matching row twice under the default optimizer, and FILTER(?x = :c || ?x != :d) returns rows with ?x = :c twice; disabling optFilterDisjunction restores the correct multiset.

Two changes together restore filter semantics:

  • Repeated disjuncts are dropped first(A || A) is A — so the degenerate duplicate that generated queries really contain (LDBC SPB emits FILTER(?pf = :c || ?pf = :c)) collapses to a single equality that grounds the pattern, rather than being either doubled (before) or left unoptimized (declined).
  • The expansion is applied only when the disjuncts are mutually exclusive: every remaining disjunct must test one and the same variable against a constant (= or sameTerm) and the constants must be pairwise known to be different values (NodeValue.notSameValueAs, with an indeterminate comparison treated as possibly equal). That makes the rewrite an exact partition. The motivating case — ?x IN (...) over distinct constants, including mixed IRI/literal lists — keeps its expansion; every other disjunction is now evaluated as the filter it is.

Several existing algebra tests pinned the unsound expansions; their expectation is now that the filter is left alone.


  • Tests are included.
  • Documentation change and updates are provided for the Apache Jena website
  • Commits have been squashed to remove intermediate development commit messages.
  • Key commit messages start with the issue number (GH-xxxx)

By submitting this pull request, I acknowledge that I am making a contribution to the Apache Software Foundation under the terms and conditions of the Contributor's Agreement.

…juncts are mutually exclusive

TransformFilterDisjunction rewrites filter(e1 || e2, P) into a
disjunction that evaluates P once per disjunct, so a solution
satisfying k disjuncts is returned k times where the filter returns it
once. FILTER(?x = :c || ?x = :c) returns every matching row twice
under the default optimizer, and FILTER(?x = :c || ?x != :d) returns
rows with ?x = :c twice; disabling optFilterDisjunction restores the
correct multiset. Several existing algebra tests pinned the unsound
expansions; their expectation is now that the filter is left alone.

Two changes together restore filter semantics:

Repeated disjuncts are dropped first — (A || A) is A — so the
degenerate duplicate that generated queries really contain (LDBC SPB
emits FILTER(?pf = :c || ?pf = :c)) collapses to a single equality
that grounds the pattern, rather than being either doubled (before) or
left unoptimized (declined).

The expansion is then applied only when every remaining disjunct tests
one and the same variable against a constant (= or sameTerm) and the
constants are pairwise known to be different values
(NodeValue.notSameValueAs, with an indeterminate comparison treated as
possibly equal), which makes the disjuncts mutually exclusive and the
rewrite an exact partition. The motivating case — ?x IN (...) over
distinct constants, including mixed IRI/literal lists — keeps its
expansion; every other disjunction is now evaluated as the filter it
is.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

TransformFilterDisjunction returns duplicate solutions for overlapping disjuncts

1 participant