Skip to content

refactor: make reranking candidate-read budgets explicit - #2606

Open
erichare wants to merge 5 commits into
hazel/refactor-farrfrom
fix/rerank-candidate-resolution
Open

erichare wants to merge 5 commits into
hazel/refactor-farrfrom
fix/rerank-candidate-resolution

Conversation

@erichare

@erichare erichare commented Oct 2, 2026 •

Copy link
Copy Markdown
Contributor

What this PR does:

Makes each findAndRerank retrieval leg use the limit in its inner find command instead of passing limits through CommandContext.

  • Passes concrete vector and lexical page sizes and the sorted-result limit to shared query resolution, without a caller-mode flag.
  • Treats a missing or non-positive internal candidate limit as an internal state error. User-supplied limits remain validated by the planner with COMMAND_FIELD_VALUE_INVALID.
  • Preserves ordinary find behavior and the vector-search limit.

Depends on #2637. PR #2607 adds pagination to collect the full candidate set.

Validation: 258 focused parser, planner, builder, resolver, and collection-operation tests pass on Java 21. Includes zero/negative user limits and candidate vector page-size bounds.

Which issue(s) this PR fixes:

Refs #2599 (first part).

Checklist

  • Changes manually tested
  • Automated Tests added/updated
  • Documentation added/updated
  • CLA Signed: DataStax CLA

@github-actions

github-actions Bot commented Oct 5, 2026 •

Copy link
Copy Markdown
Contributor

📈 Unit Test Coverage Delta vs Main Branch

Metric Value
Main Branch 54.58%
This PR 55.62%
Delta 🟢 +1.04%
✅ Coverage improved!

@github-actions

github-actions Bot commented Oct 5, 2026 •

Copy link
Copy Markdown
Contributor

Unit Test Coverage Report

Overall Project 55.62% -0.04% 🍏
Files changed 4.26% ❌

File Coverage
FindAndRerankOperationBuilder.java 93.47% 🍏
CommandContext.java 87.68% 🍏
IntermediateCollectionReadTask.java 57.14% 🍏
FindCommandResolver.java 0% -16.42% ❌

@github-actions

github-actions Bot commented Oct 5, 2026 •

Copy link
Copy Markdown
Contributor

📈 Integration Test Coverage Delta vs Main Branch (dse69-it)

Metric Value
Main Branch 72.01%
This PR 72.95%
Delta 🟢 +0.94%
✅ Coverage improved!

@github-actions

github-actions Bot commented Oct 5, 2026 •

Copy link
Copy Markdown
Contributor

Integration Test Coverage Report (dse69-it)

Overall Project 72.95% -0.01% 🍏
Files changed 80.85% 🍏

File Coverage
CommandContext.java 93.56% 🍏
FindCommandResolver.java 92.06% -3.25% 🍏
FindAndRerankOperationBuilder.java 88.4% 🍏
IntermediateCollectionReadTask.java 66.29% 🍏

@github-actions

github-actions Bot commented Oct 5, 2026 •

Copy link
Copy Markdown
Contributor

📉 Integration Test Coverage Delta vs Main Branch (hcd-it)

Metric Value
Main Branch 74.27%
This PR 74.19%
Delta 🔴 -0.09%
⚠️ Coverage decreased

@github-actions

github-actions Bot commented Oct 5, 2026 •

Copy link
Copy Markdown
Contributor

Integration Test Coverage Report (hcd-it)

Overall Project 74.19% -0.22% 🍏
Files changed 80.05% 🍏

File Coverage
LegMode.java 100% 🍏
FindAndRerankOperationBuilder.java 100% 🍏
RerankLeg.java 100% 🍏
SortException.java 100% 🍏
FindCommandResolver.java 99.64% -0.36% 🍏
CommandContext.java 93.56% 🍏
FindAndRerankPlanner.java 93.53% -6.47% 🍏
FindAndRerankCommand.java 85.05% 🍏
RerankingQuery.java 78.67% 🍏
DeferredCommandResultAction.java 73.68% 🍏
CommandFeatures.java 71.19% 🍏
LexicalLeg.java 71.05% -28.95% 🍏
VectorizeLeg.java 66.67% -33.33% 🍏
IntermediateCollectionReadTask.java 66.29% 🍏
FindAndRerankSortClauseDeserializer.java 56.99% 🍏
FindAndRerankPlan.java 50.63% -49.37% ❌
FindAndRerankSort.java 36.24% -37.16% ❌
VectorLeg.java 33.33% -66.67% ❌

Base automatically changed from fix/rerank-integration-coverage to main October 7, 2026 01:09
@erichare
erichare force-pushed the fix/rerank-candidate-resolution branch from d4c2b54 to af04e14 Compare October 7, 2026 01:09
@erichare
erichare force-pushed the fix/rerank-candidate-resolution branch from af04e14 to 8df7a12 Compare October 15, 2026 15:02
@erichare
erichare removed this pull request from stack #2608 October 15, 2026 15:04
@erichare
erichare changed the base branch from main to hazel/refactor-farr October 15, 2026 15:04
@erichare
erichare added this pull request to stack #2646 October 15, 2026 15:04
Comment thread src/main/java/io/stargate/sgv2/jsonapi/service/resolver/FindCommandResolver.java Outdated
private Operation<CollectionSchemaObject> resolveCollectionCommand(
CommandContext<CollectionSchemaObject> commandContext,
FindCommand command,
boolean candidateRead) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't really understand, a boolean flag is just another workaround/hack, and your next PR 2607 reuses this boolean. It will be hard to maintain and hard to understand if we add more boolean flags. I think we need to discuss this

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed candidateRead in cead439 and carried the change into #2607 in 60aaec1. Each entry point now supplies concrete vector/lexical page sizes and a sorted-result limit. Shared query resolution no longer branches on the caller. #2607 keeps pagination in CollectionCandidateReadOperation and supplies the full candidate budget for sorted reads. Ordinary find retains its existing page sizes and sorted-result cap.

258 focused tests pass for #2606 and 280 for #2607. Does this separation address the concern, or were you thinking of a different boundary?

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Inner reads return at most 50 documents

2 participants