Add filtering argument to Search Mode#91
Conversation
There was a problem hiding this comment.
Orca Security Scan Summary
| Status | Check | Issues by priority | |
|---|---|---|---|
| Infrastructure as Code | View in Orca | ||
| SAST | View in Orca | ||
| Secrets | View in Orca | ||
| Vulnerabilities | View in Orca |
search_strategy argument to Search Moderetrieval_strategy argument to Search Mode
retrieval_strategy argument to Search Modefiltering argument to Search Mode
| collections: list[Union[str, QueryAgentCollectionConfig]] | ||
| limit: int | ||
| offset: int | ||
| filtering: Literal["recall", "precision"] = "recall" |
There was a problem hiding this comment.
Remove defaulting it to recall.
cdpierse
left a comment
There was a problem hiding this comment.
What's the desired behaviour we would like if a user does not pass filtering? If the server handles filtering=None then we should make it a optional Literal where when the value is None it defaults to server side values.
Right now if we were to make the decision for any reason to make precision the default for search mode server side for any customer they would have to update the client in order for this to take effect.
This might not be the actual desired behaviour but I wanted to flag that this what the state would be if we ship as is. Happy to hear thoughts from @augustas1 @danmichaeljones @dannyjameswilliams also here
I think this is probably a sensible solution, as long as it's kept up to date in the docs (or probably at least the docstring of the method) what the server-side default is. Then we may also want to update |
|
|
||
| parsed_response = SearchModeResponse(**response.json()) | ||
| if parsed_response.searches: | ||
| if parsed_response.searches is not None: |
There was a problem hiding this comment.
What's the need for this additional is not None check isn't the parsed_respond.searches model the same ?
There was a problem hiding this comment.
In precision mode we are allowing Search Mode to return no searches, so we need to check that here in order to avoid re-generating searches instead of re-executing the cached one with pagination.
|
|
||
| parsed_response = AsyncSearchModeResponse(**response.json()) | ||
| if parsed_response.searches: | ||
| if parsed_response.searches is not None: |
There was a problem hiding this comment.
In precision mode we are allowing Search Mode to return no searches, so we need to check that here in order to avoid re-generating searches instead of re-executing the cached one with pagination.
Description
Adds a
filteringparameter to the Query Agent's Search Mode. This allows users to control the trade-off between recall and precision.The parameter accepts two values:
"recall"to optimize for finding all relevant results, or"precision"to optimize for the accuracy of returned results.Usage example:
Pagination Fix
Empty search lists are cached from precision mode's ability to output no searches, ensuring pagination requests are consistent.