fix(chat): broaden over-specific prediction-market queries on zero results - #22
Merged
Conversation
…sults Predexon /markets/search is strict keyword AND-matching over market titles, so model-composed queries like "World Cup final Spain vs Argentina" returned 0 results and the model hallucinated an answer (2026-07-17 prod log audit: >=10 of 124 franklin-try conversations hit this zero-result miss). - search_prediction_markets now auto-retries once on an empty first result with the query stripped of qualifier/filler words down to its 1-3 strongest entity keywords, and tells the model which broadened query actually matched. - Tool schema + FRANKLIN_TOOLS_PROMPT now tell the model up front that search is strict AND-matching and to query with 1-3 entity keywords, never full sentences.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
2026-07-17 production log audit: a user asked about betting on "the world cup final game ... spain against argentina". Franklin's
search_prediction_marketspassed the model's verbose query verbatim to Predexon, got 0 results, and the model hallucinated ("the next World Cup will be in 2026").Live probes confirmed the root cause — Predexon
/markets/searchis strict keyword AND-matching over market titles:q='World Cup final game'-> 0 resultsq='World Cup final Spain vs Argentina'-> 0 resultsq='World Cup'-> 10 markets, incl. Polymarket 'Will Spain win the 2026 FIFA World Cup?' ($58M vol) and 'Will Argentina win?' ($85M vol)At least 10 of 124 franklin-try conversations show this zero-result miss pattern on over-specific queries.
Fix
src/hooks/use-franklin-chat.ts): when the first search returns 0 results, retry once with the query stripped of qualifier/filler words (final, game, odds, vs, winner, betting, ...) down to its 1-3 strongest entity keywords. If the retry finds markets, return them with a note naming the broadened query so the model knows what actually matched. All-filler queries ("hottest markets right now") broaden to nothing and skip the retry, so a paid retry only fires when there are real entity keywords to search.FRANKLIN_TOOLS_PROMPTnow state up front that search is strict keyword AND-matching over market titles — query with 1-3 entity keywords (e.g. "World Cup", "Fed rate cut"), never full sentences or qualifiers.Broadening behaviour on audit queries:
Verification
npx tsc --noEmitcleannpx eslintclean on both changed files