Skip to content

fix(chat): broaden over-specific prediction-market queries on zero results - #22

Merged
VickyXAI merged 1 commit into
masterfrom
fix/pm-search-broadening
Jul 18, 2026
Merged

fix(chat): broaden over-specific prediction-market queries on zero results#22
VickyXAI merged 1 commit into
masterfrom
fix/pm-search-broadening

Conversation

@VickyXAI

Copy link
Copy Markdown
Contributor

Problem

2026-07-17 production log audit: a user asked about betting on "the world cup final game ... spain against argentina". Franklin's search_prediction_markets passed the model's verbose query verbatim to Predexon, got 0 results, and the model hallucinated ("the next World Cup will be in 2026").

Live probes confirmed the root cause — Predexon /markets/search is strict keyword AND-matching over market titles:

  • q='World Cup final game' -> 0 results
  • q='World Cup final Spain vs Argentina' -> 0 results
  • q='World Cup' -> 10 markets, incl. Polymarket 'Will Spain win the 2026 FIFA World Cup?' ($58M vol) and 'Will Argentina win?' ($85M vol)

At least 10 of 124 franklin-try conversations show this zero-result miss pattern on over-specific queries.

Fix

  1. Auto-broadening retry (src/hooks/use-franklin-chat.ts): when the first search returns 0 results, retry once with the query stripped of qualifier/filler words (final, game, odds, vs, winner, betting, ...) down to its 1-3 strongest entity keywords. If the retry finds markets, return them with a note naming the broadened query so the model knows what actually matched. All-filler queries ("hottest markets right now") broaden to nothing and skip the retry, so a paid retry only fires when there are real entity keywords to search.
  2. Model guidance: tool schema description + FRANKLIN_TOOLS_PROMPT now state up front that search is strict keyword AND-matching over market titles — query with 1-3 entity keywords (e.g. "World Cup", "Fed rate cut"), never full sentences or qualifiers.

Broadening behaviour on audit queries:

  • "World Cup final game" -> "World Cup"
  • "World Cup final Spain vs Argentina" -> "World Cup Spain"
  • "Fed rate cut odds" -> "Fed rate cut"
  • "hottest markets right now" -> "" (no retry)

Verification

  • npx tsc --noEmit clean
  • npx eslint clean on both changed files
  • No CJK characters introduced (repo is English-only)

…sults

Predexon /markets/search is strict keyword AND-matching over market
titles, so model-composed queries like "World Cup final Spain vs
Argentina" returned 0 results and the model hallucinated an answer
(2026-07-17 prod log audit: >=10 of 124 franklin-try conversations
hit this zero-result miss).

- search_prediction_markets now auto-retries once on an empty first
  result with the query stripped of qualifier/filler words down to its
  1-3 strongest entity keywords, and tells the model which broadened
  query actually matched.
- Tool schema + FRANKLIN_TOOLS_PROMPT now tell the model up front that
  search is strict AND-matching and to query with 1-3 entity keywords,
  never full sentences.
@VickyXAI
VickyXAI merged commit 3c4ff51 into master Jul 18, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant