Skip to content

feat: add Tavily as configurable search backend alongside Serper#4

Open
manisrinivasan2k1 wants to merge 1 commit into
InternScience:mainfrom
Tavily-FDE:feat/tavily-migration/web-search-serper-additive
Open

feat: add Tavily as configurable search backend alongside Serper#4
manisrinivasan2k1 wants to merge 1 commit into
InternScience:mainfrom
Tavily-FDE:feat/tavily-migration/web-search-serper-additive

Conversation

@manisrinivasan2k1

Copy link
Copy Markdown

Summary

Adds Tavily Search as a configurable alternative to the existing Serper backend in the WebSearch tool. Both providers coexist — Serper remains the default and is completely untouched.

A new SEARCH_PROVIDER environment variable (serper | tavily, default serper) controls which backend is used at runtime.

Changes

agent_base/tools/tool_web.py

  • Added google_search_with_tavily() method using TavilyClient.search() with output formatted to match the existing Serper snippet structure
  • Updated call() to read SEARCH_PROVIDER env var and dispatch to the appropriate backend

agent_base/tools/tooling.py

  • Added TAVILY_API_KEY to SENSITIVE_ENV_EXACT set so it is masked from subprocess environments

requirements.txt

  • Added tavily-python>=0.5.0 dependency

.env.example

  • Documented SEARCH_PROVIDER and TAVILY_API_KEY as optional env vars

Dependency changes

  • Added tavily-python>=0.5.0 to requirements.txt

Environment variable changes

  • TAVILY_API_KEY — new, optional (required when SEARCH_PROVIDER=tavily)
  • SEARCH_PROVIDER — new, optional (default: serper; values: serper | tavily)
  • SERPER_KEY — unchanged

Notes for reviewers

  • ScholarSearch is excluded from this migration (no Tavily equivalent for /scholar)
  • All existing Serper logic and retry behavior is completely untouched
  • The Tavily import is deferred (inside the method) to avoid import errors when tavily-python is not installed

Automated Review

  • Passed after 1 attempt(s)
  • Final review: The migration correctly adds Tavily as a parallel search backend behind a SEARCH_PROVIDER env var. All four reported files were modified. The Tavily SDK usage is correct (import, client instantiation, client.search() call, and response shape). Existing Serper functionality is entirely preserved with the default unchanged. Dependencies, sensitive-env masking, and .env.example documentation are all properly updated. Three minor issues noted but none are blocking.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces support for Tavily as an alternative search provider alongside Serper. It adds the google_search_with_tavily method, updates configuration files, and registers TAVILY_API_KEY as a sensitive environment variable. The review feedback suggests adding defensive type checks on the Tavily API response and individual result pages to prevent runtime errors, as well as explicitly validating the SEARCH_PROVIDER environment variable to avoid silent fallbacks.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment on lines +171 to +173
results = response.get("results", [])
if not results:
return f"No results found for '{query}'. Try with a more general query."

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Defensively check if the response returned by Tavily is a dictionary before calling .get() on it to avoid potential AttributeError if the API returns an unexpected response type.

Suggested change
results = response.get("results", [])
if not results:
return f"No results found for '{query}'. Try with a more general query."
if not isinstance(response, dict):
return f"[WebSearch] Tavily returned an invalid response format."
results = response.get("results", [])
if not results:
return f"No results found for '{query}'. Try with a more general query."

Comment on lines +176 to +181
for idx, page in enumerate(results, start=1):
title = str(page.get("title", "Untitled result"))
link = str(page.get("url", ""))
snippet = f"\n{page['content']}" if page.get("content") else ""
redacted_version = f"{idx}. [{title}]({link}){snippet}"
web_snippets.append(redacted_version)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Add a type check to ensure each page in results is a dictionary before accessing its keys, similar to the existing check in google_search_with_serp. Also, avoid querying page.get("content") twice by storing the result in a variable.

Suggested change
for idx, page in enumerate(results, start=1):
title = str(page.get("title", "Untitled result"))
link = str(page.get("url", ""))
snippet = f"\n{page['content']}" if page.get("content") else ""
redacted_version = f"{idx}. [{title}]({link}){snippet}"
web_snippets.append(redacted_version)
for idx, page in enumerate(results, start=1):
if not isinstance(page, dict):
continue
title = str(page.get("title", "Untitled result"))
link = str(page.get("url", ""))
content_val = page.get("content")
snippet = f"\n{content_val}" if content_val else ""
redacted_version = f"{idx}. [{title}]({link}){snippet}"
web_snippets.append(redacted_version)

Comment on lines +199 to 202
provider = os.getenv("SEARCH_PROVIDER", "serper").strip().lower()
if provider == "tavily":
return self.google_search_with_tavily(query.strip())
return self.google_search_with_serp(query.strip())

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Validate the SEARCH_PROVIDER environment variable. If a user configures an unsupported or misspelled provider, returning an explicit error message is much better than silently falling back to serper.

Suggested change
provider = os.getenv("SEARCH_PROVIDER", "serper").strip().lower()
if provider == "tavily":
return self.google_search_with_tavily(query.strip())
return self.google_search_with_serp(query.strip())
provider = os.getenv("SEARCH_PROVIDER", "serper").strip().lower()
if provider == "tavily":
return self.google_search_with_tavily(query.strip())
elif provider == "serper":
return self.google_search_with_serp(query.strip())
else:
return f"[WebSearch] Unknown search provider '{provider}'. Supported providers are 'serper' and 'tavily'."

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant