[heydryft]

Change Summary

Adds Outcrawl as a web search engine and as a page content fetcher. Outcrawl (https://outcrawl.ai) is a scrape and search API for AI agents (free tier: 1,000 credits a month).

• New engine outcrawl for WebSearchTool, exposed as the outcrawl_search tool. It calls POST https://api.outcrawl.ai/v1/search with query and limit. With scrape: true each hit also carries the page markdown, which prefills content so the fetch step is skipped (same mechanism as Tavily raw_content).
• New content fetcher fetcher: outcrawl_scrape, which calls POST /v1/scrape with formats: ["markdown"] and is used by fetch_page and by fetch_content in searches.
• Failures raise OutcrawlHTTPError with status, code, retry_after, is_quota and is_auth, so WebSearchTool._search_error_payload reports auth and rate limit errors to the model the same way it does for Tavily.
• Key: OUTCRAWL_API_KEY env var or outcrawl_api_key in the web_search tool config; optional outcrawl_request_timeout (default 60s). The engine raises a clear error when no key is set, like Exa.

The layout follows the Tavily engine (stdlib HTTP client, no new dependency), and the registration points match it:

• ms_agent/tools/search/outcrawl/ (new: http.py, schema.py, search.py, fetcher.py, __init__.py)
• ms_agent/tools/search/search_base.py: SearchEngineType.OUTCRAWL, ENGINE_TOOL_NAMES['outcrawl']
• ms_agent/tools/search/websearch_tool.py: get_content_fetcher, get_search_engine_class, get_search_engine, SUPPORTED_ENGINES, API key map, request timeout, connect(), outcrawl_search() method, docstrings
• ms_agent/capabilities/wrappers/web_search.py: key requirement and engine descriptions
• Docs: ms-agent-skills/references/web-search.md, ms-agent-skills/SKILL.md, examples/capability/setup.md
• Tests: tests/search/test_outcrawl.py (new), tests/capability/test_gateway.py (missing key case, mirroring the Tavily one)

Example config:

tools:
  web_search:
    engines:
      - outcrawl
    outcrawl_api_key: <OUTCRAWL_API_KEY>
    fetcher: outcrawl_scrape

Related issue number

None.

Checklist

• The pull request title is a good summary of the changes - it will be used in the changelog
• Unit tests for the changes exist
• Run pre-commit install and pre-commit run --all-files before git commit, and passed lint check.
• Documentation reflects the changes where applicable

Commands run

• pre-commit run --files <the 13 changed files>: flake8, isort, yapf and the other hooks all passed (I ran it on the changed files, not with --all-files).
• python -m pytest tests/search/test_outcrawl.py tests/search/test_tavily_keyless.py tests/capability/test_gateway.py tests/capability/test_review_regressions.py tests/tools/test_server_tools_smoke.py (with no search keys set): 57 passed, 6 skipped. The skips are the real network tests that need keys.
• OUTCRAWL_API_KEY=... python -m pytest tests/search/test_outcrawl.py: 17 passed, including the two real network tests, test_outcrawl_search_real and test_outcrawl_scrape_real.
• tests/search/test_search.py does not collect in my environment because the optional arxiv package is not installed. That happens on main too and is unrelated to this change.

Live calls through the new code

These ran through WebSearchTool with engines: [outcrawl] and fetcher: outcrawl_scrape:

• outcrawl_search with query "ModelScope MS-Agent framework" and num_results: 2: status ok, 2 results (github.com/modelscope/ms-agent and its releases page). Both pages were then fetched with the Outcrawl scrape fetcher, fetch_success: true, about 39,000 characters of markdown each.
• fetch_page on https://example.com/: status ok, and the markdown starts with "This domain is for use in documentation examples...".

Note: the HTTP client sends User-Agent: ms-agent because the API edge refuses the default Python-urllib user agent.

Disclosure: I work on Outcrawl.

This PR was prepared with AI assistance (omp with Claude). The code, tests, docs edits and this description were written with it. The tests, lint and live calls above were actually run as described. No human has reviewed it yet.