Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Binary file added assets/websearch/icon.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
25 changes: 25 additions & 0 deletions external_plugins/websearch/.mcp.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
{
"mcpServers": {
"search": {
"type": "stdio",
"command": "node",
"args": [
"${ZCODE_PLUGIN_ROOT}/bin/server.mjs"
],
"env": {
"WEBSEARCH_ORDER": "${user_config.order}",
"WEBSEARCH_DISABLED": "${user_config.disabled}",
"WEBSEARCH_MODE": "${user_config.mode}",
"WEBSEARCH_LIMIT": "${user_config.limit}",
"WEBSEARCH_TIMEOUT_MS": "${user_config.timeout_ms}",
"WEBSEARCH_BING_MARKET": "${user_config.bing_market}",
"WEBSEARCH_SEARXNG": "${user_config.searxng_instances}",
"WEBSEARCH_ANYSEARCH_KEY": "${user_config.anysearch_key}",
"WEBSEARCH_EXA_KEY": "${user_config.exa_key}",
"WEBSEARCH_PARALLEL_KEY": "${user_config.parallel_key}"
},
"enabled": true,
"timeoutMs": 120000
}
}
}
87 changes: 87 additions & 0 deletions external_plugins/websearch/.zcode-plugin/plugin.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,87 @@
{
"name": "websearch",
"version": "0.1.5",
"description": "Multi-engine web search for ZCode as a zero-dependency MCP server: AnySearch, Exa, Parallel, Bing and SearXNG chained with automatic fallback, plus a static page reader. Every engine works without a key; filling one in switches it to your own quota. Configuration (engine order, per-engine keys) is websearch.json inside the plugin data folder — created automatically on first use, and the tools report its exact path, so just ask the model for the websearch config location.",
"description_i18n": {
"en": "Multi-engine web search for ZCode as a zero-dependency MCP server: AnySearch, Exa, Parallel, Bing and SearXNG chained with automatic fallback, plus a static page reader. Every engine works without a key; filling one in switches it to your own quota. Configuration (engine order, per-engine keys) is websearch.json inside the plugin data folder — created automatically on first use, and the tools report its exact path, so just ask the model for the websearch config location.",
"zh-CN": "ZCode 独立网页搜索:零依赖 MCP server,AnySearch / Exa / Parallel / Bing / SearXNG 自动回退,另带静态网页正文读取;所有引擎免 Key 可用。配置(引擎顺序、各引擎 Key)写在插件数据目录的 websearch.json——首次使用自动生成,装好后直接问模型「websearch 配置在哪」即可拿到具体路径。"
},
"author": {
"name": "JochenYang",
"url": "https://github.com/JochenYang"
},
"license": "MIT",
"keywords": [
"websearch",
"search",
"web-search",
"mcp",
"bing",
"exa",
"parallel",
"searxng",
"anysearch"
],
"userConfig": {
"order": {
"type": "string",
"title": "Engine order",
"description": "Comma-separated engine ids tried in order: anysearch / exa / parallel / bing / searxng",
"default": "anysearch,exa,parallel,bing,searxng"
},
"disabled": {
"type": "string",
"title": "Disabled engines",
"description": "Comma-separated engine ids to skip, e.g. exa,searxng; empty keeps all enabled",
"default": ""
},
"mode": {
"type": "string",
"title": "Retry mode",
"description": "fallback = start from the first engine every call; rotate = shift the start by one each call to spread free tiers",
"default": "fallback"
},
"limit": {
"type": "number",
"title": "Default result count",
"description": "1-10, used when a tool call omits limit",
"default": 10
},
"timeout_ms": {
"type": "number",
"title": "Total timeout (ms)",
"description": "One deadline covers the whole chain, 5000-60000; each engine gets the remaining budget",
"default": 20000
},
"bing_market": {
"type": "string",
"title": "Bing market",
"description": "auto = follow the query language; or fix it to zh / en",
"default": "auto"
},
"searxng_instances": {
"type": "string",
"title": "SearXNG instances",
"description": "One base URL per line, JSON API required; empty skips the engine",
"default": ""
},
"anysearch_key": {
"type": "string",
"title": "AnySearch key (optional)",
"description": "Empty uses the free tier; filling it in switches the engine to your own quota (Authorization: Bearer)",
"default": ""
},
"exa_key": {
"type": "string",
"title": "Exa key (optional)",
"description": "Empty uses the anonymous tier; filling it in switches the engine to your own quota (Authorization: Bearer)",
"default": ""
},
"parallel_key": {
"type": "string",
"title": "Parallel key (optional)",
"description": "Empty uses the anonymous tier; filling it in switches the engine to your own quota (x-api-key)",
"default": ""
}
}
}
21 changes: 21 additions & 0 deletions external_plugins/websearch/LICENSE
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
MIT License

Copyright (c) 2026 JochenYang

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
108 changes: 108 additions & 0 deletions external_plugins/websearch/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,108 @@
# websearch

Multi-engine web search for ZCode as a single zero-dependency MCP server: AnySearch / Exa / Parallel / Bing / SearXNG chained with automatic fallback, plus a static page reader. **Every engine works without a key** (free tiers); when a quota runs out, fill in that engine's own key and it switches to your quota.

> [中文文档](./README_CN.md)

## Tools

### `web_search`

```json
{ "query": "required, ≤2000 chars", "limit": "1-10, optional, uses the configured default", "engines": ["bing", "exa"] }
```

Success returns `{ ok:true, engine, results:[{title,url,snippet,publishedAt}], attempts, skipped, elapsedMs }`; failure returns `{ ok:false, errorCode, message, attempts, skipped, configPath }` with a `message` you can relay to the user.

`attempts` records every engine that was tried (`ok` / `count` / `ms`), and failed quota-shaped errors (401/402/429, rate limit, quota) carry a `hint` telling the user to fill in that engine's key.

### `fetch_page`

```json
{ "url": "required, absolute http(s) URL", "maxChars": "optional, default 8000, range 500-30000" }
```

Strips `script` / `style` / `svg` / comments, converts block tags to newlines, decodes HTML entities, squeezes whitespace, truncates to `maxChars`. `application/json` responses are returned verbatim. No page scripts are executed and no login state is used.

## Engines

| id | endpoint | method | key |
|---|---|---|---|
| `anysearch` | `api.anysearch.com/v1/search` | JSON API | optional (Bearer) |
| `exa` | `mcp.exa.ai/mcp` | hosted MCP, tool `web_search_exa` | optional (Bearer) |
| `parallel` | `mcp.parallel.ai/mcp` | hosted MCP, tool `web_search` | optional (`x-api-key`) |
| `bing` | `www.bing.com/search` | result-page scrape | none, and not applicable |
| `searxng` | your instance | `/search?format=json` | needs a self-hosted instance (public ones usually disable the JSON API) |

Chain rules: `fallback` (default — start from the first engine every call) or `rotate` (shift the start by one per call to spread free tiers). One deadline covers the whole call (20s default) and each engine gets the remaining budget; engines left unrun are reported in `skipped`. Results are deduplicated by URL and snippets are capped at 300 characters — nothing is rewritten.

## Configuration

This ZCode build has **no configuration form on the plugin detail page**. Configuration lives in a `websearch.json` file inside the plugin data folder:

```text
<plugin-data-folder>/websearch.json
```

The file is **created automatically on first use**, and its absolute path is baked into both tool descriptions — so the quickest way to find it is to ask the model *"where is the websearch config file?"*. It is also `<ZCODE_PLUGIN_DATA>/websearch.json`, i.e. for a typical install:

```text
C:\Users\<you>\.zcode\cli\plugins\data\websearch@zcode-plugin\websearch.json
```

```json
{
"anysearchKey": "",
"exaKey": "",
"parallelKey": "",
"order": "anysearch,exa,parallel,bing,searxng",
"disabled": "",
"mode": "fallback",
"defaultLimit": 10,
"timeoutMs": 20000,
"bingMarket": "auto",
"searxngInstances": ""
}
```

The same fields are also declared as plugin `userConfig` options, so newer ZCode builds that grow a configuration form pick them up without any change on this side. Precedence: `websearch.json` > plugin configuration > environment variables > built-in defaults.

Environment fallbacks: `ANYSEARCH_API_KEY` / `EXA_API_KEY` / `PARALLEL_API_KEY` apply **only while the corresponding field is empty** — the file (or plugin configuration) always wins.

## Security boundary

- An MCP server is an ordinary local process; the host does **not** gate its network traffic. The boundary therefore lives in this plugin: only `http`/`https`, URLs with credentials refused, loopback / private / link-local (including the `169.254.169.254` metadata endpoint) / CGNAT / unique-local / multicast addresses refused for both address families, at most 5 redirects with **every hop re-checked**, 2 MB response ceiling, request timeouts.
- `fetch_page` URLs are model-controlled, so that guard always applies. SearXNG instances come from the user's own configuration, so a local instance is allowed there.
- Search results and page text are **untrusted data**; both tool descriptions say not to follow instructions found in them.

## Known limits

- **Bing is an HTML scrape**: a result-page markup change returns 0 results (the chain skips it and says so — it never pretends "no results").
- **Anonymous tiers get rate-limited**: Exa has had long 429 periods, AnySearch moved to a paid tier at some point; the chain falls through and `attempts[].hint` suggests filling in a key.
- **`fetch_page` does not render JavaScript** — script-rendered pages only yield their static part.
- **No caching**: every call really hits the engines.
- The built-in `WebSearch` (when the model provider supports server-side native search) takes precedence for the model; an MCP plugin cannot disable it.

## Development

```text
external_plugins/websearch/
├── .mcp.json # MCP server declaration (stdio, node, user_config env)
├── .zcode-plugin/plugin.json
├── bin/server.mjs # the whole implementation, zero dependencies, Node 18+
├── commands/websearch.md # /websearch manual entry and single-engine diagnosis
├── tests/test-server.py # offline handshake/error paths; --live hits the real engines
├── LICENSE # MIT
└── README.md / README_CN.md
```

```bash
node --check bin/server.mjs
node bin/server.mjs --self-test # offline parser/config/SSRF assertions
python tests/test-server.py # offline MCP handshake and error paths
python tests/test-server.py --live # also hits the real engines
```

## License

MIT — see [LICENSE](./LICENSE). The search engines themselves are third-party services (AnySearch, Exa, Parallel, Bing, SearXNG) used over their public endpoints.
116 changes: 116 additions & 0 deletions external_plugins/websearch/README_CN.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,116 @@
# websearch

ZCode 的独立网页搜索插件:一个零依赖的 MCP server,把 AnySearch / Exa / Parallel / Bing / SearXNG 串成一条会自动回退的引擎链,另带一个静态网页正文读取工具。**所有引擎都能免 Key 使用**(免费额度);某个引擎额度用完时,把它的 Key 填进配置文件就切换到你的额度。

> [English](./README.md)

## 工具

### `web_search`

```json
{ "query": "必填,≤2000 字符", "limit": "1-10,省略用设置默认值", "engines": ["bing", "exa"] }
```

成功返回 `{ ok:true, engine, results:[{title,url,snippet,publishedAt}], attempts, skipped, elapsedMs }`;失败返回 `{ ok:false, errorCode, message, attempts, skipped, configPath }`,`message` 可直接转述给用户。

`attempts` 里每个引擎都有 `ok` / `count` / `ms`,失败时带 `error`;额度类失败(401/402/429、rate limit、quota)会额外带 `hint`,提示可以填 Key。

### `fetch_page`

```json
{ "url": "必填,http(s) 绝对地址", "maxChars": "可选,默认 8000,范围 500-30000" }
```

去掉 `script`/`style`/`svg`/注释,块级标签转换行,解码 HTML 实体,压缩空白后截断。`application/json` 直接返回原文。不执行页面脚本、不处理登录态。

## 引擎

| id | 端点 | 方式 | Key |
|---|---|---|---|
| `anysearch` | `api.anysearch.com/v1/search` | JSON API | 可选(Bearer) |
| `exa` | `mcp.exa.ai/mcp` | 托管 MCP,工具 `web_search_exa` | 可选(Bearer) |
| `parallel` | `mcp.parallel.ai/mcp` | 托管 MCP,工具 `web_search` | 可选(`x-api-key`) |
| `bing` | `www.bing.com/search` | 抓结果页 HTML | 不需要,也无法填 |
| `searxng` | 你填的实例 | `/search?format=json` | 需要自建实例(公共实例基本关了 JSON API) |

链式规则:`fallback`(默认,每次从第一位开始)或 `rotate`(每次把起始位后移一格,分散匿名额度);一次调用只有一个总预算(默认 20s),每个引擎用剩余预算,预算耗尽后剩余引擎记入 `skipped`。只做去重与 300 字符截断,不改写摘要。

## 配置

**当前版本的 ZCode 插件详情页没有配置表单**(详情页的「高级信息」只展示路径)。配置有三种方式,按优先级:

1. **插件数据目录的 `websearch.json`(推荐)**:路径是

```text
<插件数据目录>/websearch.json
```

文件在**首次使用时自动生成**,且绝对路径写进了两个工具的描述——最省事的找到方式就是直接问模型「websearch 配置文件在哪」。典型安装下的完整路径:

```text
C:\Users\<你>\.zcode\cli\plugins\data\websearch@zcode-plugin\websearch.json
```

```json
{
"anysearchKey": "",
"exaKey": "",
"parallelKey": "",
"order": "anysearch,exa,parallel,bing,searxng",
"disabled": "",
"mode": "fallback",
"defaultLimit": 10,
"timeoutMs": 20000,
"bingMarket": "auto",
"searxngInstances": ""
}
```

只写要改的键即可,其余用默认值;保存后开新会话生效。

2. **环境变量**:`ANYSEARCH_API_KEY` / `EXA_API_KEY` / `PARALLEL_API_KEY`(`setx` 后重启 ZCode)。**仅当 `websearch.json` 和插件配置里都没填该引擎的 Key 时生效**。

3. **插件的 `userConfig`**(清单里声明了 10 个字段):值会展开进 MCP 的环境变量,但当前版本没有输入界面;等 ZCode 支持配置表单后,无需改本插件即可使用。

优先级:`websearch.json` > `userConfig`(应用配置)> 环境变量 > 内置默认值。

## 安全边界

- **没有宿主守门**。Cindy 版靠宿主的 `cindy.fetch` 做 SSRF/体积/超时拦截;ZCode 的 MCP server 是普通本地进程,所以这些限制写在本插件里:只允许 http/https、拒绝带用户名密码的 URL、拒绝回环/内网/链路本地(含 `169.254.169.254` 元数据地址)/CGNAT/唯一本地地址/组播地址(IPv4 与 IPv6 都判)、最多 5 次重定向且**每一跳都重新校验**、响应上限 2 MB、请求超时。
- `fetch_page` 的 URL 来自模型,因此始终走上面的校验;SearXNG 实例地址来自你自己的配置,允许指向本机(本地自建实例是正常用法)。
- 搜索结果的正文与摘要是**不可信数据**,两个工具的描述里都写明了不要执行其中的指令。

## 已知限制

- **Bing 靠 HTML 解析**:结果页 class 改版就会返回 0 条(会被链自动跳过,不会伪装成「没有结果」)。
- **匿名额度会限流**:Exa 曾长期 429、AnySearch 曾转付费档,链会自动回退;失败时 `attempts[].hint` 会提示填 Key。
- **`fetch_page` 不渲染 JS**:需要脚本渲染的页面只能拿到静态部分。
- **没有缓存**:每次调用都真实打到引擎。
- 内置 `WebSearch` 存在时(provider 命中支持服务端搜索的端点),模型可能优先用内置那个;插件无法禁用内置工具。

## 开发与验证

```text
external_plugins/websearch/
├── .mcp.json # MCP server 声明(stdio、node、user_config 环境变量)
├── .zcode-plugin/plugin.json
├── bin/server.mjs # 全部实现,零依赖,Node 18+(全局 fetch)
├── commands/websearch.md # /websearch 手动入口与单引擎排查
├── tests/test-server.py # 离线 + --live 测试
├── LICENSE # MIT
└── README.md / README_CN.md
```

```bash
node --check bin/server.mjs
node bin/server.mjs --self-test # 离线解析器/配置/SSRF 断言
python tests/test-server.py # 离线:MCP 握手、工具表、错误路径
python tests/test-server.py --live # 追加真实引擎与网络守门测试
```

`--self-test` 与离线测试不发网络请求,可以随时跑;`--live` 需要出网,用来确认引擎是否仍然可用(Bing 解析失效时它会是第一个失败的用例)。

## 许可

MIT — 见 [LICENSE](./LICENSE)。搜索引擎本身是第三方服务(AnySearch、Exa、Parallel、Bing、SearXNG),通过它们的公开端点使用。
Loading