diff --git a/docs/en/notes/guide/quickstart/dataflow_webui.md b/docs/en/notes/guide/quickstart/dataflow_webui.md index 13cfcf4fc0..351448d092 100644 --- a/docs/en/notes/guide/quickstart/dataflow_webui.md +++ b/docs/en/notes/guide/quickstart/dataflow_webui.md @@ -65,3 +65,120 @@ First, locate the `backend/app/core/config.py` file in the downloaded and extrac ``` You can import multiple dependency packages. After adding them, restart the WebUI service, and you should see the operators from DataFlow-Extension in the WebUI operator library. + +## Building Pipelines with an AI Agent + +Beyond drag-and-drop orchestration on the canvas, DataFlow-WebUI now ships with a **built-in agent** that lets you build pipelines through natural language. You describe a target in plain English (or Chinese), and the agent queries the DataFlow operator library over MCP, plans the operator chain, and renders a runnable pipeline directly onto the canvas. + +The agent layer supports multiple code-agent backends — **Claude Code**, **Codex**, and **Cursor** — in two distinct usage modes. + +> **Note:** The agent feature relies on the setup scripts and MCP/skill configuration that live in the **source repository** (`.mcp.json`, `.claude/skills/`, `.cursor/`, `scripts/`). These are **not** part of the release zip downloaded by `dataflow webui`. To use the agent, clone the source repo and run the one-command setup described below, rather than the release flow above. + +### Two usage modes + +**Mode A — WebUI Dispatch (chat from the browser).** +The WebUI backend spawns a code-agent CLI headlessly when you chat in the browser chat panel. A small **agent dropdown** in the chat header lets you pick the backend per session. Agents in this mode: **Claude Code** and **Codex**. + +**Mode B — IDE / Terminal (user-driven).** +The agent runs in your own IDE or terminal and connects to the WebUI's MCP server at `http://localhost:8000/mcp`, pushing the pipeline it builds back onto the WebUI canvas. Agents in this mode: **Cursor IDE** and **Claude Code (terminal)**. + +> **Why Cursor is not in the dropdown.** Cursor is *not* dispatched by the WebUI backend. Its intended use is to open this project in Cursor IDE, where your existing Cursor session discovers the `dataflow` MCP server (`.cursor/mcp.json`) and renders pipelines back into the WebUI canvas. There is no need to select an agent in the WebUI when you work from Cursor. + +| Agent | Mode | How to use | MCP config location | Auth | +|---|---|---|---|---| +| **Claude Code** | WebUI dispatch | Pick "Claude Code" in the chat dropdown | `--mcp-config .mcp.json` (passed by backend) | `ANTHROPIC_API_KEY` (or `ANTHROPIC_BASE_URL` for a gateway/中转) | +| **Codex** | WebUI dispatch | Pick "Codex" in the chat dropdown | `~/.codex/config.toml` `[mcp_servers.dataflow]` | `OPENAI_API_KEY` (+ optional `OPENAI_BASE_URL`), **or** `codex login` OAuth (ChatGPT Plus/Pro, no API key needed) | +| **Cursor IDE** | IDE / terminal | Open this project in Cursor; the Agent panel auto-discovers MCP | `.cursor/mcp.json` (project-scoped) | Cursor built-in auth | +| **Claude Code (terminal)** | IDE / terminal | Run `claude` in this repo; it reads `.mcp.json` automatically | `.mcp.json` at repo root | `ANTHROPIC_API_KEY` | + +### Prerequisites + +* **Python 3.10+** with pip +* **Node.js 20+** with npm +* **Git** +* At least one code-agent CLI: + - **Claude Code:** `curl -fsSL https://claude.ai/code/install.sh | sh` + - **Codex:** `npm i -g @openai/codex` + - **Cursor:** [Download Cursor IDE](https://cursor.com) + +### One-command setup + +Clone the source repository and run the installer: + +```shell +git clone https://github.com/OpenDCAI/DataFlow-WebUI.git +cd DataFlow-WebUI +./scripts/setup_all.sh +./scripts/start.sh +``` + +`setup_all.sh` is idempotent (safe to re-run). It checks prerequisites, installs DataFlow + backend dependencies, builds the frontend, initializes the DataFlow data directory, and configures MCP for all agents (delegating to `./scripts/setup_agent.sh all`). To (re)configure just one agent: + +```shell +./scripts/setup_agent.sh claude # or: cursor / codex / all +``` + +This writes the per-agent config files: + +| Agent | Files written | +|---|---| +| Claude Code | `.mcp.json` (repo root) | +| Cursor IDE | `.cursor/mcp.json` + `.cursor/rules/*.mdc` | +| Codex | `~/.codex/config.toml` (appends `[mcp_servers.dataflow]`) + `AGENTS.md` | + +### Authentication + +Export the credentials for whichever agent you use, in the same shell that starts the backend: + +```shell +# Claude Code +export ANTHROPIC_API_KEY=sk-ant-... +export ANTHROPIC_BASE_URL=https://your-gateway/v1 # only if using a gateway/中转 + +# Codex — option 1: API key +export OPENAI_API_KEY=sk-... +export OPENAI_BASE_URL=https://your-gateway/v1 # only if using a gateway/中转 + +# Codex — option 2: ChatGPT Plus/Pro OAuth (no API key) +codex login +``` + +Cursor IDE uses its own built-in authentication — no environment variable needed. + +### Using the agent + +**From the browser (Claude Code / Codex).** Start the backend with `./scripts/start.sh` and open `http://localhost:8000/`. In the chat panel header, pick **Claude Code** or **Codex** from the agent dropdown, then describe what you want to build. Switching the dropdown starts a fresh conversation (different agents do not share session ids); your last choice is remembered across reloads. The agent queries operators over MCP, plans the chain, and syncs the resulting pipeline onto the canvas. It will not auto-execute a pipeline unless you explicitly ask. + +**From Cursor IDE.** Open this project in Cursor. `.cursor/mcp.json` defines the `dataflow` MCP server, but Cursor requires you to enable it once: + +1. Open **Cursor Settings** (Cmd/Ctrl + Shift + J → Settings) +2. Go to **Features → MCP Servers** (or search "MCP") +3. Find `dataflow` and toggle it **ON** +4. If it shows "failed", make sure the backend is running: `./scripts/start.sh --status` +5. Click "Refresh" or restart Cursor to reconnect + +Then chat in Cursor's Agent panel as usual — the pipeline syncs back to the WebUI canvas via MCP. No agent selection in the WebUI is needed. + +**From the Claude Code terminal.** Run `claude` inside the cloned repo; it reads `.mcp.json` automatically and connects to the running backend. + +### Verify the agent connection + +| Agent | Verify command | +|---|---| +| Claude Code | `claude --print --mcp-config .mcp.json --output-format text "call mcp__dataflow__list_operator_categories and report the result"` | +| Codex | `codex exec --json --sandbox workspace-write "call the dataflow MCP tool list_operator_categories and report the result"` | +| Cursor IDE | In the Agent panel: "call the dataflow MCP tool list_operator_categories and report the result" | + +A correct setup returns a list of operator categories such as `core_text`, `general_text`, `reasoning`, … If it fails, the backend may not be running (`./scripts/start.sh`), MCP may not be activated (see the Cursor steps above), or authentication may be missing. + +### Troubleshooting + +| Symptom | Most likely cause | Fix | +|---|---|---| +| `: command not found` in backend logs | The CLI binary is not on the PATH the backend sees | Re-install it, or set `DATAFLOW_CLAUDE_CLI` / `DATAFLOW_CODEX_CLI=/abs/path/to/cli` before launching the backend | +| Chat replies are empty or return an immediate `done` | The agent failed to authenticate | Confirm the relevant API key is exported in the shell that started the backend; for Codex without an API key, run `codex login` first | +| Tool calls fail with "MCP server not reachable" | The agent's MCP config doesn't point at this backend | Re-run `./scripts/setup_agent.sh ` and confirm the backend is on `localhost:8000` | +| The agent invents non-existent operators | The construction skill is not loaded | For Cursor, re-run `./scripts/setup_agent.sh cursor` to regenerate `.cursor/rules/`; for Codex, regenerate `AGENTS.md` | +| Cursor IDE shows no `dataflow` MCP tools | `.cursor/mcp.json` is missing, or the backend is not running | Run `./scripts/setup_agent.sh cursor` and make sure the backend is up at `localhost:8000` | + +> For the full agent setup reference, including authorization scope and behavioral rules, see [`AGENT_SETUP.md`](https://github.com/OpenDCAI/DataFlow-WebUI/blob/main/AGENT_SETUP.md) in the DataFlow-WebUI repository. diff --git a/docs/zh/notes/guide/quickstart/dataflow_webui.md b/docs/zh/notes/guide/quickstart/dataflow_webui.md index 0f08633fa1..a9050b3ac8 100644 --- a/docs/zh/notes/guide/quickstart/dataflow_webui.md +++ b/docs/zh/notes/guide/quickstart/dataflow_webui.md @@ -48,3 +48,120 @@ dataflow webui -h ] ``` 可以导入多种依赖包,添加后重启WebUI服务,就可以在WebUI的算子库中看到DataFlow-Extension中的算子了。 + +## 使用 AI 智能体搭建流水线 + +除了在画布上拖拉拽编排,DataFlow-WebUI 现在还内置了 **AI 智能体(Agent)**,让你可以用自然语言搭建流水线。你只需用中文(或英文)描述目标,智能体就会通过 MCP 查询 DataFlow 算子库、规划算子链,并把可运行的流水线直接渲染到画布上。 + +智能体层支持多种 code-agent 后端 —— **Claude Code**、**Codex** 和 **Cursor** —— 并提供两种不同的使用模式。 + +> **注意:** 智能体功能依赖于位于**源码仓库**中的配置脚本与 MCP/skill 配置(`.mcp.json`、`.claude/skills/`、`.cursor/`、`scripts/`)。这些文件**不包含**在 `dataflow webui` 下载的发行版 zip 包中。因此使用智能体功能时,请克隆源码仓库并执行下文的一键配置,而不是走上面的发行版流程。 + +### 两种使用模式 + +**模式 A —— WebUI 调度(在浏览器中对话)。** +当你在浏览器的对话面板里聊天时,WebUI 后端会以无头(headless)方式拉起 code-agent CLI。对话标题旁有一个 **agent 下拉框**,可以按会话选择后端。此模式下的智能体:**Claude Code** 与 **Codex**。 + +**模式 B —— IDE / 终端(用户主导)。** +智能体运行在你自己的 IDE 或终端中,连接到 WebUI 的 MCP server(`http://localhost:8000/mcp`),并把它搭建好的流水线同步回 WebUI 画布。此模式下的智能体:**Cursor IDE** 与 **Claude Code(终端)**。 + +> **为什么下拉框里没有 Cursor。** Cursor **不**由 WebUI 后端调度。它的使用方式是在 Cursor IDE 中打开本项目,由你已有的 Cursor 会话自动发现 `dataflow` MCP server(`.cursor/mcp.json`),并把流水线渲染回 WebUI 画布。从 Cursor 工作时,无需在 WebUI 中选择 agent。 + +| 智能体 | 模式 | 使用方式 | MCP 配置位置 | 认证 | +|---|---|---|---|---| +| **Claude Code** | WebUI 调度 | 在对话下拉框中选择 "Claude Code" | `--mcp-config .mcp.json`(由后端传入) | `ANTHROPIC_API_KEY`(使用中转/网关时用 `ANTHROPIC_BASE_URL`) | +| **Codex** | WebUI 调度 | 在对话下拉框中选择 "Codex" | `~/.codex/config.toml` 的 `[mcp_servers.dataflow]` | `OPENAI_API_KEY`(可选 `OPENAI_BASE_URL`),**或** `codex login` OAuth(ChatGPT Plus/Pro,无需 API key) | +| **Cursor IDE** | IDE / 终端 | 在 Cursor 中打开本项目,Agent 面板自动发现 MCP | `.cursor/mcp.json`(项目级) | Cursor 内置认证 | +| **Claude Code(终端)** | IDE / 终端 | 在本仓库目录下运行 `claude`,自动读取 `.mcp.json` | 仓库根目录的 `.mcp.json` | `ANTHROPIC_API_KEY` | + +### 前置条件 + +* **Python 3.10+**,含 pip +* **Node.js 20+**,含 npm +* **Git** +* 至少一个 code-agent CLI: + - **Claude Code:** `curl -fsSL https://claude.ai/code/install.sh | sh` + - **Codex:** `npm i -g @openai/codex` + - **Cursor:** [下载 Cursor IDE](https://cursor.com) + +### 一键配置 + +克隆源码仓库并运行安装脚本: + +```shell +git clone https://github.com/OpenDCAI/DataFlow-WebUI.git +cd DataFlow-WebUI +./scripts/setup_all.sh +./scripts/start.sh +``` + +`setup_all.sh` 是幂等的(可安全重复运行)。它会检查前置条件、安装 DataFlow 与后端依赖、构建前端、初始化 DataFlow 数据目录,并为所有智能体配置 MCP(内部委托给 `./scripts/setup_agent.sh all`)。如果只想(重新)配置某一个智能体: + +```shell +./scripts/setup_agent.sh claude # 或:cursor / codex / all +``` + +它会写入各智能体对应的配置文件: + +| 智能体 | 写入的文件 | +|---|---| +| Claude Code | `.mcp.json`(仓库根目录) | +| Cursor IDE | `.cursor/mcp.json` + `.cursor/rules/*.mdc` | +| Codex | `~/.codex/config.toml`(追加 `[mcp_servers.dataflow]`)+ `AGENTS.md` | + +### 认证配置 + +在启动后端的同一个终端里,导出你所使用智能体的凭据: + +```shell +# Claude Code +export ANTHROPIC_API_KEY=sk-ant-... +export ANTHROPIC_BASE_URL=https://your-gateway/v1 # 仅在使用中转/网关时需要 + +# Codex —— 方式一:API key +export OPENAI_API_KEY=sk-... +export OPENAI_BASE_URL=https://your-gateway/v1 # 仅在使用中转/网关时需要 + +# Codex —— 方式二:ChatGPT Plus/Pro OAuth(无需 API key) +codex login +``` + +Cursor IDE 使用其自带的内置认证,无需任何环境变量。 + +### 开始使用智能体 + +**在浏览器中(Claude Code / Codex)。** 用 `./scripts/start.sh` 启动后端,打开 `http://localhost:8000/`。在对话面板标题处的 agent 下拉框中选择 **Claude Code** 或 **Codex**,然后描述你想搭建的内容。切换下拉框会开启一段全新对话(不同智能体之间不共享 session id);你上次的选择会在刷新后被记住。智能体会通过 MCP 查询算子、规划算子链,并把生成的流水线同步到画布上。除非你明确要求,否则它不会自动执行流水线。 + +**在 Cursor IDE 中。** 在 Cursor 中打开本项目。`.cursor/mcp.json` 已定义好 `dataflow` MCP server,但 Cursor 需要你手动启用一次: + +1. 打开 **Cursor Settings**(Cmd/Ctrl + Shift + J → Settings) +2. 进入 **Features → MCP Servers**(或搜索 "MCP") +3. 找到 `dataflow` 并切换为 **ON** +4. 如果显示 "failed",确认后端正在运行:`./scripts/start.sh --status` +5. 点击 "Refresh" 或重启 Cursor 以重新连接 + +随后在 Cursor 的 Agent 面板中正常对话即可 —— 流水线会通过 MCP 同步回 WebUI 画布,无需在 WebUI 中选择 agent。 + +**在 Claude Code 终端中。** 在克隆好的仓库目录下运行 `claude`,它会自动读取 `.mcp.json` 并连接到正在运行的后端。 + +### 验证智能体连接 + +| 智能体 | 验证命令 | +|---|---| +| Claude Code | `claude --print --mcp-config .mcp.json --output-format text "call mcp__dataflow__list_operator_categories and report the result"` | +| Codex | `codex exec --json --sandbox workspace-write "call the dataflow MCP tool list_operator_categories and report the result"` | +| Cursor IDE | 在 Agent 面板中:"call the dataflow MCP tool list_operator_categories and report the result" | + +配置正确时,会返回一组算子类别,例如 `core_text`、`general_text`、`reasoning` 等。如果失败,可能是后端未启动(`./scripts/start.sh`)、MCP 未激活(参考上面的 Cursor 步骤),或缺少认证。 + +### 常见问题排查 + +| 现象 | 可能原因 | 解决方法 | +|---|---|---| +| 后端日志出现 `: command not found` | CLI 可执行文件不在后端能看到的 PATH 上 | 重新安装,或在启动后端前设置 `DATAFLOW_CLAUDE_CLI` / `DATAFLOW_CODEX_CLI=/abs/path/to/cli` | +| 对话回复为空,或立即返回 `done` | 智能体认证失败 | 确认相应的 API key 已在启动后端的终端中导出;Codex 若无 API key,先运行 `codex login` | +| 工具调用报 "MCP server not reachable" | 智能体的 MCP 配置没有指向本后端 | 重新运行 `./scripts/setup_agent.sh `,并确认后端在 `localhost:8000` 上 | +| 智能体编造不存在的算子 | 构建用的 skill 没有加载 | Cursor 重新运行 `./scripts/setup_agent.sh cursor` 重新生成 `.cursor/rules/`;Codex 重新生成 `AGENTS.md` | +| Cursor IDE 中看不到 `dataflow` MCP 工具 | `.cursor/mcp.json` 缺失,或后端未启动 | 运行 `./scripts/setup_agent.sh cursor`,并确保后端在 `localhost:8000` 上运行 | + +> 完整的智能体配置参考(含授权范围与行为规范),请见 DataFlow-WebUI 仓库中的 [`AGENT_SETUP.md`](https://github.com/OpenDCAI/DataFlow-WebUI/blob/main/AGENT_SETUP.md)。