Skip to content

docs: document WebUI built-in agent (Claude Code / Codex / Cursor) #178

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
haolpku merged 1 commit into from
Jun 25, 2026
Merged

docs: document WebUI built-in agent (Claude Code / Codex / Cursor) #178

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
117 changes: 117 additions & 0 deletions docs/en/notes/guide/quickstart/dataflow_webui.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -65,3 +65,120 @@ First, locate the `backend/app/core/config.py` file in the downloaded and extrac
```

You can import multiple dependency packages. After adding them, restart the WebUI service, and you should see the operators from DataFlow-Extension in the WebUI operator library.

## Building Pipelines with an AI Agent

Beyond drag-and-drop orchestration on the canvas, DataFlow-WebUI now ships with a **built-in agent** that lets you build pipelines through natural language. You describe a target in plain English (or Chinese), and the agent queries the DataFlow operator library over MCP, plans the operator chain, and renders a runnable pipeline directly onto the canvas.

The agent layer supports multiple code-agent backends — **Claude Code**, **Codex**, and **Cursor** — in two distinct usage modes.

> **Note:** The agent feature relies on the setup scripts and MCP/skill configuration that live in the **source repository** (`.mcp.json`, `.claude/skills/`, `.cursor/`, `scripts/`). These are **not** part of the release zip downloaded by `dataflow webui`. To use the agent, clone the source repo and run the one-command setup described below, rather than the release flow above.

### Two usage modes

**Mode A — WebUI Dispatch (chat from the browser).**
The WebUI backend spawns a code-agent CLI headlessly when you chat in the browser chat panel. A small **agent dropdown** in the chat header lets you pick the backend per session. Agents in this mode: **Claude Code** and **Codex**.

**Mode B — IDE / Terminal (user-driven).**
The agent runs in your own IDE or terminal and connects to the WebUI's MCP server at `http://localhost:8000/mcp`, pushing the pipeline it builds back onto the WebUI canvas. Agents in this mode: **Cursor IDE** and **Claude Code (terminal)**.

> **Why Cursor is not in the dropdown.** Cursor is *not* dispatched by the WebUI backend. Its intended use is to open this project in Cursor IDE, where your existing Cursor session discovers the `dataflow` MCP server (`.cursor/mcp.json`) and renders pipelines back into the WebUI canvas. There is no need to select an agent in the WebUI when you work from Cursor.

| Agent | Mode | How to use | MCP config location | Auth |
|---|---|---|---|---|
| **Claude Code** | WebUI dispatch | Pick "Claude Code" in the chat dropdown | `--mcp-config .mcp.json` (passed by backend) | `ANTHROPIC_API_KEY` (or `ANTHROPIC_BASE_URL` for a gateway/中转) |
| **Codex** | WebUI dispatch | Pick "Codex" in the chat dropdown | `~/.codex/config.toml` `[mcp_servers.dataflow]` | `OPENAI_API_KEY` (+ optional `OPENAI_BASE_URL`), **or** `codex login` OAuth (ChatGPT Plus/Pro, no API key needed) |
| **Cursor IDE** | IDE / terminal | Open this project in Cursor; the Agent panel auto-discovers MCP | `.cursor/mcp.json` (project-scoped) | Cursor built-in auth |
| **Claude Code (terminal)** | IDE / terminal | Run `claude` in this repo; it reads `.mcp.json` automatically | `.mcp.json` at repo root | `ANTHROPIC_API_KEY` |

### Prerequisites

* **Python 3.10+** with pip
* **Node.js 20+** with npm
* **Git**
* At least one code-agent CLI:
- **Claude Code:** `curl -fsSL https://claude.ai/code/install.sh | sh`
- **Codex:** `npm i -g @openai/codex`
- **Cursor:** [Download Cursor IDE](https://cursor.com)

### One-command setup

Clone the source repository and run the installer:

```shell
git clone https://github.com/OpenDCAI/DataFlow-WebUI.git
cd DataFlow-WebUI
./scripts/setup_all.sh
./scripts/start.sh
```

`setup_all.sh` is idempotent (safe to re-run). It checks prerequisites, installs DataFlow + backend dependencies, builds the frontend, initializes the DataFlow data directory, and configures MCP for all agents (delegating to `./scripts/setup_agent.sh all`). To (re)configure just one agent:

```shell
./scripts/setup_agent.sh claude # or: cursor / codex / all
```

This writes the per-agent config files:

| Agent | Files written |
|---|---|
| Claude Code | `.mcp.json` (repo root) |
| Cursor IDE | `.cursor/mcp.json` + `.cursor/rules/*.mdc` |
| Codex | `~/.codex/config.toml` (appends `[mcp_servers.dataflow]`) + `AGENTS.md` |

### Authentication

Export the credentials for whichever agent you use, in the same shell that starts the backend:

```shell
# Claude Code
export ANTHROPIC_API_KEY=sk-ant-...
export ANTHROPIC_BASE_URL=https://your-gateway/v1 # only if using a gateway/中转

# Codex — option 1: API key
export OPENAI_API_KEY=sk-...
export OPENAI_BASE_URL=https://your-gateway/v1 # only if using a gateway/中转

# Codex — option 2: ChatGPT Plus/Pro OAuth (no API key)
codex login
```

Cursor IDE uses its own built-in authentication — no environment variable needed.

### Using the agent

**From the browser (Claude Code / Codex).** Start the backend with `./scripts/start.sh` and open `http://localhost:8000/`. In the chat panel header, pick **Claude Code** or **Codex** from the agent dropdown, then describe what you want to build. Switching the dropdown starts a fresh conversation (different agents do not share session ids); your last choice is remembered across reloads. The agent queries operators over MCP, plans the chain, and syncs the resulting pipeline onto the canvas. It will not auto-execute a pipeline unless you explicitly ask.

**From Cursor IDE.** Open this project in Cursor. `.cursor/mcp.json` defines the `dataflow` MCP server, but Cursor requires you to enable it once:

1. Open **Cursor Settings** (Cmd/Ctrl + Shift + J → Settings)
2. Go to **Features → MCP Servers** (or search "MCP")
3. Find `dataflow` and toggle it **ON**
4. If it shows "failed", make sure the backend is running: `./scripts/start.sh --status`
5. Click "Refresh" or restart Cursor to reconnect

Then chat in Cursor's Agent panel as usual — the pipeline syncs back to the WebUI canvas via MCP. No agent selection in the WebUI is needed.

**From the Claude Code terminal.** Run `claude` inside the cloned repo; it reads `.mcp.json` automatically and connects to the running backend.

### Verify the agent connection

| Agent | Verify command |
|---|---|
| Claude Code | `claude --print --mcp-config .mcp.json --output-format text "call mcp__dataflow__list_operator_categories and report the result"` |
| Codex | `codex exec --json --sandbox workspace-write "call the dataflow MCP tool list_operator_categories and report the result"` |
| Cursor IDE | In the Agent panel: "call the dataflow MCP tool list_operator_categories and report the result" |

A correct setup returns a list of operator categories such as `core_text`, `general_text`, `reasoning`, … If it fails, the backend may not be running (`./scripts/start.sh`), MCP may not be activated (see the Cursor steps above), or authentication may be missing.

### Troubleshooting

| Symptom | Most likely cause | Fix |
|---|---|---|
| `<cli>: command not found` in backend logs | The CLI binary is not on the PATH the backend sees | Re-install it, or set `DATAFLOW_CLAUDE_CLI` / `DATAFLOW_CODEX_CLI=/abs/path/to/cli` before launching the backend |
| Chat replies are empty or return an immediate `done` | The agent failed to authenticate | Confirm the relevant API key is exported in the shell that started the backend; for Codex without an API key, run `codex login` first |
| Tool calls fail with "MCP server not reachable" | The agent's MCP config doesn't point at this backend | Re-run `./scripts/setup_agent.sh <kind>` and confirm the backend is on `localhost:8000` |
| The agent invents non-existent operators | The construction skill is not loaded | For Cursor, re-run `./scripts/setup_agent.sh cursor` to regenerate `.cursor/rules/`; for Codex, regenerate `AGENTS.md` |
| Cursor IDE shows no `dataflow` MCP tools | `.cursor/mcp.json` is missing, or the backend is not running | Run `./scripts/setup_agent.sh cursor` and make sure the backend is up at `localhost:8000` |

> For the full agent setup reference, including authorization scope and behavioral rules, see [`AGENT_SETUP.md`](https://github.com/OpenDCAI/DataFlow-WebUI/blob/main/AGENT_SETUP.md) in the DataFlow-WebUI repository.
117 changes: 117 additions & 0 deletions docs/zh/notes/guide/quickstart/dataflow_webui.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,3 +48,120 @@ dataflow webui -h
]
```
可以导入多种依赖包，添加后重启WebUI服务，就可以在WebUI的算子库中看到DataFlow-Extension中的算子了。

## 使用 AI 智能体搭建流水线

除了在画布上拖拉拽编排，DataFlow-WebUI 现在还内置了 **AI 智能体（Agent）**，让你可以用自然语言搭建流水线。你只需用中文（或英文）描述目标，智能体就会通过 MCP 查询 DataFlow 算子库、规划算子链，并把可运行的流水线直接渲染到画布上。

智能体层支持多种 code-agent 后端 —— **Claude Code**、**Codex** 和 **Cursor** —— 并提供两种不同的使用模式。

> **注意：** 智能体功能依赖于位于**源码仓库**中的配置脚本与 MCP/skill 配置（`.mcp.json`、`.claude/skills/`、`.cursor/`、`scripts/`）。这些文件**不包含**在 `dataflow webui` 下载的发行版 zip 包中。因此使用智能体功能时，请克隆源码仓库并执行下文的一键配置，而不是走上面的发行版流程。

### 两种使用模式

**模式 A —— WebUI 调度（在浏览器中对话）。**
当你在浏览器的对话面板里聊天时，WebUI 后端会以无头（headless）方式拉起 code-agent CLI。对话标题旁有一个 **agent 下拉框**，可以按会话选择后端。此模式下的智能体：**Claude Code** 与 **Codex**。

**模式 B —— IDE / 终端（用户主导）。**
智能体运行在你自己的 IDE 或终端中，连接到 WebUI 的 MCP server（`http://localhost:8000/mcp`），并把它搭建好的流水线同步回 WebUI 画布。此模式下的智能体：**Cursor IDE** 与 **Claude Code（终端）**。

> **为什么下拉框里没有 Cursor。** Cursor **不**由 WebUI 后端调度。它的使用方式是在 Cursor IDE 中打开本项目，由你已有的 Cursor 会话自动发现 `dataflow` MCP server（`.cursor/mcp.json`），并把流水线渲染回 WebUI 画布。从 Cursor 工作时，无需在 WebUI 中选择 agent。

| 智能体 | 模式 | 使用方式 | MCP 配置位置 | 认证 |
|---|---|---|---|---|
| **Claude Code** | WebUI 调度 | 在对话下拉框中选择 "Claude Code" | `--mcp-config .mcp.json`（由后端传入） | `ANTHROPIC_API_KEY`（使用中转/网关时用 `ANTHROPIC_BASE_URL`） |
| **Codex** | WebUI 调度 | 在对话下拉框中选择 "Codex" | `~/.codex/config.toml` 的 `[mcp_servers.dataflow]` | `OPENAI_API_KEY`（可选 `OPENAI_BASE_URL`），**或** `codex login` OAuth（ChatGPT Plus/Pro，无需 API key） |
| **Cursor IDE** | IDE / 终端 | 在 Cursor 中打开本项目，Agent 面板自动发现 MCP | `.cursor/mcp.json`（项目级） | Cursor 内置认证 |
| **Claude Code（终端）** | IDE / 终端 | 在本仓库目录下运行 `claude`，自动读取 `.mcp.json` | 仓库根目录的 `.mcp.json` | `ANTHROPIC_API_KEY` |

### 前置条件

* **Python 3.10+**，含 pip
* **Node.js 20+**，含 npm
* **Git**
* 至少一个 code-agent CLI：
- **Claude Code：** `curl -fsSL https://claude.ai/code/install.sh | sh`
- **Codex：** `npm i -g @openai/codex`
- **Cursor：** [下载 Cursor IDE](https://cursor.com)

### 一键配置

克隆源码仓库并运行安装脚本：

```shell
git clone https://github.com/OpenDCAI/DataFlow-WebUI.git
cd DataFlow-WebUI
./scripts/setup_all.sh
./scripts/start.sh
```

`setup_all.sh` 是幂等的（可安全重复运行）。它会检查前置条件、安装 DataFlow 与后端依赖、构建前端、初始化 DataFlow 数据目录，并为所有智能体配置 MCP（内部委托给 `./scripts/setup_agent.sh all`）。如果只想（重新）配置某一个智能体：

```shell
./scripts/setup_agent.sh claude # 或：cursor / codex / all
```

它会写入各智能体对应的配置文件：

| 智能体 | 写入的文件 |
|---|---|
| Claude Code | `.mcp.json`（仓库根目录） |
| Cursor IDE | `.cursor/mcp.json` + `.cursor/rules/*.mdc` |
| Codex | `~/.codex/config.toml`（追加 `[mcp_servers.dataflow]`）+ `AGENTS.md` |

### 认证配置

在启动后端的同一个终端里，导出你所使用智能体的凭据：

```shell
# Claude Code
export ANTHROPIC_API_KEY=sk-ant-...
export ANTHROPIC_BASE_URL=https://your-gateway/v1 # 仅在使用中转/网关时需要

# Codex —— 方式一：API key
export OPENAI_API_KEY=sk-...
export OPENAI_BASE_URL=https://your-gateway/v1 # 仅在使用中转/网关时需要

# Codex —— 方式二：ChatGPT Plus/Pro OAuth（无需 API key）
codex login
```

Cursor IDE 使用其自带的内置认证，无需任何环境变量。

### 开始使用智能体

**在浏览器中（Claude Code / Codex）。** 用 `./scripts/start.sh` 启动后端，打开 `http://localhost:8000/`。在对话面板标题处的 agent 下拉框中选择 **Claude Code** 或 **Codex**，然后描述你想搭建的内容。切换下拉框会开启一段全新对话（不同智能体之间不共享 session id）；你上次的选择会在刷新后被记住。智能体会通过 MCP 查询算子、规划算子链，并把生成的流水线同步到画布上。除非你明确要求，否则它不会自动执行流水线。

**在 Cursor IDE 中。** 在 Cursor 中打开本项目。`.cursor/mcp.json` 已定义好 `dataflow` MCP server，但 Cursor 需要你手动启用一次：

1. 打开 **Cursor Settings**（Cmd/Ctrl + Shift + J → Settings）
2. 进入 **Features → MCP Servers**（或搜索 "MCP"）
3. 找到 `dataflow` 并切换为 **ON**
4. 如果显示 "failed"，确认后端正在运行：`./scripts/start.sh --status`
5. 点击 "Refresh" 或重启 Cursor 以重新连接

随后在 Cursor 的 Agent 面板中正常对话即可 —— 流水线会通过 MCP 同步回 WebUI 画布，无需在 WebUI 中选择 agent。

**在 Claude Code 终端中。** 在克隆好的仓库目录下运行 `claude`，它会自动读取 `.mcp.json` 并连接到正在运行的后端。

### 验证智能体连接

| 智能体 | 验证命令 |
|---|---|
| Claude Code | `claude --print --mcp-config .mcp.json --output-format text "call mcp__dataflow__list_operator_categories and report the result"` |
| Codex | `codex exec --json --sandbox workspace-write "call the dataflow MCP tool list_operator_categories and report the result"` |
| Cursor IDE | 在 Agent 面板中："call the dataflow MCP tool list_operator_categories and report the result" |

配置正确时，会返回一组算子类别，例如 `core_text`、`general_text`、`reasoning` 等。如果失败，可能是后端未启动（`./scripts/start.sh`）、MCP 未激活（参考上面的 Cursor 步骤），或缺少认证。

### 常见问题排查

| 现象 | 可能原因 | 解决方法 |
|---|---|---|
| 后端日志出现 `<cli>: command not found` | CLI 可执行文件不在后端能看到的 PATH 上 | 重新安装，或在启动后端前设置 `DATAFLOW_CLAUDE_CLI` / `DATAFLOW_CODEX_CLI=/abs/path/to/cli` |
| 对话回复为空，或立即返回 `done` | 智能体认证失败 | 确认相应的 API key 已在启动后端的终端中导出；Codex 若无 API key，先运行 `codex login` |
| 工具调用报 "MCP server not reachable" | 智能体的 MCP 配置没有指向本后端 | 重新运行 `./scripts/setup_agent.sh <kind>`，并确认后端在 `localhost:8000` 上 |
| 智能体编造不存在的算子 | 构建用的 skill 没有加载 | Cursor 重新运行 `./scripts/setup_agent.sh cursor` 重新生成 `.cursor/rules/`；Codex 重新生成 `AGENTS.md` |
| Cursor IDE 中看不到 `dataflow` MCP 工具 | `.cursor/mcp.json` 缺失，或后端未启动 | 运行 `./scripts/setup_agent.sh cursor`，并确保后端在 `localhost:8000` 上运行 |

> 完整的智能体配置参考（含授权范围与行为规范），请见 DataFlow-WebUI 仓库中的 [`AGENT_SETUP.md`](https://github.com/OpenDCAI/DataFlow-WebUI/blob/main/AGENT_SETUP.md)。
Loading