diff --git a/.claude/skills/board-ops/SKILL.md b/.claude/skills/board-ops/SKILL.md index 02808f21fa..bb75edece9 100644 --- a/.claude/skills/board-ops/SKILL.md +++ b/.claude/skills/board-ops/SKILL.md @@ -299,8 +299,10 @@ fi # Then the two chained item-edit calls from "Add a card and set its fields". ``` -**Convert** it to an issue only after the advisory is **published**. The web UI -has "Convert to issue" on the card; the API equivalent is: +**Convert** it to an issue when its fix is ready to merge, keeping its opaque +title; a follow-up card converts only after the advisory is **published**. When +each happens, and what the issue gets, is the `/security-advisory` flow. The web +UI has "Convert to issue" on the card; the API equivalent is: ```sh REPO_ID=$(gh api repos/modelcontextprotocol/servers --jq .node_id) || REPO_ID= diff --git a/.claude/skills/release/SKILL.md b/.claude/skills/release/SKILL.md index 4f3fe0bc45..8492b22ffa 100644 --- a/.claude/skills/release/SKILL.md +++ b/.claude/skills/release/SKILL.md @@ -256,11 +256,25 @@ gh workflow run prepare-python-release.yml --repo modelcontextprotocol/servers \ It opens a PR that starts with `Closes #N`, on a branch named `v2/chore/-python-calver-`, labeled `v2`, and dispatches the TypeScript, Python and DCO workflows on that branch, as the Version Packages -workflow does (not the Everything diff, whose paths it never touches). Wait -for its three `(dispatched)` statuses to pass, check that each stamped package really -changed, and a maintainer merges it. +workflow does (not the Everything diff, whose paths it never touches). No PR means no Python server has a shipped change since its last stamp. +Then, on the PR: + +1. **Link it to the release issue.** Its `Closes #N` is only a + cross-reference on `v2/main`, and the workflow makes no API call, so link + it with the `addCloseIssueReferences` mutation from `/pr-flow` (step 6) + and read it back: `closingIssuesReferences` must list `#N`. On `v2/main` + it closes nothing at merge. Find the PR by its branch: + + ```sh + gh pr list --repo modelcontextprotocol/servers --base v2/main \ + --json number,headRefName,url \ + | jq --arg p "v2/chore/$N-python-calver-" '.[] | select(.headRefName | startswith($p))' + ``` +2. **Wait for its three `(dispatched)` statuses to pass**, check that each + stamped package really changed, and a maintainer merges it. + ⚠️ GitHub only dispatches a workflow that exists on the default branch. Until the first milestone merge carries this one to `main`, do it by hand, which is the same script: @@ -274,7 +288,9 @@ node scripts/prepare-python-release.mjs # prints "name: old -> new" per stam git commit -s -am "chore: stamp Python CalVer versions for $(date -u +%Y.%-m.%-d)" ``` -then open the PR through `/pr-flow`, body starting `Closes #N`. +then open the PR through `/pr-flow`, body starting `Closes #N`. Its step 6 +links the PR to the release issue and reads the link back, as step 1 above +does for the workflow's PR: do not skip it. ## 3. The merge PR: `v2/main` → `main` @@ -453,12 +469,35 @@ gh issue list --repo modelcontextprotocol/servers --milestone "$MILESTONE" \ before and after. **4d. The ledger.** Write the results up as a published artifact (a private -page, shared with the maintainers) and link it from the merge PR's body. It -holds: +page, shared with the maintainers) and link it from the merge PR's body. + +**Title it `Servers vX.Y.Z Release Ledger`**, the milestone's version, in both +the page's `` and its `<h1>`. + +**Start from the previous release's ledger**, so every ledger reads as one +series. It is linked from the latest published Release's notes, on the +`Release ledger:` line (5a): + +```sh +gh release view --repo modelcontextprotocol/servers --json tagName,body \ + --jq '.tagName, (.body | split("\n")[] | select(startswith("Release ledger:")))' +``` + +Read that page (the artifact tool's read action), and build the new one on its +stylesheet and section structure, changing the content only. This skill +deliberately names no ledger URL: the latest Release always points at the +current one, and a URL written here would go stale with the next release. + +It holds, in this order: - **Masthead**: repository, the merge PR and its commit, the milestone, the date, and a line saying what tree was tested and that its hash matches - `origin/v2/main`. + `origin/v2/main`. Two more lines: + - **SDKs**: the MCP SDK versions the lockfiles hold (each + `@modelcontextprotocol/*` SDK package, and Python `mcp`). + - **Toolchain**: the versions used to verify: Node (with OS and + architecture), uv, the Inspector CLI and web UI, Playwright, Docker and + Claude Code. - **Verdict**: `local:gate` and `pack:verify` results, milestone issues verified as `N / N`, findings. - **The gate**: one cell per stage, with its numbers. @@ -469,13 +508,34 @@ holds: (`node scripts/npm-publish-guard.mjs src/<server>` for npm; for PyPI, `curl -s -o /dev/null -w '%{http_code}' https://pypi.org/pypi/<name>/<version>/json`, where 404 means it will publish). -- **One table per theme**: *Issue · What was driven · Observed · Status*. One - row per closed issue, linked, with the actual output in the Observed cell. +- **One table per theme** (a server, a group of servers, or the tooling), + with the columns *Issue · Title · What was driven · Observed · Status*. One + row per closed issue. The Issue cell links it and is tagged with how it was + exercised: `client` (the Inspector CLI or an SDK client), `web UI` (the + Inspector web UI through Playwright) or `probe`. The Observed cell holds the + actual output. Status is a pill: `PASS`, or a named exception. +- **Preparation PRs**: the audit PR (or, when the audit forced no fix, its + report), Version Packages and Prepare Python Release, each with what was + checked on it. +- **Tool schemas against the previous release**: each TypeScript server's + `tools/list` (input and output schemas, annotations and descriptions) + compared with the version the previous release published, in both eras. +- **LLM client**: one headless Claude Code run (`--strict-mcp-config`) + calling one tool on each of the seven release-candidate builds, with what + each returned (`/client-smoke`). +- **Also shipping: advisory fixes**: each fix that shipped through a security + advisory rather than a milestone issue. - **Findings**: anything that is a caveat rather than a pass, stated on its own. A row that says "verified" without saying what was run is not a ledger entry. +**A second merge PR.** When a fix lands after the first merge PR merged (4e; +in v2.2.0, #5342 followed #5338 for a `certifi` lockfile bump), the masthead +names both merge PRs with their heads, and the page says which checks ran on +which head and what differs between the two trees +(`git diff --stat <first head> <second head>`). + **4e. When verification finds something, the fix goes on `v2/main`, never on the merge branch.** File the issue (`/issue-create`), fix it through an ordinary PR, then fast-forward the merge branch to the new `origin/v2/main`, @@ -539,8 +599,8 @@ added to the draft by hand: - **Every advisory this release fixes**, as `--advisory GHSA-xxxx-yyyy-zzzz`, once each: the `[GHSA-…]` cards on the board for the milestone's work - (`security-advisory`). Its fix shipped from a private fork with no public - issue, so its reporters are no issue's author. A draft advisory works. + (`security-advisory`). Its issue is converted from the card by a + maintainer, so its reporters are no issue's author. A draft advisory works. - **Every outside contributor credited on one of the milestone's issues**, as `--thank "@login (#PR), <what it prototyped>, for #ISSUE"`, once per line of the prototype paragraph. The preview lists each candidate (below). diff --git a/.claude/skills/security-advisory/SKILL.md b/.claude/skills/security-advisory/SKILL.md index f63f22317d..a591c51c4a 100644 --- a/.claude/skills/security-advisory/SKILL.md +++ b/.claude/skills/security-advisory/SKILL.md @@ -26,15 +26,18 @@ things safe to say in public about the backlog are **aggregate counts** (how many are in each state). A branch named after the bug, or a commit message that describes it, discloses it as surely as an issue would. -Related: `/board-ops` holds the board IDs and the draft-card recipes, and -`/issue-create` gives public tracking its labels and milestone **after -publication**, never merely after the release (step 6). - -⚠️ **The PR flow does not apply to the fix itself.** It needs a public issue -and a public PR against `v2/main`, which is the disclosure this flow exists to -delay. The fix is reviewed inside the private fork (step 4), and `AGENTS.md` -records that fork PR as an exception to its PR rules (no public issue link, no -labels, tracked by the draft card). +Related: `/board-ops` holds the board IDs and the draft-card recipes, +`/issue-create` gives the converted issue its labels and milestone (step 5), +and `/pr-flow` opens the public PR that brings the fix to `v2/main` (step 5). + +⚠️ **The fix is written and reviewed in the private fork, then merged through +an opaque public PR.** The fork's PR (step 4) links no public issue, carries no +labels and is tracked by the draft card; it is never merged. When the fix is +ready to merge, the card converts to an issue that keeps its opaque title, and +a public PR from the same opaque branch closes it (step 5). That PR follows +`/pr-flow` except that **nothing in it describes the vulnerability** and it +gets **no Copilot review**. `AGENTS.md` records both PRs as an exception to +its PR rules. ## The flow @@ -44,8 +47,8 @@ labels, tracked by the draft card). | 2 | **Verify the claim**: which server, and does the code belong to it or to an SDK? | Judgment | | 3 | Valid and ours → **accept** (`triage` → `draft`); otherwise close with a reason | **Human only** | | 4 | Create the **private fork**, fix and review there | Mechanical | -| 5 | Merge to `v2/main`, milestone merge, **release**, close the fork's PR, then **publish** | **Human only** (closing the fork's PR is mechanical) | -| 6 | **After publication**, convert the draft card into a public issue | Mechanical | +| 5 | Close to the release: **convert the card** to an opaque issue, open the opaque **public PR**, merge it, close the fork's PR; then milestone merge, **release**, **publish** | **Human only** for merging, releasing and publishing (converting, the PR and closing the fork's PR are mechanical) | +| 6 | **After publication**, give the issue its public title and the advisory link | Mechanical | There is **one release line**. Every fix lands on `v2/main` and ships through the next milestone merge to `main` and a release, as any other change does @@ -197,7 +200,8 @@ moves the advisory `triage` → `draft`. The API shows it as `state` and normally assigning a milestone, and a draft card cannot carry one, so for an advisory draft **accepting the advisory is the approval**. `AGENTS.md` records this as the advisory exemption to its `Incoming` ⇔ milestone rule. The milestone -arrives with the public issue in step 6, where the ordinary rule resumes. +arrives when the card converts to an issue in step 5, where the ordinary rule +resumes. **Invalid, out of scope, a duplicate, or an SDK's → close** with a comment saying which, and why; for an SDK's, say where it was routed. A reporter who is @@ -249,8 +253,10 @@ any other change: a regression test that fails before the fix, and the gate under **Before pushing**. Name the branch `v2/fix/<ghsa-id>` (lowercase, **no descriptive slug**, since a branch name outlives the fork once merged), and keep commit messages equally opaque until -publication. **The fork's PR targets `v2/main`**, never `main`: check its base -before anyone merges it. +publication. **The fork's PR targets `v2/main`**, never `main`, so the diff +reviewed there is the diff that lands. It is where the review happens; it is +**not merged**. The same branch is pushed to this repository and merged +through a public PR (step 5), and the fork's PR is then closed. ⚠️ **Move the card as the work moves.** `In Progress` when the fix is started, `In Review` when the fork's PR is open. The fork is invisible to everyone not on @@ -290,8 +296,10 @@ body nobody will read again. already accepted, and asking for the work approves it. Otherwise it goes in `Incoming`. Score its Priority with step 1's table, using the severity of the residual exposure rather than the advisory's. -- **Say in the fix PR's body what the fix leaves open**, and link the - follow-up. The advisory's text must say it too when it is published (step 5). +- **Say in the fork PR's body what the fix leaves open**, and name the + follow-up card. The public PR (step 5) says nothing about it, since + describing the gap describes the vulnerability. The advisory's text must say + it too when it is published (step 5). Create it with `/board-ops`'s draft-card recipe, using the follow-up title in place of `[$GHSA]`. @@ -308,35 +316,114 @@ of three cases: - It is non-security work outside the advisory's scope, which gets a public issue **after publication**, like any other finding. -### 5. Merge, release, publish (human only) +### 5. Bring the fix to `v2/main`, release, publish Publish **after** the fix has shipped in a release, never before: publishing discloses the vulnerability, and doing it while users have no upgrade hands out a working exploit. **Shipped** means the fixed version of that server is on npm or PyPI. -⚠️ **The patch stops being secret at MERGE, not at publish.** Merging the fork -puts an ordinary public commit on `v2/main`, readable by anyone, and it then -travels through the milestone merge to `main` before `release.yml` publishes it. -The time between merge and publish is not a period of secrecy; it is a period -of **exposure to anyone reading commits**, so keep it short: merge close to the -release, and publish as soon as the release is out. +⚠️ **The patch stops being secret when its branch reaches this repository, not +at publish.** Pushing the fork's branch here (below) makes its commits readable +by anyone, the public PR shows the diff, and the merge then travels through the +milestone merge to `main` before `release.yml` publishes it. The time between +that push and the publish is not a period of secrecy; it is a period of +**exposure to anyone reading commits**, so keep it short: start this step close +to the release, merge as soon as CI passes, and publish as soon as the release +is out. The release itself is a maintainer action (`release.yml` runs only when a maintainer publishes a GitHub Release, behind the `release` environment's approval; see [`RELEASING.md`](../../../RELEASING.md)). Say which server needs a release and stop there. -⚠️ **Publishing is irreversible and human-only.** It makes the advisory public, -with no undo. Same rule as accepting: recommend, never perform. +⚠️ **Merging, releasing and publishing are human-only.** Publishing is also +irreversible: it makes the advisory public, with no undo. Same rule as +accepting: recommend, never perform. Converting the card, opening the public +PR and closing the fork's PR are mechanical. + +#### Convert the card to an opaque issue + +When the fix's review in the fork is done and the release is near, convert the +draft card to an issue, so the public PR has an issue to close and a card that +tracks it. + +⚠️ **Convert the draft; do not file a new issue.** "Convert to issue" creates a +**new** issue from the draft card, and there is no way to point an existing card +at an issue filed separately, so filing by hand and then converting produces two +issues and two cards. + +1. **Keep the opaque title**, `[GHSA-xxxx-yyyy-zzzz]`, and the + link-and-triage body (step 1). The issue takes the card's title and body, + and the advisory is still private, so no summary may appear yet (step 6 + adds it). The advisory link in the body opens only for people on the + advisory. +2. **Convert the draft card to an issue** in `modelcontextprotocol/servers` + (the card keeps its place and its field values). The recipe is in + `/board-ops`. +3. Apply the labels and milestone from `/issue-create` (steps 1 and 2): `v2`, + one type label (normally `bug`), the server's scope label, and the milestone + of the release the fix will ship in. **Skip its board step**: the converted + card is already on #43. + +A **follow-up card stays a draft** here: its write-up describes the gap, so it +converts only after publication (step 6). + +#### Open the public PR + +Push the fork's branch to this repository under the **same opaque name**, +`v2/fix/<ghsa-id>`, and open the PR with `/pr-flow` steps 4 and 6. The fork has +no CI, so run the gate on exactly what you push: + +```sh +GHSA=<GHSA_ID> +BRANCH="v2/fix/$(tr '[:upper:]' '[:lower:]' <<<"$GHSA")" +FORK=$(gh api repos/modelcontextprotocol/servers/security-advisories/$GHSA \ + --jq '.private_fork.full_name // empty') || FORK= +[ -n "$FORK" ] || echo "no private fork for $GHSA (or the lookup failed)" >&2 +WT="../$(basename "$PWD")-${BRANCH#v2/fix/}" +git fetch origin v2/main +# The push runs only after the gate exits 0, from the worktree holding the fork's head. +[ -n "$FORK" ] && git fetch "https://github.com/$FORK.git" "$BRANCH" \ + && git worktree add --detach "$WT" FETCH_HEAD \ + && cd "$WT" && npm install && npm run local:gate \ + && git push origin "HEAD:refs/heads/$BRANCH" +``` + +If the branch no longer merges cleanly onto `origin/v2/main`, resolve that in +the fork, where the review is, before pushing. + +The worktree is on a detached `HEAD`, so `gh pr create` cannot infer the +branch: add `--head "$BRANCH"` to `/pr-flow`'s create command. + +- **Title**: `fix(<server>): GHSA-xxxx-yyyy-zzzz`, and nothing more. +- **Body**: `Closes #<N>` (the converted issue) on the first line, then + `/pr-flow`'s sections, **describing no part of the vulnerability**: what it + is ("the fix for GHSA-xxxx-yyyy-zzzz, reviewed in the advisory's private + fork; details stay with the advisory until it is published"), the gate's + result, and each checklist item answered. Client evidence stays with the + advisory; say so under **How Has This Been Tested?** rather than recording + scenarios that describe the attack. +- Label it `v2`, link it with `addCloseIssueReferences` and read the link back + (`/pr-flow` step 6), and move the card to **In Review** if it is not there + already. +- ⚠️ **No Copilot review: skip `/pr-flow` steps 7 to 9.** Review comments on a + public PR are public, and a reviewer's findings would describe the + vulnerability. The review happened in the fork. Do not request one. + +#### Merge it, then close the issue + +A maintainer merges it once CI passes. Then close the issue by hand and move +its card to **Done** (`/pr-flow` step 10), as for any PR into `v2/main`: +`Done` means the work shipped, and it has. #### Close the fork's PR once its fix is on `v2/main` **GitHub will not publish an advisory while its private fork has an open pull request.** The advisory page says "you must merge or close outstanding pull requests" and **Publish advisory** stays disabled. The fix reaches `v2/main` -through a PR in this repository, so the fork's own PR is left open with -nothing still to merge, and blocks publication until someone closes it. +through the public PR above, so the fork's own PR is left open with nothing +still to merge, and blocks publication until someone closes it. Closing it is mechanical and reversible (a PR can be reopened), and it neither closes nor publishes the advisory. Close a PR **only once its head is on @@ -381,9 +468,10 @@ fork, so check each of these first: - **Nothing that matters lives only in the fork.** Publishing deletes it. Review findings worth keeping, a residual-exposure note, a follow-up's write-up: copy each onto the advisory (if it belongs in what is published) or - onto the draft card that tracks it (a follow-up's write-up goes in **its - follow-up card's** `## Write-up` section, where step 6 reads it), within step - 4's confidentiality limits, before the button is pressed. + onto the follow-up draft card that tracks it (a follow-up's write-up goes in + **its follow-up card's** `## Write-up` section, where step 6 reads it), within + step 4's confidentiality limits, before the button is pressed. Never onto the + converted issue, which is public. - **The affected product**: ecosystem (`npm` or `pip`) and package name, as published (`@modelcontextprotocol/server-filesystem`, `mcp-server-git`). @@ -402,33 +490,25 @@ fork, so check each of these first: platforms or cases are still exposed, so a user who upgrades does not believe they are fully covered. -### 6. After publication, convert the card into public tracking +### 6. After publication, give the issue its public title -**The trigger is publication, not the release.** The release ships the fix -while the advisory can still be private, and a public issue opened in that gap +**The trigger is publication, not the merge or the release.** Both happen while +the advisory can still be private, and a summary added to the issue in that gap describes a vulnerability the advisory has not disclosed yet. -⚠️ **Convert the draft; do not file a new issue.** "Convert to issue" creates a -**new** issue from the draft card, and there is no way to point an existing card -at an issue filed separately, so filing by hand and then converting produces two -issues and two cards. +1. **Retitle the issue** `[GHSA-xxxx-yyyy-zzzz] - <published summary>`. Only + now, with the advisory public, may the summary appear. +2. **Replace the body's triage lines** with a link to the now-public advisory. -1. **Give the card its public title first**, `[GHSA-xxxx-yyyy-zzzz] - <published - summary>` (edit the draft's title on the card), since the issue takes the - card's title. Only now, with the advisory public, may the summary appear. -2. **Convert the draft card to an issue** in `modelcontextprotocol/servers` - (the card keeps its place and its field values). The recipe is in - `/board-ops`. -3. Apply the labels and milestone from `/issue-create` (steps 1 and 2): `v2`, - one type label (normally `bug`), the server's scope label, and the milestone - of the release the fix shipped in. **Skip its board step**: the converted - card is already on #43. -4. Replace the body's triage lines with a link to the now-public advisory, and - **close** the issue: the work shipped before the issue existed. -5. Move the card to **`Done`**, which is right here because the fix shipped. +```sh +gh issue edit <N> --repo modelcontextprotocol/servers \ + --title "[GHSA-xxxx-yyyy-zzzz] - <published summary>" --body-file <file> +``` + +The issue is already closed and its card in `Done` (step 5); neither changes. -**A follow-up card converts at the same time, but stays open**, because its -work has not shipped: +**A follow-up card converts now, and stays open**, because its work has not +shipped: 1. Retitle it `[GHSA-xxxx-yyyy-zzzz follow-up] - <summary>`. 2. Convert it. diff --git a/.github/workflows/claude.yml b/.github/workflows/claude.yml deleted file mode 100644 index f37a77662c..0000000000 --- a/.github/workflows/claude.yml +++ /dev/null @@ -1,65 +0,0 @@ -name: Claude Code - -on: - issue_comment: - types: [created] - pull_request_review_comment: - types: [created] - issues: - types: [opened, assigned] - pull_request_review: - types: [submitted] - -jobs: - # This job holds a credential twice over (`id-token: write`, and the - # `ANTHROPIC_API_KEY` secret), so every action in it is pinned to a commit - # SHA with the release it came from in a trailing comment. A moving tag - # here would let whoever can move it run code next to both. - # `verify:action-pins` enforces it. - # - # It runs only for a trusted author (#4795). A mention is not enough: the - # repository is public, so anyone could otherwise start an agent that holds - # a shell and the API key, and steer it with the text of their comment. - # Each event checks the author of the text that carries the mention (the - # comment, the review, or the issue), never someone further up the thread, - # and only OWNER, MEMBER and COLLABORATOR count. An outside contributor's - # issue that mentions @claude starts nothing; a maintainer's comment on it - # does. `scripts/claude-workflow.test.mjs` evaluates this condition against - # each event, so a disjunct that loses its check fails the script tests. - claude: - timeout-minutes: 30 - if: | - (github.event_name == 'issue_comment' && contains(github.event.comment.body, '@claude') && contains(fromJSON('["OWNER","MEMBER","COLLABORATOR"]'), github.event.comment.author_association)) || - (github.event_name == 'pull_request_review_comment' && contains(github.event.comment.body, '@claude') && contains(fromJSON('["OWNER","MEMBER","COLLABORATOR"]'), github.event.comment.author_association)) || - (github.event_name == 'pull_request_review' && contains(github.event.review.body, '@claude') && contains(fromJSON('["OWNER","MEMBER","COLLABORATOR"]'), github.event.review.author_association)) || - (github.event_name == 'issues' && (contains(github.event.issue.body, '@claude') || contains(github.event.issue.title, '@claude')) && contains(fromJSON('["OWNER","MEMBER","COLLABORATOR"]'), github.event.issue.author_association)) - runs-on: ubuntu-latest - permissions: - contents: read - pull-requests: read - issues: read - id-token: write - actions: read - steps: - - name: Checkout repository - uses: actions/checkout@d23441a48e516b6c34aea4fa41551a30e30af803 # v6.1.0 - with: - fetch-depth: 1 - - - name: Run Claude Code - id: claude - uses: anthropics/claude-code-action@12dd8d74c712f5f3669365b2369b558c495b1104 # v1.0.238 - with: - anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }} - - # Allow Claude to read CI results on PRs - additional_permissions: | - actions: read - - # Trigger when assigned to an issue - assignee_trigger: "claude" - - claude_args: | - --mcp-config .mcp.json - --allowedTools "Bash,mcp__mcp-docs,WebFetch" - --append-system-prompt "If posting a comment to GitHub, give a concise summary of the comment at the top and put all the details in a <details> block. When working on MCP-related code or reviewing MCP-related changes, use the mcp-docs MCP server to look up the latest protocol documentation. For schema details, reference https://github.com/modelcontextprotocol/modelcontextprotocol/tree/main/schema which contains versioned schemas in JSON (schema.json) and TypeScript (schema.ts) formats." diff --git a/.github/workflows/python.yml b/.github/workflows/python.yml index f5ddd6b26a..219e515924 100644 --- a/.github/workflows/python.yml +++ b/.github/workflows/python.yml @@ -142,24 +142,16 @@ jobs: working-directory: src/${{ matrix.package }} run: uv sync --locked --all-extras --dev - # fetch's tests pin readabilipy's Readability.js path, which needs Node - # and readabilipy's JavaScript dependencies. readabilipy >= 0.3.0 ships - # only their package.json and runs `npm install` on first use, but on - # Windows its subprocess call cannot find npm (npm.cmd), so it falls back - # to the pure-Python extractor. Install them here, as an operator would. + # fetch's tests pin the Readability.js path, which needs Node and npm. + # readabilipy >= 0.3.0 ships only its JavaScript's package.json, and the + # server installs the dependencies on first use, resolving npm through + # PATH so Windows' npm.cmd is found (#5313). Nothing is pre-installed + # here, so this leg proves that install works on Windows. - uses: actions/setup-node@v7 if: matrix.package == 'fetch' with: node-version: 24 - - name: Install readabilipy's JavaScript dependencies - if: matrix.package == 'fetch' - working-directory: src/fetch - shell: bash - run: | - jsdir=$(uv run --frozen python -c "import os, readabilipy; print(os.path.join(os.path.dirname(readabilipy.__file__), 'javascript'))") - cd "$jsdir" && npm install --no-audit --no-fund - - name: Run pytest working-directory: src/${{ matrix.package }} run: uv run --frozen pytest diff --git a/.github/workflows/sdk-watch.yml b/.github/workflows/sdk-watch.yml index f0497ea490..7d9850cf71 100644 --- a/.github/workflows/sdk-watch.yml +++ b/.github/workflows/sdk-watch.yml @@ -31,7 +31,7 @@ # Tokens: `GITHUB_TOKEN` files and comments (`issues: write`). It cannot write # the org project board, so the issue arrives labeled and milestoned and # `issue-triage` boards it; boarding from here waits on a GitHub App (#5061). -# `ANTHROPIC_API_KEY` is the organization secret `claude.yml` already uses. +# `ANTHROPIC_API_KEY` is an organization secret. name: SDK Watch on: diff --git a/AGENTS.md b/AGENTS.md index 8fe94f01a7..a5e5b058d1 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -63,7 +63,7 @@ servers/ │ overview of how the rules, skills, gates and sweeps fit together ├── .github/workflows/ typescript.yml, python.yml (per-package CI), release.yml (publishes on a GitHub Release), │ version-packages.yml (the changesets PR), prepare-python-release.yml (the CalVer PR), -│ claude.yml (@claude mentions), everything-mcp-diff.yml (everything's interface diff on PRs), +│ everything-mcp-diff.yml (everything's interface diff on PRs), │ main-source.yml (a PR into main must bring nothing that is not on v2/main), │ and the scheduled sweeps that file issues: dependency-refresh.yml, dependabot-alerts.yml, sdk-watch.yml ├── .github/ISSUE_TEMPLATE/ Bug and feature issue forms; config.yml routes security and new servers away @@ -171,8 +171,8 @@ diagnosing a red stage is the `pre-push-gate` skill. A workflow job **holds a credential** when it can mint an OIDC token or push a package (`id-token: write`, `packages: write`), or is handed any secret other -than `GITHUB_TOKEN`. Today that is `release.yml`'s publish jobs, `claude.yml`, -and `sdk-watch.yml`'s `analyze` and `post` jobs (`ANTHROPIC_API_KEY`). Jobs +than `GITHUB_TOKEN`. Today that is `release.yml`'s publish jobs and +`sdk-watch.yml`'s `analyze` and `post` jobs (`ANTHROPIC_API_KEY`). Jobs that hold none can still need pinning under the rules below: `release.yml`'s build jobs and `sdk-watch.yml`'s `sweep`, whose outputs `analyze` reads. @@ -184,12 +184,6 @@ build jobs and `sdk-watch.yml`'s `sweep`, whose outputs `analyze` reads. downloads**, since what it builds is what gets published, and for **a job whose outputs a credentialed job reads**, since the reader acts on them. `verify:action-pins` enforces both. Other jobs keep moving major tags. -- **A credentialed job that a comment, review or issue can start runs only - for a trusted author**: `OWNER`, `MEMBER` or `COLLABORATOR` in the - `author_association` of the text that triggered it, never of the thread - around it. Today that is `claude.yml`, whose `if:` checks it on every event - it listens to; `scripts/claude-workflow.test.mjs` evaluates that condition - per event and fails when one loses its check. - **A job that holds a publish credential does not install dependencies or build.** `release.yml` builds, tests and verifies each package in a job with no `id-token`, and the publish job only downloads that artifact and @@ -359,15 +353,22 @@ every commit a pull request adds; one unsigned commit fails it. The repair work is invisible to the board. If there is no issue yet, create it first. This holds for a maintainer's own one-line fix as much as for a feature. -- **Exception: a security advisory's fix, in its private fork.** A PR in the - advisory's temporary private fork (`servers-ghsa-xxxx-yyyy-zzzz`) links no - public issue, carries no labels, and is tracked by the advisory's `[GHSA-…]` draft - card rather than an issue's card, because each of those would disclose the - vulnerability before a fix ships. For the same reason the work-start rules - on assigning an issue and naming the branch `v2/<type>/<ISSUE_NUMBER>-<slug>` - do not apply: its branch is `v2/fix/<ghsa-id>`, with **no descriptive - slug**. It still targets `v2/main`, and it is reviewed inside the fork. The - flow is the `security-advisory` skill. +- **Exception: a security advisory's fix.** It is written and reviewed in a + PR in the advisory's temporary private fork (`servers-ghsa-xxxx-yyyy-zzzz`), + which links no public issue, carries no labels, and is tracked by the + advisory's `[GHSA-…]` draft card rather than an issue's card, because each + of those would disclose the vulnerability before a fix ships. For the same + reason the work-start rules on assigning an issue and naming the branch + `v2/<type>/<ISSUE_NUMBER>-<slug>` do not apply: its branch is + `v2/fix/<ghsa-id>`, with **no descriptive slug**. It targets `v2/main`, and + it is **not merged**. When the fix is ready to merge, close to the release, + the draft card converts to an issue that keeps its opaque title, and that + branch is pushed here under the same name and merged through a **public PR** + that `Closes` that issue and follows every rule above, except that its + title and body describe no part of the vulnerability and it gets **no + Copilot review loop**: review comments on a public PR are public, and the + review already happened in the fork. The fork's PR is then closed. The flow + is the `security-advisory` skill. - **Exception: a milestone's merge PR.** It is the one PR that targets `main`. It is a pure merge of `v2/main` (its branch is `origin/v2/main` with no @@ -480,13 +481,16 @@ here. card** titled with the bare id, `[GHSA-xxxx-yyyy-zzzz]`, because a real issue would disclose the vulnerability before a fix exists. The title stays that opaque until publication: board access is wider than advisory access, - and a summary can name the server or the attack. It becomes a public - issue by conversion **once the advisory is published**, never merely once - the fix ships. Work the fix leaves open is tracked by a second draft card, - `[GHSA-xxxx-yyyy-zzzz follow-up]`, under the same rules. It stays open - after conversion, because its work has not shipped. The `[GHSA-` prefix is - load-bearing: **any other draft card is still a defect to delete**. The flow - is the `security-advisory` skill. + and a summary can name the server or the attack. It converts to an issue + **when its fix is ready to merge**, keeping that opaque title, so the + public PR that brings the fix to `v2/main` has an issue to close; the + summary joins the title only **once the advisory is published**, never + merely once the fix ships. Work the fix leaves open is tracked by a second + draft card, `[GHSA-xxxx-yyyy-zzzz follow-up]`, under the same rules, except + that it converts only once the advisory is published, since its write-up + describes the gap. It stays open after conversion, because its work has + not shipped. The `[GHSA-` prefix is load-bearing: **any other draft card + is still a defect to delete**. The flow is the `security-advisory` skill. **Accepting, closing and publishing an advisory, and replying to its reporter, are human-only; never automate or bulk-apply any of them.** - **Only issues go on the board, never PRs.** A PR is tracked through its linked @@ -511,8 +515,8 @@ here. approval. A `[GHSA-` advisory draft is exempt: a draft card cannot carry a milestone, so its approval act is **accepting the advisory**, which moves it - `Incoming` → `Todo`. Its milestone arrives with the public issue after - publication. + `Incoming` → `Todo`. Its milestone arrives when the card converts to an + issue, before the fix's public PR opens. - **`Done` means the work shipped**: its PR merged, or it is a parent whose last sub-issue closed. A duplicate, won't-fix, not-planned or superseded issue shipped nothing, so its card is **deleted** rather than moved to Done. @@ -573,7 +577,9 @@ here. so **link it explicitly** with the `addCloseIssueReferences` GraphQL mutation right after opening it. **On merge, close the issue by hand and move its card to Done.** Keep the `Closes` line anyway. -- **After opening a PR, run a Copilot review loop to convergence, unprompted.** +- **After opening a PR, run a Copilot review loop to convergence, unprompted** + (except a security advisory's public fix PR; see + [Contributing](#contributing)). Request a review, wait for the round to post or for Copilot's session to end without one, answer every comment, and request again only when you pushed a fix or the last round ended without a review. Stop on the **first** clean diff --git a/RELEASING.md b/RELEASING.md index c44cf95773..c1824afea2 100644 --- a/RELEASING.md +++ b/RELEASING.md @@ -39,7 +39,7 @@ node scripts/prepare-python-release.mjs # prints one "name: old -> new" line ### PRs opened by these workflows > [!NOTE] -> Both workflows push their PR's branch with the workflow token, and a push made with that token starts no workflow, so each one then dispatches the TypeScript, Python and DCO workflows on that branch, and Version Packages the Everything diff too (`workflow_dispatch` is the one event the token can start; the Python stamps touch nothing the diff runs on). A PR does not list dispatched checks, so each dispatched run also reports itself on the PR's head commit as a status: `TypeScript (dispatched)`, `Python (dispatched)` and `DCO (dispatched)`, plus `Everything diff (dispatched)` on Version Packages; wait for them to pass before merging. Closing and reopening the PR does not start the CI workflows: that fires `pull_request`, and those run on `push`. The token's `pull_request` events do not run DCO or the diff either: GitHub holds those runs as `action_required` (#5173), so the PR may also list them as pending or failed with no jobs; the `(dispatched)` statuses are the results. To run one again by hand: `gh workflow run typescript.yml --ref <branch>` (or `python.yml`, `dco.yml`, `everything-mcp-diff.yml`). The Version Packages PR's body is rewritten on every push to `v2/main`, so it is linked to the release issue with `addCloseIssueReferences` (which lives outside the body), carries a `Part of #N` comment as its textual record, and gets `Part of #N` as its body's first line only immediately before it is merged. The repository setting **Settings → Actions → General → Allow GitHub Actions to create and approve pull requests** must be on, or the workflows fail at the step that opens the PR. +> Both workflows push their PR's branch with the workflow token, and a push made with that token starts no workflow, so each one then dispatches the TypeScript, Python and DCO workflows on that branch, and Version Packages the Everything diff too (`workflow_dispatch` is the one event the token can start; the Python stamps touch nothing the diff runs on). A PR does not list dispatched checks, so each dispatched run also reports itself on the PR's head commit as a status: `TypeScript (dispatched)`, `Python (dispatched)` and `DCO (dispatched)`, plus `Everything diff (dispatched)` on Version Packages; wait for them to pass before merging. Closing and reopening the PR does not start the CI workflows: that fires `pull_request`, and those run on `push`. The token's `pull_request` events do not run DCO or the diff either: GitHub holds those runs as `action_required` (#5173), so the PR may also list them as pending or failed with no jobs; the `(dispatched)` statuses are the results. To run one again by hand: `gh workflow run typescript.yml --ref <branch>` (or `python.yml`, `dco.yml`, `everything-mcp-diff.yml`). The Version Packages PR's body is rewritten on every push to `v2/main`, so it is linked to the release issue with `addCloseIssueReferences` (which lives outside the body), carries a `Part of #N` comment as its textual record, and gets `Part of #N` as its body's first line only immediately before it is merged. The Prepare Python Release PR keeps its `Closes #N` line and is linked to the release issue with `addCloseIssueReferences` too, as every PR into `v2/main` is. The repository setting **Settings → Actions → General → Allow GitHub Actions to create and approve pull requests** must be on, or the workflows fail at the step that opens the PR. ## How publishing works @@ -78,7 +78,7 @@ A release ships one **milestone**. The step-by-step procedure, with its commands 2. **Preparation PRs land on `v2/main`**, each starting with `Closes #<release issue>`: - the **audit** (`npm audit`, and `pip-audit` over each Python server's locked requirements): the report goes on the release issue, and any fix it forces is its own commit in a PR; - the **Version Packages** PR (TypeScript), which is linked to the release issue with `addCloseIssueReferences` and a `Part of #N` comment, and gets `Part of #N` as its body's first line just before merging, because the changesets action rewrites its body on every push; - - the **Prepare Python Release** PR (Python). + - the **Prepare Python Release** PR (Python), which keeps its `Closes #N` line and is also linked to the release issue with `addCloseIssueReferences`, since on `v2/main` a closing keyword is only a cross-reference. A milestone that touches both languages and needs an audit fix has all three. They are merged before the next step starts. 3. **The merge PR** takes `v2/main` into `main`. It is a pure merge: its branch is `origin/v2/main` with no commits of its own, and the merged tree equals `origin/v2/main`'s. Its body starts with `Part of #<release issue>`, not `Closes`, because `main` is the default branch and a closing keyword there would close the issue before anything is published. It is merged with a merge commit, never squashed. diff --git a/docs/ai-software-factory.md b/docs/ai-software-factory.md index e0d5356ead..85e46296ab 100644 --- a/docs/ai-software-factory.md +++ b/docs/ai-software-factory.md @@ -212,8 +212,9 @@ A privately reported vulnerability is tracked by a draft board card titled with the bare advisory id, `[GHSA-xxxx-yyyy-zzzz]`, because a public issue would disclose it before a fix exists. The `security-advisory` skill covers checking whether the code is this repository's or the SDK's underneath, the -private fork the fix is built in, and turning the card into public tracking -once the advisory is published. **Accepting, closing and publishing an +private fork the fix is built in, converting the card to an issue with the +same opaque title so an opaque public PR can bring the fix to `v2/main`, and +giving that issue its public title once the advisory is published. **Accepting, closing and publishing an advisory, and replying to its reporter, are human-only acts**, never automated or bulk-applied. diff --git a/package-lock.json b/package-lock.json index 65623def4d..0348a6cfbf 100644 --- a/package-lock.json +++ b/package-lock.json @@ -667,6 +667,12 @@ "url": "https://github.com/sponsors/nzakas" } }, + "node_modules/@isaacs/nonrecursive-glob-match": { + "version": "1.0.3", + "resolved": "https://registry.npmjs.org/@isaacs/nonrecursive-glob-match/-/nonrecursive-glob-match-1.0.3.tgz", + "integrity": "sha512-SI7kyKHq4XXDRozw+r9GlRZkia1ZkFrpLW98r2IC2Bpr8YkFGE3fvjzmxCG0x78A++69HmDNrDvSMYBZmHGpKA==", + "license": "BlueOak-1.0.0" + }, "node_modules/@jridgewell/resolve-uri": { "version": "3.1.2", "resolved": "https://registry.npmjs.org/@jridgewell/resolve-uri/-/resolve-uri-3.1.2.tgz", @@ -1414,23 +1420,6 @@ "@types/node": "*" } }, - "node_modules/@types/yargs": { - "version": "17.0.35", - "resolved": "https://registry.npmjs.org/@types/yargs/-/yargs-17.0.35.tgz", - "integrity": "sha512-qUHkeCyQFxMXg79wQfTtfndEC+N9ZZg76HJftDJp+qH2tV7Gj4OJi7l+PiWwJ+pWtW8GwSmqsDj/oymhrTWXjg==", - "dev": true, - "license": "MIT", - "dependencies": { - "@types/yargs-parser": "*" - } - }, - "node_modules/@types/yargs-parser": { - "version": "21.0.3", - "resolved": "https://registry.npmjs.org/@types/yargs-parser/-/yargs-parser-21.0.3.tgz", - "integrity": "sha512-I4q9QU9MQv4oEOz4tAHJtNz1cwuLxn2F3xcc2iV5WdqLPpUnj30aUuxt1mAxYTG+oe8CZMV/+6rU4S4gRDzqtQ==", - "dev": true, - "license": "MIT" - }, "node_modules/@typescript-eslint/eslint-plugin": { "version": "8.71.0", "resolved": "https://registry.npmjs.org/@typescript-eslint/eslint-plugin/-/eslint-plugin-8.71.0.tgz", @@ -2053,111 +2042,6 @@ "url": "https://github.com/chalk/chalk?sponsor=1" } }, - "node_modules/cliui": { - "version": "8.0.1", - "resolved": "https://registry.npmjs.org/cliui/-/cliui-8.0.1.tgz", - "integrity": "sha512-BSeNnyus75C4//NQ9gQt1/csTXyo/8Sb+afLAkzAptFuMsod9HFokGNudZpi/oQV73hnVK+sR+5PVRMd+Dr7YQ==", - "license": "ISC", - "dependencies": { - "string-width": "^4.2.0", - "strip-ansi": "^6.0.1", - "wrap-ansi": "^7.0.0" - }, - "engines": { - "node": ">=12" - } - }, - "node_modules/cliui/node_modules/ansi-regex": { - "version": "5.0.1", - "resolved": "https://registry.npmjs.org/ansi-regex/-/ansi-regex-5.0.1.tgz", - "integrity": "sha512-quJQXlTSUGL2LH9SUXo8VwsY4soanhgo6LNSm84E1LBcE8s3O0wpdiRzyR9z/ZZJMlMWv37qOOb9pdJlMUEKFQ==", - "license": "MIT", - "engines": { - "node": ">=8" - } - }, - "node_modules/cliui/node_modules/ansi-styles": { - "version": "4.3.0", - "resolved": "https://registry.npmjs.org/ansi-styles/-/ansi-styles-4.3.0.tgz", - "integrity": "sha512-zbB9rCJAT1rbjiVDb2hqKFHNYLxgtk8NURxZ3IZwD3F6NtxbXZQCnnSi1Lkx+IDohdPlFp222wVALIheZJQSEg==", - "license": "MIT", - "dependencies": { - "color-convert": "^2.0.1" - }, - "engines": { - "node": ">=8" - }, - "funding": { - "url": "https://github.com/chalk/ansi-styles?sponsor=1" - } - }, - "node_modules/cliui/node_modules/emoji-regex": { - "version": "8.0.0", - "resolved": "https://registry.npmjs.org/emoji-regex/-/emoji-regex-8.0.0.tgz", - "integrity": "sha512-MSjYzcWNOA0ewAHpz0MxpYFvwg6yjy1NG3xteoqz644VCo/RPgnr1/GGt+ic3iJTzQ8Eu3TdM14SawnVUmGE6A==", - "license": "MIT" - }, - "node_modules/cliui/node_modules/string-width": { - "version": "4.2.3", - "resolved": "https://registry.npmjs.org/string-width/-/string-width-4.2.3.tgz", - "integrity": "sha512-wKyQRQpjJ0sIp62ErSZdGsjMJWsap5oRNihHhu6G7JVO/9jIB6UyevL+tXuOqrng8j/cxKTWyWUwvSTriiZz/g==", - "license": "MIT", - "dependencies": { - "emoji-regex": "^8.0.0", - "is-fullwidth-code-point": "^3.0.0", - "strip-ansi": "^6.0.1" - }, - "engines": { - "node": ">=8" - } - }, - "node_modules/cliui/node_modules/strip-ansi": { - "version": "6.0.1", - "resolved": "https://registry.npmjs.org/strip-ansi/-/strip-ansi-6.0.1.tgz", - "integrity": "sha512-Y38VPSHcqkFrCpFnQ9vuSXmquuv5oXOKpGeT6aGrr3o3Gc9AlVa6JBfUSOCnbxGGZF+/0ooI7KrPuUSztUdU5A==", - "license": "MIT", - "dependencies": { - "ansi-regex": "^5.0.1" - }, - "engines": { - "node": ">=8" - } - }, - "node_modules/cliui/node_modules/wrap-ansi": { - "version": "7.0.0", - "resolved": "https://registry.npmjs.org/wrap-ansi/-/wrap-ansi-7.0.0.tgz", - "integrity": "sha512-YVGIj2kamLSTxw6NsZjoBxfSwsn0ycdesmc4p+Q21c5zPuZ1pl+NfxVdxPtdHvmNVOQ6XSYG4AUtyt/Fi7D16Q==", - "license": "MIT", - "dependencies": { - "ansi-styles": "^4.0.0", - "string-width": "^4.1.0", - "strip-ansi": "^6.0.0" - }, - "engines": { - "node": ">=10" - }, - "funding": { - "url": "https://github.com/chalk/wrap-ansi?sponsor=1" - } - }, - "node_modules/color-convert": { - "version": "2.0.1", - "resolved": "https://registry.npmjs.org/color-convert/-/color-convert-2.0.1.tgz", - "integrity": "sha512-RRECPsj7iu/xb5oKYcsFHSppFNnsj/52OVTRKb4zP5onXwVF3zVmmToNcOfGC+CRDpfK/U584fMg38ZHCaElKQ==", - "license": "MIT", - "dependencies": { - "color-name": "~1.1.4" - }, - "engines": { - "node": ">=7.0.0" - } - }, - "node_modules/color-name": { - "version": "1.1.4", - "resolved": "https://registry.npmjs.org/color-name/-/color-name-1.1.4.tgz", - "integrity": "sha512-dOy+3AuW3a2wNbZHIuMZpTcgjGuLU/uBL/ubcZF9OXbDo8ff4O8yVp5Bf0efS8uEoYo5q4Fx7dY9OgQGXgAsQA==", - "license": "MIT" - }, "node_modules/content-disposition": { "version": "1.1.0", "resolved": "https://registry.npmjs.org/content-disposition/-/content-disposition-1.1.0.tgz", @@ -2378,15 +2262,6 @@ "node": ">= 0.4" } }, - "node_modules/escalade": { - "version": "3.2.0", - "resolved": "https://registry.npmjs.org/escalade/-/escalade-3.2.0.tgz", - "integrity": "sha512-WUj2qlxaQtO4g6Pq5c29GTcWGDyd8itL8zTlipgECz3JesAiiOKotd8JU6otB3PACgG6xkJUyVhboMS+bje/jA==", - "license": "MIT", - "engines": { - "node": ">=6" - } - }, "node_modules/escape-html": { "version": "1.0.3", "resolved": "https://registry.npmjs.org/escape-html/-/escape-html-1.0.3.tgz", @@ -3035,15 +2910,6 @@ "url": "https://github.com/sponsors/ljharb" } }, - "node_modules/get-caller-file": { - "version": "2.0.5", - "resolved": "https://registry.npmjs.org/get-caller-file/-/get-caller-file-2.0.5.tgz", - "integrity": "sha512-DyFP3BM/3YHTQOCUL/w0OZHR0lpKeGrxotcHWcqNEdnltqFwXVfhEBQ94eIo34AfQpo0rGki4cyIiftY06h2Fg==", - "license": "ISC", - "engines": { - "node": "6.* || 8.* || >= 10.*" - } - }, "node_modules/get-intrinsic": { "version": "1.3.0", "resolved": "https://registry.npmjs.org/get-intrinsic/-/get-intrinsic-1.3.0.tgz", @@ -3094,23 +2960,6 @@ "node": ">=6" } }, - "node_modules/glob": { - "version": "13.0.6", - "resolved": "https://registry.npmjs.org/glob/-/glob-13.0.6.tgz", - "integrity": "sha512-Wjlyrolmm8uDpm/ogGyXZXb1Z+Ca2B8NbJwqBVg0axK9GbBeoS7yGV6vjXnYdGm6X53iehEuxxbyiKp8QmN4Vw==", - "license": "BlueOak-1.0.0", - "dependencies": { - "minimatch": "^10.2.2", - "minipass": "^7.1.3", - "path-scurry": "^2.0.2" - }, - "engines": { - "node": "18 || 20 || >=22" - }, - "funding": { - "url": "https://github.com/sponsors/isaacs" - } - }, "node_modules/glob-parent": { "version": "5.1.2", "resolved": "https://registry.npmjs.org/glob-parent/-/glob-parent-5.1.2.tgz", @@ -3369,15 +3218,6 @@ "node": ">=0.10.0" } }, - "node_modules/is-fullwidth-code-point": { - "version": "3.0.0", - "resolved": "https://registry.npmjs.org/is-fullwidth-code-point/-/is-fullwidth-code-point-3.0.0.tgz", - "integrity": "sha512-zymm5+u+sCsSWyD9qNaejV3DFvhCKclKdizYaJUuHA83RLjb7nSuGnddCHGv0hk+KY7BMAlsWeK4Ueg6EV6XQg==", - "license": "MIT", - "engines": { - "node": ">=8" - } - }, "node_modules/is-glob": { "version": "4.0.3", "resolved": "https://registry.npmjs.org/is-glob/-/is-glob-4.0.3.tgz", @@ -3866,15 +3706,6 @@ "url": "https://github.com/sponsors/sindresorhus" } }, - "node_modules/lru-cache": { - "version": "11.5.2", - "resolved": "https://registry.npmjs.org/lru-cache/-/lru-cache-11.5.2.tgz", - "integrity": "sha512-4pfM1Ff0x50o0tQwb5ucw/RzNyD0/YJME6IVcStalZuMWxdt3sR3huStTtxz4PUmvZfRguvDejasvQ2kifR11g==", - "license": "BlueOak-1.0.0", - "engines": { - "node": "20 || >=22" - } - }, "node_modules/magic-string": { "version": "0.30.21", "resolved": "https://registry.npmjs.org/magic-string/-/magic-string-0.30.21.tgz", @@ -4039,6 +3870,7 @@ "version": "10.2.5", "resolved": "https://registry.npmjs.org/minimatch/-/minimatch-10.2.5.tgz", "integrity": "sha512-MULkVLfKGYDFYejP07QOurDLLQpcjk7Fw+7jXS2R2czRQzR56yHRveU5NDJEOviH+hETZKSkIk5c+T23GjFUMg==", + "dev": true, "license": "BlueOak-1.0.0", "dependencies": { "brace-expansion": "^5.0.5" @@ -4060,15 +3892,6 @@ "url": "https://github.com/sponsors/ljharb" } }, - "node_modules/minipass": { - "version": "7.1.3", - "resolved": "https://registry.npmjs.org/minipass/-/minipass-7.1.3.tgz", - "integrity": "sha512-tEBHqDnIoM/1rXME1zgka9g6Q2lcoCkxHLuc7ODJ5BxbP5d4c2Z5cGgtXAku59200Cx7diuHTOYfSBD8n6mm8A==", - "license": "BlueOak-1.0.0", - "engines": { - "node": ">=16 || 14 >=14.17" - } - }, "node_modules/ms": { "version": "2.1.3", "resolved": "https://registry.npmjs.org/ms/-/ms-2.1.3.tgz", @@ -4305,22 +4128,6 @@ "dev": true, "license": "MIT" }, - "node_modules/path-scurry": { - "version": "2.0.2", - "resolved": "https://registry.npmjs.org/path-scurry/-/path-scurry-2.0.2.tgz", - "integrity": "sha512-3O/iVVsJAPsOnpwWIeD+d6z/7PmqApyQePUtCndjatj/9I5LylHvt5qluFaBT3I5h3r1ejfR056c+FCv+NnNXg==", - "license": "BlueOak-1.0.0", - "dependencies": { - "lru-cache": "^11.0.0", - "minipass": "^7.1.2" - }, - "engines": { - "node": "18 || 20 || >=22" - }, - "funding": { - "url": "https://github.com/sponsors/isaacs" - } - }, "node_modules/path-to-regexp": { "version": "8.4.2", "resolved": "https://registry.npmjs.org/path-to-regexp/-/path-to-regexp-8.4.2.tgz", @@ -4499,9 +4306,9 @@ "license": "MIT" }, "node_modules/qs": { - "version": "6.16.0", - "resolved": "https://registry.npmjs.org/qs/-/qs-6.16.0.tgz", - "integrity": "sha512-h6fhOIaRrID2CbEY2fqs+7t+UXZo+MLAnU5gRIq85uFtdiUPCdsApMlHhXogKVM4HM2DVbIjGNTTYH2OcmP1vA==", + "version": "6.17.1", + "resolved": "https://registry.npmjs.org/qs/-/qs-6.17.1.tgz", + "integrity": "sha512-jd3COkaKVgGUW+ZHyEHCG9bH7RoaaiWuMgzsIBjUMVjLAaWGWpoUAVwljVvBHxRu/2qEHnadLGZrkutYUBPQJA==", "license": "BSD-3-Clause", "dependencies": { "es-define-property": "^1.0.1", @@ -4586,15 +4393,6 @@ "node": ">= 0.10" } }, - "node_modules/require-directory": { - "version": "2.1.1", - "resolved": "https://registry.npmjs.org/require-directory/-/require-directory-2.1.1.tgz", - "integrity": "sha512-fGxEI7+wsG9xrvdjsrlmL22OMTTiHRwAMroiEeMgq8gzoLC/PQr7RsRDSTLUg/bZAZtF+TVIkHc6/4RIKrui+Q==", - "license": "MIT", - "engines": { - "node": ">=0.10.0" - } - }, "node_modules/require-from-string": { "version": "2.0.2", "resolved": "https://registry.npmjs.org/require-from-string/-/require-from-string-2.0.2.tgz", @@ -5485,15 +5283,6 @@ "integrity": "sha512-l4Sp/DRseor9wL6EvV2+TuQn63dMkPjZ/sp9XkghTEbV9KlPS1xUsZ3u7/IQO4wxtcFB4bgpQPRcR3QCvezPcQ==", "license": "ISC" }, - "node_modules/y18n": { - "version": "5.0.8", - "resolved": "https://registry.npmjs.org/y18n/-/y18n-5.0.8.tgz", - "integrity": "sha512-0pfFzegeDWJHJIAmTLRP2DwHjdF5s7jo9tuztdQxAhINCdvS+3nGINqPd00AphqJR/0LhANUS6/+7SCb98YOfA==", - "license": "ISC", - "engines": { - "node": ">=10" - } - }, "node_modules/yaml": { "version": "2.9.1", "resolved": "https://registry.npmjs.org/yaml/-/yaml-2.9.1.tgz", @@ -5510,74 +5299,6 @@ "url": "https://github.com/sponsors/eemeli" } }, - "node_modules/yargs": { - "version": "17.7.2", - "resolved": "https://registry.npmjs.org/yargs/-/yargs-17.7.2.tgz", - "integrity": "sha512-7dSzzRQ++CKnNI/krKnYRV7JKKPUXMEh61soaHKg9mrWEhzFWhFnxPxGl+69cD1Ou63C13NUPCnmIcrvqCuM6w==", - "license": "MIT", - "dependencies": { - "cliui": "^8.0.1", - "escalade": "^3.1.1", - "get-caller-file": "^2.0.5", - "require-directory": "^2.1.1", - "string-width": "^4.2.3", - "y18n": "^5.0.5", - "yargs-parser": "^21.1.1" - }, - "engines": { - "node": ">=12" - } - }, - "node_modules/yargs-parser": { - "version": "21.1.1", - "resolved": "https://registry.npmjs.org/yargs-parser/-/yargs-parser-21.1.1.tgz", - "integrity": "sha512-tVpsJW7DdjecAiFpbIB1e3qxIQsE6NoPc5/eTdrbbIC4h0LVsWhnoa3g+m2HclBIujHzsxZ4VJVA+GUuc2/LBw==", - "license": "ISC", - "engines": { - "node": ">=12" - } - }, - "node_modules/yargs/node_modules/ansi-regex": { - "version": "5.0.1", - "resolved": "https://registry.npmjs.org/ansi-regex/-/ansi-regex-5.0.1.tgz", - "integrity": "sha512-quJQXlTSUGL2LH9SUXo8VwsY4soanhgo6LNSm84E1LBcE8s3O0wpdiRzyR9z/ZZJMlMWv37qOOb9pdJlMUEKFQ==", - "license": "MIT", - "engines": { - "node": ">=8" - } - }, - "node_modules/yargs/node_modules/emoji-regex": { - "version": "8.0.0", - "resolved": "https://registry.npmjs.org/emoji-regex/-/emoji-regex-8.0.0.tgz", - "integrity": "sha512-MSjYzcWNOA0ewAHpz0MxpYFvwg6yjy1NG3xteoqz644VCo/RPgnr1/GGt+ic3iJTzQ8Eu3TdM14SawnVUmGE6A==", - "license": "MIT" - }, - "node_modules/yargs/node_modules/string-width": { - "version": "4.2.3", - "resolved": "https://registry.npmjs.org/string-width/-/string-width-4.2.3.tgz", - "integrity": "sha512-wKyQRQpjJ0sIp62ErSZdGsjMJWsap5oRNihHhu6G7JVO/9jIB6UyevL+tXuOqrng8j/cxKTWyWUwvSTriiZz/g==", - "license": "MIT", - "dependencies": { - "emoji-regex": "^8.0.0", - "is-fullwidth-code-point": "^3.0.0", - "strip-ansi": "^6.0.1" - }, - "engines": { - "node": ">=8" - } - }, - "node_modules/yargs/node_modules/strip-ansi": { - "version": "6.0.1", - "resolved": "https://registry.npmjs.org/strip-ansi/-/strip-ansi-6.0.1.tgz", - "integrity": "sha512-Y38VPSHcqkFrCpFnQ9vuSXmquuv5oXOKpGeT6aGrr3o3Gc9AlVa6JBfUSOCnbxGGZF+/0ooI7KrPuUSztUdU5A==", - "license": "MIT", - "dependencies": { - "ansi-regex": "^5.0.1" - }, - "engines": { - "node": ">=8" - } - }, "node_modules/yocto-queue": { "version": "0.1.0", "resolved": "https://registry.npmjs.org/yocto-queue/-/yocto-queue-0.1.0.tgz", @@ -5612,7 +5333,7 @@ }, "src/everything": { "name": "@modelcontextprotocol/server-everything", - "version": "2.1.0", + "version": "2.2.1", "license": "SEE LICENSE IN LICENSE", "dependencies": { "@modelcontextprotocol/node": "^2.1.1", @@ -5642,13 +5363,12 @@ }, "src/filesystem": { "name": "@modelcontextprotocol/server-filesystem", - "version": "2.1.0", + "version": "2.2.0", "license": "SEE LICENSE IN LICENSE", "dependencies": { "@modelcontextprotocol/server": "^2.3.1", "diff": "^8.0.3", - "glob": "^13.0.6", - "minimatch": "^10.0.1", + "minimatch": "^11.0.4", "zod": "^4.2.0" }, "bin": { @@ -5668,9 +5388,25 @@ "node": ">=24" } }, + "src/filesystem/node_modules/minimatch": { + "version": "11.0.4", + "resolved": "https://registry.npmjs.org/minimatch/-/minimatch-11.0.4.tgz", + "integrity": "sha512-/UJ1dEoVQTjPNDh6SGlUo+ueLMmoHTas4q+8yaAR/5t6nLAIiBDJtoYUh20IYleix7ehiQkgczb84PE3uzHkxg==", + "license": "BlueOak-1.0.0", + "dependencies": { + "@isaacs/nonrecursive-glob-match": "^1.0.3", + "brace-expansion": "^5.0.12" + }, + "engines": { + "node": "20 || >=22" + }, + "funding": { + "url": "https://github.com/sponsors/isaacs" + } + }, "src/memory": { "name": "@modelcontextprotocol/server-memory", - "version": "2.0.1", + "version": "2.1.0", "license": "SEE LICENSE IN LICENSE", "dependencies": { "@modelcontextprotocol/server": "^2.3.1", @@ -5693,12 +5429,11 @@ }, "src/sequentialthinking": { "name": "@modelcontextprotocol/server-sequential-thinking", - "version": "2.0.0", + "version": "2.1.1", "license": "SEE LICENSE IN LICENSE", "dependencies": { "@modelcontextprotocol/server": "^2.3.1", "chalk": "^5.3.0", - "yargs": "^17.7.2", "zod": "^4.2.0" }, "bin": { @@ -5707,7 +5442,6 @@ "devDependencies": { "@modelcontextprotocol/client": "^2.3.1", "@types/node": "^24", - "@types/yargs": "^17.0.32", "@vitest/coverage-v8": "^4.1.12", "shx": "^0.4.0", "typescript": "^5.8.2", diff --git a/package.json b/package.json index e4d1170aca..34557784d1 100644 --- a/package.json +++ b/package.json @@ -70,7 +70,7 @@ "yaml": "^2.9.0" }, "overrides": { - "qs": ">=6.16.0", + "qs": ">=6.16.1", "hono": ">=4.13.7", "proxy-addr": "^2.0.8", "ip-address": "^10.7.4", diff --git a/scripts/claude-workflow.test.mjs b/scripts/claude-workflow.test.mjs deleted file mode 100644 index d1f502c6ae..0000000000 --- a/scripts/claude-workflow.test.mjs +++ /dev/null @@ -1,153 +0,0 @@ -// Tests for the trusted-author gate on `.github/workflows/claude.yml` (#4795). -// The `claude` job holds a shell, the API key and `id-token: write`, and it -// reads the text that mentions it, so who may start it is a security -// property. These tests evaluate the job's `if:` against a synthetic payload -// for every event the workflow listens to, rather than matching its text, so -// a disjunct that loses its author check, or checks the wrong author, fails -// here. Run via `npm run test:scripts`. -// -// The evaluator covers only the expression subset the condition uses -// (`==`, `&&`, `||`, `!`, parentheses, string literals, `github.*` paths, -// `contains` and `fromJSON`), with `contains` case-insensitive as in Actions. -// It refuses any other token, so widening the condition past that subset -// fails loudly instead of being evaluated wrongly. - -import { test } from "node:test"; -import assert from "node:assert/strict"; -import { readFileSync } from "node:fs"; -import path from "node:path"; -import { fileURLToPath } from "node:url"; -import { parse } from "yaml"; - -const repoRoot = path.resolve( - path.dirname(fileURLToPath(import.meta.url)), - "..", -); -const workflow = parse( - readFileSync(path.join(repoRoot, ".github/workflows/claude.yml"), "utf8"), -); -const condition = workflow.jobs.claude.if; - -const TOKEN = - /\s+|'(?:[^']|'')*'|==|&&|\|\||!|\(|\)|,|contains|fromJSON|github(?:\.[A-Za-z_][A-Za-z0-9_-]*)+/y; - -/** Compiles the expression subset above into a function of `github`. */ -function compile(expr) { - let js = ""; - TOKEN.lastIndex = 0; - while (TOKEN.lastIndex < expr.length) { - const at = TOKEN.lastIndex; - const match = TOKEN.exec(expr); - if (!match) { - throw new Error(`unsupported expression syntax at: ${expr.slice(at)}`); - } - const tok = match[0]; - if (tok.startsWith("'")) { - js += JSON.stringify(tok.slice(1, -1).replaceAll("''", "'")); - } else if (tok.startsWith("github.")) { - // A missing property reads as null, as it does in Actions. - js += `(${tok.replaceAll(".", "?.")} ?? null)`; - } else if (tok === "==") { - // Only `event_name` is compared, against exact event names. - js += "==="; - } else { - js += tok; - } - } - return new Function("github", "contains", "fromJSON", `return (${js});`); -} - -const fold = (v) => (typeof v === "string" ? v.toLowerCase() : v); -const eq = (a, b) => fold(a) === fold(b); -const contains = (haystack, needle) => - Array.isArray(haystack) - ? haystack.some((item) => eq(item, needle)) - : haystack != null && - String(haystack).toLowerCase().includes(String(needle).toLowerCase()); - -const evaluate = (github) => - Boolean(compile(condition)(github, contains, JSON.parse)); - -// One payload per event the workflow listens to, with the mention placed in -// the text that event carries, and `who` as that text's author association. -// The surrounding issue or pull request is always opened by a maintainer, so -// a gate that read the wrong author would let an outsider through. -const EVENTS = { - issue_comment: (who) => ({ - event: { - issue: { body: "", title: "", author_association: "OWNER" }, - comment: { body: "@claude review this", author_association: who }, - }, - }), - pull_request_review_comment: (who) => ({ - event: { - pull_request: { author_association: "OWNER" }, - comment: { body: "@claude fix this line", author_association: who }, - }, - }), - pull_request_review: (who) => ({ - event: { - pull_request: { author_association: "OWNER" }, - review: { body: "@claude summarize", author_association: who }, - }, - }), - issues: (who) => ({ - event: { - issue: { - body: "@claude look at this", - title: "", - author_association: who, - }, - }, - }), -}; - -const TRUSTED = ["OWNER", "MEMBER", "COLLABORATOR"]; -const UNTRUSTED = [ - "CONTRIBUTOR", - "FIRST_TIME_CONTRIBUTOR", - "FIRST_TIMER", - "MANNEQUIN", - "NONE", -]; - -test("every event the workflow listens to has a payload here", () => { - assert.deepEqual(Object.keys(workflow.on).sort(), Object.keys(EVENTS).sort()); -}); - -for (const [name, payload] of Object.entries(EVENTS)) { - const run = (who) => evaluate({ event_name: name, ...payload(who) }); - - test(`${name}: a trusted author's mention starts the job`, () => { - for (const who of TRUSTED) assert.equal(run(who), true, who); - }); - - test(`${name}: an untrusted author's mention starts nothing`, () => { - for (const who of UNTRUSTED) assert.equal(run(who), false, who); - }); - - test(`${name}: a trusted author without a mention starts nothing`, () => { - const github = { event_name: name, ...payload("OWNER") }; - const text = - github.event.comment ?? github.event.review ?? github.event.issue; - text.body = "no mention here"; - assert.equal(evaluate(github), false); - }); -} - -test("an outsider's issue title mention starts nothing", () => { - const github = { - event_name: "issues", - event: { - issue: { body: "", title: "@claude do it", author_association: "NONE" }, - }, - }; - assert.equal(evaluate(github), false); - github.event.issue.author_association = "MEMBER"; - assert.equal(evaluate(github), true); -}); - -test("the evaluator refuses syntax it does not model", () => { - assert.throws(() => compile("github.event_name != 'x'"), /unsupported/); - assert.throws(() => compile("startsWith(github.ref, 'x')"), /unsupported/); -}); diff --git a/scripts/lib/sweep.mjs b/scripts/lib/sweep.mjs index 5daea0a0aa..9e380110a7 100644 --- a/scripts/lib/sweep.mjs +++ b/scripts/lib/sweep.mjs @@ -48,6 +48,14 @@ export const AUTOMATION_LOGIN = "github-actions"; /** Labels on every sweep issue. `chore` and `dependencies` need write access. */ export const SWEEP_LABELS = ["v2", "chore", "dependencies"]; +/** + * The most stdout `gh()` reads from one call. Node's default is 1 MiB, and + * past it `spawnSync` throws ENOBUFS. The alert listing is the reply that can + * cross it: each Dependabot alert is about 7 KB of JSON, so some 150 open + * alerts overflow the default and abort the sweep (#5339). + */ +export const GH_MAX_BUFFER = 256 * 1024 * 1024; + /** The two labels the trust rule reads; `v2` is applied by an issue form. */ const TRUST_LABELS = ["chore", "dependencies"]; @@ -223,6 +231,7 @@ export function isDryRun(argv = process.argv.slice(2)) { export function gh(spawn, args, input) { const result = spawn("gh", args, { encoding: "utf8", + maxBuffer: GH_MAX_BUFFER, ...(input === undefined ? {} : { input }), }); if (result.error) throw result.error; diff --git a/scripts/lib/sweep.test.mjs b/scripts/lib/sweep.test.mjs index 949906f78b..0e8d7f60bb 100644 --- a/scripts/lib/sweep.test.mjs +++ b/scripts/lib/sweep.test.mjs @@ -5,7 +5,9 @@ import { test } from "node:test"; import assert from "node:assert/strict"; +import { spawnSync } from "node:child_process"; import { + GH_MAX_BUFFER, SWEEP_LABELS, cell, currentMilestone, @@ -176,6 +178,34 @@ test("gh throws on a spawn error, and passes stdin only when given", () => { assert.equal(calls[1].input, "body"); }); +// The real spawnSync, with "gh" swapped for a node that writes `bytes` of +// stdout, so the options gh() passes are the ones that decide the outcome. +const bigReply = (bytes) => (_cmd, _args, opts) => + spawnSync( + process.execPath, + ["-e", `process.stdout.write("x".repeat(${bytes}))`], + opts, + ); + +test("gh reads a reply over Node's 1 MiB default without ENOBUFS (#5339)", () => { + const bytes = 2 * 1024 * 1024; + assert.ok(GH_MAX_BUFFER > bytes); + const result = gh(bigReply(bytes), ["api", "x"]); + assert.equal(result.status, 0); + assert.equal(result.stdout.length, bytes); +}); + +test("gh still rethrows a genuine spawn error from the real spawnSync", () => { + assert.throws( + () => + gh( + (_cmd, args, opts) => spawnSync("/nonexistent/gh-5339", args, opts), + ["api", "x"], + ), + (err) => err.code === "ENOENT", + ); +}); + test("currentMilestone reads open milestones and fails loudly", () => { const { spawn, calls } = fakeSpawn(() => ({ status: 0, diff --git a/scripts/release-notes.mjs b/scripts/release-notes.mjs index fe5eb39362..ab7c34dfee 100644 --- a/scripts/release-notes.mjs +++ b/scripts/release-notes.mjs @@ -38,11 +38,12 @@ // Two kinds of credit never come from an issue's author, and were once pasted // in by hand, which is how v2.0.0 published without its Thanks (#5250): // -// * An advisory's fix ships from its private fork with no public issue, so -// each `--advisory GHSA-…` is read for its credits (a draft works: notes are -// drafted before advisories publish). Declined credits, bots and -// maintainers are dropped; a credit still pending acceptance is listed and -// flagged, since the reporter may yet decline it. +// * An advisory's issue is converted from its board card by a maintainer, +// never filed by its reporter, so each `--advisory GHSA-…` is read for its +// credits (a draft works: notes are drafted before advisories publish). +// Declined credits, bots and maintainers are dropped; a credit still +// pending acceptance is listed and flagged, since the reporter may yet +// decline it. // * An outside PR is closed and its author credited on an issue (Harvest), so // each `--thank "@login (#ref), <what>"` is a line of the prototype // paragraph. Which issues carry such a credit is a reading, not a lookup, @@ -68,6 +69,13 @@ import { parseArgs } from "node:util"; import { readProject } from "./prepare-python-release.mjs"; export const REPO = "modelcontextprotocol/servers"; + +// The most stdout `run()` reads from one call. Node's default is 1 MiB, and +// past it `spawnSync` throws ENOBUFS before any notes are produced: the tree +// under `src/` at v2.2.0's merge commit came back at 1,087,532 bytes (#5339), +// and it only grows. 256 MiB is far past any reply this script reads while +// still bounding a runaway one. +export const MAX_BUFFER = 256 * 1024 * 1024; const [OWNER, NAME] = REPO.split("/"); // What `collaborators/{user}/permission` reports in its `permission` field. @@ -183,7 +191,11 @@ export function parseNotesArgs(argv) { * @param {string} [input] */ function run(spawn, cmd, args, input) { - const result = spawn(cmd, args, { encoding: "utf8", input }); + const result = spawn(cmd, args, { + encoding: "utf8", + input, + maxBuffer: MAX_BUFFER, + }); if (result.error) { throw result.error; } diff --git a/scripts/release-notes.test.mjs b/scripts/release-notes.test.mjs index e4f8ed3497..c3333bba88 100644 --- a/scripts/release-notes.test.mjs +++ b/scripts/release-notes.test.mjs @@ -10,9 +10,11 @@ // `npm run test:scripts`. import assert from "node:assert/strict"; +import { spawnSync } from "node:child_process"; import { test } from "node:test"; import { ADVISORY_LEAD_IN, + MAX_BUFFER, MIXED_LEAD_IN, REPO, THANKS_LEAD_IN, @@ -20,6 +22,7 @@ import { bucket, closingKeywordIssues, dirsTouched, + filesOf, formatKnownIssues, formatPackages, formatThanks, @@ -282,6 +285,31 @@ test("bucket: one, several and no servers; unpublished and empty fall to Reposit ); }); +test("a gh reply over Node's 1 MiB default gets through without ENOBUFS (#5339)", () => { + // The real spawnSync, with "gh" swapped for a node that prints 2 MiB of + // file names, so the options run() passes are what decide the outcome. + const lines = 128 * 1024; + const spawn = (_cmd, _args, opts) => + spawnSync( + process.execPath, + ["-e", `process.stdout.write("src/everything/f.ts\\n".repeat(${lines}))`], + opts, + ); + assert.ok(MAX_BUFFER > lines * 20); + const files = filesOf(spawn, 1); + assert.equal(files.length, lines); + assert.equal(files[0], "src/everything/f.ts"); +}); + +test("a genuine spawn error is still rethrown", () => { + const spawn = (_cmd, args, opts) => + spawnSync("/nonexistent/gh-5339", args, opts); + assert.throws( + () => filesOf(spawn, 1), + (err) => err.code === "ENOENT", + ); +}); + test("closingKeywordIssues reads every closing keyword, never a cross-repo ref", () => { const body = [ "Closes #1", diff --git a/scripts/verify-action-pins.mjs b/scripts/verify-action-pins.mjs index 344dd00953..da588c3f5d 100644 --- a/scripts/verify-action-pins.mjs +++ b/scripts/verify-action-pins.mjs @@ -18,14 +18,13 @@ // // 1. can mint an OIDC token or push a package: `id-token: write` or // `packages: write` (or `write-all`), in its own `permissions:` or, absent -// one, the workflow's. Here that is `release.yml`'s two publish jobs and -// `claude.yml`; +// one, the workflow's. Here that is `release.yml`'s two publish jobs; // 2. is handed any secret other than `GITHUB_TOKEN`, its own or through the // workflow-level `env:`, in any spelling of the expression (`secrets.X`, // `secrets['X']`, `secrets[matrix.name]`), or as a reusable-workflow // call's `secrets: inherit` (a `secrets:` mapping is read like any other // expression, so one passing only `GITHUB_TOKEN` does not count). Here -// that is `claude.yml` again, for `ANTHROPIC_API_KEY`; or +// that is `sdk-watch.yml`'s `analyze` and `post`, for `ANTHROPIC_API_KEY`; or // 3. uploads an artifact that a credentialed job downloads and has anywhere // in its transitive `needs` (an artifact is scoped to the run, so a job // can download from any job that finished before it), so every producer diff --git a/src/everything/CHANGELOG.md b/src/everything/CHANGELOG.md index 44b065bcf5..8f52f8fb8f 100644 --- a/src/everything/CHANGELOG.md +++ b/src/everything/CHANGELOG.md @@ -1,5 +1,25 @@ # @modelcontextprotocol/server-everything +## 2.2.1 + +### Patch Changes + +- [#5436](https://github.com/modelcontextprotocol/servers/pull/5436) [`ee12fb1`](https://github.com/modelcontextprotocol/servers/commit/ee12fb1446585fe27407d4fe1a0205c3feffefa8) [@cliffhall](https://github.com/cliffhall) - Give the resource `gzip-file-as-resource` creates the light and dark resource icons, like every other resource. A 2025-era client now sees them on the session resource's `resources/list` entry and on the returned link; a 2026-07-28 client sees them on the minted resource link. + +## 2.2.0 + +### Minor Changes + +- [#5389](https://github.com/modelcontextprotocol/servers/pull/5389) [`35bed66`](https://github.com/modelcontextprotocol/servers/commit/35bed663e06c5985aafc04c8483baff8cad341e6) [@cliffhall](https://github.com/cliffhall) - Add light- and dark-themed icons (SEP-973) to the server info and to every tool, prompt, resource and resource template, in both protocol eras. + +- [#5365](https://github.com/modelcontextprotocol/servers/pull/5365) [`3f867a0`](https://github.com/modelcontextprotocol/servers/commit/3f867a0f70ad5427abe45b7cabc620449c70bb30) [@cliffhall](https://github.com/cliffhall) - Add a resource collection example: reading `demo://resource/collection/summer-specials`, or the `demo://resource/collection/by-category/{category}` template, returns several contents in one `resources/read`, each under its item's own `demo://resource/collection/item/{itemId}` URI, which reads back on its own. Both templates complete their variable. + +### Patch Changes + +- [#5396](https://github.com/modelcontextprotocol/servers/pull/5396) [`25b4a5d`](https://github.com/modelcontextprotocol/servers/commit/25b4a5dc32911b2054f8a9351aa4058ebd552cd0) [@cliffhall](https://github.com/cliffhall) - `trigger-long-running-operation` now honors cancellation in both protocol eras: a cancelled call clears its pending timer and sends no further progress, so it no longer delays stdio shutdown ([#1124](https://github.com/modelcontextprotocol/servers/issues/1124), [#4846](https://github.com/modelcontextprotocol/servers/issues/4846)). + +- [#5350](https://github.com/modelcontextprotocol/servers/pull/5350) [`4b54ad6`](https://github.com/modelcontextprotocol/servers/commit/4b54ad6e330f295d9c7b8701bd52a9fa36095c7e) [@cliffhall](https://github.com/cliffhall) - `resource-prompt` now names the values its arguments accept in `prompts/list`: `resourceType` must be "Text" or "Blob", and `resourceId` must be a positive integer. Prompt arguments carry no schema, so clients had no way to learn these before calling the prompt. + ## 2.1.0 ### Minor Changes diff --git a/src/everything/Dockerfile b/src/everything/Dockerfile index 3a29550add..aec26b4f5c 100644 --- a/src/everything/Dockerfile +++ b/src/everything/Dockerfile @@ -1,22 +1,64 @@ +# The `everything` server's container image (#3309). Build it from the +# repository root, which is the build context: +# +# docker build -f src/everything/Dockerfile -t mcp/everything . +# +# `everything` is an npm workspace, so its lockfile and the `overrides` that pin +# its transitive dependencies live in the root package.json and +# package-lock.json, not in src/everything. Both stages install from them with +# `npm ci`, so the image gets exactly the versions this repository tested, +# rather than whatever a lockfile-less `npm install` resolves on the day. +# Dockerfile.dockerignore (next to this file) keeps the context to what the +# build reads. +# +# The image runs the server over stdio by default (`docker run -i`); give the +# transport name as the argument to run another, e.g. `streamableHttp`, which +# listens on PORT (3001) on every interface so a published port reaches it. +# The README's "Running with Docker" section has both commands. This repository +# publishes no image: `mcp/everything` on Docker Hub is built by Docker. + FROM node:24-alpine AS builder -COPY src/everything /app -COPY tsconfig.json /tsconfig.json +WORKDIR /repo -WORKDIR /app +# Manifests first, so the dependency layer is reused until one of them changes. +COPY package.json package-lock.json .npmrc tsconfig.json ./ +COPY src/everything/package.json src/everything/ -RUN --mount=type=cache,target=/root/.npm npm install +# `--ignore-scripts` skips the workspace's `prepare` build, which needs the +# source; the build runs explicitly below. +RUN --mount=type=cache,target=/root/.npm \ + npm ci --workspace src/everything --ignore-scripts -FROM node:24-alpine AS release +COPY src/everything src/everything +RUN npm run build --workspace src/everything -WORKDIR /app -COPY --from=builder /app/dist /app/dist -COPY --from=builder /app/package.json /app/package.json -COPY --from=builder /app/package-lock.json /app/package-lock.json +FROM node:24-alpine AS release ENV NODE_ENV=production -RUN npm ci --ignore-scripts --omit-dev +WORKDIR /repo + +COPY package.json package-lock.json .npmrc ./ +COPY src/everything/package.json src/everything/ + +RUN --mount=type=cache,target=/root/.npm \ + npm ci --workspace src/everything --omit=dev --ignore-scripts + +COPY --from=builder /repo/src/everything/dist src/everything/dist + +WORKDIR /repo/src/everything + +USER node + +# The Streamable HTTP (and deprecated HTTP+SSE) transports' default port. +ENV PORT=3001 +EXPOSE 3001 + +# Both transports close cleanly on SIGINT; node running as PID 1 would ignore +# the default SIGTERM, leaving `docker stop` to wait out its timeout. +STOPSIGNAL SIGINT -CMD ["node", "dist/index.js"] \ No newline at end of file +ENTRYPOINT ["node", "dist/index.js"] +CMD ["stdio"] diff --git a/src/everything/Dockerfile.dockerignore b/src/everything/Dockerfile.dockerignore new file mode 100644 index 0000000000..fc25d62749 --- /dev/null +++ b/src/everything/Dockerfile.dockerignore @@ -0,0 +1,15 @@ +# The build context for src/everything/Dockerfile (#3309). BuildKit reads a +# `<Dockerfile>.dockerignore` beside the Dockerfile in place of a root +# .dockerignore, so this allow-list applies to this image only. The context is +# the repository root, so start from nothing and admit only what the build +# copies: the root manifests and lockfile, and the server's source without +# its build output or test artifacts. +* +!package.json +!package-lock.json +!.npmrc +!tsconfig.json +!src/everything +src/everything/node_modules +src/everything/dist +src/everything/coverage diff --git a/src/everything/README.md b/src/everything/README.md index ec0c81181a..b0ab993b7f 100644 --- a/src/everything/README.md +++ b/src/everything/README.md @@ -13,6 +13,10 @@ This MCP server attempts to exercise all the features of the MCP protocol. It is A complete list of the registered MCP primitives and other protocol features demonstrated can be found in the [Server Features](docs/features.md) document. +The resource collections (`demo://resource/collection/summer-specials`, and the `by-category/{category}` and `item/{itemId}` templates beside it) show a single `resources/read` returning several contents, each under its own URI: reading a collection returns one content per item, and each item's URI can be read on its own. + +The server info and every tool, prompt, resource and resource template carry a pair of [icons](docs/features.md#icons), one for a light theme and one for a dark theme, so a client can show the variant that suits its own background. + The `get-env` tool returns the server's environment variables, but masks the value of any credential-like name (a `KEY`, `TOKEN`, `SECRET`, `PASSWORD`, `PASSWD`, `CREDENTIAL`, `AUTH` or `COOKIE` segment, such as `OPENAI_API_KEY`) as `"<redacted>"`. To see every value unmasked while debugging, start the server with `GET_ENV_INCLUDE_SECRETS=1` (or `true`). Leave it unset otherwise: with it set, every credential in the server's environment reaches the model's context. ## Protocol versions @@ -167,6 +171,32 @@ npx @modelcontextprotocol/server-everything streamableHttp npx @modelcontextprotocol/server-everything sse ``` +## Running with Docker + +This repository publishes no image; build one from the repository root (the build context): + +```shell +docker build -f src/everything/Dockerfile -t mcp/everything . +``` + +The `mcp/everything` image on Docker Hub is built by Docker, not by this repository, and can lag behind it. + +### Run over stdio (the default) + +```shell +docker run -i --rm mcp/everything +``` + +`-i` keeps stdin open, which the stdio transport reads. This is the command an MCP client configuration launches. + +### Run the Streamable HTTP server + +```shell +docker run --rm -p 127.0.0.1:3001:3001 mcp/everything streamableHttp +``` + +The endpoint is `http://localhost:3001/mcp`. The server listens on port 3001 on every interface inside the container; `127.0.0.1:` publishes it on the host's loopback only. To use another port, set `PORT` and publish that one (`-e PORT=8080 -p 127.0.0.1:8080:8080`). The localhost-only `Origin` check described above applies inside the container too. `docker stop` ends the server cleanly. + ## License This MCP server is licensed under the MIT License. This means you are free to use, modify, and distribute the software, subject to the terms and conditions of the MIT License. For more details, please see the LICENSE file in the project repository. diff --git a/src/everything/__tests__/collections.test.ts b/src/everything/__tests__/collections.test.ts new file mode 100644 index 0000000000..d06659d711 --- /dev/null +++ b/src/everything/__tests__/collections.test.ts @@ -0,0 +1,142 @@ +/** + * Characterizes the resource collections (#3130) through the protocol: a + * single `resources/read` of a collection answers with several contents, each + * under its item's own URI, and each of those URIs reads back on its own. The + * same reads run against a 2025-era client (`initialize`) and a 2026-07-28 + * client (`server/discover`), since the collections are served to both. Also + * covers the category and item-id completers and the -32602 for a category or + * item that is not in the catalog. + */ +import { afterEach, describe, expect, it } from "vitest"; +import { connect, connectModern, type Session } from "./harness.js"; +import { + COLLECTION_CATEGORIES, + COLLECTION_ITEMS, + SUMMER_SPECIALS_URI, + collectionItemUri, +} from "../resources/collections.js"; + +let session: Session | undefined; + +afterEach(async () => { + await session?.close(); + session = undefined; +}); + +/** The contents one item is served as. */ +const contentsOf = (item: (typeof COLLECTION_ITEMS)[number]) => ({ + uri: collectionItemUri(item.id), + mimeType: "application/json", + text: JSON.stringify(item, null, 2), +}); + +const CATEGORY_URI = "demo://resource/collection/by-category"; + +describe.each([ + ["2025-era", () => connect()], + ["2026-07-28", () => connectModern()], +])("resource collections for a %s client", (_era, open) => { + it("lists the summer-specials collection and both templates", async () => { + session = await open(); + const { resources } = await session.client.listResources(); + expect(resources).toContainEqual( + expect.objectContaining({ + name: "summer-specials", + uri: SUMMER_SPECIALS_URI, + mimeType: "application/json", + }), + ); + const { resourceTemplates } = await session.client.listResourceTemplates(); + expect(resourceTemplates.map((t) => [t.name, t.uriTemplate])).toEqual( + expect.arrayContaining([ + ["collection-by-category", `${CATEGORY_URI}/{category}`], + ["collection-item", "demo://resource/collection/item/{itemId}"], + ]), + ); + }); + + it("returns every item as a separate content, each under its own URI", async () => { + session = await open(); + const { contents } = await session.client.readResource({ + uri: SUMMER_SPECIALS_URI, + }); + expect(contents).toEqual(COLLECTION_ITEMS.map(contentsOf)); + expect(new Set(contents.map((c) => c.uri)).size).toBe( + COLLECTION_ITEMS.length, + ); + }); + + it.each(COLLECTION_CATEGORIES)( + "returns only the %s items for that category", + async (category) => { + session = await open(); + const { contents } = await session.client.readResource({ + uri: `${CATEGORY_URI}/${category}`, + }); + expect(contents).toEqual( + COLLECTION_ITEMS.filter((i) => i.category === category).map(contentsOf), + ); + }, + ); + + it("reads back each member of a collection under the URI it was returned with", async () => { + session = await open(); + const { contents } = await session.client.readResource({ + uri: SUMMER_SPECIALS_URI, + }); + for (const member of contents) { + const read = await session.client.readResource({ uri: member.uri }); + expect(read.contents).toEqual([member]); + } + }); + + it.each([ + `${CATEGORY_URI}/outerwear`, + "demo://resource/collection/item/0", + "demo://resource/collection/item/6", + "demo://resource/collection/item/01", + "demo://resource/collection/item/abc", + ])("answers %s, which is not in the catalog, with -32602", async (uri) => { + session = await open(); + await expect(session.client.readResource({ uri })).rejects.toMatchObject({ + code: -32602, + data: { uri }, + }); + }); +}); + +describe("completion/complete for the collection templates", () => { + /** Complete one variable of a collection template. */ + async function complete(uri: string, name: string, value: string) { + session = await connect(); + const { completion } = await session.client.complete({ + ref: { type: "ref/resource", uri }, + argument: { name, value }, + }); + return completion.values; + } + + it.each([ + ["", ["accessories", "footwear", "skincare"]], + ["f", ["footwear"]], + ["x", []], + ])("completes category %j", async (value, expected) => { + expect( + await complete(`${CATEGORY_URI}/{category}`, "category", value), + ).toEqual(expected); + }); + + it.each([ + ["", ["1", "2", "3", "4", "5"]], + ["3", ["3"]], + ["9", []], + ])("completes itemId %j", async (value, expected) => { + expect( + await complete( + "demo://resource/collection/item/{itemId}", + "itemId", + value, + ), + ).toEqual(expected); + }); +}); diff --git a/src/everything/__tests__/gzip-file-as-resource.test.ts b/src/everything/__tests__/gzip-file-as-resource.test.ts index ddfe33b30d..7f4c6032b2 100644 --- a/src/everything/__tests__/gzip-file-as-resource.test.ts +++ b/src/everything/__tests__/gzip-file-as-resource.test.ts @@ -22,6 +22,7 @@ import { textOf, type Session, } from "./harness.js"; +import { RESOURCE_ICONS } from "../server/icons.js"; const TEXT = "hello, gzip"; const DATA_URI = `data:text/plain;base64,${Buffer.from(TEXT).toString("base64")}`; @@ -100,6 +101,7 @@ describe("gzip-file-as-resource", () => { uri: "demo://resource/session/hello.txt.gz", name: "hello.txt.gz", mimeType: "application/gzip", + icons: RESOURCE_ICONS, }, ]); const blob = await readBlob(s, "demo://resource/session/hello.txt.gz"); @@ -114,6 +116,7 @@ describe("gzip-file-as-resource", () => { uri: "demo://resource/session/listed.gz", name: "listed.gz", mimeType: "application/gzip", + icons: RESOURCE_ICONS, }); expect( ofMethod(s.notifications, "notifications/resources/list_changed").length, diff --git a/src/everything/__tests__/harness.ts b/src/everything/__tests__/harness.ts index 839ebf0ba2..baab1c4a38 100644 --- a/src/everything/__tests__/harness.ts +++ b/src/everything/__tests__/harness.ts @@ -269,3 +269,18 @@ export async function connectRaw(capabilities: Json): Promise<RawSession> { }, }; } + +/** + * A list's entries without their `icons`, for a snapshot of the rest. The + * icons are long `data:` URIs, one pair per primitive kind, and icons.test.ts + * asserts on them for every list in both eras. + */ +export function withoutIcons<T extends { icons?: unknown }>( + entries: T[], +): Omit<T, "icons">[] { + return entries.map((entry) => { + const rest: Omit<T, "icons"> & { icons?: unknown } = { ...entry }; + delete rest.icons; + return rest; + }); +} diff --git a/src/everything/__tests__/icons.test.ts b/src/everything/__tests__/icons.test.ts new file mode 100644 index 0000000000..f4b700bebf --- /dev/null +++ b/src/everything/__tests__/icons.test.ts @@ -0,0 +1,176 @@ +/** + * Icons with light and dark themes (SEP-973, #3131), as each era's client + * sees them. + * + * The server attaches one light/dark icon pair per primitive kind + * (server/icons.ts). These tests read what crosses the wire, in both + * protocol eras: `serverInfo` from `initialize` (2025-era) or + * `server/discover` (2026-07-28), and every entry of `tools/list`, + * `prompts/list`, `resources/list` and `resources/templates/list`, and the + * resource link a tool returns for a resource it creates (#5434). Each + * must carry a "light" and a "dark" icon, and each icon must be a scalable + * SVG `data:` URI that decodes to an SVG document. + */ +import { afterEach, describe, expect, it } from "vitest"; +import type { Icon } from "@modelcontextprotocol/client"; +import { + ALL_CAPABILITIES, + connect, + connectModern, + connectRaw, + contentOf, + contentOfType, + MODERN_CAPABILITIES, + type RawSession, + type Session, +} from "./harness.js"; +import { + PROMPT_ICONS, + RESOURCE_ICONS, + RESOURCE_TEMPLATE_ICONS, + SERVER_ICONS, + TOOL_ICONS, +} from "../server/icons.js"; + +let session: Session | undefined; +let raw: RawSession | undefined; + +afterEach(async () => { + await session?.close(); + await raw?.close(); + session = undefined; + raw = undefined; +}); + +const SVG_DATA_URI = "data:image/svg+xml;base64,"; + +/** A small file for gzip-file-as-resource; a data: URI needs no network. */ +const DATA_URI = `data:text/plain;base64,${Buffer.from("icons").toString("base64")}`; + +/** Assert that `icons` is a themed pair of decodable SVG icons. */ +function expectThemedPair(icons: Icon[] | undefined): void { + expect(icons).toBeDefined(); + const themes = (icons ?? []).map((icon) => icon.theme).sort(); + expect(themes).toEqual(["dark", "light"]); + for (const icon of icons ?? []) { + expect(icon.mimeType).toBe("image/svg+xml"); + expect(icon.sizes).toEqual(["any"]); + expect(icon.src.startsWith(SVG_DATA_URI)).toBe(true); + const svg = Buffer.from( + icon.src.slice(SVG_DATA_URI.length), + "base64", + ).toString("utf8"); + expect(svg).toMatch(/^<svg xmlns="http:\/\/www\.w3\.org\/2000\/svg"/); + expect(svg).toMatch(/<\/svg>$/); + } + // The two variants are drawn differently, not one image labelled twice. + expect(icons?.[0]?.src).not.toBe(icons?.[1]?.src); +} + +describe("the icon sets", () => { + it("gives every primitive kind its own light/dark pair", () => { + const sets = [ + SERVER_ICONS, + TOOL_ICONS, + PROMPT_ICONS, + RESOURCE_ICONS, + RESOURCE_TEMPLATE_ICONS, + ]; + for (const set of sets) expectThemedPair(set); + const lightSources = new Set( + sets.map((set) => set.find((icon) => icon.theme === "light")?.src), + ); + expect(lightSources.size).toBe(sets.length); + }); +}); + +describe.each([ + { + era: "2025-era", + open: () => connect({ capabilities: ALL_CAPABILITIES }), + }, + { + era: "2026-07-28", + open: () => connectModern({ capabilities: MODERN_CAPABILITIES }), + }, +])("icons on the $era surface", ({ open }) => { + it("puts the server's pair on serverInfo", async () => { + session = await open(); + const info = session.client.getServerVersion(); + expect(info?.icons).toEqual(SERVER_ICONS); + expectThemedPair(info?.icons); + }); + + it("puts the tool pair on every tool", async () => { + session = await open(); + const { tools } = await session.client.listTools(); + expect(tools.length).toBeGreaterThan(0); + for (const tool of tools) expect(tool.icons).toEqual(TOOL_ICONS); + }); + + it("puts the prompt pair on every prompt", async () => { + session = await open(); + const { prompts } = await session.client.listPrompts(); + expect(prompts.length).toBeGreaterThan(0); + for (const prompt of prompts) expect(prompt.icons).toEqual(PROMPT_ICONS); + }); + + it("puts the resource pair on every listed resource", async () => { + session = await open(); + const { resources } = await session.client.listResources(); + expect(resources.length).toBeGreaterThan(0); + for (const resource of resources) { + expect(resource.icons).toEqual(RESOURCE_ICONS); + } + }); + + it("puts the template pair on every resource template", async () => { + session = await open(); + const { resourceTemplates } = await session.client.listResourceTemplates(); + expect(resourceTemplates.length).toBeGreaterThan(0); + for (const template of resourceTemplates) { + expect(template.icons).toEqual(RESOURCE_TEMPLATE_ICONS); + } + }); + + // #5434: a resource a tool creates is a resource too: its link carries the + // resource pair, whichever era's mechanism keeps it readable (a session + // resource for a 2025-era client, a minted URI on 2026-07-28). + it("puts the resource pair on the link gzip-file-as-resource returns", async () => { + session = await open(); + const result = await session.client.callTool({ + name: "gzip-file-as-resource", + arguments: { name: "icons.gz", data: DATA_URI }, + }); + const link = contentOfType(contentOf(result)[0], "resource_link"); + expect(link.icons).toEqual(RESOURCE_ICONS); + }); +}); + +// #5434: on the 2025 era the tool also registers the file as a session +// resource, which then appears in resources/list with the pair. +it("puts the resource pair on the session resource gzip-file-as-resource lists", async () => { + session = await connect({ capabilities: ALL_CAPABILITIES }); + await session.client.callTool({ + name: "gzip-file-as-resource", + arguments: { name: "listed-icons.gz", data: DATA_URI }, + }); + const { resources } = await session.client.listResources(); + const generated = resources.find((r) => r.name === "listed-icons.gz"); + expect(generated?.icons).toEqual(RESOURCE_ICONS); + for (const resource of resources) { + expect(resource.icons).toEqual(RESOURCE_ICONS); + } +}); + +describe("icons on the raw 2026-07-28 wire", () => { + it("carries the server's pair in server/discover's serverInfo", async () => { + raw = await connectRaw({}); + const discover = await raw.result("server/discover"); + expect(discover).toMatchObject({ + _meta: { + "io.modelcontextprotocol/serverInfo": { icons: SERVER_ICONS }, + }, + }); + }); +}); diff --git a/src/everything/__tests__/long-running-cancel.test.ts b/src/everything/__tests__/long-running-cancel.test.ts new file mode 100644 index 0000000000..2ad5d3afd7 --- /dev/null +++ b/src/everything/__tests__/long-running-cancel.test.ts @@ -0,0 +1,143 @@ +/** + * Cancellation of `trigger-long-running-operation` (#1124, #4846). + * + * The tool waits on a timer per step. Before #4846 it ignored the request's + * cancellation: a cancelled call kept stepping and sending progress, and its + * pending timer held a stdio server's process open after the client went + * away. Each case cancels a call from a real SDK client, in both protocol + * eras. + * + * - Over the stdio transport manager on in-memory pipes, the server's raw + * output is read, since the SDK client silently drops progress for a + * request it has cancelled: no progress may cross the wire after the + * cancel, and the session keeps serving. + * - The step timer is what kept the process alive, and nothing on the wire + * shows whether it is still pending, so `node:timers/promises` is wrapped + * (calling through) to show the tool's wait is abandoned on the cancel. + * That wait clears its own timer when its signal aborts. + */ +import { PassThrough } from "node:stream"; +import { afterEach, describe, expect, it, vi } from "vitest"; +import { Client } from "@modelcontextprotocol/client"; +import { StdioServerTransport } from "@modelcontextprotocol/server/stdio"; +import { connect, connectModern, contentOf, textOf } from "./harness.js"; +import type { Session } from "./harness.js"; +import { main, type StdioSession } from "../transports/stdio.js"; + +const timers = vi.hoisted(() => ({ + sleeps: [] as Array<{ ms: number | undefined; settled: Promise<unknown> }>, +})); + +vi.mock("node:timers/promises", async (importOriginal) => { + const actual = await importOriginal<typeof import("node:timers/promises")>(); + return { + ...actual, + setTimeout: <T>( + ms?: number, + value?: T, + options?: Parameters<typeof actual.setTimeout>[2], + ) => { + const promise = actual.setTimeout(ms, value, options); + timers.sleeps.push({ + ms, + settled: promise.then( + () => "elapsed", + (error: unknown) => error, + ), + }); + return promise; + }, + }; +}); + +let session: Session | undefined; +const started: StdioSession[] = []; + +afterEach(async () => { + timers.sleeps.length = 0; + await session?.close(); + session = undefined; + for (const { shutdown } of started.splice(0)) { + process.off("SIGINT", shutdown); + } +}); + +const delay = (ms: number) => + new Promise<void>((resolve) => setTimeout(resolve, ms)); + +describe.each([ + ["2025-era", {}], + ["2026-07-28", { versionNegotiation: { mode: { pin: "2026-07-28" } } }], +] as const)("over stdio, a cancelled call (%s)", (_era, opts) => { + it("sends no progress after the cancel, and the session keeps serving", async () => { + const toServer = new PassThrough(); + const fromServer = new PassThrough(); + started.push(await main(toServer, fromServer)); + // Every line the server writes, read alongside the client's transport. + let wire = ""; + fromServer.on("data", (chunk: Buffer) => { + wire += chunk.toString(); + }); + const client = new Client({ name: "stdio-test", version: "0.0.0" }, opts); + await client.connect(new StdioServerTransport(fromServer, toServer)); + + // Four 50 ms steps: uncancelled, the last progress lands at about 200 ms. + const controller = new AbortController(); + const call = client.callTool( + { + name: "trigger-long-running-operation", + arguments: { duration: 0.2, steps: 4 }, + }, + { + signal: controller.signal, + onprogress: () => controller.abort("cancelled by test"), + }, + ); + await expect(call).rejects.toBeDefined(); + await delay(300); + expect(wire.match(/"notifications\/progress"/g)).toHaveLength(1); + + const echo = await client.callTool({ + name: "echo", + arguments: { message: "after cancel" }, + }); + expect(echo.content).toEqual([ + { type: "text", text: "Echo: after cancel" }, + ]); + await client.close(); + }); +}); + +describe.each([ + ["2025-era", () => connect()], + ["2026-07-28", () => connectModern()], +])("a cancelled call (%s)", (_era, open) => { + it("abandons its pending step wait, so no timer holds the process open", async () => { + session = await open(); + const controller = new AbortController(); + const call = session.client.callTool( + { + name: "trigger-long-running-operation", + arguments: { duration: 20, steps: 2 }, + }, + { signal: controller.signal }, + ); + await vi.waitFor(() => expect(timers.sleeps).toHaveLength(1)); + controller.abort("cancelled by test"); + await expect(call).rejects.toBeDefined(); + + // The 10 s wait settles at once, rejected by the abort, not elapsed. + expect(timers.sleeps[0]?.ms).toBe(10_000); + const outcome = await timers.sleeps[0]?.settled; + expect(outcome).toMatchObject({ name: "AbortError" }); + // The loop stopped: no further step was scheduled. + await delay(20); + expect(timers.sleeps).toHaveLength(1); + + const result = await session.client.callTool({ + name: "echo", + arguments: { message: "still here" }, + }); + expect(textOf(contentOf(result)[0])).toBe("Echo: still here"); + }); +}); diff --git a/src/everything/__tests__/prompts.test.ts b/src/everything/__tests__/prompts.test.ts index 993ef9a421..8606ee8626 100644 --- a/src/everything/__tests__/prompts.test.ts +++ b/src/everything/__tests__/prompts.test.ts @@ -5,7 +5,11 @@ * `{resourceId}` variable. */ import { afterEach, beforeEach, describe, expect, it } from "vitest"; -import { connect, textOf, type Session } from "./harness.js"; +import { connect, connectModern, textOf, type Session } from "./harness.js"; +import { + RESOURCE_TYPE_BLOB, + RESOURCE_TYPE_TEXT, +} from "../resources/templates.js"; let session: Session; @@ -235,3 +239,48 @@ describe("completion/complete for resource templates", () => { expect(completion.values).toEqual(expected); }); }); + +/** + * Prompt arguments carry no JSON Schema over the wire, only a name, a + * description and `required`, so the description is the one place a client + * can learn what an argument accepts (#3985). Checked in both eras, since each + * is served by its own code path. + */ +describe.each([ + ["2025-era", connect], + ["2026-07-28", connectModern], +])("prompts/list argument descriptions (%s)", (_era, open) => { + let listed: Session; + + beforeEach(async () => { + listed = await open(); + }); + + afterEach(async () => { + await listed.close(); + }); + + it("describes every argument of every prompt", async () => { + const { prompts } = await listed.client.listPrompts(); + const undescribed = prompts.flatMap((prompt) => + (prompt.arguments ?? []) + .filter((argument) => !argument.description?.trim()) + .map((argument) => `${prompt.name}.${argument.name}`), + ); + expect(undescribed).toEqual([]); + }); + + it("names resource-prompt's allowed values in its argument descriptions", async () => { + const { prompts } = await listed.client.listPrompts(); + const prompt = prompts.find((p) => p.name === "resource-prompt"); + const descriptionOf = (name: string) => + prompt?.arguments?.find((argument) => argument.name === name) + ?.description; + expect(descriptionOf("resourceType")).toBe( + `Type of resource to fetch. Must be "${RESOURCE_TYPE_TEXT}" or "${RESOURCE_TYPE_BLOB}".`, + ); + expect(descriptionOf("resourceId")).toBe( + "ID of the resource to fetch. Must be a positive integer (1 or greater).", + ); + }); +}); diff --git a/src/everything/__tests__/registrations.test.ts b/src/everything/__tests__/registrations.test.ts index 7e711e5248..463632df63 100644 --- a/src/everything/__tests__/registrations.test.ts +++ b/src/everything/__tests__/registrations.test.ts @@ -328,3 +328,154 @@ describe("Registration Index Files", () => { }); }); }); + +// Every tool carries all four annotation hints as booleans, on every path that +// registers tools (#5156). surface.test.ts snapshots the advertised values; +// this checks presence at registration, so a refactor that drops a hint fails +// here, next to the path that dropped it. The tool set is read from what each +// path registers rather than listed, so a tool added later is covered without +// editing this test. +describe("tool annotation hints at registration", () => { + const HINTS = [ + "readOnlyHint", + "destructiveHint", + "idempotentHint", + "openWorldHint", + ] as const; + + // Every capability registerConditionalTools gates on, so it registers its + // whole set. + const allCapabilities = { + roots: {}, + sampling: {}, + elicitation: { form: {}, url: {} }, + }; + + type RegisterFn = (server: McpServer, context: ServerContext) => void; + + /** Each tool a path registers, with the annotations it was registered with. */ + const registeredBy = ( + register: RegisterFn, + era: "legacy" | "modern", + ): { name: string; annotations: unknown }[] => { + const mockServer = createMockServer(); + vi.mocked(mockServer.server.getClientCapabilities).mockReturnValue( + allCapabilities, + ); + register(mockServer, contextFor(era)); + return vi.mocked(mockServer.registerTool).mock.calls.map((call) => { + const config: { annotations?: unknown } = call[1]; + return { name: call[0], annotations: config.annotations }; + }); + }; + + /** "tool: hint" for each hint a tool lacks or holds as a non-boolean. */ + const missingHints = ( + tools: { name: string; annotations: unknown }[], + ): string[] => + tools.flatMap(({ name, annotations }) => + HINTS.filter( + (hint) => + typeof annotations !== "object" || + annotations === null || + typeof (annotations as Record<string, unknown>)[hint] !== "boolean", + ).map((hint) => `${name}: ${hint}`), + ); + + const paths: { + path: string; + register: RegisterFn; + era: "legacy" | "modern"; + includes: string[]; + }[] = [ + { + path: "registerTools (2025 era)", + register: registerTools, + era: "legacy", + includes: ["echo", "get-sum", "trigger-long-running-operation"], + }, + { + path: "registerTools (2026-07-28)", + register: registerTools, + era: "modern", + includes: ["echo", "get-sum", "trigger-long-running-operation"], + }, + { + path: "registerLegacyTools (2025 era)", + register: registerLegacyTools, + era: "legacy", + includes: ["toggle-simulated-logging", "toggle-subscriber-updates"], + }, + { + path: "registerConditionalTools (2025 era, every capability declared)", + register: registerConditionalTools, + era: "legacy", + includes: [ + "get-roots-list", + "trigger-elicitation-request", + "trigger-multi-step-elicitation", + "trigger-url-elicitation", + "trigger-sampling-request", + ], + }, + { + path: "registerModernTools (2026-07-28)", + register: registerModernTools, + era: "modern", + includes: [ + // capability-gated, registered up front on this era + "get-roots-list", + "trigger-elicitation-request", + "trigger-multi-step-elicitation", + "trigger-url-elicitation", + "trigger-sampling-request", + // task tools + "simulate-research-query", + "trigger-sampling-request-async", + "trigger-elicitation-request-async", + ], + }, + ]; + + it.each(paths)( + "$path registers every tool with all four hints as booleans", + ({ register, era, includes }) => { + const tools = registeredBy(register, era); + + // A path that silently registers nothing would pass the hint check + // vacuously, so pin the tools it is known to register. + expect(tools.length).toBeGreaterThan(0); + expect(tools.map((tool) => tool.name)).toEqual( + expect.arrayContaining(includes), + ); + + expect(missingHints(tools), "tools missing an annotation hint").toEqual( + [], + ); + }, + ); + + it("reports a tool registered without a hint", () => { + // Guards the check itself: a missing, null or non-boolean hint is flagged. + expect( + missingHints([ + { name: "none", annotations: undefined }, + { + name: "partial", + annotations: { + readOnlyHint: true, + destructiveHint: "no", + idempotentHint: false, + }, + }, + ]), + ).toEqual([ + "none: readOnlyHint", + "none: destructiveHint", + "none: idempotentHint", + "none: openWorldHint", + "partial: destructiveHint", + "partial: openWorldHint", + ]); + }); +}); diff --git a/src/everything/__tests__/static-files.test.ts b/src/everything/__tests__/static-files.test.ts index b2c3f8b911..c51d7e047f 100644 --- a/src/everything/__tests__/static-files.test.ts +++ b/src/everything/__tests__/static-files.test.ts @@ -112,7 +112,9 @@ describe("static document resources", () => { const session = await harness.connect(); try { const { resources } = await session.client.listResources(); - expect(resources).toEqual([]); + expect( + resources.filter((r) => r.uri.includes("/static/document/")), + ).toEqual([]); expect(session.client.getInstructions()).toBe( "Server instructions not loaded: Error: ENOENT: no docs", ); diff --git a/src/everything/__tests__/surface.test.ts b/src/everything/__tests__/surface.test.ts index 5febd2c995..4d115487e4 100644 --- a/src/everything/__tests__/surface.test.ts +++ b/src/everything/__tests__/surface.test.ts @@ -13,6 +13,7 @@ import { connect, ofMethod, type Session, + withoutIcons, } from "./harness.js"; let session: Session | undefined; @@ -754,13 +755,12 @@ describe("prompts/list", () => { it("lists the four prompts with their arguments", async () => { session = await connect(); const { prompts } = await session.client.listPrompts(); - expect(prompts).toMatchInlineSnapshot(` + expect(withoutIcons(prompts)).toMatchInlineSnapshot(` [ { "_meta": undefined, "arguments": undefined, "description": "A prompt with no arguments", - "icons": undefined, "name": "simple-prompt", "title": "Simple Prompt", }, @@ -779,7 +779,6 @@ describe("prompts/list", () => { }, ], "description": "A prompt with two arguments, one required and one optional", - "icons": undefined, "name": "args-prompt", "title": "Arguments Prompt", }, @@ -798,7 +797,6 @@ describe("prompts/list", () => { }, ], "description": "First argument choice narrows values for second argument.", - "icons": undefined, "name": "completable-prompt", "title": "Team Management", }, @@ -806,18 +804,17 @@ describe("prompts/list", () => { "_meta": undefined, "arguments": [ { - "description": "Type of resource to fetch", + "description": "Type of resource to fetch. Must be "Text" or "Blob".", "name": "resourceType", "required": true, }, { - "description": "ID of the text resource to fetch", + "description": "ID of the resource to fetch. Must be a positive integer (1 or greater).", "name": "resourceId", "required": true, }, ], "description": "A prompt that includes an embedded resource reference", - "icons": undefined, "name": "resource-prompt", "title": "Resource Prompt", }, @@ -827,10 +824,10 @@ describe("prompts/list", () => { }); describe("resources/list and resources/templates/list", () => { - it("lists one static resource per file in docs/", async () => { + it("lists one static resource per file in docs/, and the summer-specials collection", async () => { session = await connect(); const { resources } = await session.client.listResources(); - expect(resources).toMatchInlineSnapshot(` + expect(withoutIcons(resources)).toMatchInlineSnapshot(` [ { "description": "Static document file exposed from /docs: architecture.md", @@ -874,14 +871,21 @@ describe("resources/list and resources/templates/list", () => { "name": "structure.md", "uri": "demo://resource/static/document/structure.md", }, + { + "description": "A resource collection: one read returns every item in the catalog as a separate content, each under its own demo://resource/collection/item/{itemId} URI.", + "mimeType": "application/json", + "name": "summer-specials", + "title": "Summer Specials Collection", + "uri": "demo://resource/collection/summer-specials", + }, ] `); }); - it("lists the dynamic text and blob templates", async () => { + it("lists the dynamic text and blob templates, and the collection templates", async () => { session = await connect(); const { resourceTemplates } = await session.client.listResourceTemplates(); - expect(resourceTemplates).toMatchInlineSnapshot(` + expect(withoutIcons(resourceTemplates)).toMatchInlineSnapshot(` [ { "description": "Plaintext dynamic resource fabricated from the {resourceId} variable, which must be an integer.", @@ -895,6 +899,20 @@ describe("resources/list and resources/templates/list", () => { "name": "Dynamic Blob Resource", "uriTemplate": "demo://resource/dynamic/blob/{resourceId}", }, + { + "description": "A resource collection filtered by the {category} variable (accessories, footwear, skincare): one read returns each item in that category as a separate content, under its own URI.", + "mimeType": "application/json", + "name": "collection-by-category", + "title": "Collection by Category", + "uriTemplate": "demo://resource/collection/by-category/{category}", + }, + { + "description": "One item of the resource collections, selected by the {itemId} variable (1 to 5). The collections return their items under these URIs.", + "mimeType": "application/json", + "name": "collection-item", + "title": "Collection Item", + "uriTemplate": "demo://resource/collection/item/{itemId}", + }, ] `); }); diff --git a/src/everything/docs/features.md b/src/everything/docs/features.md index 3c243ad6ff..4701836639 100644 --- a/src/everything/docs/features.md +++ b/src/everything/docs/features.md @@ -47,7 +47,7 @@ Each feature below is marked **both**, **2026-07-28** or **2025-era**. Roots, sa - `trigger-elicitation-request-async` (tools/trigger-elicitation-request-async.ts), **2026-07-28**: Runs as a task whose `input_required` status carries a form elicitation; the client answers with `tasks/update` and the task completes with a summary. Needs a client that declares the tasks extension and form-mode elicitation. - `trigger-list-changed` (tools/trigger-list-changed.ts), **both**: Sends `notifications/<list>/list_changed` for `tools`, `prompts` or `resources`: on 2026-07-28 to each `subscriptions/listen` stream that asked for that list, on a 2025-era connection unsolicited. - `trigger-log-messages` (tools/trigger-log-messages.ts), **both**, deprecated: Sends one log message at each level during the call. A 2026-07-28 request gets those at or above the level its `_meta` names (none if it names none); a 2025-era client those at or above its `logging/setLevel` level. -- `trigger-long-running-operation` (tools/trigger-long-running-operation.ts), **both**: Simulates a multi-step operation over a given `duration` and number of `steps`; reports progress via `notifications/progress` when a `progressToken` is provided by the client. +- `trigger-long-running-operation` (tools/trigger-long-running-operation.ts), **both**: Simulates a multi-step operation over a given `duration` and number of `steps`; reports progress via `notifications/progress` when a `progressToken` is provided by the client. Honors cancellation: a cancelled call clears its timer and sends no further progress. - `trigger-multi-step-elicitation` (tools/trigger-multi-step-elicitation.ts), **both**: Asks for a name, then for a favorite color addressed by that name, carrying the first answer to the second round in `requestState`. The state is HMAC-signed and expires after ten minutes; a tampered or expired one is refused with `-32602`. A blank name or a color that is not a choice is asked again. Needs form-mode elicitation. - `trigger-resource-update` (tools/trigger-resource-update.ts), **both**: Sends `notifications/resources/updated` for a URI: on 2026-07-28 to each listen stream whose `resourceSubscriptions` name it, on a 2025-era connection once the client has subscribed to it. - `trigger-sampling-request` (tools/trigger-sampling-request.ts), **both**, deprecated: Asks the client's LLM for a completion with a multi-round-trip request, using the provided `prompt` and `maxTokens`; returns the LLM's response payload. Needs client capability `sampling`. @@ -70,8 +70,9 @@ On a 2025-era connection the tools that need a client capability are registered - Static Documents: `demo://resource/static/document/<filename>` (serves files from `src/everything/docs/` as static file-based resources; a 2026-07-28 read carries a one-hour public cache hint) - Session Scoped, **2025-era**: `demo://resource/session/<name>` (per-session resources registered dynamically; available only for the lifetime of the session) - Minted, **2026-07-28**: `demo://resource/minted/<id>/<name>` (resources a tool created, under a server-minted id; readable by any client that has the URI, for ten minutes; the process keeps at most 100 of them and 64 MiB of content, dropping the oldest first) +- Collections: `demo://resource/collection/summer-specials`, `demo://resource/collection/by-category/{category}` and `demo://resource/collection/item/{itemId}` (resources/collections.ts; one `resources/read` of a collection returns several contents, one per item, each under its item's own URI, which reads back on its own; `category` and `itemId` complete) -A read of a resource that does not exist (an unknown URI, a template id that is not a positive integer, an expired minted resource) is answered with `-32602` and the URI in the error's `data`. +A read of a resource that does not exist (an unknown URI, a template id that is not a positive integer, a collection category or item that is not in the catalog, an expired minted resource) is answered with `-32602` and the URI in the error's `data`. ## Resource Subscriptions and Notifications @@ -79,6 +80,10 @@ A read of a resource that does not exist (an unknown URI, a template id that is - **2025-era**: clients subscribe and unsubscribe to resource URIs with `resources/subscribe` and `resources/unsubscribe`. `toggle-subscriber-updates` starts or stops a per-connection interval that emits `notifications/resources/updated { uri }` for the URIs that connection subscribed to, and `trigger-resource-update` sends one on demand. List changes arrive unsolicited. - Each client's subscriptions belong to the server instance serving its connection, so concurrent clients are independent. +## Icons + +**Both.** The server demonstrates icons with light and dark themes (SEP-973). Its `serverInfo` (in `initialize` and in `server/discover`) and every entry of `tools/list`, `prompts/list`, `resources/list` and `resources/templates/list` carry an `icons` array of two scalable SVG icons, one with `theme: "light"` (drawn for a light background) and one with `theme: "dark"`. Each kind has its own glyph: a server rack, a wrench for tools, a speech bubble for prompts, a document for resources, and a document with braces for templates. The icons are inline `data:` URIs, so a client fetches nothing; they are defined in `server/icons.ts`. + ## Logging Logging is deprecated as of 2026-07-28 (SEP-2577) and remains available on both eras. diff --git a/src/everything/docs/structure.md b/src/everything/docs/structure.md index b3052d9432..440702593f 100644 --- a/src/everything/docs/structure.md +++ b/src/everything/docs/structure.md @@ -28,6 +28,7 @@ src/everything │ └── resource.ts ├── resources │ ├── index.ts + │ ├── collections.ts │ ├── files.ts │ ├── minted.ts │ ├── session.ts @@ -36,6 +37,7 @@ src/everything ├── server │ ├── index.ts │ ├── context.ts + │ ├── icons.ts │ ├── logging.ts │ ├── roots.ts │ └── tasks.ts @@ -132,6 +134,8 @@ src/everything - Blob: `demo://resource/dynamic/blob/{index}` (MIME: `application/octet-stream`, Base64 payload) - The `{index}` path variable must be a finite positive integer; any other is not found (`-32602`). Content is generated on demand with a timestamp. - Exposes helpers `textResource(uri, index)`, `textResourceUri(index)`, `blobResource(uri, index)`, and `blobResourceUri(index)` so other modules can construct and embed dynamic resources directly (e.g., from prompts). +- `collections.ts` + - Registers a resource collection example: `summer-specials` (`demo://resource/collection/summer-specials`), whose read returns every item of a small fixed catalog as a separate content under its own URI; the `collection-by-category` template (`demo://resource/collection/by-category/{category}`), which returns one category's items; and the `collection-item` template (`demo://resource/collection/item/{itemId}`), which reads one item back. Both templates complete their variable, and a category or id not in the catalog is not found (`-32602`). - `files.ts` - Registers static file-based resources for each file in the `docs/` folder. - URIs follow the pattern: `demo://resource/static/document/<filename>`. @@ -152,6 +156,8 @@ src/everything - Exposes `{ server, cleanup }` to the chosen transport. Cleanup stops any running intervals in the server when the connection ends. - `context.ts` - The per-instance context features receive: the era, the change publisher, the 2025-era subscriptions and a cleanup registry. +- `icons.ts` + - The light- and dark-themed icon pairs (SEP-973) attached to the server info and to every tool, prompt, resource and resource template, one pair per kind, as inline SVG `data:` URIs. - `logging.ts` - Implements simulated logging (2025 era). Periodically sends randomized log messages at various levels to the connected client. Started/stopped on demand via a dedicated tool. - `roots.ts` diff --git a/src/everything/package.json b/src/everything/package.json index 77dac768bb..4358c5db97 100644 --- a/src/everything/package.json +++ b/src/everything/package.json @@ -1,6 +1,6 @@ { "name": "@modelcontextprotocol/server-everything", - "version": "2.1.0", + "version": "2.2.1", "description": "MCP server that exercises all the features of the MCP protocol", "license": "SEE LICENSE IN LICENSE", "mcpName": "io.github.modelcontextprotocol/server-everything", diff --git a/src/everything/prompts/args.ts b/src/everything/prompts/args.ts index d3113d3687..b0f0a0837d 100644 --- a/src/everything/prompts/args.ts +++ b/src/everything/prompts/args.ts @@ -1,5 +1,6 @@ import { z } from "zod"; import { McpServer } from "@modelcontextprotocol/server"; +import { PROMPT_ICONS } from "../server/icons.js"; /** * Register a prompt with arguments @@ -20,6 +21,7 @@ export const registerArgumentsPrompt = (server: McpServer) => { "args-prompt", { title: "Arguments Prompt", + icons: PROMPT_ICONS, description: "A prompt with two arguments, one required and one optional", argsSchema: promptArgsSchema, }, diff --git a/src/everything/prompts/completions.ts b/src/everything/prompts/completions.ts index aff3c77f77..f319ac0a72 100644 --- a/src/everything/prompts/completions.ts +++ b/src/everything/prompts/completions.ts @@ -1,5 +1,6 @@ import { z } from "zod"; import { McpServer, completable } from "@modelcontextprotocol/server"; +import { PROMPT_ICONS } from "../server/icons.js"; /** * Register a prompt with completable arguments @@ -45,6 +46,7 @@ export const registerPromptWithCompletions = (server: McpServer) => { "completable-prompt", { title: "Team Management", + icons: PROMPT_ICONS, description: "First argument choice narrows values for second argument.", argsSchema: promptArgsSchema, }, diff --git a/src/everything/prompts/resource.ts b/src/everything/prompts/resource.ts index 5b96aba194..10278ee929 100644 --- a/src/everything/prompts/resource.ts +++ b/src/everything/prompts/resource.ts @@ -13,6 +13,7 @@ import { RESOURCE_TYPE_TEXT, RESOURCE_TYPES, } from "../resources/templates.js"; +import { PROMPT_ICONS } from "../server/icons.js"; /** * Register a prompt with an embedded resource reference @@ -33,6 +34,7 @@ export const registerEmbeddedResourcePrompt = (server: McpServer) => { "resource-prompt", { title: "Resource Prompt", + icons: PROMPT_ICONS, description: "A prompt that includes an embedded resource reference", argsSchema: promptArgsSchema, }, diff --git a/src/everything/prompts/simple.ts b/src/everything/prompts/simple.ts index 13d20d6667..cea0363654 100644 --- a/src/everything/prompts/simple.ts +++ b/src/everything/prompts/simple.ts @@ -1,4 +1,5 @@ import { McpServer } from "@modelcontextprotocol/server"; +import { PROMPT_ICONS } from "../server/icons.js"; /** * Register a simple prompt with no arguments @@ -12,6 +13,7 @@ export const registerSimplePrompt = (server: McpServer) => { "simple-prompt", { title: "Simple Prompt", + icons: PROMPT_ICONS, description: "A prompt with no arguments", }, () => ({ diff --git a/src/everything/resources/collections.ts b/src/everything/resources/collections.ts new file mode 100644 index 0000000000..c4c4398a1f --- /dev/null +++ b/src/everything/resources/collections.ts @@ -0,0 +1,155 @@ +/** + * Resource collections (#3130): one `resources/read` that answers with several + * contents, each under a URI of its own. The spec allows a read to return more + * than one content ("a server could return the contents of several files when + * a directory resource is read"), and this module shows the pattern with a + * small, fixed product catalog: + * + * - `demo://resource/collection/summer-specials` (static, listed): every item. + * - `demo://resource/collection/by-category/{category}` (template, with a + * completer for `category`): the items in one category. + * - `demo://resource/collection/item/{itemId}` (template, with a completer for + * `itemId`): one item. + * + * Every content a collection returns carries its item's URI, not the + * collection's, and that URI is readable on its own through the item + * template, so a client can follow up on any one member. The catalog is a + * constant, so the module keeps no state and serves both protocol eras alike. + * Adapted from the example in olaservo's gist linked on the issue. + */ +import { + CompleteResourceTemplateCallback, + McpServer, + ResourceNotFoundError, + ResourceTemplate, +} from "@modelcontextprotocol/server"; +import { RESOURCE_ICONS, RESOURCE_TEMPLATE_ICONS } from "../server/icons.js"; + +/** One product in the example catalog. */ +export type CollectionItem = { + id: number; + name: string; + price: number; + category: string; +}; + +/** The example catalog the collections are drawn from. */ +export const COLLECTION_ITEMS: readonly CollectionItem[] = [ + { id: 1, name: "Summer Hat", price: 29.99, category: "accessories" }, + { id: 2, name: "Beach Towel", price: 19.99, category: "accessories" }, + { id: 3, name: "Sunglasses", price: 49.99, category: "accessories" }, + { id: 4, name: "Flip Flops", price: 14.99, category: "footwear" }, + { id: 5, name: "Sunscreen SPF50", price: 12.99, category: "skincare" }, +]; + +/** The distinct categories in the catalog, in first-seen order. */ +export const COLLECTION_CATEGORIES: readonly string[] = [ + ...new Set(COLLECTION_ITEMS.map((item) => item.category)), +]; + +const MIME_TYPE = "application/json"; +const collectionUriBase = "demo://resource/collection"; +const itemUriBase = `${collectionUriBase}/item`; +const categoryUriBase = `${collectionUriBase}/by-category`; + +/** The URI of the collection holding every item. */ +export const SUMMER_SPECIALS_URI = `${collectionUriBase}/summer-specials`; + +/** + * The URI under which one item can be read on its own. + * @param id + */ +export const collectionItemUri = (id: number) => `${itemUriBase}/${id}`; + +/** + * The contents of one item: JSON, under the item's own URI. + * @param item + */ +const itemContents = (item: CollectionItem) => ({ + uri: collectionItemUri(item.id), + mimeType: MIME_TYPE, + text: JSON.stringify(item, null, 2), +}); + +/** Completes a category name from its prefix. */ +const categoryCompleter: CompleteResourceTemplateCallback = (value) => + COLLECTION_CATEGORIES.filter((category) => category.startsWith(value)); + +/** Completes an item id from its prefix. */ +const itemIdCompleter: CompleteResourceTemplateCallback = (value) => + COLLECTION_ITEMS.map((item) => String(item.id)).filter((id) => + id.startsWith(value), + ); + +/** + * Register the collection resources with the MCP server. + * - `summer-specials`: a static resource whose read returns every item + * - `collection-by-category`: a template whose read returns one category's items + * - `collection-item`: a template whose read returns one item + * - A category or item id that is not in the catalog is not found (`-32602`) + * + * @param server + */ +export const registerCollectionResources = (server: McpServer) => { + // The whole collection: one read, one content per item, each with its own URI + server.registerResource( + "summer-specials", + SUMMER_SPECIALS_URI, + { + title: "Summer Specials Collection", + icons: RESOURCE_ICONS, + mimeType: MIME_TYPE, + description: + "A resource collection: one read returns every item in the catalog as a separate content, each under its own demo://resource/collection/item/{itemId} URI.", + }, + async () => ({ contents: COLLECTION_ITEMS.map(itemContents) }), + ); + + // A filtered collection: the items in one category + server.registerResource( + "collection-by-category", + new ResourceTemplate(`${categoryUriBase}/{category}`, { + list: undefined, + complete: { category: categoryCompleter }, + }), + { + title: "Collection by Category", + icons: RESOURCE_TEMPLATE_ICONS, + mimeType: MIME_TYPE, + description: `A resource collection filtered by the {category} variable (${COLLECTION_CATEGORIES.join(", ")}): one read returns each item in that category as a separate content, under its own URI.`, + }, + async (uri, variables) => { + const category = String(variables.category); + const items = COLLECTION_ITEMS.filter( + (item) => item.category === category, + ); + if (items.length === 0) throw new ResourceNotFoundError(uri.toString()); + return { contents: items.map(itemContents) }; + }, + ); + + // One member of the collections, readable on its own + server.registerResource( + "collection-item", + new ResourceTemplate(`${itemUriBase}/{itemId}`, { + list: undefined, + complete: { itemId: itemIdCompleter }, + }), + { + title: "Collection Item", + icons: RESOURCE_TEMPLATE_ICONS, + mimeType: MIME_TYPE, + description: + "One item of the resource collections, selected by the {itemId} variable (1 to 5). The collections return their items under these URIs.", + }, + async (uri, variables) => { + // Match the id's exact spelling, so "01" is not found rather than item 1 + const itemId = String(variables.itemId); + const item = COLLECTION_ITEMS.find( + (candidate) => String(candidate.id) === itemId, + ); + if (!item) throw new ResourceNotFoundError(uri.toString()); + return { contents: [itemContents(item)] }; + }, + ); +}; diff --git a/src/everything/resources/files.ts b/src/everything/resources/files.ts index 51329b36b6..c78b9a40b1 100644 --- a/src/everything/resources/files.ts +++ b/src/everything/resources/files.ts @@ -2,6 +2,7 @@ import { McpServer, type CacheHint } from "@modelcontextprotocol/server"; import { dirname, join } from "path"; import { fileURLToPath } from "url"; import { readdirSync, readFileSync, statSync } from "fs"; +import { RESOURCE_ICONS } from "../server/icons.js"; /** How long a client or shared cache may keep a static document. */ export const STATIC_DOCUMENT_CACHE_HINT: CacheHint = { @@ -54,7 +55,12 @@ export const registerFileResources = (server: McpServer) => { uri, // A static document changes only when the server is upgraded, so any // cache may keep it for an hour (2026-07-28 clients only; #4852). - { mimeType, description, cacheHint: STATIC_DOCUMENT_CACHE_HINT }, + { + mimeType, + description, + icons: RESOURCE_ICONS, + cacheHint: STATIC_DOCUMENT_CACHE_HINT, + }, async (uri) => { const text = readFileSafe(fullPath); return { diff --git a/src/everything/resources/index.ts b/src/everything/resources/index.ts index d36d948fd4..f4182f87bb 100644 --- a/src/everything/resources/index.ts +++ b/src/everything/resources/index.ts @@ -3,6 +3,7 @@ import type { ServerContext } from "../server/context.js"; import { registerResourceTemplates } from "./templates.js"; import { registerFileResources } from "./files.js"; import { registerMintedResources } from "./minted.js"; +import { registerCollectionResources } from "./collections.js"; import { fileURLToPath } from "url"; import { dirname, join } from "path"; import { readFileSync } from "fs"; @@ -19,6 +20,7 @@ export const registerResources = ( ) => { registerResourceTemplates(server); registerFileResources(server); + registerCollectionResources(server); if (context.era === "modern") registerMintedResources(server); }; diff --git a/src/everything/resources/minted.ts b/src/everything/resources/minted.ts index 2ed51ed8b1..b9211a363c 100644 --- a/src/everything/resources/minted.ts +++ b/src/everything/resources/minted.ts @@ -20,6 +20,7 @@ import { ResourceTemplate, type ResourceLink, } from "@modelcontextprotocol/server"; +import { RESOURCE_ICONS, RESOURCE_TEMPLATE_ICONS } from "../server/icons.js"; /** How long a minted resource stays readable, in milliseconds. */ export const MINTED_RESOURCE_TTL_MS = 10 * 60_000; @@ -109,6 +110,8 @@ export function mintBlobResource( name, mimeType, description: `Created by a tool call; readable for ${MINTED_RESOURCE_TTL_MS / 60_000} minutes at this URI`, + // #5434: a minted resource is a resource like any other (#3131). + icons: RESOURCE_ICONS, }; } @@ -124,6 +127,7 @@ export function registerMintedResources(server: McpServer): void { "Minted Resource", new ResourceTemplate(mintedUriTemplate, { list: undefined }), { + icons: RESOURCE_TEMPLATE_ICONS, description: "A resource a tool created for this client, such as gzip-file-as-resource's output. Readable for ten minutes at the URI the tool returned.", }, diff --git a/src/everything/resources/templates.ts b/src/everything/resources/templates.ts index 9eb101efad..71dad191af 100644 --- a/src/everything/resources/templates.ts +++ b/src/everything/resources/templates.ts @@ -6,6 +6,7 @@ import { ResourceTemplate, completable, } from "@modelcontextprotocol/server"; +import { RESOURCE_TEMPLATE_ICONS } from "../server/icons.js"; // Resource types export const RESOURCE_TYPE_TEXT = "Text" as const; @@ -26,7 +27,13 @@ export const RESOURCE_TYPES: string[] = [ * The completion logic matches the input against available resource types. */ export const resourceTypeCompleter = completable( - z.string().describe("Type of resource to fetch"), + // Prompt arguments carry no JSON Schema, only a description, so the + // description is the only place a client can learn the allowed values (#3985) + z + .string() + .describe( + `Type of resource to fetch. Must be "${RESOURCE_TYPE_TEXT}" or "${RESOURCE_TYPE_BLOB}".`, + ), (value: string) => { return RESOURCE_TYPES.filter((t) => t.startsWith(value)); }, @@ -49,7 +56,11 @@ export const resourceTypeCompleter = completable( * This helps validate and suggest appropriate resource IDs. */ export const resourceIdForPromptCompleter = completable( - z.string().describe("ID of the text resource to fetch"), + z + .string() + .describe( + "ID of the resource to fetch. Must be a positive integer (1 or greater).", + ), (value: string) => { const resourceId = Number(value); return Number.isInteger(resourceId) && resourceId > 0 ? [value] : []; @@ -176,6 +187,7 @@ export const registerResourceTemplates = (server: McpServer) => { }), { mimeType: "text/plain", + icons: RESOURCE_TEMPLATE_ICONS, description: "Plaintext dynamic resource fabricated from the {resourceId} variable, which must be an integer.", }, @@ -196,6 +208,7 @@ export const registerResourceTemplates = (server: McpServer) => { }), { mimeType: "application/octet-stream", + icons: RESOURCE_TEMPLATE_ICONS, description: "Binary (base64) dynamic resource fabricated from the {resourceId} variable, which must be an integer.", }, diff --git a/src/everything/server/icons.ts b/src/everything/server/icons.ts new file mode 100644 index 0000000000..8c70a65732 --- /dev/null +++ b/src/everything/server/icons.ts @@ -0,0 +1,75 @@ +// The icons the everything server attaches to its primitives (#3131). +// +// SEP-973 lets a server give `icons` to its implementation info and to each +// tool, prompt, resource and resource template, and an icon may name the +// `theme` it is drawn for: "light" for a light background, "dark" for a dark +// one. This module defines one light/dark pair per primitive kind, so a client +// such as the Inspector can show which kind an entry is and pick the variant +// that suits its own theme. +// +// One pair per kind, rather than one per item, keeps every registration to a +// single `icons:` line that names its kind's set. Each icon is a small inline +// SVG carried as a base64 `data:` URI, the form the spec names for embedded +// image data: a client fetches nothing, and the package ships no new files. + +import type { Icon } from "@modelcontextprotocol/server"; + +/** Stroke colour of the variant drawn for a light background. */ +const LIGHT_THEME_STROKE = "#1F2937"; + +/** Stroke colour of the variant drawn for a dark background. */ +const DARK_THEME_STROKE = "#F9FAFB"; + +/** + * Build the light and dark variants of one glyph. + * + * @param glyph - The SVG elements of a 24x24, stroke-only glyph. + * @returns Two scalable SVG icons, one per theme. + */ +const themedIcons = (glyph: string): Icon[] => + ( + [ + ["light", LIGHT_THEME_STROKE], + ["dark", DARK_THEME_STROKE], + ] as const + ).map(([theme, stroke]) => { + const svg = + `<svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24" ` + + `fill="none" stroke="${stroke}" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">` + + `${glyph}</svg>`; + return { + src: `data:image/svg+xml;base64,${Buffer.from(svg).toString("base64")}`, + mimeType: "image/svg+xml", + sizes: ["any"], + theme, + }; + }); + +/** The server itself (its `serverInfo`): a two-unit server rack. */ +export const SERVER_ICONS: Icon[] = themedIcons( + '<rect x="3" y="4" width="18" height="7" rx="2"/>' + + '<rect x="3" y="13" width="18" height="7" rx="2"/>' + + '<path d="M7 7.5h.01M7 16.5h.01"/>', +); + +/** Every tool: a wrench. */ +export const TOOL_ICONS: Icon[] = themedIcons( + '<path d="M14.7 6.3a4 4 0 0 0-5.4 5.4L3 18l3 3 6.3-6.3a4 4 0 0 0 5.4-5.4l-2.5 2.5-2.4-.6-.6-2.4z"/>', +); + +/** Every prompt: a speech bubble. */ +export const PROMPT_ICONS: Icon[] = themedIcons( + '<path d="M4 5h16v11H9l-5 4z"/><path d="M8 9h8M8 12h5"/>', +); + +/** Every concrete resource: a document. */ +export const RESOURCE_ICONS: Icon[] = themedIcons( + '<path d="M6 3h8l4 4v14H6z"/><path d="M14 3v4h4M9 12h6M9 16h6"/>', +); + +/** Every resource template: a document with a placeholder in braces. */ +export const RESOURCE_TEMPLATE_ICONS: Icon[] = themedIcons( + '<path d="M6 3h8l4 4v14H6z"/><path d="M14 3v4h4"/>' + + '<path d="M10.5 11c-1 0-1 .8-1 1.5s0 1.5-1 1.5c1 0 1 .8 1 1.5s0 1.5 1 1.5"/>' + + '<path d="M13.5 11c1 0 1 .8 1 1.5s0 1.5 1 1.5c-1 0-1 .8-1 1.5s0 1.5-1 1.5"/>', +); diff --git a/src/everything/server/index.ts b/src/everything/server/index.ts index ee56f48eb1..fb8a531423 100644 --- a/src/everything/server/index.ts +++ b/src/everything/server/index.ts @@ -46,6 +46,7 @@ import { type ServerContext, } from "./context.js"; import { SERVER_VERSION } from "../version.js"; +import { SERVER_ICONS } from "./icons.js"; /** What the factory is asked to build. */ export type ServerOptions = { @@ -127,6 +128,7 @@ export const createServer = ( { name: "mcp-servers/everything", title: "Everything Reference Server", + icons: SERVER_ICONS, version: SERVER_VERSION, }, { diff --git a/src/everything/tools/echo.ts b/src/everything/tools/echo.ts index 8ae7943ad5..a35528eb43 100644 --- a/src/everything/tools/echo.ts +++ b/src/everything/tools/echo.ts @@ -1,5 +1,6 @@ import { McpServer, CallToolResult } from "@modelcontextprotocol/server"; import { z } from "zod"; +import { TOOL_ICONS } from "../server/icons.js"; // Tool input schema export const EchoSchema = z.object({ @@ -12,6 +13,7 @@ const config = { title: "Echo Tool", description: "Echoes back the input string", inputSchema: EchoSchema, + icons: TOOL_ICONS, annotations: { readOnlyHint: true, destructiveHint: false, diff --git a/src/everything/tools/get-annotated-message.ts b/src/everything/tools/get-annotated-message.ts index b339998286..cd3c3b22de 100644 --- a/src/everything/tools/get-annotated-message.ts +++ b/src/everything/tools/get-annotated-message.ts @@ -1,6 +1,7 @@ import { McpServer, CallToolResult } from "@modelcontextprotocol/server"; import { z } from "zod"; import { MCP_TINY_IMAGE } from "./get-tiny-image.js"; +import { TOOL_ICONS } from "../server/icons.js"; // Tool input schema const GetAnnotatedMessageSchema = z.object({ @@ -20,6 +21,7 @@ const config = { description: "Demonstrates how annotations can be used to provide metadata about content.", inputSchema: GetAnnotatedMessageSchema, + icons: TOOL_ICONS, annotations: { readOnlyHint: true, // This tool only returns data, no side effects destructiveHint: false, // Does not delete or modify anything diff --git a/src/everything/tools/get-env.ts b/src/everything/tools/get-env.ts index 70122dfb0e..1d625c39ad 100644 --- a/src/everything/tools/get-env.ts +++ b/src/everything/tools/get-env.ts @@ -16,6 +16,7 @@ * the environment at call time. */ import { McpServer, CallToolResult } from "@modelcontextprotocol/server"; +import { TOOL_ICONS } from "../server/icons.js"; /** The placeholder that replaces a masked value. */ export const REDACTED = "<redacted>"; @@ -46,6 +47,7 @@ const config = { "Returns all environment variables, helpful for debugging MCP server configuration. " + `Values of credential-like names (a KEY, TOKEN, SECRET, PASSWORD, PASSWD, CREDENTIAL, AUTH or COOKIE name segment) are masked as "${REDACTED}" unless ${INCLUDE_SECRETS_VAR} is set to 1 or true in the server's environment.`, inputSchema: {}, + icons: TOOL_ICONS, annotations: { readOnlyHint: true, destructiveHint: false, diff --git a/src/everything/tools/get-header-params.ts b/src/everything/tools/get-header-params.ts index 89d3278379..03c5f6c34d 100644 --- a/src/everything/tools/get-header-params.ts +++ b/src/everything/tools/get-header-params.ts @@ -18,6 +18,7 @@ import { fromJsonSchema, type JsonSchemaType, } from "@modelcontextprotocol/server"; +import { TOOL_ICONS } from "../server/icons.js"; /** * The keyword that asks a 2026-07-28 client to mirror an argument into the @@ -81,6 +82,7 @@ const config = { "JSON Schema 2020-12 input and output schemas.", inputSchema: fromJsonSchema<HeaderParamsInput>(inputSchema), outputSchema: fromJsonSchema(outputSchema), + icons: TOOL_ICONS, annotations: { readOnlyHint: true, destructiveHint: false, diff --git a/src/everything/tools/get-resource-links.ts b/src/everything/tools/get-resource-links.ts index bf2bfd1d54..67af21e262 100644 --- a/src/everything/tools/get-resource-links.ts +++ b/src/everything/tools/get-resource-links.ts @@ -6,6 +6,7 @@ import { blobResourceUri, blobResource, } from "../resources/templates.js"; +import { TOOL_ICONS } from "../server/icons.js"; // Tool input schema const GetResourceLinksSchema = z.object({ @@ -24,6 +25,7 @@ const config = { description: "Returns up to ten resource links that reference different types of resources", inputSchema: GetResourceLinksSchema, + icons: TOOL_ICONS, annotations: { readOnlyHint: true, destructiveHint: false, diff --git a/src/everything/tools/get-resource-reference.ts b/src/everything/tools/get-resource-reference.ts index f88cb5d887..cc29841d3d 100644 --- a/src/everything/tools/get-resource-reference.ts +++ b/src/everything/tools/get-resource-reference.ts @@ -9,6 +9,7 @@ import { RESOURCE_TYPE_TEXT, RESOURCE_TYPES, } from "../resources/templates.js"; +import { TOOL_ICONS } from "../server/icons.js"; // Tool input schema const GetResourceReferenceSchema = z.object({ @@ -27,6 +28,7 @@ const config = { title: "Get Resource Reference Tool", description: "Returns a resource reference that can be used by MCP clients", inputSchema: GetResourceReferenceSchema, + icons: TOOL_ICONS, annotations: { readOnlyHint: true, destructiveHint: false, diff --git a/src/everything/tools/get-roots-list.ts b/src/everything/tools/get-roots-list.ts index bdb7769981..b3dd66891f 100644 --- a/src/everything/tools/get-roots-list.ts +++ b/src/everything/tools/get-roots-list.ts @@ -14,6 +14,7 @@ import { inputResponse, type InputRequiredResult, } from "@modelcontextprotocol/server"; +import { TOOL_ICONS } from "../server/icons.js"; // Tool configuration const name = "get-roots-list"; @@ -23,6 +24,7 @@ const config = { "Lists the current MCP roots provided by the client. Demonstrates the roots protocol capability even though this server doesn't access files. " + "Deprecated: Roots is deprecated as of protocol version 2026-07-28. The server asks for the roots with a multi-round-trip request on every protocol version.", inputSchema: {}, + icons: TOOL_ICONS, annotations: { readOnlyHint: true, destructiveHint: false, diff --git a/src/everything/tools/get-structured-content.ts b/src/everything/tools/get-structured-content.ts index 11f323e14b..17fb6943b6 100644 --- a/src/everything/tools/get-structured-content.ts +++ b/src/everything/tools/get-structured-content.ts @@ -4,6 +4,7 @@ import { CallToolResult, ContentBlock, } from "@modelcontextprotocol/server"; +import { TOOL_ICONS } from "../server/icons.js"; // Tool input schema const GetStructuredContentInputSchema = { @@ -27,6 +28,7 @@ const config = { "Returns structured content along with an output schema for client data validation", inputSchema: GetStructuredContentInputSchema, outputSchema: GetStructuredContentOutputSchema, + icons: TOOL_ICONS, annotations: { readOnlyHint: true, destructiveHint: false, diff --git a/src/everything/tools/get-sum.ts b/src/everything/tools/get-sum.ts index d8bd04234e..9a025d77e3 100644 --- a/src/everything/tools/get-sum.ts +++ b/src/everything/tools/get-sum.ts @@ -1,5 +1,6 @@ import { z } from "zod"; import { McpServer, CallToolResult } from "@modelcontextprotocol/server"; +import { TOOL_ICONS } from "../server/icons.js"; // Tool input schema const GetSumSchema = z.object({ @@ -13,6 +14,7 @@ const config = { title: "Get Sum Tool", description: "Returns the sum of two numbers", inputSchema: GetSumSchema, + icons: TOOL_ICONS, annotations: { readOnlyHint: true, destructiveHint: false, diff --git a/src/everything/tools/get-tiny-image.ts b/src/everything/tools/get-tiny-image.ts index 972906b1b9..0211d036a3 100644 --- a/src/everything/tools/get-tiny-image.ts +++ b/src/everything/tools/get-tiny-image.ts @@ -1,4 +1,5 @@ import { McpServer, CallToolResult } from "@modelcontextprotocol/server"; +import { TOOL_ICONS } from "../server/icons.js"; // A tiny encoded MCP logo image export const MCP_TINY_IMAGE = @@ -10,6 +11,7 @@ const config = { title: "Get Tiny Image Tool", description: "Returns a tiny MCP logo image.", inputSchema: {}, + icons: TOOL_ICONS, annotations: { readOnlyHint: true, destructiveHint: false, diff --git a/src/everything/tools/gzip-file-as-resource.ts b/src/everything/tools/gzip-file-as-resource.ts index 9d70a41cb7..2ad289f34c 100644 --- a/src/everything/tools/gzip-file-as-resource.ts +++ b/src/everything/tools/gzip-file-as-resource.ts @@ -34,6 +34,7 @@ import { registerSessionResource, } from "../resources/session.js"; import type { ServerContext } from "../server/context.js"; +import { RESOURCE_ICONS, TOOL_ICONS } from "../server/icons.js"; // Maximum input file size - 10 MB default const GZIP_MAX_FETCH_SIZE = Number( @@ -179,6 +180,7 @@ const config = { description: "Compresses a single file using gzip compression. Depending upon the selected output type, returns either the compressed data as a gzipped resource or a resource link, allowing it to be downloaded in a subsequent request: during the current session for a 2025-era client, or for ten minutes at a minted URI on protocol version 2026-07-28.", inputSchema: GZipFileAsResourceSchema, + icons: TOOL_ICONS, annotations: { readOnlyHint: false, destructiveHint: false, @@ -228,7 +230,9 @@ export const registerGZipFileAsResourceTool = ( const uri = getSessionResourceURI(name); const blob = compressedBuffer.toString("base64"); const mimeType = "application/gzip"; - const resource = <Resource>{ uri, name, mimeType }; + // #5434: the session resource is listed like any other, so it carries + // the resource icons (#3131) in resources/list and in its link. + const resource = <Resource>{ uri, name, mimeType, icons: RESOURCE_ICONS }; // Return the resource or a resource link that can be used to access this resource later if (outputType === "resource") { diff --git a/src/everything/tools/simulate-research-query.ts b/src/everything/tools/simulate-research-query.ts index fc558ad2db..fc64cedeea 100644 --- a/src/everything/tools/simulate-research-query.ts +++ b/src/everything/tools/simulate-research-query.ts @@ -28,6 +28,7 @@ import { type TaskInputResponses, type TaskState, } from "../server/tasks.js"; +import { TOOL_ICONS } from "../server/icons.js"; // Tool input schema const SimulateResearchQuerySchema = z.object({ @@ -70,6 +71,7 @@ const config = { "If 'ambiguous' is true and the client supports elicitation, the task pauses with status " + "input_required until the client answers its clarification request with tasks/update.", inputSchema: SimulateResearchQuerySchema, + icons: TOOL_ICONS, annotations: { // Each call creates a task in the process-wide task store. readOnlyHint: false, diff --git a/src/everything/tools/toggle-simulated-logging.ts b/src/everything/tools/toggle-simulated-logging.ts index 486a13fd8e..c63649e3ef 100644 --- a/src/everything/tools/toggle-simulated-logging.ts +++ b/src/everything/tools/toggle-simulated-logging.ts @@ -12,6 +12,7 @@ import { McpServer, CallToolResult } from "@modelcontextprotocol/server"; import { beginSimulatedLogging } from "../server/logging.js"; import type { ServerContext } from "../server/context.js"; +import { TOOL_ICONS } from "../server/icons.js"; // Tool configuration const name = "toggle-simulated-logging"; @@ -22,6 +23,7 @@ const config = { "protocol version 2026-07-28 removes logging/setLevel; use trigger-log-messages with a per-request " + "log level instead.", inputSchema: {}, + icons: TOOL_ICONS, annotations: { readOnlyHint: false, destructiveHint: false, diff --git a/src/everything/tools/toggle-subscriber-updates.ts b/src/everything/tools/toggle-subscriber-updates.ts index 596125baa2..c771c0d719 100644 --- a/src/everything/tools/toggle-subscriber-updates.ts +++ b/src/everything/tools/toggle-subscriber-updates.ts @@ -12,6 +12,7 @@ import { McpServer, CallToolResult } from "@modelcontextprotocol/server"; import { beginSimulatedResourceUpdates } from "../resources/subscriptions.js"; import type { ServerContext } from "../server/context.js"; +import { TOOL_ICONS } from "../server/icons.js"; // Tool configuration const name = "toggle-subscriber-updates"; @@ -22,6 +23,7 @@ const config = { "since protocol version 2026-07-28 removes resources/subscribe; use trigger-resource-update with a " + "subscriptions/listen stream instead.", inputSchema: {}, + icons: TOOL_ICONS, annotations: { readOnlyHint: false, destructiveHint: false, diff --git a/src/everything/tools/trigger-elicitation-request-async.ts b/src/everything/tools/trigger-elicitation-request-async.ts index d3ad5151ba..5219e8cf3b 100644 --- a/src/everything/tools/trigger-elicitation-request-async.ts +++ b/src/everything/tools/trigger-elicitation-request-async.ts @@ -18,6 +18,7 @@ import { } from "@modelcontextprotocol/server"; import { z } from "zod"; import { createTask, type TaskState } from "../server/tasks.js"; +import { TOOL_ICONS } from "../server/icons.js"; // Tool configuration const name = "trigger-elicitation-request-async"; @@ -28,6 +29,7 @@ const config = { "the user's input: the task waits in status input_required, carrying an elicitation request, until the " + "client answers it with tasks/update, then completes with a summary of the answer.", inputSchema: {}, + icons: TOOL_ICONS, annotations: { readOnlyHint: false, destructiveHint: false, diff --git a/src/everything/tools/trigger-elicitation-request.ts b/src/everything/tools/trigger-elicitation-request.ts index 688474534b..2299bdaf33 100644 --- a/src/everything/tools/trigger-elicitation-request.ts +++ b/src/everything/tools/trigger-elicitation-request.ts @@ -17,6 +17,7 @@ import { inputResponse, type InputRequiredResult, } from "@modelcontextprotocol/server"; +import { TOOL_ICONS } from "../server/icons.js"; // Tool configuration const name = "trigger-elicitation-request"; @@ -24,6 +25,7 @@ const config = { title: "Trigger Elicitation Request Tool", description: "Trigger a Request from the Server for User Elicitation", inputSchema: {}, + icons: TOOL_ICONS, annotations: { readOnlyHint: false, destructiveHint: false, diff --git a/src/everything/tools/trigger-list-changed.ts b/src/everything/tools/trigger-list-changed.ts index 69fbccad63..099a5878c4 100644 --- a/src/everything/tools/trigger-list-changed.ts +++ b/src/everything/tools/trigger-list-changed.ts @@ -12,6 +12,7 @@ import { McpServer, CallToolResult } from "@modelcontextprotocol/server"; import { z } from "zod"; import type { ServerContext } from "../server/context.js"; +import { TOOL_ICONS } from "../server/icons.js"; // Tool input schema const TriggerListChangedSchema = z.object({ @@ -29,6 +30,7 @@ const config = { "2026-07-28 it is delivered on each subscriptions/listen stream that asked for that list; a 2025-era " + "client receives it unsolicited.", inputSchema: TriggerListChangedSchema, + icons: TOOL_ICONS, annotations: { readOnlyHint: true, destructiveHint: false, diff --git a/src/everything/tools/trigger-log-messages.ts b/src/everything/tools/trigger-log-messages.ts index b626a22536..7b962fb027 100644 --- a/src/everything/tools/trigger-log-messages.ts +++ b/src/everything/tools/trigger-log-messages.ts @@ -16,6 +16,7 @@ import { CallToolResult, LoggingLevel, } from "@modelcontextprotocol/server"; +import { TOOL_ICONS } from "../server/icons.js"; /** Every logging level, from least to most severe. */ export const LOGGING_LEVELS: LoggingLevel[] = [ @@ -39,6 +40,7 @@ const config = { "are sent, and none if it names no level; a 2025-era client gets those at or above the level it set " + "with logging/setLevel. Deprecated: logging is deprecated as of protocol version 2026-07-28.", inputSchema: {}, + icons: TOOL_ICONS, annotations: { readOnlyHint: true, destructiveHint: false, diff --git a/src/everything/tools/trigger-long-running-operation.ts b/src/everything/tools/trigger-long-running-operation.ts index c91825b504..571cb2826d 100644 --- a/src/everything/tools/trigger-long-running-operation.ts +++ b/src/everything/tools/trigger-long-running-operation.ts @@ -1,5 +1,7 @@ +import { setTimeout as sleep } from "node:timers/promises"; import { z } from "zod"; import { McpServer, CallToolResult } from "@modelcontextprotocol/server"; +import { TOOL_ICONS } from "../server/icons.js"; // Tool input schema const TriggerLongRunningOperationSchema = z.object({ @@ -16,6 +18,7 @@ const config = { title: "Trigger Long Running Operation Tool", description: "Demonstrates a long running operation with progress updates.", inputSchema: TriggerLongRunningOperationSchema, + icons: TOOL_ICONS, annotations: { readOnlyHint: true, destructiveHint: false, @@ -36,6 +39,11 @@ const config = { * At the end of the operation, the tool returns a message indicating the completion of the * operation, including the total duration and steps. * + * The operation watches the request's abort signal, which the SDK fires when the client + * sends `notifications/cancelled` or the connection closes, in either protocol era. On + * cancellation it clears its pending timer and stops without sending further progress, so + * a cancelled operation never holds the process open (#1124, #4846). + * * @param {McpServer} server - The McpServer instance where the tool will be registered. */ export const registerTriggerLongRunningOperationTool = (server: McpServer) => { @@ -47,11 +55,24 @@ export const registerTriggerLongRunningOperationTool = (server: McpServer) => { const { duration, steps } = validatedArgs; const stepDuration = duration / steps; const progressToken = ctx.mcpReq._meta?.progressToken; + const signal = ctx.mcpReq.signal; for (let i = 1; i < steps + 1; i++) { - await new Promise((resolve) => - setTimeout(resolve, stepDuration * 1000), - ); + try { + // Rejects at once, clearing its timer, when the request is cancelled. + await sleep(stepDuration * 1000, undefined, { signal }); + } catch { + // The SDK drops the response to a cancelled request, so this result + // never reaches the client; returning simply ends the handler. + return { + content: [ + { + type: "text", + text: `Long running operation cancelled after ${i - 1} of ${steps} steps.`, + }, + ], + }; + } if (progressToken !== undefined) { await server.server.notification( diff --git a/src/everything/tools/trigger-multi-step-elicitation.ts b/src/everything/tools/trigger-multi-step-elicitation.ts index afb2b73b66..fc132eb2cc 100644 --- a/src/everything/tools/trigger-multi-step-elicitation.ts +++ b/src/everything/tools/trigger-multi-step-elicitation.ts @@ -26,6 +26,7 @@ import { } from "@modelcontextprotocol/server"; import { z } from "zod"; import { roundStateCodec, type RoundState } from "../server/index.js"; +import { TOOL_ICONS } from "../server/icons.js"; // Tool configuration const name = "trigger-multi-step-elicitation"; @@ -37,6 +38,7 @@ const config = { "(input_required) and requestState on protocol version 2026-07-28; 2025-era clients get the same " + "flow as elicitation requests.", inputSchema: {}, + icons: TOOL_ICONS, annotations: { readOnlyHint: false, destructiveHint: false, diff --git a/src/everything/tools/trigger-resource-update.ts b/src/everything/tools/trigger-resource-update.ts index 5f3ca68abb..e4b5ee92b4 100644 --- a/src/everything/tools/trigger-resource-update.ts +++ b/src/everything/tools/trigger-resource-update.ts @@ -12,6 +12,7 @@ import { McpServer, CallToolResult } from "@modelcontextprotocol/server"; import { z } from "zod"; import type { ServerContext } from "../server/context.js"; +import { TOOL_ICONS } from "../server/icons.js"; // Tool input schema const TriggerResourceUpdateSchema = z.object({ @@ -31,6 +32,7 @@ const config = { "delivered on each subscriptions/listen stream whose resourceSubscriptions name the URI; a 2025-era " + "client receives it once it has called resources/subscribe for the URI.", inputSchema: TriggerResourceUpdateSchema, + icons: TOOL_ICONS, annotations: { readOnlyHint: true, destructiveHint: false, diff --git a/src/everything/tools/trigger-sampling-request-async.ts b/src/everything/tools/trigger-sampling-request-async.ts index f0e90b0bbc..a0481c39a7 100644 --- a/src/everything/tools/trigger-sampling-request-async.ts +++ b/src/everything/tools/trigger-sampling-request-async.ts @@ -19,6 +19,7 @@ import { } from "@modelcontextprotocol/server"; import { z } from "zod"; import { createTask, type TaskState } from "../server/tasks.js"; +import { TOOL_ICONS } from "../server/icons.js"; // Tool input schema const TriggerSamplingRequestAsyncSchema = z.object({ @@ -50,6 +51,7 @@ const config = { "answers it with tasks/update, then completes with the result. Deprecated: Sampling is deprecated as of " + "protocol version 2026-07-28.", inputSchema: TriggerSamplingRequestAsyncSchema, + icons: TOOL_ICONS, annotations: { readOnlyHint: false, destructiveHint: false, diff --git a/src/everything/tools/trigger-sampling-request.ts b/src/everything/tools/trigger-sampling-request.ts index 688a1509ef..314943bc55 100644 --- a/src/everything/tools/trigger-sampling-request.ts +++ b/src/everything/tools/trigger-sampling-request.ts @@ -17,6 +17,7 @@ import { type InputRequiredResult, } from "@modelcontextprotocol/server"; import { z } from "zod"; +import { TOOL_ICONS } from "../server/icons.js"; // Tool input schema const TriggerSamplingRequestSchema = z.object({ @@ -35,6 +36,7 @@ const config = { "Trigger a Request from the Server for LLM Sampling. Deprecated: Sampling is deprecated as of protocol " + "version 2026-07-28. The server asks with a multi-round-trip request on every protocol version.", inputSchema: TriggerSamplingRequestSchema, + icons: TOOL_ICONS, annotations: { readOnlyHint: false, destructiveHint: false, diff --git a/src/everything/tools/trigger-url-elicitation.ts b/src/everything/tools/trigger-url-elicitation.ts index ffe08a05b6..b68dd9befc 100644 --- a/src/everything/tools/trigger-url-elicitation.ts +++ b/src/everything/tools/trigger-url-elicitation.ts @@ -24,6 +24,7 @@ import { } from "@modelcontextprotocol/server"; import { z } from "zod"; import type { ServerContext } from "../server/context.js"; +import { TOOL_ICONS } from "../server/icons.js"; // Tool input schema const TriggerUrlElicitationSchema = z.object({ @@ -65,6 +66,7 @@ const config = { "to handle URL elicitation via the error response. Set errorPath=true to use the error path. " + "On protocol version 2026-07-28 the server instead returns the URL elicitation in-band (input_required).", inputSchema: TriggerUrlElicitationSchema, + icons: TOOL_ICONS, annotations: { readOnlyHint: false, destructiveHint: false, diff --git a/src/fetch/README.md b/src/fetch/README.md index 09b4e8b5d8..ec90981f40 100644 --- a/src/fetch/README.md +++ b/src/fetch/README.md @@ -15,6 +15,8 @@ Requires MCP Python SDK 2.x (`mcp>=2.3,<3`). The fetch tool will truncate the response, but by using the `start_index` argument, you can specify where to start the content extraction. This lets models read a webpage in chunks, until they find the information they need. +The markdown of an HTML page opens with the page's title as a `#` heading (its `og:title`, else its `<h1>` when the `<title>` holds it, else its `<title>` or `<h1>`), because the extraction drops a heading that repeats the title. It is not added when the markdown already opens with it, to a raw result, or to a page that failed to simplify. + ### Available Tools - `fetch` - Fetches a URL from the internet and extracts its contents as markdown. @@ -22,6 +24,7 @@ The fetch tool will truncate the response, but by using the `start_index` argume - `max_length` (integer, optional): Maximum number of characters to return (default: 5000) - `start_index` (integer, optional): Start content from this character index (default: 0) - `raw` (boolean, optional): Get raw content without markdown conversion (default: false) + - `timeout_ms` (integer, optional): HTTP request timeout for this call, in milliseconds, from 100 to 600000 (default: the server's timeout, 30000 unless configured; see [Customization - Timeout](#customization---timeout)) - Annotations: `readOnlyHint: true`, `destructiveHint: false`, `idempotentHint: true`, `openWorldHint: true` (it only reads, and it reaches arbitrary hosts on the internet) ### Prompts @@ -34,11 +37,15 @@ The fetch tool will truncate the response, but by using the `start_index` argume ### Errors - A failed fetch (a robots.txt refusal, an unreachable page, an HTTP error status, a refused private address, an unusable proxy setting) and arguments that do not match the tool's schema are returned as a tool result with `isError: true`, whose text starts `Error executing tool fetch:` and says what went wrong, so the model can read it and decide what to do. -- Calling a tool the server does not have is a JSON-RPC protocol error, `-32602` (`Unknown tool: <name>`), as the specification requires. So is getting a prompt it does not have (`Unknown prompt: <name>`), or the `fetch` prompt without its `url` (`URL is required`). +- Calling a tool the server does not have is a JSON-RPC protocol error, `-32602` (`Unknown tool: <name>`), as the specification requires. So is getting a prompt it does not have (`Unknown prompt: <name>`), or the `fetch` prompt without its `url` (`URL is required`) or with a `url` that is not a valid URL (`Invalid URL '<url>': <reason>`). - When the `fetch` prompt's fetch fails, the prompt still returns: its message holds the reason the fetch failed. The server is built on the MCP Python SDK's high-level `MCPServer`: the tool and the prompt are decorated functions, and their type hints and field descriptions are the input schema and the prompt's arguments. +### Content negotiation + +The tool and the prompt ask for markdown first, with the request header `Accept: text/markdown, text/html;q=0.9, */*;q=0.8`. A site that serves markdown through content negotiation sends it directly, and a `text/markdown` response is returned as-is: it is not simplified and carries no raw-content notice. A site that ignores the header sends what it always did, and HTML is simplified to markdown as before. A `raw` fetch keeps the default `Accept: */*`, so it gets whatever the site serves by default (for a web page, its HTML), unsimplified; a markdown response it receives anyway is returned as-is too. + ### Protocol versions The server speaks both eras of the Model Context Protocol over stdio, with nothing to configure: a **2026-07-28** client (discovered with `server/discover`, no session) and a **2025-11-25** client (the `initialize` handshake). The client's first request decides which. Tools and prompts behave the same in both. Methods the 2026-07-28 specification removes, such as `ping`, are answered only on a 2025-11-25 connection. @@ -46,6 +53,7 @@ The server speaks both eras of the Model Context Protocol over stdio, with nothi ## Installation Optionally: Install node.js, this will cause the fetch server to use a different HTML simplifier that is more robust. +Its JavaScript dependencies are then installed with npm on first use, which contacts the npm registry. To keep the server from ever starting Node.js, see [Customization - HTML extraction](#customization---html-extraction). ### Using uv (recommended) @@ -188,6 +196,38 @@ The client receives an error naming the refused address. To fetch from local or internal hosts (a development server, an intranet page), add the argument `--allow-private-ips` to the `args` list in the configuration. +### Customization - Allowed hosts + +By default the server may fetch from any host (subject to the private-address guard above). To restrict it to the +hosts you name, add `--allowed-hosts` followed by one or more entries to the `args` list (the argument can also be +repeated). Each entry is one of: + +- an exact host name, such as `docs.python.org`, which matches that host only, not its subdomains; +- `*.` followed by a domain, such as `*.github.com`, which matches every subdomain of it at any depth + (`api.github.com`, `a.b.github.com`) but not `github.com` itself: list both to allow both; +- an IP address, such as `203.0.113.7` or `2001:db8::7`, which matches a URL that uses that address literally. + +Matching ignores case and a trailing dot, and an internationalized name may be given in either its Unicode or its +`xn--` form. An entry names a host only, so it carries no scheme, port or path. The list is checked before every request +the server sends, by the host name in the URL rather than by the addresses it resolves to: the requested page, its +robots.txt check, and every redirect hop of either, from the tool and from the prompt alike. A request to any other host +is refused before it is sent, and the client receives an error naming the refused host. + +The allowlist and the private-address guard are separate checks, and a request must pass both. Listing an internal +host, such as `localhost` or `10.0.0.5`, does not by itself let the server reach it: that also needs +`--allow-private-ips`, which in turn does not lift the allowlist. + +```json +{ + "mcpServers": { + "fetch": { + "command": "uvx", + "args": ["mcp-server-fetch", "--allowed-hosts", "docs.python.org", "*.wikipedia.org"] + } + } +} +``` + ### Customization - User-agent By default, depending on if the request came from the model (via a tool), or was user initiated (via a prompt), the @@ -202,6 +242,25 @@ ModelContextProtocol/1.0 (User-Specified; +https://github.com/modelcontextprotoc This can be customized by adding the argument `--user-agent=YourUserAgent` to the `args` list in the configuration. +### Customization - Timeout + +The HTTP request for the page waits up to 30 seconds (30000 ms) by default: to connect, and then for each read of the +response. It bounds each of those separately, not the whole download, so a large page that keeps arriving is not cut +off. A request that times out fails with a tool error saying how long it waited. + +The timeout can be set at three levels, the most specific winning: + +1. The `fetch` tool's `timeout_ms` argument, for one call (for example `120000` for a large download, or `5000` to + fail fast). +2. The argument `--timeout-ms=60000` in the `args` list in the configuration, for every call. +3. The `FETCH_TIMEOUT_MS` environment variable (for example `"env": {"FETCH_TIMEOUT_MS": "60000"}` in the + configuration), for every call when `--timeout-ms` is not given. + +The server-wide value also applies to the `fetch` prompt, and is the default the tool advertises for `timeout_ms`. +Every level takes a whole number of milliseconds from 100 to 600000 (10 minutes); the server refuses to start with a +value outside that range, and the tool refuses such a `timeout_ms`. This is the HTTP request timeout only: the +robots.txt check before an autonomous fetch keeps its own 5-second timeout, and HTML extraction is not covered. + ### Customization - Proxy The server can be configured to use a proxy by using the `--proxy-url` argument. @@ -213,11 +272,34 @@ installed with the server). A `socks://` proxy URL, as many desktop proxy settin ### Customization - HTML extraction -HTML is converted to Markdown by [readabilipy](https://github.com/alan-turing-institute/ReadabiliPy). When Node.js is -on the `PATH`, readabilipy runs Mozilla's Readability.js in a Node.js subprocess; otherwise it uses its own -pure-Python extractor, which keeps headings and text but drops link targets. The Node.js subprocess runs with no -timeout, so a Node.js that hangs, or fails in a way readabilipy does not detect, stalls the tool call until the client -gives up. +HTML is converted to Markdown with [readabilipy](https://github.com/alan-turing-institute/ReadabiliPy). When Node.js 14 +or newer is on the `PATH`, the server runs Mozilla's Readability.js (readabilipy's `ExtractArticle.js`) in a Node.js +subprocess; otherwise it uses readabilipy's pure-Python extractor, which keeps headings and text but drops link +targets. The subprocess runs off the server's event loop, so other requests are answered while it works, and under a +30 second timeout: a Node.js that hangs is killed, together with anything it started, and that page is extracted with +the pure-Python extractor instead. If Readability.js fails in any other way, the same fallback is used. Every reason +for a fallback is written to stderr (the server's log), never to stdout, which carries the protocol. + +**Readability.js's dependencies are installed with npm on first use, which reaches the npm registry.** readabilipy +ships only the `package.json` for its JavaScript (`@mozilla/readability`, `jsdom`, `minimist`). The first time the +server extracts HTML with Node.js on the `PATH` and those packages missing, it runs `npm install` in readabilipy's +`javascript` directory, inside that tool call, so: + +- `registry.npmjs.org` belongs in the egress allowlist of a deployment that restricts outbound traffic, unless the + packages are installed ahead of time (below) or Readability.js is turned off. Without it the install fails and the + server uses the pure-Python extractor. +- The JavaScript that runs is resolved from readabilipy's unpinned version ranges when it is installed, so pinning + `mcp-server-fetch` does not pin it. +- The install runs at most once per server process, under a 120 second timeout. Its output goes to stderr. If it + fails, or Node.js is missing or older than 14, the server uses the pure-Python extractor until it is restarted. + +To install the packages ahead of time (in a container image, say), run `npm install` in readabilipy's `javascript` +directory from the Python environment the server runs in; the server then finds them and installs nothing: + +```bash +cd "$(python -c "import os, readabilipy; print(os.path.join(os.path.dirname(readabilipy.__file__), 'javascript'))")" +npm install --no-audit --no-fund +``` Readability.js discards hidden elements. Pages rendered progressively on the server often send a visible loading shell, then the real content in a container hidden (`display: none`, `visibility: hidden`, `hidden`) until scripts @@ -226,7 +308,8 @@ Readability.js extracted, the server unhides it and runs Readability.js again, k text. Other pages are extracted once, exactly as before. To always use the pure-Python extractor, add the argument `--no-readability-js` to the `args` list in the -configuration. The server then never starts Node.js (or npm), whether or not it is installed. +configuration. The server then never starts Node.js (or npm), whether or not it is installed, and never contacts the +npm registry. ## Windows Configuration @@ -270,6 +353,10 @@ If you're experiencing timeout issues on Windows, you may need to set the `PYTHO This addresses character encoding issues that can cause the server to timeout on Windows systems. +Readability.js works on Windows as it does elsewhere: the server finds Node.js and npm (`npm.cmd`) on the `PATH`, and +installs Readability.js's dependencies on first use as described in +[Customization - HTML extraction](#customization---html-extraction). + ## Debugging You can use the MCP inspector to debug the server. For uvx installations: @@ -294,7 +381,7 @@ uv sync --frozen --all-extras --dev uv run --frozen pytest ``` -The tests drive `serve()` through an MCP `ClientSession` in-process (and, in `tests/test_eras.py`, through a `Client` as a 2026-07-28 and as a 2025-11-25 client), with HTTP mocked, so they need no network. The HTML extraction tests need Node.js on `PATH` (readabilipy uses Readability.js when Node is present and a pure-Python fallback when it is not). +The tests drive `serve()` through an MCP `ClientSession` in-process (and, in `tests/test_eras.py`, through a `Client` as a 2026-07-28 and as a 2025-11-25 client), with HTTP mocked, so they need no network. The HTML extraction tests need Node.js and npm on `PATH` (the server uses Readability.js when Node is present and a pure-Python fallback when it is not), and the first run installs Readability.js's dependencies from the npm registry. Coverage, measured per file with branch coverage: diff --git a/src/fetch/pyproject.toml b/src/fetch/pyproject.toml index 3c01f6919d..0f71ebd59c 100644 --- a/src/fetch/pyproject.toml +++ b/src/fetch/pyproject.toml @@ -1,6 +1,6 @@ [project] name = "mcp-server-fetch" -version = "2026.10.28" +version = "2026.11.4" description = "A Model Context Protocol server providing tools to fetch and convert web content for usage by LLMs" readme = "README.md" requires-python = ">=3.10" diff --git a/src/fetch/src/mcp_server_fetch/__init__.py b/src/fetch/src/mcp_server_fetch/__init__.py index 39ac728705..fa3a6eba88 100644 --- a/src/fetch/src/mcp_server_fetch/__init__.py +++ b/src/fetch/src/mcp_server_fetch/__init__.py @@ -1,15 +1,32 @@ +from .allowlist import parse_allowed_host from .server import serve +from .timeout import ( + DEFAULT_TIMEOUT_MS, + MAX_TIMEOUT_MS, + MIN_TIMEOUT_MS, + TIMEOUT_ENV, + parse_timeout_ms, + timeout_ms_from_env, +) def main(): """MCP Fetch Server - HTTP fetching functionality for MCP""" import argparse import asyncio + import os parser = argparse.ArgumentParser( description="give a model the ability to make web requests" ) parser.add_argument("--user-agent", type=str, help="Custom User-Agent string") + parser.add_argument( + "--timeout-ms", + type=parse_timeout_ms, + help=f"HTTP request timeout in milliseconds, {MIN_TIMEOUT_MS} to " + f"{MAX_TIMEOUT_MS} (default: ${TIMEOUT_ENV}, else {DEFAULT_TIMEOUT_MS}); " + "a tool call's timeout_ms argument overrides it", + ) parser.add_argument( "--ignore-robots-txt", action="store_true", @@ -21,6 +38,15 @@ def main(): action="store_true", help="Allow fetching private, loopback and link-local addresses (refused by default)", ) + parser.add_argument( + "--allowed-hosts", + nargs="+", + action="extend", + type=parse_allowed_host, + metavar="HOST", + help="Only fetch from these hosts: an exact host name or IP address, or " + "*.example.com for any subdomain of example.com (all hosts by default)", + ) parser.add_argument( "--no-readability-js", action="store_true", @@ -29,12 +55,21 @@ def main(): ) args = parser.parse_args() + # The server reads its timeout from the environment (see timeout.py), and + # the flag outranks a value already there; a bad value stops it here. + if args.timeout_ms is not None: + os.environ[TIMEOUT_ENV] = str(args.timeout_ms) + try: + timeout_ms_from_env() + except ValueError as e: + parser.error(str(e)) asyncio.run( serve( args.user_agent, args.ignore_robots_txt, args.proxy_url, args.allow_private_ips, + args.allowed_hosts, not args.no_readability_js, ) ) diff --git a/src/fetch/src/mcp_server_fetch/allowlist.py b/src/fetch/src/mcp_server_fetch/allowlist.py new file mode 100644 index 0000000000..102a87fba5 --- /dev/null +++ b/src/fetch/src/mcp_server_fetch/allowlist.py @@ -0,0 +1,101 @@ +# The fetch server's operator-configured host allowlist (#2317). With +# `--allowed-hosts`, every request the server sends (the tool's page, the +# prompt's page, each robots.txt check and every redirect hop of any of them) +# must name a listed host, or it is refused before it is sent. Without the flag +# nothing is restricted, so existing configurations keep working. +# +# The allowlist names hosts, and it is checked against the host the request +# connects to (`raw_host`, the ASCII form), never against resolved addresses: +# a name the operator chose is trusted by name. It is a separate check from +# the private-address guard in server.py, and both apply: listing `localhost` +# or an intranet name does not, by itself, let the server reach a private +# address; that still needs `--allow-private-ips`. Two entry forms only, to +# keep the semantics obvious: an exact host (a name or an IP literal), and +# `*.example.com`, which matches any subdomain at any depth of example.com but +# not example.com itself. Ports and schemes are not part of an entry. + +import argparse +import ipaddress +from collections.abc import Awaitable, Callable, Iterable + +import httpx +from mcp.server.mcpserver.exceptions import ToolError + + +def normalize_allowed_host(entry: str) -> str: + """The canonical form of one ``--allowed-hosts`` entry, or ``ValueError``. + + A name is lowercased, its trailing dot dropped, and IDNA-encoded to the + ASCII form requests are checked in (``bücher.example`` becomes + ``xn--bcher-kva.example``). An IP literal is written in its compressed + form, an IPv6 one with or without brackets. A wildcard is ``*.`` followed + by a name. + """ + value = entry.strip().rstrip(".").lower() + wildcard = value.startswith("*.") + name = value[2:] if wildcard else value + if not wildcard: + try: + return ipaddress.ip_address(name.strip("[]")).compressed + except ValueError: + pass + if not name or any(c in name for c in "*/\\:@[]?#% ") or ".." in name: + raise ValueError( + f"{entry!r} is not a host name, an IP address or *.<domain> " + "(no scheme, port, path or other wildcard)" + ) + try: + ascii_name = httpx.URL(f"http://{name}/").raw_host.decode("ascii") + except httpx.InvalidURL as e: + raise ValueError(f"{entry!r} is not a valid host name: {e}") from None + return f"*.{ascii_name}" if wildcard else ascii_name + + +def parse_allowed_host(entry: str) -> str: + """``normalize_allowed_host`` as an argparse ``type``, with its message.""" + try: + return normalize_allowed_host(entry) + except ValueError as e: + raise argparse.ArgumentTypeError(str(e)) from None + + +def normalize_allowed_hosts(entries: Iterable[str] | None) -> tuple[str, ...]: + """Every entry normalized; an empty tuple means no allowlist.""" + return tuple(normalize_allowed_host(entry) for entry in entries or ()) + + +def host_is_allowed(host: str, allowed_hosts: tuple[str, ...]) -> bool: + """Whether ``host`` (a request's ``raw_host``) matches a normalized entry.""" + host = host.rstrip(".").lower() + try: + host = ipaddress.ip_address(host).compressed + except ValueError: + pass + for entry in allowed_hosts: + if entry.startswith("*."): + if host.endswith(entry[1:]): + return True + elif host == entry: + return True + return False + + +def refuse_unlisted_destination( + allowed_hosts: tuple[str, ...], +) -> Callable[[httpx.Request], Awaitable[None]]: + """An httpx request hook refusing any request to a host not on the list. + + httpx runs request hooks before every request it sends, redirect hops + included, so a listed host cannot redirect the fetch to an unlisted one. + The error does not repeat the list, which is the operator's configuration. + """ + + async def hook(request: httpx.Request) -> None: + host = request.url.raw_host.decode("ascii") + if not host_is_allowed(host, allowed_hosts): + raise ToolError( + f"Refused to fetch {request.url}: {host} is not one of the " + "hosts this server is allowed to fetch from (--allowed-hosts)." + ) + + return hook diff --git a/src/fetch/src/mcp_server_fetch/readability_js.py b/src/fetch/src/mcp_server_fetch/readability_js.py new file mode 100644 index 0000000000..424778c9ad --- /dev/null +++ b/src/fetch/src/mcp_server_fetch/readability_js.py @@ -0,0 +1,274 @@ +# Runs Mozilla's Readability.js (readabilipy's `ExtractArticle.js`) under +# Node.js in a subprocess the server controls, instead of through readabilipy's +# own wrapper (`simple_json_from_html_string(use_readability=True)`). +# +# readabilipy's wrapper had four defects for a stdio MCP server, each fixed +# here rather than worked around at the call site: +# +# - No timeout (#5306). Its `subprocess.run` calls wait forever, so a Node.js +# or npm that hangs stalled the tool call. Every process here runs under a +# timeout, and on timeout its whole process tree is killed (a process group +# on POSIX, `taskkill /T` on Windows), so nothing is left running. +# - Child output on fd 1 (#5310). Its `npm install` inherits the server's +# stdout, which over stdio is the JSON-RPC channel, and it `print`s Node's +# stderr to stdout on a failure. Here every child gets stdin from the null +# device (it must never read the server's JSON-RPC input either) and pipes +# for stdout and stderr; whatever a child prints is forwarded to stderr. +# - npm not found on Windows (#5313). It runs a bare `npm`, which Windows' +# `CreateProcess` does not resolve to `npm.cmd`. Here `node` and `npm` are +# resolved with `shutil.which`, which honours `PATHEXT`. +# - A failed Readability.js run raised out of the tool call. Here any failure +# (no Node.js, an old one, no npm, a failed install, a timeout, a crash, bad +# output) returns None, and the caller falls back to readabilipy's +# pure-Python extractor, with the reason written to stderr. +# +# readabilipy >= 0.3.0 ships only `package.json` for its JavaScript and runs +# `npm install` on first use, which reaches the npm registry during a tool call +# (#4830). That stays the behaviour (the README states it), but the install now +# runs at most once per process, under a lock and a timeout. It installs into +# readabilipy's own `javascript` directory, so dependencies an operator +# installed there ahead of time (the README says how) are found and nothing is +# fetched. +# +# Everything here is synchronous: the caller runs extraction in a worker +# thread (`asyncio.to_thread`), so the server's event loop never waits on it. + +from __future__ import annotations + +import json +import os +import shutil +import signal +import subprocess +import sys +import tempfile +import threading +from pathlib import Path + +import readabilipy + +# How long one Readability.js run may take before it is killed. +EXTRACT_TIMEOUT_SECONDS = 30.0 +# How long the one-time `npm install` of readabilipy's JavaScript may take. +INSTALL_TIMEOUT_SECONDS = 120.0 +# How long `node -v` may take. +VERSION_TIMEOUT_SECONDS = 10.0 +# How long cleanup after a timeout may wait: for `taskkill`, and for the +# killed child to be reaped. +KILL_TIMEOUT_SECONDS = 5.0 +# readabilipy's `package.json` declares `"engines": {"node": ">=14.0.0"}`. +MIN_NODE_MAJOR = 14 +# What `ExtractArticle.js` requires; all three present means installed. +JS_DEPENDENCIES = ("@mozilla/readability", "jsdom", "minimist") + +_probe_lock = threading.Lock() +_probed = False +_node: str | None = None + + +def javascript_dir() -> Path: + """readabilipy's `javascript` directory: `ExtractArticle.js` and `node_modules`.""" + return Path(readabilipy.__file__).parent / "javascript" + + +def _log(message: str) -> None: + # stderr only: stdout is the JSON-RPC channel. + print(f"mcp-server-fetch: {message}", file=sys.stderr, flush=True) + + +def _forward(output: str) -> None: + """Copy a child's captured output to stderr, never to stdout.""" + if output: + sys.stderr.write(output if output.endswith("\n") else output + "\n") + sys.stderr.flush() + + +def _kill_tree(proc: subprocess.Popen[str]) -> None: + """Kill ``proc`` and everything it started (npm runs node, `npm.cmd` runs cmd).""" + if sys.platform == "win32": + try: + subprocess.run( + ["taskkill", "/F", "/T", "/PID", str(proc.pid)], + stdin=subprocess.DEVNULL, + stdout=subprocess.DEVNULL, + stderr=subprocess.DEVNULL, + check=False, + timeout=KILL_TIMEOUT_SECONDS, + ) + except (OSError, subprocess.SubprocessError): + # taskkill could not start or did not finish: the direct child + # is still killed below. + pass + else: + try: + # start_new_session made the child a process-group leader. + os.killpg(proc.pid, signal.SIGKILL) + except ProcessLookupError: + pass + # Whatever the tree kill did, the direct child is certainly gone after this. + proc.kill() + + +def _run( + argv: list[str], timeout: float, cwd: Path | None = None +) -> subprocess.CompletedProcess[str] | None: + """Run ``argv`` detached from the server's stdio, or None if it could not + start or did not finish within ``timeout`` seconds (it is killed then).""" + try: + proc = subprocess.Popen( + argv, + cwd=cwd, + stdin=subprocess.DEVNULL, + stdout=subprocess.PIPE, + stderr=subprocess.PIPE, + text=True, + encoding="utf-8", + errors="replace", + # POSIX only (ignored on Windows): its own process group, so a + # timeout can kill the processes it starts as well. + start_new_session=True, + ) + except OSError as e: + _log(f"could not start {argv[0]}: {e}") + return None + try: + stdout, stderr = proc.communicate(timeout=timeout) + except subprocess.TimeoutExpired: + _kill_tree(proc) + # Reap the child, but do not drain its pipes: a descendant that left + # its process group (or survived taskkill) can hold them open forever. + # The pipes close when `proc` is collected. + try: + proc.wait(timeout=KILL_TIMEOUT_SECONDS) + except subprocess.TimeoutExpired: + pass + _log(f"{Path(argv[0]).name} did not finish within {timeout:g} s and was killed") + return None + return subprocess.CompletedProcess(argv, proc.returncode, stdout, stderr) + + +def _node_major(version: str) -> int | None: + """The major version in `node -v` output (``v24.1.0``), or None.""" + head = version.strip().lstrip("v").split(".", 1)[0] + return int(head) if head.isdigit() else None + + +def _dependencies_installed(jsdir: Path) -> bool: + modules = jsdir / "node_modules" + return all((modules / name / "package.json").is_file() for name in JS_DEPENDENCIES) + + +def _install_dependencies(jsdir: Path) -> bool: + """`npm install` readabilipy's JavaScript dependencies; whether it worked.""" + npm = shutil.which("npm") + if npm is None: + _log( + "npm was not found on PATH, so Readability.js's dependencies cannot be " + f"installed in {jsdir}; using the pure-Python extractor" + ) + return False + _log( + f"installing Readability.js's dependencies with npm in {jsdir} (first use; " + "this contacts the npm registry)" + ) + done = _run( + [npm, "install", "--no-audit", "--no-fund"], + timeout=INSTALL_TIMEOUT_SECONDS, + cwd=jsdir, + ) + if done is None: + return False + _forward(done.stdout) + _forward(done.stderr) + if done.returncode != 0 or not _dependencies_installed(jsdir): + _log( + f"npm install failed (exit {done.returncode}); using the pure-Python extractor" + ) + return False + return True + + +def _find_node() -> str | None: + """A Node.js ready to run `ExtractArticle.js`, installing its dependencies + if they are missing, or None (the reason is logged).""" + node = shutil.which("node") + if node is None: + _log("Node.js was not found on PATH; using the pure-Python extractor") + return None + done = _run([node, "-v"], timeout=VERSION_TIMEOUT_SECONDS) + major = _node_major(done.stdout) if done and done.returncode == 0 else None + if major is None or major < MIN_NODE_MAJOR: + _log( + f"Node.js >= {MIN_NODE_MAJOR} is needed for Readability.js " + f"({node} did not report one); using the pure-Python extractor" + ) + return None + jsdir = javascript_dir() + if not _dependencies_installed(jsdir) and not _install_dependencies(jsdir): + return None + return node + + +def node_path() -> str | None: + """The Node.js to run Readability.js with, or None to use pure Python. + + Probed once per process, under a lock, so concurrent first calls run one + `npm install` between them. A failed probe is not retried until the server + restarts, so a missing or broken Node.js costs one probe, not one per call. + """ + global _probed, _node + with _probe_lock: + if not _probed: + _node = _find_node() + _probed = True + return _node + + +def extract_article(html: str) -> str | None: + """Readability.js's article HTML for ``html`` ("" when it finds none), or + None when Readability.js is unavailable or failed, to fall back on.""" + node = node_path() + if node is None: + return None + jsdir = javascript_dir() + with tempfile.TemporaryDirectory( + prefix="mcp-server-fetch-", ignore_cleanup_errors=True + ) as tmp: + page = Path(tmp, "page.html") + result = Path(tmp, "article.json") + page.write_text(html, encoding="utf-8") + done = _run( + [ + node, + str(jsdir / "ExtractArticle.js"), + "-i", + str(page), + "-o", + str(result), + ], + timeout=EXTRACT_TIMEOUT_SECONDS, + cwd=jsdir, + ) + if done is None: + _log( + "Readability.js was stopped; using the pure-Python extractor for this " + "page. Start the server with --no-readability-js to never use it." + ) + return None + if done.returncode != 0 or not result.is_file(): + _forward(done.stderr) + _log( + f"Readability.js failed (exit {done.returncode}); using the " + "pure-Python extractor for this page" + ) + return None + try: + article = json.loads(result.read_text(encoding="utf-8")) + except ValueError: + _log( + "Readability.js wrote unreadable output; using the pure-Python extractor" + ) + return None + # `Readability.parse()` returns null for a page with no article. + content = article.get("content") if isinstance(article, dict) else None + return content if isinstance(content, str) else "" diff --git a/src/fetch/src/mcp_server_fetch/server.py b/src/fetch/src/mcp_server_fetch/server.py index a36c5a2547..faed1cef40 100644 --- a/src/fetch/src/mcp_server_fetch/server.py +++ b/src/fetch/src/mcp_server_fetch/server.py @@ -14,6 +14,8 @@ # prompt the server does not have, or a prompt call without its `url`, is a # malformed request, answered with a JSON-RPC `-32602` protocol error by # `reject_unknown_names` before `MCPServer` sees it (see there for why). +# A prompt `url` that is not a valid URL gets the same `-32602` from the +# prompt handler (#4914). # `serve()` serves both protocol eras (#4853): a 2025-11-25 client through the # initialize handshake, and a 2026-07-28 client sessionlessly. No handler reads # the session or calls back to the client (roots, sampling, elicitation, @@ -23,23 +25,34 @@ import ipaddress import os import socket -from collections.abc import Awaitable, Callable, Mapping +from collections.abc import Awaitable, Callable, Mapping, Sequence from typing import Annotated, Any, Tuple from importlib.metadata import version from urllib.parse import urlparse, urlunparse import httpx +from pydantic import TypeAdapter, ValidationError import markdownify import readabilipy.simple_json +from .readability_js import extract_article from bs4 import BeautifulSoup, Tag +from .allowlist import normalize_allowed_hosts, refuse_unlisted_destination from mcp.server import MCPServer, ServerRequestContext from mcp.server.context import CallNext, HandlerResult, ServerMiddleware from mcp.server.mcpserver.exceptions import ToolError from mcp.shared.exceptions import MCPError +from .timeout import ( + DEFAULT_TIMEOUT_MS, + MAX_TIMEOUT_MS, + MIN_TIMEOUT_MS, + timeout_ms_from_env, +) from mcp.types import INVALID_PARAMS, ToolAnnotations from protego import Protego from pydantic import AnyUrl, Field +from .title import with_page_title + # The version this server reports in serverInfo, read from the installed # distribution's metadata (pyproject.toml) so it cannot drift from the # published version (#360). Without it the SDK reports its own `mcp` version. @@ -168,11 +181,15 @@ async def _refuse_private_destination(request: httpx.Request) -> None: ) -def _readability_content(html: str, use_readability_js: bool) -> str: - """readabilipy's simplified HTML for ``html``, or "" when it has none.""" +def _readability_content(html: str) -> str: + """readabilipy's pure-Python simplified HTML for ``html``, or "" when it has none. + + Readability.js is not run through readabilipy (see `readability_js`), so + this only ever asks it for its pure-Python extractor, which starts no process. + """ try: ret = readabilipy.simple_json.simple_json_from_html_string( - html, use_readability=use_readability_js + html, use_readability=False ) except IndexError: # readabilipy's pure-Python extractor wraps its simplified tree with @@ -266,8 +283,12 @@ def _recover_hidden_content(html: str, content: str) -> str: if not _hiding_declaration(d) ] tag["style"] = ";".join(style) - recovered = _readability_content(str(soup), use_readability_js=True) - if _text_length(BeautifulSoup(recovered, "html.parser")) > extracted: + recovered = extract_article(str(soup)) + # None: this second run failed or timed out, so keep the first result. + if ( + recovered is not None + and _text_length(BeautifulSoup(recovered, "html.parser")) > extracted + ): return recovered return content @@ -277,19 +298,26 @@ def extract_content_from_html(html: str, use_readability_js: bool = True) -> str Args: html: Raw HTML content to process - use_readability_js: Ask readabilipy for Readability.js, which it runs - under Node when Node is on PATH. False always uses its pure-Python - extractor, so no Node (or npm) process is started (#4199). + use_readability_js: Use Readability.js under Node.js when Node.js is + on PATH (`readability_js`), falling back to readabilipy's + pure-Python extractor when it is unavailable, fails or times out. + False always uses the pure-Python extractor, so no Node.js (or + npm) process is started (#4199). + + Synchronous and possibly slow (a Node.js subprocess, up to its timeout): + `fetch_url` runs it in a worker thread, off the event loop (#5306). Returns: Simplified markdown version of the content """ failed = "<error>Page failed to be simplified from HTML</error>" - content_html = _readability_content(html, use_readability_js) - if use_readability_js: + content_html = extract_article(html) if use_readability_js else None + if content_html is not None: # Only Readability.js drops hidden elements; the pure-Python extractor # keeps them, so its result never needs recovering. content_html = _recover_hidden_content(html, content_html) + else: + content_html = _readability_content(html) if not content_html: return failed content = markdownify.markdownify( @@ -461,6 +489,7 @@ async def check_may_autonomously_fetch_url( user_agent: str, proxy_url: str | None = None, allow_private_ips: bool = False, + allowed_hosts: tuple[str, ...] = (), ) -> None: """ Check if the URL can be fetched by the user agent according to the robots.txt file. @@ -471,8 +500,12 @@ async def check_may_autonomously_fetch_url( robot_txt_url = get_robots_txt_url(url) async with make_client(proxy_url) as client: + hooks: list[Callable[[httpx.Request], Awaitable[None]]] = [] + if allowed_hosts: + hooks.append(refuse_unlisted_destination(allowed_hosts)) if not allow_private_ips: - client.event_hooks = {"request": [_refuse_private_destination]} + hooks.append(_refuse_private_destination) + client.event_hooks = {"request": hooks} try: response = await client.get( robot_txt_url, @@ -507,6 +540,17 @@ async def check_may_autonomously_fetch_url( ) +# The Accept header of a non-raw fetch (#5198). A site that negotiates content +# can then serve native markdown, which is returned as-is, and a site that does +# not still serves HTML (preferred over everything else) to be simplified. +MARKDOWN_ACCEPT = "text/markdown, text/html;q=0.9, */*;q=0.8" + + +def _is_markdown(content_type: str) -> bool: + """Whether a Content-Type header declares markdown (``text/markdown``, RFC 7763).""" + return content_type.split(";", 1)[0].strip().lower() == "text/markdown" + + def _body_looks_like_html(page_raw: str) -> bool: """Whether a body served with no Content-Type is an HTML document. @@ -523,8 +567,10 @@ async def fetch_url( url: str, user_agent: str, force_raw: bool = False, + timeout_ms: int = DEFAULT_TIMEOUT_MS, proxy_url: str | None = None, allow_private_ips: bool = False, + allowed_hosts: tuple[str, ...] = (), check_robots_txt: bool = False, use_readability_js: bool = True, ) -> Tuple[str, str]: @@ -546,23 +592,33 @@ async def _check_redirect_robots_txt(request: httpx.Request) -> None: seen_first_request = True return await check_may_autonomously_fetch_url( - str(request.url), user_agent, proxy_url, allow_private_ips + str(request.url), user_agent, proxy_url, allow_private_ips, allowed_hosts ) async with make_client(proxy_url) as client: hooks: list[Callable[[httpx.Request], Awaitable[None]]] = [] + if allowed_hosts: + hooks.append(refuse_unlisted_destination(allowed_hosts)) if not allow_private_ips: hooks.append(_refuse_private_destination) if check_robots_txt: hooks.append(_check_redirect_robots_txt) if hooks: client.event_hooks = {"request": hooks} + # A raw fetch keeps httpx's default Accept: */*, so it gets the site's + # default representation (a page's HTML), as before this negotiation. + if not force_raw: + client.headers["Accept"] = MARKDOWN_ACCEPT try: response = await client.get( url, follow_redirects=True, headers={"User-Agent": user_agent}, - timeout=30, + timeout=timeout_ms / 1000, + ) + except httpx.TimeoutException as e: + raise ToolError( + f"Failed to fetch {url}: timed out after {timeout_ms} ms: {e!r}" ) except HTTPError as e: raise ToolError(f"Failed to fetch {url}: {e!r}") @@ -574,6 +630,10 @@ async def _check_redirect_robots_txt(request: httpx.Request) -> None: page_raw = decode_page(response) content_type = response.headers.get("content-type", "") + # Markdown is already what this tool returns, so it is not simplified, and + # it carries no raw-content prefix even under force_raw: it is unchanged. + if _is_markdown(content_type): + return page_raw, "" is_page_html = ( "<html" in page_raw[:100] or "text/html" in content_type.lower() @@ -582,7 +642,10 @@ async def _check_redirect_robots_txt(request: httpx.Request) -> None: ) if is_page_html and not force_raw: - return extract_content_from_html(page_raw, use_readability_js), "" + content = await asyncio.to_thread( + extract_content_from_html, page_raw, use_readability_js + ) + return await asyncio.to_thread(with_page_title, page_raw, content), "" # A response with no Content-Type would otherwise leave an empty slot here. declared_type = content_type or "(not declared)" @@ -597,6 +660,10 @@ async def _check_redirect_robots_txt(request: httpx.Request) -> None: Although originally you did not have internet access, and were advised to refuse and tell the user this, this tool now grants you internet access. Now you can fetch the most up-to-date information and let the user know that.""" +# Validates the prompt's `url` the way the tool's AnyUrl parameter does. +_PROMPT_URL = TypeAdapter(AnyUrl) + + def reject_unknown_names(server: MCPServer) -> ServerMiddleware[Any]: """Middleware answering a malformed tool or prompt request with `-32602`. @@ -639,6 +706,7 @@ def create_server( ignore_robots_txt: bool = False, proxy_url: str | None = None, allow_private_ips: bool = False, + allowed_hosts: Sequence[str] | None = None, use_readability_js: bool = True, ) -> MCPServer: """Build the fetch server: the `fetch` tool, the `fetch` prompt, and the guard. @@ -648,8 +716,10 @@ def create_server( if proxy_url: proxy_url = normalize_proxy_url(proxy_url) normalize_proxy_env() + allowed = normalize_allowed_hosts(allowed_hosts) user_agent_autonomous = custom_user_agent or DEFAULT_USER_AGENT_AUTONOMOUS user_agent_manual = custom_user_agent or DEFAULT_USER_AGENT_MANUAL + default_timeout_ms = timeout_ms_from_env() # subscriptions=False: the tool and prompt lists never change and there are # no resources to watch, so `subscriptions/listen` is not served and the @@ -703,20 +773,31 @@ async def fetch( strict=True, ), ] = False, + timeout_ms: Annotated[ + int, + Field( + description="How long to wait, in milliseconds, to connect to the server and for each read of its response before giving up. Raise it for a slow server or a large download; lower it to fail fast.", + ge=MIN_TIMEOUT_MS, + le=MAX_TIMEOUT_MS, + strict=True, + ), + ] = default_timeout_ms, ) -> str: page_url = str(url) if not ignore_robots_txt: await check_may_autonomously_fetch_url( - page_url, user_agent_autonomous, proxy_url, allow_private_ips + page_url, user_agent_autonomous, proxy_url, allow_private_ips, allowed ) content, prefix = await fetch_url( page_url, user_agent_autonomous, force_raw=raw, + timeout_ms=timeout_ms, proxy_url=proxy_url, allow_private_ips=allow_private_ips, + allowed_hosts=allowed, check_robots_txt=not ignore_robots_txt, use_readability_js=use_readability_js, ) @@ -742,14 +823,26 @@ async def fetch( async def fetch_prompt( url: Annotated[str, Field(description="URL to fetch")], ) -> str: + # The URL is checked as the tool's AnyUrl parameter checks it, so a + # malformed one is a bad argument (-32602) rather than an exception + # out of httpx, which the SDK would answer with code 0 (#4914). + try: + url = str(_PROMPT_URL.validate_python(url)) + except ValidationError as e: + raise MCPError( + code=INVALID_PARAMS, + message=f"Invalid URL {url!r}: {e.errors()[0]['msg']}", + ) # The user asked for this page, so robots.txt is not consulted, and a # failure is shown to them as the prompt's text rather than raised. try: content, prefix = await fetch_url( url, user_agent_manual, + timeout_ms=default_timeout_ms, proxy_url=proxy_url, allow_private_ips=allow_private_ips, + allowed_hosts=allowed, use_readability_js=use_readability_js, ) except ToolError as e: @@ -764,6 +857,7 @@ async def serve( ignore_robots_txt: bool = False, proxy_url: str | None = None, allow_private_ips: bool = False, + allowed_hosts: Sequence[str] | None = None, use_readability_js: bool = True, ) -> None: """Run the fetch MCP server over stdio. @@ -772,16 +866,22 @@ async def serve( custom_user_agent: Optional custom User-Agent string to use for requests ignore_robots_txt: Whether to ignore robots.txt restrictions proxy_url: Optional proxy URL to use for requests + allowed_hosts: When given, the only hosts the server may fetch from: + exact hosts, or ``*.example.com`` for any subdomain of it, checked + in addition to the private-address guard. Empty or None allows + every host allow_private_ips: Allow fetching private, loopback and link-local addresses, which are refused by default - use_readability_js: Extract HTML with Readability.js under Node when - Node is on PATH; False always uses the pure-Python extractor + use_readability_js: Extract HTML with Readability.js under Node.js + when Node.js is on PATH (falling back to the pure-Python extractor + when it cannot run); False always uses the pure-Python extractor """ server = create_server( custom_user_agent, ignore_robots_txt, proxy_url, allow_private_ips, + allowed_hosts, use_readability_js, ) # Both eras (#4853): the SDK's dual-era loop lets the client's first diff --git a/src/fetch/src/mcp_server_fetch/timeout.py b/src/fetch/src/mcp_server_fetch/timeout.py new file mode 100644 index 0000000000..b066949a6a --- /dev/null +++ b/src/fetch/src/mcp_server_fetch/timeout.py @@ -0,0 +1,57 @@ +# The fetch server's HTTP request timeout (#4448): its default, its bounds, and +# how a configured value is read. It applies to the page request a `fetch` tool +# call or prompt makes, and is set at three levels, the most specific winning: +# the tool's `timeout_ms` argument, the `--timeout-ms` flag, the +# `FETCH_TIMEOUT_MS` environment variable, and otherwise 30 s (the value the +# server always used). It is httpx's per-phase timeout, so it bounds the +# connect, each read and each write separately rather than the whole transfer. +# The same bounds hold at every level, so a server default always satisfies the +# tool argument's advertised minimum and maximum, and becomes its advertised +# default. robots.txt checks are not covered: they keep httpx's own default. +# +# The server-wide value has one source, the environment variable: +# `create_server` reads it, and `main()` copies `--timeout-ms` into it (the flag +# outranks a value the environment already had). So the setting reaches the +# server the same way whether it was given as a flag or as a variable, and +# `serve()`'s positional signature is unchanged for its existing callers. + +import argparse +import os + +DEFAULT_TIMEOUT_MS = 30_000 +MIN_TIMEOUT_MS = 100 +MAX_TIMEOUT_MS = 600_000 +TIMEOUT_ENV = "FETCH_TIMEOUT_MS" + + +def parse_timeout_ms(value: str) -> int: + """Parse a timeout in milliseconds, as the ``--timeout-ms`` argparse type. + + Raises ``argparse.ArgumentTypeError``, whose message argparse prints as is. + """ + try: + timeout_ms = int(value.strip()) + except ValueError: + raise argparse.ArgumentTypeError( + f"{value!r} is not a whole number of milliseconds" + ) + if not MIN_TIMEOUT_MS <= timeout_ms <= MAX_TIMEOUT_MS: + raise argparse.ArgumentTypeError( + f"{timeout_ms} ms is outside {MIN_TIMEOUT_MS}..{MAX_TIMEOUT_MS} ms" + ) + return timeout_ms + + +def timeout_ms_from_env() -> int: + """The server-wide timeout: ``FETCH_TIMEOUT_MS`` when set, else 30 s. + + Raises ``ValueError`` for an unparseable or out-of-range value, so a bad + setting stops the server at startup rather than failing every request. + """ + value = os.environ.get(TIMEOUT_ENV, "").strip() + if not value: + return DEFAULT_TIMEOUT_MS + try: + return parse_timeout_ms(value) + except argparse.ArgumentTypeError as e: + raise ValueError(f"{TIMEOUT_ENV}: {e}") from None diff --git a/src/fetch/src/mcp_server_fetch/title.py b/src/fetch/src/mcp_server_fetch/title.py new file mode 100644 index 0000000000..074387211f --- /dev/null +++ b/src/fetch/src/mcp_server_fetch/title.py @@ -0,0 +1,150 @@ +# The page title for the fetch server's simplified (Markdown) result (#2472). +# +# Readability.js removes an article heading that repeats the document title, +# and neither extractor returns the `<title>` itself in the content the server +# converts, so a page such as a Sphinx release note came back without the line +# that says what it is. This module reads the title from the raw HTML and puts +# it at the top of the Markdown as a level-one heading, unless the Markdown +# already opens with it. +# +# It works from the raw page rather than from readabilipy's result, so it is +# independent of how the extractor is invoked and is applied at the one call +# site in `fetch_url` that simplifies HTML. The raw result (`raw: true`) and +# non-HTML bodies never pass through it. + +import re + +from bs4 import BeautifulSoup, Tag + +# The marker `extract_content_from_html` returns when a page simplifies to +# nothing. A title on top of it would make the failure read as content. +_FAILURE_PREFIX = "<error>" + +# Titles are compared word by word: case, Markdown heading markers, emphasis, +# escapes, Sphinx's "¶" permalink and other punctuation are ignored, but word +# boundaries are kept, so "Rust" does not match "Trust". +_WORD = re.compile(r"[^\W_]+") +# Characters that Markdown or inline HTML would interpret in the emitted +# heading ("<T>" read as a tag, "__init__" as emphasis, "<" as an entity), +# each escaped with a backslash so the heading shows the title as written. +_MARKDOWN_SPECIAL = re.compile(r"([\\`*_\[\]<>#&|~])") + + +def _normalized(text: str) -> str: + """``text`` as its casefolded words, one space apart, padded with spaces. + + The padding makes ``in`` a whole-word, whole-phrase containment test. + """ + words = _WORD.findall(text.casefold()) + return f" {' '.join(words)} " if words else "" + + +def _starts_tag(char: str) -> bool: + return char.isascii() and (char.isalpha() or char in "/!?") + + +def _visible(line: str) -> str: + """The text a reader sees on one Markdown line. + + A link's destination (`](...)`) and an HTML tag (an autolink's URL + included) are dropped; a code span's contents are kept, tag-like text + and all. As in CommonMark, a `<` starts a tag only before a letter, `/`, + `!` or `?` (so "x < 5 and y > 3" is text), and a backslash-escaped + character is text. One pass, so a long line with an unterminated `](`, `<` or + backtick costs linear time: each closer is searched for forward from + where it is needed, and once a search for one fails it is never repeated. + """ + out: list[str] = [] + missing: set[str] = set() + i, end = 0, len(line) + while i < end: + if line[i] == "\\" and i + 1 < end: + out.append(line[i + 1]) + i += 2 + continue + for opener, closer, keep in ( + ("`", "`", True), + ("<", ">", False), + ("](", ")", False), + ): + if closer in missing or not line.startswith(opener, i): + continue + if opener == "<" and not _starts_tag(line[i + 1 : i + 2]): + continue + j = line.find(closer, i + len(opener)) + if j == -1: + missing.add(closer) + continue + out.append(line[i + len(opener) : j] if keep else " ") + i = j + 1 + break + else: + out.append(line[i]) + i += 1 + return "".join(out) + + +def _collapsed(text: str) -> str: + return " ".join(text.split()) + + +def _text(candidate: str) -> str: + # A trailing "¶" is a permalink that is not marked up as one. + return _collapsed(candidate).rstrip("¶").strip() + + +def page_title(html: str) -> str | None: + """The page's title, or None when the page names none. + + In order: `og:title`; the first `<h1>` when `<title>` holds it, since + `<title>` then only adds the site's name ("What's new — pandas docs"); + `<title>`; the first `<h1>`. + """ + soup = BeautifulSoup(html, "html.parser") + og = soup.find("meta", attrs={"property": "og:title"}) + og_title = _text(str(og.get("content", ""))) if isinstance(og, Tag) else "" + title = _text(soup.title.get_text(" ")) if soup.title else "" + h1 = soup.find("h1") + heading = "" + if isinstance(h1, Tag): + # Sphinx appends a permalink anchor ("¶", or "#" in some themes) to + # each heading; it is not part of the heading's text. + for link in h1.find_all("a", class_="headerlink"): + link.decompose() + heading = _text(h1.get_text(" ")) + heading_in_title = bool(_normalized(heading)) and _normalized( + heading + ) in _normalized(title) + if og_title: + return og_title + if heading_in_title: + return heading + return title or heading or None + + +def _opens_with_title(content: str, title: str) -> bool: + """Whether the Markdown's first non-empty line already carries the title. + + A first line holding the whole title counts (the pure-Python extractor + opens with the title in a paragraph), and so does a first heading that the + title holds: `<title>` often adds the site name ("What's new — pandas + docs") to the page's own heading ("What's new"). + """ + first = next((line for line in content.splitlines() if line.strip()), "") + line, wanted = _normalized(_visible(first)), _normalized(title) + if not line or not wanted: + return False + if wanted in line: + return True + return first.lstrip().startswith("#") and line in wanted + + +def with_page_title(html: str, content: str) -> str: + """``content`` with the page's title as a heading on top, if it lacks one.""" + if content.startswith(_FAILURE_PREFIX): + return content + title = page_title(html) + if title is None or _opens_with_title(content, title): + return content + heading = _MARKDOWN_SPECIAL.sub(r"\\\1", title) + return f"# {heading}\n\n{content.lstrip()}" diff --git a/src/fetch/tests/harness.py b/src/fetch/tests/harness.py index 2d566a62a9..f54d88a21f 100644 --- a/src/fetch/tests/harness.py +++ b/src/fetch/tests/harness.py @@ -27,7 +27,6 @@ import anyio import httpx import pytest -import readabilipy.simple_json from mcp import ClientSession from mcp.shared.memory import create_client_server_memory_streams from mcp.shared.message import SessionMessage @@ -40,6 +39,7 @@ ) import mcp_server_fetch.server as server_module +from mcp_server_fetch import readability_js Responder = Union[httpx.Response, Callable[[httpx.Request], httpx.Response]] @@ -132,31 +132,35 @@ async def fake_stdio_server() -> AsyncIterator[Any]: @pytest.fixture def node_readability() -> None: - """Require the Readability.js path that readabilipy takes when Node is on PATH. + """Require the Readability.js path the server takes when Node is on PATH. - readabilipy silently falls back to a pure-Python extractor without Node, - and the two produce different Markdown, so each HTML test names the mode - it pins. Node is a prerequisite of this repository's gate, so a missing - Node fails the test rather than skipping it. + The server silently falls back to readabilipy's pure-Python extractor + without Node, and the two produce different Markdown, so each HTML test + names the mode it pins. Node is a prerequisite of this repository's gate, + so a missing Node fails the test rather than skipping it. """ - if not readabilipy.simple_json.have_node(): - pytest.fail("Node.js >= 10 must be on PATH for the Readability.js path") + if readability_js.node_path() is None: + pytest.fail( + f"Node.js >= {readability_js.MIN_NODE_MAJOR} (and npm, or readabilipy's " + "JavaScript dependencies) must be on PATH for the Readability.js path" + ) @pytest.fixture def python_readability(monkeypatch: pytest.MonkeyPatch) -> None: - """Force readabilipy's pure-Python fallback, as on a machine without Node.""" - monkeypatch.setattr(readabilipy.simple_json, "have_node", lambda: False) + """Force the pure-Python fallback, as on a machine without Node.""" + monkeypatch.setattr(readability_js, "node_path", lambda: None) @pytest.fixture(autouse=True) def private_tempdir(monkeypatch: pytest.MonkeyPatch, tmp_path: Path) -> None: """Give each test its own ``tempfile.gettempdir()``. - readabilipy's Readability.js path writes its input and output to fixed + readabilipy 0.2's Readability.js path wrote its input and output to fixed names (``full.html``, ``article.json``) in the shared temp directory, so two test runs at once (parallel gate runs in separate worktrees, say) read - each other's pages. + each other's pages. The server's own runner uses a fresh directory per + page, so this now only keeps each test's temporary files to itself. """ monkeypatch.setattr(tempfile, "tempdir", str(tmp_path)) diff --git a/src/fetch/tests/test_allowed_hosts.py b/src/fetch/tests/test_allowed_hosts.py new file mode 100644 index 0000000000..0ba647a221 --- /dev/null +++ b/src/fetch/tests/test_allowed_hosts.py @@ -0,0 +1,305 @@ +"""The operator-configured host allowlist, ``--allowed-hosts`` (#2317). + +Entry parsing and matching are unit-tested directly in ``allowlist``; the +enforcement is driven through the real server over the MCP protocol (see +``harness.connect``), so every request path is covered as a client sees it: +the tool's page, the prompt's page, the robots.txt check, and a redirect hop +of each. ``web.urls()`` proves a refused request was never sent. +""" + +from __future__ import annotations + +import argparse +from typing import Any + +import httpx +import pytest +from mcp.types import CallToolResult + +import mcp_server_fetch +from mcp_server_fetch.allowlist import ( + host_is_allowed, + normalize_allowed_host, + normalize_allowed_hosts, + parse_allowed_host, +) + +from .harness import FakeWeb, connect + +TOOL_ERROR = "Error executing tool fetch: " +PAGE = "https://docs.example.com/guide" +ROBOTS = "https://docs.example.com/robots.txt" +OTHER = "https://other.example.org/secret" +OTHER_ROBOTS = "https://other.example.org/robots.txt" + + +def refused(url: str, host: str) -> str: + return ( + f"Refused to fetch {url}: {host} is not one of the hosts this server " + "is allowed to fetch from (--allowed-hosts)." + ) + + +def plain(text: str, status: int = 200) -> httpx.Response: + return httpx.Response( + status, content=text.encode(), headers={"content-type": "text/plain"} + ) + + +def wire(result: CallToolResult) -> dict[str, Any]: + return result.model_dump( + by_alias=True, mode="json", exclude_none=True, exclude_unset=True + ) + + +def text_of(result: CallToolResult) -> str: + data = wire(result) + assert len(data["content"]) == 1 + return data["content"][0]["text"] + + +# --- entries --------------------------------------------------------------- + + +@pytest.mark.parametrize( + "entry, normalized", + [ + ("Example.COM", "example.com"), + ("example.com.", "example.com"), + (" example.com ", "example.com"), + ("*.Example.com", "*.example.com"), + ("bücher.example", "xn--bcher-kva.example"), + ("*.bücher.example", "*.xn--bcher-kva.example"), + ("10.0.0.1", "10.0.0.1"), + ("::1", "::1"), + ("[0:0::1]", "::1"), + ("localhost", "localhost"), + ], +) +def test_entries_are_normalized(entry: str, normalized: str) -> None: + assert normalize_allowed_host(entry) == normalized + + +@pytest.mark.parametrize( + "entry", + [ + "", + ".", + "*.", + "*", + "*example.com", + "a.*.example.com", + "https://example.com", + "example.com:443", + "example.com/path", + "user@example.com", + "exa mple.com", + "example.com?x", + "a..b", + "*.*.example.com", + ], +) +def test_malformed_entries_are_rejected(entry: str) -> None: + with pytest.raises(ValueError): + normalize_allowed_host(entry) + + +def test_an_entry_httpx_cannot_encode_is_rejected() -> None: + with pytest.raises(ValueError, match="is not a valid host name"): + normalize_allowed_host("☃.example.com") + + +def test_parse_allowed_host_reports_through_argparse() -> None: + assert parse_allowed_host("Example.com") == "example.com" + with pytest.raises(argparse.ArgumentTypeError, match="no scheme, port"): + parse_allowed_host("https://example.com") + + +def test_no_entries_is_no_allowlist() -> None: + assert normalize_allowed_hosts(None) == () + assert normalize_allowed_hosts([]) == () + + +@pytest.mark.parametrize( + "host, allowed", + [ + ("example.com", True), + ("EXAMPLE.com.", True), + ("www.example.com", False), + ("api.example.org", True), + ("a.b.example.org", True), + ("example.org", False), + ("badexample.org", False), + ("example.org.evil.com", False), + ("10.0.0.1", True), + ("10.0.0.2", False), + ("0:0::1", True), + ("evil.com", False), + ], +) +def test_matching(host: str, allowed: bool) -> None: + entries = normalize_allowed_hosts(["example.com", "*.example.org", "10.0.0.1"]) + entries += normalize_allowed_hosts(["::1"]) + assert host_is_allowed(host, entries) is allowed + + +# --- command line ---------------------------------------------------------- + + +def run_main(monkeypatch: pytest.MonkeyPatch, *argv: str) -> Any: + from unittest.mock import AsyncMock + + serve = AsyncMock() + monkeypatch.setattr(mcp_server_fetch, "serve", serve) + monkeypatch.setattr("sys.argv", ["mcp-server-fetch", *argv]) + mcp_server_fetch.main() + return serve + + +def test_flag_is_normalized_and_passed_to_serve( + monkeypatch: pytest.MonkeyPatch, +) -> None: + serve = run_main( + monkeypatch, + "--allowed-hosts", + "Example.com", + "*.docs.example.org", + "--allowed-hosts", + "10.0.0.1", + ) + serve.assert_awaited_once_with( + None, + False, + None, + False, + ["example.com", "*.docs.example.org", "10.0.0.1"], + True, + ) + + +def test_malformed_flag_value_exits_with_usage_error( + monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str] +) -> None: + with pytest.raises(SystemExit) as excinfo: + run_main(monkeypatch, "--allowed-hosts", "example.com:8080") + assert excinfo.value.code == 2 + assert "argument --allowed-hosts: 'example.com:8080' is not a host name" in ( + capsys.readouterr().err + ) + + +async def test_create_server_rejects_a_malformed_entry() -> None: + from mcp_server_fetch.server import create_server + + with pytest.raises(ValueError, match="'https://x' is not a host name"): + create_server(allowed_hosts=["https://x"]) + + +# --- enforcement ----------------------------------------------------------- + + +async def test_listed_host_is_fetched_with_its_robots_txt(web: FakeWeb) -> None: + web.add(ROBOTS, plain("", status=404)) + web.add(PAGE, plain("guide")) + async with connect(allowed_hosts=["*.example.com"]) as (session, _): + result = await session.call_tool("fetch", {"url": PAGE}) + assert wire(result)["isError"] is False + assert web.urls() == [ROBOTS, PAGE] + + +async def test_unlisted_host_is_refused_before_robots_txt(web: FakeWeb) -> None: + async with connect(allowed_hosts=["docs.example.com"]) as (session, _): + result = await session.call_tool("fetch", {"url": OTHER}) + assert text_of(result) == TOOL_ERROR + refused(OTHER_ROBOTS, "other.example.org") + assert web.requests == [] + # Checked by name: nothing was resolved for the unlisted host. + assert web.lookups == [] + + +async def test_unlisted_host_is_refused_with_robots_txt_ignored( + web: FakeWeb, +) -> None: + async with connect(ignore_robots_txt=True, allowed_hosts=["docs.example.com"]) as ( + session, + _, + ): + result = await session.call_tool("fetch", {"url": OTHER}) + assert text_of(result) == TOOL_ERROR + refused(OTHER, "other.example.org") + assert web.requests == [] + + +async def test_redirect_to_unlisted_host_is_refused(web: FakeWeb) -> None: + web.add(ROBOTS, plain("", status=404)) + web.add(PAGE, httpx.Response(302, headers={"location": OTHER})) + web.add(OTHER, plain("secret")) + async with connect(allowed_hosts=["docs.example.com"]) as (session, _): + result = await session.call_tool("fetch", {"url": PAGE}) + assert text_of(result) == TOOL_ERROR + refused(OTHER, "other.example.org") + # Neither the hop's robots.txt nor the hop itself was requested. + assert web.urls() == [ROBOTS, PAGE] + + +async def test_robots_redirect_to_unlisted_host_is_refused(web: FakeWeb) -> None: + web.add(ROBOTS, httpx.Response(301, headers={"location": OTHER_ROBOTS})) + async with connect(allowed_hosts=["docs.example.com"]) as (session, _): + result = await session.call_tool("fetch", {"url": PAGE}) + assert text_of(result) == TOOL_ERROR + refused(OTHER_ROBOTS, "other.example.org") + assert web.urls() == [ROBOTS] + + +async def test_redirect_between_listed_hosts_is_followed(web: FakeWeb) -> None: + web.add(ROBOTS, plain("", status=404)) + web.add(PAGE, httpx.Response(302, headers={"location": OTHER})) + web.add(OTHER_ROBOTS, plain("", status=404)) + web.add(OTHER, plain("moved here")) + async with connect(allowed_hosts=["docs.example.com", "*.example.org"]) as ( + session, + _, + ): + result = await session.call_tool("fetch", {"url": PAGE}) + assert wire(result)["isError"] is False + assert text_of(result).endswith("moved here") + assert web.urls() == [ROBOTS, PAGE, OTHER_ROBOTS, OTHER] + + +async def test_prompt_refuses_unlisted_host_and_its_redirects(web: FakeWeb) -> None: + web.add(PAGE, httpx.Response(302, headers={"location": OTHER})) + async with connect(allowed_hosts=["docs.example.com"]) as (session, _): + direct = await session.get_prompt("fetch", {"url": OTHER}) + redirected = await session.get_prompt("fetch", {"url": PAGE}) + for prompt in (direct, redirected): + text = prompt.messages[0].content + assert getattr(text, "text") == refused(OTHER, "other.example.org") + assert web.urls() == [PAGE] + + +async def test_listed_private_address_still_needs_allow_private_ips( + web: FakeWeb, +) -> None: + # Both checks apply: listing an internal host does not lift the + # private-address guard, and --allow-private-ips does not lift the list. + internal = "http://10.0.0.1/status" + web.add(internal, plain("internal")) + async with connect(ignore_robots_txt=True, allowed_hosts=["10.0.0.1"]) as ( + session, + _, + ): + guarded = await session.call_tool("fetch", {"url": internal}) + assert "which is not a public address" in text_of(guarded) + async with connect( + ignore_robots_txt=True, + allow_private_ips=True, + allowed_hosts=["10.0.0.1"], + ) as (session, _): + allowed = await session.call_tool("fetch", {"url": internal}) + unlisted = await session.call_tool("fetch", {"url": "http://10.0.0.2/"}) + assert text_of(allowed).endswith("internal") + assert text_of(unlisted) == TOOL_ERROR + refused("http://10.0.0.2/", "10.0.0.2") + assert web.urls() == [internal] + + +async def test_no_allowlist_fetches_any_public_host(web: FakeWeb) -> None: + web.add(OTHER, plain("anywhere")) + async with connect(ignore_robots_txt=True) as (session, _): + result = await session.call_tool("fetch", {"url": OTHER}) + assert text_of(result).endswith("anywhere") diff --git a/src/fetch/tests/test_empty_tree.py b/src/fetch/tests/test_empty_tree.py index 0c13586f58..6835e49d77 100644 --- a/src/fetch/tests/test_empty_tree.py +++ b/src/fetch/tests/test_empty_tree.py @@ -73,7 +73,9 @@ def html(body: str) -> httpx.Response: @pytest.mark.parametrize("page", [SPA_SHELL, "<html><body></body></html>", ""]) def test_emulation_reproduces_the_reported_crash(page: str) -> None: with pytest.raises(IndexError, match="list index out of range"): - readabilipy.simple_json.simple_json_from_html_string(page, use_readability=True) + readabilipy.simple_json.simple_json_from_html_string( + page, use_readability=False + ) @pytest.mark.usefixtures("python_readability", "bs4_413_append") diff --git a/src/fetch/tests/test_entrypoints.py b/src/fetch/tests/test_entrypoints.py index a929c2d792..90fbcd3de9 100644 --- a/src/fetch/tests/test_entrypoints.py +++ b/src/fetch/tests/test_entrypoints.py @@ -31,7 +31,7 @@ def run_main(monkeypatch: pytest.MonkeyPatch, *argv: str) -> AsyncMock: def test_main_defaults(monkeypatch: pytest.MonkeyPatch) -> None: serve = run_main(monkeypatch) - serve.assert_awaited_once_with(None, False, None, False, True) + serve.assert_awaited_once_with(None, False, None, False, None, True) def test_main_passes_every_flag(monkeypatch: pytest.MonkeyPatch) -> None: @@ -45,7 +45,7 @@ def test_main_passes_every_flag(monkeypatch: pytest.MonkeyPatch) -> None: "--allow-private-ips", ) serve.assert_awaited_once_with( - "MyBot/1.0", True, "http://proxy.example.com:8080", True, True + "MyBot/1.0", True, "http://proxy.example.com:8080", True, None, True ) @@ -90,7 +90,7 @@ def test_package_init_run_as_main_calls_main(monkeypatch: pytest.MonkeyPatch) -> monkeypatch.setattr(mcp_server_fetch.server, "serve", serve) monkeypatch.setattr(sys, "argv", ["mcp-server-fetch"]) runpy.run_module("mcp_server_fetch.__init__", run_name="__main__") - serve.assert_awaited_once_with(None, False, None, False, True) + serve.assert_awaited_once_with(None, False, None, False, None, True) def console_script() -> Path: diff --git a/src/fetch/tests/test_hidden_ssr.py b/src/fetch/tests/test_hidden_ssr.py index 5834564de0..4bdf338997 100644 --- a/src/fetch/tests/test_hidden_ssr.py +++ b/src/fetch/tests/test_hidden_ssr.py @@ -12,7 +12,7 @@ ``hidden``; the ``@mozilla/readability`` that readabilipy 0.3 installs also drops ``visibility: hidden`` and ``aria-hidden="true"``. The protocol tests that prove the second run happens use the two hidings both versions drop. They -count readabilipy calls, so "a normal page is unchanged" is asserted as "it +count extractor calls, so "a normal page is unchanged" is asserted as "it was extracted once, as before", not inferred from the Markdown. """ @@ -26,6 +26,7 @@ from bs4 import BeautifulSoup from mcp.types import CallToolResult +import mcp_server_fetch.server as server_module from mcp_server_fetch.server import _is_hidden from .harness import FakeWeb, connect @@ -49,15 +50,28 @@ def ssr_page(hiding: str) -> str: @pytest.fixture def calls(monkeypatch: pytest.MonkeyPatch) -> list[str]: - """Record every HTML string readabilipy is asked to simplify.""" + """Record every HTML string an extractor is asked to simplify. + + Both extractors: the server's Readability.js runner and readabilipy's + pure-Python extractor, which it falls back to. + """ seen: list[str] = [] - original = readabilipy.simple_json.simple_json_from_html_string + original_js = server_module.extract_article + original_py = readabilipy.simple_json.simple_json_from_html_string + + def spy_js(html: str) -> Any: + result = original_js(html) + # None: Readability.js did not run (no Node), so nothing was extracted. + if result is not None: + seen.append(html) + return result - def spy(html: str, **kwargs: Any) -> Any: + def spy_py(html: str, **kwargs: Any) -> Any: seen.append(html) - return original(html, **kwargs) + return original_py(html, **kwargs) - monkeypatch.setattr(readabilipy.simple_json, "simple_json_from_html_string", spy) + monkeypatch.setattr(server_module, "extract_article", spy_js) + monkeypatch.setattr(readabilipy.simple_json, "simple_json_from_html_string", spy_py) return seen diff --git a/src/fetch/tests/test_markdown_negotiation.py b/src/fetch/tests/test_markdown_negotiation.py new file mode 100644 index 0000000000..197a90bef7 --- /dev/null +++ b/src/fetch/tests/test_markdown_negotiation.py @@ -0,0 +1,137 @@ +"""Protocol-level tests for asking for native markdown (#5198). + +A fetch used to send httpx's default ``Accept: */*``, so a site that can serve +markdown through content negotiation sent HTML to be simplified, and a +``text/markdown`` response was labeled as content that "cannot be simplified +to markdown" behind the raw-content prefix. Now a non-raw fetch prefers +``text/markdown`` (HTML next, anything else last), and a markdown response is +returned as-is with no prefix. A ``raw`` fetch keeps the default ``Accept``, +so it gets the site's default representation (a page's HTML) as before. These +tests drive ``serve()`` through an MCP client with HTTP served by +``harness.FakeWeb``. +""" + +from __future__ import annotations + +import httpx +import pytest + +from mcp_server_fetch.server import MARKDOWN_ACCEPT, _is_markdown + +from .harness import FakeWeb, connect + +PAGE = "https://example.com/doc" +ROBOTS = "https://example.com/robots.txt" + +# Opens with an "<html" mention inside the first 100 characters, which the +# HTML sniff would otherwise take for a page to simplify. +MARKDOWN = "# Title\n\nUse an `<html>` tag at the top.\n\n- one\n- two\n" + +ARTICLE = """<html><head><title>Test Page +
+

Hello World

+

This is a link in a test paragraph +with enough words to count as the main content of the page.

+
+""" + + +def markdown(content_type: str = "text/markdown; charset=utf-8") -> httpx.Response: + return httpx.Response( + 200, content=MARKDOWN.encode(), headers={"content-type": content_type} + ) + + +async def call_fetch(web: FakeWeb, arguments: dict[str, object]) -> str: + async with connect() as (session, _): + result = await session.call_tool("fetch", {"url": PAGE, **arguments}) + data = result.model_dump(by_alias=True, mode="json", exclude_none=True) + assert data["isError"] is False + return data["content"][0]["text"] + + +def page_requests(web: FakeWeb) -> list[httpx.Request]: + return [r for r in web.requests if str(r.url) == PAGE] + + +async def test_a_fetch_prefers_markdown_then_html(web: FakeWeb) -> None: + web.add(ROBOTS, httpx.Response(404)) + web.add(PAGE, markdown()) + await call_fetch(web, {}) + [request] = page_requests(web) + assert request.headers["accept"] == "text/markdown, text/html;q=0.9, */*;q=0.8" + assert MARKDOWN_ACCEPT == request.headers["accept"] + # The robots.txt lookup is not a page fetch and asks for nothing new. + [robots] = [r for r in web.requests if str(r.url) == ROBOTS] + assert robots.headers["accept"] == "*/*" + + +@pytest.mark.parametrize( + "content_type", ["text/markdown", "text/markdown; charset=utf-8", "Text/Markdown"] +) +async def test_markdown_is_returned_as_is_without_the_raw_prefix( + web: FakeWeb, content_type: str +) -> None: + web.add(ROBOTS, httpx.Response(404)) + web.add(PAGE, markdown(content_type)) + text = await call_fetch(web, {}) + assert text == f"Contents of {PAGE}:\n{MARKDOWN}" + + +async def test_a_raw_fetch_keeps_the_default_accept(web: FakeWeb) -> None: + web.add(ROBOTS, httpx.Response(404)) + web.add( + PAGE, httpx.Response(200, text=ARTICLE, headers={"content-type": "text/html"}) + ) + text = await call_fetch(web, {"raw": True}) + [request] = page_requests(web) + assert request.headers["accept"] == "*/*" + assert text.endswith(f"Contents of {PAGE}:\n{ARTICLE}") + + +async def test_markdown_under_raw_is_unchanged_and_unprefixed(web: FakeWeb) -> None: + # Markdown served unasked (a .md file, say) is the same either way. + web.add(ROBOTS, httpx.Response(404)) + web.add(PAGE, markdown()) + text = await call_fetch(web, {"raw": True}) + assert text == f"Contents of {PAGE}:\n{MARKDOWN}" + + +@pytest.mark.usefixtures("python_readability") +async def test_html_from_a_site_that_ignores_accept_is_still_simplified( + web: FakeWeb, +) -> None: + web.add(ROBOTS, httpx.Response(404)) + web.add( + PAGE, httpx.Response(200, text=ARTICLE, headers={"content-type": "text/html"}) + ) + text = await call_fetch(web, {}) + assert "cannot be simplified" not in text + assert "
" not in text + assert "Hello World" in text + + +async def test_the_prompt_returns_markdown_as_is(web: FakeWeb) -> None: + web.add(PAGE, markdown()) + async with connect() as (session, _): + result = await session.get_prompt("fetch", {"url": PAGE}) + message = result.model_dump(by_alias=True, mode="json")["messages"][0] + assert message["content"]["text"].endswith(MARKDOWN) + assert "cannot be simplified" not in message["content"]["text"] + [request] = page_requests(web) + assert request.headers["accept"] == MARKDOWN_ACCEPT + + +@pytest.mark.parametrize( + ("content_type", "expected"), + [ + ("text/markdown", True), + (" text/markdown ; variant=GFM", True), + ("text/markdownish", False), + ("text/x-markdown", False), + ("text/plain", False), + ("", False), + ], +) +def test_is_markdown(content_type: str, expected: bool) -> None: + assert _is_markdown(content_type) is expected diff --git a/src/fetch/tests/test_page_title.py b/src/fetch/tests/test_page_title.py new file mode 100644 index 0000000000..be4978a31b --- /dev/null +++ b/src/fetch/tests/test_page_title.py @@ -0,0 +1,225 @@ +"""Tests for the page title on the fetch tool's simplified result (#2472). + +Readability.js drops an article heading that repeats the document title, so a +page such as pandas' "What's new in 2.1.0" release note came back without the +line naming it. ``with_page_title`` now puts the title on top as a heading +unless the Markdown already opens with it. The protocol-level tests drive the +real server through an MCP client with HTTP served by ``harness.FakeWeb``, on +both extractor paths, and pin that the raw result is untouched; the unit tests +below pin each rule for choosing the title and spotting a duplicate. +""" + +from __future__ import annotations + +from typing import Any + +import httpx +import pytest +from mcp import ClientSession +from mcp.types import CallToolResult + +from mcp_server_fetch.title import page_title, with_page_title + +from .harness import FakeWeb, connect + +PAGE = "https://example.com/page" +ROBOTS = "https://example.com/robots.txt" +PREFIX = f"Contents of {PAGE}:\n" + +# The shape of a Sphinx (pydata theme) page: `` adds the site name to +# the `<h1>`, which carries a "#" permalink. +BODY = "<p>These are the changes in pandas 2.1.0, with a long enough body. " * 8 +SPHINX_PAGE = f"""<!DOCTYPE html> +<html><head><title>What’s new in 2.1.0 (Aug 30, 2023) — pandas documentation +
+

What’s new in 2.1.0 (Aug 30, 2023)#

+{BODY}

+
""" + + +def text_of(result: CallToolResult) -> str: + assert not result.is_error, result + data = result.model_dump(by_alias=True, mode="json", exclude_none=True) + assert len(data["content"]) == 1 + return data["content"][0]["text"] + + +async def fetch(web: FakeWeb, html: str, **arguments: Any) -> str: + web.add(ROBOTS, httpx.Response(404)) + web.add( + PAGE, + httpx.Response(200, text=html, headers={"content-type": "text/html"}), + ) + async with connect() as (session, _): + assert isinstance(session, ClientSession) + result = await session.call_tool("fetch", {"url": PAGE, **arguments}) + return text_of(result) + + +@pytest.mark.usefixtures("node_readability", "private_tempdir") +async def test_readability_js_result_gains_the_dropped_title(web: FakeWeb) -> None: + text = await fetch(web, SPHINX_PAGE) + assert text.startswith(f"{PREFIX}# What’s new in 2.1.0 (Aug 30, 2023)\n\n") + assert text.count("What’s new in 2.1.0") == 1 + assert "These are the changes in pandas 2.1.0" in text + + +@pytest.mark.usefixtures("python_readability") +async def test_python_result_is_not_given_a_second_title(web: FakeWeb) -> None: + # The pure-Python extractor keeps the `

` (and opens with the + # ``), so the title is already there and nothing is added. + text = await fetch(web, SPHINX_PAGE) + assert not text.startswith(f"{PREFIX}# ") + assert "What’s new in 2.1.0" in text + assert "These are the changes in pandas 2.1.0" in text + + +async def test_raw_result_is_untouched(web: FakeWeb) -> None: + text = await fetch(web, SPHINX_PAGE, raw=True) + assert text.endswith(SPHINX_PAGE) + assert "\n# What’s new" not in text + + +@pytest.mark.usefixtures("node_readability", "private_tempdir") +async def test_failed_simplification_is_not_titled(web: FakeWeb) -> None: + text = await fetch( + web, "<html><head><title>Shell" + ) + assert text == f"{PREFIX}Page failed to be simplified from HTML" + + +@pytest.mark.parametrize( + ("html", "expected"), + [ + # og:title wins over everything else. + ( + 'T

T

', + "Social title", + ), + # The `

` wins when `` holds it. + ("<title>Heading | Site

Heading

", "Heading"), + # Otherwise ``. + ("<title>Document title

Something else

", "Document title"), + # A trailing "¶" permalink that is plain text is dropped. + ("Doc ¶", "Doc"), + # No ``: the `<h1>`. + ("<body><h1>Only heading</h1></body>", "Only heading"), + # An empty og:title and title fall through to the `<h1>`. + ( + '<meta property="og:title" content=""><title>

H

', + "H", + ), + # Nothing names the page. + ("

No title

", None), + ("

#

", None), + ], +) +def test_page_title(html: str, expected: str | None) -> None: + assert page_title(html) == expected + + +@pytest.mark.parametrize( + ("content", "expected"), + [ + # No heading: the title goes on top. + ("Body text.\n", "# Title Here\n\nBody text.\n"), + # Leading blank lines are folded into the separator. + ("\n\nBody text.\n", "# Title Here\n\nBody text.\n"), + # Already opens with the title as a heading, at any level. + ("# Title Here\n\nBody.", "# Title Here\n\nBody."), + ("\n## title here ¶\n\nBody.", "\n## title here ¶\n\nBody."), + # A first line that holds the whole title, heading or not. + ("

Title Here — Site

Skip\n", "

Title Here — Site

Skip\n"), + # A first heading the title holds (the title adds a site name). + ("# Title\n\nBody.", "# Title\n\nBody."), + # A first paragraph the title holds is not a heading: title added. + ("Title\n\nBody.", "# Title Here\n\nTitle\n\nBody."), + # A first line of punctuation only cannot match. + ("---\n\nBody.", "# Title Here\n\n---\n\nBody."), + # The title only in a link destination or tag is not shown: added. + ( + "[Source](https://example.com/title-here)\n", + "# Title Here\n\n[Source](https://example.com/title-here)\n", + ), + ( + "x\n", + "# Title Here\n\nx\n", + ), + # The title as a link's visible text is shown: not added. + ("[Title Here](https://example.com)\n", "[Title Here](https://example.com)\n"), + # Empty content still gains the title. + ("", "# Title Here\n\n"), + ], +) +def test_with_page_title(content: str, expected: str) -> None: + assert with_page_title("Title Here", content) == expected + + +def test_with_page_title_leaves_untitled_page_and_failure_alone() -> None: + assert with_page_title("

x

", "Body.") == "Body." + failure = "Page failed to be simplified from HTML" + assert with_page_title("T", failure) == failure + + +@pytest.mark.parametrize( + ("title", "content"), + [ + # Matching is by whole words, in both directions. + ("Rust", "Trust the compiler."), + ("Go", "Algorithms make this possible."), + ("Rust Guide", "# Rust Gui\n"), + ], +) +def test_partial_words_do_not_match(title: str, content: str) -> None: + html = f"{title}" + assert with_page_title(html, content) == f"# {title}\n\n{content}" + + +def test_code_span_contents_count_as_visible() -> None: + # A heading's inline HTML example is text, not a tag, inside a code span. + content = "# The `` element\n\nBody." + assert ( + with_page_title("The <input> element", content) == content + ) + + +@pytest.mark.parametrize( + ("title_html", "content"), + [ + # A "<" before a space is text, not a tag. + ("Comparing x < 5 and y > 3", "# Comparing x < 5 and y > 3\n\nBody."), + # Backslash-escaped brackets are text too. + ("Using <T>", "# Using \\\n\nBody."), + ], +) +def test_literal_angle_brackets_count_as_visible(title_html: str, content: str) -> None: + assert with_page_title(f"{title_html}", content) == content + + +@pytest.mark.parametrize("opener", ["](", " None: + # Linear in the line's length: a quadratic scan of 200,000 openers would + # not finish in the test's time. The title is not on the line, so it is added. + content = opener * 200_000 + result = with_page_title("Title Here", content) + assert result == f"# Title Here\n\n{content}" + + +@pytest.mark.parametrize( + ("title_html", "heading"), + [ + ("Using <T>", "# Using \\"), + ("__init__ and *args", "# \\_\\_init\\_\\_ and \\*args"), + ( + "[Draft] R&D #3 `x` | a~b \\ c", + "# \\[Draft\\] R\\&D \\#3 \\`x\\` \\| a\\~b \\\\ c", + ), + ], +) +def test_heading_escapes_markdown_and_html(title_html: str, heading: str) -> None: + result = with_page_title(f"{title_html}", "Body.") + assert result == f"{heading}\n\nBody." + + +def test_title_with_no_word_characters_is_not_matched() -> None: + assert with_page_title("—", "Body.") == "# —\n\nBody." diff --git a/src/fetch/tests/test_prompt_url.py b/src/fetch/tests/test_prompt_url.py new file mode 100644 index 0000000000..bc19088ccb --- /dev/null +++ b/src/fetch/tests/test_prompt_url.py @@ -0,0 +1,91 @@ +"""The `fetch` prompt rejects a malformed `url` as a bad argument (#4914). + +The tool's `url` is an AnyUrl, so pydantic rejects a malformed one before the +tool runs. The prompt's `url` is a plain string (prompt arguments always are), +and it used to go to httpx unchecked: `httpx.InvalidURL` is not an +`HTTPError`, so it escaped the prompt's error handling and the SDK answered +with JSON-RPC code 0, which a client cannot tell from a server fault. These +tests pin the fix at the protocol level: the same AnyUrl check, answered with +`-32602` and the validation message, in both protocol eras and on the raw +wire, with nothing fetched; and a valid URL still fetched, normalized the way +the tool normalizes it. +""" + +import httpx +import pytest +from mcp.shared.exceptions import MCPError +from mcp.types import INVALID_PARAMS, JSONRPCError, TextContent + +from .harness import FakeWeb, connect, raw_request +from .test_eras import ERAS +from .test_eras import connect as connect_era + +# (url, the pydantic message the error carries) +INVALID = [ + pytest.param("http://[::1", "invalid IPv6 address", id="unclosed-ipv6"), + pytest.param("not a url", "relative URL without a base", id="no-scheme"), + pytest.param("http://", "empty host", id="empty-host"), + pytest.param("http://example.com:99999/", "invalid port number", id="bad-port"), +] + + +def expected_message(url: str, reason: str) -> str: + return f"Invalid URL {url!r}: Input should be a valid URL, {reason}" + + +@pytest.mark.parametrize(("url", "reason"), INVALID) +async def test_invalid_prompt_url_is_invalid_params_on_the_wire( + web: FakeWeb, url: str, reason: str +) -> None: + # The issue's reproduction, as raw JSON-RPC: code -32602, never 0. + response = await raw_request( + "prompts/get", {"name": "fetch", "arguments": {"url": url}} + ) + assert isinstance(response, JSONRPCError), response + assert response.error.code == INVALID_PARAMS + assert response.error.message == expected_message(url, reason) + assert web.requests == [] + + +@pytest.mark.parametrize(("url", "reason"), INVALID) +async def test_invalid_prompt_url_is_invalid_params_through_a_client( + web: FakeWeb, url: str, reason: str +) -> None: + async with connect() as (session, _): + with pytest.raises(MCPError) as excinfo: + await session.get_prompt("fetch", {"url": url}) + assert excinfo.value.error.code == INVALID_PARAMS + assert excinfo.value.error.message == expected_message(url, reason) + assert web.requests == [] + + +@pytest.mark.parametrize(("mode", "negotiated"), ERAS) +async def test_invalid_prompt_url_is_invalid_params_in_both_eras( + web: FakeWeb, mode: str, negotiated: str +) -> None: + async with connect_era(mode) as client: + assert client.session.protocol_version == negotiated + with pytest.raises(MCPError) as excinfo: + await client.get_prompt("fetch", {"url": "http://[::1"}) + assert excinfo.value.error.code == INVALID_PARAMS + assert excinfo.value.error.message == expected_message( + "http://[::1", "invalid IPv6 address" + ) + assert web.requests == [] + + +async def test_valid_prompt_url_is_fetched_as_the_tool_would_fetch_it( + web: FakeWeb, +) -> None: + # A bare host is normalized with its trailing slash, as the tool's AnyUrl + # normalizes it, and the page comes back as the prompt's text. + web.add( + "https://example.com/", + httpx.Response(200, content=b"hello", headers={"content-type": "text/plain"}), + ) + async with connect() as (session, _): + result = await session.get_prompt("fetch", {"url": "https://example.com"}) + content = result.messages[0].content + assert isinstance(content, TextContent) + assert content.text.endswith("hello") + assert web.urls() == ["https://example.com/"] diff --git a/src/fetch/tests/test_protocol.py b/src/fetch/tests/test_protocol.py index 8090c7e8c3..c91ca82ab4 100644 --- a/src/fetch/tests/test_protocol.py +++ b/src/fetch/tests/test_protocol.py @@ -15,7 +15,6 @@ from __future__ import annotations -import os from collections.abc import AsyncIterator from contextlib import asynccontextmanager from importlib.metadata import version @@ -187,6 +186,15 @@ async def test_list_tools_wire_shape(web: FakeWeb) -> None: "title": "Raw", "type": "boolean", }, + # #4448: the server-wide timeout is the advertised default. + "timeout_ms": { + "default": 30000, + "description": "How long to wait, in milliseconds, to connect to the server and for each read of its response before giving up. Raise it for a slow server or a large download; lower it to fail fast.", + "maximum": 600000, + "minimum": 100, + "title": "Timeout Ms", + "type": "integer", + }, }, "required": ["url"], "title": "fetchArguments", @@ -999,9 +1007,10 @@ async def test_guard_uses_the_real_resolver() -> None: ) -async def test_timeouts_are_hard_coded(web: FakeWeb) -> None: - # Characterizes #4448: the page fetch uses a fixed 30s timeout and the - # robots.txt fetch uses httpx's 5s default; neither is configurable. +async def test_default_timeouts(web: FakeWeb) -> None: + # With nothing configured the page fetch waits 30s and the robots.txt + # fetch uses httpx's 5s default. Only the page timeout is configurable + # (#4448; tests/test_timeout.py). web.add(ROBOTS, plain("", status=404)) web.add(PAGE, plain("ok")) async with connect() as (session, _): @@ -1183,26 +1192,6 @@ async def test_get_prompt_with_unknown_name_is_a_jsonrpc_error( assert web.requests == [] -async def test_get_prompt_does_not_validate_the_url( - monkeypatch: pytest.MonkeyPatch, -) -> None: - # The prompt passes the raw string to httpx; an unusable URL comes back as - # a "Failed to fetch" prompt message rather than a validation error. - # No `web` fixture: the real transport rejects the URL before any I/O. - # Proxy variables are cleared so the real client cannot pick up one from - # the environment and send the request through it. - for name in list(os.environ): - if name.lower() in ("http_proxy", "https_proxy", "all_proxy"): - monkeypatch.delenv(name) - async with connect() as (session, _): - result = await session.get_prompt("fetch", {"url": "not a url"}) - data = wire(result) - assert data["description"] == PROMPT_DESCRIPTION - assert data["messages"][0]["content"]["text"].startswith( - "Failed to fetch not a url: UnsupportedProtocol(" - ) - - # --------------------------------------------------------------------------- # Protocol era # --------------------------------------------------------------------------- diff --git a/src/fetch/tests/test_readability_js_opt_out.py b/src/fetch/tests/test_readability_js_opt_out.py index 05cb77d89b..1a428d04c1 100644 --- a/src/fetch/tests/test_readability_js_opt_out.py +++ b/src/fetch/tests/test_readability_js_opt_out.py @@ -1,15 +1,14 @@ """Tests for ``--no-readability-js`` (#4199): an operator opt-out from Node.js. -readabilipy runs Mozilla's Readability.js under Node whenever Node is on PATH -(and readabilipy 0.3 may first run ``npm install``), in a subprocess with no -timeout. A Node or npm that hangs, or a deployment that must not start them, -had no remedy short of editing the installed package. With the flag the -server asks readabilipy for its pure-Python extractor, which never probes for -Node at all. - -The opt-out tests replace readabilipy's ``subprocess`` module with one that -fails on any use, so they prove no process is started rather than inferring -it from the Markdown that comes back. The default-path test needs Node on PATH, +The server runs Mozilla's Readability.js under Node whenever Node is on PATH +(and may first run ``npm install``, see ``readability_js``). A deployment that +must not start them can opt out: with the flag the server uses readabilipy's +pure-Python extractor and never probes for Node at all. + +The opt-out tests replace the ``subprocess`` module of both the server's +Readability.js runner and readabilipy with one that fails on any use, so they +prove no process is started rather than inferring it from the Markdown that +comes back. The default-path test needs Node on PATH, like the repository's other Readability.js tests. """ @@ -25,6 +24,8 @@ from mcp.types import CallToolResult import mcp_server_fetch +import mcp_server_fetch.server as server_module +from mcp_server_fetch import readability_js from .harness import FakeWeb, connect @@ -38,16 +39,20 @@ class NoProcesses: - """Stands in for readabilipy's ``subprocess`` module: any use fails the test.""" + """Stands in for a ``subprocess`` module: any use fails the test.""" def __getattr__(self, name: str) -> Any: - raise AssertionError(f"readabilipy used subprocess.{name}") + raise AssertionError(f"a process was started: subprocess.{name}") @pytest.fixture def no_processes(monkeypatch: pytest.MonkeyPatch) -> None: - # Both readabilipy's Node probe (`have_node`) and its Readability.js call - # go through this module attribute, as does 0.3's `npm install`. + # The server's Node probe, its `npm install` and its Readability.js run + # all go through `readability_js.subprocess`; readabilipy's own Node path + # (which the server no longer takes) goes through its module's. The probe + # cache is cleared so a probe made by an earlier test cannot hide one. + monkeypatch.setattr(readability_js, "_probed", False) + monkeypatch.setattr(readability_js, "subprocess", NoProcesses()) monkeypatch.setattr(readabilipy.simple_json, "subprocess", NoProcesses()) @@ -65,10 +70,8 @@ def html(body: str) -> httpx.Response: def test_no_processes_fixture_catches_the_node_probe(no_processes: None) -> None: # The guard the opt-out tests rely on does fire on the default path. - with pytest.raises(AssertionError, match="readabilipy used subprocess.run"): - readabilipy.simple_json.simple_json_from_html_string( - ARTICLE, use_readability=True - ) + with pytest.raises(AssertionError, match="a process was started: subprocess"): + server_module.extract_content_from_html(ARTICLE) def test_flag_is_passed_to_serve(monkeypatch: pytest.MonkeyPatch) -> None: @@ -76,7 +79,7 @@ def test_flag_is_passed_to_serve(monkeypatch: pytest.MonkeyPatch) -> None: monkeypatch.setattr(mcp_server_fetch, "serve", serve) monkeypatch.setattr(sys, "argv", ["mcp-server-fetch", "--no-readability-js"]) mcp_server_fetch.main() - serve.assert_awaited_once_with(None, False, None, False, False) + serve.assert_awaited_once_with(None, False, None, False, None, False) def test_flag_is_in_help( diff --git a/src/fetch/tests/test_readability_subprocess.py b/src/fetch/tests/test_readability_subprocess.py new file mode 100644 index 0000000000..c548032dba --- /dev/null +++ b/src/fetch/tests/test_readability_subprocess.py @@ -0,0 +1,516 @@ +"""Tests for the server's own Readability.js runner (``readability_js``). + +It replaces readabilipy's Node.js wrapper, which had no timeout and blocked +the event loop (#5306), wrote ``npm install`` output to stdout, the stdio +JSON-RPC channel (#5310), and could not find ``npm.cmd`` on Windows (#5313). + +Most tests put fake ``node`` and ``npm`` shell scripts first on ``PATH`` (the +reproduction #5306 describes), point the runner at a scratch copy of +readabilipy's ``javascript`` directory, and clear its once-per-process probe, +so no real Node.js runs and nothing reaches the network. Shell scripts need a +POSIX shell, so those tests skip on Windows; there the ``node_readability`` +tests in the other files drive the real ``npm.cmd`` (the Windows CI leg no +longer installs readabilipy's JavaScript for them). +""" + +from __future__ import annotations + +import os +import shutil +import stat +import subprocess +import sys +import time +from pathlib import Path +from typing import Any, cast + +import anyio +import httpx +import pytest + +import mcp_server_fetch.server as server_module +from mcp_server_fetch import readability_js + +from .harness import FakeWeb, connect + +PAGE = "https://example.com/article" + +ARTICLE = """Article
+

Heading

+

This is a link in a paragraph with +enough words to count as the main content of the page.

+
""" + +posix_only = pytest.mark.skipif( + sys.platform == "win32", reason="the fake node and npm are POSIX shell scripts" +) + +# A `node` that reports a version and then does what each test's body says +# with ExtractArticle.js's arguments: "$1" is the script, "$3" the input +# page and "$5" the output JSON. +NODE_VERSION = 'if [ "$1" = "-v" ]; then echo v24.0.0; exit 0; fi\n' + + +def html(body: str) -> httpx.Response: + return httpx.Response( + 200, content=body.encode(), headers={"content-type": "text/html"} + ) + + +@pytest.fixture +def bindir(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> Path: + """An empty directory that is the only place the runner looks for node and npm. + + The fake scripts' own utilities (sleep, cat, mkdir) still come from the + system directories on PATH, but the runner's lookup sees this directory + alone, so a real Node.js or npm installed anywhere (/usr/bin included, as + a distribution installs it) is never found or run. + """ + path = tmp_path / "bin" + path.mkdir() + monkeypatch.setenv("PATH", os.pathsep.join([str(path), "/usr/bin", "/bin"])) + which = shutil.which # the real one, before the module's is replaced + + def only_bindir(name: str) -> str | None: + return which(name, path=str(path)) + + monkeypatch.setattr(readability_js.shutil, "which", only_bindir) + return path + + +def tool(bindir: Path, name: str, body: str) -> Path: + script = bindir / name + script.write_text("#!/bin/sh\n" + body) + script.chmod(script.stat().st_mode | stat.S_IXUSR | stat.S_IXGRP | stat.S_IXOTH) + return script + + +@pytest.fixture +def jsdir(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> Path: + """A scratch readabilipy `javascript` directory, with a fresh probe.""" + path = tmp_path / "javascript" + path.mkdir() + monkeypatch.setattr(readability_js, "javascript_dir", lambda: path) + monkeypatch.setattr(readability_js, "_probed", False) + monkeypatch.setattr(readability_js, "_node", None) + return path + + +def install(jsdir: Path) -> None: + """Lay out node_modules as a completed `npm install` would.""" + for name in readability_js.JS_DEPENDENCIES: + package = jsdir / "node_modules" / name + package.mkdir(parents=True) + (package / "package.json").write_text("{}") + + +def alive(pid: int) -> bool: + try: + os.kill(pid, 0) + except ProcessLookupError: + return False + return True + + +def wait_dead(pid: int) -> bool: + # A killed process is reaped by its parent; give the kernel a moment. + deadline = time.monotonic() + 5 + while alive(pid): + if time.monotonic() > deadline: # pragma: no cover # only on failure + return False + time.sleep(0.05) + return True + + +# --- #5306: a bounded run, killed on timeout, off the event loop ------------ + + +@posix_only +async def test_hanging_node_is_killed_and_the_call_falls_back( + web: FakeWeb, bindir: Path, jsdir: Path, monkeypatch: pytest.MonkeyPatch +) -> None: + # The reproduction from #5306: a node that never exits. It also starts a + # child of its own, which the timeout must kill as well. + install(jsdir) + pids = bindir / "pids" + tool( + bindir, + "node", + NODE_VERSION + f'sleep 3600 &\necho "$$ $!" > {pids}\nwait\n', + ) + monkeypatch.setattr(readability_js, "EXTRACT_TIMEOUT_SECONDS", 1.0) + web.add(PAGE, html(ARTICLE)) + started = time.monotonic() + async with connect(ignore_robots_txt=True) as (session, _): + result = await session.call_tool("fetch", {"url": PAGE}) + assert time.monotonic() - started < 10 + assert result.is_error is False + [content] = result.content + assert content.type == "text" + # The pure-Python extractor's Markdown: it drops link targets. + assert "This is a link in a paragraph" in content.text + assert "https://example.com/x" not in content.text + for pid in map(int, pids.read_text().split()): + assert wait_dead(pid), f"process {pid} outlived the timeout" + + +@posix_only +async def test_extraction_does_not_block_other_requests( + web: FakeWeb, bindir: Path, jsdir: Path, monkeypatch: pytest.MonkeyPatch +) -> None: + install(jsdir) + started = bindir / "started" + tool(bindir, "node", NODE_VERSION + f"touch {started}\nexec sleep 3600\n") + monkeypatch.setattr(readability_js, "EXTRACT_TIMEOUT_SECONDS", 5.0) + web.add(PAGE, html(ARTICLE)) + async with connect(ignore_robots_txt=True) as (session, _): + fetched: list[float] = [] + listed = float("inf") + + async def call() -> None: + await session.call_tool("fetch", {"url": PAGE}) + fetched.append(time.monotonic()) + + async with anyio.create_task_group() as tg: + tg.start_soon(call) + while not started.exists(): + await anyio.sleep(0.05) + # Node.js is running (and hung); the loop still answers. + tools = await session.list_tools() + listed = time.monotonic() + assert [t.name for t in tools.tools] == ["fetch"] + # The fetch finished (by falling back) only after the timeout, well + # after the listing was answered. + assert listed < fetched[0] - 1 + + +# --- #5310: nothing a child prints reaches stdout --------------------------- + + +@posix_only +def test_npm_install_output_goes_to_stderr_not_stdout( + bindir: Path, jsdir: Path, capfd: pytest.CaptureFixture[str] +) -> None: + node = tool(bindir, "node", NODE_VERSION) + # npm prints to both streams, as the real one does, and installs. + deps = " ".join(f"{jsdir}/node_modules/{n}" for n in readability_js.JS_DEPENDENCIES) + tool( + bindir, + "npm", + "echo 'added 38 packages, and audited 39 packages in 470ms'\n" + "echo 'npm warn deprecated' >&2\n" + f"for d in {deps}; do mkdir -p $d && echo '{{}}' > $d/package.json; done\n", + ) + assert readability_js.node_path() == str(node) + out, err = capfd.readouterr() + assert out == "" + assert "added 38 packages" in err + assert "npm warn deprecated" in err + assert "contacts the npm registry" in err + + +@posix_only +def test_failed_extraction_stderr_goes_to_stderr_not_stdout( + bindir: Path, jsdir: Path, capfd: pytest.CaptureFixture[str] +) -> None: + install(jsdir) + tool( + bindir, + "node", + NODE_VERSION + "echo 'on stdout'\necho 'TypeError: x' >&2\nexit 1\n", + ) + assert readability_js.extract_article(ARTICLE) is None + out, err = capfd.readouterr() + assert out == "" + assert "TypeError: x" in err + assert "Readability.js failed (exit 1)" in err + + +@posix_only +def test_children_do_not_read_the_servers_stdin( + bindir: Path, jsdir: Path, tmp_path: Path +) -> None: + # A child given the server's stdin could consume JSON-RPC input. + seen = tmp_path / "stdin" + script = tool(bindir, "reader", f"cat > {seen}\n") + r, w = os.pipe() + saved = os.dup(0) + try: + os.dup2(r, 0) + os.write(w, b"not for the child\n") + os.close(w) + done = readability_js._run([str(script)], timeout=5) + finally: + os.dup2(saved, 0) + os.close(saved) + os.close(r) + assert done is not None and done.returncode == 0 + assert seen.read_text() == "" + + +# --- #5313: node and npm are resolved through PATH (and PATHEXT) ------------ + + +def test_npm_is_run_by_the_path_shutil_which_resolves( + jsdir: Path, monkeypatch: pytest.MonkeyPatch +) -> None: + # On Windows `shutil.which("npm")` finds `npm.cmd`, which a bare "npm" + # passed to CreateProcess does not; whatever it finds is what runs. + found = {"node": r"C:\nodejs\node.exe", "npm": r"C:\nodejs\npm.CMD"} + monkeypatch.setattr(readability_js.shutil, "which", found.get) + runs: list[list[str]] = [] + + def run(argv: list[str], timeout: float, cwd: Path | None = None) -> Any: + runs.append(argv) + if argv[1] == "install": + install(jsdir) + return subprocess.CompletedProcess(argv, 0, "v24.0.0\n", "") + + monkeypatch.setattr(readability_js, "_run", run) + assert readability_js.node_path() == found["node"] + assert runs == [ + [found["node"], "-v"], + [found["npm"], "install", "--no-audit", "--no-fund"], + ] + + +# --- Each way Readability.js can be unavailable falls back ------------------- + + +@posix_only +def test_no_node_on_path( + bindir: Path, jsdir: Path, capfd: pytest.CaptureFixture[str] +) -> None: + assert readability_js.extract_article(ARTICLE) is None + assert "Node.js was not found on PATH" in capfd.readouterr().err + + +@posix_only +@pytest.mark.parametrize( + "body", ["echo v12.22.0\n", "echo garbage\n", "exit 3\n", "exec sleep 3600\n"] +) +def test_old_or_broken_node( + bindir: Path, + jsdir: Path, + body: str, + monkeypatch: pytest.MonkeyPatch, + capfd: pytest.CaptureFixture[str], +) -> None: + monkeypatch.setattr(readability_js, "VERSION_TIMEOUT_SECONDS", 0.5) + tool(bindir, "node", body) + assert readability_js.node_path() is None + assert "Node.js >= 14 is needed" in capfd.readouterr().err + + +@posix_only +def test_no_npm_for_missing_dependencies( + bindir: Path, jsdir: Path, capfd: pytest.CaptureFixture[str] +) -> None: + tool(bindir, "node", NODE_VERSION) + assert readability_js.node_path() is None + assert "npm was not found on PATH" in capfd.readouterr().err + + +@posix_only +def test_failed_npm_install( + bindir: Path, jsdir: Path, capfd: pytest.CaptureFixture[str] +) -> None: + tool(bindir, "node", NODE_VERSION) + tool(bindir, "npm", "echo 'npm error network ENOTFOUND' >&2\nexit 1\n") + assert readability_js.node_path() is None + err = capfd.readouterr().err + assert "npm error network ENOTFOUND" in err + assert "npm install failed (exit 1)" in err + + +@posix_only +def test_hanging_npm_install_is_killed( + bindir: Path, + jsdir: Path, + monkeypatch: pytest.MonkeyPatch, + capfd: pytest.CaptureFixture[str], +) -> None: + monkeypatch.setattr(readability_js, "INSTALL_TIMEOUT_SECONDS", 0.5) + tool(bindir, "node", NODE_VERSION) + tool(bindir, "npm", "exec sleep 3600\n") + assert readability_js.node_path() is None + assert "npm did not finish within 0.5 s and was killed" in capfd.readouterr().err + + +@posix_only +def test_probe_runs_once_per_process(bindir: Path, jsdir: Path) -> None: + calls = bindir / "calls" + node = tool(bindir, "node", f"echo x >> {calls}\n" + NODE_VERSION) + install(jsdir) + assert readability_js.node_path() == str(node) + assert readability_js.node_path() == str(node) + assert calls.read_text() == "x\n" + + +@posix_only +def test_unstartable_node( + bindir: Path, jsdir: Path, capfd: pytest.CaptureFixture[str] +) -> None: + # Found on PATH but not executable as a program (no interpreter line). + script = bindir / "node" + script.write_bytes(b"\x00\x01") + script.chmod(0o755) + assert readability_js.node_path() is None + assert "could not start" in capfd.readouterr().err + + +@posix_only +@pytest.mark.parametrize( + ("body", "expected"), + [ + # Readability.parse() found no article. + ('echo null > "$5"\n', ""), + ('echo \'{"content": null}\' > "$5"\n', ""), + ( + 'echo \'{"content": "

hi

"}\' > "$5"\n', + "

hi

", + ), + # Exit 0 but no output file: a failure, so fall back. + ("exit 0\n", None), + # Output that is not JSON. + ('echo "{" > "$5"\n', None), + ], +) +def test_extraction_outcomes( + bindir: Path, jsdir: Path, body: str, expected: str | None +) -> None: + install(jsdir) + tool(bindir, "node", NODE_VERSION + body) + assert readability_js.extract_article(ARTICLE) == expected + + +@posix_only +def test_extraction_gets_the_page_and_runs_extractarticle( + bindir: Path, jsdir: Path, tmp_path: Path +) -> None: + install(jsdir) + copy = tmp_path / "seen.html" + args = tmp_path / "args" + tool( + bindir, + "node", + NODE_VERSION + + f'echo "$1" > {args}\ncp "$3" {copy}\n' + + 'echo \'{"content": "

ok

"}\' > "$5"\n', + ) + assert readability_js.extract_article("

é

") == "

ok

" + assert copy.read_text(encoding="utf-8") == "

é

" + assert args.read_text().strip() == str(jsdir / "ExtractArticle.js") + + +# --- Killing the process tree ------------------------------------------------ + + +def test_kill_tree_uses_taskkill_on_windows(monkeypatch: pytest.MonkeyPatch) -> None: + runs: list[list[str]] = [] + killed: list[bool] = [] + + class Proc: + pid = 4242 + + def kill(self) -> None: + killed.append(True) + + monkeypatch.setattr(readability_js.sys, "platform", "win32") + monkeypatch.setattr( + readability_js.subprocess, "run", lambda argv, **_: runs.append(argv) + ) + readability_js._kill_tree(cast("subprocess.Popen[str]", Proc())) + assert runs == [["taskkill", "/F", "/T", "/PID", "4242"]] + assert killed == [True] + + +def test_kill_tree_kills_the_child_when_taskkill_fails( + monkeypatch: pytest.MonkeyPatch, +) -> None: + killed: list[bool] = [] + timeouts: list[Any] = [] + + class Proc: + pid = 4242 + + def kill(self) -> None: + killed.append(True) + + def run(argv: list[str], **kwargs: Any) -> None: + timeouts.append(kwargs.get("timeout")) + raise OSError("taskkill is blocked") + + monkeypatch.setattr(readability_js.sys, "platform", "win32") + monkeypatch.setattr(readability_js.subprocess, "run", run) + readability_js._kill_tree(cast("subprocess.Popen[str]", Proc())) + assert timeouts == [readability_js.KILL_TIMEOUT_SECONDS] + assert killed == [True] + + +@posix_only +def test_timeout_returns_while_a_detached_helper_holds_the_pipes( + bindir: Path, jsdir: Path, monkeypatch: pytest.MonkeyPatch, tmp_path: Path +) -> None: + # A helper that leaves the process group (its own session) survives the + # group kill and keeps stdout and stderr open; the run must still end. + install(jsdir) + pid = tmp_path / "helper.pid" + helper = ( + "import os, time; os.setsid(); " + f"open({str(pid)!r}, 'w').write(str(os.getpid())); time.sleep(3600)" + ) + tool(bindir, "node", NODE_VERSION + f'"{sys.executable}" -c "{helper}" &\nwait\n') + monkeypatch.setattr(readability_js, "EXTRACT_TIMEOUT_SECONDS", 1.0) + started = time.monotonic() + try: + assert readability_js.extract_article(ARTICLE) is None + assert ( + time.monotonic() - started < 1.0 + readability_js.KILL_TIMEOUT_SECONDS + 3 + ) + finally: + deadline = time.monotonic() + 5 + while not pid.exists() and time.monotonic() < deadline: # pragma: no cover + time.sleep(0.05) + os.kill(int(pid.read_text()), 9) + + +@posix_only +def test_kill_tree_tolerates_a_group_already_gone( + monkeypatch: pytest.MonkeyPatch, +) -> None: + proc = subprocess.Popen(["true"], start_new_session=True) + proc.wait() + + def gone(pid: int, sig: int) -> None: + raise ProcessLookupError + + monkeypatch.setattr(readability_js.os, "killpg", gone) + readability_js._kill_tree(cast("subprocess.Popen[str]", proc)) + + +# --- The server's use of the runner ----------------------------------------- + + +def test_a_failed_second_run_keeps_the_first_result( + monkeypatch: pytest.MonkeyPatch, +) -> None: + # #3878's recovery re-runs Readability.js; if that run fails (None), the + # first run's content is returned rather than nothing. + page = ( + "

short shell

" + ) + results = iter(["

short shell

", None]) + monkeypatch.setattr(server_module, "extract_article", lambda _: next(results)) + assert server_module.extract_content_from_html(page).strip() == "short shell" + + +def test_unavailable_readability_js_falls_back_to_python( + monkeypatch: pytest.MonkeyPatch, +) -> None: + monkeypatch.setattr(server_module, "extract_article", lambda _: None) + text = server_module.extract_content_from_html(ARTICLE) + assert "This is a link in a paragraph" in text + assert "https://example.com/x" not in text diff --git a/src/fetch/tests/test_timeout.py b/src/fetch/tests/test_timeout.py new file mode 100644 index 0000000000..cdfff77502 --- /dev/null +++ b/src/fetch/tests/test_timeout.py @@ -0,0 +1,287 @@ +"""Tests for the configurable HTTP request timeout (#4448). + +The timeout is set at three levels, the most specific winning: the ``fetch`` +tool's ``timeout_ms`` argument, then ``--timeout-ms`` / ``FETCH_TIMEOUT_MS``, +then the 30 s the server always used. Tool and prompt tests drive the real +server through an MCP client (``harness.connect``) with HTTP served by +``harness.FakeWeb``, and read the timeout httpx actually applied from the +request's ``timeout`` extension, so they assert what reaches the transport +rather than what a function was called with. ``main()`` runs in-process with +``serve`` patched. +""" + +from __future__ import annotations + +import os +import sys +from typing import Any +from unittest.mock import AsyncMock + +import httpx +import pytest + +import mcp_server_fetch +from mcp_server_fetch.server import create_server +from mcp_server_fetch.timeout import ( + DEFAULT_TIMEOUT_MS, + MAX_TIMEOUT_MS, + MIN_TIMEOUT_MS, + TIMEOUT_ENV, + timeout_ms_from_env, +) + +from .harness import FakeWeb, connect + +PAGE = "https://example.com/page" +TOOL_ERROR = "Error executing tool fetch: " + + +@pytest.fixture(autouse=True) +def no_timeout_env(monkeypatch: pytest.MonkeyPatch) -> None: + # A developer's own FETCH_TIMEOUT_MS must not leak into these tests, nor + # what main() writes to it into later tests. setenv first: delenv of an + # absent variable records no undo, so main()'s write would survive. + monkeypatch.setenv(TIMEOUT_ENV, "") + monkeypatch.delenv(TIMEOUT_ENV) + + +def recording_page(seen: list[dict[str, Any]]) -> Any: + def respond(request: httpx.Request) -> httpx.Response: + seen.append(dict(request.extensions["timeout"])) + return httpx.Response( + 200, content=b"hello", headers={"content-type": "text/plain"} + ) + + return respond + + +def every_phase(seconds: float) -> dict[str, float]: + return {"connect": seconds, "read": seconds, "write": seconds, "pool": seconds} + + +def wire(result: Any) -> dict[str, Any]: + return result.model_dump(by_alias=True, mode="json", exclude_none=True) + + +async def fetch_timeouts( + web: FakeWeb, arguments: dict[str, Any] +) -> list[dict[str, Any]]: + seen: list[dict[str, Any]] = [] + web.add(PAGE, recording_page(seen)) + async with connect(ignore_robots_txt=True) as (session, _): + result = await session.call_tool("fetch", {"url": PAGE, **arguments}) + assert wire(result)["isError"] is False + return seen + + +# --- tool ------------------------------------------------------------------- + + +async def test_default_is_thirty_seconds(web: FakeWeb) -> None: + assert await fetch_timeouts(web, {}) == [every_phase(30.0)] + + +async def test_environment_sets_the_server_default( + web: FakeWeb, monkeypatch: pytest.MonkeyPatch +) -> None: + monkeypatch.setenv(TIMEOUT_ENV, "5000") + assert await fetch_timeouts(web, {}) == [every_phase(5.0)] + + +async def test_tool_argument_outranks_the_server_default( + web: FakeWeb, monkeypatch: pytest.MonkeyPatch +) -> None: + monkeypatch.setenv(TIMEOUT_ENV, "5000") + assert await fetch_timeouts(web, {"timeout_ms": 120000}) == [every_phase(120.0)] + + +@pytest.mark.parametrize("timeout_ms", [MIN_TIMEOUT_MS, MAX_TIMEOUT_MS]) +async def test_tool_argument_bounds_are_inclusive( + web: FakeWeb, timeout_ms: int +) -> None: + assert await fetch_timeouts(web, {"timeout_ms": timeout_ms}) == [ + every_phase(timeout_ms / 1000) + ] + + +@pytest.mark.parametrize( + "timeout_ms", [MIN_TIMEOUT_MS - 1, MAX_TIMEOUT_MS + 1, 0, -1, "1000", 1.5, True] +) +async def test_tool_argument_outside_the_schema_is_a_tool_error( + web: FakeWeb, timeout_ms: Any +) -> None: + web.add(PAGE, httpx.Response(200, content=b"never fetched")) + async with connect(ignore_robots_txt=True) as (session, _): + result = await session.call_tool( + "fetch", {"url": PAGE, "timeout_ms": timeout_ms} + ) + data = wire(result) + assert data["isError"] is True + assert "timeout_ms" in data["content"][0]["text"] + assert web.requests == [] + + +async def test_schema_advertises_bounds_and_the_server_default( + web: FakeWeb, monkeypatch: pytest.MonkeyPatch +) -> None: + monkeypatch.setenv(TIMEOUT_ENV, "7000") + async with connect() as (session, _): + [tool] = wire(await session.list_tools())["tools"] + prop = tool["inputSchema"]["properties"]["timeout_ms"] + assert prop["type"] == "integer" + assert prop["minimum"] == MIN_TIMEOUT_MS + assert prop["maximum"] == MAX_TIMEOUT_MS + assert prop["default"] == 7000 + assert "milliseconds" in prop["description"] + assert tool["inputSchema"]["required"] == ["url"] + + +async def test_a_timeout_says_how_long_it_waited(web: FakeWeb) -> None: + def hang(request: httpx.Request) -> httpx.Response: + raise httpx.ReadTimeout("timed out", request=request) + + web.add(PAGE, hang) + async with connect(ignore_robots_txt=True) as (session, _): + result = await session.call_tool("fetch", {"url": PAGE, "timeout_ms": 2000}) + data = wire(result) + assert data["isError"] is True + [content] = data["content"] + assert content["text"].startswith( + f"{TOOL_ERROR}Failed to fetch {PAGE}: timed out after 2000 ms: ReadTimeout(" + ) + + +async def test_robots_txt_check_keeps_the_httpx_default(web: FakeWeb) -> None: + # Only the page request is configurable; the robots.txt check is not. + robots: list[dict[str, Any]] = [] + page: list[dict[str, Any]] = [] + + def robots_txt(request: httpx.Request) -> httpx.Response: + robots.append(dict(request.extensions["timeout"])) + return httpx.Response(200, content=b"User-agent: *\nAllow: /") + + web.add("https://example.com/robots.txt", robots_txt) + web.add(PAGE, recording_page(page)) + async with connect() as (session, _): + result = await session.call_tool("fetch", {"url": PAGE, "timeout_ms": 9000}) + assert wire(result)["isError"] is False + assert robots == [every_phase(5.0)] + assert page == [every_phase(9.0)] + + +# --- prompt ----------------------------------------------------------------- + + +async def test_prompt_uses_the_server_default( + web: FakeWeb, monkeypatch: pytest.MonkeyPatch +) -> None: + monkeypatch.setenv(TIMEOUT_ENV, "4000") + seen: list[dict[str, Any]] = [] + web.add(PAGE, recording_page(seen)) + async with connect() as (session, _): + result = await session.get_prompt("fetch", {"url": PAGE}) + assert "hello" in str(result.messages[0].content) + assert seen == [every_phase(4.0)] + + +# --- server-wide setting ---------------------------------------------------- + + +@pytest.mark.parametrize("value", ["", " "]) +def test_unset_or_blank_environment_is_the_default( + monkeypatch: pytest.MonkeyPatch, value: str +) -> None: + monkeypatch.setenv(TIMEOUT_ENV, value) + assert timeout_ms_from_env() == DEFAULT_TIMEOUT_MS + + +def test_environment_value_is_trimmed(monkeypatch: pytest.MonkeyPatch) -> None: + monkeypatch.setenv(TIMEOUT_ENV, " 2500 ") + assert timeout_ms_from_env() == 2500 + + +@pytest.mark.parametrize( + ("value", "message"), + [ + ("abc", "'abc' is not a whole number of milliseconds"), + ("1.5", "'1.5' is not a whole number of milliseconds"), + ("99", f"99 ms is outside {MIN_TIMEOUT_MS}..{MAX_TIMEOUT_MS} ms"), + ("600001", f"600001 ms is outside {MIN_TIMEOUT_MS}..{MAX_TIMEOUT_MS} ms"), + ], +) +def test_bad_environment_stops_the_server_at_startup( + monkeypatch: pytest.MonkeyPatch, value: str, message: str +) -> None: + monkeypatch.setenv(TIMEOUT_ENV, value) + with pytest.raises(ValueError, match=f"^{TIMEOUT_ENV}: {message}$"): + create_server() + + +# --- command line ----------------------------------------------------------- + + +def run_main(monkeypatch: pytest.MonkeyPatch, *argv: str) -> AsyncMock: + serve = AsyncMock() + monkeypatch.setattr(mcp_server_fetch, "serve", serve) + monkeypatch.setattr(sys, "argv", ["mcp-server-fetch", *argv]) + mcp_server_fetch.main() + return serve + + +def test_flag_sets_the_variable_the_server_reads( + monkeypatch: pytest.MonkeyPatch, +) -> None: + serve = run_main(monkeypatch, "--timeout-ms", "60000") + serve.assert_awaited_once() + assert os.environ[TIMEOUT_ENV] == "60000" + assert timeout_ms_from_env() == 60000 + + +def test_flag_outranks_the_environment(monkeypatch: pytest.MonkeyPatch) -> None: + # Even a bad value in the environment: the flag replaces it. + monkeypatch.setenv(TIMEOUT_ENV, "nonsense") + run_main(monkeypatch, "--timeout-ms", "1500") + assert os.environ[TIMEOUT_ENV] == "1500" + + +def test_no_flag_leaves_the_environment_alone( + monkeypatch: pytest.MonkeyPatch, +) -> None: + monkeypatch.setenv(TIMEOUT_ENV, "8000") + run_main(monkeypatch) + assert os.environ[TIMEOUT_ENV] == "8000" + + +@pytest.mark.parametrize("value", ["abc", "50", "600001"]) +def test_bad_flag_is_a_usage_error( + monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str], value: str +) -> None: + with pytest.raises(SystemExit) as excinfo: + run_main(monkeypatch, "--timeout-ms", value) + assert excinfo.value.code == 2 + assert "argument --timeout-ms:" in capsys.readouterr().err + assert TIMEOUT_ENV not in os.environ + + +def test_bad_environment_is_a_usage_error( + monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str] +) -> None: + monkeypatch.setenv(TIMEOUT_ENV, "soon") + with pytest.raises(SystemExit) as excinfo: + run_main(monkeypatch) + assert excinfo.value.code == 2 + assert ( + f"{TIMEOUT_ENV}: 'soon' is not a whole number of milliseconds" + in capsys.readouterr().err + ) + + +def test_flag_is_in_help( + monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str] +) -> None: + with pytest.raises(SystemExit) as excinfo: + run_main(monkeypatch, "--help") + assert excinfo.value.code == 0 + out = capsys.readouterr().out + assert "--timeout-ms" in out + assert TIMEOUT_ENV in out diff --git a/src/fetch/uv.lock b/src/fetch/uv.lock index 84cb448773..c544141f6f 100644 --- a/src/fetch/uv.lock +++ b/src/fetch/uv.lock @@ -745,7 +745,7 @@ wheels = [ [[package]] name = "mcp-server-fetch" -version = "2026.10.28" +version = "2026.11.4" source = { editable = "." } dependencies = [ { name = "beautifulsoup4" }, diff --git a/src/filesystem/CHANGELOG.md b/src/filesystem/CHANGELOG.md index 6000a2b949..652dd0638e 100644 --- a/src/filesystem/CHANGELOG.md +++ b/src/filesystem/CHANGELOG.md @@ -1,5 +1,47 @@ # @modelcontextprotocol/server-filesystem +## 2.2.0 + +### Minor Changes + +- [#5395](https://github.com/modelcontextprotocol/servers/pull/5395) [`99ba0c6`](https://github.com/modelcontextprotocol/servers/commit/99ba0c60a1b61556f8f30a03e0bf68328ad9331f) [@cliffhall](https://github.com/cliffhall) - Add a `--confine-roots` command-line flag. With it, a 2025-era client's Roots can narrow the command-line directories but never replace or widen them: a root inside a command-line directory is allowed, a root containing one is narrowed to it, and a root outside all of them is ignored and logged. Without the flag, Roots replace the command-line directories as before. + +- [#5359](https://github.com/modelcontextprotocol/servers/pull/5359) [`ea2997b`](https://github.com/modelcontextprotocol/servers/commit/ea2997b807b157978c0e18760433c9fedb8a20d4) [@cliffhall](https://github.com/cliffhall) - Add a `--read-only` command-line flag. With it the server registers only the tools annotated `readOnlyHint: true`, so every tool that writes (such as `write_file` and `edit_file`) is not listed and cannot be called, on both protocol eras and whether the directories come from the command line or a client's Roots. `--` now ends the options, so a directory named like an option can still be given after it. + +- [#5376](https://github.com/modelcontextprotocol/servers/pull/5376) [`af8a01b`](https://github.com/modelcontextprotocol/servers/commit/af8a01b598ba5d8037c574a7ef77f2a7bf98c444) [@cliffhall](https://github.com/cliffhall) - Add an `append_file` tool that appends text to the end of a file, exactly as given, creating the file if it does not exist ([#1208](https://github.com/modelcontextprotocol/servers/issues/1208)). + +- [#5385](https://github.com/modelcontextprotocol/servers/pull/5385) [`65e68d3`](https://github.com/modelcontextprotocol/servers/commit/65e68d3f382b8e097e16d665808d28c3f13b6f71) [@cliffhall](https://github.com/cliffhall) - Add a `copy_file` tool that copies one regular file to a new location, keeping its permissions and modification time, and never overwrites an existing destination ([#519](https://github.com/modelcontextprotocol/servers/issues/519)). + +- [#5388](https://github.com/modelcontextprotocol/servers/pull/5388) [`3a2e1fd`](https://github.com/modelcontextprotocol/servers/commit/3a2e1fde2252749df82773e7accc5d1976d20313) [@cliffhall](https://github.com/cliffhall) - Add an optional `depth` parameter to `directory_tree` that limits how many levels the tree descends; omitted, the whole tree is returned as before ([#1077](https://github.com/modelcontextprotocol/servers/issues/1077)). + +- [#5368](https://github.com/modelcontextprotocol/servers/pull/5368) [`3f173d6`](https://github.com/modelcontextprotocol/servers/commit/3f173d6f674cb1e33d45ae724dfc334e1e3a6e73) [@cliffhall](https://github.com/cliffhall) - Expose files in the allowed directories as MCP resources through a `file://{+path}` resource template, so a client can read a file with `resources/read` (text or a base64 blob by MIME type), confined to the allowed directories exactly as the tools are. `resources/list` stays empty, so the server never enumerates the disk. + +### Patch Changes + +- [#5390](https://github.com/modelcontextprotocol/servers/pull/5390) [`6017ed3`](https://github.com/modelcontextprotocol/servers/commit/6017ed304d0df3e23415182c408ee05fc0b523b9) [@cliffhall](https://github.com/cliffhall) - Log the effective allowed directories to stderr, with their source, at startup and whenever they change: from the command line, or from the client's Roots (saying whether those replaced the command-line directories), or none, with why every path is now refused. Previously the server logged only a count of valid Roots. + +- [#5426](https://github.com/modelcontextprotocol/servers/pull/5426) [`81ca196`](https://github.com/modelcontextprotocol/servers/commit/81ca19651d2b3dd09c616223f7afc35d1e0e16ba) [@cliffhall](https://github.com/cliffhall) - Move `minimatch` to `^11.0.4`. Every 10.x release is affected by GHSA-g3hv-75wg-m249 (denial of service through glob-part amplification), and the `search_files` patterns and the `excludePatterns` of `search_files` and `directory_tree` reach it from the client. The fix is outside the old `^10.0.1` range, so neither a fresh install nor `npm update` could pick it up. + +- [#5380](https://github.com/modelcontextprotocol/servers/pull/5380) [`d665e43`](https://github.com/modelcontextprotocol/servers/commit/d665e43ee75484889c6910f5fa356b530badbbbf) [@cliffhall](https://github.com/cliffhall) - Report a dangling symlink as a broken symlink. A request through a symlink whose target does not exist is still refused, but the error now names the symlink instead of saying its parent directory, which exists, does not. + +- [#5379](https://github.com/modelcontextprotocol/servers/pull/5379) [`6575d87`](https://github.com/modelcontextprotocol/servers/commit/6575d8705e95160b3dbfe380da8c0c5e01b170e0) [@cliffhall](https://github.com/cliffhall) - `edit_file`: when an edit's `oldText` matches nowhere, the error now names which edit failed, states the matching rules, and quotes the closest lines of the file with line numbers, plus the first differing line JSON-quoted so a whitespace difference is visible. The tool description now states the matching rules too. What matches is unchanged. + +- [#5384](https://github.com/modelcontextprotocol/servers/pull/5384) [`eed581e`](https://github.com/modelcontextprotocol/servers/commit/eed581eda6d76eac3229bb73b3555cc1bc18edc6) [@cliffhall](https://github.com/cliffhall) - Expand `~\` to the home directory on Windows, in command-line directories, request paths and client Roots alike, as `~/` already is. On POSIX a backslash is a filename character, so `~\name` there still names a literal file. + +- [#5377](https://github.com/modelcontextprotocol/servers/pull/5377) [`fbddfb0`](https://github.com/modelcontextprotocol/servers/commit/fbddfb02d8fe8d9fefb29f3e468241c40a490359) [@cliffhall](https://github.com/cliffhall) - `read_text_file` (and the deprecated `read_file`) now treats `head: 0` or `tail: 0` as a request for zero lines and returns empty text, instead of the whole file. `head` and `tail` accept only non-negative integers, and `head` with `tail` is rejected even when either is 0. + +- [#5353](https://github.com/modelcontextprotocol/servers/pull/5353) [`1a7b9f9`](https://github.com/modelcontextprotocol/servers/commit/1a7b9f975e0390183d18280bf898cea53e631cf3) [@cliffhall](https://github.com/cliffhall) - Every read-only tool (`read_file`, `read_text_file`, `read_media_file`, `read_multiple_files`, `list_directory`, `list_directory_with_sizes`, `directory_tree`, `search_files`, `get_file_info` and `list_allowed_directories`) now also sets `idempotentHint: true` and `destructiveHint: false`, so all four tool annotation hints are explicit on every tool and a client never falls back to the spec's defaults (destructive, not idempotent). + +- [#5383](https://github.com/modelcontextprotocol/servers/pull/5383) [`22ce3e5`](https://github.com/modelcontextprotocol/servers/commit/22ce3e52148f95bfb991dbfd84d9bcaecb4678f7) [@cliffhall](https://github.com/cliffhall) - Keep an allowed directory's name exactly as the file system spells it. A command-line directory, or a client Root's realpath, whose name ends in a space or a quote was trimmed, so the server allowed the sibling directory without that suffix (and refused the directory that was actually given). + +- [#5351](https://github.com/modelcontextprotocol/servers/pull/5351) [`13c781a`](https://github.com/modelcontextprotocol/servers/commit/13c781a79411fb04efbd540505cfe0a207f580a1) [@cliffhall](https://github.com/cliffhall) - Drop the unused `glob` runtime dependency: nothing in the server imported it (pattern matching goes through `minimatch`), so installing the server no longer downloads `glob` and its own dependencies. + +- [#5378](https://github.com/modelcontextprotocol/servers/pull/5378) [`9192ea3`](https://github.com/modelcontextprotocol/servers/commit/9192ea3655cbe4cda51f42c33749008867d298c7) [@cliffhall](https://github.com/cliffhall) - `list_directory_with_sizes` no longer reports an entry it could not stat (unreadable, a dangling symlink, removed mid-listing, or a symlink leaving the allowed directories) as `0 B`. Such an entry is shown with size `unknown`, sorts last by size, and is left out of the combined size, which notes how many files it excludes. + +- [#5367](https://github.com/modelcontextprotocol/servers/pull/5367) [`4441e23`](https://github.com/modelcontextprotocol/servers/commit/4441e23c4c9e9c6d415b0ef5abb79c00e31f08ce) [@cliffhall](https://github.com/cliffhall) - The `search_files` description now says plainly that it finds files and directories by name, not by content, and that its glob is matched case-sensitively against each entry's path relative to `path` (not the working directory), so a model no longer reaches for it to grep file contents. + +- [#5355](https://github.com/modelcontextprotocol/servers/pull/5355) [`135a925`](https://github.com/modelcontextprotocol/servers/commit/135a925c04f37ef3b80d87cddd812c964b485853) [@cliffhall](https://github.com/cliffhall) - `search_files` no longer drops a subdirectory it cannot read without saying so. It still returns the matches it found, and now follows them with a note that the search is incomplete, listing each unreadable directory and its error code. Paths outside the allowed directories are still skipped silently, and an unreadable starting directory is still an error. + ## 2.1.0 ### Minor Changes diff --git a/src/filesystem/README.md b/src/filesystem/README.md index dbb89834a9..09d13eb67d 100644 --- a/src/filesystem/README.md +++ b/src/filesystem/README.md @@ -26,11 +26,37 @@ mcp-server-filesystem /path/to/dir1 /path/to/dir2 This is the only method that works on every protocol version, so it is the one to use. +### Read-only Mode +Add `--read-only` to serve only the tools that cannot change the filesystem: +```bash +mcp-server-filesystem --read-only /path/to/dir1 /path/to/dir2 +``` + +The server then registers only the tools annotated `readOnlyHint: true` in the [Tool annotations](#tool-annotations-mcp-hints) table, the ones that read and list. Every tool not annotated `readOnlyHint: true`, that is every tool that writes (such as `write_file` and `edit_file`), is left out, so a client never lists it and a call to it fails as an unknown tool. A tool added later follows the same rule through its annotation. The server enforces this, whatever the client is configured to allow, and it works the same with `npx` as with Docker. It applies on both protocol eras, and to directories that come from a client's Roots. + +The flag may appear anywhere among the arguments. `--` ends the options, so a directory literally named `--read-only` can be given after it. + +Read-only mode covers every allowed directory. To make only some of them read-only, run two servers: one with `--read-only` for the directories to protect, and one without it for the rest. That separation holds only while the writable server keeps its command-line directories: a 2025-era client that supports Roots replaces them (see Method 2), so it could expose a protected directory to the writable server, unless that server is started with `--confine-roots` (see below). Where that matters, make the protected directories read-only at the operating-system level as well, with file permissions or, with Docker, a bind mount's `ro` flag. + ### Method 2: MCP Roots (Deprecated, 2025-era clients only) A client that connects with a 2025-era protocol version (the `initialize` handshake) and supports [Roots](https://modelcontextprotocol.io/docs/learn/client-concepts#roots) can dynamically update the Allowed directories. Roots notified by Client to Server, completely replace any server-side Allowed directories when provided. +#### Confining Roots to the command-line directories +To keep a client's Roots inside the directories you gave on the command line, add `--confine-roots`: +```bash +mcp-server-filesystem --confine-roots /path/to/dir1 /path/to/dir2 +``` + +The client's Roots can then narrow the command-line directories but never replace or widen them. The server allows the overlap of the two: + +- A root inside a command-line directory is allowed as given, so a client can narrow access to one project. +- A root that contains a command-line directory (for example `/`) is narrowed to that directory. +- A root outside every command-line directory is ignored and logged. If every root is ignored, the server treats the update as one with no valid roots: before any root has been in force the command-line directories stay; once one has, access is revoked until the client exposes a root inside them again. + +This suits an operator who decides what may be shared (the command line) while letting a client say what it wants to work on (its Roots), and it keeps a client from discarding the command-line directories by advertising its working directory. Without command-line directories the flag has no effect, and the server warns at startup. Without the flag, Roots replace the command-line directories as described above. + A Root reached through a symlink is allowed in both its advertised form and its resolved form, as a command-line directory is: a Root of `file:///tmp/project` on macOS allows `/tmp/project` and `/private/tmp/project`. Every requested path is still resolved and checked against the resolved directories, so a symlink inside a Root that points outside it is refused. **Important**: If server starts without command-line arguments AND a 2025-era client doesn't support roots protocol, the server fails at initialization: it logs the reason to stderr, closes the connection and exits with status 1. A client that supports roots but provides none (or none that are valid) keeps the connection, with no allowed directories until it sends `roots/list_changed` with valid roots. @@ -53,7 +79,7 @@ The server's directory access control follows this flow: 3. **Roots Protocol Handling** (2025-era clients that support roots) - **On initialization**: Server requests roots from client via `roots/list` - Client responds with its configured roots - - Server replaces ALL allowed directories with client's roots + - Server replaces ALL allowed directories with client's roots (with `--confine-roots`, with their overlap with the command-line directories) - Tool calls that arrive before the initial roots are loaded wait for them, so they are checked against the client's roots - **On runtime updates**: Client can send `notifications/roots/list_changed` - Server requests updated roots and replaces allowed directories again @@ -68,6 +94,11 @@ The server's directory access control follows this flow: 5. **Access Control** - All filesystem operations are restricted to allowed directories - Use `list_allowed_directories` tool to see current directories + - The server also logs the effective allowed directories to stderr, with where they came from (the command line, or the client's Roots and whether those replaced the command-line directories), at startup and whenever they change: after `initialize`, after each Roots update, and when a withdrawal leaves none. For example: + ``` + Effective allowed directories (from the client's Roots, which replaced the command-line directories): + /Users/username/project + ``` - Server requires at least ONE allowed directory to operate **Note**: The server will only allow operations within directories specified either via `args` or, for a 2025-era client, via Roots. @@ -76,14 +107,25 @@ The server's directory access control follows this flow: ## API +### Resources + +Files inside the allowed directories are also [resources](https://modelcontextprotocol.io/specification/2026-07-28/server/resources), so a client application can read one with `resources/read` and decide itself what to do with it (attach it to the model's context, show it to the user), rather than relaying it through a tool result. + +- **Resource template** `file://{+path}` (name `file`) + - Read any file by its absolute `file://` URI, for example `file:///home/user/project/README.md`. A path with spaces or other reserved characters is percent-encoded, as in any `file://` URL. + - Text files are returned as `text`, with a MIME type from the extension (`text/plain` for an unknown extension whose bytes are UTF-8). Images, audio, PDFs and other binary files are returned as a base64 `blob` with their MIME type (`application/octet-stream` when unknown). + - Confined exactly as the tools are: a URI outside the allowed directories, or a symlink whose target is outside them, is refused with an `Access denied` error (`-32602`). A missing file is "resource not found" (`-32602` with the URI in the error data), and a directory is not a resource. + - `resources/list` returns nothing: the server never enumerates the disk. Find files with `list_directory`, `directory_tree` or `search_files`, then read them by URI. + - Results carry a cache hint of `ttlMs: 0`, `cacheScope: "private"` (2026-07-28), since a file can change at any time. + ### Tools - **read_text_file** - Read complete contents of a file as text - Inputs: - `path` (string) - - `head` (number, optional): First N lines - - `tail` (number, optional): Last N lines + - `head` (non-negative integer, optional): First N lines; `0` returns no lines + - `tail` (non-negative integer, optional): Last N lines; `0` returns no lines - Always treats the file as UTF-8 text regardless of extension - Cannot specify both `head` and `tail` simultaneously @@ -118,6 +160,18 @@ The server's directory access control follows this flow: `get_file_info` reports a file's `hardLinks` count; more than 1 means the file has other names +- **append_file** + - Append text to the end of a file, creating the file if it does not exist + - Inputs: + - `path` (string): File location + - `content` (string): Text to append + - The text is appended exactly as given: no newline is added before or + after it. To build a file line by line, end each `content` with `\n` + - The existing content is never changed. The file is appended to in place, + so it keeps its inode, creation time, hard links and permissions, and a + path that is not a regular file (a directory, say) is refused. The same + hard-link caveat as `write_file` applies + - **edit_file** - Make selective edits using advanced pattern matching and formatting - Features: @@ -135,6 +189,8 @@ The server's directory access control follows this flow: - `newText` (string): Text to replace with - `dryRun` (boolean): Preview changes without applying (default: false) - Returns detailed diff and match information for dry runs, otherwise applies changes + - Matching: `oldText` must match the file's text exactly, except that line endings never matter and, when there is no exact match, a run of whole lines matches if each line is equal once leading and trailing whitespace is ignored. Whitespace inside a line must match; nothing fuzzier is tried + - If any edit has no match, no edits are applied, and the error names the edit, quotes the closest lines of the file with line numbers, and shows the first line that differs (JSON-quoted, so tabs and trailing spaces are visible) - Best Practice: Always use dryRun first to preview changes before applying them - **create_directory** @@ -154,6 +210,7 @@ The server's directory access control follows this flow: - `sortBy` (string, optional): Sort entries by "name" or "size" (default: "name") - Returns detailed listing with file sizes and summary statistics - Shows total files, directories, and combined size + - An entry that cannot be stat'd (unreadable, a dangling symlink, or a symlink leaving the allowed directories) is listed with size `unknown` and left out of the combined size - **move_file** - Move or rename files and directories @@ -168,23 +225,46 @@ The server's directory access control follows this flow: source is left in place. A directory cannot be moved across filesystems and fails with an error; move its files one at a time instead +- **copy_file** + - Copy a file to a new location, without its content passing through the + conversation + - Inputs: + - `source` (string): The file to copy + - `destination` (string): Where to create the copy + - Fails if destination exists: it never overwrites + - Copies one regular file, with its permissions and modification time, into + a newly created destination. A directory, or anything else that is not a + regular file, is refused; create the directory and copy its files one at a + time instead. If the copy fails, the partial destination is removed. The + source is never changed + - **search_files** - - Recursively search for files/directories that match or do not match patterns + - Recursively find files/directories by name (a glob over their paths), not + by content: it never reads or searches inside files - Inputs: - `path` (string): Starting directory - - `pattern` (string): Search pattern + - `pattern` (string): Glob matched, case-sensitively, against each entry's + path relative to `path`: `*.ext` matches entries directly under `path`, + `**/*.ext` or `**/name` matches at any depth - `excludePatterns` (string[]): Exclude any patterns. An excluded entry is skipped before it is resolved or read, so excluding a slow folder keeps the search out of it entirely. - Glob-style pattern matching - Returns full paths to matches - Bounded: see [Slow and very large trees](#slow-and-very-large-trees) + - A subdirectory that cannot be read (no permission, say) is skipped, and the + result says so: after the matches, a note that the search is incomplete + lists each unreadable directory with its error code. Paths outside the + allowed directories are still skipped silently. - **directory_tree** - Get recursive JSON tree structure of directory contents - Inputs: - `path` (string): Starting directory - `excludePatterns` (string[]): Exclude any patterns. Glob formats are supported. + - `depth` (positive integer, optional): How many levels to descend. `1` + lists only the entries directly in `path`, `2` adds their children, and + so on. Omitted, the whole tree is returned - Returns: - JSON array where each entry contains: - `name` (string): File/directory name @@ -192,6 +272,10 @@ The server's directory access control follows this flow: - `children` (array): Present only for directories - Empty array for empty directories - Omitted for files + - Omitted for directories at the last level when `depth` is set: they + are not read, so whether they hold anything is not reported + - A `depth` limit also keeps the walk out of deeper levels entirely, so + entries below it do not count toward the 100,000-entry bound - Output is formatted with 2-space indentation for readability - Bounded: see [Slow and very large trees](#slow-and-very-large-trees) @@ -250,21 +334,24 @@ The mapping for filesystem tools is: | Tool | readOnlyHint | idempotentHint | destructiveHint | Notes | |-----------------------------|--------------|----------------|-----------------|--------------------------------------------------| -| `read_text_file` | `true` | – | – | Pure read | -| `read_media_file` | `true` | – | – | Pure read | -| `read_multiple_files` | `true` | – | – | Pure read | -| `list_directory` | `true` | – | – | Pure read | -| `list_directory_with_sizes` | `true` | – | – | Pure read | -| `directory_tree` | `true` | – | – | Pure read | -| `search_files` | `true` | – | – | Pure read | -| `get_file_info` | `true` | – | – | Pure read | -| `list_allowed_directories` | `true` | – | – | Pure read | +| `read_file` | `true` | `true` | `false` | Deprecated alias of `read_text_file` | +| `read_text_file` | `true` | `true` | `false` | Pure read | +| `read_media_file` | `true` | `true` | `false` | Pure read | +| `read_multiple_files` | `true` | `true` | `false` | Pure read | +| `list_directory` | `true` | `true` | `false` | Pure read | +| `list_directory_with_sizes` | `true` | `true` | `false` | Pure read | +| `directory_tree` | `true` | `true` | `false` | Pure read | +| `search_files` | `true` | `true` | `false` | Pure read | +| `get_file_info` | `true` | `true` | `false` | Pure read | +| `list_allowed_directories` | `true` | `true` | `false` | Pure read | | `create_directory` | `false` | `true` | `false` | Re‑creating the same dir is a no‑op | | `write_file` | `false` | `true` | `true` | Overwrites existing files | +| `append_file` | `false` | `false` | `false` | Adds to a file; repeating it appends again | | `edit_file` | `false` | `false` | `true` | Re‑applying edits can fail or double‑apply | | `move_file` | `false` | `false` | `true` | Deletes source file | +| `copy_file` | `false` | `true` | `false` | Never overwrites; a repeat fails, changing nothing | -> Note: `idempotentHint` and `destructiveHint` are meaningful only when `readOnlyHint` is `false`, as defined by the MCP spec. Every tool also sets `openWorldHint: false` — this server only accesses the local filesystem within its allowed directories, never an open or external world. +> Note: the MCP spec gives `idempotentHint` and `destructiveHint` meaning only when `readOnlyHint` is `false`, but every tool sets them anyway, so a client that reads them without checking `readOnlyHint` first still sees the truth: a read can be repeated safely and changes nothing. Every tool also sets `openWorldHint: false` — this server only accesses the local filesystem within its allowed directories, never an open or external world. ## Requirements diff --git a/src/filesystem/__tests__/allowed-directories.test.ts b/src/filesystem/__tests__/allowed-directories.test.ts new file mode 100644 index 0000000000..59e376e317 --- /dev/null +++ b/src/filesystem/__tests__/allowed-directories.test.ts @@ -0,0 +1,207 @@ +// The effective allowed-directories summary (#4912): one stderr message naming +// the directories in force and where they came from, printed at startup and +// after every change to them. The message itself is pinned through +// describeAllowedDirectories; the protocol tests drive a real Client over the +// in-memory transport and assert the server prints it at each point an +// operator needs it: startup, after initialize (with and without Roots), after +// a Roots update, after a withdrawal, and when the initial roots/list fails. + +import fs from "fs/promises"; +import path from "path"; +import { PassThrough } from "node:stream"; +import { pathToFileURL } from "url"; +import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; +import type { Root } from "@modelcontextprotocol/server"; +import { describeAllowedDirectories } from "../allowed-directories.js"; +import { main } from "../index.js"; +import { + allowedDirectoriesOf, + connect, + makeTempDir, + quietStderr, + type Connected, +} from "./helpers.js"; + +const ROOTS = { roots: { listChanged: true } }; + +let cliDir: string; +let rootDir: string; +let stderr: ReturnType; +let closers: Array<() => Promise>; + +beforeEach(async () => { + stderr = quietStderr(); + cliDir = await makeTempDir("mcp-fs-summary-cli-"); + rootDir = await makeTempDir("mcp-fs-summary-root-"); + closers = []; +}); + +afterEach(async () => { + for (const close of closers.reverse()) await close(); + vi.restoreAllMocks(); + await fs.rm(cliDir, { recursive: true, force: true }); + await fs.rm(rootDir, { recursive: true, force: true }); +}); + +async function connectTracked( + ...args: Parameters +): Promise { + const conn = await connect(...args); + closers.push(conn.close); + return conn; +} + +function rootOf(dir: string): Root { + return { uri: pathToFileURL(dir).href, name: path.basename(dir) }; +} + +describe("describeAllowedDirectories", () => { + it("lists command-line directories one per line", () => { + expect( + describeAllowedDirectories(["/a", "/b,c"], { from: "command-line" }), + ).toBe( + "Effective allowed directories (from the command line):\n /a\n /b,c", + ); + }); + + it("says when the client's Roots replaced the command-line directories", () => { + expect( + describeAllowedDirectories(["/r"], { + from: "roots", + replacedCommandLine: true, + }), + ).toBe( + "Effective allowed directories (from the client's Roots, which replaced the command-line directories):\n /r", + ); + expect( + describeAllowedDirectories(["/r"], { + from: "roots", + replacedCommandLine: false, + }), + ).toBe("Effective allowed directories (from the client's Roots):\n /r"); + }); + + it("says why there are none, and that every path is refused", () => { + expect(describeAllowedDirectories([], { from: "command-line" })).toBe( + "Effective allowed directories: none. No command-line directories and no valid client Roots, so every path is refused", + ); + expect( + describeAllowedDirectories([], { + from: "roots", + replacedCommandLine: true, + }), + ).toBe( + "Effective allowed directories: none. The client's Roots name no valid directory, so every path is refused until it exposes one again", + ); + }); +}); + +describe("the server logs the summary", () => { + it("at startup, for the command-line directories", async () => { + const handle = await main([cliDir], new PassThrough(), new PassThrough()); + closers.push(() => handle.close()); + expect(stderr).toHaveBeenCalledWith( + describeAllowedDirectories([cliDir], { from: "command-line" }), + ); + }); + + it("after initialize, when the client does not support Roots", async () => { + await connectTracked([cliDir]); + await vi.waitFor(() => + expect(stderr).toHaveBeenCalledWith( + describeAllowedDirectories([cliDir], { from: "command-line" }), + ), + ); + }); + + it("after initialize, when the client's Roots replace the command-line directories", async () => { + await connectTracked([cliDir], { + capabilities: ROOTS, + listRoots: () => [rootOf(rootDir)], + }); + await vi.waitFor(() => + expect(stderr).toHaveBeenCalledWith( + describeAllowedDirectories([rootDir], { + from: "roots", + replacedCommandLine: true, + }), + ), + ); + }); + + it("after initialize, when Roots supply the only directories", async () => { + await connectTracked([], { + capabilities: ROOTS, + listRoots: () => [rootOf(rootDir)], + }); + await vi.waitFor(() => + expect(stderr).toHaveBeenCalledWith( + describeAllowedDirectories([rootDir], { + from: "roots", + replacedCommandLine: false, + }), + ), + ); + }); + + it("after initialize, keeping the command-line directories when no root is valid", async () => { + await connectTracked([cliDir], { + capabilities: ROOTS, + listRoots: () => [rootOf(path.join(rootDir, "missing"))], + }); + await vi.waitFor(() => + expect(stderr).toHaveBeenCalledWith( + describeAllowedDirectories([cliDir], { from: "command-line" }), + ), + ); + }); + + it("after initialize, when the initial roots/list fails", async () => { + await connectTracked([cliDir], { + capabilities: ROOTS, + listRoots: () => { + throw new Error("client has no roots today"); + }, + }); + await vi.waitFor(() => + expect(stderr).toHaveBeenCalledWith( + describeAllowedDirectories([cliDir], { from: "command-line" }), + ), + ); + }); + + it("after a Roots update, and after the client withdraws its last root", async () => { + const other = await makeTempDir("mcp-fs-summary-other-"); + closers.push(() => fs.rm(other, { recursive: true, force: true })); + let roots = [rootOf(rootDir)]; + const { client } = await connectTracked([cliDir], { + capabilities: ROOTS, + listRoots: () => roots, + }); + await vi.waitFor(async () => + expect(await allowedDirectoriesOf(client)).toEqual([rootDir]), + ); + + roots = [rootOf(other)]; + await client.sendRootsListChanged(); + await vi.waitFor(() => + expect(stderr).toHaveBeenCalledWith( + describeAllowedDirectories([other], { + from: "roots", + replacedCommandLine: true, + }), + ), + ); + + roots = []; + await client.sendRootsListChanged(); + await vi.waitFor(() => + expect(stderr).toHaveBeenCalledWith( + describeAllowedDirectories([], { + from: "roots", + replacedCommandLine: true, + }), + ), + ); + }); +}); diff --git a/src/filesystem/__tests__/append-file.test.ts b/src/filesystem/__tests__/append-file.test.ts new file mode 100644 index 0000000000..fc2a99d099 --- /dev/null +++ b/src/filesystem/__tests__/append-file.test.ts @@ -0,0 +1,330 @@ +// Protocol-level tests for the append_file tool (#1208), driven through a real +// Client over an in-memory transport. They pin the contract the README states: +// text is appended exactly as given (no newline added), a missing file is +// created, the existing content and the file's identity are kept, and the +// write takes the same symlink-safe path as write_file, so nothing swapped in +// after validation is written through. + +import fs from "fs/promises"; +import fsSync from "fs"; +import path from "path"; +import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; +import type { Client } from "@modelcontextprotocol/client"; +import { + call, + connect, + makeTempDir, + quietStderr, + textOf, + type Connected, +} from "./helpers.js"; + +let dir: string; +let conn: Connected; +let client: Client; + +beforeEach(async () => { + quietStderr(); + dir = await makeTempDir("mcp-fs-append-"); + conn = await connect([dir]); + client = conn.client; +}); + +afterEach(async () => { + vi.restoreAllMocks(); + await conn.close(); + await fs.rm(dir, { recursive: true, force: true }); +}); + +describe("append_file", () => { + it("is listed right after write_file, with truthful annotations", async () => { + const { tools } = await client.listTools(); + const names = tools.map((t) => t.name); + expect(names.indexOf("append_file")).toBe(names.indexOf("write_file") + 1); + const tool = tools.find((t) => t.name === "append_file"); + expect(tool?.title).toBe("Append File"); + expect(tool?.annotations).toEqual({ + readOnlyHint: false, + idempotentHint: false, + destructiveHint: false, + openWorldHint: false, + }); + }); + + it("appends to an existing file exactly as given, adding no newline", async () => { + const file = path.join(dir, "log.txt"); + await fs.writeFile(file, "first"); + const result = await call(client, "append_file", { + path: file, + content: " second", + }); + expect(result.isError).toBeFalsy(); + expect(textOf(result)).toBe(`Successfully appended to ${file}`); + expect(result.structuredContent).toEqual({ content: textOf(result) }); + expect(await fs.readFile(file, "utf-8")).toBe("first second"); + }); + + it("builds a file up line by line when each append ends in a newline", async () => { + const file = path.join(dir, "lines.txt"); + for (const line of ["one\n", "two\n", "three\n"]) { + const result = await call(client, "append_file", { + path: file, + content: line, + }); + expect(result.isError).toBeFalsy(); + } + expect(await fs.readFile(file, "utf-8")).toBe("one\ntwo\nthree\n"); + }); + + it("keeps CRLF and other content byte for byte", async () => { + const file = path.join(dir, "crlf.txt"); + await fs.writeFile(file, "a\r\n"); + await call(client, "append_file", { path: file, content: "b\r\nc" }); + expect(await fs.readFile(file, "utf-8")).toBe("a\r\nb\r\nc"); + }); + + it("creates a file that does not exist yet", async () => { + const file = path.join(dir, "new.txt"); + const result = await call(client, "append_file", { + path: file, + content: "hello", + }); + expect(result.isError).toBeFalsy(); + expect(await fs.readFile(file, "utf-8")).toBe("hello"); + }); + + it("keeps an append that lands on a new file before the creating write", async () => { + const file = path.join(dir, "race.txt"); + // The creating call's write is held back until another caller's append + // has reached the new file; without O_APPEND the held write would land at + // offset 0 over it. + const realWriteFile = fsSync.writeFile; + let raced = false; + vi.spyOn(fsSync, "writeFile").mockImplementation((( + target: number, + ...rest: unknown[] + ) => { + if (!raced && typeof target === "number") { + raced = true; + fsSync.appendFileSync(file, "SECOND"); + } + return (realWriteFile as (...args: unknown[]) => void)(target, ...rest); + }) as typeof fsSync.writeFile); + const result = await call(client, "append_file", { + path: file, + content: "FIRST", + }); + expect(result.isError).toBeFalsy(); + expect(raced).toBe(true); + expect(await fs.readFile(file, "utf-8")).toBe("SECONDFIRST"); + }); + + it("leaves a file unchanged when the content is empty", async () => { + const file = path.join(dir, "same.txt"); + await fs.writeFile(file, "kept"); + const result = await call(client, "append_file", { + path: file, + content: "", + }); + expect(result.isError).toBeFalsy(); + expect(await fs.readFile(file, "utf-8")).toBe("kept"); + }); + + it("appends through the existing file, keeping its inode and hard links", async () => { + const file = path.join(dir, "linked.txt"); + const link = path.join(dir, "other-name.txt"); + await fs.writeFile(file, "base"); + await fs.link(file, link); + const before = await fs.stat(file, { bigint: true }); + await call(client, "append_file", { path: file, content: "+more" }); + const after = await fs.stat(file, { bigint: true }); + expect(after.ino).toBe(before.ino); + expect(after.nlink).toBe(2n); + expect(await fs.readFile(link, "utf-8")).toBe("base+more"); + }); + + it.skipIf(process.platform === "win32" || process.getuid?.() === 0)( + "appends to a write-only file, which it never reads", + async () => { + const file = path.join(dir, "write-only.log"); + await fs.writeFile(file, "a"); + await fs.chmod(file, 0o222); + try { + const result = await call(client, "append_file", { + path: file, + content: "b", + }); + expect(result.isError, textOf(result)).toBeFalsy(); + } finally { + await fs.chmod(file, 0o644); + } + expect(await fs.readFile(file, "utf-8")).toBe("ab"); + }, + ); + + it.skipIf(process.platform === "win32")( + "refuses a FIFO without blocking", + async () => { + const fifo = path.join(dir, "pipe"); + const { execFileSync } = await import("child_process"); + execFileSync("mkfifo", [fifo]); + const result = await call(client, "append_file", { + path: fifo, + content: "x", + }); + expect(result.isError).toBe(true); + expect(textOf(result)).toBe( + `Refusing to append: not a regular file: ${fifo}`, + ); + }, + ); + + it("refuses a path outside the allowed directories", async () => { + const outside = await makeTempDir("mcp-fs-append-outside-"); + try { + const target = path.join(outside, "x.txt"); + const result = await call(client, "append_file", { + path: target, + content: "nope", + }); + expect(result.isError).toBe(true); + expect(textOf(result)).toMatch(/^Access denied/); + expect(fsSync.existsSync(target)).toBe(false); + } finally { + await fs.rm(outside, { recursive: true, force: true }); + } + }); + + it("refuses a directory", async () => { + const sub = path.join(dir, "sub"); + await fs.mkdir(sub); + const result = await call(client, "append_file", { + path: sub, + content: "x", + }); + expect(result.isError).toBe(true); + expect(textOf(result)).toBe( + `Refusing to append: not a regular file: ${sub}`, + ); + }); + + it("fails, creating nothing, when the parent directory does not exist", async () => { + const result = await call(client, "append_file", { + path: path.join(dir, "missing", "x.txt"), + content: "x", + }); + expect(result.isError).toBe(true); + expect(textOf(result)).toMatch(/ENOENT/); + expect(fsSync.existsSync(path.join(dir, "missing"))).toBe(false); + }); + + it("refuses when the file at the path is not the one it opened", async () => { + const file = path.join(dir, "target.txt"); + const decoy = path.join(dir, "decoy.txt"); + await fs.writeFile(file, "original"); + await fs.writeFile(decoy, "decoy"); + // The lstat that names the file to append to describes another file, as + // if the target had been replaced between that lstat and the open. + const realLstat = fs.lstat.bind(fs); + vi.spyOn(fs, "lstat").mockImplementation(((p: string, opts?: unknown) => + realLstat(p === file ? decoy : p, opts as never)) as typeof fs.lstat); + const result = await call(client, "append_file", { + path: file, + content: "pwned", + }); + expect(result.isError).toBe(true); + expect(textOf(result)).toBe( + `Refusing to append: ${file} was replaced while it was being opened`, + ); + expect(await fs.readFile(file, "utf-8")).toBe("original"); + expect(await fs.readFile(decoy, "utf-8")).toBe("decoy"); + }); + + it.skipIf(process.platform === "win32" || process.getuid?.() === 0)( + "reports a file it cannot create", + async () => { + const locked = path.join(dir, "locked"); + await fs.mkdir(locked); + await fs.chmod(locked, 0o500); + try { + const result = await call(client, "append_file", { + path: path.join(locked, "x.txt"), + content: "x", + }); + expect(result.isError).toBe(true); + expect(textOf(result)).toMatch(/EACCES/); + } finally { + await fs.chmod(locked, 0o700); + } + }, + ); +}); + +// The TOCTOU window, as server-tools.test.ts stages it for write_file: a +// directory on the path is swapped for a symlink to a directory outside the +// allowed one right after validation resolved the path. append_file must not +// write through it, whether it would create the file or append to one. +describe.skipIf(process.platform === "win32")( + "append_file with a directory on the path swapped for a symlink after validation", + () => { + let outside: string; + let flip: string; + let flipReal: string; + + beforeEach(async () => { + outside = await makeTempDir("mcp-fs-append-outside-"); + flip = path.join(dir, "flip"); + flipReal = path.join(dir, "flip-real"); + await fs.mkdir(flip); + }); + + afterEach(async () => { + await fs.rm(outside, { recursive: true, force: true }); + }); + + /** Swap `flip` for a symlink to `outside` once validation realpath'd `target`. */ + const swapAfterRealpath = (target: string) => { + const realRealpath = fs.realpath.bind(fs); + let swapped = false; + vi.spyOn(fs, "realpath").mockImplementation(async (p, opts) => { + const resolved = await realRealpath(p, opts); + if (!swapped && p === target) { + swapped = true; + fsSync.renameSync(flip, flipReal); + fsSync.symlinkSync(outside, flip); + } + return resolved; + }); + }; + + it("does not create a file through it", async () => { + swapAfterRealpath(flip); + const result = await call(client, "append_file", { + path: path.join(flip, "created.txt"), + content: "pwned", + }); + expect(result.isError).toBe(true); + expect(textOf(result)).toMatch(/^Access denied/); + expect(await fs.readdir(outside)).toEqual([]); + }); + + it("does not append to a file through it", async () => { + const victim = path.join(flip, "victim.txt"); + await fs.writeFile(victim, "original"); + await fs.writeFile(path.join(outside, "victim.txt"), "SECRET"); + swapAfterRealpath(victim); + const result = await call(client, "append_file", { + path: victim, + content: "pwned", + }); + expect(result.isError).toBe(true); + expect(textOf(result)).toMatch(/^Access denied/); + expect(await fs.readFile(path.join(outside, "victim.txt"), "utf-8")).toBe( + "SECRET", + ); + expect( + await fs.readFile(path.join(flipReal, "victim.txt"), "utf-8"), + ).toBe("original"); + }); + }, +); diff --git a/src/filesystem/__tests__/confine-roots.test.ts b/src/filesystem/__tests__/confine-roots.test.ts new file mode 100644 index 0000000000..963e484cfd --- /dev/null +++ b/src/filesystem/__tests__/confine-roots.test.ts @@ -0,0 +1,284 @@ +// --confine-roots (#3133): a 2025-era client's Roots may narrow the operator's +// command-line directories but never replace or widen them. The overlap rule +// is pinned on confineRootsToDirectories directly; the protocol tests drive a +// real Client that answers roots/list over the in-memory transport and assert +// what the server then allows, logs and refuses, including the withdrawal and +// symlink cases the unconfined flow already guards. The 2026-07-28 era never +// asks for Roots, so there the flag changes nothing, which main() pins. + +import fs from "fs/promises"; +import path from "path"; +import { PassThrough } from "node:stream"; +import { pathToFileURL } from "url"; +import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; +import { Client, InMemoryTransport } from "@modelcontextprotocol/client"; +import type { Root } from "@modelcontextprotocol/server"; +import { + confineRootsToDirectories, + describeAllowedDirectories, +} from "../allowed-directories.js"; +import { main, parseArgs } from "../index.js"; +import { createServer } from "../server.js"; +import { + allowedDirectoriesOf, + call, + makeTempDir, + quietStderr, + textOf, +} from "./helpers.js"; + +let base: string; +let outside: string; +let stderr: ReturnType; +let closers: Array<() => Promise>; + +beforeEach(async () => { + stderr = quietStderr(); + base = await makeTempDir("mcp-fs-confine-base-"); + outside = await makeTempDir("mcp-fs-confine-outside-"); + await fs.mkdir(path.join(base, "inner")); + await fs.writeFile(path.join(base, "inner", "in.txt"), "inside"); + await fs.writeFile(path.join(base, "top.txt"), "top"); + await fs.writeFile(path.join(outside, "out.txt"), "outside"); + closers = []; +}); + +afterEach(async () => { + for (const close of closers.reverse()) await close(); + vi.restoreAllMocks(); + await fs.rm(base, { recursive: true, force: true }); + await fs.rm(outside, { recursive: true, force: true }); +}); + +function rootOf(dir: string): Root { + return { uri: pathToFileURL(dir).href, name: path.basename(dir) }; +} + +/** + * A 2025-era client that supports Roots and answers roots/list with + * `roots()`, connected to a server for `directories` with confinement on. + */ +async function connect( + directories: string[], + roots: () => Root[], + confineRoots = true, +): Promise { + const server = createServer(directories, { confineRoots }); + const client = new Client( + { name: "fs-confine-test", version: "0.0.0" }, + { capabilities: { roots: { listChanged: true } } }, + ); + client.setRequestHandler("roots/list", async () => ({ roots: roots() })); + const [clientTransport, serverTransport] = + InMemoryTransport.createLinkedPair(); + await Promise.all([ + server.connect(serverTransport), + client.connect(clientTransport), + ]); + closers.push(async () => { + await client.close(); + await server.close(); + }); + return client; +} + +describe("confineRootsToDirectories", () => { + it("keeps a root inside a command-line directory", () => { + expect(confineRootsToDirectories(["/b/x"], ["/b"])).toEqual({ + confined: ["/b/x"], + ignored: [], + }); + }); + + it("narrows a root that contains command-line directories to them", () => { + expect(confineRootsToDirectories(["/"], ["/b", "/c/d"])).toEqual({ + confined: ["/b", "/c/d"], + ignored: [], + }); + }); + + it("ignores a root that overlaps no command-line directory", () => { + expect(confineRootsToDirectories(["/elsewhere", "/b"], ["/b"])).toEqual({ + confined: ["/b"], + ignored: ["/elsewhere"], + }); + }); + + it("does not treat a sibling with a common prefix as inside", () => { + expect(confineRootsToDirectories(["/base-other"], ["/base"])).toEqual({ + confined: [], + ignored: ["/base-other"], + }); + }); + + it("lists each directory once", () => { + expect(confineRootsToDirectories(["/b", "/", "/b"], ["/b"])).toEqual({ + confined: ["/b"], + ignored: [], + }); + }); +}); + +describe("with --confine-roots on 2025-11-25", () => { + it("narrows the allowed directories to a root inside them", async () => { + const inner = path.join(base, "inner"); + const client = await connect([base], () => [rootOf(inner)]); + await vi.waitFor(async () => + expect(await allowedDirectoriesOf(client)).toEqual([inner]), + ); + expect(stderr).toHaveBeenCalledWith( + describeAllowedDirectories([inner], { + from: "roots", + replacedCommandLine: true, + confined: true, + }), + ); + const top = await call(client, "read_text_file", { + path: path.join(base, "top.txt"), + }); + expect(textOf(top)).toMatch(/^Access denied/); + }); + + it("ignores a root outside them and keeps the command-line directories", async () => { + const client = await connect([base], () => [rootOf(outside)]); + await vi.waitFor(() => + expect(stderr).toHaveBeenCalledWith( + `Ignored client roots outside the command-line directories (--confine-roots): ${outside}`, + ), + ); + expect(await allowedDirectoriesOf(client)).toEqual([base]); + const out = await call(client, "read_text_file", { + path: path.join(outside, "out.txt"), + }); + expect(textOf(out)).toMatch(/^Access denied/); + const top = await call(client, "read_text_file", { + path: path.join(base, "top.txt"), + }); + expect(textOf(top)).toBe("top"); + }); + + it("narrows a root that contains a command-line directory to that directory", async () => { + const client = await connect([base], () => [ + rootOf(path.dirname(base)), + rootOf(outside), + ]); + await vi.waitFor(async () => + expect(await allowedDirectoriesOf(client)).toEqual([base]), + ); + const out = await call(client, "read_text_file", { + path: path.join(outside, "out.txt"), + }); + expect(textOf(out)).toMatch(/^Access denied/); + }); + + it("refuses a symlink inside the directories that points outside them", async () => { + const link = path.join(base, "link"); + await fs.symlink(outside, link); + const client = await connect([base], () => [rootOf(link)]); + await vi.waitFor(() => + expect(stderr).toHaveBeenCalledWith( + `Ignored client roots outside the command-line directories (--confine-roots): ${outside}`, + ), + ); + const out = await call(client, "read_text_file", { + path: path.join(link, "out.txt"), + }); + expect(textOf(out)).toMatch(/^Access denied/); + }); + + it("revokes access once roots were in force and every new root is outside", async () => { + const inner = path.join(base, "inner"); + let roots = [rootOf(inner)]; + const client = await connect([base], () => roots); + await vi.waitFor(async () => + expect(await allowedDirectoriesOf(client)).toEqual([inner]), + ); + roots = [rootOf(outside)]; + await client.sendRootsListChanged(); + await vi.waitFor(async () => + expect(await allowedDirectoriesOf(client)).toEqual([]), + ); + const out = await call(client, "read_text_file", { + path: path.join(outside, "out.txt"), + }); + expect(textOf(out)).toMatch(/^Access denied/); + }); + + it("lets the roots set the directories when none were given on the command line", async () => { + const client = await connect([], () => [rootOf(outside)]); + await vi.waitFor(async () => + expect(await allowedDirectoriesOf(client)).toEqual([outside]), + ); + }); +}); + +describe("without --confine-roots", () => { + it("still lets a root outside the command-line directories replace them", async () => { + const client = await connect([base], () => [rootOf(outside)], false); + await vi.waitFor(async () => + expect(await allowedDirectoriesOf(client)).toEqual([outside]), + ); + }); +}); + +describe("the command line", () => { + it("parses --confine-roots anywhere before --", () => { + expect(parseArgs(["/a", "--confine-roots"])).toEqual({ + directories: ["/a"], + readOnly: false, + confineRoots: true, + }); + expect(parseArgs(["--", "--confine-roots"])).toEqual({ + directories: ["--confine-roots"], + readOnly: false, + confineRoots: false, + }); + }); + + it("warns that --confine-roots does nothing without directories", async () => { + const handle = await main( + ["--confine-roots"], + new PassThrough(), + new PassThrough(), + ); + closers.push(() => handle.close()); + expect(stderr).toHaveBeenCalledWith( + "Warning: --confine-roots has no effect without command-line directories; a client's Roots will set the allowed directories", + ); + }); + + it("serves the command-line directories on 2026-07-28, as without the flag", async () => { + const stdin = new PassThrough(); + const stdout = new PassThrough(); + stdout.setEncoding("utf8"); + const handle = await main(["--confine-roots", base], stdin, stdout); + closers.push(() => handle.close()); + const line = new Promise((resolve) => { + let buffered = ""; + stdout.on("data", (chunk: string) => { + buffered += chunk; + const newline = buffered.indexOf("\n"); + if (newline !== -1) resolve(buffered.slice(0, newline)); + }); + }); + stdin.write( + `${JSON.stringify({ + jsonrpc: "2.0", + id: 1, + method: "tools/call", + params: { + _meta: { + "io.modelcontextprotocol/protocolVersion": "2026-07-28", + "io.modelcontextprotocol/clientCapabilities": { roots: {} }, + }, + name: "list_allowed_directories", + arguments: {}, + }, + })}\n`, + ); + const { result } = JSON.parse(await line) as { + result: { content: Array<{ text: string }> }; + }; + expect(result.content[0].text).toBe(`Allowed directories:\n${base}`); + }); +}); diff --git a/src/filesystem/__tests__/copy-file.test.ts b/src/filesystem/__tests__/copy-file.test.ts new file mode 100644 index 0000000000..e2e587c8cd --- /dev/null +++ b/src/filesystem/__tests__/copy-file.test.ts @@ -0,0 +1,325 @@ +// Protocol-level tests for the copy_file tool (#519), driven through a real +// Client over an in-memory transport. They pin the contract the README states: +// one regular file is copied with its permissions and modification time, the +// source is left as it was, an existing destination is never overwritten, +// directories and anything else that is not a regular file are refused, and +// both paths go through the allowed-directory checks. + +import fs from "fs/promises"; +import fsSync from "fs"; +import path from "path"; +import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; +import type { Client } from "@modelcontextprotocol/client"; +import { + call, + connect, + makeTempDir, + quietStderr, + textOf, + type Connected, +} from "./helpers.js"; + +let dir: string; +let conn: Connected; +let client: Client; + +beforeEach(async () => { + quietStderr(); + dir = await makeTempDir("mcp-fs-copy-"); + conn = await connect([dir]); + client = conn.client; +}); + +afterEach(async () => { + vi.restoreAllMocks(); + await conn.close(); + await fs.rm(dir, { recursive: true, force: true }); +}); + +describe("copy_file", () => { + it("is listed right after move_file, with truthful annotations", async () => { + const { tools } = await client.listTools(); + const names = tools.map((t) => t.name); + expect(names.indexOf("copy_file")).toBe(names.indexOf("move_file") + 1); + const tool = tools.find((t) => t.name === "copy_file"); + expect(tool?.title).toBe("Copy File"); + expect(tool?.annotations).toEqual({ + readOnlyHint: false, + idempotentHint: true, + destructiveHint: false, + openWorldHint: false, + }); + }); + + it("copies a file, leaving the source as it was", async () => { + const source = path.join(dir, "a.txt"); + const destination = path.join(dir, "b.txt"); + await fs.writeFile(source, "alpha\r\nbeta"); + const result = await call(client, "copy_file", { source, destination }); + expect(result.isError).toBeFalsy(); + expect(textOf(result)).toBe( + `Successfully copied ${source} to ${destination}`, + ); + expect(result.structuredContent).toEqual({ content: textOf(result) }); + expect(await fs.readFile(destination, "utf-8")).toBe("alpha\r\nbeta"); + expect(await fs.readFile(source, "utf-8")).toBe("alpha\r\nbeta"); + const [sourceStat, copyStat] = await Promise.all([ + fs.stat(source, { bigint: true }), + fs.stat(destination, { bigint: true }), + ]); + expect(copyStat.ino).not.toBe(sourceStat.ino); + }); + + it("copies binary content byte for byte, into another directory", async () => { + const bytes = Buffer.from( + Array.from({ length: 200_000 }, (_, i) => (i * 7) % 256), + ); + const source = path.join(dir, "blob.bin"); + await fs.mkdir(path.join(dir, "sub")); + const destination = path.join(dir, "sub", "blob-copy.bin"); + await fs.writeFile(source, bytes); + const result = await call(client, "copy_file", { source, destination }); + expect(result.isError).toBeFalsy(); + expect((await fs.readFile(destination)).equals(bytes)).toBe(true); + }); + + it.skipIf(process.platform === "win32")( + "keeps the source's permissions and modification time", + async () => { + const source = path.join(dir, "script.sh"); + const destination = path.join(dir, "script-copy.sh"); + await fs.writeFile(source, "#!/bin/sh\n"); + await fs.chmod(source, 0o750); + const mtime = new Date("2001-02-03T04:05:06Z"); + await fs.utimes(source, mtime, mtime); + const result = await call(client, "copy_file", { source, destination }); + expect(result.isError).toBeFalsy(); + const copied = await fs.stat(destination); + expect(copied.mode & 0o777).toBe(0o750); + expect(copied.mtime.getTime()).toBe(mtime.getTime()); + }, + ); + + it("refuses an existing destination and leaves it untouched", async () => { + const source = path.join(dir, "a.txt"); + const destination = path.join(dir, "b.txt"); + await fs.writeFile(source, "new"); + await fs.writeFile(destination, "keep me"); + const result = await call(client, "copy_file", { source, destination }); + expect(result.isError).toBe(true); + expect(textOf(result)).toBe(`Destination already exists: ${destination}`); + expect(await fs.readFile(destination, "utf-8")).toBe("keep me"); + }); + + it("refuses a copy onto itself", async () => { + const source = path.join(dir, "a.txt"); + await fs.writeFile(source, "same"); + const result = await call(client, "copy_file", { + source, + destination: source, + }); + expect(result.isError).toBe(true); + expect(textOf(result)).toBe(`Destination already exists: ${source}`); + expect(await fs.readFile(source, "utf-8")).toBe("same"); + }); + + it.skipIf(process.platform === "win32")( + "refuses a destination occupied by a dangling symlink, without following it", + async () => { + const source = path.join(dir, "a.txt"); + const destination = path.join(dir, "link.txt"); + const target = path.join(dir, "would-be-created.txt"); + await fs.writeFile(source, "x"); + await fs.symlink(target, destination); + const result = await call(client, "copy_file", { source, destination }); + expect(result.isError).toBe(true); + expect(fsSync.existsSync(target)).toBe(false); + }, + ); + + it("refuses a directory", async () => { + const source = path.join(dir, "folder"); + await fs.mkdir(source); + const result = await call(client, "copy_file", { + source, + destination: path.join(dir, "folder-copy"), + }); + expect(result.isError).toBe(true); + expect(textOf(result)).toMatch(/^Cannot copy a directory: /); + expect(fsSync.existsSync(path.join(dir, "folder-copy"))).toBe(false); + }); + + it.skipIf(process.platform === "win32")( + "refuses a source that is not a regular file", + async () => { + const source = path.join(dir, "fifo"); + const { execFileSync } = await import("child_process"); + execFileSync("mkfifo", [source]); + const result = await call(client, "copy_file", { + source, + destination: path.join(dir, "fifo-copy"), + }); + expect(result.isError).toBe(true); + expect(textOf(result)).toBe(`Cannot copy: not a regular file: ${source}`); + }, + ); + + it("removes the copy when closing the source fails after it", async () => { + const source = path.join(dir, "a.txt"); + const destination = path.join(dir, "b.txt"); + await fs.writeFile(source, "data"); + // The copy closes the destination first, then the source: fail the + // second close, after really closing the descriptor. + const realClose = fsSync.close; + let closes = 0; + vi.spyOn(fsSync, "close").mockImplementation((( + fd: number, + callback: (error: NodeJS.ErrnoException | null) => void, + ) => + realClose(fd, (error) => + ++closes === 2 + ? callback( + Object.assign(new Error("EIO: i/o error, close"), { + code: "EIO", + }), + ) + : callback(error), + )) as typeof fsSync.close); + const result = await call(client, "copy_file", { source, destination }); + expect(closes).toBe(2); + expect(result.isError).toBe(true); + expect(textOf(result)).toBe("EIO: i/o error, close"); + expect(fsSync.existsSync(destination)).toBe(false); + expect(await fs.readFile(source, "utf-8")).toBe("data"); + }); + + it("reports a missing source", async () => { + const result = await call(client, "copy_file", { + source: path.join(dir, "missing.txt"), + destination: path.join(dir, "copy.txt"), + }); + expect(result.isError).toBe(true); + expect(textOf(result)).toMatch(/ENOENT/); + expect(fsSync.existsSync(path.join(dir, "copy.txt"))).toBe(false); + }); + + it("refuses a source or destination outside the allowed directories", async () => { + const outside = await makeTempDir("mcp-fs-copy-outside-"); + try { + const secret = path.join(outside, "secret.txt"); + await fs.writeFile(secret, "SECRET"); + const inside = path.join(dir, "a.txt"); + await fs.writeFile(inside, "a"); + + const fromOutside = await call(client, "copy_file", { + source: secret, + destination: path.join(dir, "stolen.txt"), + }); + expect(fromOutside.isError).toBe(true); + expect(textOf(fromOutside)).toMatch(/^Access denied/); + expect(fsSync.existsSync(path.join(dir, "stolen.txt"))).toBe(false); + + const toOutside = await call(client, "copy_file", { + source: inside, + destination: path.join(outside, "leaked.txt"), + }); + expect(toOutside.isError).toBe(true); + expect(textOf(toOutside)).toMatch(/^Access denied/); + expect(fsSync.existsSync(path.join(outside, "leaked.txt"))).toBe(false); + } finally { + await fs.rm(outside, { recursive: true, force: true }); + } + }); + + it.skipIf(process.platform === "win32")( + "refuses a symlink inside that points outside", + async () => { + const outside = await makeTempDir("mcp-fs-copy-outside-"); + try { + await fs.writeFile(path.join(outside, "secret.txt"), "SECRET"); + const link = path.join(dir, "link.txt"); + await fs.symlink(path.join(outside, "secret.txt"), link); + const result = await call(client, "copy_file", { + source: link, + destination: path.join(dir, "stolen.txt"), + }); + expect(result.isError).toBe(true); + expect(textOf(result)).toMatch(/^Access denied/); + expect(fsSync.existsSync(path.join(dir, "stolen.txt"))).toBe(false); + } finally { + await fs.rm(outside, { recursive: true, force: true }); + } + }, + ); + + it("reports a destination whose state it cannot read", async () => { + const source = path.join(dir, "a.txt"); + await fs.writeFile(source, "x"); + const destination = path.join(dir, "copy.txt"); + // Validation lstats the missing destination first; the existence check + // after it is the call that fails here, as it would for a directory made + // unreadable in between. + const realLstat = fs.lstat.bind(fs); + let seen = 0; + vi.spyOn(fs, "lstat").mockImplementation(((p: string, opts?: unknown) => { + if (p === destination && ++seen === 2) { + return Promise.reject( + Object.assign(new Error("EACCES: permission denied, lstat"), { + code: "EACCES", + }), + ); + } + return realLstat(p, opts as never); + }) as typeof fs.lstat); + const result = await call(client, "copy_file", { source, destination }); + expect(seen).toBe(2); + expect(result.isError).toBe(true); + expect(textOf(result)).toBe("EACCES: permission denied, lstat"); + expect(fsSync.existsSync(destination)).toBe(false); + }); +}); + +// The TOCTOU window, as server-tools.test.ts stages it for move_file: the +// destination's directory is swapped for a symlink to a directory outside the +// allowed one right after validation resolved it. copy_file must not create +// its copy through it. +describe.skipIf(process.platform === "win32")( + "copy_file with the destination's directory swapped for a symlink after validation", + () => { + let outside: string; + let flip: string; + + beforeEach(async () => { + outside = await makeTempDir("mcp-fs-copy-outside-"); + flip = path.join(dir, "flip"); + await fs.mkdir(flip); + }); + + afterEach(async () => { + await fs.rm(outside, { recursive: true, force: true }); + }); + + it("does not create the copy through it", async () => { + const source = path.join(dir, "a.txt"); + await fs.writeFile(source, "data"); + const realRealpath = fs.realpath.bind(fs); + let swapped = false; + vi.spyOn(fs, "realpath").mockImplementation(async (p, opts) => { + const resolved = await realRealpath(p, opts); + if (!swapped && p === flip) { + swapped = true; + fsSync.renameSync(flip, path.join(dir, "flip-real")); + fsSync.symlinkSync(outside, flip); + } + return resolved; + }); + const result = await call(client, "copy_file", { + source, + destination: path.join(flip, "copy.txt"), + }); + expect(result.isError).toBe(true); + expect(textOf(result)).toMatch(/^Access denied/); + expect(await fs.readdir(outside)).toEqual([]); + }); + }, +); diff --git a/src/filesystem/__tests__/directory-tree-depth.test.ts b/src/filesystem/__tests__/directory-tree-depth.test.ts new file mode 100644 index 0000000000..1cb9ed92a3 --- /dev/null +++ b/src/filesystem/__tests__/directory-tree-depth.test.ts @@ -0,0 +1,174 @@ +// Protocol-level tests for directory_tree's optional `depth` (#1077), driven +// through a real Client over an in-memory transport. Omitted, the tree is the +// whole tree, as before; set, the walk stops at that level, directories there +// are listed without a children array, and nothing below them is read, so a +// shallow view of a huge tree costs neither the readdirs nor the entry budget +// of the bounded walk (#4162). + +import fs from "fs/promises"; +import path from "path"; +import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; +import type { Client } from "@modelcontextprotocol/client"; +import { + call, + connect, + makeTempDir, + quietStderr, + textOf, + type Connected, +} from "./helpers.js"; + +interface TreeEntry { + name: string; + type: "file" | "directory"; + children?: TreeEntry[]; +} + +/** Sort a tree by name at every level; readdir order is the OS's. */ +function sortTree(entries: TreeEntry[]): TreeEntry[] { + return [...entries] + .sort((a, b) => a.name.localeCompare(b.name)) + .map((entry) => + entry.children ? { ...entry, children: sortTree(entry.children) } : entry, + ); +} + +let dir: string; +let conn: Connected; +let client: Client; + +// dir/ +// top.txt +// empty/ +// a/ +// a.txt +// b/ +// b.txt +// c/ +// c.txt +beforeEach(async () => { + quietStderr(); + dir = await makeTempDir("mcp-fs-tree-depth-"); + await fs.mkdir(path.join(dir, "a", "b", "c"), { recursive: true }); + await fs.mkdir(path.join(dir, "empty")); + await fs.writeFile(path.join(dir, "top.txt"), ""); + await fs.writeFile(path.join(dir, "a", "a.txt"), ""); + await fs.writeFile(path.join(dir, "a", "b", "b.txt"), ""); + await fs.writeFile(path.join(dir, "a", "b", "c", "c.txt"), ""); + conn = await connect([dir]); + client = conn.client; +}); + +afterEach(async () => { + vi.restoreAllMocks(); + await conn.close(); + await fs.rm(dir, { recursive: true, force: true }); +}); + +async function tree(args: Record): Promise { + const result = await call(client, "directory_tree", { path: dir, ...args }); + expect(result.isError, textOf(result)).toBeFalsy(); + return sortTree(JSON.parse(textOf(result)) as TreeEntry[]); +} + +const FULL: TreeEntry[] = [ + { + name: "a", + type: "directory", + children: [ + { name: "a.txt", type: "file" }, + { + name: "b", + type: "directory", + children: [ + { name: "b.txt", type: "file" }, + { + name: "c", + type: "directory", + children: [{ name: "c.txt", type: "file" }], + }, + ], + }, + ], + }, + { name: "empty", type: "directory", children: [] }, + { name: "top.txt", type: "file" }, +]; + +describe("directory_tree depth", () => { + it("returns the whole tree when depth is omitted", async () => { + expect(await tree({})).toEqual(FULL); + }); + + it("lists only the entries directly in path at depth 1, directories without children", async () => { + expect(await tree({ depth: 1 })).toEqual([ + { name: "a", type: "directory" }, + { name: "empty", type: "directory" }, + { name: "top.txt", type: "file" }, + ]); + }); + + it("descends one more level at depth 2", async () => { + expect(await tree({ depth: 2 })).toEqual([ + { + name: "a", + type: "directory", + children: [ + { name: "a.txt", type: "file" }, + { name: "b", type: "directory" }, + ], + }, + { name: "empty", type: "directory", children: [] }, + { name: "top.txt", type: "file" }, + ]); + }); + + it("returns the whole tree when depth reaches past its bottom", async () => { + expect(await tree({ depth: 4 })).toEqual(FULL); + expect(await tree({ depth: 100 })).toEqual(FULL); + }); + + it("applies excludePatterns within the depth", async () => { + expect(await tree({ depth: 2, excludePatterns: ["a.txt"] })).toEqual([ + { + name: "a", + type: "directory", + children: [{ name: "b", type: "directory" }], + }, + { name: "empty", type: "directory", children: [] }, + { name: "top.txt", type: "file" }, + ]); + }); + + it("never reads a directory below the depth", async () => { + const readdir = vi.spyOn(fs, "readdir"); + await tree({ depth: 2 }); + const read = readdir.mock.calls.map(([target]) => String(target)); + expect(read).toContain(path.join(dir, "a")); + expect(read).not.toContain(path.join(dir, "a", "b")); + expect(read).not.toContain(path.join(dir, "a", "b", "c")); + }); + + it.each([0, -1, 1.5, "2"])("rejects depth %j", async (depth) => { + const result = await call(client, "directory_tree", { path: dir, depth }); + expect(result.isError).toBe(true); + expect(textOf(result)).toMatch( + /^Input validation error: Invalid arguments for tool directory_tree/, + ); + }); + + it("advertises depth as an optional positive integer", async () => { + const { tools } = await client.listTools(); + const schema = tools.find((t) => t.name === "directory_tree") + ?.inputSchema as { + properties: Record>; + required?: string[]; + }; + expect(schema.properties.depth).toMatchObject({ + type: "integer", + exclusiveMinimum: 0, + }); + expect(schema.properties.depth.description).toMatch(/^How many levels/); + expect(schema.required).toEqual(["path"]); + }); +}); diff --git a/src/filesystem/__tests__/edit-file-mismatch.test.ts b/src/filesystem/__tests__/edit-file-mismatch.test.ts new file mode 100644 index 0000000000..25ee424a2c --- /dev/null +++ b/src/filesystem/__tests__/edit-file-mismatch.test.ts @@ -0,0 +1,228 @@ +// The edit_file no-match error (#2034). An edit whose oldText matches nowhere +// used to fail with only "Could not find exact match", leaving a model to +// guess at an invisible whitespace difference. The error now names the edit, +// states what the matcher tolerates, and quotes the closest lines of the file +// with line numbers and the first differing line JSON-quoted. These tests pin +// that report, and that it stays a report: a near match is never applied. + +import fs from "fs/promises"; +import path from "path"; +import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; +import { describeEditMismatch } from "../lib.js"; +import { + call, + connect, + makeTempDir, + quietStderr, + textOf, + type Connected, +} from "./helpers.js"; + +describe("edit_file no-match error, over the protocol", () => { + let dir: string; + let conn: Connected; + let file: string; + const original = + "function greet() {\n" + + " const name = 'world';\n" + + " console.log('hello', name);\n" + + " return name;\n" + + "}\n"; + + beforeEach(async () => { + quietStderr(); + dir = await makeTempDir("mcp-fs-edit-mismatch-"); + conn = await connect([dir]); + file = path.join(dir, "greet.ts"); + await fs.writeFile(file, original); + }); + + afterEach(async () => { + vi.restoreAllMocks(); + await conn.close(); + await fs.rm(dir, { recursive: true, force: true }); + }); + + it("names the failed edit, quotes the closest lines and writes nothing", async () => { + const result = await call(conn.client, "edit_file", { + path: file, + edits: [ + { oldText: "return name;", newText: "return name.length;" }, + // A tab where the file has a space, inside the line: not tolerated. + { + oldText: " const name = 'world';\n console.log('hello',\tname);", + newText: " console.log('bye');", + }, + ], + }); + expect(result.isError).toBe(true); + expect(textOf(result)).toBe( + "Could not find exact match for edit 2 of 2. oldText must match the " + + "file's text exactly; only line endings, and leading and trailing " + + "whitespace on whole lines, may differ. No edits were applied.\n\n" + + "oldText:\n const name = 'world';\n console.log('hello',\tname);\n\n" + + "Closest text in the file (lines 1-5 of 5, after the earlier edits " + + "in this call):\n" + + "1: function greet() {\n" + + "2: const name = 'world';\n" + + "3: console.log('hello', name);\n" + + "4: return name.length;\n" + + "5: }\n\n" + + "First difference, oldText line 2 against file line 3:\n" + + " oldText: \" console.log('hello',\\tname);\"\n" + + " file: \" console.log('hello', name);\"\n\n" + + "Retry with oldText copied exactly from the file, without the " + + "line-number prefixes.", + ); + // The first edit matched, but a failed call applies none of them. + expect(await fs.readFile(file, "utf-8")).toBe(original); + }); + + it("still applies an edit that differs only in indentation", async () => { + await call(conn.client, "edit_file", { + path: file, + edits: [{ oldText: " return name;\n}", newText: " return 0;\n}" }], + }); + const edited = await fs.readFile(file, "utf-8"); + expect(edited).toContain("\n return 0;\n}"); + expect(edited).not.toContain(" return 0;"); + }); + + it("states the matching rules in the tool description", async () => { + const { tools } = await conn.client.listTools(); + const description = tools.find((t) => t.name === "edit_file")?.description; + expect(description).toContain("Whitespace inside a line must match."); + expect(description).toContain("line endings (LF or CRLF) never matter"); + expect(description).toContain("nothing is written"); + }); +}); + +describe("describeEditMismatch", () => { + const lines = (n: number, prefix = "line") => + Array.from({ length: n }, (_, i) => `${prefix} ${i + 1}`); + + it("omits the earlier-edits note for the first edit", () => { + const message = describeEditMismatch("alpha\nbeta\n", "betta", 0, 1); + expect(message).toContain("edit 1 of 1"); + expect(message).toContain("(lines 1-2 of 2):\n1: alpha\n2: beta"); + expect(message).toContain(' oldText: "betta"\n file: "beta"'); + }); + + it("picks the most similar run of lines, the earliest on a tie", () => { + const content = [...lines(10), "target x", ...lines(10), "target x"].join( + "\n", + ); + const message = describeEditMismatch(content, "target x", 0, 1); + expect(message).toContain("(lines 9-13 of 22):"); + expect(message).toContain("11: target x"); + expect(message).toContain("oldText line 1 against file line 11"); + }); + + it("normalizes CRLF in oldText before comparing", () => { + const message = describeEditMismatch("a\nb\nc\n", "a\r\nB\r\n", 0, 1); + expect(message).toContain("oldText line 2 against file line 2"); + }); + + it("reports oldText that runs past the end of the file", () => { + const message = describeEditMismatch("a\nb\n", "a\nb\nc", 0, 1); + expect(message).toContain("(lines 1-2 of 2):"); + expect(message).toContain( + 'oldText line 3 ("c") runs past the end of the file, which ends at line 2.', + ); + }); + + it("says so when no line resembles oldText", () => { + const message = describeEditMismatch("abc\ndef\n", "xyz", 0, 1); + expect(message).toContain("No line of the file resembles oldText."); + expect(message).not.toContain("Closest text"); + }); + + it("says so for an empty file", () => { + const message = describeEditMismatch("", "x", 0, 1); + expect(message).toContain("The file is empty."); + expect(message).not.toContain("Closest text"); + }); + + it("finds a near match that ends at the end of the file", () => { + const message = describeEditMismatch( + "header\nalpha\nbeta\n", + "alpha\nbeta\ngamma", + 0, + 1, + ); + expect(message).toContain( + "(lines 1-3 of 3):\n1: header\n2: alpha\n3: beta", + ); + expect(message).toContain( + 'oldText line 3 ("gamma") runs past the end of the file, which ends at line 3.', + ); + }); + + it("lines up a terminal newline in oldText with the file's", () => { + const content = "function f() {\n return name;\n}\n"; + const message = describeEditMismatch(content, "return naem;\n}\n", 0, 1); + expect(message).toContain("(lines 1-3 of 3):"); + expect(message).toContain( + "oldText line 1 against file line 2:\n" + + ' oldText: "return naem;"\n' + + ' file: " return name;"', + ); + }); + + it("reports an extra blank line at the end of the file at its own line", () => { + const message = describeEditMismatch("alpha\n", "alpha\n\n", 0, 1); + expect(message).toContain("(lines 1-1 of 1):\n1: alpha"); + expect(message).toContain( + 'oldText line 2 ("") runs past the end of the file, which ends at line 1.', + ); + }); + + it("cuts a long file line, in the region and in the difference", () => { + const long = "x".repeat(1_000_000); + const message = describeEditMismatch(`${long}\n`, "xxxx yyyy", 0, 1); + expect(message.length).toBeLessThan(2_000); + expect(message).toContain( + `1: ${"x".repeat(500)} [... cut: the line has 1000000 characters]`, + ); + expect(message).toContain( + ` file: "${"x".repeat(500)} [... cut: the line has 1000000 characters]"`, + ); + expect(message).toContain( + "Some lines above were cut. Read the file to copy oldText from it exactly.", + ); + expect(message).not.toContain("without the line-number prefixes"); + }); + + it("never starts the closest text after the file's terminal newline", () => { + const message = describeEditMismatch( + "function f() {\nreturn value;\n}\n", + "\nreturn vaLue;", + 0, + 1, + ); + expect(message).toContain("(lines 1-3 of 3):"); + expect(message).not.toContain("runs past the end"); + expect(message).toContain("oldText line 1 against file line 1"); + }); + + it("caps the quoted region at 30 lines", () => { + const content = lines(50).join("\n"); + const oldText = [...lines(39), "line 40 changed"].join("\n"); + const message = describeEditMismatch(content, oldText, 0, 1); + expect(message).toContain("(lines 1-30 of 50):"); + expect(message).toContain("\n30: line 30\n"); + expect(message).not.toContain("\n31: "); + expect(message).toContain("oldText line 40 against file line 40"); + }); + + it("skips the search on a file too large to compare", () => { + const content = lines(2000).join("\n"); + const oldText = lines(1000, "other").join("\n"); + const message = describeEditMismatch(content, oldText, 1, 3); + expect(message).toContain("edit 2 of 3"); + expect(message).toContain( + "The file has 2000 lines, too many to search for the closest text to this oldText.", + ); + expect(message).not.toContain("Closest text"); + }); +}); diff --git a/src/filesystem/__tests__/file-resources.test.ts b/src/filesystem/__tests__/file-resources.test.ts new file mode 100644 index 0000000000..76bf103dcc --- /dev/null +++ b/src/filesystem/__tests__/file-resources.test.ts @@ -0,0 +1,345 @@ +// Files as resources (#297), driven through a real Client over an in-memory +// transport: the file:// template is advertised, resources/list enumerates +// nothing, a read returns text or a base64 blob by type, and a read is +// confined exactly as the tools are (allowed directories only, no symlink out, +// the roots barrier on 2025-11-25, the no-directories error on 2026-07-28). + +import fs from "fs/promises"; +import path from "path"; +import { pathToFileURL } from "url"; +import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; +import { + Client, + InMemoryTransport, + ProtocolError, + ResourceNotFoundError, +} from "@modelcontextprotocol/client"; +import { McpServer } from "@modelcontextprotocol/server"; +import { serveStdio } from "@modelcontextprotocol/server/stdio"; +import { + createServer, + NO_ALLOWED_DIRECTORIES_MODERN_ERROR, +} from "../server.js"; +import { + FILE_RESOURCE_TEMPLATE, + fileResourceContents, + registerFileResources, +} from "../file-resources.js"; +import { FsTimeoutError } from "../lib.js"; +import { connect, makeTempDir, quietStderr } from "./helpers.js"; + +let allowed: string; +let other: string; +let closers: Array<() => Promise>; + +const uriOf = (p: string) => pathToFileURL(p).href; + +beforeEach(async () => { + quietStderr(); + allowed = await makeTempDir("mcp-fs-res-allowed-"); + other = await makeTempDir("mcp-fs-res-other-"); + await fs.writeFile(path.join(other, "secret.txt"), "secret"); + closers = []; +}); + +afterEach(async () => { + for (const close of closers.reverse()) await close(); + vi.restoreAllMocks(); + await fs.rm(allowed, { recursive: true, force: true }); + await fs.rm(other, { recursive: true, force: true }); +}); + +async function connectTo(directories: string[]): Promise { + const { client, close } = await connect(directories); + closers.push(close); + return client; +} + +/** The error a resources/read rejects with. */ +async function readError(client: Client, uri: string): Promise { + const error = await client.readResource({ uri }).then( + () => undefined, + (e: unknown) => e, + ); + expect(error).toBeInstanceOf(ProtocolError); + return error as ProtocolError; +} + +describe("resource discovery", () => { + it("advertises the file:// template and lists no resources", async () => { + const client = await connectTo([allowed]); + expect(client.getServerCapabilities()?.resources).toBeDefined(); + const { resourceTemplates } = await client.listResourceTemplates(); + expect(resourceTemplates).toEqual([ + expect.objectContaining({ + name: "file", + uriTemplate: FILE_RESOURCE_TEMPLATE, + }), + ]); + const { resources } = await client.listResources(); + expect(resources).toEqual([]); + }); +}); + +describe("resources/read", () => { + it("returns a text file as text", async () => { + const file = path.join(allowed, "notes.md"); + await fs.writeFile(file, "# héllo\n"); + const client = await connectTo([allowed]); + const { contents } = await client.readResource({ uri: uriOf(file) }); + expect(contents).toEqual([ + { uri: uriOf(file), mimeType: "text/markdown", text: "# héllo\n" }, + ]); + }); + + it("returns an image as a base64 blob with its MIME type", async () => { + const file = path.join(allowed, "pixel.png"); + const bytes = Buffer.from([0x89, 0x50, 0x4e, 0x47, 0x00, 0xff]); + await fs.writeFile(file, bytes); + const client = await connectTo([allowed]); + const { contents } = await client.readResource({ uri: uriOf(file) }); + expect(contents).toEqual([ + { + uri: uriOf(file), + mimeType: "image/png", + blob: bytes.toString("base64"), + }, + ]); + }); + + it("decodes a percent-encoded path", async () => { + const file = path.join(allowed, "with space.txt"); + await fs.writeFile(file, "spaced"); + const client = await connectTo([allowed]); + const uri = uriOf(file); + expect(uri).toContain("%20"); + const { contents } = await client.readResource({ uri }); + expect(contents[0]).toMatchObject({ text: "spaced" }); + }); + + it("rejects a file outside the allowed directories", async () => { + const client = await connectTo([allowed]); + const error = await readError( + client, + uriOf(path.join(other, "secret.txt")), + ); + expect(error.code).toBe(-32602); + expect(error.message).toContain("Access denied"); + }); + + it("rejects a dot-dot URI that climbs out of an allowed directory", async () => { + const client = await connectTo([allowed]); + const climbing = `${uriOf(allowed)}/../${path.basename(other)}/secret.txt`; + const error = await readError(client, climbing); + expect(error.message).toContain("Access denied"); + }); + + it("rejects a symlink whose target is outside the allowed directories", async () => { + const link = path.join(allowed, "link.txt"); + await fs.symlink(path.join(other, "secret.txt"), link); + const client = await connectTo([allowed]); + const error = await readError(client, uriOf(link)); + expect(error.message).toContain("symlink target outside"); + }); + + it("reports a missing file as resource not found", async () => { + const client = await connectTo([allowed]); + const uri = uriOf(path.join(allowed, "missing.txt")); + const error = await readError(client, uri); + expect(error).toBeInstanceOf(ResourceNotFoundError); + expect(error.data).toEqual({ uri }); + }); + + it("reports a dangling symlink as resource not found", async () => { + const link = path.join(allowed, "dangling.txt"); + await fs.symlink(path.join(allowed, "gone", "nowhere.txt"), link); + const client = await connectTo([allowed]); + const uri = uriOf(link); + const error = await readError(client, uri); + expect(error).toBeInstanceOf(ResourceNotFoundError); + expect(error.data).toEqual({ uri }); + }); + + it("reports a path beneath a regular file as resource not found", async () => { + await fs.writeFile(path.join(allowed, "parent.txt"), "x"); + const client = await connectTo([allowed]); + const uri = uriOf(path.join(allowed, "parent.txt", "child.txt")); + const error = await readError(client, uri); + expect(error).toBeInstanceOf(ResourceNotFoundError); + expect(error.data).toEqual({ uri }); + }); + + it("reports a directory as not a resource", async () => { + await fs.mkdir(path.join(allowed, "sub")); + const client = await connectTo([allowed]); + const uri = uriOf(path.join(allowed, "sub")); + const error = await readError(client, uri); + expect(error).toBeInstanceOf(ResourceNotFoundError); + expect(error.message).toContain("is a directory"); + }); + + it.skipIf(process.platform === "win32")( + "rejects a file URI with a remote host", + async () => { + const client = await connectTo([allowed]); + const error = await readError(client, "file://elsewhere/etc/passwd"); + expect(error.code).toBe(-32602); + expect(error.message).toContain("Invalid file URI"); + }, + ); + + it("reports a failure while resolving the path as an internal error", async () => { + const file = path.join(allowed, "unlucky.txt"); + await fs.writeFile(file, "x"); + const client = await connectTo([allowed]); + vi.spyOn(fs, "realpath").mockRejectedValueOnce( + Object.assign(new Error("i/o error"), { code: "EIO" }), + ); + const error = await readError(client, uriOf(file)); + expect(error.code).toBe(-32603); + expect(error.message).toContain("Failed to resolve"); + }); + + it("reports a path resolution timeout as an internal error", async () => { + const file = path.join(allowed, "slow.txt"); + await fs.writeFile(file, "x"); + const client = await connectTo([allowed]); + vi.spyOn(fs, "realpath").mockRejectedValueOnce( + new FsTimeoutError("realpath", file), + ); + const error = await readError(client, uriOf(file)); + expect(error.code).toBe(-32603); + expect(error.message).toContain("timed out"); + }); + + it("reports any other read failure as an internal error", async () => { + const file = path.join(allowed, "locked.txt"); + await fs.writeFile(file, "x"); + const client = await connectTo([allowed]); + vi.spyOn(fs, "readFile").mockRejectedValueOnce( + Object.assign(new Error("permission denied"), { code: "EACCES" }), + ); + const error = await readError(client, uriOf(file)); + expect(error.code).toBe(-32603); + expect(error.message).toContain("permission denied"); + }); +}); + +describe("fileResourceContents", () => { + const uri = "file:///x"; + + it("sniffs an unknown extension: UTF-8 is text/plain", () => { + expect( + fileResourceContents(uri, "/a/Makefile", Buffer.from("all:\n")), + ).toEqual({ uri, mimeType: "text/plain", text: "all:\n" }); + }); + + it("sniffs an unknown extension: NUL bytes are an octet-stream blob", () => { + const bytes = Buffer.from([0x41, 0x00, 0x42]); + expect(fileResourceContents(uri, "/a/data.bin", bytes)).toEqual({ + uri, + mimeType: "application/octet-stream", + blob: bytes.toString("base64"), + }); + }); + + it("returns a textual type that is not valid UTF-8 as a blob of that type", () => { + const bytes = Buffer.from([0xc3, 0x28]); + expect(fileResourceContents(uri, "/a/broken.JSON", bytes)).toEqual({ + uri, + mimeType: "application/json", + blob: bytes.toString("base64"), + }); + }); +}); + +describe("protocol eras", () => { + async function connectEra( + directories: string[], + mode: "auto" | "legacy", + listRoots?: () => Promise<{ uri: string }[]>, + ): Promise { + const [clientTransport, serverTransport] = + InMemoryTransport.createLinkedPair(); + const handle = serveStdio(({ era }) => createServer(directories, { era }), { + transport: serverTransport, + }); + const client = new Client( + { name: "fs-res-era-test", version: "0.0.0" }, + { + capabilities: listRoots ? { roots: {} } : {}, + versionNegotiation: { mode }, + }, + ); + if (listRoots) { + client.setRequestHandler("roots/list", async () => ({ + roots: await listRoots(), + })); + } + await client.connect(clientTransport); + closers.push( + () => handle.close(), + () => client.close(), + ); + return client; + } + + it("reads a file on 2026-07-28", async () => { + const file = path.join(allowed, "a.txt"); + await fs.writeFile(file, "modern"); + const client = await connectEra([allowed], "auto"); + expect(client.getProtocolEra()).toBe("modern"); + const { contents } = await client.readResource({ uri: uriOf(file) }); + expect(contents[0]).toMatchObject({ text: "modern" }); + }); + + it("explains on 2026-07-28 that a server with no directories cannot operate", async () => { + const client = await connectEra([], "auto"); + const error = await readError( + client, + uriOf(path.join(other, "secret.txt")), + ); + expect(error.message).toContain(NO_ALLOWED_DIRECTORIES_MODERN_ERROR); + }); + + it("waits on 2025-11-25 for the client's roots before reading", async () => { + let answer!: (roots: { uri: string }[]) => void; + const roots = new Promise<{ uri: string }[]>((r) => (answer = r)); + const client = await connectEra([allowed], "legacy", () => roots); + expect(client.getProtocolEra()).toBe("legacy"); + const read = client.readResource({ + uri: uriOf(path.join(other, "secret.txt")), + }); + answer([{ uri: uriOf(other) }]); + const { contents } = await read; + expect(contents[0]).toMatchObject({ text: "secret" }); + }); +}); + +describe("registerFileResources", () => { + it("waits again when the roots barrier is replaced while it waits", async () => { + const file = path.join(allowed, "a.txt"); + await fs.writeFile(file, "after"); + const first = Promise.resolve(); + const second = Promise.resolve(); + const barriers = [first, second, second, second]; + const rootsSettled = vi.fn(() => barriers.shift() ?? second); + const server = new McpServer({ name: "t", version: "0.0.0" }); + registerFileResources(server, { + allowedDirectories: () => [allowed], + rootsSettled, + unavailableReason: () => undefined, + }); + const client = new Client({ name: "t", version: "0.0.0" }); + const [ct, st] = InMemoryTransport.createLinkedPair(); + await Promise.all([server.connect(st), client.connect(ct)]); + closers.push( + () => server.close(), + () => client.close(), + ); + const { contents } = await client.readResource({ uri: uriOf(file) }); + expect(contents[0]).toMatchObject({ text: "after" }); + // first, then (replaced) second, then second again: settled. + expect(rootsSettled).toHaveBeenCalledTimes(4); + }); +}); diff --git a/src/filesystem/__tests__/lib.test.ts b/src/filesystem/__tests__/lib.test.ts index 15ee98abf1..b1e720ec86 100644 --- a/src/filesystem/__tests__/lib.test.ts +++ b/src/filesystem/__tests__/lib.test.ts @@ -402,6 +402,24 @@ describe("Lib Functions", () => { ).rejects.toThrow("EACCES lstat"); }); + // #5191: only an ENOENT from an existing entry's realpath means a + // broken symlink; any other failure is reported as it is. + it("reports a realpath failure other than ENOENT on an existing component as is", async () => { + const base = + process.platform === "win32" ? "C:\\Users\\test" : "/home/user"; + const enoentError = new Error("ENOENT") as NodeJS.ErrnoException; + enoentError.code = "ENOENT"; + mockFs.realpath + .mockRejectedValueOnce(enoentError) + .mockResolvedValueOnce(base) + .mockRejectedValueOnce(new Error("EACCES realpath")); + mockReaddirNames.mockResolvedValueOnce(["link"]); + + await expect( + validatePath(path.join(base, "link", "a"), allowedDirs), + ).rejects.toThrow("EACCES realpath"); + }); + it("rejects when no ancestor directory exists at all", async () => { const newFilePath = process.platform === "win32" diff --git a/src/filesystem/__tests__/path-utils.test.ts b/src/filesystem/__tests__/path-utils.test.ts index 4534014560..38a3330e9f 100644 --- a/src/filesystem/__tests__/path-utils.test.ts +++ b/src/filesystem/__tests__/path-utils.test.ts @@ -264,6 +264,19 @@ describe("Path Utilities", () => { it("leaves other paths unchanged", () => { expect(expandHome("C:/test")).toBe("C:/test"); }); + + // On POSIX a backslash is a filename character, so ~\notes names a + // file in the working directory, not one in the home directory (#5180). + // win32.test.ts covers the Windows expansion. + it("leaves ~\\ alone on POSIX", () => { + const realPlatform = process.platform; + Object.defineProperty(process, "platform", { value: "linux" }); + try { + expect(expandHome("~\\notes")).toBe("~\\notes"); + } finally { + Object.defineProperty(process, "platform", { value: realPlatform }); + } + }); }); describe("WSL path handling (issue #2795 fix)", () => { diff --git a/src/filesystem/__tests__/pinned-directory.test.ts b/src/filesystem/__tests__/pinned-directory.test.ts index a395187025..38c0e80b34 100644 --- a/src/filesystem/__tests__/pinned-directory.test.ts +++ b/src/filesystem/__tests__/pinned-directory.test.ts @@ -908,6 +908,10 @@ describe("createDirectoryInPlace", () => { execFileSync("hdiutil", ["detach", mount, "-quiet"]); } }, + // hdiutil create, attach and detach together take about 1.5 s on an idle + // machine and can pass vitest's 5 s default under load (#5314). A + // timeout sized for them, never a retry, which would hide a real failure. + 30_000, ); it("reports a failure to enter a directory other than a missing one as is", () => { diff --git a/src/filesystem/__tests__/read-only.test.ts b/src/filesystem/__tests__/read-only.test.ts new file mode 100644 index 0000000000..21fd1d25d5 --- /dev/null +++ b/src/filesystem/__tests__/read-only.test.ts @@ -0,0 +1,221 @@ +// Read-only mode (#632, #1160): `--read-only` on the command line, or +// `readOnly` in createServer's options, serves only the tools whose +// annotations declare `readOnlyHint: true`. The write tools are not merely +// refused at call time but absent, so a client never lists them. These tests +// pin the exact read-only tool set on both protocol eras, prove a write tool +// cannot be called and leaves the disk untouched, and drive the flag through +// main() so the command-line wiring is covered too. + +import fs from "fs/promises"; +import path from "path"; +import { PassThrough } from "node:stream"; +import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; +import { + Client, + InMemoryTransport, + type CallToolResult, +} from "@modelcontextprotocol/client"; +import { serveStdio } from "@modelcontextprotocol/server/stdio"; +import { main, parseArgs } from "../index.js"; +import { createServer } from "../server.js"; +import { makeTempDir, quietStderr } from "./helpers.js"; + +// Every tool that cannot change the filesystem. A tool added to the server +// must be added here if it is read-only; a write tool is withheld without any +// change, which is the point of keying the mode on the annotation. +const READ_ONLY_TOOLS = [ + "directory_tree", + "get_file_info", + "list_allowed_directories", + "list_directory", + "list_directory_with_sizes", + "read_file", + "read_media_file", + "read_multiple_files", + "read_text_file", + "search_files", +]; + +let dir: string; +let stderr: ReturnType; +let closers: Array<() => Promise>; + +beforeEach(async () => { + stderr = quietStderr(); + dir = await makeTempDir("mcp-fs-read-only-"); + await fs.writeFile(path.join(dir, "note.txt"), "original"); + closers = []; +}); + +afterEach(async () => { + for (const close of closers.reverse()) await close(); + vi.restoreAllMocks(); + await fs.rm(dir, { recursive: true, force: true }); +}); + +/** A client connected to a server for `dir`, negotiating per `mode`. */ +async function connect( + readOnly: boolean, + mode: "auto" | "legacy", +): Promise { + const [clientTransport, serverTransport] = + InMemoryTransport.createLinkedPair(); + const handle = serveStdio( + ({ era }) => createServer([dir], { era, readOnly }), + { transport: serverTransport }, + ); + const client = new Client( + { name: "fs-read-only-test", version: "0.0.0" }, + { versionNegotiation: { mode } }, + ); + await client.connect(clientTransport); + closers.push( + () => handle.close(), + () => client.close(), + ); + return client; +} + +describe.each([ + ["2026-07-28", "auto"], + ["2025-11-25", "legacy"], +] as const)("read-only mode on %s", (_label, mode) => { + it("lists only the read-only tools, each annotated readOnlyHint: true", async () => { + const client = await connect(true, mode); + const { tools } = await client.listTools(); + expect(tools.map((tool) => tool.name).sort()).toEqual(READ_ONLY_TOOLS); + for (const tool of tools) { + expect(tool.annotations?.readOnlyHint).toBe(true); + } + }); + + it("withholds exactly the tools not annotated read-only", async () => { + const full = await connect(false, mode); + const { tools } = await full.listTools(); + const writeTools = tools + .filter((tool) => tool.annotations?.readOnlyHint !== true) + .map((tool) => tool.name); + expect(writeTools).toEqual( + expect.arrayContaining([ + "write_file", + "edit_file", + "create_directory", + "move_file", + ]), + ); + const readOnly = await connect(true, mode); + const listed = (await readOnly.listTools()).tools.map((t) => t.name); + for (const name of writeTools) expect(listed).not.toContain(name); + }); + + it("refuses a call to a write tool and leaves the file untouched", async () => { + const client = await connect(true, mode); + const target = path.join(dir, "note.txt"); + const result = (await client + .callTool({ + name: "write_file", + arguments: { path: target, content: "changed" }, + }) + .catch((error: unknown) => ({ + isError: true, + content: [{ type: "text", text: String(error) }], + }))) as CallToolResult; + expect(result.isError).toBe(true); + expect(JSON.stringify(result.content)).toMatch(/write_file/); + expect(await fs.readFile(target, "utf8")).toBe("original"); + }); + + it("still serves the read tools", async () => { + const client = await connect(true, mode); + const result = (await client.callTool({ + name: "read_text_file", + arguments: { path: path.join(dir, "note.txt") }, + })) as CallToolResult; + expect(result.isError).toBeFalsy(); + expect(result.content).toEqual([{ type: "text", text: "original" }]); + }); +}); + +describe("parseArgs", () => { + it("takes every argument as a directory when no option is given", () => { + expect(parseArgs(["/a", "/b"])).toEqual({ + directories: ["/a", "/b"], + readOnly: false, + confineRoots: false, + }); + }); + + it("recognizes --read-only before, between or after the directories", () => { + for (const args of [ + ["--read-only", "/a", "/b"], + ["/a", "--read-only", "/b"], + ["/a", "/b", "--read-only"], + ]) { + expect(parseArgs(args)).toEqual({ + directories: ["/a", "/b"], + readOnly: true, + confineRoots: false, + }); + } + }); + + it("treats everything after -- as a directory", () => { + expect(parseArgs(["--read-only", "--", "--read-only", "/a"])).toEqual({ + directories: ["--read-only", "/a"], + readOnly: true, + confineRoots: false, + }); + }); +}); + +describe("main() with --read-only", () => { + it("serves only the read-only tools over stdio and says so on stderr", async () => { + const stdin = new PassThrough(); + const stdout = new PassThrough(); + stdout.setEncoding("utf8"); + const handle = await main(["--read-only", dir], stdin, stdout); + closers.push(() => handle.close()); + const line = new Promise((resolve) => { + let buffered = ""; + stdout.on("data", (chunk: string) => { + buffered += chunk; + const newline = buffered.indexOf("\n"); + if (newline !== -1) resolve(buffered.slice(0, newline)); + }); + }); + stdin.write( + `${JSON.stringify({ + jsonrpc: "2.0", + id: 1, + method: "tools/list", + params: { + _meta: { + "io.modelcontextprotocol/protocolVersion": "2026-07-28", + "io.modelcontextprotocol/clientCapabilities": {}, + }, + }, + })}\n`, + ); + const { result } = JSON.parse(await line) as { + result: { tools: Array<{ name: string }> }; + }; + expect(result.tools.map((tool) => tool.name).sort()).toEqual( + READ_ONLY_TOOLS, + ); + expect(stderr).toHaveBeenCalledWith( + "Read-only mode: only tools that cannot change the filesystem are served", + ); + }); + + it("prints usage when --read-only is the only argument", async () => { + const handle = await main( + ["--read-only"], + new PassThrough(), + new PassThrough(), + ); + closers.push(() => handle.close()); + expect(stderr).toHaveBeenCalledWith( + "Usage: mcp-server-filesystem [--read-only] [--confine-roots] [allowed-directory] [additional-directories...]", + ); + }); +}); diff --git a/src/filesystem/__tests__/roots-utils.test.ts b/src/filesystem/__tests__/roots-utils.test.ts index 411aa84e48..862998173c 100644 --- a/src/filesystem/__tests__/roots-utils.test.ts +++ b/src/filesystem/__tests__/roots-utils.test.ts @@ -13,6 +13,7 @@ import { mkdirSync, writeFileSync, realpathSync, + symlinkSync, } from "fs"; import { homedir, tmpdir } from "os"; import { join } from "path"; @@ -139,3 +140,35 @@ describe("getValidRootDirectories", () => { }); }); }); + +// #5320: a Root's realpath is kept as the file system spells it. Trimming a +// trailing space or quote from it names the sibling without the suffix. +// Windows refuses or strips these characters in names, so POSIX only. +describe.skipIf(process.platform === "win32")( + "getValidRootDirectories: a realpath ending in a space or quote (#5320)", + () => { + let base: string; + + beforeEach(() => { + base = realpathSync.native(mkdtempSync(join(tmpdir(), "mcp-fs-5320-"))); + }); + + afterEach(() => { + rmSync(base, { recursive: true, force: true }); + }); + + it.each([" ", "'", '"'])( + "keeps the resolved form untrimmed when it ends in %j", + async (suffix) => { + const target = join(base, `real${suffix}`); + mkdirSync(target); + mkdirSync(join(base, "real")); + const link = join(base, "link"); + symlinkSync(target, link); + expect( + await getValidRootDirectories([{ uri: pathToFileURL(link).href }]), + ).toEqual([link, target]); + }, + ); + }, +); diff --git a/src/filesystem/__tests__/search-files-description.test.ts b/src/filesystem/__tests__/search-files-description.test.ts new file mode 100644 index 0000000000..5c0401f6b7 --- /dev/null +++ b/src/filesystem/__tests__/search-files-description.test.ts @@ -0,0 +1,82 @@ +// What the search_files description tells a model, checked against what the +// tool does (#896, #3419). Models read "search for files matching a pattern" +// as a content search (grep) and got "No matches found" for text that was in +// the files, so the description now says the tool matches names, never +// contents. These tests pin that wording and each matching rule it states, so +// the description cannot drift from the behavior again. + +import fs from "fs/promises"; +import path from "path"; +import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; +import type { Client } from "@modelcontextprotocol/client"; +import { + call, + connect, + makeTempDir, + quietStderr, + textOf, + type Connected, +} from "./helpers.js"; + +let dir: string; +let conn: Connected; +let client: Client; + +beforeEach(async () => { + quietStderr(); + dir = await makeTempDir("mcp-fs-search-desc-"); + await fs.mkdir(path.join(dir, "src", "config"), { recursive: true }); + await fs.writeFile(path.join(dir, "notes.md"), "needle"); + await fs.writeFile(path.join(dir, "src", "app.md"), ""); + await fs.writeFile(path.join(dir, "config"), ""); + await fs.writeFile(path.join(dir, "README.MD"), ""); + conn = await connect([dir]); + client = conn.client; +}); + +afterEach(async () => { + vi.restoreAllMocks(); + await conn.close(); + await fs.rm(dir, { recursive: true, force: true }); +}); + +async function search(pattern: string): Promise { + const text = textOf( + await call(client, "search_files", { path: dir, pattern }), + ); + return text.split("\n").sort(); +} + +describe("search_files description (#896, #3419)", () => { + it("says it matches names, not file contents", async () => { + const { tools } = await client.listTools(); + const description = tools.find( + (t) => t.name === "search_files", + )?.description; + expect(description).toMatch( + /^Find files and directories by NAME, not by content/, + ); + expect(description).toContain("never reads or searches inside files"); + expect(description).toContain("relative to 'path'"); + expect(description).not.toContain("working directory"); + }); + + it("does not match a file's contents", async () => { + expect(await search("**/needle")).toEqual(["No matches found"]); + }); + + it("matches '*.ext' only directly under path", async () => { + expect(await search("*.md")).toEqual([path.join(dir, "notes.md")]); + }); + + it("matches '**/name' at any depth, the top level included", async () => { + expect(await search("**/config")).toEqual([ + path.join(dir, "config"), + path.join(dir, "src", "config"), + ]); + }); + + it("matches case-sensitively", async () => { + expect(await search("*.MD")).toEqual([path.join(dir, "README.MD")]); + }); +}); diff --git a/src/filesystem/__tests__/search-unreadable.test.ts b/src/filesystem/__tests__/search-unreadable.test.ts new file mode 100644 index 0000000000..9b0d1fbe45 --- /dev/null +++ b/src/filesystem/__tests__/search-unreadable.test.ts @@ -0,0 +1,169 @@ +// search_files over a tree with a subdirectory it cannot read (#5189). The +// walk used to swallow that readdir failure with the validation errors it +// skips on purpose, so the result came back short with nothing to say so. Now +// the walk skips the directory, keeps going, and the tool names it after the +// matches. These tests make readdir fail for one directory by spying on the +// fs/promises object the server shares with this file, which works on every +// platform; one more test uses a real chmod 000 directory where POSIX +// permissions apply. + +import fs from "fs/promises"; +import type { PathLike } from "fs"; +import path from "path"; +import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; +import type { Client } from "@modelcontextprotocol/client"; +import { + call, + connect, + makeTempDir, + quietStderr, + textOf, + type Connected, +} from "./helpers.js"; +import { searchFilesWithValidation } from "../lib.js"; + +const realReaddir = fs.readdir.bind(fs); + +let dir: string; +let conn: Connected; +let client: Client; + +beforeEach(async () => { + quietStderr(); + dir = await makeTempDir("mcp-fs-unreadable-"); + await fs.mkdir(path.join(dir, "open")); + await fs.mkdir(path.join(dir, "locked")); + await fs.mkdir(path.join(dir, "sealed")); + await fs.writeFile(path.join(dir, "open", "a.txt"), ""); + await fs.writeFile(path.join(dir, "locked", "hidden.txt"), ""); + await fs.writeFile(path.join(dir, "sealed", "also-hidden.txt"), ""); + conn = await connect([dir]); + client = conn.client; +}); + +afterEach(async () => { + vi.restoreAllMocks(); + await conn.close(); + await fs.chmod(path.join(dir, "locked"), 0o755); + await fs.rm(dir, { recursive: true, force: true }); +}); + +/** Make readdir reject with `error` for each directory in `failing`. */ +function failReaddir(failing: string[], error: Error): void { + vi.spyOn(fs, "readdir").mockImplementation((( + target: PathLike, + ...rest: unknown[] + ) => + failing.includes(target.toString()) + ? Promise.reject(error) + : ( + realReaddir as ( + target: PathLike, + ...rest: unknown[] + ) => Promise + )(target, ...rest)) as typeof fs.readdir); +} + +function eacces(): Error { + return Object.assign(new Error("EACCES: permission denied"), { + code: "EACCES", + }); +} + +describe("search_files with an unreadable subdirectory (#5189)", () => { + it("returns the matches it found and names the directory it could not read", async () => { + const locked = path.join(dir, "locked"); + failReaddir([locked], eacces()); + const result = await call(client, "search_files", { + path: dir, + pattern: "**/*.txt", + }); + expect(result.isError).toBeFalsy(); + const text = textOf(result); + expect(text.split("\n\n")).toEqual([ + expect.stringContaining(path.join(dir, "open", "a.txt")), + "The search is incomplete: 1 directory could not be read, and any matches inside are missing:\n" + + `${locked} (EACCES)`, + ]); + expect(text).toContain(path.join(dir, "sealed", "also-hidden.txt")); + expect(text).not.toContain(path.join(locked, "hidden.txt")); + expect(result.structuredContent).toEqual({ content: text }); + }); + + it('says "No matches found" and lists every unreadable directory when nothing else matched', async () => { + const locked = path.join(dir, "locked"); + const sealed = path.join(dir, "sealed"); + failReaddir([locked, sealed], eacces()); + const result = await call(client, "search_files", { + path: dir, + pattern: "**/*hidden*", + }); + expect(result.isError).toBeFalsy(); + const [matches, note] = textOf(result).split("\n\n"); + expect(matches).toBe("No matches found"); + const [heading, ...listed] = note.split("\n"); + expect(heading).toBe( + "The search is incomplete: 2 directories could not be read, and any matches inside are missing:", + ); + expect(listed.sort()).toEqual([`${locked} (EACCES)`, `${sealed} (EACCES)`]); + }); + + it("names the directory without a code when the error carries none", async () => { + const locked = path.join(dir, "locked"); + failReaddir([locked], new Error("unreadable")); + const result = await call(client, "search_files", { + path: dir, + pattern: "**/a.txt", + }); + expect(textOf(result)).toBe( + `${path.join(dir, "open", "a.txt")}\n\n` + + "The search is incomplete: 1 directory could not be read, and any matches inside are missing:\n" + + locked, + ); + }); + + it("still fails the request when the starting directory cannot be read", async () => { + failReaddir([dir], eacces()); + const result = await call(client, "search_files", { + path: dir, + pattern: "**/*.txt", + }); + expect(result.isError).toBe(true); + expect(textOf(result)).toContain("EACCES"); + }); + + it("reports nothing extra when every directory reads", async () => { + const result = await call(client, "search_files", { + path: dir, + pattern: "**/a.txt", + }); + expect(textOf(result)).toBe(path.join(dir, "open", "a.txt")); + }); + + it.skipIf(process.platform === "win32" || process.getuid?.() === 0)( + "reports a real chmod 000 directory", + async () => { + const locked = path.join(dir, "locked"); + await fs.chmod(locked, 0o000); + const result = await call(client, "search_files", { + path: dir, + pattern: "**/*.txt", + }); + expect(result.isError).toBeFalsy(); + expect(textOf(result)).toContain(`\n${locked} (EACCES)`); + expect(textOf(result)).toContain(path.join(dir, "open", "a.txt")); + }, + ); +}); + +describe("searchFilesWithValidation without an onUnreadable callback", () => { + it("skips the unreadable directory and returns the rest", async () => { + const locked = path.join(dir, "locked"); + failReaddir([locked], eacces()); + const realDir = await fs.realpath(dir); + const results = await searchFilesWithValidation(realDir, "**/a.txt", [ + realDir, + ]); + expect(results).toEqual([path.join(realDir, "open", "a.txt")]); + }); +}); diff --git a/src/filesystem/__tests__/server-paths.test.ts b/src/filesystem/__tests__/server-paths.test.ts index 803cfb41a6..d24ee98076 100644 --- a/src/filesystem/__tests__/server-paths.test.ts +++ b/src/filesystem/__tests__/server-paths.test.ts @@ -314,7 +314,7 @@ describe("symlinks", () => { ).toBe("x"); }); - it("refuses to write under a respelled dangling symlink, reporting a missing parent", async () => { + it("refuses to write under a respelled dangling symlink, naming it as broken (#5191)", async () => { await fs.symlink( path.join(outside, "not-yet"), path.join(dir, "dangling"), @@ -324,24 +324,51 @@ describe("symlinks", () => { content: "x", }); expect(textOf(result)).toBe( - `Parent directory does not exist: ${path.join(dir, "DANGLING")}`, + `Broken symlink: ${path.join(dir, "DANGLING")} points to a target that does not exist`, ); await expect(fs.stat(path.join(outside, "not-yet"))).rejects.toThrow(); }); }, ); - it("refuses to write through a dangling symlink, reporting a missing parent", async () => { + // #5191: the refusal is right, and the message names the symlink rather + // than reporting its parent, which exists, as missing. + it("refuses to write through a dangling symlink, naming it as broken (#5191)", async () => { const link = path.join(dir, "dangling"); await fs.symlink(path.join(outside, "not-yet"), link); const result = await call(client, "write_file", { path: link, content: "x", }); - expect(textOf(result)).toBe(`Parent directory does not exist: ${dir}`); + expect(textOf(result)).toBe( + `Broken symlink: ${link} points to a target that does not exist`, + ); await expect(fs.stat(path.join(outside, "not-yet"))).rejects.toThrow(); }); + it("refuses to read a dangling symlink, naming it as broken (#5191)", async () => { + const link = path.join(dir, "dangling"); + await fs.symlink(path.join(dir, "gone.txt"), link); + const result = await read(link); + expect(result.isError).toBe(true); + expect(textOf(result)).toBe( + `Broken symlink: ${link} points to a target that does not exist`, + ); + }); + + it("names a dangling symlink in the middle of a path as broken (#5191)", async () => { + const link = path.join(dir, "dangling-dir"); + await fs.symlink(path.join(dir, "gone"), link); + const result = await call(client, "write_file", { + path: path.join(link, "f.txt"), + content: "x", + }); + expect(textOf(result)).toBe( + `Broken symlink: ${link} points to a target that does not exist`, + ); + await expect(fs.stat(path.join(dir, "gone"))).rejects.toThrow(); + }); + it("reports a realpath failure other than ENOENT as is", async () => { await fs.writeFile(path.join(dir, "file.txt"), ""); const result = await read(path.join(dir, "file.txt", "child")); diff --git a/src/filesystem/__tests__/server-tools.test.ts b/src/filesystem/__tests__/server-tools.test.ts index 20bded6aeb..2be36723cd 100644 --- a/src/filesystem/__tests__/server-tools.test.ts +++ b/src/filesystem/__tests__/server-tools.test.ts @@ -31,12 +31,14 @@ const TOOL_NAMES = [ "read_media_file", "read_multiple_files", "write_file", + "append_file", "edit_file", "create_directory", "list_directory", "list_directory_with_sizes", "directory_tree", "move_file", + "copy_file", "search_files", "get_file_info", "list_allowed_directories", @@ -83,25 +85,32 @@ describe("server identity and tool list", () => { }); }); - it("advertises only the tools capability", () => { + it("advertises the tools and resources capabilities", () => { expect(client.getServerCapabilities()).toEqual({ tools: { listChanged: true }, + resources: { listChanged: true }, }); }); - it("lists the 14 tools in registration order", async () => { + it("lists the tools in registration order", async () => { const { tools } = await client.listTools(); expect(tools.map((t) => t.name)).toEqual(TOOL_NAMES); }); - // #3402: the read-only tools carry no idempotentHint or destructiveHint. - // When that issue is fixed, this table is the test to change. + // #3402: every tool sets all four hints explicitly, the read-only ones + // included (idempotent, never destructive), so a client never falls back + // to the spec's defaults (destructive, not idempotent, open world). it("pins each tool's title and annotations (#3402)", async () => { const { tools } = await client.listTools(); const table = Object.fromEntries( tools.map((t) => [t.name, { title: t.title, ...t.annotations }]), ); - const readOnly = { readOnlyHint: true, openWorldHint: false }; + const readOnly = { + readOnlyHint: true, + idempotentHint: true, + destructiveHint: false, + openWorldHint: false, + }; expect(table).toEqual({ read_file: { title: "Read File (Deprecated)", ...readOnly }, read_text_file: { title: "Read Text File", ...readOnly }, @@ -114,6 +123,13 @@ describe("server identity and tool list", () => { destructiveHint: true, openWorldHint: false, }, + append_file: { + title: "Append File", + readOnlyHint: false, + idempotentHint: false, + destructiveHint: false, + openWorldHint: false, + }, edit_file: { title: "Edit File", readOnlyHint: false, @@ -141,6 +157,13 @@ describe("server identity and tool list", () => { destructiveHint: true, openWorldHint: false, }, + copy_file: { + title: "Copy File", + readOnlyHint: false, + idempotentHint: true, + destructiveHint: false, + openWorldHint: false, + }, search_files: { title: "Search Files", ...readOnly }, get_file_info: { title: "Get File Info", ...readOnly }, list_allowed_directories: { @@ -209,11 +232,15 @@ describe("server identity and tool list", () => { properties: { path: { type: "string" }, tail: { - type: "number", + type: "integer", + minimum: 0, + maximum: Number.MAX_SAFE_INTEGER, description: "If provided, returns only the last N lines of the file", }, head: { - type: "number", + type: "integer", + minimum: 0, + maximum: Number.MAX_SAFE_INTEGER, description: "If provided, returns only the first N lines of the file", }, @@ -449,12 +476,55 @@ describe("read_text_file and read_file", () => { }); }); - it("treats head: 0 as absent and returns the whole file", async () => { - const result = await call(client, "read_text_file", { - path: file, - head: 0, + describe("a zero count (#5184)", () => { + it.each(["head", "tail"])("%s: 0 returns no lines", async (key) => { + const result = await call(client, "read_text_file", { + path: file, + [key]: 0, + }); + expect(result.isError).toBeFalsy(); + expect(result.structuredContent).toEqual({ content: "" }); + }); + + it.each(["head", "tail"])( + "%s: 0 still reports a missing file", + async (key) => { + const result = await call(client, "read_text_file", { + path: path.join(dir, "missing.txt"), + [key]: 0, + }); + expect(result.isError).toBe(true); + expect(textOf(result)).toMatch(/^ENOENT: no such file or directory/); + }, + ); + + it("rejects head: 0 and tail: 0 together", async () => { + const result = await call(client, "read_text_file", { + path: file, + head: 0, + tail: 0, + }); + expect(result.isError).toBe(true); + expect(textOf(result)).toBe( + "Cannot specify both head and tail parameters simultaneously", + ); + }); + + it.each([ + ["head", -1], + ["tail", -1], + ["head", 1.5], + ["tail", 1.5], + ])("rejects %s: %s as invalid input", async (key, value) => { + for (const tool of ["read_text_file", "read_file"]) { + const result = await call(client, tool, { + path: file, + [key]: value, + }); + expect(result.isError, tool).toBe(true); + expect(textOf(result), tool).toMatch(/Input validation error/); + } }); - expect(textOf(result)).toBe("one\ntwo\nthree\nfour\n"); }); it("rejects head and tail together", async () => { @@ -871,8 +941,9 @@ describe("edit_file", () => { edits: [{ oldText: "return 9;", newText: "x" }], }); expect(result.isError).toBe(true); - expect(textOf(result)).toBe( - "Could not find exact match for edit:\nreturn 9;", + // #2034: the error also quotes the closest lines of the file. + expect(textOf(result)).toMatch( + /^Could not find exact match for edit 1 of 1\. [^]*\n\noldText:\nreturn 9;\n\nClosest text in the file \(lines 1-4 of 4\):\n[^]*3: {3}return 1;\n/, ); }); @@ -1174,15 +1245,38 @@ describe("list_directory_with_sizes", () => { expect(files).toEqual(["b.txt", "a.txt"]); }); - it("lists an entry it cannot stat (a dangling symlink) as size 0", async () => { + it("lists an entry it cannot stat (a dangling symlink) as unknown size, left out of the total (#5190)", async () => { await fs.symlink(path.join(dir, "gone"), path.join(dir, "dangling")); const result = await call(client, "list_directory_with_sizes", { path: dir, }); expect(textOf(result)).toContain( - `[FILE] ${"dangling".padEnd(30)} ${"0 B".padStart(10)}`, + `[FILE] ${"dangling".padEnd(30)} ${"unknown".padStart(10)}`, ); + expect(textOf(result)).not.toContain(" 0 B"); expect(textOf(result)).toContain("Total: 3 files, 1 directories"); + expect(textOf(result)).toContain( + "Combined size: 2.00 KB (excludes 1 file of unknown size)", + ); + }); + + it("sorts entries of unknown size last when sorting by size (#5190)", async () => { + await fs.symlink(path.join(dir, "gone"), path.join(dir, "dangling")); + await fs.symlink(path.join(dir, "gone2"), path.join(dir, "dangling2")); + await fs.writeFile(path.join(dir, "empty.txt"), ""); + const result = await call(client, "list_directory_with_sizes", { + path: dir, + sortBy: "size", + }); + const files = textOf(result) + .split("\n") + .filter((line) => line.startsWith("[FILE]")) + .map((line) => line.split(/\s+/)[1]); + expect(files.slice(0, 3)).toEqual(["b.txt", "a.txt", "empty.txt"]); + expect(files.slice(3).sort()).toEqual(["dangling", "dangling2"]); + expect(textOf(result)).toContain( + "Combined size: 2.00 KB (excludes 2 files of unknown size)", + ); }); it("does not report the size of a symlink target outside the allowed directories", async () => { @@ -1197,10 +1291,12 @@ describe("list_directory_with_sizes", () => { path: dir, }); expect(textOf(result)).toContain( - `[FILE] ${"link-out".padEnd(30)} ${"0 B".padStart(10)}`, + `[FILE] ${"link-out".padEnd(30)} ${"unknown".padStart(10)}`, ); expect(textOf(result)).not.toContain("1.50 KB"); - expect(textOf(result)).toContain("Combined size: 2.00 KB"); + expect(textOf(result)).toContain( + "Combined size: 2.00 KB (excludes 1 file of unknown size)", + ); } finally { await fs.rm(outside, { recursive: true, force: true }); } diff --git a/src/filesystem/__tests__/stdio.test.ts b/src/filesystem/__tests__/stdio.test.ts index 196fa30f59..841e5c4432 100644 --- a/src/filesystem/__tests__/stdio.test.ts +++ b/src/filesystem/__tests__/stdio.test.ts @@ -129,7 +129,7 @@ describe("startup", () => { it("prints usage, and waits for roots, when started with no directories", async () => { await session.start([]); expect(stderrText()).toContain( - "Usage: mcp-server-filesystem [allowed-directory] [additional-directories...]", + "Usage: mcp-server-filesystem [--read-only] [--confine-roots] [allowed-directory] [additional-directories...]", ); expect(stderrText()).toContain( "At least one directory must be provided by EITHER method for the server to operate.", @@ -224,7 +224,13 @@ describe("the stdio transport", () => { session.send(7, "tools/list"); const response = await session.response(7); expect(response.error).toBeUndefined(); - expect(response.result?.tools).toHaveLength(14); + // Not a count: a test that pins the number breaks on every new tool; + // server-tools.test.ts pins the full list. + expect(response.result?.tools).toEqual( + expect.arrayContaining([ + expect.objectContaining({ name: "list_allowed_directories" }), + ]), + ); }); // #4206: a malformed line is reported to the transport's onerror (unset, @@ -248,7 +254,11 @@ describe("the stdio transport", () => { message: "Method not found", }); session.send(3, "tools/list"); - expect((await session.response(3)).result?.tools).toHaveLength(14); + expect((await session.response(3)).result?.tools).toEqual( + expect.arrayContaining([ + expect.objectContaining({ name: "list_allowed_directories" }), + ]), + ); }); }); @@ -275,3 +285,76 @@ describe("isEntryPoint", () => { expect(isEntryPoint(undefined, moduleUrl)).toBe(false); }); }); + +// #5320: a command-line directory whose name ends in a space or quote is +// allowed under that exact name. Trimming it, in its given form or its +// realpath, names the sibling without the suffix and allows that instead. +// Windows refuses or strips these characters in names, so POSIX only. +describe.skipIf(process.platform === "win32").each([ + ["a space", " "], + ["a single quote", "'"], + ["a double quote", '"'], +])("an allowed directory whose name ends in %s (#5320)", (_, suffix) => { + let allowed: string; + let sibling: string; + + beforeEach(async () => { + allowed = path.join(dir, `project${suffix}`); + sibling = path.join(dir, "project"); + await fs.mkdir(allowed); + await fs.mkdir(sibling); + await fs.writeFile(path.join(allowed, "mine.txt"), "allowed"); + await fs.writeFile(path.join(sibling, "secret.txt"), "not allowed"); + }); + + async function readOver(args: string[], file: string) { + await session.start(args); + await session.initialize(); + session.send(2, "tools/call", { + name: "list_allowed_directories", + arguments: {}, + }); + session.send(3, "tools/call", { + name: "read_text_file", + arguments: { path: file }, + }); + return { + listed: (await session.response(2)).result?.structuredContent, + read: (await session.response(3)).result, + }; + } + + it("lists it untrimmed and refuses the sibling, given directly", async () => { + const { listed, read } = await readOver( + [allowed], + path.join(sibling, "secret.txt"), + ); + expect(listed).toEqual({ content: `Allowed directories:\n${allowed}` }); + expect(read?.isError).toBe(true); + expect(JSON.stringify(read?.content)).toMatch( + /Access denied - path outside allowed directories/, + ); + }); + + it("keeps the realpath untrimmed and refuses the sibling, given through a symlink", async () => { + const link = path.join(dir, "link"); + await fs.symlink(allowed, link); + const { listed, read } = await readOver( + [link], + path.join(sibling, "secret.txt"), + ); + expect(listed).toEqual({ + content: `Allowed directories:\n${link}\n${allowed}`, + }); + expect(read?.isError).toBe(true); + expect(JSON.stringify(read?.content)).toMatch( + /Access denied - path outside allowed directories/, + ); + }); + + it("still reads the allowed directory's own file", async () => { + const { read } = await readOver([allowed], path.join(allowed, "mine.txt")); + expect(read?.isError).toBeFalsy(); + expect(read?.structuredContent).toEqual({ content: "allowed" }); + }); +}); diff --git a/src/filesystem/__tests__/tool-annotations.test.ts b/src/filesystem/__tests__/tool-annotations.test.ts new file mode 100644 index 0000000000..fe768465de --- /dev/null +++ b/src/filesystem/__tests__/tool-annotations.test.ts @@ -0,0 +1,84 @@ +// Protocol-level invariants on the tool annotations every filesystem tool +// advertises in tools/list (#3402). server-tools.test.ts pins each tool's exact +// hints; this file holds the rules any tool, existing or added later, must +// keep: all four hints set explicitly (the spec's defaults for a missing hint +// are the pessimistic ones, so leaving one out misdescribes the tool), a +// read-only tool idempotent and never destructive, and no tool open-world, +// since every tool is confined to the allowed directories. + +import fs from "fs/promises"; +import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; +import type { Tool } from "@modelcontextprotocol/client"; +import { + connect, + makeTempDir, + quietStderr, + type Connected, +} from "./helpers.js"; + +let dir: string; +let conn: Connected; +let tools: Tool[]; + +beforeEach(async () => { + // The server logs its roots decision to stderr on every connect. + quietStderr(); + dir = await makeTempDir("mcp-fs-annotations-"); + conn = await connect([dir]); + ({ tools } = await conn.client.listTools()); +}); + +afterEach(async () => { + vi.restoreAllMocks(); + await conn.close(); + await fs.rm(dir, { recursive: true, force: true }); +}); + +describe("tool annotations (#3402)", () => { + it("sets all four hints, as booleans, on every tool", () => { + expect(tools.length).toBeGreaterThan(0); + for (const tool of tools) { + const a = tool.annotations; + expect( + { + readOnlyHint: typeof a?.readOnlyHint, + idempotentHint: typeof a?.idempotentHint, + destructiveHint: typeof a?.destructiveHint, + openWorldHint: typeof a?.openWorldHint, + }, + tool.name, + ).toEqual({ + readOnlyHint: "boolean", + idempotentHint: "boolean", + destructiveHint: "boolean", + openWorldHint: "boolean", + }); + } + }); + + it("marks every read-only tool idempotent and not destructive", () => { + const readOnly = tools.filter((t) => t.annotations?.readOnlyHint === true); + expect(readOnly.length).toBeGreaterThan(0); + for (const tool of readOnly) { + expect(tool.annotations, tool.name).toMatchObject({ + idempotentHint: true, + destructiveHint: false, + }); + } + }); + + it("marks no tool open-world", () => { + for (const tool of tools) { + expect(tool.annotations?.openWorldHint, tool.name).toBe(false); + } + }); + + it("keeps the edits that compound non-idempotent", () => { + // Re-running edit_file or move_file with the same arguments does not + // leave the same state (the edit no longer matches; the source is gone), + // so neither may be advertised as safe to retry. + const byName = new Map(tools.map((t) => [t.name, t.annotations])); + expect(byName.get("edit_file")?.idempotentHint).toBe(false); + expect(byName.get("move_file")?.idempotentHint).toBe(false); + }); +}); diff --git a/src/filesystem/__tests__/win32.test.ts b/src/filesystem/__tests__/win32.test.ts index 4042d1cabe..a9fa9dc65c 100644 --- a/src/filesystem/__tests__/win32.test.ts +++ b/src/filesystem/__tests__/win32.test.ts @@ -3,6 +3,8 @@ // Windows-only branches of path-utils.ts and path-validation.ts run here: // drive roots, bare drive letters, backslash conversion and UNC shares. +import { promises as fsp, type Stats } from "fs"; +import os from "os"; import { afterAll, beforeAll, describe, expect, it, vi } from "vitest"; vi.mock("path", async () => { @@ -10,7 +12,8 @@ vi.mock("path", async () => { return { ...actual.win32, default: actual.win32 }; }); -const { normalizePath } = await import("../path-utils.js"); +const { normalizePath, expandHome } = await import("../path-utils.js"); +const { getValidRootDirectories } = await import("../roots-utils.js"); const { isPathWithinAllowedDirectories } = await import("../path-validation.js"); @@ -141,3 +144,43 @@ describe("isPathWithinAllowedDirectories on win32", () => { ).toBe(false); }); }); + +// Windows users write a home-relative path with a backslash (#5180). The +// home directory is stubbed to a Windows spelling, since the host's is POSIX. +describe("~\\ on win32", () => { + const home = "C:\\Users\\me"; + + it("expands ~\\ to the home directory, as ~/ is", () => { + const spy = vi.spyOn(os, "homedir").mockReturnValue(home); + try { + expect(expandHome("~\\projects")).toBe("C:\\Users\\me\\projects"); + expect(expandHome("~\\projects")).toBe(expandHome("~/projects")); + } finally { + spy.mockRestore(); + } + }); + + it("leaves a ~ that does not start a segment alone", () => { + expect(expandHome("~user\\projects")).toBe("~user\\projects"); + expect(expandHome("C:\\~\\x")).toBe("C:\\~\\x"); + }); + + it("expands a ~\\ client Root through the same expandHome", async () => { + const spies = [ + vi.spyOn(os, "homedir").mockReturnValue(home), + // The directory does not exist on this host: realpath and stat answer + // as Windows would for an existing directory. + vi.spyOn(fsp, "realpath").mockImplementation(async (p) => String(p)), + vi + .spyOn(fsp, "stat") + .mockResolvedValue({ isDirectory: () => true } as Stats), + ]; + try { + expect(await getValidRootDirectories([{ uri: "~\\projects" }])).toEqual([ + "C:\\Users\\me\\projects", + ]); + } finally { + for (const spy of spies) spy.mockRestore(); + } + }); +}); diff --git a/src/filesystem/allowed-directories.ts b/src/filesystem/allowed-directories.ts new file mode 100644 index 0000000000..45627d46b8 --- /dev/null +++ b/src/filesystem/allowed-directories.ts @@ -0,0 +1,80 @@ +// How the filesystem server reports its allow-list to the operator (#4912), +// and how --confine-roots keeps a client's Roots inside it (#3133). +// The allowed directories in force can come from the command line or, on a +// 2025-era connection, from the client's Roots, and they change at runtime as +// the client's Roots do. index.ts logs them at startup and server.ts after +// every change, both through describeAllowedDirectories, so the message reads +// the same wherever it is printed and an operator can tell the effective +// access state and its source from stderr alone, without calling a tool. +// confineRootsToDirectories is a pure function of the two lists, so the +// policy is testable apart from the roots/list exchange that feeds it. + +import { isPathWithinAllowedDirectories } from "./path-validation.js"; + +/** + * Where the allowed directories in force came from (#4912): the command line, + * or the client's Roots, which on a 2025-era connection replace the + * command-line directories (`replacedCommandLine` says whether there were any + * to replace). + */ +export type AllowedDirectoriesSource = + | { from: "command-line" } + | { from: "roots"; replacedCommandLine: boolean; confined?: boolean }; + +/** + * The one-message summary of the allowed directories in force, logged to + * stderr at startup and after every change, so an operator can read the + * effective access state and its source without calling a tool (#4912). Each + * directory is on a line of its own, since a path may contain a comma. + */ +export function describeAllowedDirectories( + directories: string[], + source: AllowedDirectoriesSource, +): string { + if (directories.length === 0) { + return source.from === "roots" + ? "Effective allowed directories: none. The client's Roots name no valid directory, so every path is refused until it exposes one again" + : "Effective allowed directories: none. No command-line directories and no valid client Roots, so every path is refused"; + } + const origin = + source.from === "command-line" + ? "from the command line" + : source.confined + ? "from the client's Roots, confined to the command-line directories" + : source.replacedCommandLine + ? "from the client's Roots, which replaced the command-line directories" + : "from the client's Roots"; + return [ + `Effective allowed directories (${origin}):`, + ...directories.map((dir) => ` ${dir}`), + ].join("\n"); +} + +/** + * The client's Roots confined to the operator's command-line directories + * (`--confine-roots`, #3133): the overlap of the two lists, so a client's + * Roots can narrow the command-line directories but never widen them. A root + * inside a command-line directory is kept; a root that contains one is + * narrowed to that directory; a root that overlaps none is ignored, and + * returned in `ignored` so the caller can log it. Both lists hold resolved, + * normalized directories (each spelling of a symlinked one), and every request + * is still realpath'd and checked against the result, so a root's advertised + * spelling kept here cannot reach past its target. + */ +export function confineRootsToDirectories( + rootDirectories: string[], + commandLineDirectories: string[], +): { confined: string[]; ignored: string[] } { + const confined = new Set(); + const ignored: string[] = []; + for (const root of rootDirectories) { + const inside = isPathWithinAllowedDirectories(root, commandLineDirectories); + const contained = commandLineDirectories.filter((dir) => + isPathWithinAllowedDirectories(dir, [root]), + ); + if (inside) confined.add(root); + for (const dir of contained) confined.add(dir); + if (!inside && contained.length === 0) ignored.push(root); + } + return { confined: [...confined], ignored }; +} diff --git a/src/filesystem/file-resources.ts b/src/filesystem/file-resources.ts new file mode 100644 index 0000000000..cc4bc8d3a4 --- /dev/null +++ b/src/filesystem/file-resources.ts @@ -0,0 +1,247 @@ +// Files as MCP resources (#297). The server exposes one resource template, +// `file://{+path}`, so a client can read any file inside the allowed +// directories with resources/read and get it back as text or as a base64 blob +// with its MIME type, instead of only through the read tools. A resource is +// read by the client application, which can put it in model context or show it +// to the user, rather than returned inside a tool result the model must relay. +// +// The template has no list callback, on purpose: resources/list stays empty, so +// the server never enumerates the disk. A client reaches a file by building its +// URI from the template (or from a path a tool returned), and every read goes +// through the same validatePath as the tools, so a resource can reach nothing a +// tool cannot: no path outside the allowed directories, and no symlink whose +// target is outside them. +// +// It lives in a module of its own, registered from createServer() with one +// call, so the server factory's tool code is untouched. The factory hands it +// the live allow-list, the roots barrier and the reason a 2026-07-28 +// connection with no directories cannot operate, so a read sees the same +// directories, at the same moment, as a tool call would. +import fs from "fs/promises"; +import path from "path"; +import { fileURLToPath } from "url"; +import { + McpServer, + ProtocolError, + ProtocolErrorCode, + ResourceNotFoundError, + ResourceTemplate, +} from "@modelcontextprotocol/server"; +import type { ReadResourceResult } from "@modelcontextprotocol/server"; +import { FsTimeoutError, validatePath } from "./lib.js"; + +/** The URI template every file resource matches (RFC 6570 reserved expansion). */ +export const FILE_RESOURCE_TEMPLATE = "file://{+path}"; + +/** MIME types by lowercase extension; anything else is sniffed (see below). */ +const MIME_TYPES: Readonly> = { + ".txt": "text/plain", + ".md": "text/markdown", + ".csv": "text/csv", + ".html": "text/html", + ".htm": "text/html", + ".css": "text/css", + ".js": "text/javascript", + ".mjs": "text/javascript", + ".ts": "text/typescript", + ".json": "application/json", + ".xml": "application/xml", + ".yaml": "application/yaml", + ".yml": "application/yaml", + ".svg": "image/svg+xml", + ".png": "image/png", + ".jpg": "image/jpeg", + ".jpeg": "image/jpeg", + ".gif": "image/gif", + ".webp": "image/webp", + ".bmp": "image/bmp", + ".ico": "image/x-icon", + ".mp3": "audio/mpeg", + ".wav": "audio/wav", + ".ogg": "audio/ogg", + ".flac": "audio/flac", + ".mp4": "video/mp4", + ".pdf": "application/pdf", + ".zip": "application/zip", + ".gz": "application/gzip", +}; + +/** MIME types outside text/* whose content is still text. */ +const TEXTUAL_MIME_TYPES = new Set([ + "application/json", + "application/xml", + "application/yaml", + "image/svg+xml", +]); + +function isTextualMimeType(mimeType: string): boolean { + return mimeType.startsWith("text/") || TEXTUAL_MIME_TYPES.has(mimeType); +} + +/** The bytes as UTF-8 text, or undefined when they are not valid UTF-8 text. */ +function decodeText(bytes: Buffer): string | undefined { + // A NUL byte is valid UTF-8 but marks binary content in practice. + if (bytes.includes(0)) return undefined; + try { + return new TextDecoder("utf-8", { fatal: true }).decode(bytes); + } catch { + return undefined; + } +} + +/** + * The contents of one file resource: text when its type is textual (or, for an + * unknown extension, when its bytes decode as UTF-8), a base64 blob otherwise. + */ +export function fileResourceContents( + uri: string, + filePath: string, + bytes: Buffer, +): ReadResourceResult["contents"][number] { + const known = MIME_TYPES[path.extname(filePath).toLowerCase()]; + if (known === undefined || isTextualMimeType(known)) { + const text = decodeText(bytes); + if (text !== undefined) { + return { uri, mimeType: known ?? "text/plain", text }; + } + } + return { + uri, + mimeType: known ?? "application/octet-stream", + blob: bytes.toString("base64"), + }; +} + +/** The errno codes that mean a path does not exist. */ +const MISSING_PATH_CODES = new Set(["ENOENT", "ENOTDIR"]); + +/** + * Whether `error` says the path does not exist, directly or through its + * cause: validatePath wraps a missing parent's ENOENT in an Error of its own. + */ +function isMissingPathError(error: unknown): boolean { + for (let e = error; e instanceof Error; e = e.cause) { + const code = (e as NodeJS.ErrnoException).code; + if (code !== undefined && MISSING_PATH_CODES.has(code)) return true; + } + return false; +} + +/** + * Whether `error` is the filesystem failing (an errno error, or a call that + * timed out) rather than validatePath refusing the path, which it does with a + * plain Error and no errno code. + */ +function isOperationalError(error: unknown): boolean { + return ( + error instanceof FsTimeoutError || + (error instanceof Error && + (error as NodeJS.ErrnoException).code !== undefined) + ); +} + +export interface FileResourceContext { + /** The directories reads are confined to, read at the moment of each read. */ + allowedDirectories: () => readonly string[]; + /** + * The current roots barrier: settles when no roots refresh is pending. It + * is read again after it settles, since a newer refresh can replace it. + */ + rootsSettled: () => Promise; + /** Why the server cannot serve a read at all right now, if it cannot. */ + unavailableReason: () => string | undefined; +} + +/** Register the `file://{+path}` resource template on `server`. */ +export function registerFileResources( + server: McpServer, + context: FileResourceContext, +): void { + server.registerResource( + "file", + new ResourceTemplate(FILE_RESOURCE_TEMPLATE, { list: undefined }), + { + title: "File", + description: + "A file inside the allowed directories, by its absolute file:// URI " + + "(for example file:///home/user/project/README.md). Text files are " + + "returned as text, other files as a base64 blob with their MIME type. " + + "Directories are not resources; use the list_directory tool to find files.", + // A file can change at any time, and its contents belong to this user. + cacheHint: { ttlMs: 0, cacheScope: "private" }, + }, + async (uri): Promise => { + // Wait out a pending roots refresh, as tool calls do, so a read is + // checked against the directories that refresh brings in. + let settled: Promise; + do { + settled = context.rootsSettled(); + await settled; + } while (settled !== context.rootsSettled()); + + const unavailable = context.unavailableReason(); + if (unavailable !== undefined) { + throw new ProtocolError(ProtocolErrorCode.InvalidParams, unavailable); + } + + let requestedPath: string; + try { + requestedPath = fileURLToPath(uri); + } catch (error) { + throw new ProtocolError( + ProtocolErrorCode.InvalidParams, + `Invalid file URI ${uri.href}: ${(error as Error).message}`, + ); + } + + let validPath: string; + try { + validPath = await validatePath( + requestedPath, + context.allowedDirectories(), + ); + } catch (error) { + // A path that does not exist (a dangling symlink, a missing parent, a + // path beneath a regular file) is a missing resource, so a client can + // recognise it by the URI in the error data. + if (isMissingPathError(error)) { + throw new ResourceNotFoundError(uri.href, (error as Error).message); + } + // The filesystem failed while resolving the path (an I/O error, a + // timeout): the request was fine, so it is the server's error. + if (isOperationalError(error)) { + throw new ProtocolError( + ProtocolErrorCode.InternalError, + `Failed to resolve ${uri.href}: ${(error as Error).message}`, + ); + } + // Outside the allowed directories, or a symlink out of them: the + // message says which, as the tools do. + throw new ProtocolError( + ProtocolErrorCode.InvalidParams, + (error as Error).message, + ); + } + + let bytes: Buffer; + try { + bytes = await fs.readFile(validPath); + } catch (error) { + if (isMissingPathError(error)) { + throw new ResourceNotFoundError(uri.href); + } + if ((error as NodeJS.ErrnoException).code === "EISDIR") { + throw new ResourceNotFoundError( + uri.href, + `Not a file: ${uri.href} is a directory; directories are not resources`, + ); + } + throw new ProtocolError( + ProtocolErrorCode.InternalError, + `Failed to read ${uri.href}: ${(error as Error).message}`, + ); + } + return { contents: [fileResourceContents(uri.href, validPath, bytes)] }; + }, + ); +} diff --git a/src/filesystem/index.ts b/src/filesystem/index.ts index 160948c18b..7e2fd762c1 100644 --- a/src/filesystem/index.ts +++ b/src/filesystem/index.ts @@ -1,6 +1,7 @@ #!/usr/bin/env node // The stdio entry point of the filesystem server (the package's bin). It -// resolves the allowed directories named on the command line and serves +// parses the command line (its options, such as --read-only, and the allowed +// directories), resolves the allowed directories and serves // createServer() on stdio through the SDK's serveStdio entry, which answers // 2026-07-28 clients (server/discover, no handshake) and 2025-era clients // (initialize, then Roots) from the same factory (#4852). Everything runs inside @@ -19,8 +20,13 @@ import { StdioServerTransport, type StdioServerHandle, } from "@modelcontextprotocol/server/stdio"; -import { normalizePath, expandHome, realpathWithRetry } from "./path-utils.js"; +import { + normalizePathFormat, + expandHome, + realpathWithRetry, +} from "./path-utils.js"; import { createServer } from "./server.js"; +import { describeAllowedDirectories } from "./allowed-directories.js"; /** * Resolve the allowed directories given on the command line, warning about and @@ -39,12 +45,18 @@ export async function resolveAllowedDirectories( args.map(async (dir) => { const expanded = expandHome(dir); const absolute = path.resolve(expanded); - const normalizedOriginal = normalizePath(absolute); + // normalizePathFormat, not normalizePath, for both forms: a trailing + // space or quote is part of a POSIX filename, so trimming it would + // allow the sibling that lacks it (#5320). Trimming after + // path.resolve could only ever remove a trailing character, never + // shell-quoting leftovers at the start, so the argument is taken + // exactly as the shell passed it. + const normalizedOriginal = normalizePathFormat(absolute); try { // Security: Resolve symlinks in allowed directories during startup // This ensures we know the real paths and can validate against them later const resolved = await realpathWithRetry(absolute); - const normalizedResolved = normalizePath(resolved); + const normalizedResolved = normalizePathFormat(resolved); // Return both original and resolved paths if they differ // This allows matching against either /tmp or /private/tmp on macOS if (normalizedOriginal !== normalizedResolved) { @@ -84,6 +96,43 @@ export async function resolveAllowedDirectories( return accessibleDirectories; } +/** What the command line asks for: the directories, and the options. */ +export interface ParsedArgs { + /** The allowed directories, as given (resolveAllowedDirectories resolves them). */ + directories: string[]; + /** `--read-only`: serve only the tools that cannot change anything (#632). */ + readOnly: boolean; + /** `--confine-roots`: keep a client's Roots inside the directories (#3133). */ + confineRoots: boolean; +} + +/** + * Split the command line into options and allowed directories. An option may + * appear anywhere; `--` ends the options, so a directory whose name is an + * option's can still be given after it. Any other argument is a directory, as + * every argument was before options existed. + */ +export function parseArgs(args: string[]): ParsedArgs { + const parsed: ParsedArgs = { + directories: [], + readOnly: false, + confineRoots: false, + }; + let options = true; + for (const arg of args) { + if (options && arg === "--") { + options = false; + } else if (options && arg === "--read-only") { + parsed.readOnly = true; + } else if (options && arg === "--confine-roots") { + parsed.confineRoots = true; + } else { + parsed.directories.push(arg); + } + } + return parsed; +} + /** * Serve the server over stdio with the allowed directories in `args`. The * first message the client sends picks the connection's protocol era, and the @@ -95,9 +144,10 @@ export async function main( stdin: Readable = process.stdin, stdout: Writable = process.stdout, ): Promise { - if (args.length === 0) { + const { directories, readOnly, confineRoots } = parseArgs(args); + if (directories.length === 0) { console.error( - "Usage: mcp-server-filesystem [allowed-directory] [additional-directories...]", + "Usage: mcp-server-filesystem [--read-only] [--confine-roots] [allowed-directory] [additional-directories...]", ); console.error("Note: Allowed directories can be provided via:"); console.error(" 1. Command-line arguments (shown above)"); @@ -109,7 +159,7 @@ export async function main( ); } - const allowedDirectories = await resolveAllowedDirectories(args); + const allowedDirectories = await resolveAllowedDirectories(directories); // With no directories and a 2025-era client without Roots, the server // closes the connection after initialize (#4992); exit non-zero so the // failure shows. @@ -117,6 +167,8 @@ export async function main( ({ era }) => createServer(allowedDirectories, { era, + readOnly, + confineRoots, onCannotOperate: () => process.exit(1), }), { @@ -131,10 +183,26 @@ export async function main( }, ); console.error("Secure MCP Filesystem Server running on stdio"); + if (readOnly) { + console.error( + "Read-only mode: only tools that cannot change the filesystem are served", + ); + } + if (confineRoots && allowedDirectories.length === 0) { + console.error( + "Warning: --confine-roots has no effect without command-line directories; a client's Roots will set the allowed directories", + ); + } if (allowedDirectories.length === 0) { console.error( "Started without allowed directories - waiting for a 2025-era client to provide roots via MCP protocol", ); + } else { + // The directories in force until a 2025-era client's Roots replace them; + // on 2026-07-28 they are final (#4912). + console.error( + describeAllowedDirectories(allowedDirectories, { from: "command-line" }), + ); } return handle; } diff --git a/src/filesystem/lib.ts b/src/filesystem/lib.ts index 775145e708..335ee27142 100644 --- a/src/filesystem/lib.ts +++ b/src/filesystem/lib.ts @@ -7,9 +7,11 @@ import { minimatch } from "minimatch"; import { normalizePathFormat, expandHome } from "./path-utils.js"; import { isPathWithinAllowedDirectories } from "./path-validation.js"; import { + appendFileInDirectory, createDirectoryInPlace, createFileInDirectory, moveFileAcrossDevices, + copyFileInDirectory, overwriteFileInDirectory, renameIntoDirectory, } from "./pinned-directory.js"; @@ -28,6 +30,12 @@ interface FileInfo { export interface SearchOptions { excludePatterns?: string[]; + /** + * Called for each subdirectory below the starting path that could not be + * read (EACCES, say). The walk skips it and goes on, so the matches it + * returns are partial; this is how the caller finds out (#5189). + */ + onUnreadable?: (directory: string, error: unknown) => void; } export interface SearchResult { @@ -429,7 +437,21 @@ async function resolveUnicodeEquivalentPath( return path.join(currentPath, ...relativeParts.slice(index)); } - currentPath = await realpathWithTimeout(candidate); + try { + currentPath = await realpathWithTimeout(candidate); + } catch (error) { + // The entry exists (lstat found it just above) but realpath cannot + // follow it, so it is a symlink whose target is missing. Name it: a bare + // ENOENT reaches validatePath's catch, which would report its parent, + // a directory that plainly exists, as missing (#5191). + if ((error as NodeJS.ErrnoException).code === "ENOENT") { + throw new Error( + `Broken symlink: ${candidate} points to a target that does not exist`, + { cause: error }, + ); + } + throw error; + } if ( !isPathWithinAllowedDirectories( normalizePathFormat(currentPath), @@ -615,6 +637,38 @@ async function overwriteInPlace( await overwriteFileInDirectory(filePath, content, expected); } +/** + * Append `content` to the file at `filePath`, exactly as given: no newline is + * added before or after it (#1208). A file that does not exist yet is created + * with `content`, the way write_file creates one. + * + * Security: the same path as writeFileContent. Exclusive creation never + * follows a symlink; an existing entry must be a regular file, and is appended + * to through a handle checked to be that very file (see appendFileInDirectory + * in pinned-directory.ts), so nothing swapped in after validation is written + * through. + */ +export async function appendFileContent( + filePath: string, + content: string, +): Promise { + try { + // O_APPEND here too: another append can reach the file between its + // creation and this write, and must not be overwritten. + await createFileInDirectory(filePath, content, true); + return; + } catch (error) { + if ((error as NodeJS.ErrnoException).code !== "EEXIST") { + throw error; + } + } + const expected = await fs.lstat(filePath, { bigint: true }); + if (!expected.isFile()) { + throw new Error(`Refusing to append: not a regular file: ${filePath}`); + } + await appendFileInDirectory(filePath, content, expected); +} + /** Create a directory and any missing parents (see pinned-directory.ts). */ export async function createDirectory(directoryPath: string): Promise { createDirectoryInPlace(directoryPath); @@ -650,6 +704,29 @@ export async function moveFile( throw new Error(`Destination already exists: ${destinationPath}`); } +/** + * Copy the regular file at `sourcePath` to `destinationPath` (#519). Like + * move_file, it never overwrites: anything at the destination (a file, a + * directory or a symlink, which lstat sees rather than follows) fails it, and + * the exclusive create in copyFileInDirectory refuses one that appears in + * between. Directories are not copied. + */ +export async function copyFile( + sourcePath: string, + destinationPath: string, +): Promise { + try { + await fs.lstat(destinationPath); + } catch (error) { + if ((error as NodeJS.ErrnoException).code === "ENOENT") { + await copyFileInDirectory(sourcePath, destinationPath); + return; + } + throw error; + } + throw new Error(`Destination already exists: ${destinationPath}`); +} + // File Editing Functions interface FileEdit { oldText: string; @@ -729,6 +806,180 @@ function reindentLine( return indent + body; } +// What the no-match error shows (#2034). A model that sent oldText which +// differs from the file in a way the matcher does not tolerate (whitespace +// inside a line, a typo, stale content) needs the file's real text to retry +// with, so the error quotes the closest run of lines with their line +// numbers. Finding it compares oldText against the lines at every position +// of the file, so it is skipped past a fixed budget of bigram lookups rather +// than stalling the error on a huge file or a huge oldText. +const MISMATCH_WORK_LIMIT = 10_000_000; +const MISMATCH_CONTEXT_LINES = 2; +const MISMATCH_MAX_REGION_LINES = 30; +// A quoted file line is cut at this many characters, so a long line (minified +// code, one-line JSON) cannot blow the error up to the size of the file. +const MISMATCH_MAX_LINE_CHARS = 500; + +interface LineGrams { + grams: Map; + size: number; +} + +/** A line's character bigrams with all whitespace removed, for similarity. */ +function lineGrams(line: string): LineGrams { + const text = line.replace(/\s+/g, ""); + const grams = new Map(); + if (text.length === 1) grams.set(text, 1); + for (let i = 0; i < text.length - 1; i++) { + const gram = text.slice(i, i + 2); + grams.set(gram, (grams.get(gram) ?? 0) + 1); + } + let size = 0; + for (const count of grams.values()) size += count; + return { grams, size }; +} + +/** Dice similarity of two lines' bigrams: 1 for two blank lines, 0 for nothing shared. */ +function lineSimilarity(a: LineGrams, b: LineGrams): number { + if (a.size + b.size === 0) return 1; + let shared = 0; + for (const [gram, count] of a.grams) { + shared += Math.min(count, b.grams.get(gram) ?? 0); + } + return (2 * shared) / (a.size + b.size); +} + +/** + * The error message for an edit that matched nowhere in `content` (the file + * as the earlier edits in the same call left it). It names the edit, says + * what the matcher tolerates, and quotes the run of file lines most similar + * to oldText, numbered, with the first line that differs shown quoted so + * whitespace is visible. It only reports: it never changes what matches. + */ +export function describeEditMismatch( + content: string, + oldText: string, + editIndex: number, + editCount: number, +): string { + const oldLines = normalizeLineEndings(oldText).split("\n"); + // The lines as the matcher sees them, so a terminal newline in oldText + // lines up with the file's. The empty element after a terminal newline is + // not a line of the file, so it is left out of what is counted and shown. + const fileLines = content.split("\n"); + const lineCount = content.endsWith("\n") + ? fileLines.length - 1 + : content === "" + ? 0 + : fileLines.length; + + const parts = [ + `Could not find exact match for edit ${editIndex + 1} of ${editCount}. ` + + "oldText must match the file's text exactly; only line endings, and " + + "leading and trailing whitespace on whole lines, may differ. " + + "No edits were applied.", + `oldText:\n${oldText}`, + ]; + + if (lineCount === 0) { + parts.push("The file is empty."); + return parts.join("\n\n"); + } + + const oldGrams = oldLines.map(lineGrams); + let oldWork = 0; + for (const line of oldGrams) oldWork += line.grams.size + 1; + if (fileLines.length * oldWork > MISMATCH_WORK_LIMIT) { + parts.push( + `The file has ${lineCount} lines, too many to search for the closest ` + + "text to this oldText. Read the file and copy oldText from it exactly.", + ); + return parts.join("\n\n"); + } + + // Every starting line is scored, including those near the end where fewer + // lines than oldText has remain, so a near match at the end of the file is + // still found. The earliest start wins a tie. A start is always a real + // line, never the empty element after a terminal newline. + const fileGrams = fileLines.map(lineGrams); + let best = 0; + let bestScore = 0; + for (let i = 0; i < lineCount; i++) { + const width = Math.min(oldLines.length, fileLines.length - i); + let score = 0; + for (let j = 0; j < width; j++) { + score += lineSimilarity(oldGrams[j], fileGrams[i + j]); + } + if (score > bestScore) { + best = i; + bestScore = score; + } + } + + if (bestScore === 0) { + parts.push( + "No line of the file resembles oldText. Read the file and copy " + + "oldText from it exactly.", + ); + return parts.join("\n\n"); + } + + const start = Math.min( + Math.max(0, best - MISMATCH_CONTEXT_LINES), + lineCount - 1, + ); + const end = Math.min( + lineCount, + best + oldLines.length + MISMATCH_CONTEXT_LINES, + start + MISMATCH_MAX_REGION_LINES, + ); + let clipped = false; + const clip = (line: string): string => { + if (line.length <= MISMATCH_MAX_LINE_CHARS) return line; + clipped = true; + return ( + line.slice(0, MISMATCH_MAX_LINE_CHARS) + + ` [... cut: the line has ${line.length} characters]` + ); + }; + const numbered = fileLines + .slice(start, end) + .map((line, k) => `${start + k + 1}: ${clip(line)}`); + const after = editIndex > 0 ? ", after the earlier edits in this call" : ""; + parts.push( + `Closest text in the file (lines ${start + 1}-${end} of ` + + `${lineCount}${after}):\n${numbered.join("\n")}`, + ); + + // Some line differs beyond leading and trailing whitespace, or oldText runs + // past the file's end: otherwise the line-by-line matcher would have + // matched here. Math.max only keeps the index a valid one. + const differs = Math.max( + 0, + oldLines.findIndex( + (line, j) => + best + j >= lineCount || line.trim() !== fileLines[best + j].trim(), + ), + ); + parts.push( + best + differs >= lineCount + ? `oldText line ${differs + 1} (${JSON.stringify(clip(oldLines[differs]))}) ` + + `runs past the end of the file, which ends at line ${lineCount}.` + : `First difference, oldText line ${differs + 1} against file line ` + + `${best + differs + 1}:\n` + + ` oldText: ${JSON.stringify(clip(oldLines[differs]))}\n` + + ` file: ${JSON.stringify(clip(fileLines[best + differs]))}`, + ); + parts.push( + clipped + ? "Some lines above were cut. Read the file to copy oldText from it " + + "exactly." + : "Retry with oldText copied exactly from the file, without the " + + "line-number prefixes.", + ); + return parts.join("\n\n"); +} + export async function applyFileEdits( filePath: string, edits: FileEdit[], @@ -742,7 +993,7 @@ export async function applyFileEdits( // Apply edits sequentially let modifiedContent = content; - for (const edit of edits) { + for (const [index, edit] of edits.entries()) { const normalizedOld = normalizeLineEndings(edit.oldText); const normalizedNew = normalizeLineEndings(edit.newText); @@ -798,7 +1049,14 @@ export async function applyFileEdits( } if (!matchFound) { - throw new Error(`Could not find exact match for edit:\n${edit.oldText}`); + throw new Error( + describeEditMismatch( + modifiedContent, + edit.oldText, + index, + edits.length, + ), + ); } } @@ -833,7 +1091,8 @@ export async function tailFile( const stats = await fs.stat(filePath); const fileSize = stats.size; - if (fileSize === 0) return ""; + // Zero lines is an empty result; slice(-0) below would return every line. + if (fileSize === 0 || numLines === 0) return ""; // Open file for reading const fileHandle = await fs.open(filePath, "r"); @@ -930,12 +1189,25 @@ export async function searchFilesWithValidation( allowedDirectories: readonly string[], options: SearchOptions = {}, ): Promise { - const { excludePatterns = [] } = options; + const { excludePatterns = [], onUnreadable } = options; const results: string[] = []; const visit = createEntryBudget("search_files", rootPath); async function search(currentPath: string) { - const entries = await readdirWithTimeout(currentPath); + let entries: Dirent[]; + try { + entries = await readdirWithTimeout(currentPath); + } catch (error) { + // The starting path failing is the request failing, and a timeout ends + // the whole search (see FsTimeoutError). A subdirectory that cannot be + // read is skipped, but reported to the caller, so the result never reads + // as complete when it is not (#5189). + if (currentPath === rootPath || error instanceof FsTimeoutError) { + throw error; + } + onUnreadable?.(currentPath, error); + return; + } for (const entry of entries) { const fullPath = path.join(currentPath, entry.name); @@ -951,29 +1223,27 @@ export async function searchFilesWithValidation( visit(); + // Only validation is guarded: an entry refused by validatePath (outside + // the allowed directories, a dangling link) is skipped silently. A + // timeout still ends the whole search rather than returning partial + // results as if they were complete. try { await validatePath(fullPath, allowedDirectories); - - // Use glob matching for the search pattern - if (minimatch(relativePath, pattern, { dot: true })) { - results.push(fullPath); - } - - if (entry.isDirectory()) { - await search(fullPath); - } } catch (error) { - // A timeout or the entry cap ends the whole search with its error - // rather than returning partial results as if they were complete; any - // other failure (an entry refused or unreadable) skips that entry. - if ( - error instanceof FsTimeoutError || - error instanceof WalkLimitError - ) { + if (error instanceof FsTimeoutError) { throw error; } continue; } + + // Use glob matching for the search pattern + if (minimatch(relativePath, pattern, { dot: true })) { + results.push(fullPath); + } + + if (entry.isDirectory()) { + await search(fullPath); + } } } diff --git a/src/filesystem/package.json b/src/filesystem/package.json index 432fafe77c..c159303dd5 100644 --- a/src/filesystem/package.json +++ b/src/filesystem/package.json @@ -1,6 +1,6 @@ { "name": "@modelcontextprotocol/server-filesystem", - "version": "2.1.0", + "version": "2.2.0", "description": "MCP server for filesystem access", "license": "SEE LICENSE IN LICENSE", "mcpName": "io.github.modelcontextprotocol/server-filesystem", @@ -36,8 +36,7 @@ "dependencies": { "@modelcontextprotocol/server": "^2.3.1", "diff": "^8.0.3", - "glob": "^13.0.6", - "minimatch": "^10.0.1", + "minimatch": "^11.0.4", "zod": "^4.2.0" }, "devDependencies": { diff --git a/src/filesystem/path-utils.ts b/src/filesystem/path-utils.ts index 58a73b8cc9..987cf7a5b8 100644 --- a/src/filesystem/path-utils.ts +++ b/src/filesystem/path-utils.ts @@ -124,12 +124,18 @@ export function normalizePathFormat(p: string): string { } /** - * Expands home directory tildes in paths + * Expands home directory tildes in paths: a bare `~`, `~/…` and, on Windows + * only, `~\…` (#5180). On POSIX a backslash is an ordinary filename + * character, so `~\notes` there names a file and is left as it is. * @param filepath The path to expand * @returns Expanded path */ export function expandHome(filepath: string): string { - if (filepath.startsWith("~/") || filepath === "~") { + if ( + filepath === "~" || + filepath.startsWith("~/") || + (process.platform === "win32" && filepath.startsWith("~\\")) + ) { return path.join(os.homedir(), filepath.slice(1)); } return filepath; diff --git a/src/filesystem/pinned-directory.ts b/src/filesystem/pinned-directory.ts index 8b2e63ad6d..f0b137a3ff 100644 --- a/src/filesystem/pinned-directory.ts +++ b/src/filesystem/pinned-directory.ts @@ -229,10 +229,15 @@ function openInDirectory( }); } -/** Create `filePath`, which must not exist, and write `content` to it. */ +/** + * Create `filePath`, which must not exist, and write `content` to it. With + * `append`, the descriptor is opened O_APPEND, so an append that lands on the + * new file before this write does is kept rather than overwritten (#1208). + */ export async function createFileInDirectory( filePath: string, content: string, + append: boolean = false, ): Promise { // O_EXCL: an existing file or symlink fails with EEXIST, never followed. const { fd } = openInDirectory( @@ -240,6 +245,7 @@ export async function createFileInDirectory( fs.constants.O_WRONLY | fs.constants.O_CREAT | fs.constants.O_EXCL | + (append ? fs.constants.O_APPEND : 0) | noFollow(), false, ); @@ -280,6 +286,42 @@ export async function overwriteFileInDirectory( } } +/** + * Append `content` to the end of the regular file at `filePath`, which must be + * the file `expected` (its BigInt lstat) describes, through a single + * descriptor (#1208). The same checks as overwriteFileInDirectory guard the + * open; O_APPEND makes every write land at the file's end as it is at that + * moment, so a concurrent append is never overwritten. + */ +export async function appendFileInDirectory( + filePath: string, + content: string, + expected: EntryIdentity, +): Promise { + // O_WRONLY, so a write-only file (a 0222 log) can be appended to: append + // never reads. O_NONBLOCK keeps a FIFO swapped in from blocking the open + // (it fails with ENXIO without a reader; a regular file ignores the flag; + // it is undefined on Windows, where the lstat identity checks stand in). + const { fd, opened } = openInDirectory( + filePath, + fs.constants.O_WRONLY | + fs.constants.O_APPEND | + (fs.constants.O_NONBLOCK ?? 0) | + noFollow(), + onWindows(), + ); + try { + if (!opened.isFile() || !isSameEntry(expected, opened)) { + throw new Error( + `Refusing to append: ${filePath} was replaced while it was being opened`, + ); + } + await writeAll(fd, content); + } finally { + await close(fd); + } +} + /** * Rename `sourcePath` (absolute, outside this fix's scope) to the final * component of `destinationPath`, relative to its anchored directory. @@ -347,9 +389,13 @@ function keepModeAndTimes(fd: number, source: fs.BigIntStats): Promise { * Open the regular file at `sourcePath` for reading, relative to its anchored * directory. A directory, a symlink or anything else that is not a regular * file is refused: a symlink is not followed or recreated, since its target - * could name anything, and a directory is not copied recursively. + * could name anything, and a directory is not copied recursively. `operation` + * names the caller in those refusals: a cross-device move, or copy_file. */ -function openSourceInDirectory(sourcePath: string): { +function openSourceInDirectory( + sourcePath: string, + operation: "move" | "copy", +): { fd: number; opened: fs.BigIntStats; } { @@ -358,13 +404,18 @@ function openSourceInDirectory(sourcePath: string): { const found = fs.lstatSync(name, { bigint: true }); if (found.isDirectory()) { throw new Error( - `Cannot move a directory across filesystems: ${sourcePath}. ` + - "Create the directory at the destination and move its files one at a time.", + operation === "move" + ? `Cannot move a directory across filesystems: ${sourcePath}. ` + + "Create the directory at the destination and move its files one at a time." + : `Cannot copy a directory: ${sourcePath}. copy_file copies one regular file; ` + + "create the directory at the destination and copy its files one at a time.", ); } if (!found.isFile()) { throw new Error( - `Cannot move across filesystems: not a regular file: ${sourcePath}`, + operation === "move" + ? `Cannot move across filesystems: not a regular file: ${sourcePath}` + : `Cannot copy: not a regular file: ${sourcePath}`, ); } const fd = fs.openSync(name, readFlags()); @@ -372,7 +423,7 @@ function openSourceInDirectory(sourcePath: string): { const opened = fs.fstatSync(fd, { bigint: true }); if (!opened.isFile() || !isSameEntry(found, opened)) { throw new Error( - `Refusing to move: ${sourcePath} was replaced while it was being opened`, + `Refusing to ${operation}: ${sourcePath} was replaced while it was being opened`, ); } return { fd, opened }; @@ -511,7 +562,7 @@ export async function moveFileAcrossDevices( sourcePath: string, destinationPath: string, ): Promise { - const { fd, opened: source } = openSourceInDirectory(sourcePath); + const { fd, opened: source } = openSourceInDirectory(sourcePath, "move"); // Set once the copy is complete; until then copyIntoNewFile cleans up. let created: EntryIdentity | undefined; try { @@ -535,6 +586,37 @@ export async function moveFileAcrossDevices( } } +/** + * Copy the regular file at `sourcePath` to `destinationPath`, which must not + * exist (#519): the copy half of the cross-device move above, with the same + * checks. The source is opened relative to its anchored directory and must be + * a regular file; the destination is created exclusively relative to its own, + * owner-only until it is complete, and then takes the source's permissions and + * times. A copy that fails is removed. The source is never changed. + */ +export async function copyFileInDirectory( + sourcePath: string, + destinationPath: string, +): Promise { + const { fd, opened: source } = openSourceInDirectory(sourcePath, "copy"); + // Set once the copy is complete; until then copyIntoNewFile cleans up. + let created: EntryIdentity | undefined; + try { + try { + created = await copyIntoNewFile(fd, source, destinationPath); + } finally { + await close(fd); + } + } catch (error) { + // A failure after the copy completed (closing the source) still fails + // the call, so the copy goes too, as the move's does. + if (created) { + discardCopy(destinationPath, created); + } + throw error; + } +} + /** * Create `directory` and any missing parents, like mkdir -p: anchor at the * deepest ancestor that exists, then step down one component at a time, diff --git a/src/filesystem/roots-utils.ts b/src/filesystem/roots-utils.ts index 9b2b7a14ff..4d4dd06ab8 100644 --- a/src/filesystem/roots-utils.ts +++ b/src/filesystem/roots-utils.ts @@ -1,10 +1,9 @@ import { promises as fs, type Stats } from "fs"; import path from "path"; -import os from "os"; import { - normalizePath, normalizePathFormat, realpathWithRetry, + expandHome, } from "./path-utils.js"; import type { Root } from "@modelcontextprotocol/server"; import { fileURLToPath } from "url"; @@ -29,17 +28,15 @@ async function parseRootUri(rootUri: string): Promise { const rawPath = rootUri.startsWith("file://") ? fileURLToPath(rootUri) : rootUri; - const expandedPath = - rawPath.startsWith("~/") || rawPath === "~" - ? path.join(os.homedir(), rawPath.slice(1)) - : rawPath; + const expandedPath = expandHome(rawPath); const absolutePath = path.resolve(expandedPath); const resolvedPath = await realpathWithRetry(absolutePath); return { - // normalizePathFormat, not normalizePath: trimming whitespace or quotes - // from the client's spelling would name a different directory. + // normalizePathFormat, not normalizePath, for both: trimming whitespace + // or quotes from the client's spelling or from the realpath would name + // a different directory, and allow it (#5183, #5320). advertised: normalizePathFormat(absolutePath), - resolved: normalizePath(resolvedPath), + resolved: normalizePathFormat(resolvedPath), }; } catch { return null; // Path doesn't exist or other error diff --git a/src/filesystem/server.ts b/src/filesystem/server.ts index a5127f8dbf..6adbba0648 100644 --- a/src/filesystem/server.ts +++ b/src/filesystem/server.ts @@ -13,14 +13,20 @@ import { pathToFileURL } from "url"; import { z } from "zod"; import { minimatch } from "minimatch"; import { getValidRootDirectories } from "./roots-utils.js"; +import { + confineRootsToDirectories, + describeAllowedDirectories, +} from "./allowed-directories.js"; import { formatSize, validatePath, getFileStats, readFileContent, writeFileContent, + appendFileContent, createDirectory, moveFile, + copyFile, searchFilesWithValidation, applyFileEdits, tailFile, @@ -28,6 +34,7 @@ import { readdirWithTimeout, createEntryBudget, } from "./lib.js"; +import { registerFileResources } from "./file-resources.js"; import { SERVER_VERSION } from "./version.js"; // The JSON Schema dialect every advertised tool schema declares (#4841). The @@ -51,10 +58,14 @@ const ReadTextFileArgsSchema = z.object({ path: z.string(), tail: z .number() + .int() + .min(0) .optional() .describe("If provided, returns only the last N lines of the file"), head: z .number() + .int() + .min(0) .optional() .describe("If provided, returns only the first N lines of the file"), }); @@ -77,6 +88,15 @@ const WriteFileArgsSchema = z.object({ content: z.string(), }); +const AppendFileArgsSchema = z.object({ + path: z.string(), + content: z + .string() + .describe( + "Text to append, exactly as given. No newline is added: end it with \\n if the next append should start on a new line.", + ), +}); + const EditOperation = z.object({ oldText: z.string().describe("Text to search for - must match exactly"), newText: z.string().describe("Text to replace with"), @@ -111,6 +131,15 @@ const ListDirectoryWithSizesArgsSchema = z.object({ const DirectoryTreeArgsSchema = z.object({ path: z.string(), excludePatterns: z.array(z.string()).optional().default([]), + depth: z + .number() + .int() + .positive() + .optional() + .describe( + "How many levels to descend: 1 lists only the entries directly in path. " + + "Directories at the last level have no children array. Omit for the whole tree.", + ), }); const MoveFileArgsSchema = z.object({ @@ -118,6 +147,13 @@ const MoveFileArgsSchema = z.object({ destination: z.string(), }); +const CopyFileArgsSchema = z.object({ + source: z.string().describe("The file to copy"), + destination: z + .string() + .describe("Where to create the copy; nothing may exist there yet"), +}); + const SearchFilesArgsSchema = z.object({ path: z.string(), pattern: z.string(), @@ -177,6 +213,19 @@ export interface ServerOptions { * Roots (#4992). The stdio entry point exits the process from here. */ onCannotOperate?: (error: Error) => void; + /** + * Serve only the tools that cannot change the filesystem (#632, #1160): + * every tool whose annotations do not declare `readOnlyHint: true` is left + * out of the server, so clients never see it and cannot call it. + */ + readOnly?: boolean; + /** + * Confine the client's Roots to the command-line directories (#3133): on a + * 2025-era connection a root may narrow them but never widen them, and a + * root outside every one of them is ignored. Has no effect when the server + * was started with no command-line directories. + */ + confineRoots?: boolean; } /** @@ -250,6 +299,25 @@ export function createServer( } return handler(...args); }) as never)) as typeof server.registerTool; + // Read-only mode (#632, #1160) withholds every tool that does not declare + // itself read-only. Keying on the annotation rather than on a list of write + // tools fails closed: a new tool is withheld unless its registration says + // `readOnlyHint: true`, the hint the repository already requires to be set + // truthfully, so a new write tool needs no change here. A withheld tool is + // registered and then removed, before any client connects, so it is absent + // from tools/list and a call to it is answered as an unknown tool. + if (options.readOnly) { + const registerReadWrite = server.registerTool.bind(server); + server.registerTool = (( + name: string, + config: Parameters[1], + handler: Parameters[2], + ) => { + const tool = registerReadWrite(name, config, handler); + if (config.annotations?.readOnlyHint !== true) tool.remove(); + return tool; + }) as typeof server.registerTool; + } // Tool registrations @@ -259,16 +327,18 @@ export function createServer( ) => { const validPath = await validatePath(args.path, allowedDirectories); - if (args.head && args.tail) { + // Compared against undefined, not tested for truthiness: a count of 0 + // asks for zero lines, not for the whole file (#5184). + if (args.head !== undefined && args.tail !== undefined) { throw new Error( "Cannot specify both head and tail parameters simultaneously", ); } let content: string; - if (args.tail) { + if (args.tail !== undefined) { content = await tailFile(validPath, args.tail); - } else if (args.head) { + } else if (args.head !== undefined) { content = await headFile(validPath, args.head); } else { content = await readFileContent(validPath); @@ -288,7 +358,12 @@ export function createServer( "Read the complete contents of a file as text. DEPRECATED: Use read_text_file instead.", inputSchema: jsonSchema2020(ReadTextFileArgsSchema.shape), outputSchema: jsonSchema2020({ content: z.string() }), - annotations: { readOnlyHint: true, openWorldHint: false }, + annotations: { + readOnlyHint: true, + idempotentHint: true, + destructiveHint: false, + openWorldHint: false, + }, }, readTextFileHandler, ); @@ -309,15 +384,24 @@ export function createServer( path: z.string(), tail: z .number() + .int() + .min(0) .optional() .describe("If provided, returns only the last N lines of the file"), head: z .number() + .int() + .min(0) .optional() .describe("If provided, returns only the first N lines of the file"), }), outputSchema: jsonSchema2020({ content: z.string() }), - annotations: { readOnlyHint: true, openWorldHint: false }, + annotations: { + readOnlyHint: true, + idempotentHint: true, + destructiveHint: false, + openWorldHint: false, + }, }, readTextFileHandler, ); @@ -351,7 +435,12 @@ export function createServer( ]), ), }), - annotations: { readOnlyHint: true, openWorldHint: false }, + annotations: { + readOnlyHint: true, + idempotentHint: true, + destructiveHint: false, + openWorldHint: false, + }, }, async (args: z.infer) => { const validPath = await validatePath(args.path, allowedDirectories); @@ -407,7 +496,12 @@ export function createServer( "the entire operation. Only works within allowed directories.", inputSchema: jsonSchema2020(ReadMultipleFilesArgsSchema.shape), outputSchema: jsonSchema2020({ content: z.string() }), - annotations: { readOnlyHint: true, openWorldHint: false }, + annotations: { + readOnlyHint: true, + idempotentHint: true, + destructiveHint: false, + openWorldHint: false, + }, }, async (args: z.infer) => { const results = await Promise.all( @@ -459,13 +553,49 @@ export function createServer( }, ); + server.registerTool( + "append_file", + { + title: "Append File", + description: + "Append text to the end of a file, creating the file if it does not exist. " + + "The existing content is never changed or replaced. The text is appended exactly " + + "as given, with no newline added before or after it, so end it with a newline " + + "if the next append should start on a new line. Only works within allowed directories.", + inputSchema: jsonSchema2020(AppendFileArgsSchema.shape), + outputSchema: jsonSchema2020({ content: z.string() }), + annotations: { + readOnlyHint: false, + idempotentHint: false, + destructiveHint: false, + openWorldHint: false, + }, + }, + async (args: z.infer) => { + const validPath = await validatePath(args.path, allowedDirectories); + await appendFileContent(validPath, args.content); + const text = `Successfully appended to ${args.path}`; + return { + content: [{ type: "text" as const, text }], + structuredContent: { content: text }, + }; + }, + ); + server.registerTool( "edit_file", { title: "Edit File", description: - "Make line-based edits to a text file. Each edit replaces exact line sequences " + - "with new content. Returns a git-style diff showing the changes made. " + + "Make line-based edits to a text file. Edits apply in order, each to the " + + "result of the ones before it, and each replaces the first match of its " + + "oldText with newText. oldText must match the file's text exactly, except " + + "that line endings (LF or CRLF) never matter and, when there is no exact " + + "match, a run of whole lines matches if each line is equal once leading and " + + "trailing whitespace is ignored (newText then takes the file's indentation). " + + "Whitespace inside a line must match. If any edit has no match, nothing is " + + "written and the error shows the closest lines of the file, numbered, to " + + "copy oldText from. Returns a git-style diff showing the changes made. " + "Only works within allowed directories.", inputSchema: jsonSchema2020(EditFileArgsSchema.shape), outputSchema: jsonSchema2020({ content: z.string() }), @@ -526,7 +656,12 @@ export function createServer( "finding specific files within a directory. Only works within allowed directories.", inputSchema: jsonSchema2020(ListDirectoryArgsSchema.shape), outputSchema: jsonSchema2020({ content: z.string() }), - annotations: { readOnlyHint: true, openWorldHint: false }, + annotations: { + readOnlyHint: true, + idempotentHint: true, + destructiveHint: false, + openWorldHint: false, + }, }, async (args: z.infer) => { const validPath = await validatePath(args.path, allowedDirectories); @@ -555,15 +690,26 @@ export function createServer( "finding specific files within a directory. Only works within allowed directories.", inputSchema: jsonSchema2020(ListDirectoryWithSizesArgsSchema.shape), outputSchema: jsonSchema2020({ content: z.string() }), - annotations: { readOnlyHint: true, openWorldHint: false }, + annotations: { + readOnlyHint: true, + idempotentHint: true, + destructiveHint: false, + openWorldHint: false, + }, }, async (args: z.infer) => { const validPath = await validatePath(args.path, allowedDirectories); const entries = await fs.readdir(validPath, { withFileTypes: true }); - // Get detailed information for each entry + // Get detailed information for each entry; size is undefined when the + // entry could not be stat'd + type SizedEntry = { + name: string; + isDirectory: boolean; + size: number | undefined; + }; const detailedEntries = await Promise.all( - entries.map(async (entry) => { + entries.map(async (entry): Promise => { const entryPath = path.join(validPath, entry.name); try { // fs.stat follows symlinks, so validate each entry first: a @@ -578,14 +724,16 @@ export function createServer( name: entry.name, isDirectory: entry.isDirectory(), size: stats.size, - mtime: stats.mtime, }; } catch { + // The entry could not be stat'd (unreadable, a dangling symlink, + // removed since readdir, or outside the allowed directories). + // Its size is unknown, not 0: inventing one would list it as an + // empty file and count it in the total (#5190). return { name: entry.name, isDirectory: entry.isDirectory(), - size: 0, - mtime: new Date(0), + size: undefined, }; } }), @@ -594,7 +742,8 @@ export function createServer( // Sort entries based on sortBy parameter const sortedEntries = [...detailedEntries].sort((a, b) => { if (args.sortBy === "size") { - return b.size - a.size; // Descending by size + // Descending by size; an entry of unknown size sorts last + return (b.size ?? -1) - (a.size ?? -1); } // Default sort by name return a.name.localeCompare(b.name); @@ -604,7 +753,12 @@ export function createServer( const formattedEntries = sortedEntries.map( (entry) => `${entry.isDirectory ? "[DIR]" : "[FILE]"} ${entry.name.padEnd(30)} ${ - entry.isDirectory ? "" : formatSize(entry.size).padStart(10) + entry.isDirectory + ? "" + : (entry.size === undefined + ? "unknown" + : formatSize(entry.size) + ).padStart(10) }`, ); @@ -612,14 +766,20 @@ export function createServer( const totalFiles = detailedEntries.filter((e) => !e.isDirectory).length; const totalDirs = detailedEntries.filter((e) => e.isDirectory).length; const totalSize = detailedEntries.reduce( - (sum, entry) => sum + (entry.isDirectory ? 0 : entry.size), + (sum, entry) => sum + (entry.isDirectory ? 0 : (entry.size ?? 0)), 0, ); + const unknownFiles = detailedEntries.filter( + (e) => !e.isDirectory && e.size === undefined, + ).length; const summary = [ "", `Total: ${totalFiles} files, ${totalDirs} directories`, - `Combined size: ${formatSize(totalSize)}`, + `Combined size: ${formatSize(totalSize)}` + + (unknownFiles > 0 + ? ` (excludes ${unknownFiles} ${unknownFiles === 1 ? "file" : "files"} of unknown size)` + : ""), ]; const text = [...formattedEntries, ...summary].join("\n"); @@ -638,11 +798,17 @@ export function createServer( description: "Get a recursive tree view of files and directories as a JSON structure. " + "Each entry includes 'name', 'type' (file/directory), and 'children' for directories. " + - "Files have no children array, while directories always have a children array (which may be empty). " + + "Files have no children array, while directories always have a children array (which may be empty), " + + "except at the last level when 'depth' limits the tree: those directories are not read and have no children array. " + "The output is formatted with 2-space indentation for readability. Only works within allowed directories.", inputSchema: jsonSchema2020(DirectoryTreeArgsSchema.shape), outputSchema: jsonSchema2020({ content: z.string() }), - annotations: { readOnlyHint: true, openWorldHint: false }, + annotations: { + readOnlyHint: true, + idempotentHint: true, + destructiveHint: false, + openWorldHint: false, + }, }, async (args: z.infer) => { interface TreeEntry { @@ -653,9 +819,13 @@ export function createServer( const rootPath = args.path; const visit = createEntryBudget("directory_tree", rootPath); + // `level` is the depth of currentPath's entries: 1 for those directly + // in the requested path. A directory at the depth limit is listed but + // not read, so its subtree costs no readdir and no entry budget (#1077). async function buildTree( currentPath: string, excludePatterns: string[] = [], + level: number = 1, ): Promise { const validPath = await validatePath(currentPath, allowedDirectories); const entries = await readdirWithTimeout(validPath); @@ -689,9 +859,16 @@ export function createServer( type: entry.isDirectory() ? "directory" : "file", }; - if (entry.isDirectory()) { + if ( + entry.isDirectory() && + (args.depth === undefined || level < args.depth) + ) { const subPath = path.join(currentPath, entry.name); - entryData.children = await buildTree(subPath, excludePatterns); + entryData.children = await buildTree( + subPath, + excludePatterns, + level + 1, + ); } result.push(entryData); @@ -747,29 +924,86 @@ export function createServer( }, ); + server.registerTool( + "copy_file", + { + title: "Copy File", + description: + "Copy a file to a new location, without reading its content into the conversation. " + + "Copies one regular file, keeping its permissions and modification time; directories " + + "are not copied. If the destination exists, the operation will fail: it never " + + "overwrites. Both source and destination must be within allowed directories.", + inputSchema: jsonSchema2020(CopyFileArgsSchema.shape), + outputSchema: jsonSchema2020({ content: z.string() }), + annotations: { + readOnlyHint: false, + idempotentHint: true, + destructiveHint: false, + openWorldHint: false, + }, + }, + async (args: z.infer) => { + const validSourcePath = await validatePath( + args.source, + allowedDirectories, + ); + const validDestPath = await validatePath( + args.destination, + allowedDirectories, + ); + await copyFile(validSourcePath, validDestPath); + const text = `Successfully copied ${args.source} to ${args.destination}`; + return { + content: [{ type: "text" as const, text }], + structuredContent: { content: text }, + }; + }, + ); + server.registerTool( "search_files", { title: "Search Files", description: - "Recursively search for files and directories matching a pattern. " + - "The patterns should be glob-style patterns that match paths relative to the working directory. " + - "Use pattern like '*.ext' to match files in current directory, and '**/*.ext' to match files in all subdirectories. " + + "Find files and directories by NAME, not by content: this tool never reads or searches inside files. " + + "It walks the tree under 'path' and matches a glob pattern against each entry's path relative to 'path' (case-sensitive). " + + "Use '*.ext' to match entries directly under 'path', and '**/*.ext' or '**/name' to match at any depth. " + "Returns full paths to all matching items. Great for finding files when you don't know their exact location. " + "Only searches within allowed directories.", inputSchema: jsonSchema2020(SearchFilesArgsSchema.shape), outputSchema: jsonSchema2020({ content: z.string() }), - annotations: { readOnlyHint: true, openWorldHint: false }, + annotations: { + readOnlyHint: true, + idempotentHint: true, + destructiveHint: false, + openWorldHint: false, + }, }, async (args: z.infer) => { const validPath = await validatePath(args.path, allowedDirectories); + // A subdirectory that cannot be read is skipped, and named after the + // matches so the caller knows they are partial (#5189). + const unreadable: string[] = []; const results = await searchFilesWithValidation( validPath, args.pattern, allowedDirectories, - { excludePatterns: args.excludePatterns }, + { + excludePatterns: args.excludePatterns, + onUnreadable: (directory, error) => { + const code = (error as NodeJS.ErrnoException).code; + unreadable.push(code ? `${directory} (${code})` : directory); + }, + }, ); - const text = results.length > 0 ? results.join("\n") : "No matches found"; + let text = results.length > 0 ? results.join("\n") : "No matches found"; + if (unreadable.length > 0) { + text += + `\n\nThe search is incomplete: ${unreadable.length} ` + + `${unreadable.length === 1 ? "directory" : "directories"} could not be read, ` + + "and any matches inside are missing:\n" + + unreadable.join("\n"); + } return { content: [{ type: "text" as const, text }], structuredContent: { content: text }, @@ -788,7 +1022,12 @@ export function createServer( "without reading the actual content. Only works within allowed directories.", inputSchema: jsonSchema2020(GetFileInfoArgsSchema.shape), outputSchema: jsonSchema2020({ content: z.string() }), - annotations: { readOnlyHint: true, openWorldHint: false }, + annotations: { + readOnlyHint: true, + idempotentHint: true, + destructiveHint: false, + openWorldHint: false, + }, }, async (args: z.infer) => { const validPath = await validatePath(args.path, allowedDirectories); @@ -814,7 +1053,12 @@ export function createServer( "before trying to access files.", inputSchema: jsonSchema2020({}), outputSchema: jsonSchema2020({ content: z.string() }), - annotations: { readOnlyHint: true, openWorldHint: false }, + annotations: { + readOnlyHint: true, + idempotentHint: true, + destructiveHint: false, + openWorldHint: false, + }, }, async () => { const text = `Allowed directories:\n${allowedDirectories.join("\n")}`; @@ -825,19 +1069,31 @@ export function createServer( }, ); + // Files as resources (#297): reads see the same allow-list, roots barrier + // and 2026-07-28 no-directories error as the tools above. + registerFileResources(server, { + allowedDirectories: () => allowedDirectories, + rootsSettled: () => barrier.settled, + unavailableReason: () => + options.era === "modern" && allowedDirectories.length === 0 + ? NO_ALLOWED_DIRECTORIES_MODERN_ERROR + : undefined, + }); + // Updates allowed directories based on MCP client roots, unless a newer // refresh started after this one's `generation` was taken (#5097). async function updateAllowedDirectoriesFromRoots( requestedRoots: Root[], generation: number, ) { - const validatedRootDirs = await getValidRootDirectories(requestedRoots); + const validRootDirs = await getValidRootDirectories(requestedRoots); if (generation !== rootsGeneration) { console.error( "Discarded a stale roots/list answer: a newer roots update has started", ); return; } + const validatedRootDirs = confineRoots(validRootDirs); if (validatedRootDirs.length > 0) { allowedDirectories = [...validatedRootDirs]; rootsInForce = true; @@ -852,6 +1108,46 @@ export function createServer( } else { console.error("No valid root directories provided by client"); } + logAllowedDirectories(); + } + + // Log the allowed directories now in force and where they came from (#4912). + function logAllowedDirectories(): void { + console.error( + describeAllowedDirectories( + allowedDirectories, + rootsInForce + ? { + from: "roots", + replacedCommandLine: initialAllowedDirectories.length > 0, + confined: confining, + } + : { from: "command-line" }, + ), + ); + } + + // True when the client's Roots are kept inside the command-line directories + // (#3133). With none given there is nothing to confine them to. + const confining = + options.confineRoots === true && initialAllowedDirectories.length > 0; + + // The client's valid roots as the server will apply them: unchanged, or, + // with --confine-roots, cut down to their overlap with the command-line + // directories. A root outside all of them is dropped like an invalid one, + // so roots that are all outside leave the state as invalid roots would. + function confineRoots(validRootDirs: string[]): string[] { + if (!confining) return validRootDirs; + const { confined, ignored } = confineRootsToDirectories( + validRootDirs, + initialAllowedDirectories, + ); + if (ignored.length > 0) { + console.error( + `Ignored client roots outside the command-line directories (--confine-roots): ${ignored.join(", ")}`, + ); + } + return confined; } // Handles dynamic roots updates during runtime, when client sends "roots/list_changed" notification, server fetches the updated roots and replaces all allowed directories with the new roots. @@ -910,6 +1206,7 @@ export function createServer( "Failed to request initial roots from client:", error instanceof Error ? error.message : String(error), ); + logAllowedDirectories(); } } else { if (allowedDirectories.length > 0) { @@ -917,6 +1214,7 @@ export function createServer( "Client does not support MCP Roots, using allowed directories set from server args:", allowedDirectories, ); + logAllowedDirectories(); } else { // Nothing could ever be allowed in this session, so fail visibly // rather than serve every call with "Access denied" (#4992): log the diff --git a/src/git/README.md b/src/git/README.md index 01ae034782..b4a22edf86 100644 --- a/src/git/README.md +++ b/src/git/README.md @@ -35,12 +35,12 @@ Please note that mcp-server-git is currently in early development. The functiona - Returns: Diff output of staged changes 4. `git_diff` - - Shows differences between branches or commits + - Shows differences between branches or commits, or across a revision range - Inputs: - `repo_path` (string): Path to Git repository - - `target` (string): Target branch or commit to compare with + - `target` (string): Target branch or commit to compare with, or a revision range: `a..b` (from `a` to `b`) or `a...b` (from the merge base of `a` and `b` to `b`). An omitted endpoint means `HEAD`; each endpoint must resolve and cannot start with `-` - `context_lines` (number, optional): Number of context lines to show (default: 3) - - Returns: Diff output comparing current state with target + - Returns: Diff output comparing the working tree with target, or the two endpoints of a range with each other 5. `git_commit` - Records changes to the repository @@ -84,14 +84,22 @@ Please note that mcp-server-git is currently in early development. The functiona - `repo_path` (string): Path to Git repository - `branch_name` (string): Name of branch to checkout - Returns: Confirmation of branch switch, or `HEAD is now detached at ` when `branch_name` is not a branch (a sha, tag or other revision) -11. `git_show` +11. `git_merge` + - Merges a branch into the current branch, or changes nothing + - Inputs: + - `repo_path` (string): Path to Git repository + - `source_branch` (string): Branch, tag or commit to merge into the current branch (check out the branch to merge into first with `git_checkout`) + - `fast_forward_only` (boolean, optional): Only fast-forward, failing if the branches have diverged (default `true`). `false` lets Git create a merge commit when they have + - Returns: `Fast-forwarded`, `Merged ... with merge commit `, or `Already up to date` + - Refuses a detached HEAD, a working tree with uncommitted changes, a merge already in progress, and a merge that would overwrite an ignored local file. A merge that stops on conflicts is aborted (`git merge --abort`) and the error names the conflicting files, so the repository is left as it was +12. `git_show` - Shows the contents of a commit, or of a file or directory at a revision - Inputs: - `repo_path` (string): Path to Git repository - `revision` (string): The revision (commit hash, branch name, tag) or object spec (`HEAD:path/to/file`) to show - Returns: Contents of the specified commit, file content, or directory listing -12. `git_branch` +13. `git_branch` - List Git branches - Inputs: - `repo_path` (string): Path to the Git repository. @@ -100,6 +108,13 @@ Please note that mcp-server-git is currently in early development. The functiona - `not_contains` (string, optional): The commit sha that branch should NOT contain. Do not pass anything to this param if no commit sha is specified - Returns: List of branches +14. `git_remote` + - Lists the repository's remotes with their fetch and push URLs, like `git remote -v` + - Inputs: + - `repo_path` (string): Path to the Git repository. + - Returns: One line per remote and direction, `\t (fetch)` or `(push)` (a partial clone's fetch line ends with its filter, such as ` [blob:none]`), or `No remotes configured` + - A credential embedded in a URL is never returned: the whole userinfo of a URL such as `https://user:token@host/...` or `https://token@host/...` is shown as ``. An SSH username (`ssh://git@host/...`) is kept, since it names an account rather than authenticating one; a password after it is redacted. A remote-helper address (`https::https://user:token@host/...`) has its inner URL redacted the same way. scp-style addresses (`git@host:path`) and local paths are shown as configured. A line that cannot be parsed is shown as ``, and if a configured remote URL (or `insteadOf` rewrite) contains a newline, which git itself refuses, the tool returns an error instead of a listing. + ### Protocol versions The server speaks both eras of the Model Context Protocol over stdio, with nothing to configure: a **2026-07-28** client (discovered with `server/discover`, no session) and a **2025-11-25** client (the `initialize` handshake). The client's first request decides which. Tools behave the same in both. @@ -111,6 +126,25 @@ Roots is deprecated in 2026-07-28, and this server never asks a client for Roots - **A tool that fails** returns a result with `isError: true` whose text is `Error executing tool : `, so the model can read it and correct the call. This covers git errors (an unknown revision, a failed command), a `repo_path` that is not a repository or is outside `--repository`, the flag-injection guards, and arguments that do not match the tool's input schema. - **An unknown tool name** is a protocol error, JSON-RPC `-32602` (`Unknown tool: `), as the specification requires in both eras. Before 2.0.0 it was an `isError` result. +### Prompts + +Beside its tools, the server offers two prompts, which a client typically shows the user as slash commands. Each reads the repository when it is requested and returns one user message that embeds the changes, ready to send to the model. They open the repository the way the tools do, so `--repository` confines them too, and the same flag-injection guards apply. They behave the same in both protocol eras. + +1. `git_commit_message` + - Asks the model for a commit message for the staged changes + - Arguments: + - `repo_path` (string, required): Path to the Git repository + - Returns: A message embedding the staged diff and asking for a commit message, or one saying nothing is staged + +2. `git_summarize_changes` + - Asks the model to summarize the working tree's changes, or the diff against a branch or commit + - Arguments: + - `repo_path` (string, required): Path to the Git repository + - `target` (string, optional): Branch or commit to diff against. Without it, the working tree's status and its staged and unstaged diffs are summarized + - Returns: A message embedding those changes and asking for a summary, or one saying there are none + +A prompt request that fails (a `repo_path` that is not a repository or is outside `--repository`, an unknown `target`, a `target` starting with `-`) is a JSON-RPC `-32602` error carrying the message a tool would report, since `prompts/get` has no `isError` result. + ## Installation ### Using uv (recommended) @@ -135,12 +169,49 @@ python -m mcp_server_git ## Configuration > [!IMPORTANT] -> **Without `--repository`, the server reads and writes any Git repository the server process can reach.** Each tool call names its own `repo_path`, and that holds for the tools that change a repository too (`git_add`, `git_commit`, `git_reset`, `git_create_branch`, `git_checkout`). `--repository` is the only confinement the server offers: it never asks the client for Roots, in either protocol era. Set `--repository` whenever the server is used with prompts or content that are not fully trusted. It admits the whole enclosing repository, not only the directory you name, as the next paragraph describes. +> **Without `--repository`, the server reads and writes any Git repository the server process can reach.** Each tool call names its own `repo_path`, and that holds for the tools that change a repository too (`git_add`, `git_commit`, `git_reset`, `git_create_branch`, `git_checkout`, `git_merge`). `--repository` is the only confinement the server offers: it never asks the client for Roots, in either protocol era. Set `--repository` whenever the server is used with prompts or content that are not fully trusted. It admits the whole enclosing repository, not only the directory you name, as the next paragraph describes. `--repository` (`-r`) accepts any path inside a Git working tree. At startup the server walks up to the enclosing repository root, the way `git rev-parse --show-toplevel` does, so `--repository .` works when a shared config is launched from any subdirectory of the repository. A path inside the repository therefore allows tool calls anywhere in that repository, not only under the subdirectory given. A path that does not exist, or is not inside a Git repository, is logged as a one-line error at startup. A tool call's `repo_path` must be inside that repository, and so must the directories Git then uses for it: a repository nested inside whose `.git` file, `commondir` or `core.worktree` points outside is refused. A linked worktree can still be the `--repository`, though its Git directory lives under the main repository's `.git`. +### Commit identity + +`git_commit` has no identity setting of its own. It takes the author and the committer from the same places `git commit` does, in this order: + +1. The server process's environment: `GIT_AUTHOR_NAME` and `GIT_AUTHOR_EMAIL` for the author, `GIT_COMMITTER_NAME` and `GIT_COMMITTER_EMAIL` for the committer. The author and the committer are resolved separately, so set all four (usually to the same name and email). +2. `user.name` and `user.email` from Git config: the repository's `.git/config`, then `~/.gitconfig`, then `$XDG_CONFIG_HOME/git/config` (by default `~/.config/git/config`), then `/etc/gitconfig`. + +Unlike `git commit`, the server does not refuse to commit when neither is set. It falls back to the operating-system user and host name, so a commit made in a container is recorded as something like `root `. A local install usually finds your own `~/.gitconfig`; a Docker container has none, so give it an identity explicitly. + +For a local command (`uvx`, `python -m mcp_server_git`), add an `env` block to the server's entry in your client configuration: + +```json +"mcpServers": { + "git": { + "command": "uvx", + "args": ["mcp-server-git", "--repository", "path/to/git/repo"], + "env": { + "GIT_AUTHOR_NAME": "Your Name", + "GIT_AUTHOR_EMAIL": "you@example.com", + "GIT_COMMITTER_NAME": "Your Name", + "GIT_COMMITTER_EMAIL": "you@example.com" + } + } +} +``` + +For Docker, pass the variables to `docker run` with `-e`, as the Docker examples below do. Alternatively, mount your gitconfig read-only in place of the four `-e` pairs. The image runs as `root`, so it goes at `/root/.gitconfig`: + +```json +"args": [ + "run", "--rm", "-i", + "--mount", "type=bind,src=/Users/username/.gitconfig,dst=/root/.gitconfig,readonly", + "--mount", "type=bind,src=/Users/username,dst=/Users/username", + "mcp/git" +] +``` + ### Usage with Claude Desktop Add this to your `claude_desktop_config.json`: @@ -162,12 +233,21 @@ Add this to your `claude_desktop_config.json`: Using docker * Note: replace '/Users/username' with a path that you want to be accessible by this tool +* Note: the `-e` lines set the identity `git_commit` records; see [Commit identity](#commit-identity) ```json "mcpServers": { "git": { "command": "docker", - "args": ["run", "--rm", "-i", "--mount", "type=bind,src=/Users/username,dst=/Users/username", "mcp/git"] + "args": [ + "run", "--rm", "-i", + "-e", "GIT_AUTHOR_NAME=Your Name", + "-e", "GIT_AUTHOR_EMAIL=you@example.com", + "-e", "GIT_COMMITTER_NAME=Your Name", + "-e", "GIT_COMMITTER_EMAIL=you@example.com", + "--mount", "type=bind,src=/Users/username,dst=/Users/username", + "mcp/git" + ] } } ``` @@ -343,6 +423,10 @@ There are two ways to try your changes in a client: "run", "--rm", "-i", + "-e", "GIT_AUTHOR_NAME=Your Name", + "-e", "GIT_AUTHOR_EMAIL=you@example.com", + "-e", "GIT_COMMITTER_NAME=Your Name", + "-e", "GIT_COMMITTER_EMAIL=you@example.com", "--mount", "type=bind,src=/Users/username/Desktop,dst=/projects/Desktop", "--mount", "type=bind,src=/path/to/other/allowed/dir,dst=/projects/other/allowed/dir,ro", "--mount", "type=bind,src=/path/to/file.txt,dst=/projects/path/to/file.txt", diff --git a/src/git/pyproject.toml b/src/git/pyproject.toml index 1bc4739d7c..1d3d4fa43d 100644 --- a/src/git/pyproject.toml +++ b/src/git/pyproject.toml @@ -1,6 +1,6 @@ [project] name = "mcp-server-git" -version = "2026.10.28" +version = "2026.11.4" description = "A Model Context Protocol server providing tools to read, search, and manipulate Git repositories programmatically via LLMs" readme = "README.md" requires-python = ">=3.10" diff --git a/src/git/src/mcp_server_git/server.py b/src/git/src/mcp_server_git/server.py index 8ced4d97cd..ca69538746 100644 --- a/src/git/src/mcp_server_git/server.py +++ b/src/git/src/mcp_server_git/server.py @@ -1,8 +1,9 @@ # The git MCP server: tools that read and change a Git repository through -# GitPython. Repository confinement (`--repository`, `validate_repo_path` on the -# path given, `validate_opened_repo` on the directories git then uses) and the -# flag-injection guards live beside the handlers because every tool call must -# pass them before any git command runs. +# GitPython, and prompts that embed a repository's changes in a message for the +# model (#5195). Repository confinement (`--repository`, `validate_repo_path` on +# the path given, `validate_opened_repo` on the directories git then uses) and +# the flag-injection guards live beside the handlers because every tool call and +# prompt must pass them before any git command runs. # # Built on the MCP Python SDK's high-level `MCPServer` (#5174), the API these # reference servers are meant to teach: each tool is a typed function, and the @@ -18,6 +19,7 @@ # logging), so nothing differs between them. import logging +import re import threading from collections.abc import Callable, Iterator from contextlib import contextmanager @@ -69,9 +71,11 @@ class GitTools(str, Enum): LOG = "git_log" CREATE_BRANCH = "git_create_branch" CHECKOUT = "git_checkout" + MERGE = "git_merge" SHOW = "git_show" BRANCH = "git_branch" + REMOTE = "git_remote" def _git_text(repo: git.Repo, command: str, *args: str) -> str: @@ -117,6 +121,37 @@ def git_diff_staged(repo: git.Repo, context_lines: int = DEFAULT_CONTEXT_LINES) return _git_text(repo, "diff", f"--unified={context_lines}", "--cached") +def _validate_diff_target(repo: git.Repo, target: str) -> None: + """Check that git_diff's target is one revision, or a range of two (#5194). + + `rev_parse` resolves a single revision, so it rejects `main..feature` and + `main...feature` as a whole although `git diff` accepts them. The target is + tried whole first, because one revision may itself contain `..` (the + commit-message search `:/fix..bug`). Only when that fails is it split at + its first `..` or `...`, and each endpoint is held to the single-revision + rules: no leading '-', and it must resolve. An empty endpoint means HEAD, + as in git; both empty (`..`) is not a range. + """ + try: + repo.rev_parse(target) + return + except (ODBError, ValueError, KeyError): + # BadName (an ODBError) for an unknown name; ValueError and KeyError + # for a spec GitPython cannot apply. Re-raise unless it is a range. + start = target.find("..") + if start == -1 or target in ("..", "..."): + raise + end = start + 3 if target.startswith("...", start) else start + 2 + for endpoint in (target[:start], target[end:]): + # The same flag-injection guard the whole target gets in git_diff. + if endpoint.startswith("-"): + raise FlagInjectionError( + f"Invalid target: '{target}' - range endpoint '{endpoint}' " + "cannot start with '-'" + ) + repo.rev_parse(endpoint or "HEAD") # BadName names the bad endpoint + + def git_diff( repo: git.Repo, target: str, context_lines: int = DEFAULT_CONTEXT_LINES ) -> str: @@ -124,7 +159,8 @@ def git_diff( # even if a malicious ref with that name exists (e.g. via filesystem manipulation) if target.startswith("-"): raise FlagInjectionError(f"Invalid target: '{target}' - cannot start with '-'") - repo.rev_parse(target) # Validates target is a real git ref, throws BadName if not + # Validates target is a real revision or range, throws BadName if not + _validate_diff_target(repo, target) return _git_text(repo, "diff", f"--unified={context_lines}", target) @@ -345,6 +381,124 @@ def git_checkout(repo: git.Repo, branch_name: str) -> str: return f"Switched to branch '{repo.active_branch.name}'" +def _git_stderr(error: git.GitCommandError) -> str: + """Git's own message from a failed command, without GitPython's framing. + + GitPython formats `stderr` as `\n stderr: ''`. + """ + message = str(error.stderr).strip().removeprefix("stderr: '").removesuffix("'") + return message or str(error) + + +def _merge_failure(repo: git.Repo, error: git.GitCommandError, what: str) -> str: + """Why `git merge` failed, after undoing whatever it left behind. + + A merge that stops on conflicts, or fails to write its merge commit, + leaves MERGE_HEAD and a half-merged index and working tree. The tool + required a clean tree before merging, so `git merge --abort` restores + exactly the state the call started from. `what` names the merge, as + "Merge of '' into ''". + """ + unchanged = "The merge was aborted and the repository is unchanged." + if not (Path(repo.git_dir) / "MERGE_HEAD").exists(): + # Git refused before changing anything. + return f"{what} failed. {unchanged} Git reported:\n{_git_stderr(error)}" + # Through _git_text, so a non-ASCII name prints as written, not as + # core.quotepath's octal escapes. + conflicts = _git_text(repo, "diff", "--name-only", "--diff-filter=U").splitlines() + repo.git.merge("--abort") + if conflicts: + return ( + f"{what} stopped on conflicts in: {', '.join(conflicts)}. {unchanged} " + "Resolve the conflicts by hand." + ) + return f"{what} failed. {unchanged} Git reported:\n{_git_stderr(error)}" + + +def git_merge( + repo: git.Repo, source_branch: str, fast_forward_only: bool = True +) -> str: + """Merge `source_branch` into the current branch, or change nothing. + + Conservative by design (#659): the merge goes into the branch HEAD is on, + never another, and it starts only from a clean working tree, so a merge + that stops on conflicts is aborted and the repository is left as it was. + The conflicts are reported for the caller to resolve by hand. By default + only a fast-forward is allowed; `fast_forward_only=False` lets git create + a merge commit when the branches have diverged. + """ + # Defense in depth: reject sources starting with '-' to prevent flag + # injection, even if a malicious ref with that name exists. + if source_branch.startswith("-"): + raise FlagInjectionError( + f"Invalid source branch: '{source_branch}' - cannot start with '-'" + ) + if repo.head.is_detached: + raise ValueError( + "HEAD is detached; check out the branch to merge into first (git_checkout)" + ) + target = repo.active_branch.name + if not repo.head.is_valid(): + raise ValueError(f"Branch '{target}' has no commits yet; nothing to merge into") + if (Path(repo.git_dir) / "MERGE_HEAD").exists(): + raise ValueError( + "A merge is already in progress; conclude it with git_commit first" + ) + if repo.is_dirty(): + raise ValueError( + f"Cannot merge into '{target}': the working tree has uncommitted " + "changes. Commit or discard them first, so a failed merge can be " + "undone cleanly." + ) + # Resolves the revision (BadName if unknown) and requires a commit. + source = repo.commit(source_branch) + head = repo.head.commit + if repo.is_ancestor(source, head): + return f"Already up to date: '{target}' already contains '{source_branch}'" + fast_forward = repo.is_ancestor(head, source) + if fast_forward_only and not fast_forward: + raise ValueError( + f"Cannot fast-forward '{target}' to '{source_branch}': the branches " + "have diverged. Nothing was changed. Pass fast_forward_only=false " + "to create a merge commit." + ) + # The flags pin the outcome this tool describes, whatever the user's + # config says (merge.ff, branch..mergeOptions): --ff/--ff-only + # choose fast-forwarding, --commit and --no-squash make a merge that + # succeeds a completed one that moves HEAD, and --no-edit keeps git from + # opening an editor for the message. --no-overwrite-ignore refuses to + # replace an ignored local file (a `.env`, say) that the source tracks: + # `is_dirty()` cannot see one, and git would overwrite it unrecoverably. + # + # Git is given the commit validated above, not the name: a name can stand + # for more than one commit (`git merge FETCH_HEAD` merges every head + # FETCH_HEAD lists, as an octopus), and only the one checked is merged. + # With a sha, git's default message would name the sha, so the message + # names what the caller asked for. + mode = "--ff-only" if fast_forward_only else "--ff" + try: + repo.git.merge( + mode, + "--commit", + "--no-squash", + "--no-edit", + "--no-overwrite-ignore", + "-m", + f"Merge '{source_branch}' into '{target}'", + source.hexsha, + ) + except git.GitCommandError as e: + what = f"Merge of '{source_branch}' into '{target}'" + raise ValueError(_merge_failure(repo, e, what)) from e + merged = repo.head.commit + if fast_forward: + return f"Fast-forwarded '{target}' to '{source_branch}' ({merged.hexsha[:7]})" + return ( + f"Merged '{source_branch}' into '{target}' with merge commit " + f"{merged.hexsha[:7]}" + ) + + def git_show(repo: git.Repo, revision: str) -> str: # Defense in depth: reject revisions starting with '-' to prevent flag injection, # even if a malicious ref with that name exists (e.g. via filesystem manipulation) @@ -517,6 +671,118 @@ def git_branch( return branch_info +# A URL with an authority part: `scheme://[userinfo@]host...`. The userinfo is +# everything up to the last '@' before the first '/', which is where git itself +# stops reading it. +_URL_USERINFO = re.compile( + r"^(?P[A-Za-z][A-Za-z0-9+.-]*)://(?P[^/]*)@" +) + +# An explicit remote-helper address, `::
` +# (`https::https://user:token@host/...`, gitremote-helpers(7)). The address +# after the `::` is a URL of its own, and is redacted as one. +_REMOTE_HELPER = re.compile(r"^(?P[A-Za-z][A-Za-z0-9+.-]*)::") + +# One line of `git remote -v`: `\t (fetch|push)`, and on a +# partial clone's fetch line the promisor filter after it (` [blob:none]`). +# `.` matches everything but '\n', so a URL holding another line separator +# (U+2028, say) stays whole. +_REMOTE_LINE = re.compile( + r"^(?P[^\t]*)\t(?P.*) \((?Pfetch|push)\)" + r"(?P \[.*\])?$" +) + +# Schemes whose username is an account name rather than a secret, so it is kept +# (`ssh://git@host/...`) and only a password after it is redacted. +_SSH_SCHEMES = frozenset({"ssh", "git+ssh", "ssh+git"}) + +REDACTED = "" + + +def redact_url(url: str) -> str: + """`url` with any credential in its userinfo replaced by `` (#1611). + + A remote URL can carry a credential (`https://user:token@host/...`), and + the tool's output goes to the model, so it must not echo one. Over HTTP(S) + and every other scheme a token is as often the username as the password + (`https://@github.com/...`), so the whole userinfo is redacted. An + SSH username names the account and authenticates nothing, so it stays, + with only a password after it redacted. A remote-helper address + (`::`) has its inner URL redacted. An scp-like address + (`git@host:path`) or a local path has no userinfo to carry a password and + is returned as is. + """ + helper = _REMOTE_HELPER.match(url) + if helper is not None: + return url[: helper.end()] + redact_url(url[helper.end() :]) + match = _URL_USERINFO.match(url) + if match is None: + return url + scheme, userinfo = match["scheme"], match["userinfo"] + if scheme.lower() in _SSH_SCHEMES: + user, has_password, _ = userinfo.partition(":") + kept = f"{user}:{REDACTED}" if has_password else user + else: + kept = REDACTED + return f"{scheme}://{kept}@{url[match.end() :]}" + + +# The config values `git remote -v` builds its URLs from: each remote's +# `url` and `pushurl`, and the `insteadOf` / `pushInsteadOf` rewrites. +_REMOTE_URL_KEYS = r"^(remote\..*\.(url|pushurl)|url\..*\.(insteadof|pushinsteadof))$" + + +def _remote_url_has_newline(repo: git.Repo) -> bool: + """Whether a value `git remote -v` prints from holds a newline. + + Such a URL is split across lines of `git remote -v`, and a fragment can + look like a whole line (`https://alice:SECRET (fetch)`) with its + credential's `@` on the next one, so no line parser can redact it. Read + the raw values NUL-separated (`-z`: `\\n\\0`), where a newline + cannot hide. `with_exceptions=False` because `--get-regexp` exits 1 when + nothing matches; a real failure leaves the output empty, and the + `git remote -v` that follows reports it. + """ + entries = repo.git.config( + "-z", "--get-regexp", _REMOTE_URL_KEYS, with_exceptions=False + ) + return any("\n" in entry.partition("\n")[2] for entry in entries.split("\0")) + + +def git_remote(repo: git.Repo) -> str: + """The repository's remotes with their fetch and push URLs, as `git remote -v`. + + Each line is `\\t (fetch|push)`, a partial clone's fetch line + followed by its promisor filter (` [blob:none]`); the URL goes through + `redact_url` so no credential reaches the client. Lines are split on + '\\n' alone, the only separator git prints, not on every separator + `str.splitlines` knows, which would cut a URL holding U+2028 in two. A + URL holding a newline itself is refused (`_remote_url_has_newline`). + """ + if _remote_url_has_newline(repo): + raise ValueError( + "A configured remote URL contains a newline, so the remotes cannot " + "be listed without risking a credential in it; git refuses such " + "URLs as well" + ) + output = _git_text(repo, "remote", "-v").rstrip("\n") + if not output: + return "No remotes configured" + lines = [] + for line in output.split("\n"): + match = _REMOTE_LINE.match(line) + if match is None: + # Not a line in the shape `git remote -v` prints (a URL holding a + # newline, say): it cannot be parsed, so none of it is returned. + lines.append(REDACTED) + continue + url = redact_url(match["url"]) + lines.append( + f"{match['name']}\t{url} ({match['direction']}){match['filter'] or ''}" + ) + return "\n".join(lines) + + # The failures a tool reports to the model as a `ToolError`, each with the # message it was raised with: the guards' `ValueError`s and # `FlagInjectionError`s, git and GitPython errors (`GitError`: a failed @@ -596,6 +862,93 @@ def writes(*, destructive: bool, idempotent: bool) -> ToolAnnotations: ) +class GitPrompts(str, Enum): + COMMIT_MESSAGE = "git_commit_message" + SUMMARIZE_CHANGES = "git_summarize_changes" + + +def _fenced(text: str, info: str = "diff") -> str: + """`text` in a Markdown code fence longer than any backtick run inside it. + + A diff can itself hold a ``` line (a Markdown file's code block), which + would close a fixed three-backtick fence early. + """ + # One pass for the longest run: growing the fence and searching again for + # each extra backtick is quadratic in a long run, under the git lock. + longest = run = 0 + for char in text: + run = run + 1 if char == "`" else 0 + longest = max(longest, run) + fence = "`" * max(3, longest + 1) + body = text.rstrip("\n") + return f"{fence}{info}\n{body}\n{fence}" + + +def commit_message_prompt(repo: git.Repo) -> str: + """The git_commit_message prompt's text: the staged diff and the request.""" + diff = git_diff_staged(repo) + if not diff.strip(): + return ( + "There are no staged changes in this repository, so there is nothing " + "to write a commit message for. Tell the user to stage changes first " + "(git_add); git_status shows what is modified or untracked." + ) + return ( + "Write a commit message for the staged changes below. Start with an " + "imperative subject line of at most 72 characters, then a blank line and " + "a short body explaining what changed and why, only if the subject alone " + "does not say enough. Follow the repository's existing convention if " + "its history shows one (git_log).\n\n" + f"Staged changes:\n{_fenced(diff)}" + ) + + +def summarize_changes_prompt(repo: git.Repo, target: str | None = None) -> str: + """The git_summarize_changes prompt's text: the changes and the request.""" + if target: + # git_diff applies the flag-injection guard and resolves the target. + diff = git_diff(repo, target) + scope = f"compared with `{target}`" + changes = f"Diff against {target}:\n{_fenced(diff)}" if diff.strip() else "" + else: + staged = git_diff_staged(repo) + unstaged = git_diff_unstaged(repo) + scope = "in the working tree" + parts = [f"Status:\n{_fenced(git_status(repo), 'text')}"] + if staged.strip(): + parts.append(f"Staged changes:\n{_fenced(staged)}") + if unstaged.strip(): + parts.append(f"Unstaged changes:\n{_fenced(unstaged)}") + # The porcelain status is empty only on a clean tree; an untracked file + # alone leaves both diffs empty but is still a change to summarize. + clean = not repo.git.status("--porcelain").strip() + changes = "" if clean else "\n\n".join(parts) + if not changes: + return f"There are no changes {scope} to summarize. Tell the user so." + return ( + f"Summarize the changes {scope} below for a reviewer, in plain " + "language. Group related changes, explain the intent behind them rather " + "than restating each line, and point out anything that looks unfinished " + "or risky.\n\n" + f"{changes}" + ) + + +@contextmanager +def reported_as_invalid_params() -> Iterator[None]: + """Turn an anticipated failure in a prompt into a JSON-RPC -32602 error. + + `prompts/get` has no `isError` result: the spec's error handling for it is + a JSON-RPC error, and a repository path, target or guard the request named + is an invalid argument. Without this, MCPServer hides the message behind + "Error rendering prompt" (code 0 or -32603, depending on the era). + """ + try: + yield + except ANTICIPATED_ERRORS as e: + raise MCPError(code=INVALID_PARAMS, message=str(e)) from e + + def create_server(repository: Path | None) -> MCPServer: """The git server, confined to `repository` when one is given.""" @@ -636,6 +989,43 @@ def open_repo(repo_path: str) -> git.Repo: raise return repo + # Prompts (#5195): each embeds what the repository holds now in one user + # message, through the same `open_repo` (and so the same confinement) and + # the same guarded helpers as the tools, under the same lock. + @server.prompt( + name=GitPrompts.COMMIT_MESSAGE.value, + title="Draft a commit message", + description="Asks the model for a commit message for the staged changes", + ) + def commit_message( + repo_path: Annotated[str, Field(description="The path to the Git repository.")], + ) -> str: + with _GIT_LOCK, reported_as_invalid_params(), open_repo(repo_path) as repo: + return commit_message_prompt(repo) + + @server.prompt( + name=GitPrompts.SUMMARIZE_CHANGES.value, + title="Summarize changes", + description=( + "Asks the model to summarize the working tree's staged and unstaged " + "changes, or the diff against a branch or commit" + ), + ) + def summarize_changes( + repo_path: Annotated[str, Field(description="The path to the Git repository.")], + target: Annotated[ + Optional[str], + Field( + description=( + "Branch or commit to diff against. Defaults to the working " + "tree's staged and unstaged changes" + ) + ), + ] = None, + ) -> str: + with _GIT_LOCK, reported_as_invalid_params(), open_repo(repo_path) as repo: + return summarize_changes_prompt(repo, target) + @server.tool( name=GitTools.STATUS.value, description="Shows the working tree status", @@ -672,7 +1062,7 @@ def diff_staged(repo_path: str, context_lines: int = DEFAULT_CONTEXT_LINES) -> s @server.tool( name=GitTools.DIFF.value, - description="Shows differences between branches or commits", + description="Shows differences between branches or commits, or across a revision range (main..feature, main...feature)", annotations=READ_ONLY, structured_output=False, ) @@ -774,6 +1164,37 @@ def create_branch( def checkout(repo_path: str, branch_name: str) -> str: return git_checkout(open_repo(repo_path), branch_name) + @server.tool( + name=GitTools.MERGE.value, + description=( + "Merges a branch into the current branch. Fast-forward only unless " + "fast_forward_only is false; refuses a dirty working tree, and " + "aborts a merge that conflicts, leaving the repository unchanged" + ), + annotations=writes(destructive=False, idempotent=True), + structured_output=False, + ) + @reports_failures + def merge( + repo_path: str, + source_branch: Annotated[ + str, + Field( + description=("Branch, tag or commit to merge into the current branch") + ), + ], + fast_forward_only: Annotated[ + bool, + Field( + description=( + "Only fast-forward, failing if the branches have diverged. " + "False allows a merge commit" + ) + ), + ] = True, + ) -> str: + return git_merge(open_repo(repo_path), source_branch, fast_forward_only) + @server.tool( name=GitTools.SHOW.value, description="Shows the contents of a commit, or of a file or directory given as :", @@ -814,6 +1235,22 @@ def branch( ) -> str: return git_branch(open_repo(repo_path), branch_type, contains, not_contains) + @server.tool( + name=GitTools.REMOTE.value, + description=( + "Lists the repository's remotes with their fetch and push URLs, " + "like `git remote -v`. Credentials embedded in a URL are shown as " + "" + ), + annotations=READ_ONLY, + structured_output=False, + ) + @reports_failures + def remote( + repo_path: Annotated[str, Field(description="The path to the Git repository.")], + ) -> str: + return git_remote(open_repo(repo_path)) + return server diff --git a/src/git/tests/snapshots/list_tools.json b/src/git/tests/snapshots/list_tools.json index 3cef80b843..1bad348fa8 100644 --- a/src/git/tests/snapshots/list_tools.json +++ b/src/git/tests/snapshots/list_tools.json @@ -81,7 +81,7 @@ }, { "name": "git_diff", - "description": "Shows differences between branches or commits", + "description": "Shows differences between branches or commits, or across a revision range (main..feature, main...feature)", "inputSchema": { "properties": { "repo_path": { @@ -318,6 +318,41 @@ "openWorldHint": false } }, + { + "name": "git_merge", + "description": "Merges a branch into the current branch. Fast-forward only unless fast_forward_only is false; refuses a dirty working tree, and aborts a merge that conflicts, leaving the repository unchanged", + "inputSchema": { + "properties": { + "repo_path": { + "title": "Repo Path", + "type": "string" + }, + "source_branch": { + "description": "Branch, tag or commit to merge into the current branch", + "title": "Source Branch", + "type": "string" + }, + "fast_forward_only": { + "default": true, + "description": "Only fast-forward, failing if the branches have diverged. False allows a merge commit", + "title": "Fast Forward Only", + "type": "boolean" + } + }, + "required": [ + "repo_path", + "source_branch" + ], + "type": "object", + "title": "mergeArguments" + }, + "annotations": { + "readOnlyHint": false, + "destructiveHint": false, + "idempotentHint": true, + "openWorldHint": false + } + }, { "name": "git_show", "description": "Shows the contents of a commit, or of a file or directory given as :", @@ -401,6 +436,30 @@ "idempotentHint": true, "openWorldHint": false } + }, + { + "name": "git_remote", + "description": "Lists the repository's remotes with their fetch and push URLs, like `git remote -v`. Credentials embedded in a URL are shown as ", + "inputSchema": { + "properties": { + "repo_path": { + "description": "The path to the Git repository.", + "title": "Repo Path", + "type": "string" + } + }, + "required": [ + "repo_path" + ], + "type": "object", + "title": "remoteArguments" + }, + "annotations": { + "readOnlyHint": true, + "destructiveHint": false, + "idempotentHint": true, + "openWorldHint": false + } } ] } diff --git a/src/git/tests/test_identity.py b/src/git/tests/test_identity.py new file mode 100644 index 0000000000..8fbfd9b356 --- /dev/null +++ b/src/git/tests/test_identity.py @@ -0,0 +1,118 @@ +"""Where `git_commit` takes the commit's author and committer from (#619). + +The server sets no identity of its own: `git_commit` goes through GitPython's +`IndexFile.commit`, whose `Actor` reads `GIT_AUTHOR_NAME`/`GIT_AUTHOR_EMAIL` +and `GIT_COMMITTER_NAME`/`GIT_COMMITTER_EMAIL` from the server's environment +first, then `user.name`/`user.email` from the repository's config, the user's +`~/.gitconfig` and `$XDG_CONFIG_HOME/git/config`. The README's "Commit +identity" section tells users to configure the server exactly that way (an +`env` block, `docker run -e`, or a mounted gitconfig), so these tests pin each +route through the protocol: a dependency upgrade or a refactor that stopped +honoring one would otherwise silently commit as `root@`. +""" + +from pathlib import Path + +import git +import pytest + +from conftest import connect, disable_autocrlf, text_result, wire + +IDENTITY_VARS = ( + "GIT_AUTHOR_NAME", + "GIT_AUTHOR_EMAIL", + "GIT_COMMITTER_NAME", + "GIT_COMMITTER_EMAIL", +) + + +@pytest.fixture(autouse=True) +def clean_identity_env(monkeypatch: pytest.MonkeyPatch) -> None: + """Start every test with no identity in the environment, whatever CI sets.""" + for name in IDENTITY_VARS: + monkeypatch.delenv(name, raising=False) + + +async def commit_new_file(repo: git.Repo) -> git.Commit: + """Stage a new file, commit it through the protocol, return the commit.""" + assert repo.working_dir is not None + root = Path(repo.working_dir) + (root / "new.txt").write_text("new\n", newline="\n") + repo.index.add(["new.txt"]) + async with connect(None) as session: + result = wire( + await session.call_tool( + "git_commit", {"repo_path": str(root), "message": "add new"} + ) + ) + head = repo.head.commit + assert result == text_result( + f"Changes committed successfully with hash {head.hexsha}" + ) + return head + + +async def test_environment_variables_set_author_and_committer( + repo: git.Repo, monkeypatch: pytest.MonkeyPatch +): + # The `repo` fixture configures user.name/user.email repo-locally; the + # environment wins over it, as it does for `git commit`. + monkeypatch.setenv("GIT_AUTHOR_NAME", "Ada Author") + monkeypatch.setenv("GIT_AUTHOR_EMAIL", "ada@example.com") + monkeypatch.setenv("GIT_COMMITTER_NAME", "Cy Committer") + monkeypatch.setenv("GIT_COMMITTER_EMAIL", "cy@example.com") + + head = await commit_new_file(repo) + + assert (head.author.name, head.author.email) == ("Ada Author", "ada@example.com") + assert (head.committer.name, head.committer.email) == ( + "Cy Committer", + "cy@example.com", + ) + + +async def test_author_variables_alone_leave_the_committer_to_config( + repo: git.Repo, monkeypatch: pytest.MonkeyPatch +): + # Author and committer are resolved separately, which is why the README + # tells users to set all four variables. + monkeypatch.setenv("GIT_AUTHOR_NAME", "Ada Author") + monkeypatch.setenv("GIT_AUTHOR_EMAIL", "ada@example.com") + + head = await commit_new_file(repo) + + assert (head.author.name, head.author.email) == ("Ada Author", "ada@example.com") + assert (head.committer.name, head.committer.email) == ( + "Test User", + "test@example.com", + ) + + +async def test_user_gitconfig_in_home_sets_the_identity( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch +): + # The mounted-gitconfig route: no environment variables and no repo-local + # identity, only a ~/.gitconfig under the server's HOME. + home = tmp_path / "home" + home.mkdir() + (home / ".gitconfig").write_text( + "[user]\n\tname = Gita Config\n\temail = gita@example.com\n" + ) + monkeypatch.setenv("HOME", str(home)) + monkeypatch.setenv("USERPROFILE", str(home)) # what expanduser reads on Windows + monkeypatch.delenv("XDG_CONFIG_HOME", raising=False) + + repo = git.Repo.init(tmp_path / "repo", initial_branch="main") + try: + disable_autocrlf(repo) + head = await commit_new_file(repo) + assert (head.author.name, head.author.email) == ( + "Gita Config", + "gita@example.com", + ) + assert (head.committer.name, head.committer.email) == ( + "Gita Config", + "gita@example.com", + ) + finally: + repo.close() diff --git a/src/git/tests/test_merge.py b/src/git/tests/test_merge.py new file mode 100644 index 0000000000..879931e0f3 --- /dev/null +++ b/src/git/tests/test_merge.py @@ -0,0 +1,368 @@ +"""Protocol-level tests for the git_merge tool (#659). + +git_merge merges a source into the current branch and either succeeds or +leaves the repository exactly as it was: it refuses a dirty tree, a detached +HEAD and a merge already in progress, allows only a fast-forward unless told +otherwise, and aborts a merge that stops on conflicts. Each test drives the +real server through a client (`conftest.connect`) and checks both the reply +and the repository afterwards. +""" + +import stat +import sys +from pathlib import Path +from typing import Any + +import git +import pytest + +from conftest import connect, text_result, tool_error, wire + + +def root_of(repo: git.Repo) -> Path: + assert repo.working_dir is not None + return Path(repo.working_dir) + + +async def merge( + repo: git.Repo, source: str, *, repository: Path | None = None, **extra: Any +) -> dict[str, Any]: + async with connect(repository) as session: + return wire( + await session.call_tool( + "git_merge", + {"repo_path": str(root_of(repo)), "source_branch": source, **extra}, + ) + ) + + +def commit_file(repo: git.Repo, name: str, content: str) -> git.Commit: + (root_of(repo) / name).write_text(content, newline="\n") + repo.index.add([name]) + return repo.index.commit(f"write {name}") + + +def diverge(repo: git.Repo, *, conflicting: bool) -> None: + """Give `feature` and `main` one commit each since they forked. + + With `conflicting`, both change test.txt; otherwise they touch different + files. Leaves `main` checked out. + """ + repo.git.checkout("-b", "feature") + commit_file(repo, "test.txt" if conflicting else "feature.txt", "feature\n") + repo.git.checkout("main") + commit_file(repo, "test.txt" if conflicting else "main.txt", "main\n") + + +def assert_untouched(repo: git.Repo, head: str) -> None: + assert repo.head.commit.hexsha == head + assert repo.active_branch.name == "main" + assert not (Path(repo.git_dir) / "MERGE_HEAD").exists() + assert not repo.is_dirty(untracked_files=True) + + +async def test_fast_forward(repo: git.Repo): + repo.git.checkout("-b", "feature") + tip = commit_file(repo, "feature.txt", "feature\n") + repo.git.checkout("main") + + result = await merge(repo, "feature") + + assert result == text_result( + f"Fast-forwarded 'main' to 'feature' ({tip.hexsha[:7]})" + ) + assert repo.active_branch.name == "main" + assert repo.head.commit == tip + assert (root_of(repo) / "feature.txt").read_text() == "feature\n" + + +async def test_repeating_a_merge_changes_nothing(repo: git.Repo): + # Why the tool is annotated idempotent: once merged, the source is an + # ancestor of HEAD, so the same call again is a no-op. + repo.git.checkout("-b", "feature") + commit_file(repo, "feature.txt", "feature\n") + repo.git.checkout("main") + await merge(repo, "feature") + head = repo.head.commit.hexsha + + assert await merge(repo, "feature") == text_result( + "Already up to date: 'main' already contains 'feature'" + ) + assert_untouched(repo, head) + + +async def test_diverged_with_fast_forward_only_is_refused(repo: git.Repo): + diverge(repo, conflicting=False) + head = repo.head.commit.hexsha + + assert await merge(repo, "feature") == tool_error( + "git_merge", + "Cannot fast-forward 'main' to 'feature': the branches have diverged. " + "Nothing was changed. Pass fast_forward_only=false to create a merge " + "commit.", + ) + assert_untouched(repo, head) + + +async def test_diverged_without_fast_forward_only_creates_merge_commit( + repo: git.Repo, +): + diverge(repo, conflicting=False) + main_tip = repo.head.commit + feature_tip = repo.commit("feature") + + result = await merge(repo, "feature", fast_forward_only=False) + + merged = repo.head.commit + assert result == text_result( + f"Merged 'feature' into 'main' with merge commit {merged.hexsha[:7]}" + ) + assert merged.parents == (main_tip, feature_tip) + assert repo.active_branch.name == "main" + assert not repo.is_dirty(untracked_files=True) + assert (root_of(repo) / "feature.txt").exists() + + +async def test_fast_forward_still_fast_forwards_when_merge_commits_allowed( + repo: git.Repo, +): + # merge.ff=false in the config would make a merge commit; the tool passes + # --ff, so a fast-forward stays a fast-forward. + with repo.config_writer() as config: + config.set_value("merge", "ff", "false") + repo.git.checkout("-b", "feature") + tip = commit_file(repo, "feature.txt", "feature\n") + repo.git.checkout("main") + + result = await merge(repo, "feature", fast_forward_only=False) + + assert result == text_result( + f"Fast-forwarded 'main' to 'feature' ({tip.hexsha[:7]})" + ) + assert repo.head.commit == tip + + +async def test_conflict_is_aborted_and_reported(repo: git.Repo): + diverge(repo, conflicting=True) + head = repo.head.commit.hexsha + + assert await merge(repo, "feature", fast_forward_only=False) == tool_error( + "git_merge", + "Merge of 'feature' into 'main' stopped on conflicts in: test.txt. " + "The merge was aborted and the repository is unchanged. " + "Resolve the conflicts by hand.", + ) + assert_untouched(repo, head) + assert (root_of(repo) / "test.txt").read_text() == "main\n" + + +async def test_conflict_in_non_ascii_file_names_it_as_written(repo: git.Repo): + name = "日本語.txt" + commit_file(repo, name, "base\n") + repo.git.checkout("-b", "feature") + commit_file(repo, name, "feature\n") + repo.git.checkout("main") + commit_file(repo, name, "main\n") + head = repo.head.commit.hexsha + + assert await merge(repo, "feature", fast_forward_only=False) == tool_error( + "git_merge", + f"Merge of 'feature' into 'main' stopped on conflicts in: {name}. " + "The merge was aborted and the repository is unchanged. " + "Resolve the conflicts by hand.", + ) + assert_untouched(repo, head) + + +async def test_merge_commit_message_names_the_source(repo: git.Repo): + diverge(repo, conflicting=False) + + await merge(repo, "feature", fast_forward_only=False) + + assert repo.head.commit.message.strip() == "Merge 'feature' into 'main'" + + +async def test_fetch_head_merges_only_the_commit_it_resolves_to(repo: git.Repo): + # FETCH_HEAD can list several heads, and `git merge FETCH_HEAD` would + # merge them all as an octopus. The tool merges only the one it resolved. + repo.git.checkout("-b", "first") + first = commit_file(repo, "first.txt", "first\n") + repo.git.checkout("-b", "second", "main") + second = commit_file(repo, "second.txt", "second\n") + repo.git.checkout("main") + (Path(repo.git_dir) / "FETCH_HEAD").write_text( + f"{first.hexsha}\t\tbranch 'first' of origin\n" + f"{second.hexsha}\t\tbranch 'second' of origin\n" + ) + + result = await merge(repo, "FETCH_HEAD", fast_forward_only=False) + + assert result == text_result( + f"Fast-forwarded 'main' to 'FETCH_HEAD' ({first.hexsha[:7]})" + ) + assert repo.head.commit == first + assert not (root_of(repo) / "second.txt").exists() + + +@pytest.mark.skipif( + sys.platform == "win32", reason="git for Windows runs hooks through sh" +) +async def test_failed_merge_commit_is_aborted(repo: git.Repo): + # A merge with no conflicts can still fail to commit; here a + # pre-merge-commit hook refuses it, which leaves MERGE_HEAD behind. + diverge(repo, conflicting=False) + head = repo.head.commit.hexsha + hook = Path(repo.git_dir) / "hooks" / "pre-merge-commit" + hook.parent.mkdir(exist_ok=True) + hook.write_text("#!/bin/sh\necho 'merge refused by hook' >&2\nexit 1\n") + hook.chmod(hook.stat().st_mode | stat.S_IEXEC) + + result = await merge(repo, "feature", fast_forward_only=False) + + assert result["isError"] is True + text = result["content"][0]["text"] + assert text.startswith( + "Error executing tool git_merge: Merge of 'feature' into 'main' failed. " + "The merge was aborted and the repository is unchanged. Git reported:\n" + ) + assert "merge refused by hook" in text + assert_untouched(repo, head) + + +async def test_git_refusal_before_merging_is_reported(repo: git.Repo): + # An untracked file the merge would overwrite: git refuses before it + # changes anything, so there is no merge state to abort. + repo.git.checkout("-b", "feature") + commit_file(repo, "new.txt", "feature\n") + repo.git.checkout("main") + head = repo.head.commit.hexsha + (root_of(repo) / "new.txt").write_text("untracked\n") + + result = await merge(repo, "feature") + + assert result["isError"] is True + text = result["content"][0]["text"] + assert text.startswith( + "Error executing tool git_merge: Merge of 'feature' into 'main' failed. " + "The merge was aborted and the repository is unchanged. Git reported:\n" + ) + assert "untracked working tree files would be overwritten" in text + assert repo.head.commit.hexsha == head + assert (root_of(repo) / "new.txt").read_text() == "untracked\n" + + +async def test_ignored_file_the_source_tracks_is_not_overwritten(repo: git.Repo): + # is_dirty() does not see an ignored file, and git overwrites one by + # default; the tool passes --no-overwrite-ignore so git refuses instead. + commit_file(repo, ".gitignore", ".env\n") + repo.git.checkout("-b", "feature") + (root_of(repo) / ".env").write_text("tracked\n") + repo.git.add("-f", ".env") + repo.index.commit("track .env") + repo.git.checkout("main") + head = repo.head.commit.hexsha + (root_of(repo) / ".env").write_text("SECRET=local\n") + + result = await merge(repo, "feature") + + assert result["isError"] is True + assert "would be overwritten by merge" in result["content"][0]["text"] + assert repo.head.commit.hexsha == head + assert (root_of(repo) / ".env").read_text() == "SECRET=local\n" + + +@pytest.mark.parametrize("options", ["--no-commit", "--squash"]) +async def test_configured_merge_options_still_complete_the_merge( + repo: git.Repo, options: str +): + # branch..mergeOptions could make git stop short of a commit (or + # squash); the tool overrides them, so a reported merge commit is real. + diverge(repo, conflicting=False) + with repo.config_writer() as config: + config.set_value('branch "main"', "mergeOptions", options) + main_tip = repo.head.commit + feature_tip = repo.commit("feature") + + result = await merge(repo, "feature", fast_forward_only=False) + + merged = repo.head.commit + assert result == text_result( + f"Merged 'feature' into 'main' with merge commit {merged.hexsha[:7]}" + ) + assert merged.parents == (main_tip, feature_tip) + assert not (Path(repo.git_dir) / "MERGE_HEAD").exists() + assert not repo.is_dirty(untracked_files=True) + + +async def test_dirty_working_tree_is_refused(repo: git.Repo): + repo.git.checkout("-b", "feature") + commit_file(repo, "feature.txt", "feature\n") + repo.git.checkout("main") + head = repo.head.commit.hexsha + (root_of(repo) / "test.txt").write_text("edited\n") + + assert await merge(repo, "feature") == tool_error( + "git_merge", + "Cannot merge into 'main': the working tree has uncommitted changes. " + "Commit or discard them first, so a failed merge can be undone cleanly.", + ) + assert repo.head.commit.hexsha == head + assert (root_of(repo) / "test.txt").read_text() == "edited\n" + + +async def test_merge_in_progress_is_refused(repo: git.Repo): + diverge(repo, conflicting=False) + repo.git.merge("--no-commit", "--no-ff", "feature") + + assert await merge(repo, "feature") == tool_error( + "git_merge", + "A merge is already in progress; conclude it with git_commit first", + ) + assert (Path(repo.git_dir) / "MERGE_HEAD").exists() + + +async def test_detached_head_is_refused(repo: git.Repo): + repo.git.checkout("--detach") + + assert await merge(repo, "main") == tool_error( + "git_merge", + "HEAD is detached; check out the branch to merge into first (git_checkout)", + ) + + +async def test_unborn_branch_is_refused(repo: git.Repo): + repo.git.checkout("--orphan", "empty") + repo.git.rm("-rf", "--cached", ".") + + assert await merge(repo, "main") == tool_error( + "git_merge", "Branch 'empty' has no commits yet; nothing to merge into" + ) + + +async def test_unknown_source_is_error(repo: git.Repo): + assert await merge(repo, "nope") == tool_error( + "git_merge", "Ref 'nope' did not resolve to an object" + ) + + +async def test_source_that_is_not_a_commit_is_error(repo: git.Repo): + result = await merge(repo, "HEAD^{tree}") + assert result["isError"] is True + assert repo.head.commit.hexsha == repo.commit("main").hexsha + + +async def test_rejects_flag_injection(repo: git.Repo): + assert await merge(repo, "--abort") == tool_error( + "git_merge", "Invalid source branch: '--abort' - cannot start with '-'" + ) + + +async def test_repo_path_outside_repository_is_refused(repo: git.Repo, tmp_path: Path): + allowed = tmp_path / "allowed" + allowed.mkdir() + git.Repo.init(allowed).close() + + result = await merge(repo, "main", repository=allowed) + + assert result["isError"] is True + assert "is outside the allowed repository" in result["content"][0]["text"] diff --git a/src/git/tests/test_prompts.py b/src/git/tests/test_prompts.py new file mode 100644 index 0000000000..75f6de9fa9 --- /dev/null +++ b/src/git/tests/test_prompts.py @@ -0,0 +1,291 @@ +# The git prompts (#5195), driven through a real `mcp` 2.x `Client` in both +# protocol eras, plus the prompt-text builders called directly. +# +# The prompts exist to demonstrate the Prompts feature beside the tools, so +# what matters is what a client receives from prompts/list and prompts/get: +# the names and arguments, one user message embedding the repository's +# changes, and a JSON-RPC -32602 error (prompts/get has no isError result) when +# the repository path, the target or a guard rejects the request. Each era test +# is parametrized over both eras, since nothing about prompts should differ +# between them. + +from pathlib import Path +from typing import Any + +import git +import pytest +from mcp.shared.exceptions import MCPError +from mcp.types import INVALID_PARAMS, GetPromptResult, TextContent + +from mcp_server_git.server import ( + _fenced, + commit_message_prompt, + summarize_changes_prompt, +) + +from test_eras import ERAS, MODERN, connect + +COMMIT_MESSAGE = "git_commit_message" +SUMMARIZE_CHANGES = "git_summarize_changes" + + +def repo_dir(repo: git.Repo) -> Path: + return Path(repo.working_dir) + + +def message_text(result: GetPromptResult) -> str: + """The text of a prompt result holding one user message of text.""" + assert len(result.messages) == 1 + message = result.messages[0] + assert message.role == "user" + assert isinstance(message.content, TextContent) + return message.content.text + + +def prompt_wire(prompts: Any) -> list[dict[str, Any]]: + return [ + p.model_dump(by_alias=True, mode="json", exclude_none=True) for p in prompts + ] + + +EXPECTED_PROMPTS = [ + { + "name": COMMIT_MESSAGE, + "title": "Draft a commit message", + "description": "Asks the model for a commit message for the staged changes", + "arguments": [ + { + "name": "repo_path", + "description": "The path to the Git repository.", + "required": True, + } + ], + }, + { + "name": SUMMARIZE_CHANGES, + "title": "Summarize changes", + "description": ( + "Asks the model to summarize the working tree's staged and unstaged " + "changes, or the diff against a branch or commit" + ), + "arguments": [ + { + "name": "repo_path", + "description": "The path to the Git repository.", + "required": True, + }, + { + "name": "target", + "description": ( + "Branch or commit to diff against. Defaults to the working " + "tree's staged and unstaged changes" + ), + "required": False, + }, + ], + }, +] + + +# --- Over the protocol, in both eras ---------------------------------------- + + +@pytest.mark.parametrize(("mode", "negotiated"), ERAS) +async def test_prompts_are_listed_in_each_era( + repo: git.Repo, mode: str, negotiated: str +) -> None: + async with connect(mode, repo_dir(repo)) as client: + assert client.session.protocol_version == negotiated + prompts = prompt_wire((await client.list_prompts()).prompts) + assert prompts == EXPECTED_PROMPTS + + +@pytest.mark.parametrize(("mode", "negotiated"), ERAS) +async def test_commit_message_embeds_the_staged_diff( + repo: git.Repo, mode: str, negotiated: str +) -> None: + (repo_dir(repo) / "feature.txt").write_text("hello prompt world\n") + repo.index.add(["feature.txt"]) + (repo_dir(repo) / "test.txt").write_text("unstaged edit\n") + async with connect(mode, repo_dir(repo)) as client: + result = await client.get_prompt( + COMMIT_MESSAGE, {"repo_path": str(repo_dir(repo))} + ) + assert result.description == ( + "Asks the model for a commit message for the staged changes" + ) + text = message_text(result) + assert text.startswith("Write a commit message for the staged changes below.") + assert "Staged changes:\n```diff\ndiff --git a/feature.txt b/feature.txt" in text + assert "+hello prompt world" in text + # Only what is staged: the unstaged edit to test.txt is not in it. + assert "unstaged edit" not in text + + +@pytest.mark.parametrize(("mode", "negotiated"), ERAS) +async def test_summarize_changes_embeds_the_working_tree( + repo: git.Repo, mode: str, negotiated: str +) -> None: + (repo_dir(repo) / "feature.txt").write_text("staged line\n") + repo.index.add(["feature.txt"]) + (repo_dir(repo) / "test.txt").write_text("unstaged edit\n") + (repo_dir(repo) / "untracked.txt").write_text("new\n") + async with connect(mode, repo_dir(repo)) as client: + result = await client.get_prompt( + SUMMARIZE_CHANGES, {"repo_path": str(repo_dir(repo))} + ) + text = message_text(result) + assert text.startswith("Summarize the changes in the working tree below") + assert "Status:\n```text\n" in text + assert "untracked.txt" in text + assert "Staged changes:\n```diff\n" in text + assert "+staged line" in text + assert "Unstaged changes:\n```diff\n" in text + assert "+unstaged edit" in text + + +@pytest.mark.parametrize(("mode", "negotiated"), ERAS) +async def test_summarize_changes_against_a_target( + repo: git.Repo, mode: str, negotiated: str +) -> None: + (repo_dir(repo) / "test.txt").write_text("committed edit\n") + repo.index.add(["test.txt"]) + repo.index.commit("second commit") + async with connect(mode, repo_dir(repo)) as client: + result = await client.get_prompt( + SUMMARIZE_CHANGES, + {"repo_path": str(repo_dir(repo)), "target": "HEAD~1"}, + ) + text = message_text(result) + assert text.startswith("Summarize the changes compared with `HEAD~1` below") + assert "Diff against HEAD~1:\n```diff\n" in text + assert "+committed edit" in text + + +@pytest.mark.parametrize(("mode", "negotiated"), ERAS) +async def test_target_flag_injection_is_invalid_params( + repo: git.Repo, mode: str, negotiated: str +) -> None: + # git_diff's guard applies unchanged, and its message reaches the client. + async with connect(mode, repo_dir(repo)) as client: + with pytest.raises(MCPError) as excinfo: + await client.get_prompt( + SUMMARIZE_CHANGES, + {"repo_path": str(repo_dir(repo)), "target": "--output=/tmp/x"}, + ) + assert excinfo.value.error.code == INVALID_PARAMS + assert excinfo.value.error.message == ( + "Invalid target: '--output=/tmp/x' - cannot start with '-'" + ) + + +@pytest.mark.parametrize(("mode", "negotiated"), ERAS) +async def test_unknown_target_is_invalid_params( + repo: git.Repo, mode: str, negotiated: str +) -> None: + async with connect(mode, repo_dir(repo)) as client: + with pytest.raises(MCPError) as excinfo: + await client.get_prompt( + SUMMARIZE_CHANGES, + {"repo_path": str(repo_dir(repo)), "target": "no-such-ref"}, + ) + assert excinfo.value.error.code == INVALID_PARAMS + assert "no-such-ref" in excinfo.value.error.message + + +@pytest.mark.parametrize("prompt", [COMMIT_MESSAGE, SUMMARIZE_CHANGES]) +@pytest.mark.parametrize(("mode", "negotiated"), ERAS) +async def test_repository_confinement_applies_to_prompts( + tmp_path: Path, mode: str, negotiated: str, prompt: str +) -> None: + # Both prompts open the repository through open_repo, so --repository + # confines them exactly as it confines the tools. + allowed = git.Repo.init(tmp_path / "allowed") + other = git.Repo.init(tmp_path / "other") + try: + async with connect(mode, repo_dir(allowed)) as client: + with pytest.raises(MCPError) as excinfo: + await client.get_prompt(prompt, {"repo_path": str(repo_dir(other))}) + finally: + allowed.close() + other.close() + assert excinfo.value.error.code == INVALID_PARAMS + assert "is outside the allowed repository" in excinfo.value.error.message + + +async def test_a_path_that_is_not_a_repository_is_invalid_params( + tmp_path: Path, +) -> None: + async with connect(MODERN, None) as client: + with pytest.raises(MCPError) as excinfo: + await client.get_prompt(COMMIT_MESSAGE, {"repo_path": str(tmp_path)}) + assert excinfo.value.error.code == INVALID_PARAMS + + +async def test_unknown_prompt_and_missing_argument_are_protocol_errors( + repo: git.Repo, +) -> None: + # Both are rejected by MCPServer before a prompt function runs. The code + # is the SDK's to choose (python-sdk#3669 moves both to -32602), so only + # that each is a JSON-RPC error, not a result, is pinned here. + async with connect(MODERN, repo_dir(repo)) as client: + with pytest.raises(MCPError): + await client.get_prompt("git_no_such_prompt", {}) + with pytest.raises(MCPError): + await client.get_prompt(COMMIT_MESSAGE, {}) + + +# --- The prompt text, called directly ---------------------------------------- + + +def test_commit_message_with_nothing_staged(repo: git.Repo) -> None: + (repo_dir(repo) / "test.txt").write_text("unstaged edit\n") + text = commit_message_prompt(repo) + assert text.startswith("There are no staged changes in this repository") + + +def test_summarize_a_clean_working_tree(repo: git.Repo) -> None: + assert summarize_changes_prompt(repo) == ( + "There are no changes in the working tree to summarize. Tell the user so." + ) + + +def test_summarize_only_unstaged_changes(repo: git.Repo) -> None: + (repo_dir(repo) / "test.txt").write_text("unstaged edit\n") + text = summarize_changes_prompt(repo) + assert "Unstaged changes:" in text + assert "Staged changes:" not in text + + +def test_summarize_only_staged_changes(repo: git.Repo) -> None: + (repo_dir(repo) / "test.txt").write_text("staged edit\n") + repo.index.add(["test.txt"]) + text = summarize_changes_prompt(repo) + assert "Staged changes:" in text + assert "Unstaged changes:" not in text + + +def test_summarize_only_untracked_files(repo: git.Repo) -> None: + # Both diffs are empty, but the status lists the new file. + (repo_dir(repo) / "untracked.txt").write_text("new\n") + text = summarize_changes_prompt(repo) + assert text.startswith("Summarize the changes in the working tree below") + assert "untracked.txt" in text + assert "Staged changes:" not in text + assert "Unstaged changes:" not in text + + +def test_summarize_against_a_target_with_no_difference(repo: git.Repo) -> None: + assert summarize_changes_prompt(repo, "HEAD") == ( + "There are no changes compared with `HEAD` to summarize. Tell the user so." + ) + + +def test_fence_outgrows_backticks_in_the_content() -> None: + # A diff of a Markdown file can hold its own ``` fence; the embedding + # fence must be longer so it is not closed early. + assert _fenced("plain\n") == "```diff\nplain\n```" + assert _fenced("+```python\n+x\n+```\n") == ("````diff\n+```python\n+x\n+```\n````") + assert _fenced("a ```` b", "text") == "`````text\na ```` b\n`````" + # A long run is measured in one pass, not by growing the fence. + assert _fenced("`" * 10_000).startswith("`" * 10_001 + "diff\n") diff --git a/src/git/tests/test_protocol.py b/src/git/tests/test_protocol.py index 370ff2b692..4a7d7c4b80 100644 --- a/src/git/tests/test_protocol.py +++ b/src/git/tests/test_protocol.py @@ -37,8 +37,9 @@ SNAPSHOTS = Path(__file__).parent / "snapshots" -# What the server advertises in either era. MCPServer serves empty prompt and -# resource lists, so both appear; nothing ever changes, so no listChanged. +# What the server advertises in either era. MCPServer serves prompt and +# resource lists (the git prompts, #5195; no resources), so both appear; +# nothing ever changes, so no listChanged. CAPABILITIES = { "prompts": {"listChanged": False}, "resources": {"listChanged": False, "subscribe": False}, @@ -78,8 +79,8 @@ async def test_initialize_reports_package_version_as_server_version(repo: git.Re "version": version("mcp-server-git"), } assert version("mcp") != version("mcp-server-git") - # MCPServer (#5174) always serves prompts/list and resources/list (empty - # here), so it advertises both capabilities beside tools. + # MCPServer (#5174) always serves prompts/list and resources/list (the + # resource list is empty here), so it advertises both beside tools. assert init["capabilities"] == CAPABILITIES @@ -104,8 +105,10 @@ async def test_list_tools_names_in_order(): "git_log", "git_create_branch", "git_checkout", + "git_merge", "git_show", "git_branch", + "git_remote", ] @@ -316,6 +319,58 @@ async def test_git_diff_rejects_flag_injection(repo: git.Repo): ) +def _diverge(repo: git.Repo) -> None: + """Give `main` and a new `feature` branch one commit each past their base.""" + root = root_of(repo) + repo.git.checkout("-b", "feature") + (root / "feature.txt").write_text("FEATURE\n", newline="\n") + repo.index.add(["feature.txt"]) + repo.index.commit("feature commit") + repo.git.checkout("main") + (root / "main.txt").write_text("MAIN\n", newline="\n") + repo.index.add(["main.txt"]) + repo.index.commit("main commit") + + +@pytest.mark.parametrize("target", ["main..feature", "main...feature"]) +async def test_git_diff_accepts_revision_range(repo: git.Repo, target: str): + """A range reaches `git diff` as given (#5194); rev_parse refuses it whole.""" + _diverge(repo) + result = await call( + None, "git_diff", {"repo_path": str(root_of(repo)), "target": target} + ) + assert result == text_result( + f"Diff with {target}:\n" + repo.git.diff("--unified=3", target) + ) + text = result["content"][0]["text"] + assert "+FEATURE" in text + # Two dots compare the tips, so main's own commit shows as removed; three + # dots start from the merge base, so it does not appear at all. + assert ("-MAIN" in text) is (target == "main..feature") + + +@pytest.mark.parametrize( + ("target", "message"), + [ + ("main..nope", "Ref 'nope' did not resolve to an object"), + ("nope...main", "Ref 'nope' did not resolve to an object"), + ( + "main..-output=/tmp/evil", + "Invalid target: 'main..-output=/tmp/evil' - range endpoint " + "'-output=/tmp/evil' cannot start with '-'", + ), + ("..", "Ref '..' did not resolve to an object"), + ], +) +async def test_git_diff_range_endpoints_are_validated( + repo: git.Repo, target: str, message: str +): + """Each endpoint gets the single-target checks: it resolves, no leading '-'.""" + assert await call( + None, "git_diff", {"repo_path": str(root_of(repo)), "target": target} + ) == tool_error("git_diff", message) + + async def test_git_diff_of_non_utf8_content_is_a_normal_result(repo: git.Repo): """A Latin-1 file's diff reaches the client with U+FFFD in place of the undecodable bytes, as git_show's does, and serializes as UTF-8 (#5193).""" diff --git a/src/git/tests/test_remote.py b/src/git/tests/test_remote.py new file mode 100644 index 0000000000..242789f08d --- /dev/null +++ b/src/git/tests/test_remote.py @@ -0,0 +1,197 @@ +"""Tests for git_remote (#1611): listing remotes without leaking credentials. + +The tool's output goes to the model, and a remote URL can carry a credential +(`https://user:token@host/...`), so most of these pin `redact_url`'s rule: the +whole userinfo is redacted except an SSH username, which names an account and +authenticates nothing. The protocol tests drive the real server through +`conftest.connect`; the unit tests cover URL shapes git would not let a test +configure as easily. +""" + +from pathlib import Path +from typing import Any +from unittest import mock + +import git +import pytest + +from mcp_server_git import server +from mcp_server_git.server import git_remote, redact_url + +from conftest import connect, make_repo, text_result, tool_error, wire + + +async def call_remote(repository: Path | None, repo: git.Repo) -> dict[str, Any]: + assert repo.working_dir is not None + async with connect(repository) as session: + return wire( + await session.call_tool("git_remote", {"repo_path": repo.working_dir}) + ) + + +async def test_no_remotes(repo: git.Repo): + assert await call_remote(None, repo) == text_result("No remotes configured") + + +async def test_lists_fetch_and_push_urls(repo: git.Repo): + repo.create_remote("origin", "https://example.com/org/repo.git") + repo.create_remote("upstream", "git@github.com:org/repo.git") + repo.git.remote("set-url", "--push", "upstream", "/srv/mirror.git") + assert await call_remote(None, repo) == text_result( + "origin\thttps://example.com/org/repo.git (fetch)\n" + "origin\thttps://example.com/org/repo.git (push)\n" + "upstream\tgit@github.com:org/repo.git (fetch)\n" + "upstream\t/srv/mirror.git (push)" + ) + + +async def test_credentials_are_redacted_over_the_wire(repo: git.Repo): + repo.create_remote("origin", "https://user:s3cret-token@example.com/org/repo.git") + repo.create_remote("fork", "https://ghp_abc123@github.com/me/repo.git") + repo.git.remote("set-url", "--push", "fork", "ssh://git:hunter2@example.com/r") + result = await call_remote(None, repo) + assert result == text_result( + "fork\thttps://@github.com/me/repo.git (fetch)\n" + "fork\tssh://git:@example.com/r (push)\n" + "origin\thttps://@example.com/org/repo.git (fetch)\n" + "origin\thttps://@example.com/org/repo.git (push)" + ) + text = result["content"][0]["text"] + for secret in ("s3cret-token", "ghp_abc123", "hunter2"): + assert secret not in text + + +async def test_respects_repository_confinement(tmp_path: Path, repo: git.Repo): + # --repository must name a real repository, or serve() exits before the + # client connects. + allowed_repo = make_repo(tmp_path / "allowed") + allowed_repo.close() + allowed = (tmp_path / "allowed").resolve() + assert await call_remote(allowed, repo) == tool_error( + "git_remote", + f"Repository path '{repo.working_dir}' is outside the allowed repository " + f"'{allowed}'", + ) + + +async def test_not_a_repository(tmp_path: Path): + plain = tmp_path / "plain" + plain.mkdir() + async with connect(None) as session: + result = wire(await session.call_tool("git_remote", {"repo_path": str(plain)})) + assert result == tool_error("git_remote", str(plain)) + + +@pytest.mark.parametrize( + ("url", "expected"), + [ + # HTTP(S): a token is as often the username as the password. + ( + "https://user:token@example.com/r.git", + "https://@example.com/r.git", + ), + ("https://token@example.com/r.git", "https://@example.com/r.git"), + ("http://u:p@example.com:8080/r", "http://@example.com:8080/r"), + ("HTTPS://u:p@example.com/r", "HTTPS://@example.com/r"), + # Any other scheme with userinfo is treated the same way. + ("ftps://u:p@example.com/r", "ftps://@example.com/r"), + # A password holding an unescaped '@': the userinfo runs to the last + # '@' before the first '/', so none of it survives. + ("https://u:p@ss@example.com/r", "https://@example.com/r"), + # No path at all. + ("https://u:p@example.com", "https://@example.com"), + # SSH: the username stays, a password does not. + ("ssh://git@example.com/r.git", "ssh://git@example.com/r.git"), + ("ssh://git:pw@example.com/r.git", "ssh://git:@example.com/r.git"), + ("git+ssh://me:pw@example.com/r", "git+ssh://me:@example.com/r"), + ("SSH+GIT://me@example.com/r", "SSH+GIT://me@example.com/r"), + # No userinfo: unchanged. + ("https://example.com/r.git", "https://example.com/r.git"), + ("https://example.com/path@x/r.git", "https://example.com/path@x/r.git"), + ("git@github.com:org/r.git", "git@github.com:org/r.git"), + ("/srv/repos/r.git", "/srv/repos/r.git"), + ("file:///srv/repos/r.git", "file:///srv/repos/r.git"), + # A remote-helper address: the URL after `::` is redacted as a URL. + ( + "https::https://alice:token@example.com/r.git", + "https::https://@example.com/r.git", + ), + ("ext::ssh://git:pw@example.com/r", "ext::ssh://git:@example.com/r"), + ("https::https://example.com/r.git", "https::https://example.com/r.git"), + # A password holding a line separator other than '\n'. + ("https://u:s3cret\u2028tok@example.com/r", "https://@example.com/r"), + ], +) +def test_redact_url(url: str, expected: str): + assert redact_url(url) == expected + + +def test_unparseable_line_is_withheld(repo: git.Repo): + # `git remote -v` always prints `\t ()`. A line in + # any other shape (a URL holding a raw newline is split across two) cannot + # be parsed, so none of it is returned. + with mock.patch.object( + server, + "_git_text", + return_value=( + "origin\thttps://u:pa\nss@example.com/r (fetch)\n" + "origin\tno-direction\n" + "up\thttps://example.com/r (push)\n" + ), + ): + assert git_remote(repo) == ( + "\n\n\nup\thttps://example.com/r (push)" + ) + + +async def test_line_separator_in_password_is_not_split(repo: git.Repo): + # str.splitlines() would cut this URL at U+2028 and leak both halves of + # the password (Copilot review on #5373). + repo.create_remote("origin", "https://alice:s3cret\u2028token@example.com/r.git") + result = await call_remote(None, repo) + assert result == text_result( + "origin\thttps://@example.com/r.git (fetch)\n" + "origin\thttps://@example.com/r.git (push)" + ) + assert "s3cret" not in result["content"][0]["text"] + assert "token" not in result["content"][0]["text"] + + +async def test_partial_clone_filter_is_kept(repo: git.Repo): + # A promisor remote's fetch line ends in its filter (Copilot review on + # #5373); the line still parses, and only the URL is redacted. + repo.create_remote("origin", "https://alice:token@example.com/r.git") + with repo.config_writer() as config: + config.set_value('remote "origin"', "promisor", "true") + config.set_value('remote "origin"', "partialclonefilter", "blob:none") + assert await call_remote(None, repo) == text_result( + "origin\thttps://@example.com/r.git (fetch) [blob:none]\n" + "origin\thttps://@example.com/r.git (push)" + ) + + +@pytest.mark.parametrize( + "key", + ["remote.origin.url", "remote.origin.pushurl", "url.https://x/.insteadOf"], +) +async def test_newline_in_a_url_refuses_to_list(repo: git.Repo, key: str): + # Copilot review on #5373: with a newline after ` (fetch)`, the fragment + # before it reads as a whole line with no '@', so a line parser would + # return `https://alice:SECRET` unredacted. + repo.create_remote("origin", "https://example.com/r.git") + repo.git.config(key, "https://alice:SECRET (fetch)\nrest@example.com/r.git") + result = await call_remote(None, repo) + assert result == tool_error( + "git_remote", + "A configured remote URL contains a newline, so the remotes cannot be " + "listed without risking a credential in it; git refuses such URLs as well", + ) + assert "SECRET" not in result["content"][0]["text"] + + +async def test_remote_helper_url_is_redacted(repo: git.Repo): + repo.git.remote("add", "origin", "https::https://alice:token@example.com/r.git") + assert await call_remote(None, repo) == text_result( + "origin\thttps::https://@example.com/r.git (fetch)\n" + "origin\thttps::https://@example.com/r.git (push)" + ) diff --git a/src/git/tests/test_server.py b/src/git/tests/test_server.py index 538d168a80..86a2b3dd12 100644 --- a/src/git/tests/test_server.py +++ b/src/git/tests/test_server.py @@ -268,6 +268,38 @@ def test_git_diff(test_repository): assert "feature changes" in result +def test_git_diff_range_with_omitted_endpoint_means_head(test_repository): + """`a..` and `..b` default the missing endpoint to HEAD, as git does.""" + default_branch = test_repository.active_branch.name + test_repository.git.checkout("-b", "feature-range") + file_path = Path(test_repository.working_dir) / "test.txt" + file_path.write_text("range changes") + test_repository.index.add(["test.txt"]) + test_repository.index.commit("range commit") + + assert git_diff(test_repository, f"{default_branch}..") == git_diff( + test_repository, f"{default_branch}..HEAD" + ) + assert "range changes" in git_diff(test_repository, f"{default_branch}..") + assert git_diff(test_repository, "..HEAD") == "" + + +def test_git_diff_single_revision_containing_dots_is_not_split(test_repository): + """A revision that resolves whole is used whole, even with '..' inside it.""" + test_repository.index.commit("fix..bug") + + # ':/fix..bug' names the commit whose message matches, not a range. + assert git_diff(test_repository, ":/fix..bug") == "" + + +@pytest.mark.parametrize("target", ["HEAD..feature..x", "...."]) +def test_git_diff_rejects_malformed_ranges(test_repository, target): + """Only the first '..' or '...' splits; anything left must still resolve.""" + test_repository.git.branch("feature") + with pytest.raises(BadName): + git_diff(test_repository, target) + + # A non-ASCII name, and the octal escape git prints for its first character # under its default core.quotepath=true (#5129). NON_ASCII_FILENAME = "日本語.txt" diff --git a/src/git/uv.lock b/src/git/uv.lock index 3a5cd3826c..8b15a59638 100644 --- a/src/git/uv.lock +++ b/src/git/uv.lock @@ -509,7 +509,7 @@ wheels = [ [[package]] name = "mcp-server-git" -version = "2026.10.28" +version = "2026.11.4" source = { editable = "." } dependencies = [ { name = "click" }, diff --git a/src/memory/CHANGELOG.md b/src/memory/CHANGELOG.md index 97355c7e3a..e1220d3a57 100644 --- a/src/memory/CHANGELOG.md +++ b/src/memory/CHANGELOG.md @@ -1,5 +1,21 @@ # @modelcontextprotocol/server-memory +## 2.1.0 + +### Minor Changes + +- [#5394](https://github.com/modelcontextprotocol/servers/pull/5394) [`27148ef`](https://github.com/modelcontextprotocol/servers/commit/27148ef87d8836a0ca138b807866967cca04ca7a) [@cliffhall](https://github.com/cliffhall) - Add optional lightweight read modes, so a large graph need not be returned whole: `observationLimit` on `read_graph`, `search_nodes` and `open_nodes` caps the observations returned per entity and adds each entity's total as `observationCount` (`0` gives an overview of names, types and counts), `entityTypes` on `read_graph` keeps only entities of those types, and `limit` on `search_nodes` caps the number of matches. Calls without them return what they always have. The `read_graph` description now says when to use it rather than `search_nodes` or `open_nodes`. + +### Patch Changes + +- [#5362](https://github.com/modelcontextprotocol/servers/pull/5362) [`3a412f5`](https://github.com/modelcontextprotocol/servers/commit/3a412f57262e9ecada94dd7d44fc5367c5e64c80) [@cliffhall](https://github.com/cliffhall) - Warn on stderr at startup when the memory file is inside a `node_modules` directory, where reinstalling or updating the server, or clearing the npx cache, can delete it, and recommend in the README setting `MEMORY_FILE_PATH` to an absolute path you own. The default path is unchanged. + +- [#5349](https://github.com/modelcontextprotocol/servers/pull/5349) [`d7ebd6f`](https://github.com/modelcontextprotocol/servers/commit/d7ebd6fbce355230b0ab1c383da07d23f90368d8) [@cliffhall](https://github.com/cliffhall) - `create_entities`, `create_relations` and `add_observations` now declare `idempotentHint: true`. Each already skips what is stored, so repeating a call with the same arguments has no additional effect on the graph, as with the delete tools. + +- [#5357](https://github.com/modelcontextprotocol/servers/pull/5357) [`01062fc`](https://github.com/modelcontextprotocol/servers/commit/01062fc80b9c70355a6216d10c564cf9459e2680) [@cliffhall](https://github.com/cliffhall) - `add_observations` stores an observation repeated within one call once, and reports it once in `addedObservations`, instead of storing it twice. + +- [#5393](https://github.com/modelcontextprotocol/servers/pull/5393) [`942d8f1`](https://github.com/modelcontextprotocol/servers/commit/942d8f155e44242a34aa89f6943f2c21eac3047b) [@cliffhall](https://github.com/cliffhall) - The `search_nodes` and `open_nodes` tool descriptions now say when to use each one and which sibling read tool to prefer instead, so a model looks an entity up by name with `open_nodes`, searches by keyword with `search_nodes`, and leaves `read_graph` for when it needs the whole graph. + ## 2.0.1 ### Patch Changes diff --git a/src/memory/README.md b/src/memory/README.md index f56252a06c..b36aa143c7 100644 --- a/src/memory/README.md +++ b/src/memory/README.md @@ -109,17 +109,23 @@ Example: - **read_graph** - Read the entire knowledge graph - - No input required + - Optional input: + - `entityTypes` (string[], at least one): return only entities of these types, and the relations that touch one of them + - `observationLimit` (integer, 0 or more): see below - Returns complete graph structure with all entities and relations + - With `observationLimit`, each entity carries at most that many observations (the first ones, in the order they were added) and an `observationCount` with its total. `observationLimit: 0` returns names, types and counts only: a cheap overview of a large graph, from which `open_nodes` reads the entities you need in full - **search_nodes** - Search for nodes based on query - - Input: `query` (string) + - Input: `query` (string, at most 2048 characters; a longer query is rejected) - Searches across: - Entity names - Entity types - Observation content - Returns matching entities and their relations + - Optional input: + - `limit` (integer, 1 or more): return at most this many matches (the first, in stored order), with the relations that touch them + - `observationLimit` (integer, 0 or more): as for `read_graph` - **open_nodes** - Retrieve specific nodes by name @@ -128,6 +134,7 @@ Example: - Requested entities - Relations between requested entities - Silently skips non-existent nodes + - Optional input: `observationLimit` (integer, 0 or more), as for `read_graph` ### Resources @@ -240,10 +247,20 @@ On Windows, use: } ``` -- `MEMORY_FILE_PATH`: Path to the memory storage JSONL file (default: `memory.jsonl` in the server directory) +- `MEMORY_FILE_PATH`: Path to the memory storage JSONL file (default: `memory.jsonl` in the server directory). A leading `~` expands to your home directory; a relative path resolves against the server directory. Set it to an absolute path you own; see [Choosing where the graph is stored](#choosing-where-the-graph-is-stored). When `MEMORY_FILE_PATH` is not set and the server directory still holds a legacy `memory.json` but no `memory.jsonl`, the server renames `memory.json` to `memory.jsonl` at startup. If that rename fails, the server exits with an error naming both paths and the cause, rather than starting on an empty graph; fix the cause (a permission problem, say) and restart, or set `MEMORY_FILE_PATH` to choose a file explicitly. +#### Choosing where the graph is stored + +**Set `MEMORY_FILE_PATH` to an absolute path you own**, such as `~/memory.jsonl`, as in the configuration above. The default `memory.jsonl` sits in the server's own directory, next to its code. Under `npx` or an npm install, that directory is inside a `node_modules` tree, which npm manages as its own: reinstalling or updating the server, or clearing the npx cache, can delete the file and the knowledge graph in it. A relative `MEMORY_FILE_PATH` resolves against that same directory, so it has the same problem. + +- When the memory file it resolves is inside a `node_modules` directory, the server prints a warning on stderr at startup naming the file; most clients show a server's stderr in its log. Move the file to the new path when you set the variable, or the server starts on an empty graph. +- The directory must already exist: the server creates the memory file, not the directories above it. +- The default stays where it is so that existing graphs keep loading. The [Docker](#docker) setup above keeps it in a named volume mounted over `/app/dist`, so the graph there survives restarts and draws no warning. + +Saves are atomic. Every write tool writes the whole graph to a temporary file in the same directory and renames it over the memory file, so a server stopped or killed mid-save leaves the previous graph intact rather than a half-written file. The file's permission bits are kept across the rename. Writes from several processes are serialized by a lock, described next. + #### Sharing one memory file between clients Each MCP client starts its own server process, so two clients using the same `MEMORY_FILE_PATH` (or both using the default) are two processes writing one file. Every write tool takes an exclusive lock file, `.lock`, next to the memory file while it reads, changes and saves the graph, so writes from different processes take turns and none is lost. The lock file exists only while a write is in progress, so the directory holding the memory file must be writable (it already must be, for the atomic save). diff --git a/src/memory/__tests__/entry-point.test.ts b/src/memory/__tests__/entry-point.test.ts index 5593f0d64b..2dba5e8e20 100644 --- a/src/memory/__tests__/entry-point.test.ts +++ b/src/memory/__tests__/entry-point.test.ts @@ -1,5 +1,6 @@ // Tests for the memory server's startup path (#4854): main() run in-process -// over an in-memory transport, so its MEMORY_FILE_PATH handling is covered, +// over an in-memory transport, so its MEMORY_FILE_PATH handling and its +// warning for a graph file inside node_modules (#4117) are covered, // and isMainModule(), the guard that keeps importing index.ts from starting a // stdio server. The bin itself is exercised over real stdio by // stdio-smoke.test.ts. @@ -52,11 +53,34 @@ describe("main", () => { expect(errorSpy).toHaveBeenCalledWith( "Knowledge Graph MCP Server running on stdio", ); + // A file outside node_modules draws no location warning (#4117). + expect(errorSpy).not.toHaveBeenCalledWith( + expect.stringContaining("node_modules"), + ); } finally { await client.close(); await server.close(); } }); + + it("warns on stderr when the graph file is inside node_modules (#4117)", async () => { + const installDir = path.join(dir, "node_modules", "server-memory", "dist"); + await fs.mkdir(installDir, { recursive: true }); + const graphPath = path.join(installDir, "memory.jsonl"); + process.env.MEMORY_FILE_PATH = graphPath; + const errorSpy = vi.spyOn(console, "error").mockImplementation(() => {}); + + const server = await main(InMemoryTransport.createLinkedPair()[1]); + try { + expect(errorSpy).toHaveBeenCalledWith( + expect.stringMatching( + /^WARNING: the memory file .*memory\.jsonl is inside a node_modules directory/, + ), + ); + } finally { + await server.close(); + } + }); }); describe("isMainModule", () => { diff --git a/src/memory/__tests__/file-path.test.ts b/src/memory/__tests__/file-path.test.ts index 7ba32b2c40..8633ef3f5f 100644 --- a/src/memory/__tests__/file-path.test.ts +++ b/src/memory/__tests__/file-path.test.ts @@ -1,5 +1,6 @@ // Tests for how the memory server picks its graph file: MEMORY_FILE_PATH -// handling, the memory.json -> memory.jsonl migration, and "~" expansion. +// handling, the memory.json -> memory.jsonl migration, "~" expansion, and the +// startup warning for a graph file npm could delete (#4117). // // Every test runs ensureMemoryFilePath against its own temporary directory. // Vitest runs test files in parallel, so files written beside the source are @@ -14,6 +15,7 @@ import { ensureMemoryFilePath, defaultMemoryPath, expandHome, + memoryFileLocationWarning, } from "../index.js"; describe("ensureMemoryFilePath", () => { @@ -292,3 +294,53 @@ describe("expandHome", () => { ); }); }); + +describe("memoryFileLocationWarning", () => { + it("warns about a file inside an npx or npm install's node_modules", () => { + const installed = path.join( + os.homedir(), + ".npm", + "_npx", + "abc123", + "node_modules", + "@modelcontextprotocol", + "server-memory", + "dist", + "memory.jsonl", + ); + const warning = memoryFileLocationWarning(installed); + expect(warning).toContain(installed); + expect(warning).toContain("node_modules"); + expect(warning).toContain("MEMORY_FILE_PATH"); + }); + + it("warns about a Windows path inside node_modules on any platform", () => { + expect( + memoryFileLocationWarning( + "C:\\Users\\me\\AppData\\npm\\node_modules\\@modelcontextprotocol\\server-memory\\dist\\memory.jsonl", + ), + ).toContain("node_modules"); + }); + + it("does not warn about a file outside node_modules", () => { + expect( + memoryFileLocationWarning(path.join(os.homedir(), "memory.jsonl")), + ).toBeUndefined(); + }); + + it("does not warn about a directory whose name only contains node_modules", () => { + expect( + memoryFileLocationWarning( + path.join(os.homedir(), "my_node_modules_notes", "memory.jsonl"), + ), + ).toBeUndefined(); + }); + + it("does not warn about the Docker image's /app/dist, which the setup mounts as a volume", () => { + expect( + memoryFileLocationWarning( + path.join(path.sep, "app", "dist", "memory.jsonl"), + ), + ).toBeUndefined(); + }); +}); diff --git a/src/memory/__tests__/knowledge-graph.test.ts b/src/memory/__tests__/knowledge-graph.test.ts index 4ccb70a168..d8ad963070 100644 --- a/src/memory/__tests__/knowledge-graph.test.ts +++ b/src/memory/__tests__/knowledge-graph.test.ts @@ -236,6 +236,21 @@ describe("KnowledgeGraphManager", () => { ]); }).rejects.toThrow("Entity with name NonExistent not found"); }); + + it("should store and report an observation repeated in one call once", async () => { + await manager.createEntities([ + { name: "Alice", entityType: "person", observations: [] }, + ]); + + const results = await manager.addObservations([ + { entityName: "Alice", contents: ["x", "y", "x"] }, + ]); + + expect(results[0].addedObservations).toEqual(["x", "y"]); + const graph = await manager.readGraph(); + const alice = graph.entities.find((e) => e.name === "Alice"); + expect(alice?.observations).toEqual(["x", "y"]); + }); }); describe("deleteEntities", () => { diff --git a/src/memory/__tests__/read-options.test.ts b/src/memory/__tests__/read-options.test.ts new file mode 100644 index 0000000000..ec0435bf38 --- /dev/null +++ b/src/memory/__tests__/read-options.test.ts @@ -0,0 +1,249 @@ +// Tests for the read tools' optional inputs (#3953), driven through an SDK +// Client over an in-memory transport: read_graph's entityTypes filter, the +// observationLimit every read tool takes (with the observationCount it adds), +// and search_nodes' limit. Each option is optional, so the default results +// are pinned too: a call without them returns what it always has, and the +// knowledge-graph resource never carries observationCount. +import { describe, it, expect, beforeEach, afterEach } from "vitest"; +import type { Client } from "@modelcontextprotocol/client"; +import { call, connect, makeTempGraph, textOf } from "./helpers.js"; +import type { Connection } from "./helpers.js"; + +const alice = { + name: "Alice", + entityType: "person", + observations: ["works at Acme Corp", "likes tea", "plays chess"], +}; +const bob = { + name: "Bob", + entityType: "person", + observations: ["likes programming"], +}; +const acme = { + name: "Acme Corp", + entityType: "organization", + observations: ["makes anvils", "founded in 1900"], +}; +const empty = { name: "Nobody", entityType: "placeholder", observations: [] }; +const aliceKnowsBob = { from: "Alice", to: "Bob", relationType: "knows" }; +const aliceWorksAtAcme = { + from: "Alice", + to: "Acme Corp", + relationType: "works_at", +}; +const bobKnowsAlice = { from: "Bob", to: "Alice", relationType: "knows" }; + +describe("read tool options (#3953)", () => { + let conn: Connection; + let client: Client; + let cleanup: () => Promise; + + beforeEach(async () => { + const graph = await makeTempGraph(); + cleanup = graph.cleanup; + conn = await connect(graph.filePath); + client = conn.client; + await call(client, "create_entities", { + entities: [alice, bob, acme, empty], + }); + await call(client, "create_relations", { + relations: [aliceKnowsBob, aliceWorksAtAcme, bobKnowsAlice], + }); + }); + + afterEach(async () => { + await conn.close(); + await cleanup(); + }); + + // Call a tool and return its structured result, checking that the text + // block carries the same graph. + async function read(name: string, args: Record) { + const result = await call(client, name, args); + expect(result.isError).toBeFalsy(); + expect(textOf(result)).toBe( + JSON.stringify(result.structuredContent, null, 2), + ); + return result.structuredContent; + } + + // The text of a call that must fail input validation. + async function rejected(name: string, args: Record) { + const result = await call(client, name, args); + expect(result.isError).toBe(true); + return textOf(result); + } + + describe("defaults", () => { + it("read_graph without options returns every observation and no counts", async () => { + expect(await read("read_graph", {})).toEqual({ + entities: [alice, bob, acme, empty], + relations: [aliceKnowsBob, aliceWorksAtAcme, bobKnowsAlice], + }); + }); + + it("search_nodes and open_nodes without options return full entities", async () => { + expect(await read("search_nodes", { query: "acme" })).toEqual({ + entities: [alice, acme], + relations: [aliceKnowsBob, aliceWorksAtAcme, bobKnowsAlice], + }); + expect(await read("open_nodes", { names: ["Bob"] })).toEqual({ + entities: [bob], + relations: [aliceKnowsBob, bobKnowsAlice], + }); + }); + + it("the knowledge-graph resource is unaffected", async () => { + const { contents } = await client.readResource({ + uri: "memory://knowledge-graph", + }); + const [content] = contents; + expect("text" in content && JSON.parse(content.text)).toEqual({ + entities: [alice, bob, acme, empty], + relations: [aliceKnowsBob, aliceWorksAtAcme, bobKnowsAlice], + }); + }); + }); + + describe("observationLimit", () => { + it("0 gives read_graph an overview of names, types and counts", async () => { + expect(await read("read_graph", { observationLimit: 0 })).toEqual({ + entities: [ + { ...alice, observations: [], observationCount: 3 }, + { ...bob, observations: [], observationCount: 1 }, + { ...acme, observations: [], observationCount: 2 }, + { ...empty, observationCount: 0 }, + ], + relations: [aliceKnowsBob, aliceWorksAtAcme, bobKnowsAlice], + }); + }); + + it("keeps the first observations, in the order they were added", async () => { + expect(await read("read_graph", { observationLimit: 2 })).toEqual({ + entities: [ + { + ...alice, + observations: ["works at Acme Corp", "likes tea"], + observationCount: 3, + }, + { ...bob, observationCount: 1 }, + { ...acme, observationCount: 2 }, + { ...empty, observationCount: 0 }, + ], + relations: [aliceKnowsBob, aliceWorksAtAcme, bobKnowsAlice], + }); + }); + + it("applies to search_nodes", async () => { + expect( + await read("search_nodes", { query: "acme", observationLimit: 1 }), + ).toEqual({ + entities: [ + { + ...alice, + observations: ["works at Acme Corp"], + observationCount: 3, + }, + { ...acme, observations: ["makes anvils"], observationCount: 2 }, + ], + relations: [aliceKnowsBob, aliceWorksAtAcme, bobKnowsAlice], + }); + }); + + it("applies to open_nodes", async () => { + expect( + await read("open_nodes", { + names: ["Alice", "Missing"], + observationLimit: 1, + }), + ).toEqual({ + entities: [ + { + ...alice, + observations: ["works at Acme Corp"], + observationCount: 3, + }, + ], + relations: [aliceKnowsBob, aliceWorksAtAcme, bobKnowsAlice], + }); + }); + + it.each([ + ["read_graph", {}], + ["search_nodes", { query: "a" }], + ["open_nodes", { names: ["Alice"] }], + ])("%s rejects a negative or fractional limit", async (name, args) => { + expect(await rejected(name, { ...args, observationLimit: -1 })).toContain( + "observationLimit", + ); + expect( + await rejected(name, { ...args, observationLimit: 1.5 }), + ).toContain("observationLimit"); + }); + }); + + describe("read_graph entityTypes", () => { + it("keeps entities of the given types and the relations that touch them", async () => { + expect( + await read("read_graph", { entityTypes: ["organization"] }), + ).toEqual({ + entities: [acme], + relations: [aliceWorksAtAcme], + }); + }); + + it("accepts several types, and combines with observationLimit", async () => { + expect( + await read("read_graph", { + entityTypes: ["organization", "placeholder"], + observationLimit: 0, + }), + ).toEqual({ + entities: [ + { ...acme, observations: [], observationCount: 2 }, + { ...empty, observationCount: 0 }, + ], + relations: [aliceWorksAtAcme], + }); + }); + + it("matches types exactly", async () => { + expect(await read("read_graph", { entityTypes: ["Person"] })).toEqual({ + entities: [], + relations: [], + }); + }); + + it("rejects an empty list", async () => { + expect(await rejected("read_graph", { entityTypes: [] })).toContain( + "entityTypes", + ); + }); + }); + + describe("search_nodes limit", () => { + it("keeps the first matches and the relations that touch them", async () => { + expect(await read("search_nodes", { query: "acme", limit: 1 })).toEqual({ + entities: [alice], + relations: [aliceKnowsBob, aliceWorksAtAcme, bobKnowsAlice], + }); + expect(await read("search_nodes", { query: "o", limit: 2 })).toEqual({ + entities: [alice, bob], + relations: [aliceKnowsBob, aliceWorksAtAcme, bobKnowsAlice], + }); + }); + + it("returns every match when the limit exceeds them", async () => { + expect(await read("search_nodes", { query: "acme", limit: 10 })).toEqual({ + entities: [alice, acme], + relations: [aliceKnowsBob, aliceWorksAtAcme, bobKnowsAlice], + }); + }); + + it("rejects a limit below 1", async () => { + expect( + await rejected("search_nodes", { query: "a", limit: 0 }), + ).toContain("limit"); + }); + }); +}); diff --git a/src/memory/__tests__/tools.test.ts b/src/memory/__tests__/tools.test.ts index 170d5933e5..f800f4c17a 100644 --- a/src/memory/__tests__/tools.test.ts +++ b/src/memory/__tests__/tools.test.ts @@ -60,6 +60,39 @@ const graphOutputSchema = { additionalProperties: false, }; +// The read tools' output (#3953): an entity may also carry observationCount. +const readGraphOutputSchema = { + ...graphOutputSchema, + properties: { + ...graphOutputSchema.properties, + entities: { + type: "array", + items: { + ...entityItem(true), + properties: { + ...entityItem(true).properties, + observationCount: { + type: "integer", + minimum: 0, + maximum: Number.MAX_SAFE_INTEGER, + description: + "How many observations the entity holds in total; present only when observationLimit was given", + }, + }, + }, + }, + }, +}; + +// The observationLimit input every read tool takes (#3953). +const observationLimitInput = { + type: "integer", + minimum: 0, + maximum: Number.MAX_SAFE_INTEGER, + description: + "Return at most this many observations per entity (the first ones, in the order they were added) and add each entity's total as observationCount. Use 0 for names, types and counts only: a cheap overview of a large graph, after which open_nodes reads the entities you need in full. Omit to return every observation.", +}; + const statusOutputSchema = { type: "object", properties: { success: { type: "boolean" }, message: { type: "string" } }, @@ -68,10 +101,12 @@ const statusOutputSchema = { additionalProperties: false, }; +// create_entities, create_relations and add_observations skip what is already +// stored, so repeating a call with the same arguments changes nothing (#5153). const writeAnnotations = { readOnlyHint: false, destructiveHint: false, - idempotentHint: false, + idempotentHint: true, openWorldHint: false, }; const deleteAnnotations = { @@ -270,20 +305,31 @@ const expectedTools = [ { name: "read_graph", title: "Read Graph", - description: "Read the entire knowledge graph", + description: + "Read the entire knowledge graph: every entity with its observations, and every relation. The result grows with the graph, so use this only when you need everything; prefer search_nodes to find entities by keyword and open_nodes to read entities whose names you know.", inputSchema: { type: "object", - properties: {}, + properties: { + entityTypes: { + type: "array", + items: { type: "string" }, + minItems: 1, + description: + "Return only entities whose entityType is one of these (exact match), and only the relations that touch one of them. Omit to return every entity.", + }, + observationLimit: observationLimitInput, + }, $schema: JSON_SCHEMA_2020_12, }, - outputSchema: graphOutputSchema, + outputSchema: readGraphOutputSchema, annotations: readAnnotations, execution, }, { name: "search_nodes", title: "Search Nodes", - description: "Search for nodes in the knowledge graph based on a query", + description: + "Search the knowledge graph for entities whose name, type or observations contain the query (case-insensitive). Use this when you have a keyword, a topic or part of a name but not the exact entity names. Prefer open_nodes when you already know the exact names, and use read_graph only when you need the whole graph. Returns the matching entities and every relation that touches one of them.", inputSchema: { type: "object", properties: { @@ -293,18 +339,27 @@ const expectedTools = [ description: "The search query to match against entity names, types, and observation content", }, + limit: { + type: "integer", + minimum: 1, + maximum: Number.MAX_SAFE_INTEGER, + description: + "Return at most this many matching entities (the first matches, in stored order), with the relations that touch them. Omit to return every match.", + }, + observationLimit: observationLimitInput, }, required: ["query"], $schema: JSON_SCHEMA_2020_12, }, - outputSchema: graphOutputSchema, + outputSchema: readGraphOutputSchema, annotations: readAnnotations, execution, }, { name: "open_nodes", title: "Open Nodes", - description: "Open specific nodes in the knowledge graph by their names", + description: + "Retrieve specific entities by their exact names, with every relation that touches one of them. Use this when you already know the names (from the conversation, or from an earlier search_nodes or read_graph result); it is the cheapest way to read entities you can name. Prefer search_nodes when you only have a keyword or part of a name. Names that do not exist are skipped.", inputSchema: { type: "object", properties: { @@ -313,11 +368,12 @@ const expectedTools = [ items: { type: "string" }, description: "An array of entity names to retrieve", }, + observationLimit: observationLimitInput, }, required: ["names"], $schema: JSON_SCHEMA_2020_12, }, - outputSchema: graphOutputSchema, + outputSchema: readGraphOutputSchema, annotations: readAnnotations, execution, }, @@ -392,6 +448,30 @@ describe("memory tools over the protocol", () => { expect(tools).toEqual(expectedTools); }); + // The idempotentHint the three write tools declare: a repeat of the same + // call leaves the graph as the first call left it (#5153). + it("changes nothing when a write tool is repeated with the same arguments (#5153)", async () => { + const calls: [string, Record][] = [ + ["create_entities", { entities: [alice, bob] }], + ["create_relations", { relations: [aliceKnowsBob] }], + [ + "add_observations", + { observations: [{ entityName: "Bob", contents: ["plays chess"] }] }, + ], + ]; + for (const [name, args] of calls) { + expect((await call(client, name, args)).isError).toBeUndefined(); + } + // Compare around each repeat on its own, so a later repeat cannot mask + // a change an earlier one made. + for (const [name, args] of calls) { + const before = await readGraph(); + const repeat = await call(client, name, args); + expect(repeat.isError).toBeUndefined(); + expect(await readGraph()).toEqual(before); + } + }); + it("offers no prompts", async () => { // A raw request: SDK v2's listPrompts() answers an empty list without // asking a server that does not advertise prompts. @@ -651,6 +731,26 @@ describe("memory tools over the protocol", () => { relations: [], }); }); + + it("stores and reports an observation repeated in one call once", async () => { + const result = await call(client, "add_observations", { + observations: [ + { entityName: "Alice", contents: ["plays chess", "plays chess"] }, + ], + }); + const expected = [ + { entityName: "Alice", addedObservations: ["plays chess"] }, + ]; + expect(result.isError).toBeUndefined(); + expect(result.structuredContent).toEqual({ results: expected }); + expect(await readGraph()).toEqual({ + entities: [ + { ...alice, observations: [...alice.observations, "plays chess"] }, + bob, + ], + relations: [], + }); + }); }); describe("delete_entities", () => { diff --git a/src/memory/index.ts b/src/memory/index.ts index 4892ee85a2..b81397ee60 100644 --- a/src/memory/index.ts +++ b/src/memory/index.ts @@ -107,6 +107,32 @@ async function pathExists(filePath: string): Promise { } } +// The startup warning for a memory file inside a node_modules tree, or +// undefined for any other path. npx and npm install the server under +// node_modules, so the default memory.jsonl beside dist/index.js lands there, +// and npm treats that tree as its own: a reinstall, an update or a cleared npx +// cache can delete the file and the graph in it (#4117). The default is not +// moved, since that would strand every existing graph; the warning says how to +// choose a durable path instead. Only node_modules is matched, not the server's +// directory as such: the Docker setup mounts a volume over /app/dist, and a +// source checkout's dist survives a rebuild, so there the default is durable. +export function memoryFileLocationWarning( + memoryFilePath: string, +): string | undefined { + // Both separators, so a Windows path is matched on any platform. + const segments = path.resolve(memoryFilePath).split(/[\\/]/); + if (!segments.includes("node_modules")) { + return undefined; + } + return ( + `WARNING: the memory file ${memoryFilePath} is inside a node_modules directory, ` + + "which npm manages: reinstalling or updating the server, or clearing the npx cache, " + + "can delete it and the knowledge graph it holds. Set MEMORY_FILE_PATH to an absolute " + + "path outside node_modules, in a directory that exists (for example ~/memory.jsonl), " + + "and move this file there to keep the graph." + ); +} + // We are storing our memory using entities, relations, and observations in a graph structure export interface Entity { name: string; @@ -633,7 +659,9 @@ export class KnowledgeGraphManager { if (!entity) { throw new Error(`Entity with name ${o.entityName} not found`); } - const newObservations = o.contents.filter( + // A Set keeps the first occurrence of each string, so a repeat within + // one call is stored and reported once (#5201). + const newObservations = [...new Set(o.contents)].filter( (content) => !entity.observations.includes(content), ); entity.observations.push(...newObservations); @@ -707,23 +735,50 @@ export class KnowledgeGraphManager { }); } - async readGraph(): Promise { - return this.loadGraph(); + // With no options, the whole graph, as the knowledge-graph resource serves + // it. entityTypes keeps only entities of those types, and the relations + // that touch one of them; observationLimit caps each entity's observations + // (see limitObservations). + async readGraph(options: ReadGraphOptions = {}): Promise { + const graph = await this.loadGraph(); + const { entityTypes, observationLimit } = options; + if (entityTypes === undefined) { + return { + entities: limitObservations(graph.entities, observationLimit), + relations: graph.relations, + }; + } + const entities = graph.entities.filter((e) => + entityTypes.includes(e.entityType), + ); + const names = new Set(entities.map((e) => e.name)); + return { + entities: limitObservations(entities, observationLimit), + relations: graph.relations.filter( + (r) => names.has(r.from) || names.has(r.to), + ), + }; } - // Very basic search function - async searchNodes(query: string): Promise { + // Very basic search function. limit keeps the first matches in stored + // order; observationLimit caps each entity's observations. + async searchNodes( + query: string, + options: SearchNodesOptions = {}, + ): Promise { const graph = await this.loadGraph(); // Filter entities - const filteredEntities = graph.entities.filter( - (e) => - e.name.toLowerCase().includes(query.toLowerCase()) || - e.entityType.toLowerCase().includes(query.toLowerCase()) || - e.observations.some((o) => - o.toLowerCase().includes(query.toLowerCase()), - ), - ); + const filteredEntities = graph.entities + .filter( + (e) => + e.name.toLowerCase().includes(query.toLowerCase()) || + e.entityType.toLowerCase().includes(query.toLowerCase()) || + e.observations.some((o) => + o.toLowerCase().includes(query.toLowerCase()), + ), + ) + .slice(0, options.limit); // Create a Set of filtered entity names for quick lookup const filteredEntityNames = new Set(filteredEntities.map((e) => e.name)); @@ -734,15 +789,16 @@ export class KnowledgeGraphManager { (r) => filteredEntityNames.has(r.from) || filteredEntityNames.has(r.to), ); - const filteredGraph: KnowledgeGraph = { - entities: filteredEntities, + return { + entities: limitObservations(filteredEntities, options.observationLimit), relations: filteredRelations, }; - - return filteredGraph; } - async openNodes(names: string[]): Promise { + async openNodes( + names: string[], + options: OpenNodesOptions = {}, + ): Promise { const graph = await this.loadGraph(); // Filter entities @@ -761,13 +817,57 @@ export class KnowledgeGraphManager { (r) => filteredEntityNames.has(r.from) || filteredEntityNames.has(r.to), ); - const filteredGraph: KnowledgeGraph = { - entities: filteredEntities, + return { + entities: limitObservations(filteredEntities, options.observationLimit), relations: filteredRelations, }; + } +} + +// Options of the read tools (#3953), all optional, so a call without them +// returns what it always has. They let a client read a large graph in +// pieces: an overview of names, types and observation counts first +// (observationLimit 0), then the entities it needs in full. +export interface OpenNodesOptions { + observationLimit?: number; +} + +export interface ReadGraphOptions extends OpenNodesOptions { + entityTypes?: string[]; +} + +export interface SearchNodesOptions extends OpenNodesOptions { + limit?: number; +} + +// An entity as a read tool returns it. observationCount is set only when +// observationLimit was given, so a count above observations.length tells the +// caller that observations were left out. +export interface ReadEntity extends Entity { + observationCount?: number; +} + +export interface ReadGraphResult { + entities: ReadEntity[]; + relations: Relation[]; +} - return filteredGraph; +// Keep each entity's first observationLimit observations, in the order they +// were added, and record how many it holds. Without a limit the entities are +// returned as they are. +function limitObservations( + entities: Entity[], + observationLimit: number | undefined, +): ReadEntity[] { + if (observationLimit === undefined) { + return entities; } + return entities.map((e) => ({ + name: e.name, + entityType: e.entityType, + observations: e.observations.slice(0, observationLimit), + observationCount: e.observations.length, + })); } // Zod schemas for entities and relations @@ -796,6 +896,47 @@ export const SearchNodesQuerySchema = z "The search query to match against entity names, types, and observation content", ); +// The read tools' optional inputs (#3953). Their descriptions carry the +// guidance on when to use them, so the tool descriptions need not. +const ObservationLimitSchema = z + .number() + .int() + .min(0) + .optional() + .describe( + "Return at most this many observations per entity (the first ones, in the order they were added) and add each entity's total as observationCount. Use 0 for names, types and counts only: a cheap overview of a large graph, after which open_nodes reads the entities you need in full. Omit to return every observation.", + ); + +const ReadGraphEntityTypesSchema = z + .array(z.string()) + .min(1) + .optional() + .describe( + "Return only entities whose entityType is one of these (exact match), and only the relations that touch one of them. Omit to return every entity.", + ); + +const SearchNodesLimitSchema = z + .number() + .int() + .min(1) + .optional() + .describe( + "Return at most this many matching entities (the first matches, in stored order), with the relations that touch them. Omit to return every match.", + ); + +// An entity in a read tool's result: observationCount appears only when the +// call gave observationLimit. +const ReadEntitySchema = EntitySchema.extend({ + observationCount: z + .number() + .int() + .min(0) + .optional() + .describe( + "How many observations the entity holds in total; present only when observationLimit was given", + ), +}); + // Build a memory server over the graph file at memoryFilePath: the nine // tools, the knowledge-graph resource and its subscriptions. main() serves the // result on stdio; tests connect it to an in-memory transport, so each call @@ -853,7 +994,7 @@ export function createServer( annotations: { readOnlyHint: false, destructiveHint: false, - idempotentHint: false, + idempotentHint: true, openWorldHint: false, }, }, @@ -898,7 +1039,7 @@ export function createServer( annotations: { readOnlyHint: false, destructiveHint: false, - idempotentHint: false, + idempotentHint: true, openWorldHint: false, }, }, @@ -944,7 +1085,7 @@ export function createServer( annotations: { readOnlyHint: false, destructiveHint: false, - idempotentHint: false, + idempotentHint: true, openWorldHint: false, }, }, @@ -1092,10 +1233,14 @@ export function createServer( "read_graph", { title: "Read Graph", - description: "Read the entire knowledge graph", - inputSchema: z.object({}), + description: + "Read the entire knowledge graph: every entity with its observations, and every relation. The result grows with the graph, so use this only when you need everything; prefer search_nodes to find entities by keyword and open_nodes to read entities whose names you know.", + inputSchema: z.object({ + entityTypes: ReadGraphEntityTypesSchema, + observationLimit: ObservationLimitSchema, + }), outputSchema: z.object({ - entities: z.array(EntitySchema), + entities: z.array(ReadEntitySchema), relations: z.array(RelationSchema), }), annotations: { @@ -1105,8 +1250,11 @@ export function createServer( openWorldHint: false, }, }, - async () => { - const graph = await knowledgeGraphManager.readGraph(); + async ({ entityTypes, observationLimit }) => { + const graph = await knowledgeGraphManager.readGraph({ + entityTypes, + observationLimit, + }); return { content: [ { type: "text" as const, text: JSON.stringify(graph, null, 2) }, @@ -1121,12 +1269,15 @@ export function createServer( "search_nodes", { title: "Search Nodes", - description: "Search for nodes in the knowledge graph based on a query", + description: + "Search the knowledge graph for entities whose name, type or observations contain the query (case-insensitive). Use this when you have a keyword, a topic or part of a name but not the exact entity names. Prefer open_nodes when you already know the exact names, and use read_graph only when you need the whole graph. Returns the matching entities and every relation that touches one of them.", inputSchema: z.object({ query: SearchNodesQuerySchema, + limit: SearchNodesLimitSchema, + observationLimit: ObservationLimitSchema, }), outputSchema: z.object({ - entities: z.array(EntitySchema), + entities: z.array(ReadEntitySchema), relations: z.array(RelationSchema), }), annotations: { @@ -1136,8 +1287,11 @@ export function createServer( openWorldHint: false, }, }, - async ({ query }) => { - const graph = await knowledgeGraphManager.searchNodes(query); + async ({ query, limit, observationLimit }) => { + const graph = await knowledgeGraphManager.searchNodes(query, { + limit, + observationLimit, + }); return { content: [ { type: "text" as const, text: JSON.stringify(graph, null, 2) }, @@ -1152,14 +1306,16 @@ export function createServer( "open_nodes", { title: "Open Nodes", - description: "Open specific nodes in the knowledge graph by their names", + description: + "Retrieve specific entities by their exact names, with every relation that touches one of them. Use this when you already know the names (from the conversation, or from an earlier search_nodes or read_graph result); it is the cheapest way to read entities you can name. Prefer search_nodes when you only have a keyword or part of a name. Names that do not exist are skipped.", inputSchema: z.object({ names: z .array(z.string()) .describe("An array of entity names to retrieve"), + observationLimit: ObservationLimitSchema, }), outputSchema: z.object({ - entities: z.array(EntitySchema), + entities: z.array(ReadEntitySchema), relations: z.array(RelationSchema), }), annotations: { @@ -1169,8 +1325,10 @@ export function createServer( openWorldHint: false, }, }, - async ({ names }) => { - const graph = await knowledgeGraphManager.openNodes(names); + async ({ names, observationLimit }) => { + const graph = await knowledgeGraphManager.openNodes(names, { + observationLimit, + }); return { content: [ { type: "text" as const, text: JSON.stringify(graph, null, 2) }, @@ -1241,6 +1399,10 @@ export function registerKnowledgeGraphSubscriptions( // including one that stops the transport from starting, is reported on stderr. export async function main(transport?: Transport): Promise { const memoryFilePath = await ensureMemoryFilePath(); + const locationWarning = memoryFileLocationWarning(memoryFilePath); + if (locationWarning) { + console.error(locationWarning); + } const handle = serveStdio(({ era }) => createServer(memoryFilePath, era), { transport, // serveStdio reports a transport that fails, including one that cannot diff --git a/src/memory/package.json b/src/memory/package.json index cb1ea6ae7f..8ad287740f 100644 --- a/src/memory/package.json +++ b/src/memory/package.json @@ -1,6 +1,6 @@ { "name": "@modelcontextprotocol/server-memory", - "version": "2.0.1", + "version": "2.1.0", "description": "MCP server for enabling memory for Claude through a knowledge graph", "license": "SEE LICENSE IN LICENSE", "mcpName": "io.github.modelcontextprotocol/server-memory", diff --git a/src/sequentialthinking/CHANGELOG.md b/src/sequentialthinking/CHANGELOG.md index 7a6ecaa755..9700d8625e 100644 --- a/src/sequentialthinking/CHANGELOG.md +++ b/src/sequentialthinking/CHANGELOG.md @@ -1,5 +1,25 @@ # @modelcontextprotocol/server-sequential-thinking +## 2.1.1 + +### Patch Changes + +- [#5435](https://github.com/modelcontextprotocol/servers/pull/5435) [`8da06bc`](https://github.com/modelcontextprotocol/servers/commit/8da06bc446aff78ff8829534aa3dc78d12b44c8b) [@cliffhall](https://github.com/cliffhall) - Bound the thought box logged to stderr. Since the multi-line box, every row was padded to the thought's longest line, so a thought with one long line and many short ones logged their product: a 60 KB thought wrote 400 million characters and could exhaust the server's memory. The box is now at most 100 characters wide (longer lines, and a long branch ID in the header, wrap) and draws at most 1000 rows of thought text (it then says how many lines it left out), so the log no longer grows with the thought's line count times its longest line. The thought kept in the server's history and the tool result are unchanged. + +## 2.1.0 + +### Minor Changes + +- [#5397](https://github.com/modelcontextprotocol/servers/pull/5397) [`93f89b1`](https://github.com/modelcontextprotocol/servers/commit/93f89b1ecb1443714b28cdb542969363bb932fcb) [@cliffhall](https://github.com/cliffhall) - Bound the thought history each connection keeps: at most `MAX_THOUGHT_HISTORY` thoughts (default 1000) are retained, the oldest dropped first and from its branch too, so a long-lived connection no longer grows memory without bound. + +### Patch Changes + +- [#5361](https://github.com/modelcontextprotocol/servers/pull/5361) [`9a1cdf3`](https://github.com/modelcontextprotocol/servers/commit/9a1cdf38348b3ee0752e48ea61295889a11b63b0) [@cliffhall](https://github.com/cliffhall) - Draw a multi-line thought one row per line in the stderr thought box, sized to the longest line, instead of as one broken row ([#5205](https://github.com/modelcontextprotocol/servers/issues/5205)) + +- [#5352](https://github.com/modelcontextprotocol/servers/pull/5352) [`59b7f9d`](https://github.com/modelcontextprotocol/servers/commit/59b7f9d15150217217997f6b523cf8afeca1e0f9) [@cliffhall](https://github.com/cliffhall) - Remove the unused `yargs` dependency (and `@types/yargs`), so installing the server no longer pulls in yargs and its transitive packages. + +- [#5374](https://github.com/modelcontextprotocol/servers/pull/5374) [`c4d0cc4`](https://github.com/modelcontextprotocol/servers/commit/c4d0cc4e0041f72f9f05c6eb16fe818753f9aeac) [@cliffhall](https://github.com/cliffhall) - Trim the `sequentialthinking` tool description from 2,781 to 1,009 characters (about 566 to 198 tokens), so it fits the 1,024-character cap OpenAI and Azure OpenAI put on tool descriptions ([#799](https://github.com/modelcontextprotocol/servers/issues/799)). The "Key features" and "Parameters explained" sections are gone: each parameter is still described by the input schema, which is unchanged, and the guidance those field descriptions do not carry is kept ([#4507](https://github.com/modelcontextprotocol/servers/issues/4507)). + ## 2.0.0 ### Major Changes diff --git a/src/sequentialthinking/README.md b/src/sequentialthinking/README.md index 390d09646d..81ae0ae26a 100644 --- a/src/sequentialthinking/README.md +++ b/src/sequentialthinking/README.md @@ -15,7 +15,7 @@ Published on npm as [`@modelcontextprotocol/server-sequential-thinking`](https:/ ## Tool -### sequential_thinking +### sequentialthinking Facilitates a detailed, step-by-step thinking process for problem-solving and analysis. @@ -40,7 +40,7 @@ The Sequential Thinking tool is designed for: - Tasks that need to maintain context over multiple steps - Situations where irrelevant information needs to be filtered out -In practice, you do not call `sequential_thinking` directly by hand unless your client exposes raw tool calls. Instead, connect the server to an MCP-aware host and ask the model to think through a problem step by step. The host can then decide to call the tool one or more times while it works. +In practice, you do not call `sequentialthinking` directly by hand unless your client exposes raw tool calls. Instead, connect the server to an MCP-aware host and ask the model to think through a problem step by step. The host can then decide to call the tool one or more times while it works. ### What it looks like in use @@ -52,7 +52,7 @@ Example prompts that typically benefit from this tool: ### How to tell it is working -If your host or inspector shows tool activity, you should see repeated calls to `sequential_thinking` with fields such as: +If your host or inspector shows tool activity, you should see repeated calls to `sequentialthinking` with fields such as: - `thought` - `thoughtNumber` @@ -66,7 +66,7 @@ When the reasoning changes course, you may also see revision or branching fields After installing the server in your MCP host: 1. Restart or reload the host so it reconnects to the server. -2. Confirm the `sequential_thinking` tool appears in the host's MCP tool list or inspector. +2. Confirm the `sequentialthinking` tool appears in the host's MCP tool list or inspector. 3. Ask the host to solve a non-trivial problem in a step-by-step way. 4. Verify that the host invokes the tool multiple times instead of returning a one-shot answer. @@ -134,6 +134,10 @@ On Windows, use `cmd /c` to launch `npx`: To disable logging of thought information set env var: `DISABLE_THOUGHT_LOGGING` to `true`. +Each thought is logged to stderr as a box, one row per line of the thought. The box is at most 100 characters wide, with longer lines (and a long branch ID in the header) wrapped, and draws at most 1000 rows of thought text; past that it ends with a row saying how many lines it left out. Only the log is shortened: the server keeps the whole thought in its history, and the tool result is unchanged. + +To bound memory use, the server retains at most `MAX_THOUGHT_HISTORY` thoughts per connection (a positive integer; default `1000`). Past the cap the oldest thought is dropped, from its branch too, so `thoughtHistoryLength` in the result stops at the cap and a branch whose thoughts have all been dropped is no longer listed in `branches`. + ### Usage with VS Code For quick installation, click one of the installation buttons below... diff --git a/src/sequentialthinking/__tests__/thinking.test.ts b/src/sequentialthinking/__tests__/thinking.test.ts index e13f08e98b..5410ebb284 100644 --- a/src/sequentialthinking/__tests__/thinking.test.ts +++ b/src/sequentialthinking/__tests__/thinking.test.ts @@ -4,7 +4,8 @@ // in-process client and reads only the result's text and structuredContent. // Includes the #4813 regression tests: a branchId that names an // Object.prototype key is an ordinary branch, and a failed call leaves the -// state unchanged. +// state unchanged; and the #2912 tests: the history is capped at +// MAX_THOUGHT_HISTORY, evicting the oldest thought from its branch too. import { describe, it, expect, beforeEach, afterEach, vi } from "vitest"; import { @@ -14,6 +15,30 @@ import { thought, type Connected, } from "./helpers.js"; +import { DEFAULT_MAX_THOUGHT_HISTORY } from "../lib.js"; + +/** + * Connect to a server built under MAX_THOUGHT_HISTORY=`value` (`undefined` = + * unset). The cap is read once, when the server is built, so the variable is + * restored as soon as the connection is up. + */ +async function connectCapped(value: string | undefined): Promise { + const previous = process.env.MAX_THOUGHT_HISTORY; + if (value === undefined) { + delete process.env.MAX_THOUGHT_HISTORY; + } else { + process.env.MAX_THOUGHT_HISTORY = value; + } + try { + return await connect(); + } finally { + if (previous === undefined) { + delete process.env.MAX_THOUGHT_HISTORY; + } else { + process.env.MAX_THOUGHT_HISTORY = previous; + } + } +} describe("sequentialthinking thought processing", () => { let conn: Connected; @@ -392,4 +417,136 @@ describe("sequentialthinking thought processing", () => { ).toEqual([]); }); }); + + // #2912: a server lives as long as its connection, so the history it keeps + // is capped, oldest thought first, and the branches shrink with it. + describe("#2912: bounded thought history", () => { + let capped: Connected | undefined; + + afterEach(async () => { + await capped?.close(); + capped = undefined; + vi.restoreAllMocks(); + }); + + it(`retains at most ${DEFAULT_MAX_THOUGHT_HISTORY} thoughts by default`, async () => { + capped = await connectCapped(undefined); + let length = 0; + for (let n = 1; n <= DEFAULT_MAX_THOUGHT_HISTORY + 5; n++) { + length = parseOk( + await capped.think(thought({ thoughtNumber: n })), + ).thoughtHistoryLength; + expect(length).toBe(Math.min(n, DEFAULT_MAX_THOUGHT_HISTORY)); + } + expect(length).toBe(DEFAULT_MAX_THOUGHT_HISTORY); + }); + + it("stops thoughtHistoryLength at MAX_THOUGHT_HISTORY", async () => { + capped = await connectCapped("3"); + const lengths: number[] = []; + for (let n = 1; n <= 6; n++) { + lengths.push( + parseOk(await capped.think(thought({ thoughtNumber: n }))) + .thoughtHistoryLength, + ); + } + expect(lengths).toEqual([1, 2, 3, 3, 3, 3]); + }); + + it("drops a branch once all of its thoughts are evicted", async () => { + capped = await connectCapped("2"); + await capped.think(thought({ branchFromThought: 1, branchId: "alt" })); + expect( + parseOk(await capped.think(thought({ thoughtNumber: 2 }))).branches, + ).toEqual(["alt"]); + expect( + parseOk(await capped.think(thought({ thoughtNumber: 3 }))), + ).toMatchObject({ branches: [], thoughtHistoryLength: 2 }); + }); + + it("keeps a branch while any of its thoughts is retained", async () => { + capped = await connectCapped("2"); + await capped.think(thought({ branchFromThought: 1, branchId: "alt" })); + await capped.think( + thought({ thoughtNumber: 2, branchFromThought: 1, branchId: "alt" }), + ); + // Evicts the first "alt" thought; the second is still retained. + expect( + parseOk(await capped.think(thought({ thoughtNumber: 3 }))), + ).toMatchObject({ branches: ["alt"], thoughtHistoryLength: 2 }); + // Evicts the second; nothing on "alt" is left. + expect( + parseOk(await capped.think(thought({ thoughtNumber: 4 }))).branches, + ).toEqual([]); + }); + + it("lists a branch recreated after eviction as newly seen", async () => { + capped = await connectCapped("3"); + await capped.think(thought({ branchFromThought: 1, branchId: "a" })); + await capped.think( + thought({ thoughtNumber: 2, branchFromThought: 1, branchId: "b" }), + ); + await capped.think( + thought({ thoughtNumber: 3, branchFromThought: 1, branchId: "b" }), + ); + // Evicts the only "a" thought. + expect( + parseOk(await capped.think(thought({ thoughtNumber: 4 }))).branches, + ).toEqual(["b"]); + // "a" is recorded again, after "b"; the first "b" thought is evicted. + expect( + parseOk( + await capped.think( + thought({ thoughtNumber: 5, branchFromThought: 1, branchId: "a" }), + ), + ).branches, + ).toEqual(["b", "a"]); + }); + + it("evicts a branch whose id names an Object.prototype key", async () => { + capped = await connectCapped("1"); + await capped.think( + thought({ branchFromThought: 1, branchId: "__proto__" }), + ); + expect( + parseOk(await capped.think(thought({ thoughtNumber: 2 }))), + ).toMatchObject({ branches: [], thoughtHistoryLength: 1 }); + }); + + it("accepts a value padded with whitespace", async () => { + capped = await connectCapped(" 1 "); + await capped.think(thought()); + expect( + parseOk(await capped.think(thought({ thoughtNumber: 2 }))) + .thoughtHistoryLength, + ).toBe(1); + }); + + it("uses the default for an empty value, without a warning", async () => { + const errors = vi.spyOn(console, "error").mockImplementation(() => {}); + capped = await connectCapped(""); + await capped.think(thought()); + expect( + parseOk(await capped.think(thought({ thoughtNumber: 2 }))) + .thoughtHistoryLength, + ).toBe(2); + expect(errors).not.toHaveBeenCalled(); + }); + + it.each(["0", "-1", "1.5", "abc", "1e3", "99999999999999999999"])( + "reports %j on stderr and keeps the default cap", + async (value) => { + const errors = vi.spyOn(console, "error").mockImplementation(() => {}); + capped = await connectCapped(value); + expect(errors).toHaveBeenCalledWith( + `MAX_THOUGHT_HISTORY must be a positive integer, got "${value}"; using ${DEFAULT_MAX_THOUGHT_HISTORY}`, + ); + await capped.think(thought()); + expect( + parseOk(await capped.think(thought({ thoughtNumber: 2 }))) + .thoughtHistoryLength, + ).toBe(2); + }, + ); + }); }); diff --git a/src/sequentialthinking/__tests__/thought-logging.test.ts b/src/sequentialthinking/__tests__/thought-logging.test.ts index 7ed39ae1a2..6b9345e295 100644 --- a/src/sequentialthinking/__tests__/thought-logging.test.ts +++ b/src/sequentialthinking/__tests__/thought-logging.test.ts @@ -1,6 +1,8 @@ // The server's other observable output: unless DISABLE_THOUGHT_LOGGING is // "true" (any case), each accepted thought is drawn as a box on stderr, with -// a header that differs for plain thoughts, revisions and branches. Thoughts +// a header that differs for plain thoughts, revisions and branches, and one +// row per line of the thought, wrapped at MAX_BOX_TEXT_WIDTH and cut off +// after MAX_BOX_ROWS. Thoughts // are driven through the in-process client and stderr is captured by spying // on console.error, which also keeps the boxes off the test output. chalk's // colour level depends on the terminal the tests run in, so it is pinned to 0 @@ -17,6 +19,7 @@ import { type MockInstance, } from "vitest"; import chalk from "chalk"; +import { MAX_BOX_ROWS, MAX_BOX_TEXT_WIDTH } from "../lib.js"; import { connect, thought, type Connected } from "./helpers.js"; describe("sequentialthinking thought logging", () => { @@ -41,14 +44,20 @@ describe("sequentialthinking thought logging", () => { return errorSpy.mock.calls.map((args) => String(args[0])); } - /** The box the server draws, for an uncoloured header. */ - function box(header: string, text: string): string { - const border = "─".repeat(Math.max(header.length, text.length) + 4); + /** + * The box the server draws, for an uncoloured header: one row per line of + * the thought (given as an array), sized to the longest row. + */ + function box(header: string, text: string | string[]): string { + const lines = typeof text === "string" ? [text] : text; + const width = Math.max(header.length, ...lines.map((l) => l.length)); + const border = "─".repeat(width + 4); + const rows = lines.map((l) => `│ ${l.padEnd(border.length - 2)} │`); return ` ┌${border}┐ │ ${header.padEnd(border.length - 2)} │ ├${border}┤ -│ ${text.padEnd(border.length - 2)} │ +${rows.join("\n")} └${border}┘`; } @@ -178,6 +187,169 @@ describe("sequentialthinking thought logging", () => { expect(new Set(rows.map((row) => row.length)).size).toBe(1); }); + // #5205: a thought with line breaks is drawn one row per line, the box as + // wide as its longest line, whichever line ending the client sent. + describe("a multi-line thought", () => { + beforeEach(async () => { + conn = await connect({ disableThoughtLogging: undefined }); + }); + + it.each([ + ["\\n", "\n"], + ["\\r\\n", "\r\n"], + ["a lone \\r", "\r"], + ])("draws one row per line split by %s", async (_l, eol) => { + const lines = [ + "first line", + "a much longer second line of the thought", + "3", + ]; + await conn!.think(thought({ thought: lines.join(eol) })); + expect(logged()).toEqual([box("💭 Thought 1/3", lines)]); + expect(logged()[0]).not.toContain("\r"); + }); + + it("sizes the box from the longest line, not the whole thought", async () => { + await conn!.think(thought({ thought: "ab\ncd\nef" })); + // Shorter than the header, so the header sets the width. + expect(logged()).toEqual([box("💭 Thought 1/3", ["ab", "cd", "ef"])]); + }); + + it("keeps blank lines as blank rows", async () => { + await conn!.think(thought({ thought: "top\n\nbottom\n" })); + expect(logged()).toEqual([ + box("💭 Thought 1/3", ["top", "", "bottom", ""]), + ]); + }); + + it("draws every row the same width", async () => { + await conn!.think( + thought({ thought: "short\na considerably longer line here\nmid" }), + ); + const rows = logged()[0].split("\n").slice(1); + expect(rows).toHaveLength(7); + expect(new Set(rows.map((row) => row.length)).size).toBe(1); + }); + + // #5433: past MAX_BOX_ROWS the box stops drawing and says how many lines + // it left out; the call itself is unaffected. + it("draws a thought with very many lines", async () => { + const lines = Array.from({ length: 200_000 }, (_, i) => `${i % 10}`); + const result = await conn!.think(thought({ thought: lines.join("\n") })); + expect(result.isError).toBeFalsy(); + const out = logged()[0].split("\n"); + expect(out).toHaveLength(MAX_BOX_ROWS + 1 + 5); + expect(out.at(-2)).toContain( + `… ${200_000 - MAX_BOX_ROWS} more lines not shown`, + ); + // The summary is wider than every one-character line and the header, + // so it sets the box's width; it must not stick out of it. + expect(new Set(out.slice(1).map((r) => r.length)).size).toBe(1); + }); + }); + + // #5433: the box is never wider than MAX_BOX_TEXT_WIDTH. A longer line, or + // a longer header, wraps onto rows of that width, so the box grows linearly + // with the thought instead of as its longest line times its line count. + describe("a thought wider than the box", () => { + const W = MAX_BOX_TEXT_WIDTH; + const border = "─".repeat(W + 4); + const row = (text: string) => `│ ${text.padEnd(border.length - 2)} │`; + + beforeEach(async () => { + conn = await connect({ disableThoughtLogging: undefined }); + }); + + it("wraps a long line onto rows of the capped width", async () => { + const text = `${"a".repeat(W)}${"b".repeat(W)}${"c".repeat(5)}`; + await conn!.think(thought({ thought: `${text}\nshort` })); + expect(logged()).toEqual([ + ` +┌${border}┐ +${row("💭 Thought 1/3")} +├${border}┤ +${row("a".repeat(W))} +${row("b".repeat(W))} +${row("c".repeat(5))} +${row("short")} +└${border}┘`, + ]); + }); + + it("stays bounded for one long line and many short ones", async () => { + // The case from the review of #5430: before the cap, every one of the + // 19,999 short rows was padded to 20,000 characters, 400 million in all. + const n = 20_000; + const text = `${"x".repeat(n)}${"\ny".repeat(n - 1)}`; + const result = await conn!.think(thought({ thought: text })); + expect(result.isError).toBeFalsy(); + const out = logged()[0]; + // 200 wrapped rows for the long line, short lines up to the row cap, + // then the summary row; each row is W + 6 characters and a newline. + const shortShown = MAX_BOX_ROWS - n / W; + expect(out.split("\n")).toHaveLength(MAX_BOX_ROWS + 1 + 5); + expect(out).toContain(`… ${n - 1 - shortShown} more lines not shown`); + expect(out.length).toBeLessThan((MAX_BOX_ROWS + 10) * (W + 7)); + }); + + it("counts a line the row cap cuts off as not shown", async () => { + const text = `${"s\n".repeat(MAX_BOX_ROWS - 1)}${"L".repeat(2 * W)}`; + await conn!.think(thought({ thought: text })); + const rows = logged()[0].split("\n"); + expect(rows).toHaveLength(MAX_BOX_ROWS + 1 + 5); + expect(rows.at(-3)).toBe(row("L".repeat(W))); + expect(rows.at(-2)).toBe(row("… 1 more line not shown")); + }); + + it("draws a thought that exactly fills the cap with no summary row", async () => { + const text = Array.from({ length: MAX_BOX_ROWS }, () => "s").join("\n"); + await conn!.think(thought({ thought: text })); + const rows = logged()[0].split("\n"); + expect(rows).toHaveLength(MAX_BOX_ROWS + 5); + expect(logged()[0]).not.toContain("not shown"); + }); + + it("draws every row the same width", async () => { + await conn!.think(thought({ thought: `${"z".repeat(3 * W + 7)}\nend` })); + const rows = logged()[0].split("\n").slice(1); + expect(new Set(rows.map((r) => r.length))).toEqual(new Set([W + 6])); + }); + + it("wraps a header longer than the cap, and keeps the box linear", async () => { + const branchId = "b".repeat(2 * W); + const lines = 1_000; + await conn!.think( + thought({ + thought: Array.from({ length: lines }, () => "s").join("\n"), + branchFromThought: 1, + branchId, + }), + ); + const out = logged()[0]; + const rows = out.split("\n").slice(1); + expect(new Set(rows.map((r) => r.length))).toEqual(new Set([W + 6])); + const header = `🌿 Branch 1/3 (from thought 1, ID: ${branchId})`; + const headerRows = rows.slice(1, rows.indexOf(`├${border}┤`)); + expect(headerRows.map((r) => r.slice(2, -2).trimEnd()).join("")).toBe( + header, + ); + // Each row is W + 6 characters and a newline. Uncapped, the header would + // have set every row's width to its own length of more than 2 × W. + expect(out.length).toBeLessThan((lines + 10) * (W + 7)); + }); + + it("never splits an emoji across rows", async () => { + // The emoji's two UTF-16 halves would straddle the cut at W. + const text = `${"a".repeat(W - 1)}😀tail`; + await conn!.think(thought({ thought: text })); + const rows = logged()[0].split("\n").slice(4, -1); + expect(rows.map((r) => r.slice(2, -2).trimEnd())).toEqual([ + "a".repeat(W - 1), + "😀tail", + ]); + }); + }); + // #5000: when colour is on, the header carries chalk's escape codes, but the // border is sized from the visible text, so the box matches the uncoloured one. it("sizes the border from the visible header, not its escape codes", async () => { diff --git a/src/sequentialthinking/__tests__/tools-list.test.ts b/src/sequentialthinking/__tests__/tools-list.test.ts index a145dbd6a3..1f62176064 100644 --- a/src/sequentialthinking/__tests__/tools-list.test.ts +++ b/src/sequentialthinking/__tests__/tools-list.test.ts @@ -3,8 +3,9 @@ // annotations, input schema and output schema, exactly as the SDK emits them // today. These are characterization tests. The annotations assert the values // fixed in #4721 (the tool is stateful, so neither read-only nor idempotent). -// The 2781-character description (#799) is pinned as current design: the -// 1024 cap is an OpenAI/Azure client limit, not the MCP spec. +// The description is held to the 1024-character cap OpenAI and Azure OpenAI +// put on tool descriptions (#799), and leaves explaining each parameter to the +// input schema's own field descriptions (#4507). import { describe, it, expect, beforeAll, afterAll } from "vitest"; import { createRequire } from "node:module"; @@ -72,18 +73,35 @@ describe("sequentialthinking: initialize and tools/list", () => { expect(tool.execution).toBeUndefined(); }); - // #799: some clients cap a tool description at 1024 characters. This one is - // 2781 today; asserting <= 1024 would fail, so the current length is pinned. - it("has a 2781-character description today (#799)", () => { - expect(tool.description).toHaveLength(2781); + // #799: OpenAI and Azure OpenAI reject a tool description over 1024 + // characters. The MCP spec sets no cap; this keeps the tool usable there. + it("keeps the description within 1024 characters (#799)", () => { + expect(tool.description).toBeDefined(); + expect(tool.description?.length ?? 0).toBeLessThanOrEqual(1024); expect(tool.description).toMatch( /^A detailed tool for dynamic and reflective problem-solving through thoughts\.\n/, ); + expect(tool.description).toContain("When to use this tool:"); expect(tool.description).toMatch( - /11\. Only set nextThoughtNeeded to false when truly done and a satisfactory answer is reached$/, + /Set nextThoughtNeeded to false only when truly done and a satisfactory answer is reached$/, ); }); + // #4507: the parameters are explained by the input schema, not the + // description, so every property must carry a description of its own, and + // the description no longer restates them in a "Parameters explained" list. + it("leaves explaining each parameter to the input schema (#4507)", () => { + const properties = tool.inputSchema.properties ?? {}; + expect(Object.keys(properties)).toHaveLength(9); + for (const [name, schema] of Object.entries(properties)) { + expect( + (schema as { description?: unknown }).description, + `inputSchema property ${name}`, + ).toEqual(expect.any(String)); + } + expect(tool.description).not.toMatch(/Parameters explained|Key features/); + }); + it("advertises the input schema as JSON Schema 2020-12", () => { expect(tool.inputSchema).toEqual({ $schema: "https://json-schema.org/draft/2020-12/schema", diff --git a/src/sequentialthinking/index.ts b/src/sequentialthinking/index.ts index c7015df353..be0aaa2eb2 100644 --- a/src/sequentialthinking/index.ts +++ b/src/sequentialthinking/index.ts @@ -54,8 +54,11 @@ export function createServer(): McpServer { "sequentialthinking", { title: "Sequential Thinking", + // At most 1024 characters, the cap OpenAI and Azure OpenAI put on a tool + // description (#799). Each parameter is explained by its own inputSchema + // description, which clients send too, so this text carries only when to + // call the tool and the guidance those field descriptions do not (#4507). description: `A detailed tool for dynamic and reflective problem-solving through thoughts. -This tool helps analyze problems through a flexible thinking process that can adapt and evolve. Each thought can build on, question, or revise previous insights as understanding deepens. When to use this tool: @@ -63,51 +66,15 @@ When to use this tool: - Planning and design with room for revision - Analysis that might need course correction - Problems where the full scope might not be clear initially -- Problems that require a multi-step solution - Tasks that need to maintain context over multiple steps - Situations where irrelevant information needs to be filtered out -Key features: -- You can adjust total_thoughts up or down as you progress -- You can question or revise previous thoughts -- You can add more thoughts even after reaching what seemed like the end -- You can express uncertainty and explore alternative approaches -- Not every thought needs to build linearly - you can branch or backtrack -- Generates a solution hypothesis -- Verifies the hypothesis based on the Chain of Thought steps -- Repeats the process until satisfied -- Provides a correct answer - -Parameters explained: -- thought: Your current thinking step, which can include: - * Regular analytical steps - * Revisions of previous thoughts - * Questions about previous decisions - * Realizations about needing more analysis - * Changes in approach - * Hypothesis generation - * Hypothesis verification -- nextThoughtNeeded: True if you need more thinking, even if at what seemed like the end -- thoughtNumber: Current number in sequence (can go beyond initial total if needed) -- totalThoughts: Current estimate of thoughts needed (can be adjusted up/down) -- isRevision: A boolean indicating if this thought revises previous thinking -- revisesThought: If is_revision is true, which thought number is being reconsidered -- branchFromThought: If branching, which thought number is the branching point -- branchId: Identifier for the current branch (if any) -- needsMoreThoughts: If reaching end but realizing more thoughts needed - You should: -1. Start with an initial estimate of needed thoughts, but be ready to adjust -2. Feel free to question or revise previous thoughts -3. Don't hesitate to add more thoughts if needed, even at the "end" -4. Express uncertainty when present -5. Mark thoughts that revise previous thinking or branch into new paths -6. Ignore information that is irrelevant to the current step -7. Generate a solution hypothesis when appropriate -8. Verify the hypothesis based on the Chain of Thought steps -9. Repeat the process until satisfied with the solution -10. Provide a single, ideally correct answer as the final output -11. Only set nextThoughtNeeded to false when truly done and a satisfactory answer is reached`, +1. Estimate totalThoughts up front and adjust it up or down as you go; thoughtNumber may exceed it +2. Question or revise earlier thoughts (isRevision with revisesThought), or branch (branchFromThought with branchId) +3. Express uncertainty, and ignore information irrelevant to the current step +4. Generate a solution hypothesis, verify it against the earlier steps, and repeat until satisfied +5. Set nextThoughtNeeded to false only when truly done and a satisfactory answer is reached`, inputSchema: z.object({ thought: z.string().describe("Your current thinking step"), nextThoughtNeeded: coercedBoolean.describe( diff --git a/src/sequentialthinking/lib.ts b/src/sequentialthinking/lib.ts index 3e836fce65..b9299910bc 100644 --- a/src/sequentialthinking/lib.ts +++ b/src/sequentialthinking/lib.ts @@ -12,16 +12,93 @@ export interface ThoughtData { nextThoughtNeeded: boolean; } +/** + * The widest the text inside a thought box gets (#5433). A longer line, or a + * longer header, wraps onto further rows of this width. Without the cap every + * row was padded to the longest line, so one long line and many short ones + * made the box grow as their product: a 60 KB thought logged 400 million + * characters. Capped, the box grows linearly with the thought. + */ +export const MAX_BOX_TEXT_WIDTH = 100; + +/** + * The most rows of thought text a box draws (#5433). Each row is padded to + * the box's width, so a thought of many short lines would still log about a + * hundred times its own size; past this many rows the box ends with one row + * saying how many lines it left out. Header rows (a long branchId) are not + * counted. Only stderr is cut: the thought kept in thoughtHistory is whole, + * and the tool result, which carries no thought text, is unchanged. + */ +export const MAX_BOX_ROWS = 1000; + +/** + * `text` cut into rows of at most `width` UTF-16 code units, never between + * the two halves of a surrogate pair, so an emoji is never split across rows. + * An empty string is one empty row. + */ +function wrapRow(text: string, width: number): string[] { + const rows: string[] = []; + let start = 0; + do { + let end = Math.min(start + width, text.length); + const last = text.charCodeAt(end - 1); + if ( + end < text.length && + end - start > 1 && + last >= 0xd800 && + last <= 0xdbff + ) { + end -= 1; + } + rows.push(text.slice(start, end)); + start = end; + } while (start < text.length); + return rows; +} + +/** + * How many thoughts one server instance retains when MAX_THOUGHT_HISTORY is + * unset (#2912). A server lives as long as its client's connection, so without + * a cap a long-lived connection grows its history without bound. + */ +export const DEFAULT_MAX_THOUGHT_HISTORY = 1000; + +/** + * The thought cap from MAX_THOUGHT_HISTORY: a positive integer, or the default + * when the variable is unset or empty. Any other value is reported on stderr + * and replaced by the default, so a typo cannot remove the bound. + */ +function readMaxThoughtHistory(): number { + const raw = (process.env.MAX_THOUGHT_HISTORY ?? "").trim(); + if (raw === "") { + return DEFAULT_MAX_THOUGHT_HISTORY; + } + const value = Number(raw); + if (/^\d+$/.test(raw) && Number.isSafeInteger(value) && value >= 1) { + return value; + } + console.error( + `MAX_THOUGHT_HISTORY must be a positive integer, got "${raw}"; using ${DEFAULT_MAX_THOUGHT_HISTORY}`, + ); + return DEFAULT_MAX_THOUGHT_HISTORY; +} + export class SequentialThinkingServer { + // At most maxThoughtHistory thoughts, oldest first. Each branch holds the + // retained thoughts recorded on it, so evicting the oldest thought also + // removes it from its branch, and a branch left empty is dropped: the + // branches together never hold more than the history does. private thoughtHistory: ThoughtData[] = []; // Null prototype, so any string branchId (even "constructor" or "__proto__") // is an ordinary own key rather than an inherited Object.prototype member. private branches: Record = Object.create(null); private disableThoughtLogging: boolean; + private maxThoughtHistory: number; constructor() { this.disableThoughtLogging = (process.env.DISABLE_THOUGHT_LOGGING || "").toLowerCase() === "true"; + this.maxThoughtHistory = readMaxThoughtHistory(); } private formatThought(thoughtData: ThoughtData): string { @@ -64,20 +141,61 @@ export class SequentialThinkingServer { const counts = ` ${thoughtNumber}/${totalThoughts}${context}`; const visibleHeader = `${label}${counts}`; - const border = "─".repeat( - Math.max(visibleHeader.length, thought.length) + 4, + // #5205: each line of the thought is a row of its own, and the box is as + // wide as the longest of them (or the header). Any line ending (\r\n, \n + // or a lone \r) ends a row: a \r left inside a row would send the cursor + // back to the left border and draw the rest of the row over it. + const lines = thought.split(/\r\n|\r|\n/); + // A reduce rather than Math.max(...lengths): spreading one argument per + // line would overflow the call stack on a thought with very many lines. + // #5433: never wider than MAX_BOX_TEXT_WIDTH; longer lines wrap below. + const wrapWidth = Math.min( + MAX_BOX_TEXT_WIDTH, + lines.reduce( + (widest, line) => Math.max(widest, line.length), + visibleHeader.length, + ), + ); + // A header longer than the cap (a long branchId) wraps like a thought + // line; only its first row, which starts with the label, is coloured. + const headerTexts = wrapRow(visibleHeader, wrapWidth); + // Rows are drawn up to MAX_BOX_ROWS; the lines left over are counted, not + // drawn, so a huge thought costs no more than the cap. + const texts: string[] = []; + // Lines drawn in full; a line cut off by the cap counts as hidden. + let whole = 0; + for (const line of lines) { + const wrapped = wrapRow(line, wrapWidth); + const room = MAX_BOX_ROWS - texts.length; + texts.push(...wrapped.slice(0, room)); + if (wrapped.length > room) break; + whole++; + } + const hidden = lines.length - whole; + const summary = + hidden > 0 + ? `… ${hidden} more line${hidden === 1 ? "" : "s"} not shown` + : ""; + if (summary) texts.push(summary); + // The summary row can be wider than every line it follows, so the box is + // sized after it. Widening cannot change the wrapping above: below the cap + // no line wrapped, and at the cap the summary (under 40 characters) fits. + const width = Math.max(wrapWidth, summary.length); + const border = "─".repeat(width + 4); + // Every row, header rows included, is padded to the border's width by the + // visible text's length. Each label's emoji is one surrogate pair, so its + // length of 2 is also its display width in a terminal. + const row = (text: string) => `│ ${text.padEnd(border.length - 2)} │`; + const headerRows = headerTexts.map((text, i) => + i === 0 ? row(text).replace(label, colour(label)) : row(text), ); - // The header row is padded to the border's width like the thought row, by - // the visible text's length. Each label's emoji is one surrogate pair, so - // its length of 2 is also its display width in a terminal. - const padding = " ".repeat(border.length - 2 - visibleHeader.length); - const header = `${colour(label)}${counts}${padding}`; + const rows = texts.map(row); return ` ┌${border}┐ -│ ${header} │ +${headerRows.join("\n")} ├${border}┤ -│ ${thought.padEnd(border.length - 2)} │ +${rows.join("\n")} └${border}┘`; } @@ -109,6 +227,19 @@ export class SequentialThinkingServer { this.branches[input.branchId].push(input); } + // One thought in, at most one out. Thoughts enter a branch in the order + // they enter the history, so the evicted thought is its branch's first. + if (this.thoughtHistory.length > this.maxThoughtHistory) { + const [oldest] = this.thoughtHistory.splice(0, 1); + if (oldest.branchFromThought && oldest.branchId) { + const branch = this.branches[oldest.branchId]; + branch.shift(); + if (branch.length === 0) { + delete this.branches[oldest.branchId]; + } + } + } + return { content: [ { @@ -118,6 +249,9 @@ export class SequentialThinkingServer { thoughtNumber: input.thoughtNumber, totalThoughts: input.totalThoughts, nextThoughtNeeded: input.nextThoughtNeeded, + // Both describe the retained thoughts: once the cap is + // reached the length stays at MAX_THOUGHT_HISTORY, and a + // branch whose thoughts were all evicted is no longer listed. branches: Object.keys(this.branches), thoughtHistoryLength: this.thoughtHistory.length, }, diff --git a/src/sequentialthinking/package.json b/src/sequentialthinking/package.json index 24e203f537..18b019f929 100644 --- a/src/sequentialthinking/package.json +++ b/src/sequentialthinking/package.json @@ -1,6 +1,6 @@ { "name": "@modelcontextprotocol/server-sequential-thinking", - "version": "2.0.0", + "version": "2.1.1", "description": "MCP server for sequential thinking and problem solving", "license": "SEE LICENSE IN LICENSE", "mcpName": "io.github.modelcontextprotocol/server-sequential-thinking", @@ -36,13 +36,11 @@ "dependencies": { "@modelcontextprotocol/server": "^2.3.1", "chalk": "^5.3.0", - "yargs": "^17.7.2", "zod": "^4.2.0" }, "devDependencies": { "@modelcontextprotocol/client": "^2.3.1", "@types/node": "^24", - "@types/yargs": "^17.0.32", "@vitest/coverage-v8": "^4.1.12", "shx": "^0.4.0", "typescript": "^5.8.2", diff --git a/src/time/README.md b/src/time/README.md index b4dfa76613..b9fc205394 100644 --- a/src/time/README.md +++ b/src/time/README.md @@ -11,14 +11,21 @@ Requires MCP Python SDK 2.x (`mcp>=2.3,<3`). ### Available Tools - `get_current_time` - Get current time in a specific timezone or system timezone. - - Required arguments: - - `timezone` (string): IANA timezone name (e.g., 'America/New_York', 'Europe/London') + - Optional arguments: + - `timezone` (string): IANA timezone name (e.g., 'America/New_York', 'Europe/London'). Defaults to the server's local timezone. - `convert_time` - Convert time between timezones. - Required arguments: - - `source_timezone` (string): Source IANA timezone name - `time` (string): Time in 24-hour format (HH:MM) - - `target_timezone` (string): Target IANA timezone name + - Optional arguments: + - `source_timezone` (string): Source IANA timezone name. Defaults to the server's local timezone. + - `target_timezone` (string): Target IANA timezone name. Defaults to the server's local timezone. + +- `get_time_ago` - Get the time a given number of days, hours, minutes and seconds ago in a specific timezone, for questions such as "what happened in the last 5 hours?". + - Optional arguments: + - `timezone` (string): IANA timezone name; defaults to the server's local timezone + - `days_ago`, `hours_ago`, `minutes_ago`, `seconds_ago` (non-negative JSON integers, default 0; a fraction, numeric string or boolean is rejected): how far back to go. They add up, so two and a half hours ago is `hours_ago: 2, minutes_ago: 30`. + - The offset is elapsed time, counted back from the current instant, so across a daylight-saving change "a day ago" is exactly 24 hours ago and its wall-clock reading can differ from now by an hour. An offset reaching before year 1 is an error. A timezone name, here and in `--local-timezone`, must be an exact IANA key, case included: `Europe/Warsaw` is accepted and `europe/warsaw` is rejected on every platform, including macOS, whose case-insensitive filesystem would otherwise resolve it. @@ -31,6 +38,7 @@ The server speaks both eras of the Model Context Protocol over stdio, with nothi The server is built on the SDK's high-level `MCPServer`. Each tool declares an `outputSchema` and returns its result twice: as `structuredContent`, and as the same object in indented JSON in a text block, for a client that does not read structured output. - **A tool failure is an `isError` result** whose text the model can read and act on, prefixed `Error executing tool : `. That covers an unknown or non-exact timezone (`Invalid timezone: ...`), an empty timezone (`Missing required argument: `), a time not in `HH:MM` 24-hour form, a wall-clock time skipped by a clock change, and an argument missing or of the wrong type (the SDK's validation message). +- **An ambiguous time is reported, not guessed.** On the day a clock change repeats a wall-clock time (a DST fall-back, such as 01:30 on 2024-11-03 in `America/New_York`), `convert_time` returns its first occurrence in `source` and `target`, and the second in `later_occurrence`: a `note` saying the time occurs twice, with that occurrence's own `source`, `target` and `time_difference`. For any other time `later_occurrence` is `null`. - **An unknown tool name is a JSON-RPC `-32602` error** (`Unknown tool: `), not an `isError` result, as both protocol versions specify. - The server registers no prompts or resources. Their list methods answer with empty lists. @@ -208,7 +216,7 @@ Optionally, you can add it to a file called `.vscode/mcp.json` in your workspace ### Customization - System Timezone -By default, the server automatically detects your system's timezone. You can override this by adding the argument `--local-timezone` to the `args` list in the configuration. +By default, the server automatically detects your system's timezone. You can override this by adding the argument `--local-timezone` to the `args` list in the configuration. This local timezone is also what a tool uses for a timezone argument the call omits, and each tool's input schema names it as that argument's `default`. An explicitly empty timezone is still an error. Example: ```json @@ -267,7 +275,29 @@ Response (`structuredContent`, and the same JSON as text): "day_of_week": "Tuesday", "is_dst": false }, - "time_difference": "+14.0h" + "time_difference": "+14.0h", + "later_occurrence": null +} +``` + +3. Get a past time: +```json +{ + "name": "get_time_ago", + "arguments": { + "timezone": "Europe/Warsaw", + "hours_ago": 2, + "minutes_ago": 30 + } +} +``` +Response (`structuredContent`, and the same JSON as text): +```json +{ + "timezone": "Europe/Warsaw", + "datetime": "2024-01-01T10:30:00+01:00", + "day_of_week": "Monday", + "is_dst": false } ``` @@ -288,10 +318,11 @@ npx @modelcontextprotocol/inspector uv run mcp-server-time ## Examples of Questions for Claude -1. "What time is it now?" (will use system timezone) +1. "What time is it now?" (the timezone can be omitted: the server uses its local timezone) 2. "What time is it in Tokyo?" 3. "When it's 4 PM in New York, what time is it in London?" 4. "Convert 9:30 AM Tokyo time to New York time" +5. "What time was it two and a half hours ago?" ## Development diff --git a/src/time/pyproject.toml b/src/time/pyproject.toml index d124841d71..109a35476b 100644 --- a/src/time/pyproject.toml +++ b/src/time/pyproject.toml @@ -1,6 +1,6 @@ [project] name = "mcp-server-time" -version = "2026.10.28" +version = "2026.11.4" description = "A Model Context Protocol server providing tools for time queries and timezone conversions for LLMs" readme = "README.md" requires-python = ">=3.10" diff --git a/src/time/src/mcp_server_time/server.py b/src/time/src/mcp_server_time/server.py index e0b15c23c2..f9c2ece10e 100644 --- a/src/time/src/mcp_server_time/server.py +++ b/src/time/src/mcp_server_time/server.py @@ -40,6 +40,7 @@ class TimeTools(str, Enum): GET_CURRENT_TIME = "get_current_time" CONVERT_TIME = "convert_time" + GET_TIME_AGO = "get_time_ago" class TimeResult(BaseModel): @@ -53,6 +54,26 @@ class TimeConversionResult(BaseModel): source: TimeResult target: TimeResult time_difference: str + later_occurrence: "LaterOccurrence | None" = Field( + default=None, + description=( + "Set only when the source time occurs twice on this date because " + "a clock change repeats it (a DST fall-back): the same conversion " + "at its second occurrence. source and target are the first." + ), + ) + + +class LaterOccurrence(BaseModel): + """The second reading of an ambiguous source time (#5196).""" + + note: str + source: TimeResult + target: TimeResult + time_difference: str + + +TimeConversionResult.model_rebuild() class TimeConversionInput(BaseModel): @@ -115,6 +136,37 @@ def get_current_time(self, timezone_name: str) -> TimeResult: is_dst=bool(current_time.dst()), ) + def get_time_ago( + self, + timezone_name: str, + days: int = 0, + hours: int = 0, + minutes: int = 0, + seconds: int = 0, + ) -> TimeResult: + """Get the time an elapsed span before now, in specified timezone. + + The span is subtracted from the current instant in UTC, not from the + local wall clock, so "5 hours ago" across a clock change is the instant + five real hours back (#2710). An offset that reaches past what + `datetime` can hold is a `ToolError`, not an `OverflowError`. + """ + timezone = get_zoneinfo(timezone_name) + try: + span = timedelta(days=days, hours=hours, minutes=minutes, seconds=seconds) + past_time = (datetime.now(ZoneInfo("UTC")) - span).astimezone(timezone) + except OverflowError: + raise ToolError( + "Time offset out of range: the result would be before year 1" + ) + + return TimeResult( + timezone=timezone_name, + datetime=past_time.isoformat(timespec="seconds"), + day_of_week=past_time.strftime("%A"), + is_dst=bool(past_time.dst()), + ) + def convert_time( self, source_tz: str, time_str: str, target_tz: str ) -> TimeConversionResult: @@ -146,31 +198,63 @@ def convert_time( f"{source_time.date().isoformat()} (skipped by a clock change)" ) - target_time = source_time.astimezone(target_timezone) - source_offset = source_time.utcoffset() or timedelta() - target_offset = target_time.utcoffset() or timedelta() - hours_difference = (target_offset - source_offset).total_seconds() / 3600 + def convert(at: datetime) -> tuple[TimeResult, TimeResult, str]: + """`at` as a source and target `TimeResult`, and their difference.""" + target_time = at.astimezone(target_timezone) + source_offset = at.utcoffset() or timedelta() + target_offset = target_time.utcoffset() or timedelta() + hours_difference = (target_offset - source_offset).total_seconds() / 3600 + + if hours_difference.is_integer(): + time_diff_str = f"{hours_difference:+.1f}h" + else: + # For fractional hours like Nepal's UTC+5:45 + time_diff_str = f"{hours_difference:+.2f}".rstrip("0").rstrip(".") + "h" + + return ( + TimeResult( + timezone=source_tz, + datetime=at.isoformat(timespec="seconds"), + day_of_week=at.strftime("%A"), + is_dst=bool(at.dst()), + ), + TimeResult( + timezone=target_tz, + datetime=target_time.isoformat(timespec="seconds"), + day_of_week=target_time.strftime("%A"), + is_dst=bool(target_time.dst()), + ), + time_diff_str, + ) - if hours_difference.is_integer(): - time_diff_str = f"{hours_difference:+.1f}h" - else: - # For fractional hours like Nepal's UTC+5:45 - time_diff_str = f"{hours_difference:+.2f}".rstrip("0").rstrip(".") + "h" + source, target, time_difference = convert(source_time) + + # A wall-clock time a clock change repeats (e.g. 01:30 on a DST + # fall-back day) has two instants. The datetime above is the first + # (fold=0); fold=1 is the second, at a different UTC offset. Report + # both rather than presenting the first as the only answer (#5196). + later_occurrence = None + later_time = source_time.replace(fold=1) + if later_time.utcoffset() != source_time.utcoffset(): + later_source, later_target, later_difference = convert(later_time) + later_occurrence = LaterOccurrence( + note=( + f"{time_str} occurs twice in {source_tz} on " + f"{source_time.date().isoformat()} because a clock change " + f"repeats it: first at {source.datetime}, then at " + f"{later_source.datetime}. source and target are the first " + "occurrence; this is the second." + ), + source=later_source, + target=later_target, + time_difference=later_difference, + ) return TimeConversionResult( - source=TimeResult( - timezone=source_tz, - datetime=source_time.isoformat(timespec="seconds"), - day_of_week=source_time.strftime("%A"), - is_dst=bool(source_time.dst()), - ), - target=TimeResult( - timezone=target_tz, - datetime=target_time.isoformat(timespec="seconds"), - day_of_week=target_time.strftime("%A"), - is_dst=bool(target_time.dst()), - ), - time_difference=time_diff_str, + source=source, + target=target, + time_difference=time_difference, + later_occurrence=later_occurrence, ) @@ -230,9 +314,9 @@ def get_current_time( timezone: Annotated[ str, Field( - description=f"IANA timezone name (e.g., 'America/New_York', 'Europe/London'). Use '{local_tz}' as local timezone if no timezone provided by the user." + description=f"IANA timezone name (e.g., 'America/New_York', 'Europe/London'). Optional: defaults to the server's local timezone, '{local_tz}', when omitted." ), - ], + ] = local_tz, ) -> TimeResult: require("timezone", timezone) return time_server.get_current_time(timezone) @@ -243,26 +327,62 @@ def get_current_time( annotations=READ_ONLY, ) def convert_time( + # Keyword-only, so the required `time` can keep its place between the + # two optional timezones and the schema keeps its property order. + *, source_timezone: Annotated[ str, Field( - description=f"Source IANA timezone name (e.g., 'America/New_York', 'Europe/London'). Use '{local_tz}' as local timezone if no source timezone provided by the user." + description=f"Source IANA timezone name (e.g., 'America/New_York', 'Europe/London'). Optional: defaults to the server's local timezone, '{local_tz}', when omitted." ), - ], + ] = local_tz, time: Annotated[ str, Field(description="Time to convert in 24-hour format (HH:MM)") ], target_timezone: Annotated[ str, Field( - description=f"Target IANA timezone name (e.g., 'Asia/Tokyo', 'America/San_Francisco'). Use '{local_tz}' as local timezone if no target timezone provided by the user." + description=f"Target IANA timezone name (e.g., 'Asia/Tokyo', 'America/San_Francisco'). Optional: defaults to the server's local timezone, '{local_tz}', when omitted." ), - ], + ] = local_tz, ) -> TimeConversionResult: require("source_timezone", source_timezone) require("target_timezone", target_timezone) return time_server.convert_time(source_timezone, time, target_timezone) + @server.tool( + name=TimeTools.GET_TIME_AGO.value, + description=( + "Get the time a given number of days, hours, minutes and seconds " + "ago in a specific timezone" + ), + annotations=READ_ONLY, + ) + def get_time_ago( + timezone: Annotated[ + str, + Field( + description=f"IANA timezone name (e.g., 'America/New_York', 'Europe/London'). Optional: defaults to the server's local timezone, '{local_tz}', when omitted." + ), + ] = local_tz, + days_ago: Annotated[ + int, Field(description="Whole days before now", ge=0, strict=True) + ] = 0, + hours_ago: Annotated[ + int, Field(description="Whole hours before now", ge=0, strict=True) + ] = 0, + minutes_ago: Annotated[ + int, Field(description="Whole minutes before now", ge=0, strict=True) + ] = 0, + seconds_ago: Annotated[ + int, Field(description="Whole seconds before now", ge=0, strict=True) + ] = 0, + ) -> TimeResult: + require("timezone", timezone) + return time_server.get_time_ago( + timezone, days_ago, hours_ago, minutes_ago, seconds_ago + ) + return server diff --git a/src/time/tests/test_entrypoints.py b/src/time/tests/test_entrypoints.py index 2daaf6f433..7790627910 100644 --- a/src/time/tests/test_entrypoints.py +++ b/src/time/tests/test_entrypoints.py @@ -113,7 +113,11 @@ async def test_console_script_boots_over_stdio() -> None: assert init.model_dump(by_alias=True, mode="json")["serverInfo"]["name"] == ( "mcp-time" ) - assert [tool.name for tool in tools.tools] == ["get_current_time", "convert_time"] + assert [tool.name for tool in tools.tools] == [ + "get_current_time", + "convert_time", + "get_time_ago", + ] assert result.model_dump(by_alias=True, mode="json")["isError"] is False diff --git a/src/time/tests/test_eras.py b/src/time/tests/test_eras.py index d619fab639..e45a109624 100644 --- a/src/time/tests/test_eras.py +++ b/src/time/tests/test_eras.py @@ -104,7 +104,11 @@ async def test_tools_list_is_the_same_in_both_eras() -> None: again = tool_wire((await client.list_tools()).tools) async with connect("legacy") as client: legacy = tool_wire((await client.list_tools()).tools) - assert [t["name"] for t in modern] == ["get_current_time", "convert_time"] + assert [t["name"] for t in modern] == [ + "get_current_time", + "convert_time", + "get_time_ago", + ] assert modern == again == legacy diff --git a/src/time/tests/test_protocol.py b/src/time/tests/test_protocol.py index ff17d34895..72efc7fe3d 100644 --- a/src/time/tests/test_protocol.py +++ b/src/time/tests/test_protocol.py @@ -205,12 +205,12 @@ def expected_tools(local_tz: str) -> list[dict[str, Any]]: "title": "Timezone", "description": ( "IANA timezone name (e.g., 'America/New_York', " - f"'Europe/London'). Use '{local_tz}' as local timezone " - "if no timezone provided by the user." + "'Europe/London'). Optional: defaults to the " + f"server's local timezone, '{local_tz}', when omitted." ), + "default": local_tz, } }, - "required": ["timezone"], }, "outputSchema": TIME_RESULT_SCHEMA, "annotations": READ_ONLY_ANNOTATIONS, @@ -227,9 +227,10 @@ def expected_tools(local_tz: str) -> list[dict[str, Any]]: "title": "Source Timezone", "description": ( "Source IANA timezone name (e.g., 'America/New_York', " - f"'Europe/London'). Use '{local_tz}' as local timezone " - "if no source timezone provided by the user." + "'Europe/London'). Optional: defaults to the " + f"server's local timezone, '{local_tz}', when omitted." ), + "default": local_tz, }, "time": { "type": "string", @@ -241,12 +242,13 @@ def expected_tools(local_tz: str) -> list[dict[str, Any]]: "title": "Target Timezone", "description": ( "Target IANA timezone name (e.g., 'Asia/Tokyo', " - f"'America/San_Francisco'). Use '{local_tz}' as local " - "timezone if no target timezone provided by the user." + "'America/San_Francisco'). Optional: defaults to the " + f"server's local timezone, '{local_tz}', when omitted." ), + "default": local_tz, }, }, - "required": ["source_timezone", "time", "target_timezone"], + "required": ["time"], }, "outputSchema": { "type": "object", @@ -255,10 +257,77 @@ def expected_tools(local_tz: str) -> list[dict[str, Any]]: "source": {"$ref": "#/$defs/TimeResult"}, "target": {"$ref": "#/$defs/TimeResult"}, "time_difference": {"title": "Time Difference", "type": "string"}, + "later_occurrence": { + "anyOf": [ + {"$ref": "#/$defs/LaterOccurrence"}, + {"type": "null"}, + ], + "default": None, + "description": ( + "Set only when the source time occurs twice on this " + "date because a clock change repeats it (a DST " + "fall-back): the same conversion at its second " + "occurrence. source and target are the first." + ), + }, }, "required": ["source", "target", "time_difference"], - "$defs": {"TimeResult": TIME_RESULT_SCHEMA}, + "$defs": { + "LaterOccurrence": { + "type": "object", + "title": "LaterOccurrence", + "description": ( + "The second reading of an ambiguous source time (#5196)." + ), + "properties": { + "note": {"title": "Note", "type": "string"}, + "source": {"$ref": "#/$defs/TimeResult"}, + "target": {"$ref": "#/$defs/TimeResult"}, + "time_difference": { + "title": "Time Difference", + "type": "string", + }, + }, + "required": ["note", "source", "target", "time_difference"], + }, + "TimeResult": TIME_RESULT_SCHEMA, + }, + }, + "annotations": READ_ONLY_ANNOTATIONS, + }, + { + "name": "get_time_ago", + "description": ( + "Get the time a given number of days, hours, minutes and seconds " + "ago in a specific timezone" + ), + "inputSchema": { + "type": "object", + "title": "get_time_agoArguments", + "properties": { + "timezone": { + "type": "string", + "title": "Timezone", + "default": local_tz, + "description": ( + "IANA timezone name (e.g., 'America/New_York', " + "'Europe/London'). Optional: defaults to the server's " + f"local timezone, '{local_tz}', when omitted." + ), + }, + **{ + f"{unit}_ago": { + "type": "integer", + "title": f"{unit.capitalize()} Ago", + "default": 0, + "minimum": 0, + "description": f"Whole {unit} before now", + } + for unit in ("days", "hours", "minutes", "seconds") + }, + }, }, + "outputSchema": TIME_RESULT_SCHEMA, "annotations": READ_ONLY_ANNOTATIONS, }, ] @@ -512,9 +581,9 @@ async def test_get_current_time_ignores_extra_arguments() -> None: assert result["datetime"] == "2024-01-01T12:00:00+00:00" -async def test_get_current_time_does_not_default_to_the_local_timezone() -> None: - # The local timezone only appears in the schema descriptions; the server - # never substitutes it for an empty argument. +async def test_get_current_time_does_not_default_an_empty_timezone() -> None: + # Only an omitted timezone takes the local default (#2853). An explicitly + # empty one is a mistake, reported as missing rather than guessed at. async with connected("Europe/Warsaw") as session: result = await call(session, "get_current_time", {"timezone": ""}) assert result == tool_error( @@ -522,11 +591,70 @@ async def test_get_current_time_does_not_default_to_the_local_timezone() -> None ) +async def test_get_current_time_defaults_to_the_local_timezone() -> None: + # An omitted timezone is the server's local one (#2853): the + # --local-timezone override here. + async with connected("Asia/Kolkata") as session: + with frozen_at("2024-01-01 12:00:00+00:00"): + result = await call_json(session, "get_current_time", {}) + assert result == { + "timezone": "Asia/Kolkata", + "datetime": "2024-01-01T17:30:00+05:30", + "day_of_week": "Monday", + "is_dst": False, + } + + +async def test_get_current_time_defaults_to_the_detected_local_timezone() -> None: + # With no --local-timezone, the default is the detected zone, the same one + # the argument's description names. + with patch("mcp_server_time.server.get_localzone_name", return_value="Asia/Tokyo"): + async with connected(None) as session: + with frozen_at("2024-01-01 12:00:00+00:00"): + result = await call_json(session, "get_current_time", {}) + assert result["timezone"] == "Asia/Tokyo" + assert result["datetime"] == "2024-01-01T21:00:00+09:00" + + +@pytest.mark.parametrize( + "arguments,source,target", + [ + # Either timezone, or both, may be omitted (#2853); each omitted one + # is the local timezone, Europe/Warsaw here. + ( + {"time": "12:00", "target_timezone": "Asia/Tokyo"}, + ("Europe/Warsaw", "2024-01-01T12:00:00+01:00"), + ("Asia/Tokyo", "2024-01-01T20:00:00+09:00"), + ), + ( + {"source_timezone": "Asia/Tokyo", "time": "12:00"}, + ("Asia/Tokyo", "2024-01-01T12:00:00+09:00"), + ("Europe/Warsaw", "2024-01-01T04:00:00+01:00"), + ), + ( + {"time": "12:00"}, + ("Europe/Warsaw", "2024-01-01T12:00:00+01:00"), + ("Europe/Warsaw", "2024-01-01T12:00:00+01:00"), + ), + ], +) +async def test_convert_time_defaults_omitted_timezones_to_the_local_timezone( + arguments: dict[str, Any], + source: tuple[str, str], + target: tuple[str, str], +) -> None: + async with connected("Europe/Warsaw") as session: + with frozen_at("2024-01-01 06:00:00+00:00"): + result = await call_json(session, "convert_time", arguments) + assert (result["source"]["timezone"], result["source"]["datetime"]) == source + assert (result["target"]["timezone"], result["target"]["datetime"]) == target + + @pytest.mark.parametrize( "arguments,needles", [ # Rejected by MCPServer against the tool's signature, before it runs. - ({}, ["timezone\n Field required [type=missing"]), + # An omitted timezone is not an error: it defaults (#2853). ({"timezone": 5}, ["timezone\n Input should be a valid string"]), ({"timezone": None}, ["timezone\n Input should be a valid string"]), ], @@ -599,6 +727,8 @@ async def test_convert_time_wire_result() -> None: "is_dst": False, }, "time_difference": "+8.0h", + # Only an ambiguous source time sets it (#5196). + "later_occurrence": None, } assert result == { "content": [{"type": "text", "text": json.dumps(expected, indent=2)}], @@ -795,8 +925,10 @@ async def test_convert_time_into_a_nonexistent_local_time() -> None: async def test_convert_time_at_an_ambiguous_local_time() -> None: - # 01:30 on 2024-11-03 happens twice in New York. fold=0 picks the first - # occurrence, still on EDT. + # 01:30 on 2024-11-03 happens twice in New York: first on EDT, then, after + # the clocks go back at 02:00, on EST. source and target are the first + # occurrence; later_occurrence carries the second, with a note saying the + # time is ambiguous rather than presenting one answer as certain (#5196). async with connected() as session: with frozen_at("2024-11-03 12:00:00+00:00"): result = await call_json( @@ -816,6 +948,122 @@ async def test_convert_time_at_an_ambiguous_local_time() -> None: } assert result["target"]["datetime"] == "2024-11-03T05:30:00+00:00" assert result["time_difference"] == "+4.0h" + assert result["later_occurrence"] == { + "note": ( + "01:30 occurs twice in America/New_York on 2024-11-03 because a " + "clock change repeats it: first at 2024-11-03T01:30:00-04:00, then " + "at 2024-11-03T01:30:00-05:00. source and target are the first " + "occurrence; this is the second." + ), + "source": { + "timezone": "America/New_York", + "datetime": "2024-11-03T01:30:00-05:00", + "day_of_week": "Sunday", + "is_dst": False, + }, + "target": { + "timezone": "UTC", + "datetime": "2024-11-03T06:30:00+00:00", + "day_of_week": "Sunday", + "is_dst": False, + }, + "time_difference": "+5.0h", + } + + +async def test_convert_time_ambiguity_is_in_the_structured_result() -> None: + # The structured content carries the same later_occurrence as the text. + async with connected() as session: + with frozen_at("2024-10-27 12:00:00+00:00"): + result = await call( + session, + "convert_time", + { + "source_timezone": "Europe/Warsaw", + "time": "02:30", + "target_timezone": "UTC", + }, + ) + structured = result["structuredContent"] + assert structured == json.loads(result["content"][0]["text"]) + assert structured["source"]["datetime"] == "2024-10-27T02:30:00+02:00" + assert structured["target"]["datetime"] == "2024-10-27T00:30:00+00:00" + later = structured["later_occurrence"] + assert later["source"]["datetime"] == "2024-10-27T02:30:00+01:00" + assert later["target"]["datetime"] == "2024-10-27T01:30:00+00:00" + assert later["time_difference"] == "-1.0h" + + +@pytest.mark.parametrize( + "instant,source,time,target,later_target,later_difference", + [ + # Lord Howe Island moves its clocks back by only 30 minutes (02:00 -> + # 01:30), so 01:45 repeats with a fractional difference. + ( + "2024-04-07 00:00:00+00:00", + "Australia/Lord_Howe", + "01:45", + "UTC", + "2024-04-06T15:15:00+00:00", + "-10.5h", + ), + # The target's own zone does not matter: only the source is ambiguous. + ( + "2024-11-03 12:00:00+00:00", + "America/New_York", + "01:00", + "Asia/Kolkata", + "2024-11-03T11:30:00+05:30", + "+10.5h", + ), + ], +) +async def test_convert_time_later_occurrence_in_other_zones( + instant: str, + source: str, + time: str, + target: str, + later_target: str, + later_difference: str, +) -> None: + async with connected() as session: + with frozen_at(instant): + result = await call_json( + session, + "convert_time", + {"source_timezone": source, "time": time, "target_timezone": target}, + ) + later = result["later_occurrence"] + assert later["note"].startswith(f"{time} occurs twice in {source} on ") + assert later["source"]["timezone"] == source + assert later["target"]["datetime"] == later_target + assert later["time_difference"] == later_difference + + +@pytest.mark.parametrize( + "instant,source,time", + [ + # The hour after the repeated one, on the fall-back day itself. + ("2024-11-03 12:00:00+00:00", "America/New_York", "02:00"), + # The hour before it, also on the fall-back day. + ("2024-11-03 12:00:00+00:00", "America/New_York", "00:59"), + # The same wall time on an ordinary day. + ("2024-11-10 12:00:00+00:00", "America/New_York", "01:30"), + # The same date in a zone with no clock change. + ("2024-11-03 12:00:00+00:00", "UTC", "01:30"), + ], +) +async def test_convert_time_unambiguous_time_has_no_later_occurrence( + instant: str, source: str, time: str +) -> None: + async with connected() as session: + with frozen_at(instant): + result = await call_json( + session, + "convert_time", + {"source_timezone": source, "time": time, "target_timezone": "UTC"}, + ) + assert result["later_occurrence"] is None async def test_convert_time_difference_uses_offsets_at_the_converted_instant() -> None: @@ -867,13 +1115,14 @@ async def test_convert_time_ignores_extra_arguments() -> None: "arguments,needles", [ # Rejected by MCPServer against the tool's signature, before it runs. + # `time` stays required; the timezones default when omitted (#2853). ( - {"source_timezone": "UTC", "time": "12:00"}, - ["target_timezone\n Field required [type=missing"], + {"source_timezone": "UTC", "target_timezone": "UTC"}, + ["time\n Field required [type=missing"], ), ( - {"time": "12:00", "target_timezone": "UTC"}, - ["source_timezone\n Field required [type=missing"], + {"source_timezone": 5, "time": "12:00"}, + ["source_timezone\n Input should be a valid string"], ), ( {"source_timezone": "UTC", "time": 1200, "target_timezone": "UTC"}, @@ -949,6 +1198,164 @@ async def test_convert_time_checks_timezones_before_the_time_string() -> None: ) +# --------------------------------------------------------------------------- +# tools/call get_time_ago +# --------------------------------------------------------------------------- + + +async def test_get_time_ago_wire_result() -> None: + # The issue's own example (#2710): two and a half hours ago. + async with connected() as session: + with frozen_at("2024-01-01 12:00:00+00:00"): + result = await call( + session, + "get_time_ago", + {"timezone": "UTC", "hours_ago": 2, "minutes_ago": 30}, + ) + expected = { + "timezone": "UTC", + "datetime": "2024-01-01T09:30:00+00:00", + "day_of_week": "Monday", + "is_dst": False, + } + assert result == { + "content": [{"type": "text", "text": json.dumps(expected, indent=2)}], + "structuredContent": expected, + "isError": False, + } + + +@pytest.mark.parametrize( + "arguments,expected", + [ + # No offset at all is now. + ({}, "2024-01-01T12:00:00+00:00"), + ({"seconds_ago": 1}, "2024-01-01T11:59:59+00:00"), + ({"minutes_ago": 90}, "2024-01-01T10:30:00+00:00"), + # Every unit at once, crossing back into the previous year. + ( + {"days_ago": 1, "hours_ago": 2, "minutes_ago": 3, "seconds_ago": 4}, + "2023-12-31T09:56:56+00:00", + ), + # 2023 is not a leap year: 365 days back is the same date. + ({"days_ago": 365}, "2023-01-01T12:00:00+00:00"), + ], +) +async def test_get_time_ago_offsets(arguments: dict[str, Any], expected: str) -> None: + async with connected() as session: + with frozen_at("2024-01-01 12:00:00+00:00"): + result = await call_json( + session, "get_time_ago", {"timezone": "UTC", **arguments} + ) + assert result["datetime"] == expected + + +async def test_get_time_ago_defaults_to_the_local_timezone() -> None: + # An omitted timezone is the server's local timezone, as the schema's + # `default` says. + async with connected("Europe/Warsaw") as session: + with frozen_at("2024-01-01 12:00:00+00:00"): + result = await call_json(session, "get_time_ago", {"hours_ago": 1}) + assert result == { + "timezone": "Europe/Warsaw", + "datetime": "2024-01-01T12:00:00+01:00", + "day_of_week": "Monday", + "is_dst": False, + } + + +@pytest.mark.parametrize( + "arguments,expected,is_dst", + [ + # 03:30 CEST on the spring-forward day is 01:30 UTC; two real hours + # back is 23:30 UTC, 00:30 CET. Subtracting from the wall clock would + # wrongly give 01:30. + ({"hours_ago": 2}, "2024-03-31T00:30:00+01:00", False), + # A day ago is 24 elapsed hours, 01:30 UTC the day before, so across + # the change the wall clock reads 02:30 CET, not 03:30. + ({"days_ago": 1}, "2024-03-30T02:30:00+01:00", False), + ({"minutes_ago": 30}, "2024-03-31T03:00:00+02:00", True), + ], +) +async def test_get_time_ago_counts_elapsed_time_across_a_clock_change( + arguments: dict[str, Any], expected: str, is_dst: bool +) -> None: + async with connected() as session: + with frozen_at("2024-03-31 01:30:00+00:00"): + result = await call_json( + session, "get_time_ago", {"timezone": "Europe/Warsaw", **arguments} + ) + assert (result["datetime"], result["is_dst"]) == (expected, is_dst) + + +async def test_get_time_ago_echoes_the_requested_name_not_a_canonical_one() -> None: + async with connected() as session: + with frozen_at("2024-01-01 12:00:00+00:00"): + result = await call_json( + session, "get_time_ago", {"timezone": "US/Eastern", "days_ago": 1} + ) + assert result["timezone"] == "US/Eastern" + assert result["datetime"] == "2023-12-31T07:00:00-05:00" + assert result["day_of_week"] == "Sunday" + + +@pytest.mark.parametrize( + "arguments,needles", + [ + # Rejected by MCPServer against the tool's signature, before it runs. + ( + {"hours_ago": -1}, + ["hours_ago\n Input should be greater than or equal to 0"], + ), + # The offsets are strict integers: no fraction, numeric string or + # boolean is coerced into a different cutoff. + ({"days_ago": 1.5}, ["days_ago\n Input should be a valid integer"]), + ({"hours_ago": "2"}, ["hours_ago\n Input should be a valid integer"]), + ({"hours_ago": True}, ["hours_ago\n Input should be a valid integer"]), + ({"minutes_ago": "ten"}, ["minutes_ago\n Input should be a valid integer"]), + ({"seconds_ago": None}, ["seconds_ago\n Input should be a valid integer"]), + ({"timezone": None}, ["timezone\n Input should be a valid string"]), + ], +) +async def test_get_time_ago_argument_validation( + arguments: dict[str, Any], needles: list[str] +) -> None: + async with connected() as session: + result = await call(session, "get_time_ago", arguments) + assert_validation_error(result, "get_time_ago", *needles) + + +@pytest.mark.parametrize( + "arguments,message", + [ + # Raised by the tool as a ToolError. + ({"timezone": ""}, "Missing required argument: timezone"), + ( + {"timezone": "europe/warsaw"}, + "Invalid timezone: 'No time zone found with key europe/warsaw'", + ), + # Past what timedelta can hold, and past year 1. + ( + {"days_ago": 10**10}, + "Time offset out of range: the result would be before year 1", + ), + ( + {"days_ago": 999_999_999}, + "Time offset out of range: the result would be before year 1", + ), + # The timezone is checked before the offset. + ( + {"timezone": "Bad/Zone", "days_ago": 10**10}, + "Invalid timezone: 'No time zone found with key Bad/Zone'", + ), + ], +) +async def test_get_time_ago_errors(arguments: dict[str, Any], message: str) -> None: + async with connected() as session: + result = await call(session, "get_time_ago", arguments) + assert result == tool_error("get_time_ago", message) + + # --------------------------------------------------------------------------- # Unknown tools # --------------------------------------------------------------------------- diff --git a/src/time/uv.lock b/src/time/uv.lock index 5cbafc4db7..d030a69906 100644 --- a/src/time/uv.lock +++ b/src/time/uv.lock @@ -497,7 +497,7 @@ wheels = [ [[package]] name = "mcp-server-time" -version = "2026.10.28" +version = "2026.11.4" source = { editable = "." } dependencies = [ { name = "mcp" },