diff --git a/.claude/skills/board-ops/SKILL.md b/.claude/skills/board-ops/SKILL.md index 0a8f58251..c34f1c165 100644 --- a/.claude/skills/board-ops/SKILL.md +++ b/.claude/skills/board-ops/SKILL.md @@ -26,9 +26,57 @@ The two projects have their own field and option IDs and none of them are interchangeable — a #28 id passed to #11 is rejected with "option Id does not belong to the field", so the mistake is at least loud. +## Finding a card without trusting `--limit` + +⚠️ **`gh project item-list --limit N` truncates silently.** Past `N` it returns +the first `N` items with no error and no warning, so a `select` over the result +matches nothing and a card that exists reads as missing. Board #28 passed 500 +items in September 2026 — double the figure quoted here two months earlier — and +the old `--limit 500` lookups reported a carded issue as unboarded and 16 GHSA +drafts as absent in one session (#2451). A limit is a guess about the board's +size; don't make the recipes depend on it being right. + +- **An issue's card is looked up from the issue**, which is independent of board + size — see [Move an existing card](#move-an-existing-card). +- **A draft card or a whole-board dump** (the GHSA lookup, the snapshot, the + recovery dump, `/issue-triage`'s sweep and audit) genuinely needs the full + listing. Those recipes use a limit with headroom **and** compare the result's + `.items | length` against the `.totalCount` that `item-list --format json` + also returns, so a truncated listing fails loudly instead of passing as + complete. The check also catches a failed `gh` call, whose empty output has + neither key. Where a later step reads the dump from a file, an incomplete dump + is deleted, so that step fails on the missing file rather than running on + partial data. + **Only issues go on a board — never PRs, never draft cards.** A PR is tracked through the card of the issue it closes. +**The one exception is a GitHub security advisory**, tracked by a draft card +titled `[GHSA-xxxx-yyyy-zzzz] - …` because a real issue would disclose it before +a fix exists. The flow is `/security-advisory`. + +⚠️ **A draft card has no repository and no issue number, so the issue-side +lookup below cannot find one**, and `item-add --url` has no URL to be given. +Look it up by **title** in the full listing instead, then feed that item id to +`item-edit` or `item-delete` exactly as usual: + +```sh +GHSA=GHSA-xxxx-yyyy-zzzz # the advisory's real id +ITEM_ID= # never let an earlier lookup's id survive a failed one +BOARD=$(gh project item-list 28 --owner modelcontextprotocol --format json --limit 2000) +if jq -e '(.items | length) == .totalCount' <<<"$BOARD" >/dev/null; then + ITEM_ID=$(jq -r '.items[] | select(.content.type=="DraftIssue") + | select(.content.title | startswith("['"$GHSA"']")) | .id' <<<"$BOARD") + [ -n "$ITEM_ID" ] || echo "no draft card titled [$GHSA] on #28" >&2 +else + echo "item-list incomplete or failed — raise --limit; not concluding anything" >&2 +fi +``` + +Match on the **bracketed GHSA id**, not on words from the summary — a summary is +free text and two advisories can share one. Advisory drafts live on #28 only; +`/issue-triage`'s audit reports one found anywhere else. + ## V2 board (#28) IDs The project node id and the field ids are stable. The **option** ids are **not** — @@ -123,24 +171,48 @@ gh project item-edit --project-id PVT_kwDOCt2Azc4BA5sz --id "$ITEM_ID" \ ### Move an existing card -Look the item id up by issue number rather than re-adding it. Keep `--limit` -above the board's item count (~265 as of 2026-08-01) — past it `item-list` -truncates **silently**, `select` matches nothing, and `item-edit --id ""` fails -with an opaque node-resolution error rather than saying the limit was too low. +Look the item id up **from the issue** rather than re-adding it. An issue's +`projectItems` lists the cards it has on every board, so the lookup does not +depend on how many items the board holds (see [Finding a card without trusting +`--limit`](#finding-a-card-without-trusting---limit)). Select the card by the +board's **node id**, not its number: project numbers are per-owner, and an issue +can also sit on a user-owned project that happens to be numbered 28. Querying +through the repository also means the issue number cannot match another repo's +issue — board #11 really does carry a `modelcontextprotocol/servers` card. + +**For a v1 card on #11, swap every #28 id, not just the lookup's.** #11's node +id `PVT_kwDOCt2Azc4BA5sz` goes in both the lookup's `select` and the edit's +`--project-id`; the edit also takes #11's own Status field +`PVTSSF_lADOCt2Azc4BA5szzgzkS-g` and an option id from [its +table](#v1-board-11-ids); and a delete is `item-delete 11`. + +The mutation runs only on a non-empty id: `item-edit --id ""` fails with an +opaque node-resolution error rather than saying the card was not found. The +`|| ITEM_ID=` matters too — on a GraphQL error (a number that is a PR, not an +issue; a rate limit) `gh api` still prints the raw error JSON to stdout, which +would otherwise land in `ITEM_ID` as a non-empty "id". `first:100` is the +connection's maximum page; it counts the boards one issue is on, not the cards +on a board, so it has no board-size exposure. + +```sh +N= +ITEM_ID=$(gh api graphql -F n="$N" -f query='query($n:Int!){ + repository(owner:"modelcontextprotocol",name:"inspector"){issue(number:$n){ + projectItems(first:100){nodes{id project{id}}}}}}' \ + --jq '.data.repository.issue.projectItems.nodes[] + | select(.project.id=="PVT_kwDOCt2Azc4BJVxt") | .id') || ITEM_ID= +[ -n "$ITEM_ID" ] || echo "#$N has no card on #28 (or the lookup failed)" >&2 +``` -⚠️ **Filter by repository, not by number alone.** These are **org** projects and -issue numbers are **repo-local**, so an unfiltered `select` can match another -repo's issue that happens to share the number — board #11 really does carry a -`modelcontextprotocol/servers` card — and then moves or deletes the wrong card, -or passes two ids at once (Copilot). +Then edit it — e.g. Status → In Review, when its PR opens: ```sh -ITEM_ID=$(gh project item-list 28 --owner modelcontextprotocol --format json --limit 500 \ - --jq '.items[] | select(.content.repository=="modelcontextprotocol/inspector" - and .content.number==) | .id') -# e.g. Status → In Review, when its PR opens -gh project item-edit --project-id PVT_kwDOCt2Azc4BJVxt --id "$ITEM_ID" \ - --field-id PVTSSF_lADOCt2Azc4BJVxtzg5iI8c --single-select-option-id 159c8a02 +if [ -n "$ITEM_ID" ]; then + gh project item-edit --project-id PVT_kwDOCt2Azc4BJVxt --id "$ITEM_ID" \ + --field-id PVTSSF_lADOCt2Azc4BJVxtzg5iI8c --single-select-option-id 159c8a02 +else + echo "no ITEM_ID — nothing edited" >&2 +fi ``` ### Delete a card @@ -150,10 +222,13 @@ not planned / obsolete / superseded shipped nothing, so its card is **deleted**, not parked in Done: ```sh -ITEM_ID=$(gh project item-list 28 --owner modelcontextprotocol --format json --limit 500 \ - --jq '.items[] | select(.content.repository=="modelcontextprotocol/inspector" - and .content.number==) | .id') -gh project item-delete 28 --owner modelcontextprotocol --id "$ITEM_ID" +# ITEM_ID from the issue-side LOOKUP block in "Move an existing card" above — +# the lookup only, not the item-edit that follows it. +if [ -n "$ITEM_ID" ]; then + gh project item-delete 28 --owner modelcontextprotocol --id "$ITEM_ID" +else + echo "no ITEM_ID — nothing deleted" >&2 +fi ``` Deleting the card removes it from the board only — **the issue itself is @@ -198,7 +273,8 @@ Safe alternatives, in order of preference: **including its `id`**, appending only the new one. `ProjectV2SingleSelectFieldOptionInput.id` is an optional `String`, so a mixed list works. Verify afterward that no card lost its value — snapshot - `gh project item-list … --format json` before and after and diff; don't just + `gh project item-list … --format json --limit 2000` before and after, check + each is complete the way the snapshot below does, and diff; don't just spot-check. Send those dumps to `$BOARD_TMP` too, for the reason above. Both the `Incoming` Status option and the Urgent/High/Medium/Low Priority @@ -220,11 +296,17 @@ PR (Copilot). ```sh BOARD_TMP=$(mktemp -d) -gh project item-list 28 --owner modelcontextprotocol --format json --limit 600 \ +gh project item-list 28 --owner modelcontextprotocol --format json --limit 2000 \ > "$BOARD_TMP/board-snapshot.json" -echo "snapshot: $BOARD_TMP/board-snapshot.json" # note the path; you need it to recover +# A truncated snapshot cannot restore the cards it dropped — refuse to proceed on one. +jq -e '(.items | length) == .totalCount' "$BOARD_TMP/board-snapshot.json" >/dev/null \ + && echo "snapshot: $BOARD_TMP/board-snapshot.json" \ + || { echo "SNAPSHOT INCOMPLETE — raise --limit and retake it before editing options" >&2 + rm -f "$BOARD_TMP/board-snapshot.json"; false; } ``` +Note the printed path; you need it to recover. + ### Recovering from a deleted option This has happened twice — once via the API (~197 items, reconstructed by @@ -241,25 +323,40 @@ and pass the Priority field id `PVTSSF_lADOCt2Azc4BJVxtzg5iJE4`. # 0. Same temp dir the snapshot went to — keep every dump out of the worktree. BOARD_TMP=${BOARD_TMP:-$(mktemp -d)} -# 1. Which cards lost their value, and what did they hold? -gh project item-list 28 --owner modelcontextprotocol --format json --limit 600 \ +# 1. Which cards lost their value, and what did they hold? lost-ids.json is +# kept ONLY when the dump is complete AND the snapshot reports what those cards +# held — step 3 refuses to run without it, so neither a truncated dump nor a +# missing snapshot can turn into a silent no-op or an unconfirmed re-apply. +rm -f "$BOARD_TMP/lost-ids.json" +gh project item-list 28 --owner modelcontextprotocol --format json --limit 2000 \ > "$BOARD_TMP/board-broken.json" -jq -r '[.items[]|select(.status==null)|.id]' "$BOARD_TMP/board-broken.json" \ - > "$BOARD_TMP/lost-ids.json" -jq -r --slurpfile L "$BOARD_TMP/lost-ids.json" '($L[0]) as $lost - | [.items[] | select(.id as $i | $lost|index($i)) | .status // "(none)"] - | group_by(.) | map({s:.[0],c:length}) | .[] | "was \(.s): \(.c)"' \ - "$BOARD_TMP/board-snapshot.json" +if jq -e '(.items | length) == .totalCount' "$BOARD_TMP/board-broken.json" >/dev/null; then + jq -r '[.items[]|select(.status==null)|.id]' "$BOARD_TMP/board-broken.json" \ + > "$BOARD_TMP/lost-ids.json" || rm -f "$BOARD_TMP/lost-ids.json" + jq -r --slurpfile L "$BOARD_TMP/lost-ids.json" '($L[0]) as $lost + | [.items[] | select(.id as $i | $lost|index($i)) | .status // "(none)"] + | group_by(.) | map({s:.[0],c:length}) | .[] | "was \(.s): \(.c)"' \ + "$BOARD_TMP/board-snapshot.json" \ + || { echo "no usable snapshot — cannot confirm what these cards held; not re-applying" >&2 + rm -f "$BOARD_TMP/lost-ids.json"; } +else + echo "board-broken.json INCOMPLETE — raise --limit and re-run step 1" >&2 + rm -f "$BOARD_TMP/board-broken.json" +fi # 2. Recreate the option, echoing every surviving option's id (see above). # NOTE: the recreated option gets a NEW id — the deleted one never comes back. # 3. Re-apply it to the orphaned cards. -for id in $(jq -r '.[]' "$BOARD_TMP/lost-ids.json"); do - gh project item-edit --project-id PVT_kwDOCt2Azc4BJVxt --id "$id" \ - --field-id PVTSSF_lADOCt2Azc4BJVxtzg5iI8c --single-select-option-id - sleep 0.4 -done +if [ -s "$BOARD_TMP/lost-ids.json" ]; then + for id in $(jq -r '.[]' "$BOARD_TMP/lost-ids.json"); do + gh project item-edit --project-id PVT_kwDOCt2Azc4BJVxt --id "$id" \ + --field-id PVTSSF_lADOCt2Azc4BJVxtzg5iI8c --single-select-option-id + sleep 0.4 + done +else + echo "no lost-ids.json — step 1 did not complete; nothing re-applied" >&2 +fi ``` Step 1's grouping is the safety check: confirm the orphaned set is exactly the diff --git a/.claude/skills/issue-create/SKILL.md b/.claude/skills/issue-create/SKILL.md index 5fe54eca0..9e50fde1a 100644 --- a/.claude/skills/issue-create/SKILL.md +++ b/.claude/skills/issue-create/SKILL.md @@ -29,7 +29,10 @@ query, and an unmilestoned one drops out of release planning silently. **Never create a duplicate.** Check the board for a matching item first. **Never create a draft card** (a board card with no issue number) — every board -item is a real GitHub issue. +item is a real GitHub issue. The single exception is a **GitHub security +advisory**, which is private until it is published and so cannot be tracked by +an issue at all; see `/security-advisory`. Nothing you reach through *this* +flow is that case. ## 0. Check the board first diff --git a/.claude/skills/issue-triage/SKILL.md b/.claude/skills/issue-triage/SKILL.md index 0511119e1..ba34eca10 100644 --- a/.claude/skills/issue-triage/SKILL.md +++ b/.claude/skills/issue-triage/SKILL.md @@ -51,13 +51,19 @@ double-boarded (a real defect a past sweep introduced — #1929 reproduced it). D=$(mktemp -d) gh issue list --repo modelcontextprotocol/inspector --state open --limit 1000 \ --json number,milestone > "$D/open.json" -# Union of BOTH boards, filtered to this repo — org boards can hold other repos' issues. +# item-list truncates SILENTLY past --limit (and a failed call writes nothing), and a +# missing card reads as an "unboarded" issue that then gets double-carded — so an +# incomplete dump is deleted, and the steps below fail on the missing file. for P in 28 11; do - gh project item-list $P --owner modelcontextprotocol --format json --limit 700 \ - | jq '[.items[] | select(.content.type=="Issue" - and .content.repository=="modelcontextprotocol/inspector") - | .content.number]' -done | jq -s 'add' > "$D/boarded.json" + gh project item-list $P --owner modelcontextprotocol --format json --limit 2000 > "$D/b$P.json" + jq -e '(.items | length) == .totalCount' "$D/b$P.json" >/dev/null \ + || { echo "board #$P listing INCOMPLETE — raise --limit and re-run" >&2; rm -f "$D/b$P.json"; false; } +done +# Union of BOTH boards, filtered to this repo — org boards can hold other repos' issues. +jq -s '[.[].items[] | select(.content.type=="Issue" + and .content.repository=="modelcontextprotocol/inspector") + | .content.number]' "$D/b28.json" "$D/b11.json" > "$D/boarded.json" \ + || rm -f "$D/boarded.json" # Prints the destination too: milestoned already → Todo, otherwise → Incoming. jq -r --slurpfile b "$D/boarded.json" \ '.[] | select(.number as $n | ($b[0]|index($n))|not) @@ -200,8 +206,9 @@ count means the board contradicts a rule, not that the rule needs revisiting. | Check | Invariant | Fix | | --- | --- | --- | | Double-boarded | An issue has a card on **one** board, the one matching its version label | Delete the wrong-board card | -| Non-Issue items | **Only issues go on a board** — never PRs, never drafts | Delete the item | +| Non-Issue items | **Only issues go on a board** — never PRs, never drafts, *except* a `[GHSA-…]` **draft** on **#28** | Delete the item | | No Status | Every card carries a Status | Set one — `Incoming` if unmilestoned, else by where it actually is | +| GHSA draft missing Status/Priority | An exempted advisory draft still carries both | Set them — `/security-advisory` | | `Incoming` **with** a milestone (#28) | Incoming ⇔ no milestone | Approval was never recorded: move to **Todo**, or clear the milestone | | Past Incoming **without** a milestone (#28) | Everything past Incoming ⇔ milestoned | Claims an approval nobody made: milestone it, or move back to Incoming | | Wrong board for label | `v1` → #11, `v2` → #28 | Move the card to the right board | @@ -217,8 +224,14 @@ D=$(mktemp -d); R=modelcontextprotocol/inspector # the last check below reads closed issues' state reasons. gh issue list --repo $R --state all --limit 2000 \ --json number,state,stateReason,labels,milestone > "$D/i.json" +# item-list truncates SILENTLY past --limit (and a failed call writes nothing); an +# incomplete dump would make every check below lie, so it is deleted and the audit +# fails on the missing file instead. for P in 28 11; do gh project item-list $P --owner modelcontextprotocol \ - --format json --limit 700 > "$D/b$P.json"; done + --format json --limit 2000 > "$D/b$P.json" + jq -e '(.items | length) == .totalCount' "$D/b$P.json" >/dev/null \ + || { echo "board #$P listing INCOMPLETE — raise --limit and re-run" >&2; rm -f "$D/b$P.json"; false; } +done jq -nr --slurpfile o "$D/i.json" --slurpfile a "$D/b28.json" --slurpfile b "$D/b11.json" --arg R "$R" ' ($o[0] | map({key:(.number|tostring), value:{st:.state, sr:(.stateReason // ""), lab:[.labels[].name], ms:(.milestone.title // null)}}) | from_entries) as $M @@ -235,7 +248,25 @@ jq -nr --slurpfile o "$D/i.json" --slurpfile a "$D/b28.json" --slurpfile b "$D/b | [own($b)[] | select(.content.type=="Issue") | {n:.content.number, s:.status}] as $B11 | { "double-boarded": [$B28[].n | select(. as $n | [$B11[].n]|index($n))], - "non-Issue on a board": [(own($a)[], own($b)[]) | select(.content.type!="Issue") | .content.number], + # An advisory draft card is the ONE legitimate non-Issue item (see AGENTS.md). + # The exemption is narrowed three ways, and each one matters: DRAFTS only + # (a GHSA-titled PR is still reported), board #28 ONLY (an advisory has no + # business on #11), and the `[GHSA-` title prefix (a stray draft is still + # reported). Reports the TITLE, since a draft has no number. + "non-Issue on a board": [(own($a)[] | select(.content.type!="Issue" + and ((.content.type=="DraftIssue" + and ((.content.title // "") | startswith("[GHSA-"))) | not))), + (own($b)[] | select(.content.type!="Issue"))] + | map(.content.title // "(untitled)"), + # $B28/$B11 hold only Issue items, so the Status and Priority checks below + # cannot see an advisory draft. Exempting drafts from the check above would + # therefore have made a half-made advisory card invisible to the whole + # audit; this is the narrow replacement. + "GHSA draft missing Status/Priority": + [own($a)[] | select(.content.type=="DraftIssue" + and ((.content.title // "") | startswith("[GHSA-"))) + | select(.status==null or .priority==null) + | (.content.title[0:24])], "no Status": [($B28[], $B11[]) | select(.s==null) | .n], "Incoming w/ milestone": [$B28[] | select(.s=="Incoming" and ms(.n)!=null) | .n], "past Incoming, no ms": [$B28[] | select(.s!=null and .s!="Incoming" and .s!="Done" @@ -276,7 +307,12 @@ Two things the queries must account for, both learned the hard way: - **The two milestone checks are #28-only.** Every milestone in this repo is a v2 release bucket, so a `v1` issue has none it could take — running the Incoming⇔milestone invariant over board #11 would flag every card on it for a - state it cannot reach. + state it cannot reach. They also read **Issue items only**, which is what + exempts a `[GHSA-` advisory draft on #28: it cannot carry a milestone, and its + approval is the advisory's **acceptance** (`/security-advisory` step 3), + which lives on the advisory rather than the board. The audit cannot see that, + so a GHSA draft past `Incoming` is correct once its advisory is accepted and + is not reported. - **Count the labels; don't test for presence.** The invariant is *exactly one*, so a predicate that only asks "is any version label present" passes an issue carrying **both** `v1` and `v2` — which belongs to two lines at once @@ -286,6 +322,31 @@ Two things the queries must account for, both learned the hard way: and that check then reports `0` while the invariant it states (no drafts) is being violated (Copilot). The filter admits an item with no repository and excludes only cards that name a *different* one. +- **Advisory drafts are carved out of that check by TITLE, not by type.** A + GitHub security advisory is private until it is published, so it is tracked by + a draft card titled `[GHSA-xxxx-yyyy-zzzz] - …` — the one exception `AGENTS.md` + grants to "no draft cards", and the `security-advisory` skill is the flow. There + are enough of them open at any time that counting them would pin this check + permanently non-zero, and a check that never prints `0` stops being read at + all. The discriminator is deliberately the **title prefix** and nothing + broader: exempting *all* drafts, or every card whose Status is `Incoming`, + would let an ordinary stray draft through, which is the defect the check + exists for. So a draft titled anything else is still reported — by title, + since a draft has no issue number to print. + ⚠️ **The title prefix alone is not enough, because a title is not a type and + not a board.** Matched on its own it would also exempt a **pull request** + whose title happens to start `[GHSA-` — a plausible title for a security fix + — and an advisory draft misfiled on **#11**, where the replacement field check + below does not look either, so both checks would read `0`. The exemption is + therefore `DraftIssue` **and** `[GHSA-` **and** board #28; #11 still reports + every non-Issue item it carries. + ⚠️ **The exemption had to come with a replacement check.** `$B28` and `$B11` + are built from `Issue` items only, so the `no Status` and `no Priority` + checks never see a draft — before the carve-out the non-Issue check was the + *only* thing looking at one, and exempting drafts there alone would have made + a half-made advisory card invisible to the entire audit. Hence + `GHSA draft missing Status/Priority`, which reads the item-level `.status` + and `.priority` that `item-list` exposes for a draft as it does for an issue. - **`$M` holds closed issues too** — the lookup is built from `gh issue list --state all`, which it has to be, because the last check reads closed issues' state reasons. So `isopen` is not there to cope with a missing diff --git a/.claude/skills/security-advisory/SKILL.md b/.claude/skills/security-advisory/SKILL.md new file mode 100644 index 000000000..404feca08 --- /dev/null +++ b/.claude/skills/security-advisory/SKILL.md @@ -0,0 +1,355 @@ +--- +name: security-advisory +description: "Take a privately reported vulnerability through this repo's security advisory flow — board it, verify who owns the code path, accept or reject, fix it in the private fork, ship to every affected release line, publish, then turn the card into public tracking. Use when a vulnerability is reported privately; when deciding whether an advisory is ours to fix; when looking up or creating its private fork; when answering a reporter; or when a GHSA-titled board card needs handling." +disable-model-invocation: false +--- + +# Handling a security advisory + +Private vulnerability reporting is enabled on this repo and +[`SECURITY.md`](../../../SECURITY.md) routes every report to it — the issue +chooser deliberately has no security template, because a vulnerability report +must not open a public issue. So an advisory never arrives as an issue, and for +most of its life it must **not** become one. + +Two steps in this flow are **outward-facing, and both stay human-gated**: +**accepting** an advisory (the reporter sees it) and **publishing** it (it +becomes public, and there is no unpublish). Never automate either, never +bulk-apply them, and never take either step because a checklist said to. +Everything else here is mechanics. + +⚠️ **A CVE and the credits are *choices made at publish time*, not effects of +publishing.** Requesting a CVE is an optional action on the advisory, and a +credit appears only when someone is explicitly added **and accepts** it. They +are named here because they are the parts a maintainer must not forget — the +reporter's credit especially, since nothing prompts for it — not because +publishing performs them. + +Related: `/board-ops` (the card IDs and recipes) and `/issue-create`, for the +labels, milestone and board that public tracking takes — **after publication, +never merely after the release**, since the release ships the fix while the +advisory may still be private. + +⚠️ **How that tracking is created depends on the affected lines**, and only the +v2 path is a conversion: a v2 issue is **converted** from the draft (filing one +separately would duplicate both the issue and the card), while a v1 issue is +**filed** on #11, because the draft is on #28 and cannot move there. Step 6 has +the per-line sequence. + +⚠️ **`/pr-flow` does not apply to the fix itself.** It requires a public issue +and a public PR against the release branch — the disclosure this flow exists to +delay. The fix is reviewed inside the private fork (step 4), and `/pr-flow` +becomes relevant only once the advisory is published. + +## The flow + +| # | Step | Gate | +| --- | --- | --- | +| 1 | Advisory lands in state `triage` → **draft card** on board #28 | Mechanical | +| 2 | **Verify the claim** — who owns the code path, and **which release lines are affected** | Judgment | +| 3 | Valid → **accept** (`triage` → `draft`); invalid → close with a reason | **Human only** | +| 4 | Create the **private fork**, fix and review there | Mechanical | +| 5 | Merge **to every affected line**, release each, then **publish** the advisory | **Human only** | +| 6 | **After publication**, turn the card into public tracking — per line: convert (v2), or file on #11 and delete the draft (v1) | Mechanical | + +### 1. Board it as a draft card + +An advisory is private, so a public issue tracking it would disclose it before a +fix exists. It therefore gets a **draft card** — the one documented exception to +[`AGENTS.md`](../../../AGENTS.md#issue-driven-work-style)'s "every board item is +a real GitHub issue". + +- **Title:** `[GHSA-xxxx-yyyy-zzzz] - `. That `[GHSA-` prefix + is not cosmetic: the board audit in `/issue-triage` keys its draft carve-out + on it, so a card titled any other way is reported as a stray draft. +- **Body:** `**Advisory:** ` on the first line, then severity and + reported date. The link first, because a maintainer reading the card has no + other route back to the private advisory. + ⚠️ **Do not copy the vulnerability description onto the card.** Project + access and advisory access are **separate permission sets**, so the board's + audience is not the advisory's audience — anyone with project access reads + the card, whether or not they are an advisory collaborator. The boards are + private ([`/issue-triage`](../issue-triage/SKILL.md)), so this is a wider + audience than intended rather than a public leak, but a reproduction or a PoC + is the part worth keeping to the people handling it. The card carries the + **link and triage metadata only**; the link is how a reader with access gets + the details, and the absence of details is how a reader without access is + told they do not have them. +- **Status `Incoming`**, plus a **provisional** Priority scored with the + `/issue-triage` rubric. `Incoming` is correct even though somebody clearly + triaged it to make the card: nobody has approved shipping a fix yet, and a + draft card has no milestone to carry the approval. + ⚠️ **Provisional is not a hedge — it is the only honest score at this + point.** The rubric's first axis is *severity*, and step 2 says ownership is + established **before** severity, precisely because #2409 looked severe right + up until it turned out not to be ours. At step 1 you have a report and + nothing verified, so score what the report claims, mark it provisional in the + body, and **re-score it at the end of step 2**, when you know whether the + code is ours and which lines it reaches. An advisory that turns out to be + upstream has its card deleted rather than re-scored (step 3). + ⚠️ **Put the score's arithmetic in the draft body**, marked provisional and + dated. `/issue-triage` says to record it as an issue comment, and a draft + card has no comments — so without this the Priority is a bare word with + nothing behind it, and the step-2 re-score cannot tell what it is revising. + Write the two axes, the bonuses you claimed, and the total as **numbers and + rubric names only** — `Severity 4`, `+1 security`, `Total 6`. That is + deliberately *not* the comment form: its template follows each axis with a + free-text justification ("Severity 3 — a real feature is broken…"), and for + an advisory that justification is the impact and the affected surface, which + is exactly what the warning above keeps off the card. The reasoning behind a + number belongs in the private advisory. Leave the provisional line in place + when you re-score and add the new one under it, so the change of view is + legible. + ⚠️ **Set both fields.** The board audit's non-Issue check now exempts + `[GHSA-` drafts, so a half-made card no longer trips it; the audit carries a + narrow replacement check (see `/issue-triage`) and it is the only thing + looking. + +The card is made **by hand**. There is no `PROJECT_TOKEN` in this org and +`organization projects: write` is a permission `GITHUB_TOKEN` structurally +cannot hold, so a board write is unreachable from Actions — the same constraint +`AGENTS.md` records for the dependency sweeps. **Do not propose a nightly +workflow for this;** that approach was tried and abandoned for exactly this +reason. + +```sh +# --paginate: this endpoint returns 30 per page, and an inventory that silently +# stops at the first page is worse than none — it reads as "nothing pending". +gh api --paginate repos/modelcontextprotocol/inspector/security-advisories \ + --jq '.[] | select(.state=="triage") + | "\(.ghsa_id)\t\(.severity)\t\(.summary)"' +``` + +### 2. Verify the claim — who owns the code path, and which lines it affects + +Before assessing severity, establish that the vulnerable code is **ours**. A +report can be entirely accurate about behavior the Inspector merely exhibits +because an SDK does it. + +⚠️ **This is not hypothetical.** #2409 — a loopback/HTTPS-exemption finding — +read as an Inspector defect and turned out to live in +`@modelcontextprotocol/client` (`typescript-sdk#2591`). The reporter withdrew +it. Had ownership been checked after the severity assessment rather than before, +the fix would have been written against the wrong repo. + +So: reproduce it, find the code, and check whether that code is first-party or +reached through a dependency. An advisory against upstream code is not ours to +accept or publish. + +⚠️ **"Upstream's problem" is not a reason to say it in public.** A genuine +unfixed vulnerability handed to a public upstream issue is disclosed — by us, +on someone else's behalf, before they have a fix. Route it through **that +project's own private reporting channel** (its `SECURITY.md`, or its advisory +form), and only reference a public upstream issue once the upstream has +published. Where the reporter would rather carry it over themselves, say so and +let them. #2409 took the benign version of this path: the reporter withdrew the +report here and raised it upstream. + +#### Which release lines are affected — ask it here, not at merge time + +⚠️ **An advisory is very nearly the only work the v1 line ever receives**, so +this is exactly where assuming v2 does the most damage. `SECURITY.md` supports +v1 for **security fixes only**, published under the `v1-latest` dist-tag, and +its "What to Include" asks the reporter to state "whether it affects v2, v1, or +both". Read what they said and then check it yourself — it is a request, not a +required form field, so it is often absent and it is never authoritative when +present. A v1-only advisory assumed to be +v2 gets merged to a branch where the bug does not exist, and one affecting both +lines leaves v1 **unpatched** while the advisory is published, which is the +worst outcome this whole flow can produce. + +So the outcome of step 2 is a **set** of affected lines, and each one is +shipped on its own terms: + +| Line | Branch | Flow | Publishes to | +| --- | --- | --- | --- | +| v2 | `v2/main` | `fix branch → v2/main → (milestone) main` | `latest` | +| v1 | `v1/main` | `fix branch → v1/main`, flat — **no merge into `main`** | `v1-latest` | + +**The two lines publish independently under separate dist-tags, so a v1 fix is +not forward-ported** — if v2 is affected too, that is a second fix on `v2/main`, +not a merge. Branch names carry the version segment either way +(`v1/fix/…`, `v2/fix/…`). + +**Now re-score the card's Priority**, replacing the provisional one from step 1. +This is the first point at which the rubric's severity axis has anything solid +under it: you know the code is ours, you have reproduced it, and you know how +many lines it reaches — and "affects both lines" is itself a severity input the +provisional score could not have had. + +### 3. Accept, or close + +**Valid and ours → accept.** In the UI this is "Accept and open as draft"; it +moves the advisory `triage` → `draft`. The state is readable as `state` and +`submission.accepted` on the API object. + +**Then move the card `Incoming` → `Todo`.** On #28 the approval act is +normally assigning a milestone, and a draft card cannot carry one — so for an +advisory draft, **accepting the advisory is the approval**, and it is what +licenses the card to leave `Incoming`. `AGENTS.md` records this as the +advisory exemption to its `Incoming` ⇔ milestone invariant. The milestone +arrives with the public issue in step 6, where the ordinary rule resumes. + +**Invalid, out of scope, or upstream → close** with a comment saying which, and +why. A reporter who is told nothing reasonably assumes they were ignored. + +⚠️ **Closing an advisory leaves its draft card behind — delete it.** Nothing +shipped, so `Done` would be a false record and `Incoming` would claim work is +still queued; `AGENTS.md` deletes a card in exactly this situation, and a +rejected advisory's card is now invisible to the audit's non-Issue check by +construction. The delete recipe is in `/board-ops`. + +⚠️ **Accepting is a human act, always.** It is visible to the reporter and it +commits this project to treating the report as a real vulnerability. Nothing in +this skill authorizes taking it — surface the recommendation and let a +maintainer click. + +⚠️ **There is no comment API for security advisories.** Not in REST (the +advisory object exposes no comments endpoint) and not in GraphQL +(`RepositoryAdvisory` is not commentable, and no advisory-comment mutation +exists). Comments are **UI-only**, so every exchange with a reporter is manual — +you cannot script the reply, and you cannot read the thread back with `gh`. + +### 4. The private fork + +Accepted advisories are fixed in a **private fork** GitHub creates for the +advisory: a private repo named `-` in the org. + +⚠️ **Read `private_fork` FIRST. The POST is not a probe — it CREATES one.** +Calling it to "check whether a fork exists" makes one, in the org, which then +needs cleaning up. This was learned the hard way. + +```sh +# Idempotency check — does one already exist? +gh api repos/modelcontextprotocol/inspector/security-advisories/ \ + --jq '.private_fork // "none"' + +# Only if that printed "none": +gh api -X POST \ + repos/modelcontextprotocol/inspector/security-advisories//forks +# → 202 Accepted; the fork appears shortly afterwards. +``` + +⚠️ **Deleting a private fork needs the `delete_repo` OAuth scope, which a +default `gh` token does not carry.** So a fork created by mistake is not +something you can quietly undo — it takes a re-scoped token or an admin in the +UI. That asymmetry is the whole reason for the read-first rule above. + +Fix and review inside the fork. Its PRs and commits are private, so none of the +normal public review flow applies; the diff comes back as an ordinary commit at +merge time, **to the branch of each line step 2 found affected** — `v2/main` +for v2, `v1/main` for v1. + +⚠️ **Move the card as the work moves.** `AGENTS.md`'s lifecycle applies to this +card like any other: **`In Progress`** when the fix is started, **`In Review`** +when the fork's PR is open. The card being private is not a reason to skip it — +it is the reason to do it, since the fork is invisible to everyone who is not on +the advisory, and this card is the only place the rest of the team can see the +work exists at all. A card that sits in `Incoming` until it jumps to `Done` +reports "unreviewed, nobody committed to it" for the entire time somebody is +actively fixing it. + +### 5. Merge to every affected line, release, publish + +Publish **after** the fix has shipped in a release, never before — publishing +discloses the vulnerability, so doing it while users have no upgrade available +hands out a working exploit. + +⚠️ **"Shipped" means shipped on *every* affected line.** The two lines release +independently under separate dist-tags, so v2 reaching `latest` says nothing +about `v1-latest`. Publishing with one line still unpatched discloses a live +vulnerability to the users who have no fix — and they are the users least able +to move, since v1 is the deprecated line they are on because upgrading is hard. + +⚠️ **The patch stops being secret at MERGE, not at publish — and no release +path changes that.** Merging the private fork puts an ordinary public commit on +`v2/main` or `v1/main`, readable by anyone, and a v2 release then moves it +through **two public PRs** on its way to `main`. So the window between merge and +publish is not a period of secrecy to protect; it is a period of **exposure to +anyone reading commits**, which is why it should be short. Merge close to the +release rather than early, and publish as soon as the release is out. + +**Do not hand this off to the release skill.** It is `disable-model-invocation: +true`, so a pointer to it from here is a dead end for the model anyway — a +maintainer invokes `/release` themselves. Say which lines need a release and +stop there. A v1 fix takes no merge into `main` at all and publishes straight +from `v1/main`, so it does not go through that procedure. + +⚠️ **Publishing is irreversible and human-gated.** It makes the advisory +public, and there is no undo. Same rule as accepting: recommend, never perform. + +**Before publishing, do the two things publishing will not do for you:** +request the **CVE** (optional, and the advisory is the only place to ask) and +**add the reporter to the credits** — a credit is an explicit addition the +person then has to accept, so an unadded reporter is simply never credited, and +that is the failure nobody notices because nothing reports it. + +### 6. After publication, turn the card into public tracking + +**The trigger is publication, not the release.** The release ships the fix +while the advisory can still be private, and a public issue opened in that gap +describes a vulnerability the advisory has not disclosed yet. Wait for step 5 +to finish. + +Once it has, the work becomes ordinary board history — by conversion for v2, by +filing for v1. + +⚠️ **Convert FIRST — the order is not interchangeable.** GitHub's "Convert to +issue" creates a **new** issue from the draft; there is no way to point an +existing card at an issue you filed separately. Filing the issue by hand and +then converting produces two issues and two cards, which is why this step reads +the way it does: + +1. **Convert the draft card to an issue** on board #28 (the card keeps its + place and its field values; the issue is created from the card's title and + body). +2. Apply a **type label** and the **version label of the line the fix shipped + on**, then a milestone — and those two are not independent: + + | Affected | Version label | Milestone | Board | + | --- | --- | --- | --- | + | v2 | `v2` | the release the fix shipped in | #28 — the converted card is already there | + | v1 | `v1` | **none** — every milestone is a v2 release bucket | **#11**, which has no Priority field | + | both | **two issues**, one per line — see below | | | + + Do **not** run `/issue-create`'s add-card step for the converted card: it + already exists. + + **The draft converts exactly once, so "both" needs a stated order.** Every + issue carries exactly one version label and lives on one board, and there is + only ever one draft card — so one line inherits it and the other gets a + fresh issue: + + 1. **Convert the draft into the `v2` issue on #28.** v2 takes the + conversion because the draft is already on #28 and v2 is the line with a + milestone to record. + 2. **File the `v1` issue separately** through `/issue-create` — `v1`, a type + label, **no milestone**, and a card on **#11** (Status only; that board + has no Priority field). This one *is* filed rather than converted, which + is not a contradiction of step 6: there is no second draft to convert. + 3. Cross-link the two so neither reads as the whole story, then **close + both** and move both cards to `Done`. + + **For a v1-only advisory** the draft is on the wrong board and cannot be + moved there by converting: file the `v1` issue on #11 as in (2), then + **delete** the #28 draft rather than converting it — a converted card would + put a `v1` issue on #28, which the board audit reports as a wrong-board + card. +3. **Close it.** The work shipped before the issue existed. +4. Move the card to **`Done`** — correct here, because the fix genuinely + shipped. + +## API facts worth not re-deriving + +All verified against the live API. + +| Thing | Fact | +| --- | --- | +| States | `triage` → `draft` (accepted) → `published`; or `closed` | +| Accepted? | `submission.accepted` on the advisory object, alongside `state` | +| Private fork | `POST …/security-advisories/{ghsa_id}/forks` → `202`, private repo `-` in the org | +| Fork idempotency | Read `.private_fork` first — the POST creates, it does not probe | +| Fork deletion | Needs the `delete_repo` OAuth scope; a default `gh` token lacks it | +| Comments | **No API at all**, REST or GraphQL. UI-only | +| Board writes | Not automatable — no `PROJECT_TOKEN`, and `GITHUB_TOKEN` cannot hold `organization projects: write` | +| Affected lines | `SECURITY.md` **asks** for v2 / v1 / both — a request, not a required field. Read it, never rely on it | diff --git a/.claude/skills/security-advisory/evals/evals.json b/.claude/skills/security-advisory/evals/evals.json new file mode 100644 index 000000000..06743379e --- /dev/null +++ b/.claude/skills/security-advisory/evals/evals.json @@ -0,0 +1,34 @@ +[ + { + "prompt": "Someone just reported a vulnerability against this repo privately. What do I do with it?", + "expect": "security-advisory" + }, + { + "prompt": "How do I fix a privately reported vulnerability without the patch being visible before the release goes out?", + "expect": "security-advisory" + }, + { + "prompt": "How do I reply to a reporter who filed a vulnerability privately?", + "expect": "security-advisory" + }, + { + "prompt": "A privately reported vulnerability turns out to be in an upstream package rather than our own code. How do I close it out?", + "expect": "security-advisory" + }, + { + "prompt": "When is it safe to make a privately reported vulnerability public, and who makes that call here?", + "expect": "security-advisory" + }, + { + "prompt": "Walk me through taking an accepted vulnerability report all the way to a shipped, disclosed fix.", + "expect": "security-advisory" + }, + { + "prompt": "What is the capital of Portugal?", + "expect": null + }, + { + "prompt": "Rename the local variable `tmp` to `buffer` in this snippet: `const tmp = 1; return tmp + 1;`", + "expect": null + } +] diff --git a/.claude/skills/testing/SKILL.md b/.claude/skills/testing/SKILL.md index 6da0df69b..59604fd62 100644 --- a/.claude/skills/testing/SKILL.md +++ b/.claude/skills/testing/SKILL.md @@ -16,6 +16,16 @@ test goes, how to run it, and how to clear the gate. **If it does, load the `test-servers` skill now — that is step one, before choosing a location or writing a line.** +⚠️ **Load it before searching the code, not after.** A task phrased as +end-to-end or integration coverage of an MCP operation — listing tools, +paginating a list, calling a tool, reading a resource — almost always stands a +fixture up, so treat that phrasing as the answer to the question above and load +`test-servers` *first*. Grepping for an existing test to copy is not a +substitute: the fixture you find that way (a config under +`test-servers/configs/`) does not tell you which of the three shapes below +drives it, or that it can be stale. If the skill then shows the case needs no +fixture, you have lost one skill load. + The condition is **"does this test depend on a fixture from `test-servers/`?"** — not which tier it lands in, and not which directory it lands in. There are two ways to depend on one, and they need different halves of that skill: diff --git a/AGENTS.md b/AGENTS.md index d05cc5a74..1c0b7d906 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -25,6 +25,7 @@ users invoke them by name. | [`pr-flow`](.claude/skills/pr-flow/SKILL.md) | Branch naming, DCO signoff, screenshots, opening the PR, requesting a Copilot review, responding, closing out | Model-invoked, or `/pr-flow` | | [`pre-push-gate`](.claude/skills/pre-push-gate/SKILL.md) | Running `npm run local:gate` and diagnosing a failing stage | Model-invoked, or `/pre-push-gate` | | [`release`](.claude/skills/release/SKILL.md) | Cutting a release: bump on `v2/main`, milestone merge, tag `origin/main`, publish | `/release` | +| [`security-advisory`](.claude/skills/security-advisory/SKILL.md) | A privately reported vulnerability end to end: the draft card, who owns the code path, which release lines are affected, accepting, the private fork, publishing, public tracking per line (v2 converts; v1 files) | Model-invoked, or `/security-advisory` | | [`test-servers`](.claude/skills/test-servers/SKILL.md) | Picking and running a showcase test server; the stale-build hazard | Model-invoked, or `/test-servers` | Longer-form human documentation lives in [`docs/`](./docs) — see the table in the @@ -255,7 +256,7 @@ node/field/option IDs, and the option-deletion hazard` was cut at `#28`, so 90 skill at all: it is absent from the listing and the Skill tool refuses it. The costs are asymmetric — a spurious load costs ~250 characters, a missed one costs a wrong base branch or an unsigned commit — and the budget is not tight - (nine of the ten are model-invoked today and total ~3.2k of 4k). Reserve + (ten of the eleven are model-invoked today and total ~3.7k of 4k). Reserve `true` for a procedure that is genuinely only ever started deliberately — `release` is the only one left, because nobody cuts a release by implication. ⚠️ **A `true` skill cannot be reached by another skill either.** If a @@ -267,7 +268,7 @@ node/field/option IDs, and the option-deletion hazard` was cut at `#28`, so 90 cases (n=4) and `testing` from 3/5 to 2/5, while the six new skills all measured 100% and every negative case stayed clean. So the ceiling is attention, not characters — we were at 2.8k of a 4k budget throughout _that - experiment_ (it is ~3.2k now; the point is that nothing was near the cap). Adding + experiment_ (it is ~3.7k now; the point is that nothing was near the cap). Adding a skill therefore has a cost paid by the _existing_ ones, which only `skills:eval` can see. **Re-run the full eval after any flip _or description edit_**, not just the changed skill's own cases. @@ -295,8 +296,14 @@ node/field/option IDs, and the option-deletion hazard` was cut at `#28`, so 90 variance: each prompt is scored on its own `passes / RUNS`, so more prompts steady nothing, they cover more of the ways someone might reach the skill and expose a description that only fires on one narrow phrasing; `npm run skills:eval` actually runs them (it needs the - `claude` CLI and real model calls, so it is deliberately **not** in the gate — + selected agent's CLI — `claude` by default, `copilot` with `AGENT=copilot` — and real model calls, so it is deliberately **not** in the gate — run it when adding a skill or editing a model-invoked description). + **The skills serve GitHub Copilot users too, from where they are.** The + Copilot CLI discovers `.claude/skills/` alongside `.github/skills/`, so a + Copilot session reaches the same procedures with no second copy — and none + may be added. `AGENT=copilot npm run skills:eval` measures that side with the + same cases; one run measures one agent and never folds the two rates together + (#2397, details in `docs/skill-authoring.md`). Two things learned writing the first set, both of which make a case measure the wrong thing: a prompt whose answer is **already in this file** is not a trigger case — the model answers correctly without the skill, and the case @@ -335,7 +342,7 @@ node/field/option IDs, and the option-deletion hazard` was cut at `#28`, so 90 overflows, and drops the least-invoked entries **first** — which are exactly the model-invoked skills that must fire on their own. `verify:skills` prints the current cost against the budget recorded in `scripts/lib/skill-manifest.mjs` - (3,234/4,000 characters as of this writing) and fails when it is exceeded. Raise + (3,729/4,000 characters as of this writing) and fails when it is exceeded. Raise the budget deliberately, or tighten a description; each entry is capped at 1,536 characters regardless, so **put the key use case first**. @@ -358,12 +365,13 @@ skills; the rules are here. - **Before starting work, check the board for the relevant item.** - **Every board item is a real GitHub issue.** No draft cards. Before creating a new issue, check the board for a matching item — **never create a duplicate**. + - **The one exception is a GitHub security advisory**, which is tracked by a **draft card** titled `[GHSA-xxxx-yyyy-zzzz] - `. An advisory is private until it is published, so a real issue would disclose the vulnerability before a fix exists — the thing the whole advisory flow is for. The card is made by hand (no `PROJECT_TOKEN` exists in this org, and `GITHUB_TOKEN` cannot hold `organization projects: write`), and it becomes public tracking **once the advisory is published** — never merely once the fix ships, since the release can precede publication. How depends on the line: a `v2` issue is **converted** from the draft on #28, while a `v1` issue is **filed** on #11 and the #28 draft is deleted, because a draft cannot convert onto another board and a `v1` issue on #28 is a wrong-board card. An advisory affecting both lines produces one issue per line. The `[GHSA-` prefix is load-bearing: it is what the board audit's draft carve-out keys on, so **any other draft card is still a defect to delete**. The flow itself — verifying who owns the code path, accepting, the private fork, publishing — is the `security-advisory` skill. **Accepting and publishing an advisory are outward-facing and stay human-gated; never automate or bulk-apply either.** - **Only issues go on a board — never PRs.** A PR gets the `v2` label but is tracked through its linked issue's card (via `Closes #N`), not its own board item. - **Label by version — every issue and every PR, no exceptions.** Exactly one of `v1` (work targeting `v1/main`, the deprecated security-fix-only line) or `v2` (active development; the default for anything new). There is no unlabeled state and no "decide later": an issue with neither label belongs to no version line and is invisible to every version-filtered query. Set it at **create time** (`gh issue create --label v2 …`), never by backfilling. **If the target version isn't obvious, it's `v2`.** - **Label by type — exactly one of `bug` / `enhancement` / `documentation` / `chore` / `question`** on every issue you create or triage. The version label says which line the work belongs to; the type label says what kind of work it is, and the two are independent. Don't force the binary: pressing a docs task or a dependency pin into `enhancement` degrades it to "not a bug", at which point filtering by it stops telling you anything. A **PR** needs no type label — it is classified through the issue it closes. - **Every v2 issue you create gets a milestone.** Milestones are _release_ buckets, so pick by when the work ships. Never leave a v2 issue you filed unmilestoned pending a decision. Two exceptions, both deliberate: an issue that arrives **unboarded** stays unmilestoned in `Incoming` until a maintainer approves it — there, the _absence_ of a milestone is the signal; and **every milestone is a v2 release bucket, so a `v1` issue has none to take**. Say so when filing one rather than dropping it in a v2.x bucket. - **Every v2 board item has a Priority.** Priority is a **board field**, not a label, so an unboarded issue has nowhere to store it. Derive it with the rubric in the `issue-triage` skill rather than asserting it. Board #11 has no Priority field; a v1 issue gets a Status and nothing else. -- **`Incoming` ⇔ no milestone; everything past it ⇔ milestoned — on board #28.** Board #11 is exempt for the reason above: a v1 issue has no bucket to take, so its Status is set on its own and the audit's milestone checks do not apply to it. The rest of the invariant is unchanged: assigning the milestone _is_ the approval act, so the two always go together. `Todo` asserts a maintainer signed off, so never park an unreviewed issue there — that erases the distinction and quietly promotes unreviewed work into the queue. An issue created through the documented flow skips `Incoming` entirely, because filing it _was_ the approval. +- **`Incoming` ⇔ no milestone; everything past it ⇔ milestoned — on board #28.** Board #11 is exempt for the reason above: a v1 issue has no bucket to take, so its Status is set on its own and the audit's milestone checks do not apply to it. A `[GHSA-` **advisory draft** on #28 is exempt too, for a different reason: a draft card cannot carry a milestone, so its approval act is **accepting the advisory**, which moves it `Incoming` → `Todo`; its milestone arrives with the public issue after publication. The rest of the invariant is unchanged: assigning the milestone _is_ the approval act, so the two always go together. `Todo` asserts a maintainer signed off, so never park an unreviewed issue there — that erases the distinction and quietly promotes unreviewed work into the queue. An issue created through the documented flow skips `Incoming` entirely, because filing it _was_ the approval. - **`Done` means the work shipped.** Exactly two things earn a card a place in Done: its **PR merged**, or it is a **parent whose last sub-issue closed**. Anything else — duplicate, won't fix, not planned, obsolete, superseded — means nothing shipped, so the card is **deleted**. Done is read as the record of what a milestone actually delivered; a duplicate sitting there makes that record wrong in a way nobody can detect later. Deleting a card touches the board only — the issue keeps its labels and comments and stays searchable forever. - **When work begins**, create a feature branch and set Status to **In Progress**. **Branch names start with the target version segment** — `v2/fix/2071-oauth-resource-metadata`, `v1/fix/proxy-ssrf-pin` — matching the base branches themselves. - **When work is complete**, run `npm run format` then `npm run local:gate`, **sign off every commit** (`git commit -s` — the DCO check is a hard merge gate with no partial credit), open a PR against the matching base branch with **`Closes #` as the body's first line**, and set Status to **In Review**. diff --git a/LICENSE b/LICENSE new file mode 100644 index 000000000..4a9398576 --- /dev/null +++ b/LICENSE @@ -0,0 +1,216 @@ +The MCP project is undergoing a licensing transition from the MIT License to the Apache License, Version 2.0 ("Apache-2.0"). All new code and specification contributions to the project are licensed under Apache-2.0. Documentation contributions (excluding specifications) are licensed under CC-BY-4.0. + +Contributions for which relicensing consent has been obtained are licensed under Apache-2.0. Contributions made by authors who originally licensed their work under the MIT License and who have not yet granted explicit permission to relicense remain licensed under the MIT License. + +No rights beyond those granted by the applicable original license are conveyed for such contributions. + +--- + + Apache License + Version 2.0, January 2004 + http://www.apache.org/licenses/ + + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + + 1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to the Licensor for inclusion in the Work by the copyright + owner or by an individual or Legal Entity authorized to submit on behalf + of the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + + 2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + + 3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated in this section) patent license to make, have made, + use, offer to sell, sell, import, and otherwise transfer the Work, + where such license applies only to those patent claims licensable + by such Contributor that are necessarily infringed by their + Contribution(s) alone or by combination of their Contribution(s) + with the Work to which such Contribution(s) was submitted. If You + institute patent litigation against any entity (including a + cross-claim or counterclaim in a lawsuit) alleging that the Work + or a Contribution incorporated within the Work constitutes direct + or contributory patent infringement, then any patent licenses + granted to You under this License for that Work shall terminate + as of the date such litigation is filed. + + 4. Redistribution. You may reproduce and distribute copies of the + Work or Derivative Works thereof in any medium, with or without + modifications, and in Source or Object form, provided that You + meet the following conditions: + + (a) You must give any other recipients of the Work or + Derivative Works a copy of this License; and + + (b) You must cause any modified files to carry prominent notices + stating that You changed the files; and + + (c) You must retain, in the Source form of any Derivative Works + that You distribute, all copyright, patent, trademark, and + attribution notices from the Source form of the Work, + excluding those notices that do not pertain to any part of + the Derivative Works; and + + (d) If the Work includes a "NOTICE" text file as part of its + distribution, then any Derivative Works that You distribute must + include a readable copy of the attribution notices contained + within such NOTICE file, excluding those notices that do not + pertain to any part of the Derivative Works, in at least one + of the following places: within a NOTICE text file distributed + as part of the Derivative Works; within the Source form or + documentation, if provided along with the Derivative Works; or, + within a display generated by the Derivative Works, if and + wherever such third-party notices normally appear. The contents + of the NOTICE file are for informational purposes only and + do not modify the License. You may add Your own attribution + notices within Derivative Works that You distribute, alongside + or as an addendum to the NOTICE text from the Work, provided + that such additional attribution notices cannot be construed + as modifying the License. + + You may add Your own copyright statement to Your modifications and + may provide additional or different license terms and conditions + for use, reproduction, or distribution of Your modifications, or + for any such Derivative Works as a whole, provided Your use, + reproduction, and distribution of the Work otherwise complies with + the conditions stated in this License. + + 5. Submission of Contributions. Unless You explicitly state otherwise, + any Contribution intentionally submitted for inclusion in the Work + by You to the Licensor shall be under the terms and conditions of + this License, without any additional terms or conditions. + Notwithstanding the above, nothing herein shall supersede or modify + the terms of any separate license agreement you may have executed + with Licensor regarding such Contributions. + + 6. Trademarks. This License does not grant permission to use the trade + names, trademarks, service marks, or product names of the Licensor, + except as required for reasonable and customary use in describing the + origin of the Work and reproducing the content of the NOTICE file. + + 7. Disclaimer of Warranty. Unless required by applicable law or + agreed to in writing, Licensor provides the Work (and each + Contributor provides its Contributions) on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied, including, without limitation, any warranties or conditions + of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A + PARTICULAR PURPOSE. You are solely responsible for determining the + appropriateness of using or redistributing the Work and assume any + risks associated with Your exercise of permissions under this License. + + 8. Limitation of Liability. In no event and under no legal theory, + whether in tort (including negligence), contract, or otherwise, + unless required by applicable law (such as deliberate and grossly + negligent acts) or agreed to in writing, shall any Contributor be + liable to You for damages, including any direct, indirect, special, + incidental, or consequential damages of any character arising as a + result of this License or out of the use or inability to use the + Work (including but not limited to damages for loss of goodwill, + work stoppage, computer failure or malfunction, or any and all + other commercial damages or losses), even if such Contributor + has been advised of the possibility of such damages. + + 9. Accepting Warranty or Additional Liability. While redistributing + the Work or Derivative Works thereof, You may choose to offer, + and charge a fee for, acceptance of support, warranty, indemnity, + or other liability obligations and/or rights consistent with this + License. However, in accepting such obligations, You may act only + on Your own behalf and on Your sole responsibility, not on behalf + of any other Contributor, and only if You agree to indemnify, + defend, and hold each Contributor harmless for any liability + incurred by, or claims asserted against, such Contributor by reason + of your accepting any such warranty or additional liability. + + END OF TERMS AND CONDITIONS + +--- + +MIT License + +Copyright (c) 2024-2025 Model Context Protocol a Series of LF Projects, LLC. + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. + +--- + +Creative Commons Attribution 4.0 International (CC-BY-4.0) + +Documentation in this project (excluding specifications) is licensed under +CC-BY-4.0. See https://creativecommons.org/licenses/by/4.0/legalcode for +the full license text. diff --git a/README.md b/README.md index 29b74bd93..06ae527f9 100644 --- a/README.md +++ b/README.md @@ -14,6 +14,9 @@ npx @modelcontextprotocol/inspector --cli # CLI npx @modelcontextprotocol/inspector --tui # TUI ``` +> [!WARNING] +> **On a machine with no OS keychain, secrets are saved to a plaintext file by default.** That covers Linux without libsecret or a Secret Service, headless and SSH sessions, Termux, and containers with a mounted volume. OAuth client secrets and stdio `env:` values then go to `~/.mcp-inspector/secrets.json`, unencrypted unless you supply a key. See [Where secrets are stored](./docs/secret-storage.md) for how to get a keychain back, encrypt the file, or keep secrets in memory only. + > **Upgrading from v1?** Read the [v1 → v2 migration guide](./docs/v1-to-v2-migration.md) — CLI flags, the new `--config` vs. `--catalog` split, the Node engine bump, and what no longer ships. > **Repo status.** This is the **v2** line of the Inspector. Active development happens on **`v2/main`** (the develop branch — all v2 PRs target it), which is merged into **`main`** at milestone releases; `main` is the default branch and holds the latest released v2, published to the npm `latest` tag. The legacy **v1** line lives on **`v1/main`** — security fixes only, published straight from that branch to the npm `v1-latest` tag (`npx @modelcontextprotocol/inspector@v1-latest`). See [`AGENTS.md`](./AGENTS.md) for branch/board conventions. @@ -74,13 +77,15 @@ Each client has its own README with client-specific detail: | [Writing a skill](./docs/skill-authoring.md) | How to write a skill description that actually fires, and eval cases that measure it — the case shapes that work, and the tuning loop | | [Test servers](./docs/test-servers.md) | The composable test servers and the showcase config for every feature — what to run, what to click, and what the broken build did | | [Publishing](./docs/publishing.md) | What ships in the tarball, the packaging invariants, and `pack:verify` | -| [Docker](./docs/docker.md) | Running the container image — ports, volumes, and where secrets go | +| [Docker](./docs/docker.md) | Running the container image — ports, volumes, and making secrets durable in a container | +| [Where secrets are stored](./docs/secret-storage.md) | How the secret store is chosen on every runtime — OS keychain, `secrets.json` or memory — plus file encryption, locking, and moving back to a keychain | | [Migrating from v1 to v2](./docs/v1-to-v2-migration.md) | CLI flag mapping, `--config` vs. `--catalog`, the Node engine bump, env-var renames | | [Environment variables](./docs/environment-variables.md) | Every variable that changes runtime behavior — auth, ports, storage, the secret store, logging, proxies — plus the Node TLS variables for a self-signed server | | [MCP server configuration](./docs/mcp-server-configuration.md) | Which server(s) the Inspector connects to, and the config file format | | [Reviewing an MCP App](./docs/mcp-app-review.md) | The CLI-first → one-shot-web recipe for automated App-tool review | | [Smoke-testing an MCP server](./docs/cli-smoke-testing.md) | The connect → list → call → assert workflow for a shell or CI job: `--format json` + `jq`, the exit-code map, and keeping OAuth non-interactive | | [Launcher and config consolidation](./docs/launcher-config-consolidation-plan.md) | Why the launcher runs a client in-process rather than spawning it | +| [Roadmap, Aug 2026 → Feb 2027](./docs/inspector-roadmap-2026-h2.md) | The six-month plan: spec-following work aligned to the published MCP roadmap, official extension support, and the experience work we choose | ## Testing and the quality gate @@ -106,4 +111,4 @@ A key rule worth surfacing here: **all work is issue-driven.** Before starting, ## License -MIT. +See [`LICENSE`](./LICENSE). The MCP project is transitioning from the MIT License to Apache-2.0: new code contributions are licensed under Apache-2.0, documentation (excluding specifications) under CC-BY-4.0, and contributions whose authors originally licensed them under MIT and have not granted relicensing consent remain under MIT. The file carries the full Apache-2.0 and MIT texts and links the CC-BY-4.0 legal code. diff --git a/clients/cli/package-lock.json b/clients/cli/package-lock.json index a8e46a61e..9d514f281 100644 --- a/clients/cli/package-lock.json +++ b/clients/cli/package-lock.json @@ -6,7 +6,7 @@ "packages": { "": { "name": "@modelcontextprotocol/inspector-cli", - "license": "MIT", + "license": "SEE LICENSE IN ../../LICENSE", "bin": { "mcp-inspector-cli": "build/index.js" }, diff --git a/clients/cli/package.json b/clients/cli/package.json index 5bcf04cd9..79ef02d32 100644 --- a/clients/cli/package.json +++ b/clients/cli/package.json @@ -2,7 +2,7 @@ "name": "@modelcontextprotocol/inspector-cli", "private": true, "description": "CLI for the Model Context Protocol inspector", - "license": "MIT", + "license": "SEE LICENSE IN ../../LICENSE", "type": "module", "main": "build/index.js", "exports": { diff --git a/clients/launcher/package-lock.json b/clients/launcher/package-lock.json index 555e8caf2..cf9841527 100644 --- a/clients/launcher/package-lock.json +++ b/clients/launcher/package-lock.json @@ -6,7 +6,7 @@ "packages": { "": { "name": "@modelcontextprotocol/inspector-launcher", - "license": "MIT", + "license": "SEE LICENSE IN ../../LICENSE", "bin": { "mcp-inspector": "build/index.js" } diff --git a/clients/launcher/package.json b/clients/launcher/package.json index 4ea7560f2..9b5781b58 100644 --- a/clients/launcher/package.json +++ b/clients/launcher/package.json @@ -2,7 +2,7 @@ "name": "@modelcontextprotocol/inspector-launcher", "private": true, "description": "Launcher for MCP Inspector (web, CLI, TUI)", - "license": "MIT", + "license": "SEE LICENSE IN ../../LICENSE", "type": "module", "main": "build/index.js", "bin": { diff --git a/clients/tui/__tests__/ToolTestModal.test.tsx b/clients/tui/__tests__/ToolTestModal.test.tsx index bf9af6883..1bf58296e 100644 --- a/clients/tui/__tests__/ToolTestModal.test.tsx +++ b/clients/tui/__tests__/ToolTestModal.test.tsx @@ -121,6 +121,52 @@ describe("ToolTestModal", () => { api.unmount(); }); + it("resolves a $ref'd union branch before checking its required arguments (#2321)", async () => { + // Unresolved, a `$ref` branch's requirements read as unknown and the call + // would go out missing `address`; inlined, the branch is checked as written. + const callTool = vi.fn(); + const tool = makeTool({ + inputSchema: { + type: "object", + oneOf: [{ $ref: "#/$defs/Email" }, { $ref: "#/$defs/Sms" }], + $defs: { + Email: { + type: "object", + properties: { + kind: { type: "string", const: "email" }, + address: { type: "string" }, + }, + required: ["kind", "address"], + }, + Sms: { + type: "object", + properties: { + kind: { type: "string", const: "sms" }, + phone: { type: "string" }, + }, + required: ["kind", "phone"], + }, + }, + }, + }); + const api = render( + , + ); + await tick(); + setSubmitValue({ __variant: "0", __b0__kind: "email" }); + api.stdin.write("\r"); + await tick(); + await tick(); + expect(callTool).not.toHaveBeenCalled(); + api.unmount(); + }); + it("names every missing required argument (#2123)", async () => { const callTool = vi.fn(); const tool = makeTool({ diff --git a/clients/tui/package-lock.json b/clients/tui/package-lock.json index d4907b518..6f92f7691 100644 --- a/clients/tui/package-lock.json +++ b/clients/tui/package-lock.json @@ -6,7 +6,7 @@ "packages": { "": { "name": "@modelcontextprotocol/inspector-tui", - "license": "MIT", + "license": "SEE LICENSE IN ../../LICENSE", "dependencies": { "ink": "^6.0.0", "ink-form": "^2.0.1", diff --git a/clients/tui/package.json b/clients/tui/package.json index 42f8571e0..81c6ddb92 100644 --- a/clients/tui/package.json +++ b/clients/tui/package.json @@ -2,7 +2,7 @@ "name": "@modelcontextprotocol/inspector-tui", "private": true, "description": "Terminal User Interface (TUI) for the Model Context Protocol inspector", - "license": "MIT", + "license": "SEE LICENSE IN ../../LICENSE", "type": "module", "main": "build/index.js", "exports": { diff --git a/clients/tui/src/components/ToolTestModal.tsx b/clients/tui/src/components/ToolTestModal.tsx index 1b0b726af..4e7f693df 100644 --- a/clients/tui/src/components/ToolTestModal.tsx +++ b/clients/tui/src/components/ToolTestModal.tsx @@ -11,6 +11,7 @@ import { schemaToForm, } from "../utils/schemaToForm.js"; import { ScrollView, type ScrollViewRef } from "ink-scroll-view"; +import { inlineLocalRefs } from "@inspector/core/json/localRefs.js"; interface ToolTestModalProps { tool: Tool; @@ -63,8 +64,13 @@ export function ToolTestModal({ }; }, [width, height]); - const formStructure = tool?.inputSchema - ? schemaToForm(tool.inputSchema, tool.name || "Unknown Tool") + // Same-document `$ref`s inlined once, for the form and the decode alike: a + // property declared as a bare `$ref` has no `type` to build a field from, so + // a deduplicated Zod schema would otherwise lose its string input (#2321). + const inputSchema = inlineLocalRefs(tool?.inputSchema); + + const formStructure = inputSchema + ? schemaToForm(inputSchema, tool?.name || "Unknown Tool") : { title: `Test Tool: ${tool?.name || "Unknown"}`, sections: [{ title: "Parameters", fields: [] }], @@ -125,14 +131,14 @@ export function ToolTestModal({ // field names, because ink-form scopes values by name across the whole form // (#2123). This turns them back into the arguments the server declared: // the base fields plus the chosen branch's, and nothing from the others. - const values = decodeFormValues(tool.inputSchema, rawValues); + const values = decodeFormValues(inputSchema, rawValues); // A branch's fields are rendered optional — only one alternative applies to // a call, and requiring every branch's would deadlock a static form — so // the chosen shape's own requirements are checked here instead. Reported // rather than sent: a call known to violate the schema teaches the user // nothing about the server (#2123). - const missing = missingRequiredFields(tool.inputSchema, values, rawValues); + const missing = missingRequiredFields(inputSchema, values, rawValues); if (missing.length > 0) { setResult({ input: values, diff --git a/clients/web/package-lock.json b/clients/web/package-lock.json index c666dc79a..9f7affb97 100644 --- a/clients/web/package-lock.json +++ b/clients/web/package-lock.json @@ -5,6 +5,7 @@ "packages": { "": { "name": "@modelcontextprotocol/inspector-web", + "license": "SEE LICENSE IN ../../LICENSE", "dependencies": { "@dnd-kit/core": "^6.3.1", "@dnd-kit/sortable": "^8.0.0", diff --git a/clients/web/package.json b/clients/web/package.json index 4c8cb7957..1c84c3250 100644 --- a/clients/web/package.json +++ b/clients/web/package.json @@ -1,6 +1,7 @@ { "name": "@modelcontextprotocol/inspector-web", "private": true, + "license": "SEE LICENSE IN ../../LICENSE", "type": "module", "main": "build/index.js", "bin": { diff --git a/clients/web/server/resolve-bind-host.ts b/clients/web/server/resolve-bind-host.ts index f18945ce0..c978392e0 100644 --- a/clients/web/server/resolve-bind-host.ts +++ b/clients/web/server/resolve-bind-host.ts @@ -18,10 +18,13 @@ export const BIND_ALL_INTERFACES_ENV = "DANGEROUSLY_BIND_ALL_INTERFACES"; /** * An explicit, unambiguous opt-in. Unlike a bare `!!value` (which treats the - * string `"false"` as truthy), only `"true"`/`"1"` (case-insensitive) enable - * the override, so `DANGEROUSLY_BIND_ALL_INTERFACES=false` reads as "off". + * string `"false"` as truthy), only `"true"`/`"1"` (trimmed, case-insensitive) + * enable the override, so `DANGEROUSLY_BIND_ALL_INTERFACES=false` reads as + * "off". Exported so every `DANGEROUSLY_*` safety flag parses the same way — + * `DANGEROUSLY_OMIT_AUTH` once used `!!value`, and `=false` turned auth off + * (#2331). Anything unrecognized fails closed. */ -function isEnabled(value: string | undefined): boolean { +export function isEnvFlagEnabled(value: string | undefined): boolean { const v = value?.trim().toLowerCase(); return v === "true" || v === "1"; } @@ -66,7 +69,10 @@ export function resolveBindHostname( env: NodeJS.ProcessEnv = process.env, ): string { const host = (env.HOST ?? DEFAULT_BIND_HOST).trim(); - if (isAllInterfacesHost(host) && !isEnabled(env[BIND_ALL_INTERFACES_ENV])) { + if ( + isAllInterfacesHost(host) && + !isEnvFlagEnabled(env[BIND_ALL_INTERFACES_ENV]) + ) { // Show the resolved address when it differs from the typed spelling — the // guard now catches forms the resolver folds to the wildcard (a fullwidth // `HOST="０"` renders like `0`, `HOST=0` / `0x0` / `::0` bind `0.0.0.0`), and diff --git a/clients/web/server/web-server-config.ts b/clients/web/server/web-server-config.ts index 1b9c45397..55283953e 100644 --- a/clients/web/server/web-server-config.ts +++ b/clients/web/server/web-server-config.ts @@ -20,7 +20,7 @@ import { secretStorageSummary } from "../../../core/auth/secret-storage-info.ts" import { readInspectorVersionSafe } from "../../../core/node/version.ts"; import { resolveSandboxPort } from "./sandbox-controller.js"; import { resolveAppOriginPort } from "./app-origin-controller.js"; -import { resolveBindHostname } from "./resolve-bind-host.js"; +import { isEnvFlagEnabled, resolveBindHostname } from "./resolve-bind-host.js"; import { APP_ORIGIN_FULL_ADDRESS_ENV, resolveAppOriginPublicOrigin, @@ -386,7 +386,11 @@ export function buildWebServerConfig( ); } const hostname = resolveBindHostname(); - const dangerouslyOmitAuth = !!process.env.DANGEROUSLY_OMIT_AUTH; + // Only an explicit `true`/`1` omits auth — `!!value` read `=false` as "on" + // and silently disabled the /api/* bearer gate (#2331). + const dangerouslyOmitAuth = isEnvFlagEnabled( + process.env.DANGEROUSLY_OMIT_AUTH, + ); const authToken = dangerouslyOmitAuth ? "" : ((process.env[API_SERVER_ENV_VARS.AUTH_TOKEN] as string | undefined) ?? diff --git a/clients/web/src/components/elements/SecretStorageFooter/SecretStorageFooter.test.tsx b/clients/web/src/components/elements/SecretStorageFooter/SecretStorageFooter.test.tsx index 84717d96d..b171b918c 100644 --- a/clients/web/src/components/elements/SecretStorageFooter/SecretStorageFooter.test.tsx +++ b/clients/web/src/components/elements/SecretStorageFooter/SecretStorageFooter.test.tsx @@ -109,6 +109,19 @@ describe("SecretStorageFooter", () => { expect(band).toHaveAttribute("data-tone", "warn"); }); + it("offers both key variables in the plaintext tooltip", async () => { + // Either variable clears the condition, and the file form is the one a + // container should use (#2447), so the advice names both. + const user = userEvent.setup(); + renderWithMantine(); + await user.hover(screen.getByRole("button", { name: /Copy secrets file/ })); + expect( + await screen.findByText( + /Set MCP_INSPECTOR_SECRET_KEY or MCP_INSPECTOR_SECRET_KEY_FILE to encrypt\./, + ), + ).toBeInTheDocument(); + }); + it("does not claim exposure in the tooltip when encryption is unknown", async () => { // Mirrors `secretStorageCaveat`: `plaintext` is absent alongside // `encryptionUnknown`, and a two-way `!== false` test would assert that diff --git a/clients/web/src/components/elements/SecretStorageFooter/SecretStorageFooter.tsx b/clients/web/src/components/elements/SecretStorageFooter/SecretStorageFooter.tsx index 979555574..751772e7d 100644 --- a/clients/web/src/components/elements/SecretStorageFooter/SecretStorageFooter.tsx +++ b/clients/web/src/components/elements/SecretStorageFooter/SecretStorageFooter.tsx @@ -189,7 +189,7 @@ function footerTooltip(info: SecretStorageInfo): string | undefined { parts.push( info.pendingEncryption ? "Re-encrypted the next time a secret is saved." - : "Set MCP_INSPECTOR_SECRET_KEY to encrypt.", + : "Set MCP_INSPECTOR_SECRET_KEY or MCP_INSPECTOR_SECRET_KEY_FILE to encrypt.", ); } if (info.looseMode !== undefined) { diff --git a/clients/web/src/components/groups/SchemaForm/SchemaForm.test.tsx b/clients/web/src/components/groups/SchemaForm/SchemaForm.test.tsx index e8b23b6dd..45df1ddc6 100644 --- a/clients/web/src/components/groups/SchemaForm/SchemaForm.test.tsx +++ b/clients/web/src/components/groups/SchemaForm/SchemaForm.test.tsx @@ -1067,6 +1067,34 @@ describe("SchemaForm nullable unions", () => { expect(onChange).toHaveBeenCalledWith({ direction: "envio" }); }); + it("renders a string input for a property that is a bare $ref to a string (#2321)", async () => { + const user = userEvent.setup(); + const onChange = vi.fn(); + // What a Zod → JSON Schema converter emits for one `z.string().regex(…)` + // instance used by two fields: the second use is only a pointer, with no + // `type` of its own. Through `toFormSchema`, as the Tools panel does. + const schema = toFormSchema({ + type: "object", + properties: { + dateRangeBegin: { type: "string", pattern: "^\\d{4}-\\d{2}-\\d{2}$" }, + dateRangeEnd: { + $ref: "#/$defs/DateString", + description: "end date, yyyy-MM-dd", + }, + }, + $defs: { + DateString: { type: "string", pattern: "^\\d{4}-\\d{2}-\\d{2}$" }, + }, + }); + renderWithMantine( + , + ); + await user.type(screen.getByRole("textbox", { name: "dateRangeEnd" }), "2"); + expect(onChange).toHaveBeenCalledWith({ dateRangeEnd: "2" }); + expect(screen.getByText("end date, yyyy-MM-dd")).toBeInTheDocument(); + expect(screen.queryByText(/Not valid JSON/)).not.toBeInTheDocument(); + }); + it("renders a TextInput for a type: [string, null] field", async () => { const user = userEvent.setup(); const onChange = vi.fn(); diff --git a/clients/web/src/components/groups/ServerSettingsForm/ServerSettingsForm.stories.tsx b/clients/web/src/components/groups/ServerSettingsForm/ServerSettingsForm.stories.tsx index 245feb668..b79cae575 100644 --- a/clients/web/src/components/groups/ServerSettingsForm/ServerSettingsForm.stories.tsx +++ b/clients/web/src/components/groups/ServerSettingsForm/ServerSettingsForm.stories.tsx @@ -112,6 +112,12 @@ function InteractiveRender(args: ServerSettingsFormProps) { settings: { ...args.settings, paginatedLists: value }, }); }} + onSuppressNotificationStreamChange={(value) => { + args.onSuppressNotificationStreamChange(value); + updateArgs({ + settings: { ...args.settings, suppressNotificationStream: value }, + }); + }} onAdvertisedExtensionChange={(key, checked) => { args.onAdvertisedExtensionChange(key, checked); updateArgs({ @@ -182,6 +188,7 @@ const meta: Meta = { onTimeoutChange: fn(), onAutoRefreshChange: fn(), onPaginatedListsChange: fn(), + onSuppressNotificationStreamChange: fn(), onAdvertisedExtensionChange: fn(), onMaxFetchRequestsChange: fn(), onSkillCatalogLimitChange: fn(), diff --git a/clients/web/src/components/groups/ServerSettingsForm/ServerSettingsForm.test.tsx b/clients/web/src/components/groups/ServerSettingsForm/ServerSettingsForm.test.tsx index 26488227b..d57fa3657 100644 --- a/clients/web/src/components/groups/ServerSettingsForm/ServerSettingsForm.test.tsx +++ b/clients/web/src/components/groups/ServerSettingsForm/ServerSettingsForm.test.tsx @@ -68,6 +68,7 @@ const baseHandlers = { onTimeoutChange: vi.fn(), onAutoRefreshChange: vi.fn(), onPaginatedListsChange: vi.fn(), + onSuppressNotificationStreamChange: vi.fn(), onAdvertisedExtensionChange: vi.fn(), onMaxFetchRequestsChange: vi.fn(), onSkillCatalogLimitChange: vi.fn(), @@ -452,6 +453,86 @@ describe("ServerSettingsForm", () => { expect(onAutoRefreshChange).toHaveBeenCalledWith(true); }); + describe("Suppress Notification Stream (#2317)", () => { + const name = /Suppress Notification Stream/; + + it("is unchecked by default and reflects an explicit true", () => { + const { rerender } = renderWithMantine( + , + ); + expect(screen.getByRole("checkbox", { name })).not.toBeChecked(); + rerender( + , + ); + expect(screen.getByRole("checkbox", { name })).toBeChecked(); + }); + + it("invokes onSuppressNotificationStreamChange when toggled", async () => { + const user = userEvent.setup(); + const onSuppressNotificationStreamChange = vi.fn(); + renderWithMantine( + , + ); + await user.click(screen.getByRole("checkbox", { name })); + expect(onSuppressNotificationStreamChange).toHaveBeenCalledWith(true); + }); + + it("is hidden for a server pinned to the modern era, which never opens the stream", () => { + renderWithMantine( + , + ); + expect(screen.queryByRole("checkbox", { name })).not.toBeInTheDocument(); + }); + + it("stays visible for an auto-era server, which may resolve to legacy", () => { + renderWithMantine( + , + ); + expect(screen.getByRole("checkbox", { name })).toBeInTheDocument(); + }); + + it.each(["sse", "stdio"] as const)( + "is hidden for a %s server, which has no standalone GET stream", + (serverType) => { + renderWithMantine( + , + ); + expect( + screen.queryByRole("checkbox", { name }), + ).not.toBeInTheDocument(); + }, + ); + }); + it("renders the Advertised Extensions section with Tasks checked by default", () => { renderWithMantine( void; onAutoRefreshChange: (value: boolean) => void; onPaginatedListsChange: (value: boolean) => void; + /** Toggle the standalone `GET` notification stream suppression (#2317). */ + onSuppressNotificationStreamChange: (value: boolean) => void; /** * Toggle whether the Inspector advertises the extension `key` to this server. * `checked` is the new advertise state; the modal folds it into @@ -479,6 +481,7 @@ export function ServerSettingsForm({ onTimeoutChange, onAutoRefreshChange, onPaginatedListsChange, + onSuppressNotificationStreamChange, onAdvertisedExtensionChange, onMaxFetchRequestsChange, onSkillCatalogLimitChange, @@ -513,6 +516,11 @@ export function ServerSettingsForm({ // `auto` server that is either not yet connected or resolved to modern, keep // it visible. const configuredEra = settings.protocolEra ?? DEFAULT_PROTOCOL_ERA; + // The standalone GET stream exists only on a legacy-era Streamable HTTP + // connection; a pinned-modern server never opens it, so the box would be a + // no-op there (#2317). `auto` keeps it, since it may resolve to legacy. + const showSuppressNotificationStream = + serverType === "streamable-http" && configuredEra !== "modern"; const showModernLogLevel = configuredEra === "modern" || (configuredEra === "auto" && negotiatedEra !== "legacy"); @@ -708,6 +716,16 @@ export function ServerSettingsForm({ checked={settings.paginatedLists ?? false} onChange={(e) => onPaginatedListsChange(e.currentTarget.checked)} /> + {showSuppressNotificationStream ? ( + + onSuppressNotificationStreamChange(e.currentTarget.checked) + } + /> + ) : null} { ); }); + // #2317 — omit-when-off, so an untouched server writes no field. + it("maps the notification-stream suppression into settings, and back to unset", async () => { + const user = userEvent.setup(); + const onSettingsChange = vi.fn(); + const { rerender } = renderWithMantine( + , + ); + const name = /Suppress Notification Stream/; + await user.click(screen.getByRole("checkbox", { name })); + expect(onSettingsChange).toHaveBeenLastCalledWith( + expect.objectContaining({ suppressNotificationStream: true }), + ); + + rerender( + , + ); + await user.click(screen.getByRole("checkbox", { name })); + expect(onSettingsChange).toHaveBeenLastCalledWith( + expect.objectContaining({ suppressNotificationStream: undefined }), + ); + }); + // #2144 — same omit-the-default shape as the refresh-token pair above. it("maps the revoke-on-clear opt-out into settings, and back to unset", async () => { const user = userEvent.setup(); diff --git a/clients/web/src/components/groups/ServerSettingsModal/ServerSettingsModal.tsx b/clients/web/src/components/groups/ServerSettingsModal/ServerSettingsModal.tsx index db2b63a43..1f57b9ecc 100644 --- a/clients/web/src/components/groups/ServerSettingsModal/ServerSettingsModal.tsx +++ b/clients/web/src/components/groups/ServerSettingsModal/ServerSettingsModal.tsx @@ -206,6 +206,14 @@ export function ServerSettingsModal({ onSettingsChange({ ...settings, paginatedLists: value }); } + function handleSuppressNotificationStreamChange(value: boolean) { + // Omit-when-off, so a server that never touched the box writes no field. + onSettingsChange({ + ...settings, + suppressNotificationStream: value ? true : undefined, + }); + } + function handleAdvertisedExtensionChange(key: string, checked: boolean) { const next = { ...settings.advertisedExtensions }; const ext = ADVERTISABLE_EXTENSIONS.find((e) => e.key === key); @@ -306,6 +314,9 @@ export function ServerSettingsModal({ onTimeoutChange={handleTimeoutChange} onAutoRefreshChange={handleAutoRefreshChange} onPaginatedListsChange={handlePaginatedListsChange} + onSuppressNotificationStreamChange={ + handleSuppressNotificationStreamChange + } onAdvertisedExtensionChange={handleAdvertisedExtensionChange} onMaxFetchRequestsChange={handleMaxFetchRequestsChange} onSkillCatalogLimitChange={handleSkillCatalogLimitChange} diff --git a/clients/web/src/test/core/auth/oidcDiscoveryCompat.test.ts b/clients/web/src/test/core/auth/oidcDiscoveryCompat.test.ts index c9ea94b98..5cc8d2011 100644 --- a/clients/web/src/test/core/auth/oidcDiscoveryCompat.test.ts +++ b/clients/web/src/test/core/auth/oidcDiscoveryCompat.test.ts @@ -608,6 +608,77 @@ describe("probe cancellation (#2319)", () => { ).rejects.toBe(reason); }); + /** + * A response whose body `cancel()` never settles — the shape `ReadableStream` + * permits — running `onCancel` when the release starts. + */ + function stallingCancelResponse( + status: number, + text: string, + onCancel: () => void = () => {}, + ): Response { + return new Response( + new ReadableStream({ + start(ctrl) { + ctrl.enqueue(new TextEncoder().encode(text)); + ctrl.close(); + }, + cancel() { + onCancel(); + return new Promise(() => {}); + }, + }), + { status }, + ); + } + + it("rejects with the caller's reason when it aborts during a non-OK probe's release (#2389)", async () => { + const caller = new AbortController(); + const reason = new Error("gave up during release"); + const inner = vi.fn((input) => + Promise.resolve( + String(input) === RFC8414 + ? new Response(null, { status: 404 }) + : // The abort lands while the probe's body is being discarded. + stallingCancelResponse(404, "nope", () => caller.abort(reason)), + ), + ); + const wrapped = withRfc8414OidcCompat(inner); + + // Would hang if the release were awaited: the recheck sits after it. + await expect(wrapped(RFC8414, { signal: caller.signal })).rejects.toBe( + reason, + ); + // Stopped at the first candidate rather than probing on. + expect(inner).toHaveBeenCalledTimes(2); + }); + + it("walks past a non-OK probe whose release never settles (#2389)", async () => { + vi.spyOn(console, "warn").mockImplementation(() => {}); + const cancelled: string[] = []; + const inner = vi.fn((input) => { + const url = String(input); + if (url === OIDC_APPENDED) return Promise.resolve(json(RFC8414_DOC)); + if (url === OIDC_SUFFIXED) { + return Promise.resolve( + stallingCancelResponse(404, "nope", () => cancelled.push("probe")), + ); + } + return Promise.resolve( + stallingCancelResponse(404, "nope", () => cancelled.push("original")), + ); + }); + const wrapped = withRfc8414OidcCompat(inner); + + // Reaching the second candidate and substituting it is only possible if + // neither the probe's release nor the original's was waited on. + const response = await wrapped(RFC8414); + await expect(response.json()).resolves.toEqual(RFC8414_DOC); + expect(inner).toHaveBeenCalledTimes(3); + // Still released, just not waited on. + expect(cancelled.sort()).toEqual(["original", "probe"]); + }); + it("still falls back to the original response on an ordinary probe failure", async () => { const inner = vi.fn((input) => { if (String(input) === RFC8414) { diff --git a/clients/web/src/test/core/auth/secret-storage-info.test.ts b/clients/web/src/test/core/auth/secret-storage-info.test.ts index 481f9e0d5..b9cbae2cc 100644 --- a/clients/web/src/test/core/auth/secret-storage-info.test.ts +++ b/clients/web/src/test/core/auth/secret-storage-info.test.ts @@ -82,7 +82,9 @@ describe("secretStorageCaveat", () => { it("names the fix for an unencrypted file, not just the problem", () => { const caveat = secretStorageCaveat(plaintextFile); expect(caveat).toContain("unencrypted"); - expect(caveat).toContain("MCP_INSPECTOR_SECRET_KEY"); + expect(caveat).toContain( + "Set MCP_INSPECTOR_SECRET_KEY or MCP_INSPECTOR_SECRET_KEY_FILE to encrypt them.", + ); }); it("changes the advice once a passphrase is set but not yet applied", () => { diff --git a/clients/web/src/test/core/jsonUtils.test.ts b/clients/web/src/test/core/jsonUtils.test.ts index 12f4877b9..d5adcdc20 100644 --- a/clients/web/src/test/core/jsonUtils.test.ts +++ b/clients/web/src/test/core/jsonUtils.test.ts @@ -193,6 +193,23 @@ describe("JSON Utils", () => { }, }; + it("coerces a value whose property is a bare $ref (#2321)", () => { + const refTool: Tool = { + name: "ref-tool", + inputSchema: { + type: "object", + properties: { + first: { type: "integer" }, + second: { $ref: "#/$defs/Count" }, + }, + $defs: { Count: { type: "integer" } }, + }, + }; + expect( + convertToolParameters(refTool, { first: "1", second: "2" }), + ).toEqual({ first: 1, second: 2 }); + }); + it("coerces a value whose schema lives on a root union branch (#2123)", () => { const unionTool: Tool = { name: "union-tool", diff --git a/clients/web/src/test/core/localRefs.test.ts b/clients/web/src/test/core/localRefs.test.ts new file mode 100644 index 000000000..39b1841fa --- /dev/null +++ b/clients/web/src/test/core/localRefs.test.ts @@ -0,0 +1,298 @@ +import { describe, it, expect } from "vitest"; +import { + EXPANSION_BUDGET, + MAX_DEPTH, + inlineLocalRefs, +} from "@inspector/core/json/localRefs.js"; + +// Zod → JSON Schema converters deduplicate a reused schema instance into +// `$defs` and point each use at it with a bare `$ref`, which has no `type` for +// a form builder to dispatch on (#2321). +describe("inlineLocalRefs", () => { + const date = { type: "string", pattern: "^\\d{4}-\\d{2}-\\d{2}$" }; + + it("inlines a $defs reference, letting the use site's siblings win", () => { + const schema = { + type: "object", + properties: { + end: { $ref: "#/$defs/Date", description: "end date" }, + }, + $defs: { Date: { ...date, description: "a date" } }, + }; + const resolved = inlineLocalRefs(schema); + expect(resolved.properties.end).toEqual({ + ...date, + description: "end date", + }); + // The input is never mutated. + expect(schema.properties.end).toEqual({ + $ref: "#/$defs/Date", + description: "end date", + }); + }); + + it("inlines a definitions reference nested in anyOf and items", () => { + const resolved = inlineLocalRefs({ + type: "object", + properties: { + maybe: { anyOf: [{ $ref: "#/definitions/D" }, { type: "null" }] }, + list: { type: "array", items: { $ref: "#/definitions/D" } }, + }, + definitions: { D: date }, + }); + expect(resolved.properties.maybe.anyOf[0]).toEqual(date); + expect(resolved.properties.list.items).toEqual(date); + }); + + it("resolves chained references", () => { + const resolved = inlineLocalRefs({ + properties: { a: { $ref: "#/$defs/A" } }, + $defs: { A: { $ref: "#/$defs/B" }, B: date }, + }); + expect(resolved.properties.a).toEqual(date); + }); + + it("returns the same reference when there is nothing to inline", () => { + const schema = { type: "object", properties: { a: date } }; + expect(inlineLocalRefs(schema)).toBe(schema); + expect(inlineLocalRefs(null)).toBeNull(); + expect(inlineLocalRefs(["x"])).toEqual(["x"]); + }); + + it("returns the same resolved object for repeated calls", () => { + const schema = { + properties: { a: { $ref: "#/$defs/A" } }, + $defs: { A: date }, + }; + expect(inlineLocalRefs(schema)).toBe(inlineLocalRefs(schema)); + }); + + it("stops at a recursive reference instead of looping", () => { + const resolved = inlineLocalRefs({ + properties: { root: { $ref: "#/$defs/Node" } }, + $defs: { + Node: { + type: "object", + properties: { child: { $ref: "#/$defs/Node" } }, + }, + }, + }); + expect(resolved.properties.root).toEqual({ + type: "object", + properties: { child: { $ref: "#/$defs/Node" } }, + }); + }); + + it("leaves remote, unresolvable and non-object references in place", () => { + const resolved = inlineLocalRefs({ + properties: { + remote: { $ref: "https://example.com/s.json" }, + notPointer: { $ref: "#Anchor" }, + missing: { $ref: "#/$defs/Nope" }, + badEscape: { $ref: "#/$defs/%E0%A4%A" }, + scalar: { $ref: "#/$defs/S/type" }, + pastEnd: { $ref: "#/$defs/L/5" }, + badIndex: { $ref: "#/$defs/L/01" }, + badEscape2: { $ref: "#/$defs/A~2B" }, + }, + $defs: { S: date, L: [date], "A~2B": date }, + }); + expect(resolved.properties).toEqual({ + remote: { $ref: "https://example.com/s.json" }, + notPointer: { $ref: "#Anchor" }, + missing: { $ref: "#/$defs/Nope" }, + badEscape: { $ref: "#/$defs/%E0%A4%A" }, + scalar: { $ref: "#/$defs/S/type" }, + pastEnd: { $ref: "#/$defs/L/5" }, + badIndex: { $ref: "#/$defs/L/01" }, + badEscape2: { $ref: "#/$defs/A~2B" }, + }); + }); + + it("follows array indices, escaped segments and the document root", () => { + const resolved = inlineLocalRefs({ + type: "object", + properties: { + indexed: { $ref: "#/$defs/L/0" }, + escaped: { $ref: "#/$defs/a~1b~0c%20d" }, + encodedSlashes: { $ref: "#%2F$defs%2FL%2F0" }, + encodedSeparator: { $ref: "#/$defs/N%2Finner" }, + self: { anyOf: [{ $ref: "#" }] }, + }, + $defs: { L: [date], "a/b~c d": date, N: { inner: date } }, + }); + expect(resolved.properties.indexed).toEqual(date); + expect(resolved.properties.escaped).toEqual(date); + expect(resolved.properties.encodedSlashes).toEqual(date); + expect(resolved.properties.encodedSeparator).toEqual(date); + // `#` is the schema being inlined, so it is kept rather than recursed. + expect(resolved.properties.self.anyOf[0]).toEqual({ + type: "object", + properties: expect.any(Object), + $defs: expect.any(Object), + }); + }); + + it("treats data keywords as data, but property NAMES as schemas", () => { + const pointer = { $ref: "#/$defs/D" }; + const resolved = inlineLocalRefs({ + type: "object", + default: pointer, + properties: { + default: pointer, + enum: pointer, + ["__proto__"]: pointer, + }, + $defs: { D: date }, + }); + expect(resolved.default).toBe(pointer); + expect(resolved.properties.default).toEqual(date); + expect(resolved.properties.enum).toEqual(date); + expect(Object.hasOwn(resolved.properties, "__proto__")).toBe(true); + }); + + it("walks the legacy dependencies map by name, passing name lists through", () => { + const resolved = inlineLocalRefs({ + type: "object", + dependencies: { + default: { properties: { x: { $ref: "#/$defs/D" } } }, + other: ["default"], + }, + $defs: { D: date }, + }); + expect(resolved.dependencies.default.properties.x).toEqual(date); + expect(resolved.dependencies.other).toEqual(["default"]); + }); + + it("finds a reference that sits only under a data-keyword-named property", () => { + const schema = { + properties: { const: { $ref: "#/$defs/D" } }, + $defs: { D: date }, + }; + expect(inlineLocalRefs(schema).properties.const).toEqual(date); + }); + it("merges annotation siblings but declines a $ref whose siblings constrain", () => { + const resolved = inlineLocalRefs({ + properties: { + annotated: { $ref: "#/$defs/E", title: "T", default: "a" }, + widened: { $ref: "#/$defs/E", enum: ["a", "b"] }, + }, + $defs: { E: { type: "string", enum: ["a"] } }, + }); + expect(resolved.properties.annotated).toEqual({ + type: "string", + enum: ["a"], + title: "T", + default: "a", + }); + // Conjunctive in JSON Schema, so a merge would admit "b"; left as written. + expect(resolved.properties.widened).toEqual({ + $ref: "#/$defs/E", + enum: ["a", "b"], + }); + }); + + it("inlines a root $ref beside its definitions", () => { + const resolved = inlineLocalRefs({ + $ref: "#/definitions/Args", + definitions: { Args: { type: "object", properties: { a: date } } }, + }); + expect(resolved).toMatchObject({ + type: "object", + properties: { a: date }, + }); + }); + + it("leaves everything under a nested $id unresolved", () => { + const embedded = { + $id: "https://example.com/inner", + properties: { x: { $ref: "#/$defs/D" } }, + $defs: { D: { type: "integer" } }, + }; + const resolved = inlineLocalRefs({ + $id: "https://example.com/outer", + properties: { outer: { $ref: "#/$defs/D" }, inner: embedded }, + $defs: { D: date }, + }); + expect(resolved.properties.outer).toEqual(date); + expect(resolved.properties.inner).toBe(embedded); + }); + + it("returns the schema unresolved when expansion would exceed the budget", () => { + // Each level uses the previous one twice: 2^20 nodes from 20 definitions. + const $defs: Record = { L0: { type: "string" } }; + for (let n = 1; n <= 20; n++) { + $defs[`L${n}`] = { + type: "object", + properties: { + a: { $ref: `#/$defs/L${n - 1}` }, + b: { $ref: `#/$defs/L${n - 1}` }, + }, + }; + } + const schema = { properties: { top: { $ref: "#/$defs/L20" } }, $defs }; + expect(2 ** 20).toBeGreaterThan(EXPANSION_BUDGET); + expect(inlineLocalRefs(schema)).toBe(schema); + }); + it("copies extension keywords as data even when they hold a $ref", () => { + const resolved = inlineLocalRefs({ + type: "object", + "x-vendor": { $ref: "#/$defs/D" }, + properties: { a: { $ref: "#/$defs/D" } }, + $defs: { D: date }, + }); + expect(resolved["x-vendor"]).toEqual({ $ref: "#/$defs/D" }); + expect(resolved.properties.a).toEqual(date); + }); + + it("walks every subschema keyword shape", () => { + const pointer = { $ref: "#/$defs/D" }; + const resolved = inlineLocalRefs({ + not: pointer, + items: [pointer], + prefixItems: [pointer], + patternProperties: { "^x": pointer }, + allOf: "not an array", + properties: "not a map", + $defs: { D: date }, + }); + expect(resolved).toMatchObject({ + not: date, + items: [date], + prefixItems: [date], + patternProperties: { "^x": date }, + allOf: "not an array", + properties: "not a map", + }); + }); + + it("keeps enumNames beside a $ref as an annotation", () => { + const resolved = inlineLocalRefs({ + properties: { c: { $ref: "#/$defs/C", enumNames: ["Red"] } }, + $defs: { C: { type: "string", enum: ["r"] } }, + }); + expect(resolved.properties.c).toEqual({ + type: "string", + enum: ["r"], + enumNames: ["Red"], + }); + }); + + it("returns the schema unresolved past MAX_DEPTH, in either pass", () => { + // Deeper than the stack allows, with the $ref at the bottom: the scan bails. + let deep: Record = { $ref: "#/$defs/D" }; + for (let n = 0; n < 6000; n++) deep = { items: deep }; + const scanned = { ...deep, $defs: { D: date } }; + expect(inlineLocalRefs(scanned)).toBe(scanned); + + // Within the bound where it is declared (`$defs/T`, one level down) but + // past it once inlined under `properties/a` (two down): only inlining bails. + let tall: Record = { type: "string" }; + for (let n = 0; n < MAX_DEPTH - 1; n++) tall = { items: tall }; + const inlined = { + properties: { a: { $ref: "#/$defs/T" } }, + $defs: { T: tall }, + }; + expect(inlineLocalRefs(inlined)).toBe(inlined); + }); +}); diff --git a/clients/web/src/test/core/mcp/node/notificationHeadersFetch.test.ts b/clients/web/src/test/core/mcp/node/notificationHeadersFetch.test.ts new file mode 100644 index 000000000..0df1c0eb9 --- /dev/null +++ b/clients/web/src/test/core/mcp/node/notificationHeadersFetch.test.ts @@ -0,0 +1,120 @@ +import { describe, it, expect, vi } from "vitest"; +import { PROTOCOL_VERSION_META_KEY } from "@modelcontextprotocol/client"; +import { createNotificationHeadersFetch } from "@inspector/core/mcp/node/notificationHeadersFetch.js"; +import { MODERN_PROTOCOL_VERSION } from "@inspector/core/mcp/types.js"; + +const URL_ = "https://example.com/mcp"; + +function cancelled(version: string | undefined): Record { + return { + jsonrpc: "2.0", + method: "notifications/cancelled", + params: { + requestId: 7, + ...(version !== undefined && { + _meta: { [PROTOCOL_VERSION_META_KEY]: version }, + }), + }, + }; +} + +/** Run one call through the wrapper; return the init the base fetch saw. */ +async function send(init: RequestInit | undefined): Promise { + const baseFetch = vi.fn( + async () => new Response(null, { status: 202 }), + ); + await createNotificationHeadersFetch(baseFetch)(URL_, init); + expect(baseFetch).toHaveBeenCalledTimes(1); + return baseFetch.mock.calls[0]![1] ?? {}; +} + +function post(body: unknown, headers?: HeadersInit): RequestInit { + return { + method: "POST", + body: typeof body === "string" ? body : JSON.stringify(body), + headers, + }; +} + +describe("createNotificationHeadersFetch", () => { + it("stamps Mcp-Method and MCP-Protocol-Version on a modern notification", async () => { + const seen = await send( + post(cancelled(MODERN_PROTOCOL_VERSION), { + "content-type": "application/json", + "mcp-session-id": "abc", + }), + ); + const headers = new Headers(seen.headers); + expect(headers.get("mcp-method")).toBe("notifications/cancelled"); + expect(headers.get("mcp-protocol-version")).toBe(MODERN_PROTOCOL_VERSION); + // Existing headers and the body survive. + expect(headers.get("mcp-session-id")).toBe("abc"); + expect(headers.get("content-type")).toBe("application/json"); + expect(seen.body).toBe(JSON.stringify(cancelled(MODERN_PROTOCOL_VERSION))); + expect(headers.has("mcp-name")).toBe(false); + }); + + it("accepts a Headers instance and a lowercase method", async () => { + const init = post( + cancelled(MODERN_PROTOCOL_VERSION), + new Headers({ accept: "application/json" }), + ); + const seen = await send({ ...init, method: "post" }); + const headers = new Headers(seen.headers); + expect(headers.get("mcp-method")).toBe("notifications/cancelled"); + expect(headers.get("accept")).toBe("application/json"); + }); + + it("stamps a revision later than 2026-07-28 too", async () => { + const seen = await send(post(cancelled("2027-01-01"))); + expect(new Headers(seen.headers).get("mcp-protocol-version")).toBe( + "2027-01-01", + ); + }); + + const untouched: [string, RequestInit | undefined][] = [ + ["no init", undefined], + ["a GET", { method: "GET" }], + [ + "a POST with no method", + { body: JSON.stringify(cancelled("2026-07-28")) }, + ], + ["a non-string body", { method: "POST", body: new Blob(["{}"]) }], + ["a non-JSON body", post("not json")], + ["a JSON non-object body", post([cancelled(MODERN_PROTOCOL_VERSION)])], + ["a null body", post("null")], + [ + "a request (has an id)", + post({ ...cancelled(MODERN_PROTOCOL_VERSION), id: 1 }), + ], + ["a response (no method)", post({ jsonrpc: "2.0", result: {} })], + [ + "a non-string method", + post({ ...cancelled(MODERN_PROTOCOL_VERSION), method: 5 }), + ], + ["a legacy-era notification", post(cancelled("2025-11-25"))], + ["an unclaimed notification", post(cancelled(undefined))], + [ + "a notification with no params", + post({ jsonrpc: "2.0", method: "notifications/initialized" }), + ], + [ + "a notification whose _meta is not an object", + post({ jsonrpc: "2.0", method: "x", params: { _meta: "nope" } }), + ], + [ + "a non-string protocol version claim", + post({ + jsonrpc: "2.0", + method: "x", + params: { _meta: { [PROTOCOL_VERSION_META_KEY]: 20260728 } }, + }), + ], + ]; + + it.each(untouched)("passes %s through untouched", async (_label, init) => { + const seen = await send(init); + expect(seen).toEqual(init ?? {}); + expect(new Headers(seen.headers).has("mcp-method")).toBe(false); + }); +}); diff --git a/clients/web/src/test/core/mcp/node/suppressNotificationStreamFetch.test.ts b/clients/web/src/test/core/mcp/node/suppressNotificationStreamFetch.test.ts new file mode 100644 index 000000000..8db313a44 --- /dev/null +++ b/clients/web/src/test/core/mcp/node/suppressNotificationStreamFetch.test.ts @@ -0,0 +1,102 @@ +/** + * Request classification for `createSuppressNotificationStreamFetch` (#2317). + * + * The wrapper sits on a fetch the SDK uses for more than the MCP endpoint — + * OAuth metadata discovery shares it — so the cost of a loose predicate is + * broken auth, not merely a missing stream. These cases pin the match to + * exactly the standalone SSE `GET`: every other request, including the + * near-misses (another path, a non-SSE `Accept`, a `Last-Event-ID` + * resumption), must reach the network. The live-transport half — that the SDK + * really accepts the synthetic 405 and carries on — is + * `integration/mcp/suppress-notification-stream.test.ts`. + */ +import { describe, it, expect, vi } from "vitest"; +import { createSuppressNotificationStreamFetch } from "@inspector/core/mcp/node/suppressNotificationStreamFetch.js"; + +const ENDPOINT = "https://example.com/mcp"; +const SSE = { accept: "text/event-stream" }; + +function setup() { + const baseFetch = vi.fn(async () => new Response("ok", { status: 200 })); + return { + baseFetch, + fetchFn: createSuppressNotificationStreamFetch( + baseFetch, + new URL(ENDPOINT), + ), + }; +} + +describe("createSuppressNotificationStreamFetch (#2317)", () => { + it("answers the standalone SSE GET with a local 405 and never sends it", async () => { + const { baseFetch, fetchFn } = setup(); + const res = await fetchFn(ENDPOINT, { + method: "GET", + headers: new Headers({ accept: "application/json, text/event-stream" }), + }); + expect(res.status).toBe(405); + expect(baseFetch).not.toHaveBeenCalled(); + }); + + it("matches a URL input and a Request input for the endpoint", async () => { + const { baseFetch, fetchFn } = setup(); + expect((await fetchFn(new URL(ENDPOINT), { headers: SSE })).status).toBe( + 405, + ); + expect( + (await fetchFn(new Request(ENDPOINT, { headers: SSE }))).status, + ).toBe(405); + expect(baseFetch).not.toHaveBeenCalled(); + }); + + it.each([ + [ + "an OAuth protected-resource metadata GET", + "https://example.com/.well-known/oauth-protected-resource/mcp", + { accept: "application/json" }, + ], + [ + "an authorization-server metadata GET on another origin", + "https://auth.example.com/.well-known/oauth-authorization-server", + { accept: "application/json" }, + ], + ["an SSE GET to another path", "https://example.com/other", SSE], + ["an SSE GET with a different query", `${ENDPOINT}?x=1`, SSE], + ["a non-SSE GET to the endpoint", ENDPOINT, { accept: "application/json" }], + ["a GET to the endpoint with no Accept", ENDPOINT, {}], + ["an unparseable URL", "not a url", SSE], + ])("passes %s through", async (_label, url, headers) => { + const { baseFetch, fetchFn } = setup(); + const init = { method: "GET", headers }; + expect((await fetchFn(url, init)).status).toBe(200); + expect(baseFetch).toHaveBeenCalledWith(url, init); + }); + + it("passes a resumption GET (Last-Event-ID) through", async () => { + const { baseFetch, fetchFn } = setup(); + const init = { + method: "get", + headers: { ...SSE, "last-event-id": "42" }, + }; + expect((await fetchFn(ENDPOINT, init)).status).toBe(200); + expect(baseFetch).toHaveBeenCalledWith(ENDPOINT, init); + }); + + it.each(["POST", "DELETE"])("passes %s through", async (method) => { + const { baseFetch, fetchFn } = setup(); + const init = { method, headers: SSE }; + expect((await fetchFn(ENDPOINT, init)).status).toBe(200); + expect(baseFetch).toHaveBeenCalledWith(ENDPOINT, init); + }); + + it("reads the method off a Request input", async () => { + const { baseFetch, fetchFn } = setup(); + const post = new Request(ENDPOINT, { + method: "POST", + headers: SSE, + body: "{}", + }); + expect((await fetchFn(post)).status).toBe(200); + expect(baseFetch).toHaveBeenCalledTimes(1); + }); +}); diff --git a/clients/web/src/test/core/mcp/serverList.test.ts b/clients/web/src/test/core/mcp/serverList.test.ts index eb3ebc220..d660ecd00 100644 --- a/clients/web/src/test/core/mcp/serverList.test.ts +++ b/clients/web/src/test/core/mcp/serverList.test.ts @@ -347,6 +347,41 @@ describe("serverEntriesToMcpConfig", () => { expect(round).toEqual(original); }); + it("round-trips suppressNotificationStream: lifts true to settings and back to disk (#2317)", () => { + const original: MCPConfig = { + mcpServers: { + delta: { + type: "streamable-http", + url: "https://x.test/mcp", + suppressNotificationStream: true, + }, + }, + }; + const [entry] = mcpConfigToServerEntries(original); + expect(entry?.settings?.suppressNotificationStream).toBe(true); + const round = serverEntriesToMcpConfig(mcpConfigToServerEntries(original)); + expect(round).toEqual(original); + }); + + it("drops a non-true suppressNotificationStream on read and omits it on write (#2317)", () => { + const original = { + mcpServers: { + epsilon: { + type: "streamable-http", + url: "https://x.test/mcp", + suppressNotificationStream: false, + }, + }, + } satisfies MCPConfig; + const [entry] = mcpConfigToServerEntries(original); + expect(entry?.settings).toBeDefined(); + expect(entry?.settings?.suppressNotificationStream).toBeUndefined(); + const round = serverEntriesToMcpConfig(mcpConfigToServerEntries(original)); + expect( + "suppressNotificationStream" in (round.mcpServers.epsilon ?? {}), + ).toBe(false); + }); + it("round-trips advertisedExtensions: lifts a non-empty map to settings and back to disk", () => { const original: MCPConfig = { mcpServers: { diff --git a/clients/web/src/test/core/schemaLint.test.ts b/clients/web/src/test/core/schemaLint.test.ts index cb2db9d20..6cecffc49 100644 --- a/clients/web/src/test/core/schemaLint.test.ts +++ b/clients/web/src/test/core/schemaLint.test.ts @@ -303,10 +303,35 @@ describe("lintToolSchemas — type-union", () => { }), ); expect(rules(findings)).toEqual(["type-union"]); + // A warning, not an error: `--strict` exits 6 only on error findings, and + // the array form is provider-recommended, so it must not fail CI (#2286). expect(findings[0]!.severity).toBe("warning"); + // Framed as a portability trade that acknowledges the provider guidance. + expect(findings[0]!.issue).toContain("some model providers recommend it"); + expect(findings[0]!.issue).toContain("less portable"); + // A weaker-class rule must name its dialect, not a generic "some clients". + expect(findings[0]!.issue).toContain( + "OpenAPI subset used for Gemini function declarations", + ); expect(findings[0]!.suggestion).toContain( '{"anyOf": [{"type": "null"}, {"type": "boolean"}]}', ); + // The `null` branch is not expressible in the named OpenAPI 3.0 dialect, + // so a null union's suggestion must say so rather than overclaim. + expect(findings[0]!.suggestion).toContain("nullable: true"); + }); + + it("omits the null-branch caveat when the union has no null", () => { + const findings = lintToolSchemas( + tool({ + inputSchema: { + type: "object", + properties: { a: { type: ["string", "number"] } }, + }, + }), + ); + expect(rules(findings)).toEqual(["type-union"]); + expect(findings[0]!.suggestion).not.toContain("nullable"); }); it("never suggests un-requiring the property as the equivalent fix", () => { diff --git a/clients/web/src/test/integration/auth/node/file-secret-store.test.ts b/clients/web/src/test/integration/auth/node/file-secret-store.test.ts index 8ab964339..711977c4f 100644 --- a/clients/web/src/test/integration/auth/node/file-secret-store.test.ts +++ b/clients/web/src/test/integration/auth/node/file-secret-store.test.ts @@ -16,6 +16,9 @@ import * as path from "node:path"; import { FileSecretStore, readSecretFilePermissions, + resolveSecretPassphrase, + SECRET_KEY_ENV, + SECRET_KEY_FILE_ENV, SecretFileKeyMismatchError, tightenSecretFilePermissions, } from "@inspector/core/auth/node/file-secret-store.js"; @@ -1531,3 +1534,209 @@ describe("tightenSecretFilePermissions reports what it could not fix", () => { }); }); }); + +describe("resolveSecretPassphrase (MCP_INSPECTOR_SECRET_KEY_FILE, #2447)", () => { + const keyFile = (): string => path.join(tmpDir, "secret-key"); + + it("returns nothing when neither variable is set", () => { + expect(resolveSecretPassphrase({})).toEqual({}); + }); + + it("uses MCP_INSPECTOR_SECRET_KEY verbatim", () => { + expect( + resolveSecretPassphrase({ [SECRET_KEY_ENV]: " pass phrase " }), + ).toEqual({ passphrase: " pass phrase " }); + }); + + it("reads the key file, stripping only the trailing line break", async () => { + await fs.writeFile(keyFile(), " from file \r\n\n"); + expect( + resolveSecretPassphrase({ [SECRET_KEY_FILE_ENV]: keyFile() }), + ).toEqual({ passphrase: " from file " }); + }); + + it("treats a blank MCP_INSPECTOR_SECRET_KEY as unset, so the file is used", async () => { + await fs.writeFile(keyFile(), "from-file\n"); + expect( + resolveSecretPassphrase({ + [SECRET_KEY_ENV]: " ", + [SECRET_KEY_FILE_ENV]: keyFile(), + }), + ).toEqual({ passphrase: "from-file" }); + }); + + it("resolves a relative key-file path against the working directory", async () => { + await fs.writeFile(keyFile(), "relative\n"); + const relative = path.relative(process.cwd(), keyFile()); + expect( + resolveSecretPassphrase({ [SECRET_KEY_FILE_ENV]: relative }), + ).toEqual({ passphrase: "relative" }); + }); + + it("refuses both variables at once rather than picking one", async () => { + await fs.writeFile(keyFile(), "from-file\n"); + const result = resolveSecretPassphrase({ + [SECRET_KEY_ENV]: "direct", + [SECRET_KEY_FILE_ENV]: keyFile(), + }); + expect(result.passphrase).toBeUndefined(); + expect(result.problem).toMatch( + /both MCP_INSPECTOR_SECRET_KEY and MCP_INSPECTOR_SECRET_KEY_FILE are set/, + ); + }); + + it("reports a missing key file as a problem, not as no passphrase", () => { + const result = resolveSecretPassphrase({ + [SECRET_KEY_FILE_ENV]: path.join(tmpDir, "nope"), + }); + expect(result.passphrase).toBeUndefined(); + expect(result.problem).toMatch(/could not be read: .*ENOENT/); + }); + + it("strips a lone trailing carriage return", async () => { + await fs.writeFile(keyFile(), "classic-mac\r"); + expect( + resolveSecretPassphrase({ [SECRET_KEY_FILE_ENV]: keyFile() }), + ).toEqual({ passphrase: "classic-mac" }); + }); + + it("reports a blank MCP_INSPECTOR_SECRET_KEY_FILE as a problem, not as unset", () => { + // A template whose path did not expand must not quietly mean plaintext. + const result = resolveSecretPassphrase({ [SECRET_KEY_FILE_ENV]: " " }); + expect(result.passphrase).toBeUndefined(); + expect(result.problem).toBe( + "MCP_INSPECTOR_SECRET_KEY_FILE is set but empty", + ); + }); + + it("still refuses both variables when the file variable is blank", () => { + const result = resolveSecretPassphrase({ + [SECRET_KEY_ENV]: "direct", + [SECRET_KEY_FILE_ENV]: "", + }); + expect(result.passphrase).toBeUndefined(); + expect(result.problem).toMatch(/both .* are set/); + }); + + it("reports an empty key file as a problem", async () => { + await fs.writeFile(keyFile(), "\n"); + const result = resolveSecretPassphrase({ + [SECRET_KEY_FILE_ENV]: keyFile(), + }); + expect(result.passphrase).toBeUndefined(); + expect(result.problem).toMatch(/is empty/); + }); +}); + +describe("FileSecretStore with MCP_INSPECTOR_SECRET_KEY_FILE (#2447)", () => { + const keyFile = (): string => path.join(tmpDir, "secret-key"); + + afterEach(() => { + vi.unstubAllEnvs(); + }); + + it("encrypts with the passphrase read from the file", async () => { + await fs.writeFile(keyFile(), "hunter2\n"); + vi.stubEnv(SECRET_KEY_ENV, ""); + vi.stubEnv(SECRET_KEY_FILE_ENV, keyFile()); + const store = new FileSecretStore({ filePath: filePath() }); + expect(store.encrypted).toBe(true); + expect(store.keyProblem).toBeUndefined(); + await store.set("alpha", "env:A", "super-secret"); + const raw = await fs.readFile(filePath(), "utf-8"); + expect(JSON.parse(raw).encryption).toBe("aes-256-gcm"); + // The same passphrase supplied directly opens it: the newline is not + // part of the key. + const direct = new FileSecretStore({ + filePath: filePath(), + passphrase: "hunter2", + }); + expect(await direct.get("alpha", "env:A")).toBe("super-secret"); + }); + + it("refuses to write, rather than writing plaintext, when the key file is missing", async () => { + vi.stubEnv(SECRET_KEY_ENV, ""); + vi.stubEnv(SECRET_KEY_FILE_ENV, path.join(tmpDir, "missing-key")); + const store = new FileSecretStore({ filePath: filePath() }); + expect(store.encrypted).toBe(false); + expect(store.keyProblem).toMatch(/could not be read/); + await expect(store.set("alpha", "env:A", "v")).rejects.toThrow( + SecretStoreUnavailableError, + ); + await expect(store.set("alpha", "env:A", "v")).rejects.toThrow( + /Refusing to read or write it rather than store secrets unencrypted/, + ); + expect(existsSyncFile(filePath())).toBe(false); + expect(await store.get("alpha", "env:A")).toBeNull(); + await expect(store.readAll()).rejects.toThrow(SecretStoreUnavailableError); + expect(await store.readOnDiskEncryption()).toEqual({ + state: "unreadable", + detail: expect.stringMatching( + /MCP_INSPECTOR_SECRET_KEY_FILE .* could not be read/, + ), + }); + }); + + it("leaves an existing file untouched while the key is unavailable", async () => { + const plain = new FileSecretStore({ filePath: filePath() }); + await plain.set("alpha", "env:A", "1"); + const before = await fs.readFile(filePath(), "utf-8"); + vi.stubEnv(SECRET_KEY_ENV, "direct"); + vi.stubEnv(SECRET_KEY_FILE_ENV, keyFile()); + const store = new FileSecretStore({ filePath: filePath() }); + expect(store.keyProblem).toMatch(/both/); + await expect(store.set("alpha", "env:B", "2")).rejects.toThrow( + SecretStoreUnavailableError, + ); + await expect(store.delete("alpha", "env:A")).resolves.toBeUndefined(); + expect(await fs.readFile(filePath(), "utf-8")).toBe(before); + }); + + it("refuses a key file that is the secrets file itself", async () => { + // Otherwise the plaintext JSON becomes the passphrase, the next save + // replaces it with ciphertext, and the following start cannot open it. + const plain = new FileSecretStore({ filePath: filePath() }); + await plain.set("alpha", "env:A", "1"); + const before = await fs.readFile(filePath(), "utf-8"); + vi.stubEnv(SECRET_KEY_ENV, ""); + vi.stubEnv(SECRET_KEY_FILE_ENV, filePath()); + const store = new FileSecretStore({ filePath: filePath() }); + expect(store.keyProblem).toMatch(/is the secrets file itself/); + await expect(store.set("alpha", "env:B", "2")).rejects.toThrow( + SecretStoreUnavailableError, + ); + expect(await fs.readFile(filePath(), "utf-8")).toBe(before); + }); + + it("refuses a symlink or hard link to the secrets file", async () => { + await fs.writeFile(filePath(), "{}\n"); + const link = path.join(tmpDir, "key-symlink"); + const hard = path.join(tmpDir, "key-hardlink"); + await fs.symlink(filePath(), link); + await fs.link(filePath(), hard); + for (const keyPath of [link, hard]) { + expect( + resolveSecretPassphrase({ [SECRET_KEY_FILE_ENV]: keyPath }, filePath()) + .problem, + ).toMatch(/is the secrets file itself/); + } + }); + + it("matches by path when the secrets file does not exist yet", () => { + const result = resolveSecretPassphrase( + { [SECRET_KEY_FILE_ENV]: filePath() }, + filePath(), + ); + expect(result.problem).toMatch(/is the secrets file itself/); + }); + + it("an explicit passphrase option ignores the environment", () => { + vi.stubEnv(SECRET_KEY_FILE_ENV, path.join(tmpDir, "missing-key")); + const store = new FileSecretStore({ + filePath: filePath(), + passphrase: "hunter2", + }); + expect(store.keyProblem).toBeUndefined(); + expect(store.encrypted).toBe(true); + }); +}); diff --git a/clients/web/src/test/integration/auth/node/secret-store-selection.test.ts b/clients/web/src/test/integration/auth/node/secret-store-selection.test.ts index e560a4792..b31e1c518 100644 --- a/clients/web/src/test/integration/auth/node/secret-store-selection.test.ts +++ b/clients/web/src/test/integration/auth/node/secret-store-selection.test.ts @@ -26,6 +26,7 @@ import { chooseFallbackKind, isOnMountPoint, parseSecretStoreEnv, + SECRET_STORAGE_DOCS_URL, warnAboutSecretStorage, } from "@inspector/core/auth/node/secret-store-selection.js"; import { @@ -38,6 +39,7 @@ const ENV_KEYS = [ "MCP_INSPECTOR_SECRET_STORE", "MCP_INSPECTOR_SECRET_FILE", "MCP_INSPECTOR_SECRET_KEY", + "MCP_INSPECTOR_SECRET_KEY_FILE", "MCP_STORAGE_DIR", "KUBERNETES_SERVICE_HOST", ]; @@ -433,6 +435,48 @@ describe("warnAboutSecretStorage", () => { expect(warn).not.toHaveBeenCalled(); }); + it("points a plaintext fallback at the secret-storage guide", () => { + // The Linux-without-libsecret case (#2447): the one run where a user + // gets a plaintext file without asking for it. + const warn = vi.spyOn(console, "warn").mockImplementation(() => {}); + warnAboutSecretStorage({ + kind: "file", + reason: "fallback", + durable: true, + path: "/home/u/.mcp-inspector/secrets.json", + plaintext: true, + detail: "no Secret Service", + }); + const output = warn.mock.calls.flat().join("\n"); + expect(output).toContain("Secrets are stored unencrypted"); + expect(output).toContain(SECRET_STORAGE_DOCS_URL); + expect(SECRET_STORAGE_DOCS_URL).toMatch(/\/docs\/secret-storage\.md$/); + }); + + it("points a configured store with a caveat at the guide too", () => { + const warn = vi.spyOn(console, "warn").mockImplementation(() => {}); + warnAboutSecretStorage({ + kind: "memory", + reason: "configured", + durable: false, + }); + expect(warn.mock.calls.flat().join("\n")).toContain( + SECRET_STORAGE_DOCS_URL, + ); + }); + + it("does not print the guide link for a configured store with nothing to say", () => { + const warn = vi.spyOn(console, "warn").mockImplementation(() => {}); + warnAboutSecretStorage({ + kind: "file", + reason: "configured", + durable: true, + path: "/x/secrets.json", + plaintext: false, + }); + expect(warn).not.toHaveBeenCalled(); + }); + it("announces a fallback with no cause to name", () => { // `detail` is optional — an explicitly configured store has no keychain // error behind it, and the banner must not print an empty error line. diff --git a/clients/web/src/test/integration/mcp/inspectorClient-subscriptions-era.test.ts b/clients/web/src/test/integration/mcp/inspectorClient-subscriptions-era.test.ts index d41f0da28..e724d8d5c 100644 --- a/clients/web/src/test/integration/mcp/inspectorClient-subscriptions-era.test.ts +++ b/clients/web/src/test/integration/mcp/inspectorClient-subscriptions-era.test.ts @@ -1,7 +1,10 @@ import { describe, it, expect, afterEach, vi } from "vitest"; import { InspectorClient } from "@inspector/core/mcp/inspectorClient.js"; import { createTransportNode } from "@inspector/core/mcp/node/transport.js"; -import { eraToVersionNegotiation } from "@inspector/core/mcp/types.js"; +import { + eraToVersionNegotiation, + MODERN_PROTOCOL_VERSION, +} from "@inspector/core/mcp/types.js"; import type { McpSubscription } from "@modelcontextprotocol/client"; import { createTestServerHttp, @@ -101,6 +104,19 @@ describe("resource subscriptions era fork (#1630)", () => { return { connected, messages }; } + /** A fetch that records each string-bodied request's body and final headers. */ + function recordingFetch( + inner: typeof fetch, + sent: { body: string; headers: Headers }[], + ): typeof fetch { + return (input, init) => { + if (typeof init?.body === "string") { + sent.push({ body: init.body, headers: new Headers(init.headers) }); + } + return inner(input, init); + }; + } + function methodsSent(messages: MessageEntry[]): string[] { return messages .filter((m) => m.direction === "request") @@ -196,6 +212,48 @@ describe("resource subscriptions era fork (#1630)", () => { expect(methodsSent(messages)).not.toContain("subscriptions/listen"); }); + it("sends the listen stream's notifications/cancelled with Mcp-Method (#2385)", async () => { + // The SDK POSTs a `notifications/cancelled` when a listen stream closes, + // and stamps the SEP-2243 headers on requests only — so a strict modern + // server refused the unsubscribe `400 Mcp-Method is required`. The test + // server is lenient, so the wire headers are asserted directly. + const started = await startServer({}); + const sent: { body: string; headers: Headers }[] = []; + const connected = new InspectorClient( + { type: "streamable-http", url: started.url }, + { + environment: { + transport: (config, options) => + createTransportNode(config, { + ...options, + fetchFn: recordingFetch( + options?.fetchFn ?? globalThis.fetch, + sent, + ), + }), + }, + versionNegotiation: eraToVersionNegotiation("modern"), + listChangedNotifications: NO_LIST_CHANGED, + }, + ); + client = connected; + await connected.connect(); + await connected.subscribeToResource(RESOURCE_URI); + await connected.unsubscribeFromResource(RESOURCE_URI); + + await vi.waitFor(() => { + const cancelled = sent.find((request) => + request.body.includes('"notifications/cancelled"'), + ); + expect(cancelled?.headers.get("mcp-method")).toBe( + "notifications/cancelled", + ); + expect(cancelled?.headers.get("mcp-protocol-version")).toBe( + MODERN_PROTOCOL_VERSION, + ); + }); + }); + it("keeps the stream open past the last subscription when a listChanged opt-in remains", async () => { // The other half of the above: with an advertised listChanged the filter // is still non-empty, so the last unsubscribe re-lists rather than diff --git a/clients/web/src/test/integration/mcp/remote/server-extra-coverage.test.ts b/clients/web/src/test/integration/mcp/remote/server-extra-coverage.test.ts index 4ebd734b3..bee1d3e33 100644 --- a/clients/web/src/test/integration/mcp/remote/server-extra-coverage.test.ts +++ b/clients/web/src/test/integration/mcp/remote/server-extra-coverage.test.ts @@ -827,6 +827,40 @@ describe("server.ts supplemental coverage", () => { expect((await res.json()).error).toMatch(/paginatedLists/); }); + it("rejects a non-boolean suppressNotificationStream (#2317)", async () => { + const res = await postSettings({ + ...base, + suppressNotificationStream: "yes", + }); + expect((await res.json()).error).toMatch(/suppressNotificationStream/); + }); + + // #2317 — a 200 only proves the payload validated; read the entry back so + // dropping the field from `normalizeSettings` cannot pass silently. + async function readSuppressNotificationStream() { + const res = await fetch(`${h.baseUrl}/api/servers`); + const body = (await res.json()) as { + mcpServers: Record>; + }; + return body.mcpServers.srv?.suppressNotificationStream; + } + + it("persists suppressNotificationStream through a save (#2317)", async () => { + expect( + (await postSettings({ ...base, suppressNotificationStream: true })) + .status, + ).toBe(200); + expect(await readSuppressNotificationStream()).toBe(true); + }); + + it("writes no suppressNotificationStream field when off (#2317)", async () => { + expect( + (await postSettings({ ...base, suppressNotificationStream: false })) + .status, + ).toBe(200); + expect(await readSuppressNotificationStream()).toBeUndefined(); + }); + it("rejects a negative maxFetchRequests", async () => { const res = await postSettings({ ...base, maxFetchRequests: -2 }); expect((await res.json()).error).toMatch(/maxFetchRequests/); diff --git a/clients/web/src/test/integration/mcp/suppress-notification-stream.test.ts b/clients/web/src/test/integration/mcp/suppress-notification-stream.test.ts new file mode 100644 index 000000000..d35e27cd4 --- /dev/null +++ b/clients/web/src/test/integration/mcp/suppress-notification-stream.test.ts @@ -0,0 +1,78 @@ +import { describe, it, expect, afterEach } from "vitest"; +import { InspectorClient } from "@inspector/core/mcp/inspectorClient.js"; +import { createTransportNode } from "@inspector/core/mcp/node/transport.js"; +import type { InspectorServerSettings } from "@inspector/core/mcp/types.js"; +import { + createTestServerHttp, + type TestServerHttp, + createTestServerInfo, + createNumberedTools, +} from "@modelcontextprotocol/inspector-test-server"; + +/** + * Live coverage of the `suppressNotificationStream` setting (#2317) through + * the real SDK transport: `createSuppressNotificationStreamFetch` is only + * useful if the transport really does treat the synthetic 405 as "no + * standalone stream" and carry on, which a unit test of the wrapper cannot + * show. The control arm proves the recorder would have seen the GET. + */ +describe("suppressNotificationStream (#2317)", () => { + let client: InspectorClient | null = null; + let server: TestServerHttp | null = null; + + afterEach(async () => { + await client?.disconnect().catch(() => {}); + client = null; + await server?.stop().catch(() => {}); + server = null; + }); + + function settings(suppress: boolean): InspectorServerSettings { + return { + headers: [], + metadata: {}, + env: [], + connectionTimeout: 0, + requestTimeout: 0, + taskTtl: 60000, + maxFetchRequests: 1000, + roots: [], + ...(suppress && { suppressNotificationStream: true }), + }; + } + + async function connectRecording(suppress: boolean) { + server = createTestServerHttp({ + serverInfo: createTestServerInfo("suppress-stream-test", "1.0.0"), + tools: createNumberedTools(2), + }); + await server.start(); + const methods: string[] = []; + const recordingFetch: typeof fetch = (input, init) => { + methods.push((init?.method ?? "GET").toUpperCase()); + return fetch(input, init); + }; + client = new InspectorClient( + { type: "streamable-http", url: server.url }, + { + environment: { transport: createTransportNode, fetch: recordingFetch }, + serverSettings: settings(suppress), + }, + ); + await client.connect(); + return { client, methods }; + } + + it("opens the standalone GET stream by default (control)", async () => { + const { client: connected, methods } = await connectRecording(false); + await expect.poll(() => methods.includes("GET")).toBe(true); + expect((await connected.listTools()).tools).toHaveLength(2); + }); + + it("never sends the GET when suppressed, and requests still work", async () => { + const { client: connected, methods } = await connectRecording(true); + expect((await connected.listTools()).tools).toHaveLength(2); + expect(methods).toContain("POST"); + expect(methods).not.toContain("GET"); + }); +}); diff --git a/clients/web/src/test/integration/server/web-server-config.test.ts b/clients/web/src/test/integration/server/web-server-config.test.ts index d3477d115..8682dd42b 100644 --- a/clients/web/src/test/integration/server/web-server-config.test.ts +++ b/clients/web/src/test/integration/server/web-server-config.test.ts @@ -154,6 +154,30 @@ describe("buildWebServerConfigFromEnv", () => { expect(cfg.authToken).toBe(""); }); + // #2331: `!!value` read every non-empty string as "on", so a deployment that + // set DANGEROUSLY_OMIT_AUTH=false to keep auth on silently turned it off. + it.each(["true", "TRUE", " True ", "1", " 1 "])( + "omits auth for the explicit opt-in DANGEROUSLY_OMIT_AUTH=%j", + (value) => { + process.env.DANGEROUSLY_OMIT_AUTH = value; + process.env[API_SERVER_ENV_VARS.AUTH_TOKEN] = "ignored"; + const cfg = buildWebServerConfigFromEnv(); + expect(cfg.dangerouslyOmitAuth).toBe(true); + expect(cfg.authToken).toBe(""); + }, + ); + + it.each(["false", "FALSE", "0", "", " ", "no", "yes", "on", "2"])( + "keeps auth on for DANGEROUSLY_OMIT_AUTH=%j", + (value) => { + process.env.DANGEROUSLY_OMIT_AUTH = value; + process.env[API_SERVER_ENV_VARS.AUTH_TOKEN] = "kept"; + const cfg = buildWebServerConfigFromEnv(); + expect(cfg.dangerouslyOmitAuth).toBe(false); + expect(cfg.authToken).toBe("kept"); + }, + ); + it("uses API_SERVER_ENV_VARS.AUTH_TOKEN when present", () => { process.env[API_SERVER_ENV_VARS.AUTH_TOKEN] = "primary"; const cfg = buildWebServerConfigFromEnv(); diff --git a/clients/web/src/utils/jsonUtils.ts b/clients/web/src/utils/jsonUtils.ts index 24a31e91d..a04396e78 100644 --- a/clients/web/src/utils/jsonUtils.ts +++ b/clients/web/src/utils/jsonUtils.ts @@ -2,6 +2,7 @@ import { admitsNull, normalizeNullableUnion, } from "@inspector/core/json/nullableUnion.js"; +import { inlineLocalRefs } from "@inspector/core/json/localRefs.js"; import { branchAcceptsValues, declaresAnyFields, @@ -92,8 +93,10 @@ export function toFormSchema(schema: unknown): InspectorFormSchema | null { } // Structural narrow: the SDK schema's fields are a superset of what the form // reads (`type`, `properties`, `required`, `items`, …); the values the form - // never dereferences don't affect rendering. - return schema as InspectorFormSchema; + // never dereferences don't affect rendering. Same-document `$ref`s are + // inlined first, since every widget is chosen by a property's own `type` and + // a deduplicated Zod schema carries none (#2321). + return inlineLocalRefs(schema) as InspectorFormSchema; } export type DataType = diff --git a/core/auth/node/file-secret-store.ts b/core/auth/node/file-secret-store.ts index 4d8127518..1d7292b68 100644 --- a/core/auth/node/file-secret-store.ts +++ b/core/auth/node/file-secret-store.ts @@ -9,8 +9,9 @@ * off disk. Writing secrets back to disk here is therefore a decision, not * an oversight, and it is bounded three ways: the file is separate from * `mcp.json` (so a user pasting their catalog into an issue does not paste - * their secrets), it is written `0600`, and it is encrypted whenever - * `MCP_INSPECTOR_SECRET_KEY` is set. What it buys is the alternative: + * their secrets), it is written `0600`, and it is encrypted whenever a + * passphrase is supplied, through `MCP_INSPECTOR_SECRET_KEY` or the file + * named by `MCP_INSPECTOR_SECRET_KEY_FILE`. What it buys is the alternative: * before this, those users could not persist a secret at all — `set` threw * and the route answered 503. * @@ -30,10 +31,13 @@ * of them you hold an OAuth client secret for. That index is worth * roughly as much to an attacker as some of the values. * - * **Key.** `MCP_INSPECTOR_SECRET_KEY` is a passphrase, not a key: it is - * stretched with scrypt against a per-file random salt stored beside the - * ciphertext, so the same passphrase produces a different key for a - * different file and a precomputed table buys an attacker nothing. + * **Key.** The value of `MCP_INSPECTOR_SECRET_KEY`, or the contents of the + * file named by `MCP_INSPECTOR_SECRET_KEY_FILE` (see + * {@link resolveSecretPassphrase}), is a passphrase, not a key: it is + * stretched with scrypt against a random salt, regenerated on every write + * and stored beside the ciphertext, so the same passphrase produces a + * different key for every write and a precomputed table buys an attacker + * nothing. * * That is **not** a licence to use a short, memorable one. The salt * defeats precomputation; it does nothing against guessing, and the cost @@ -59,6 +63,7 @@ */ import * as crypto from "node:crypto"; +import { readFileSync, statSync } from "node:fs"; import * as fs from "node:fs/promises"; import * as path from "node:path"; import { readStoreFile, writeStoreFile } from "../../storage/store-io.js"; @@ -108,6 +113,114 @@ const SCRYPT_P = 1; /** Env var holding the passphrase. Absent → the file is written in the clear. */ export const SECRET_KEY_ENV = "MCP_INSPECTOR_SECRET_KEY"; +/** + * Env var naming a file that holds the passphrase — the `_FILE` convention + * Docker and Compose secrets are built around (#2447). A secret mounted at + * `/run/secrets/` never appears in `docker inspect`, the process + * environment, or a Compose file, which is where `SECRET_KEY_ENV` has to + * live and where it is readable by anyone who can reach the container. + */ +export const SECRET_KEY_FILE_ENV = "MCP_INSPECTOR_SECRET_KEY_FILE"; + +/** + * Where the passphrase came from, or why it could not be had. + * + * `problem` is the state this type exists for. A key file that is named but + * missing, unreadable or empty is a user who asked for encryption and did not + * get a key — and treating that as "no passphrase" would write every secret + * in the clear, the one outcome they configured this to prevent. So it is + * carried as its own state, and the store refuses to read or write under it. + */ +export interface SecretPassphrase { + passphrase?: string; + problem?: string; +} + +/** + * Whether two paths name the same file. Equal resolved paths always do; past + * that, matching device and inode catch a symlink or hard link. A path that + * cannot be stat'd (it does not exist yet, as a secrets file before its first + * save often does) matches only by path. + */ +function isSameFile(a: string, b: string): boolean { + if (path.resolve(a) === path.resolve(b)) return true; + try { + const sa = statSync(a); + const sb = statSync(b); + return sa.dev === sb.dev && sa.ino === sb.ino; + } catch { + return false; + } +} + +/** + * Resolve the passphrase from `SECRET_KEY_ENV` or `SECRET_KEY_FILE_ENV`. + * + * Both set is a misconfiguration rather than a precedence question: they can + * disagree, and silently preferring one would encrypt with a key the user did + * not mean — which is only discovered when the file will not open. Refusing + * names both variables at startup instead. + * + * The file's trailing line breaks (`\n`, `\r\n` or a lone `\r`) are stripped + * (`echo … > key` writes one, and the passphrase is not meant to include it); + * anything else is kept as-is, matching how the env var is used verbatim. + * + * Blank is a problem, not "off", at both levels — a blank + * `SECRET_KEY_FILE_ENV` and a key file that is empty after stripping. Unlike + * an empty `MCP_INSPECTOR_SECRET_KEY=`, naming a key file is an explicit + * request for encryption. + */ +export function resolveSecretPassphrase( + env: NodeJS.ProcessEnv = process.env, + secretFilePath?: string, +): SecretPassphrase { + const direct = env[SECRET_KEY_ENV]; + const hasDirect = direct !== undefined && direct.trim() !== ""; + // Presence, not content, decides whether a key file was asked for. Unlike a + // blank `MCP_INSPECTOR_SECRET_KEY=` (a user switching encryption off), a + // blank `MCP_INSPECTOR_SECRET_KEY_FILE=` is almost always a template whose + // path did not expand — and reading it as "unset" would write plaintext. + const rawKeyFile = env[SECRET_KEY_FILE_ENV]; + const wantsKeyFile = rawKeyFile !== undefined; + const keyFile = rawKeyFile?.trim(); + if (hasDirect && wantsKeyFile) { + return { + problem: `both ${SECRET_KEY_ENV} and ${SECRET_KEY_FILE_ENV} are set; set only one`, + }; + } + if (hasDirect) return { passphrase: direct }; + if (!wantsKeyFile) return {}; + if (!keyFile) { + return { problem: `${SECRET_KEY_FILE_ENV} is set but empty` }; + } + const resolved = path.resolve(keyFile); + // A key file that *is* the secrets file reads the plaintext JSON as the + // passphrase, and the next save replaces it with ciphertext — so on the + // following start the key has changed to that ciphertext and nothing it + // wrote can be opened again. Checked before reading, by path and by inode + // (a symlink or hard link names the same file under another path). + if (secretFilePath !== undefined && isSameFile(resolved, secretFilePath)) { + return { + problem: `${SECRET_KEY_FILE_ENV} (${resolved}) is the secrets file itself; point it at a separate key file`, + }; + } + let contents: string; + try { + contents = readFileSync(resolved, "utf-8"); + } catch (err) { + return { + // `String(err)` rather than `.message`: `readFileSync` only ever throws + // an `Error`, so an `instanceof` guard would be a branch no test can take. + problem: `${SECRET_KEY_FILE_ENV} (${resolved}) could not be read: ${String(err)}`, + }; + } + const passphrase = contents.replace(/[\r\n]+$/, ""); + if (passphrase.trim() === "") { + return { problem: `${SECRET_KEY_FILE_ENV} (${resolved}) is empty` }; + } + return { passphrase }; +} + interface KdfParams { algorithm: "scrypt"; salt: string; @@ -168,8 +281,8 @@ export class SecretFileKeyMismatchError extends SecretStoreUnavailableError { constructor(filePath: string, hasKey: boolean) { super( hasKey - ? `The secrets file at ${filePath} could not be decrypted with the current ${SECRET_KEY_ENV}. Refusing to write, which would overwrite the existing secrets. Restore the original passphrase, or delete the file to start over.` - : `The secrets file at ${filePath} is encrypted but ${SECRET_KEY_ENV} is not set. Refusing to write, which would overwrite the existing secrets. Set the passphrase this file was written with, or delete the file to start over.`, + ? `The secrets file at ${filePath} could not be decrypted with the current ${SECRET_KEY_ENV} (or ${SECRET_KEY_FILE_ENV}). Refusing to write, which would overwrite the existing secrets. Restore the original passphrase, or delete the file to start over.` + : `The secrets file at ${filePath} is encrypted but ${SECRET_KEY_ENV} is not set (nor ${SECRET_KEY_FILE_ENV}). Refusing to write, which would overwrite the existing secrets. Set the passphrase this file was written with, or delete the file to start over.`, ); this.name = "SecretFileKeyMismatchError"; } @@ -294,9 +407,10 @@ export interface FileSecretStoreOptions { /** Absolute path of the secrets file. */ filePath: string; /** - * Passphrase. Defaults to `process.env[SECRET_KEY_ENV]`; an empty or - * whitespace-only value counts as absent, since `MCP_INSPECTOR_SECRET_KEY=` - * in a compose file is a user who meant "off", not a one-character key. + * Passphrase. Defaults to {@link resolveSecretPassphrase} over + * `process.env`; an empty or whitespace-only value counts as absent, since + * `MCP_INSPECTOR_SECRET_KEY=` in a compose file is a user who meant "off", + * not a one-character key. */ passphrase?: string; } @@ -304,10 +418,20 @@ export interface FileSecretStoreOptions { export class FileSecretStore implements SecretStore { readonly filePath: string; private readonly passphrase: string | undefined; + /** + * Why a configured key could not be obtained. While set, every read throws + * and every write refuses — see {@link SecretPassphrase}. + */ + readonly keyProblem: string | undefined; constructor(options: FileSecretStoreOptions) { this.filePath = options.filePath; - const raw = options.passphrase ?? process.env[SECRET_KEY_ENV]; + const resolved = + options.passphrase !== undefined + ? { passphrase: options.passphrase } + : resolveSecretPassphrase(process.env, this.filePath); + const raw = resolved.passphrase; this.passphrase = raw && raw.trim() ? raw : undefined; + this.keyProblem = resolved.problem; } /** @@ -324,7 +448,8 @@ export class FileSecretStore implements SecretStore { * no file yet, or when it cannot be read or parsed. * * Separate from {@link encrypted} because the two genuinely disagree for a - * whole session: adding `MCP_INSPECTOR_SECRET_KEY` to an install that + * whole session: adding a passphrase (`MCP_INSPECTOR_SECRET_KEY` or + * `MCP_INSPECTOR_SECRET_KEY_FILE`) to an install that * already has a plaintext file flips `encrypted` to true immediately, * while the bytes stay readable until the next `set`. A descriptor built * from the policy would tell that user "File (encrypted)" while their @@ -348,6 +473,12 @@ export class FileSecretStore implements SecretStore { * `set` is in fact about to refuse. */ async readOnDiskEncryption(): Promise { + // Before looking at the file at all: with no key to open it and no + // permission to write it in the clear, neither "absent" (which would fall + // back to reporting the write policy) nor "plaintext" is true of it. + if (this.keyProblem !== undefined) { + return { state: "unreadable", detail: this.keyProblem }; + } let raw: string | null; try { raw = await readStoreFile(this.filePath); @@ -412,7 +543,7 @@ export class FileSecretStore implements SecretStore { state: "unreadable", detail: err instanceof SecretFileKeyMismatchError - ? `it cannot be decrypted with the current ${SECRET_KEY_ENV}` + ? `it cannot be decrypted with the current ${SECRET_KEY_ENV} (or ${SECRET_KEY_FILE_ENV})` : err instanceof Error ? err.message : String(err), @@ -469,6 +600,13 @@ export class FileSecretStore implements SecretStore { } private async readMap(): Promise | null> { + // Every read and write passes through here, so this one check is what + // stops a missing key file from degrading to plaintext writes. + if (this.keyProblem !== undefined) { + throw new SecretStoreUnavailableError( + `The secrets file at ${this.filePath} cannot be used: ${this.keyProblem}. Refusing to read or write it rather than store secrets unencrypted.`, + ); + } const raw = await readStoreFile(this.filePath); if (raw === null) return null; diff --git a/core/auth/node/secret-store-selection.ts b/core/auth/node/secret-store-selection.ts index 3a55ec1f5..6603d88fa 100644 --- a/core/auth/node/secret-store-selection.ts +++ b/core/auth/node/secret-store-selection.ts @@ -88,6 +88,14 @@ export const SECRET_FILE_ENV = "MCP_INSPECTOR_SECRET_FILE"; */ export const STORAGE_DIR_ENV = "MCP_STORAGE_DIR"; +/** + * The user-facing guide the fallback and caveat warnings point to. On + * `main`, the release branch, so it describes the behavior that was + * published rather than whatever `v2/main` is mid-way through. + */ +export const SECRET_STORAGE_DOCS_URL = + "https://github.com/modelcontextprotocol/inspector/blob/main/docs/secret-storage.md"; + const KINDS: SecretStoreKind[] = ["keyring", "file", "memory"]; export interface ResolvedSecretStore { @@ -462,6 +470,16 @@ export function warnAboutSecretStorage(info: SecretStorageInfo): void { } const caveat = secretStorageCaveat(info); if (caveat) console.warn(`[mcp-inspector] ${caveat}`); + // Only after something was actually said: the ordinary keychain run stays + // silent. This is the one place a headless or SSH user is guaranteed to + // look, and the fallback it reports — a plaintext file on a Linux box with + // no Secret Service — is otherwise explained only in docs they would have + // no reason to open (#2447). + if (info.reason === "fallback" || caveat) { + console.warn( + `[mcp-inspector] How the secret store is chosen, and how to secure it: ${SECRET_STORAGE_DOCS_URL}`, + ); + } } let resolved: Promise | undefined; diff --git a/core/auth/node/secret-store.ts b/core/auth/node/secret-store.ts index 49612b492..2ea87a4b4 100644 --- a/core/auth/node/secret-store.ts +++ b/core/auth/node/secret-store.ts @@ -285,7 +285,8 @@ const PROBE_ACCOUNT = "__inspector:probe"; * - `KeyringSecretStore` — the OS keychain, and the default wherever one * is reachable. * - `FileSecretStore` — `~/.mcp-inspector/secrets.json`, `0600`, - * encrypted when `MCP_INSPECTOR_SECRET_KEY` is set. The fallback on a + * encrypted when `MCP_INSPECTOR_SECRET_KEY` or + * `MCP_INSPECTOR_SECRET_KEY_FILE` supplies a passphrase. The fallback on a * host with no keychain, and on a container with a mounted volume. * - `InMemorySecretStore` — the session-scoped store. Used by the test * suite (so CI needs no libsecret), and as the container fallback when diff --git a/core/auth/oidcDiscoveryCompat.ts b/core/auth/oidcDiscoveryCompat.ts index ce0ace876..e92a1971d 100644 --- a/core/auth/oidcDiscoveryCompat.ts +++ b/core/auth/oidcDiscoveryCompat.ts @@ -176,9 +176,10 @@ async function releaseBody(response: Response): Promise { * ⚠️ `ReadableStream.cancel()` adopts the underlying source's cancel promise, * which is permitted never to settle — so awaiting it on a path that is * *propagating a cancellation* can hang the very thing that was meant to end - * (Copilot). Every exceptional exit below uses this: the release is a courtesy - * to the connection pool, and the caller's abort or timeout must reach it - * regardless. The `void` is the documented case where the callee owns its + * (Copilot), and awaiting it on the normal path leaves the caller's abort + * unable to end a stalled release (#2389). Every release below uses this: the + * release is a courtesy to the connection pool, and neither progress nor the + * caller's abort or timeout may wait on it. The `void` is the documented case where the callee owns its * failures — `releaseBody` swallows its own — and the caller genuinely cannot * await. */ @@ -334,12 +335,16 @@ export function withRfc8414OidcCompat(fetchFn: typeof fetch): typeof fetch { // Node/undici, and this loop can run on every OAuth attempt — so // release it rather than letting repeated discovery against a 404 // candidate exhaust the origin's pool (Copilot). Same discipline as - // `core/mcp/node/authChallengeFetch.ts`. - await releaseBody(probe); - // Rechecked after the release, which awaits: an abort that lands while - // the body is being discarded must not be answered with the preceding - // response either, and `continue` would otherwise carry on probing for - // a caller that has stopped waiting. + // `core/mcp/node/authChallengeFetch.ts`. Detached, not awaited + // (#2389): the next step — the abort check, the next candidate — must + // not depend on a cancel that is permitted never to settle, or a + // stalled release would hang discovery with the caller's abort unable + // to end it. + releaseBodyDetached(probe); + // Rechecked before moving on: an abort that landed while the probe was + // answering must not be met with the preceding response either, and + // `continue` would otherwise carry on probing for a caller that has + // stopped waiting. if (callerSignal?.aborted) { releaseBodyDetached(response); throw callerSignal.reason; @@ -386,8 +391,10 @@ export function withRfc8414OidcCompat(fetchFn: typeof fetch): typeof fetch { // (Copilot). const source = probe.url || candidate; // The original failed response is about to be dropped in favour of the - // substitution, so release its connection too. - await releaseBody(response); + // substitution, so release its connection too — detached, for the same + // reason as the non-OK probe above (#2389): the substitution must not + // wait on a cancel that may never settle. + releaseBodyDetached(response); console.warn( `[oauth] ${source} returned RFC 8414 OAuth 2.0 authorization server ` + `metadata, not an OpenID provider document. The MCP TypeScript SDK ` + diff --git a/core/auth/secret-storage-info.ts b/core/auth/secret-storage-info.ts index c63220e2c..2b9b84d61 100644 --- a/core/auth/secret-storage-info.ts +++ b/core/auth/secret-storage-info.ts @@ -45,9 +45,9 @@ export interface SecretStorageInfo { * True when the secrets file is *currently* in the clear. Read off the * file's own envelope rather than off whether a passphrase is * configured, because those two disagree for a whole session: adding - * `MCP_INSPECTOR_SECRET_KEY` to an install that already has a plaintext - * file makes the next write encrypt, while the existing bytes stay - * readable until then. Reporting the intent would tell that user their + * `MCP_INSPECTOR_SECRET_KEY` (or `MCP_INSPECTOR_SECRET_KEY_FILE`) to an + * install that already has a plaintext file makes the next write encrypt, + * while the existing bytes stay readable until then. Reporting the intent would tell that user their * secrets were encrypted while they were not. * * **File-only, and omitted entirely for the other kinds** — not "false". @@ -200,7 +200,7 @@ export function secretStorageCaveat( if (info.plaintext) { return info.pendingEncryption ? "Existing secrets in this file are still unencrypted (file mode 0600). They are re-encrypted the next time a secret is saved." - : "Secrets are stored unencrypted (file mode 0600). Set MCP_INSPECTOR_SECRET_KEY to encrypt them."; + : "Secrets are stored unencrypted (file mode 0600). Set MCP_INSPECTOR_SECRET_KEY or MCP_INSPECTOR_SECRET_KEY_FILE to encrypt them."; } return undefined; } diff --git a/core/json/jsonUtils.ts b/core/json/jsonUtils.ts index 527f626e6..aee0986ec 100644 --- a/core/json/jsonUtils.ts +++ b/core/json/jsonUtils.ts @@ -1,4 +1,5 @@ import type { Tool } from "@modelcontextprotocol/client"; +import { inlineLocalRefs } from "./localRefs.js"; import { normalizeNullableUnion } from "./nullableUnion.js"; import { narrowBySuppliedNames, @@ -317,8 +318,10 @@ export function convertParametersForSchema( // A property's schema can live on a root composition branch rather than on // the root itself (#2123); see `coercionProperties` for how the branch is // identified when it does. + // Same-document `$ref`s are inlined first: a property declared as a bare + // `$ref` has no `type` of its own to convert by (#2321). const { base, branches } = resolveRootUnion( - (inputSchema ?? {}) as RootUnionSchema, + inlineLocalRefs(inputSchema ?? {}) as RootUnionSchema, ); const properties = coercionProperties(base, branches, params); for (const [key, value] of Object.entries(params)) { diff --git a/core/json/localRefs.ts b/core/json/localRefs.ts new file mode 100644 index 000000000..905e399a2 --- /dev/null +++ b/core/json/localRefs.ts @@ -0,0 +1,301 @@ +/** + * Inlining of same-document `$ref`s (`#/$defs/…`, `#/definitions/…`) into the + * schema that uses them, so a form builder sees the referent's `type`. + * + * Every form builder here — the web `SchemaForm`, the TUI's `schemaToForm` — + * and the argument conversion in {@link ./jsonUtils.ts} dispatch on a + * property's own `type`. Zod → JSON Schema converters deduplicate a schema + * instance used twice by emitting it once under `$defs` and pointing both + * uses at it with a bare `$ref`, which carries no `type` at all. So the second + * of two fields sharing `z.string().regex(…)` fell through to the raw JSON + * editor, and a date typed into it was rejected as invalid JSON and dropped + * from the call (#2321). Resolving here, once, keeps the three consumers from + * disagreeing about which fields are strings. + * + * Deliberately narrow: + * - **Local pointers only.** A remote or relative `$ref` names a document this + * code cannot fetch, and is left in place (the field keeps its JSON editor). + * - **Unresolvable pointers are left in place**, for the same reason. + * - **Recursive references stop at the recursion.** A `$ref` to a schema that + * is already being inlined above it is kept as a `$ref`, so a tree type + * renders its first level and edits the rest as JSON rather than looping. + * - **Only annotation siblings are merged.** `{ $ref, description }` is exactly + * what `.optional().describe(…)` on a shared instance produces, and the + * description written at the use site is the one the user should see. A + * sibling that *constrains* (`enum`, `minLength`, …) applies in conjunction + * with the referent rather than replacing it, which a merge cannot express — + * so a `$ref` carrying one is left unresolved rather than loosened. + * - **Embedded resources are left alone.** A nested `$id` starts a new base + * URI, and a `#/…` pointer beneath it means that resource's root, not the + * document's. Rather than track bases, nothing under a nested `$id` is + * resolved. + * - **Expansion is bounded.** Inlining copies a referent at every use, so a + * chain of definitions each using the previous one twice grows as `2^n` + * from an `O(n)` schema. A server controls the schema, so past + * {@link EXPANSION_BUDGET} nodes — or {@link MAX_DEPTH} levels of nesting, + * which would otherwise overflow the stack — the whole schema is returned + * unresolved rather than freezing or crashing the form that renders it. + */ + +/* + * Where subschemas live, and nowhere else. Every other keyword's value is data + * — `const`, `enum`, an `x-vendor` extension — and is copied untouched even + * when it happens to hold a `$ref`-shaped object. Same lists as + * `schemaLint.ts`. + */ + +/** Keywords whose value is one subschema (or, for draft-04 `items`, an array). */ +const SUBSCHEMA_KEYWORDS = new Set([ + "items", + "contains", + "not", + "propertyNames", + "if", + "then", + "else", + "additionalProperties", + "unevaluatedProperties", + "additionalItems", + "unevaluatedItems", + "contentSchema", +]); + +/** Keywords whose value is an array of subschemas. */ +const SUBSCHEMA_ARRAY_KEYWORDS = new Set([ + "allOf", + "anyOf", + "oneOf", + "prefixItems", +]); + +/** + * Keywords whose values map arbitrary NAMES to subschemas. Their keys are + * user-chosen, so a property called `default` is a schema, not data. + */ +const SUBSCHEMA_MAP_KEYWORDS = new Set([ + "properties", + "patternProperties", + "dependentSchemas", + // Pre-2019 spelling of `dependentSchemas`; its array values are property + // name lists, which are not schemas and pass through untouched. + "dependencies", + "$defs", + "definitions", +]); + +/** + * Keywords that may sit beside a `$ref` without blocking its inlining: they + * annotate or organize, and constrain nothing, so the use site's value can + * safely replace the referent's. + */ +const NON_CONSTRAINT_SIBLINGS = new Set([ + "title", + "description", + "default", + "examples", + "deprecated", + "readOnly", + "writeOnly", + // Non-standard labels for `enum` values, read by both form builders. + "enumNames", + "$comment", + "$schema", + "$id", + "$defs", + "definitions", +]); + +/** Most schema nodes one inlining may produce before it gives up. */ +export const EXPANSION_BUDGET = 10_000; + +/** + * Deepest subschema nesting either pass walks before it gives up — the same + * bound `schemaLint.ts` uses. Both passes recurse, and a server can nest far + * deeper than the call stack allows. + */ +export const MAX_DEPTH = 64; + +/** Thrown to unwind a traversal that hit a bound; the input is returned. */ +class Bail extends Error {} + +type JsonRecord = Record; + +interface Traversal { + root: JsonRecord; + /** References being inlined above the current node, for cycle detection. */ + active: Set; + remaining: number; +} + +/** Charge one produced node against the traversal's budget. */ +function spend(traversal: Traversal): void { + traversal.remaining -= 1; + if (traversal.remaining < 0) throw new Bail(); +} + +/** + * Rebuild `node` with `visit` applied to each subschema it holds directly, + * copying every other value untouched. + */ +function mapSubschemas( + node: JsonRecord, + visit: (child: unknown) => unknown, +): JsonRecord { + const result: JsonRecord = {}; + for (const [key, value] of Object.entries(node)) { + let next: unknown = value; + if (SUBSCHEMA_KEYWORDS.has(key)) { + next = Array.isArray(value) ? value.map(visit) : visit(value); + } else if (SUBSCHEMA_ARRAY_KEYWORDS.has(key) && Array.isArray(value)) { + next = value.map(visit); + } else if (SUBSCHEMA_MAP_KEYWORDS.has(key) && isRecord(value)) { + next = mapNames(value, visit); + } + define(result, key, next); + } + return result; +} + +/** A name → subschema map with `visit` applied to each value. */ +function mapNames( + map: JsonRecord, + visit: (child: unknown) => unknown, +): JsonRecord { + const result: JsonRecord = {}; + for (const [name, value] of Object.entries(map)) { + define(result, name, Array.isArray(value) ? value : visit(value)); + } + return result; +} + +/** + * `defineProperty`, not assignment: `__proto__` is a legal property name in a + * schema's `properties`, and assigning it would set the prototype instead. + */ +function define(target: JsonRecord, key: string, value: unknown): void { + Object.defineProperty(target, key, { + value, + writable: true, + enumerable: true, + configurable: true, + }); +} + +function isRecord(value: unknown): value is JsonRecord { + return typeof value === "object" && value !== null && !Array.isArray(value); +} + +/** The referent of a `#/…` JSON Pointer within `root`, or `undefined`. */ +function resolvePointer(root: unknown, ref: string): unknown { + if (!ref.startsWith("#")) return undefined; + let pointer: string; + try { + // The whole fragment is URI-decoded BEFORE it is split into tokens (RFC + // 6901 §6): `#%2F$defs%2FDate` is the pointer `/$defs/Date`, and + // `#/a%2Fb` is the path `a` → `b`. A literal `/` inside a name is `~1`. + pointer = decodeURIComponent(ref.slice(1)); + } catch { + return undefined; + } + if (pointer === "") return root; + if (!pointer.startsWith("/")) return undefined; + let current: unknown = root; + for (const token of pointer.slice(1).split("/")) { + // `~0` and `~1` are the only escapes RFC 6901 defines; anything else makes + // the pointer malformed rather than naming a key that happens to match. + if (/~(?![01])/.test(token)) return undefined; + // `~1` before `~0`, so `~01` decodes to `~1`. + const segment = token.replace(/~1/g, "/").replace(/~0/g, "~"); + if (Array.isArray(current)) { + const index = Number(segment); + if (!/^(0|[1-9]\d*)$/.test(segment) || index >= current.length) { + return undefined; + } + current = current[index]; + } else if (isRecord(current) && Object.hasOwn(current, segment)) { + current = current[segment]; + } else { + return undefined; + } + } + return current; +} + +/** Whether any subschema in `node` holds a `$ref` string. */ +function containsRef(node: unknown, depth: number): boolean { + if (depth > MAX_DEPTH) throw new Bail(); + if (!isRecord(node)) return false; + if (typeof node.$ref === "string") return true; + let found = false; + mapSubschemas(node, (child) => { + found ||= containsRef(child, depth + 1); + return child; + }); + return found; +} + +function inline(node: unknown, traversal: Traversal, depth: number): unknown { + if (depth > MAX_DEPTH) throw new Bail(); + if (!isRecord(node)) return node; + // An embedded resource: its pointers are relative to itself (see the header). + if (node !== traversal.root && typeof node.$id === "string") return node; + spend(traversal); + const visit = (child: unknown) => inline(child, traversal, depth + 1); + + const ref = node.$ref; + const siblingsConstrain = Object.keys(node).some( + (key) => key !== "$ref" && !NON_CONSTRAINT_SIBLINGS.has(key), + ); + if ( + typeof ref === "string" && + !siblingsConstrain && + !traversal.active.has(ref) + ) { + const target = resolvePointer(traversal.root, ref); + if (isRecord(target)) { + traversal.active.add(ref); + const resolved = visit(target) as JsonRecord; + traversal.active.delete(ref); + const siblings: JsonRecord = { ...node }; + delete siblings.$ref; + return { ...resolved, ...mapSubschemas(siblings, visit) }; + } + } + return mapSubschemas(node, visit); +} + +// Keyed on the input object: form panels call this on every render, and a +// fresh tree each time would defeat anything downstream keyed on identity. +const cache = new WeakMap(); + +/** + * `schema` with every resolvable same-document `$ref` replaced by its referent. + * + * Returns `schema` itself — same reference — when it contains no `$ref`, or + * when inlining would exceed {@link EXPANSION_BUDGET} or {@link MAX_DEPTH}; + * and the same resolved object for repeated calls with the same input. Never + * mutates the input. + */ +export function inlineLocalRefs(schema: T): T { + if (!isRecord(schema)) return schema; + const cached = cache.get(schema); + if (cached !== undefined) return cached as T; + // The result is the input's own shape with references expanded, so it is + // still a `T` to every caller that reads it as one. + let resolved: T; + try { + resolved = containsRef(schema, 0) + ? (inline( + schema, + { root: schema, active: new Set(), remaining: EXPANSION_BUDGET }, + 0, + ) as T) + : schema; + } catch (error) { + /* v8 ignore next -- Bail is the only thing either traversal throws */ + if (!(error instanceof Bail)) throw error; + resolved = schema; + } + cache.set(schema, resolved); + return resolved; +} diff --git a/core/json/schemaLint.ts b/core/json/schemaLint.ts index e6d82c316..737cc675a 100644 --- a/core/json/schemaLint.ts +++ b/core/json/schemaLint.ts @@ -16,7 +16,12 @@ * conformance check would report nothing on essentially every real server. * What bites instead is the narrower subset each consumer accepts, which is * what the rules below encode. Every rule is a construct that is legal JSON - * Schema and is known to be refused or quietly mishandled by real MCP clients. + * Schema and is either known to be refused or quietly mishandled by real MCP + * clients, or — a weaker class, and only ever at `warning` severity — outside + * a documented schema dialect that consumers translate tool schemas into + * (`type-union`, #2286). A rule in the weaker class names that dialect at its + * call site, and moves to the stronger class only once a shipping client that + * mishandles it is recorded there. * * Kept pure and dependency-free so all three clients share one verdict: the * CLI's `--strict` report, the TUI's tool detail pane, and the web Tools tab @@ -465,6 +470,17 @@ function walk( }); } +/** + * Appended to the `type-union` suggestion when the union includes `null`. + * `anyOf` fixes the single-`type` problem, but OpenAPI 3.0 — the dialect the + * warning names — has no `null` type at all and spells nullability + * `nullable: true`, so the `{"type": "null"}` branch is not itself portable + * there (#2395 review). The suggestion stays JSON Schema rather than + * recommending `nullable`, which is not a JSON Schema keyword, and says so. + */ +const NULL_BRANCH_CAVEAT = + " A dialect with no `null` type, such as OpenAPI 3.0, still cannot express the `null` branch directly; there nullability is written `nullable: true`, which is not JSON Schema, so no single spelling is portable to both."; + /** Rules that apply to a single schema object, ignoring its children. */ function lintNode( node: SchemaRecord, @@ -482,17 +498,29 @@ function lintNode( // omission — a different contract, not the same one spelled portably. // `anyOf` branches each carrying a single `type` are equivalent, and this // lint treats them as portable. + // + // Deliberately a `warning` and worded as a trade, not a defect (#2286). + // The array form is what some model providers' own tool guidance + // recommends for a nullable field (OpenAI's structured outputs), so an + // author may be using it on purpose. What it costs is portability to a + // consumer that translates tool schemas into a single-`type` dialect — + // the OpenAPI 3.0 subset Gemini's function declarations use, where `type` + // is one enum value and nullability is `nullable: true`. A warning never + // fails the CLI's `--strict` exit code (only `error` does), so keeping the + // rule informs without turning a deliberate choice into a red CI job. add( ctx, "type-union", "warning", path, - `\`type\` is an array (${JSON.stringify(type)}). The array form is legal JSON Schema, but several MCP clients read \`type\` as a single string and either reject the tool or drop the constraint.`, + `\`type\` is an array (${JSON.stringify(type)}). This is legal JSON Schema, and some model providers recommend it for nullable fields, but it is less portable: a client that maps tool schemas onto a single-\`type\` dialect (such as the OpenAPI subset used for Gemini function declarations) may reject the tool or drop the constraint.`, `Split it into \`anyOf\` branches, each with a single \`type\` — \`{"anyOf": [${type .map((t) => `{"type": "${t}"}`) .join( ", ", - )}]}\`. (Making the property optional instead is a different contract: absent is not the same as \`null\`.)`, + )}]}\`. (Making the property optional instead is a different contract: absent is not the same as \`null\`.)${ + type.includes("null") ? NULL_BRANCH_CAVEAT : "" + }`, ); } diff --git a/core/mcp/node/notificationHeadersFetch.ts b/core/mcp/node/notificationHeadersFetch.ts new file mode 100644 index 000000000..0f9633b17 --- /dev/null +++ b/core/mcp/node/notificationHeadersFetch.ts @@ -0,0 +1,72 @@ +import { PROTOCOL_VERSION_META_KEY } from "@modelcontextprotocol/client"; +import { MODERN_PROTOCOL_VERSION } from "../types.js"; + +/** + * Wrap fetch so a modern-era JSON-RPC **notification** POST carries the + * SEP-2243 standard headers the SDK only stamps on requests (#2385). + * + * This is a compatibility workaround, not a protocol mandate. The 2026-07-28 + * Streamable HTTP spec defines no client-to-server notifications — closing the + * SSE stream is the cancellation signal, and "header requirements for + * notification POSTs are not defined by this revision". The SDK nonetheless + * POSTs a `notifications/cancelled` whenever a `subscriptions/listen` stream + * closes (every resource unsubscribe re-listens), and its + * `_applyBodyDerivedHeaders` stamps nothing on a non-request. Servers that + * apply their request-header validation to every POST — as SEP-2243's draft + * example did for notifications — refused it `400 Header mismatch: Mcp-Method + * is required`, failing the unsubscribe. Stamping the headers is harmless to a + * server that ignores them. + * + * This mirrors the SDK's own request rule exactly, so the two cannot disagree + * about era: the message's `_meta` protocol-version claim is the signal, and a + * message without a modern claim is passed through untouched — a legacy + * exchange never gains a 2026 header. `Mcp-Name` is not added: the spec + * defines it only for `tools/call`, `resources/read` and `prompts/get` + * requests. + * + * Remove once the SDK stops POSTing that notification on Streamable HTTP, or + * stamps it itself. + */ +export function createNotificationHeadersFetch( + baseFetch: typeof fetch, +): typeof fetch { + return (input, init) => { + const method = modernNotificationMethod(init); + if (method === undefined) return baseFetch(input, init); + const headers = new Headers(init?.headers); + headers.set("mcp-protocol-version", method.version); + headers.set("mcp-method", method.method); + return baseFetch(input, { ...init, headers }); + }; +} + +/** + * The method and modern protocol version of a single JSON-RPC notification in + * a POST body, or `undefined` for anything else (a request, a response, a + * batch, a legacy or unclaimed message, a non-string or non-JSON body). + */ +function modernNotificationMethod( + init: RequestInit | undefined, +): { method: string; version: string } | undefined { + if (init?.method?.toUpperCase() !== "POST") return undefined; + if (typeof init.body !== "string") return undefined; + let message: unknown; + try { + message = JSON.parse(init.body); + } catch { + return undefined; + } + if (!isRecord(message) || "id" in message) return undefined; + if (typeof message.method !== "string") return undefined; + const meta = isRecord(message.params) ? message.params._meta : undefined; + const version = isRecord(meta) ? meta[PROTOCOL_VERSION_META_KEY] : undefined; + // Dated revision tokens order lexically — the SDK's own era test. + if (typeof version !== "string" || version < MODERN_PROTOCOL_VERSION) { + return undefined; + } + return { method: message.method, version }; +} + +function isRecord(value: unknown): value is Record { + return typeof value === "object" && value !== null && !Array.isArray(value); +} diff --git a/core/mcp/node/suppressNotificationStreamFetch.ts b/core/mcp/node/suppressNotificationStreamFetch.ts new file mode 100644 index 000000000..2981b51d1 --- /dev/null +++ b/core/mcp/node/suppressNotificationStreamFetch.ts @@ -0,0 +1,81 @@ +/** + * Suppress the Streamable HTTP client's standalone `GET` notification stream + * (#2317). + * + * The SDK's `StreamableHTTPClientTransport` opens a long-lived `GET` SSE stream + * as soon as `notifications/initialized` is accepted, and exposes no option to + * skip it. Against a server that can serve only one request per client at a + * time, that stream occupies the only slot: `initialize` succeeds and every + * later request hangs until the per-request timeout fires (#2187). + * + * The transport already treats `405 Method Not Allowed` on that `GET` as "this + * server offers no standalone stream" and carries on with POST-only traffic, so + * answering the `GET` locally with a synthetic 405 — without sending it — + * reuses the SDK's own spec-conformant path (the client MAY open the stream; + * it is never required to). + * + * Only the *standalone* stream is suppressed, and the match is deliberately + * narrow because the SDK routes more than MCP traffic through this fetch: + * + * - **OAuth discovery.** The transport hands the same fetch to protected + * resource and authorization server metadata discovery, which are plain + * `GET`s to other URLs. Suppressing those would break SDK-managed auth. + * So the request must target the MCP endpoint itself and ask for + * `text/event-stream`. + * - **Resumption.** A `GET` carrying `Last-Event-ID` is the transport resuming + * a POST response stream that dropped mid-request. That belongs to + * request/response traffic and must keep reaching the server. + * + * Only the legacy (initialize-handshake) era opens this stream. A modern-era + * connection never sends it, so the wrapper is inert there. + */ +export function createSuppressNotificationStreamFetch( + baseFetch: typeof fetch, + endpoint: URL, +): typeof fetch { + return async (input, init) => { + if (isStandaloneStreamRequest(input, init, endpoint)) { + return new Response(null, { + status: 405, + statusText: "Method Not Allowed", + }); + } + return baseFetch(input, init); + }; +} + +function requestUrl(input: Parameters[0]): URL | undefined { + const raw = + input instanceof Request + ? input.url + : input instanceof URL + ? input.href + : input; + try { + return new URL(raw); + } catch { + return undefined; + } +} + +function isStandaloneStreamRequest( + input: Parameters[0], + init: Parameters[1], + endpoint: URL, +): boolean { + const request = input instanceof Request ? input : undefined; + const method = (init?.method ?? request?.method ?? "GET").toUpperCase(); + if (method !== "GET") return false; + const url = requestUrl(input); + if ( + !url || + url.origin !== endpoint.origin || + url.pathname !== endpoint.pathname || + url.search !== endpoint.search + ) { + return false; + } + const headers = new Headers(init?.headers ?? request?.headers); + const accept = headers.get("accept")?.toLowerCase() ?? ""; + return accept.includes("text/event-stream") && !headers.has("last-event-id"); +} diff --git a/core/mcp/node/transport.ts b/core/mcp/node/transport.ts index 08792f0c6..842997846 100644 --- a/core/mcp/node/transport.ts +++ b/core/mcp/node/transport.ts @@ -17,6 +17,8 @@ import { createAuthChallengeObserverFetch, } from "./authChallengeFetch.js"; import { createProxyFetch } from "./proxyFetch.js"; +import { createNotificationHeadersFetch } from "./notificationHeadersFetch.js"; +import { createSuppressNotificationStreamFetch } from "./suppressNotificationStreamFetch.js"; /** * Build the wire `headers` record from `settings.headers`, dropping rows with @@ -172,10 +174,24 @@ export function createTransportNode( ...(headers && { headers }), }; + // Both wrappers sit above the tracker. Header stamping, so the tracker + // records the headers actually sent (#2385); stream suppression outermost + // of all, so a suppressed GET is answered before it reaches the tracker — + // it is never sent, and the Network log should not show a request the + // server never saw (#2317). They touch disjoint requests (notification + // POSTs vs. the endpoint's SSE GET), so their relative order is free. + const stampedFetch = createNotificationHeadersFetch( + fetchWithOptionalAuthIntercept, + ); + const httpFetch = + settings?.suppressNotificationStream === true + ? createSuppressNotificationStreamFetch(stampedFetch, url) + : stampedFetch; + const transport = new StreamableHTTPClientTransport(url, { authProvider, requestInit, - fetch: fetchWithOptionalAuthIntercept, + fetch: httpFetch, // SEP-2350: how the transport reacts to a `403 insufficient_scope` // challenge. Defaults to the SDK's `reauthorize` when unset. ...(settings?.oauthOnInsufficientScope && { diff --git a/core/mcp/remote/node/server.ts b/core/mcp/remote/node/server.ts index 5257aec1c..dbeb87ce6 100644 --- a/core/mcp/remote/node/server.ts +++ b/core/mcp/remote/node/server.ts @@ -1957,6 +1957,16 @@ export function createRemoteApp( error: "settings.paginatedLists must be a boolean", }; } + // Optional on the wire; boolean when present, else unset (off) (#2317). + if ( + obj.suppressNotificationStream !== undefined && + typeof obj.suppressNotificationStream !== "boolean" + ) { + return { + ok: false, + error: "settings.suppressNotificationStream must be a boolean", + }; + } // maxFetchRequests is optional on the wire (older clients won't send it); // when present it must be a non-negative number (0 = unlimited), otherwise // it defaults below. @@ -2114,6 +2124,10 @@ export function createRemoteApp( autoRefreshOnListChanged: obj.autoRefreshOnListChanged === true, // Absent → false, matching the read side (omit-on-false on the write side). paginatedLists: obj.paginatedLists === true, + // Absent → unset (off); only an explicit true is carried (#2317). + ...(obj.suppressNotificationStream === true && { + suppressNotificationStream: true, + }), // Absent → product default, matching the read side. The default is the // omit-sentinel in inspectorSettingsToStoredFields, so a client that // didn't send one writes no spurious maxFetchRequests to disk. diff --git a/core/mcp/serverList.ts b/core/mcp/serverList.ts index a0b427006..9814008ed 100644 --- a/core/mcp/serverList.ts +++ b/core/mcp/serverList.ts @@ -163,6 +163,7 @@ type StoredInspectorFields = Pick< | "taskTtl" | "autoRefreshOnListChanged" | "paginatedLists" + | "suppressNotificationStream" | "advertisedExtensions" | "maxFetchRequests" | "skillCatalogMaxSkills" @@ -526,6 +527,7 @@ export function storedFieldsToInspectorSettings( stored.taskTtl !== undefined || stored.autoRefreshOnListChanged !== undefined || stored.paginatedLists !== undefined || + stored.suppressNotificationStream !== undefined || stored.advertisedExtensions !== undefined || stored.maxFetchRequests !== undefined || stored.skillCatalogMaxSkills !== undefined || @@ -576,6 +578,11 @@ export function storedFieldsToInspectorSettings( if (isSkillCatalogLimit(stored.skillCatalogMaxBytes)) { settings.skillCatalogMaxBytes = stored.skillCatalogMaxBytes; } + // Hand-edited non-boolean values are dropped (→ off) rather than coerced; + // only an explicit `true` suppresses the stream (#2317). + if (stored.suppressNotificationStream === true) { + settings.suppressNotificationStream = true; + } // Absent on disk reads back as the default era; the write side then omits the // default so a byte-stable round-trip never injects `protocolEra` into files // that never set it. An unknown literal from a hand-edited file is dropped @@ -717,6 +724,11 @@ export function inspectorSettingsToStoredFields( out.paginatedLists = true; } + // Persist only when enabled — absent reads back as unset (off) (#2317). + if (settings.suppressNotificationStream) { + out.suppressNotificationStream = true; + } + // Persist only when the user has toggled at least one extension override; // an empty map reads back as unset (above), keeping the diff minimal for the // common (no-override) case. @@ -848,6 +860,7 @@ const INSPECTOR_FIELD_KEY_MAP = { taskTtl: true, autoRefreshOnListChanged: true, paginatedLists: true, + suppressNotificationStream: true, advertisedExtensions: true, maxFetchRequests: true, skillCatalogMaxSkills: true, diff --git a/core/mcp/types.ts b/core/mcp/types.ts index f6a673036..d954b8a21 100644 --- a/core/mcp/types.ts +++ b/core/mcp/types.ts @@ -157,6 +157,13 @@ export type StoredMCPServer = MCPServerConfig & { * false (the default). (#1721) */ paginatedLists?: boolean; + /** + * When true, a legacy-era Streamable HTTP connection does not open the + * standalone `GET` notification stream. See + * {@link InspectorServerSettings.suppressNotificationStream}. + * Inspector-specific. Omitted on disk when false (the default). (#2317) + */ + suppressNotificationStream?: boolean; /** * Per-extension overrides for which extensions the Inspector advertises to * this server (keyed by extension id; a present key wins over the registry @@ -928,6 +935,25 @@ export interface InspectorServerSettings { * Default false. Server-wide; the per-list sidebar toggle edits this. (#1721) */ paginatedLists?: boolean; + /** + * When true, a Streamable HTTP connection on the **legacy** (initialize + * handshake) era does not open the standalone `GET` notification stream, + * which the client MAY open but is never required to (#2317). Two uses: a + * one-click diagnostic for a server that cannot serve a second concurrent + * request — which the long-lived stream otherwise occupies, hanging every + * request after `initialize` (#2187) — and an escape hatch that makes such a + * server inspectable. The cost is that server→client messages not carried on + * a request's own response stream (list_changed, resource updates, + * standalone logs) do not arrive. + * + * Two things it does not change. A `Last-Event-ID` `GET` resuming a dropped + * POST response stream is part of request/response traffic and still goes + * out. And a modern-era connection never opens the standalone stream (its + * notifications arrive over POST `subscriptions/listen`), so the setting has + * no effect there. Read at connect time; no effect on stdio or legacy SSE + * transports. Default false. + */ + suppressNotificationStream?: boolean; /** * Maximum number of HTTP fetch requests retained in the Network log for this * server. When exceeded, the oldest entries rotate out (and any deferred diff --git a/docs/docker.md b/docs/docker.md index 5b06c0f54..ae39754f0 100644 --- a/docs/docker.md +++ b/docs/docker.md @@ -42,44 +42,58 @@ docker run --rm -p 127.0.0.1:6274:6274 \ The same volume also persists OAuth tokens and stored state, so an authorized server stays authorized across runs. Use `-e MCP_CATALOG_PATH=/some/other/path.json` to put the catalog somewhere else — mount a volume covering whatever directory you point it at. If you **bind-mount a host directory** instead of a named volume (`-v "$PWD/inspector-data:/home/node/.mcp-inspector"`), the directory keeps its host ownership, so on Linux add `--user "$(id -u):$(id -g)"` or `chown` it to uid `1000` — otherwise the non-root `node` user can't write and adding a server fails with `EACCES`. -**Where secrets go, and how to make them survive (#1950).** The Inspector keeps the values it deliberately does _not_ write to `mcp.json` — an OAuth client secret, an enterprise IdP client secret, each stdio `env:` value — in the **OS keychain**. A container has no keychain (the published image has no D-Bus session), so on startup the Inspector probes for one and falls back, saying so in the logs and in a permanent footer at the bottom of the Client Settings and Server Settings dialogs. Which fallback you get depends on whether the directory it would write to is going to survive: - -| Situation | Store | Secrets survive a restart? | -| -------------------------------------------------------------- | -------------------------------------------- | -------------------------- | -| Keychain reachable (a normal desktop install) | OS keychain | Yes | -| Container, **no volume** on `/home/node/.mcp-inspector` | Memory | No — session only | -| Container **with** that volume, or any host without a keychain | `~/.mcp-inspector/secrets.json`, mode `0600` | Yes | - -So the same volume that keeps your server list also switches secrets from session-scoped to durable — nothing extra to configure. The in-memory default for an unmounted container is deliberate: a file in the writable layer is discarded by `--rm` and by every image update, and promising durability it can't deliver is worse than declining to. - -**A file-backed store is unencrypted unless you give it a key.** Set `MCP_INSPECTOR_SECRET_KEY` and the file is encrypted with AES-256-GCM (the passphrase is stretched with scrypt against a per-file random salt). Without it the file is still `0600`, but the values are readable to anyone who can read the file — which the startup log and the settings footer both say, every session, in a warning tone: - -```bash -docker run --rm -p 127.0.0.1:6274:6274 \ - -v mcp-inspector-data:/home/node/.mcp-inspector \ - -e MCP_INSPECTOR_SECRET_KEY="$MY_PASSPHRASE" \ - ghcr.io/modelcontextprotocol/inspector -``` - -**Use a high-entropy passphrase — generated, not chosen.** The random salt stops an attacker precomputing a table across files; it does nothing against _guessing_, and the scrypt cost is deliberately low because the derivation runs on every read and write. Anyone who obtains `secrets.json` can therefore test candidate passphrases quickly and offline, so treat this value like any other credential rather than like a memorable password. - -Setting the passphrase later is safe — the next write upgrades an existing plaintext file in place. Until that write happens the existing values really are still readable, and the banner and footer keep saying so rather than reporting the file as encrypted the moment the variable appears. **Changing or losing the passphrase is not safe**: a file that can no longer be decrypted is read as empty and _refuses to be written_, rather than being silently replaced with a new one holding only your latest secret. Restore the original passphrase, or delete `secrets.json` and re-enter the values. - -The Inspector writes the file `0600` and re-tightens it at startup if something loosened it. If it _cannot_ — the file belongs to another user, or the mount is read-only — it says so in the log rather than continuing to describe the file as protected, since on that box the mode claim above is not true. - -**Two Inspectors, one file.** Within a process, mutations are serialized per file path, so a web session's own concurrent saves cannot lose each other. Across processes — a CLI run beside a web session — each mutation takes an exclusive lock on `secrets.json.lock` for the whole read-modify-write, using [`proper-lockfile`](https://github.com/moxystudio/node-proper-lockfile) (the same library npm itself locks with). The lock expires 10 seconds after its holder stops refreshing it, so an Inspector that is killed mid-save does not leave the file unwritable. - -Two running Inspectors are therefore genuinely serialized. What a lock file cannot make single-winner is the *takeover of a lock whose holder died* — that needs a compare-and-swap on a directory entry (`renameat2`) which Node does not expose, and it is what an earlier hand-rolled attempt failed three review rounds on. `proper-lockfile` does not close that race either. The window opens only after a holder dies without releasing. - -The Inspector adds one thing on top: every lock-directory removal the library makes on its behalf — on release, and from its exit handler — is guarded by a check that the directory is still the one it created (by inode and birth time, which survive the library's own refresh but not a delete-and-recreate). That matters because those removals are otherwise unconditional, so a holder whose lock had been replaced would delete the *winner's* lock on the way out, turning one compromised writer into two unprotected ones. It also surfaces the takeover as a warning. Treat all of this as **best-effort**: the guard is still a check followed by an act, so it makes the destructive case rare rather than impossible, and it rests on filesystem metadata that not every filesystem reports. - -Which is why, underneath the lock, each mutation still reads the file, applies its change, writes, then reads back and compares the whole map; if something wrote in between it re-applies onto what was left and retries, failing loudly after five lost rounds rather than returning as though the value were saved. That check is what still catches a clobber inside that window — and it covers what no lock can, since a lock only orders the writers that *take* it: an editor, a restored backup, or an Inspector older than this release. - -If another process holds the lock and will not let go, the save **fails** rather than going ahead unlocked — waiting past the stale window first, so a crashed Inspector resolves itself rather than failing everyone else's saves. Writing alongside a writer you can see is the one case where degrading would lose the secret it was trying to protect. - -It is also what covers the lock being unavailable. This store exists for boxes where the usual mechanism isn't there, so a directory that can't hold a lock file — a read-only `$HOME`, a mount owned by another uid — makes the save proceed unlocked with a warning, rather than turning every `set` into a failure on exactly the deployments the store was written for. - -Three env vars affect where the file lands. `MCP_INSPECTOR_SECRET_STORE=keyring|file|memory` picks the store outright, bypassing the probe. `MCP_INSPECTOR_SECRET_FILE` names the file. Failing both, the file follows `MCP_STORAGE_DIR` — the same variable that relocates OAuth tokens and `client.json` — so mounting a volume at your configured storage directory is enough to make secrets durable there. These variables apply outside a container too; every runtime variable is listed in [Environment variables](./environment-variables.md). +**Where secrets go, and how to make them survive (#1950).** The Inspector keeps an OAuth client secret, an enterprise IdP client secret and each stdio `env:` value out of `mcp.json`, in the OS keychain. A container has no keychain (the published image has no D-Bus session), so it falls back, and which fallback you get depends on whether the secrets directory is going to survive: + +| Situation | Store | Secrets survive a restart? | +| ------------------------------------------------------- | -------------------------------------------- | -------------------------- | +| **No volume** on `/home/node/.mcp-inspector` | Memory | No — session only | +| **With** that volume | `~/.mcp-inspector/secrets.json`, mode `0600` | Yes | + +So the same volume that keeps your server list also switches secrets from session-scoped to durable — nothing extra to configure. The in-memory default for an unmounted container is deliberate: a file in the writable layer is discarded by `--rm` and by every image update, and promising durability it can't deliver is worse than declining to. The check looks at the **directory that holds the secrets file**, so if you relocate it with `-e MCP_STORAGE_DIR=…` or `-e MCP_INSPECTOR_SECRET_FILE=…`, mount a volume at that file's parent directory. Don't bind-mount the file on its own: it is not recognized as durable, so you get the memory store, and even with `-e MCP_INSPECTOR_SECRET_STORE=file` it cannot be written, because every save replaces the file by renaming a temporary file over it. + +> [!WARNING] +> **Mounting that volume turns on file storage of secrets, and without a key the file is plaintext.** Every OAuth client secret, IdP client secret and stdio `env:` value you save is then written to `secrets.json` on the volume, readable by anyone who can read the volume: root and every member of the `docker` group on the host, and anyone who gets a backup, snapshot or copy of it. Mode `0600` only keeps out other non-root users. +> +> **Give it a key, and keep that key only where the Inspector can read it.** Generate one into a file only you can read, outside the volume, backups and any repository that holds the secrets file: +> +> ```bash +> mkdir -p ~/.config/mcp-inspector +> (umask 077 && openssl rand -base64 32 > ~/.config/mcp-inspector/secret-key) +> ``` +> +> Then hand it to the container **as a file** with `MCP_INSPECTOR_SECRET_KEY_FILE`, not as an environment variable: +> +> ```bash +> docker run --rm -p 127.0.0.1:6274:6274 \ +> -v mcp-inspector-data:/home/node/.mcp-inspector \ +> -v "$HOME/.config/mcp-inspector/secret-key:/run/secrets/mcp_inspector_secret_key:ro" \ +> -e MCP_INSPECTOR_SECRET_KEY_FILE=/run/secrets/mcp_inspector_secret_key \ +> ghcr.io/modelcontextprotocol/inspector +> ``` +> +> Or with Compose secrets: +> +> ```yaml +> services: +> inspector: +> image: ghcr.io/modelcontextprotocol/inspector +> ports: ["127.0.0.1:6274:6274"] +> volumes: ["mcp-inspector-data:/home/node/.mcp-inspector"] +> environment: +> MCP_INSPECTOR_SECRET_KEY_FILE: /run/secrets/mcp_inspector_secret_key +> secrets: [mcp_inspector_secret_key] +> secrets: +> mcp_inspector_secret_key: +> file: ${HOME}/.config/mcp-inspector/secret-key +> volumes: +> mcp-inspector-data: +> ``` +> +> A key passed as a file stays out of `docker inspect`, the container's environment, your shell history and the Compose file. The container runs as uid `1000`, and without Swarm, Compose secrets are bind mounts that keep the host file's owner and mode, so the `0600` file must be owned by uid `1000`. On a Linux host where your uid is not `1000`, `sudo chown 1000 ~/.config/mcp-inspector/secret-key`; if the container can't read it, the log and the settings footer say the key file could not be read. Don't loosen the mode to make it readable instead: that hands the key to every other user on the host, and anyone who also gets a copy of the secrets file can then open it. `MCP_INSPECTOR_SECRET_KEY` still works, but a key passed that way is readable by anyone who can run `docker inspect` or `docker exec` against the container. If the key file is missing, unreadable or empty, `MCP_INSPECTOR_SECRET_KEY_FILE` is set to an empty value, or it is set together with a non-blank `MCP_INSPECTOR_SECRET_KEY`, the Inspector **refuses to read or write the secrets file** rather than falling back to plaintext, and says why in the log and the settings footer. +> +> **Even encrypted, secrets on disk carry moderate risk.** Encryption protects against the file leaking **on its own**. It does not protect against anyone who can also reach the key, which on a single host usually includes root and the `docker` group. Read [what the file store protects against](./secret-storage.md#what-the-file-store-protects-against) before relying on it. If that is not acceptable, don't mount the volume (secrets then stay in memory for the session), or run the Inspector outside a container, where it uses the OS keychain. + +⚠️ Keep supplying the **same** passphrase on every run: a file that can no longer be decrypted is read as empty and refuses to be written. Everything else about the store — the selection order, the file's location, encryption, permissions, locking, and choosing a store explicitly with `MCP_INSPECTOR_SECRET_STORE` — applies to every runtime and is in [Where secrets are stored](./secret-storage.md). **Upgrading from an image before this fix?** Earlier images did not create `/home/node/.mcp-inspector`, so Docker created the volume's mount point as `root` and the non-root `node` user couldn't write to it. An **empty** volume repairs itself on the first run of a current image (Docker applies the image directory's ownership to an empty volume), but one that already has files in it keeps its old `root` ownership and still fails with `EACCES`. Fix it once: diff --git a/docs/environment-variables.md b/docs/environment-variables.md index 0c7cf6ce6..cc25f2d13 100644 --- a/docs/environment-variables.md +++ b/docs/environment-variables.md @@ -16,7 +16,7 @@ These guard the web backend, which spawns processes on request. Read [Host bindi | --------------------------------- | -------- | ------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `MCP_INSPECTOR_API_TOKEN` | web, CLI | a random token per launch | Bearer token guarding every `/api/*` route (`x-mcp-remote-auth: Bearer `). Set it to use a known token instead of the generated one printed in the launch banner. The CLI reads it only to fill the `autoConnect` parameter of the deep link it emits. | | `MCP_PROXY_AUTH_TOKEN` | web | — | **Deprecated** v1 name for `MCP_INSPECTOR_API_TOKEN`, used only when the new name is unset. | -| `DANGEROUSLY_OMIT_AUTH` | web | unset | Disables the API token entirely. ⚠️ **Any non-empty value turns auth off, including `false` and `0`** — unset the variable to keep auth on. | +| `DANGEROUSLY_OMIT_AUTH` | web | unset | Disables the API token entirely when set to `true` or `1` (trimmed, case-insensitive). Any other value — including `false`, `0` and empty — keeps auth on. | | `HOST` | web, CLI | `127.0.0.1` | Address the web server binds. An all-interfaces host (`0.0.0.0`, `::`, an empty string, and equivalent spellings) is **refused** unless `DANGEROUSLY_BIND_ALL_INTERFACES` is enabled. The CLI reads it only to build its deep link. | | `DANGEROUSLY_BIND_ALL_INTERFACES` | web | off | Opts in to an all-interfaces `HOST`. Only `true` or `1` (case-insensitive) enable it, so `false` reads as off. The Docker image sets it. | | `ALLOWED_ORIGINS` | web | derived from `HOST` | Comma-separated origins allowed to call the API. Unset, the list follows `HOST` at `CLIENT_PORT`: the loopback origins for a loopback host, the loopback origins plus `http://0.0.0.0` and `http://[::]` for an all-interfaces bind, and otherwise only the configured host's own origin (so binding a LAN address does **not** also allow `localhost`). **Replaces** the default list rather than adding to it, so list every form you browse from. Each entry must include the scheme (`http://localhost:6274`). The same list is the MCP Apps sandbox proxy's embedder allow-list (its `frame-ancestors` header and its referrer check), so a public Inspector origin must be listed here for the Apps tab to render. | @@ -71,13 +71,17 @@ Every default above that starts with `~` is built from the home directory the pr ## Secret store -Where server secrets (headers, client secrets) are kept. The details — the keychain probe, the file format, encryption and locking — are in the [Docker guide](./docker.md); these variables apply to every install, not only containers. +Where the Inspector's secrets (OAuth client secrets, the enterprise IdP client secret, stdio `env:` values) are kept. How the store is chosen, and the details of the file store — its location, encryption, permissions and locking — are in [Where secrets are stored](./secret-storage.md); these variables apply to every install, not only containers. + +> [!WARNING] +> On a host with no OS keychain (Linux without libsecret or a Secret Service, headless or SSH sessions, Termux), the Inspector **automatically** stores secrets in a file that is **plaintext** unless `MCP_INSPECTOR_SECRET_KEY_FILE` or `MCP_INSPECTOR_SECRET_KEY` is set. See [the warning in Where secrets are stored](./secret-storage.md#how-the-store-is-chosen). | Variable | Read by | Default | Effect | | ---------------------------- | ------------- | --------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `MCP_INSPECTOR_SECRET_STORE` | web, CLI, TUI | probe the OS keychain | `keyring`, `file`, or `memory` (case-insensitive) picks the store outright and skips the probe. An empty or whitespace-only value counts as unset and silently runs automatic selection; any other value is ignored with a warning and also falls back to automatic selection. | | `MCP_INSPECTOR_SECRET_FILE` | web, CLI, TUI | `~/.mcp-inspector/secrets.json` | Path of the file store. Lookup order: this variable, then `secrets.json` in `MCP_STORAGE_DIR` when that is set, then `~/.mcp-inspector/secrets.json`. ⚠️ The default sits **beside** the storage directory, not inside it. | -| `MCP_INSPECTOR_SECRET_KEY` | web, CLI, TUI | unset (file is plaintext, `0600`) | Passphrase that encrypts the file store; an empty or whitespace-only value counts as unset. Use a generated, high-entropy value. ⚠️ Changing or losing it makes the existing file unreadable; see the Docker guide before rotating it. | +| `MCP_INSPECTOR_SECRET_KEY` | web, CLI, TUI | unset (file is plaintext, `0600`) | Passphrase that encrypts the file store; an empty or whitespace-only value counts as unset. Use a generated, high-entropy value. ⚠️ Changing or losing it makes the existing file unreadable; see [Where secrets are stored](./secret-storage.md#encryption) before rotating it. | +| `MCP_INSPECTOR_SECRET_KEY_FILE` | web, CLI, TUI | unset | Path of a file holding the passphrase; trailing line breaks are removed. Use this for Docker or Compose secrets, so the key stays out of the environment. Setting it together with a non-blank `MCP_INSPECTOR_SECRET_KEY` is an error, and so is setting it to an empty value. ⚠️ If the file is missing, unreadable or empty, or is the secrets file itself, the file store refuses to read or write rather than fall back to plaintext. | When no store is configured, the choice also depends on whether the Inspector is running in a container, which it detects from `KUBERNETES_SERVICE_HOST` (or Docker's and Podman's marker files). That variable is set by the orchestrator, not by you. diff --git a/docs/inspector-roadmap-2026-h2.md b/docs/inspector-roadmap-2026-h2.md index 64da1509c..62a0a3a49 100644 --- a/docs/inspector-roadmap-2026-h2.md +++ b/docs/inspector-roadmap-2026-h2.md @@ -6,45 +6,47 @@ **Horizon:** 2026-08-11 → 2027-02-11 (~26 weekly milestones, `v2.2.0` → ~`v2.27.0`) **Owner:** [Inspector V2 WG](https://modelcontextprotocol.io/community/working-groups/inspector-v2) -**Status:** Draft for WG review +**Status:** Draft for WG review — **revised 2026-09-17** against the published MCP roadmap of 2026-08-22 (#2400) + +## Work we can start now, no external blockers + +[Inspector Unblocked Work](https://claude.ai/artifact/MTFGsTVbKCqMchA1JYHo83) lists the roadmap items that depend on nothing outside this repo. --- ## Table of Contents +- [Work we can start now, no external blockers](#work-we-can-start-now-no-external-blockers) - [1. Why this document exists](#1-why-this-document-exists) - [2. The two tracks](#2-the-two-tracks) - [3. Track A — following the spec](#3-track-a--following-the-spec) - - [3.1 Transport evolution and scalability](#31-transport-evolution-and-scalability) - - [3.2 Server Cards](#32-server-cards) - - [3.3 Agent communication and Tasks](#33-agent-communication-and-tasks) - - [3.4 Enterprise readiness](#34-enterprise-readiness) - - [3.5 Triggers and events](#35-triggers-and-events) - - [3.6 Result type improvements](#36-result-type-improvements) - - [3.7 Interceptors](#37-interceptors) - - [3.8 File uploads](#38-file-uploads) - - [3.9 Skills over MCP](#39-skills-over-mcp) - - [3.10 Primitive grouping and tool annotations](#310-primitive-grouping-and-tool-annotations) - - [3.11 Conformance and validation](#311-conformance-and-validation) -- [4. Track B — experience work we choose](#4-track-b--experience-work-we-choose) - - [4.1 The zoomable timeline (headline)](#41-the-zoomable-timeline-headline) - - [4.2 Session record, replay, and share](#42-session-record-replay-and-share) - - [4.3 Diff and compare](#43-diff-and-compare) - - [4.4 Command palette and global search](#44-command-palette-and-global-search) - - [4.5 Saved calls and collections](#45-saved-calls-and-collections) - - [4.6 Assertions and CI flows](#46-assertions-and-ci-flows) - - [4.7 The argument editor workstream](#47-the-argument-editor-workstream) - - [4.8 Connection Doctor](#48-connection-doctor) - - [4.9 Server management and portability](#49-server-management-and-portability) - - [4.10 Workspace and layout](#410-workspace-and-layout) - - [4.11 Performance at scale](#411-performance-at-scale) - - [4.12 Accessibility and keyboard-first operation](#412-accessibility-and-keyboard-first-operation) - - [4.13 Onboarding](#413-onboarding) - - [4.14 Plugin architecture](#414-plugin-architecture) -- [5. Sequencing](#5-sequencing) -- [6. What we are deliberately not doing](#6-what-we-are-deliberately-not-doing) -- [7. Open questions](#7-open-questions) -- [8. Sources](#8-sources) + - [3.1 Agentic messaging primitives](#31-agentic-messaging-primitives) + - [3.2 HTTP-native transport unification and hardening](#32-http-native-transport-unification-and-hardening) + - [3.3 Agent identity and enterprise-ready security](#33-agent-identity-and-enterprise-ready-security) + - [3.4 Improved primitives](#34-improved-primitives) + - [3.5 Improved SDK developer experience](#35-improved-sdk-developer-experience) + - [3.6 Conformance and validation](#36-conformance-and-validation) + - [3.7 Off the published roadmap — watch only](#37-off-the-published-roadmap--watch-only) +- [4. Official extensions](#4-official-extensions) +- [5. Track B — experience work we choose](#5-track-b--experience-work-we-choose) + - [5.1 The zoomable timeline (headline)](#51-the-zoomable-timeline-headline) + - [5.2 Session record, replay, and share](#52-session-record-replay-and-share) + - [5.3 Diff and compare](#53-diff-and-compare) + - [5.4 Command palette and global search](#54-command-palette-and-global-search) + - [5.5 Saved calls and collections](#55-saved-calls-and-collections) + - [5.6 Assertions and CI flows](#56-assertions-and-ci-flows) + - [5.7 Observability export](#57-observability-export) + - [5.8 Connection Doctor](#58-connection-doctor) + - [5.9 Server management and portability](#59-server-management-and-portability) + - [5.10 Large servers: grouping and performance](#510-large-servers-grouping-and-performance) + - [5.11 Workspace and layout](#511-workspace-and-layout) + - [5.12 Accessibility and keyboard-first operation](#512-accessibility-and-keyboard-first-operation) + - [5.13 Onboarding](#513-onboarding) + - [5.14 Plugin architecture](#514-plugin-architecture) +- [6. Sequencing](#6-sequencing) +- [7. What we are deliberately not doing](#7-what-we-are-deliberately-not-doing) +- [8. Open questions](#8-open-questions) +- [9. Sources](#9-sources) --- @@ -54,7 +56,7 @@ Through v1, the Inspector was a **follow-along project**. The spec moved, we cha whatever planning capacity remained went to keeping up rather than to the tool's own design. Every release was reactive by necessity. -That constraint has lifted. v2 meets the 2026-07-28 spec across all three clients, on SDK v2, +That constraint has lifted. v2 meets the 2026-07-28 spec across all three clients (the open gaps are in official extensions, not the base protocol: the Tasks-extension `Mcp-Name` header, #1917, waits on an SDK release, and the rest are tracked in §4), on SDK v2, with a shared `core/`, a ≥90% per-file coverage gate, and a smoke/e2e apparatus that catches packaging failures. For the first time we can spend planned effort on **what the Inspector should be**, not only on what the spec just became. @@ -63,24 +65,47 @@ This document splits the next six months into those two kinds of work, so that n starves the other. The explicit intent is a **roughly even split of capacity** — spec-following work is non-negotiable but bounded, and the remaining capacity is ours to direct. -> **Sourcing note.** The MCP roadmap circulated as a Google Doc ("MCP Roadmap Process and -> Timeline") requires authentication and could not be read directly. This plan is built from -> the **published** roadmap at `modelcontextprotocol.io/development/roadmap` (last updated -> 2026-03-05) plus the current WG and IG charters, which together cover the same themes at -> more implementation-relevant detail. If the private doc contains timelines or themes absent -> from the public page, §3 should be revised against it before the plan is adopted. +> **Sourcing note.** The first draft (#1980) was written when the MCP roadmap could not be read +> directly, and was built from the 2026-03-05 public page plus WG charters. This revision (#2400) +> re-aligns §3 with the **published** roadmap at +> [`modelcontextprotocol.io/development/roadmap`](https://modelcontextprotocol.io/development/roadmap), +> last updated **2026-08-22**, which organizes the next spec cycle into five priority areas — +> §3.1 to §3.5 follow them one to one. The roadmap itself states it "reflects current thinking +> rather than firm commitments" and carries **no per-item dates**, only a "six to twelve months" +> window, so the phase placements in §6 remain our estimate. It also adds §4, a standing +> section for **official extensions**, which the roadmap does not list and which we must track +> separately. + +### Already shipped since the first draft + +Worth recording, because much of the first draft's "build now" list is done and should not +be re-planned: + +| Item | Issue(s) | +| ------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `Last-Event-ID` resumption (legacy Streamable HTTP only; the 2026-07-28 era removed SSE resumability) | [#920](https://github.com/modelcontextprotocol/inspector/issues/920) | +| `server.json` support | [#922](https://github.com/modelcontextprotocol/inspector/issues/922) | +| Discover checkmarks for task extensions | [#1887](https://github.com/modelcontextprotocol/inspector/issues/1887) | +| Tool-schema portability lint (`--strict`) | [#1005](https://github.com/modelcontextprotocol/inspector/issues/1005), [#1015](https://github.com/modelcontextprotocol/inspector/issues/1015) | +| The argument editor workstream (all six issues) | [#1853](https://github.com/modelcontextprotocol/inspector/issues/1853), [#1856](https://github.com/modelcontextprotocol/inspector/issues/1856), [#1885](https://github.com/modelcontextprotocol/inspector/issues/1885), [#1928](https://github.com/modelcontextprotocol/inspector/issues/1928), [#1919](https://github.com/modelcontextprotocol/inspector/issues/1919), [#1910](https://github.com/modelcontextprotocol/inspector/issues/1910) | +| Connection fixes (version-negotiation DX, dev containers, ghost entry) and self-signed `https://localhost` guidance (documented trust configuration, not a code fix) | [#962](https://github.com/modelcontextprotocol/inspector/issues/962), [#1936](https://github.com/modelcontextprotocol/inspector/issues/1936), [#1951](https://github.com/modelcontextprotocol/inspector/issues/1951), [#1914](https://github.com/modelcontextprotocol/inspector/issues/1914) | +| Server config: paste-JSON, custom headers, auth URL overrides, file-backed secrets | [#904](https://github.com/modelcontextprotocol/inspector/issues/904), [#1915](https://github.com/modelcontextprotocol/inspector/issues/1915), [#1906](https://github.com/modelcontextprotocol/inspector/issues/1906), [#1950](https://github.com/modelcontextprotocol/inspector/issues/1950) | +| IdP OIDC option (EMA itself, #1509, predates the first draft) | [#1937](https://github.com/modelcontextprotocol/inspector/issues/1937) | +| Skills over MCP (SEP-2640) across web, CLI and TUI | [#2234](https://github.com/modelcontextprotocol/inspector/issues/2234), [#2248](https://github.com/modelcontextprotocol/inspector/issues/2248) | + +Closed as **not planned**, so not carried forward: custom transports ([#1741](https://github.com/modelcontextprotocol/inspector/issues/1741)), the configurable-proxy base ([#1684](https://github.com/modelcontextprotocol/inspector/issues/1684)), the readiness summary ([#1916](https://github.com/modelcontextprotocol/inspector/issues/1916)), full panel collapse ([#928](https://github.com/modelcontextprotocol/inspector/issues/928)), `*.localhost` domains ([#1944](https://github.com/modelcontextprotocol/inspector/issues/1944)), and the trusted-local-host OAuth HTTP exception ([#1911](https://github.com/modelcontextprotocol/inspector/issues/1911)). --- ## 2. The two tracks -| | **Track A — Spec-following** | **Track B — Experience** | -| --------------------------- | ----------------------------------------------------- | ------------------------------------------- | -| **Driver** | MCP roadmap, WG deliverables, SEP acceptance | Our own judgment about the tool | -| **Trigger to start** | A SEP reaches Draft with a Tier-1 SDK reference impl | Whenever we have capacity | -| **Risk** | Slips when upstream slips; we cannot control the date | We control the date entirely | -| **Failure mode if starved** | Inspector stops being the reference test client | Inspector stays a protocol dump, not a tool | -| **Target capacity** | ~50% | ~50% | +| | **Track A — Spec-following** | **Track B — Experience** | +| --------------------------- | ----------------------------------------------------------------- | ------------------------------------------- | +| **Driver** | MCP roadmap, WG deliverables, SEP acceptance, approved extensions | Our own judgment about the tool | +| **Trigger to start** | A SEP reaches Draft with a Tier-1 SDK reference impl, or is Final; or an extension is approved as official (§4) | Whenever we have capacity | +| **Risk** | Slips when upstream slips; we cannot control the date | We control the date entirely | +| **Failure mode if starved** | Inspector stops being the reference test client | Inspector stays a protocol dump, not a tool | +| **Target capacity** | ~50% | ~50% | The two tracks are not independent. Several Track B items — the timeline, session record/replay, diff — are **force multipliers for Track A**: each new protocol feature @@ -90,227 +115,257 @@ general surfaces early so the spec work that lands later is cheap to display.** ### How the Inspector's role is changing -Worth stating plainly, because it shapes the priorities below. The roadmap's Validation -section names **conformance test suites**, **SDK tiers**, and **reference implementations** as -standing investments, and SEP-2484 now requires conformance tests for final SEPs. The -Inspector is the most visible MCP client in the ecosystem and is already the thing people -reach for when a server misbehaves. +Worth stating plainly, because it shapes the priorities below. The roadmap's SDK area makes +the **conformance test suite** the source of truth that SDKs and quickstarts are validated +against, and SEP-2484 (Final) requires conformance tests for Standards Track SEPs that change observable protocol behavior to reach +Final. The Inspector is the most visible MCP client in the ecosystem and is already the thing +people reach for when a server misbehaves. That points at an expanded role: not just _"show me the traffic"_ but _"tell me whether this -server is correct."_ Several items below (Server Card diffing, the conformance runner, -assertions, the readiness summary) are steps toward that, and they should be evaluated as a +server is correct."_ Several items below (the conformance runner, assertions, cache-hint +validation, the capability diff) are steps toward that, and they should be evaluated as a group rather than individually. --- ## 3. Track A — following the spec -Each subsection states the upstream theme, our read on what it means for the Inspector, and a -concrete feature list. **Confidence** flags how much of the list we can commit to now: +§3.1–§3.5 mirror the five priority areas of the published roadmap, in its order. Each states +the upstream area, our read on what it means for the Inspector, and a concrete feature list. +**Confidence** flags how much of the list we can commit to now: -- 🟢 **Build now** — the shape is known; blocked only on our own capacity. +- 🟢 **Build now** — the shape is known (the SEP is Final, or the work is ours alone); blocked only on our own capacity. - 🟡 **Design now, build on signal** — enough detail to design against; wait for a Draft SEP or a Tier-1 SDK impl before building. - 🔴 **Watch** — too early to predict a UI; keep a tracking issue and a WG liaison. - -### 3.1 Transport evolution and scalability - -**Upstream:** Transports WG. Next-generation Streamable HTTP that runs statelessly across -multiple instances and behaves correctly behind load balancers and proxies; a session model -covering creation, resumption, and migration; conformance guidance for SDK authors. The -roadmap is explicit that **no additional official transports** ship this cycle. - -**Read:** This is the theme most likely to produce breaking wire changes, and the one where -the Inspector is most useful — session resumption and proxy behavior are exactly the failures -nobody can reproduce by reading code. Our era model (`legacy` / `modern` / `auto`) already -gives us the negotiation seam to add a third era behind. - -| Feature | Confidence | Notes | -| ------------------------------------------------------------------------------------------------------------------------------------------------------------ | ---------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| **Session lifecycle lane** — session id, creation, resumption, migration, and expiry as first-class events, not log lines | 🟢 | Renders into the timeline (§4.1). Buildable against today's session model; extends to the new one. | -| **`Last-Event-ID` resumption support and display** | 🟢 | Existing gap — [#920](https://github.com/modelcontextprotocol/inspector/issues/920). Do it now; it is table stakes for the new session work. | -| **Proxy / intermediary harness** — route through a configurable proxy, then deliberately misbehave: rewrite headers, drop the GET stream, close mid-response | 🟡 | Builds on [#1684](https://github.com/modelcontextprotocol/inspector/issues/1684). Needs a `misbehaving-proxy` preset in `test-servers/`. | -| **Stateless-mode verification** — issue the same request across N synthetic instances and diff the responses | 🟡 | Directly tests the property the WG is specifying. Pairs with §4.3. | -| **Third protocol era behind the existing negotiation seam** | 🟡 | Cost is low _if_ we keep era-conditional exposure rather than replacing the legacy path. | -| **Custom transport support** | 🟢 | [#1741](https://github.com/modelcontextprotocol/inspector/issues/1741). The roadmap pushes experimentation to custom transports, so the Inspector should be able to load one. | - -### 3.2 Server Cards - -**Upstream:** Server Card WG, [SEP-2127](https://github.com/modelcontextprotocol/modelcontextprotocol/pull/2127) (Draft). A standard `.well-known` document exposing structured server metadata so browsers, crawlers, and registries can discover capabilities **without connecting**. Deliberately kept close to a subset of `server.json`. - -**Read:** This is the single highest-leverage Track A item for us, because it creates a new -Inspector capability rather than a new panel: **inspect before connect**. It also creates an -obvious correctness question that only a tool like ours can answer. - -| Feature | Confidence | Notes | -| ----------------------------------------------------------------------------------------------------------- | ---------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| **Card preview** — paste a URL, fetch the card, render the capability surface, one-click add to catalog | 🟡 | The pre-connection entry point. Wait for the format to settle. | -| **Card-vs-reality diff** — compare the advertised card against what `initialize` + `*/list` actually return | 🟡 | _The_ Inspector-shaped feature here. Nobody else in the ecosystem is positioned to check this. Shares machinery with [#1034](https://github.com/modelcontextprotocol/inspector/issues/1034) and §4.3. | -| **`mcp-inspector --card-lint `** — validate a card, non-zero exit on drift | 🟡 | CI-usable; a natural companion to the conformance runner (§3.11). | -| **`server.json` support** | 🟢 | [#922](https://github.com/modelcontextprotocol/inspector/issues/922). Prerequisite — the card is a subset, so this lands first regardless. | - -### 3.3 Agent communication and Tasks - -**Upstream:** Agents WG. Tasks (`io.modelcontextprotocol/tasks`, SEP-2663) is being -**stabilized and promoted from an extension into core**. Named open gaps: **retry semantics** -(what happens on transient failure, who decides to retry) and **expiry policies** (result -retention, how clients learn a result expired). An Agents Extension is under evaluation. - -**Read:** We already drive the modern Tasks extension ourselves over a raw-wire channel, -because SDK v2 era-gates `tasks/*` out. Promotion to core will move that back under the SDK — -plan for the migration, but **keep the era-conditional exposure**; the legacy `capabilities.tasks` -path must keep working. - -| Feature | Confidence | Notes | -| -------------------------------------------------------------------------------------------------------------------- | ---------- | ------------------------------------------------------------------------------------------- | -| **Retry visualization** — attempts, backoff, who initiated each retry | 🟡 | Design against the WG's gap list now. | -| **Expiry / TTL surfacing** — retention countdown on a completed task, distinct rendering for an expired-result error | 🟡 | Cheap once the semantics land; easy to get wrong if we guess early. | -| **Tasks as timeline spans** — a long-running task is a span, not a row | 🟢 | Falls out of §4.1 for free. The strongest argument for building the timeline first. | -| **Extension → core migration** | 🟡 | Retire the raw-wire channel when the SDK covers it; keep both paths during overlap. | -| **`Mcp-Name` header on Tasks over Streamable HTTP** | 🟢 | [#1917](https://github.com/modelcontextprotocol/inspector/issues/1917) — open bug, fix now. | -| **Discover checkmarks for task extensions** | 🟢 | [#1887](https://github.com/modelcontextprotocol/inspector/issues/1887). | - -### 3.4 Enterprise readiness - -**Upstream:** An Enterprise WG is expected to form. Four named areas: **audit trails and -observability**, **enterprise-managed auth** (Cross-App Access / ID-JAG), **gateway and proxy -patterns**, and **configuration portability**. Most output is expected as extensions rather -than core spec changes. Related: the Enterprise-Managed Authorization IG, and sponsored work -on [SEP-1932 (DPoP)](https://github.com/modelcontextprotocol/modelcontextprotocol/pull/1932) and [SEP-1933 (Workload Identity Federation)](https://github.com/modelcontextprotocol/modelcontextprotocol/pull/1933). - -**Read:** "Audit trails and observability, in a form enterprises can feed into their existing -pipelines" is a description of something the Inspector nearly already has. We hold the entire -session; we simply cannot **export** it in any pipeline-shaped format. That gap is cheap to -close and disproportionately valuable. - -| Feature | Confidence | Notes | -| ------------------------------------------------------------------------------------------------------ | ---------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| **OTLP export** — emit the session as OpenTelemetry spans; show trace/span ids inline; "copy as trace" | 🟢 | SEP-414 already puts trace context in `_meta`. Buildable today, no upstream dependency. | -| **Structured audit transcript** — the full session as a stable, documented JSON artifact | 🟢 | Shares its format with §4.2 record/replay. Build once, use for both. | -| **Machine-readable readiness summary** | 🟢 | [#1916](https://github.com/modelcontextprotocol/inspector/issues/1916). | -| **ID-JAG / Cross-App Access test flow** | 🟡 | The EMA IG exists specifically because this only works when IdP + client + AS interoperate. A test client is exactly what they lack. Related: [#1937](https://github.com/modelcontextprotocol/inspector/issues/1937), [#571](https://github.com/modelcontextprotocol/inspector/issues/571). | -| **DPoP and Workload Identity Federation** | 🔴 | Both sponsored but pre-acceptance. Watch; do not build. | -| **Gateway mode** — declare an intermediary, then show what we sent vs. what the gateway forwarded | 🟡 | Depends on the Gateways IG settling propagation semantics. | -| **Configuration portability** | 🟢 | [#1912](https://github.com/modelcontextprotocol/inspector/issues/1912), [#904](https://github.com/modelcontextprotocol/inspector/issues/904), plus `server.json` (§3.2). | - -### 3.5 Triggers and events - -**Upstream:** Triggers and Events WG. A standardized server→client callback mechanism -(webhooks or similar), with subscription lifecycle and **ordering guarantees that hold across -all transports**. Status: "SEP: Events in MCP v1 RFC" — **Ideating**. - -**Read:** ⚠️ **This is the largest architectural change on the horizon for us, and the one we -are least prepared for.** Every Inspector surface today assumes we are the party that -_initiated_ the connection. A webhook mechanism makes us a **server** — we must host a -publicly reachable callback endpoint, which for a tool that usually runs on `localhost` is a -real problem (tunnels, port forwarding, or a relay). - -We should start the design conversation **now**, well ahead of the SEP, and bring it to the -WG as implementation feedback. The ordering-guarantee requirement in particular is -untestable without a client that records arrival order — which is us. - -| Feature | Confidence | Notes | -| ----------------------------------------------------------------------------------------- | ---------- | ------------------------------------------------------------------------------------------------------------------------------------------------ | -| **Callback receiver** — backend-hosted endpoint, its URL registered as the trigger target | 🔴 | Needs design now, build later. Security review mandatory: an inbound public endpoint on a process that spawns subprocesses is a serious surface. | -| **Local reachability story** — tunnel integration or documented guidance | 🔴 | Likely the hardest UX problem of the whole six months. | -| **Delivery log with ordering and duplicate assertions** | 🔴 | The conformance value: did events arrive in the promised order? were any redelivered? | - -### 3.6 Result type improvements - -**Upstream:** "On the Horizon." **Streamed results** (incremental output for generated text, -audio, video frames) and **reference-based results** (client decides when to pull a large -payload into context). Explicitly cross-cutting — streaming touches transport, references -touch the schema. - -**Read:** Streaming changes how every result panel renders: today we display a _result_, and -we would need to display a _stream that becomes a result_. Worth a rendering abstraction -before the SEP, not after. - -| Feature | Confidence | Notes | -| ------------------------------------------------------------------------------------------------------- | ---------- | ----------------------------------------------------------------------------------- | -| **Incremental result rendering** — progressive display, with time-to-first-chunk and inter-chunk timing | 🔴 | The timing view is Inspector-shaped; the timeline is the natural home. | -| **Reference-result handling** — show a handle plus an explicit "pull payload", with size accounting | 🔴 | Also a good default for large payloads _today_, independent of the SEP (see §4.11). | - -### 3.7 Interceptors - -**Upstream:** Interceptors WG, [SEP-1763](https://github.com/modelcontextprotocol/modelcontextprotocol/pull/2076) (Draft). Interceptors as a new primitive with two types — **validators** (pass/fail) and **mutators** (transform payloads) — across in-process, sidecar, and remote deployment models, with priority-based chain ordering and audit-mode semantics. A **CLI client for interceptor invocation and testing** is a listed WG deliverable (Ideating, unowned). - -**Read:** Two things stand out. First, "CLI client for interceptor invocation and testing" is -**an unclaimed deliverable that describes our CLI**. Worth raising with the WG — Ola co-leads -both groups, so the liaison already exists. Second, an interceptor chain is a -_before → after payload transformation_, which is a diff, which we should already be able to -render (§4.3). - -| Feature | Confidence | Notes | -| ---------------------------------------------------------------------------------------------------------- | ---------- | ---------------------------------------------------------------------------------------------------------------------- | -| **Interceptor test bench** — register a chain, show before/after diff per hop, visualize priority ordering | 🟡 | The clearest "Inspector as the reference tool" opportunity of the six months. | -| **Audit-mode rendering** — what _would_ have been blocked or mutated | 🟡 | Follows the SEP's audit semantics. | -| **CLI interceptor invocation** | 🟡 | **Action: raise with the Interceptors WG.** If we take it, it needs its own milestone allocation. | -| **Our plugin architecture as an interceptor host** | 🟡 | [#1025](https://github.com/modelcontextprotocol/inspector/issues/1025). Prevents us building two extension mechanisms. | - -### 3.8 File uploads - -**Upstream:** File Uploads WG, [SEP-2356](https://github.com/modelcontextprotocol/modelcontextprotocol/pull/2356) (Draft, TS SDK reference impl targeted End May). Declarative `FileInputDescriptor` on tool input schemas and elicitation schemas, so hosts render native file pickers. Success criteria explicitly include **"at least one production host rendering a native file picker from the descriptor."** - -**Read:** The most tractable Track A item on the list — narrow, well-specified, with a TS SDK -reference implementation coming, and we are a credible candidate for that "production host." -It touches three surfaces: `SchemaForm` (Tools), elicitation forms, and MCP Apps. - -| Feature | Confidence | Notes | -| ------------------------------------------------------------------------------------ | ---------- | ------------------------------------------------ | -| **File picker in `SchemaForm`** when a descriptor is present, with data-URI encoding | 🟡 | Wait for the TS SDK types, then build. Low risk. | -| **Same in elicitation forms** | 🟡 | Shared component. | -| **Size guardrails and host-side validation** | 🟡 | The SEP references OWASP ASVS V5. | - -### 3.9 Skills over MCP - -**Upstream:** Skills Over MCP WG, [SEP-2640](https://github.com/modelcontextprotocol/modelcontextprotocol/pull/2640) (In Review, Extensions Track). Resources-based; a reference implementation is also In Review. - -**Read:** Because it is Resources-based, the incremental cost is low — a Skills view over the -existing resource machinery rather than a new subsystem. - -| Feature | Confidence | Notes | -| -------------------------------------------------------- | ---------- | -------------------------------------------------------------------- | -| **Skills view** — list, preview content, show activation | 🟡 | Gate on the negotiated extension, the way the Tasks tab gates today. | - -### 3.10 Primitive grouping and tool annotations - -**Upstream:** Two IGs. **Primitive Grouping** explores organizing Tools/Resources/Prompts -beyond flat lists — deliberately not picking one canonical pattern early. **Tool Annotations** -is consolidating six independent annotation SEPs and considering runtime annotations and tool -_response_ annotations. - -**Read:** Grouping is the rare case where the spec-following work and the UX work are the same -work. Flat lists are already our weakest surface on large servers — [#1957](https://github.com/modelcontextprotocol/inspector/issues/1957) (duplicate tool names) was a symptom. **Build the grouped sidebar as a UX -improvement now**, and adopt whatever grouping the IG lands as a data source later. - -| Feature | Confidence | Notes | -| ------------------------------------------------------------------ | ---------- | --------------------------------------------------------------------------------------------------------------------- | -| **Grouped / tree sidebars with group-aware search** | 🟢 | Build now on client-side heuristics (name prefixes, annotations). Ship value immediately; swap the data source later. | -| **Richer annotation rendering** | 🟢 | Extends the existing `AnnotationBadge`. | -| **Annotation-driven confirmation** before a `destructiveHint` call | 🟢 | Small, obviously correct, no upstream dependency. | -| **Runtime / response annotations** | 🔴 | Watch. | - -### 3.11 Conformance and validation - -**Upstream:** Standing investment — conformance test suites, SDK tiers ([SEP-1730](https://github.com/modelcontextprotocol/modelcontextprotocol/issues/1730)), reference implementations. [SEP-2484](https://github.com/modelcontextprotocol/modelcontextprotocol/pull/2484) now **requires conformance tests for final SEPs**, and the EMA IG is explicitly contributing scenarios to the `modelcontextprotocol/conformance` repository. +- ✅ **Shipped** — already in the Inspector; listed for completeness, not scheduled. + +### 3.1 Agentic messaging primitives + +**Upstream:** Triggers & Events, Agents, and Transports WGs. Messaging beyond +request/response: work that runs for minutes, servers that push, results that stream, and +steering work mid-flight. This period: **server-initiated events** ("channels and +subscriptions for push delivery, including webhooks") and a **composition review** so Tasks, +triggers, `subscriptions/listen` and progress notifications share "a lifecycle, a cancellation +model, [and] an error surface". **Beyond this period:** Tasks (SEP-2663) toward eventual +inclusion in core. + +**Read:** Two changes from the first draft. First, **Tasks moving into core is no longer a +this-period item**, so the raw-wire Tasks channel stays for the whole horizon and its +retirement drops out of the plan. Second, the composition review names the exact thing a +timeline can show better than any list: three kinds of "not done yet" work side by side. That +argues for **one lane for in-flight work** rather than a tasks lane and a subscriptions lane. + +The webhook half remains the largest architectural change on the horizon for us. Every +Inspector surface assumes we initiated the connection; a webhook makes us a **server** that +must be publicly reachable, which a tool usually run on `localhost` is not. Start the design +conversation now and bring it to the WG as implementation feedback. + +| Feature | Confidence | Notes | +| -------------------------------------------------------------------------------------------------------------------------------------------------- | ---------- | ------------------------------------------------------------------------------------------------------------------------------------------------- | +| **In-flight work lane** — tasks, open `subscriptions/listen` streams and progress-reporting requests as spans on one timeline lane (§5.1) | 🟢 | `subscriptions/listen` and progress are in the 2026-07-28 spec; Tasks is the official `io.modelcontextprotocol/tasks` extension (§4). Makes composition gaps (mismatched cancellation, divergent errors) visible, which the WG can use. | +| **Cancellation and error comparison** — show how each in-flight kind ended (completed, cancelled, errored, server-closed) with the same vocabulary | 🟢 | A small, direct contribution to the composition review. | +| **Callback receiver** — backend-hosted endpoint registered as a push target | 🔴 | Design now, build when the SEP lands. Security review mandatory: an inbound public endpoint on a process that spawns subprocesses. | +| **Local reachability story** — tunnel integration or documented guidance | 🔴 | Likely the hardest UX problem of the six months. | +| **Delivery log with ordering and duplicate assertions** | 🔴 | The conformance value: did events arrive in order? were any redelivered? | +| **`Mcp-Name` header on Tasks over Streamable HTTP** | 🟡 | [#1917](https://github.com/modelcontextprotocol/inspector/issues/1917) — a current non-conformance: the fix is merged upstream but unreleased, so the pinned SDK still omits the header SEP-2663 requires. Waits on the next SDK release. | +| **Tasks extension → core migration** | 🔴 | Moved to "Beyond" upstream. Keep the era-conditional exposure; the legacy `capabilities.tasks` path must keep working. | + +### 3.2 HTTP-native transport unification and hardening + +**Upstream:** Transports WG. "The 2026-07-28 release made a remote MCP server a normal HTTP +workload." The goal is **one transport model**: **HTTP over stdio** (Streamable HTTP as the +single binding, possibly HTTP/2 over stdin/stdout for multiplexing) and **caching** — SEP-2549 +(Final) added `ttlMs` and `cacheScope` to list results and resource reads, with **ETags** next, +including for tool-call results. **Beyond:** standardized error handling across all surfaces, +capability scoping for tool lists after SEP-2575, and a secure way to hand servers +configuration. + +**Read:** For **modern** (2026-07-28) connections, the first draft's §3.1 (stateless Streamable +HTTP, session creation / resumption / migration) is **largely obsolete**: SEP-2575 (stateless) +and SEP-2567 (sessionless) are Final and already shipped, so a session lifecycle lane has nothing +to show there. The Inspector is still a dual-era client, though, and **legacy** Streamable HTTP +keeps `initialize` and session-scoped state; a session lifecycle lane for legacy connections stays +a valid, **deferred** timeline follow-up (§5.1) rather than being dropped. Caching, on the other hand, is +Final and we already parse the fields — we just do not render them, and a client that shows +cache hints is exactly how a server author finds out theirs are wrong. + +HTTP over stdio would change how every stdio server connects, and our transport layer is +where the Inspector is thinnest over the SDK. Watch closely. + +| Feature | Confidence | Notes | +| ------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| **Cache hint display** — `ttlMs` / `cacheScope` on the SEP-2549 surfaces (`tools/list`, `prompts/list`, `resources/list`, `resources/templates/list`, `resources/read`), plus modern (2026-07-28+) `skills/list` and `skills/get`, which the stable ext-skills spec requires to carry both fields (our `skills/get` validation still treats them as optional: [#2404](https://github.com/modelcontextprotocol/inspector/issues/2404)); legacy results do not require the fields but are shown when a server sends them, with freshness countdown and "stale" marking | 🟢 | SEP-2549 is Final. The runtime parses the hints everywhere and honors them through the SDK cache for the four `*/list` methods; `resources/read`, `skills/list` and `skills/get` go through plain requests that validate but do not honor them, so this item includes that plumbing as well as the display. | +| **Cache behavior observations** — note a re-fetch of a still-fresh result, and a list that changed inside its declared TTL, as diagnostics rather than errors | 🟢 | Inspector-shaped: nobody else observes both the hint and the reality. `ttlMs` is a freshness hint, so both are compliant. | +| **Stateful-tool workflow investigation** — how to help a user carry an SEP-2567-style handle from one tool result into the next call | 🟡 | Replaces the first draft's "session lifecycle lane". The protocol has no concept of a handle (it is ordinary tool data), so a generic view would be inference; investigate before designing. | +| **ETag support** — send `If-None-Match`, show 304s and version changes | 🔴 | Watch until a SEP reaches Draft with an SDK impl. | +| **HTTP over stdio** | 🔴 | Watch. If it lands, the Network screen becomes meaningful for stdio servers too — a large win. | +| **Standardized error rendering** | 🔴 | "Beyond". Our Protocol-vs-Network error split (#1628) is the seam to adopt it into. | + +### 3.3 Agent identity and enterprise-ready security + +**Upstream:** Agent Identity WG (forming this period), coordinated with the IETF OAuth and +WIMSE WGs. MCP authorization assumes a person at a browser; increasingly the caller is an +agent. This period: **finalize DPoP** and drive adoption; an opinionated **agent identity and +delegation** model built on **Workload Identity Federation** ([SEP-1933](https://github.com/modelcontextprotocol/modelcontextprotocol/pull/1933)), **ID-JAG** as used by +Enterprise-Managed Authorization, and **RFC 8693 token exchange**. **Beyond:** +human-presence attestation. + +**Read:** DPoP was 🔴 in the first draft and is now a named deliverable, so it moves up. Our +EMA work (#1509) already gives us the ID-JAG leg, which makes the Inspector a credible test +client for the whole identity chain. The first draft's audit trails and gateway mode are **no longer on the MCP roadmap**, and +configuration ("providing servers with configuration options in a secure way") is now a +"Beyond" item (§3.2), outside this horizon; OTLP export and the audit +transcript are still worth building, but as our own Track B work (§5.7), not as spec-following. + +| Feature | Confidence | Notes | +| -------------------------------------------------------------------------------------------------- | ---------- | ----------------------------------------------------------------------------------------------- | +| **OAuth Client Credentials extension** — client-secret and JWT-bearer assertion flows | 🟢 | An **approved** official extension (§4) we do not support. No upstream dependency. | +| **Token exchange (RFC 8693) test flow** | 🟡 | Named in the roadmap; the RFC is stable, the MCP profile of it is not. | +| **DPoP** — generate a proof key, send `DPoP` proofs, show proof/nonce exchange in the Network view | 🟡 | Design against [SEP-1932](https://github.com/modelcontextprotocol/modelcontextprotocol/pull/1932); build when it is Final or has a Tier-1 SDK impl. | +| **Workload Identity Federation** | 🟡 | SEP-1933. Needs a way to present a workload credential from a developer machine — design first. | +| **Human-presence attestation** | 🔴 | "Beyond". | + +### 3.4 Improved primitives + +**Upstream:** Core Primitives WG (forming this period); File Uploads WG. This period: a +**`tools/call` result-shape redesign** to resolve the `content` vs `structuredContent` +confusion; **progressive discovery**, where clients learn tools and resources as needed +instead of ingesting the whole catalog, interacting with the caching work; and a review of +**primitive annotations** (audience and priority), which "most implementers haven't adopted" +and which may be deprecated. The File Uploads WG continues on **scoped file operations and +filesystem-like resource semantics** (range reads, hierarchical listing). + +**Read:** Every item here touches a panel we own. The result-shape redesign rewrites the tool +result view; progressive discovery breaks the assumption behind every list we render (that +`*/list` returns everything); and a possible annotation deprecation means we should not invest +in richer annotation rendering now. The first draft's §3.6 (streamed and reference results) +and §3.8 (the SEP-2356 file picker) are **not prioritized deliverables for this period** — the +roadmap mentions "results that stream" only in framing — so they move to watch. + +What we _can_ do now is show one concrete symptom of the problem the redesign is solving: a +server that returns `structuredContent` without the serialized-JSON text block the spec asks +for can hide its structured data from older clients today. + +| Feature | Confidence | Notes | +| ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------- | ------------------------------------------------------------------------------------------------------------------------ | +| **Serialized-JSON check for `structuredContent`** — when a result carries `structuredContent`, flag the absence of a `TextContent` block holding its serialized JSON, the one relationship the spec defines (a SHOULD, "for backwards compatibility"). Reported as a diagnostic, never an error; any other text is a legitimate summary and is not compared | 🟢 | Useful today, and implementation evidence for the Core Primitives WG. A missing `structuredContent` under a declared `outputSchema` is already flagged by `validateToolOutput` (shipped). | +| **New tool result shape** | 🔴 | WG still forming. Keep both renderings behind the era seam when it lands. | +| **Progressive discovery** | 🔴 | Design the lists (§5.10) so "not loaded yet" is a state, not an empty list. | +| **Annotation-driven confirmation** before a `destructiveHint` call | 🟢 | Tool annotations are not the audience/priority content annotations under review. Small and obviously correct. | +| **Richer audience / priority annotation rendering** | 🔴 | Paused: may be deprecated. | +| **Range reads and hierarchical resource listing** | 🟡 | We already render `resources/directory/read` for Skills (#2248); generalize it when the File Uploads WG publishes a SEP. | + +### 3.5 Improved SDK developer experience + +**Upstream:** SDK WG with the Core Maintainers. This period: **the extension contract** — +which role an extension binds (host, client, server, agent), what each does when the +capability is declared, what SDKs must support natively, packaging, and capability additions +as versioned changes; and **the generated-artifacts experiment** — generate a Tier-1 SDK and its +quickstarts from the spec, validated against the conformance suite. + +**Read:** The extension contract decides how we present extensions: today our capability view +lists advertised extension ids, and a contract that names roles and versions gives us +something to validate declarations against. The generated-artifacts experiment makes the +conformance suite central, which strengthens §3.6. + +| Feature | Confidence | Notes | +| ------------------------------------------------------------------------------------------------------------------------------ | ---------- | ----------------------------------------------------------------------------------------------- | +| **Extension declaration view** — for each advertised extension: identifier, settings object, whether the Inspector supports it | 🟢 | Buildable on today's negotiation (#1738); extend with role and version once the contract lands. | +| **Extension contract validation** | 🟡 | Validate a server's declaration against the contract once published. | +| **Run generated quickstart servers as fixtures** | 🔴 | If the experiment publishes them, they are free test servers. | + +### 3.6 Conformance and validation + +**Upstream:** Standing investment rather than a priority area — the conformance suite, SDK +tiers ([SEP-1730](https://modelcontextprotocol.io/seps/1730-sdks-tiering-system)), and +[SEP-2484](https://modelcontextprotocol.io/seps/2484-conformance-tests-required-for-final-seps) +(Final), which requires conformance tests for Standards Track SEPs that change observable protocol behavior to reach Final. §3.5 makes +the suite the validation target for generated SDKs. **Read:** A conformance suite needs a driver and a report. We are the natural driver, and we -already have a CLI that exits non-zero. This is the clearest path to the expanded role -described in §2 — and unlike most of Track A, **it is not gated on any SEP**. +already have a CLI that exits non-zero. The runner itself needs agreement with the suite's +maintainers on a programmatic interface; the **assertion engine** it would share with §5.6 +does not. + +| Feature | Confidence | Notes | +| ------------------------------------------------------------------------------------------------- | ---------- | --------------------------------------------------------------------------------------------------------------------------------------------------------- | +| **Conformance runner** — run the suite against a connected server, render pass/fail per assertion | 🟡 | **Action: open a conversation with the conformance maintainers.** Build the shared assertion engine (§5.6) first. | +| **`mcp-inspector --conformance` for CI** | 🟡 | Same engine, CLI report, exit code. | +| **Tool-schema portability lint (`--strict`)** — not a full JSON Schema validator | ✅ | Shipped — [#1005](https://github.com/modelcontextprotocol/inspector/issues/1005), [#1015](https://github.com/modelcontextprotocol/inspector/issues/1015). | + +### 3.7 Off the published roadmap — watch only + +The first draft planned build work for several WG efforts that are not priority deliverables in +the 2026-08-22 roadmap (some, such as streamed results, appear only in its framing). They are not cancelled upstream — WGs keep working outside the priority areas — but the +roadmap says SEPs outside those areas "expect a longer queue", so **we do not schedule build +work for them this horizon**. Each keeps a tracking issue and a liaison. + +| Effort | First-draft plan | Now | +| ----------------------------------------------------- | ------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------- | +| **Server Cards** (SEP-2127) | Card preview, card-vs-reality diff, `--card-lint` in Phase 3 | 🔴 Watch. [#1857](https://github.com/modelcontextprotocol/inspector/issues/1857)'s **registry** half does not depend on it (§5.9). | +| **Interceptors** ([SEP-2624](https://github.com/modelcontextprotocol/modelcontextprotocol/pull/2624); originally SEP-1763) | Test bench, audit mode, CLI invocation in Phase 4 | 🔴 Watch. The WG's unowned "CLI client for interceptor invocation" is still worth raising (§8). | +| **Primitive grouping** (IG) | Grouped sidebars | The **UX** half proceeds as Track B (§5.10) on client-side heuristics; no spec data source is expected this horizon. | +| **Streamed and reference results** | Incremental rendering, reference handles | 🔴 Watch. Planned payload truncation (§5.10) will cover the large-result case; result views render full payloads today. | +| **File picker from `FileInputDescriptor`** (SEP-2356) | `SchemaForm` + elicitation picker | 🔴 Watch. The File Uploads WG's published direction is now filesystem-like resources (§3.4). | +| **Gateways, audit trails** | Gateway mode; OTLP as spec work | Gateway mode dropped. OTLP and the audit transcript continue as Track B (§5.7). (Secure server configuration is not off the roadmap: it is a "Beyond" item, §3.2; our rich server configuration and registry browsing, #1857, continue in §5.9.) | + +--- -| Feature | Confidence | Notes | -| ------------------------------------------------------------------------------------------------- | ---------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| **Conformance runner** — run the suite against a connected server, render pass/fail per assertion | 🟡 | Needs coordination on the suite's programmatic interface. **Action: open a conversation with the conformance maintainers.** | -| **`mcp-inspector --conformance` for CI** | 🟡 | Same engine, CLI report, exit code. | -| **Strict schema validation with actionable errors** | 🟢 | [#1005](https://github.com/modelcontextprotocol/inspector/issues/1005), [#1015](https://github.com/modelcontextprotocol/inspector/issues/1015). No dependency; start here. | +## 4. Official extensions + +The MCP roadmap mentions Tasks (§3.1) but carries no inventory of official extensions, and **approved extensions are spec-following work** — +a client that ignores them stops being a reference client. The list lives at +[`/extensions/overview`](https://modelcontextprotocol.io/extensions/overview); implementations +are recorded in the community-maintained +[client matrix](https://modelcontextprotocol.io/extensions/client-matrix), and extensions reach +official status through the Extensions Track of +[SEP-2133](https://modelcontextprotocol.io/seps/2133-extensions), optionally after incubating in an +`experimental-ext-*` repository (encouraged, not required). + +### Current support (as of 2026-09-17) + +| Extension | Identifier | Web | CLI | TUI | Upstream matrix | Notes | +| -------------------------------- | ---------------------------------------------------------- | --- | --- | --- | ----------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| MCP Apps | `io.modelcontextprotocol/ui` | ✅ | — | — | Inspector row, cell blank | Apps tab. Columns are rendering support: rendering needs a browser, so neither CLI nor TUI renders Apps (the CLI does offer an `--app-info` metadata probe). The shared client still advertises the extension from CLI and TUI, which is a compatibility bug tracked in [#2403](https://github.com/modelcontextprotocol/inspector/issues/2403). | +| Tasks | `io.modelcontextprotocol/tasks` | 🟡 | 🟡 | 🟡 | No column in the matrix | Raw-wire channel; stays for the horizon (§3.1). All three clients are currently partial, for two separate reasons. (1) Over Streamable HTTP, the shared `InspectorClient` sends modern `tasks/*` requests without the `Mcp-Name` header SEP-2663 requires, so strict servers reject them; stdio is unaffected. It is fixed once the upstream SDK change tracked by #1917 is released. (2) CLI and TUI have no user-facing task surface: `mcp-inspector --cli` rejects `tasks/*` (they are not in `ONE_SHOT_METHODS`), and the TUI has no Tasks pane. #1917 does not change that. | +| Skills over MCP | `io.modelcontextprotocol/skills` | ✅ | ✅ | ✅ | "Partial" (CLI README) | [#2234](https://github.com/modelcontextprotocol/inspector/issues/2234), [#2248](https://github.com/modelcontextprotocol/inspector/issues/2248). Checks mean the inspection surface is complete (list, get, digest and frontmatter verification), with one open validation gap: modern `skills/get` results missing the now-required cache fields are still accepted ([#2404](https://github.com/modelcontextprotocol/inspector/issues/2404)). Host behaviors (activation, per-skill consent, content-bound approval) are out of scope by design, since the Inspector is not a host (`core/mcp/skills.ts`); that is also why the upstream matrix says "Partial". | +| Enterprise-Managed Authorization | `io.modelcontextprotocol/enterprise-managed-authorization` | ✅ | 🟡 | 🟡 | Inspector row, cell blank | [#1509](https://github.com/modelcontextprotocol/inspector/issues/1509). CLI and TUI have no in-client Client Settings surface; they consume the `client.json` / `mcp.json` and keychain state the web settings flows write (or hand-edited files), and terminal EMA follow-ups remain (`specification/v2_auth_ema.md`). | +| OAuth Client Credentials | `io.modelcontextprotocol/oauth-client-credentials` | ❌ | ❌ | ❌ | Inspector row, cell blank | **Gap** (§3.3). [#1225](https://github.com/modelcontextprotocol/inspector/issues/1225) was closed only because v1 is frozen. | + +**Actions:** implement OAuth Client Credentials; and, with maintainer sign-off, open a PR on +`modelcontextprotocol/modelcontextprotocol` to update the Inspector row. That matrix has one row per product, +so it cannot represent the Inspector's separate Web, CLI and TUI clients: mark Apps as partial with a link explaining the split (Apps renders in +Web only; the CLI has a metadata probe), or propose separate Web/CLI/TUI rows. Enterprise Auth can be a plain check; Skills stays "Partial" upstream, because the Inspector is not a host. The same PR should propose a **Tasks** column: Tasks is an official extension the matrix cannot currently represent at all. + +### Keeping up as extensions are approved + +We picked up Skills because someone noticed, not because anything told us. Make it a +mechanism, the way SDK releases already are: + +- **An extension-watch sweep**, modelled on `scripts/sdk-watch.mjs`: on a schedule, treat + `/extensions/overview` as the authoritative set of official extensions (Tasks, for one, has no + `ext-*` repository), enumerate the org's `experimental-ext-*` repositories to discover + experimental entries and `ext-*` repositories only to enrich official ones, and read each extension's identifier from its own specification or repository (the + overview lists names and links, not identifiers), and file one issue per entry it has not filed before. As in the SDK watch, the **issue markers + are the source of truth** for idempotency: an entry whose marker is on an existing issue (open or + closed) authored by the automation is skipped, so nothing needs committing back. It **files + issues, never PRs**. Two details are left to the sweep's own design issue: which labels a trusted + marker issue must also carry (as `sdk-watch` requires), and the first-run bootstrap for extensions + already tracked by hand-filed issues (Apps #1740, Tasks #1887, Skills #2234, EMA #1509), so that it does not file duplicates. OAuth Client Credentials is the exception: its only hand-filed issue, #1225, was closed on the frozen v1 line, so a new v2 issue is filed for it deliberately, cross-referencing #1225. +- **Official extension** → a `v2` + `enhancement` issue to implement it, filed with the current milestone as `sdk-watch` does; only when no dated milestone is open is it left unmilestoned for triage to place in Incoming. +- **Experimental extension** → a `v2` + `question` tracking issue, filed **unmilestoned and + unboarded** so triage places it in Incoming (the documented exception for unapproved work); it + gets a milestone only if a maintainer approves design work against it before its SEP. +- **This table is maintainer-maintained.** The sweep never edits it; a maintainer adds a row when + an extension's issue is triaged and moves its cells as support lands. --- -## 4. Track B — experience work we choose +## 5. Track B — experience work we choose -Nothing in this section waits on a SEP. Ordered by leverage, not by effort. +No item here waits on a SEP or another project to **start**. Some later parts depend on each other or on Track A (for example, cross-server timeline correlation needs §5.11, and the assertion engine is shared with §3.6). Ordered by leverage, not by effort. -### 4.1 The zoomable timeline (headline) +### 5.1 The zoomable timeline (headline) **Committed.** The single feature that most changes what the Inspector _is_. @@ -327,7 +382,7 @@ at a glance. Timeline become three renderings of one session. This keeps the coverage gate and the existing `protocolUtils` derivations intact. - **Lanes**, each independently collapsible: - `client → server` · `server → client` · notifications · tasks · subscription streams · OAuth/auth · errors + `client → server` · `server → client` · notifications · **in-flight work** (tasks, subscriptions, progress — §3.1) · OAuth/auth · errors - **Spans, not points.** A request occupies from send to response; a task occupies its whole lifetime; a stream is a bar with events on it. Duration becomes visible, which is most of the value. @@ -342,20 +397,20 @@ at a glance. - **Latency distribution** as a secondary view — per method, so a slow tool is obvious. - **Virtualized**, keyboard-navigable, and rendered from the same store the other views use. -**Deliberately out of scope for v1 of this feature:** cross-server correlation (needs §4.10), -and OTLP-shaped nesting (needs §3.4). +**Deliberately out of scope for v1 of this feature:** cross-server correlation (needs §5.11), +and OTLP-shaped nesting (needs §5.7). -### 4.2 Session record, replay, and share +### 5.2 Session record, replay, and share Save a complete session — protocol log, network log, server config, negotiated capabilities — to a single file. Reopen it later, on another machine, with no server running. Attach it to a bug report. This changes issue triage from "works on my machine" into an artifact, and it is the same -serialization format as the enterprise audit transcript (§3.4) — **build the format once**. -Replay also gives us fixtures: a recorded session is a regression test. +serialization format as the audit transcript (§5.7) — **build the format once**. Replay also +gives us fixtures: a recorded session is a regression test. -### 4.3 Diff and compare +### 5.3 Diff and compare Two sessions, or two servers, side by side. Concretely: @@ -364,166 +419,154 @@ Two sessions, or two servers, side by side. Concretely: - **Session diff** — same calls, two servers, what differed. - **Payload diff** — before/after for any pair of JSON documents. -The payload differ is a **shared primitive**: interceptor before/after (§3.7), Server -Card-vs-reality (§3.2), and stateless-instance comparison (§3.1) are all the same widget with -different inputs. Build it as a component first, then wire the three consumers. +The payload differ is a **shared primitive**: capability diff, session diff, the cache checks +(§3.2) and any later card-vs-reality or interceptor view are the same widget with different +inputs. Build it as a component first, then wire the consumers. -### 4.4 Command palette and global search +### 5.4 Command palette and global search `⌘K` to jump to any server, tool, resource, or prompt; re-run the last call; switch tabs. Plus full-text search across the protocol log with a real filter syntax (`method:tools/call status:error duration:>500ms`). The Inspector is currently a mouse-driven app; for a developer tool that is a daily tax. -### 4.5 Saved calls and collections +### 5.5 Saved calls and collections Name a tool call with its arguments, save it, re-run it, parameterize it, share it. A Postman-collection model for MCP. The single most requested shape of workflow improvement for -any protocol client, and it composes directly with §4.6. +any protocol client, and it composes directly with §5.6. -### 4.6 Assertions and CI flows +### 5.6 Assertions and CI flows Attach expectations to a saved call — result matches schema, field equals value, latency under a bound — and run the collection from the CLI with a non-zero exit on failure. This turns the Inspector from an interactive tool into part of a server author's test suite, and it shares an -engine with the conformance runner (§3.11). -Related: [#1005](https://github.com/modelcontextprotocol/inspector/issues/1005), [#1886](https://github.com/modelcontextprotocol/inspector/issues/1886), [#1916](https://github.com/modelcontextprotocol/inspector/issues/1916). +engine with the conformance runner (§3.6). -### 4.7 The argument editor workstream +### 5.7 Observability export -Six open issues are all the same defect class — the argument editor is not schema-aware: +Moved here from the first draft's enterprise section: the roadmap no longer lists audit trails, +but we hold the entire session and cannot export it in any pipeline-shaped form. -| Issue | Symptom | -| ---------------------------------------------------------------------- | ------------------------------------------------------------------- | -| [#1853](https://github.com/modelcontextprotocol/inspector/issues/1853) | JSON parameter editor escaping while typing | -| [#1856](https://github.com/modelcontextprotocol/inspector/issues/1856) | Backspace recursively escapes JSON tool inputs | -| [#1885](https://github.com/modelcontextprotocol/inspector/issues/1885) | Null values corrupted with cascading escapes | -| [#1928](https://github.com/modelcontextprotocol/inspector/issues/1928) | Nullable enums fall back to a broken raw Textarea (v1.x regression) | -| [#1919](https://github.com/modelcontextprotocol/inspector/issues/1919) | Resource templates lack RFC 6570 expansion | -| [#1910](https://github.com/modelcontextprotocol/inspector/issues/1910) | Complex `_meta` not expressible | +- **OTLP export** — emit the session as OpenTelemetry spans; show trace/span ids from `_meta` + (SEP-414) inline; "copy as trace". +- **Structured audit transcript** — the §5.2 session file, documented as a stable format. -**Fix them as one workstream, not six bugs.** A proper schema-aware editor (CodeMirror or -Monaco with JSON Schema integration) resolves the class and unblocks file inputs (§3.8) and -strict validation (§3.11). Treating them individually has already produced one regression from -v1. +### 5.8 Connection Doctor -### 4.8 Connection Doctor +The connection fixes listed in §1 have shipped (and #1944 and #1911 were closed as not planned), but a failure is still reported as a +single error. Run an ordered checklist on failure — DNS · TCP · TLS (including local-cert +cases) · `/.well-known` discovery · protocol version negotiation · auth — and report **which +step failed and what to do about it**. First-connection success is the entire first impression +of the tool. -Connection failures are currently opaque, and five open issues say so -([#962](https://github.com/modelcontextprotocol/inspector/issues/962), [#1936](https://github.com/modelcontextprotocol/inspector/issues/1936), [#1951](https://github.com/modelcontextprotocol/inspector/issues/1951), [#1944](https://github.com/modelcontextprotocol/inspector/issues/1944), [#1914](https://github.com/modelcontextprotocol/inspector/issues/1914)). +### 5.9 Server management and portability -Run an ordered checklist on failure — DNS · TCP · TLS (including local-cert cases) · -`/.well-known` discovery · protocol version negotiation · auth — and report **which step -failed and what to do about it**. First-connection success is the entire first impression of -the tool, and today a `https://localhost` server or a dev container silently fails. +Most of the first draft's list has shipped (§1). What remains is +[#1857](https://github.com/modelcontextprotocol/inspector/issues/1857), rich server configuration, +whose **registry** half — browse an MCP Registry, pick a server, generate its configuration +form from `server.json` — needs nothing but the Registry API and our existing `server.json` +support (#922). Its Server Card half waits on SEP-2127 (§3.7). -Bundle the related fixes: `*.localhost` domains ([#1944](https://github.com/modelcontextprotocol/inspector/issues/1944)), the trusted-local-host OAuth HTTP -exception ([#1911](https://github.com/modelcontextprotocol/inspector/issues/1911)), and the ghost-server entry left by a failed manual connect ([#1914](https://github.com/modelcontextprotocol/inspector/issues/1914)). +### 5.10 Large servers: grouping and performance -### 4.9 Server management and portability +A 1000-tool server or a long-running session should not degrade. -Already well represented on the board; grouping it here so it is scheduled as a theme rather -than piecemeal: rich server configuration ([#1857](https://github.com/modelcontextprotocol/inspector/issues/1857)), custom headers and cookies ([#1915](https://github.com/modelcontextprotocol/inspector/issues/1915)), -auth/token URL overrides ([#1906](https://github.com/modelcontextprotocol/inspector/issues/1906)), file-backed secrets where no OS keychain exists ([#1950](https://github.com/modelcontextprotocol/inspector/issues/1950)), -paste-MCP-JSON ([#904](https://github.com/modelcontextprotocol/inspector/issues/904)), and registry discovery ([#1101](https://github.com/modelcontextprotocol/inspector/issues/1101)). +- **Grouped / tree lists with group-aware search**, built on client-side heuristics (name + prefixes, annotations). No spec data source is expected this horizon (§3.7). +- **Virtualize** the long lists and logs; cap in-memory protocol history with spill-to-disk, so evicted entries still reach the §5.2 session file; truncate large + payloads by default with explicit expansion. +- Design lists so **"not loaded yet" is a state**, ready for progressive discovery (§3.4). -### 4.10 Workspace and layout +### 5.11 Workspace and layout Multiple servers side by side — the actual shape of debugging a gateway, or comparing a server against a reference implementation. Detachable/resizable panels, remembered layout per -server, density modes, and full-collapse ([#928](https://github.com/modelcontextprotocol/inspector/issues/928)). Prerequisite for cross-server timeline -correlation. - -### 4.11 Performance at scale +server, and density modes. Prerequisite for cross-server timeline correlation. -A 1000-tool server or a long-running session should not degrade. Virtualize the long lists and -logs; cap in-memory protocol history with spill-to-disk; truncate large payloads by default -with explicit expansion (which is also the right default for reference results, §3.6). - -### 4.12 Accessibility and keyboard-first operation +### 5.12 Accessibility and keyboard-first operation Full keyboard operation across every tab, correct roles and labels, high-contrast support, and `prefers-reduced-motion` (which the timeline's animations will make newly relevant). We have a Storybook a11y harness already; the gap is coverage, not tooling. -### 4.13 Onboarding +### 5.13 Onboarding A first run currently presents an empty server list and no path forward. Add a guided first connection, one-click example servers drawn from `test-servers/`, and inline links from each panel to the relevant spec section. -### 4.14 Plugin architecture +### 5.14 Plugin architecture -[#1025](https://github.com/modelcontextprotocol/inspector/issues/1025). The multiplier on everything above — custom panels, custom transports (§3.1), -interceptor hosting (§3.7), and community-contributed views without core changes. Sequenced -late deliberately: designing a plugin API before the timeline, diff, and session format exist -would mean designing it against the wrong surfaces. +[#1025](https://github.com/modelcontextprotocol/inspector/issues/1025) recorded the placeholder +spec. The multiplier on everything above — custom panels and community-contributed views +without core changes. Sequenced late deliberately: designing a plugin API before the timeline, +diff, and session format exist would mean designing it against the wrong surfaces. --- -## 5. Sequencing +## 6. Sequencing Four phases of roughly six weekly milestones each. Track A items appear where their upstream -signal is expected; Track B items are placed to unblock Track A wherever possible. - -### Phase 1 — Foundations (~`v2.2` – `v2.7`, Aug–Sep 2026) +signal is expected; Track B items are placed to unblock Track A wherever possible. Phase 1 is +annotated with a selection of what has already shipped; §1 has the full list. -_Build the general surfaces the rest of the plan renders into, and clear the debt that makes -first impressions bad._ +### Phase 1 — Foundations (~`v2.2` – `v2.9`, Aug–Sep 2026) -- 🅑 **Zoomable timeline v1** — lanes, spans, zoom/pan, click-through -- 🅑 **Argument editor workstream** (§4.7) — closes six issues as one -- 🅑 **Connection Doctor** (§4.8) + the local-host connection fixes -- 🅐 `Last-Event-ID` resumption ([#920](https://github.com/modelcontextprotocol/inspector/issues/920)); `Mcp-Name` on Tasks ([#1917](https://github.com/modelcontextprotocol/inspector/issues/1917)); discover checkmarks ([#1887](https://github.com/modelcontextprotocol/inspector/issues/1887)) -- 🅐 `server.json` support ([#922](https://github.com/modelcontextprotocol/inspector/issues/922)) — prerequisite for Server Cards -- ⚙️ Windows CI/gate fixes already in `v2.2.0` +- ✅ `Last-Event-ID` resumption, legacy only (#920); discover checkmarks (#1887); `server.json` (#922) +- ✅ Argument editor workstream (six issues); connection fixes (§1) +- ✅ Skills over MCP (#2234, #2248) +- 🅑 **Zoomable timeline v1** — carried into Phase 2 +- 🅑 **Connection Doctor** (§5.8) — carried into Phase 2 -### Phase 2 — Artifacts and comparison (~`v2.8` – `v2.13`, Sep–Nov 2026) +### Phase 2 — Artifacts, comparison, and cheap spec wins (~`v2.10` – `v2.15`, Oct–Nov 2026) -_Make sessions into things you can keep, share, and compare._ +_Make sessions into things you can keep, share, and compare; take the Final-SEP and extension +items that need no upstream work._ -- 🅑 **Session record / replay / share** (§4.2) — format shared with audit transcript -- 🅑 **Diff primitive** (§4.3) — then wire capability diff ([#1034](https://github.com/modelcontextprotocol/inspector/issues/1034)) -- 🅑 **Command palette and global search** (§4.4) -- 🅐 **OTLP export and audit transcript** (§3.4) — no upstream dependency -- 🅐 **Grouped sidebars** (§3.10) on client-side heuristics -- 🅐 Strict schema validation ([#1005](https://github.com/modelcontextprotocol/inspector/issues/1005), [#1015](https://github.com/modelcontextprotocol/inspector/issues/1015)) -- 🅐 Timeline lanes for tasks and sessions (falls out of Phase 1) +- 🅑 **Zoomable timeline v1**, including the **in-flight work lane** (§3.1) +- 🅑 **Session record / replay / share** (§5.2) — format shared with the audit transcript +- 🅑 **Diff primitive** (§5.3) — then capability diff (#1034) +- 🅑 **Command palette and global search** (§5.4); **Connection Doctor** (§5.8) +- 🅐 **Cache hint display and observations** (§3.2) — SEP-2549 is Final +- 🅐 **OAuth Client Credentials extension** (§3.3, §4) +- 🅐 **Serialized-JSON check for `structuredContent`** and **destructive-call confirmation** (§3.4) +- 🅐 **Extension-watch sweep** (§4) -### Phase 3 — Automation and spec catch-up (~`v2.14` – `v2.20`, Nov 2026 – Jan 2027) +### Phase 3 — Automation (~`v2.16` – `v2.21`, Nov 2026 – Jan 2027) -_Turn the Inspector into something you can run in CI, and absorb the SEPs that have landed._ +_Turn the Inspector into something you can run in CI._ -- 🅑 **Saved calls / collections** (§4.5) → **assertions and CI flows** (§4.6) -- 🅐 **Conformance runner** (§3.11) — shares the assertion engine -- 🅐 **File uploads** (§3.8) — assumes the TS SDK reference impl has shipped -- 🅐 **Server Card preview + card-vs-reality diff** (§3.2) — assumes SEP-2127 has settled -- 🅐 **Skills view** (§3.9) — assumes SEP-2640 accepted -- 🅑 Performance at scale (§4.11); accessibility pass (§4.12) +- 🅑 **Saved calls / collections** (§5.5) → **assertions and CI flows** (§5.6) +- 🅐 **Conformance runner** (§3.6) — shares the assertion engine, if the maintainers agree an interface +- 🅑 **OTLP export** (§5.7); **registry browsing** (§5.9) +- 🅑 **Grouping and performance at scale** (§5.10); accessibility pass (§5.12) +- 🅐 **Stateful-tool workflow investigation** (§3.2); **extension declaration view** (§3.5) -### Phase 4 — Frontier (~`v2.21` – `v2.27`, Jan–Feb 2027) +### Phase 4 — Frontier (~`v2.22` – `v2.27`, Jan–Feb 2027) _The items whose shape we cannot yet commit to, plus the multiplier._ -- 🅐 **Interceptor test bench** (§3.7) — and a decision on owning the WG's CLI deliverable -- 🅐 **Triggers/events receiver** (§3.5) — design throughout, build only if the SEP lands -- 🅐 **Transport/session work** (§3.1) — proxy harness, stateless verification, third era -- 🅐 **ID-JAG / Cross-App Access flow** (§3.4) -- 🅑 **Plugin architecture** (§4.14) — designed against surfaces that now exist -- 🅑 Workspace and layout (§4.10); onboarding (§4.13) +- 🅐 **DPoP**, **token exchange**, **Workload Identity Federation** (§3.3) — as each reaches Final or a Tier-1 SDK impl +- 🅐 **Server-initiated events receiver** (§3.1) — design throughout, build only if the SEP lands +- 🅐 **ETags** (§3.2), only if a SEP reaches Draft with an SDK impl; **extension contract validation** (§3.5), only once the contract is published +- 🅑 **Plugin architecture** (§5.14) — designed against surfaces that now exist +- 🅑 Workspace and layout (§5.11); onboarding (§5.13) ### Standing commitments across all phases -- **Weekly milestone cadence** and the pre-push gate (`npm run local:gate`, renamed off `ci` in #2146) are unchanged. +- **Weekly milestone cadence** and the pre-push gate (`npm run local:gate`) are unchanged. - **Bug and triage capacity is reserved, not scheduled.** The board's Incoming queue keeps flowing regardless of phase. -- **WG liaison**: attend Transports, Agents, Triggers, Interceptors, and Server Card sessions - and feed implementation experience back. Several items above are as much _inputs to_ the - spec as outputs of it. +- **Re-read the MCP roadmap when it changes.** It carries a "Last updated" date; a change there + is the trigger to revisit §3 and §6, the way #2400 revisited this draft. +- **WG liaison**: attend Triggers & Events, Transports, Agents, Agent Identity, Core Primitives, + and SDK sessions and feed implementation experience back. Several items above are as much + _inputs to_ the spec as outputs of it. --- -## 6. What we are deliberately not doing +## 7. What we are deliberately not doing Stating these so they are decisions rather than oversights. @@ -531,40 +574,48 @@ Stating these so they are decisions rather than oversights. the diff, or the session format, it does. A new top-level tab needs justification. - **Not chasing pre-Draft SEPs.** 🔴 items get a tracking issue and a WG liaison, not code. We were burned by this in v1. +- **Not scheduling build work outside the published priority areas** (§3.7). A WG effort that + the roadmap does not list gets a liaison, not milestones. Approved official extensions (§4) + are exempt: they count as spec-following work even though the roadmap does not list them. - **Not publishing `core/` as a package this cycle.** [#1636](https://github.com/modelcontextprotocol/inspector/issues/1636) stays deferred; it adds an API compatibility obligation we cannot yet afford. -- **Not adding transports beyond what the spec blesses**, per the roadmap — but §3.1 makes - _custom_ transports loadable so the community can experiment. -- **Not building a second extension mechanism.** If we host interceptors, they run on the - plugin architecture (§4.14). +- **Not adding transports beyond what the spec blesses.** Custom transports were closed as not + planned ([#1741](https://github.com/modelcontextprotocol/inspector/issues/1741)). +- **Not investing in audience/priority annotation rendering** while their deprecation is under + review (§3.4). +- **Not building a second extension mechanism.** Anything pluggable runs on the plugin + architecture (§5.14). --- -## 7. Open questions - -For WG discussion before this plan is adopted. - -1. **Does the private roadmap doc change §3?** This plan is built from the public roadmap; the - private doc may carry timelines or themes it omits. -2. **Do we claim the Interceptors WG's "CLI client for interceptor invocation and testing"?** - It is Ideating and unowned, it describes our CLI, and we have a co-lead in common. If yes, - it needs milestone allocation in Phase 3, not Phase 4. -3. **How far do we take the conformance role?** §3.11 and §4.6 point at "the Inspector tells - you whether your server is correct." That is a real expansion of mission — worth an - explicit yes or no, and possibly a charter amendment. -4. **Who owns the triggers/events reachability problem?** A publicly reachable callback - endpoint on a localhost dev tool is a security question as much as a UX one, and it needs - an owner before Phase 4. +## 8. Open questions + +For WG discussion. + +1. **Do we claim the Interceptors WG's "CLI client for interceptor invocation and testing"?** + It is unowned and describes our CLI, but Interceptors is no longer on the published roadmap + (§3.7). If yes, it needs its own allocation rather than borrowed Phase 4 capacity. +2. **How far do we take the conformance role?** §3.6 and §5.6 point at "the Inspector tells + you whether your server is correct." With conformance now central to the SDK area (§3.5), + that is worth an explicit yes or no, and possibly a charter amendment. +3. **Who owns the server-initiated events reachability problem?** A publicly reachable + callback endpoint on a localhost dev tool is a security question as much as a UX one, and + it needs an owner before Phase 4. +4. **Should the Inspector feed the composition review directly?** The in-flight work lane + (§3.1) produces exactly the evidence the review needs; decide whether we bring it to the + Agents / Triggers & Events WGs as a demo. 5. **Is the ~50/50 capacity split right?** It is an assertion in this draft, not a measurement. -6. **Timeline v1 scope.** The §4.1 sketch is deliberately broad. Which parts are v1 and which - are follow-ups should be settled before Phase 1 starts. +6. **Timeline v1 scope.** The §5.1 sketch is deliberately broad. Which parts are v1 and which + are follow-ups should be settled before it starts. --- -## 8. Sources +## 9. Sources -- [MCP Roadmap](https://modelcontextprotocol.io/development/roadmap) (last updated 2026-03-05) -- WG charters: [Inspector V2](https://modelcontextprotocol.io/community/working-groups/inspector-v2) · [Server Card](https://modelcontextprotocol.io/community/working-groups/server-card) · [Triggers & Events](https://modelcontextprotocol.io/community/working-groups/triggers-events) · [Agents](https://modelcontextprotocol.io/community/working-groups/agents) · [Interceptors](https://modelcontextprotocol.io/community/working-groups/interceptors) · [File Uploads](https://modelcontextprotocol.io/community/working-groups/file-uploads) · [Skills Over MCP](https://modelcontextprotocol.io/community/working-groups/skills-over-mcp) -- IG charters: [Primitive Grouping](https://modelcontextprotocol.io/community/interest-groups/primitive-grouping) · [Tool Annotations](https://modelcontextprotocol.io/community/interest-groups/tool-annotations) · [Enterprise-Managed Authorization](https://modelcontextprotocol.io/community/interest-groups/enterprise-managed-authorization) +- [MCP Roadmap](https://modelcontextprotocol.io/development/roadmap) (last updated 2026-08-22) +- [Extensions overview](https://modelcontextprotocol.io/extensions/overview) · [Extension support matrix](https://modelcontextprotocol.io/extensions/client-matrix) · [SEP-2133: Extensions](https://modelcontextprotocol.io/seps/2133-extensions) +- Final SEPs cited: [SEP-2549 (TTL for list results)](https://modelcontextprotocol.io/seps/2549-TTL-for-list-results) · [SEP-2567 (sessionless)](https://modelcontextprotocol.io/seps/2567-sessionless-mcp) · [SEP-2575 (stateless)](https://modelcontextprotocol.io/seps/2575-stateless-mcp) · [SEP-2663 (Tasks extension)](https://modelcontextprotocol.io/seps/2663-tasks-extension) · [SEP-2640 (Skills extension)](https://modelcontextprotocol.io/seps/2640-skills-extension) · [SEP-2484 (conformance tests)](https://modelcontextprotocol.io/seps/2484-conformance-tests-required-for-final-seps) · [SEP-414 (request `_meta`, trace context)](https://modelcontextprotocol.io/seps/414-request-meta) +- WG charters: [Inspector V2](https://modelcontextprotocol.io/community/working-groups/inspector-v2) · [Triggers & Events](https://modelcontextprotocol.io/community/working-groups/triggers-events) · [Agents](https://modelcontextprotocol.io/community/working-groups/agents) · [Transports](https://modelcontextprotocol.io/community/working-groups/transports) · [File Uploads](https://modelcontextprotocol.io/community/working-groups/file-uploads) · [SDK](https://modelcontextprotocol.io/community/working-groups/sdk) +- [SDK tiers and conformance testing](https://modelcontextprotocol.io/community/sdk-tiers) - Internal: [`specification/v2_new_spec_impact.md`](../specification/v2_new_spec_impact.md) · [`specification/v2_scope.md`](../specification/v2_scope.md) · [`specification/v2_ux_features.md`](../specification/v2_ux_features.md) - [Inspector V2 project board (#28)](https://github.com/orgs/modelcontextprotocol/projects/28) diff --git a/docs/mcp-server-configuration.md b/docs/mcp-server-configuration.md index f1abe7452..3a9c5add1 100644 --- a/docs/mcp-server-configuration.md +++ b/docs/mcp-server-configuration.md @@ -187,6 +187,7 @@ These have no analog in the broader `mcp.json` ecosystem. Each is **omitted on w | `taskTtl` | `60000` | TTL in ms for tasks created via "Run as task" (`DEFAULT_TASK_TTL_MS`) | | `autoRefreshOnListChanged` | `false` | Refresh lists automatically on `*/list_changed` instead of only flagging the indicator | | `paginatedLists` | `false` | Fetch tools/resources/prompts one page at a time instead of auto-aggregating | +| `suppressNotificationStream` | `false` | Streamable HTTP, legacy era only: don't open the standalone `GET` notification stream. Server→client messages not carried on a request's own response stream won't arrive. `Last-Event-ID` resumption `GET`s still go out, and modern-era connections are unaffected (they never open this stream). A diagnostic and escape hatch for a server that times out every request after `initialize` because it cannot serve a second concurrent request ([#2317](https://github.com/modelcontextprotocol/inspector/issues/2317)) | | `advertisedExtensions` | — | Per-extension overrides for what the Inspector declares in `capabilities.extensions` | | `maxFetchRequests` | `1000` | Network-log retention for this server (`DEFAULT_MAX_FETCH_REQUESTS`); `0` means unlimited | | `skillCatalogMaxSkills` | `256` | The maximum number of skills whose files are read in one verification run (`SKILL_MAX_CATALOG_SKILLS`) — the CLI's `--verify` and the TUI Skills pane. Positive integer; there is no unlimited value | @@ -271,6 +272,21 @@ A catalog carrying these fields: } ``` +## Reading this file from other tools + +A catalog you have already reviewed in the Inspector is a natural input for other tooling — a CI job or a reliability harness that connects to the same servers non-interactively. Reading the file is a supported interoperability use case. It is **not a versioned interchange format**: the Inspector makes no compatibility promise beyond what this page documents, the Inspector-specific fields above grow as features land, and a standard MCP client-configuration shape may supersede this one. Pin the Inspector version you validated against, and say so in your own documentation. + +A tool that consumes the file should: + +- **Treat it as read-only.** Don't rewrite it, and don't convert it into another format as a copy that users then maintain. The Inspector owns what it writes back — it omits fields equal to their defaults and upgrades older shapes (such as the pair-array `metadata`) on save — so a second writer drifts from it. +- **Preserve stdio argument boundaries.** `command` and each `args` element are separate argv entries. Spawn them directly rather than joining them into a string for a shell, which re-splits on whitespace and interprets quoting, globs and metacharacters. Keep `cwd` and the `env` key set as given. +- **Decide on unknown fields explicitly.** Either honor an Inspector-specific field, or reject the entry naming the field you don't support. Silently ignoring one can change behavior — `protocolEra`, `headers` or `oauth` alter what connects and how. +- **Expect secrets to be absent, and supply them itself.** When the Inspector saves an entry to a durable secret store (the OS keychain, or `~/.mcp-inspector/secrets.json` — see [Where secrets are stored](./secret-storage.md)) it moves two kinds of value out of this file — each stdio `env` value and `oauth.clientSecret` — leaving each `env` key in place with an empty value and the client secret omitted. Under the session-only `memory` store it keeps plaintext that was already on disk, so `mcp.json` stays the durable copy of those values rather than a store that is lost on exit, while new or changed values still stay out of it; the same file can therefore hold a mix of placeholders and real values. Nothing else is stripped: `headers` are saved as written, so they can still hold a credential. That store is not part of the file's interface, so a file-only reader sees `"API_KEY": ""` and cannot tell an intentionally empty value from a stored one. Inject those values from the tool's own secret source, or reject the entry naming the missing key — don't launch the server with the empty placeholders. +- **Keep credential values out of its output.** A hand-written or imported file can still carry plaintext in `env` or `oauth.clientSecret`, and any saved `headers` value may be one. Don't copy those values into logs, reports, evidence bundles or generated files; key names are usually enough. +- **Describe its own scope without implying endorsement.** Which fields and Inspector versions it supports is that tool's claim to document; reading this file does not make it Inspector- or MCP-certified. + +Connecting is not side-effect free. Connecting to a stdio entry **runs its `command`** on the consumer's machine before any MCP message is exchanged, and calling a server's tools can change state wherever that server acts. A tool that goes beyond reading the file should leave both decisions to its user — authorizing the launch, not only the tool calls. + ## Per-client behavior | | Web | CLI | TUI | diff --git a/docs/publishing.md b/docs/publishing.md index bad47b7f7..452d55b86 100644 --- a/docs/publishing.md +++ b/docs/publishing.md @@ -31,5 +31,7 @@ job or the coverage gate red. ## Docker -The container image and everything about running it — ports, volumes, where -secrets go — is in [Running the Inspector in Docker](./docker.md). +The container image and everything about running it — ports, volumes, making +secrets durable — is in [Running the Inspector in Docker](./docker.md). How the +secret store is chosen on every runtime is in +[Where secrets are stored](./secret-storage.md). diff --git a/docs/secret-storage.md b/docs/secret-storage.md new file mode 100644 index 000000000..e276c0ee6 --- /dev/null +++ b/docs/secret-storage.md @@ -0,0 +1,145 @@ +# Where secrets are stored + +The Inspector keeps a few values out of `mcp.json` and `client.json` and puts them in a **secret store** instead. This guide explains which store you get, why, where it lives, and how to change it. It applies to every runtime: a desktop install, a Linux server or SSH session, Android/Termux, and a container. For the container-specific parts (volumes, ownership), also read the [Docker guide](./docker.md). + +## What counts as a secret + +Three kinds of value are stored as secrets: + +| Value | Saved from | +| --------------------------------- | -------------------------------------- | +| A server's OAuth client secret | The server's OAuth settings | +| The enterprise IdP client secret | Client Settings (install-level) | +| Each stdio server's `env:` value | A stdio server's environment variables | + +They are kept out of `mcp.json` so that sharing, committing or syncing the file does not leak credentials (#1356). When the Inspector saves an entry to a durable store, it leaves each `env` key in `mcp.json` with an empty value and omits the client secret; the real values live in the store. `headers` are **not** moved: they are saved in `mcp.json` exactly as written, so a header that carries a credential stays in the file. [MCP server configuration](./mcp-server-configuration.md) describes what that means for other tools reading the same file. + +## How the store is chosen + +Each process picks one store, once, the first time it needs it: the web backend at startup, the CLI and TUI on their first access to a secret. Every client (web, CLI and TUI) goes through the same selection, in this order: + +1. **`MCP_INSPECTOR_SECRET_STORE`**, if it is `keyring`, `file` or `memory` (case-insensitive). That store is used and nothing is probed. An empty or whitespace-only value counts as unset. Any other value is ignored with a warning, and selection continues as if it were unset. +2. **The OS keychain**, if a probe can reach it: Keychain on macOS, Credential Manager on Windows, and the Secret Service (libsecret, for example GNOME Keyring or KWallet) on Linux. Entries are stored under the service name `mcp-inspector`. Most desktop installs stop here. +3. **A fallback**, when the probe fails. The Inspector says so on stderr when it selects the store, and names the store it picked (see [Where the active store is reported](#where-the-active-store-is-reported)): + - `memory` if it is running in a container **and** the directory the secrets file would go in is not on a mounted volume, because a file in a container's writable layer is lost on `docker run --rm` and on every image update; + - `file` everywhere else. + +| Where you run it | Store | Secrets survive a restart? | +| ----------------------------------------------------------------------- | ----------------------------------- | -------------------------- | +| Desktop macOS or Windows, or Linux with a Secret Service running | OS keychain | Yes | +| Linux without libsecret or a Secret Service | File (`secrets.json`, mode `0600`) | Yes | +| Headless server or SSH session with no D-Bus session | File | Yes | +| Android/Termux | File | Yes | +| Container with **no volume** on the secrets directory | Memory | No, this session only | +| Container **with** a volume on the secrets directory | File | Yes | +| Any of the above with `MCP_INSPECTOR_SECRET_STORE` set | The store you named | Not with `memory`; with `file` in a container, only if the file is on a volume | + +> [!WARNING] +> **With no keychain, secrets go to a plaintext file, and you did not have to ask for it.** On a host where the keychain probe fails (Linux without libsecret or a running Secret Service such as GNOME Keyring or KWallet, a headless server or SSH session with no D-Bus session, or Android/Termux), the Inspector falls back **automatically** to `~/.mcp-inspector/secrets.json`. Unless you supply a key, that file is **unencrypted**. Mode `0600` keeps out other non-root users, but not root, not backups or copies of your home directory, and not any program running as you. The only signs are a warning on stderr when the store is selected and the footer in the web settings dialogs. +> +> Pick one: +> +> - **Get a keychain back**: install libsecret and run a Secret Service (for example `gnome-keyring`), or run the Inspector inside a desktop session. On the next start the Inspector moves the file's secrets into the keychain and deletes the file ([details](#getting-a-keychain-back)). +> - **Encrypt the file**: supply a generated key with `MCP_INSPECTOR_SECRET_KEY_FILE` (preferred) or `MCP_INSPECTOR_SECRET_KEY` ([details](#encryption)). +> - **Don't write secrets to disk at all**: `MCP_INSPECTOR_SECRET_STORE=memory`, and re-enter them each session. +> +> Even encrypted, secrets on disk carry moderate risk. See [what the file store protects against](#what-the-file-store-protects-against). + +The Inspector decides that it is in a container from `KUBERNETES_SERVICE_HOST`, Docker's `/.dockerenv`, Podman's `/run/.containerenv`, or the process's cgroup. The container check only chooses between `memory` and `file`; the mount check is what actually decides. + +The choice is made once per process. Installing a keychain while the Inspector is running takes effect on the next start. + +### The memory store + +`memory` keeps secrets for this process only; nothing is written anywhere and they are gone when it exits. Because it is not durable, the Inspector does **not** remove plaintext values that are already in `mcp.json` or `client.json` while it is active: in that case the file on disk is still the durable copy. New or changed values are still kept out of the file. + +## The file store + +### Where the file is + +The path is the first of these that applies: + +1. `MCP_INSPECTOR_SECRET_FILE`, if set; +2. `secrets.json` inside `MCP_STORAGE_DIR`, if that is set; +3. `~/.mcp-inspector/secrets.json`. + +⚠️ The default sits **beside** the default storage directory (`~/.mcp-inspector/storage`), not inside it. Setting `MCP_STORAGE_DIR` moves the secrets file together with the OAuth state (`oauth.json`), for every client. It also moves `client.json` for the **web** backend only; the CLI and TUI find `client.json` through `MCP_CLIENT_CONFIG_PATH` instead. + +### Encryption + +**A file store is unencrypted unless you give it a passphrase.** Set `MCP_INSPECTOR_SECRET_KEY`, or point `MCP_INSPECTOR_SECRET_KEY_FILE` at a file containing it, and the file is encrypted with AES-256-GCM, with the passphrase stretched by scrypt against a random salt that is regenerated on every write. Without it, the file is still mode `0600`, but anyone who can read the file can read the values. The startup log and the settings footer say so every session, as a warning. + +**Use a high-entropy passphrase: generate it, don't choose it.** The random salt stops an attacker from precomputing a table, but it does nothing against guessing. The scrypt cost is deliberately low because the derivation runs on every read and write. Anyone who obtains `secrets.json` can therefore test candidate passphrases quickly and offline, so treat this value like any other credential, not like a memorable password. + +**Prefer the key file.** `MCP_INSPECTOR_SECRET_KEY_FILE` reads the passphrase from a file, with trailing line breaks removed, so it never has to sit in the environment, a shell profile, an `.env` file or a Compose file. It is the variable Docker and Compose secrets are built for (see the [Docker guide](./docker.md)). Setting a non-blank `MCP_INSPECTOR_SECRET_KEY` together with `MCP_INSPECTOR_SECRET_KEY_FILE` is an error; a blank `MCP_INSPECTOR_SECRET_KEY` still counts as unset, so the key file is used. Setting `MCP_INSPECTOR_SECRET_KEY_FILE` to an empty value is also an error: unlike an empty `MCP_INSPECTOR_SECRET_KEY`, which switches encryption off, it is taken as a key file that failed to arrive. If the key file is missing, unreadable or empty, the variable is blank, or both are set, the file store **refuses to read or write** instead of falling back to plaintext: saves fail, and the startup warning and settings footer report the file as unreadable, with the reason. + +**Adding a passphrase later is safe.** The next write upgrades an existing plaintext file in place. Until that write happens the existing values are still readable, and the banner and footer keep saying so. They do not report the file as encrypted just because the variable is now set. + +**Changing or losing the passphrase is not safe.** A file that can no longer be decrypted is read as empty, and the Inspector **refuses to write to it** rather than replacing it with a new file that holds only your latest secret. To recover, restore the original passphrase, or delete the secrets file at its configured path (see [Where the file is](#where-the-file-is); the path is also shown in the startup warning and the settings footer) and enter the values again. + +### Permissions + +The Inspector writes the file with mode `0600` and tightens it again when the store is selected if something loosened it. If it _cannot_ tighten it (the file belongs to another user, or the mount is read-only), it says so in the log and the footer instead of continuing to describe the file as protected. + +### What the file store protects against + +The file store is a fallback for machines without a keychain, and it is weaker than a keychain. Treat keeping secrets in it, even encrypted, as a **moderate risk**. Here is what it does and does not defend against. + +**Without a passphrase (plaintext, mode `0600`):** + +- ✅ Other non-root users on the same machine, as long as the mode holds. +- ❌ Root, and on a container host, every member of the `docker` group, which is equivalent to root. +- ❌ Anyone who gets a copy of the file: a backup, a disk or volume snapshot, a synced home directory, or an accidental `git add` of a bind-mounted directory. +- ❌ Any program running as your user, including the stdio MCP servers the Inspector starts. + +**With `MCP_INSPECTOR_SECRET_KEY` set (AES-256-GCM):** + +- ✅ **The file leaking on its own.** A backup, snapshot, copy or commit of `secrets.json` is useless without the key, _provided_ the passphrase is high-entropy (see [Encryption](#encryption)) and the key did not leak with it. This is the threat encryption at rest is for. +- ❌ **Anyone who can read the key where it lives.** With `MCP_INSPECTOR_SECRET_KEY` the key is in the Inspector's environment, readable through `/proc//environ` by the same user or root, through `docker inspect` and `docker exec` for a container, and wherever you stored it for launching, such as a shell profile, an `.env` file or a Compose file. `MCP_INSPECTOR_SECRET_KEY_FILE` narrows this to whoever can read the key file, but the Inspector must be able to read it, so the same user can too. If the key sits next to the secrets file (in the same backup, volume or repository), encryption buys nothing. +- ❌ **Root on the host, or the `docker` group.** They can read both the file and the key, or the process memory holding the decrypted values. +- ❌ **Code running as the same user.** The Inspector does **not** pass its own environment to the stdio servers it starts: they get a short allowlist (`HOME`, `LOGNAME`, `PATH`, `SHELL`, `TERM`, `USER` on macOS and Linux) plus their configured `env:`. But a server runs as the same user, so it can open the secrets file directly and can usually read the Inspector's environment through `/proc`. Only run servers you would trust with these secrets. +- ❌ **A weak passphrase.** Anyone with the file can guess offline. + +In short, encryption turns "the file leaked" into "the file **and** the key leaked". It does not help against anyone who already has access to the machine or the container as root, or as the user the Inspector runs as. When that is not acceptable, use a keychain (install libsecret or run a Secret Service on Linux), or `MCP_INSPECTOR_SECRET_STORE=memory` and re-enter secrets each session. + +### Two Inspectors, one file + +Within a process, changes are serialized per file path, so a web session's own concurrent saves cannot overwrite each other. Across processes, for example a CLI run next to a web session, each change takes an exclusive lock on `.lock`, a lock directory beside the secrets file named after it (`secrets.json.lock` by default), for the whole read-modify-write. The lock uses [`proper-lockfile`](https://github.com/moxystudio/node-proper-lockfile), the same library npm uses for its own locks. The lock expires 10 seconds after its holder stops refreshing it, so an Inspector that is killed mid-save does not leave the file unwritable. + +So two running Inspectors are genuinely serialized. What a lock file cannot make single-winner is the _takeover of a lock whose holder died_. That needs a compare-and-swap on a directory entry (`renameat2`), which Node does not expose, and `proper-lockfile` does not close that race either. The window only opens after a holder dies without releasing its lock. + +The Inspector adds one check on top. Every lock-directory removal the library makes on its behalf, on release and from its exit handler, first checks that the directory is still the one it created (by inode and birth time, which survive the library's own refresh but not a delete-and-recreate). Without that check the removals are unconditional, so a holder whose lock had been replaced would delete the _winner's_ lock on the way out, turning one compromised writer into two unprotected ones. The check also reports the takeover as a warning. Treat all of this as **best-effort**: the check is still followed by a separate act, so it makes the destructive case rare rather than impossible, and it relies on filesystem metadata that not every filesystem reports. + +That is why, under the lock, each change still reads the file, applies the change, writes, then reads back and compares the whole map. If something wrote in between, it re-applies the change to what is there now and retries, and it fails loudly after five lost rounds instead of reporting the value as saved. That check catches a clobber inside the takeover window. It also covers writers that no lock can order, because a lock only orders the writers that _take_ it: an editor, a restored backup, or an older Inspector. + +If another process holds the lock and does not release it, the save **fails** rather than going ahead unlocked. It waits past the stale window first, so a crashed Inspector clears itself instead of failing everyone else's saves. When there is another writer you can see, writing anyway is the one case where continuing would lose the secret the save was meant to protect. + +The same read-back check covers a lock that cannot be taken at all. The file store exists for machines where the usual mechanism is missing, so when a directory cannot hold a lock file (a read-only `$HOME`, or a mount owned by another uid), the save goes ahead unlocked with a warning. Otherwise every save would fail on exactly the setups this store was written for. + +## Getting a keychain back + +If you install libsecret (or start a Secret Service) on a machine that was using the file store, the next start probes successfully, selects the keychain, and **moves the contents of `secrets.json` into it**: + +- **The keychain wins on conflict.** A value already in the keychain is kept and the file's value is not copied, because it is treated as the older copy. Only entries the keychain does not have are written. +- **The file is removed only when every entry is accounted for**, meaning each one either was already in the keychain or was written there. If any entry could not be handled, or a keychain read or write fails, the file is left as it was and the next start tries again. A file with no entries is also left in place. +- **An unreadable file is not deleted.** If the file cannot be decrypted (the passphrase changed or is now unset), the Inspector reports it and leaves the file in place. + +A successful move prints a message naming the file it removed. The same hand-off runs when you select the keychain explicitly with `MCP_INSPECTOR_SECRET_STORE=keyring`. It does not run in the other direction: choosing `file` or `memory` does not copy anything out of the keychain. + +## Where the active store is reported + +- **When the store is selected** (at startup for the web backend, on first use for the CLI and TUI), every client prints a warning on stderr if it falls back from the keychain, including the keychain error, and another if the file is unencrypted, has loose permissions, or cannot be read. Either warning is followed by a link to this guide. The web client's startup banner also has a `Secrets:` line on every run. +- **`GET /api/config`** (web) includes a `secretStorage` object describing the active store. +- **In the web UI**, a footer at the bottom of the **Client Settings**, **Server Settings** and **Add / Edit / Clone server** dialogs names the store, and turns into a warning when it is memory-only, unencrypted, loosely permissioned, or unreadable. It is shown where you type a secret, not only once at startup. + +## Changing the store + +| To | Set | +| ------------------------------------------ | -------------------------------------------------------------------- | +| Always use the keychain | `MCP_INSPECTOR_SECRET_STORE=keyring` | +| Use a file even though a keychain exists | `MCP_INSPECTOR_SECRET_STORE=file` | +| Never write secrets to disk | `MCP_INSPECTOR_SECRET_STORE=memory` | +| Put the file somewhere else | `MCP_INSPECTOR_SECRET_FILE=/path/to/secrets.json`, or `MCP_STORAGE_DIR` | +| Encrypt the file | `MCP_INSPECTOR_SECRET_KEY_FILE=/path/to/key-file` (preferred), or `MCP_INSPECTOR_SECRET_KEY=` | + +Every variable is also listed in [Environment variables](./environment-variables.md#secret-store). diff --git a/docs/skill-authoring.md b/docs/skill-authoring.md index 8582c7325..945b47273 100644 --- a/docs/skill-authoring.md +++ b/docs/skill-authoring.md @@ -418,8 +418,8 @@ CHAIN_THRESHOLD=0.4 CHAIN_MAX_TURNS=20 npm run skills:eval -- test-servers The summary is two lines, never one: ``` -7/7 first-move cases at or above 80%. -2/2 hand-off cases above 50%. +7/7 claude first-move cases at or above 80%. +2/2 claude hand-off cases above 50%. ``` Narrowing the run never narrows what a **negative** case is scored against — a @@ -443,6 +443,114 @@ metered calls, it is non-deterministic by construction, and it goes red on a rate limit. A case below threshold is a signal to investigate, not a build break. +## Measuring GitHub Copilot + +Some maintainers work on this repo with the [GitHub Copilot +CLI](https://www.npmjs.com/package/@github/copilot), so the same committed cases +also run through it (#2397): + +```sh +npm install -g @github/copilot # then sign in once: `copilot`, then `/login` +AGENT=copilot npm run skills:eval +AGENT=copilot RUNS=5 npm run skills:eval -- pr-flow +``` + +**No coercion is needed: Copilot reads `.claude/skills/` as it is.** Its project +skill sources are `.github/skills/`, `.agents/skills/` **and** `.claude/skills/`, +and `copilot skill list` on 1.0.85 shows all ten of ours. So there is no symlink, +no `.github/skills/` copy and no pointer file, and there must not be one — two +copies of a procedure is how the stale one gets read. + +What was verified about how Copilot treats the files, on 1.0.85: + +- **It honors `disable-model-invocation: true`, but only as far as its tool + goes.** Its `skill` tool refused `release` with `Skill not found`, and the + model then opened `.claude/skills/release/SKILL.md` with `view` and read it + anyway. A name-only skill is kept out of the automatic listing, not made + unreadable — which is equally true of Claude, which can `Read` the file. It + still offers `/release` in its interactive slash-command menu. +- **It honors `user-invocable: false` in that menu.** Driven through a real + pty, typing `/pro` lists `pr-flow` and `pre-push-gate` but not + `project-structure`, while `/testin` lists `testing`. The model can still load + it through its `skill` tool, which is what `user-invocable: false` is for. + (Headless `-p "/name"` is no test of this: prompt mode does not expand slash + commands, so the model simply loads the named skill as a tool call.) +- **`AGENTS.md` is loaded as custom instructions**, so the rule in + [Do not write a case `AGENTS.md` already answers](#do-not-write-a-case-agentsmd-already-answers) + applies to Copilot runs unchanged. + +How the Copilot run differs from the Claude run, since a rate only means +something next to the harness that produced it: + +| | Claude | Copilot | +| --- | --- | --- | +| Turn budget | `--max-turns` | none exists; `runPrompt` stops the process once the stream shows that many model calls | +| Availability (the real bound) | `--tools Read,Glob,Grep,Skill` | `--available-tools view,glob,grep,skill` | +| Pre-approval only | `--allowedTools` | `--allow-tool` | +| Unconditional deny | `--disallowedTools` by tool name | `--deny-tool shell`, `write`, `url` — permission **kinds**, not names | +| MCP servers | `--strict-mcp-config` | `--disable-builtin-mcps`; it does not read `.mcp.json` | +| Model | whatever `claude` defaults to | whatever Copilot's model picker defaults to (Claude Sonnet 5 when measured) | + +**Each invocation measures one agent, and every heading and summary line names +it.** The two rates come from different models behind different harnesses, so +they are compared side by side and never summed, for the same reason first-move +and hand-off cases are not. + +To probe one prompt the way the Copilot run does: + +```sh +printf '%s' "" \ + | copilot --output-format json \ + --available-tools view,glob,grep,skill --allow-tool view,glob,grep,skill \ + --deny-tool shell --deny-tool write --deny-tool url \ + --disable-builtin-mcps --disallow-temp-dir --no-ask-user --no-auto-update \ + | jq -r 'select(.type == "assistant.message") | .data.toolRequests[] + | if .name == "skill" then "skill:" + .arguments.skill else .name end' \ + | head -3 +``` + +⚠️ Like the Claude probe above, **these flags are a copy of `agentArgs` in +`scripts/skill-eval.mjs`** — change both in the same edit. Unlike the Claude +probe, nothing here stops the run after the first move, so `head -3` only +trims the output; the session carries on until it answers. + +**First measurement** (2026-09-16, Copilot CLI 1.0.85, Claude Sonnet 5): the +full suite at `RUNS=3` put **63/63 first-move cases at 100%**, every negative +clean. The hand-offs were then re-measured at `RUNS=5`, since `RUNS=3` is too +coarse to read a chain: + +| Hand-off case | Copilot, `RUNS=5` | Claude, `RUNS=5` (#2247) | +| --- | --- | --- | +| `testing → test-servers`, "Write an integration test that exercises tool listing end to end." | 100% | 100% | +| `testing → test-servers`, "Add end-to-end coverage for the tool-list pagination path." | **40%** | 100% | + +So the pagination prompt was a **Copilot-specific shortfall**, not noise: it +held below the 50% bar at both sample sizes, while the same case cleared 100% +under Claude. + +**Where it stopped** (#2399), from five recorded Copilot runs of that prompt, +two of which made the hand-off: one loaded `testing` then `test-servers` as its +first two moves; one opened with `grep` and reached `testing → test-servers` +only after about ten searches, inside the turn budget. Of the three misses, two +loaded `testing` and then went straight to `grep`, never following its pointer, +and one searched the code throughout without loading any skill. The pointer was +conditional on "does this test use a `test-servers/` fixture?", and a prompt +about pagination does not say so, so the model went to the code to find out and +did not come back once it found `pagination-http.json`. The fix is in +`testing`'s body only (the description, and so the listing, is unchanged): the +pointer now names end-to-end or integration coverage of an MCP operation as the +signal to load `test-servers` **before** searching the code. + +| Hand-off case | Copilot, `RUNS=5`, after | Claude, `RUNS=5`, after | +| --- | --- | --- | +| `testing → test-servers`, "Write an integration test that exercises tool listing end to end." | 100%, 100% | 100% | +| `testing → test-servers`, "Add end-to-end coverage for the tool-list pagination path." | **60%, 60%** | 100% | + +Two independent Copilot runs are shown because 3/5 sits one run above the bar. +It clears it without lowering the Claude rate, but it is the weakest hand-off +measured under either agent, and the first case to re-check when a Copilot +release changes the default model. + ## Checklist for a new or edited skill 1. Frontmatter opens on line 1 (no BOM, no blank line), YAML is valid, and any diff --git a/docs/v1-to-v2-migration.md b/docs/v1-to-v2-migration.md index 520d6b9a5..727b11324 100644 --- a/docs/v1-to-v2-migration.md +++ b/docs/v1-to-v2-migration.md @@ -278,7 +278,7 @@ This table maps v1 names to v2. Defaults, accepted values, and the variables wit | `CLIENT_PORT` | same | Web UI port, default `6274`. Must be a fixed port — `0`/dynamic is rejected, since the origin allow-list and sandbox CSP derive from it | | `HOST` | same, **guarded** | An all-interfaces host (`0.0.0.0`, `::`, and equivalent spellings) is now **refused** unless `DANGEROUSLY_BIND_ALL_INTERFACES=true`. Binding a specific IP or hostname needs no opt-in | | `ALLOWED_ORIGINS` | same | Still comma-separated, still **replaces** the default list rather than merging. Entries must include the scheme | -| `DANGEROUSLY_OMIT_AUTH` | same | | +| `DANGEROUSLY_OMIT_AUTH` | same, **stricter** | Only `true` / `1` (trimmed, case-insensitive) disable auth now; v1 treated any non-empty value — even `false` — as on | | `MCP_AUTO_OPEN_ENABLED` | same | Also governs opening the OAuth page in the CLI (`true` opens it even when stderr is not a TTY). The TUI does not read it | | — | `DANGEROUSLY_BIND_ALL_INTERFACES` | New opt-in for a wildcard bind (the Docker image sets it) | | — | `MCP_CATALOG_PATH` | Default catalog path | diff --git a/package-lock.json b/package-lock.json index 65ffe271e..3a3ec4d2a 100644 --- a/package-lock.json +++ b/package-lock.json @@ -1,14 +1,14 @@ { "name": "@modelcontextprotocol/inspector", - "version": "2.7.0", + "version": "2.8.0", "lockfileVersion": 3, "requires": true, "packages": { "": { "name": "@modelcontextprotocol/inspector", - "version": "2.7.0", + "version": "2.8.0", "hasInstallScript": true, - "license": "MIT", + "license": "SEE LICENSE IN LICENSE", "dependencies": { "@hono/node-server": "^2.0.12", "@modelcontextprotocol/client": "2.0.0", diff --git a/package.json b/package.json index 15423643b..01bef624a 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "@modelcontextprotocol/inspector", - "version": "2.7.0", + "version": "2.8.0", "description": "The Model Context Protocol Inspector", "keywords": [ "MCP", @@ -14,7 +14,7 @@ "type": "git", "url": "git+https://github.com/modelcontextprotocol/inspector.git" }, - "license": "MIT", + "license": "SEE LICENSE IN LICENSE", "author": "The MCP Maintainers and Community", "type": "module", "bin": { diff --git a/scripts/lib/claude-cli.mjs b/scripts/lib/claude-cli.mjs index d7cbdb62f..060fa3768 100644 --- a/scripts/lib/claude-cli.mjs +++ b/scripts/lib/claude-cli.mjs @@ -12,6 +12,10 @@ // space-joined string, so any argument holding a metacharacter becomes syntax — // so arguments go through the same `winShellArgs` quoting the npm/npx call // sites use. This is deliberately the ONLY place that decides either question. +// +// The same two questions apply to every agent CLI the skills eval drives — +// `copilot` is an npm-installed `.cmd` shim on Windows too (#2397) — so the +// decision is made once, for any command, and the `claude` helpers delegate. import { spawnSync } from "node:child_process"; import { winShellArgs } from "./win-shell-args.mjs"; @@ -28,9 +32,27 @@ export function claudeSpawnArgs( args, options = {}, platform = process.platform, +) { + return cliSpawnArgs("claude", args, options, platform); +} + +/** + * `spawn` arguments for any npm-installed agent CLI, correct on every platform. + * + * @param {string} command + * @param {string[]} args + * @param {object} [options] Passed through to the spawn call. + * @param {string} [platform] Defaults to the current platform; injectable for tests. + * @returns {{ command: string, args: string[], options: object }} + */ +export function cliSpawnArgs( + command, + args, + options = {}, + platform = process.platform, ) { return { - command: "claude", + command, args: winShellArgs(args, platform), options: { ...options, shell: platform === "win32" }, }; @@ -53,11 +75,26 @@ export function claudeSpawnArgs( * @param {{ spawn?: typeof spawnSync, platform?: string }} [io] * @returns {T | null} */ -export function probeClaudeVersion( +export function probeClaudeVersion(parseVersion, io = {}) { + return probeCliVersion("claude", parseVersion, io); +} + +/** + * Read any agent CLI's version, or null when there is no usable one. + * + * @template T + * @param {string} cli + * @param {(text: string) => T | null} parseVersion + * @param {{ spawn?: typeof spawnSync, platform?: string }} [io] + * @returns {T | null} + */ +export function probeCliVersion( + cli, parseVersion, { spawn = spawnSync, platform } = {}, ) { - const { command, args, options } = claudeSpawnArgs( + const { command, args, options } = cliSpawnArgs( + cli, ["--version"], { encoding: "utf8" }, platform, diff --git a/scripts/lib/claude-cli.test.mjs b/scripts/lib/claude-cli.test.mjs index 066fa58cb..b590da109 100644 --- a/scripts/lib/claude-cli.test.mjs +++ b/scripts/lib/claude-cli.test.mjs @@ -7,7 +7,12 @@ import { test } from "node:test"; import assert from "node:assert/strict"; -import { claudeSpawnArgs, probeClaudeVersion } from "./claude-cli.mjs"; +import { + claudeSpawnArgs, + cliSpawnArgs, + probeClaudeVersion, + probeCliVersion, +} from "./claude-cli.mjs"; test("spawns without a shell off Windows", () => { const { command, args, options } = claudeSpawnArgs( @@ -74,3 +79,28 @@ test("probeClaudeVersion asks for the shell on Windows", () => { assert.equal(seen.shell, true); assert.equal(seen.encoding, "utf8"); }); + +test("the generic helpers name the command they were given (#2397)", () => { + // `copilot` is an npm `.cmd` shim on Windows as well, so it takes the same + // shell-and-quoting decision rather than a second copy of it. + const { command, options, args } = cliSpawnArgs( + "copilot", + ["--prompt", "a & b"], + {}, + "win32", + ); + assert.equal(command, "copilot"); + assert.equal(options.shell, true); + assert.deepEqual(args, ["--prompt", '"a & b"']); + + let spawned; + const version = probeCliVersion("copilot", (t) => t.trim(), { + spawn: (c) => { + spawned = c; + return { status: 0, stdout: "GitHub Copilot CLI 1.0.85.\n" }; + }, + platform: "linux", + }); + assert.equal(spawned, "copilot"); + assert.equal(version, "GitHub Copilot CLI 1.0.85."); +}); diff --git a/scripts/skill-eval.mjs b/scripts/skill-eval.mjs index ed7fc245f..735643d77 100755 --- a/scripts/skill-eval.mjs +++ b/scripts/skill-eval.mjs @@ -29,6 +29,17 @@ // npm run skills:eval -- testing # one skill's cases // npm run skills:eval -- testing test-servers # several skills' cases // RUNS=5 THRESHOLD=0.8 npm run skills:eval +// AGENT=copilot npm run skills:eval # the same cases, driven through GitHub Copilot +// +// Two agents, measured separately (#2397). Some maintainers work on this repo +// with the GitHub Copilot CLI, which discovers project skills from +// `.claude/skills/` as well as `.github/skills/` (verified on 1.0.85: `copilot +// skill list` shows all ten). Discovery is not triggering, though, so the same +// committed cases run through `copilot` when `AGENT=copilot`. One invocation +// measures ONE agent and says which in every heading: a Copilot rate and a +// Claude rate come from different models behind different harnesses, so they +// are never folded into one number, for the same reason first-move and +// hand-off cases are not. // // Two kinds of case, measured and reported separately (#2204). A `expect` case // is a FIRST-MOVE measurement: one turn, does the model reach for the skill @@ -41,7 +52,7 @@ import { spawn } from "node:child_process"; import { readFileSync, existsSync, readdirSync, statSync } from "node:fs"; import path from "node:path"; import { fileURLToPath } from "node:url"; -import { claudeSpawnArgs, probeClaudeVersion } from "./lib/claude-cli.mjs"; +import { cliSpawnArgs, probeCliVersion } from "./lib/claude-cli.mjs"; import { isChainCase, parseClaudeVersion, @@ -94,6 +105,10 @@ const CONCURRENCY = Number(process.env.CONCURRENCY ?? 4); */ const CHAIN_MAX_TURNS = Number(process.env.CHAIN_MAX_TURNS ?? 14); +/** The agent CLIs this eval can drive. `claude` is the default. */ +export const AGENTS = ["claude", "copilot"]; +const AGENT = process.env.AGENT ?? "claude"; + /** Collect the committed cases for every model-invoked skill (optionally one). */ /** * Collect the committed cases, and the set of skill names that are OURS. @@ -243,6 +258,57 @@ export function collectSkillInvocations(text, turnOffset = 0) { return { invoked, rest, result, nextTurn: turn }; } +/** + * Extract the skills the Copilot CLI's `skill` tool was asked for, from a chunk + * of `copilot --output-format json` output (#2397). + * + * The same contract as `collectSkillInvocations`, over a different event shape. + * Copilot emits one `assistant.message` per model call, carrying every tool + * request that call made in `data.toolRequests` — so one such event is one + * turn, and two `skill` requests inside it are concurrent guesses rather than a + * hand-off, exactly as for Claude. The terminal event is `{type: "result", + * exitCode}`; it is reported as `success` for exit 0 and `exit_` + * otherwise, so `runRejection` classifies both CLIs with one table. + * + * A request is counted whether or not the tool then succeeded. That matches + * the Claude side, which scores the `tool_use` block, and it is what a trigger + * eval asks: did the model reach for the skill. (Copilot refuses a request + * for a `disable-model-invocation: true` skill with "Skill not found", and + * no case here names such a skill.) + * + * @param {string} text Newline-delimited JSON events; a trailing partial line + * is held back. + * @param {number} [turnOffset] + * @returns {{ invoked: {payload: string, turn: number}[], rest: string, + * result: string | null, nextTurn: number }} + */ +export function collectCopilotSkillInvocations(text, turnOffset = 0) { + const lines = text.split("\n"); + const rest = lines.pop() ?? ""; + const invoked = []; + let result = null; + let turn = turnOffset; + for (const line of lines) { + if (!line.trim()) continue; + let evt; + try { + evt = JSON.parse(line); + } catch { + continue; + } + if (evt?.type === "result") { + result = evt.exitCode === 0 ? "success" : `exit_${evt.exitCode}`; + } + if (evt?.type !== "assistant.message") continue; + turn++; + for (const req of evt.data?.toolRequests ?? []) { + if (req?.name !== "skill") continue; + invoked.push({ payload: JSON.stringify(req.arguments ?? {}), turn }); + } + } + return { invoked, rest, result, nextTurn: turn }; +} + /** * The skill names one recorded invocation asked for. * @@ -262,7 +328,13 @@ function entryNames(entry) { * reject exactly the runs the eval is trying to count. Verified against the * CLI: a firing prompt ends `{subtype: "error_max_turns", num_turns: 2}`. */ -const CONCLUSIVE_RESULTS = new Set(["success", "error_max_turns"]); +const CONCLUSIVE_RESULTS = new Set([ + "success", + "error_max_turns", + // Copilot has no `--max-turns`; `runPrompt` stops it at the budget itself and + // records this. It is the same observation as `error_max_turns` (#2397). + "turn_budget", +]); /** * Whether a finished run produced a usable observation. @@ -447,6 +519,138 @@ const DISALLOWED_TOOLS = [ "KillShell", ]; +/** + * The Copilot CLI's equivalents of the three lists above (#2397). + * + * Its permission model has the same split Claude's does, under other names: + * `--available-tools` decides what the model can SEE ("disables all other + * tools"), while `--allow-tool` / `--deny-tool` only decide approval and + * "do not expose tools that were filtered out" — so availability is the + * restriction here too, and the deny patterns are the unconditional second + * layer. `view`/`glob`/`grep` are Copilot's read tools and `skill` is its + * skill loader, confirmed from a live run's `toolRequests`. + * + * Copilot's deny list takes permission KINDS rather than tool names: `shell` + * (every shell command), `write` (every file-modifying tool) and `url` (every + * URL the shell or web-fetch tools would reach). `--disable-builtin-mcps` + * drops the one MCP server it ships (`github-mcp-server`); it does not read + * this checkout's `.mcp.json`, and any server a contributor configured is + * invisible past `--available-tools` anyway. + */ +const COPILOT_AVAILABLE_TOOLS = ["view", "glob", "grep", "skill"]; +const COPILOT_DENIED_KINDS = ["shell", "write", "url"]; + +/** + * The command line for one headless run of `agent`. + * + * @param {string} agent One of `AGENTS`. + * @param {number} maxTurns + * @returns {string[]} + */ +export function agentArgs(agent, maxTurns) { + if (agent === "copilot") { + return [ + // No `-p`: with none, the CLI reads the prompt from piped stdin, which + // keeps it out of argv for the same reasons as the Claude run below. + "--output-format", + "json", + "--available-tools", + COPILOT_AVAILABLE_TOOLS.join(","), + "--allow-tool", + COPILOT_AVAILABLE_TOOLS.join(","), + ...COPILOT_DENIED_KINDS.flatMap((kind) => ["--deny-tool", kind]), + "--disable-builtin-mcps", + "--disallow-temp-dir", + "--no-ask-user", + // A measurement should run the CLI version it reports, not one it + // downloaded partway through the suite. + "--no-auto-update", + ]; + } + if (agent !== "claude") throw new Error(`unknown agent \`${agent}\``); + return [ + "-p", + "--output-format", + "stream-json", + "--verbose", + "--max-turns", + String(maxTurns), + // Keep the run read-only, across every turn it is given: what the + // harness needs, minus what it must never do, minus every MCP server + // this checkout or the contributor happens to configure. + "--tools", + ALLOWED_TOOLS.join(","), + "--allowedTools", + ALLOWED_TOOLS.join(","), + "--disallowedTools", + DISALLOWED_TOOLS.join(","), + "--strict-mcp-config", + ]; +} + +/** + * Stop a child and everything it started. + * + * `copilot` is a Node wrapper around a native binary, and SIGTERM to the + * wrapper does NOT reach the binary: measured on 1.0.85, the native process was + * still running (and still spending its model call) ten seconds later. So on + * POSIX a Copilot run is spawned as the leader of its own process group and the + * whole group is signalled (Copilot). On Windows the CLI runs under `cmd.exe`, + * where `taskkill /T` takes the tree. + * + * @param {{ pid?: number, kill: (signal: string) => unknown }} child + * @param {string} platform + * @param {{ spawnFn?: typeof spawn, killProcess?: typeof process.kill }} [io] + */ +export function killTree( + child, + platform, + { spawnFn = spawn, killProcess = process.kill } = {}, +) { + if (child.pid === undefined) return; + if (platform === "win32") { + spawnFn("taskkill", ["/pid", String(child.pid), "/T", "/F"], { + stdio: "ignore", + }); + return; + } + try { + killProcess(-child.pid, "SIGTERM"); + } catch { + // The group is already gone — the run finished as the budget was reached. + } +} + +/** + * Copilot runs still in flight, so an interrupted eval can stop them. + * + * A child in its own process group no longer receives the terminal's Ctrl-C, + * which is the price of being able to signal the group — so without this, an + * interrupted suite would leave every in-flight session running unattended. + */ +const liveCopilotRuns = new Set(); + +/** + * Stop every Copilot run still in flight. + * + * Every way the eval can end early has to come through here: an interrupt, and + * just as much one sample rejecting — `pool` rejects on the first failure and + * `main().catch` exits while the other detached groups are still running, and + * a process exit does not reach them (Copilot). + * + * @param {Set<{child: object, platform: string, killFn: Function}>} [runs] + * @returns {number} How many runs were signalled. + */ +export function stopLiveCopilotRuns(runs = liveCopilotRuns) { + let n = 0; + for (const run of runs) { + run.killFn(run.child, run.platform); + n++; + } + runs.clear(); + return n; +} + /** * Drive one fresh session and return the payloads the `Skill` tool was called * with. @@ -468,6 +672,8 @@ export function runPrompt( cwd = ROOT, platform = process.platform, maxTurns = 1, + agent = "claude", + killFn = killTree, } = {}, ) { return new Promise((resolve, reject) => { @@ -477,29 +683,31 @@ export function runPrompt( // through a shell, and `cmd.exe` would re-parse any prompt containing a // metacharacter as syntax (Copilot). It also keeps the prompt out of the // process table. - const { command, args, options } = claudeSpawnArgs( - [ - "-p", - "--output-format", - "stream-json", - "--verbose", - "--max-turns", - String(maxTurns), - // Keep the run read-only, across every turn it is given: what the - // harness needs, minus what it must never do, minus every MCP server - // this checkout or the contributor happens to configure. - "--tools", - ALLOWED_TOOLS.join(","), - "--allowedTools", - ALLOWED_TOOLS.join(","), - "--disallowedTools", - DISALLOWED_TOOLS.join(","), - "--strict-mcp-config", - ], - { cwd, stdio: ["pipe", "pipe", "inherit"] }, + const { command, args, options } = cliSpawnArgs( + agent, + agentArgs(agent, maxTurns), + { + cwd, + stdio: ["pipe", "pipe", "inherit"], + // Its own process group, so `killTree` can reach the native binary the + // wrapper starts. Windows has no groups; `taskkill /T` covers it. + ...(agent === "copilot" && platform !== "win32" + ? { detached: true } + : {}), + }, platform, ); const p = spawnFn(command, args, options); + const run = { child: p, platform, killFn }; + if (agent === "copilot") liveCopilotRuns.add(run); + const collect = + agent === "copilot" + ? collectCopilotSkillInvocations + : collectSkillInvocations; + // Copilot has no `--max-turns`, so the budget is enforced from outside: once + // the stream has shown `maxTurns` model calls, the run has made every move + // it is being scored on, and the rest would only spend metered calls. + let stopped = false; let buf = ""; const invoked = []; @@ -508,20 +716,41 @@ export function runPrompt( // stream rather than restarting at each read. let turnOffset = 0; p.stdout.on("data", (chunk) => { - const parsed = collectSkillInvocations( - buf + chunk.toString(), - turnOffset, - ); + if (stopped) return; + const parsed = collect(buf + chunk.toString(), turnOffset); buf = parsed.rest; turnOffset = parsed.nextTurn; - for (const entry of parsed.invoked) invoked.push(entry); + for (const entry of parsed.invoked) { + // A pipe does not preserve event boundaries, so one chunk can carry a + // model call past the budget — even the whole rest of the run, result + // included. Scoring is bounded by turn number rather than by when the + // stop happened to land (Copilot). Claude's own `--max-turns` already + // bounds its stream. + if (agent === "copilot" && entry.turn > maxTurns) continue; + invoked.push(entry); + } if (parsed.result !== null) result = parsed.result; + if (agent === "copilot" && turnOffset >= maxTurns && result === null) { + stopped = true; + result = "turn_budget"; + killFn(p, platform); + } }); p.on("error", reject); p.on("close", (code) => { + liveCopilotRuns.delete(run); const rejection = runRejection({ result, code }); if (rejection !== null) { - reject(new Error(`\`claude -p\` ${rejection} for prompt: ${prompt}`)); + // An unauthenticated `copilot` answers `--version` fine, then prints + // its login hint to the stdout this run captures and exits with no + // events — so the one actionable fact would otherwise be swallowed. + const hint = + agent === "copilot" && result === null + ? " — if it is not signed in, run `copilot` and `/login`, or export COPILOT_GITHUB_TOKEN" + : ""; + reject( + new Error(`\`${agent}\` ${rejection}${hint} for prompt: ${prompt}`), + ); return; } resolve(invoked); @@ -530,6 +759,16 @@ export function runPrompt( }); } +/** + * Read `copilot --version` (`GitHub Copilot CLI 1.0.85.`) down to its version. + * + * @param {string} text + * @returns {string | null} + */ +export function parseCopilotVersion(text) { + return /^GitHub Copilot CLI (\d+\.\d+\.\d+)/m.exec(text)?.[1] ?? null; +} + async function pool(items, n, fn) { const out = new Array(items.length); let i = 0; @@ -574,7 +813,8 @@ export function passesThreshold(rate, threshold, strict) { * @param {object[]} cases * @param {{c: object, invoked: Iterable}[]} results One per sample. * @param {Set | null} ours - * @param {{threshold: number, chainThreshold: number, chainMaxTurns: number}} opts + * @param {{threshold: number, chainThreshold: number, chainMaxTurns: number, + * agent?: string}} opts * @returns {{ lines: string[], failed: number }} */ export function formatReport(cases, results, ours, opts) { @@ -603,15 +843,18 @@ export function formatReport(cases, results, ours, opts) { const direct = cases.filter((c) => !isChainCase(c)); const chained = cases.filter(isChainCase); + // Every heading names the agent, so a Copilot report pasted next to a Claude + // one cannot be read as the same measurement (#2397). + const agent = opts.agent ?? "claude"; const directShort = group( direct, - "First move (1 turn)", + `First move (1 turn) — ${agent}`, opts.threshold, false, ); const chainedShort = group( chained, - `Hand-off (${opts.chainMaxTurns} turns)`, + `Hand-off (${opts.chainMaxTurns} turns) — ${agent}`, opts.chainThreshold, true, ); @@ -624,13 +867,13 @@ export function formatReport(cases, results, ours, opts) { lines.push(""); if (direct.length > 0) { lines.push( - `${direct.length - directShort}/${direct.length} first-move cases at or above ${opts.threshold * 100}%.`, + `${direct.length - directShort}/${direct.length} ${agent} first-move cases at or above ${opts.threshold * 100}%.`, ); } lines.push( chained.length === 0 - ? "No hand-off cases in this selection." - : `${chained.length - chainedShort}/${chained.length} hand-off cases above ${opts.chainThreshold * 100}%.`, + ? `No ${agent} hand-off cases in this selection.` + : `${chained.length - chainedShort}/${chained.length} ${agent} hand-off cases above ${opts.chainThreshold * 100}%.`, ); return { lines, failed }; } @@ -660,9 +903,23 @@ async function main() { ); process.exit(1); } - if (probeClaudeVersion(parseClaudeVersion) === null) { + if (!AGENTS.includes(AGENT)) { console.error( - "skills:eval — no usable `claude` CLI on PATH. This eval needs one.", + `skills:eval — AGENT must be one of ${AGENTS.join(", ")} (got \`${AGENT}\`).`, + ); + process.exit(1); + } + const version = probeCliVersion( + AGENT, + AGENT === "claude" ? parseClaudeVersion : parseCopilotVersion, + ); + if (version === null) { + console.error( + AGENT === "claude" + ? "skills:eval — no usable `claude` CLI on PATH. This eval needs one." + : "skills:eval — no usable `copilot` CLI on PATH. Install it with " + + "`npm install -g @github/copilot` and sign in (`copilot` then " + + "`/login`, or export COPILOT_GITHUB_TOKEN), then re-run.", ); process.exit(1); } @@ -675,11 +932,19 @@ async function main() { process.exit(1); } + for (const signal of ["SIGINT", "SIGTERM"]) { + process.once(signal, () => { + stopLiveCopilotRuns(); + process.exit(130); + }); + } + const jobs = cases.flatMap((c) => Array.from({ length: RUNS }, () => c)); const results = await pool(jobs, CONCURRENCY, async (c) => ({ c, invoked: await runPrompt(c.prompt, { maxTurns: isChainCase(c) ? CHAIN_MAX_TURNS : 1, + agent: AGENT, }), })); @@ -687,6 +952,7 @@ async function main() { threshold: THRESHOLD, chainThreshold: CHAIN_THRESHOLD, chainMaxTurns: CHAIN_MAX_TURNS, + agent: AGENT, }); for (const line of lines) console.log(line); process.exit(failed > 0 ? 1 : 0); @@ -697,6 +963,7 @@ if ( path.resolve(process.argv[1]) === fileURLToPath(import.meta.url) ) { main().catch((e) => { + stopLiveCopilotRuns(); console.error(e.message ?? e); process.exit(1); }); diff --git a/scripts/skill-eval.test.mjs b/scripts/skill-eval.test.mjs index 97d4b5b5a..d9a38d004 100644 --- a/scripts/skill-eval.test.mjs +++ b/scripts/skill-eval.test.mjs @@ -23,6 +23,11 @@ import { passesThreshold, collectCases, collectSkillInvocations, + collectCopilotSkillInvocations, + agentArgs, + parseCopilotVersion, + killTree, + stopLiveCopilotRuns, runRejection, invokedSkillNames, runPrompt, @@ -478,8 +483,8 @@ test("the report keeps the two measurements in separate columns", () => { const text = lines.join("\n"); assert.match(text, /First move \(1 turn\)/); assert.match(text, /Hand-off \(14 turns\)/); - assert.match(text, /1\/1 first-move cases at or above 80%\./); - assert.match(text, /0\/1 hand-off cases above 50%\./); + assert.match(text, /1\/1 claude first-move cases at or above 80%\./); + assert.match(text, /0\/1 claude hand-off cases above 50%\./); // One summary line per kind, and no line that merges them. assert.equal(text.match(/cases (at or above|above)/g).length, 2); assert.equal(failed, 1, "the chained case is short, the direct one is not"); @@ -510,7 +515,10 @@ test("a single-kind selection reports only that kind, and says so", () => { new Set(["test-servers"]), OPTS, ); - assert.match(only.lines.join("\n"), /No hand-off cases in this selection\./); + assert.match( + only.lines.join("\n"), + /No claude hand-off cases in this selection\./, + ); assert.doesNotMatch(only.lines.join("\n"), /Hand-off \(14 turns\)/); assert.equal(only.failed, 0); @@ -524,7 +532,7 @@ test("a single-kind selection reports only that kind, and says so", () => { ); const text = chainOnly.lines.join("\n"); assert.doesNotMatch(text, /first-move cases/); - assert.match(text, /1\/1 hand-off cases above 50%\./); + assert.match(text, /1\/1 claude hand-off cases above 50%\./); assert.equal(chainOnly.failed, 0); }); @@ -737,3 +745,311 @@ test("collection fails loudly rather than silently shrinking the set", () => { ); rmSync(root, { recursive: true, force: true }); }); + +// --- Copilot (#2397) -------------------------------------------------------- + +/** One Copilot model call, with the tool requests it made. */ +const copilotMessage = (...requests) => + JSON.stringify({ + type: "assistant.message", + data: { + toolRequests: requests.map(([name, args]) => ({ + name, + arguments: args, + type: "function", + })), + }, + }) + "\n"; +const copilotSkill = (skill) => ["skill", { skill }]; +const copilotResult = (exitCode) => + JSON.stringify({ type: "result", exitCode }) + "\n"; + +test("collectCopilotSkillInvocations reads skill requests, one turn per model call", () => { + const text = + copilotMessage(copilotSkill("pr-flow"), ["view", { path: "/x" }]) + + JSON.stringify({ type: "assistant.message_delta", data: {} }) + + "\nnot json\n" + + copilotMessage(copilotSkill("board-ops"), copilotSkill("issue-create")) + + copilotMessage() + + copilotResult(0); + const parsed = collectCopilotSkillInvocations(text); + assert.deepEqual(parsed.invoked, [ + { payload: '{"skill":"pr-flow"}', turn: 1 }, + { payload: '{"skill":"board-ops"}', turn: 2 }, + { payload: '{"skill":"issue-create"}', turn: 2 }, + ]); + assert.equal(parsed.nextTurn, 3); + assert.equal(parsed.result, "success"); + // Two skills in one model call are not a hand-off, exactly as for Claude. + assert.equal(chainHit(["board-ops", "issue-create"], parsed.invoked), false); + assert.equal(chainHit(["pr-flow", "board-ops"], parsed.invoked), true); +}); + +test("collectCopilotSkillInvocations maps the exit code and holds back a partial line", () => { + const whole = copilotMessage(copilotSkill("testing")); + const first = collectCopilotSkillInvocations(whole.slice(0, 20)); + assert.deepEqual(first.invoked, []); + const second = collectCopilotSkillInvocations( + first.rest + whole.slice(20) + copilotResult(1), + first.nextTurn, + ); + assert.equal(second.invoked.length, 1); + assert.equal(second.result, "exit_1"); + assert.match(runRejection({ result: second.result, code: 1 }), /exit_1/); + // A request with no arguments is recorded, and simply names nothing. + const bare = collectCopilotSkillInvocations( + JSON.stringify({ + type: "assistant.message", + data: { toolRequests: [{ name: "skill" }] }, + }) + "\n", + ); + assert.deepEqual(bare.invoked, [{ payload: "{}", turn: 1 }]); +}); + +test("a Copilot run bounds availability, not just approval", () => { + const args = agentArgs("copilot", 1); + const after = (flag) => args[args.indexOf(flag) + 1]; + // `--available-tools` is what the model can see; `--allow-tool` only spares + // a prompt, so it alone would bound nothing. + assert.equal(after("--available-tools"), "view,glob,grep,skill"); + assert.equal(after("--allow-tool"), "view,glob,grep,skill"); + const denied = args.flatMap((a, i) => + args[i - 1] === "--deny-tool" ? [a] : [], + ); + assert.deepEqual(denied, ["shell", "write", "url"]); + for (const flag of [ + "--disable-builtin-mcps", + "--disallow-temp-dir", + "--no-ask-user", + "--no-auto-update", + ]) { + assert.ok(args.includes(flag), `${flag} must be set`); + } + // The prompt arrives on stdin; `-p` would demand it in argv. + assert.ok(!args.includes("-p") && !args.includes("--prompt")); + // Copilot has no turn flag at all — the budget is enforced by `runPrompt`. + assert.ok(!args.includes("--max-turns")); + assert.throws(() => agentArgs("cursor", 1), /unknown agent `cursor`/); +}); + +/** A fake child that emits the given chunks, recording whether it was killed. */ +function fakeCopilot(chunks, { code = 0 } = {}) { + const state = { command: null, killed: 0, written: null }; + const spawnFn = (command) => { + state.command = command; + const child = new EventEmitter(); + child.stdout = new EventEmitter(); + child.stdin = { end: (t) => (state.written = t) }; + queueMicrotask(() => { + for (const c of chunks) child.stdout.emit("data", Buffer.from(c)); + child.emit("close", state.killed > 0 ? null : code); + }); + return child; + }; + const killFn = () => state.killed++; + return { state, spawnFn, killFn }; +} + +test("runPrompt stops a Copilot run once it has made its budgeted moves", async () => { + const { state, spawnFn, killFn } = fakeCopilot([ + copilotMessage(copilotSkill("pr-flow")), + copilotMessage(copilotSkill("board-ops")), + ]); + const invoked = await runPrompt("take it to a PR", { + agent: "copilot", + spawnFn, + killFn, + platform: "linux", + }); + assert.equal(state.command, "copilot"); + assert.equal(state.written, "take it to a PR"); + assert.equal(state.killed, 1, "stopped exactly once"); + // Only the first move is scored: the second call arrived after the stop. + assert.deepEqual( + invoked.map((e) => e.payload), + ['{"skill":"pr-flow"}'], + ); +}); + +test("runPrompt never scores a Copilot turn past the budget, however the pipe splits it", async () => { + // Both calls and the result in ONE chunk: stopping the process is too late + // to keep turn 2 out, and the result event alone would skip the stop. + const { spawnFn, killFn } = fakeCopilot([ + copilotMessage() + + copilotMessage(copilotSkill("board-ops")) + + copilotResult(0), + ]); + const invoked = await runPrompt("p", { + agent: "copilot", + spawnFn, + killFn, + }); + assert.deepEqual(invoked, []); + assert.equal(sampleHit(null, invoked, new Set(["board-ops"])), true); +}); + +test("runPrompt accepts a Copilot run that finished inside its budget", async () => { + const { state, spawnFn, killFn } = fakeCopilot([ + copilotMessage(copilotSkill("testing")), + copilotMessage(), + copilotResult(0), + ]); + const invoked = await runPrompt("p", { + agent: "copilot", + maxTurns: 14, + spawnFn, + killFn, + }); + assert.equal(state.killed, 0); + assert.equal(sampleHit("testing", invoked), true); +}); + +test("runPrompt rejects a Copilot run that never observed anything", async () => { + // An unauthenticated CLI prints its login hint and exits 1 with no events; + // that must not read as "no skill fired". + const { spawnFn, killFn } = fakeCopilot(["To authenticate…\n"], { + code: 1, + }); + await assert.rejects( + runPrompt("p", { agent: "copilot", spawnFn, killFn }), + /`copilot` produced no terminal `result` event \(exit 1\) — if it is not signed in, run `copilot` and `\/login`/, + ); + const failed = fakeCopilot([copilotResult(2)], { code: 2 }); + await assert.rejects( + runPrompt("p", { agent: "copilot", ...failed }), + (e) => /ended `exit_2`/.test(e.message) && !/signed in/.test(e.message), + ); +}); + +test("parseCopilotVersion reads the CLI's banner", () => { + assert.equal( + parseCopilotVersion("GitHub Copilot CLI 1.0.85.\nRun 'copilot update'"), + "1.0.85", + ); + assert.equal(parseCopilotVersion("copilot: command not found"), null); +}); + +test("the report names the agent it measured", () => { + const direct = { prompt: "d", expect: "testing" }; + const text = formatReport( + [direct], + samples(direct, 3, 3), + new Set(["testing"]), + { + ...OPTS, + agent: "copilot", + }, + ).lines.join("\n"); + assert.match(text, /First move \(1 turn\) — copilot/); + assert.match(text, /1\/1 copilot first-move cases at or above 80%\./); + assert.match(text, /No copilot hand-off cases in this selection\./); + assert.doesNotMatch(text, /claude/); +}); + +test("an unknown AGENT is rejected before anything runs", () => { + const res = spawnSync(process.execPath, [SCRIPT_PATH], { + env: { ...process.env, AGENT: "cursor" }, + encoding: "utf8", + }); + assert.equal(res.status, 1); + assert.match( + res.stderr, + /AGENT must be one of claude, copilot \(got `cursor`\)/, + ); +}); + +test("a Copilot run gets its own process group on POSIX, and none on Windows", () => { + // The native binary the wrapper starts ignores the wrapper's SIGTERM, so + // only a group signal reaches it. + const seen = {}; + for (const platform of ["linux", "win32"]) { + const { spawnFn, killFn } = fakeCopilot([copilotResult(0)]); + runPrompt("p", { + agent: "copilot", + platform, + killFn, + spawnFn: (c, a, options) => { + seen[platform] = options.detached; + return spawnFn(c, a, options); + }, + }).catch(() => {}); + } + assert.equal(seen.linux, true); + assert.equal(seen.win32, undefined); + // Claude is bounded by `--max-turns` and never killed, so it stays attached. + let claudeDetached; + runPrompt("p", { + platform: "linux", + spawnFn: (_c, _a, options) => { + claudeDetached = options.detached; + const c = new EventEmitter(); + c.stdout = new EventEmitter(); + c.stdin = { end: () => {} }; + queueMicrotask(() => c.emit("close", 0)); + return c; + }, + }).catch(() => {}); + assert.equal(claudeDetached, undefined); +}); + +test("killTree signals the whole group on POSIX and the tree on Windows", () => { + const kills = []; + killTree( + { pid: 4242, kill: () => assert.fail("not the leader alone") }, + "darwin", + { + killProcess: (pid, signal) => kills.push([pid, signal]), + }, + ); + assert.deepEqual(kills, [[-4242, "SIGTERM"]]); + + // A group that already exited is not an error. + assert.doesNotThrow(() => + killTree({ pid: 4242, kill: () => {} }, "linux", { + killProcess: () => { + throw Object.assign(new Error("ESRCH"), { code: "ESRCH" }); + }, + }), + ); + + const spawned = []; + killTree({ pid: 4242, kill: () => {} }, "win32", { + spawnFn: (c, a) => spawned.push([c, ...a]), + }); + assert.deepEqual(spawned, [["taskkill", "/pid", "4242", "/T", "/F"]]); + + // A child that never started has nothing to stop. + killTree({ pid: undefined, kill: () => {} }, "linux", { + killProcess: () => assert.fail("no pid, no signal"), + }); +}); + +test("stopLiveCopilotRuns stops every run in flight, once", () => { + // The early-exit path: one sample rejects, `main().catch` exits, and the + // other detached groups would otherwise keep spending model calls. + const stopped = []; + const runs = new Set( + ["a", "b"].map((id) => ({ + child: { id }, + platform: "linux", + killFn: (child, platform) => stopped.push([child.id, platform]), + })), + ); + assert.equal(stopLiveCopilotRuns(runs), 2); + assert.deepEqual(stopped, [ + ["a", "linux"], + ["b", "linux"], + ]); + assert.equal(stopLiveCopilotRuns(runs), 0, "nothing is signalled twice"); +}); + +test("a rejected Copilot run leaves the in-flight set", async () => { + // A run that closed is no longer live, so the cleanup never signals a pid + // that may since have been reused. + const { spawnFn, killFn } = fakeCopilot([copilotResult(3)], { code: 3 }); + await assert.rejects( + runPrompt("p", { agent: "copilot", spawnFn, killFn }), + /exit_3/, + ); + assert.equal(stopLiveCopilotRuns(), 0); +});