feat(cli): add agent-relay observer to mint read-only follow-along links - #1422
Conversation
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (11)
🚧 Files skipped from review as they are similar to previous changes (9)
Included review availability: Your plan provides up to 4 included reviews per hour; 3 remain after this review. 📝 WalkthroughWalkthroughThe PR adds scoped observer-token creation, listing, revocation, and URL generation to the SDK, CLI, and MCP interface. It updates observation guidance and reclassifies the unreleased version as Minor. ChangesObserver token workflows
Estimated code review effort: 3 (Moderate) | ~25 minutes Merge Risk: ⚪ Minimal · up to The PR adds scoped observer-token workflows and updates related guidance; no actionable merge-blocking risk remains after normal checks and review. Sequence Diagram(s)sequenceDiagram
participant Agent
participant MCP
participant RelayCast
participant ObserverDashboard
Agent->>MCP: request get_observer_url
MCP->>RelayCast: create scoped read-only token
RelayCast-->>MCP: return token metadata and token material
MCP->>ObserverDashboard: construct observer URL
MCP-->>Agent: return URL and token metadata
Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches 💡 1📝 Generate docstrings 💡
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 992f1d9df6
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
🧹 Nitpick comments (1)
packages/cli/src/cli/agent-relay-mcp.ts (1)
582-643: 🧹 Nitpick | 🔵 TrivialConsider an audit trail for agent-minted observer links.
get_observer_urllets an AI agent mint a follow-along link without an explicit CLI invocation by a human. Consider emitting a telemetry event or log line when a token is created through this tool, distinct fromagent-relay observer, so workspace owners can audit who exposed a follow-along view and when.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/cli/src/cli/agent-relay-mcp.ts` around lines 582 - 643, The get_observer_url tool currently mints observer tokens without recording an audit event. Add a telemetry event or log entry immediately after successful createObserverToken validation, including the workspace, creation time, token identifier, and MCP-tool source; keep it distinct from the agent-relay observer command and avoid logging the token material or generated URL.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In `@packages/cli/src/cli/agent-relay-mcp.ts`:
- Around line 582-643: The get_observer_url tool currently mints observer tokens
without recording an audit event. Add a telemetry event or log entry immediately
after successful createObserverToken validation, including the workspace,
creation time, token identifier, and MCP-tool source; keep it distinct from the
agent-relay observer command and avoid logging the token material or generated
URL.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 44ceb656-b252-46d8-ac8c-270d1bb53ad2
📒 Files selected for processing (11)
CHANGELOG.mdpackages/cli/src/cli/agent-relay-mcp.tspackages/cli/src/cli/bootstrap.test.tspackages/cli/src/cli/bootstrap.tspackages/cli/src/cli/commands/observer.test.tspackages/cli/src/cli/commands/observer.tspackages/cli/src/cli/lib/observer-url.tspackages/sdk/src/__tests__/thin-client.test.tspackages/sdk/src/messaging/thin-client.tsplugins/codex-relay-skill/SKILL.mdplugins/gemini-relay-extension/GEMINI.md
There was a problem hiding this comment.
All reported issues were addressed
Reply with feedback, questions, or to request a fix.
Re-trigger cubic
…n observer link (#92) The orchestrator is exactly the role that should give a watching human a way to follow a run, and this skill never mentioned observation at all — no observer link, no warning against building one from the workspace key. That silence is why leads reach for `?key=<rk_live_...>`: it is the only pattern they have seen. Adds `agent-relay observer` (and the `get_observer_url` MCP equivalent) in the four places a lead actually looks: the Quick Reference table, a new bootstrap step after spawning, the paste-worthy protocol template, and the Common Mistakes table. Each states the default posture — read-only, 24h expiry, agent DMs excluded — and why a workspace key can never take its place. Requires the command and tool from AgentWorkforce/relay#1422. Claude-Session: https://claude.ai/code/session_01Jmke9G9s7ftrN49opNmdx1 Co-authored-by: Claude <noreply@anthropic.com>
…links Sharing a live view of a workspace had no supported path. The engine rejects a workspace key on the realtime endpoint and only accepts a scoped `ot_live_` observer token, but nothing in the CLI or the MCP server minted one — the only routes were a hand-rolled `POST /v1/observer-tokens` or pasting an admin key into the dashboard login. So the bundled instructions told agents that observation "requires a separately provisioned observer token" and to omit the link when none exists, which in practice meant never showing one. Adds the missing primitive: - `agent-relay observer` mints a scoped token and prints the observer URL built from it. `observer list` / `observer revoke <id>` manage tokens. - `get_observer_url` MCP tool does the same for an orchestrating agent. - `@agent-relay/sdk` exports `createObserverToken`, `listObserverTokens`, and `revokeObserverToken`. Defaults are deliberately tighter than the observer dashboard's own auto-minted token (30 days, DMs included): that one backs a browser session behind an httpOnly cookie, while this one is printed as a URL meant to be pasted into chat. 24 hours, agent DMs excluded, widened explicitly via `--expires` / `--include-dms` / `--channels`. `observerUrl` refuses any credential that is not an `ot_live_` token, so the workspace-key-in-a-URL failure this command exists to prevent cannot be reintroduced by a later caller. Updates the Codex skill and Gemini extension instructions to point at the command instead of describing the link as unobtainable. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Jmke9G9s7ftrN49opNmdx1
Five issues from review, all confirmed:
- `observer list` / `observer revoke` read only their local option object,
but Commander binds a repeated option to the ancestor that declared it
first. Verified empirically: the subcommand's local opts are `{}`, so
`--json` and `--workspace-key` were silently dropped and ambient
credentials used instead. Both now read `optsWithGlobals()`.
- `resolveObserverBaseUrl` accepted any parseable URL. Since the token is
appended to the query string, a `data:`/custom-scheme value would carry
a live credential somewhere unintended. Restricted to http/https.
- Both the CLI command and the `get_observer_url` MCP tool minted the
token before resolving the dashboard URL, so an invalid
`RELAY_OBSERVER_URL` left a live 24-hour token behind and then threw
before returning it. The URL is now resolved first.
- `--channels` enforced its cap before collapsing duplicates, so
`a,a,a,…` could be rejected for exceeding a limit it never reached.
- The test suite mutated `process.env` directly, leaking a workspace key
into every later test file. Uses `vi.stubEnv` + `vi.unstubAllEnvs`.
Also fixes the test harness, which returned its default mocks rather
than the overrides actually registered — an overridden mock could never
be asserted against. Adds four regression tests (subcommand flags,
validate-before-mint, duplicate channels, rejected schemes).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Jmke9G9s7ftrN49opNmdx1
fcaacb0 to
552944f
Compare
|
Note GitHub couldn't provide a complete incremental comparison for this pull request, so CodeRabbit is performing a full review instead. This review may take a little longer. |
|
Note GitHub couldn't provide a complete incremental comparison for this pull request, so CodeRabbit is performing a full review instead. This review may take a little longer. |
…tokens The three pattern skills each mandated printing the raw workspace key in an observer URL — "Do not print a placeholder — print the real URL the user can click. This is mandatory." A workspace key is an administrative credential; a URL query string is the worst place to put one. They also required the key be copied into every worker prompt. That is unnecessary: `create_workspace` and `set_workspace_key` both pin the workspace to the project, and the relay MCP server a `relay-worker` subagent starts resolves that pin on its own. The mandate put an admin credential into N prompts and N transcripts to achieve nothing. - Step 3 now calls `get_observer_url`, which mints a scoped, expiring, read-only token. Requires the tool added in AgentWorkforce/relay#1422. - Worker prompts no longer carry the workspace key; the worker agent definition and SubagentStart hook now say the workspace is inherited and that a key must never be printed or requested. - The three skills' shared setup steps are now byte-identical, and all three carry the worker/stage tracking table and the "do not self-release" instruction that only relay-team had. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Jmke9G9s7ftrN49opNmdx1
Seven findings, all confirmed: - **Foreground pipeline deadlock (P1).** "Do NOT release yourself — stay idle" is right for backgrounded team/fanout workers but wrong for pipeline, whose stages run in the foreground: a worker that stays idle never returns, so the blocking Agent call never completes and the lead can neither read the DONE nor spawn the next stage. Pipeline workers now end their turn after DONE. The distinction between ending a turn and releasing a relay identity (`remove_agent`) is now stated explicitly in all three skills, since conflating them caused this. - **Concurrent runs share the project pin (P1).** The pin is last-writer-wins, so two leads in one checkout can cross workers into each other's workspace. Documented the one-team-per-checkout constraint and pointed at the ACK gate that already detects it — a worker in the wrong workspace finds no assignment and cannot ACK. - **Registration failure had no usable channel.** The old text said to report it "to your lead", but `send_dm` needs the registration that just failed. The worker now stops without retrying and makes the error its final response, which is what the lead gets back from the Agent call. Same for an empty inbox, the symptom of the pin race above. - **Plugin version was not bumped.** Marketplace clients use it to detect updates, so existing installs would have stayed on the prompts that leak workspace keys. Bumped to 0.2.0 in `plugin.json`, `package.json`, and the marketplace entry. - **`get_observer_url` may not be present yet** — it ships in AgentWorkforce/relay#1422. The instruction now falls back to `agent-relay observer` so a lead is never stuck on a missing tool. - **Later pipeline stages had no handoff requirement**, unlike stage 1; the final stage now also owes evidence. - **MD040**: gave the spawn examples a `text` language. Also drops the now-stale "if any of steps 1-2 fail, retry once" from the worker definition — step 2 became a standing prohibition, not a fallible action. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Jmke9G9s7ftrN49opNmdx1
…tokens The three pattern skills each mandated printing the raw workspace key in an observer URL — "Do not print a placeholder — print the real URL the user can click. This is mandatory." A workspace key is an administrative credential; a URL query string is the worst place to put one. They also required the key be copied into every worker prompt. That is unnecessary: `create_workspace` and `set_workspace_key` both pin the workspace to the project, and the relay MCP server a `relay-worker` subagent starts resolves that pin on its own. The mandate put an admin credential into N prompts and N transcripts to achieve nothing. - Step 3 now calls `get_observer_url`, which mints a scoped, expiring, read-only token. Requires the tool added in AgentWorkforce/relay#1422. - Worker prompts no longer carry the workspace key; the worker agent definition and SubagentStart hook now say the workspace is inherited and that a key must never be printed or requested. - The three skills' shared setup steps are now byte-identical, and all three carry the worker/stage tracking table and the "do not self-release" instruction that only relay-team had. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Jmke9G9s7ftrN49opNmdx1
Observer had no page in the current docs. The slug exists only in the
v7.1.1 archive and the legacy nav, so `/docs/observer` redirects into the
archive and the current sidebar never mentions the feature at all.
The archived page also documents the only method it knew:
https://agentrelay.com/observer?key=<workspace_key>
A workspace key is an administrative credential — it can send messages,
spawn agents, and change workspace settings — and a query string is not a
place to put one. Current releases ship `agent-relay observer`, which
mints a scoped, expiring, read-only `ot_live_` token and builds the link
from that.
- Adds `web/content/docs/observer.mdx` covering `agent-relay observer`,
the `--channels` / `--include-dms` / `--expires` narrowing, `observer
list` / `revoke`, the `get_observer_url` MCP tool, and a capability
table contrasting a workspace key with an observer token.
- Registers the slug under Interfaces in `docsNav`, so the page is
reachable and `/docs/observer` resolves to current docs instead of
redirecting to the archive.
- Leaves the archived v7.1.1 page in place but adds a note that the
pattern is unsafe on a current release, linking to the new page. The
link is absolute because `rewriteLegacyDocsLinks` rewrites bare
`/docs/` links back into `/docs/7.1.1/`, which would have pointed it at
itself.
Depends on the `agent-relay observer` command in AgentWorkforce/relay#1422.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Jmke9G9s7ftrN49opNmdx1
Seven findings, all confirmed: - **Foreground pipeline deadlock (P1).** "Do NOT release yourself — stay idle" is right for backgrounded team/fanout workers but wrong for pipeline, whose stages run in the foreground: a worker that stays idle never returns, so the blocking Agent call never completes and the lead can neither read the DONE nor spawn the next stage. Pipeline workers now end their turn after DONE. The distinction between ending a turn and releasing a relay identity (`remove_agent`) is now stated explicitly in all three skills, since conflating them caused this. - **Concurrent runs share the project pin (P1).** The pin is last-writer-wins, so two leads in one checkout can cross workers into each other's workspace. Documented the one-team-per-checkout constraint and pointed at the ACK gate that already detects it — a worker in the wrong workspace finds no assignment and cannot ACK. - **Registration failure had no usable channel.** The old text said to report it "to your lead", but `send_dm` needs the registration that just failed. The worker now stops without retrying and makes the error its final response, which is what the lead gets back from the Agent call. Same for an empty inbox, the symptom of the pin race above. - **Plugin version was not bumped.** Marketplace clients use it to detect updates, so existing installs would have stayed on the prompts that leak workspace keys. Bumped to 0.2.0 in `plugin.json`, `package.json`, and the marketplace entry. - **`get_observer_url` may not be present yet** — it ships in AgentWorkforce/relay#1422. The instruction now falls back to `agent-relay observer` so a lead is never stuck on a missing tool. - **Later pipeline stages had no handoff requirement**, unlike stage 1; the final stage now also owes evidence. - **MD040**: gave the spawn examples a `text` language. Also drops the now-stale "if any of steps 1-2 fail, retry once" from the worker definition — step 2 became a standing prohibition, not a fallible action. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Jmke9G9s7ftrN49opNmdx1
* docs: add a current Observer page built on scoped observer tokens
Observer had no page in the current docs. The slug exists only in the
v7.1.1 archive and the legacy nav, so `/docs/observer` redirects into the
archive and the current sidebar never mentions the feature at all.
The archived page also documents the only method it knew:
https://agentrelay.com/observer?key=<workspace_key>
A workspace key is an administrative credential — it can send messages,
spawn agents, and change workspace settings — and a query string is not a
place to put one. Current releases ship `agent-relay observer`, which
mints a scoped, expiring, read-only `ot_live_` token and builds the link
from that.
- Adds `web/content/docs/observer.mdx` covering `agent-relay observer`,
the `--channels` / `--include-dms` / `--expires` narrowing, `observer
list` / `revoke`, the `get_observer_url` MCP tool, and a capability
table contrasting a workspace key with an observer token.
- Registers the slug under Interfaces in `docsNav`, so the page is
reachable and `/docs/observer` resolves to current docs instead of
redirecting to the archive.
- Leaves the archived v7.1.1 page in place but adds a note that the
pattern is unsafe on a current release, linking to the new page. The
link is absolute because `rewriteLegacyDocsLinks` rewrites bare
`/docs/` links back into `/docs/7.1.1/`, which would have pointed it at
itself.
Depends on the `agent-relay observer` command in AgentWorkforce/relay#1422.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Jmke9G9s7ftrN49opNmdx1
* docs: document observer in the CLI and MCP references
Review found the new Observer page pointing at two reference pages that
did not actually cover what it promised, and one link into archived docs.
- `reference-cli.mdx` called itself the complete command matrix but had
no `observer` entry. Adds an Observer section covering the three
commands and every flag.
- `agent-relay-mcp.mdx` did not list `get_observer_url`. Adds it to the
workspace tool inventory plus a section with its inputs, its return
shape, and the workspace-key warning.
- The "When to use it" link to `/docs/relay-dashboard` resolved to a
legacy-only slug, so it silently redirected into the v7.1.1 archive —
the same failure this PR set out to fix for Observer. The link is now
explicitly labelled as the archive, since Relay Dashboard has no
current page yet.
Every internal `/docs/` link on the three touched pages now resolves
within the current docs, except the one deliberate archive link.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Jmke9G9s7ftrN49opNmdx1
* docs: clarify that observer parses its own connection flags
Review flagged an internal inconsistency: the Observer section says all
three commands accept `--workspace-key` and `--base-url`, but the
"Common SDK Options" list of SDK-backed groups does not include
`observer`.
Adding it to that list would have been wrong. `observer` does not use
`addSdkOptions` — it declares those two flags itself and has no `--token`
at all, because minting an observer token requires the workspace key and
an agent token cannot do it. Listing it as SDK-backed would have implied
`--token` works there.
Says so explicitly instead.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Jmke9G9s7ftrN49opNmdx1
---------
Co-authored-by: Claude <noreply@anthropic.com>
Summary
Letting a human watch a workspace live had no supported path, so the guidance we ship told agents not to offer one.
The engine rejects a workspace key on the realtime endpoint (
GET /v1/ws) and accepts only a scopedot_live_observer token. But nothing in the CLI or MCP server minted one — greppingpackages/cli/srcfor "observer" returned exactly one hit,cloud-room.ts:197, which rejects observer tokens. The only real routes were a hand-rolledPOST /v1/observer-tokensor pasting an admin key into the dashboard login so it mints one server-side.So
plugins/codex-relay-skill/SKILL.mdandplugins/gemini-relay-extension/GEMINI.md(from #1405) said observation "requires a separately provisioned, read-only observer token" and to "omit the observer link" when none exists. Correct about the danger, but with no reachable happy path it means nobody ever gets a link. Meanwhile the skills repo'srelay-team/relay-fanout/relay-pipelineskills mandate the opposite — printing the rawrk_live_key in a URL, "This is mandatory" — which is the thing #1380 is meant to stop. This PR gives all of them one satisfiable answer.What's added
agent-relay observer— mints a scoped token and prints the observer URL built from it.observer listandobserver revoke <id>manage existing tokens.get_observer_urlMCP tool — same thing for an orchestrating agent, so a lead can hand the user a link without shelling out.createObserverToken,listObserverTokens,revokeObserverTokenfrom@agent-relay/sdk.The frontend already works:
RelaySessionProvider.tsx:29in relaycast acceptsot_live_in?key=exactly likerk_live_, so a scoped token is a drop-in URL replacement with no dashboard change.Defaults, and why they differ from the dashboard's
The observer dashboard already auto-mints a token on workspace-key login (
observer-dashboard/src/lib/observer-token.ts): 30-day expiry,include_dms: true. That is right for a token backing a browser session behind an httpOnly cookie. It is wrong for a token printed as a URL and pasted into chat, so this command defaults to 24 hours with agent DMs excluded, widened explicitly:These two defaults are the main judgement calls in this PR — easy to change if you'd rather match the dashboard.
observerUrl()throws on any credential that isn't anot_live_token, so the workspace-key-in-a-URL failure this command exists to prevent can't be reintroduced by a later caller.Also updates the Codex skill and Gemini extension text to point at the command rather than describing the link as unobtainable.
Review fixes (552944f)
Five findings, all confirmed and fixed:
observer list/revokeread only their local option object, but Commander binds a repeated option to the ancestor that declared it first. Reproduced directly — the child's local opts are{}, so both--jsonand--workspace-keywere ignored. Now readsoptsWithGlobals().data:/custom-scheme value would carry a live credential somewhere unintended. Restricted to http/https (plain http kept for local dashboards).RELAY_OBSERVER_URLleft a live 24-hour token behind and then threw before returning it. URL now resolved first.--channelsenforced its cap before deduping, soa,a,a,…could be rejected for a limit it never reached.process.envdirectly, leaking a workspace key into every later test file. Nowvi.stubEnv+vi.unstubAllEnvs.Fixing these also surfaced a bug in the test harness itself:
setup()returned its default mocks rather than the overrides actually registered, so an overridden mock could never be asserted against. Fixed, with four new regression tests.Test Plan
CHANGELOG.mdandbootstrap.test.tspackages/cli/src/cli/commands/observer.test.ts— 13 tests, including regressions for all four behavioural fixes abovepackages/sdk/src/__tests__/thin-client.test.ts— 3 tests covering the SDK helper's own scope/filter defaults, which the CLI tests stub outpackages/clisuite — 1222 passed, 11 skippednpm run typecheckclean (exit 0)node-definition-loader.bun.test.tsare pre-existing — verified identical failures on a cleanorigin/mainworktree (Bun isn't installed in this environment)bootstrap.test.tsinventory updated.observeris a group with a default action, so the leaf-path walk skips it — added an explicit assertion so the primary command can't be dropped silentlyagent-relay observerbefore merge to confirm the minted token opens the dashboardScreenshots
n/a