diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 63c8555b..4ed3b0f1 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -474,6 +474,31 @@ Remove the `mobile-dev` development entry before installing the plugin to avoid duplicate servers. Public releases use `npm run build:release` and `npm run package:cursor -- --release`. +### T3 Code local development + +T3 Code runs the Claude Code and Codex plugins that its providers have installed. +To test a local build in T3 Code, package both plugins and install them: + +```sh +npm run build +npm run package +npm run package:claude +codex plugin marketplace add ./release/marketplace +codex plugin add mobile-dev@mobile-dev-local +claude plugin marketplace add ./release/claude-plugin +claude plugin install mobile-dev@mobile-dev-local +``` + +Set `MOBILE_DEV_PANEL=browser` on T3 Code's Codex instance, as the +[installation steps](README.md#t3-code) describe. The Codex manifest forwards that +variable to the server, which then serves the panel as a browser page like Claude's +server does: `mobile_open_simulator` has no `ui.resourceUri`, so T3 Code renders no +inline app, and its result carries the `browserPanel` link that the agent opens with +`preview_open`. The shared server in `dist/server.mjs` reads the variable too. An +unknown value logs a warning to the server's stderr and keeps the default placement. +`npm run test:package` checks both placements of the packaged Codex server. After you +reinstall a plugin, start a new thread so that T3 Code starts the new copy. + ## Live React UI development In Codex, run `npm run dev` after installing the plugin. It watches the UI sources and diff --git a/README.md b/README.md index d40cb85f..c021ff6e 100644 --- a/README.md +++ b/README.md @@ -110,6 +110,32 @@ It sends plugin errors and aggregate measurements to Sentry, as described under [Privacy and support](#privacy-and-support), and nothing else to Callstack; profiles go only to the TraceSift service on your Mac. +### T3 Code + +[T3 Code](https://github.com/pingdotgg/t3code) runs Claude Code and Codex, so it loads +the Mobile Dev plugin that each of them has installed. On the Mac that runs T3 Code, +install the [Claude Code plugin](#claude-code-and-claude-desktop) for Claude threads +and the [Codex plugin](#codex) for Codex threads. In both, the panel opens in T3 Code's +Browser. + +Turn off T3 Code's own device integration while you use Mobile Dev. In +**Settings → Devices**, turn off **Use devices**, which also turns off **Agents can use +devices**. Otherwise T3 Code gives agents its own `device_*` tools, with the same names +as Mobile Dev's, and tells them to prefer those tools. + +Codex threads need one setting, because T3 Code renders Codex's panel in the chat +and asks you to confirm each action that the panel takes, such as each tap: + +1. Open **Settings → Providers**, then the Codex instance. +2. Under **Variables**, click **Add variable**. Set `MOBILE_DEV_PANEL` to `browser`. +3. Start a new thread. + +Ask the agent to open Mobile Dev. It opens the panel with T3 Code's `preview_open` +tool, and T3 Code shows the page as a floating preview. To dock the panel beside the +chat, click the preview's **Open in right panel** button. The agent reads the devices +you select in the panel and the screenshots and screen notes you add from it. +Performance recordings still render as cards in the chat. + ### Cursor: manual installation On an Apple Silicon Mac with Node.js 22.19 or later on `PATH`, obtain the diff --git a/apps/claude/skills/mobile-dev/SKILL.md b/apps/claude/skills/mobile-dev/SKILL.md index f55b8100..453550ad 100644 --- a/apps/claude/skills/mobile-dev/SKILL.md +++ b/apps/claude/skills/mobile-dev/SKILL.md @@ -25,7 +25,13 @@ result: open it in your in-app browser in the foreground, and reuse a tab that a shows Mobile Dev. Opening a page can leave the Browser pane hidden behind a card in the chat, so in the Claude desktop app's Code tab then call `mcp__ccd_view__show_pane` with `pane: "preview"` (load it with ToolSearch first if it is deferred), so the panel shows -beside the chat without the user clicking Open. Open only that link: the `baseUrl` in device results and other +beside the chat without the user clicking Open. In T3 Code, open it with the +`t3-code` server's `preview_open` (load it with ToolSearch first if it is deferred). T3 +shows a page an agent opened as a floating preview that the user can dock beside the +chat, so a result with `visible: false` still counts as shown: do not open it again, +and leave the tab open. +T3's own `mcp__t3-code__device_*` tools drive T3's separate Device panel, so use Mobile +Dev's `device_*` tools for the devices in Mobile Dev's panel. Open only that link: the `baseUrl` in device results and other `127.0.0.1` addresses belong to Mobile Dev's backends, not to the panel, so never open or show them. Without a browser, give the user the link. Then read `mobile_read_panel_context` for the devices the user selected there, and again when diff --git a/apps/codex-plugin/.mcp.json b/apps/codex-plugin/.mcp.json index cf0d2bdb..936aa5c5 100644 --- a/apps/codex-plugin/.mcp.json +++ b/apps/codex-plugin/.mcp.json @@ -14,7 +14,8 @@ "TRACE_SIFT_HOME", "TRACE_SIFT_COMMAND", "MOBILE_DEV_TELEMETRY", - "MOBILE_DEV_ENVIRONMENT" + "MOBILE_DEV_ENVIRONMENT", + "MOBILE_DEV_PANEL" ] } } diff --git a/apps/codex-plugin/server/index.ts b/apps/codex-plugin/server/index.ts index 35e490fb..7ad2f7d3 100644 --- a/apps/codex-plugin/server/index.ts +++ b/apps/codex-plugin/server/index.ts @@ -2,8 +2,12 @@ import "@mobile-dev/core/instrument.ts"; import { runStdioServer } from "@mobile-dev/core/stdio-server.ts"; import { createPanel } from "@mobile-dev/core/panel-tools.ts"; import { agentDeviceTools, readAgentDeviceCatalog } from "@mobile-dev/core/agent-device-tools.ts"; +import { browserPanel } from "@mobile-dev/core/browser-panel.ts"; +import { panelPlacement } from "@mobile-dev/core/panel-placement.ts"; import { codexUpdates } from "./codex-updates.ts"; // Codex's server: the shared server with Mobile Dev's panel and agent-device's commands, plus // the updater for Codex plugin installs. -await runStdioServer({ panel: createPanel(), features: [agentDeviceTools(readAgentDeviceCatalog()), codexUpdates] }); +const inline = panelPlacement("inline") === "inline"; +const features = [agentDeviceTools(readAgentDeviceCatalog()), codexUpdates]; +await runStdioServer({ panel: createPanel({ inline }), features: inline ? features : [...features, browserPanel({ always: true })] }); diff --git a/apps/codex-plugin/test/mcp-launch.test.ts b/apps/codex-plugin/test/mcp-launch.test.ts index 90976b7a..463d42e9 100644 --- a/apps/codex-plugin/test/mcp-launch.test.ts +++ b/apps/codex-plugin/test/mcp-launch.test.ts @@ -32,11 +32,13 @@ test("the launcher uses Codex's cached Node with spaces, no Node on PATH, and no const traceSiftHomeForwarded = config.env_vars.includes("TRACE_SIFT_HOME"); const traceSiftCommandForwarded = config.env_vars.includes("TRACE_SIFT_COMMAND"); const profileForwarded = config.env_vars.includes("CODEX_HOME"); + const panelForwarded = config.env_vars.includes("MOBILE_DEV_PANEL"); assert.ok(homeForwarded); assert.ok(telemetryForwarded); assert.ok(traceSiftHomeForwarded); assert.ok(traceSiftCommandForwarded); assert.ok(profileForwarded); + assert.ok(panelForwarded); const script = 'const args = process.argv.slice(1); const text = JSON.stringify(args); console.log(text);'; const result = launch(fixtureHome, ["--eval", script, "a b", "literal$argument"]); assert.equal(result.error, undefined); diff --git a/apps/codex-plugin/tooling/mcp-tool-budget.mjs b/apps/codex-plugin/tooling/mcp-tool-budget.mjs index 2b5f3b13..508c7b4b 100644 --- a/apps/codex-plugin/tooling/mcp-tool-budget.mjs +++ b/apps/codex-plugin/tooling/mcp-tool-budget.mjs @@ -12,12 +12,15 @@ const { schemaCompactBytes, agentToolBytes, agentTotalBytes } = CODEX_TOOL_LIMIT const plugin = resolve(process.argv[2] ?? "release/marketplace/plugins/mobile-dev"); const manifest = JSON.parse(await readFile(join(plugin, ".mcp.json"), "utf8")); const agentPlugin = await access(join(plugin, "plugin.json")).then(() => true, () => false); +const env = { ...process.env, MOBILE_DEV_TELEMETRY: "off" }; +// Measure the default inline panel, even from a host set up for the browser panel. +delete env.MOBILE_DEV_PANEL; let total = 0; let count = 0; const oversized = []; for (const [name, config] of Object.entries(manifest.mcpServers)) { const client = new Client({ name: "mobile-dev-catalog-budget", version: "1" }); - const transport = new StdioClientTransport({ command: process.execPath, args: [config.args[1]], cwd: plugin, stderr: "pipe", env: { ...process.env, MOBILE_DEV_TELEMETRY: "off" } }); + const transport = new StdioClientTransport({ command: process.execPath, args: [config.args[1]], cwd: plugin, stderr: "pipe", env }); try { await client.connect(transport); const tools = []; diff --git a/apps/codex-plugin/tooling/smoke-codex-plugin.mjs b/apps/codex-plugin/tooling/smoke-codex-plugin.mjs index 1c89832f..35bc20fd 100644 --- a/apps/codex-plugin/tooling/smoke-codex-plugin.mjs +++ b/apps/codex-plugin/tooling/smoke-codex-plugin.mjs @@ -32,6 +32,8 @@ try { await writeFile(catalogPath, catalogText); const fixtureHome = await createLaunchHome(temporary, process.execPath); const env = { ...process.env, CODEX_HOME: profile, HOME: fixtureHome, MOBILE_DEV_TELEMETRY: "off" }; + // These checks expect the default inline panel, even from a host set up for the browser panel. + delete env.MOBILE_DEV_PANEL; function codex(args) { const output = execFileSync(codexCli, args, { env, encoding: "utf8", timeout: 30000 }); return JSON.parse(output); diff --git a/apps/codex-plugin/tooling/smoke-package.mjs b/apps/codex-plugin/tooling/smoke-package.mjs index 7ef976c1..0b9a5b71 100644 --- a/apps/codex-plugin/tooling/smoke-package.mjs +++ b/apps/codex-plugin/tooling/smoke-package.mjs @@ -13,6 +13,7 @@ const temporary = await mkdtemp(join(tmpdir(), "mobile-dev-package-test-")); const plugin = join(temporary, "mobile-dev"); let transport; let runtimeTransport; +let browserTransport; function uiIdentity(text) { const user = text.match(/name="mobile-dev-user-id" content="(anon_[a-f0-9]{32})"/); @@ -43,10 +44,13 @@ try { assert.ok(serverConfig.env_vars.includes("TRACE_SIFT_HOME")); assert.ok(serverConfig.env_vars.includes("TRACE_SIFT_COMMAND")); assert.ok(serverConfig.env_vars.includes("MOBILE_DEV_TELEMETRY")); + assert.ok(serverConfig.env_vars.includes("MOBILE_DEV_PANEL")); const serverCwd = resolve(plugin, serverConfig.cwd); // Supply the Codex runtime cache layout with no Node on MCP PATH. const fixtureHome = await createLaunchHome(temporary, process.execPath); const serverEnv = { ...process.env, HOME: fixtureHome, PATH: "/usr/bin:/bin:/usr/sbin:/sbin" }; + // These checks expect the default inline panel, even from a host set up for the browser panel. + delete serverEnv.MOBILE_DEV_PANEL; const launcher = serverConfig.args[0]; const nodeVersion = spawnSync(serverConfig.command, [launcher, "--version"], { cwd: serverCwd, env: serverEnv, encoding: "utf8" }); assert.equal(nodeVersion.error, undefined); @@ -241,7 +245,29 @@ try { } assert.equal(stopped, true, "The bundled backend should stop when MCP closes."); console.log("MCP shutdown stopped the bundled backend. No simulator was booted or changed."); + // T3 Code users set MOBILE_DEV_PANEL=browser, so the panel opens on a loopback page beside the chat. + browserTransport = new StdioClientTransport({ + command: serverConfig.command, args: serverConfig.args, cwd: serverCwd, stderr: "pipe", env: { ...serverEnv, MOBILE_DEV_PANEL: "browser" }, + }); + const browser = new Client({ name: "mobile-dev-package-browser", version: "1" }); + await browser.connect(browserTransport); + const browserTools = await browser.listTools(); + const browserEntrypoint = browserTools.tools.find(tool => tool.name === "mobile_open_simulator"); + assert.equal(browserEntrypoint._meta.ui?.resourceUri, undefined, "A browser panel must not render in the chat."); + assert.equal(browserEntrypoint._meta["ui/resourceUri"], undefined, "A browser panel must not render in the chat."); + assert.equal(browserEntrypoint._meta["mobile-dev/panel"].resourceUri, `ui://mobile-dev/${manifest.version}/simulator.html`); + assert.ok(browserTools.tools.some(tool => tool.name === "mobile_read_panel_context")); + await access(join(plugin, "dist/browser-panel.html")); + const browserOpening = await browser.callTool({ name: "mobile_open_simulator", arguments: {} }, undefined, { timeout: 30000 }); + assert.ok(browserOpening.structuredContent?.browserPanel, JSON.stringify(browserOpening.content)); + const browserPanelUrl = new URL(browserOpening.structuredContent.browserPanel.url); + assert.equal(browserPanelUrl.hostname, "127.0.0.1"); + const browserPage = await fetch(browserPanelUrl); + assert.equal(browserPage.status, 200); + await browser.close(); + console.log("With MOBILE_DEV_PANEL=browser, the packaged server names its panel for the browser and serves the page on loopback."); } finally { + await browserTransport?.close(); await runtimeTransport?.close(); await transport?.close(); await rm(temporary, { recursive: true, force: true }); diff --git a/docs/architecture.md b/docs/architecture.md index 4a312879..1624f2c4 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -25,7 +25,11 @@ A host adds to the shared pieces without changing them: resource. Claude's entry (`apps/claude/server`) passes the panel with `inline: false`, so even MCP Apps hosts such as the desktop app's Code tab render none in the chat, and the browser panel feature with `always: true`, which shows it beside the chat in a - browser; it adds agent-device's commands. Every host lists simulators for logs, CPU and device choices with + browser; it adds agent-device's commands. `MOBILE_DEV_PANEL=browser` + (`packages/core/src/panel-placement.ts`) gives the Codex and shared entries Claude's setup, + and recording cards then offer no workspace. MCP Apps hosts advertise the same capabilities + whether or not an inline panel suits them. T3 Code, for example, asks the user to confirm every + panel tool call that is not read-only, so its Codex threads need `browser`. Every host lists simulators for logs, CPU and device choices with simctl, without starting Baguette. - **Runtime:** a host's `plugin-host.json` can leave out the panel's parts of `dist/` (`"panel": false`, see `PANEL_PARTS` in `tooling/host-runtime.mjs`), download them diff --git a/docs/telemetry.md b/docs/telemetry.md index 904f299f..8027a984 100644 --- a/docs/telemetry.md +++ b/docs/telemetry.md @@ -17,7 +17,8 @@ Behavior never depends on this value; it only attributes measurements and errors Cursor sessions were reported as `other`. Calls before connecting carry no host. - The server labels itself from the connecting MCP client's name on initialize and reports `other` until then. -- In hosts that render no MCP Apps panel, such as Claude Code, the panel can open in +- In hosts that render no MCP Apps panel, such as Claude Code, and with + `MOBILE_DEV_PANEL=browser`, as in T3 Code's Codex threads, the panel can open in a browser beside the chat. Its page names itself with the server's label for the chat host, so the panel is attributed to that host. Its requests join the host's server session with the same trace propagation; the browser page adds no diff --git a/packages/core/src/browser-panel.ts b/packages/core/src/browser-panel.ts index 632816ad..a1253b49 100644 --- a/packages/core/src/browser-panel.ts +++ b/packages/core/src/browser-panel.ts @@ -52,7 +52,7 @@ export function browserPanel({ template = PAGE, always = false }: BrowserPanelOp page ??= startPanelPage(connections, host, template); const opened = await page; const url = opened.url(tool, target); - const instructions = `Mobile Dev's panel is a page to show beside the chat, not in it, and it is not open yet. Show it to the user now: open ${url} in your in-app browser or side panel, in the foreground, so the page is visible without another click. Opening a page can leave the browser pane hidden: when your host has a tool that brings its browser or preview pane into view, such as a show_pane tool for its preview pane, load it if needed and call it right after. Reuse a tab that already shows Mobile Dev. Open no other address for it, such as the backend's baseUrl. If you cannot open pages, give the user the link. The page shares the user's devices and what they add to chat through mobile_read_panel_context; read it before device work.`; + const instructions = `Mobile Dev's panel is a page to show beside the chat, not in it, and it is not open yet. Show it to the user now: open ${url} in your in-app browser or side panel, in the foreground, so the page is visible without another click. This page is how Mobile Dev shows its panel, so opening it is part of opening the panel. Use your host's own browser or preview tool, and load it first if it is deferred. Never open the page with a computer-use tool that drives the screen. Opening a page can leave the browser pane hidden: when your host has a tool that brings its browser or preview pane into view, such as a show_pane tool for its preview pane, load it if needed and call it right after. A host can also show the page as a floating preview and report it as not visible. That still counts as shown, so do not open it again. Leave the tab open after you reply. Reuse a tab that already shows Mobile Dev. Open no other address for it, such as the backend's baseUrl. If you cannot open pages, give the user the link. The page shares the user's devices and what they add to chat through mobile_read_panel_context; read it before device work.`; return { url, instructions }; }); // A browser has no model context of its own, so the agent asks the server for it. diff --git a/packages/core/src/device-apps/foreground.ts b/packages/core/src/device-apps/foreground.ts index 0b5671ce..6deff28a 100644 --- a/packages/core/src/device-apps/foreground.ts +++ b/packages/core/src/device-apps/foreground.ts @@ -3,6 +3,7 @@ import { fileURLToPath } from "node:url"; import { z } from "zod"; import { adbPath } from "../native-logs.ts"; import { baguetteEnvironment, BAGUETTE_RUNTIME_TIMEOUT_MS } from "../baguette-runtime.ts"; +import { panelRuntimePath } from "../panel-runtime.ts"; import type { ForegroundApp } from "@mobile-dev/shared/device-apps.ts"; import { ExpectedOperationError } from "@mobile-dev/shared/error-reporting.ts"; @@ -29,9 +30,8 @@ export type SimulatorForeground = (deviceId: string, signal?: AbortSignal) => Pr /** Servers without the panel ship no Baguette, so a simulator's foreground app is unknown. */ export const unknownSimulatorForeground: SimulatorForeground = async () => null; -export async function foregroundSimulatorPid(deviceId: string, signal?: AbortSignal, - helper = new URL("./baguette/Baguette", import.meta.url), runtime = baguetteEnvironment): Promise { - const path = fileURLToPath(helper); +export async function foregroundSimulatorPid(deviceId: string, signal?: AbortSignal, runtime = baguetteEnvironment): Promise { + const path = await panelRuntimePath("baguette/Baguette"); const cancellation = signal ?? AbortSignal.timeout(10000); const startedAt = performance.now(); let environment: NodeJS.ProcessEnv; diff --git a/packages/core/src/index.ts b/packages/core/src/index.ts index b49815c4..cce9f8e5 100644 --- a/packages/core/src/index.ts +++ b/packages/core/src/index.ts @@ -3,7 +3,9 @@ import { runStdioServer } from "./stdio-server.ts"; import { createPanel } from "./panel-tools.ts"; import { agentDeviceTools, readAgentDeviceCatalog } from "./agent-device-tools.ts"; import { browserPanel } from "./browser-panel.ts"; +import { panelPlacement } from "./panel-placement.ts"; // The shared server with Mobile Dev's panel and agent-device's commands, as Cursor and other // MCP hosts run it. Hosts that render no MCP Apps panel can show it in a browser beside the chat. -await runStdioServer({ panel: createPanel(), features: [agentDeviceTools(readAgentDeviceCatalog()), browserPanel()] }); +const inline = panelPlacement("inline") === "inline"; +await runStdioServer({ panel: createPanel({ inline }), features: [agentDeviceTools(readAgentDeviceCatalog()), browserPanel({ always: !inline })] }); diff --git a/packages/core/src/panel-placement.ts b/packages/core/src/panel-placement.ts new file mode 100644 index 00000000..4acc7ad0 --- /dev/null +++ b/packages/core/src/panel-placement.ts @@ -0,0 +1,13 @@ +export type PanelPlacement = "inline" | "browser"; + +/** + * MCP Apps hosts advertise the same capabilities whether or not an inline panel suits them, + * such as T3 Code, which asks the user to confirm each app tool call that is not read-only, + * so only the user can choose the placement. + */ +export function panelPlacement(fallback: PanelPlacement, env: NodeJS.ProcessEnv = process.env): PanelPlacement { + const requested = env.MOBILE_DEV_PANEL?.trim().toLowerCase(); + if (requested === "inline" || requested === "browser") return requested; + if (requested) console.error(`[mobile-dev] Ignoring MOBILE_DEV_PANEL=${JSON.stringify(env.MOBILE_DEV_PANEL)}; it accepts "inline" or "browser". Using "${fallback}".`); + return fallback; +} diff --git a/packages/core/src/panel-tools.ts b/packages/core/src/panel-tools.ts index 86fa25a5..8c86d9fd 100644 --- a/packages/core/src/panel-tools.ts +++ b/packages/core/src/panel-tools.ts @@ -72,6 +72,8 @@ const INSTRUCTIONS = "For mobile app development, open mobile_open_simulator bef /** Mobile Dev's own device views, which a host adds to the server it runs (see createPanel). */ export type Panel = { + /** Whether the panel renders in the chat, where it can also open recordings in the workspace. */ + readonly inline: boolean; instructions: string; simulatorForeground: SimulatorForeground; register(server: McpServer, shared: PanelShared): () => Promise; @@ -112,6 +114,7 @@ export type PanelOptions = { */ export function createPanel({ baguette = new Baguette(), simulatorInput, copyScreenshot = copyPNGToClipboard, inline = true }: PanelOptions = {}): Panel { return { + inline, instructions: INSTRUCTIONS, simulatorForeground: foregroundSimulatorPid, register: (server, shared) => registerPanel(server, shared, { diff --git a/packages/core/src/plugin.ts b/packages/core/src/plugin.ts index 3208b720..b6e88280 100644 --- a/packages/core/src/plugin.ts +++ b/packages/core/src/plugin.ts @@ -123,8 +123,8 @@ export async function createPlugin(html: string | (() => Promise), o }) }] }; }); const readCard = (view: string) => async (uri: URL) => { - // Cards from a server with the workspace can open recordings there. - const content = (await page()).replace('data-view="panel"', `data-view="${view}"${panel ? " data-workspace" : ""}`); + // Cards from a server whose panel opens recordings in the workspace can open them there. + const content = (await page()).replace('data-view="panel"', `data-view="${view}"${panel?.inline ? " data-workspace" : ""}`); return { contents: [{ uri: uri.href, mimeType: RESOURCE_MIME_TYPE, text: content, _meta: { ui: { prefersBorder: true, permissions: { clipboardWrite: {} }, csp: { connectDomains: [SENTRY_ORIGIN], resourceDomains: [] } }, "openai/ui": { preferredDisplayMode: "inline", availableDisplayModes: ["inline", "fullscreen"] }, diff --git a/packages/core/test/cpu-foreground.test.ts b/packages/core/test/cpu-foreground.test.ts index 221ec79c..c0dd08a3 100644 --- a/packages/core/test/cpu-foreground.test.ts +++ b/packages/core/test/cpu-foreground.test.ts @@ -4,19 +4,21 @@ import { mkdir, mkdtemp, rm, writeFile } from "node:fs/promises"; import { tmpdir } from "node:os"; import { join } from "node:path"; import { pathToFileURL } from "node:url"; +import { usePanelRuntime } from "../src/panel-runtime.ts"; import { foregroundPhysicalPid, foregroundSimulatorPid, foregroundAndroidApp, parseAndroidForegroundPackage, unknownSimulatorForeground } from "../src/device-apps/foreground.ts"; import { deviceAppSources, readDeviceApps } from "../src/device-apps/sources.ts"; import type { DeviceApp } from "../../shared/src/device-apps.ts"; const udid = "00008150-001068280AE8C01C"; -test("simulator foreground detection passes the selected Xcode library environment to Baguette", async t => { +test("simulator foreground detection runs the panel runtime's Baguette with the selected Xcode library environment", async t => { const temporary = tmpdir(); const prefix = join(temporary, "mobile-dev-simulator-foreground-"); const root = await mkdtemp(prefix); t.after(() => rm(root, { recursive: true, force: true })); - const path = join(root, "helper.cjs"); - const url = pathToFileURL(path); + await mkdir(join(root, "baguette")); + const path = join(root, "baguette", "Baguette"); + usePanelRuntime(async () => root); const script = `#!${process.execPath} if (process.argv[2] !== 'foreground' || process.argv[3] !== '--udid') process.exit(2); if (process.env.DYLD_LIBRARY_PATH !== '/selected/libraries') process.exit(3); @@ -24,7 +26,7 @@ process.stdout.write('{"pid":123}'); `; await writeFile(path, script, { mode: 0o700 }); const controller = new AbortController(); - const pid = await foregroundSimulatorPid(udid, controller.signal, url, async (executable, signal) => { + const pid = await foregroundSimulatorPid(udid, controller.signal, async (executable, signal) => { assert.equal(executable, path); assert.equal(signal, controller.signal); return { DYLD_LIBRARY_PATH: "/selected/libraries" }; diff --git a/packages/core/test/panel-placement.test.ts b/packages/core/test/panel-placement.test.ts new file mode 100644 index 00000000..65b4ec88 --- /dev/null +++ b/packages/core/test/panel-placement.test.ts @@ -0,0 +1,26 @@ +import test from "node:test"; +import assert from "node:assert/strict"; +import { panelPlacement } from "../src/panel-placement.ts"; + +test("MOBILE_DEV_PANEL overrides a server's panel placement only with a known placement", t => { + const error = t.mock.method(console, "error", () => {}); + const warnings = () => { + const lines = error.mock.calls.map(call => String(call.arguments[0])); + error.mock.resetCalls(); + return lines; + }; + for (const fallback of ["inline", "browser"] as const) { + assert.equal(panelPlacement(fallback, {}), fallback); + assert.equal(panelPlacement(fallback, { MOBILE_DEV_PANEL: "" }), fallback); + assert.equal(panelPlacement(fallback, { MOBILE_DEV_PANEL: "browser" }), "browser"); + assert.equal(panelPlacement(fallback, { MOBILE_DEV_PANEL: "inline" }), "inline"); + assert.equal(panelPlacement(fallback, { MOBILE_DEV_PANEL: " Browser\n" }), "browser"); + assert.equal(panelPlacement(fallback, { MOBILE_DEV_PANEL: "INLINE" }), "inline"); + assert.deepEqual(warnings(), []); + assert.equal(panelPlacement(fallback, { MOBILE_DEV_PANEL: "sidebar" }), fallback); + const [warning, ...more] = warnings(); + assert.deepEqual(more, []); + assert.match(warning, /MOBILE_DEV_PANEL="sidebar"/); + assert.match(warning, /"inline" or "browser"/); + } +}); diff --git a/packages/core/test/plugin.test.ts b/packages/core/test/plugin.test.ts index 529d40b6..58ef5696 100644 --- a/packages/core/test/plugin.test.ts +++ b/packages/core/test/plugin.test.ts @@ -245,6 +245,17 @@ test("MCP tools expose native entrypoints and complete the simulator workflow", const recordingResource = await client.readResource({ uri: RECORDING_URI }); // Cards from a server with the workspace can open recordings there. assert.match(recordingResource.contents[0].text as string, /data-view="recording" data-workspace/); + // A panel beside the chat opens no recordings, so its cards offer no workspace. + const besideChat = await createTestPlugin('', { panel: createPanel({ inline: false }) }); + const [besideChatClientTransport, besideChatServerTransport] = InMemoryTransport.createLinkedPair(); + const besideChatClient = new Client({ name: "browser-panel-host", version: "1" }); + t.after(async () => { await besideChatClient.close(); await besideChat.close(); }); + await besideChat.server.connect(besideChatServerTransport); + await besideChatClient.connect(besideChatClientTransport); + for (const [uri, view] of [[RECORDING_URI, "recording"], [COMPARISON_URI, "comparison"]]) { + const card = String((await besideChatClient.readResource({ uri })).contents[0].text); + assert.equal(card, ``); + } assert.equal(recordingResource.contents[0].mimeType, "text/html;profile=mcp-app"); assert.deepEqual(recordingResource.contents[0]._meta?.["openai/ui"], { preferredDisplayMode: "inline", availableDisplayModes: ["inline", "fullscreen"] }); assert.deepEqual(recordingResource.contents[0]._meta?.ui, { prefersBorder: true, permissions: { clipboardWrite: {} }, csp: { connectDomains: [SENTRY_ORIGIN], resourceDomains: [] } }); diff --git a/packages/skills/mobile-dev/SKILL.md b/packages/skills/mobile-dev/SKILL.md index 0524c245..2ee58a6c 100644 --- a/packages/skills/mobile-dev/SKILL.md +++ b/packages/skills/mobile-dev/SKILL.md @@ -20,7 +20,7 @@ Do not assume the chat predates installation or continue device work through a substitute backend. Source edits or compilation can continue when they need no device tools. -1. Open the panel beside the chat with `mobile_open_simulator` by default. For an existing app, open it when starting device work; for a new app, open it before the first device launch. Reuse an open Mobile Dev panel. Use `mobile_open_workspace` when the user asks for fullscreen. Use device-list tools without opening the panel when the user requests a tool-only workflow or the host cannot show panels. A request limited to planning, docs, code review, or compilation does not need a panel or a device launch. +1. Open the panel beside the chat with `mobile_open_simulator` by default. For an existing app, open it when starting device work; for a new app, open it before the first device launch. Reuse an open Mobile Dev panel. Use `mobile_open_workspace` when the user asks for fullscreen. When the result carries `browserPanel.url`, the panel is a page beside the chat, and opening that page is part of opening the panel. Open it right away with the host's own browser or preview tool, such as `preview_open` (load it through tool discovery first if it is deferred), never with a computer-use tool, and leave the tab open. A host can show the page as a floating preview and report it as not visible; it still counts as shown, so do not open it again. Reuse a tab that already shows Mobile Dev. Drive the panel's devices with this plugin's own `device_*` tools, not another server's tools of the same name. Use device-list tools without opening the panel when the user requests a tool-only workflow or the host cannot show panels. A request limited to planning, docs, code review, or compilation does not need a panel or a device launch. Both opening tools accept optional `platform` (`ios` or `android`), `kind` (`simulator`, `emulator`, or `physical`), and `deviceId`. Pass only intent established by the user's request or app project: for example, `{ platform: "android", kind: "emulator" }` for Android emulator work, or `{ platform: "ios", kind: "physical" }` for an explicitly requested physical iPhone. Omit undecided fields for open-ended work. Pass `deviceId` only after discovering the exact device; discovery can precede opening when needed to identify a named target. A missing matching device stays unselected instead of connecting to another target. Opening selects a stopped simulator or AVD without booting it. Follow later manual selections. 2. Use the task and app project to choose the platform. Follow the user's device choice; otherwise reuse a compatible device shared by the panel, then a suitable running simulator or emulator. If no suitable device runs, choose a compatible installed simulator or AVD. Pick among equivalent devices yourself for routine app development. When the intended target is ambiguous (for example, the app is open on several devices for a performance run), ask the user with `mobile_choose_devices`, or in chat when the host has no form. Read device IDs from tool results or panel context; never invent them. When the panel opened in a browser, read its context with `mobile_read_panel_context`. 3. Boot a chosen stopped iOS simulator with `mobile_boot_simulator`, or an Android AVD with `mobile_boot_android_emulator`. A mobile app development request permits choosing and booting a suitable installed simulator without a separate device-choice question. Reuse running devices without rebooting them. Follow the Android section below for device discovery and serials. Opening the panel alone does not boot a device. diff --git a/website/src/content/docs/docs/getting-started.md b/website/src/content/docs/docs/getting-started.md index 67091cf1..1f335cc7 100644 --- a/website/src/content/docs/docs/getting-started.md +++ b/website/src/content/docs/docs/getting-started.md @@ -1,6 +1,6 @@ --- title: Getting started -description: Install Mobile Dev in ChatGPT, Codex, Claude, Cursor, or another compatible MCP environment, then connect an iOS or Android device. +description: Install Mobile Dev in ChatGPT, Codex, Claude, Cursor, T3 Code, or another compatible MCP environment, then connect an iOS or Android device. --- Install Mobile Dev for your development environment, open a new conversation, and ask **“Open Mobile Dev.”** Choose a device in the panel, then continue with your app project. @@ -90,6 +90,20 @@ Cursor must allow local plugin imports. On managed Enterprise accounts, your adm For a local MCP connection, run `npm run build` and `npm run cursor:config` from the checkout. Merge the generated `.local-dev/cursor-mcp.json` entry into the `mcpServers` object in your global `~/.cursor/mcp.json`, then reload Cursor. Preserve your other servers and use either the plugin or this local connection. +## T3 Code + +[T3 Code](https://github.com/pingdotgg/t3code) runs Claude Code and Codex, so it loads the Mobile Dev plugin that each of them has installed. On the Mac that runs T3 Code, install the [Claude Code](#claude-code) plugin for Claude threads and the [Codex](#codex-cli) plugin for Codex threads. In both, the panel opens in T3 Code's Browser. + +Turn off T3 Code's own device integration while you use Mobile Dev. In **Settings → Devices**, turn off **Use devices**, which also turns off **Agents can use devices**. Otherwise T3 Code gives agents its own `device_*` tools, with the same names as Mobile Dev's, and tells them to prefer those tools. + +Codex threads need one setting, because T3 Code renders Codex's panel in the chat and asks you to confirm each action that the panel takes, such as each tap: + +1. Open **Settings → Providers**, then the Codex instance. +2. Under **Variables**, click **Add variable**. Set `MOBILE_DEV_PANEL` to `browser`. +3. Start a new thread. + +Ask **“Open Mobile Dev.”** The agent opens the panel with T3 Code's `preview_open` tool, and T3 Code shows the page as a floating preview. To dock the panel beside the chat, click the preview's **Open in right panel** button. Performance recordings still render as cards in the chat. + ## Other environments The shared server supports local MCP connections over stdio. After completing the [source-build setup](https://github.com/callstackincubator/mobile-dev/blob/main/CONTRIBUTING.md#build-and-package), configure your client's local MCP server connection to launch Node.js 22.19 or later with the absolute path to the built `dist/server.mjs` as its argument. Keep the built distribution and its bundled helpers together. diff --git a/website/src/site/content.ts b/website/src/site/content.ts index 141c790d..970152ab 100644 --- a/website/src/site/content.ts +++ b/website/src/site/content.ts @@ -47,6 +47,7 @@ export const installOptions = [ { name: 'ChatGPT / Codex', detail: 'Open the ChatGPT plugin store.', href: pluginStoreUrl, external: true }, { name: 'Claude', detail: 'Manual setup for Desktop and Claude Code.', href: `${docs.start}#claude`, external: false }, { name: 'Cursor', detail: 'Manual setup for the local Cursor plugin.', href: `${docs.start}#cursor`, external: false }, + { name: 'T3 Code', detail: 'Manual setup for Claude and Codex threads.', href: `${docs.start}#t3-code`, external: false }, ]; type Link = { text: string; href: string };