feat(coda): three-mode framework + coda_interactive, agent/setup fixes, and skill-bundle refresh (Databricks + Flutter/Dart/shadcn)

[datasciencemonkey]

Summary

Two changes that together establish the three-mode framework for the CoDA MCP server:

1. Narrow coda_run to replay-only URLs — its returned viewer_url now always serves a static transcript from disk, never a live PTY attach. Drops the 5-minute grace machinery introduced in feat: CoDA MCP live session URL — watch hermes execute live + replay #66 (which was actually never wired in production — see commit 193c9a3 for the dead-code analysis).
2. Add new coda_interactive MCP tool — for human handoff from upstream MCP clients (Genie Code, Claude Desktop, Cursor). Caller passes a Databricks Workspace Git Folder path, optional branch, and a kickoff prompt. Coda exports the file tree, launches the chosen agent (claude default; also hermes/codex/gemini/opencode), auto-types the prompt, and returns a viewer_url for the human to attach to.

The three-mode framework now:

Mode	Tool	URL semantics	PTY lifecycle
1. Direct launch	Web UI tab	n/a (no external URL)	24h idle / WS-heartbeat extends
2. coda_interactive (new)	MCP tool	Live attach	24h idle / WS extends
3. coda_run (narrowed)	MCP tool	Replay only	Immediate teardown on hermes exit

Why Workspace Git Folders (not GitHub clone)

coda_interactive materializes project files via the Databricks Workspace API — uses Coda's existing DATABRICKS_TOKEN, no new credentials needed. Trade-off: git history is unavailable inside the session (files-only export). If history matters for a session, the MCP caller can include a git log summary in the prompt string.

Pre-existing security fix bundled

mcp_create_pty_session was stripping only 5 env vars; the HTTP create_session path was stripping NPM_TOKEN, UV_*, and npm_config_//* patterns too. Refactored to share _build_terminal_shell_env (commit ef15ef7). Closes a latent registry-credential leak into MCP-created PTYs.

Subsumes PR #66

PR #66 introduced live-attach + 5-min grace on coda_run. This PR keeps the viewer URL feature but narrows it to replay-only (since live attach moves to coda_interactive). Close #66 if this lands.

Spec + Plan artifacts (in the diff)

• Todo 1 (Mode 3 narrowing): docs/superpowers/specs/2026-05-28-coda-run-replay-only-design.md + plans/2026-05-28-coda-run-replay-only.md
• Todo 2 (Mode 2 addition): docs/superpowers/specs/2026-05-28-coda-interactive-mcp-tool-design.md + plans/2026-05-28-coda-interactive-mcp-tool.md
• Both passed independent critic-agent reviews at spec, plan, and per-task stages.

Test plan

• Local: 551 passed, 21 skipped — confirmed (pytest tests/ --ignore=tests/e2e)
• PTY-gated tests skip cleanly on Mac dev environment; should pass on Linux CI/deployed app
• Manual smoke against deployed CoDA: invoke coda_run from Genie Code; confirm viewer URL is replay-only
• Manual smoke against deployed CoDA: invoke coda_interactive with a Workspace Git Folder; confirm agent launches in exported project dir with prompt typed
• Confirm coda_inbox does NOT show interactive sessions; coda_get_result returns nothing for them
• Verify _build_terminal_shell_env strip on deployed PTY — env | grep -E 'NPM_TOKEN|UV_' should be empty

---

Follow-up: broadened source contract (commits 326e19a..a555602)

coda_interactive no longer requires the workspace_path to be a Databricks Workspace Git Folder. Any Workspace directory (Git Folder or plain Workspace folder) is accepted. The branch parameter has been removed — callers manage Git Folder branch state themselves before calling.

API change (no shipped consumers — safe):

• coda_interactive(prompt, workspace_path, branch=..., agent=..., email=...) → coda_interactive(prompt, workspace_path, agent=..., email=...)
• Return shape: "branch" key dropped.

Validation: replaced repos.list + exact-match filter + optional repos.update with a single workspace.get_status call + directory-type check (_is_directory from workspace_export.py). Clean errors for "path not found" and "path is not a directory" — both return before any PTY allocation.

Server-level instructions string rewritten to:

• Tell callers that plain Workspace folders work.
• Surface the upload-then-handoff pattern explicitly (workspace.import first if files aren't in the Workspace yet) so an upstream LLM knows the tool doesn't accept inline file payloads.

Quality fix bundled: hoisted the _app_send_input is None guard above PTY creation so an unwired send-hook can no longer orphan a PTY + project dir.

Test delta: −3 / +4. All 13 tests in tests/test_coda_interactive.py pass.

Artifacts:

• Spec: docs/superpowers/specs/2026-05-28-coda-interactive-broaden-source-design.md
• Plan: docs/superpowers/plans/2026-05-28-coda-interactive-broaden-source.md
• Original spec marked as Amended by: for traceability.

---

Follow-up #2: Workflow protocol + Databricks orientation (commits 6ff6a9b..77321dc)

coda_run now injects two new sections into prompt.txt:

• CAPABILITIES — tells hermes about the Databricks CLI (pre-authed), the 16 Databricks skills under ~/.claude/skills/, and the DeepWiki / Exa / CoDA MCP servers.
• WORKFLOW PROTOCOL — imposes a 3-phase pipeline (PLAN → EXECUTE → SYNTHESIZE) with a critique step after each phase (self-review or sub-agent — agent's choice). Max 2 iterations per phase to keep token cost bounded.

New terminal result.json status "info_needed" with a required feedback field gives the calling client a structured iteration loop when the agent is blocked. The existing "needs_approval" status is preserved with explicit disambiguation in the protocol: info_needed = "caller must add context"; needs_approval = "caller must approve a destructive action".

Three upstream-facing surfaces updated so calling LLMs know about the new statuses:

• coda_inbox counts dict gains info_needed and needs_approval keys.
• coda_get_result docstring lists all four valid statuses + the new feedback field.
• FastMCP server-level instructions gain an INFO_NEEDED HANDOFF paragraph teaching upstream LLMs to read feedback and resubmit with previous_session_id.

Flag: coda_run(..., workflow_protocol=True) is the default. Set False to skip both new sections for non-Databricks tasks.

Artifacts:

• Spec: docs/superpowers/specs/2026-05-28-coda-run-workflow-protocol-design.md
• Plan: docs/superpowers/plans/2026-05-28-coda-run-workflow-protocol.md

Discipline gates run:

• Spec critic → APPROVE-WITH-FIXES → 5 fixes applied (counts dict, needs_approval disambig, MCP instructions, canonical skill test, token estimate)
• Plan critic → APPROVE-WITH-FIXES → 2 fixes applied (correct _write_json stub, expanded regression coverage)
• Per task (1-4): spec + code-quality reviews, all approvals; minor fixes applied (section_noise dead code, JSON union-syntax placeholder)

---

Follow-up #3: coda_interactive pulls files in the terminal (fixes empty-session bug)

Bug: Calling coda_interactive launched the agent over an empty directory — the agent had no idea about the user's Workspace files.

Root cause: The MCP server's WorkspaceClient() resolves to the app's service principal (app-167dcd …), which can get_status the user's /Users/<user>/… folder (so the tool reported "launched") but cannot list/export its contents. workspace_export.py swallowed those errors → empty dir + misleading success. Verified: the CoDA terminal's CLI runs as the user (databricks current-user me → the user), and databricks workspace list/export of the folder works as the user via REST.

Fix: Stop exporting server-side. coda_interactive now types cd <project_dir> && databricks workspace export-dir <source> ./<name> && cd <name> into the PTY (authenticated as the user), waits for the pull to settle, then does a server-side filesystem post-check (identity-independent — it stats the local disk the terminal wrote). If files landed it launches the agent and seeds the prompt with a context line naming the source; if nothing landed it returns a real status=error and never launches.

Design highlights:

• Split waits — wait for the pull to go idle (reliable stabilization), THEN the existing agent-ready wait after launch. Avoids pasting the prompt into a half-initialized agent.
• databricks workspace export-dir natively handles notebook extensions, replacing the hand-rolled workspace_export.py (deleted, with its tests).
• _wait_for_agent_ready is now a thin wrapper over a generalized _wait_for_output_stable(pty, max_wait, stability); coda_run is unchanged.
• New helpers _safe_dirname (sanitized basename, rejects ./..) and _normalize_workspace_path (drops the /Workspace FUSE prefix).
• MCP instructions wording updated; no more "server-side snapshot".

Tests: test_coda_interactive.py rewritten to the pull contract (pull-first, FS-check failure → no launch, prompt context line, agent matrix); _safe_dirname/_normalize_workspace_path + the wait-wrapper covered in test_mcp_server.py. Suite: 117 passed (only the documented PTY-fd flake fails in multi-file runs; passes in isolation).

Artifacts:

• Spec: docs/superpowers/specs/2026-05-28-coda-interactive-terminal-pull-design.md
• Plan: docs/superpowers/plans/2026-05-28-coda-interactive-terminal-pull.md

Discipline gates: brainstorm → design critic (SOUND-WITH-FIXES, folded in) → spec → plan → plan critic (SOUND-WITH-FIXES, folded in) → TDD implementation → final critic (SHIP-WITH-FOLLOWUPS; the one Important finding — .. path-traversal in _safe_dirname — fixed before push).

This branch also merges the latest main (Dependabot deps bump #68: urllib3 2.7.0 / gitpython 3.1.50 / idna 3.16).

---

Follow-up #4: OpenCode + Codex + Claude-subagent fixes (the fec2152 file-move regression class)

The CoDA-MCP commit (fec2152) relocated every setup script into setup/ (and the install scripts into scripts/), but several scripts kept resolving sibling resources via Path(__file__).parent — which then pointed at setup/ instead of the repo root. Three resources were silently broken; all fixed and pinned with regression tests.

• OpenCode (444e133): setup_proxy.py launched a nonexistent setup/content_filter_proxy.py, so the content-filter proxy never started and OpenCode — the only agent routed through 127.0.0.1:4000 — failed with Cannot connect to API. Other agents talk to the gateway directly and were unaffected. Now resolves from the repo root via resolve_proxy_script_path() (script body guarded under main() so the path logic is importable/testable). Also registered databricks-claude-opus-4-7 in the OpenCode model map and dropped a duplicate gemini-2-5-pro key.
• Claude subagents (7bdc3f3): setup_claude.py looked in setup/agents/, silently hitting the "No agents directory found" branch — the TDD subagents (build-feature, prd-writer, test-generator, implementer) were never installed into ~/.claude/agents. Fixed via resolve_agents_src() → repo root.
• Codex model catalog (7bdc3f3): setup_codex.py looked in setup/.codex/databricks-models.json; the catalog was never copied into ~/.codex while config.toml still referenced it. Fixed via resolve_codex_catalog_src() → repo root.

Regression tests pin each resolved resource path to a real file so a future move can't silently regress it again: tests/test_setup_proxy.py, tests/test_setup_resource_paths.py.

Follow-up #5: flaky PTY test-suite hardening (32624c7)

The full unit suite failed intermittently (a different set each run); two independent causes:

• terminate_session double-closed master_fd (production bug): both the explicit close path (mcp_close_pty_session) and the read-thread exit path (read_pty_output) call it for the same session, but the kill/os.close block ran unconditionally — so the second os.close() could land on a since-reused fd (e.g. an asyncio event loop's self-pipe from a later test), surfacing as intermittent OSError: [Errno 9] Bad file descriptor. Now atomically claims the session (sessions.pop) and closes exactly once. Covered by tests/test_terminate_session_idempotent.py.
• App-hook leak across test files: tests/test_mcp_server.py::_reset_hooks set mcp_server's process-wide PTY hooks to None in teardown, leaking into later files so coda_run created no PTY (pty_id is None) — but only in full-suite runs. Fixed with a tests/conftest.py autouse fixture that re-establishes app's real hooks after each test, making hook state order-independent.
• TestNpmVersionLive now skips when the npm registry is unreachable (the skipif probe was raising TimeoutExpired as a collection error; the body asserted on a None result) instead of erroring/failing offline.

Full unit suite now green across 3 consecutive runs: 613 passed, 2 skipped.

Follow-up #6: skill-bundle refresh + Flutter / Dart / shadcn (f2ca8c7, 37df2f8)

• Refreshed Databricks skills from databricks-solutions/ai-dev-kit (local was ~3 months stale: 2026-02-22 → upstream 76c774f, 2026-05-28). 24 → 26 skills: new databricks-ai-functions, databricks-execution-compute, databricks-iceberg; renamed app-python→apps-python, asset-bundles→bundles, synthetic-data-generation→synthetic-data-gen; dropped databricks-app-apx (no longer shipped upstream); content refreshed across the rest. The curated 16-skill list (coda_mcp/databricks_preamble.py) and the CLAUDE.md Databricks Skills table are kept in lockstep by test_databricks_preamble.py.
• Bundled Flutter + Dart agent skills (per docs.flutter.dev/ai/agent-skills) so CoDA's agents can build Flutter/Dart apps: 10 flutter-* (flutter/skills), 9 dart-* (dart-lang/skills), and shadcn-ui-flutter (nank1ro/flutter-shadcn-ui). Documented in CLAUDE.md (skill count 30 → 50).

All of the above is verified live on the deployed test app mcp-test-coda-labs-feat (refreshed Databricks skills + the 20 new Flutter/Dart/shadcn skills confirmed in the deployed .claude/skills/).

[mpkrass7]

Regression defect on gemini-cli

[mpkrass7]

Opencode works as long as you don't change the model which defeats the purpose of opencode

[mpkrass7]

Likely not the PR but same problem in Hermes

[mpkrass7]

Again not the PR but we seriously need to figure out the cludgy terminal UI

[mpkrass7]

MCPs from Genie Code are working

[mpkrass7]

[datasciencemonkey]

Review feedback addressed (22e3c59..3124da7 — the branch also now carries latest main: dep bumps, README refresh, #77 requests-from-PyPI fix).

1. Gemini CLI regression → fixed (3124da7). Root cause: newer gemini-cli releases attach id to functionCall/functionResponse history parts, and the Databricks /gemini route rejects unknown proto fields — hence the 400 Unknown name "id" at contents[].parts[0].function_call on every tool-call turn (we install "latest stable ≥7 days" so the CLI drifted under us; main would hit this on a fresh deploy too). Gemini CLI now routes through the local content-filter proxy (the one OpenCode already uses): it strips those ids, forwards /gemini/* to the gateway's Gemini route, and detects Gemini streaming from the URL (:streamGenerateContent). Bonus: the proxy injects the freshest PAT, so gemini sessions survive token rotation now.

2. OpenCode model switch → reasoning_effort 400 → fixed (3124da7). OpenCode sends reasoning_effort after a model change; Databricks chat-completions rejects unknown params ("Extra inputs are not permitted"). The proxy now strips reasoning_effort/reasoning/verbosity from OpenAI-format request bodies.

3. Hermes → partially fixed (3124da7). The bug on our side: setup_hermes.py's known_models catalog listed databricks-gemini-2-5-pro twice and omitted the default model databricks-claude-opus-4-7 entirely — fixed. The rest is upstream: the AI Gateway doesn't expose a model-listing endpoint at {gw}/mlflow/v1/models, and "No authenticated providers found" is Hermes' scan for known-provider API keys (custom providers don't count). Not fixable in this repo.

4. Cludgy terminal UI — agreed; deferring to its own issue/PR. It's xterm.js rendering/reflow work, orthogonal to this PR.

5. MCPs from Genie Code working — 🎉

6. Spec/plan artifacts → removed (d18d6ac). All 13 files: docs/superpowers/{plans,specs}/* + docs/plans/2026-05-01-coda-mcp-server.md.

21 new regression tests (proxy sanitizers, /gemini routing, stream detection, setup wiring pins, hermes catalog hygiene). Full suite: 634 passed, 5 skipped. Gemini/OpenCode fixes need a redeploy of mcp-test-coda-labs-feat to verify live.

[datasciencemonkey]

Follow-up (95005e8, fd971de): Codex now survives PAT rotation mid-session. Codex reads OPENAI_API_KEY once at startup, so a running session 401'd after the 10-min rotation revoked its token. It now routes through the content-filter proxy via a transparent /openai/* prefix — byte-identical relay (no sanitizers, no JSON re-serialization, all HTTP methods, no read timeout) with only the Authorization header replaced by the freshest PAT per request. Zero Responses-API capability lost by construction; pinned by live-socket tests (byte-identical bodies incl. reasoning fields, fresh-token injection, verbatim SSE). Proxy also bumped to HTTP/1.1 (chunked framing legality for strict clients). Independent code review: SHIP-WITH-FIXES, both blockers (upstream connection leak on client disconnect; prefix-routing fall-through) fixed before push. Suite: 646 passed.

Proxy topology now: OpenCode + Gemini + Codex via 127.0.0.1:4000; Claude + Hermes direct.

[dgokeeffe]

Review — sole maintainer doing a backlog pass. Short version: the MCP feature is worth landing, but not in this shape. Three things need resolving first, one of them a security regression. Detail below so this is actionable rather than just blocked.

Note #64 and #66 are strict ancestors of this branch, so I've closed them in favour of this one — git merge-base --is-ancestor confirms it contains their commits.

---

1. /mcp is auth-exempt, and that undoes #44

app.py adds /mcp to the before_request exempt list:

if request.path in (...) or request.path.startswith("/socket.io") or request.path.startswith("/mcp"):

and coda_mcp/mcp_endpoint.py declines to do its own checking:

53:    Permissive CORS for /mcp — the Databricks Apps proxy handles auth.
90:    # Origin validation skipped — Databricks Apps proxy handles auth.

The Apps proxy does force SSO to reach the app, so this isn't open to the internet. But CoDA's model is single-owner, fail-closed: check_authorization() compares X-Forwarded-Email against app_owner and denies everyone else. /mcp opts out of that, so any workspace user who can reach the app can drive it — and coda_run in mcp_server.py spawns PTY sessions, i.e. arbitrary code execution as the app identity, with the owner's PAT in ~/.databrickscfg.

Combined with permissive CORS and origin validation explicitly skipped on a browser-reachable endpoint, a cross-origin request from any site the owner visits would carry their SSO cookies.

This is the same class of hole #44 closed (which removed /api/setup-status, /api/pat-status and /api/app-state from that exact list). Landing this as-is reopens it on a more dangerous endpoint.

Suggested fix: owner-gate /mcp through the existing check_authorization(), and either restore origin validation or narrow CORS to a known allowlist. If MCP clients can't carry SSO, use a shared-secret header gate like /api/inject-pat does with CODA_BOOTSTRAP_SECRET — that's the established pattern here for non-SSO callers.

2. The entrypoint change is a much bigger deal than the diff suggests

-  - gunicorn
-  - app:app
+  - uvicorn
+  - coda_mcp.mcp_asgi:app

This moves production from WSGI to ASGI. Two consequences worth surfacing explicitly:

• The comment concedes "WebSocket transport falls back to HTTP polling under uvicorn". That's a degradation of the core browser-terminal path, justified as acceptable because poll-worker.js exists. For a product whose whole value is a responsive terminal, that deserves to be an explicit decision, not a side effect of adding MCP.
• CoDA's threading model assumes a single gunicorn worker with process-local PTY state (gunicorn.conf.py, and docs/agent-instructions.md §1). Whether that invariant still holds under uvicorn is unverified.

This cannot be validated without deploying. There is currently no working CI on this repo (see #120 — repository-defined workflows never schedule a job), so an entrypoint swap is exactly the change that must be proven on a live app first.

3. ~200 of the 234 files are unrelated churn

The feature itself is small and reviewable:

Part	Size
coda_mcp/	7 files, ~1,875 lines
tests	9 MCP-related files
docs	3 files

The rest is a repo-wide relocation (setup_*.py → setup/, install scripts → scripts/) plus a 176-file .claude/ skills refresh. That churn is what makes this unreviewable, and it maximises conflict surface against everything else in flight — it's also why the diff reads as +34k/-8k.

Suggested split:

1. coda_mcp/ + its tests + docs + the minimal app.py wiring, with the auth fix from §1.
2. The setup/ + scripts/ relocation as its own mechanical PR, if it's wanted at all.
3. The .claude/ refresh via the existing refresh-databricks-skills skill, separately.

---

Happy to do the split myself if you'd rather — say the word. But §1 needs an answer from you either way, since it's a deliberate design choice in the current code rather than an oversight, and I'd rather not guess at the intent.