Skip to content

[wip] Add local canvases to the Agents Window - #335902

Draft
Ulugbek Abdullaev (ulugbekna) wants to merge 6 commits into
mainfrom
ulugbekna/agents/gh-copilot-canvases-support-vscode
Draft

[wip] Add local canvases to the Agents Window#335902
Ulugbek Abdullaev (ulugbekna) wants to merge 6 commits into
mainfrom
ulugbekna/agents/gh-copilot-canvases-support-vscode

Conversation

@ulugbekna

@ulugbekna Ulugbek Abdullaev (ulugbekna) commented Sep 11, 2026

Copy link
Copy Markdown
Contributor

Summary

Add a default-off local canvas development preview to the Agents Window, backed by approved Copilot SDK extension packages and source-resolved native browser surfaces.

This is a work-in-progress integration PR, not a request to enable the feature generally or publish the companion SDK/runtime changes.

Included

  • Canonical AHP canvas state and operations, generation/revision guards, bounded operation deduplication, and exact owning-chat routing.
  • Inert package preparation, immutable reviewed snapshots, explicit revision/workspace or shared-host approval, revocation, and document data kept outside code.
  • Canvas-first session retention, provider lifecycle/disposal fixes, explicit restart versus page reload, and stable restoration without persisting live endpoint authority.
  • Provider-neutral Sessions controls, authoring starter, visible/removable request-frozen canvas context, automatic active-chat reveal, and inert hidden/restored instances.
  • Native Integrated Browser app policy, semantic themes, permissions/navigation controls and ownership/lifecycle handling.
  • Genuine joined-SDK user-message projection so canvas-originated requests appear in chat with normal tool confirmation and cancellation. The host does not resend the prompt.
  • Focused unit/integration coverage and isolated developer launch/runbook support.

The five JavaScript allowlist additions are executable extension/browser fixtures and the no-build authoring starter; those files intentionally run as delivered rather than requiring TypeScript transpilation. The allowlist still requires normal CODEOWNERS review.

Validation

  • Latest affected Copilot live-session/history selection: 492 passing, with targeted typecheck and uncached lint clean.
  • Automatic presentation/facade selection: 99 passing, plus actual native model-driven reveal and hidden-tab refresh checks.
  • Earlier scoped native/lifecycle/load qualification is documented in the implementation runbooks. Counts from overlapping runs are not added together.
  • Real macOS arm64 native chess exercised actual model actions, explicit per-call approval, denial, cancellation, a permission wait beyond 120 seconds, reload, and provider restart while preserving data and the complete approved snapshot.
  • The standalone repaired chess ZIP is a separate local deliverable, not included in this source PR; its 125 packaged core/browser tests passed.
  • With the corrected companion runtime, the complete opt-in public SDK integration file is now 3 passing, including the formerly failing package-approval-after-an-existing-turn case.
  • Final commit preparation passed the normal VS Code pre-commit hygiene hook (181 checked files). Its formatting findings were fixed without changing behavior; protocol braces were corrected in the canonical AHP source and re-synced, not patched in generated copies.

Public companion drafts

The compatible runtime change is tracked separately in its existing private repository.

Remaining WIP gates

  • The separately reproduced Hook processor is not configured resume failure was fixed in the companion runtime's lifecycle handling, without a VS Code workaround. Repeated unchanged SDK reproductions and actual native existing-chat package approval now pass. Consuming released dependency versions is still separate from the isolated development-build validation.
  • The normal-workspace execution route is qualified only for the local macOS arm64 development setup with explicitly selected compatible SDK/runtime artifacts. Other native platforms, built-product enablement and normal dependency-release integration remain gated.
  • Real assistive-technology qualification remains outstanding.
  • Coordinated dependency releases remain separate from this draft. AHP's seven language/check jobs now pass without a provisional version bump.

Review and verification follow-up (2026-09-13)

  • All reported inline review threads are addressed: resource trust is requested unconditionally for the captured effective worktree, and package selection/approval use that same directory.
  • Native testing exposed and fixed first-worktree materialization revoking a newly bound launch lease, and immediate opening after approval failing to refresh an already initialized empty catalog.
  • Windows failures are repaired without weakening package quotas or execution authority: platform-correct file identity, verified immutable-snapshot reuse, and actual process termination rather than Unix signal-handler breadcrumbs. The deliberately large 2,048-file quota test has an explicit filesystem I/O budget.
  • Local validation: 20 tests across the three real canvas integration files, final 17-test lifecycle and 32-test package selections, client typecheck, scoped lint, and normal commit hygiene pass. These are overlapping selections, not an additive total.
  • Real macOS arm64 OSS: a first worktree turn succeeded; approval/open worked immediately afterward; two independent chess documents and the public greeting package worked in one retained conversation. A canvas-originated model request appeared in chat and waited for explicit tool approval. A rejected invalid move input caused no mutation; the legal retry moved exactly one piece.
  • Package refresh and explicit provider restart preserved documents and replaced origins. The old origin was explicitly CSP-blocked. Hide/reveal, attaching/removing/submitting a URL-free canvas reference, and closing just one logical instance passed. All 18 captured owned process-tree PIDs exited afterward.
  • All 32 executable checks pass at final head 5d9864b8adb, including compile, unit, browser, integration and smoke checks across Windows, macOS and Linux.
  • The screenshot failure was an actual fixture setup regression: NewChatView required the view-state service but the new-chat fixture did not register it. Signed follow-up 5d9864b8adb adds that real fixture-local service. All 42 affected Dark/Light fixtures render and dispose without errors; representative composers and attachment controls render again. All 16 blank-image baseline replacements were rejected, and no blocking hashes were changed. Exact-head screenshot/fixture CI passes, as does exact-head platform CI. Only the human PR review policy remains pending here.
  • SDK generation separately depends on an authorized runtime release containing the new contract.

No private user profiles, credentials, local verification logs, heap dumps, SDK/runtime build artifacts or session documents are included.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Workspace-scoped approvals can bypass target-folder trust checks and fail for isolated worktrees because repository roots are used instead of effective working directories.

Get a fresh assessment by requesting another Copilot review.

Review tier: Balanced (auto)
Findings: 1 High severity · 2 Medium severity

Note

Copilot is running an experiment and ran this review at Balanced.

Open findings (3)
What changed in this PR

Adds an opt-in local canvas development preview spanning Agent Host state, package authorization, Sessions UI, and confined Integrated Browser rendering.

Changes:

  • Adds canonical canvas protocol, persistence, context, and SDK integration.
  • Adds reviewed package snapshots with scoped approvals.
  • Adds Sessions controls, browser confinement, tooling, and tests.
File Description
src/​vs/​platform/​agentHost/​** Canvas protocol, runtime, package authorization, context, and tests
src/​vs/​sessions/​contrib/​canvases/​** Canvas presentation and actions
src/​vs/​sessions/​contrib/​providers/​agentHost/​** Package management and session integration
src/​vs/​workbench/​contrib/​browserView/​** Source-backed browser editors
src/​vs/​platform/​browserView/​** Native browser confinement and lifecycle
src/​vs/​workbench/​contrib/​chat/​** Canvas request context and draft transfer
scripts/​local-canvas-* Development setup and launch tooling
.agents/​skills/​launch/​** Isolated launch workflow updates
build/​** Build, localization, and style metadata
src/​**/​test/​** Unit and integration coverage

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread src/vs/sessions/contrib/canvases/browser/sessionCanvasActions.ts Outdated
Add the default-off local canvas preview across Agent Host, the canonical
protocol adapter, Sessions presentation, package authoring and native browser
surfaces. Preserve exact workspace/revision authority, external document data,
canvas-first retention, live source ownership and request-frozen context.

Project genuine joined-SDK requests into chat turns so canvas-originated
requests use visible tool approvals and cancellation. Include regression
coverage, runnable fixtures, launch tooling and the required JavaScript
allowlist entries for source packages that run without transpilation.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: a069b1a2-65a9-4427-b3fe-6546a3bffc9e
@ulugbekna
Ulugbek Abdullaev (ulugbekna) force-pushed the ulugbekna/agents/gh-copilot-canvases-support-vscode branch from 5de6afa to 730ecfb Compare September 13, 2026 15:30
Supply the upstream primary-view observable in the canvas binding fixture. Resolve the public SDK through Node in the Electron test runner, and accept the two platform fetch error messages for retired endpoints.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: a069b1a2-65a9-4427-b3fe-6546a3bffc9e
@github-actions

github-actions Bot commented Sep 13, 2026

Copy link
Copy Markdown
Contributor

Base: 7fe7e982 Current: 71e47dd2

No screenshot changes.

Fix the noUnusedLocals error reported by the complete CI compilation after switching dependency resolution to Node.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: a069b1a2-65a9-4427-b3fe-6546a3bffc9e
Scope launch retirement to captured working directories and refresh newly approved canvas catalogs without another turn. Correct Windows file identity and snapshot reuse, and prove child termination directly across platforms.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: a069b1a2-65a9-4427-b3fe-6546a3bffc9e
Replace remaining Unix-only exit-audit assumptions with observed process state. Prove the old backend is gone before checking endpoint unavailability, independently of platform-specific fetch errors. Give the 2048-file production-quota fixture an explicit I/O budget without changing its thresholds.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: a069b1a2-65a9-4427-b3fe-6546a3bffc9e
Register the real fixture-local view-state service required by NewChatView. All 42 affected themed fixtures render and dispose without errors; keep the existing blocking screenshot baselines unchanged.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: a069b1a2-65a9-4427-b3fe-6546a3bffc9e
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants