Skip to content

AH: adopt sdk llm-judged tools#325547

Draft
justschen wants to merge 12 commits into
mainfrom
justin/orbeetle
Draft

AH: adopt sdk llm-judged tools#325547
justschen wants to merge 12 commits into
mainfrom
justin/orbeetle

Conversation

@justschen

@justschen justschen commented Jul 13, 2026

Copy link
Copy Markdown
Collaborator

auto-permissions to Agent Host permission controls, delegating approval decisions to the Copilot SDK’s model-based judge.

Screenshot 2026-07-13 at 1 14 04 AM Screenshot 2026-07-13 at 9 34 53 PM

Implementation note

The permission.requested event wrapper is intentionally restored in this PR. It was previously removed when the SDK 1.0.7 tool-call ordering fix made its old ordering-workaround consumer obsolete. Auto-permissions now needs the raw event again to capture and correlate the SDK judge’s autoApproval recommendation and reason by tool call ID; the permission handler callback does not include that metadata.

Copilot AI review requested due to automatic review settings July 13, 2026 08:16

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds model-judged “Auto-permissions” to Agent Host sessions and surfaces approval explanations in confirmation UI.

Changes:

  • Adds Assisted approval schemas, settings, pickers, warnings, and policy handling.
  • Integrates SDK permission modes and model recommendations with live session configuration.
  • Propagates approval reasons through AHP and expands tests/documentation.
Show a summary per file
File Description
src/vs/platform/agentHost/common/agentHostSchema.ts Adds Assisted approval schema.
src/vs/platform/agentHost/common/agentService.ts Adds live config callback.
src/vs/platform/agentHost/common/sessionConfigKeys.ts Updates recognized approval values.
src/vs/platform/agentHost/common/state/protocol/channels-chat/actions.ts Adds approval reason to actions.
src/vs/platform/agentHost/common/state/protocol/channels-chat/reducer.ts Reduces approval reasons into state.
src/vs/platform/agentHost/common/state/protocol/channels-chat/state.ts Stores pending approval reasons.
src/vs/platform/agentHost/node/agentSideEffects.ts Forwards live config changes.
src/vs/platform/agentHost/node/copilot/copilotAgent.ts Enables SDK judging and synchronization.
src/vs/platform/agentHost/node/copilot/copilotAgentSession.ts Implements SDK permission modes and recommendations.
src/vs/platform/agentHost/node/sessionPermissions.ts Exposes effective approval levels.
src/vs/platform/agentHost/test/common/agentHostSchema.test.ts Tests Assisted schema validation.
src/vs/platform/agentHost/test/node/agentSideEffects.test.ts Tests config forwarding.
src/vs/platform/agentHost/test/node/copilotAgentSession.test.ts Tests SDK permission behavior.
src/vs/platform/agentHost/test/node/mockAgent.ts Records live config callbacks.
src/vs/sessions/SESSIONS.md Documents permission-mode behavior.
src/vs/sessions/contrib/providers/agentHost/browser/agentHostPermissionPickerDelegate.ts Offers Assisted permissions.
src/vs/sessions/contrib/providers/agentHost/browser/agentHostSessionConfigPicker.ts Styles and confirms Assisted selections.
src/vs/sessions/contrib/providers/agentHost/browser/baseAgentHostSessionsProvider.ts Updates approval normalization documentation.
src/vs/sessions/contrib/providers/agentHost/test/browser/agentHost/agentHostPermissionPickerDelegate.test.ts Tests Assisted picker behavior.
src/vs/sessions/contrib/providers/copilotChatSessions/browser/permissionPicker.ts Adds Assisted picker metadata.
src/vs/workbench/contrib/chat/browser/agentSessions/agentHost/agentHostChatInputPicker.ts Supports Assisted in Agent Host picker.
src/vs/workbench/contrib/chat/browser/agentSessions/agentHost/stateToProgressAdapter.ts Adapts approval reasons to confirmations.
src/vs/workbench/contrib/chat/browser/chat.shared.contribution.ts Registers Assisted default configuration.
src/vs/workbench/contrib/chat/browser/widget/chatContentParts/toolInvocationParts/abstractToolConfirmationSubPart.ts Renders approval-reason badges.
src/vs/workbench/contrib/chat/browser/widget/chatContentParts/toolInvocationParts/chatTerminalToolConfirmationSubPart.ts Adds terminal approval-reason badges.
src/vs/workbench/contrib/chat/browser/widget/chatContentParts/toolInvocationParts/toolRiskBadgeHelper.ts Creates badges from approval reasons.
src/vs/workbench/contrib/chat/browser/widget/input/permissionPickerActionItem.ts Adds Assisted labels and styling.
src/vs/workbench/contrib/chat/common/chatPermissionStorageKeys.ts Adds Assisted warning storage key.
src/vs/workbench/contrib/chat/common/chatPermissionWarnings.ts Adds Assisted warning flow.
src/vs/workbench/contrib/chat/common/chatService/chatServiceTelemetry.ts Documents Assisted telemetry values.
src/vs/workbench/contrib/chat/common/constants.ts Defines Assisted permission level.
src/vs/workbench/contrib/chat/common/tools/languageModelToolsService.ts Adds confirmation approval reasons.
src/vs/workbench/contrib/chat/test/browser/agentSessions/stateToProgressAdapter.test.ts Tests approval-reason propagation.
src/vs/workbench/contrib/chat/test/common/chatPermissionWarnings.test.ts Tests Assisted warning precedence.

Review details

Comments suppressed due to low confidence (2)

src/vs/platform/agentHost/node/copilot/copilotAgent.ts:2886

  • A replace-style config update can clear autoApprove by omitting the property, but this early return then leaves every live SDK chat in its previous on/auto mode while session state shows Default. This is especially unsafe when clearing Bypass during an active turn. Synchronize on every session-config change (the sync method already no-ops when the mode is unchanged), or pass the replace flag and handle omissions explicitly.

	respondToPermissionRequest(requestId: string, approved: boolean): void {
		for (const entry of this._sessions.values()) {
			for (const chat of entry.allChatSessions()) {

src/vs/platform/agentHost/node/copilot/copilotAgentSession.ts:2272

  • Global auto-approve participates in _getSdkPermissionMode, but live synchronization is only triggered at turn start and by SessionConfigChanged. Disabling the global setting during an active turn therefore leaves an already-applied SDK on mode active until the next turn, contrary to the documented immediate update and allowing subsequent tool calls to bypass confirmation. Root-config changes to this key also need to synchronize every in-memory chat.
	 * the in-memory write completes (e.g. the session was aborted), the
	 * just-written entry is deleted so it cannot leak.
	 */
	private async _buildEditsForPermission(request: ITypedPermissionRequest, toolCallId: string): Promise<{ items: FileEdit[] } | undefined> {
		if (request.kind !== 'write') {
  • Files reviewed: 34/34 changed files
  • Comments generated: 3
  • Review effort level: Medium

Comment thread src/vs/platform/agentHost/node/agentSideEffects.ts Outdated
Comment thread src/vs/platform/agentHost/node/copilot/copilotAgentSession.ts Outdated
@justschen

Copy link
Copy Markdown
Collaborator Author

@copilot resolve the merge conflicts in this pull request

Copilot AI and others added 2 commits July 13, 2026 21:30
Co-authored-by: justschen <54879025+justschen@users.noreply.github.com>
Co-authored-by: justschen <54879025+justschen@users.noreply.github.com>

Copilot AI commented Jul 13, 2026

Copy link
Copy Markdown
Contributor

@copilot resolve the merge conflicts in this pull request

Resolved in fd1d1325; follow-up compatibility fix is in 02b0ec2d.

Comment thread src/vs/platform/agentHost/node/copilot/copilotAgent.ts Outdated
@github-actions

github-actions Bot commented Jul 13, 2026

Copy link
Copy Markdown
Contributor

Base: 47657dea Current: c8d08df2

No screenshot changes.

@justschen

Copy link
Copy Markdown
Collaborator Author

@copilot resolve the merge conflicts in this pull request

Co-authored-by: justschen <54879025+justschen@users.noreply.github.com>

Copilot AI commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

@copilot resolve the merge conflicts in this pull request

Resolved in merge commit 9234924a216.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants