FEAT: Block GUI sends when target doesn't support the modality#1692
Open
romanlutz wants to merge 11 commits intomicrosoft:mainfrom
Open
FEAT: Block GUI sends when target doesn't support the modality#1692romanlutz wants to merge 11 commits intomicrosoft:mainfrom
romanlutz wants to merge 11 commits intomicrosoft:mainfrom
Conversation
Expose supported_input_data_types from TargetCapabilities through the backend DTO and mapper so the frontend knows which input modalities a target accepts. ChatInputArea now shows a non-blocking warning when the user attaches a file whose type (image, audio, video, file) is not in the target's supported input data types. Changes: - Add supported_input_data_types field to TargetInstance backend model - Flatten input_modalities in target_object_to_instance mapper - Add supported_input_data_types to frontend TargetInstance type - Add file->binary_path mapping in converterTypes - Derive unsupported attachment types in ChatInputArea and show warning - Add backend mapper tests and frontend component tests Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Change the unsupported modality warning from non-blocking to blocking: - Disable the send button when unsupported attachment types are present - Guard handleSend against unsupported types - Update warning text to instruct user to remove the attachment - Update tests to verify send is disabled Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Extend modality validation to also check converter output data types, not just raw attachment types. When a converter transforms text into an unsupported type (e.g., text-to-image on a text-only target), the send button is now disabled with a warning. Changes: - Add outputDataType to PieceConversion interface - Pass output type from ConverterPanel through ConverterPreview - Pass converterOutputDataTypes from ChatWindow to ChatInputArea - Validate converter outputs against target supported_input_data_types - Update all affected tests Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Increase textInput and convertedTextarea maxHeight from 96px/80px to 30vh each, allowing each to use up to ~30% of viewport height - Remove JS-side 96px cap on auto-resize so CSS maxHeight controls it - Add matching webkit scrollbar styling to convertedTextarea - Use alignItems flex-start on both rows so they grow naturally Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
When a conversion is active, both textareas now cap at 15vh each (half of the 30vh total) via inline style overrides. Without a conversion, the original textarea gets the full 30vh. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Wrap both original and converted textareas in a single scrollable container (textScrollArea) with max-height 60vh and resize: vertical - Textareas grow naturally inside it with overflow hidden (no per-textarea scrollbars), the outer container handles scrolling - Move clear-conversion button to columnRight below send with tooltip - Both textareas share space equally within the scrollable area Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Remove overflow:hidden from inputWrapper that was blocking scroll - Add clearConversionButton style with circular border (matches send button shape but uses subtle neutral colors for discoverability) - Increase columnRight gap to spacingVerticalS for more breathing room between info icon and send button - Add marginRight to convertedBadge matching originalBadge for consistent indentation Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Textarea sizing: - Each textarea auto-resizes via JS (height = scrollHeight) and is capped by CSS max-height with overflowY: auto for scrolling - Original textarea: 60vh when solo, 30vh when conversion active (via textInputShared class toggled with mergeClasses) - Converted textarea: 30vh with matching scrollbar styling - Both use identical webkit scrollbar styles - Converted textarea also auto-resizes via its own ref + useLayoutEffect Button styling: - Attach and converter buttons get round border (iconButton) - Clear conversion button has distinct circular style with subtle border - More gap between attach/converter buttons (spacingVerticalS) - More gap in columnRight between info/send/clear (spacingVerticalS) Badge alignment: - convertedBadge now has marginRight matching originalBadge Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Replace the ad-hoc supports_multi_turn and supported_input_data_types top-level fields on TargetInstance with a single nested TargetCapabilitiesInfo DTO that mirrors the full TargetCapabilities domain dataclass: - 6 boolean capability flags (supports_multi_turn, supports_json_*, etc.) - supported_input_data_types: flattened sorted unique input data types - supported_output_data_types: flattened sorted unique output data types Filtering of capability-named identifier params in the mapper now covers the full CapabilityName enum so future flags don't accidentally leak into target_specific_params. Frontend: TargetInstance.capabilities is optional (consumer code uses ?. ) so missing/old responses fail open. All consumers updated. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The GUI previously didn't tell users when a target doesn't support a modality they're sending (e.g., attaching an image to a text-only target, or applying a text-to-image converter against a text-only target). This resulted in cryptic backend errors.
This PR exposes target input modality capabilities to the frontend and blocks sends with a clear warning when there's a mismatch.
Changes
Backend
pyrit/backend/models/targets.py— Addedsupported_input_data_types: list[str]toTargetInstanceDTOpyrit/backend/mappers/target_mappers.py— Flatteninput_modalitiesfromTargetCapabilitiesinto the new fieldFrontend — Modality validation
frontend/src/types/index.ts— Addedsupported_input_data_typestoTargetInstancetypefrontend/src/components/Chat/converterTypes.ts— Addedfile → binary_pathmapping; addedoutputDataTypetoPieceConversionfrontend/src/components/Chat/ChatInputArea.tsx— Validates attachment types AND converter output types against target capabilities; disables send button with warning when mismatchedfrontend/src/components/Chat/ChatWindow.tsx— PassesconverterOutputDataTypesto ChatInputAreafrontend/src/components/Chat/ConverterPanel/ConverterPanel.tsx— Passes selected converter's output type to ConverterPreviewfrontend/src/components/Chat/ConverterPanel/ConverterPreview.tsx— IncludesoutputDataTypein PieceConversionFrontend — Input area styling
Tests
supported_input_data_typesScreenshot
This shows the shared space with two scroll bars, realigned buttons.