Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/instructions/converters.instructions.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,7 @@ from pyrit.identifiers import ComponentIdentifier

For LLM-based converters, also import:
```python
from pyrit.prompt_target import PromptChatTarget
from pyrit.prompt_target import PromptTarget
```

## Constructor Pattern
Expand All @@ -77,7 +77,7 @@ from pyrit.common.apply_defaults import apply_defaults

class MyConverter(PromptConverter):
@apply_defaults
def __init__(self, *, target: PromptChatTarget, template: str = "default") -> None:
def __init__(self, *, target: PromptTarget, template: str = "default") -> None:
...
```

Expand Down
2 changes: 1 addition & 1 deletion .github/instructions/scenarios.instructions.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ class MyScenario(Scenario):
def __init__(
self,
*,
adversarial_chat: PromptChatTarget | None = None,
adversarial_chat: PromptTarget | None = None,
objective_scorer: TrueFalseScorer | None = None,
scenario_result_id: str | None = None,
) -> None:
Expand Down
2 changes: 1 addition & 1 deletion doc/code/datasets/5_simulated_conversation.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -544,7 +544,7 @@
"| Parameter | Type | Description |\n",
"|-----------|------|-------------|\n",
"| `objective` | `str` | The goal the adversarial chat works toward |\n",
"| `adversarial_chat` | `PromptChatTarget` | The LLM that generates attack prompts (also plays the simulated target) |\n",
"| `adversarial_chat` | `PromptTarget` | The LLM that generates attack prompts (also plays the simulated target). Must declare `supports_multi_turn=True` and `supports_editable_history=True`. |\n",
"| `objective_scorer` | `TrueFalseScorer` | Evaluates whether the final turn achieved the objective |\n",
"| `num_turns` | `int` | Number of conversation turns to generate (default: 3) |\n",
"| `adversarial_chat_system_prompt_path` | `str \\| Path` | System prompt for the adversarial chat role |\n",
Expand Down
2 changes: 1 addition & 1 deletion doc/code/datasets/5_simulated_conversation.py
Original file line number Diff line number Diff line change
Expand Up @@ -125,7 +125,7 @@
# | Parameter | Type | Description |
# |-----------|------|-------------|
# | `objective` | `str` | The goal the adversarial chat works toward |
# | `adversarial_chat` | `PromptChatTarget` | The LLM that generates attack prompts (also plays the simulated target) |
# | `adversarial_chat` | `PromptTarget` | The LLM that generates attack prompts (also plays the simulated target). Must declare `supports_multi_turn=True` and `supports_editable_history=True`. |
# | `objective_scorer` | `TrueFalseScorer` | Evaluates whether the final turn achieved the objective |
# | `num_turns` | `int` | Number of conversation turns to generate (default: 3) |
# | `adversarial_chat_system_prompt_path` | `str \| Path` | System prompt for the adversarial chat role |
Expand Down
2 changes: 1 addition & 1 deletion doc/code/executor/attack/2_red_teaming_attack.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -1456,7 +1456,7 @@
"source": [
"## Other Multi-Turn Attacks\n",
"\n",
"The above examples should work using other multi-turn attacks with minimal modification. Check out attacks under `pyrit.executor.attack.multi_turn` for other examples, like Crescendo and Tree of Attacks. These algorithms are always more effective than `RedTeamingAttack`, which is a simple algorithm. However, `RedTeamingAttack` by its nature supports more targets - because it doesn't modify conversation history it can support any `PromptTarget` and not only `PromptChatTargets`."
" \"The above examples should work using other multi-turn attacks with minimal modification. Check out attacks under `pyrit.executor.attack.multi_turn` for other examples, like Crescendo and Tree of Attacks. These algorithms are always more effective than `RedTeamingAttack`, which is a simple algorithm. However, `RedTeamingAttack` by its nature supports more targets - because it doesn't modify conversation history it can support any `PromptTarget`, not only chat-style targets that declare `supports_multi_turn=True` and `supports_editable_history=True`.\""
]
}
],
Expand Down
2 changes: 1 addition & 1 deletion doc/code/executor/attack/2_red_teaming_attack.py
Original file line number Diff line number Diff line change
Expand Up @@ -314,4 +314,4 @@
# %% [markdown]
# ## Other Multi-Turn Attacks
#
# The above examples should work using other multi-turn attacks with minimal modification. Check out attacks under `pyrit.executor.attack.multi_turn` for other examples, like Crescendo and Tree of Attacks. These algorithms are always more effective than `RedTeamingAttack`, which is a simple algorithm. However, `RedTeamingAttack` by its nature supports more targets - because it doesn't modify conversation history it can support any `PromptTarget` and not only `PromptChatTargets`.
# The above examples should work using other multi-turn attacks with minimal modification. Check out attacks under `pyrit.executor.attack.multi_turn` for other examples, like Crescendo and Tree of Attacks. These algorithms are always more effective than `RedTeamingAttack`, which is a simple algorithm. However, `RedTeamingAttack` by its nature supports more targets - because it doesn't modify conversation history it can support any `PromptTarget`, not only chat-style targets that declare `supports_multi_turn=True` and `supports_editable_history=True`.
2 changes: 1 addition & 1 deletion doc/code/setup/default_values.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ from pyrit.common.apply_defaults import apply_defaults

class MyConverter(PromptConverter):
@apply_defaults
def __init__(self, *, converter_target: Optional[PromptChatTarget] = None, temperature: Optional[float] = None):
def __init__(self, *, converter_target: Optional[PromptTarget] = None, temperature: Optional[float] = None):
self.converter_target = converter_target
self.temperature = temperature
```
Expand Down
89 changes: 81 additions & 8 deletions doc/code/targets/0_prompt_targets.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,20 +19,93 @@ async def send_prompt_async(self, *, message: Message) -> Message:

A `Message` object is a normalized object with all the information a target will need to send a prompt, including a way to get a history for that prompt (in the cases that also needs to be sent). This is discussed in more depth [here](../memory/3_memory_data_types.md).

## PromptChatTargets vs PromptTargets
## Chat-style targets vs general targets
Comment thread
romanlutz marked this conversation as resolved.

A `PromptTarget` is a generic place to send a prompt. With PyRIT, the idea is that it will eventually be consumed by an AI application, but that doesn't have to be immediate. For example, you could have a SharePoint target. Everything you send a prompt to is a `PromptTarget`. Many attacks work generically with any `PromptTarget` including `RedTeamingAttack` and `PromptSendingAttack`.

With some algorithms, you want to send a prompt, set a system prompt, and modify conversation history (including PAIR [@chao2023pair], TAP [@mehrotra2023tap], and flip attack [@li2024flipattack]). These often require a `PromptChatTarget`, which implies you can modify a conversation history. `PromptChatTarget` is a subclass of `PromptTarget`.
With some algorithms, you want to send a prompt, set a system prompt, and modify conversation history (including PAIR [@chao2023pair], TAP [@mehrotra2023tap], and flip attack [@li2024flipattack]). These algorithms require a target whose [`TargetCapabilities`](#target-capabilities) declare both `supports_multi_turn=True` and `supports_editable_history=True` — i.e. you can modify a conversation history. Consumers express this requirement via `CHAT_TARGET_REQUIREMENTS` and validate it against `target.configuration` at construction time. See [Target Capabilities](#target-capabilities) below for the full list of capabilities and how they compose into a `TargetConfiguration`.

Note: The previous `PromptChatTarget` class is **deprecated** as of v0.13.0 and will be removed in v0.15.0. Use `PromptTarget` directly with a `TargetConfiguration` declaring `supports_multi_turn=True` and `supports_editable_history=True`. See [Target Capabilities](#target-capabilities) for details.


Here are some examples:

| Example | Is `PromptChatTarget`? | Notes |
|-------------------------------------|---------------------------------------|-------------------------------------------------------------------------------------------------|
| **OpenAIChatTarget** (e.g., GPT-4) | **Yes** (`PromptChatTarget`) | Designed for conversational prompts (system messages, conversation history, etc.). |
| **OpenAIImageTarget** | **No** (not a `PromptChatTarget`) | Used for image generation; does not manage conversation history. |
| **HTTPTarget** | **No** (not a `PromptChatTarget`) | Generic HTTP target. Some apps might allow conversation history, but this target doesn't handle it. |
| **AzureBlobStorageTarget** | **No** (not a `PromptChatTarget`) | Used primarily for storage; not for conversation-based AI. |
| Example | Chat-style target? | Notes |
|-------------------------------------|---------------------------------------------------|-------------------------------------------------------------------------------------------------|
| **OpenAIChatTarget** (e.g., GPT-4) | **Yes** (multi-turn + editable history) | Designed for conversational prompts (system messages, conversation history, etc.). |
| **OpenAIImageTarget** | **No** | Used for image generation; does not manage conversation history. |
| **HTTPTarget** | **No** | Generic HTTP target. Some apps might allow conversation history, but this target doesn't handle it. |
| **AzureBlobStorageTarget** | **No** | Used primarily for storage; not for conversation-based AI. |

## Target Capabilities

Every `PromptTarget` exposes a `TargetConfiguration` (via `target.configuration`) that declares what the target natively supports. This lets attacks, converters, and scorers reason about whether a given target is suitable for a given workflow — and, where possible, adapt automatically when a capability is missing.

A `TargetConfiguration` composes three concerns:

- **`TargetCapabilities`** — an immutable, declarative description of what the target natively supports.
- **`CapabilityHandlingPolicy`** — for each capability that *can* be adapted, whether to `ADAPT` (apply a normalization step to work around the gap) or `RAISE` (fail immediately).
- **`ConversationNormalizationPipeline`** — the ordered set of normalizers derived from the gap between the declared capabilities and the policy.

### Capabilities

`TargetCapabilities` declares the following flags:

| Capability | Meaning |
|----------------------------------|------------------------------------------------------------------------------------------------------------------------------------------|
| `supports_multi_turn` | The target accepts and uses conversation history (or maintains state externally, e.g. via a WebSocket). |
| `supports_multi_message_pieces` | The target accepts more than one `MessagePiece` in a single request (e.g. text + image in one message). |
| `supports_editable_history` | The conversation history can be modified after the fact. Implies `supports_multi_turn`. Required for attacks that rewrite prior turns. |
| `supports_system_prompt` | The target natively supports a system-role message. |
| `supports_json_output` | The target supports a "json" response format that guarantees valid JSON output. |
| `supports_json_schema` | The target supports constraining output to a caller-provided JSON schema. |
| `input_modalities` | The set of input modality combinations the target accepts (e.g. `{text}`, `{text, image_path}`). |
| `output_modalities` | The set of output modality combinations the target produces. |

Each target class defines defaults; instances can override individual capabilities when they depend on deployment configuration (e.g. `HTTPTarget`, `PlaywrightTarget`).

For well-known underlying models, you can look up a profile with `TargetCapabilities.get_known_capabilities(underlying_model="gpt-4o")`.

### How consumers use capabilities

Components that need a particular capability declare it as a `TargetRequirements` and validate at construction time:

```python
from pyrit.prompt_target import CHAT_TARGET_REQUIREMENTS

CHAT_TARGET_REQUIREMENTS.validate(target=target)
```

`TargetRequirements.validate` collects every missing capability and raises a single `ValueError`. For one-off checks against a single capability you can also call `target.configuration.ensure_can_handle(capability=...)` directly.

### Adapting vs raising

Some capability gaps can be papered over by PyRIT itself. For example, a single-turn target can be made to *appear* multi-turn by flattening the conversation history into a single prompt before sending. The `CapabilityHandlingPolicy` controls this on a per-capability basis:

- `UnsupportedCapabilityBehavior.RAISE` — fail at construction time. This is the safe default.
- `UnsupportedCapabilityBehavior.ADAPT` — run the corresponding normalizer in the conversation pipeline before the target sees the messages.

Non-adaptable capabilities (e.g. `supports_editable_history`) are not represented in the policy at all; requesting them on a target that lacks them always raises.

### Overriding capabilities per instance

For targets whose capabilities depend on deployment (HTTP endpoints, Playwright-driven UIs, custom backends), pass a `TargetConfiguration` to the constructor:

```python
from pyrit.prompt_target.common.target_capabilities import TargetCapabilities
from pyrit.prompt_target.common.target_configuration import TargetConfiguration

config = TargetConfiguration(
capabilities=TargetCapabilities(
supports_multi_turn=True,
supports_editable_history=True,
supports_system_prompt=True,
),
)
target = MyHTTPTarget(custom_configuration=config, ...)
```

The full implementation lives in [`pyrit/prompt_target/common/target_capabilities.py`](https://github.com/microsoft/PyRIT/blob/main/pyrit/prompt_target/common/target_capabilities.py) and [`pyrit/prompt_target/common/target_configuration.py`](https://github.com/microsoft/PyRIT/blob/main/pyrit/prompt_target/common/target_configuration.py). For runnable examples — inspecting capabilities on a real target, comparing known model profiles, and `ADAPT` vs `RAISE` in action — see [Target Capabilities](./6_1_target_capabilities.ipynb).

## Multi-Modal Targets

Expand Down
2 changes: 1 addition & 1 deletion doc/code/targets/10_3_websocket_copilot_target.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,7 @@
"\n",
"The `WebSocketCopilotTarget` supports multi-turn conversations by leveraging Copilot's server-side conversation management. It automatically generates consistent `session_id` and `conversation_id` values for each PyRIT conversation, enabling Copilot to maintain context across multiple turns.\n",
"\n",
"However, this target does not support setting a system prompt nor modifying conversation history. As a result, it cannot be used with attack strategies that require altering prior messages (such as PAIR, TAP, or flip attack) or in contexts where a `PromptChatTarget` is required.\n",
" \"However, this target does not support setting a system prompt nor modifying conversation history. As a result, it cannot be used with attack strategies that require altering prior messages (such as PAIR, TAP, or flip attack) or in contexts where a chat-style target (one that declares `supports_multi_turn=True` and `supports_editable_history=True`) is required.\\n\",\n",
"\n",
"Here is a simple multi-turn conversation example:"
]
Expand Down
2 changes: 1 addition & 1 deletion doc/code/targets/10_3_websocket_copilot_target.py
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@
#
# The `WebSocketCopilotTarget` supports multi-turn conversations by leveraging Copilot's server-side conversation management. It automatically generates consistent `session_id` and `conversation_id` values for each PyRIT conversation, enabling Copilot to maintain context across multiple turns.
#
# However, this target does not support setting a system prompt nor modifying conversation history. As a result, it cannot be used with attack strategies that require altering prior messages (such as PAIR, TAP, or flip attack) or in contexts where a `PromptChatTarget` is required.
# However, this target does not support setting a system prompt nor modifying conversation history. As a result, it cannot be used with attack strategies that require altering prior messages (such as PAIR, TAP, or flip attack) or in contexts where a chat-style target (one that declares `supports_multi_turn=True` and `supports_editable_history=True`) is required.
#
# Here is a simple multi-turn conversation example:

Expand Down
Loading
Loading