[BREAKING] FEAT: Normalize simulated conversation prompts to SeedPrompt

[varunj-msft]

Description

Simulated-conversation techniques carried three YAML file paths through the seed model, the attack parameters, and the async executor, which read those files once per objective in the middle of attack execution. A technique was therefore only inspectable by chasing a file path, and its identity was a set of absolute, machine-specific path strings.

The adversarial, simulated-target, and next-message system prompts are now canonical SeedPrompt values on SeedSimulatedConversation. YAML is loaded at the setup and composition boundaries, and only prompts travel from there, so the executor no longer touches disk. Serialized configuration and identity now describe prompt content instead of file locations, which makes a technique portable across machines and lets an edit to a prompt change its identity.

This follows direction from the review on #2503. That PR was closed unmerged, so none of it was on main — all three prompts needed normalizing, not just the two mentioned in the follow-up work item:

adversarial_chat_system_prompt: SeedPrompt is canonical — not SeedPrompt | Path.
YAML is loaded with SeedPrompt.from_yaml_file(...) at setup/composition boundaries.
Only SeedPrompt is carried through SeedSimulatedConversation, AttackParameters, and the executor.
*_path remains only as a deprecated compatibility adapter and for reading legacy records.
The *_system_prompt_path inputs stay accepted on the model, the technique factory, and the public generator. They are resolved once, emit a DeprecationWarning (removed_in="1.4.0"), and never become model fields — so they never reach persistence or identity. Every shipped technique was migrated off them, so a normal run emits no deprecation warnings.

Composition now reads prompt files where it previously only built paths, so the scenario and initializer entry points that build techniques from an async method run that work in a worker thread.

Defaults, template rendering, response schemas, sequence behavior, and the database schema are unchanged. No Alembic migration is needed — only the JSON payload inside the existing unbounded, unindexed SeedPromptEntries.value column changes shape.

Why this is BREAKING
SeedSimulatedConversation canonical fields are renamed and retyped: adversarial_chat_system_prompt_path: Path becomes adversarial_chat_system_prompt: SeedPrompt, and likewise for the simulated-target and next-message prompts. The *_path spellings still work as deprecated constructor inputs, so existing callers keep working with a warning.
SeedSimulatedConversation.load_simulated_target_system_prompt is deprecated, not removed.

Things reviewers should know
Identity churn. Every simulated-conversation seed's value_sha256 changes. This hash already churned on each release, because pyrit_version is embedded in value.
Row size. Serialized value grows from roughly 200 B to roughly 4–13 KB per technique, since it now carries prompt text. This is the intended cost of carrying prompts instead of paths. SeedEntry.value and AttackResultEntry.atomic_attack_identifier are unbounded, unindexed JSON.
Legacy records with a missing file. A record written earlier names prompt files by absolute path. If those files are not present on the reading machine, get_seed() now fails at read naming the offending record (id, name, dataset) rather than failing later at execution. A silent placeholder was deliberately rejected — it would fabricate prompt content.
Legacy read then re-add. Reading a legacy path-shaped record and re-adding it can hit a pre-existing hash/id collision in add_seeds_to_memory_async, which dedupes purely by hash and reuses the row id. The same collision already occurs on main whenever value drifts across versions. Not fixed here, since it is in shared insert logic and out of scope for this change.
Overlap with #2494. That PR touches six of the same files and adds adversarial_chat_system_prompt_prefix to the same two identity methods (_compute_value, get_identifier). Now that the adversarial prompt is a real SeedPrompt, that prefix can be prepended to prompt.value directly. Worth agreeing on merge order and whether the prefix is still needed. #2551 also touches three of these files and gates on next_message_system_prompt_path, which becomes next_message_system_prompt.

Tests and Documentation

9369 unit tests pass across models, executor/attack, scenario, setup, memory, and datasets.
Ruff check/format, ty, check-async-suffix, validate-docs, and memory-migrations check ("No new upgrade operations detected") all pass.
New coverage: canonical construction and defaults; a guard that *_path never becomes a model field; deprecated-input warnings; supplying both a prompt and its path; content-based identity, including that editing one word changes the hash and that two copies of the same text from different files match; model_dump/model_validate stability in both modes; legacy and canonical record reconstruction; a named error for a legacy record whose file is missing; and a test asserting the async path performs no YAML loading at all.
Regression coverage for template preparation: a prompt is prepared exactly once, so {% raw %} deferred template syntax survives construction, the deprecated path input, and reconstruction from a persisted record.
JupyText: updated both halves of the doc/code/datasets/5_simulated_conversation and doc/code/datasets/2_seed_programming pairs inline, then verified parity by regenerating .py from each .ipynb with jupytext --to py:percent and diffing against the committed .py. Both are in sync. Also updated doc/code/memory/3_memory_data_types.md.
Exercised end to end against live Azure endpoints: output matches the pre-change baseline (4 prepended turns plus a generated next message, with preparation and adversarial conversation references preserved).