FEAT: Save targets created through the API (part 8)

.pyrit_conf_example

@@ -161,6 +161,21 @@ allow_custom_initializers: false
 # as scenario_presets_source. Defaults to ~/.pyrit/scenario_suites.
 # scenario_suites_source: https://account.blob.core.windows.net/container/scenario_suites
 
+# Optional storage for saved targets, converters, scorers, and techniques, so they
+# come back when the backend restarts. Each kind has its own folder under the source.
+# This supports the same local directory and Azure Blob sources as
+# scenario_presets_source. If any saved entry can't be restored, the backend stays
+# not ready and lists the failures in its runtime status. Fix or delete the listed
+# files, then apply the configuration or restart the backend. Defaults to
+# ~/.pyrit/registry_recipes.
+#
+# ⚠️  WARNING: The backend rebuilds every saved entry in this source as an
+# administrator, at startup and live reinitialization. Anyone who can write to this
+# source can make the backend run a model's repository code or use its identity.
+# Restrict write access to trusted administrators, and use a dedicated folder or
+# blob prefix.
+# registry_recipes_source: https://account.blob.core.windows.net/container/registry_recipes
+
 # Upload Folder for API-Created Targets
 # -------------------------------------
 # Folder that targets created through the REST API may upload local files from

doc/getting_started/pyrit_conf.md

@@ -194,6 +194,20 @@ Suites store ordered references to scenario presets and resolve each preset's cu
 
 A SAS query string may be included; otherwise, PyRIT uses `DefaultAzureCredential`. The default is `~/.pyrit/scenario_suites`. Local writes use the same hidden lock-file and atomic replacement behavior as scenario presets. Startup scans the configured source so malformed suite documents are reported immediately without preventing the backend from starting.
 
+### `registry_recipes_source`
+
+Stores saved targets, converters, scorers, and techniques in a local directory or Azure Blob container, so they come back when the backend restarts. Targets created through `POST /api/targets` with `"persist": true` are saved here. An Azure URI may include a blob-name prefix:
+
+```yaml
+registry_recipes_source: https://account.blob.core.windows.net/pyrit-storage/registry-recipes
+```
+
+Each kind has its own folder under the source (`targets/`, `converters/`, `scorers/`, `techniques/`), with one JSON file per saved entry named after the entry. A file holds the create request the entry was made with, and restore builds the entry through the API's create for its kind. A file never holds a credential: restore refuses a file that stores a value for a parameter its class marks as a credential, such as `api_key`. A saved target also can't use a key from this server: it signs in with Microsoft Entra ID (`"auth_mode": "identity"`), which uses this server's Azure identity for everyone using the app, or needs no key. Targets from the server's configuration that it uses, such as a round-robin's members, keep their own sign-in. So restore refuses a target file that would read an empty key from an environment variable or leave an address to this server's settings. The addresses each target type declares (such as `endpoint`, `container_url` for Blob storage, or an A2A agent card path) can't hold a user name, password, SAS token, fragment, path parameters (`;`, `&`, `=`), or a query other than `api-version`, and a path can't have a query at all. An address the server would otherwise fill in must be a full http(s) or ws(s) address. In identity mode, an OpenAI target file sets `"headers": "{}"` in its `params` so that the server's `OPENAI_ADDITIONAL_REQUEST_HEADERS` aren't added. The OpenAI SDK adds the headers in `OPENAI_CUSTOM_HEADERS` to every request, so while the server sets it, OpenAI targets can't be saved and saved ones aren't restored. The SDK's `OPENAI_ORG_ID` and `OPENAI_PROJECT_ID` still apply to every OpenAI target on this server. GitHub Copilot, Hugging Face, HackAPrompt, LiteLLM, HTTP, and Copilot WebSocket targets can't be saved yet. A SAS query string may be included in the source; otherwise, PyRIT uses `DefaultAzureCredential`. The default is `~/.pyrit/registry_recipes`.
+
+Restore can't tell who wrote a saved file, even one that names the user who saved it, so the backend rebuilds every entry in this source as an administrator, at startup and after a live apply. A file here can, for example, make the backend run a model's repository code, or use the server's identity on a storage container that the file names. Write access to this source is administrator authority on the backend: grant it only to principals you already trust as backend administrators, and use a dedicated directory or blob prefix that less-trusted users and services cannot write to.
+
+The backend restores saved entries at startup and after a live apply, once the configured initializers have registered theirs. If any saved entry can't be restored, the backend doesn't become ready (state `failed`, outcome `restore-failed`) and lists each failure with its reason in `GET /api/config/runtime`, so a scan can't quietly run without an entry it expects. Until then, only the configuration and status routes answer; the rest return 503. Fix or delete the listed files, or, when a folder couldn't be listed, check that the backend can read this source. Then apply the configuration, if `enable_live_reinitialization` is on, or restart the backend. Files whose names aren't valid entry names (up to 64 lowercase letters, digits, and underscores, starting with a letter) are skipped without blocking startup; administrators can see them under `restore_ignored` in `GET /api/config/runtime`.
+
 ### `initialization_scripts`
 
 Local paths to custom Python scripts containing `PyRITInitializer` subclasses. Paths can be absolute or relative to the current working directory.

frontend/e2e/fixtures/manual_send_backend.py

@@ -4,6 +4,7 @@
 """Offline backend for manual-send browser tests, with no real configuration or providers."""
 
 import os
+import tempfile
 from collections.abc import AsyncGenerator
 from contextlib import asynccontextmanager
 from unittest.mock import AsyncMock, patch
@@ -27,19 +28,22 @@ async def _initialize_offline_async(self: ConfigurationLoader, *, raise_on_initi
 
 @asynccontextmanager
 async def _offline_lifespan_async(application: FastAPI) -> AsyncGenerator[None, None]:
-    config = ConfigurationLoader(memory_db_type="in_memory", env_files=[], env_akv_ref=[])
-    with (
-        patch.dict(os.environ, {"PYRIT_AUTH_MODE": "local", "PYRIT_DEV_MODE": "true"}, clear=True),
-        patch.object(
-            ConfigurationFileService,
-            "_read_source_async",
-            new=AsyncMock(return_value="memory_db_type: in_memory\nenv_files: []\ninitializers: []\n"),
-        ),
-        patch.object(ConfigurationLoader, "load_with_overrides", return_value=config),
-        patch.object(ConfigurationLoader, "initialize_pyrit_async", new=_initialize_offline_async),
-    ):
-        async with lifespan(application):
-            yield
+    with tempfile.TemporaryDirectory() as recipes:
+        config = ConfigurationLoader(
+            memory_db_type="in_memory", env_files=[], env_akv_ref=[], registry_recipes_source=recipes
+        )
+        with (
+            patch.dict(os.environ, {"PYRIT_AUTH_MODE": "local", "PYRIT_DEV_MODE": "true"}, clear=True),
+            patch.object(
+                ConfigurationFileService,
+                "_read_source_async",
+                new=AsyncMock(return_value="memory_db_type: in_memory\nenv_files: []\ninitializers: []\n"),
+            ),
+            patch.object(ConfigurationLoader, "load_with_overrides", return_value=config),
+            patch.object(ConfigurationLoader, "initialize_pyrit_async", new=_initialize_offline_async),
+        ):
+            async with lifespan(application):
+                yield
 
 
 app.router.lifespan_context = _offline_lifespan_async

frontend/playwright.config.ts

@@ -1,3 +1,7 @@
+import { mkdtempSync, readFileSync, rmSync, writeFileSync } from "node:fs";
+import { tmpdir } from "node:os";
+import path from "node:path";
+
 import { defineConfig, devices } from "@playwright/test";
 
 import type { UserPreferences } from "./src/types";
@@ -24,6 +28,28 @@ const VITE_SERVER_OPTIONS =
 const E2E_BACKEND_PORT = process.env.PYRIT_E2E_BACKEND_PORT ?? "18000";
 const E2E_BACKEND_URL = `http://127.0.0.1:${E2E_BACKEND_PORT}`;
 
+/**
+ * Copies the seeded backend's configuration into a fresh folder that also holds
+ * its saved targets, converters, scorers, and techniques, so a run never reads
+ * or writes the ones saved on the machine running it.
+ */
+function createSeededBackendConfigFile(): string {
+  const folder = mkdtempSync(path.join(tmpdir(), "pyrit-seeded-backend-"));
+  // The runner stops the backend before it exits, so nothing still uses the folder by then.
+  process.once("exit", () => rmSync(folder, { recursive: true, force: true }));
+  const configFile = path.join(folder, "browser_config.yaml");
+  const config = readFileSync(new URL("../tests/end_to_end/browser_config.yaml", import.meta.url), "utf-8");
+  const recipesSource = JSON.stringify(path.join(folder, "registry_recipes"));
+  writeFileSync(configFile, `${config}\nregistry_recipes_source: ${recipesSource}\n`);
+  return configFile;
+}
+
+// Workers load this file too, but only the main process starts the backend.
+const SEEDED_BACKEND_CONFIG_FILE =
+  SEEDED_MODE && process.env.TEST_WORKER_INDEX === undefined
+    ? createSeededBackendConfigFile()
+    : "";
+
 export default defineConfig({
   testDir: "./e2e",
   testIgnore: ["**/analytics.spec.ts", "**/paired-outcomes.spec.ts", "**/scenario-hierarchy.spec.ts", "**/scenario-datasets.spec.ts", "**/scenario-techniques.spec.ts", "**/scan-launch.seeded.spec.ts"],
@@ -113,7 +139,7 @@ export default defineConfig({
           command:
             `cd .. && uv run --no-sync python -m pyrit.backend.pyrit_backend ` +
             `--auth-mode local --host 127.0.0.1 --port ${E2E_BACKEND_PORT} --log-level warning ` +
-            "--config-file tests/end_to_end/browser_config.yaml",
+            `--config-file "${SEEDED_BACKEND_CONFIG_FILE}"`,
           env: { PYRIT_DEV_MODE: "true" },
           url: `${E2E_BACKEND_URL}/api/health`,
           reuseExistingServer: false,

frontend/src/components/Configuration/Reinitialize.test.tsx

@@ -124,6 +124,26 @@ describe('Reinitialize', () => {
     expect(screen.getByRole('button', { name: 'Reinitialize PyRIT' })).toBeDisabled()
   })
 
+  it('should let a failed restore be retried by reinitializing', async () => {
+    const user = userEvent.setup()
+    api.getRuntimeStatus.mockResolvedValue({
+      ...ready,
+      state: 'failed',
+      outcome: 'restore-failed',
+      message: "1 saved registry entry couldn't be restored.",
+    })
+    render(
+      <TestWrapper>
+        <Reinitialize version="saved-v1" hasUnsavedChanges={false} liveReinitializationEnabled />
+      </TestWrapper>,
+    )
+    expect(await screen.findByText(/Runtime: failed\. 1 saved registry entry/)).toBeInTheDocument()
+    await waitFor(() => expect(screen.getByRole('button', { name: 'Reinitialize PyRIT' })).toBeEnabled())
+    await user.click(screen.getByRole('button', { name: 'Reinitialize PyRIT' }))
+    await user.click(within(await screen.findByRole('dialog')).getByRole('button', { name: 'Reinitialize PyRIT' }))
+    await waitFor(() => expect(api.reinitialize).toHaveBeenCalledWith('saved-v1'))
+  })
+
   describe('runtime status polling', () => {
     beforeEach(() => {
       jest.useFakeTimers()

pyrit/backend/README.md

@@ -54,6 +54,7 @@ its database collation.
 ### Targets
 - `GET /api/targets` - List available prompt targets
 - `GET /api/targets/{id}` - Get target details
+- `POST /api/targets` - Create a target
 - `POST /api/targets/{id}/detect-capabilities` - Run live capability checks and
   apply the result to the registered target. Send `{"expected_hash": "<identifier hash>"}`.
   Detection uses a private target, has a four-minute operation deadline, and
@@ -82,6 +83,38 @@ overrides; reconstruction preserves declarations from the source target.
 Per-send values use
 [`request_generation_config` metadata](../../doc/code/targets/0_prompt_targets.md#runtime-generation-configuration).
 
+A created target is Temporary: a restart or a configuration apply removes it. With `"persist": true` (and `register`
+left on), the target is saved under `registry_recipes_source` and restored at every start, and the signed-in user is
+recorded as the one who saved it. A saved target can't hold or use a key: it signs in with Microsoft Entra ID
+(`"auth_mode": "identity"`, through this server's Azure identity) or needs no key. An OpenAI target's empty `headers`
+is set to `"{}"` first, so a saved one doesn't send this server's `OPENAI_ADDITIONAL_REQUEST_HEADERS`.
+
+| A request with `persist` that... | Gets |
+|---|---|
+| names the target with anything but up to 64 lowercase letters, digits, and underscores, starting with a letter | **422** |
+| would save a key, or use one of this server's keys or address settings | **400** |
+| gives an address that can carry a credential, or, where this server's setting would apply, one that isn't a full http(s) or ws(s) address | **400** |
+| saves a target type that can't be saved, or one that a scan's launch scripts added or put in place of another | **400** |
+| saves an OpenAI target while this server sets `OPENAI_CUSTOM_HEADERS`, whose headers the OpenAI SDK adds to every request | **400** |
+| uses a Temporary target | **400** |
+| takes a name that a target or a saved file already has | **409** |
+| can't be written to storage | **503** |
+
+A request refused with **400** can be sent again without `persist`. When storage can't be written, the server log names
+the target and its folder, since the target comes back at the next start if the write reached storage anyway.
+
+Each listed target's `origin` is `saved`, `configuration` (registered by the server's startup scripts), or `temporary`
+(gone after a restart or a configuration apply, such as a target created without `persist` or registered by a scan's
+launch scripts). A saved target can only use targets that come back after a restart; targets from configuration keep
+their own sign-in.
+
+A saved target records the create request without its name: the target type, `auth_mode`, and the parameters as the
+API accepted them. Restore sends it through the same create, so it passes the same checks and follows the class's
+constructor. A class that moves within PyRIT still restores. Renaming or removing a class or a constructor parameter
+changes the saved format: such a change raises `schema_version` and adds a migration for older files, with a test that
+restores one. A file that still doesn't match fails to restore with the reason, and the backend stays not ready until
+the file is fixed or deleted, so nothing runs without the saved target.
+
 ### Dataset creation, batch mutation, and deletion
 
 `POST /api/datasets` creates a named dataset synchronously from multipart form data.
@@ -691,8 +724,9 @@ The chat preserves failed drafts across replacement, but requires converter choi
 to be reviewed again rather than restoring outputs tied to the previous registry.
 
 The offline browser fixture `frontend.e2e.fixtures.manual_send_backend:app` uses the real
-backend lifecycle with isolated in-memory SQLite, no environment files, and no default
-providers. The recovery tests register deterministic loopback targets. For example, start
+backend lifecycle with isolated in-memory SQLite, no environment files, no saved registry
+entries, and no default providers. The recovery tests register deterministic loopback
+targets. For example, start
 it with `uv run python -m uvicorn frontend.e2e.fixtures.manual_send_backend:app --host 127.0.0.1 --port 18213`
 (set `PYTHONUTF8=1` on Windows). In another shell, set `PYRIT_BACKEND_URL=http://127.0.0.1:18213`
 and `E2E_FRONTEND_PORT=31213`, then run `npx playwright test chat-recovery --project seeded --workers 1`

pyrit/backend/mappers/target_mappers.py

@@ -15,7 +15,7 @@
 from typing import Any
 
 from pyrit.models import CapabilityName, TargetIdentifier
-from pyrit.models.catalog import TargetInstance
+from pyrit.models.catalog import RegistryEntryOrigin, TargetInstance
 from pyrit.prompt_target import PromptTarget
 
 # Capability flag names that should never be surfaced as identifier-level params:
@@ -52,7 +52,9 @@ def _target_specific_params(target_identifier: TargetIdentifier) -> dict[str, An
     return {**extra, **explicit_specific} or None
 
 
-def target_object_to_instance(target_registry_name: str, target_obj: PromptTarget) -> TargetInstance:
+def target_object_to_instance(
+    target_registry_name: str, target_obj: PromptTarget, *, origin: RegistryEntryOrigin | None = None
+) -> TargetInstance:
     """
     Build a TargetInstance DTO from a registry target object.
 
@@ -63,6 +65,7 @@ def target_object_to_instance(target_registry_name: str, target_obj: PromptTarge
     Args:
         target_registry_name: The human-friendly target registry name.
         target_obj: The domain PromptTarget object from the registry.
+        origin: Where the registered target comes from, or None for an inner target.
 
     Returns:
         TargetInstance DTO with metadata derived from the object.
@@ -83,6 +86,7 @@ def target_object_to_instance(target_registry_name: str, target_obj: PromptTarge
         supports_history_replay=supports_history_replay,
         target_specific_params=_target_specific_params(target_identifier),
         inner_targets=_build_inner_targets(target_obj),
+        origin=origin,
     )
 
 

pyrit/backend/middleware/error_handlers.py

@@ -15,6 +15,7 @@
 from pyrit.backend.models import FieldError, ProblemDetail
 from pyrit.exceptions.analytics_exception import AnalyticsException
 from pyrit.exceptions.dataset_exception import DatasetAPIException
+from pyrit.registry.file_document_storage import DocumentStorageUnavailableError, DocumentUnreadableError
 
 logger = logging.getLogger(__name__)
 
@@ -192,6 +193,38 @@ async def permission_error_handler(  # pyrit-async-suffix-exempt
             content=problem.model_dump(exclude_none=True),
         )
 
+    @app.exception_handler(DocumentStorageUnavailableError)
+    async def storage_unavailable_handler(  # pyrit-async-suffix-exempt
+        request: Request,
+        exc: DocumentStorageUnavailableError,
+    ) -> JSONResponse:  # pyrit-keyword-only-exempt Starlette exception handler (request, exc)
+        """
+        Handle stored documents that can't be read or written as 503 Service Unavailable.
+
+        The log names the storage location, without credentials, and the reason. It leaves out the traceback,
+        since the error behind it can carry a SAS URL. The response never carries a path or URL.
+
+        Returns:
+            JSONResponse: RFC 7807 problem detail response with 503 status.
+        """
+        logger.warning(f"{exc} ({request.method} {request.url.path})")
+        if isinstance(exc, DocumentUnreadableError):
+            detail = f"{exc.label} '{exc.name}' can't be read: {exc.reason}"
+        else:
+            detail = f"{exc.label} storage is unavailable"
+        problem = ProblemDetail(
+            type="/errors/storage-unavailable",
+            title="Service Unavailable",
+            status=status.HTTP_503_SERVICE_UNAVAILABLE,
+            detail=detail,
+            instance=str(request.url.path),
+        )
+
+        return JSONResponse(
+            status_code=status.HTTP_503_SERVICE_UNAVAILABLE,
+            content=problem.model_dump(exclude_none=True),
+        )
+
     @app.exception_handler(NotImplementedError)
     async def not_implemented_handler(  # pyrit-async-suffix-exempt
         request: Request,