[varunj-msft]

Description

Part 2 of saving Registry entries across restarts. Stacked on part 1 (#3891); part 3 (#3908) builds on it. This PR's own changes: varunj-msft/PyRIT@varunj-msft/13178-Unreadable-Document-Listing...varunj-msft/13178-Read-Registry-Recipes

Design: 4.3 What a recipe holds, 4.2 Where recipes live. Where every part fits: PR map.

This is one of the changes for saving Registry entries across restarts. It builds on #3891. Until #3891 merges, the diff here also shows its changes.

Problem. There's no file format or storage for a saved target, converter, scorer or technique.

Change.

• pyrit/models/catalog/registry_recipe.py: the recipe models, exported from pyrit.models.catalog.
  • A target, converter or scorer recipe holds schema_version, class_module, class_name and params. A target recipe also holds auth_mode next to params, as the target create request does; any other kind with an auth_mode, or a recipe filed under the wrong kind, is rejected.
  • One RECIPE_SCHEMA_VERSION sets the format version for both models and the storage check.
  • A technique recipe holds the fields of the technique create request, without the name, with the same limits and tag rules. The library can't import the backend, so they're copied, and FEAT: Rebuild saved registry entries in dependency order (part 3) #3908 (part 3) adds a test that the two models accept and reject the same documents.
  • Unknown fields are rejected, so a file with an extra field such as credentials is reported, not silently read.
  • Also adds the failure and reference types.
• pyrit/models/request_limits.py: adds MAX_TEXT_LENGTH for the technique description. FIX: Apply the request limits to request fields added since the last limits PR #3595 adds the same line, so the two merge cleanly; until it lands, backend/models/common.py keeps its own copy of the same value.
• pyrit/registry/registry_recipe_storage.py: RegistryRecipeStorage, a FileDocumentStorage with one folder per registry (targets/, converters/, scorers/, techniques/) under one source.
  • On Blob the folder goes on the URL path, so a SAS token stays in the query.
  • list_recipes() uses the listing from the previous PR. It returns the parsed recipes with their versions (the SHA-256 of the bytes), a failure for each unreadable, invalid or newer-format file, and the ignored files.
  • The format version is checked first, so a file from a newer PyRIT says so instead of failing on a field it doesn't know.
• Failure messages and file names in the report go through one helper that escapes control characters and caps the length, so a stored key like "\n\x1b[31mFAKE" or a 4,000-digit version can't break a log line. A recipe with text that isn't valid UTF-8 anywhere in it is rejected as invalid, and file names with invalid bytes are escaped in backslash form, so the report can always be sent as JSON. Files are read as utf-8-sig, and a document nested too deeply is reported as such.
• The file name is the entry name, so only names that fit the document rule (up to 64 lowercase letters, digits and underscores, starting with a letter) can be saved. The PRs that save entries will refuse other names.

What stays the same. Nothing calls this yet; #3908 (part 3) rebuilds entries from these recipes. No API, config or behavior change.

Tests and Documentation

• New tests/unit/registry/test_registry_recipe_storage.py: recipes and their versions, field limits and unknown fields, failures for unreadable, invalid and newer-format files, ignored names, Blob folders that keep a SAS token in the query, and report text that isn't valid UTF-8 (a recipe holding it is invalid, and a file name with invalid bytes is escaped).
• The tests also pin the default format version for both models, a control-character key, a 4,000-digit version, and params nested 300 levels deep.
• pytest tests/unit/registry tests/unit/models -n 6: 3,979 passed, 5 skipped, 4 xfailed
• ruff, ruff format and ty on the changed files: passed (the pre-commit hooks can't install in my environment, so CI runs them)
• No doc change: nothing calls this yet. JupyText: not run (no notebooks changed).

[Copilot]

🟢 Approval recommended

No active review issues require changes to the registry recipe models or storage reader.

No open issues 🎉

What changed in this PR

Adds the models and storage reader needed to preserve registry entries across restarts. This second stacked PR builds on #3891’s document-listing support; entry rebuilding and startup restoration remain for later PRs.

Changes:

• Defines versioned component and technique recipes, plus restore-report types with JSON-safe text.
• Reads recipes from per-registry local or Azure Blob folders, preserving SAS queries and reporting invalid, unreadable, newer-format, and ignored files.
• Adds unit coverage for parsing, versions, validation, storage failures, and report serialization without changing existing callers.

File	Description
tests/​unit/​registry/​test_registry_recipe_storage.py	Tests recipe loading, validation, versions, Blob paths, and report serialization.
tests/​unit/​registry/​test_file_document_storage.py	Tests issue-aware local and Blob listings and safe error messages.
pyrit/​registry/​registry_recipe_storage.py	Adds per-registry storage, recipe parsing, and failure reporting.
pyrit/​registry/​file_document_storage.py	Adds issue-aware listings while retaining legacy listing behavior.
pyrit/​models/​request_limits.py	Adds the shared technique-description text limit.
pyrit/​models/​catalog/​registry_recipe.py	Defines recipe schemas, references, and restore-report models.
pyrit/​models/​catalog/​__init__.py	Exposes the new models through lazy exports.

_{Review effort: Balanced}

[Richard Lundeen (richlundeen)]

Requesting changes. My review on #3908 has the main comment about the direction for the restore work; this PR stays as it is apart from the inline comments below. Inline comments marked Blocking need to be addressed before merge; Non-blocking ones are judgment calls.

[Richard Lundeen (richlundeen)]

Automated review completed. No blocking concerns remain. This review records source coverage; merging is handled separately.