Feat/garak exploitation scenario

[Joel (aspire488)]

Description

Tests and Documentation

Summary

Adds a PyRIT-native implementation of Garak's exploitation probe family.

What's included

• "Exploitation" scenario with:
  • "JinjaTemplatePythonInjection"
  • "SQLInjectionEcho"
• Curated Garak-derived payload datasets and reusable echo template.
• Payload provenance and expected-output metadata.
• Payload-specific "SubStringScorer" instances.
• Stable payload-hash-based atomic attack names.
• Bounded payload execution through "max_payloads_per_technique".
• Generic baseline disabled because exploitation success is payload-specific.
• Focused unit tests covering:
  • Technique expansion
  • Seed-group construction
  • Payload metadata
  • Scorer selection
  • Stable attack naming
• Updated scanner documentation and notebook examples.

Safety / scope

The implementation tests whether a target model emits exploit material; it does not execute the generated Python or SQL against a runtime, database, or external system.

The initial datasets use a small non-destructive subset of the upstream Garak payload families.

Reference

Inspired by Garak's "exploitation" probe:

https://github.com/NVIDIA/garak/blob/main/garak/probes/exploitation.py

Closes #2534.

[Roman Lutz (romanlutz)]

Seems like a copy of #2576 which is covering the exploitation scenario. I'll close unless there's a reason for having this additionally.

[Joel (aspire488)]

Thanks for pointing that out — I hadn’t caught that #2576 was already covering the exploitation scenario.

That explains the overlap, so no issue with closing this one. I appreciate you flagging the existing work rather than letting a duplicate contribution move forward.

I’ll take the existing implementation in #2576 as the canonical direction and look for non-overlapping contribution opportunities in PyRIT going forward.

[Roman Lutz (romanlutz)]

Joel (@aspire488) inopened a bunch of new issues today and there are a few older ones unassigned that are marked as "help wanted". DM me on discord if you want help identifying any

[Joel (aspire488)]

Joel (@aspire488) inopened a bunch of new issues today and there are a few older ones unassigned that are marked as "help wanted". DM me on discord if you want help identifying any

Thanks Roman, I really appreciate that.

I’d definitely be interested in getting help identifying a suitable issue. I’m trying to focus on contributions where there’s a clear engineering problem and where I can properly understand, test, and contribute to the existing direction rather than just opening PRs for the sake of activity.

I’ll take a look through the older unassigned help wanted issues as well, and I’ll DM you on Discord with the ones I’m considering.

Thanks again for the offer!

[Joel (aspire488)]

Joel (@aspire488) inopened a bunch of new issues today and there are a few older ones unassigned that are marked as "help wanted". DM me on discord if you want help identifying any

Thanks Roman! I’d definitely appreciate the help. I’ll take a look through the older unassigned help wanted issues as well.

Also, what’s your Discord username/handle? I couldn’t find a publicly listed one, so I want to make sure I’m DMing the right account.

[Roman Lutz (romanlutz)]

This page links our Discord https://microsoft.github.io/PyRIT/latest/contributing/readme/#ways-to-contribute and my username is identical there 😉

[Joel (aspire488)]

This page links our Discord https://microsoft.github.io/PyRIT/latest/contributing/readme/#ways-to-contribute and my username is identical there 😉

Roman Lutz (@romanlutz) Got it 😄 Thanks for pointing me to the official Discord and confirming your username. I’ll join through the contribution page and reach out there about the older help wanted issues. I’ll also make sure to focus on one where I can understand the existing direction and contribute something genuinely useful. Appreciate the help!