feat(ai): add pyoaev support for AI adversarial exposure validation (#295)#296
feat(ai): add pyoaev support for AI adversarial exposure validation (#295)#296SamuelHassine wants to merge 4 commits into
Conversation
There was a problem hiding this comment.
Pull request overview
Adds initial SDK primitives for the AI adversarial exposure validation (AI red-team injector + AI defense collector) domain by introducing new signature types, a shared deterministic marker helper, an expectations polling helper for AI collectors, and a new AI Target CRUD manager exposed on the client.
Changes:
- Add AI-related
SignatureTypesand a deterministicbuild_marker()helper for correlating inject executions to AI defense telemetry. - Add
InjectExpectationManager.ai_expectations_for_source(source_id)for polling agentless AI DETECTION/PREVENTION expectations. - Add
AiTargetManager(CRUD for/ai_targets) and wire it ontoOpenAEVasclient.ai_target.
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| pyoaev/signatures/types.py | Adds new AI signature enum values used for AI validation correlation. |
| pyoaev/signatures/ai_marker.py | Introduces deterministic per-inject marker helper shared by injector/collectors. |
| pyoaev/apis/inject_expectation/inject_expectation.py | Adds API helper to fetch AI-specific expectations for a collector/source. |
| pyoaev/apis/ai_target.py | Adds new REST manager/object for AI Target assets CRUD. |
| pyoaev/apis/init.py | Exposes the new AI Target API module via package exports. |
| pyoaev/client.py | Wires AiTargetManager onto the main OpenAEV client. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
…295) Add AI request marker / target endpoint signature types, a shared deterministic per-inject canary marker helper, ai_expectations_for_source to poll agentless detection/prevention expectations, and an AiTargetManager for AI Target assets.
Adds ARTIFICIAL_INTELLIGENCE to SecurityDomains so AI red-team contracts can be bucketed under the AI security domain.
) Add unit tests for the new AI SDK building blocks and tighten a type annotation flagged in review: - build_marker: lock the prefix, length, determinism and exact value so the injector and collectors stay byte-for-byte compatible. - AiTargetManager: validate request construction (method/path/payload) for create/get/update/delete against /ai_targets. - SignatureTypes: cover the new ai_request_marker / ai_target_endpoint values and confirm they are usable by SignatureType. - inject_expectation.ai_expectations_for_source: return List[Dict] to match the docstring (the endpoint returns a collection).
44e6876 to
8ff0bf3
Compare
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #296 +/- ##
==========================================
+ Coverage 73.58% 75.96% +2.38%
==========================================
Files 54 56 +2
Lines 2404 2426 +22
==========================================
+ Hits 1769 1843 +74
+ Misses 635 583 -52
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Review and fix summaryReviewed the full changed files (not just the diff) and reconciled the branch with What was done:
Verification (local): CI: CircleCI build / test / linter / formatting, codecov, signed commits, linked issue and PR title are all green. Remaining (non-code blockers):
|
Summary
ai_request_marker/ai_target_endpointsignature types and a shared deterministic per-inject canary marker helper (pyoaev/signatures/ai_marker.py).inject_expectation.ai_expectations_for_source(source_id)to poll agentless DETECTION / PREVENTION expectations for AI defense collectors.AiTargetManager(CRUD for AI Target assets) wired on the client asclient.ai_target.ARTIFICIAL_INTELLIGENCEsecurity domain so AI red-team contracts can be bucketed under the AI security domain.These are the SDK building blocks for the AI adversarial exposure validation domain (AI red-team injector + AI guardrail collector + openaev backend).
Dependency / merge order
Dependency root of the feature. Merge and release before:
ai-redteaminjectorai-guardrailandmitre-atlascollectorsPairs with the openaev backend endpoints
/api/injects/expectations/ai/{sourceId}and/api/ai_targets.Tests
test/signatures/test_ai_marker.py: locks the marker prefix, length, hex suffix, determinism, empty-agent default and exact value so the injector and collectors stay byte-for-byte compatible across runs and languages.test/apis/ai_target/test_ai_target.py: validates request construction (method / path / payload) forcreate,get,updateanddeleteagainst/ai_targets.test/signatures/test_ai_signature_types.py: covers the newSignatureTypesvalues and confirms they are usable bySignatureType.Notes
mainand resolved thepyoaev/signatures/types.pyconflict (kept both the new cloud signature types and the AI ones).ai_expectations_for_sourcenow returnsList[Dict[str, Any]](matching the docstring), the AI expectations path is built with a single f-string, and the three test files above were added.security/snyk (Filigran)check is failing with "You have used your limit of private tests" (Snyk account quota, unrelated to this change). All other checks (CircleCI build / test / linter / formatting, coverage, signed commits, linked issue, PR title) are green.Closes #295