feat(AF-626): result-export governance & DLP — policies, watermarked exports, RESULT_EXPORTED audit - #726
Merged
Conversation
Per-datasource export policies (ALLOW/WATERMARK/ROW_CAP/DENY_CLASSIFIED, most-restrictive-wins, no admin bypass) gate a new signed CSV/PDF result export endpoint and the recurring-email results attachment. Classified detection reuses AF-447 tags at export time; watermarks (exporter, UTC timestamp, query id) are baked into the signed bytes; every export writes a RESULT_EXPORTED audit row and classified exports raise the new SENSITIVE_RESULT_EXPORTED notification to org admins. Closes #626
Contributor
Frontend Test Results1 530 tests +23 1 530 ✅ +23 5m 16s ⏱️ + 1m 11s Results for commit 2fefc95. ± Comparison against base commit ec3a0be. This pull request removes 1 and adds 24 tests. Note that renamed tests count towards both.♻️ This comment has been updated with latest results. |
Contributor
Coverage Report for Frontend Coverage (frontend)
File Coverage
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Contributor
Contributor
The tenth datasource-settings tab overflows a 1280px viewport, putting AntD Tabs into transform-scrolled mode where a physical Playwright click on a scrolled-out tab is silently swallowed (classification-tags spec: clicking Masking while Classification is active). Shorten the export tab label in all seven locales and switch the affected specs to a shared dispatch-based clickTab helper. Full main e2e suite green on a fresh stack: 283 passed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #626
What
Result-export governance & DLP: masking and row-level security govern what a user sees; this adds per-datasource policies governing what leaves.
export_policy(V145): per-datasource rows with a mode (ALLOW < WATERMARK < ROW_CAP < DENY_CLASSIFIED, most-restrictive-wins), optionalrow_cap/deny_classifications, and the row-securityapplies_to_*polarity (all empty ⇒ every exporter, no admin bypass). CRUD at/api/v1/datasources/{id}/export-policiesbehind the newEXPORT_POLICY_MANAGEpermission (V146 seeds ADMIN).GET /api/v1/queries/{id}/results/export?format=CSV|PDFrenders the persisted result snapshot governed by the caller's effective decision — deny ⇒ 403RESULT_EXPORT_DENIED; watermark/cap ⇒ theResultExportWatermarkstamp (exporter email, UTC timestamp, query id) baked into the bytes before RS256 signing, so stripping it invalidates the signature. Classified detection runs at export time: persisted result columns + snapshotreferenced_tablesmatched against the AF-447 classification tags (table-level tags classify all returned columns; column-level match by name).RESULT_EXPORTEDaudit rows: fail-hard on the download path (no audit row ⇒ no bytes), best-effort (trigger=email_attachment, actor = recipient) on the email path.SENSITIVE_RESULT_EXPORTEDnotification: fires only when an export actually moved data AND classified columns were present; advisory fan-out to org admins on all active channels (GRANT_STALE shape — never pages, never opens a ticket). Full 16-site fan-out incl. Thymeleaf template, 4 chat factories, email subject/template, in-app bell, 14 locale files.utils/watermarkPreview.tsmirrors the backend template byte-for-byte).Deliberately out of scope: MCP
get_query_resultstays view-parity (same data the in-app table shows).Design notes
/queries/...): compliance already depends onworkflow.api(snapshots) +core.apiand owns the signing pipeline; the reverse edge from workflow would cycle. New module edgenotifications → compliance.api/eventsis acyclic (ApplicationModulesTestgreen).SensitiveResultExportedEventis published outside any transaction ⇒ consumed with a plain@EventListener(theQuerySnapshotListenertrap).deny_classificationsstores enum names asTEXT[](theapplies_to_rolesprecedent) rather than a PG enum array — Hibernate's enum-array binding is unreliable; the service validates values.Docs & website (same commit set)
docs/03-data-model.md(export_policy, audit actions),docs/04-api-spec.md(CRUD + export + decision endpoints),docs/05-backend.md(new section),docs/06-frontend.md,docs/07-security.md,docs/08-notifications.md,docs/09-deployment.md(ACCESSFLOW_COMPLIANCE_RESULT_EXPORT_MAX[_PDF]_ROWS),docs/12-roadmap.md(v2.2 entry),README.md,website/index.html+website/docs/index.html+sitemap.xml+website/README.md.Testing
ApplicationModulesTest,ApiPackageDependencyTest,MessagesParityTestgreen; fullmvn verify -Pcoveragegreen.export-policies.spec.ts(deny 403 + decision, signed & watermarked download, disabled button with reason, admin tab UI) plus the drift-risk specs (query-execute,datasource-settings,masking-policies,query-list,admin-audit-log,custom-roles,auth-guard-roles) — 35/35 green. Local note: host port 8080 is occupied on this machine, so the stack ran via a local-only compose override (backend on 18080 + bind-mounted runtime-config.js); no repo files affected.Review notes
Four independent review agents ran pre-PR (af-verifier, af-reviewer, af-java-reviewer, af-frontend-reviewer). Everything each of them flagged was either fixed in this branch or is recorded below.
Fixed in response to review:
NotificationListener.onQueryExecutedandQuerySnapshotListener.onQueryExecutedare unordered peer listeners on the same event, anddecide()returned allow-all when the snapshot was missing. Fixed twice over:QuerySnapshotListenernow carries@Order(HIGHEST_PRECEDENCE)so the snapshot exists before any peer consumes the event, anddecide()now fails closed (deny ⇒ attachment suppressed, mail still delivers) when no snapshot is resolvable — with tests for both.@Access(AccessType.FIELD)onExportPolicyEntity; WARN log on unparseable stored rows before the 404; debug log +Collectorsimport in the context builder;400documented on the CRUD@ApiResponses.docs/configuration/datasources/chapter gained an "Export policies" paragraph (withdateModified/sitemap bumps) so the hub card's promise is backed by content; CLAUDE.md's module map now mentions compliance: result-export governance & DLP — per-role export policies, watermarked exports, RESULT_EXPORTED audit rows #626 undercompliance/.Surviving concerns (acknowledged, not changed):
RowSecurityTab; fixing it is a cross-tab change out of scope here.SENSITIVE_RESULT_EXPORTEDnotification per recipient attachment. Intentional: each attachment is a distinct egress with its own audit row; docs/08 treats each egress as a distinct event.SensitiveResultExportNotificationTest.neverPages/neverOpensATicketand documented in docs/08.