diff --git a/README.md b/README.md index 629ec93115c..929907e9888 100644 --- a/README.md +++ b/README.md @@ -34,7 +34,7 @@ Pro Edition: [pro.demo.defectdojo.com](https://pro.demo.defectdojo.com) OWASP Community Edition: [demo.defectdojo.org](https://demo.defectdojo.org) -Either demo enviornment can be logged into with username `admin` and password `1Defectdojo@demo#appsec`. Please note that the demos are publicly accessible +Either demo environment can be logged into with username `admin` and password `1Defectdojo@demo#appsec`. Please note that the demos are publicly accessible and reset every day. Do not put sensitive data in the demo. An easy way to test DefectDojo is to upload some [sample scan reports](https://github.com/DefectDojo/django-DefectDojo/tree/master/unittests/scans). ## Quick Start for Docker Compose diff --git a/docs/assets/images/PT_ss2.png b/docs/assets/images/PT_ss2.png new file mode 100644 index 00000000000..1602a7bb2c8 Binary files /dev/null and b/docs/assets/images/PT_ss2.png differ diff --git a/docs/assets/images/org_ss1.png b/docs/assets/images/org_ss1.png new file mode 100644 index 00000000000..4fe9abedc29 Binary files /dev/null and b/docs/assets/images/org_ss1.png differ diff --git a/docs/assets/images/org_ss2.png b/docs/assets/images/org_ss2.png new file mode 100644 index 00000000000..d01d68e88da Binary files /dev/null and b/docs/assets/images/org_ss2.png differ diff --git a/docs/content/asset_modelling/engagements_tests/OS__engagements.md b/docs/content/asset_modelling/engagements_tests/OS__engagements.md index 781f654af44..824644d611d 100644 --- a/docs/content/asset_modelling/engagements_tests/OS__engagements.md +++ b/docs/content/asset_modelling/engagements_tests/OS__engagements.md @@ -66,7 +66,7 @@ Alternatively, Engagements within a particular Product can be accessed from the Engagements sit below Products and above Tests in the object hierarchy. As such, access to a Product automatically grants access to all Engagements within that Product. Engagements do not have independent access control lists. -## Engagement Lifecycle +## Working with Engagements ### Create Engagements diff --git a/docs/content/asset_modelling/engagements_tests/OS__products.md b/docs/content/asset_modelling/engagements_tests/OS__products.md index 0a2fa253a25..d0b07c5fd5c 100644 --- a/docs/content/asset_modelling/engagements_tests/OS__products.md +++ b/docs/content/asset_modelling/engagements_tests/OS__products.md @@ -91,7 +91,7 @@ Product views contain a variety of tables and charts to interpret a Product’s - **Notifications** - Toggles notifications on and off depending on specific events (e.g., an Engagement has been added or closed) -## Product Lifecycle +## Working with Products ### Create Products diff --git a/docs/content/asset_modelling/engagements_tests/OS__tests.md b/docs/content/asset_modelling/engagements_tests/OS__tests.md index bda58c23bbf..799a29d7303 100644 --- a/docs/content/asset_modelling/engagements_tests/OS__tests.md +++ b/docs/content/asset_modelling/engagements_tests/OS__tests.md @@ -104,7 +104,7 @@ The following settings are available within each Test view: - **View History** - Opens a history of edits made to the Test for tracking, reporting, and auditing purposes. -## Test Lifecycle +## Working with Tests ### Create Tests diff --git a/docs/content/asset_modelling/engagements_tests/OS_producttype.md b/docs/content/asset_modelling/engagements_tests/OS_producttype.md new file mode 100644 index 00000000000..4fb37c688be --- /dev/null +++ b/docs/content/asset_modelling/engagements_tests/OS_producttype.md @@ -0,0 +1,135 @@ +--- +title: "Product Types" +description: "Understanding Product Types in DefectDojo OS" +audience: opensource +weight: 1 +--- +**PRODUCT TYPES** → Products → Engagements → Tests → Findings + +## Overview + +**Product Types** sit at the very top of DefectDojo’s product hierarchy. Product Types are distinct from the descending objects in the hierarchy—Products, Engagements, Tests, and Findings—because they are not technical scan targets, but rather serve primarily as organizational abstractions that compartmentalize your security efforts according to: +- Business domain +- Development team +- Security team +- Software applications +- Overarching product family +- Customer or subsidiary +- Reporting structure +- etc. + +The theme of the above examples exemplifies the essential utility of Product Types: they should generally represent stable, long-lived boundaries within your security program. + +## Product Type Data and Structure + +As Product Types are not scanned directly, the only mandatory field required to create them is a name. Beyond that, they act as containers for Products and their descending Engagements, Tests, and Findings. + +When creating a Product Type, consider how their structure will inform your reporting. Do you primarily need Product Types to represent the teams working on the projects (Products) that Product Types will contain? Or would Product Types better represent overarching projects that contain different iterations of the projects (Products) within it? + +If you have a single Product Type that contains all of the relevant information for a given business domain or development team, having that represented as a Product Type will facilitate smoother reporting, rather than having to pull together a report from various Products and Product Types. + +If a particular software project has many distinct deployments or versions, it may be worth creating a single Product Type which covers the scope of the entire project and having each version exist as individual Products. In some workflows, Product Types may also be used to separate software lifecycle stages: one Product Type for “In Development,” one Product Type for “In Production,” etc. +​ +Product Types can be used to determine access to subsidiaries, acquired companies, or other regulated business units for RBAC purposes. In complex businesses, where there are a lot of unique projects with different access rules, Product Types are particularly relevant. + +Ultimately, the decision of how to use Product Types and Products depends on how you best wish to reflect your unique organizational structure and the needs of your security team. + +Below are some example structures to inform how you designate your objects as either Product Types or Products. + +- **Product Type**: Payments Division + - Product: Payments API - Production + - Product: Payments API - Staging + - Product: Billing Worker + +- **Product Type**: Software Product A + - Product: Web Portal + - Product: Mobile Backend + +Additionally, the following is an illustrative guide as to whether a something is better represented by a Product Type or an Product: + +| Product Types | Products | +|--------------|--------| +| Business units | Individual applications | +| Departments | Deployments/environments | +| Security ownership domains | Infrastructure components | +| Product families | Specific microservices | +| Portfolio-level reporting | Scan targets | +| Customers | Specific software versions | + +As noted, your structure may differ depending on your unique security needs. + +## Accessing Product Types + +Product Types are accessible via the sidebar. The submenu also provides the option to create new Product Types. + +![image](images/PT_ss2.png) + +### Product Type View + +A Product Type’s view contains a variety of tables and charts to interpret its status at a glance. This includes: +- **Description** +- **Key/Critical Checkbox** + - Checking Critical or Key is used solely for filtering purposes +- **List of Products within the Product Type** +- **Authorized Users** (DefectDojo Users) + +## Working with Product Types + +### Create Product Types + +There are two ways to create Product Types: + +- From the **Add Product Type** option in the side menu +- From the **Add Product Type** button at the top of the All Product Type list + +### Edit Product Types + +Product Types can be edited by clicking **Edit** from within the dropdown menu at the top right of the Description table in the Product Type’s view. The same menu can also be accessed by clicking the ⋮ kebab menu to the left of the Product Type in the All Product Type list. + +All ensuing fields that can be edited are also available when the Product Type is being created. + +### Delete Product Types + +Deleting a Product Type can be performed by selecting **Delete Product Type** from the Product Type’s settings. + +Because Product Types sit at the top of the hierarchy, deleting them removes all downstream security history, relationships, and child objects, such as: +- Any Products, Engagements, and Tests contained within the Product Type +- All associated security history, including Findings and integrations +- Any linked Jira Epics +- All notes and file uploads associated with the Products, Engagements, and Tests within that Product Type + +Deleting a Product Type can’t be undone. If you would like to “decommission” a Product Type without deleting underlying data (for example, preserving legacy software testing records for audit purposes), you can change the Product Type’s name or add a Tag to indicate that it is in a deprecated state. + +## Product Types vs. Metadata + +Product Types are intended to represent structural ownership or reporting boundaries, rather than lightweight classifications. Attributes such as deployment status, internal labels, or temporary workflow states may be better represented through tags or metadata rather than separate Product Types. + +## Product Type Boundaries + +Product Types establish both reporting and access boundaries within DefectDojo. Because integrations, RBAC permissions, ownership, metrics, and deduplication models frequently inherit Product Types’ structure, designing clear boundaries early helps avoid hierarchy sprawl and reporting fragmentation later. + +### Findings and Automation + +Although integrations are typically configured on lower-level objects such as Product Types, Engagements, or Findings, Product Types still define the ownership, reporting, and access boundaries within which those integrations operate. + +Permissions cascade downward, meaning that access to a Product Type automatically grants access to all objects within that Product Type (e.g., Product Types, Engagements, Tests, and Findings). + +The DefectDojo RBAC model can be used to gate human user access, but can also restrict API tokens’ access to particular Product Types. + +For more information on user roles, see our [Permissions](/admin/user_management/os__authorized_users/) article. + +### Ownership + +As top-level objects, Product Types also imply ownership over the child objects within them. SLA tracking, remediation workflows, ticket routing, and general governance all flow more smoothly when Product Types have been set up to accurately reflect the individuals accountable for them. + +### Metrics/Reporting + +Metrics dashboards, tiles and views can be filtered per Product Type, making them a critical component in how your security data is calculated, visualized, and ultimately exported. + +For reporting purposes, it is generally easier to combine multiple Product Types into a single document than it is to subdivide a single Product Type into separate documents. Therefore, we recommend setting up Product Types at as granular a level as makes sense for your team’s reports. For example, there is no need to represent a large business division as a Product Type if you’re primarily going to be reporting to individual departments within that division. + +Effectively structuring your Product Types to reflect your reporting needs is critical to accurately assessing your security posture. For more information on Metrics, click [here](/metrics_reports/dashboards/introduction_dashboard/). + +### Deduplication + +Deduplication in DefectDojo occurs at the Product level, and is not affected by the parent Product Type. diff --git a/docs/content/asset_modelling/engagements_tests/PRO__assets.md b/docs/content/asset_modelling/engagements_tests/PRO__assets.md index 131561b68ee..e8d171b69f4 100644 --- a/docs/content/asset_modelling/engagements_tests/PRO__assets.md +++ b/docs/content/asset_modelling/engagements_tests/PRO__assets.md @@ -93,7 +93,7 @@ Asset views contain a variety of tables and charts to interpret an Asset’s sta - **All Engagements** - A list of Engagements contained within the Asset. -## Asset Lifecycle +## Working with Assets ### Create Assets diff --git a/docs/content/asset_modelling/engagements_tests/PRO__engagements.md b/docs/content/asset_modelling/engagements_tests/PRO__engagements.md index 06973a2e490..fee73acb9f8 100644 --- a/docs/content/asset_modelling/engagements_tests/PRO__engagements.md +++ b/docs/content/asset_modelling/engagements_tests/PRO__engagements.md @@ -66,7 +66,7 @@ Alternatively, Engagements within an Asset can be accessed in the window at the Engagements sit below Assets and above Tests in the object hierarchy. As such, access to an Asset automatically grants access to all Engagements within that Asset. Engagements do not have independent access control lists. -## Engagement Lifecycle +## Working with Engagements ### Create Engagements diff --git a/docs/content/asset_modelling/engagements_tests/PRO__organizations.md b/docs/content/asset_modelling/engagements_tests/PRO__organizations.md new file mode 100644 index 00000000000..617b61aca12 --- /dev/null +++ b/docs/content/asset_modelling/engagements_tests/PRO__organizations.md @@ -0,0 +1,139 @@ +--- +title: "Organizations" +description: "Understanding Organizations in DefectDojo Pro" +audience: pro +weight: 1 +--- +**ORGANIZATIONS** → Assets → Engagements → Tests → Findings + +## Overview + +**Organizations** sit at the very top of DefectDojo’s product hierarchy. Organizations are distinct from the descending objects in the hierarchy—Assets, Engagements, Tests, and Findings—because they are not technical scan targets, but rather serve primarily as organizational abstractions that compartmentalize your security efforts according to: +- Business domain +- Development team +- Security team +- Software applications +- Overarching product family +- Customer or subsidiary +- Reporting structure +- etc. + +The theme of the above examples exemplifies the essential utility of Organizations: they should generally represent stable, long-lived boundaries within your security program. + +## Organization Data and Structure + +As Organizations are not scanned directly, the only mandatory field required to create them is a name. Beyond that, they act as containers for Assets and their descending Engagements, Tests, and Findings. + +When creating an Organization, consider how their structure will inform your reporting. Do you primarily need Organizations to represent the teams working on the projects (Assets) that Organizations will contain? Or would Organizations better represent overarching projects that contain different iterations of the projects (Assets) within it? + +If you have a single Organization that contains all of the relevant information for a given business domain or development team, having that represented as an Organization will facilitate smoother reporting, rather than having to pull together a report from various Assets and Organizations. + +If a particular software project has many distinct deployments or versions, it may be worth creating a single Organization which covers the scope of the entire project and having each version exist as individual Assets. In some workflows, Organizations may also be used to separate software lifecycle stages: one Organization for “In Development,” one Organization for “In Production,” etc. +​ +Organizations can be used to determine access to subsidiaries, acquired companies, or other regulated business units for RBAC purposes. In complex businesses, where there are a lot of unique projects with different access rules, Organizations are particularly relevant. + +Ultimately, the decision of how to use Organizations and Assets depends on how you best wish to reflect your unique organizational structure and the needs of your security team. + +Below are some example structures to inform how you designate your objects as either Organizations or Assets. + +- **Organization**: Payments Division + - Asset: Payments API - Production + - Asset: Payments API - Staging + - Asset: Billing Worker + +- **Organization**: Software Product A + - Asset: Web Portal + - Asset: Mobile Backend + +Additionally, the following is an illustrative guide as to whether a something is better represented by an Organization or an Asset: + +| Organizations | Assets | +|--------------|--------| +| Business units | Individual applications | +| Departments | Deployments/environments | +| Security ownership domains | Infrastructure components | +| Product families | Specific microservices | +| Portfolio-level reporting | Scan targets | +| Customers | Specific software versions | + +As noted, your structure may differ depending on your unique security needs. + +## Accessing Organizations + +Organizations are accessible via the sidebar. The submenu provides access to All Organizations as well as the option to create a new Organization. + +![image](images/org_ss1.png) + +## Organization View + +An Organization’s view contains a variety of tables and charts to interpret its status at a glance. This includes: + +- **Description** +- **Commerce** + - Whether the Organization has been determined to be Critical or Key + - Checking Critical or Key is used solely for filtering purposes +- **Assigned Members** (DefectDojo Users) +- **Assigned User Groups** + - User groups that have been assigned to the Organization for permission control. More information about user groups can be found [here](/admin/user_management/create_user_group/). +- **List of Assets within the Organization** + +## Working with Organizations + +### Create Organizations + +There are two ways to create Organizations: + +- From the **New Organization** option in the side menu +- From the **New Organization** button at the top of the All Organizations list + +### Edit Organizations + +Organizations can be edited by clicking **Edit Organization** from within the gear menu at the top right of the Organization’s view. The same menu can also be accessed by clicking the ⋮ kebab menu to the left of the Organization in the All Organization view. + +All ensuing fields that can be edited are also available when the Organization is being created. + +### Delete Organizations + +Deleting an Organization can be performed by selecting **Delete Organization** from the Organization’s settings. + +Because Organizations sit at the top of the hierarchy, deleting them removes all downstream security history, relationships, and child objects, such as: +- Any Assets, Engagements, and Tests contained within the Organization +- All associated security history, including Findings and integrations +- Any linked Jira Epics +- All notes and file uploads associated with the Assets, Engagements, and Tests within that Organization + +Deleting an Organization can’t be undone. If you would like to “decommission” an organization without deleting underlying data (for example, preserving legacy software testing records for audit purposes), you can change the Organization’s name or add a Tag to indicate that it is in a deprecated state. + +## Organiations vs. Metadata + +Organizations are intended to represent structural ownership or reporting boundaries, rather than lightweight classifications. Attributes such as deployment status, internal labels, or temporary workflow states may be better represented through tags or metadata rather than separate Organizations. + +## Organization Boundaries + +Organizations establish both reporting and access boundaries within DefectDojo. Because integrations, RBAC permissions, ownership, metrics, and deduplication models frequently inherit Organizations’ structure, designing clear boundaries early helps avoid hierarchy sprawl and reporting fragmentation later. + +### Findings and Automation + +Although integrations are typically configured on lower-level objects such as Assets, Engagements, or Findings, Organizations still define the ownership, reporting, and access boundaries within which those integrations operate. + +Permissions cascade downward, meaning that access to an Organization automatically grants access to all objects within that Organization (e.g., Assets, Engagements, Tests, and Findings). + +The DefectDojo RBAC model can be used to gate human user access, but can also restrict API tokens’ access to particular Organizations. + +For more information on user roles, see our [Introduction To Permission Types](/admin/user_management/set_user_permissions/#introduction-to-permission-types) article. + +### Ownership + +As top-level objects, Organizations also imply ownership over the child objects within them. SLA tracking, remediation workflows, ticket routing, and general governance all flow more smoothly when Organizations have been set up to accurately reflect the individuals accountable for them. + +### Metrics/Reporting + +Metrics dashboards, tiles and views can be filtered per Organization, making them a critical component in how your security data is calculated, visualized, and ultimately exported. + +For reporting purposes, it is generally easier to combine multiple Organizations into a single document than it is to subdivide a single Organization into separate documents. Therefore, we recommend setting up Organizations at as granular a level as makes sense for your team’s reports. For example, there is no need to represent a large business division as an Organization if you’re primarily going to be reporting to individual departments within that division. + +Effectively structuring your Organizations to reflect your reporting needs is critical to accurately assessing your security posture. For more information on Metrics, click [here](/metrics_reports/pro_metrics/pro__overview/). + +### Deduplication + +Deduplication in DefectDojo occurs at the Asset level, and is not affected by the parent Organization. \ No newline at end of file diff --git a/docs/content/asset_modelling/engagements_tests/PRO__tests.md b/docs/content/asset_modelling/engagements_tests/PRO__tests.md index 9eb3c550729..baecc9fedbd 100644 --- a/docs/content/asset_modelling/engagements_tests/PRO__tests.md +++ b/docs/content/asset_modelling/engagements_tests/PRO__tests.md @@ -104,7 +104,7 @@ Tests can be accessed from various sections of the DefectDojo UI. ![image](images/tests_ss16.png) -## Test Lifecycle +## Working with Tests ### Create Tests diff --git a/docs/content/metrics_reports/pro_metrics/PRO__overview.md b/docs/content/metrics_reports/pro_metrics/PRO__overview.md index c161154f3e3..2e21dadc11d 100644 --- a/docs/content/metrics_reports/pro_metrics/PRO__overview.md +++ b/docs/content/metrics_reports/pro_metrics/PRO__overview.md @@ -6,11 +6,11 @@ weight: 2 --- The DefectDojo Pro UI has various Metrics dashboards to help visualize your current security posture. Each dashboard allows stakeholders at different levels of the organization to make informed decisions without needing to interpret raw data or navigate individual Findings. These dashboards include: -* [Executive Insights](#executive-insights) -* [Priority Insights](#priority-insights) -* [Program Insights](#program-insights) -* [Remediation Insights](#remediation-insights) -* [Tool Insights](#tool-insights) +* [Executive Insights](/metrics_reports/pro_metrics/pro__executive_insights/#main-content) +* [Priority Insights](/metrics_reports/pro_metrics/pro__priority_insights/#main-content) +* [Program Insights](/metrics_reports/pro_metrics/pro__program_insights/#main-content) +* [Remediation Insights](/metrics_reports/pro_metrics/pro__remediation_insights/#main-content) +* [Tool Insights](/metrics_reports/pro_metrics/pro__tool_insights/#main-content) ![Metrics overview](images/metrics_image1.png) diff --git a/docs/content/releases/os_upgrading/3.0.100.md b/docs/content/releases/os_upgrading/3.0.100.md new file mode 100644 index 00000000000..8c2a3973513 --- /dev/null +++ b/docs/content/releases/os_upgrading/3.0.100.md @@ -0,0 +1,21 @@ +--- +title: 'Upgrading to DefectDojo Version 3.0.100' +toc_hide: true +weight: -20260622 +description: Xygeni parser keeps repeated SAST/Secrets occurrences in the same file as distinct findings. +--- + +## Xygeni parser: repeated SAST/Secrets occurrences are now distinct findings + +The Xygeni parser previously deduplicated away legitimate findings when the same secret value or code pattern appeared more than once in a single file, so only the first occurrence survived an import. + +Xygeni reuses one `uniqueHash` across every occurrence of the same value in a file (it hashes the value, not the location) while giving each occurrence a distinct `issueId` that encodes the file path and line. The SAST and Secrets scan types deduplicate on `unique_id_from_tool`, which was set to `uniqueHash`, so occurrences after the first were treated as duplicates and hidden. + +Starting in 3.0.100, for SAST and Secrets findings the parser keys `unique_id_from_tool` on the per-occurrence `issueId` (falling back to `uniqueHash` when `issueId` is absent) and keeps `uniqueHash` as `vuln_id_from_tool`. Each occurrence is now its own finding, and `vuln_id_from_tool` still groups occurrences of the same value. SCA findings are unchanged: there `uniqueHash` is unique per finding while `issueId` collides across packages, so `uniqueHash` remains the correct dedup key. + +### Required actions + +- **No action required for new imports.** Repeated occurrences that were previously collapsed now appear as separate findings. +- **Reimport behavior:** on the first reimport of an existing Xygeni SAST or Secrets test after upgrading, the previously-imported findings carry the old `uniqueHash`-based `unique_id_from_tool` and will not match the new `issueId`-based ids. Those findings are closed as no longer present and a fresh set is created with the corrected ids. This is a one-time effect; subsequent reimports match normally. SCA tests are not affected. + +For more information, check the [Release Notes](https://github.com/DefectDojo/django-DefectDojo/releases/tag/3.0.100). diff --git a/docs/content/releases/pro/changelog.md b/docs/content/releases/pro/changelog.md index 1bba4e3ad50..38301b3256d 100644 --- a/docs/content/releases/pro/changelog.md +++ b/docs/content/releases/pro/changelog.md @@ -12,6 +12,34 @@ For Open Source release notes, please see the [Releases page on GitHub](https:// ## June 2026: v3.0 +### June 22, 2026: v3.0.100 + +* **(Pro UI)** Added native Excel (`.xlsx`) export for Findings, Engagements, and Users. +* **(Pro UI)** Bulk "Add to Existing Finding Group" no longer fails with an "Invalid pk 'None'" error. +* **(Classic UI)** Fixed the disclaimer border rendering in the new UI. +* **(Findings)** Blank component values are now normalized to NULL for consistent matching and filtering. +* **(Reports)** Added DefectDojo Pro Report Builder guides (UI, API, and LLM). +* **(Tools)** Added a PICUS Breach and Attack Simulation CSV parser. +* **(Tools)** Added a Govulncheck Scanner V2 parser. +* **(Tools)** cargo-audit parser now parses CVSS vectors and derives severity from them. + +### June 18, 2026: v3.0.2 + +* **(SSO)** SAML now keeps the Pro group-mapping backend as the active authentication backend. +* **(API)** Restored `members` and `authorization_groups` fields on the Asset and Organization serializers. +* **(API)** Registered the `asset_*` / `organization_*` RBAC alias routes. +* **(API)** Restored RBAC fields on `/api/v2/user_profile/`. + +### June 17, 2026: v3.0.1 + +* **(Pro UI)** Calendar sidebar now honors the "Enable Calendar" system setting. +* **(Pro UI)** Keyword search no longer blanks the Components table. +* **(Reports)** Added a Finding quick report via the reporting engine. +* **(SSO)** Azure AD configuration now requires and defaults the Application ID URI. +* **(Locations)** Single-location filter now resolves correctly against the Location model. +* **(API)** Fixed a 500 error when deleting an Organization/Asset that still has deprecated endpoints; the new-UI banner now points at 3.3.0. +* **(API)** Refactored and enhanced API permissions. + ### June 15, 2026: v3.0.0 * **(Locations)** Locations are now enabled by default, superseding the legacy Endpoint model. The legacy Endpoint API stays read-compatible and your data is preserved. See [Locations enabled by default](/releases/os_upgrading/3.0/#locations-enabled-by-default). diff --git a/docs/content/supported_tools/parsers/file/xygeni.md b/docs/content/supported_tools/parsers/file/xygeni.md index 1b8ec32f116..54cf2c7f50e 100644 --- a/docs/content/supported_tools/parsers/file/xygeni.md +++ b/docs/content/supported_tools/parsers/file/xygeni.md @@ -64,16 +64,24 @@ Sample Xygeni JSON reports can be found ### Deduplication -Every finding carries `unique_id_from_tool` (set from Xygeni's vendor-stable -`uniqueHash`) and `vuln_id_from_tool` (set from `issueId`). The deduplication -algorithm is configured per scan type: +Every finding carries both `unique_id_from_tool` and `vuln_id_from_tool`, and +the deduplication algorithm is configured per scan type: -| Scan type | Algorithm | Hash-code fields (fallback) | -| -------------------- | ---------------------------------- | -------------------------------------------------------------- | -| Xygeni SAST Scan | `unique_id_from_tool` | n/a | -| Xygeni SCA Scan | `unique_id_from_tool_or_hash_code` | `vulnerability_ids`, `component_name`, `component_version` | -| Xygeni Secrets Scan | `unique_id_from_tool` | n/a | +| Scan type | Algorithm | `unique_id_from_tool` | `vuln_id_from_tool` | Hash-code fields (fallback) | +| -------------------- | ---------------------------------- | --------------------- | ------------------- | --------------------------------------------------------- | +| Xygeni SAST Scan | `unique_id_from_tool` | `issueId` | `uniqueHash` | n/a | +| Xygeni SCA Scan | `unique_id_from_tool_or_hash_code` | `uniqueHash` | `issueId` | `vulnerability_ids`, `component_name`, `component_version` | +| Xygeni Secrets Scan | `unique_id_from_tool` | `issueId` | `uniqueHash` | n/a | -For SCA the hash-code fallback enables cross-tool deduplication: the same -CVE on the same package@version reported by Xygeni and another SCA scanner -(Snyk, Trivy, etc.) collapse into a single Finding. +For SAST and Secrets the dedup key is the per-occurrence `issueId` (which +encodes the file path and line). The same secret value or code pattern can +appear several times in one file; Xygeni reuses a single `uniqueHash` across +those occurrences, so keying dedup on `uniqueHash` would collapse them into one +Finding and underreport the occurrences. Keying on `issueId` keeps each +occurrence as its own Finding, while `uniqueHash` is retained as the +`vuln_id_from_tool` that groups occurrences of the same value. + +For SCA the dedup key stays `uniqueHash` (it encodes CVE + package + version, +unique per finding) and the hash-code fallback enables cross-tool +deduplication: the same CVE on the same package@version reported by Xygeni and +another SCA scanner (Snyk, Trivy, etc.) collapse into a single Finding. diff --git a/dojo/authorization/query_registrations.py b/dojo/authorization/query_registrations.py index e5a2941b487..f27aba3950b 100644 --- a/dojo/authorization/query_registrations.py +++ b/dojo/authorization/query_registrations.py @@ -362,6 +362,13 @@ def _get_authorized_endpoints(permission, user=None): register_auth_filter("endpoint.get_authorized_endpoints", _get_authorized_endpoints) +def _get_authorized_endpoints_for_queryset(permission, queryset, user=None): + return _filter_by_authorized_products(queryset, "product", permission, user=user) + + +register_auth_filter("endpoint.get_authorized_endpoints_for_queryset", _get_authorized_endpoints_for_queryset) + + def _get_authorized_endpoint_status(permission, user=None): return _filter_by_authorized_products( Endpoint_Status.objects.all(), "endpoint__product", permission, user=user, @@ -371,6 +378,13 @@ def _get_authorized_endpoint_status(permission, user=None): register_auth_filter("endpoint.get_authorized_endpoint_status", _get_authorized_endpoint_status) +def _get_authorized_endpoint_status_for_queryset(permission, queryset, user=None): + return _filter_by_authorized_products(queryset, "endpoint__product", permission, user=user) + + +register_auth_filter("endpoint.get_authorized_endpoint_status_for_queryset", _get_authorized_endpoint_status_for_queryset) + + # --------------------------------------------------------------------------- # Findings / Vulnerability_Ids # --------------------------------------------------------------------------- @@ -387,6 +401,7 @@ def _get_authorized_findings(permission, queryset=None, user=None): register_auth_filter("finding.get_authorized_findings", _get_authorized_findings) +register_auth_filter("finding.get_authorized_findings_for_queryset", _get_authorized_findings) def _get_authorized_vulnerability_ids(permission, queryset=None, user=None): @@ -400,6 +415,7 @@ def _get_authorized_vulnerability_ids(permission, queryset=None, user=None): register_auth_filter("finding.get_authorized_vulnerability_ids", _get_authorized_vulnerability_ids) +register_auth_filter("finding.get_authorized_vulnerability_ids_for_queryset", _get_authorized_vulnerability_ids) # --------------------------------------------------------------------------- @@ -413,7 +429,14 @@ def _get_authorized_users(permission, user=None): return Dojo_User.objects.none() if _is_unrestricted(user, permission_to_action(permission)) or user.is_staff: return Dojo_User.objects.all().order_by("first_name", "last_name") - return Dojo_User.objects.filter(pk=user.pk) + # OS: collaborators — users sharing the caller's authorized products / + # product types (via authorized_users), plus superusers. Mirrors 2.58.4, + # which returned co-members of the caller's authorized products/types. + return Dojo_User.objects.filter( + Q(authorized_products__id__in=_authorized_product_ids(user)) + | Q(authorized_product_types__id__in=_authorized_product_type_ids(user)) + | Q(is_superuser=True), + ).distinct().order_by("first_name", "last_name") register_auth_filter("user.get_authorized_users", _get_authorized_users) @@ -427,7 +450,14 @@ def _get_authorized_users_for_product_type(users, product_type, permission): return users.none() if _is_unrestricted(user, permission_to_action(permission)) or user.is_staff: return users - return users.none() + if product_type is None: + return users.none() + # OS: users authorized on this product type via authorized_users, plus + # superusers (2.58.4 always surfaced is_superuser users as candidates). + return users.filter( + Q(id__in=product_type.authorized_users.values("id")) + | Q(is_superuser=True), + ) register_auth_filter("user.get_authorized_users_for_product_type", _get_authorized_users_for_product_type) @@ -441,7 +471,16 @@ def _get_authorized_users_for_product_and_product_type(users, product, permissio return users.none() if _is_unrestricted(user, permission_to_action(permission)) or user.is_staff: return users - return users.none() + if product is None: + return users.none() + # OS: users authorized on this product via authorized_users (directly on + # the product or via its product type), plus superusers (2.58.4 always + # surfaced is_superuser users as candidates). + return users.filter( + Q(id__in=product.authorized_users.values("id")) + | Q(id__in=product.prod_type.authorized_users.values("id")) + | Q(is_superuser=True), + ) register_auth_filter("user.get_authorized_users_for_product_and_product_type", _get_authorized_users_for_product_and_product_type) diff --git a/dojo/db_migrations/0270_finding_visibility_perf_indexes.py b/dojo/db_migrations/0270_finding_visibility_perf_indexes.py new file mode 100644 index 00000000000..1cd4291c4b8 --- /dev/null +++ b/dojo/db_migrations/0270_finding_visibility_perf_indexes.py @@ -0,0 +1,31 @@ +from django.contrib.postgres.operations import AddIndexConcurrently +from django.db import migrations, models + + +class Migration(migrations.Migration): + # CREATE INDEX CONCURRENTLY cannot run inside a transaction block, and avoids + # an ACCESS EXCLUSIVE lock on the (large) dojo_finding table. + atomic = False + + dependencies = [ + ("dojo", "0269_normalize_blank_finding_components"), + ] + + operations = [ + AddIndexConcurrently( + model_name="finding", + index=models.Index( + fields=["severity", "-numerical_severity"], + name="idx_finding_sev_active", + condition=models.Q(active=True), + ), + ), + AddIndexConcurrently( + model_name="finding", + index=models.Index( + fields=["-date"], + name="idx_finding_riskaccepted_date", + condition=models.Q(risk_accepted=True), + ), + ), + ] diff --git a/dojo/finding/models.py b/dojo/finding/models.py index 36f3554137d..3e7789a7b98 100644 --- a/dojo/finding/models.py +++ b/dojo/finding/models.py @@ -12,6 +12,7 @@ from django.conf import settings from django.core.validators import MaxValueValidator, MinValueValidator from django.db import models +from django.db.models import Q from django.urls import reverse from django.utils import timezone from django.utils.functional import cached_property @@ -476,6 +477,16 @@ class Meta: models.Index(fields=["known_exploited"]), models.Index(fields=["ransomware_used"]), models.Index(fields=["kev_date"]), + models.Index( + fields=["severity", "-numerical_severity"], + name="idx_finding_sev_active", + condition=Q(active=True), + ), + models.Index( + fields=["-date"], + name="idx_finding_riskaccepted_date", + condition=Q(risk_accepted=True), + ), ] def __init__(self, *args, **kwargs): diff --git a/dojo/templates/dojo/metrics.html b/dojo/templates/dojo/metrics.html index 2c7ab40fbf6..b35c7bb24ee 100644 --- a/dojo/templates/dojo/metrics.html +++ b/dojo/templates/dojo/metrics.html @@ -360,7 +360,7 @@

{% trans "Risk accepted bug count by month" %} {% endif %} - {% if not critical_prods %} + {% if not critical_prods and name != labels.ASSET_METRICS_CRITICAL_LABEL %}
@@ -943,7 +943,7 @@

{% trans "Closed in Period" %}

medium1.push([{{ week.epoch }}, {{ week.medium }}]); low1.push([{{ week.epoch }}, {{ week.low }}]); {% endfor %} - {% if not critical_prods %} + {% if not critical_prods and name != labels.ASSET_METRICS_CRITICAL_LABEL %} opened_per_week_2(critical, high, medium, low); accepted_per_week_2(critical1, high1, medium1, low1); {% endif %} diff --git a/dojo/templates_classic/dojo/metrics.html b/dojo/templates_classic/dojo/metrics.html index 777467e0e59..881abbef4cb 100644 --- a/dojo/templates_classic/dojo/metrics.html +++ b/dojo/templates_classic/dojo/metrics.html @@ -368,7 +368,7 @@

{% trans "Risk accepted bug count by month" %}

{% endif %} - {% if not critical_prods %} + {% if not critical_prods and name != labels.ASSET_METRICS_CRITICAL_LABEL %}
@@ -958,7 +958,7 @@

{% trans "Closed in Period" %}

medium1.push([{{ week.epoch }}, {{ week.medium }}]); low1.push([{{ week.epoch }}, {{ week.low }}]); {% endfor %} - {% if not critical_prods %} + {% if not critical_prods and name != labels.ASSET_METRICS_CRITICAL_LABEL %} opened_per_week_2(critical, high, medium, low); accepted_per_week_2(critical1, high1, medium1, low1); {% endif %} diff --git a/dojo/tools/jfrog_xray_api_summary_artifact/parser.py b/dojo/tools/jfrog_xray_api_summary_artifact/parser.py index 2e8cba494a4..175af230e51 100644 --- a/dojo/tools/jfrog_xray_api_summary_artifact/parser.py +++ b/dojo/tools/jfrog_xray_api_summary_artifact/parser.py @@ -86,7 +86,10 @@ def get_item( with contextlib.suppress(CVSS3RHScoreDoesNotMatch, CVSS3RHMalformedError): cvssv3 = CVSS3.from_rh_vector(cvss_v3).clean_vector() - impact_paths = vulnerability.get("impact_path", []) + # JFrog returns impact_path in arbitrary order; sort so file_path, + # description, and unique_id stay stable across re-imports — otherwise + # one CVE flaps into multiple findings as the order changes between scans. + impact_paths = sorted(vulnerability.get("impact_path", [])) if len(impact_paths) > 0: impact_path = decode_impact_path(impact_paths[0]) @@ -112,6 +115,16 @@ def get_item( result.update(unique_id.encode()) unique_id_from_tool = result.hexdigest() + description = ( + impact_path.name + + ":" + + impact_path.version + + " -> " + + vulnerability["description"] + ) + if len(impact_paths) > 1: + description += "\n\n**Impact paths:**\n" + "\n".join(f"- {p}" for p in impact_paths) + finding = Finding( vuln_id_from_tool=vuln_id_from_tool, service=service, @@ -119,11 +132,7 @@ def get_item( cwe=cwe, cvssv3=cvssv3, severity=severity, - description=impact_path.name - + ":" - + impact_path.version - + " -> " - + vulnerability["description"], + description=description, test=test, file_path=impact_paths[0], component_name=artifact_name, diff --git a/dojo/tools/sarif/parser.py b/dojo/tools/sarif/parser.py index cd7107d5352..0ec6800d568 100644 --- a/dojo/tools/sarif/parser.py +++ b/dojo/tools/sarif/parser.py @@ -632,6 +632,9 @@ def get_fingerprints_hashes(values): key_method = key key_method_version = 0 value = values[key] + # some tools (e.g. BlackDuck) wrap the hash as {"value": ""} instead of a plain string + if isinstance(value, dict): + value = value.get("value", "") if fingerprints.get(key_method): if fingerprints[key_method]["version"] < key_method_version: fingerprints[key_method] = { diff --git a/dojo/tools/xygeni/sast.py b/dojo/tools/xygeni/sast.py index 1e7782e5100..62b0048cb1b 100644 --- a/dojo/tools/xygeni/sast.py +++ b/dojo/tools/xygeni/sast.py @@ -35,8 +35,13 @@ def _build_finding(vuln, test): cwe=parse_cwe(cwes=vuln.get("cwes"), cwe=vuln.get("cwe"), tags=vuln.get("tags")), static_finding=True, dynamic_finding=False, - unique_id_from_tool=vuln.get("uniqueHash"), - vuln_id_from_tool=vuln.get("issueId"), + # One detector can flag the same code pattern at several locations. Xygeni reuses a + # single ``uniqueHash`` across those occurrences but gives each a distinct ``issueId`` + # (which encodes filepath + line). Dedup is keyed on ``unique_id_from_tool``, so use + # the per-occurrence ``issueId`` to keep each occurrence as its own Finding; + # ``uniqueHash`` groups them as the vuln id. + unique_id_from_tool=vuln.get("issueId") or vuln.get("uniqueHash"), + vuln_id_from_tool=vuln.get("uniqueHash"), ) _apply_code_flow_fields(finding, vuln.get("codeFlows") or []) diff --git a/dojo/tools/xygeni/secrets.py b/dojo/tools/xygeni/secrets.py index ac5b6584b71..c35a5aecb67 100644 --- a/dojo/tools/xygeni/secrets.py +++ b/dojo/tools/xygeni/secrets.py @@ -44,6 +44,11 @@ def _build_finding(secret, test): mitigation=f"Rotate this {secret_type} secret immediately and remove it from version-control history.", static_finding=True, dynamic_finding=False, - unique_id_from_tool=secret.get("uniqueHash"), - vuln_id_from_tool=secret.get("issueId"), + # The same secret value can appear several times in one file. Xygeni assigns every + # occurrence the same ``uniqueHash`` (it hashes the secret value, not the location) + # but a distinct ``issueId`` (which encodes filepath + line). Dedup is keyed on + # ``unique_id_from_tool``, so use the per-occurrence ``issueId`` to keep each + # occurrence as its own Finding; ``uniqueHash`` groups them as the vuln id. + unique_id_from_tool=secret.get("issueId") or secret.get("uniqueHash"), + vuln_id_from_tool=secret.get("uniqueHash"), ) diff --git a/tests/check_various_pages.py b/tests/check_various_pages.py index aa1188253cc..d03ec3e0cd5 100644 --- a/tests/check_various_pages.py +++ b/tests/check_various_pages.py @@ -10,6 +10,10 @@ def test_user_status(self): driver = self.driver driver.get(self.base_url + "user") + def test_critical_asset_metrics_status(self): + driver = self.driver + driver.get(self.base_url + "critical_asset_metrics") + def test_calendar_status(self): driver = self.driver driver.get(self.base_url + "calendar") @@ -42,6 +46,7 @@ def suite(): suite = unittest.TestSuite() suite.addTest(BaseTestCase("test_login")) suite.addTest(VariousPagesTest("test_user_status")) + suite.addTest(VariousPagesTest("test_critical_asset_metrics_status")) suite.addTest(VariousPagesTest("test_calendar_status")) suite.addTest(VariousPagesTest("test_finding_group_open_status")) suite.addTest(VariousPagesTest("test_finding_group_all_status")) diff --git a/unittests/scans/sarif/blackduck_nested_fingerprints.sarif b/unittests/scans/sarif/blackduck_nested_fingerprints.sarif new file mode 100644 index 00000000000..d02abf2b8fb --- /dev/null +++ b/unittests/scans/sarif/blackduck_nested_fingerprints.sarif @@ -0,0 +1,292 @@ +{ + "$schema": "https://json.schemastore.org/sarif-2.1.0.json", + "version": "2.1.0", + "runs": [ + { + "tool": { + "driver": { + "name": "Example-Application", + "rules": [ + { + "id": "BDSA-2020-1001 (CVE-2020-11022)", + "name": "BDSA-2020-1001 (CVE-2020-11022)", + "shortDescription": { + "text": "BDSA-2020-1001 (CVE-2020-11022)" + }, + "fullDescription": { + "text": "jQuery is vulnerable to XSS via the HTML passed to certain jQuery methods." + }, + "helpUri": "https://nvd.nist.gov/vuln/detail/CVE-2020-11022", + "properties": { + "tags": ["BDSA", "CWE-79", "SBOM"], + "cwe": ["CWE-79"] + } + }, + { + "id": "BDSA-2021-1002 (CVE-2021-23337)", + "name": "BDSA-2021-1002 (CVE-2021-23337)", + "shortDescription": { + "text": "BDSA-2021-1002 (CVE-2021-23337)" + }, + "fullDescription": { + "text": "Lodash is vulnerable to command injection via the template function." + }, + "helpUri": "https://nvd.nist.gov/vuln/detail/CVE-2021-23337", + "properties": { + "tags": ["BDSA", "CWE-78", "SBOM"], + "cwe": ["CWE-78"] + } + }, + { + "id": "BDSA-2019-1003 (CVE-2019-10744)", + "name": "BDSA-2019-1003 (CVE-2019-10744)", + "shortDescription": { + "text": "BDSA-2019-1003 (CVE-2019-10744)" + }, + "fullDescription": { + "text": "Lodash is vulnerable to prototype pollution via the defaultsDeep function." + }, + "helpUri": "https://nvd.nist.gov/vuln/detail/CVE-2019-10744", + "properties": { + "tags": ["BDSA", "CWE-1321", "SBOM"], + "cwe": ["CWE-1321"] + } + }, + { + "id": "BDSA-2022-1004", + "name": "BDSA-2022-1004", + "shortDescription": { + "text": "BDSA-2022-1004" + }, + "fullDescription": { + "text": "Example-lib is affected by a denial of service vulnerability." + }, + "helpUri": "https://nvd.nist.gov/vuln/search", + "properties": { + "tags": ["BDSA", "CWE-400", "SBOM"], + "cwe": ["CWE-400"] + } + }, + { + "id": "BDSA-2023-1005 (CVE-2023-26115)", + "name": "BDSA-2023-1005 (CVE-2023-26115)", + "shortDescription": { + "text": "BDSA-2023-1005 (CVE-2023-26115)" + }, + "fullDescription": { + "text": "word-wrap is vulnerable to ReDoS via the wrap function." + }, + "helpUri": "https://nvd.nist.gov/vuln/detail/CVE-2023-26115", + "properties": { + "tags": ["BDSA", "CWE-1333", "SBOM"], + "cwe": ["CWE-1333"] + } + } + ] + } + }, + "results": [ + { + "ruleId": "BDSA-2020-1001 (CVE-2020-11022)", + "level": "warning", + "message": { + "text": "BDSA-2020-1001 (CVE-2020-11022) in example-frontend-lib 1.0.0" + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "uri": "npm/example-frontend-lib@1.0.0" + } + } + } + ], + "fixes": [ + { + "description": { + "text": "Upgrade/patch dependency: example-frontend-lib 1.0.0." + } + } + ], + "properties": { + "securityRisk": "MEDIUM", + "component": "example-frontend-lib", + "componentVersion": "1.0.0", + "componentOrigin": "npm", + "matchType": "exact", + "exploitAvailable": false, + "solutionAvailable": true, + "workaroundAvailable": false, + "remediationStatus": "New", + "reachable": "unknown" + }, + "fingerprints": { + "csvRowSha256": { + "value": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa" + } + } + }, + { + "ruleId": "BDSA-2021-1002 (CVE-2021-23337)", + "level": "error", + "message": { + "text": "BDSA-2021-1002 (CVE-2021-23337) in example-util-lib 4.17.20" + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "uri": "npm/example-util-lib@4.17.20" + } + } + } + ], + "fixes": [ + { + "description": { + "text": "Upgrade/patch dependency: example-util-lib 4.17.20." + } + } + ], + "properties": { + "securityRisk": "HIGH", + "component": "example-util-lib", + "componentVersion": "4.17.20", + "componentOrigin": "npm", + "matchType": "exact", + "exploitAvailable": true, + "solutionAvailable": true, + "workaroundAvailable": false, + "remediationStatus": "New", + "reachable": "unknown" + }, + "fingerprints": { + "csvRowSha256": { + "value": "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb" + } + } + }, + { + "ruleId": "BDSA-2019-1003 (CVE-2019-10744)", + "level": "error", + "message": { + "text": "BDSA-2019-1003 (CVE-2019-10744) in example-util-lib 4.17.20" + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "uri": "npm/example-util-lib@4.17.20" + } + } + } + ], + "fixes": [ + { + "description": { + "text": "Upgrade/patch dependency: example-util-lib 4.17.20." + } + } + ], + "properties": { + "securityRisk": "HIGH", + "component": "example-util-lib", + "componentVersion": "4.17.20", + "componentOrigin": "npm", + "matchType": "exact", + "exploitAvailable": false, + "solutionAvailable": true, + "workaroundAvailable": false, + "remediationStatus": "New", + "reachable": "unknown" + }, + "fingerprints": { + "csvRowSha256": { + "value": "cccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc" + } + } + }, + { + "ruleId": "BDSA-2022-1004", + "level": "note", + "message": { + "text": "BDSA-2022-1004 in example-common-lib 2.3.1" + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "uri": "maven/example-common-lib@2.3.1" + } + } + } + ], + "fixes": [ + { + "description": { + "text": "Upgrade/patch dependency: example-common-lib 2.3.1." + } + } + ], + "properties": { + "securityRisk": "LOW", + "component": "example-common-lib", + "componentVersion": "2.3.1", + "componentOrigin": "maven", + "matchType": "exact", + "exploitAvailable": false, + "solutionAvailable": false, + "workaroundAvailable": true, + "remediationStatus": "Ignored", + "reachable": "unknown" + }, + "fingerprints": { + "csvRowSha256": { + "value": "dddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddd" + } + } + }, + { + "ruleId": "BDSA-2023-1005 (CVE-2023-26115)", + "level": "error", + "message": { + "text": "BDSA-2023-1005 (CVE-2023-26115) in example-text-lib 1.2.5" + }, + "locations": [ + { + "physicalLocation": { + "artifactLocation": { + "uri": "npm/example-text-lib@1.2.5" + } + } + } + ], + "fixes": [ + { + "description": { + "text": "Upgrade/patch dependency: example-text-lib 1.2.5." + } + } + ], + "properties": { + "securityRisk": "CRITICAL", + "component": "example-text-lib", + "componentVersion": "1.2.5", + "componentOrigin": "npm", + "matchType": "exact", + "exploitAvailable": true, + "solutionAvailable": true, + "workaroundAvailable": false, + "remediationStatus": "New", + "reachable": "unknown" + }, + "fingerprints": { + "csvRowSha256": { + "value": "eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee" + } + } + } + ] + } + ] +} diff --git a/unittests/test_auth_filter_registry.py b/unittests/test_auth_filter_registry.py new file mode 100644 index 00000000000..37d64d5137f --- /dev/null +++ b/unittests/test_auth_filter_registry.py @@ -0,0 +1,37 @@ +""" +Registry-completeness guard for the auth-filter layer. + +Every key looked up via ``get_auth_filter("...")`` must be registered via +``register_auth_filter("...")``. An unregistered key makes the per-app +wrapper fall back to its default (passthrough / None), bypassing membership +scoping, so this guards against a key being looked up but never wired up. +""" + +import re +from pathlib import Path + +import dojo +import dojo.authorization.query_registrations # noqa: F401 -- ensure OS registrations run +from dojo.authorization.query_filters import get_auth_filter + +from .dojo_test_case import DojoTestCase + +_KEY_CALL = re.compile(r'get_auth_filter\(\s*"([^"]+)"') + + +def _looked_up_keys() -> set[str]: + root = Path(dojo.__file__).resolve().parent + keys: set[str] = set() + for path in root.rglob("*.py"): + keys |= set(_KEY_CALL.findall(path.read_text(encoding="utf-8"))) + return keys + + +class TestAuthFilterRegistryComplete(DojoTestCase): + + def test_all_looked_up_keys_are_registered(self): + missing = sorted(key for key in _looked_up_keys() if get_auth_filter(key) is None) + self.assertEqual( + missing, [], + msg=f"auth-filter keys looked up but never registered (silent fallback): {missing}", + ) diff --git a/unittests/test_authorization_queryset_coverage.py b/unittests/test_authorization_queryset_coverage.py new file mode 100644 index 00000000000..a894ec0d219 --- /dev/null +++ b/unittests/test_authorization_queryset_coverage.py @@ -0,0 +1,142 @@ +""" +Coverage matrix for the OS object-scoping auth-filter queries. + +For every product-scoped filter, a non-staff user authorized on one product +(via authorized_users) must see only objects under that product, a user with +no access must see nothing, and a superuser must see everything. This is the +breadth guard that the earlier (superuser-only) coverage lacked — it would +have caught an allow-all `_for_queryset` regression on any of these filters. +""" + +from unittest.mock import patch + +from dojo.endpoint.queries import get_authorized_endpoint_status, get_authorized_endpoints +from dojo.engagement.queries import get_authorized_engagements +from dojo.finding.queries import ( + get_authorized_findings, + get_authorized_findings_for_queryset, + get_authorized_vulnerability_ids, + get_authorized_vulnerability_ids_for_queryset, +) +from dojo.finding_group.queries import get_authorized_finding_groups +from dojo.models import Dojo_User, Endpoint, Finding, Test, Vulnerability_Id +from dojo.product.queries import ( + get_authorized_app_analysis, + get_authorized_engagement_presets, + get_authorized_languages, + get_authorized_product_api_scan_configurations, + get_authorized_products, +) +from dojo.product_type.queries import get_authorized_product_types +from dojo.risk_acceptance.queries import get_authorized_risk_acceptances +from dojo.test.queries import get_authorized_test_imports, get_authorized_tests + +from .dojo_test_case import DojoTestCase, skip_unless_v2, versioned_fixtures + +_GCU = "dojo.authorization.query_registrations.get_current_user" + + +@versioned_fixtures +class TestAuthorizationQuerysetCoverage(DojoTestCase): + + fixtures = ["dojo_testdata.json"] + + def setUp(self): + super().setUp() + self.product = Test.objects.get(id=3).engagement.product + self.scoped_user = Dojo_User.objects.create(username="cov_scoped", is_active=True) + self.product.authorized_users.add(self.scoped_user) + self.no_access_user = Dojo_User.objects.create(username="cov_noaccess", is_active=True) + self.superuser = Dojo_User.objects.create(username="cov_super", is_active=True, is_superuser=True) + # The fixture must have data outside the authorized product, otherwise + # the "no leak" assertions would be vacuous. + self.assertTrue(Finding.objects.exclude(test__engagement__product=self.product).exists()) + + # (label, callable -> queryset, "") + @property + def _cases(self): + return [ + ("engagements", lambda: get_authorized_engagements("view"), "product__id"), + ("tests", lambda: get_authorized_tests("view"), "engagement__product__id"), + ("test_imports", lambda: get_authorized_test_imports("view"), "test__engagement__product__id"), + ("risk_acceptances", lambda: get_authorized_risk_acceptances("view"), "engagement__product__id"), + ("finding_groups", lambda: get_authorized_finding_groups("view"), "test__engagement__product__id"), + ("findings", lambda: get_authorized_findings("view"), "test__engagement__product__id"), + ("findings_for_queryset", + lambda: get_authorized_findings_for_queryset("view", Finding.objects.all()), + "test__engagement__product__id"), + ("vulnerability_ids", lambda: get_authorized_vulnerability_ids("view"), + "finding__test__engagement__product__id"), + ("vulnerability_ids_for_queryset", + lambda: get_authorized_vulnerability_ids_for_queryset("view", Vulnerability_Id.objects.all()), + "finding__test__engagement__product__id"), + ("app_analysis", lambda: get_authorized_app_analysis("view"), "product__id"), + ("languages", lambda: get_authorized_languages("view"), "product__id"), + ("engagement_presets", lambda: get_authorized_engagement_presets("view"), "product__id"), + ("product_api_scan_configurations", + lambda: get_authorized_product_api_scan_configurations("view"), "product__id"), + ("products", lambda: get_authorized_products("view"), "id"), + ] + + def test_scoped_user_sees_only_authorized_product(self): + with patch(_GCU, return_value=self.scoped_user): + for label, call, id_field in self._cases: + with self.subTest(filter=label): + leaked = set(call().values_list(id_field, flat=True)) - {self.product.id} + self.assertEqual(leaked, set(), msg=f"{label} returned objects outside the authorized product") + + def test_no_access_user_sees_nothing(self): + with patch(_GCU, return_value=self.no_access_user): + for label, call, _ in self._cases: + with self.subTest(filter=label): + self.assertEqual(call().count(), 0, msg=f"{label} leaked objects to an unauthorized user") + + def test_superuser_sees_everything(self): + with patch(_GCU, return_value=self.superuser): + for label, call, _ in self._cases: + with self.subTest(filter=label): + qs = call() + self.assertEqual( + qs.count(), qs.model.objects.count(), + msg=f"{label} did not return all objects for a superuser", + ) + + @skip_unless_v2 + def test_endpoints_scoping(self): + # Endpoint is deprecated under V3 (Locations); exercise under V2 only. + ep_in = Endpoint.objects.create(product=self.product, host="cov-in.example.com") + other_product = Test.objects.exclude(engagement__product=self.product).first().engagement.product + ep_out = Endpoint.objects.create(product=other_product, host="cov-out.example.com") + + with patch(_GCU, return_value=self.scoped_user): + for label, call in [ + ("endpoints", lambda: get_authorized_endpoints("view")), + ("endpoints_for_queryset", lambda: get_authorized_endpoints("view")), + ]: + with self.subTest(filter=label): + eps = call() + self.assertIn(ep_in, eps) + self.assertNotIn(ep_out, eps) + + with patch(_GCU, return_value=self.no_access_user): + self.assertEqual(get_authorized_endpoints("view").count(), 0) + self.assertEqual(get_authorized_endpoint_status("view").count(), 0) + + with patch(_GCU, return_value=self.superuser): + self.assertEqual(get_authorized_endpoints("view").count(), Endpoint.objects.count()) + + def test_product_types_scoping(self): + # get_authorized_product_types keys off product_type.authorized_users, so + # a product-only member sees none; a product-type member sees their type. + with patch(_GCU, return_value=self.no_access_user): + self.assertEqual(get_authorized_product_types("view").count(), 0) + with patch(_GCU, return_value=self.superuser): + self.assertEqual( + get_authorized_product_types("view").count(), + self.product.prod_type.__class__.objects.count(), + ) + pt_user = Dojo_User.objects.create(username="cov_pt", is_active=True) + self.product.prod_type.authorized_users.add(pt_user) + with patch(_GCU, return_value=pt_user): + pts = get_authorized_product_types("view") + self.assertEqual(set(pts.values_list("id", flat=True)), {self.product.prod_type_id}) diff --git a/unittests/test_bulk_finding_authorization.py b/unittests/test_bulk_finding_authorization.py new file mode 100644 index 00000000000..de076f6f487 --- /dev/null +++ b/unittests/test_bulk_finding_authorization.py @@ -0,0 +1,58 @@ +""" +Authorization scoping for the bulk finding endpoint. + +A product-scoped, non-staff user must not be able to bulk-delete (or edit) +findings belonging to products they are not authorized for via +``finding_bulk_update_all`` (``/finding/bulk``), even by POSTing arbitrary +finding ids. +""" + +from django.urls import reverse + +from dojo.models import Dojo_User, Finding, Test + +from .dojo_test_case import DojoTestCase, versioned_fixtures + + +@versioned_fixtures +class TestBulkFindingAuthorizationScoping(DojoTestCase): + + fixtures = ["dojo_testdata.json"] + + def setUp(self): + super().setUp() + self.user = Dojo_User.objects.create(username="bulk_scoped", is_active=True) + # The user is authorized on this product only (via authorized_users). + self.product = Test.objects.get(id=3).engagement.product + self.product.authorized_users.add(self.user) + # A finding that belongs to a DIFFERENT product. + self.other_finding = Finding.objects.exclude( + test__engagement__product=self.product, + ).first() + self.assertIsNotNone(self.other_finding) + self.client.force_login(self.user) + + def test_scoped_user_cannot_bulk_delete_other_products_findings(self): + response = self.client.post(reverse("finding_bulk_update_all"), { + "finding_to_update": [self.other_finding.id], + "delete_bulk_findings": "1", + }) + self.assertLess(response.status_code, 500) + self.assertTrue( + Finding.objects.filter(id=self.other_finding.id).exists(), + msg="scoped user deleted a finding outside their authorized products", + ) + + def test_scoped_user_cannot_bulk_edit_other_products_findings(self): + original_severity = self.other_finding.severity + new_severity = "Low" if original_severity != "Low" else "High" + response = self.client.post(reverse("finding_bulk_update_all"), { + "finding_to_update": [self.other_finding.id], + "severity": new_severity, + }) + self.assertLess(response.status_code, 500) + self.other_finding.refresh_from_db() + self.assertEqual( + self.other_finding.severity, original_severity, + msg="scoped user edited a finding outside their authorized products", + ) diff --git a/unittests/test_importers_performance.py b/unittests/test_importers_performance.py index 30ecfeb00f3..47d30a99824 100644 --- a/unittests/test_importers_performance.py +++ b/unittests/test_importers_performance.py @@ -343,13 +343,13 @@ def test_import_reimport_reimport_performance_pghistory_async(self): configure_pghistory_triggers() self._import_reimport_performance( - expected_num_queries1=156, + expected_num_queries1=157, expected_num_async_tasks1=2, - expected_num_queries2=121, + expected_num_queries2=122, expected_num_async_tasks2=1, - expected_num_queries3=28, + expected_num_queries3=29, expected_num_async_tasks3=1, - expected_num_queries4=99, + expected_num_queries4=100, expected_num_async_tasks4=0, ) @@ -367,13 +367,13 @@ def test_import_reimport_reimport_performance_pghistory_no_async(self): testuser.usercontactinfo.save() self._import_reimport_performance( - expected_num_queries1=170, + expected_num_queries1=173, expected_num_async_tasks1=2, - expected_num_queries2=129, + expected_num_queries2=130, expected_num_async_tasks2=1, - expected_num_queries3=36, + expected_num_queries3=37, expected_num_async_tasks3=1, - expected_num_queries4=99, + expected_num_queries4=100, expected_num_async_tasks4=0, ) @@ -392,13 +392,13 @@ def test_import_reimport_reimport_performance_pghistory_no_async_with_product_gr self.system_settings(enable_product_grade=True) self._import_reimport_performance( - expected_num_queries1=180, + expected_num_queries1=183, expected_num_async_tasks1=4, - expected_num_queries2=139, + expected_num_queries2=140, expected_num_async_tasks2=3, - expected_num_queries3=43, + expected_num_queries3=44, expected_num_async_tasks3=3, - expected_num_queries4=108, + expected_num_queries4=109, expected_num_async_tasks4=2, ) @@ -524,9 +524,9 @@ def test_deduplication_performance_pghistory_async(self): self.system_settings(enable_deduplication=True) self._deduplication_performance( - expected_num_queries1=92, + expected_num_queries1=93, expected_num_async_tasks1=2, - expected_num_queries2=72, + expected_num_queries2=73, expected_num_async_tasks2=2, check_duplicates=False, # Async mode - deduplication happens later ) @@ -545,9 +545,9 @@ def test_deduplication_performance_pghistory_no_async(self): testuser.usercontactinfo.save() self._deduplication_performance( - expected_num_queries1=106, + expected_num_queries1=109, expected_num_async_tasks1=2, - expected_num_queries2=87, + expected_num_queries2=90, expected_num_async_tasks2=2, ) @@ -633,13 +633,13 @@ def test_import_reimport_reimport_performance_pghistory_async(self): configure_pghistory_triggers() self._import_reimport_performance( - expected_num_queries1=163, + expected_num_queries1=164, expected_num_async_tasks1=2, - expected_num_queries2=130, + expected_num_queries2=131, expected_num_async_tasks2=1, - expected_num_queries3=36, + expected_num_queries3=37, expected_num_async_tasks3=1, - expected_num_queries4=100, + expected_num_queries4=101, expected_num_async_tasks4=0, ) @@ -657,13 +657,13 @@ def test_import_reimport_reimport_performance_pghistory_no_async(self): testuser.usercontactinfo.save() self._import_reimport_performance( - expected_num_queries1=179, + expected_num_queries1=182, expected_num_async_tasks1=2, - expected_num_queries2=140, + expected_num_queries2=141, expected_num_async_tasks2=1, - expected_num_queries3=46, + expected_num_queries3=47, expected_num_async_tasks3=1, - expected_num_queries4=100, + expected_num_queries4=101, expected_num_async_tasks4=0, ) @@ -682,13 +682,13 @@ def test_import_reimport_reimport_performance_pghistory_no_async_with_product_gr self.system_settings(enable_product_grade=True) self._import_reimport_performance( - expected_num_queries1=192, + expected_num_queries1=195, expected_num_async_tasks1=4, - expected_num_queries2=153, + expected_num_queries2=154, expected_num_async_tasks2=3, - expected_num_queries3=53, + expected_num_queries3=54, expected_num_async_tasks3=3, - expected_num_queries4=112, + expected_num_queries4=113, expected_num_async_tasks4=2, ) @@ -789,9 +789,9 @@ def test_deduplication_performance_pghistory_async(self): self.system_settings(enable_deduplication=True) self._deduplication_performance( - expected_num_queries1=99, + expected_num_queries1=100, expected_num_async_tasks1=2, - expected_num_queries2=75, + expected_num_queries2=76, expected_num_async_tasks2=2, check_duplicates=False, # Async mode - deduplication happens later ) @@ -809,8 +809,8 @@ def test_deduplication_performance_pghistory_no_async(self): testuser.usercontactinfo.save() self._deduplication_performance( - expected_num_queries1=115, + expected_num_queries1=118, expected_num_async_tasks1=2, - expected_num_queries2=198, + expected_num_queries2=201, expected_num_async_tasks2=2, ) diff --git a/unittests/test_user_queries.py b/unittests/test_user_queries.py index cfd951de512..71974ee9067 100644 --- a/unittests/test_user_queries.py +++ b/unittests/test_user_queries.py @@ -89,26 +89,41 @@ def test_user_admin(self, mock_current_user): @patch("dojo.authorization.query_registrations.get_current_user") def test_user_global_permission_legacy(self, mock_current_user): - # Legacy: Global_Role(role=Reader) is inert. The user has no - # is_staff / is_superuser flag, so only their own row is returned. + # Carrier Global_Role(role=Reader) is inert in OS. Without any + # authorized_users membership the user sees only superusers (always + # surfaced, matching 2.58.4). mock_current_user.return_value = self.global_permission_user self.assertQuerySetEqual( - Dojo_User.objects.filter(pk=self.global_permission_user.pk), + Dojo_User.objects.filter(is_superuser=True).order_by("first_name", "last_name"), get_authorized_users(Permissions.Product_View), ) @patch("dojo.authorization.query_registrations.get_current_user") def test_user_regular_legacy(self, mock_current_user): - # Legacy: per-product RBAC role is inert. A non-staff non-superuser - # only sees themselves. + # Carrier Product_Member / Product_Type_Member are inert in OS. Without + # any authorized_users membership the user sees only superusers. mock_current_user.return_value = self.regular_user self.assertQuerySetEqual( - Dojo_User.objects.filter(pk=self.regular_user.pk), + Dojo_User.objects.filter(is_superuser=True).order_by("first_name", "last_name"), get_authorized_users(Permissions.Product_View), ) + @patch("dojo.authorization.query_registrations.get_current_user") + def test_user_collaborators_via_authorized_users(self, mock_current_user): + # v2 parity: a non-staff user sees co-members of the products/types + # they are authorized on (via authorized_users), plus superusers. + self.product_1.authorized_users.add(self.regular_user) + self.product_1.authorized_users.add(self.product_user) + mock_current_user.return_value = self.regular_user + + users = get_authorized_users(Permissions.Product_View) + self.assertIn(self.regular_user, users) + self.assertIn(self.product_user, users) + self.assertIn(self.admin_user, users) + self.assertNotIn(self.invisible_user, users) + class TestGetAuthorizedUsersForProductType(DojoTestCase): @@ -185,16 +200,18 @@ def test_superuser_caller_sees_all(self, mock_get_current_user): self.assertIn(self.user_global_reader, users) @patch("dojo.authorization.query_registrations.get_current_user") - def test_non_staff_caller_sees_none(self, mock_get_current_user): - # Legacy: a non-staff non-superuser caller can't enumerate users - # for a product_type — including users with explicit memberships. + def test_non_staff_caller_sees_only_superusers(self, mock_get_current_user): + # OS: carrier Product_Type_Member is inert. A non-staff caller without + # authorized_users membership sees only superusers (2.58.4 parity). mock_get_current_user.return_value = self.user_product_type_member users = get_authorized_users_for_product_type( Dojo_User.objects.all(), self.product_type, Permissions.Product_Type_View, ) - self.assertEqual(users.count(), 0) + self.assertIn(self.superuser, users) + self.assertNotIn(self.user_product_type_member, users) + self.assertNotIn(self.user_no_perms, users) @patch("dojo.authorization.query_registrations.get_current_user") def test_anonymous_caller_sees_none(self, mock_get_current_user): @@ -311,14 +328,18 @@ def test_superuser_caller_sees_all(self, mock_get_current_user): self.assertIn(self.user_global_reader, users) @patch("dojo.authorization.query_registrations.get_current_user") - def test_non_staff_caller_sees_none(self, mock_get_current_user): + def test_non_staff_caller_sees_only_superusers(self, mock_get_current_user): + # OS: carrier Product_Member is inert. A non-staff caller without + # authorized_users membership sees only superusers (2.58.4 parity). mock_get_current_user.return_value = self.user_product_member users = get_authorized_users_for_product_and_product_type( None, self.product, Permissions.Product_View, ) - self.assertEqual(users.count(), 0) + self.assertIn(self.superuser, users) + self.assertNotIn(self.user_product_member, users) + self.assertNotIn(self.user_no_perms, users) @patch("dojo.authorization.query_registrations.get_current_user") def test_anonymous_caller_sees_none(self, mock_get_current_user): @@ -344,3 +365,54 @@ def test_users_parameter_filters_base_queryset(self, mock_get_current_user): ) for user in users: self.assertTrue(user.is_active) + + +class TestGetAuthorizedUsersViaAuthorizedUsers(DojoTestCase): + + """ + OS authorized_users-based resolution for the product / product-type user + queries (regression test for issue #15062 — empty Testing Lead selector). + """ + + @classmethod + def setUpTestData(cls): + cls.product_type = Product_Type.objects.create(name="AU Test PT") + cls.product = Product.objects.create( + name="AU Test Product", description="t", prod_type=cls.product_type, + ) + + cls.product_member = Dojo_User.objects.create(username="au_product_member", is_active=True) + cls.product_type_member = Dojo_User.objects.create(username="au_pt_member", is_active=True) + cls.unrelated = Dojo_User.objects.create(username="au_unrelated", is_active=True) + # Superuser not in any authorized_users — must still surface (2.58.4 parity). + cls.superuser = Dojo_User.objects.create(username="au_superuser", is_active=True, is_superuser=True) + + # The "authorized users" sections the reporter used in the UI. + cls.product.authorized_users.add(cls.product_member) + cls.product_type.authorized_users.add(cls.product_type_member) + + @patch("dojo.authorization.query_registrations.get_current_user") + def test_product_and_product_type_returns_authorized_users(self, mock_get_current_user): + # #15062: a non-staff user authorized on the product (via authorized_users) + # must get a non-empty list so they can pick a Testing Lead. The list + # contains users authorized directly on the product and via its type, + # plus superusers (2.58.4 parity). + mock_get_current_user.return_value = self.product_member + users = get_authorized_users_for_product_and_product_type( + None, self.product, Permissions.Product_View, + ) + self.assertIn(self.product_member, users) + self.assertIn(self.product_type_member, users) + self.assertIn(self.superuser, users) + self.assertNotIn(self.unrelated, users) + + @patch("dojo.authorization.query_registrations.get_current_user") + def test_product_type_returns_authorized_users(self, mock_get_current_user): + mock_get_current_user.return_value = self.product_type_member + users = get_authorized_users_for_product_type( + None, self.product_type, Permissions.Product_Type_View, + ) + self.assertIn(self.product_type_member, users) + self.assertIn(self.superuser, users) + self.assertNotIn(self.product_member, users) + self.assertNotIn(self.unrelated, users) diff --git a/unittests/tools/test_sarif_parser.py b/unittests/tools/test_sarif_parser.py index 77fe9e9a7fe..d9036043d18 100644 --- a/unittests/tools/test_sarif_parser.py +++ b/unittests/tools/test_sarif_parser.py @@ -580,6 +580,46 @@ def test_get_fingerprints_hashes(self): get_fingerprints_hashes(data2["fingerprints"]), ) + # some tools (e.g. BlackDuck) wrap the hash as {"value": ""} instead of a plain string + data3 = {"fingerprints": {"csvRowSha256": {"value": "abc123"}}} + self.assertEqual( + {"csvRowSha256": {"version": 0, "value": "abc123"}}, + get_fingerprints_hashes(data3["fingerprints"]), + ) + + # nested dict with no "value" key falls back to empty string rather than raising KeyError + data4 = {"fingerprints": {"csvRowSha256": {"other": "data"}}} + self.assertEqual( + {"csvRowSha256": {"version": 0, "value": ""}}, + get_fingerprints_hashes(data4["fingerprints"]), + ) + + def test_blackduck_nested_fingerprints(self): + """ + BlackDuck wraps fingerprint values as {"value": ""} instead of a plain string. + Verify unique_id_from_tool is extracted as a string, not left as a dict. + """ + with (get_unit_tests_scans_path("sarif") / "blackduck_nested_fingerprints.sarif").open(encoding="utf-8") as testfile: + parser = SarifParser() + findings = parser.get_findings(testfile, Test()) + self.assertEqual(5, len(findings)) + for finding in findings: + self.common_checks(finding) + self.assertIsInstance(finding.unique_id_from_tool, str) + self.assertFalse(finding.unique_id_from_tool.startswith("{")) + with self.subTest(i=0): + finding = findings[0] + self.assertEqual("Medium", finding.severity) + self.assertEqual("aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", finding.unique_id_from_tool) + with self.subTest(i=1): + finding = findings[1] + self.assertEqual("High", finding.severity) + self.assertEqual("bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb", finding.unique_id_from_tool) + with self.subTest(i=3): + finding = findings[3] + self.assertEqual("Info", finding.severity) + self.assertEqual("dddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddd", finding.unique_id_from_tool) + def test_tags_from_result_properties(self): with (get_unit_tests_scans_path("sarif") / "taint-python-report.sarif").open(encoding="utf-8") as testfile: parser = SarifParser() diff --git a/unittests/tools/test_xygeni_parser.py b/unittests/tools/test_xygeni_parser.py index ea161a096bd..e1caf9a0f44 100644 --- a/unittests/tools/test_xygeni_parser.py +++ b/unittests/tools/test_xygeni_parser.py @@ -42,10 +42,15 @@ def test_sast_many_findings(self): self.assertIsNotNone(finding.unique_id_from_tool) match = next( - (f for f in findings if f.unique_id_from_tool == "nsXRi+PTLom/sG8m6weOXw"), + ( + f for f in findings + if f.unique_id_from_tool == "SAS.injection.python.code_injection_deserialization.dockerized_labs/insec_des_lab/main.py.36" + ), None, ) - self.assertIsNotNone(match, "expected the deserialization SAST finding by uniqueHash") + self.assertIsNotNone(match, "expected the deserialization SAST finding by issueId") + # uniqueHash is kept as the vuln id; the per-occurrence issueId drives dedup + self.assertEqual("nsXRi+PTLom/sG8m6weOXw", match.vuln_id_from_tool) self.assertEqual("python.code_injection_deserialization", match.title) self.assertEqual("Critical", match.severity) self.assertEqual(502, match.cwe) @@ -57,6 +62,20 @@ def test_sast_many_findings(self): self.assertEqual("serialized_data", match.sast_source_object) self.assertIn("Data flow", match.description) + def test_sast_repeated_detector_in_same_file_stays_distinct(self): + # One detector flagging the same pattern at several lines shares a uniqueHash but + # carries a distinct issueId per line. Each occurrence must remain its own Finding. + with self._load("sast_many_findings.json") as testfile: + findings = XygeniParser().get_findings(testfile, Test()) + + occurrences = [ + f for f in findings + if f.vuln_id_from_tool == "VsqoC9U6q8EYG0QZ5UqxXw" # forms_without_csrf_protection, 4 lines + ] + self.assertEqual(4, len(occurrences), "all occurrences of the repeated detector must be kept") + unique_ids = {f.unique_id_from_tool for f in occurrences} + self.assertEqual(4, len(unique_ids), "each occurrence needs a distinct unique_id_from_tool") + def test_sca_many_findings(self): with self._load("sca_many_findings.json") as testfile: findings = XygeniParser().get_findings(testfile, Test()) @@ -97,16 +116,34 @@ def test_secrets_many_findings(self): self.assertIsNotNone(finding.cwe) match = next( - (f for f in findings if f.unique_id_from_tool == "LVAjuA4Z40VxktixjtztXg"), + (f for f in findings if f.unique_id_from_tool == "SEC.private_key.private_key..ssh/id_rsa.1"), None, ) self.assertIsNotNone(match, "expected the .ssh/id_rsa private-key secret finding") + # uniqueHash is kept as the vuln id; the per-occurrence issueId drives dedup + self.assertEqual("LVAjuA4Z40VxktixjtztXg", match.vuln_id_from_tool) self.assertEqual("Critical", match.severity) self.assertEqual(".ssh/id_rsa", match.file_path) self.assertEqual(1, match.line) self.assertIn("private_key", match.title) self.assertIn("Rotate", match.mitigation) + def test_secrets_repeated_in_same_file_stay_distinct(self): + # A secret value repeated in one file shares a uniqueHash across occurrences but + # carries a distinct issueId per line. The parser must surface each occurrence as + # its own Finding (distinct unique_id_from_tool) so dedup does not collapse them. + with self._load("secrets_many_findings.json") as testfile: + findings = XygeniParser().get_findings(testfile, Test()) + + occurrences = [ + f for f in findings + if f.vuln_id_from_tool == "1yvAV2ndtW4yYG+TJQhhXg" # .docker/.dockercfg, lines 9 and 29 + ] + self.assertEqual(2, len(occurrences), "both occurrences of the repeated secret must be kept") + unique_ids = {f.unique_id_from_tool for f in occurrences} + self.assertEqual(2, len(unique_ids), "each occurrence needs a distinct unique_id_from_tool") + self.assertEqual({9, 29}, {f.line for f in occurrences}) + # ----- dispatch + error cases ----- def test_dispatches_on_metadata_scan_type(self): @@ -127,7 +164,8 @@ def test_dispatches_on_metadata_scan_type(self): } findings = XygeniParser().get_findings(io.StringIO(json.dumps(report)), Test()) self.assertEqual(1, len(findings)) - self.assertEqual("abc123", findings[0].unique_id_from_tool) + self.assertEqual("SECRETS.aws-access-key.config.ini:12", findings[0].unique_id_from_tool) + self.assertEqual("abc123", findings[0].vuln_id_from_tool) self.assertEqual(798, findings[0].cwe) def test_raises_on_missing_scan_type(self):