Skip to content

Add XML support for Checkmarx CxFlow SAST parser and regression test#15049

Open
goutham-hari wants to merge 48 commits into
DefectDojo:masterfrom
goutham-hari:checkmarx-cxflow-xml-support
Open

Add XML support for Checkmarx CxFlow SAST parser and regression test#15049
goutham-hari wants to merge 48 commits into
DefectDojo:masterfrom
goutham-hari:checkmarx-cxflow-xml-support

Conversation

@goutham-hari

@goutham-hari goutham-hari commented Jun 21, 2026

Copy link
Copy Markdown

Description
The Checkmarx CxFlow SAST parser (dojo/tools/checkmarx_cxflow_sast/parser.py)
previously only supported JSON report imports — XML files hit a TODO stub and
returned no findings.

This PR adds XML support by delegating to the existing CheckmarxParser
(dojo/tools/checkmarx/parser.py, detailed mode) instead of duplicating its
parsing logic.

Test results
Added test_xml_parse_file_with_single_vulnerability_has_single_finding in
unittests/tools/test_checkmarx_cxflow_sast_parser.py, using the existing
sample file checkmarx/single_finding.xml. Verifies a single finding is
returned with the correct title, "High" severity, active=True, verified=False.

Documentation
No documentation changes — this extends file-format support within the existing
Checkmarx CxFlow SAST parser; no new parser/page is introduced.

Extra information
No pre-approval issue was filed for this one, given the narrow scope (closing a
TODO/format gap in an existing parser rather than adding a new integration).
Happy to open one if a maintainer would prefer that first.

DefectDojo release bot and others added 21 commits June 15, 2026 22:37
….0.0-3.1.0-dev

Release: Merge back 3.0.0 into dev from: master-into-dev/3.0.0-3.1.0-dev
…jo#14962)

* Added global required fields notice for WCAG H90 compliance

* display_tags.py add and removed blank line

* Added setting/env variable DD_SHOW_A11Y_REQUIRED_FIELDS_NOTICE

---------

Co-authored-by: symon.vezina <symon.vezina@hrsdc-rhdcc.gc.ca>
….0.1-3.1.0-dev

Release: Merge back 3.0.1 into dev from: master-into-dev/3.0.1-3.1.0-dev
* perf(importers): batch vulnerability_id inserts

Replace per-row Vulnerability_Id saves with bulk_create in two layers:

- fix sanitize_vulnerability_ids to return filtered list (was a no-op
  bug — reassigned local variable, caller never saw the result)
- save_vulnerability_ids now uses bulk_create per finding instead of
  one INSERT per ID; fixes all callers including the reimporter path
- DefaultImporter.store_vulnerability_ids accumulates Vulnerability_Id
  objects across all findings in a batch; flush_vulnerability_ids()
  does a single bulk_create at each batch boundary (alongside
  location_handler.persist())

For a scan with 1000 findings × 5 CVEs each: 5000 INSERT queries
reduced to O(batches) bulk_create calls.

* perf(reimporter): batch vulnerability_id reconciliation

Extend the cross-finding accumulation pattern to DefaultReImporter:

- reconcile_vulnerability_ids now accumulates changed findings into
  pending_vuln_id_deletes / pending_vulnerability_ids instead of
  issuing per-finding DELETE + INSERT immediately
- flush_vulnerability_ids (BaseImporter) runs one bulk DELETE WHERE
  finding_id IN (...) followed by one bulk_create for all new IDs
- flush called at both dedupe batch boundaries (alongside
  location_handler.persist()) and after the mitigation loop

Early-exit path (unchanged IDs) never touches either buffer, so the
common case pays zero extra cost.

Add two unit tests: cross-finding batch (3 findings, 2 changed + 1
unchanged, verify buffer contents before flush and DB state after) and
unchanged-IDs early-exit (verify buffers stay empty).

* test(performance): re-baseline importer query counts

Remove pending-rebaseline skips from TestDojoImporterPerformanceSmall
and TestDojoImporterPerformanceSmallLocations. Update all expected
query counts to reflect the batch Vulnerability_Id insert optimisation
(counts decrease by 1-20 queries per step depending on the scan size
and code path).

* fix(test): update TestSaveVulnerabilityIds mock for bulk_create

The test mocked Vulnerability_Id.save (individual saves) but
save_vulnerability_ids now uses bulk_create. Django's bulk_create
validates FK references before issuing SQL, raising ValueError when
the finding has no pk. Mock bulk_create instead and assert on the
deduplicated object list passed to it.

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
…f tests (DefectDojo#14969)

* perf(importers): batch BurpRawRequestResponse inserts + re-enable perf tests

Replace per-finding save() calls in process_request_response_pairs with
bulk_create at batch boundaries, mirroring the location_handler pattern.
Reduces DB round-trips proportionally to findings with req/resp data.
Drops the no-op clean() calls (BurpRawRequestResponse has no custom clean).

Re-enable TestDojoImporterPerformanceSmall and TestDojoImporterPerformanceSmallLocations
with recalibrated query counts after the RBAC→legacy authorization migration.

* test(perf): recalibrate tag inheritance ZAP query counts

Batch BurpRawRequestResponse inserts reduce per-finding saves for the ZAP
parser (which emits req/resp pairs). Update expected counts to match.
…ocker-compose.yml) (DefectDojo#15022)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
…(.github/workflows/release-x-manual-tag-as-latest.yml) (DefectDojo#14990)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Cody Maffucci <46459665+Maffooch@users.noreply.github.com>
…v3.0.3 to v3.1.0 (.github/workflows/detect-merge-conflicts.yaml) (DefectDojo#14992)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Cody Maffucci <46459665+Maffooch@users.noreply.github.com>
….23.1 (docker-compose.override.dev.yml) (DefectDojo#14997)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Cody Maffucci <46459665+Maffooch@users.noreply.github.com>
Bumps [ruff](https://github.com/astral-sh/ruff) from 0.15.15 to 0.15.16.
- [Release notes](https://github.com/astral-sh/ruff/releases)
- [Changelog](https://github.com/astral-sh/ruff/blob/main/CHANGELOG.md)
- [Commits](astral-sh/ruff@0.15.15...0.15.16)

---
updated-dependencies:
- dependency-name: ruff
  dependency-version: 0.15.16
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Cody Maffucci <46459665+Maffooch@users.noreply.github.com>
…Dojo#14996)

Bumps [pdfmake](https://github.com/bpampuch/pdfmake) from 0.3.8 to 0.3.10.
- [Release notes](https://github.com/bpampuch/pdfmake/releases)
- [Changelog](https://github.com/bpampuch/pdfmake/blob/master/CHANGELOG.md)
- [Commits](bpampuch/pdfmake@0.3.8...0.3.10)

---
updated-dependencies:
- dependency-name: pdfmake
  dependency-version: 0.3.10
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Cody Maffucci <46459665+Maffooch@users.noreply.github.com>
Bumps [vcrpy](https://github.com/kevin1024/vcrpy) from 8.1.1 to 8.2.1.
- [Release notes](https://github.com/kevin1024/vcrpy/releases)
- [Changelog](https://github.com/kevin1024/vcrpy/blob/master/docs/changelog.rst)
- [Commits](kevin1024/vcrpy@v8.1.1...v8.2.1)

---
updated-dependencies:
- dependency-name: vcrpy
  dependency-version: 8.2.1
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [sqlalchemy](https://github.com/sqlalchemy/sqlalchemy) from 2.0.50 to 2.0.51.
- [Release notes](https://github.com/sqlalchemy/sqlalchemy/releases)
- [Changelog](https://github.com/sqlalchemy/sqlalchemy/blob/main/CHANGES.rst)
- [Commits](https://github.com/sqlalchemy/sqlalchemy/commits)

---
updated-dependencies:
- dependency-name: sqlalchemy
  dependency-version: 2.0.51
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…3 (.github/workflows/test-helm-chart.yml) (DefectDojo#15021)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Bumps [@tailwindcss/cli](https://github.com/tailwindlabs/tailwindcss/tree/HEAD/packages/@tailwindcss-cli) from 4.3.0 to 4.3.1.
- [Release notes](https://github.com/tailwindlabs/tailwindcss/releases)
- [Changelog](https://github.com/tailwindlabs/tailwindcss/blob/main/CHANGELOG.md)
- [Commits](https://github.com/tailwindlabs/tailwindcss/commits/v4.3.1/packages/@tailwindcss-cli)

---
updated-dependencies:
- dependency-name: "@tailwindcss/cli"
  dependency-version: 4.3.1
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps vulners from 3.1.10 to 3.1.11.

---
updated-dependencies:
- dependency-name: vulners
  dependency-version: 3.1.11
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…jo#15040)

Move the Finding Groups links (Open/All/Closed Findings Groups) into the
Findings dropdown in the new-UI sidebar and remove the standalone top-level
"Finding Groups" nav item. A subtle divider separates the groups links within
the dropdown. This declutters the top-level navigation and keeps related
finding views together.
…ectDojo#15006)

Trivy legacy-format reports have no "Class" field, so the parser set
unsaved_tags entries to None (e.g. ['debian', None]). clean_tags then
crashed the whole import with TypeError in TAG_PATTERN.sub, after
parsing had already succeeded (regression from DefectDojo#14111 in 2.55.0).

- clean_tags: drop None entries instead of crashing (defends every parser)
- trivy parser: filter falsy values at all four unsaved_tags sites
- regression tests: clean_tags None handling; legacy fixture tags contain
  no None

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
DefectDojo release bot and others added 5 commits June 22, 2026 16:58
….0.100-3.1.0-dev

Release: Merge back 3.0.100 into dev from: master-into-dev/3.0.100-3.1.0-dev
…efectDojo#14999)

Several new-UI templates hardcoded brand colors that should reference the
design tokens defined in components/tailwind.css:

- calendar.html: active engagement/test events used the legacy Bootstrap
  primary blue (#337ab7) instead of the brand blue.
- benchmark.html: table link color, same legacy #337ab7.
- base.html: sidebar background (DefectDojo#3864) and sub-nav link color (#82B0D9)
  hardcoded the Fuji Blue brand hues.

These now use the matching var(--color-dd-primary-*) tokens (500/900/200).
Token values are identical to the hardcoded hex, so there is no visual change.

Intentionally left as-is:
- The sidebar lighter link color (#C6DDF2): the matching shade
  (dd-primary-100) is not emitted in the compiled CSS, since Tailwind v4 only
  outputs theme variables referenced by a generated utility.
- PDF report templates also contain #337ab7, but CSS custom properties do not
  resolve in the PDF renderer.
- Generic white/black, neutral grays, and the custom #002a4d shade have no
  design token.

FullCalendar 3.10.5 applies the event color verbatim as an inline style, so
the variable resolves at render time. Verified in the running new UI: sidebar
bg = rgb(0,56,100), sub-nav link = rgb(130,176,217), calendar event =
rgb(23,121,197) -- each identical to the hex it replaced.
…gin page (DefectDojo#14998)

The new Tailwind login page hardcoded hex color values that exactly
duplicate the design tokens defined in components/tailwind.css. Swap them
for the corresponding var(--color-*) tokens so the page stays in sync with
the design system if the palette changes. The token values are identical to
the previously hardcoded hex, so there is no visual change.

The control-label color (#333333) and the alpha-channel rgba() shadows are
left as-is since they have no exact token equivalent.
renovate Bot and others added 15 commits June 23, 2026 16:40
…yml) (DefectDojo#14896)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
…fectdojo/chart.yaml) (DefectDojo#14917)

* chore(deps): update valkey docker tag from 0.20.2 to v0.22.1 (helm/defectdojo/chart.yaml)

* update Helm documentation

---------

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
…ithub/workflows/test-helm-chart.yml) (DefectDojo#15067)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
…github/workflows/k8s-tests.yml) (DefectDojo#15068)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
…ile.integration-tests-debian) (DefectDojo#15069)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
… v3.0.1 (.github/workflows/release-x-manual-helm-chart.yml) (DefectDojo#15070)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
…tDojo#15071)

Bumps [django-debug-toolbar](https://github.com/django-commons/django-debug-toolbar) from 6.3.0 to 7.0.0.
- [Release notes](https://github.com/django-commons/django-debug-toolbar/releases)
- [Changelog](https://github.com/django-commons/django-debug-toolbar/blob/main/docs/changes.rst)
- [Commits](django-commons/django-debug-toolbar@6.3.0...7.0.0)

---
updated-dependencies:
- dependency-name: django-debug-toolbar
  dependency-version: 7.0.0
  dependency-type: direct:development
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [ruff](https://github.com/astral-sh/ruff) from 0.15.16 to 0.15.19.
- [Release notes](https://github.com/astral-sh/ruff/releases)
- [Changelog](https://github.com/astral-sh/ruff/blob/main/CHANGELOG.md)
- [Commits](astral-sh/ruff@0.15.16...0.15.19)

---
updated-dependencies:
- dependency-name: ruff
  dependency-version: 0.15.19
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
)

Bumps [django-environ](https://github.com/joke2k/django-environ) from 0.13.0 to 0.14.0.
- [Release notes](https://github.com/joke2k/django-environ/releases)
- [Changelog](https://github.com/joke2k/django-environ/blob/v0.14.0/CHANGELOG.rst)
- [Commits](joke2k/django-environ@v0.13.0...v0.14.0)

---
updated-dependencies:
- dependency-name: django-environ
  dependency-version: 0.14.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [redis](https://github.com/redis/redis-py) from 8.0.0 to 8.0.1.
- [Release notes](https://github.com/redis/redis-py/releases)
- [Changelog](https://github.com/redis/redis-py/blob/master/CHANGES)
- [Commits](redis/redis-py@v8.0.0...v8.0.1)

---
updated-dependencies:
- dependency-name: redis
  dependency-version: 8.0.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…tDojo#15075)

Bumps [pdfmake](https://github.com/bpampuch/pdfmake) from 0.3.10 to 0.3.11.
- [Release notes](https://github.com/bpampuch/pdfmake/releases)
- [Changelog](https://github.com/bpampuch/pdfmake/blob/master/CHANGELOG.md)
- [Commits](bpampuch/pdfmake@0.3.10...0.3.11)

---
updated-dependencies:
- dependency-name: pdfmake
  dependency-version: 0.3.11
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
… (.github/workflows/test-helm-chart.yml) (DefectDojo#15076)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
…/workflows/validate_docs_build.yml) (DefectDojo#15077)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
….23.1 (docker-compose.override.integration_tests.yml) (DefectDojo#15078)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
…m v7.22.0 to v7.23.0 (dockerfile.integration-tests-debian) (DefectDojo#15079)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
@goutham-hari goutham-hari force-pushed the checkmarx-cxflow-xml-support branch 2 times, most recently from 7a55ed9 to 007df3b Compare June 26, 2026 02:23
@github-actions github-actions Bot added docker settings_changes Needs changes to settings.py based on changes in settings.dist.py included in this PR apiv2 docs ui helm labels Jun 26, 2026
@goutham-hari

Copy link
Copy Markdown
Author

Description

Extend the Checkmarx CxFlow SAST parser to support both JSON and XML input formats.

Changes

  • Modified: dojo/tools/checkmarx_cxflow_sast/parser.py
    • Added CheckmarxParser import for XML support
    • Updated get_findings() to detect file format by extension (.json or .xml)
    • JSON files continue to use the existing _get_findings_json() method
    • XML files now delegate to the established CheckmarxParser with "detailed" mode for consistent output

Motivation

The Checkmarx CxFlow SAST parser previously only supported JSON export format, with a TODO comment indicating XML support was needed. This change enables users to import Checkmarx reports in both JSON and XML formats, providing greater flexibility in vulnerability management workflows.

Implementation Details

  • Reuses the existing, well-tested CheckmarxParser logic for XML handling to avoid code duplication
  • Both formats produce consistent Finding objects with the same field mappings
  • Maintains backward compatibility—existing JSON parsing remains unchanged
  • Follows the established pattern of format detection by file extension

Testing

The existing unit tests continue to pass for JSON inputs. XML support leverages the established Checkmarx XML parser, which is independently tested through test_checkmarx_parser.py.

Related Issues

Resolves the # TODO: support CxXML format comment in the CxFlow SAST parser.

@Maffooch Maffooch left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@goutham-hari please rebase the PR to the dev branch, and correct unit test failures

@Maffooch Maffooch marked this pull request as draft June 26, 2026 16:54
@goutham-hari goutham-hari marked this pull request as ready for review June 27, 2026 04:14
@github-actions

Copy link
Copy Markdown
Contributor

This pull request has conflicts, please resolve those before we can evaluate the pull request.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

apiv2 conflicts-detected docker docs helm parser settings_changes Needs changes to settings.py based on changes in settings.dist.py included in this PR ui unittests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

9 participants