Skip to content

perf(findings): load vulnerability_ids for the whole findings list page - #16659

Open
jensholdgaard wants to merge 1 commit into
DefectDojo:devfrom
jensholdgaard:fix/finding-api-vulnerability-id-prefetch
Open

jensholdgaard wants to merge 1 commit into
DefectDojo:devfrom
jensholdgaard:fix/finding-api-vulnerability-id-prefetch

Conversation

@jensholdgaard

Copy link
Copy Markdown

GET /api/v2/findings/ ran one dojo_findingvulnerabilityreference query per finding on the page, because vulnerability_ids is serialized on every row and FindingViewSet's queryset doesn't prefetch it.

Adding vulnerability_id_prefetch() to get_queryset() would break a subclass or plugin that adds its own (Django raises "lookup was already seen with a different queryset"), which is why #16512 left it out. Instead, FindingViewSet.paginate_queryset() loads the references for the page with prefetch_related_objects(), skipping findings whose references are already prefetched, so such a queryset keeps working and nothing is loaded twice.

The list scaling tests no longer leave dojo_findingvulnerabilityreference out of the count, give each finding two vulnerability ids, check their order in the response, and cover a queryset that already prefetches them.

Fixes #16652

Claude-Session: https://claude.ai/code/session_0125HXBmURB5ZfBkH3T4LtME

⚠️ Pre-Approval check ⚠️

We don't want to waste your time, so if you're unsure whether your hypothetical enhancement meets the criteria for approval, please file an issue to get pre-approval before beginning work on a PR.
Learn more here: https://github.com/DefectDojo/django-DefectDojo/blob/master/readme-docs/CONTRIBUTING.md#submission-pre-approval

Description

Describe the feature / bug fix implemented by this PR.
If this is a new parser, the parser guide may be worth (re)reading.

Test results

Ideally you extend the test suite in tests/ and dojo/unittests to cover the changed in this PR.
Alternatively, describe what you have and haven't tested.

Documentation

Please update any documentation when needed in the documentation folder)

Checklist

This checklist is for your information.

  • Make sure to rebase your PR against the very latest dev.
  • Submit all PRs, features and bug fixes alike, against the dev branch.
  • Give a meaningful name to your PR, as it may end up being used in the release notes.
  • Your code is Ruff compliant (see ruff.toml).
  • Your code is python 3.13 compliant.
  • If this is a new feature and not a bug fix, you've included the proper documentation in the docs at https://github.com/DefectDojo/django-DefectDojo/tree/dev/docs as part of this PR.
  • Model changes must include the necessary migrations in the dojo/db_migrations folder.
  • Add applicable tests to the unit tests.
  • Add the proper label to categorize your PR.

Extra information

Please clear everything below when submitting your pull request, it's here purely for your information.

Moderators: Labels currently accepted for PRs:

  • Import Scans (for new scanners/importers)
  • enhancement
  • performance
  • feature
  • bugfix
  • maintenance (a.k.a chores)
  • dependencies
  • New Migration (when the PR introduces a DB migration)
  • settings_changes (when the PR introduces changes or new settings in settings.dist.py)

Contributors: Git Tips

Rebase on dev branch

If the dev branch has changed since you started working on it, please rebase your work after the current dev.

On your working branch mybranch:

git rebase dev mybranch

In case of conflict:

 git mergetool
 git rebase --continue

When everything's fine on your local branch, force push to your myOrigin remote:

git push myOrigin --force-with-lease

To cancel everything:

git rebase --abort

Squashing commits

git rebase -i origin/dev
  • Replace pick by fixup on the commits you want squashed out
  • Replace pick by reword on the first commit if you want to change the commit message
  • Save the file and quit your editor

Force push to your myOrigin remote:

git push myOrigin --force-with-lease

GET /api/v2/findings/ ran one dojo_findingvulnerabilityreference query per
finding on the page, because vulnerability_ids is serialized on every row
and FindingViewSet's queryset doesn't prefetch it.

Adding vulnerability_id_prefetch() to get_queryset() would break a subclass
or plugin that adds its own (Django raises "lookup was already seen with a
different queryset"), which is why DefectDojo#16512 left it out. Instead,
FindingViewSet.paginate_queryset() loads the references for the page with
prefetch_related_objects(), skipping findings whose references are already
prefetched, so such a queryset keeps working and nothing is loaded twice.

The list scaling tests no longer leave dojo_findingvulnerabilityreference
out of the count, give each finding two vulnerability ids, check their
order in the response, and cover a queryset that already prefetches them.

Fixes DefectDojo#16652

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0125HXBmURB5ZfBkH3T4LtME

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant