Skip to content

Add GitHub OSV Live V2 Importer Pipeline#1977

Open
michaelehab wants to merge 3 commits intomainfrom
1904-github-osv-live-importer
Open

Add GitHub OSV Live V2 Importer Pipeline#1977
michaelehab wants to merge 3 commits intomainfrom
1904-github-osv-live-importer

Conversation

@michaelehab
Copy link
Copy Markdown
Collaborator

@michaelehab michaelehab commented Aug 18, 2025

@michaelehab michaelehab changed the title Add GitHub OSV Live V2 Importer Pipeline #1904 Add GitHub OSV Live V2 Importer Pipeline Aug 18, 2025
ziadhany
ziadhany requested changes Aug 25, 2025
View reviewed changes
Copy link
Copy Markdown
Collaborator

@ziadhany ziadhany left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@michaelehab Nice work! Just a few nits for your consideration.

Comment thread vulnerabilities/pipelines/v2_importers/github_osv_live_importer.py Outdated
Comment thread vulnerabilities/pipelines/v2_importers/github_osv_live_importer.py Outdated
Comment thread vulnerabilities/pipelines/v2_importers/github_osv_live_importer.py Outdated
Comment thread vulnerabilities/pipelines/v2_importers/github_osv_live_importer.py Outdated
Comment thread vulnerabilities/pipelines/v2_importers/github_osv_live_importer.py Outdated
Comment thread vulnerabilities/pipelines/v2_importers/github_osv_live_importer.py Outdated
Comment thread vulnerabilities/pipelines/v2_importers/github_osv_live_importer.py Outdated
Comment thread vulnerabilities/pipelines/v2_importers/github_osv_live_importer.py Outdated
@michaelehab @ziadhany
Add GitHub OSV Live V2 Importer Pipeline #1904
ba8a920 
* Add GitHub OSV Live V2 Importer

* Add tests for the GitHub OSV Live V2 Importer

* Tested functionally using the Live Evaluation API in #1969

Signed-off-by: Michael Ehab Mikhail <michael.ehab@hotmail.com>
@michaelehab @ziadhany
Use Optional to support Python 3.9 #1904
ec077d7 
Signed-off-by: Michael Ehab Mikhail <michael.ehab@hotmail.com>
@ziadhany
Copy link
Copy Markdown
Collaborator

ziadhany commented Apr 24, 2026 •
edited
Loading

I don't think fetching https://api.osv.dev/v1/query is a good idea and return the full response.
It can pull from multiple data sources, not just https://github.com/github/advisory-database, and it modifies the data in some way by adding upstream repositories and computing affected and fixed versions.

Options:

  • We can fetch the OSV API to get only the GitHub advisory ID, and then fetch the GitHub Advisory Database. However, this would require fetching unnecessary data. ( relying on the OSV API).

  • Use the VulnerableCode database to get the package-related GitHub advisory ID. However, this would require running the GitHub OSV importer continuous and would not provide live data about new vulnerabilities.

@ziadhany
refactor: Refactor the GithubOSVLiveImporterPipeline pipeline and add…
5c4696d 
… a test using real data

Signed-off-by: ziad hany <ziadhany2016@gmail.com>
@ziadhany ziadhany force-pushed the 1904-github-osv-live-importer branch from f666bc1 to 5c4696d Compare April 29, 2026 00:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ziadhany ziadhany ziadhany requested changes

Requested changes must be addressed to merge this pull request.

Assignees

@ziadhany ziadhany

Labels

blocked

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@michaelehab @ziadhany