Skip to content

fix: optimize archive access#10

Open
tiran wants to merge 1 commit intopython-wheel-build:mainfrom
tiran:optimize-archive
Open

fix: optimize archive access#10
tiran wants to merge 1 commit intopython-wheel-build:mainfrom
tiran:optimize-archive

Conversation

@tiran
Copy link
Copy Markdown
Collaborator

@tiran tiran commented Apr 20, 2026

Fix performance issues in analyze_zipmember and analyze_tarmember functions. The elftools ELFFile class performance a lot of seek and small read operations. Seeking in a zipfile and tarfile is slow.

The old implementation takes about 1.56s to analyze Pillow's 12.2.0 wheel for x86_64. The new code takes 0.22s. The performance difference is even more dramatic with Torch. With the new code, it takes less than 2 seconds to analyze a Torch 2.10.0+cpu wheel. The old code never finished in a reasonable amount of time.

Fix performance issues in `analyze_zipmember` and `analyze_tarmember`
functions. The elftools `ELFFile` class performance a lot of seek and
small read operations. Seeking in a zipfile and tarfile is slow.

The old implementation takes about 1.56s to analyze Pillow's 12.2.0 wheel
for x86_64. The new code takes 0.22s. The performance difference is even
more dramatic with Torch. With the new code, it takes less than 2
seconds to analyze a Torch 2.10.0+cpu wheel. The old code never finished
in a reasonable amount of time.

Signed-off-by: Christian Heimes <cheimes@redhat.com>
Copy link
Copy Markdown

@rd4398 rd4398 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good!
The 7x speedup on Pillow and making Torch analysis finish in <2s vs timing out is really impressive

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants