perf: replace lzma-rs with lzma-rust2 and enable zlib-rs backend - #3
Open
maboloshi wants to merge 1 commit into
Open
perf: replace lzma-rs with lzma-rust2 and enable zlib-rs backend#3maboloshi wants to merge 1 commit into
maboloshi wants to merge 1 commit into
Conversation
Switch the LZMA/LZMA2 decoder from lzma-rs to the faster, maintained lzma-rust2 (~1.4x faster on LZMA1 in our benchmarks, still pure Rust), and enable the zlib-rs backend for flate2 instead of the default miniz_oxide. Inno's non-standard 5-byte LZMA1 properties header is padded with the 8-byte unknown-size sentinel (u64::MAX) to form the 13-byte LZMA-Alone header lzma-rust2 expects; the LZMA2 branch uses Lzma2Reader with a 32 MiB dictionary default. lzma-rs is kept as a dev-dependency because the chunk reader tests still build fixtures with lzma_rs::lzma_compress. No public API changes; Error::Decompress mapping preserved. All 79 unit tests pass.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Switch LZMA/LZMA2 decoding from lzma-rs to lzma-rust2, and flate2 to the zlib-rs backend
Motivation
While extracting Inno Setup installers (solid LZMA1/LZMA2 chunks and setup-0
blocks) for a Scoop-like package manager, we measured the pure-Rust
lzma-rsdecoder to be the dominant cost of decompression.
lzma-rust2is amaintained, actively optimized LZMA/LZMA2 decoder that in our benchmarks
(21 MB LZMA1 stream) decodes roughly 1.4x faster than
lzma-rswhileremaining 100% Rust.
Similarly,
flate2's defaultrust_backend(miniz_oxide) is replaced with thenewer, faster zlib-rs backend for
Compression=zlibinstallers.Changes
Cargo.toml: replacelzma-rs = "0.3.0"withlzma-rust2 = { version = "0.18", default-features = false, features = ["std", "optimization"] };switch
flate2todefault-features = false, features = ["zlib-rs"].lzma-rsis kept as a dev-dependency only, because the chunk reader'stests still use
lzma_rs::lzma_compressto build compressed fixtures.src/decompress/block.rs(decompress_inno_lzma1): decode withlzma_rust2::LzmaReader::new_mem_limit+std::io::copyinstead oflzma_rs::lzma_decompress_with_options+UnpackedSize::UseProvided(None).Inno's non-standard 5-byte LZMA1 properties header is padded with the
8-byte unknown-size sentinel (
u64::MAX) to form the 13-byte LZMA-Aloneheader lzma-rust2 expects; the end-of-payload marker terminates the stream.
src/extract/chunk.rs: same swap fordecompress_lzma1, and the LZMA2branch now uses
lzma_rust2::Lzma2Reader(32 MiB dictionary default; Inno's1-byte property prefix is skipped) +
io::copy.Behavior notes
stream (CRC, padding) are left unread — the same data is produced as before.
Error::Decompress) ispreserved.
Compression=lzma/lzma2chunks, setup-0 blocks, solid chunks shared across files) with
tree-identical extraction results, and the existing unit tests pass.