Repository navigation
Add opt-in low-memory XLSX conversion for .NET - #207
Conversation
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
Caution Review failedThe pull request is closed. Walkthrough
Merge Risk:
|
There was a problem hiding this comment.
🟡 Changes recommended
Temporary files can be created with world-readable permissions on shared Unix hosts, exposing worksheet text and generated page content to other local accounts.
7 open issues
Create temporary files with owner-only permissions · New Reuse column-default lookup across recursive row reads · New Validate fit-to-height settings before filtering sheets · New Resolve worksheet parts through workbook relationships · New Match print-area names case-insensitively · New Reject extensions inside consumed row subtrees · New Avoid retaining all text blocks during font diagnostics · New
What changed in this PR
Adds opt-in disk-backed XLSX conversion to reduce .NET memory usage while preserving the default conversion path and reusing existing rendering logic.
Changes:
- Adds the
LowMemoryAPI option and CLI--low-memoryflag. - Stores worksheet rows and PDF pages on disk, generates content page by page, and rejects unsupported features.
- Adds compatibility documentation, regression tests, and an isolated-process memory benchmark.
| File | Description |
|---|---|
tests/MiniPdf.Tests/LowMemoryXlsxTests.cs |
Tests output equivalence, restrictions, stream handling, and cleanup. |
src/MiniPdf/PdfWriter.cs |
Generates low-memory PDF content one page at a time. |
src/MiniPdf/PdfDocument.cs |
Adds disposable, disk-backed page storage. |
src/MiniPdf/MiniPdf.cs |
Exposes LowMemory and spools non-seekable inputs to disk. |
src/MiniPdf/LowMemoryWorkbook.cs |
Extracts worksheet rows and validates unsupported features. |
src/MiniPdf/FileBackedList.cs |
Implements temporary storage and row/page serialization. |
src/MiniPdf/ExcelToPdfConverter.cs |
Integrates low-memory storage, validation, and resource cleanup. |
src/MiniPdf/ExcelReader.cs |
Supports disk-backed rows and shared strings. |
src/MiniPdf.Cli/Program.cs |
Adds --low-memory parsing and help text. |
scripts/Measure-DotNet-XlsxMemory.ps1 |
Compares memory usage, timing, and PDF hashes. |
documents/README.nuget.md |
Documents API usage, compatibility limits, and memory tradeoffs. |
documents/README.nuget.cli.md |
Documents the CLI flag and links to limitations. |
Review effort: Balanced
| internal static FileStream CreateTemporaryStream() | ||
| => new(Path.Combine(Path.GetTempPath(), "minipdf-" + Guid.NewGuid().ToString("N") + ".tmp"), | ||
| FileMode.CreateNew, FileAccess.ReadWrite, FileShare.None, 65536, FileOptions.DeleteOnClose); |
| storedRows.Add(ReadSheet(entry, sharedStrings, boldPrefixLengths, fontStyles, fillColors, borders, | ||
| numberFormats, cellXfFontIndices, cellXfFillIndices, cellXfNumFmtIds, cellXfAlignments, | ||
| cellXfVerticalAlignments, cellXfTextRotations, cellXfBorderIndices, cellXfWrapTexts, | ||
| cellXfIndents, hyperlinks, culture, worksheet: rowDocument)[0]); |
| if (options.LowMemory) | ||
| foreach (var sheet in sheets) | ||
| if (sheet.FitToPage && sheet.FitToHeight > 0) | ||
| LowMemoryWorkbook.Unsupported("fit-to-height pagination"); |
| if (entry.FullName.StartsWith("xl/worksheets/", StringComparison.Ordinal) && | ||
| entry.FullName.EndsWith(".xml", StringComparison.Ordinal) && | ||
| !entry.FullName.Contains("/_rels/")) |
| if (value is "_xlnm.Print_Area" or "_xlnm.Print_Titles") | ||
| Unsupported(value); |
| if (row.Descendants().Any(element => element.Name.LocalName == "f")) | ||
| Unsupported("formulas"); |
| var contentStreams = new List<byte[]>(document.LowMemory ? 0 : pageCount); | ||
| if (!document.LowMemory) | ||
| for (var i = 0; i < pageCount; i++) | ||
| contentStreams.Add(Compat.Latin1.GetBytes(BuildContentStream(pages[i], embeddedFonts.Count > 0, cpToFontSlot, embeddedFonts, fontNameToSlot))); |


Summary
Add opt-in .NET XLSX low-memory conversion for large, simple tables without changing the default conversion path.
MiniPdfConversionOptions.LowMemoryand the CLI--low-memoryflag.The initial scope excludes formulas, merged cells, conditional formatting, drawings/charts/images, Excel tables, hyperlinks, headers/footers, worksheet extensions, custom/hidden row heights, vertical centering, print areas/titles and fit-to-height pagination.
MaxRows,MaxColumnsandRowsPerPageare also unsupported in this mode. Worksheet restrictions apply to the entire workbook. Existing behavior is unchanged whenLowMemoryis omitted.Memory is bounded by the decoded row/page buffers plus offset indexes, workbook/style metadata and font resources; this is disk-backed conversion, not constant-memory or disk-free streaming. Byte-array overloads still allocate the complete output.
Validation
Validated on Windows using .NET SDK 10.0.103:
Passed for
netstandard2.0,net462,net6.0,net8.0andnet9.0; existing warnings remain.Passed: 437 tests, zero failures and zero skips. Coverage includes output equivalence, Unicode, styles, wrapping, pagination, stream ownership, unsupported features, row/column limits, output failures and temporary-file cleanup.
Passed: identical PDF SHA-256 hashes and at least 30% peak working-set reduction on the largest case.
The largest case reduced peak working set by 82.8%. These measurements are not a guarantee for every workbook.
Completed the full 191-case classic XLSX benchmark with all 191 candidate PDFs regenerated, no missing Microsoft 365 or LibreOffice reference PDFs, and all 1,538 expected image references verified. Overall average was 0.9578, unchanged from the preserved report; no case score or page count changed. The threshold was explicitly set to zero to retain known pre-existing low-scoring cases, not to claim that every case meets 0.95.
Additional isolated CLI checks across all 191 fixtures verified:
mainat3fa32683.git diff --checkPassed.
Generated validation outputs are excluded from the commit and retained locally under
artifacts/xlsx-low-memory; they have not been uploaded as PR attachments.Checklist
Summary by CodeRabbit