recipe: pymupdf 1.27.2.3 - #114
Open
ndonkoHenri wants to merge 7 commits into
Open
Conversation
… re-pointed toolchain
build.py re-points cc/cxx/ar/strip/ranlib to the installed NDK (NDK_HOME) when
the sysconfigdata-baked paths are absent on the host, and stores those in env[].
But script_vars is built as {**env, **sysconfig_data, ...}, so **sysconfig_data
re-shadows CC/CXX/AR with the original embedded-NDK paths. On the Android 3.14
support tree those are absolute paths into a vendored NDK that isn't present in
CI, so any recipe that passes {CC}/{CXX}/{AR} into a sub-make gets a
non-existent compiler ('clang: not found', make Error 127). Re-assert the env
compiler values into script_vars after the merge so the template vars match the
toolchain forge actually exports. No-op where sysconfigdata paths are already
valid (iOS, Android 3.12).
(cherry picked from commit f436a94)
Adds a recipe for [PyMuPDF](https://pymupdf.readthedocs.io/) 1.27.2.3 — MuPDF behind a Python API: open PDF/XPS/EPUB/CBZ/images, render pages to bitmaps, extract and search text, and compose documents. Requested in [flet#3400](flet-dev/flet#3400) (15 reactions, still open and unanswered). ## Recipe shape Self-contained, and deliberately not a `flet-libmupdf` + consumer chain: PyMuPDF downloads and builds its own MuPDF, then generates the C++ wrapper and the SWIG layer from *those* headers, so an externally-built MuPDF adds a compile without removing a step. A `flet-libmupdf` recipe was written and proven green before that became clear; it is not part of this branch. The hard part is the wrapper codegen, which runs on the build host under crossenv's cross-python. That interpreter reports the target — `platform.system()` is `Android`/`iOS` — so every upstream `== 'Linux'` / `startswith('darwin')` test picks the wrong branch, and libclang is handed no sysroot, so MuPDF's generator falls back to hardcoded 64-bit type sizes. `crossenv-codegen.patch` covers that, the iOS install-name wiring for serious-python #223, and the link flags pipcl drops; both patches explain themselves in their preambles. Also included: the `src/forge/build.py` fix that keeps `script_env`'s `{CC}`/`{CXX}`/`{AR}` in step with the toolchain forge actually exports. `**sysconfig_data` re-shadowed them with paths into an embedded NDK that isn't present in CI, which any recipe passing `{CC}` to a sub-make would trip over — this one does. ## Changes since the earlier prototype - **ZXing is no longer compiled in.** MuPDF's wrapper script appends `barcode=yes` after the recipe's own make arguments and the last assignment wins, so `barcode=no` in `meta.yaml` never took effect. No PyMuPDF release exposes a barcode API and `barcode.c` keeps a raising stub, so the patch flips that token: ~16 MB of object code, 2 MB off `libmupdf`, 1.2 MB off the wheel, no reachable functionality lost. - **`_extra` gets the iOS deployment target.** pipcl links it from its own flag list and never reads `$LDFLAGS`, so it carried a legacy `LC_VERSION_MIN_IPHONEOS 7.0` while the other three libraries had `LC_BUILD_VERSION` / `minos 13.0`. Same cause as the Android 16 KB alignment flag, fixed the same way. - **`pipcl` is pinned.** PyMuPDF requires it unpinned; it is the build backend *and* the linker, the patch monkeypatches it, and it shipped twelve releases in four months. - **The libclang `-target` is now mandatory**, not best-effort — unset, it silently produced host type sizes for a 32-bit parse. - **Tests: 2 → 14**, and both patches gained the explanatory preamble the repo convention asks for. Stays on 1.27.2.3 rather than 1.28.2: 1.28 rewrote `setup.py` around pipcl's API (five of eight hunks reject), removed `PYMUPDF_SETUP_FLAVOUR` so the dev headers would ship unconditionally, and vendors `cmark-gfm`, an unproven C dependency for these five slices. Worth doing as its own change. ## Validation All six slices build locally. On-device 12/12 on an Android arm64 emulator and on an iOS 18.6 simulator — covering rendering to real pixels, the compiled-in base-14 fonts, PNG encoding, search geometry and page surgery, not just import. ## Consumer notes PyMuPDF lets a Flet app render a PDF page to bytes an `ft.Image` can display, pull its text back out with coordinates, and build documents — all on device, with no upload. Two things to know before adding it: iOS needs Flet 0.86+ (the wheel depends on serious-python's dylib relocation), and PyMuPDF does not support concurrent use, so renders belong on one thread behind a lock. Full guidance, the size and feature tables, and the platform notes are in [`recipes/pymupdf/README.md`](recipes/pymupdf/README.md); a runnable app is in [`recipes/pymupdf/examples/render-and-read`](recipes/pymupdf/examples/render-and-read).
…, pipcl pin) [skip ci] Three things this recipe cost a cycle to learn: - forge-ci said never to dispatch `mobile_test_pythons=ALL`. That was true under flet 0.85's cp312-only packager, and stopped being true on 2026-07-14; the pinned PYTHON_BUILD_RELEASE has carried the dc76612 fix since 20260730, so a plain dispatch can now test every leg on device. - local-recipe-testing's loop used `uvx --with flet-cli`, which resolves flet-cli 0.85.2 and rejects `--python-version` outright. Replaced with the form CI uses. - forge-error-catalogue gains three build-time entries: a sub-make flag that never takes effect because upstream appends its own arguments after yours; the one extension whose build backend ignores $LDFLAGS (Android 16 KB alignment, iOS deployment target); and an unpinned build backend in [build-system] requires breaking a build with no change on your side.
…ents [skip ci] Follow-ups from checking the claims against the built wheels rather than the recipe flags: - The OCR and barcode bullets now quote what MuPDF actually answers -- `OCR Disabled in this build` and `Barcode functionality not included`, both verified present in the shipped libmupdf on each platform. - Pixmap memory was understated. Re-measured on a text-filled A4: 1.4 MB of samples at 72 dpi, 5.7 at 144, 24.9 at 300, against PNGs of 14 KB / 248 KB / 522 KB. It grows with the square of the scale, which is the point. - Added the Python-layer comparison against the same-version desktop wheel: the same 13 files, nine byte-identical, and the four that differ are the patched __init__, build metadata and the two SWIG-generated layers. That is the evidence for "upstream's documentation applies unchanged". - The tests paragraph listed coverage that predates the non-PDF-format and encryption tests, and now says what is genuinely untested. - Spelling normalised to the repo's -ise.
…kip ci] Driving the example on an iOS simulator turned a typed `quartz` into `Quarts` before it ever reached search_for, which then reported no hits on a page that plainly contains the word. search_for matches the literal string, so the field now sets autocorrect=False, enable_suggestions=False and capitalization=NONE. Verified on both platforms afterwards: 4 hits, highlighted.
The slider changed the render scale but not the displayed size: the image sits in a BoxFit.CONTAIN container, so every scale was drawn into the same box. Above the 2x default each step was also downscaled back to the same ~770 physical pixels, so the control did nothing you could see. Making it work would have meant sizing the image in logical pixels, scrolling it in both axes, and tracking page.width through an on_resize handler -- a lot of apparatus for a point that does not land on a phone screen. Removed instead, at the maintainer's call. The framing had leaked well past the widget: page 1 of the generated PDF invited you to "Zoom in", page 3 explained that extraction is unaffected by zoom, and vector_page's docstring opened by naming the slider. All of that is now written without it -- the vector page still makes its point, that shapes are page operators and the renderer decides how many pixels each becomes. Rendering is a fixed RENDER_SCALE, and the caption reports what it cost.
…e var
Follow-up to the {CC}/{CXX}/{AR} fix, from review. Re-asserting five keys after
the merge treated the symptom I had hit; the same shadowing applies to every
variable forge adjusts. sysconfig_data was merged last, so a recipe template got
python-build's build-time constant instead of the value forge exports -- the
compiler paths, and equally the CFLAGS/CPPFLAGS/LDFLAGS forge extends with SDK,
sysroot and opt/lib search paths.
Ordering env last states the rule once and stops the allowlist growing the next
time forge adjusts something sysconfig also defines.
Measured on pymupdf/android arm64-v8a, diffing the expansions against the
previous build: XCFLAGS is unchanged (forge's include paths were already in
sysconfigdata's copy) and XLDFLAGS gains exactly one entry --
-L.../site-packages/opt/lib, the path a flet-lib* host dep installs into.
Nothing is lost from either. That gap was latent rather than biting: no recipe
in the tree hands {LDFLAGS} to a sub-make and links a flet-lib*, and pymupdf
passes XLDFLAGS explicitly, which is why every slice built green without it.
Also moves scheme_paths above sysconfig_data. That is a no-op -- get_paths()
and get_config_vars() share no keys -- but it keeps the dict ordered from least
to most specific.
Beyond the compiler and flags keys, the only variable whose source changes is
ANDROID_API_LEVEL, which 13 recipes interpolate into CMake arguments. sysconfig
holds int 24 and forge sets str(sdk_version), which cross.py pins to "24", so
both render identically.
The comment now says what is actually true: script_vars is forge's environment,
not the recipe's. The loop below appends to env's LDFLAGS/CFLAGS/CPPFLAGS while
processing script_env, and script_vars is a snapshot taken before that, so a
recipe that both sets one of those and refers to it as {...} sees the value
without its own additions. pymupdf relies on exactly that.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds a recipe for PyMuPDF 1.27.2.3 — MuPDF behind a Python API: open PDF/XPS/EPUB/CBZ/image files, render pages to bitmaps, extract and search text with coordinates, and compose documents. Requested in flet#3400.
Recipe shape
Self-contained, and deliberately not a
flet-libmupdf+ consumer chain. PyMuPDF downloads and builds its own MuPDF, then generates the C++ wrapper and the SWIG layer from those exact headers — so a separately-built MuPDF adds a compile without removing a step. Aflet-libmupdfrecipe was written and proven green before that became clear; it is not part of this branch. The wheel therefore ships four interdependent native libraries:libmupdf,libmupdfcpp,_mupdfand_extra.The hard part is the wrapper codegen, which runs on the build host under crossenv's cross-python. That interpreter reports the target —
platform.system()isAndroid/iOS— so every upstream== 'Linux'/startswith('darwin')test picks the wrong branch, and libclang is handed no sysroot, so MuPDF's generator falls back to hardcoded 64-bit type sizes. Both patches explain themselves in their preambles; everymeta.yamlsetting is justified in a comment beside it.Changes
recipes/pymupdf/—meta.yaml, two patches, 14 on-device tests,README.md, and a runnable example.src/forge/build.py— required by this recipe.script_varsmergedsysconfig_datalast, so a recipe template got python-build's build-time constant instead of the value forge exports.{CC}expanded to a path inside an embedded NDK absent from the host, which is a hard failure on the Android 3.14 support tree for any recipe handing it to a sub-make— this one does. Orderingenvlast fixes that and the same shadowing of{CFLAGS}/{CPPFLAGS}/{LDFLAGS}..claude/skills/— three failure-catalogue entries and two corrections found while building this (separable; drop the commit if you'd rather keep skills out of a recipe PR).Notable decisions
Stays on 1.27.2.3 rather than 1.28.2: PyMuPDF 1.28 rewrote
setup.pyaroundpipcl's API — five of the eight hunks reject — and removedPYMUPDF_SETUP_FLAVOUR, so the dev headers and static library this recipe drops would ship unconditionally and need a new hunk. MuPDF 1.28 also vendorscmark-gfm, an unproven C dependency for these five slices. The MuPDF-script surgery applies unchanged, so the bump is tractable — but as its own change with its own CI run.ZXing is compiled out: MuPDF's wrapper script appends
barcode=yesafter the recipe's own make arguments and the last assignment wins, sobarcode=noinmeta.yamlnever took effect. No PyMuPDF release exposes a barcode API andbarcode.ckeeps a raising stub, so the patch flips that token: ~16 MB of object code, 2 MB offlibmupdf, 1.2 MB off the wheel, nothing reachable lost.*
pipclis pinned: PyMuPDF requires it unpinned; it is both the PEP 517 backend and the linker for_mupdf/_extra, the patch monkeypatches it, and it shipped twelve releases in four months.