Formulae incorrectly rendered as black boxes

[lucabaldini]

Description of the bug

I am posting this after pisa-conf/easely#66

[Some context: we are developing a small framework that supports the display of conference posters on screen, and in the process we routinely convert poster files from pdf to png.]

Basically PyMuPDF is not rendering the formulae in one of the posters for a conference we support, specifically
https://indico.global/event/14708/contributions/146662/attachments/69943/135661/PO_102.pdf
For this particular file the two equations in the bottom left box appear as black boxes.

How to reproduce the bug

This is the relevant code snippet

def run_pymupdf(input_file_path: PathLike, output_file_path: PathLike,
                target_width: int, page_number: int = 0) -> pathlib.Path:
    """Convert a .pdf file to a .png file using pymupdf under the hood.

    Arguments
    ---------
    input_file_path : PathLike
        The path to the input pdf file.

    output_file_path : PathLike
        The path to the output rasterized (png) file.

    target_width : int
        The target width for the output png file in pixels.

    page_number : int, optional
        The page number to convert (0-indexed).

    Returns
    -------
    pathlib.Path
        The path to the output rasterized (png) file.
    """
    # Open the PDF document and get the first page.
    with fitz.open(input_file_path) as document:
        page = document[page_number]
        # Calculate the zoom factor to achieve the target width.
        page_width = page.rect.width
        zoom_factor = target_width / page_width
        # Render the page as a pixmap with the specified zoom factor.
        pixmap = page.get_pixmap(matrix=fitz.Matrix(zoom_factor, zoom_factor))
        pixmap.save(output_file_path)
    return output_file_path

Link to the actual file: https://github.com/pisa-conf/easely/blob/main/src/easely/pdf.py

PyMuPDF version

1.27.2.3

Operating system

Linux

Python version

3.13

[lucabaldini]

[JorjMcKie]

This is an issue caused by the base software MuPDF. Obviously some of the embedded images (the formulae?) are incorrectly stored or damaged.
Forwarding this to mupdf's issue tracker.
Here is the link: https://bugs.ghostscript.com/show_bug.cgi?id=709424

[julian-smith-artifex-com]

This is fixed in current mupdf branch master.

So this will be fixed in a future pymupdf release, possibly the next release depending on which mupdf it is built with.

[ViktorBarzin]

We hit what looks like the same bug in production (transport e-tickets previewing with a black rectangle where an invisible white placeholder image sits) and isolated a minimal trigger — posting it here in case it's useful as a regression test, and to confirm the mupdf-master fix covers this case too.

An image using an /Indexed colorspace renders all-black when both of these hold:

1. the Indexed base colorspace is an indirect reference: /ColorSpace [/Indexed 6 0 R 1 <000000FFFFFF>], and
2. another image on the same page uses that same colorspace object directly: /ColorSpace 6 0 R.

Either condition alone renders correctly, so it looks like a colorspace-cache interaction. Rewriting the image dict to inline the base ([/Indexed [/ICCBased 7 0 R] 1 <...>]) renders correctly — that's the workaround we shipped. Notably, fitz.Pixmap(doc, xref) extraction decodes the image correctly (all-white) even while page rendering paints it black; and poppler renders the same bytes correctly.

Environment: PyMuPDF 1.27.2.3 (MuPDF 1.27.2), Linux x86_64.

Self-contained reproducer (builds the PDF in memory, renders, prints the offending pixel)

import fitz  # PyMuPDF 1.27.2.3
from PIL import ImageCms  # only to fabricate an ICC profile

icc = ImageCms.ImageCmsProfile(ImageCms.createProfile("sRGB")).tobytes()
bitmap = b"\xff" * 8          # 8x8 @ 1bpc, every bit 1 -> palette index 1 = FFFFFF (white)
rgb = b"\x00\x80\xff" * 4     # 2x2 companion image using the colorspace directly
draw = b"q 10 0 0 10 0 0 cm /Im1 Do Q q 80 0 0 30 10 10 cm /Im0 Do Q"
objs = {
    1: ("<< /Type /Catalog /Pages 2 0 R >>", None),
    2: ("<< /Type /Pages /Kids [3 0 R] /Count 1 >>", None),
    3: ("<< /Type /Page /Parent 2 0 R /MediaBox [0 0 100 50] "
        "/Resources << /XObject << /Im0 5 0 R /Im1 8 0 R >> >> /Contents 4 0 R >>", None),
    4: (f"<< /Length {len(draw)} >>", draw),
    5: ("<< /Type /XObject /Subtype /Image /Width 8 /Height 8 /BitsPerComponent 1 "
        f"/ColorSpace [/Indexed 6 0 R 1 <000000FFFFFF>] /Length {len(bitmap)} >>", bitmap),
    6: ("[/ICCBased 7 0 R]", None),
    7: (f"<< /N 3 /Length {len(icc)} >>", icc),
    8: ("<< /Type /XObject /Subtype /Image /Width 2 /Height 2 /BitsPerComponent 8 "
        f"/ColorSpace 6 0 R /Length {len(rgb)} >>", rgb),
}
out, offsets = bytearray(b"%PDF-1.4\n"), {}
for num in sorted(objs):
    head, data = objs[num]
    offsets[num] = len(out)
    out += f"{num} 0 obj\n{head}\n".encode()
    if data is not None:
        out += b"stream\n" + data + b"\nendstream\n"
    out += b"endobj\n"
xref_pos = len(out)
out += f"xref\n0 {len(objs)+1}\n0000000000 65535 f \n".encode()
for num in sorted(objs):
    out += f"{offsets[num]:010d} 00000 n \n".encode()
out += f"trailer\n<< /Size {len(objs)+1} /Root 1 0 R >>\nstartxref\n{xref_pos}\n%%EOF".encode()

doc = fitz.open(stream=bytes(out), filetype="pdf")
pix = doc[0].get_pixmap(dpi=150)
print("center of the indexed image:", pix.pixel(int(50 * 150 / 72), int(25 * 150 / 72)))
# actual: (0, 0, 0) — expected: (255, 255, 255)
# drop the /Im1 draw (or inline the Indexed base) and it renders white correctly

Happy to open a separate issue with this if you would rather track it independently.

[julian-smith-artifex-com]

Thanks for this, especially for the reproducer. I can confirm that current mupdf master fixes your test.