Insert_textbox incorrectly splits English words in CJK-mixed text

[yinghao-xue]

When using Page.insert_textbox()to insert mixed Chinese and English text

“Van Dyke等人进行的一项回顾性病例系列研究强调了三份独立病例中Elevess引起的严重急性局部反应和结节性无菌性脓肿的形成。”

the automatic line-breaking logic treats English as words and CJK as characters. This causes premature line breaks after English words. It looks like this:

Van Dyke
等人进行的一项回顾性病例系列研究强调了三份独立病
例中 Elevess
引起的严重急性局部反应和结节性无菌性脓肿的形成。

we tried add or remove space between Chinese and English characters, but none looks normal. How can we achieve proper line breaking?

[JorjMcKie]

You did not share any code. So I do not know which font you were using. What your post looks like to me, it seems that the Chinese text part does not contain spaces.

The algorithm uses the standard space character 0x20 to generate a line break if the subsequent string will not fit in the remaining line. A "word" in the sense used by the algorithm is a string not containing spaces.
If the string is longer than a complete line, it will be broken in fitting parts.

[yinghao-xue]

The font is default cjk font. pymupdf.Font("cjk")
In Chinese layout, characters are not separated by spaces, and line breaks can occur anywhere between characters. Inserting spaces between Chinese characters to force line breaks results in poor typography. Are there alternative solutions for handling mixed CJK-English text?

[JorjMcKie]

We do not do any inspection for the language present in the text or being supported by the font, and we certainly do not plan to do this because of the scope creep.

We might consider adding characters that are considered as line respectively word breaks. A natural candidate I can think of is Zero Width Space (ZWSP), 0x200b. This can be present anywhere in a string without impact on the visual appearance.

[JorjMcKie]

Just had another look:

The problem does not occur in other languages than Chinese and Japanese. So I think there is a way to adjust our existing logic for the common Unicode range of these two languages only.

This appears to be a manageable scope. I will convert this post to an "enhancement" issue.