Allow CJK adjacency in emphasis delimiter runs

[JoshuaKirby88]

Addresses #650.

Problem

CommonMark's punctuation-based flanking rules can leave ordinary CJK emphasis
visible as literal * characters. Japanese and Chinese generally do not add
spaces at these boundaries, and Korean particles attach directly to the text
they follow.

Before and after

Japanese

Markdown:

これは**「重要なこと」**です。

Current CommonMark HTML:

<p>これは**「重要なこと」**です。</p>

Proposed HTML:

<p>これは<strong>「重要なこと」</strong>です。</p>

Chinese

Markdown:

这是**我想做的事。**所以继续。

Current CommonMark HTML:

<p>这是**我想做的事。**所以继续。</p>

Proposed HTML:

<p>这是<strong>我想做的事。</strong>所以继续。</p>

Korean

Markdown:

문서를 작성할 때 **마크다운(Markdown)**을 사용하세요.

Current CommonMark HTML:

<p>문서를 작성할 때 **마크다운(Markdown)**을 사용하세요.</p>

Proposed HTML:

<p>문서를 작성할 때 <strong>마크다운(Markdown)</strong>을 사용하세요.</p>

Proposed rule

This PR defines a CJK character as an assigned Unicode character that meets
either of these conditions:

• its East_Asian_Width property is W, F, or H, excluding characters
  with Emoji_Presentation;
• its Script property is Hangul.

A delimiter run adjacent to such a character can satisfy the punctuation
portion of the left-flanking or right-flanking definition.

The existing whitespace conditions still apply. The existing restrictions on
underscore delimiters also remain unchanged.

Conformance examples

The PR adds nine examples covering Japanese, Chinese, Korean, strong and single
emphasis, ASCII punctuation inside CJK text, and protected behavior involving
ambiguous-width punctuation, underscores, whitespace, and emoji presentation.

Five examples demonstrate newly supported emphasis. Four protect behavior that
must remain unchanged.

Proof implementation

The complete cmark proof is available as an
exact comparison against upstream cmark.
It includes the implementation, regression tests, and a Unicode 17 range
generator that pins and verifies every input file by SHA-256.

The classifier uses binary search over 68 non-overlapping ranges, requiring at
most seven range comparisons for each adjacent code point.

Reproduce the verification

The following starts from public repositories and does not require this branch
to be checked out already:

workdir=$(mktemp -d)

git clone https://github.com/commonmark/commonmark-spec.git "$workdir/spec"
git -C "$workdir/spec" fetch origin pull/839/head:pr-839
git -C "$workdir/spec" switch pr-839

git clone https://github.com/JoshuaKirby88/cmark.git "$workdir/cmark"
git -C "$workdir/cmark" checkout 836a2757facef61aea2dad3cf3573123b3bf1793

python3 "$workdir/cmark/tools/generate_cjk_ranges.py" --check
make -C "$workdir/cmark" test

python3 "$workdir/spec/test/spec_tests.py" \
  --spec "$workdir/spec/spec.txt" \
  --program "$workdir/cmark/build/src/cmark --unsafe"

Expected results:

• the generated Unicode 17 table matches all 68 checked-in ranges;
• all nine native cmark test groups pass;
• all 664 examples in this proposed specification pass.

[Copilot]

Pull request overview

Updates the CommonMark specification to allow emphasis delimiter runs to open/close when adjacent to CJK characters, addressing cases where CJK punctuation/adjacency previously caused literal * to remain visible.

Changes:

• Defines a new “CJK character” based on Unicode East_Asian_Width / Script properties (excluding Emoji_Presentation).
• Extends left-/right-flanking delimiter run definitions to permit a CJK-adjacency alternative when punctuation would otherwise block emphasis.
• Adds conformance examples covering new supported cases and protected non-changes (underscores, whitespace, ambiguous-width punctuation, emoji presentation).

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[jgm]

@tats-u I would be grateful for your review of this.

[tats-u]

This is essentially, to put it nicely, an ultra-simplified version, or to put it bluntly, a poor man's copy of https://github.com/tats-u/markdown-cjk-friendly. It becomes equivalent to that if you remove the support for variation sequences before delimiter runs.

While it is commendable for handling many cases with minimal effort, it is regrettable that the PR description makes no mention of my repository. Also, although cases that cannot be covered by this are rare, Japanese or Chinese people who are knowledgeable about Unicode could easily list them.

The test cases lack coverage for at least the following:

• Delimiter runs sandwiched between non-CJK characters and characters with an East Asian Width of H (e.g., half-width katakana) or F (full-width parentheses, commas, exclamation marks, and question marks)
• Hangul in NFD representation (parts of Hangul whose East Asian Width is N)
• Supplementary characters (characters U+10000 or higher counted as two characters in UTF-16)
• CJK characters that are emoji characters but not default emoji presentation characters

[tats-u]

By the way, do CommonMark maintainers like you consider the change in 0.31.2, where emojis are now treated as punctuation (meaning, for example, a**😀**a is no longer recognized as bold), to be an expected change?