fix: pair code regions by CommonMark rules in markdown preprocessing#228
Open
PathGao wants to merge 1 commit into
Open
fix: pair code regions by CommonMark rules in markdown preprocessing#228PathGao wants to merge 1 commit into
PathGao wants to merge 1 commit into
Conversation
The five source-level rewrites (![[embeds]], [[#wikilinks]], ^block-ids, ==highlight==, ^[inline footnotes]) protected code regions with the regex alternation ```.*?```|`.*?`, which cannot express CommonMark pairing: fences close only on a line-leading run of the same character at least as long as the opener, and a span of N backticks closes only on a run of exactly N. A single 4-backtick inline sample (or any ~~~ fence, which the pattern did not know at all) desynchronized protection for the entire rest of the document — later inline code containing ![[x]] was rewritten into <img> tags, with nonlocal collateral damage in unrelated blocks. Replace the alternation with a small scanner (code_region_ranges) that walks fences and backtick runs by the CommonMark rules, and skip any match that starts inside a protected range. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #227.
The five source-level rewrites (
![[embeds]],[[#wikilinks]],^block-ids,==highlight==,^[inline footnotes]) protected code regions with the alternation(?s)```.*?```|`.*?`|…, which cannot express CommonMark pairing. One 4-backtick inline sample — or any~~~fence, which the pattern did not know at all — desynchronized protection for the entire rest of the document, rewriting later inline code containing![[x]]into<img>tags with nonlocal collateral damage (details and minimal repro in #227).Change: a small scanner
code_region_ranges()computes protected byte ranges by the CommonMark rules —~~~, up to 3 leading spaces) closes only on a line-leading run of the same character at least as long as the opener (unclosed fence protects to EOF; backtick-fence info strings may not contain backticks);Each rewrite pass then skips matches that start inside a protected range. Ranges are recomputed per pass since replacements shift offsets. No new dependencies; rewrite behavior outside code regions is unchanged.
Tests: 8 new unit tests covering both rupture modes (4-backtick desync,
~~~fences), pairing rules (longer-run close, exact-N spans, unclosed fence), and unchanged rewrites outside code.cargo test: 10/10.🤖 Generated with Claude Code