Skip to content

Fix < in code blocks being escaped twice in safe mode - #738

Open
LuisFigueroaG wants to merge 3 commits into
trentm:masterfrom
LuisFigueroaG:fix/safe-mode-code-block-escaping
Open

LuisFigueroaG wants to merge 3 commits into
trentm:masterfrom
LuisFigueroaG:fix/safe-mode-code-block-escaping

Conversation

@LuisFigueroaG

Copy link
Copy Markdown
Contributor

Fixes #240

In safe mode, a < that looks like the start of a tag (a<b, i<3) is replaced with &lt; early on, before the block gamut runs. When that text is inside a code block, the code block encoding escapes the & again, so the example from the issue renders as:

<pre><code>for (i=0; i&amp;lt;3; i++) {

The blockquote from the original report isn't swallowed any more, but the double escaping is still there with the same input. It also happens in fenced code blocks (without pygments), in code blocks nested in lists, and in Markdown Documentation - Syntax.text when converted in safe mode.

What I changed: the escaped < is now hashed, the same way complete tags are already hashed in safe mode, so code blocks leave it alone and it's restored as &lt; at the end. Header ids unescape it before slugifying, so # Comparing a<b keeps its current id.

Tests:

  • New case tm-cases/safe_mode_incomplete_tags_in_code_blocks covers indented, fenced and list-nested code blocks, a heading with a<b, and a literal &lt; in a code block (which should still come out as &amp;lt;). It fails on master.
  • test.py -- -knownfailure: OK
  • test/test_redos.py: 10 tests, OK
  • pycodestyle lib/markdown2.py: same warning count as master

I also ran every .text file under test/ through master and this branch, with no safe mode, escape and replace, and with and without a large set of extras. The only difference outside email obfuscation (random on every run) was the intended one in the Syntax document.

In safe mode, incomplete tags like the `<3` in `i<3` are swapped for a
literal `&lt;` before the block gamut runs. Code blocks then encode the
`&` again, so `i<3` inside an indented or fenced code block came out as
`i&amp;lt;3`.

Hash the `&lt;` instead, the same way complete tags are hashed, so code
blocks leave it alone. The hash is also added to the code table so header
ids and TOC entries keep using the plain text.

Fixes trentm#240
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Angle Brackets Bug

1 participant