Skip to content

Emoji: Skip ignored tags with attributes in wp_staticize_emoji(). #13918

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
sanjithsarkar wants to merge 8 commits into
base: trunk
Choose a base branch
from
+296 −53
Open

Emoji: Skip ignored tags with attributes in wp_staticize_emoji(). #13918

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
1dd4604
Emoji: Skip ignored tags with attributes in `wp_staticize_emoji()`.
sanjithsarkar Oct 2, 2026
a38c9cb
Tests: Make data provider static and add docblock spacing in `convert…
sanjithsarkar Oct 22, 2026
832f3fe
Merge remote-tracking branch 'upstream/trunk' into fix/66134-staticiz…
sanjithsarkar Nov 11, 2026
8ae84b5
Emoji: Use `WP_HTML_Tag_Processor` to find text in `wp_staticize_emoj…
sanjithsarkar Nov 11, 2026
cae86eb
Formatting: Use `WP_HTML_Tag_Processor` to find text in `convert_smil…
sanjithsarkar Nov 12, 2026
a6f7089
Formatting: Return non-string values unchanged in `convert_smilies()`.
sanjithsarkar Nov 12, 2026
b8749f2
Formatting: Inline the Tag Processor loops and track ignored tags wit…
sanjithsarkar Nov 12, 2026
38ef2a9
Formatting: Close only the nearest matching ignored tag.
sanjithsarkar Nov 12, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
196 changes: 143 additions & 53 deletions src/wp-includes/formatting.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3612,49 +3612,91 @@ function translate_smiley( $matches ) {
function convert_smilies( $text ) {
global $wp_smiliessearch;

if ( ! get_option( 'use_smilies' ) || empty( $wp_smiliessearch ) ) {
if ( ! is_string( $text ) || ! get_option( 'use_smilies' ) || empty( $wp_smiliessearch ) ) {
// Return default text.
return $text;
}

// HTML loop taken from texturize function, could possible be consolidated.
$textarr = preg_split( '/(<[^>]*>)/U', $text, -1, PREG_SPLIT_DELIM_CAPTURE ); // Capture the tags as well as in between.

if ( false === $textarr ) {
// Return default text.
return $text;
// Without any markup the whole text is a single text node.
if ( ! str_contains( $text, '<' ) ) {
return preg_replace_callback( $wp_smiliessearch, 'translate_smiley', $text ) ?? $text;
}

// Loop stuff.
$stop = count( $textarr );
// Anonymous subclass of WP_HTML_Tag_Processor which exposes underlying bookmark spans.
$processor = new class( $text ) extends WP_HTML_Tag_Processor {
/**
* Gets the span for the current token.
*
* @return WP_HTML_Span Current token span.
*/
public function get_span(): WP_HTML_Span {
// Always called while matched on a token, so setting the bookmark never fails.
$this->set_bookmark( 'here' );
return $this->bookmarks['here'];
}
};

/*
* The smilies are replaced by raw HTML, which set_modifiable_text() would
* escape, so the output is assembled here from the unchanged parts of the
* text and the converted text nodes.
*/
$output = '';
$was_at = 0;

// Ignore processing of specific tags.
$tags_to_ignore = 'code|pre|style|script|textarea';
$ignore_block_element = '';
/*
* Smilies are not converted inside CODE and PRE elements. The Tag Processor
* does not track document structure, so the open CODE and PRE elements are
* tracked here. Text is ignored for as long as any of them is open.
*/
$ignored_tags = array();

for ( $i = 0; $i < $stop; $i++ ) {
$content = $textarr[ $i ];
while ( $processor->next_token() ) {
$token_type = $processor->get_token_type();

// If we're in an ignore block, wait until we find its closing tag.
if ( '' === $ignore_block_element && preg_match( '/^<(' . $tags_to_ignore . ')[^>]*>/', $content, $matches ) ) {
$ignore_block_element = $matches[1];
if ( '#tag' === $token_type ) {
$tag_name = $processor->get_tag();
$is_closer = $processor->is_tag_closer();

// A closing tag is unexpected when no CODE or PRE element is open.
if ( ( 'CODE' !== $tag_name && 'PRE' !== $tag_name ) || ( $is_closer && 0 === count( $ignored_tags ) ) ) {
continue;
}

if ( ! $is_closer ) {
$ignored_tags[] = $tag_name;
continue;
}

/*
* A closing tag closes the nearest open element of its own name and
* leaves the others open. A CODE element which is still open when its
* containing PRE closes continues into the text which follows.
*/
$opener_at = array_search( $tag_name, array_reverse( $ignored_tags, true ), true );

@dmsnell dmsnell Nov 12, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I may have misspoken, or at least have been too terse. this is also not true.

keep in mind that we’re only examining selected elements, which actually is not a great way of parsing HTML. it’s more or less okay because a failure mode is failing to handle emoji.

if we are going to use the Tag Processor, and assume things about the markup, a closing tag should match the last item on the array of open ignored elements.

if ( false !== $opener_at ) {
array_splice( $ignored_tags, $opener_at, 1 );
}
continue;
}

// If it's not a tag and not in ignore block.
if ( '' === $ignore_block_element && strlen( $content ) > 0 && '<' !== $content[0] ) {
$content = preg_replace_callback( $wp_smiliessearch, 'translate_smiley', $content );
if ( '#text' !== $token_type || count( $ignored_tags ) > 0 ) {
continue;
}

// Did we exit ignore block?
if ( '' !== $ignore_block_element && '</' . $ignore_block_element . '>' === $content ) {
$ignore_block_element = '';
$span = $processor->get_span();
$content = substr( $text, $span->start, $span->length );
$converted = preg_replace_callback( $wp_smiliessearch, 'translate_smiley', $content );

if ( null === $converted || $converted === $content ) {

@dmsnell dmsnell Nov 12, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good job skipping the update if the preg_ call didn’t change it.

continue;
}

$output .= $content;
$output .= substr( $text, $was_at, $span->start - $was_at ) . $converted;
$was_at = $span->start + $span->length;
}

return $output;
return $output . substr( $text, $was_at );
}

/**
Expand Down Expand Up @@ -6228,52 +6270,100 @@ function wp_staticize_emoji( $text ) {
/** This filter is documented in wp-includes/formatting.php */
$ext = apply_filters( 'emoji_ext', '.png' );

// Anonymous subclass of WP_HTML_Tag_Processor which exposes underlying bookmark spans.
$processor = new class( $text ) extends WP_HTML_Tag_Processor {
/**
* Gets the span for the current token.
*
* @return WP_HTML_Span Current token span.
*/
public function get_span(): WP_HTML_Span {
// Always called while matched on a token, so setting the bookmark never fails.
$this->set_bookmark( 'here' );
return $this->bookmarks['here'];
}
};

/*
* The emoji are replaced by raw HTML, which set_modifiable_text() would
* escape, so the output is assembled here from the unchanged parts of the
* text and the converted text nodes.
*/
$output = '';
$was_at = 0;

/*
* HTML loop taken from smiley function, which was taken from texturize function.
* It'll never be consolidated.
*
* First, capture the tags as well as in between.
* Emoji are not converted inside CODE and PRE elements. The Tag Processor
* does not track document structure, so the open CODE and PRE elements are
* tracked here. Text is ignored for as long as any of them is open.
*/
$textarr = preg_split( '/(<.*>)/U', $text, -1, PREG_SPLIT_DELIM_CAPTURE );
$stop = count( $textarr );
$ignored_tags = array();

// Ignore processing of specific tags.
$tags_to_ignore = 'code|pre|style|script|textarea';
$ignore_block_element = '';
while ( $processor->next_token() ) {
$token_type = $processor->get_token_type();

for ( $i = 0; $i < $stop; $i++ ) {
$content = $textarr[ $i ];
if ( '#tag' === $token_type ) {
$tag_name = $processor->get_tag();
$is_closer = $processor->is_tag_closer();

// If we're in an ignore block, wait until we find its closing tag.
if ( '' === $ignore_block_element && preg_match( '/^<(' . $tags_to_ignore . ')>/', $content, $matches ) ) {
$ignore_block_element = $matches[1];
// A closing tag is unexpected when no CODE or PRE element is open.
if ( ( 'CODE' !== $tag_name && 'PRE' !== $tag_name ) || ( $is_closer && 0 === count( $ignored_tags ) ) ) {
continue;
}

if ( ! $is_closer ) {
$ignored_tags[] = $tag_name;
continue;
}

/*
* A closing tag closes the nearest open element of its own name and
* leaves the others open. A CODE element which is still open when its
* containing PRE closes continues into the text which follows.
*/
$opener_at = array_search( $tag_name, array_reverse( $ignored_tags, true ), true );
if ( false !== $opener_at ) {
array_splice( $ignored_tags, $opener_at, 1 );
}
continue;
}

// If it's not a tag and not in ignore block.
if ( '' === $ignore_block_element && strlen( $content ) > 0 && '<' !== $content[0] && str_contains( $content, '&#x' ) ) {
foreach ( $possible_emoji as $emojum => $emoji_char ) {
if ( ! str_contains( $content, $emojum ) ) {
continue;
}
if ( '#text' !== $token_type || count( $ignored_tags ) > 0 ) {
continue;
}

$span = $processor->get_span();
$content = substr( $text, $span->start, $span->length );

$file = str_replace( ';&#x', '-', $emojum );
$file = str_replace( array( '&#x', ';' ), '', $file );
if ( ! str_contains( $content, '&#x' ) ) {
continue;
}

$entity = sprintf( '<img src="%s" alt="%s" class="wp-smiley" style="height: 1em; max-height: 1em;" />', $cdn_url . $file . $ext, $emoji_char );
$converted = $content;

$content = str_replace( $emojum, $entity, $content );
foreach ( $possible_emoji as $emojum => $emoji_char ) {
if ( ! str_contains( $converted, $emojum ) ) {
continue;
}

$file = str_replace( ';&#x', '-', $emojum );
$file = str_replace( array( '&#x', ';' ), '', $file );

$entity = sprintf( '<img src="%s" alt="%s" class="wp-smiley" style="height: 1em; max-height: 1em;" />', $cdn_url . $file . $ext, $emoji_char );

$converted = str_replace( $emojum, $entity, $converted );
}

// Did we exit ignore block?
if ( '' !== $ignore_block_element && '</' . $ignore_block_element . '>' === $content ) {
$ignore_block_element = '';
if ( $converted === $content ) {
continue;
}

$output .= $content;
$output .= substr( $text, $was_at, $span->start - $was_at ) . $converted;
$was_at = $span->start + $span->length;
}

$output .= substr( $text, $was_at );

// Finally, remove any stray U+FE0F characters.
$output = str_replace( '&#xfe0f;', '', $output );

Expand Down
70 changes: 70 additions & 0 deletions tests/phpunit/tests/formatting/convertSmilies.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -163,6 +163,49 @@ public static function data_ignore_smilies_in_tags() {
);
}

/**
* Tests where the ignore block starts and ends, by counting converted smilies.
*
* @ticket 66134
*
* @dataProvider data_ignore_smilies_block_boundaries
*
* @param string $input The content to convert.
* @param int $expected The expected number of converted smilies.
*/
public function test_ignore_smilies_block_boundaries( $input, $expected ) {
$this->assertSame( $expected, substr_count( convert_smilies( $input ), "\xf0\x9f\x99\x82" ) );
}

/**
* Data provider.
*
* @return array[]
*/
public static function data_ignore_smilies_block_boundaries() {
return array(
'smiley after closing tag is converted' => array( '<pre class="foo"> :) </pre> :)', 1 ),
'uppercase tags' => array( '<PRE class="foo"> :) </PRE> :)', 1 ),
'whitespace in closing tag' => array( '<pre> :) </pre > :)', 1 ),
'custom element starting with ignored name' => array( '<code-snippet> :) </code-snippet> :)', 2 ),
'unknown element starting with ignored name' => array( '<preview> :) </preview> :)', 2 ),
'element name prefixed by ignored script name' => array( '<script-loader> :) </script-loader> :)', 2 ),
'greater-than sign in attribute value' => array( '<img alt="1 > 0 :) " src="a.png"> :)', 1 ),
'ignored tag with greater-than in attribute' => array( '<pre title="a > b"> :) </pre> :)', 1 ),
'closing tag lookalike in attribute value' => array( '<pre><span title="</pre>"> :) </span></pre> :)', 1 ),
'smiley inside HTML comment' => array( '<!-- 1 > 0 :) --> :)', 1 ),
'ignored tag inside HTML comment' => array( '<!-- > <pre> --> :)', 1 ),
'closing tag inside HTML comment' => array( '<pre><!-- > </pre> --> :) </pre> :)', 1 ),
'nested ignored tags' => array( '<pre><code> :) </code> :) </pre> :)', 1 ),
'same ignored tag nested' => array( '<code><code> :) </code> :) </code> :)', 1 ),
'ignored tag still open after its parent' => array( '<pre><code> :) </pre> :) </code> :)', 1 ),
'ignored tag never closed inside its parent' => array( '<pre><code> :) </pre> :)', 0 ),
'parent still open after ignored tag closes' => array( '<code><pre> :) </code> :) </pre> :)', 1 ),
'unmatched closing tag inside ignored tag' => array( '<pre> :) </code> :) </pre> :)', 1 ),
'less-than sign in text' => array( '1 < 2 :) <p> :) </p>', 2 ),
);
}

/**
* Tests that combinations of smilies separated by a single space
* are converted correctly.
Expand Down Expand Up @@ -373,4 +416,31 @@ public function test_smilies_with_large_text_input() {
$text = '<p><img alt="" src="data:image/png;base64,' . str_repeat( 'iVBORw0KGgoAAAAN', 65536 ) . '="></p> :)';
$this->assertStringContainsString( "\xf0\x9f\x99\x82", convert_smilies( $text ) );
}

/**
* Tests that a non-string value is returned as is, rather than causing a fatal error.
*
* @ticket 66134
*
* @dataProvider data_non_string_input_is_returned_unchanged
*
* @param mixed $input The non-string value to pass.
*/
public function test_non_string_input_is_returned_unchanged( $input ) {
$this->assertSame( $input, convert_smilies( $input ) );
}

/**
* Data provider.
*
* @return array[]
*/
public static function data_non_string_input_is_returned_unchanged() {
return array(
'null' => array( null ),
'false' => array( false ),
'integer' => array( 123 ),
'array' => array( array( ':)' ) ),
);
}
}
Loading
Loading