Survive replay contexts and boxed comments - #23
Open
phucnht wants to merge 1 commit into
Open
Conversation
A change command is protected, so its tokens travel unexecuted into anything LaTeX reads twice: a table of contents, this package's own report file, the running header and footer. Every one of those replays was measured to fail, not assumed: - an id inside a heading was a fatal duplicate-ID error on the second pass, because validation ran again for the replayed copy - one change recorded once per echo in the change report: a heading with one change and a two-page document produced four report lines for it, not one - a heading change's hyperref bookmark carried the raw command tokens concatenated, "OLDHNEWH", because hyperref's string expansion has no definition for \txreplace and drops the command while keeping every argument - a margin or todo comment inside a float, a footnote, or a minipage was a fatal "Float(s) lost", because \marginpar is illegal in a box The fix is a replay depth counter, incremented and decremented by kernel hooks on \@starttoc and \@outputpage, so a change command renders its resolved text instead of markup wherever it replays and skips validation and counting entirely there. Depth rather than a flag, because a page can break inside a long contents listing and nest one replay context inside the other. hyperref gets its own expandable stand-ins, since \pdfstringdefDisableCommands runs the disable list itself rather than executing the document's, and the disable list only \let-remaps onto them, avoiding the hash-doubling that macro parameters inside that list would cause. The margin fallback measures four signals against the actual kernel contexts rather than guessing: \l_tx_boxed_bool set by the same kernel hooks around \@makefntext and \@floatboxreset, \@CapType existing only inside a float environment, \@MPFN reading mpfootnote only inside a minipage, and \mode_if_inner: for a plain restricted box. Falls back to an inline comment with a warning that fires once per document. Report text is scanned for tokens that only work in the context they were written in (math shift, alignment, sub/superscript) and shown as literal source only when one is present; detokenizing unconditionally was tried first and mangles UTF-8 prose under pdfTeX, where accented characters are active and arrive as raw bytes. tests/robust.tex exercises all of it together: a heading with an id, a rejected heading change, inline and display math, a float with a removed span and a boxed comment, a caption, a running header across a page break, and a galley-mode comment that must remain a real margin note. case_robust in scripts/test.sh pins one assertion per finding above. Verified each guard is not vacuous by disabling it in turn: no replay hooks brings back the duplicate-ID failure, no margin-context detection brings back "Float(s) lost", no report-text scan brings back "Missing $ inserted". Restoring each passes again. Full suite green at 23 cases.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
First slice of the roadmap's "Robust and accessible authoring" group, and the current Limitations section: markup in headings, captions, footnotes, floats and math.
Started by writing a failure matrix, not by guessing where the package might struggle: 16 probe documents, one context each, compiled against the unpatched package. Three failed outright and one (headings) silently miscounted and mis-bookmarked without erroring.
What was actually broken, measured before any fix
idinside a heading, caption, or anything else LaTeX reads twice (contents, running headers) is a protected command, so its tokens replay unexecuted and re-validate as a second occurrence of the same ID. Fatal.\txreportlineoccurrences before and after.\txreplaceduring PDF-string expansion, drops the command, keeps every argument: a heading bookmark readOLDHNEWHconcatenated. Read directly out of the PDF's/Titlebytes, not eyeballed.\marginparis illegal inside a float, footnote, or minipage. A margin or todo comment there aborted the document. Reproduced in all three contexts.The fix
A replay-depth counter, incremented/decremented by kernel hooks on
\@starttocand\@outputpage(verified these fire with a throwaway\typeoutprobe before relying on them). Depth rather than a boolean, because a page break inside a long contents listing nests one replay context inside another. Inside a replay: skip ID validation, skip counting and the report line, render resolved text instead of markup.hyperref gets expandable stand-ins
\let-remapped onto the public command names inside its own disable list, rather than trying to feed it macros with parameters (that's a hash-doubling trap in\pdfstringdefDisableCommands).The margin fallback checks four measured signals rather than guessing: a kernel-hook flag for footnotes/floats,
\@captype(exists only in a float),\@mpfn(readsmpfootnoteonly in a minipage),\mode_if_inner:for a plain restricted box. Falls back to an inline comment, warns once per document.Report text: scanned for tokens that only work in the context they were written in (math shift, alignment, sub/superscript), shown as literal source only when one is found. Unconditional detokenizing was tried first and measured to mangle UTF-8 prose under pdfTeX (accented chars are active, arrive as raw bytes) — reverted once that showed up in a probe.
Verification
Every guard proven non-vacuous by disabling it and watching
case_robustfail the specific way it should, then restoring:All 16 original probes recompile clean. Full suite green at 23 cases including the new
robustcase andl3build.