fix(core): one compiled handshake registry, so every partitioner sees a mid-file trigger — closes #2848 - #2943
Merged
Conversation
… a mid-file trigger -- closes #2848 Every handshake trigger is `^`-anchored, so a partitioner that compiles it without re.M only fires on a file whose very FIRST byte opens the block. Three modules compiled the registry independently and the flags had drifted: detector had re.I|re.M, prism and language_lens had re.I alone. In every real file prism's embedded segment was therefore never formed and the HOST language's comment rules ran over the embedded body -- html's <!-- --> rules over a JavaScript //, so the comment stayed in code_stream (counted as code, and subtracted from the derived doc_loc) and its commented-out eval( read as executable risk. language_lens' copy meant _detect_hybrids reported a polyglot lang_mix only for the same degenerate offset-0 file. #1183 fixed the anchor half of this drift in detector; the flag half was never applied. Rather than add re.M in two more places, the compilation moves next to the patterns it compiles (_lens_config.COMPILED_HANDSHAKE_REGISTRY) and all three consumers share it -- compiled patterns are immutable, so this is one list, not three copies, and a test pins the identity so a fourth cannot appear. Measured: rosetta_audit 46/46, 0 regressions -- no planted count moves, so no manifest rebless is owed. Golden masters blessed both legs: 194 substantive diffs in html (101) and groovy (36) only, the Coding/Documentation LOC redistribution the issue predicted (11 html files move 64 JavaScript // lines out of code and into doc; math.html 192/2 -> 171/23, conserved per file). The groovy entry is the boundary case, read in full: gradle's EnrichedReportRenderer emits injected JavaScript from a triple-quoted string (marked // language=javascript), and its 61 lines of real JS logic were being recorded as documentation -- they are code now, with the JS signals that were invisible before. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BLztFhLH42oxWBKEvwfdBb
Contributor
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #2848.
What was wrong (and one thing the issue did not know)
Every handshake trigger is
^-anchored, so compiling it withoutre.Mmeans the partitioner only fires on a file whose very first byte opens the block. The issue found two copies of the registry; there are three, and two were missing the flag:detector.pyre.I | re.Mprism.pyre.Ilanguage_lens.pyre.I_detect_hybridsreported a polyglotlang_mixonly for the degenerate offset-0 fileThe issue's micro-repro, before → after: mid-file
<script>read coding/doc LOC 7/0 with// eval("nope");surviving intocode_streamas executable risk; it now reads 6/1 with the comment incomment_stream, matching the offset-0 case in kind.The fix
The flags have now drifted twice on the same three patterns (#1183 the anchor, this the multiline flag) because three modules each compiled them. So rather than adding
re.Min two more places, the compilation moves next to the patterns as_lens_config.COMPILED_HANDSHAKE_REGISTRY, and all three consumers share it. Compiled patterns are immutable and thread-safe, so this is one list rather than three copies, andtest_every_partitioner_shares_one_compiled_registry_2848pins the identity (not a copy of the flags) so a fourth copy cannot quietly appear.The test that documented this asymmetry as a deliberate workaround (
test_both_partitioners_move_the_same_boundary, from #2549) is now parameterised over both trigger positions instead.Measured
rosetta_audit46/46, 0 regressions. No planted signal count moves, so no manifest rebless is owed — contrary to the issue's expectation of a corpus-wide re-bless. (The manifests gate signal counts; the derived LOC columns live in the bias report, which regenerates on its own schedule.) Reachability scan: rosetta has mid-file triggers in html only.bless_scope: 194 substantive diffs, html (101) and groovy (36) only — theCoding LOC/Documentation LOCredistribution the issue predicted. 11 html files move 64 JavaScript//lines out of code and into doc, conserved per file (math.html192/2 → 171/23,webWorkerExtensionHostIframe.html139/0 → 124/15).EnrichedReportRenderer.groovyemits injected JavaScript from a triple-quoted string (marked// language=javascript). Its ~61 lines of genuine JS logic were recorded as documentation (103 doc LOC for a file with ~10 real comments); they are code now, and the JS signals that were invisible — 8 branches, 13 state mutations, 5 closures — are counted. Worth a reviewer's eye: this is embedded code inside a host string literal, which is exactly the case Residual function-extraction gap post-#1193: contiguous dead zone + near-total loss in embedded-foreign-code-heavy files #1198's line-anchoring was designed to keep honest, and here the content is real injected JavaScript rather than fixture data describing it.test_security_auditorfailures are this venv missing pandas/xgboost — identical on untouched main; CI installs them).audit_checkall clear; ruff baseline clean.🤖 Generated with Claude Code
https://claude.ai/code/session_01BLztFhLH42oxWBKEvwfdBb