You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Should Parser.revise(name, suffix=name.suffix) be the identity? (John Smith MD PhD gives MD PhD, and revising with that string gives MD, PhD back) #511
Parser.revise(name, suffix=...) classifies the new value by a full sub-parse of the bare string and then gives every harvested token the named role. Its docstring says the value "is classified ON ITS OWN", and that is the limit this issue is about: a suffix string carries no comma the entry rule can route by, and the sub-parse reads its words as a name, so the entry structure #436 derives from the written text is never derived for a revised field.
The user-visible shape is that feeding a name's own suffix back is not the identity:
p=parse("John Smith MD PhD")
p.suffix# 'MD PhD'Parser().revise(p, suffix=p.suffix).suffix# 'MD, PhD'
Over the 1117 distinct names in tools/differential/corpus*.jsonl, revise(p, suffix=p.suffix).suffix != p.suffix for 38. 36 are a space-joined run coming back comma-joined; the other two are the Ph., D. split and the dash-as-entry above. Before #510 it was 24 of 1116: #510 made more suffix views space-joined, so more of them now differ from what the sub-parse reconstructs. The parse side is unchanged — revise never round-tripped these, and the release-log bullet for #436 says so.
Why the sub-parse cannot get it right as it stands
The sub-parse of "MD PhD" is a full pipeline over a two-word string with no comma: MD reads as a given name and PhD as its suffix, so the #436 pass sees one SUFFIX token and joins nothing. revise then forces both tokens to SUFFIX, after the pass has run, and _with_field_tokens re-runs nothing. The joined tag is the persisted form of an entry (mechanisms.md#MARK-DONT-STRIP; _facade.__setstate__ re-derives it from the entry strings on unpickle), and nothing in revise writes it.
Options
Re-derive the entries after the forced roles. Run the parse("John Smith MD PhD").suffix returns "MD, PhD" — a run without a comma renders with one #436 pass (rules.md#R1) over the harvested tokens once they carry the named role, using the sub-parse's own spans and comma offsets. The rule is the same one the main parse applies: a comma between two suffix words parts them, spaces join them. The within-piece heal needs the same treatment or the Ph. D. case stays split. Smallest change; keeps revise's "one string, one field" API.
Accept a list.revise(p, suffix=["MD PhD", "FACS"]), one string per entry, the way suffix_list reads. Makes the entry structure explicit and sidesteps the sub-parse entirely, but widens the API and leaves the string form with the limit.
Document the limit and leave it.revise already says the value is classified on its own; add the entry sentence and a test pinning the current values, so the next person does not rediscover it.
Option 1 is the honest fix if revise is meant to accept what the views produce; option 3 is fine if revise is meant for fresh input rather than round-trips. Either way the Ph., D. case is worth a pin, since it is the v1 fix_phd mechanism failing on the path nobody tested.
Where this is written down
Parser.revise's docstring (the "classified ON ITS OWN" limit), the C1 entry of docs/design/decisions.md (the NOT FIXED clause the #436 bundle added, with the 24 → 38 measurement), and the 2.3.0 release-log bullet for #436.
Rationale
Parser.revise(name, suffix=...)classifies the new value by a full sub-parse of the bare string and then gives every harvested token the named role. Its docstring says the value "is classified ON ITS OWN", and that is the limit this issue is about: a suffix string carries no comma the entry rule can route by, and the sub-parse reads its words as a name, so the entry structure #436 derives from the written text is never derived for a revised field.The user-visible shape is that feeding a name's own suffix back is not the identity:
Nor is the within-piece heal kept:
And a delimiter core comes back as an entry of its own:
Measurement (2026-09-06, master at
330ee55)Over the 1117 distinct names in
tools/differential/corpus*.jsonl,revise(p, suffix=p.suffix).suffix != p.suffixfor 38. 36 are a space-joined run coming back comma-joined; the other two are thePh., D.split and the dash-as-entry above. Before #510 it was 24 of 1116: #510 made more suffix views space-joined, so more of them now differ from what the sub-parse reconstructs. The parse side is unchanged —revisenever round-tripped these, and the release-log bullet for #436 says so.RECOMPUTE:
Why the sub-parse cannot get it right as it stands
The sub-parse of
"MD PhD"is a full pipeline over a two-word string with no comma:MDreads as a given name andPhDas its suffix, so the #436 pass sees one SUFFIX token and joins nothing.revisethen forces both tokens to SUFFIX, after the pass has run, and_with_field_tokensre-runs nothing. Thejoinedtag is the persisted form of an entry (mechanisms.md#MARK-DONT-STRIP;_facade.__setstate__re-derives it from the entry strings on unpickle), and nothing inrevisewrites it.Options
Ph. D.case stays split. Smallest change; keepsrevise's "one string, one field" API.revise(p, suffix=["MD PhD", "FACS"]), one string per entry, the waysuffix_listreads. Makes the entry structure explicit and sidesteps the sub-parse entirely, but widens the API and leaves the string form with the limit.revisealready says the value is classified on its own; add the entry sentence and a test pinning the current values, so the next person does not rediscover it.Option 1 is the honest fix if
reviseis meant to accept what the views produce; option 3 is fine ifreviseis meant for fresh input rather than round-trips. Either way thePh., D.case is worth a pin, since it is the v1fix_phdmechanism failing on the path nobody tested.Where this is written down
Parser.revise's docstring (the "classified ON ITS OWN" limit), theC1entry ofdocs/design/decisions.md(the NOT FIXED clause the #436 bundle added, with the 24 → 38 measurement), and the 2.3.0 release-log bullet for #436.