Skip to content

Should Parser.revise(name, suffix=name.suffix) be the identity? (John Smith MD PhD gives MD PhD, and revising with that string gives MD, PhD back) #511

Description

@derek73

Rationale

Parser.revise(name, suffix=...) classifies the new value by a full sub-parse of the bare string and then gives every harvested token the named role. Its docstring says the value "is classified ON ITS OWN", and that is the limit this issue is about: a suffix string carries no comma the entry rule can route by, and the sub-parse reads its words as a name, so the entry structure #436 derives from the written text is never derived for a revised field.

The user-visible shape is that feeding a name's own suffix back is not the identity:

p = parse("John Smith MD PhD")
p.suffix                                   # 'MD PhD'
Parser().revise(p, suffix=p.suffix).suffix # 'MD, PhD'

Nor is the within-piece heal kept:

p = parse("John Smith, Ph. D.")
Parser().revise(p, suffix=p.suffix).suffix # 'Ph., D.'   (was 'Ph. D.')

And a delimiter core comes back as an entry of its own:

p = parse("Doe, John, MD PhD - FACS Fellow")
Parser().revise(p, suffix=p.suffix).suffix # 'MD, PhD, -, FACS, Fellow'

Measurement (2026-09-06, master at 330ee55)

Over the 1117 distinct names in tools/differential/corpus*.jsonl, revise(p, suffix=p.suffix).suffix != p.suffix for 38. 36 are a space-joined run coming back comma-joined; the other two are the Ph., D. split and the dash-as-entry above. Before #510 it was 24 of 1116: #510 made more suffix views space-joined, so more of them now differ from what the sub-parse reconstructs. The parse side is unchanged — revise never round-tripped these, and the release-log bullet for #436 says so.

RECOMPUTE:

from nameparser import Parser
P = Parser()
bad = [n for n in corpus_names
       if (p := P.parse(n)).suffix
       and P.revise(p, suffix=p.suffix).suffix != p.suffix]

Why the sub-parse cannot get it right as it stands

The sub-parse of "MD PhD" is a full pipeline over a two-word string with no comma: MD reads as a given name and PhD as its suffix, so the #436 pass sees one SUFFIX token and joins nothing. revise then forces both tokens to SUFFIX, after the pass has run, and _with_field_tokens re-runs nothing. The joined tag is the persisted form of an entry (mechanisms.md#MARK-DONT-STRIP; _facade.__setstate__ re-derives it from the entry strings on unpickle), and nothing in revise writes it.

Options

  1. Re-derive the entries after the forced roles. Run the parse("John Smith MD PhD").suffix returns "MD, PhD" — a run without a comma renders with one #436 pass (rules.md#R1) over the harvested tokens once they carry the named role, using the sub-parse's own spans and comma offsets. The rule is the same one the main parse applies: a comma between two suffix words parts them, spaces join them. The within-piece heal needs the same treatment or the Ph. D. case stays split. Smallest change; keeps revise's "one string, one field" API.
  2. Accept a list. revise(p, suffix=["MD PhD", "FACS"]), one string per entry, the way suffix_list reads. Makes the entry structure explicit and sidesteps the sub-parse entirely, but widens the API and leaves the string form with the limit.
  3. Document the limit and leave it. revise already says the value is classified on its own; add the entry sentence and a test pinning the current values, so the next person does not rediscover it.

Option 1 is the honest fix if revise is meant to accept what the views produce; option 3 is fine if revise is meant for fresh input rather than round-trips. Either way the Ph., D. case is worth a pin, since it is the v1 fix_phd mechanism failing on the path nobody tested.

Where this is written down

Parser.revise's docstring (the "classified ON ITS OWN" limit), the C1 entry of docs/design/decisions.md (the NOT FIXED clause the #436 bundle added, with the 24 → 38 measurement), and the 2.3.0 release-log bullet for #436.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions