diff --git a/docs/design/decisions.md b/docs/design/decisions.md index 5eed4346..e57afe91 100644 --- a/docs/design/decisions.md +++ b/docs/design/decisions.md @@ -413,17 +413,31 @@ Excluded (the never-given / ambiguous particle line, nameparser/config/particles Open (contested vocabulary memberships — the rule is right, the word's set is questioned; the issue is canonical): [#342](https://github.com/derek73/python-nameparser/issues/342) -rai in SUFFIX_ACRONYMS vs. the South Asian surname · -[#346](https://github.com/derek73/python-nameparser/issues/346) -swami and the renunciate titles absent from the given-name-title set · -[#343](https://github.com/derek73/python-nameparser/issues/343) / -[#344](https://github.com/derek73/python-nameparser/issues/344) -Bengali and Devanagari honorific vocabulary. +rai in SUFFIX_ACRONYMS vs. the South Asian surname. (#346, #343 and +#344 were the other two entries here and were answered together on +2026-09-06 — see #indic-honorifics.) Excluded (SUFFIX_ACRONYMS / SUFFIX_WORDS — the esq dual membership, deliberate; AGENTS.md's gotcha carries the full algebra): - esq is in BOTH sets and must not be "deduplicated". The load-bearing membership is the acronym one (it carries the multi-dot spellings: removing it costs "John Smith E.S.Q." its family name); the word membership is inert as shipped but is what keeps "Esq" matching for a caller who edits suffix_acronyms themselves. esq is the ONLY member of SUFFIX_ACRONYMS ∩ SUFFIX_WORDS — that singleton is why the two sets cannot carry a disjointness assert, which is the standing cost this entry defends. Deliberately no changed-parse count — the count is a property of the measuring grid, not of the code. +### indic-honorifics — the renunciate class and the Indic honorific vocabulary (2026-09-06, #346/#344/#343) + +Closes #346, #344 and #343 as one bundle: each CLASS is decided once, and each script's membership in it is then argued word by word, so the three lists overlap without matching. No parser code moves in it. Four existing forks carry the whole thing — the renunciate fold (rules.md#H1), the spaced trailing suffix path, title vocabulary beating the one-letter-plus-period initial shape, and the unvalidated given_name_titles — and all four were confirmed on the pre-bundle tree with a throwaway override before any wordlist was edited — the recipe is `Parser(lexicon=Lexicon.default().add(given_name_titles={...}, titles={...}, suffix_words={...})).parse(text)`, and the fourth fork is the one a reader will not expect: `Lexicon` deliberately does not validate `given_name_titles` against `titles` (see #given-name-titles), so an entry in the given-name set alone is never recognized as a title at all. In the shipped data that is free, TITLES being GIVEN_NAME_TITLES | {...}. + +- **The renunciate class (#346).** What is decided once is the CLASS — renunciate given-name titles — against rules.md's H Background, which already carried the two criteria: membership in the given-name-title list follows HOW THE TITLE ADDRESSES, and whether an empty family is correct output follows SURNAME RETENTION. Renunciation answers both at once, so "Swami Vivekananda" reads given Vivekananda, family "". Each script's MEMBERSHIP is its own list, and the three overlap rather than coincide: Latin swami/guru/baba/lama (moved out of the TITLES-only block), Devanagari स्वामी/गुरु/बाबा/संत, Bengali স্বামী/শ্রীল/গুরু/বাবা. lama is a Tibetan title whose native script is Tibetan, and the Hindi and Bengali borrowings लामा/লামা are rare enough that no entry was argued for either — "Lama Zopa" folds, "लामा जोपा" and "লামা জোপা" do not. संत (Sant) and শ্রীল (Srila) run the other way: renunciate titles native to those scripts with NO Latin entry, — Latin sant collides with a real given name (the Excluded block below) and srila has no transliteration in use — "संत कबीर" folds, "Sant Kabir" does not. A sweep "harmonizing" the three lists must therefore argue each word on the two criteria, never on symmetry. rabbi and imam address by TITLE rather than by given name, so they fail the first criterion and stay out; the second agrees, those traditions keeping surnames. "Rabbi Cohen" keeps family Cohen and is pinned as the control beside the Swami row. The four Latin entries were already TITLES vocabulary, so the promotion changes only which field the one name word lands in, and C-i's position test was not re-run for them; the known leading-position collision is Guru Dutt, whose given name is Guru — family went from Dutt to "" — accepted as the pre-existing cost of guru being a title at all, and pinned by the row renunciate_title_swallows_a_real_given_name. +- **venerable is left in TITLES only, and is deliberately NOT an Excluded entry.** The traditions split on surname retention — Buddhist monastics drop the family name, Anglican archdeacons and Catholic candidates keep it — so neither answer is right for the single-name case. TITLES has no ambiguous subset and no AmbiguityKind, so, as for king in #v1-xfail-triage and roz in the maiden markers, the only two expressions available are ship and do not ship; this is the third thing, a membership that is right and an OUTPUT that is undecidable, and it stays where it is. +- **Rinpoche is postpositional** ("Sogyal Rinpoche", "Lama Zopa Rinpoche"), so it joins SUFFIX_WORDS and no title set. The Tibetan -la particle is vocative and glued and needs no entry. The Tibetan PREFIX titles (Tulku, Khenpo, Geshe, Kyabje, Jetsun) are out of this bundle — each re-asks the surname-retention question for a different tradition — and are deferred to a follow-up issue (#514). +- **The trailing set ships spaced only.** rinpoche, जी/साहब/साहिब/साहेब/महाराज and সাহেব/বাবু/মহারাজ are SUFFIX_WORDS entries and none is a honorific_tails entry. The standing prohibition below is what generalizes: it was recorded for जी alone and now covers the set, on the same evidence and with the same accepted cost. +- **Bengali বাবু is trailing where Devanagari बाबू is leading** ("অমল বাবু" against a leading बाबू). Different codepoints, so the two cannot interact; the divergence is stated in both wordlist comments so that a sweep "harmonizing" them has to argue with the data module rather than discover the split afterwards. +- **The Latin md history, corrected.** md entered TITLES in 8cd3123 (2015-09-05, #32) as the medical degree written before the name, alongside do, which #296's audit later took out (#vocabulary-collisions). The Muhammad reading is a 2026 reinterpretation, recorded in the case row audit_md_leading_stays_a_title; both readings agree on the field, which is why the entry survived #296's audit as the one deviation from that audit's table. Recorded here so the record stops implying md was added for Muhammad. +- **মোঃ/মো and মোসাঃ/মোছাঃ go into TITLES**, mirroring Latin md. Semantically they are religious name prefixes rather than titles. TITLES is the FUNCTIONAL home because `given` must stay the name the person is addressed by — মোঃ আবদুল করিম is called আবদুল করিম — and the parser has no name-prefix field. The bound-given-name set is the wrong home for the same kind of reason it is the right one for Arabic abdul: abdul is an inseparable half of one name, মোঃ is detachable. Latin Mst waits for the opt-in pack (#345); the most already in TITLES is the English "Most Rev." word and is unrelated. +- **Abbreviation marks and periods.** An entry carrying an abbreviation mark is spelled with it: the Bengali visarga ঃ (U+0983) is a spacing combining mark (Unicode category Mc, not a letter and not punctuation), and `_lexicon._normalize` lowercases and strips edge whitespace and ASCII periods only — no NFC, no NFKC, no casefold anywhere in that module — so the mark is simply never touched and মোঃ reaches the lexicon and matches intact. Recompute with `Lexicon.default().add(titles={"মোঃ"}).titles & {"মোঃ"}`, which is non-empty iff the mark survived, and `unicodedata.category("ঃ")`, which is `Mc`. Period spellings (মো., মোসা., ডা., प्रो., पं.) match the bare stem through the same edge-period strip, so every BENGALI abbreviation carrying the mark ships as two entries, the visarga form and the bare stem (মোঃ and মো, মোসাঃ and মোসা, মোছাঃ and মোছা, মিঃ and মি), while the Devanagari abbreviations carry no such mark and ship as the bare stem alone (प्रो, पं, प्रा, डा); the dotted form is never listed in either script. +- **Vocabulary beats the initial reading.** A one-letter-plus-period Bengali abbreviation is is_initial-shaped and the title entry wins: "ড. মুহাম্মদ ইউনূস" reads title ড., given মুহাম্মদ, family ইউনূস, while "র. কে. নারায়ণ" — real initials with no entry behind them — is unchanged. #343 verified the same fork at 2.1.0; it is re-verified here on the shipped set. This is MEASURED behavior pinned by rows, not a normative rule — no rule in rules.md states that title vocabulary outranks the one-letter-plus-period initial shape, rules.md#H2 speaking only to the bare-initial and the shape-versus-suffix cases — so a future shape veto must re-check "ড. মুহাম্মদ ইউনূস" before it lands. +- **rules.md#H2's Accepted clause keeps its claim and loses its example.** The clause says an unlisted abugida honorific stays a name word, because the shape rule cannot see an unbroken run of letters through combining vowel signs; that is still true. Its example used প্রফেসর, which this bundle lists, so the example is now "প্রকৌশলী. Sen" → given="প্রকৌশলী." — Engineer, a loanword deliberately not shipped. The example is a contract-tier name in corpus_rules.jsonl, so the swap landed in the Bengali commit rather than in the records commit; leaving it would have exited the gate 1 there. +- **Not recorded as excluded:** the Tier 4 candidates — transliterated military ranks, Advocate, Engineer, Marhum, and the kinship forms — are skipped pending a corpus check, not declined, and मां/माता/মা and the hafiz/syed/shahid forms are left alone the same way. An Excluded entry is a standing prohibition, and none of these has been argued to that standard. One of them carries a duty anyway: প্রকৌশলী (Engineer) is rules.md#H2's executable example of an unlisted abugida honorific, so whoever ships it must move the example to another unlisted word first, or the doc test and the contract-tier gate fail together. +- **Measurement (2026-09-06).** Four corpus names move, all under the renunciate fold, all in one direction (the family empties and the given fills): Swami Vivekananda, Guru Nanak, Baba Ramdev and Lama Zopa, every one from the radar-tier corpus_issues.jsonl. Recompute by parsing every name in the tools/differential/corpus*.jsonl glob twice — once with the shipped lexicon, once with `Lexicon.default().remove(given_name_titles={"swami","guru","baba","lama"})` — and diffing the seven name fields. The multi-name spellings are the boundary and do not move: Swami Vivekananda Saraswati and Guru Gobind Singh keep families Saraswati and Singh, rules.md#H1's fold requiring exactly one name word. The Devanagari and Bengali additions move no corpus name at all, the corpora holding no name written in either script apart from rules.md's own H2 example, which this bundle rewrote. The gate's intentional counts rose by four at every baseline; read today's from the `corpus:` line of `uv run python tools/differential/compare.py --baseline X`. + ### cjk-comma-demotion — the script shapes are pure, the wrappers are tolerated (2026-09-01, #469) Closes #469, and continues the corpus-tier arc below rather than standing apart from it: the tier split gave the differential somewhere to WATCH a name without promising it, and this is the first doctrine narrowing to spend that. No parser behavior moves anywhere in it. Counts are this session's and every one is recomputable from the checked-in tree — `wc -l` over tools/differential/, `uv run python tools/differential/build_shapes_corpus.py --coverage`, and one `uv run python tools/differential/compare.py --baseline X` run per baseline, read off its `corpora:` and `corpus:` lines. @@ -516,6 +530,7 @@ Excluded (given-name titles and post-nominals — the 2026-07-19 transliteration - Ottoman post-nominals باشا/بك/أفندي — surname collisions. - Hebrew bare רב (an ordinary word, "many") and בר (Bar is a common modern Israeli given name) — deferred, #269's territory. - Latin sri/shri deliberately absent while Devanagari श्री ships: the transliteration collides (Sri Mulyani), the native script cannot. +- Latin sant and pandit deliberately absent while Devanagari संत and पंडित ship: Sant is a Punjabi given name (Sant Singh) and Pandit a common surname (Vijaya Lakshmi Pandit), so the Latin forms fail #vocabulary-collisions C-i's position test at the position the claim acts on, and TITLES has no ambiguous subset to express a partial answer in. The native spellings carry neither collision. The Latin forms belong, if anywhere, to the opt-in packs of #345 (2026-09-06, #344). Excluded (multi-word vocabulary entries — a set outside #M2's two exceptions matches one written word, so a multi-word entry there is silently inert and warns at configuration; given_name_titles and, since #434, maiden_markers are the exceptions, and they match phrases by different mechanisms): @@ -528,11 +543,17 @@ Excluded (Lexicon.honorific_tails — a glued tail peels only if it could never - 博士 — 田中博士 is Tanaka Hiroshi as readily as Doctor Tanaka. - 殿 — Japanese surnames end in it (鵜殿, 真殿, four-figure populations); peeling it would cut a real family name in two. - 君 — 王君 is a complete Chinese name; its kana spelling くん does peel. -- जी (standing prohibition for #344's implementation) — Banerjee, Mukherjee and Chatterjee end in the substring (बनर्जी/मुखर्जी/चटर्जी), and glued peeling strands a fragment on a bare virama (बनर् + जी). The 殿 criterion, in a non-CJK script — which also shows the criterion is not CJK-specific. +- जी, and with it the whole spaced trailing set the 2026-09-06 Indic bundle shipped — जी, साहब, साहिब, साहेब, महाराज, সাহেব, বাবু, মহারাজ and Latin rinpoche. None is a honorific_tails entry and none may become one. The evidence is जी's and it carries the set: Banerjee, Mukherjee and Chatterjee end in the substring (बनर्जी/मुखर्जी/चटर्जी), and glued peeling strands a fragment on a bare virama (बनर् + जी). The 殿 criterion, in a non-CJK script — which also shows the criterion is not CJK-specific. The accepted cost is that गांधीजी stays unpeeled, and it is the right trade: an unrecognized honorific costs one field, a strand cuts a real family name in two. Both shapes are pinned in tests/v2/test_locales.py. See #indic-honorifics. Excluded (TITLES): - ঠাকুর — a genuine Bengali honorific (lord/master) that is also Tagore, the surname (#343 records it so a wordlist sweep does not ship it). +- ठाकुर — the Devanagari spelling of the entry above, excluded on the same ground and recorded with it so a sweep of one script does not ship the other (2026-09-06, #344). +- कुमारी / কুমারী (Kumari) — a given name and a family name in the very script the honorific would be written in, so #vocabulary-collisions C-i's position test fails at the position the claim acts on. TITLES has no ambiguous subset to express a partial answer in, so under C-i's "under uncertainty, default to ambiguous" the only available answer is do not ship (2026-09-06, #344/#343). +- बेगम / বেগম (Begum) — a trailing name element far more often than a leading title, so the leading claim would be wrong more often than right (2026-09-06, #343/#344). +- शेख / শেখ (Sheikh) — a clan name and a family name in both scripts ("শেখ হাসিনা"). The divergence from Latin is deliberate and is the sri/shri case run backwards: the Latin sheikh/sheik/shaykh/shaikh cluster SHIPS as given-name titles for the Arabic addressing form, while the Indic spellings name the family (2026-09-06, #344/#343). +- आचार्य (Acharya) — a Brahmin surname (2026-09-06, #344). +- राजा / रानी (Raja/Rani) — common given names (2026-09-06, #344). - The trailing-position rule that must NOT be adopted: TITLES holds hundreds of words in no suffix set, at least nineteen of them ordinary English surnames (king, judge, bishop, baron, sheriff, ...), so a blanket "vocabulary outranks position in the trailing slot" reading would turn "Mary Jane King" into title="King" with the family name gone. The leading half of this argument is AGENTS.md's "Dean is deliberately absent" gotcha; this is the trailing half, and it shadows the family name rather than the given (#316). Excluded (Policy.script_orders defaults): Script.KATAKANA is deliberately absent — a pure-katakana token is predominantly a transcribed foreign name kept in its source order, so nothing defaults on it (rule W4's boundary). Noted 2026-08-15: of the three Script-keyed axes, this is the one with no force-a-decision guard (mechanisms.md#FORCE-A-DECISION-TABLE), so a new Script member silently gets no order. diff --git a/docs/design/rules.md b/docs/design/rules.md index 56644205..9d914629 100644 --- a/docs/design/rules.md +++ b/docs/design/rules.md @@ -74,8 +74,9 @@ H2. Rationale: before a name, an abbreviation is almost always a letters, so an abbreviation in a script whose letters carry combining vowel signs (Bengali, Devanagari) never reads as a title by shape — an unlisted abugida honorific stays a name - word, and only vocabulary (#343) can recognize it. - "প্রফেসর. Sen" → given="প্রফেসর." + word, and only vocabulary can recognize it — which is what #343 + and #344 supply for Bengali and Devanagari. + "প্রকৌশলী. Sen" → given="প্রকৌশলী." Accepted: before a family comma the pre-comma text is wholly the family name (C1), so no shape or vocabulary reading makes a title there. diff --git a/docs/locales.rst b/docs/locales.rst index 3ad31fd9..6eaed404 100644 --- a/docs/locales.rst +++ b/docs/locales.rst @@ -14,9 +14,13 @@ What works without a pack -------------------------- Most international names need no pack at all. The default vocabulary -covers five scripts — Latin, Cyrillic, Greek, Arabic and Hebrew, plus -Devanagari titles — so honorifics, conjunctions and name particles in -those scripts are recognized out of the box: +covers seven scripts — Latin, Cyrillic, Greek, Arabic, Hebrew, +Devanagari and Bengali. Honorifics, conjunctions and name particles +written in them are recognized out of the box, as far as each script's +vocabulary reaches: all seven ship honorifics, while conjunctions and +particles reach fewer — Cyrillic and Greek ship conjunctions and no +particle, Hebrew particles and no conjunction, Devanagari and Bengali +honorifics only. .. doctest:: diff --git a/docs/release_log.rst b/docs/release_log.rst index 6a5b7779..b5241b03 100644 --- a/docs/release_log.rst +++ b/docs/release_log.rst @@ -14,6 +14,16 @@ Release Log - **Fix Parser.revise() splitting a space-separated suffix value into comma-separated entries.** ``Parser().revise(n, suffix="MD PhD").suffix`` is ``MD PhD`` and was ``MD, PhD``, and a name's own rendered suffix now revises back to itself for all but one of the 38 differential-corpus names that failed to on 2026-09-06 (368 of the 1117 carry a suffix) (recipe in the ``C1`` entry of ``docs/design/decisions.md``; the one left is a Korean honorific glued to an initial, which the value's own parse peels where the whole name kept it glued -- a word read on its own, not an entry boundary). A suffix value's entries are now derived from the value's own commas by the same rule a whole name uses: a comma parts two credentials and a space joins them, so ``revise(n, suffix="MD, PhD")`` is still two entries. ``revise(n, suffix="Ph. D.")`` renders ``Ph. D.`` where the 2.2.0 note below accepted ``Ph., D.``; the head-position merge that note describes still does not fire, the pair joining under the entry rule instead. One limit: a delimiter configured through ``extra_suffix_delimiters`` parts a value only where the value's own words read as a name with a tail segment, so in a run of post-nominals it stays a word; write a comma at the boundary instead. ``ParsedName.replace()`` is unchanged (closes #511) + **Additions** + + - **Add the renunciate titles to the given-name title list, so a renunciate's one name is a given name.** ``HumanName("Swami Vivekananda")`` gives first ``Vivekananda`` with an empty last name, where every release since 1.4.0 gave last ``Vivekananda``; ``Guru Nanak``, ``Baba Ramdev`` and ``Lama Zopa`` move the same way, and so do the Devanagari and Bengali spellings added below. Two name words behind the title are unchanged -- ``Swami Vivekananda Saraswati`` keeps last ``Saraswati`` -- and a surname-retaining title is untouched: ``Rabbi Cohen`` still gives last ``Cohen``. ``venerable`` is deliberately not in the list, the traditions using it splitting on whether the family name survives. See the ``indic-honorifics`` entry of ``docs/design/decisions.md`` (closes #346) + + - **Add trailing honorifics as post-nominal vocabulary:** Latin ``rinpoche``, Devanagari ``जी``, ``साहब``, ``साहिब``, ``साहेब``, ``महाराज``, and Bengali ``সাহেব``, ``বাবু``, ``মহারাজ``. ``HumanName("Lama Zopa Rinpoche")`` reads title ``Lama``, first ``Zopa``, suffix ``Rinpoche``; ``नरेन्द्र मोदी जी`` reads last ``मोदी``, suffix ``जी``. They are recognized SPACED only and are deliberately absent from ``Lexicon.honorific_tails``: Banerjee, Mukherjee and Chatterjee end in the ``जी`` substring (``बनर्जी``, ``मुखर्जी``, ``चटर्जी``), so a glued peel would cut a real family name in two, and ``गांधीजी`` staying unpeeled is the accepted cost. Bengali ``বাবু`` is trailing where Devanagari ``बाबू`` is a leading title (#344, #343) + + - **Add Devanagari honorifics (#344):** ``डॉक्टर``, ``डा``, ``प्रो``, ``प्रोफेसर``, ``प्राध्यापक``, ``प्रा``, ``पंडित``, ``पं``, ``सरदार``, ``सुश्री``, ``श्रीयुत``, ``श्रीमान``, ``सौ``, ``बाबू``, ``महात्मा``, ``न्यायमूर्ति``, ``मौलाना``, ``जनाब`` and ``महाराजा`` as titles, beside the ``श्री``/``श्रीमती``/``डॉ`` that shipped in 2.1.0, and ``स्वामी``, ``गुरु``, ``बाबा``, ``संत`` as given-name titles. ``डॉक्टर शर्मा`` reads title ``डॉक्टर``, last ``शर्मा``; ``स्वामी विवेकानंद`` reads first ``विवेकानंद`` with no last name. Dotted spellings (``प्रो.``, ``पं.``) match the same entries. Excluded under the collision rule: ``कुमारी`` (Kumari is a given and a family name), ``बेगम``, ``शेख``, ``आचार्य``, ``राजा``/``रानी`` and ``ठाकुर``, all borne as ordinary names (closes #344) + + - **Add Bengali honorifics -- the first Bengali vocabulary in the default lexicon (#343):** ``ড``, ``ডঃ``, ``ডক্টর``, ``ডাঃ``, ``ডা``, ``ডাক্তার``, ``শ্রী``, ``শ্রীমতী``, ``জনাব``, ``অধ্যাপক``, ``প্রফেসর``, ``বিচারপতি``, ``মাওলানা``, ``মুফতি``, ``আলহাজ্ব``, ``আলহাজ``, ``মিঃ``, ``মি``, ``মিসেস``, ``মোঃ``, ``মো``, ``মোসাঃ``, ``মোসা``, ``মোছাঃ`` and ``মোছা`` as titles, and ``স্বামী``, ``শ্রীল``, ``গুরু``, ``বাবা`` as given-name titles. ``ড. মুহাম্মদ ইউনূস`` reads title ``ড.``, first ``মুহাম্মদ``, last ``ইউনূস`` -- the vocabulary beats the initial reading -- while real initials are untouched: ``র. কে. নারায়ণ`` is unchanged. ``মোঃ আবদুল করিম`` reads title ``মোঃ``, first ``আবদুল``, last ``করিম``, the mirror of Latin ``Md``; the visarga spelling and the ``মো.`` period spelling both match, and the women's ``মোসাঃ``/``মোসা.`` rides the same pair of entries. ``ঠাকুর`` stays out, being Tagore. Latin transliterations (``Sri``, ``Pandit``, ``Mst``) are not added -- they collide with real given names where the native scripts cannot -- and belong to the opt-in packs of #345 (closes #343) + * 2.2.0 - August 31, 2026 nameparser 2.2 is a rename plus about thirty parsing fixes. diff --git a/nameparser/config/suffixes.py b/nameparser/config/suffixes.py index 58ca748e..a27b9fd7 100644 --- a/nameparser/config/suffixes.py +++ b/nameparser/config/suffixes.py @@ -93,6 +93,28 @@ 'さま', # ja the kana spelling of 様 'くん', # ja the kana spelling of 君 'ちゃん', # ja familiar/diminutive + # #346/#344/#343: spaced trailing honorifics, South Asian and + # Tibetan. SPACED ONLY -- none of these is in GLUED_HONORIFICS + # below, and none may be added to it. जी is the reason and the + # reason generalizes: Banerjee, Mukherjee and Chatterjee are + # written बनर्जी, मुखर्जी, चटर्जी, so a glued peel would strand a + # fragment on a bare virama (बनर् + जी) -- the 殿 criterion in a + # non-CJK script. गांधीजी therefore stays unpeeled, which is the + # accepted trade (decisions.md#indic-honorifics). + # NOTE the leading/trailing split across the two scripts, which is + # real and must not be "harmonized": Devanagari बाबू is a LEADING + # honorific -- a TITLES entry (#344), not a suffix -- while Bengali + # বাবু is TRAILING (অমল বাবু). Different codepoints, so the two + # entries cannot interact. + 'rinpoche', # bo postpositional (Sogyal Rinpoche, Lama Zopa Rinpoche) + 'जी', # hi/mr the universal respect particle (मोदी जी) + 'साहब', # hi Sahib + 'साहिब', # hi/pa the Sahib spelling with the i-matra + 'साहेब', # mr the Marathi Saheb spelling + 'महाराज', # hi Maharaj, trailing; the leading महाराजा is a TITLES entry (#344) + 'সাহেব', # bn Saheb (রহমান সাহেব) + 'বাবু', # bn Babu -- TRAILING; see the note above + 'মহারাজ', # bn Maharaj }) """ @@ -158,6 +180,11 @@ (Madono) with four-figure populations, so peeling it would cut a real family name in two. Spaced 殿 is safe for the reason 양/군 are: a 殿-surnamed person's name LEADS, and the suffix gate is trailing-only. +* The Indic trailing set (जी, साहब, साहिब, साहेब, महाराज, সাহেব, বাবু, + মহারাজ) and Latin rinpoche are spaced-only for the same reason 殿 is: + जी ends Banerjee/Mukherjee/Chatterjee (बनर्जी, मुखर्जी, चटर्जी) and a + glued peel would strand बनर् on a bare virama. The criterion is not + CJK-specific, which is the point (decisions.md#indic-honorifics). Three more are in NEITHER set, so neither spelling is recognized. 君: 王君 is a complete Chinese name (君 is a common given-name final), so the honorific diff --git a/nameparser/config/titles.py b/nameparser/config/titles.py index 331d5d52..7f9856f6 100644 --- a/nameparser/config/titles.py +++ b/nameparser/config/titles.py @@ -3,10 +3,27 @@ GIVEN_NAME_TITLES = frozenset({ 'aunt', 'auntie', + # #346: the renunciate class -- 'baba' here, and 'guru', 'lama', + # 'swami' in their own alphabetical places below. All four were in + # the TITLES-only block until 2026-09-06. Two separate criteria + # both say yes (rules.md#H Background): the title addresses by the + # GIVEN name, which is what membership here means, and the + # traditions renounce the surname, which is what makes the empty + # family the right output for "Swami Vivekananda". 'rabbi' and + # 'imam' address by TITLE rather than by given name ('Rabbi + # Cohen'), so they fail the first criterion and stay out, and the + # second agrees, those traditions keeping surnames; + # 'venerable' stays in TITLES only, because the traditions using it + # split on surname retention (Buddhist monastics drop the family + # name, Anglican archdeacons keep it) and this set has no way to + # say "sometimes". + 'baba', 'brother', 'dame', 'father', + 'guru', 'king', + 'lama', 'maid', 'master', 'mother', @@ -14,6 +31,7 @@ 'queen', 'sir', 'sister', + 'swami', 'uncle', 'sheikh', 'sheik', @@ -48,6 +66,31 @@ 'الحاجة', # hajj honorific (f) 'الشيخة', # female counterpart of الشيخ 'مهندس', # engineer (a genuine title in Egyptian usage) + + # #344: Devanagari renunciate titles, the native-script twins of + # baba/guru/swami above. NO Latin twins for संत on purpose. No NEW + # Latin transliterations here: Sri and Sant collide with real + # given names (and Pandit, in the civil block below, with a + # surname) where the native script cannot (the sri/shri + # precedent in the TITLES block below); baba/guru/swami/lama are + # the pre-existing Latin entries above. Being in this literal puts + # them in TITLES too, TITLES being GIVEN_NAME_TITLES | {...}, which + # is what makes them titles at all -- Lexicon does not validate one + # set against the other. + 'स्वामी', # Swami + 'गुरु', # Guru + 'बाबा', # Baba + 'संत', # Sant + + # #343: Bengali renunciate titles. Same two criteria as the Latin + # and Devanagari sets (rules.md#H Background), and no NEW Latin + # transliterations -- srila has none; baba/guru/swami already ship + # above. শ্রীল is the Vaishnava honorific (শ্রীল প্রভুপাদ); + # unprefixed শ্রী is civil and sits in TITLES below. + 'স্বামী', # Swami + 'শ্রীল', # Srila + 'গুরু', # Guru + 'বাবা', # Baba }) """ When these titles appear with a single other name, that name is a given name, e.g. @@ -142,7 +185,6 @@ 'author', 'award-winning', 'ayatollah', - 'baba', 'bailiff', 'ballet', 'bandleader', @@ -355,7 +397,6 @@ 'großfürst', 'group', 'guitarist', - 'guru', 'gyani', 'gysgt', 'hajji', @@ -400,7 +441,6 @@ 'kingdom', 'knowledge', 'lady', - 'lama', 'lamido', 'law', 'lawyer', @@ -664,7 +704,6 @@ 'superior', 'supreme', 'surgeon', - 'swami', 'swordbearer', 'sysselmann', 'tax', @@ -776,6 +815,81 @@ 'श्री', # Shri (Mr.) 'श्रीमती', # Shrimati (Mrs.) 'डॉ', # Dr. abbreviation + # #344: the civil set, same no-NEW-Latin-transliterations rule as + # the three above. + # Excluded under the collision rule and recorded here so a sweep + # does not ship them -- ठाकुर (Tagore, the surname), कुमारी, बेगम, + # शेख, आचार्य, राजा/रानी are all borne as ordinary names. Full + # argument: decisions.md Excluded (TITLES). + 'डॉक्टर', # Doctor, full form + 'डा', # Dr. -- the डा. abbreviation (Nepali and older Hindi), + # edge-period normalized + 'प्रो', # Prof. abbreviation + 'प्रोफेसर', # Professor, full form + 'प्राध्यापक', # Professor (the Sanskritic form) + 'प्रा', # Prof. abbreviation, Marathi + 'पंडित', # Pandit + 'पं', # Pandit abbreviation + 'सरदार', # Sardar + 'सुश्री', # Ms. + 'श्रीयुत', # Shriyut (Mr.) + 'श्रीमान', # Shriman (Mr.) + 'सौ', # Marathi Sau. (Mrs.) + 'बाबू', # Babu -- LEADING in Hindi; Bengali বাবু is trailing + # and lives in suffixes.py. Different codepoints. + 'महात्मा', # Mahatma -- civil, not renunciate: it addresses by + # surname ("महात्मा गांधी"), so TITLES only + 'न्यायमूर्ति', # Justice + 'मौलाना', # Maulana + 'जनाब', # Janab (Mr.) + 'महाराजा', # Maharaja, LEADING (trailing महाराज is a suffix word) + + # #343: Bengali (bn). NO Latin twins, for the reason the + # Devanagari block above gives -- transliterated Sri/Md/Mst + # collide with real given names where the native script cannot. + # No NEW Latin transliterations here (sri, mst); Latin md already + # ships, and for the medical degree + # (decisions.md#indic-honorifics). Doctor abbreviations + # split by profession: ড./ডঃ is the PhD's, ডাঃ/ডা. the Bangladeshi + # physician's. Abbreviation marks are spelled as written -- the + # visarga ঃ (U+0983) is a spacing combining mark, and the lookup + # fold strips only edge periods and whitespace, so it reaches the + # lexicon intact -- while period spellings (মো., ডা.) match the + # bare stem through the edge-period normalization, which is why + # each stem is listed once and the dotted form is not listed at + # all. + # Excluded under the collision rule and recorded here so a sweep + # does not ship them -- ঠাকুর (Tagore, the surname), কুমারী, বেগম + # and শেখ are all borne as ordinary names. Full argument: + # decisions.md Excluded (TITLES). + 'ড', # Dr. abbreviation (PhD) + 'ডঃ', # Dr. abbreviation, visarga spelling + 'ডক্টর', # Doctor, full form + 'ডাঃ', # Dr. abbreviation (physician), visarga spelling + 'ডা', # Dr. abbreviation (physician) + 'ডাক্তার', # Doctor, full form (physician) + 'শ্রী', # Shri (Mr.) + 'শ্রীমতী', # Shrimati (Mrs.) + 'জনাব', # Janab (Mr.) + 'অধ্যাপক', # Professor + 'প্রফেসর', # Professor, borrowed form + 'বিচারপতি', # Justice + 'মাওলানা', # Maulana + 'মুফতি', # Mufti (Latin mufti ships too) + 'আলহাজ্ব', # Alhaj + 'আলহাজ', # Alhaj, the spelling without the ব-phala + 'মিঃ', # Mr., borrowed + 'মি', # the same, period spelling মি. + 'মিসেস', # Mrs., borrowed + 'মোঃ', # Md. (Mohammad) -- a name PREFIX, not a title + # semantically; TITLES is the functional home + # because `given` must stay the name the person is + # addressed by (decisions.md#indic-honorifics) + 'মো', # the same, period spelling মো. + 'মোসাঃ', # Mst. (Mosammat), the women's counterpart + 'মোসা', # the same, period spelling মোসা. + 'মোছাঃ', # the same, ছ spelling + 'মোছা', # the same, period spelling মোছা. } diff --git a/tests/v2/cases.py b/tests/v2/cases.py index 9dddd3e7..650e59b5 100644 --- a/tests/v2/cases.py +++ b/tests/v2/cases.py @@ -2552,6 +2552,65 @@ def _check_cjk_shape_purity(self) -> None: {"family": "Smith", "suffix": "MD"}, classification="fix(#296)", notes="the third leg of 'position decides' for 'md'"), + # -- #346: the renunciate class. A title of religious renunciation + # addresses the GIVEN name AND belongs to a tradition that + # abolishes the surname, so rules.md#H1's given-name-title fold is + # the right reading and family="" is the right output. The two + # criteria are separate (rules.md#H Background) and the control + # row below is what keeps them separate. + Case("renunciate_title_keeps_the_given_name", "Swami Vivekananda", + {"title": "Swami", "given": "Vivekananda"}, + classification="fix(#346)", + notes="pins the #346 membership change the ledger " + "classifies: 'swami' moved from the TITLES-only block " + "into GIVEN_NAME_TITLES, so H1's fold fires and the " + "family is empty where 1.4.0 through 2.2.0 read family " + "'Vivekananda'. The fold itself is already pinned by " + "the Sir John rows; this is the name the release note " + "advertises"), + Case("surname_retaining_title_keeps_the_family", + "Rabbi Cohen", + {"title": "Rabbi", "family": "Cohen"}, + notes="makes rules.md#H Background's 'Rabbi Cohen' sentence " + "executable -- no other row parses a Rabbi. 'rabbi' " + "addresses by title and the tradition keeps surnames, " + "so it stays out of GIVEN_NAME_TITLES and H1 families " + "the one name word. The control that keeps 'addresses " + "by the given name' and 'has no surname' separate"), + Case("renunciate_title_with_a_trailing_honorific", + "Lama Zopa Rinpoche", + {"title": "Lama", "given": "Zopa", "suffix": "Rinpoche"}, + classification="feat(#346)", + notes="the renunciate fold and the spaced trailing honorific " + "together. 'rinpoche' is postpositional in Tibetan " + "usage ('Sogyal Rinpoche'), so it is SUFFIX_WORDS " + "vocabulary rather than a title; that takes the third " + "word out of the name and leaves H1 exactly one given " + "to fold, which is why the family is empty. Without " + "the suffix entry this reads family 'Rinpoche'" + "; classified feat, not fix -- the fold was already " + "pinned by the Swami row, and this row's delta is the " + "rinpoche entry"), + Case("renunciate_title_with_two_name_words_keeps_the_family", + "Swami Vivekananda Saraswati", + {"title": "Swami", "given": "Vivekananda", + "family": "Saraswati"}, + notes="H1's fold requires exactly one name word. The ledger " + "carries this name as a _MUST_NOT_MATCH probe, which " + "pins the regex and not the parse -- and it is " + "radar-tier in corpus_issues.jsonl, so the gate cannot " + "fail on it either. The release note advertises it"), + Case("renunciate_title_swallows_a_real_given_name", "Guru Dutt", + {"title": "Guru", "given": "Dutt"}, + classification="fix(#346)", + notes="the known Latin collision: Guru IS the filmmaker's " + "given name. 'guru' was already a title, so the " + "leading-position swallow predates #346; the promotion " + "into GIVEN_NAME_TITLES is what moved 'Dutt' from " + "family to given. Accepted as the pre-existing cost of " + "'guru' being a title at all " + "(decisions.md#indic-honorifics); TITLES has no " + "ambiguous subset to say otherwise"), Case("audit_do_leading_is_a_name", "Do Nguyen", {"given": "Do", "family": "Nguyen"}, ambiguities=("particle-or-given",), diff --git a/tests/v2/test_ledger_guards.py b/tests/v2/test_ledger_guards.py index d427d4c6..8f945351 100644 --- a/tests/v2/test_ledger_guards.py +++ b/tests/v2/test_ledger_guards.py @@ -883,6 +883,17 @@ def test_case_shape_ids_exist_in_the_inventory() -> None: "fix(#436/#437) the space-separated post-nominal run compounds with the bound-given reserve": ("abdul Smith V", "abdul Smith Jr Ma", "abdul Smith Jr", "Smith Jr V"), + # #346's rule is a literal alternation of four names, so a + # widening reaching only names the corpora lack would leave + # _CORPUS_CLAIMS unmoved. These probes are the wall, and each is a + # name that must not move: two name words after the title (H1 does + # not fold), a surname-retaining title, and the bare renunciate + # title standing alone -- and 'Mr Guru Nanak', which the anchored + # regex rejects but an alternation that lost its left anchor would + # claim (the guard uses re.search). + "fix(#346) a renunciate title and one name word leave the name a given name": + ("Swami Vivekananda Saraswati", "Guru Gobind Singh", + "Rabbi Cohen", "Swami", "Mr Guru Nanak"), } @@ -1508,6 +1519,15 @@ class _LatinCopy(NamedTuple): "The Rt Hon Kenneth Clarke QC MP, HMG", "Washington Jr\\. MD, Franklin", "abdul Smith Jr Ma", "abdul Smith Jr V"}), + # #346's movers, one corpus name per alternative -- a list of + # names, not a copy of any wordlist, so there is no vocabulary for + # it to drift from. The rule's subject is a SHAPE the vocabulary + # participates in (a renunciate title plus exactly one name word), + # and a member copying GIVEN_NAME_TITLES would reach 'Swami + # Vivekananda Saraswati' and 'Guru Gobind Singh', which do not + # move. One set, identical in all four ledgers. + frozenset({"Baba Ramdev", "Guru Nanak", "Lama Zopa", + "Swami Vivekananda"}), # fix(#445)'s movers, one corpus name per alternative -- a list of # names, not a copy of any wordlist, so there is no vocabulary for # it to drift from. Two sets because the ledgers group the nine @@ -1994,6 +2014,12 @@ def _claim(rule: dict) -> _Claim: # change the row here before it reached the gate. "fix(#436/#437) a space-separated post-nominal run renders with spaces, not commas": _Claim(10, ('suffix',), "30f5314a2662", None), + # #346's alternation. Four corpus names, `family` and + # `given` together: the fold moves both roles at once, so a + # widening taking one alone would change the roles here + # before it reached the gate. + "fix(#346) a renunciate title and one name word leave the name a given name": + _Claim(4, ('family', 'given'), "a3399ee7b21e", None), "fix(A2) content-free input names nobody, so every role empties": _Claim(5, ('given',), "1af8d718688b", None), "fix(#335) a marker-led clause leaves the one name word its bare reading": @@ -2202,6 +2228,12 @@ def _claim(rule: dict) -> _Claim: # change the row here before it reached the gate. "fix(#436/#437) a space-separated post-nominal run renders with spaces, not commas": _Claim(10, ('suffix',), "30f5314a2662", None), + # #346's alternation. Four corpus names, `family` and + # `given` together: the fold moves both roles at once, so a + # widening taking one alone would change the roles here + # before it reached the gate. + "fix(#346) a renunciate title and one name word leave the name a given name": + _Claim(4, ('family', 'given'), "a3399ee7b21e", None), # The compound rule, at the two baselines where 'abdul Smith # Jr V' already diffs {family, given} under fix(#401) and the # widened diff leaves that rule's `fields`. Three roles here @@ -2371,6 +2403,12 @@ def _claim(rule: dict) -> _Claim: # change the row here before it reached the gate. "fix(#436/#437) a space-separated post-nominal run renders with spaces, not commas": _Claim(10, ('suffix',), "30f5314a2662", None), + # #346's alternation. Four corpus names, `family` and + # `given` together: the fold moves both roles at once, so a + # widening taking one alone would change the roles here + # before it reached the gate. + "fix(#346) a renunciate title and one name word leave the name a given name": + _Claim(4, ('family', 'given'), "a3399ee7b21e", None), # The four one-name CJK rules, literal-anchored, at the two # baselines where the render is the whole of what moved. A # reach of 1 is one _CORPUS_CLAIMS cannot police on its own -- @@ -2394,6 +2432,12 @@ def _claim(rule: dict) -> _Claim: # change the row here before it reached the gate. "fix(#436/#437) a space-separated post-nominal run renders with spaces, not commas": _Claim(10, ('suffix',), "30f5314a2662", None), + # #346's alternation. Four corpus names, `family` and + # `given` together: the fold moves both roles at once, so a + # widening taking one alone would change the roles here + # before it reached the gate. + "fix(#346) a renunciate title and one name word leave the name a given name": + _Claim(4, ('family', 'given'), "a3399ee7b21e", None), # The four one-name CJK rules, literal-anchored, at the two # baselines where the render is the whole of what moved. A # reach of 1 is one _CORPUS_CLAIMS cannot police on its own -- diff --git a/tests/v2/test_locales.py b/tests/v2/test_locales.py index 64d98f8e..c7f23d42 100644 --- a/tests/v2/test_locales.py +++ b/tests/v2/test_locales.py @@ -1005,7 +1005,8 @@ def test_non_interference_all_packs_combined() -> None: assert declared >= len(all_rotators) -# -- #269: non-Latin default vocabulary (Cyrillic, Greek, Arabic, Hebrew) -- +# -- #269: non-Latin default vocabulary (Cyrillic, Greek, Arabic, +# Hebrew, Devanagari, Bengali) -- # # This is DEFAULT vocabulary (nameparser/config/titles.py, # conjunctions.py, prefixes.py), not a locale pack -- it lives here @@ -1014,6 +1015,10 @@ def test_non_interference_all_packs_combined() -> None: # before the data landed (2026-07-17), so these pin actual observed # behavior, not guesses -- see the per-script comments below for the # rows that came out differently than a first guess would suggest. +# The Devanagari block grew and a Bengali one opened on 2026-09-06 +# with the Indic honorific bundle (#346/#344/#343); those rows were confirmed the +# same way, live against a runtime-augmented Lexicon.default() before +# the data landed. See decisions.md#indic-honorifics. @pytest.mark.parametrize("name, field, expected", [ # Cyrillic (ru/uk) titles. ("г-н Иван Петров", "title", "г-н"), @@ -1165,6 +1170,100 @@ def test_non_interference_all_packs_combined() -> None: ("גב' דוד לוי", "title", "גב'"), ("ד״ר דוד לוי", "title", "ד״ר"), ("גב׳ דוד לוי", "title", "גב׳"), + # Indic trailing honorifics (#344/#343) -- SUFFIX_WORDS, spaced + # only. These rows need none of the Devanagari/Bengali title + # vocabulary; the trailing path reads the suffix set alone. + ("नरेन्द्र मोदी जी", "suffix", "जी"), + ("नरेन्द्र मोदी जी", "family", "मोदी"), + # The glued prohibition, both shapes. जी is deliberately NOT a + # honorific_tails entry: Banerjee/Mukherjee/Chatterjee end in the + # substring and a glued peel strands a fragment on a bare virama + # (बनर् + जी). गांधीजी staying unpeeled is the accepted cost. + ("अमित बनर्जी", "family", "बनर्जी"), + ("अमित बनर्जी", "suffix", ""), + ("बनर्जी", "given", "बनर्जी"), + ("बनर्जी", "suffix", ""), + ("गांधीजी", "given", "गांधीजी"), + ("गांधीजी", "suffix", ""), + # Bengali trailing honorific. One name word plus a suffix reads + # GIVEN, not family -- H1 never fires, there being no title. + ("রহমান সাহেব", "suffix", "সাহেব"), + ("রহমান সাহেব", "given", "রহমান"), + ("রহমান সাহেব", "family", ""), + # বাবু is TRAILING in Bengali where Devanagari बाबू is LEADING. + # Different codepoints, so the two cannot interact -- and a sweep + # "harmonizing" them changes four parses and, without these rows, + # reddens nothing (decisions.md#indic-honorifics). + ("অমল বাবু", "suffix", "বাবু"), + ("অমল বাবু", "given", "অমল"), + ("बाबू अमल", "title", "बाबू"), + ("बाबू अमल", "family", "अमल"), + # Devanagari honorifics (#344). The renunciate set folds like the + # Latin one: स्वामी/गुरु/बाबा/संत are given-name titles, so one + # following name is a GIVEN name and the family is empty. + ("स्वामी विवेकानंद", "title", "स्वामी"), + ("स्वामी विवेकानंद", "given", "विवेकानंद"), + ("स्वामी विवेकानंद", "family", ""), + # Two name words: H1's fold needs exactly one, so the family + # survives. This is the boundary the ledger's probes also pin. + ("स्वामी शिवानंद सरस्वती", "given", "शिवानंद"), + ("स्वामी शिवानंद सरस्वती", "family", "सरस्वती"), + # संत ships and Latin Sant deliberately does not (Sant Singh is a + # Punjabi given name) -- the script asymmetry, executable. + ("संत कबीर", "title", "संत"), + ("संत कबीर", "given", "कबीर"), + ("Sant Kabir", "given", "Sant"), + ("Sant Kabir", "family", "Kabir"), + # The civil control: a TITLES-only Devanagari honorific families + # the one name word, exactly as श्री/डॉ have since 2.0.0. + ("डॉक्टर शर्मा", "title", "डॉक्टर"), + ("डॉक्टर शर्मा", "family", "शर्मा"), + # Bengali honorifics (#343) -- a new #269 script. The renunciate + # set folds as it does in Devanagari and Latin. + ("স্বামী বিবেকানন্দ", "title", "স্বামী"), + ("স্বামী বিবেকানন্দ", "given", "বিবেকানন্দ"), + ("স্বামী বিবেকানন্দ", "family", ""), + # শ্রীল is renunciate and folds the same way; unprefixed শ্রী is + # civil, so it families the one name word instead. + ("শ্রীল প্রভুপাদ", "given", "প্রভুপাদ"), + ("শ্রীল প্রভুপাদ", "family", ""), + ("শ্রী সেন", "title", "শ্রী"), + ("শ্রী সেন", "family", "সেন"), + # Vocabulary beats the initial reading: 'ড.' is is_initial-shaped + # and the title entry wins, which is the fork #343 asked about. + ("ড. মুহাম্মদ ইউনূস", "title", "ড."), + ("ড. মুহাম্মদ ইউনূস", "given", "মুহাম্মদ"), + ("ড. মুহাম্মদ ইউনূস", "family", "ইউনূস"), + # ... and real Bengali initials are untouched, because no entry + # matches them. The negative control for the row above. + ("র. কে. নারায়ণ", "given", "র."), + ("র. কে. নারায়ণ", "middle", "কে."), + ("র. কে. নারায়ণ", "family", "নারায়ণ"), + # The mirror of cases.py's audit_md_leading_stays_a_title. Both + # spellings of the abbreviation ship: the visarga form মোঃ (the visarga ঃ (U+0983) is a spacing + # combining mark, and the lookup fold strips only edge periods and + # whitespace, so it reaches the lexicon intact) and the period + # form মো., which matches the bare stem মো through the edge-period + # normalization. + ("মোঃ আবদুল করিম", "title", "মোঃ"), + ("মোঃ আবদুল করিম", "given", "আবদুল"), + ("মোঃ আবদুল করিম", "family", "করিম"), + ("মো. আবদুল করিম", "title", "মো."), + ("মো. আবদুল করিম", "given", "আবদুল"), + ("মো. আবদুল করিম", "family", "করিম"), + # the women's form, period spelling; the visarga form rides the same entry pattern + ("মোসা. ফাতেমা বেগম", "title", "মোসা."), + ("মোসা. ফাতেমা বেগম", "given", "ফাতেমা"), + ("মি. রহমান", "title", "মি."), + ("মি. রহমান", "family", "রহমান"), + # The exclusion holds: ঠাকুর is Tagore, a surname, and is + # deliberately not a title (decisions.md's Excluded (TITLES)). + # LEADING position is where the exclusion is load-bearing -- + # TITLES is leading-only, so the trailing spelling reads the same + # whether or not the word ships and cannot guard anything. + ("রবীন্দ্রনাথ ঠাকুর", "family", "ঠাকুর"), + ("ঠাকুর রবীন্দ্রনাথ", "given", "ঠাকুর"), + ("ঠাকুর রবীন্দ্রনাথ", "title", ""), ]) def test_269_nonlatin_vocabulary_parses( name: str, field: str, expected: str) -> None: diff --git a/tools/differential/corpus_rules.jsonl b/tools/differential/corpus_rules.jsonl index 79b972cf..894a5e5b 100644 --- a/tools/differential/corpus_rules.jsonl +++ b/tools/differential/corpus_rules.jsonl @@ -223,7 +223,7 @@ "Сидоров Иван Петрович" "Хосе И Мария Сантос" "سلمان، محمد" -"প্রফেসর. Sen" +"প্রকৌশলী. Sen" "マイケル ジャクソン" "マイケル・ジャクソン" "威廉·莎士比亚" diff --git a/tools/differential/expected_since_1.4.0.toml b/tools/differential/expected_since_1.4.0.toml index b64dadb4..9ac500d6 100644 --- a/tools/differential/expected_since_1.4.0.toml +++ b/tools/differential/expected_since_1.4.0.toml @@ -84,6 +84,35 @@ issue = "fix(#436/#437) a space-separated post-nominal run renders with spaces, name_regex = "^(?:JOHN DOE PHD MD|John Doe MD PhD|John Smith MD PhD|John Smith Mc V|Kenneth Clarke QC MP|Smith, John PhD I\\.|The Rt Hon Kenneth Clarke QC MP, HMG|Washington Jr\\. MD, Franklin|abdul Smith Jr Ma|abdul Smith Jr V)$" fields = ["suffix"] +# #346: swami, guru, baba and lama moved from the TITLES-only block +# into GIVEN_NAME_TITLES on 2026-09-06. rules.md#H1's Accepted clause +# -- "a given-name title plus one name word leaves the family empty" +# -- now reaches them, so the one name word behind the title is the +# GIVEN name and the family is empty, where every baseline read it as +# the family. Both roles move together and `fields` declares both: a +# name whose TITLE moved would arrive UNEXPLAINED, as it should. +# +# Four corpus names, all from the radar-tier corpus_issues.jsonl and +# all moving the same way. The two- and three-word spellings are the +# boundary and are _MUST_NOT_MATCH probes: 'Swami Vivekananda +# Saraswati' and 'Guru Gobind Singh' keep their family names, because +# H1's fold requires exactly one name word, and a rule reaching them +# would stand ready to explain the very regression that boundary +# exists to catch. +# +# An anchored alternation of the four NAMES rather than a shape, +# under the ledger discipline expected_since_1.4.0.toml's header +# states for all four files: keep each entry's `name_regex` as tight +# as the diff allows. Four names moved, so a four-name list is the +# tightest expression of that diff; anything shape-shaped is wider +# than what was measured. It copies no wordlist either, which is what +# _NOT_A_VOCABULARY_COPY in tests/v2/test_ledger_guards.py records; +# _CORPUS_CLAIMS pins the reach at 4 with its digest. +[[change]] +issue = "fix(#346) a renunciate title and one name word leave the name a given name" +name_regex = "^(?:Baba Ramdev|Guru Nanak|Lama Zopa|Swami Vivekananda)$" +fields = ["family", "given"] + [[change]] issue = "fix(#271/#272/#298) native-script CJK: family-first order, hangul segmentation, the kana license and the dots" # '毛 泽东', '김민준': script_orders flips first/last for a name written diff --git a/tools/differential/expected_since_2.0.0.toml b/tools/differential/expected_since_2.0.0.toml index de5932a9..d8104140 100644 --- a/tools/differential/expected_since_2.0.0.toml +++ b/tools/differential/expected_since_2.0.0.toml @@ -57,6 +57,35 @@ issue = "fix(#436/#437) a space-separated post-nominal run renders with spaces, name_regex = "^(?:JOHN DOE PHD MD|John Doe MD PhD|John Smith MD PhD|John Smith Mc V|Kenneth Clarke QC MP|Smith, John PhD I\\.|The Rt Hon Kenneth Clarke QC MP, HMG|Washington Jr\\. MD, Franklin|abdul Smith Jr Ma|abdul Smith Jr V)$" fields = ["suffix"] +# #346: swami, guru, baba and lama moved from the TITLES-only block +# into GIVEN_NAME_TITLES on 2026-09-06. rules.md#H1's Accepted clause +# -- "a given-name title plus one name word leaves the family empty" +# -- now reaches them, so the one name word behind the title is the +# GIVEN name and the family is empty, where every baseline read it as +# the family. Both roles move together and `fields` declares both: a +# name whose TITLE moved would arrive UNEXPLAINED, as it should. +# +# Four corpus names, all from the radar-tier corpus_issues.jsonl and +# all moving the same way. The two- and three-word spellings are the +# boundary and are _MUST_NOT_MATCH probes: 'Swami Vivekananda +# Saraswati' and 'Guru Gobind Singh' keep their family names, because +# H1's fold requires exactly one name word, and a rule reaching them +# would stand ready to explain the very regression that boundary +# exists to catch. +# +# An anchored alternation of the four NAMES rather than a shape, +# under the ledger discipline expected_since_1.4.0.toml's header +# states for all four files: keep each entry's `name_regex` as tight +# as the diff allows. Four names moved, so a four-name list is the +# tightest expression of that diff; anything shape-shaped is wider +# than what was measured. It copies no wordlist either, which is what +# _NOT_A_VOCABULARY_COPY in tests/v2/test_ledger_guards.py records; +# _CORPUS_CLAIMS pins the reach at 4 with its digest. +[[change]] +issue = "fix(#346) a renunciate title and one name word leave the name a given name" +name_regex = "^(?:Baba Ramdev|Guru Nanak|Lama Zopa|Swami Vivekananda)$" +fields = ["family", "given"] + [[change]] issue = "fix(#271/#272/#298) native-script CJK: family-first order, hangul segmentation, the kana license and the dots" # The 2.1 East Asian defaults, in the fields they move name pieces diff --git a/tools/differential/expected_since_2.1.0.toml b/tools/differential/expected_since_2.1.0.toml index 03f013aa..63ab7ae3 100644 --- a/tools/differential/expected_since_2.1.0.toml +++ b/tools/differential/expected_since_2.1.0.toml @@ -115,6 +115,35 @@ issue = "fix(#436/#437) the glued honorific and the generational suffix are one name_regex = "^김민준씨 Jr\\.$" fields = ["suffix"] +# #346: swami, guru, baba and lama moved from the TITLES-only block +# into GIVEN_NAME_TITLES on 2026-09-06. rules.md#H1's Accepted clause +# -- "a given-name title plus one name word leaves the family empty" +# -- now reaches them, so the one name word behind the title is the +# GIVEN name and the family is empty, where every baseline read it as +# the family. Both roles move together and `fields` declares both: a +# name whose TITLE moved would arrive UNEXPLAINED, as it should. +# +# Four corpus names, all from the radar-tier corpus_issues.jsonl and +# all moving the same way. The two- and three-word spellings are the +# boundary and are _MUST_NOT_MATCH probes: 'Swami Vivekananda +# Saraswati' and 'Guru Gobind Singh' keep their family names, because +# H1's fold requires exactly one name word, and a rule reaching them +# would stand ready to explain the very regression that boundary +# exists to catch. +# +# An anchored alternation of the four NAMES rather than a shape, +# under the ledger discipline expected_since_1.4.0.toml's header +# states for all four files: keep each entry's `name_regex` as tight +# as the diff allows. Four names moved, so a four-name list is the +# tightest expression of that diff; anything shape-shaped is wider +# than what was measured. It copies no wordlist either, which is what +# _NOT_A_VOCABULARY_COPY in tests/v2/test_ledger_guards.py records; +# _CORPUS_CLAIMS pins the reach at 4 with its digest. +[[change]] +issue = "fix(#346) a renunciate title and one name word leave the name a given name" +name_regex = "^(?:Baba Ramdev|Guru Nanak|Lama Zopa|Swami Vivekananda)$" +fields = ["family", "given"] + [[change]] issue = "fix(#369) a given-name title licenses the bound given-name join with one word to spare" # 'Sheik abdul salam': rules.md#P5 -- "or after a given-name title, diff --git a/tools/differential/expected_since_2.2.0.toml b/tools/differential/expected_since_2.2.0.toml index 22aeb839..72bf3bf2 100644 --- a/tools/differential/expected_since_2.2.0.toml +++ b/tools/differential/expected_since_2.2.0.toml @@ -104,6 +104,35 @@ issue = "fix(#436/#437) the glued honorific and the generational suffix are one name_regex = "^김민준씨 Jr\\.$" fields = ["suffix"] +# #346: swami, guru, baba and lama moved from the TITLES-only block +# into GIVEN_NAME_TITLES on 2026-09-06. rules.md#H1's Accepted clause +# -- "a given-name title plus one name word leaves the family empty" +# -- now reaches them, so the one name word behind the title is the +# GIVEN name and the family is empty, where every baseline read it as +# the family. Both roles move together and `fields` declares both: a +# name whose TITLE moved would arrive UNEXPLAINED, as it should. +# +# Four corpus names, all from the radar-tier corpus_issues.jsonl and +# all moving the same way. The two- and three-word spellings are the +# boundary and are _MUST_NOT_MATCH probes: 'Swami Vivekananda +# Saraswati' and 'Guru Gobind Singh' keep their family names, because +# H1's fold requires exactly one name word, and a rule reaching them +# would stand ready to explain the very regression that boundary +# exists to catch. +# +# An anchored alternation of the four NAMES rather than a shape, +# under the ledger discipline expected_since_1.4.0.toml's header +# states for all four files: keep each entry's `name_regex` as tight +# as the diff allows. Four names moved, so a four-name list is the +# tightest expression of that diff; anything shape-shaped is wider +# than what was measured. It copies no wordlist either, which is what +# _NOT_A_VOCABULARY_COPY in tests/v2/test_ledger_guards.py records; +# _CORPUS_CLAIMS pins the reach at 4 with its digest. +[[change]] +issue = "fix(#346) a renunciate title and one name word leave the name a given name" +name_regex = "^(?:Baba Ramdev|Guru Nanak|Lama Zopa|Swami Vivekananda)$" +fields = ["family", "given"] + [[change]] issue = "fix(#462) the facade keeps an initial-shaped conjunction letter" # 'Scott E. Werner', 'Amy E Maid', 'Jose E. Maria Santos':