From 44456553fef6262a30526fce0821ddf8c7cc7fb3 Mon Sep 17 00:00:00 2001 From: Derek Gulbranson Date: Mon, 7 Sep 2026 20:08:05 -0700 Subject: [PATCH 1/3] =?UTF-8?q?docs(design):=20order=20declarations=20are?= =?UTF-8?q?=20authoritative=20=E2=80=94=20#471=20declined,=20#384=20on=20o?= =?UTF-8?q?ption=201?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A name order is a property of the data source, not of a string: the caller declares it to match how their records are written, and the declaration outranks any string-level inference. rules.md's Name order Background states that once, and two open questions rest on it. #471 DECLINED: a never-given particle marks where a surname BEGINS, so the fold takes the rest of the name: `de la Torre Vega` is a surname-only string read right and `de la Cruz Juan Carlos` is a family-first listing read under the wrong order, whose remedy is the declaration or the family comma. P1's reach paragraph says so and carries both faces as examples. This supersedes the reason the 2026-08-17 P1 entry gave for the same outcome, and answers the #364 entry's "nothing ever argued for takes everything". #384 on option 1: the ru and tr_az patronymic rotations exist to restore the default reading from a family-first listing, so under a declared family-first order they stand down and position decides. The code did not do that on its own -- the rotation reads whatever word sits in the family slot, which under FAMILY_FIRST is the first word -- so the next commit gates it on the order assign read; this entry records the decision and the measured gap. The divergent row (ru + FAMILY_FIRST on natural-order input) is the declaration being honored and is recorded as Accepted, pinned by a tests/v2/test_locales.py row because a rules.md example line takes one annotation, a pack or an order, never both. Options 2 and 3 are Declined with reasons. No parse changes and no release-note bullet. The new P1 example joins the contract corpus (corpus_rules.jsonl, 250 -> 251 names) and, at the 1.4.0 baseline only, the already-recorded initials-view class fix(initials-per-word) a particle chain inside a name part, beside de la Cruz Juan Carlos -- so the 1.4.0 intentional count goes 367 -> 368 and that rule's recorded claim is 108 names with its new digest. Gate: 368 / 263 / 175 / 37, 0 unexplained. Closes #471 Closes #384 Co-Authored-By: Claude Fable 5.1 --- docs/design/decisions.md | 25 +++++++++----- docs/design/rules.md | 47 ++++++++++++++++++++++++--- tests/v2/cases.py | 6 +++- tests/v2/test_ledger_guards.py | 6 ++-- tests/v2/test_locales.py | 32 +++++++++++++++++- tools/differential/corpus_rules.jsonl | 1 + 6 files changed, 101 insertions(+), 16 deletions(-) diff --git a/docs/design/decisions.md b/docs/design/decisions.md index d372d244..37485dbe 100644 --- a/docs/design/decisions.md +++ b/docs/design/decisions.md @@ -56,6 +56,13 @@ the 2026-08-16 entries below. The survivor is the degenerate bare The order is READ, not re-derived: assign now records the order it actually used on ParseState.order, and the fold keys on that. policy.name_order would have been wrong — a script_orders entry can put the family first under a given-first policy, and the roles assign already wrote would then disagree with the roles the fold computes. The same reasoning already appears one function away, where the PARTICLE_OR_GIVEN emitter reads the role off the token rather than assuming given. A unit, not a token: a particle chain (P2), a conjunction join (P3) and a bound given-name pair (P5) each count once. All three are read off the TAGS rather than the pieces, but for two different reasons, and the difference matters to anyone who tries to simplify this. The conjunction join grouping DID build and the prefix chain then swallowed: the name reaches the fold as [de][la Vega y Santos Juan], the ambiguous particle having chained forward over the join, so the JOIN's boundary is gone. The leading particle keeps its own piece, and must — the fold's site test wants a lone piece, so a one-piece name would not fold at all. The bound-given join grouping never built at all: P5 joins only where the bound word is the first non-title piece, and at a fold site the first piece is the particle, so "ibn Awf abdul Rahman" arrives as four separate pieces. Restoring piece boundaries in group would fix the first and silently split the second. It also makes P5's new implemented: pointer stronger than a citation — this is the only place that join happens in this shape. This is what keeps "de la Vega y Santos Juan" from stopping between Vega and Santos — the case rules.md#P3 was amended for on 2026-08-17, now executable. Measured, with the denominator this log requires: of 751 corpus names, SEVEN reach P1's leading fold (DE MESNIL, De Groot, de Mesnil, de Mesnil Garcia, de Mesnil Jr., de la Vega, dos Santos) and exactly ONE of those has anything past the run to lose, so one moves (de Mesnil Garcia) under each family-first order and none under the default. The single mover is a fact about the corpus holding no family-first listing with leftovers, not about the blast radius. The differential harness cannot see any of it — it runs the corpus under the default policy against 1.4.0, which has no name_order — so its exit 0 is evidence for the accepted cost and for nothing else. The verification that counts is the two-leftover shape: mutation-checked, discarding name_order from the leftover placement fails three tests across three layers — the FAMILY_FIRST_GIVEN_LAST case row (one of the two, the FAMILY_FIRST row being blind to it by construction), test_post_rules.py's test_the_two_family_first_orders_differ_at_two_leftovers, and rules.md's P1 example for that order. An earlier draft of this sentence said "exactly one, and no other test in the suite": true when measured, false one commit later, because two of the three guards were added after the measurement and it was not re-run. Recording the failure mode as well as the number, since it is this log's recurring one. +- 2026-09-07 #471 DECLINED — the greedy reach under the default order is the READING, not a gap in one, and what decides it is a principle rather than a limit: a name order is a property of the DATA SOURCE, not of a string. The caller declares it to match how their records are written, and the declaration outranks anything the parser could infer from one name. rules.md's Name order Background now states that once, and decisions.md#O1's 2026-09-07 entry applies the same principle to the patronymic rotations. Applied here: a never-given particle marks where a surname BEGINS, multi-word surnames are ordinary — Spanish double surnames above all — and under the default order nothing has said this surname ends before the string does. So `de la Torre Vega` is a surname-only string read right, and `de la Cruz Juan Carlos` is a family-first listing read under the wrong order. The remedy is the declaration or the comma: FAMILY_FIRST already gives family `de la Cruz`, given `Juan`, middle `Carlos` (#395, the 2026-08-18 entry above), and the family comma fixes the surname whatever order is declared. `de la Family Family Given` and `de la Family Given Given` are undecidable under EITHER order, which is what makes the comma the answer rather than a better heuristic. + This SUPERSEDES the REASON the 2026-08-17 entry above gives for the same outcome — "The mixed-language shapes are what make the greedy reading look wrong, and the parser cannot see language." That was right and weaker. The order is not a language judgement at all: it is a declaration about the data, and a caller who has one supplies it whether or not a parser could ever have guessed it. The outcome that entry reached, and everything else in it, stands. + It also answers the 2026-08-16 #364 entry's "Nothing ever argued for 'takes everything'; it was the shape of v1's handle_non_first_name_prefix, not a decision." This entry argues for it. The v1 provenance is unchanged and the argument is new, which is the only thing that had been missing. + Measured 2026-09-07 on the tree, and on the 1.4.0, 2.0.0, 2.1.0 and 2.2.0 wheels, which agree on every default-order row: `de la Torre Vega` gives family `de la Torre Vega`; `de la Cruz Juan Carlos` gives family `de la Cruz Juan Carlos`; `de Mesnil Jean` gives family `de Mesnil Jean`; and the comma form `de la Torre Vega, Juan` gives family `de la Torre Vega`, given `Juan` under both orders. Under FAMILY_FIRST `de la Torre Vega` reads family `de la Torre`, given `Vega` — the cost the declaration accepts, and the reason a surname-only record in a family-first source still wants its comma. Recompute with `Parser().parse(name)` and `Parser(policy=Policy(name_order=FAMILY_FIRST)).parse(name)`; for the wheels, `HumanName(name).last` from a directory outside the checkout so the tree cannot shadow them. + Nothing moves. No parse changes; the `parity` row `leading_never_given_particle_two_leftovers` in tests/v2/cases.py and the guard `test_the_family_first_fold_is_not_explained_under_the_default_order` in tests/v2/test_ledger_guards.py stay as the pins of the decided behavior; no release-note bullet is written, there being no behavior to note and no by-design precedent in docs/release_log.rst. rules.md#P1 gains `de la Torre Vega` as an example, which puts the name in the contract corpus corpus_rules.jsonl. Its parity is over the SEVEN roles, not the `initials` view: at the 1.4.0 baseline only, the name falls into the already-recorded class `fix(initials-per-word) a particle chain inside a name part` beside its sibling `de la Cruz Juan Carlos` — the v1→v2 per-word initialling shipped since 2.0.0, not a regression — so the 1.4.0 intentional count goes 367 → 368 and that rule's recorded claim goes 107 → 108 names with a new digest. The three 2.x baselines are unmoved at 263 / 175 / 37, and `unexplained: 0` at all four. + The `Open:` pointer to #471 that stood at the end of decisions.md#P6's 2026-08-30 #467 entry is removed with this entry, which is where the question is now answered. P1's own `Open:` block below is #360's and is untouched. + Deliberately NOT decided here: reporting an ambiguity on the shapes neither order settles — the honest-output flag — would be an AmbiguityKind and belongs with the ambiguity bundle (#449/#491/#348) if anywhere. Declining the reach change does not decline that. Declined: @@ -134,7 +141,6 @@ Open: [#380](https://github.com/derek73/python-nameparser/issues/380) covers "Be WHAT REVIEW FOUND, because the corpus could not. Four agents on PR #472 found the site firing outside the order it was reasoned from, in two directions, and the differential corpus reported neither: 52 default-order names (a conjunction stops a particle's forward chain and leaves it standing in a middle — `Maria Luisa y de la Cruz` gave family 'de la Cruz' where master gives 'la Cruz') and 366 FAMILY_FIRST_GIVEN_LAST folds of a particle that does not end the name (`de Anke van y` gave family 'van de Anke' with the given name still behind it). Of 1099 corpus names only 6 hold a conjunction-then-particle adjacency and none in this shape, so the REACH recipe honestly reported 0 default-order movers throughout. The count was right and the population was blind; "none under the default order" was a property of the corpus, not of the code. Both are now `· boundary` examples in rules.md so the doc runner holds them. The repair is one clause — the run must END the name — and it subsumes the order test a first repair added: measured over 542,592 generated parses, an explicit `state.order[0] is Role.FAMILY` never decides anything the ends-the-name test has not already decided, so it is not in the code. What IS kept is `state.order is not None`, which is what keeps this site off the family-comma path (0 co-firings with P6's comma site over ~892,000 parses). A stale index list was repaired with it: O1 and O2 retag between roles without recomputing, and this site was the first consumer of `families` after them (33,840 stale observations instrumented). No shipped input reaches a wrong answer through it — no `_TURKIC`/`_EAST_SLAVIC`-matching word is particle vocabulary — so the fix is unwitnessed by any test and is recorded here instead. - Open: [#471](https://github.com/derek73/python-nameparser/issues/471) is the leading fold's REACH under the default order (`de Mesnil Jean` reports the whole string as the family). Split out of this change deliberately: it is a 1.4.0 parity break on the order every existing caller is on — prototyped, 5 unexplained diffs at that baseline and a failing v1 parity test — where nothing under a family-first order has a v1 answer at all. ### M2 — the maiden-marker rule @@ -689,10 +695,16 @@ Declined: - 2026-07-12 (landed in the v2 core, PR #288) — v1 parity pinned live: the rotation reconstructs token position from assigned roles, which is faithful to v1 only under the default given-first order. - 2026-08-15 — the rotation × non-default name_order interaction question rode #270, which closed 2026-07-28 with the order constants and no recorded answer for the rotations. +- 2026-09-07 #384 — OPTION 1: the rotations are scoped to the DEFAULT order, and a gate on the read order is what makes that true. The principle is decisions.md#P1's 2026-09-07 entry and rules.md's Name order Background — a name order is a property of the data source, the caller declares it, and the declaration outranks string-level inference. This rotation exists to RESTORE the default reading from a family-first listing, which is the whole of its job, so a caller who has DECLARED a family-first order has already supplied what it would have inferred and position decides instead. + The first draft of this entry said the scoping was by CONSTRUCTION and there was nothing to implement, on the reasoning that the rotation asks whether the word standing in the FAMILY position carries a patronymic ending and only the default order puts a name's last word there. That is false, and measuring it is what found it: under a family-first order the family slot holds the FIRST word, and a patronymic-DERIVED surname standing first carries the same ending, so such a name reached the rotation and was rotated out of the reading the caller declared. Measured 2026-09-07 before the gate, `Мицкевич Адам Юзеф` under `parser_for(locales.RU, base=Parser(policy=Policy(name_order=FAMILY_FIRST)))` read family `Адам`, given `Юзеф`, middle `Мицкевич`, where plain FAMILY_FIRST reads family `Мицкевич`, given `Адам`, middle `Юзеф`; `Рабинович Иван Лев` read family `Иван` the same way. After the gate both read as plain FAMILY_FIRST does. The condition is on `state.order` — the order assign actually READ — rather than `policy.name_order`, for the reason decisions.md#P1's fold gives: a script_orders entry can override the policy, and the roles the rotation reads are the ones assign made. No corpus name moves at any of the four baselines, and sweeping the corpora under each pack with FAMILY_FIRST declared shows no difference either: they hold no three-word name whose FIRST word carries a patronymic ending. The general lesson is mechanisms.md's about capability claims — an option chosen because it needs no code is exactly where the no-code claim goes unmeasured. + Measured 2026-09-07 with `parser_for(locales.RU, base=Parser(policy=Policy(name_order=FAMILY_FIRST)))`. The family-first listing reads the same either way: `Сидоров Иван Петрович` gives family `Сидоров`, given `Иван`, middle `Петрович` under the default order by rotation, and the identical parse under FAMILY_FIRST by position. The divergence is on natural-order input — `Иван Петрович Сидоров` gives family `Сидоров` by rotation under the default order, and family `Иван`, given `Петрович`, middle `Сидоров` under FAMILY_FIRST. That row is ACCEPTED, not a defect: the caller said this source is written family-first and is being taken at their word. rules.md#O1 carries it as an Accepted consequence. + O2 is the same decision on the Turkic marker and its 2026-09-07 entry says so. The shapes differ, which is why the two rules state their own consequences rather than sharing one: under FAMILY_FIRST `Ali Ahmad Vali oglu` reads family `Ali` exactly as the rotation would, and it is under FAMILY_FIRST_GIVEN_LAST that it reads given `oglu` — the marker in a name field, which is O2's existing Accepted consequence reached by a second route. + Where the row is pinned, and why not in rules.md. The example grammar admits ONE annotation, a policy or a locale, and `tests/v2/rules_doc.py`'s `resolve_annotation` returns exactly one of the two, so a pack and an order cannot be written on the same example line. Registering a combined policy would be a test change made inside a docs decision, and #470 already holds the question of what an example line should be able to declare. So the parse is a `tests/v2/test_locales.py` row beside the pack's other rows, and rules.md#O1 states it in prose. -Open: -[#384](https://github.com/derek73/python-nameparser/issues/384) -what the rotations should do under a non-default name_order (the divergent measurement is in the issue). +Declined: + +- Option 2, suppressing the rotation and re-deriving the order from the patronymic trace under a declared family-first order — the declaration already carries the information this would recompute, and recomputing it means overriding the caller with a string-level guess, which is the principle above inverted. +- Option 3, reporting an ambiguity wherever a patronymic trace disagrees with the declared order — the report would fire on natural-order input inside a family-first source, and the caller cannot tell that from a mis-declared record, which is the only thing the report would be for. It is NOT true that it would fire on every correct family-first parse: a family-first listing reads the same under both orders, so the trace and the position agree there and nothing is reported. What is left is a signal on the one shape where the declaration is doing its job, indistinguishable from the shape where the declaration is wrong. A report that cannot be filtered on is the failure decisions.md#P6's 2026-08-29 entry records for a different rule; here it is avoidable by not adding it. ### O2 — Turkic rotation @@ -700,10 +712,7 @@ what the rotations should do under a non-default name_order (the divergent measu - 2026-07-02 (design, orthography coverage) — dotless ı does not case-fold to i, so qızı is a literal alternative in the pattern (re.I cannot bridge it; all-caps QIZI falls to the ASCII alternative); the patterns are NFC literals and NFD input does not match. Scoping decline: suffix-attached Kazakh/Uzbek patronymics (Әбішұлы) are bound-suffix morphology, not a standalone marker, and deliberately out. - 2026-08-15 — same rotation/name_order interaction status as O1. - -Open: -[#384](https://github.com/derek73/python-nameparser/issues/384) -same question as O1. +- 2026-09-07 #384 — OPTION 1, the same answer as decisions.md#O1's entry of the same date and on the same principle: the rotation restores the default reading from a family-first listing, so a declared family-first order supplies that reading instead and the rotation stands down. It stands down by the same gate on the read order, for the reason O1's entry records — the marker test reads the word in the FAMILY position, which under a family-first order is the FIRST word, so a name opening with the marker reached it. Measured 2026-09-07 before the gate, `oglu Ahmad Vali Ali` with Turkic handling and FAMILY_FIRST read family `Ahmad`; after it, family `oglu`, which is what plain FAMILY_FIRST reads and this rule's own 2026-07-02 Accepted consequence — the marker in a name field. Measured 2026-09-07: with Turkic handling active, `Ali Ahmad Vali oglu` reads family `Ali` under FAMILY_FIRST, exactly as the rotation would, and given `oglu` under FAMILY_FIRST_GIVEN_LAST — the marker in a name field, which is this rule's own 2026-07-02 Accepted consequence reached by a second route rather than a new cost. Options 2 and 3 are Declined for O1's reasons. ### differential-ledger — tooling decisions (2.1.0 release arc) diff --git a/docs/design/rules.md b/docs/design/rules.md index 4fc7fd37..184f64c8 100644 --- a/docs/design/rules.md +++ b/docs/design/rules.md @@ -134,10 +134,13 @@ P1. Rationale: a particle OPENING a name has the whole rest of the How MANY name words it attaches to depends on the order. Under a family-first order it takes exactly ONE — declaring that order asserts that what follows the family is not more surname. Under - the default order it takes the rest of the name: nothing there - marks where the surname ends. Whether the default order should - narrow the same way is #471 — today `de Mesnil Jean` reports the - whole string as the family, and that reach is 1.4.0's. One name + the default order it takes the rest of the name, and that is a + reading rather than a gap in one: a never-given particle marks + where a surname BEGINS, surnames are routinely several words, + and no declaration has said this one ends before the string + does. So `de Mesnil Jean` reports the whole string as the + family, which is 1.4.0's reach as well (#471, declined by + design). One name word means one UNIT — a particle chain (P2), a conjunction join (P3) or a bound given-name pair (P5) is taken whole or not at all. A title does not move the opening position (P4), but a @@ -161,6 +164,18 @@ P1. Rationale: a particle OPENING a name has the whole rest of the "Mc Donald" → family="Mc Donald" "de los Santos" → family="de los Santos" "van Gogh" → given="van" · boundary + Accepted: the default-order reach has two faces, and the second + is a real cost. A surname-only string is read right, which is + what the reach is for; a family-first listing read under the + default order is read wrong, all of it landing in the family. + The remedy is the declaration or the comma — declare a + family-first order, which gives family `de la Cruz`, given + `Juan`, middle `Carlos`, or write the family comma. What + NEITHER order settles is the comma's alone: `de la Family + Family Given` and `de la Family Given Given` are the same + string shape, and no reading of the words tells them apart. + "de la Torre Vega" → family="de la Torre Vega" + "de la Cruz Juan Carlos" → family="de la Cruz Juan Carlos" Accepted: a bare "de" stays the given name — there is nothing to fold into, and inventing a surname would be worse. "de" → given="de" @@ -951,6 +966,7 @@ C2. Rationale: text beyond the recognized comma parts should be ## Name order (O) Background: written name order varies by convention: given-first (the library's default reading), family-first, and family-first with the given name last (Vietnamese, where the person is called by the last element, given names are frequently two syllables — the given_names view stays correct wherever the internal boundary falls — and quốc ngữ is Latin script, so no native-script signal exists at all). The order is declared by the caller or a locale pack, never detected — but a few conventions leave a recognizable trace in the name itself. Patronymics are one: East Slavic names carry a father's-name derivative with distinctive endings between given and family, and Turkic names use a standalone marker word ("oglu" son-of, "qizi" daughter-of) after the father's name. Where such a trace is present and unambiguous, an opted-in parser can restore the intended reading from a family-first listing. +A declared order is a property of the DATA SOURCE rather than of any one string: the caller sets it to match how their records are written, and it governs what no vocabulary and no script license has already claimed (O4) — it outranks what the parser could infer from the shape of a particular name, and yields where a name's own script settles the order instead (W4, where a name written wholly in an East Asian script reads family-first whatever order the caller declared). Two consequences run through this document. Under the default given-first order a string opening with a never-given particle is a surname whose given name is simply absent, so the fold takes the rest of it (P1). Under a declared family-first order the caller has already said that what follows the family is not more surname — so the fold stops there, and the rotations below, whose whole job is to RESTORE the default reading from a family-first listing, have nothing left to restore. A shape neither order settles is the family comma's job, and the parser does not guess at it. O1. Rationale: an East Slavic name written family-first still shows its patronymic — the distinctive ending identifies which word is @@ -962,9 +978,25 @@ O1. Rationale: an East Slavic name written family-first still shows the words are restored to given, patronymic, family. A middle word that also carries the ending blocks the reading, because the surname itself may be patronymic-derived. + The restoration is the default order's work: it recovers the + given-first reading a family-first listing hides, so where the + caller has DECLARED a family-first order there is nothing left + to restore and position decides. "Сидоров Иван Петрович" [ru] → family="Сидоров" "Sidorov Ivan Petrovich Jr." [ru] → family="Sidorov" "Иван Петрович Абрамович" [ru] → family="Абрамович" · boundary + Accepted: under a declared family-first order the readings part + on natural-order input, and the declaration wins. A family-first + listing reads the same either way — the first example above is + that same parse with FAMILY_FIRST declared — while with East + Slavic handling active and FAMILY_FIRST declared the + natural-order Иван Петрович Сидоров reads family Иван, given + Петрович, middle Сидоров. That is the caller's declaration being + honored on input they said was written family-first, not a + defect (#384). The example grammar takes one annotation, so a + pack and an order cannot share an example line; the parse is + pinned in tests/v2/test_locales.py and decisions.md#O1 records + why options 2 and 3 were declined. history: decisions.md#O1 · implemented: nameparser/_pipeline/_post_rules.py O2. Rationale: a Turkic patronymic marker is a separate word that @@ -975,12 +1007,19 @@ O2. Rationale: a Turkic patronymic marker is a separate word that nicknames aside — ending in a standalone patronymic marker reads family-first: the first name word is the family name, and the marker stays beside the father's name in the middle. + The scope is O1's: the restoration recovers the default reading, + so a declared family-first order stands in its place. "Ali Ahmad Vali oglu" [tr_az] → family="Ali" "Ali Ahmad Vali oglu Jr." [tr_az] → family="Ali" Accepted: any other count of name words keeps its positional reading, even when that leaves the marker itself in a name field. "Ali Ahmad oglu" [tr_az] → family="oglu" · boundary + Accepted: the order scope above has this rule's own consequence + reached by a second route — with Turkic handling active and + FAMILY_FIRST_GIVEN_LAST declared, Ali Ahmad Vali oglu reads + given oglu, the marker standing in a name field because the + declaration put the name's last word there (#384). history: decisions.md#O2 · implemented: nameparser/_pipeline/_post_rules.py O3. Rationale: several traditions write compound family names diff --git a/tests/v2/cases.py b/tests/v2/cases.py index 3ecf38a2..85fac3bb 100644 --- a/tests/v2/cases.py +++ b/tests/v2/cases.py @@ -691,7 +691,11 @@ def _check_cjk_shape_purity(self) -> None: "ambiguous, so this fold cannot fire on it. See " "decisions.md#P1, 2026-08-17.) 1.4.0 gives last 'de la " "Cruz Juan Carlos' too, so this row must not move when " - "#395 lands"), + "#395 lands. #471 DECLINED 2026-09-07: the reach is " + "the reading and not a gap in one -- a declared order " + "is a property of the data source, so under the " + "default order nothing has said this surname ends " + "before the string does. See decisions.md#P1"), Case("leading_never_given_particle_two_leftovers_family_first", "de la Cruz Juan Carlos", {"family": "de la Cruz", "given": "Juan", "middle": "Carlos"}, diff --git a/tests/v2/test_ledger_guards.py b/tests/v2/test_ledger_guards.py index 48de98fb..d297eae5 100644 --- a/tests/v2/test_ledger_guards.py +++ b/tests/v2/test_ledger_guards.py @@ -2243,7 +2243,9 @@ def _claim(rule: dict) -> _Claim: # its parse is otherwise identical across the two surfaces. # Reach against explained, measured 2026-09-02 at baseline # 1.4.0 -- a snapshot of that run, not a standing count: - # 27/27, 1/1, 96/66, 41/19, 107/11, 18/2. The reach half is + # 27/27, 1/1, 96/66, 41/19, 107/11, 18/2 -- the particle chain's + # reach is 108 since 2026-09-07, when rules.md#P1's `de la Torre + # Vega` example joined the contract corpus (#471). The reach half is # what this roster holds; the explained half moves with the # corpus and is re-read from the gate. The phd rule's 18 is the widest # gap and the most literal regex -- `\bph\. d\.` matches every @@ -2259,7 +2261,7 @@ def _claim(rule: dict) -> _Claim: "fix(initials-per-word) a bound-given run initials each word (facade, since 2.0.0)": _Claim(41, ('_initials',), "e99f56c955d5", ('DEFAULT',)), "fix(initials-per-word) a particle chain inside a name part initials each word (facade, since 2.0.0)": - _Claim(107, ('_initials',), "bdc4da864f59", ('DEFAULT',)), + _Claim(108, ('_initials',), "45f0b2c1a7d4", ('DEFAULT',)), "fix(initials-per-word) the Ph. D. merge initials each word (facade, since 2.0.0)": _Claim(18, ('_initials',), "f67d8ebddd56", ('DEFAULT',)), }, diff --git a/tests/v2/test_locales.py b/tests/v2/test_locales.py index c7f23d42..ea1b160a 100644 --- a/tests/v2/test_locales.py +++ b/tests/v2/test_locales.py @@ -14,7 +14,7 @@ from nameparser import Locale, Parser, locales, parse, parser_for from nameparser._lexicon import _VOCAB_FIELDS, Lexicon from nameparser._policy import ( - UNSET, PatronymicRule, Policy, Script, _SCRIPT_RANGES, + FAMILY_FIRST, UNSET, PatronymicRule, Policy, Script, _SCRIPT_RANGES, ) from nameparser._types import AmbiguityKind from nameparser.locales import ja as _ja @@ -730,6 +730,36 @@ def test_parser_for_results_chain_as_bases() -> None: assert chained.parse("Mammadova Aygun Ali kizi").family == "Mammadova" +def test_the_rotation_stands_down_under_a_declared_family_first_order( + ) -> None: + """#384 option 1: O1's rotation is the DEFAULT order's work. + + It exists to RESTORE the given-first reading from a family-first + listing, so a caller who has DECLARED FAMILY_FIRST has already + supplied what it would have inferred and position decides + instead. Scoped by construction, not by an order test: the + rotation reads the word in the FAMILY position for a patronymic + ending, and only the default order puts a name's last word + there. + + Both halves are one measurement. The family-first listing reads + the same either way, and the natural-order name reads as the + caller declared it -- family 'Иван' -- which is the divergence + #384 measured, Accepted in docs/design/decisions.md#O1 rather + than treated as a defect. rules.md#O1 states it in prose because + an example line there takes ONE annotation, a pack or an order, + never both. + """ + ff = parser_for(locales.RU, + base=Parser(policy=Policy(name_order=FAMILY_FIRST))) + listing = ff.parse("Сидоров Иван Петрович") + assert (listing.family, listing.given, listing.middle) == ( + "Сидоров", "Иван", "Петрович") + natural = ff.parse("Иван Петрович Сидоров") + assert (natural.family, natural.given, natural.middle) == ( + "Иван", "Петрович", "Сидоров") + + def test_locales_import_is_lazy(monkeypatch: pytest.MonkeyPatch) -> None: # importing the package must not import any pack module; PEP 562 # loads them on first attribute access (the lazy-access contract: diff --git a/tools/differential/corpus_rules.jsonl b/tools/differential/corpus_rules.jsonl index 894a5e5b..f2eef6c3 100644 --- a/tools/differential/corpus_rules.jsonl +++ b/tools/differential/corpus_rules.jsonl @@ -207,6 +207,7 @@ "de Anke van y" "de Mesnil Jean" "de la Cruz Juan Carlos" +"de la Torre Vega" "de la Vega" "de la Vega y Santos Juan" "de los Santos" From 4523155b9bf7b3c0202db29d24f0a5f687e153b6 Mon Sep 17 00:00:00 2001 From: Derek Gulbranson Date: Mon, 7 Sep 2026 20:58:55 -0700 Subject: [PATCH 2/3] fix(post_rules): the patronymic rotations stand down under a declared family-first order MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit rules.md#O1/#O2 exist to RESTORE the given-first reading a family-first listing hides. The code tested the word in the FAMILY slot regardless of the order assign read, so under FAMILY_FIRST -- where that slot holds the FIRST word -- a patronymic-derived surname fired the rotation: ru + FAMILY_FIRST read "Мицкевич Адам Юзеф" as family 'Адам' where plain FAMILY_FIRST reads family 'Мицкевич'. The rotations now apply only when the read order is the default given-first one (`state.order`, not policy.name_order, for the reason the P1 fold gives). No corpus name moves at any baseline, and the corpus swept under both packs with FAMILY_FIRST declared shows no difference either -- the corpora hold no three-word name whose first word carries a patronymic ending. Pinned by the firing control. Refs #384 Co-Authored-By: Claude Fable 5.1 --- nameparser/_pipeline/_post_rules.py | 13 +++- tests/v2/pipeline/test_post_rules.py | 89 ++++++++++++++++++++++++---- tests/v2/test_locales.py | 37 ++++++++---- 3 files changed, 114 insertions(+), 25 deletions(-) diff --git a/nameparser/_pipeline/_post_rules.py b/nameparser/_pipeline/_post_rules.py index 1b94b845..a0a900aa 100644 --- a/nameparser/_pipeline/_post_rules.py +++ b/nameparser/_pipeline/_post_rules.py @@ -462,7 +462,18 @@ def post_rules(state: ParseState) -> ParseState: # middle_as_family fold below runs comma or not (v1 order: # patronymics first, then handle_middle_name_as_last) rules = state.policy.patronymic_rules - rotations_apply = state.structure is Structure.NO_COMMA + # Both rotations RESTORE the given-first reading a family-first + # listing hides, so rules.md#O1's scope clause holds them to the + # default order: a caller who declared family-first has already + # said what the rotation would infer, and position decides + # (decisions.md#O1, the 2026-09-07 entry on #384). `state.order`, + # not policy.name_order, for the reason the P1 fold above gives -- + # a script_orders entry can override the policy, and the roles the + # rotations read are the ones assign actually made. None means + # assign consulted no order at all, which is the default reading. + read_order = state.order + rotations_apply = state.structure is Structure.NO_COMMA and ( + read_order is None or read_order[0] is Role.GIVEN) # rules.md#O1: "a name of exactly three name words — titles, # suffixes and nicknames aside — whose last name word carries a # patronymic ending and whose middle name word does not reads as diff --git a/tests/v2/pipeline/test_post_rules.py b/tests/v2/pipeline/test_post_rules.py index 2ffc3980..dd7a0c4f 100644 --- a/tests/v2/pipeline/test_post_rules.py +++ b/tests/v2/pipeline/test_post_rules.py @@ -106,6 +106,50 @@ def test_no_swap_when_more_fields_present() -> None: _ES = Policy(patronymic_rules=frozenset({PatronymicRule.EAST_SLAVIC})) _TK = Policy(patronymic_rules=frozenset({PatronymicRule.TURKIC})) +_FF = Policy(name_order=FAMILY_FIRST) +_FFGL = Policy(name_order=FAMILY_FIRST_GIVEN_LAST) + +#: Both family-first orders, since the rules claimed of a family-first +#: order are claimed of every one of them and only one was ever parsed. +#: They differ in where the given name lands behind the family, which +#: is exactly what a leading shape must not depend on; the +#: leading-particle cases below fold identically under both. +_FAMILY_FIRST = [pytest.param(_FF, id="FAMILY_FIRST"), + pytest.param(_FFGL, id="FAMILY_FIRST_GIVEN_LAST")] + +#: What the SHIPPED 2.2.0 wheel reads as the family, per (input, +#: declared order) -- the rotation firing on the family slot whatever +#: order assign read, which under a family-first order is the FIRST +#: word. This column is the negative control: it is the reading the +#: two stand-down tests below exist to prevent, recorded as data and +#: re-measurable (`pip install nameparser==2.2.0`) rather than left in +#: a comment. Nothing asserts on it -- what the tests assert is the +#: expected column -- but a failure quotes it, so a rotation that +#: fires again says so in the message. +_PRE_GATE_FAMILY = { + ("Мицкевич Адам Юзеф", FAMILY_FIRST): "Адам", + ("Мицкевич Адам Юзеф", FAMILY_FIRST_GIVEN_LAST): "Юзеф", + ("oglu Ahmad Vali Ali", FAMILY_FIRST): "Ahmad", + ("oglu Ahmad Vali Ali", FAMILY_FIRST_GIVEN_LAST): "Ali", +} + + +def _assert_rotation_stood_down(pack: Policy, order: Policy, text: str, + family: str) -> None: + """The pack opted in under `order` reads `text` exactly as `order` + alone reads it, family included.""" + out = _parsed(text, dataclasses.replace(pack, + name_order=order.name_order)) + plain = _parsed(text, order) + roles = (Role.TITLE, Role.GIVEN, Role.MIDDLE, Role.FAMILY, + Role.SUFFIX) + pre_gate = _PRE_GATE_FAMILY[text, order.name_order] + assert [_by_role(out, r) for r in roles] == \ + [_by_role(plain, r) for r in roles], ( + f"the pack changed the reading; before the gate the 2.2.0 " + f"wheel read family {pre_gate!r} here") + assert _by_role(out, Role.FAMILY) == family + def test_east_slavic_rotation() -> None: out = _parsed("Сидоров Иван Петрович", _ES) @@ -137,6 +181,25 @@ def test_east_slavic_skips_when_middle_is_also_patronymic() -> None: assert _by_role(out, Role.FAMILY) == "Abramovich" +@pytest.mark.parametrize("order", _FAMILY_FIRST) +def test_east_slavic_stands_down_under_a_declared_family_first_order( + order: Policy) -> None: + # rules.md#O1's scope clause (decisions.md#O1, 2026-09-07, #384): + # the rotation RESTORES the default reading a family-first listing + # hides, so a declared family-first order supplies that reading and + # the rotation stands down. The gate is on the order assign READ, + # so it needs an input the rotation would otherwise fire on: under + # a family-first order the family slot holds the FIRST word, and a + # patronymic-DERIVED surname written first carries the ending. + # + # BOTH family-first orders, because the gate is claimed of the + # order class and the two place the given name differently: the + # rotation rewrote a different word under each (_PRE_GATE_FAMILY), + # so one order alone would leave the other's reading unpinned. + _assert_rotation_stood_down(_ES, order, "Мицкевич Адам Юзеф", + "Мицкевич") + + def test_east_slavic_off_by_default() -> None: out = _parsed("Сидоров Иван Петрович") assert _by_role(out, Role.GIVEN) == "Сидоров" @@ -149,6 +212,20 @@ def test_turkic_rotation() -> None: assert _by_role(out, Role.FAMILY) == "Mammadova" +@pytest.mark.parametrize("order", _FAMILY_FIRST) +def test_turkic_stands_down_under_a_declared_family_first_order( + order: Policy) -> None: + # The other pack, gated by the same read: rules.md#O2 is the same + # restoration of a hidden given-first reading, and the gate is one + # condition covering both rotations -- so pinning only the East + # Slavic half would leave half the gate free to come back out. + # 'oglu' is the Turkic patronymic marker standing where a + # family-first order puts the surname, which is what makes O2's + # trailing-marker test see it. + _assert_rotation_stood_down(_TK, order, "oglu Ahmad Vali Ali", + "oglu") + + def test_leading_never_given_particle_folds_into_family() -> None: # v1 handle_non_first_name_prefix: a leading particle that is never # a given name ('de') means the whole name is a surname @@ -172,18 +249,6 @@ def test_degenerate_bare_particle_stays_given() -> None: assert not _by_role(out, Role.FAMILY) -_FF = Policy(name_order=FAMILY_FIRST) -_FFGL = Policy(name_order=FAMILY_FIRST_GIVEN_LAST) - -#: Both family-first orders, since the rule is claimed of every one of -#: them and only one was ever parsed. They differ in where the given -#: name lands behind the family, which is exactly what the leading -#: shape must not depend on; the cases below fold identically under -#: both. -_FAMILY_FIRST = [pytest.param(_FF, id="FAMILY_FIRST"), - pytest.param(_FFGL, id="FAMILY_FIRST_GIVEN_LAST")] - - # --- rule 1b under the family-first orders (#359) --------------------- # The fold keys on POSITION, not on the GIVEN role: a never-given # particle keeps its particle whatever name_order says. diff --git a/tests/v2/test_locales.py b/tests/v2/test_locales.py index ea1b160a..dacc87d4 100644 --- a/tests/v2/test_locales.py +++ b/tests/v2/test_locales.py @@ -737,21 +737,34 @@ def test_the_rotation_stands_down_under_a_declared_family_first_order( It exists to RESTORE the given-first reading from a family-first listing, so a caller who has DECLARED FAMILY_FIRST has already supplied what it would have inferred and position decides - instead. Scoped by construction, not by an order test: the - rotation reads the word in the FAMILY position for a patronymic - ending, and only the default order puts a name's last word - there. - - Both halves are one measurement. The family-first listing reads - the same either way, and the natural-order name reads as the - caller declared it -- family 'Иван' -- which is the divergence - #384 measured, Accepted in docs/design/decisions.md#O1 rather - than treated as a defect. rules.md#O1 states it in prose because - an example line there takes ONE annotation, a pack or an order, - never both. + instead. That scoping is a GATE, not a property of the shape: + the rotation reads the word in the FAMILY position for a + patronymic ending, and under FAMILY_FIRST that slot holds the + name's FIRST word. + + 'Мицкевич Адам Юзеф' is the input that makes the difference + visible -- a patronymic-DERIVED surname written first. Without + the gate the rotation fires on it and reports family 'Адам'; + with the gate it reads exactly as plain FAMILY_FIRST does. + + The other two are one measurement of the accepted divergence. + The family-first listing reads the same either way, and the + natural-order name reads as the caller declared it -- family + 'Иван' -- which is what #384 measured, Accepted in + docs/design/decisions.md#O1 rather than treated as a defect. + rules.md#O1 states it in prose because an example line there + takes ONE annotation, a pack or an order, never both. """ ff = parser_for(locales.RU, base=Parser(policy=Policy(name_order=FAMILY_FIRST))) + # the firing control: without the gate this reads family 'Адам' + derived = ff.parse("Мицкевич Адам Юзеф") + assert (derived.family, derived.given, derived.middle) == ( + "Мицкевич", "Адам", "Юзеф") + plain = Parser(policy=Policy(name_order=FAMILY_FIRST)).parse( + "Мицкевич Адам Юзеф") + assert (derived.family, derived.given, derived.middle) == ( + plain.family, plain.given, plain.middle) listing = ff.parse("Сидоров Иван Петрович") assert (listing.family, listing.given, listing.middle) == ( "Сидоров", "Иван", "Петрович") From 0b6a792791bc35d99ee56f5fa1bc6e6dcbdcdee7 Mon Sep 17 00:00:00 2001 From: Derek Gulbranson Date: Mon, 7 Sep 2026 21:47:24 -0700 Subject: [PATCH 3/3] review round: PR #517 findings Three review agents on the whole PR. The order Background is narrowed to what the code can implement -- a declared FAMILY-FIRST order outranks the shape; an explicit GIVEN_FIRST is the default and cannot be told apart from it -- and the two decisions.md restatements become citations. The #470 citation is corrected (closed not-planned 2026-09-02, and it decided that prose in an Accepted clause is the right home, not a workaround); the option-3 decline is scoped to listings carrying the trace; the capability-claim lesson is attributed to AGENTS.md axis 10; the zero-movers reason covers the Turkic trigger too; the gate's unwitnessable conditions are recorded as such. The gate becomes a module-level helper so a reachability probe can patch it back to the ungated condition and demand the 2.2.0 readings return -- without it a refactor that stops the rotation reaching family-first placement leaves all five stand-down rows green while asserting nothing. O2's Accepted clause gains its pin; a peeled-title parameter joins the East Slavic stand-down test. The #384 gate gets its release-note bullet: it changes what 2.2.0 does for a supported configuration, and corpus movement is not the house exemption. Co-Authored-By: Claude Fable 5.1 --- docs/design/decisions.md | 16 ++++----- docs/design/rules.md | 45 +++++++++++------------ docs/release_log.rst | 2 ++ nameparser/_pipeline/_post_rules.py | 30 +++++++++------- tests/v2/pipeline/test_post_rules.py | 53 ++++++++++++++++++++++++---- tests/v2/test_locales.py | 27 +++++++++++++- 6 files changed, 123 insertions(+), 50 deletions(-) diff --git a/docs/design/decisions.md b/docs/design/decisions.md index 37485dbe..fb3b84e8 100644 --- a/docs/design/decisions.md +++ b/docs/design/decisions.md @@ -56,8 +56,8 @@ the 2026-08-16 entries below. The survivor is the degenerate bare The order is READ, not re-derived: assign now records the order it actually used on ParseState.order, and the fold keys on that. policy.name_order would have been wrong — a script_orders entry can put the family first under a given-first policy, and the roles assign already wrote would then disagree with the roles the fold computes. The same reasoning already appears one function away, where the PARTICLE_OR_GIVEN emitter reads the role off the token rather than assuming given. A unit, not a token: a particle chain (P2), a conjunction join (P3) and a bound given-name pair (P5) each count once. All three are read off the TAGS rather than the pieces, but for two different reasons, and the difference matters to anyone who tries to simplify this. The conjunction join grouping DID build and the prefix chain then swallowed: the name reaches the fold as [de][la Vega y Santos Juan], the ambiguous particle having chained forward over the join, so the JOIN's boundary is gone. The leading particle keeps its own piece, and must — the fold's site test wants a lone piece, so a one-piece name would not fold at all. The bound-given join grouping never built at all: P5 joins only where the bound word is the first non-title piece, and at a fold site the first piece is the particle, so "ibn Awf abdul Rahman" arrives as four separate pieces. Restoring piece boundaries in group would fix the first and silently split the second. It also makes P5's new implemented: pointer stronger than a citation — this is the only place that join happens in this shape. This is what keeps "de la Vega y Santos Juan" from stopping between Vega and Santos — the case rules.md#P3 was amended for on 2026-08-17, now executable. Measured, with the denominator this log requires: of 751 corpus names, SEVEN reach P1's leading fold (DE MESNIL, De Groot, de Mesnil, de Mesnil Garcia, de Mesnil Jr., de la Vega, dos Santos) and exactly ONE of those has anything past the run to lose, so one moves (de Mesnil Garcia) under each family-first order and none under the default. The single mover is a fact about the corpus holding no family-first listing with leftovers, not about the blast radius. The differential harness cannot see any of it — it runs the corpus under the default policy against 1.4.0, which has no name_order — so its exit 0 is evidence for the accepted cost and for nothing else. The verification that counts is the two-leftover shape: mutation-checked, discarding name_order from the leftover placement fails three tests across three layers — the FAMILY_FIRST_GIVEN_LAST case row (one of the two, the FAMILY_FIRST row being blind to it by construction), test_post_rules.py's test_the_two_family_first_orders_differ_at_two_leftovers, and rules.md's P1 example for that order. An earlier draft of this sentence said "exactly one, and no other test in the suite": true when measured, false one commit later, because two of the three guards were added after the measurement and it was not re-run. Recording the failure mode as well as the number, since it is this log's recurring one. -- 2026-09-07 #471 DECLINED — the greedy reach under the default order is the READING, not a gap in one, and what decides it is a principle rather than a limit: a name order is a property of the DATA SOURCE, not of a string. The caller declares it to match how their records are written, and the declaration outranks anything the parser could infer from one name. rules.md's Name order Background now states that once, and decisions.md#O1's 2026-09-07 entry applies the same principle to the patronymic rotations. Applied here: a never-given particle marks where a surname BEGINS, multi-word surnames are ordinary — Spanish double surnames above all — and under the default order nothing has said this surname ends before the string does. So `de la Torre Vega` is a surname-only string read right, and `de la Cruz Juan Carlos` is a family-first listing read under the wrong order. The remedy is the declaration or the comma: FAMILY_FIRST already gives family `de la Cruz`, given `Juan`, middle `Carlos` (#395, the 2026-08-18 entry above), and the family comma fixes the surname whatever order is declared. `de la Family Family Given` and `de la Family Given Given` are undecidable under EITHER order, which is what makes the comma the answer rather than a better heuristic. - This SUPERSEDES the REASON the 2026-08-17 entry above gives for the same outcome — "The mixed-language shapes are what make the greedy reading look wrong, and the parser cannot see language." That was right and weaker. The order is not a language judgement at all: it is a declaration about the data, and a caller who has one supplies it whether or not a parser could ever have guessed it. The outcome that entry reached, and everything else in it, stands. +- 2026-09-07 #471 DECLINED — the greedy reach under the default order is the READING, not a gap in one, and what decides it is a principle rather than a limit: a name order is a property of the DATA SOURCE, not of a string. The caller declares it to match how their records are written: the declaration governs what no vocabulary has claimed and a declared family-first order outranks the shape; rules.md's Name order Background states it. decisions.md#O1's 2026-09-07 entry applies the same principle to the patronymic rotations. Applied here: a never-given particle marks where a surname BEGINS, multi-word surnames are ordinary — Spanish double surnames above all — and under the default order nothing has said this surname ends before the string does. So `de la Torre Vega` is a surname-only string read right, and `de la Cruz Juan Carlos` is a family-first listing read under the wrong order. The remedy is the declaration or the comma: FAMILY_FIRST already gives family `de la Cruz`, given `Juan`, middle `Carlos` (#395, the 2026-08-18 entry above), and the family comma fixes the surname whatever order is declared. `de la Family Family Given` and `de la Family Given Given` are undecidable under EITHER order, which is what makes the comma the answer rather than a better heuristic. + This SUPERSEDES the REASON the 2026-08-17 entry above gives for the same outcome — "The mixed-language shapes are what make the greedy reading look wrong, and the parser cannot see language." That was right and weaker. The order is not a language judgement at all: it is a declaration about the data, and a caller who has one supplies it whether or not a parser could ever have guessed it. The outcome that entry reached, and everything else in it, stands — and so does its "Why the question kept thrashing" paragraph: the REACH question is a language judgement, which is why the order declaration is where the caller supplies it. What is superseded is only the claim that the parser's blindness to language is the reason. It also answers the 2026-08-16 #364 entry's "Nothing ever argued for 'takes everything'; it was the shape of v1's handle_non_first_name_prefix, not a decision." This entry argues for it. The v1 provenance is unchanged and the argument is new, which is the only thing that had been missing. Measured 2026-09-07 on the tree, and on the 1.4.0, 2.0.0, 2.1.0 and 2.2.0 wheels, which agree on every default-order row: `de la Torre Vega` gives family `de la Torre Vega`; `de la Cruz Juan Carlos` gives family `de la Cruz Juan Carlos`; `de Mesnil Jean` gives family `de Mesnil Jean`; and the comma form `de la Torre Vega, Juan` gives family `de la Torre Vega`, given `Juan` under both orders. Under FAMILY_FIRST `de la Torre Vega` reads family `de la Torre`, given `Vega` — the cost the declaration accepts, and the reason a surname-only record in a family-first source still wants its comma. Recompute with `Parser().parse(name)` and `Parser(policy=Policy(name_order=FAMILY_FIRST)).parse(name)`; for the wheels, `HumanName(name).last` from a directory outside the checkout so the tree cannot shadow them. Nothing moves. No parse changes; the `parity` row `leading_never_given_particle_two_leftovers` in tests/v2/cases.py and the guard `test_the_family_first_fold_is_not_explained_under_the_default_order` in tests/v2/test_ledger_guards.py stay as the pins of the decided behavior; no release-note bullet is written, there being no behavior to note and no by-design precedent in docs/release_log.rst. rules.md#P1 gains `de la Torre Vega` as an example, which puts the name in the contract corpus corpus_rules.jsonl. Its parity is over the SEVEN roles, not the `initials` view: at the 1.4.0 baseline only, the name falls into the already-recorded class `fix(initials-per-word) a particle chain inside a name part` beside its sibling `de la Cruz Juan Carlos` — the v1→v2 per-word initialling shipped since 2.0.0, not a regression — so the 1.4.0 intentional count goes 367 → 368 and that rule's recorded claim goes 107 → 108 names with a new digest. The three 2.x baselines are unmoved at 263 / 175 / 37, and `unexplained: 0` at all four. @@ -695,16 +695,16 @@ Declined: - 2026-07-12 (landed in the v2 core, PR #288) — v1 parity pinned live: the rotation reconstructs token position from assigned roles, which is faithful to v1 only under the default given-first order. - 2026-08-15 — the rotation × non-default name_order interaction question rode #270, which closed 2026-07-28 with the order constants and no recorded answer for the rotations. -- 2026-09-07 #384 — OPTION 1: the rotations are scoped to the DEFAULT order, and a gate on the read order is what makes that true. The principle is decisions.md#P1's 2026-09-07 entry and rules.md's Name order Background — a name order is a property of the data source, the caller declares it, and the declaration outranks string-level inference. This rotation exists to RESTORE the default reading from a family-first listing, which is the whole of its job, so a caller who has DECLARED a family-first order has already supplied what it would have inferred and position decides instead. - The first draft of this entry said the scoping was by CONSTRUCTION and there was nothing to implement, on the reasoning that the rotation asks whether the word standing in the FAMILY position carries a patronymic ending and only the default order puts a name's last word there. That is false, and measuring it is what found it: under a family-first order the family slot holds the FIRST word, and a patronymic-DERIVED surname standing first carries the same ending, so such a name reached the rotation and was rotated out of the reading the caller declared. Measured 2026-09-07 before the gate, `Мицкевич Адам Юзеф` under `parser_for(locales.RU, base=Parser(policy=Policy(name_order=FAMILY_FIRST)))` read family `Адам`, given `Юзеф`, middle `Мицкевич`, where plain FAMILY_FIRST reads family `Мицкевич`, given `Адам`, middle `Юзеф`; `Рабинович Иван Лев` read family `Иван` the same way. After the gate both read as plain FAMILY_FIRST does. The condition is on `state.order` — the order assign actually READ — rather than `policy.name_order`, for the reason decisions.md#P1's fold gives: a script_orders entry can override the policy, and the roles the rotation reads are the ones assign made. No corpus name moves at any of the four baselines, and sweeping the corpora under each pack with FAMILY_FIRST declared shows no difference either: they hold no three-word name whose FIRST word carries a patronymic ending. The general lesson is mechanisms.md's about capability claims — an option chosen because it needs no code is exactly where the no-code claim goes unmeasured. +- 2026-09-07 #384 — OPTION 1: the rotations are scoped to the DEFAULT order, and a gate on the read order is what makes that true. The principle is decisions.md#P1's 2026-09-07 entry and rules.md's Name order Background — a name order is a property of the data source, the caller declares it, the declaration governs what no vocabulary has claimed, and a declared family-first order outranks the shape. This rotation exists to RESTORE the default reading from a family-first listing, which is the whole of its job, so a caller who has DECLARED a family-first order has already supplied what it would have inferred and position decides instead. + The first draft of this entry said the scoping was by CONSTRUCTION and there was nothing to implement, on the reasoning that the rotation asks whether the word standing in the FAMILY position carries a patronymic ending and only the default order puts a name's last word there. That is false, and measuring it is what found it: under a family-first order the family slot holds the FIRST word, and a patronymic-DERIVED surname standing first carries the same ending, so such a name reached the rotation and was rotated out of the reading the caller declared. Measured 2026-09-07 before the gate, `Мицкевич Адам Юзеф` under `parser_for(locales.RU, base=Parser(policy=Policy(name_order=FAMILY_FIRST)))` read family `Адам`, given `Юзеф`, middle `Мицкевич`, where plain FAMILY_FIRST reads family `Мицкевич`, given `Адам`, middle `Юзеф`; `Рабинович Иван Лев` read family `Иван` the same way. After the gate both read as plain FAMILY_FIRST does. The condition is on `state.order` — the order assign actually READ — rather than `policy.name_order`, for the reason decisions.md#P1's fold gives: a script_orders entry can override the policy, and the roles the rotation reads are the ones assign made. No corpus name moves at any of the four baselines, and sweeping the corpora under each pack with FAMILY_FIRST declared shows no difference either: they hold no three-word name whose FIRST word carries an East Slavic patronymic ending, and no four-word name whose FIRST word is a standalone Turkic marker (both counts 0 over the `tools/differential/corpus*.jsonl` union, measured 2026-09-07 by applying `_post_rules`'s own four patronymic patterns to the first word). The general lesson is docs/design/AGENTS.md's axis 10, re-run every capability claim — an option chosen because it needs no code is exactly where the no-code claim goes unmeasured. Two of the gate's three conditions have no possible witness and are recorded here instead: `read_order is None` cannot co-occur with a rotation shape, since assign leaves the order unrecorded only where it positioned at most one name word, and `state.order` differs from `policy.name_order` only through `script_orders`, whose scripts are East Asian and so cannot match the patronymic regexes. Unwitnessed by any test, on the precedent decisions.md#P6's 2026-08-30 #467 entry sets. Measured 2026-09-07 with `parser_for(locales.RU, base=Parser(policy=Policy(name_order=FAMILY_FIRST)))`. The family-first listing reads the same either way: `Сидоров Иван Петрович` gives family `Сидоров`, given `Иван`, middle `Петрович` under the default order by rotation, and the identical parse under FAMILY_FIRST by position. The divergence is on natural-order input — `Иван Петрович Сидоров` gives family `Сидоров` by rotation under the default order, and family `Иван`, given `Петрович`, middle `Сидоров` under FAMILY_FIRST. That row is ACCEPTED, not a defect: the caller said this source is written family-first and is being taken at their word. rules.md#O1 carries it as an Accepted consequence. - O2 is the same decision on the Turkic marker and its 2026-09-07 entry says so. The shapes differ, which is why the two rules state their own consequences rather than sharing one: under FAMILY_FIRST `Ali Ahmad Vali oglu` reads family `Ali` exactly as the rotation would, and it is under FAMILY_FIRST_GIVEN_LAST that it reads given `oglu` — the marker in a name field, which is O2's existing Accepted consequence reached by a second route. - Where the row is pinned, and why not in rules.md. The example grammar admits ONE annotation, a policy or a locale, and `tests/v2/rules_doc.py`'s `resolve_annotation` returns exactly one of the two, so a pack and an order cannot be written on the same example line. Registering a combined policy would be a test change made inside a docs decision, and #470 already holds the question of what an example line should be able to declare. So the parse is a `tests/v2/test_locales.py` row beside the pack's other rows, and rules.md#O1 states it in prose. + O2 is the same decision on the Turkic marker and its 2026-09-07 entry says so. The shapes differ, which is why the two rules state their own consequences rather than sharing one: under FAMILY_FIRST `Ali Ahmad Vali oglu` reads family `Ali` exactly as the rotation would, and it is under FAMILY_FIRST_GIVEN_LAST that it reads given `oglu`, middle `Ahmad Vali` — the marker in a name field, which is O2's existing Accepted consequence reached by a second route. Both readings are pinned in tests/v2/test_locales.py beside O1's row (measured 2026-09-07). + Where the row is pinned, and why not in rules.md. The example grammar admits ONE annotation, and `tests/v2/rules_doc.py`'s `resolve_annotation` resolves it to a policy, a locale or a gated locale — never a pair. Composite keys exist for other pairings (`family-first+middle_as_family`, `[ja+segmenter]`), but no registered annotation combines a pack with an order, so the two cannot be written on the same example line. Registering a combined policy would be a test change made inside a docs decision. #470 asked whether an example line should declare its format and was closed not-planned on 2026-09-02 — "no notation is needed. The Accepted clause is already the right home for this" — so prose in the rule is the decided home here, not a workaround. The parse is a `tests/v2/test_locales.py` row beside the pack's other rows, and rules.md#O1 states it in prose. Declined: - Option 2, suppressing the rotation and re-deriving the order from the patronymic trace under a declared family-first order — the declaration already carries the information this would recompute, and recomputing it means overriding the caller with a string-level guess, which is the principle above inverted. -- Option 3, reporting an ambiguity wherever a patronymic trace disagrees with the declared order — the report would fire on natural-order input inside a family-first source, and the caller cannot tell that from a mis-declared record, which is the only thing the report would be for. It is NOT true that it would fire on every correct family-first parse: a family-first listing reads the same under both orders, so the trace and the position agree there and nothing is reported. What is left is a signal on the one shape where the declaration is doing its job, indistinguishable from the shape where the declaration is wrong. A report that cannot be filtered on is the failure decisions.md#P6's 2026-08-29 entry records for a different rule; here it is avoidable by not adding it. +- Option 3, reporting an ambiguity wherever a patronymic trace disagrees with the declared order — the report would fire on natural-order input inside a family-first source, and the caller cannot tell that from a mis-declared record, which is the only thing the report would be for. It is NOT true that it would fire on every correct family-first parse: a family-first listing CARRYING THE TRACE reads the same under both orders — the rotation fires only where the last word carries the ending, and it then makes the first word the family exactly as position does — so the trace and the position agree there and nothing is reported. Scoped deliberately, the unscoped claim being false: `Сидоров Иван Лев` is a family-first listing with no trace, and it reads family `Лев` under the ru pack's default order against family `Сидоров` under FAMILY_FIRST (measured 2026-09-07). What is left is a signal on the one shape where the declaration is doing its job, indistinguishable from the shape where the declaration is wrong. A report that cannot be filtered on is the failure decisions.md#P6's 2026-08-29 entry records for a different rule; here it is avoidable by not adding it. ### O2 — Turkic rotation @@ -712,7 +712,7 @@ Declined: - 2026-07-02 (design, orthography coverage) — dotless ı does not case-fold to i, so qızı is a literal alternative in the pattern (re.I cannot bridge it; all-caps QIZI falls to the ASCII alternative); the patterns are NFC literals and NFD input does not match. Scoping decline: suffix-attached Kazakh/Uzbek patronymics (Әбішұлы) are bound-suffix morphology, not a standalone marker, and deliberately out. - 2026-08-15 — same rotation/name_order interaction status as O1. -- 2026-09-07 #384 — OPTION 1, the same answer as decisions.md#O1's entry of the same date and on the same principle: the rotation restores the default reading from a family-first listing, so a declared family-first order supplies that reading instead and the rotation stands down. It stands down by the same gate on the read order, for the reason O1's entry records — the marker test reads the word in the FAMILY position, which under a family-first order is the FIRST word, so a name opening with the marker reached it. Measured 2026-09-07 before the gate, `oglu Ahmad Vali Ali` with Turkic handling and FAMILY_FIRST read family `Ahmad`; after it, family `oglu`, which is what plain FAMILY_FIRST reads and this rule's own 2026-07-02 Accepted consequence — the marker in a name field. Measured 2026-09-07: with Turkic handling active, `Ali Ahmad Vali oglu` reads family `Ali` under FAMILY_FIRST, exactly as the rotation would, and given `oglu` under FAMILY_FIRST_GIVEN_LAST — the marker in a name field, which is this rule's own 2026-07-02 Accepted consequence reached by a second route rather than a new cost. Options 2 and 3 are Declined for O1's reasons. +- 2026-09-07 #384 — OPTION 1, the same answer as decisions.md#O1's entry of the same date and on the same principle: the rotation restores the default reading from a family-first listing, so a declared family-first order supplies that reading instead and the rotation stands down. It stands down by the same gate on the read order, for the reason O1's entry records — the marker test reads the word in the FAMILY position, which under a family-first order is the FIRST word, so a name opening with the marker reached it. Measured 2026-09-07 before the gate, `oglu Ahmad Vali Ali` with Turkic handling and FAMILY_FIRST read family `Ahmad`; after it, family `oglu`, which is what plain FAMILY_FIRST reads and this rule's own 2026-07-02 Accepted consequence — the marker in a name field. The natural-order twin `Ali Ahmad Vali oglu` is decisions.md#O1's entry of the same date, which measures it under both family-first orders and pins it in tests/v2/test_locales.py. Options 2 and 3 are Declined for O1's reasons. ### differential-ledger — tooling decisions (2.1.0 release arc) diff --git a/docs/design/rules.md b/docs/design/rules.md index 184f64c8..fb49dfe5 100644 --- a/docs/design/rules.md +++ b/docs/design/rules.md @@ -140,17 +140,17 @@ P1. Rationale: a particle OPENING a name has the whole rest of the and no declaration has said this one ends before the string does. So `de Mesnil Jean` reports the whole string as the family, which is 1.4.0's reach as well (#471, declined by - design). One name - word means one UNIT — a particle chain (P2), a conjunction join - (P3) or a bound given-name pair (P5) is taken whole or not at - all. A title does not move the opening position (P4), but a - family comma does end the question: the comma has already fixed - the surname, so there is no positional read left for an order to - narrow, and a particle opening the part AFTER it takes the rest - of that part whatever order is declared. What is left over is not read by O4's rule for a whole name, - which would make the first leftover a second family name; it is - laid out as the positions AFTER the family in the declared order, - the family slot being already filled. + design). One name word means one UNIT — a particle chain (P2), + a conjunction join (P3) or a bound given-name pair (P5) is + taken whole or not at all. A title does not move the opening + position (P4), but a family comma does end the question: the + comma has already fixed the surname, so there is no positional + read left for an order to narrow, and a particle opening the + part AFTER it takes the rest of that part whatever order is + declared. What is left over is not read by O4's rule for a + whole name, which would make the first leftover a second family + name; it is laid out as the positions AFTER the family in the + declared order, the family slot being already filled. "de la Vega" → family="de la Vega" "Sir de Mesnil" → family="de Mesnil" "Mesnil de" family-first → given="de" @@ -966,7 +966,7 @@ C2. Rationale: text beyond the recognized comma parts should be ## Name order (O) Background: written name order varies by convention: given-first (the library's default reading), family-first, and family-first with the given name last (Vietnamese, where the person is called by the last element, given names are frequently two syllables — the given_names view stays correct wherever the internal boundary falls — and quốc ngữ is Latin script, so no native-script signal exists at all). The order is declared by the caller or a locale pack, never detected — but a few conventions leave a recognizable trace in the name itself. Patronymics are one: East Slavic names carry a father's-name derivative with distinctive endings between given and family, and Turkic names use a standalone marker word ("oglu" son-of, "qizi" daughter-of) after the father's name. Where such a trace is present and unambiguous, an opted-in parser can restore the intended reading from a family-first listing. -A declared order is a property of the DATA SOURCE rather than of any one string: the caller sets it to match how their records are written, and it governs what no vocabulary and no script license has already claimed (O4) — it outranks what the parser could infer from the shape of a particular name, and yields where a name's own script settles the order instead (W4, where a name written wholly in an East Asian script reads family-first whatever order the caller declared). Two consequences run through this document. Under the default given-first order a string opening with a never-given particle is a surname whose given name is simply absent, so the fold takes the rest of it (P1). Under a declared family-first order the caller has already said that what follows the family is not more surname — so the fold stops there, and the rotations below, whose whole job is to RESTORE the default reading from a family-first listing, have nothing left to restore. A shape neither order settles is the family comma's job, and the parser does not guess at it. +A declared order is a property of the DATA SOURCE rather than of any one string: the caller sets it to match how their records are written, and it governs what no vocabulary and no script license has already claimed (O4). A declared FAMILY-FIRST order outranks what the parser could infer from the shape of a particular name; the given-first reading is the parser's DEFAULT rather than a declaration it can tell apart from one, and the traces above are what refine it (O1, O2). And the declaration yields where a name's own script settles the order instead (W4, where a name written wholly in an East Asian script reads family-first whatever order the caller declared). Two consequences run through this document. Under the default given-first order a string opening with a never-given particle is a surname whose given name is simply absent, so the fold takes the rest of it (P1). Under a declared family-first order the caller has already said that what follows the family is not more surname — so the fold stops there, and the rotations below, whose whole job is to RESTORE the default reading from a family-first listing, have nothing left to restore. A shape neither order settles is the family comma's job, and the parser does not guess at it. O1. Rationale: an East Slavic name written family-first still shows its patronymic — the distinctive ending identifies which word is @@ -987,16 +987,17 @@ O1. Rationale: an East Slavic name written family-first still shows "Иван Петрович Абрамович" [ru] → family="Абрамович" · boundary Accepted: under a declared family-first order the readings part on natural-order input, and the declaration wins. A family-first - listing reads the same either way — the first example above is - that same parse with FAMILY_FIRST declared — while with East - Slavic handling active and FAMILY_FIRST declared the - natural-order Иван Петрович Сидоров reads family Иван, given - Петрович, middle Сидоров. That is the caller's declaration being - honored on input they said was written family-first, not a - defect (#384). The example grammar takes one annotation, so a - pack and an order cannot share an example line; the parse is - pinned in tests/v2/test_locales.py and decisions.md#O1 records - why options 2 and 3 were declined. + listing reads the same either way under FAMILY_FIRST — the first + example above is that same parse with FAMILY_FIRST declared, + and under FAMILY_FIRST_GIVEN_LAST the listing's given and middle + swap, so only the family is invariant — while with East Slavic + handling active and FAMILY_FIRST declared the natural-order + Иван Петрович Сидоров reads family Иван, given Петрович, middle + Сидоров. That is the caller's declaration being honored on input + they said was written family-first, not a defect (#384). No + registered example annotation combines a pack with an order, so + the parse is pinned in tests/v2/test_locales.py instead, and + decisions.md#O1 records why options 2 and 3 were declined. history: decisions.md#O1 · implemented: nameparser/_pipeline/_post_rules.py O2. Rationale: a Turkic patronymic marker is a separate word that diff --git a/docs/release_log.rst b/docs/release_log.rst index 4c61008d..5d14c287 100644 --- a/docs/release_log.rst +++ b/docs/release_log.rst @@ -18,6 +18,8 @@ Release Log - **Mark ba as an acronym that is also an ordinary name, so a bare trailing Ba keeps the family name.** ``HumanName("Anna Ba")`` gives last ``Ba`` and reports a suffix-or-name ambiguity, where 2.0.0 through 2.2.0 gave suffix ``Ba`` and no last name. The SPACED full-name form keeps the credential reading: ``John Smith BA`` still gives suffix ``BA``, now flagged, and the dotted ``John Smith B.A.`` is an unflagged suffix, the periods settling it. The COMMA forms move, and this is the marking's real cost: ``Smith, BA`` gives first ``BA``, and ``John Smith, BA`` gives first ``BA``, last ``John Smith``, where 2.0.0 through 2.2.0 gave suffix ``BA`` for both -- what ``Smith, Ed`` costs, which S2 already accepted for the other ambiguous acronyms. A bracketed or quoted ``John Smith (BA)`` falls through to nickname parsing, as the 2.0 note for ``ma``/``do`` below recorded for that pair. Write ``B.A.`` to keep the credential reading. BA is a common credential and Ba a real surname in Vietnamese and Senegalese Fula, which is the ``ma``/``Ma`` shape exactly. No corpus name moves (#342) + - **Fix the East Slavic and Turkic patronymic rotations overriding a declared family-first name order.** With ``patronymic_rules`` opted in and ``Policy(name_order=FAMILY_FIRST)``, ``Мицкевич Адам Юзеф`` gave last ``Адам`` through 2.2.0 and now gives last ``Мицкевич`` -- the reading the declaration asks for -- and ``oglu Ahmad Vali Ali`` with Turkic handling gave last ``Ahmad`` and now ``oglu``. The rotations exist to restore the given-first reading a family-first listing hides, so under a declared family-first order the declaration decides. No corpus name moves. See the ``O1`` entry of ``docs/design/decisions.md`` (closes #384) + **Additions** - **Add the renunciate titles to the given-name title list, so a renunciate's one name is a given name.** ``HumanName("Swami Vivekananda")`` gives first ``Vivekananda`` with an empty last name, where every release since 1.4.0 gave last ``Vivekananda``; ``Guru Nanak``, ``Baba Ramdev`` and ``Lama Zopa`` move the same way, and so do the Devanagari and Bengali spellings added below. Two name words behind the title are unchanged -- ``Swami Vivekananda Saraswati`` keeps last ``Saraswati`` -- and a surname-retaining title is untouched: ``Rabbi Cohen`` still gives last ``Cohen``. ``venerable`` is deliberately not in the list, the traditions using it splitting on whether the family name survives. See the ``indic-honorifics`` entry of ``docs/design/decisions.md`` (closes #346) diff --git a/nameparser/_pipeline/_post_rules.py b/nameparser/_pipeline/_post_rules.py index a0a900aa..8efa051b 100644 --- a/nameparser/_pipeline/_post_rules.py +++ b/nameparser/_pipeline/_post_rules.py @@ -65,6 +65,23 @@ _RENDERS_ELSEWHERE = frozenset({Role.TITLE, Role.NICKNAME, Role.MAIDEN}) +def _rotations_apply(state: ParseState) -> bool: + # Both patronymic rotations RESTORE the given-first reading a + # family-first listing hides, so rules.md#O1's scope clause holds + # them to the default order: a caller who declared family-first has + # already said what the rotation would infer, and position decides + # (decisions.md#O1, the 2026-09-07 entry on #384). `state.order`, + # not policy.name_order, for the reason the P1 fold gives -- a + # script_orders entry can override the policy, and the roles the + # rotations read are the ones assign actually made. None means + # assign positioned nothing: a family comma (which the NO_COMMA + # test already excludes) or an early return with no name piece to + # position, so there is no declaration to defer to and the + # rotation's own shape test decides. + return state.structure is Structure.NO_COMMA and ( + state.order is None or state.order[0] is Role.GIVEN) + + def _mark_suffix_entries(tokens: list[WorkToken], state: ParseState) -> None: # In place over the caller's token list, the way every other rule # in post_rules writes: a state-in/state-out spelling here cost @@ -462,18 +479,7 @@ def post_rules(state: ParseState) -> ParseState: # middle_as_family fold below runs comma or not (v1 order: # patronymics first, then handle_middle_name_as_last) rules = state.policy.patronymic_rules - # Both rotations RESTORE the given-first reading a family-first - # listing hides, so rules.md#O1's scope clause holds them to the - # default order: a caller who declared family-first has already - # said what the rotation would infer, and position decides - # (decisions.md#O1, the 2026-09-07 entry on #384). `state.order`, - # not policy.name_order, for the reason the P1 fold above gives -- - # a script_orders entry can override the policy, and the roles the - # rotations read are the ones assign actually made. None means - # assign consulted no order at all, which is the default reading. - read_order = state.order - rotations_apply = state.structure is Structure.NO_COMMA and ( - read_order is None or read_order[0] is Role.GIVEN) + rotations_apply = _rotations_apply(state) # rules.md#O1: "a name of exactly three name words — titles, # suffixes and nicknames aside — whose last name word carries a # patronymic ending and whose middle name word does not reads as diff --git a/tests/v2/pipeline/test_post_rules.py b/tests/v2/pipeline/test_post_rules.py index dd7a0c4f..73bf4dab 100644 --- a/tests/v2/pipeline/test_post_rules.py +++ b/tests/v2/pipeline/test_post_rules.py @@ -123,19 +123,25 @@ def test_no_swap_when_more_fields_present() -> None: #: word. This column is the negative control: it is the reading the #: two stand-down tests below exist to prevent, recorded as data and #: re-measurable (`pip install nameparser==2.2.0`) rather than left in -#: a comment. Nothing asserts on it -- what the tests assert is the -#: expected column -- but a failure quotes it, so a rotation that -#: fires again says so in the message. +#: a comment. The stand-down tests do not assert on it -- what they +#: assert is the expected column -- but a failure quotes it, so a +#: rotation that fires again says so in the message. The vacuity probe +#: below DOES assert on it, which is what keeps this column honest. +#: The key is (input, declared order): each input picks out its own +#: pack, `Мицкевич` being East Slavic and `oglu` Turkic, so no pack +#: identifier is needed to disambiguate a row. _PRE_GATE_FAMILY = { ("Мицкевич Адам Юзеф", FAMILY_FIRST): "Адам", ("Мицкевич Адам Юзеф", FAMILY_FIRST_GIVEN_LAST): "Юзеф", + ("Dr. Мицкевич Адам Юзеф", FAMILY_FIRST): "Адам", + ("Dr. Мицкевич Адам Юзеф", FAMILY_FIRST_GIVEN_LAST): "Юзеф", ("oglu Ahmad Vali Ali", FAMILY_FIRST): "Ahmad", ("oglu Ahmad Vali Ali", FAMILY_FIRST_GIVEN_LAST): "Ali", } def _assert_rotation_stood_down(pack: Policy, order: Policy, text: str, - family: str) -> None: + family: str, title: str = "") -> None: """The pack opted in under `order` reads `text` exactly as `order` alone reads it, family included.""" out = _parsed(text, dataclasses.replace(pack, @@ -149,6 +155,7 @@ def _assert_rotation_stood_down(pack: Policy, order: Policy, text: str, f"the pack changed the reading; before the gate the 2.2.0 " f"wheel read family {pre_gate!r} here") assert _by_role(out, Role.FAMILY) == family + assert _by_role(out, Role.TITLE) == title def test_east_slavic_rotation() -> None: @@ -181,9 +188,18 @@ def test_east_slavic_skips_when_middle_is_also_patronymic() -> None: assert _by_role(out, Role.FAMILY) == "Abramovich" +@pytest.mark.parametrize("text,title", [ + ("Мицкевич Адам Юзеф", ""), + # a peeled title must not move the shape the gate is claimed of: + # rules.md#O1 counts name words "titles, suffixes and nicknames + # aside", so the rotation still reaches this input pre-gate + # (_PRE_GATE_FAMILY records the same family as the bare form) and + # the title has to survive the stand-down + ("Dr. Мицкевич Адам Юзеф", "Dr."), +]) @pytest.mark.parametrize("order", _FAMILY_FIRST) def test_east_slavic_stands_down_under_a_declared_family_first_order( - order: Policy) -> None: + order: Policy, text: str, title: str) -> None: # rules.md#O1's scope clause (decisions.md#O1, 2026-09-07, #384): # the rotation RESTORES the default reading a family-first listing # hides, so a declared family-first order supplies that reading and @@ -196,8 +212,7 @@ def test_east_slavic_stands_down_under_a_declared_family_first_order( # order class and the two place the given name differently: the # rotation rewrote a different word under each (_PRE_GATE_FAMILY), # so one order alone would leave the other's reading unpinned. - _assert_rotation_stood_down(_ES, order, "Мицкевич Адам Юзеф", - "Мицкевич") + _assert_rotation_stood_down(_ES, order, text, "Мицкевич", title) def test_east_slavic_off_by_default() -> None: @@ -226,6 +241,30 @@ def test_turkic_stands_down_under_a_declared_family_first_order( "oglu") +def test_the_order_gate_is_what_stands_the_rotations_down( + monkeypatch: pytest.MonkeyPatch) -> None: + """Reachability probe: with the gate patched back to the ungated + condition, every stand-down input must rotate again, reading the + family _PRE_GATE_FAMILY records from the 2.2.0 wheel. If this + fails green-side, the stand-down rows above have gone vacuous -- + the rotation no longer reaches their inputs.""" + from nameparser._pipeline import _post_rules + monkeypatch.setattr( + _post_rules, "_rotations_apply", + lambda state: state.structure is _post_rules.Structure.NO_COMMA) + for pack, text in ((_ES, "Мицкевич Адам Юзеф"), + (_ES, "Dr. Мицкевич Адам Юзеф"), + (_TK, "oglu Ahmad Vali Ali")): + for order in (_FF, _FFGL): + policy = dataclasses.replace(pack, + name_order=order.name_order) + out = _parsed(text, policy) + assert _by_role(out, Role.FAMILY) == \ + _PRE_GATE_FAMILY[text, policy.name_order], ( + f"{text!r} no longer reaches the rotation under " + f"{policy.name_order}; the stand-down rows are vacuous") + + def test_leading_never_given_particle_folds_into_family() -> None: # v1 handle_non_first_name_prefix: a leading particle that is never # a given name ('de') means the whole name is a surname diff --git a/tests/v2/test_locales.py b/tests/v2/test_locales.py index dacc87d4..30a14c69 100644 --- a/tests/v2/test_locales.py +++ b/tests/v2/test_locales.py @@ -14,7 +14,8 @@ from nameparser import Locale, Parser, locales, parse, parser_for from nameparser._lexicon import _VOCAB_FIELDS, Lexicon from nameparser._policy import ( - FAMILY_FIRST, UNSET, PatronymicRule, Policy, Script, _SCRIPT_RANGES, + FAMILY_FIRST, FAMILY_FIRST_GIVEN_LAST, UNSET, PatronymicRule, Policy, + Script, _SCRIPT_RANGES, ) from nameparser._types import AmbiguityKind from nameparser.locales import ja as _ja @@ -773,6 +774,30 @@ def test_the_rotation_stands_down_under_a_declared_family_first_order( "Иван", "Петрович", "Сидоров") +def test_the_turkic_marker_lands_in_a_name_field_under_both_orders( + ) -> None: + """Pins the prose decisions.md#O1's 2026-09-07 entry states for O2. + + Gate-independent: `Ali Ahmad Vali oglu` is natural-order input, + so the rotation would not fire on it under the default order + either. What the two readings show is O2's own 2026-07-02 + Accepted consequence reached by a second route -- a declared + family-first order leaves the marker standing in a name field. + Under FAMILY_FIRST that costs nothing visible, the family being + the word the rotation would have chosen anyway; under + FAMILY_FIRST_GIVEN_LAST the marker becomes the GIVEN name, which + is the reading rules.md#O2 carries as Accepted. + """ + ff = parser_for(locales.TR_AZ, base=Parser( + policy=Policy(name_order=FAMILY_FIRST))) + n = ff.parse("Ali Ahmad Vali oglu") + assert (n.family, n.given, n.middle) == ("Ali", "Ahmad", "Vali oglu") + ffgl = parser_for(locales.TR_AZ, base=Parser( + policy=Policy(name_order=FAMILY_FIRST_GIVEN_LAST))) + n = ffgl.parse("Ali Ahmad Vali oglu") + assert (n.family, n.given, n.middle) == ("Ali", "oglu", "Ahmad Vali") + + def test_locales_import_is_lazy(monkeypatch: pytest.MonkeyPatch) -> None: # importing the package must not import any pack module; PEP 562 # loads them on first attribute access (the lazy-access contract: