nameparser 2.0 implementation

.github/workflows/python-package.yml

@@ -20,14 +20,22 @@ jobs:
     strategy:
       fail-fast: false
       matrix:
-        python-version: ["3.10", "3.11", "3.12", "3.13", "3.14"]
+        python-version: ["3.11", "3.12", "3.13", "3.14", "3.15"]
+    # 3.15 is pre-release until 2026-10-01 (#259): surface breakage
+    # without blocking the branch; flip to blocking when it goes final.
+    continue-on-error: ${{ matrix.python-version == '3.15' }}
+    env:
+      # Without this, uv sync honors .python-version (3.11) over the
+      # matrix interpreter and every job silently tests 3.11.
+      UV_PYTHON: ${{ matrix.python-version }}
 
     steps:
     - uses: actions/checkout@v7
     - name: Set up Python ${{ matrix.python-version }}
       uses: actions/setup-python@v6
       with:
         python-version: ${{ matrix.python-version }}
+        allow-prereleases: true
     - name: Install uv
       uses: astral-sh/setup-uv@v6
       with:

.python-version

@@ -1 +1 @@
-3.10
+3.11

AGENTS.md

@@ -29,7 +29,9 @@ uv run mypy
 # Lint
 uv run ruff check nameparser/
 
-# Debug how a specific name string is parsed (prints HumanName repr)
+# Debug how a specific name string is parsed: 2.0 core parse() --
+# prints the ParsedName repr raw and capitalized, then initials;
+# --json emits the component dict
 uv run python -m nameparser "Dr. Juan Q. Xavier de la Vega III"
 
 # Build docs
@@ -107,6 +109,22 @@ Parse flow:
 
 Each named attribute (`title`, `first`, etc.) is a `@property` that joins its corresponding `_list`. Setters call `_set_list()` which runs the value through `parse_pieces()`, so assigning `hn.last = "de la Vega"` correctly re-parses prefix tokens.
 
+## 2.0 API modules (`nameparser/_*.py`, `tests/v2/`)
+
+The 2.0 rewrite lands as underscore-private modules alongside the v1 code. These conventions apply to all new-API code and are stricter than the v1 sections above. The full design record (rationale, settled-decision logs, dated amendments) lives in untracked `docs/superpowers/specs/`; this section is the enforceable subset. **A commit that establishes or amends one of these conventions must update this section in the same commit** — grep-driven staleness sweeps miss paraphrased prose (see Workflow above), so write-time maintenance is the mechanism, audits are the backstop.
+
+- **Module layout**: every new module is underscore-private (`_types.py`, `_lexicon.py`, `_policy.py`, `_locale.py`, `_render.py`, `_pipeline/`, `_parser.py`, plus the facade layer: `_facade.py`, `_config_shim.py`). The public import surface is exactly `nameparser` and `nameparser.locales`; `nameparser/__init__.py` holds re-exports and `__all__` only — no logic. Since the M11 swap, the old paths are import-path-preserving re-exports: `nameparser.parser` re-exports the `_facade` `HumanName`, `nameparser.config` re-exports the `_config_shim` names (`Constants`, `CONSTANTS`, `SetManager`, `TupleManager`, `RegexTupleManager`); the `config/` DATA modules stay the vocabulary source through 2.x. The whole facade layer is deleted in 3.0.
+- **Facade layer** (`_facade.py`, `_config_shim.py`): the v1-compat `HumanName`/`Constants` over the core. Key mechanisms: `Constants._generation` dirty-tracking (every mutation bumps; facades resolve their `Parser` lazily via `_cached_parser(lexicon, policy)`); `Constants._snapshot()` mirrors `_lexicon._default_lexicon()` (equality-pinned); the facade pickles v1-SHAPED state (component lists, one `__setstate__` path for 1.4 and 2.x blobs; components rebuild via `replace()`, never a re-parse); `_V1_HOOKS` overrides warn once per subclass (#280). The compat contract is the migration spec's promise: warning-free 1.4 code behaves identically except release-log-classified fixes — `tools/differential/` (dev-only, not shipped) verifies this against 1.4-on-PyPI over a 486-name corpus. `parser.py:NNNN` citations throughout the 2.0 code refer to the PRE-swap v1 file, deleted at the M11 swap; resolve them with `git show 2d5d8c2:nameparser/parser.py`.
+- **Layering is enforced by `tests/v2/test_layering.py`** (exact-module matching; `if TYPE_CHECKING:` imports don't count): `_types` imports nothing internal at module level — its rendering delegates import `_render` at call time; `_lexicon` and `_policy` sit above `_types` independently (`_lexicon` may import `nameparser.config.*` DATA modules only — vocabulary is single-sourced from the v1 data modules through 2.x, e.g. `config/maiden_markers.py`); `_locale` sits on `_lexicon`+`_policy` (plus `_types` for the shared pickle mixin); `_render` imports `_types` and `_lexicon` (for `Lexicon.default()` and `_normalize`); `_pipeline/*` imports `_types`+`_lexicon`+`_policy` plus in-package `_pipeline` helpers; `_parser` sits on everything except `_render`; the facade layer (`_facade`, `_config_shim`, `parser`, `config/__init__`, `__main__`) may import anything public plus `_render`; locale pack modules (`locales/*.py`) import `_locale`/`_lexicon`/`_policy` only, and the `locales/__init__` additionally lazy-imports its packs (PEP 562). Extend the test's `ALLOWED` table when adding a module.
+- **Canonical field order** — the seven roles in `Role` enum declaration order, defined once and derived everywhere (properties, `as_dict`, reprs, `comparison_key`). Never restate the order literally.
+- **Method organization**, fixed section order in every class: fields + `__post_init__` validation → alternative constructors → dunders (construction/equality → protocol → operators) → properties → public methods by concern (access → editing → comparison → rendering delegates) → private helpers last, except a helper serving exactly one section may sit at that section's head. Sanctioned deviation, facade layer only: `HumanName` and the shim `Constants` organize by v1 concern groups (`# -- render defaults --`, `# -- config / parsing --`, `# -- fields --`, ..., dunders and pickle last) — the classes mirror v1's own surface and die in 3.0; the canonical order still binds every core type.
+- **Validation is eager and fail-loud**: every `raise` states the offending value, the expected form, and the fix. Exception taxonomy: wrong type — including wrong element type inside a collection, bare `str` where an iterable of strings is expected, or a `Mapping` where a plain iterable is expected — raises `TypeError`; well-typed but unacceptable values raise `ValueError`; failed enum lookups stay `ValueError` for any input (stdlib `EnumType` precedent).
+- **Reprs are bounded**: render which fields deviate from a named baseline and by how much, never contents (`Lexicon(default + titles: +2)`).
+- **Typing/docs**: `from __future__ import annotations`; `frozen=True, slots=True` on every public dataclass; strict-profile mypy flags via per-module overrides in pyproject (`strict = true` itself is not valid per-module). Docstrings state contracts in prose with **no doctest blocks** — `--doctest-modules` makes every example a test; behavior examples go to unit tests per the lean-docs rule.
+- **Pickling**: v2 types must round-trip (`Parser` will be picklable by construction, and it holds a `Lexicon`). Every frozen type assigns `_guarded_getstate`/`_guarded_setstate` (`_types.py`) in its class body (`@dataclass(slots=True)` would override inherited pickle methods) — unpickling fails at the LOAD site on field-layout skew, and values are deliberately NOT re-validated (pickle is not a security boundary; canonical state comes from a validated instance). `Lexicon` keeps its own copy of the guard (layering) plus the `mappingproxy` slot rebuild; a new unpicklable slot type needs the same treatment plus a round-trip test.
+- **One sanctioned global**: the (future) cached default `Parser`. Lazily cached FROZEN singletons (`Lexicon.default()`'s `functools.cache`, the future default parser) are constants, not state; any second piece of module-level MUTABLE state requires amending the conventions doc, on purpose, in review. Sanctioned exceptions, facade layer only: `_config_shim.CONSTANTS` (the v1 shared singleton, mutable by design) and `_facade._WARNED_SUBCLASSES` (the once-per-subclass hook-warning dedup set) — both deleted with the layer in 3.0.
+- **Tests**: all v2 tests live in the `tests/v2/` package (its `conftest.py` overrides the v1 dual-run fixture — v2 code never reads shared `CONSTANTS`), one test module per source module plus cross-cutting `test_reprs.py` and `test_layering.py`, names stating behavior. Never assert `Lexicon.default()` contents; the narrow sourcing spot-checks in `test_default_sources_v1_vocabulary` that pin the v1→v2 migration contract (e.g. the flipped `particles_ambiguous` model) are the sanctioned exception.
+
 ## Extension Patterns
 
 **Adding a scalar `Constants` attribute + `HumanName` kwarg** (e.g. `initials_separator`, `suffix_delimiter`):
@@ -120,7 +138,7 @@ Each named attribute (`title`, `first`, etc.) is a `@property` that joins its co
 
 Add a dedicated `copy.deepcopy()` round-trip test for it too (see `test_regexes_deepcopy_roundtrip`/`test_nickname_delimiters_deepcopy_roundtrip` in `tests/test_constants.py`), not just reliance on conftest's autouse snapshot/restore exercising it incidentally. `TupleManager`/`RegexTupleManager.__getattr__` answer *any* unknown attribute lookup — dunder probes like `__deepcopy__` raise `AttributeError` (never answered as config keys), everything else still returns `self.get(attr)`/`EMPTY_REGEX` — so a new manager subtype or a `__getattr__` tweak can silently break `copy.deepcopy` (this bit `RegexTupleManager` before the dunder-lookup guard was added). A direct test on the new attribute's own manager instance catches that where the conftest fixture, which never asserts on the copy, would not. As with the scalar-attribute pattern above, also check `AGENTS.md` itself for now-stale references when you touch this.
 
-**Unknown-key attribute access on `TupleManager`/`RegexTupleManager` warns (1.4, #256)** — a key not currently in the dict emits `DeprecationWarning` naming the miss and the known keys (`_warn_unknown_key`), before falling back to the same `None`/`EMPTY_REGEX` default as before; `.get()` stays silent. Dunder probes (`__deepcopy__`) still raise `AttributeError` outright, and single-underscore probes (`_repr_html_`, IPython's `_ipython_canary_method_should_not_exist_`, etc.) are excluded from the warning too — no real config key starts with `_`, so both guards just keep protocol/introspection probes from misfiring as "typo" warnings. This means internal parser code that reads `self.C.regexes.<key>` unconditionally (e.g. `squash_bidi`'s `bidi`) now warns if a caller's custom `regexes` dict omits that key — a previously-silent partial-override pattern is on the same deprecation path as an actual typo.
+**Unknown-key attribute access on `TupleManager`/`RegexTupleManager` warns (1.4, #256) — 2.0 note: the warning became a hard `AttributeError` naming the known keys (shim `TupleManager`); the paragraph below describes the deleted v1 machinery and applies only when reading v1 history.** — a key not currently in the dict emits `DeprecationWarning` naming the miss and the known keys (`_warn_unknown_key`), before falling back to the same `None`/`EMPTY_REGEX` default as before; `.get()` stays silent. Dunder probes (`__deepcopy__`) still raise `AttributeError` outright, and single-underscore probes (`_repr_html_`, IPython's `_ipython_canary_method_should_not_exist_`, etc.) are excluded from the warning too — no real config key starts with `_`, so both guards just keep protocol/introspection probes from misfiring as "typo" warnings. This means internal parser code that reads `self.C.regexes.<key>` unconditionally (e.g. `squash_bidi`'s `bidi`) now warns if a caller's custom `regexes` dict omits that key — a previously-silent partial-override pattern is on the same deprecation path as an actual typo.
 
 **Adding a word to a config set** — first check the *other* sets for the same word (grep `nameparser/config/` or intersect the sets in a `python3 -c`). Real overlaps exist: `do`/`st`/`mc` ∈ `PREFIXES` ∩ `TITLES`/`SUFFIX_ACRONYMS`; `abd` = "ABD" ∈ `SUFFIX_ACRONYMS`; `abu` ∈ `PREFIXES` ∩ `bound_first_names` (position-dependent: leading token → first-name join, mid-name → last-name join). Usually position-dependent and harmless, but can force a guard or an exclusion (the `last_base` all-particles guard; dropping `abd` from `bound_first_names`).
 

README.rst

@@ -3,67 +3,9 @@ Name Parser
 
 |Build Status| |PyPI| |PyPI version| |Documentation| |License| |Downloads| |Codecov|
 
-📣 **nameparser 2.0 is in design** — a new immutable core API, with full
-compatibility for existing code through 2.x. Read the `design RFC
-<https://github.com/derek73/python-nameparser/pull/285>`__ and share
-feedback on `the discussion issue
-<https://github.com/derek73/python-nameparser/issues/284>`__.
-
-A simple Python (3.10+) module for parsing human names into their
-individual components.
-
-* hn.title
-* hn.first
-* hn.middle
-* hn.last
-* hn.suffix
-* hn.nickname
-* hn.maiden
-* hn.surnames *(middle + last)*
-* hn.given_names *(first + middle)*
-* hn.initials *(first initial of each name part)*
-* hn.last_base *(last, minus any prefixes)*
-* hn.last_prefixes *(leading last-name particles, e.g. "van der")*
-
-Supported Name Structures
-~~~~~~~~~~~~~~~~~~~~~~~~~
-
-The supported name structure is generally "Title First Middle Last Suffix", where all pieces 
-are optional. Comma-separated format like "Last, First" is also supported.
-
-1. Title Firstname "Nickname" Middle Middle Lastname Suffix
-2. Lastname [Suffix], Title Firstname (Nickname) Middle Middle[,] Suffix [, Suffix]
-3. Title Firstname M Lastname [Suffix], Suffix [Suffix] [, Suffix]
-
-How It Works
-~~~~~~~~~~~~
-
-The parser works in two layers.
-
-A **vocabulary layer** recognizes name pieces by what they are, using
-configurable sets of known words: titles ("Dr."), suffixes ("III", "PhD"),
-last-name prefixes ("de la"), conjunctions ("y", "&"), and delimited
-nicknames ("Doc"). Titles and conjunctions chain together to handle complex
-titles like "Asst Secretary of State"; prefixes join forward so "de la Vega"
-stays one last name. This layer doesn't care where in the string a word
-appears — and it's the layer you customize, by adding or removing entries
-in the sets to fit your dataset.
-
-A **positional layer** then assigns everything the vocabulary layer didn't
-claim, based purely on where it sits: the first unclaimed word is the first
-name, the last one is the last name, and anything between them is a middle
-name. There is no semantic understanding — "Dr" is a title when it comes
-before a name and a suffix when it comes after ("pre-nominal" and
-"post-nominal" would probably be better names) — and no attempt to correct
-mistakes in the input.
-
-It attempts the best guess that can be made with a simple, deterministic,
-rule-based approach — no statistical models or machine learning; the same
-input always parses the same way. The positional layer assumes Western name
-order (given name first), so the main use case is English and other
-languages that share that structure. It can also try to correct the
-capitalization of names that are all upper- or lowercase. It's not perfect,
-but it gets you pretty far.
+nameparser parses human names into seven fields — title, given, middle,
+family, suffix, nickname, maiden. Results are immutable, configuration is
+composable, and locale packs are opt-in.
 
 Installation
 ------------
@@ -72,95 +14,50 @@ Installation
 
   pip install nameparser
 
-If you want to try out the latest code from GitHub you can
-install with pip using the command below.
-
-``pip install -e git+https://github.com/derek73/python-nameparser.git``
-
-If you need to handle lists of names, check out
-`namesparser <https://github.com/gwu-libraries/namesparser>`_, a
-compliment to this module that handles multiple names in a string.
-
+Requires Python 3.11+.
 
 Quick Start Example
--------------------
-
-::
-
-    >>> from nameparser import HumanName
-    >>> name = HumanName("Dr. Juan Q. Xavier de la Vega III (Doc Vega)")
-    >>> name 
-    <HumanName : [
-        title: 'Dr.'
-        first: 'Juan'
-        middle: 'Q. Xavier'
-        last: 'de la Vega'
-        suffix: 'III'
-        nickname: 'Doc Vega'
-        maiden: ''
-    ]>
-    >>> name.last
-    'de la Vega'
-    >>> name.as_dict()
-    {'title': 'Dr.', 'first': 'Juan', 'middle': 'Q. Xavier', 'last': 'de la Vega', 'suffix': 'III', 'nickname': 'Doc Vega', 'maiden': ''}
-    >>> str(name)
-    'Dr. Juan Q. Xavier de la Vega III (Doc Vega)'
-    >>> name.string_format = "{first} {last}"
-    >>> str(name)
-    'Juan de la Vega'
-
-
-Because the positional layer has no semantic understanding, position is
-everything:
+--------------------
 
-::
-
-    >>> name = HumanName("1 & 2, 3 4 5, Mr.")
-    >>> name 
-    <HumanName : [
-        title: ''
-        first: '3'
-        middle: '4 5'
-        last: '1 & 2'
-        suffix: 'Mr.'
-        nickname: ''
-        maiden: ''
-    ]>
-
-Customization
--------------
+.. code-block:: python
 
-Your project may need some adjustment for your dataset. Most customization
-is vocabulary — `customizing the configured pre-defined sets`_ of titles,
-prefixes, etc. that the vocabulary layer matches against. You can also do
-your own pre- or post-processing, or subclass the `HumanName` class for
-deeper changes. See the `full documentation`_ for more information.
+    >>> from nameparser import parse
+    >>> name = parse("Dr. Juan Q. Xavier de la Vega III")
+    >>> name.given, name.family
+    ('Juan', 'de la Vega')
+    >>> name.render("{family}, {given}")
+    'de la Vega, Juan'
 
+Those seven fields are ``title``, ``given``, ``middle``, ``family``,
+``suffix``, ``nickname``, and ``maiden`` — plus aggregate views like
+``given_names``, ``surnames``, ``family_base``, and ``family_particles``
+for combining or splitting them further.
 
-`Full documentation`_
-~~~~~~~~~~~~~~~~~~~~~
+Learn more
+----------
 
-.. _customizing the configured pre-defined sets: http://nameparser.readthedocs.org/en/latest/customize.html
-.. _Full documentation: http://nameparser.readthedocs.org/en/latest/
-
-
-Contributing
-------------
+* `Getting started <https://nameparser.readthedocs.io/en/latest/usage.html>`__ — the full tour: parsing, aggregates, dicts, rendering, dedup
+* `Customizing the parser <https://nameparser.readthedocs.io/en/latest/customize.html>`__ — vocabulary, behavior, and presentation
+* `Locale packs <https://nameparser.readthedocs.io/en/latest/locales.html>`__ — opt-in bundles for East Slavic patronymics, Turkic markers, and more
+* There's also a CLI: ``python -m nameparser --json "Doe, John"``
 
-If you come across name piece that you think should be in the default config, you're
-probably right. `Start a New Issue`_ and we can get them added. 
+Coming from 1.x
+----------------
 
-Please let me know if there are ways this library could be structured to make
-it easier for you to use in your projects. Read CONTRIBUTING.md_ for more info
-on running the tests and contributing to the project.
+Nothing breaks. 2.0 keeps ``HumanName`` and ``CONSTANTS`` working exactly
+as before — same imports, same attributes, same mutation API. See
+`Migrating from HumanName <https://nameparser.readthedocs.io/en/latest/migrate.html>`__
+for translating a v1 customization into the new API, whenever that's
+convenient for you.
 
-**GitHub Project**
+See the `release log <https://nameparser.readthedocs.io/en/latest/release_log.html>`__
+for the full list of changes in the 2.0 series.
 
-https://github.com/derek73/python-nameparser
+License
+-------
 
-.. _CONTRIBUTING.md: https://github.com/derek73/python-nameparser/tree/master/CONTRIBUTING.md
-.. _Start a New Issue: https://github.com/derek73/python-nameparser/issues
-.. _click here to propose changes to the titles: https://github.com/derek73/python-nameparser/edit/master/nameparser/config/titles.py
+LGPL licensed. See `LICENSE <https://github.com/derek73/python-nameparser/blob/master/LICENSE>`__
+for details.
 
 .. |Build Status| image:: https://github.com/derek73/python-nameparser/actions/workflows/python-package.yml/badge.svg
    :target: https://github.com/derek73/python-nameparser/actions/workflows/python-package.yml