Skip to content

Commit bfa798f

Browse files
derek73claude
andcommitted
Read S2's run after P3's joins, counting the chain's units (#614 review)
The review found that reading the trailing run before every join broke what the corpus never tests: a bare salutation lost its connective to the family name (`Mr. and Mrs.` -> family 'and'), a joined leading salutation counted as three words (`Mr. and Mrs. Jr`), and a particle chain or a connective join counted word by word where rules.md#P3 says a joined part is one name word (`Freiherr von Berg X.Y.Z.` took the acronym against every release). Decided by Derek: connectives join first. - The read now runs after P3's joins and before the particle chain and the bound-given join, on segments of three or more pieces, and counts a particle run (duals behind a particle included) with the plain words after it as one unit, in a copy (`_counting_view`); roman numerals by shape stay out of the unit (`Juan de la Vega VI`, the #610 class). `_take_trailing_duals` is gone: the view covers it. - The chain's S2 emitter reports again where the read did not weigh the member (`Freiherr von Berg Ed Do` reports 'Ed'). - Restored to master: the P3 group tests, the one-word-gate case row, `Jane née Jr y Jones` (its ledger rule withdrawn, #418's no longer dormant). Moves now: `John van der Berg Prof.` (title), P5's `abdul J. V` (1.4.0's reading), report wording, one garbage fallback. - New tests, each killed by a recorded mutation: salutation, unit-count, numeral, dual-at-the-head, contiguity and fallback rows; direct tests of `read_trailing_run`; group tests of `tail_reads` and the narrowed emitter. P5 tests renamed to what they assert. - Docs: S2/P2/P5/H5 statements, the decisions entry (the first design and its reversal, recipe with draw mechanics, residuals), P3/P5/H5/P2 and parse-cost bullets, mechanisms, release log narrowed to what holds behind a comma, AGENTS.md's sweep lesson. Frames 317 -> 309 (3.11), 296 -> 290 (3.12-3.15) against adf6da8; segments under three pieces unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
1 parent 7ed94d5 commit bfa798f

21 files changed

Lines changed: 497 additions & 347 deletions

‎AGENTS.md‎

Lines changed: 2 additions & 2 deletions
Large diffs are not rendered by default.

‎docs/design/decisions.md‎

Lines changed: 14 additions & 12 deletions
Large diffs are not rendered by default.

‎docs/design/mechanisms.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -47,7 +47,7 @@ How it works. The two layers compose without ordering bugs because the positiona
4747

4848
## READ-WITHOUT-THEN-BIND — read the name without the construct, then fix that reading
4949

50-
Problem shape. An optional construct sits inside a name — a maiden clause, a trailing title — and the requirement is that writing it changes nothing else about the name. Checked after the fact, that invariant has to be defended at every place the construct can end, and each defense has to model what later stages would do to the words it gives up. Contract statement. Read the name as if the construct were not written, using the rule that reads that position, not the whole parse. The construct takes only what that reading leaves. Where later stages could still move what the reading decided, give those words their roles at the take so no later join can reach them. The invariant then holds by construction and is not a property each stop must defend. How it works. M2 before #601 scanned forward, stopped at suffix-looking words, and kept a release check asking at each stop whether the name left standing would read what was given up as post-nominals. That check modelled the P2 chain, the P5 bound join and P6's attachment, and #548 was it failing at the one stop that never asked. #601 replaced the stops and the check with the clause-free reading plus the bind: a net 689 lines out of `_group.py` (+210/−899 in the #601 commit), and #535's grids went from 15,586 and 8,713 invariant violations to 0 (prototype measurement, 2026-10-03, #601's comments). The first half already appears twice. H5's trailing title is TRANSPARENT: "what stands once the chain is taken reads exactly as it would read written without the title, plus the title". P3 asks its word count and its one-case question of the name's OWN words, so a clause beside the name changes neither. M2 is the first to bind as well, because its construct sits in front of the run it reads and the joins come later. C1 is the second (#613): the part after the comma is read once, after classify and before any join, by the reading a part holding no name word gets, and a postnominal part's roles are set there. segment had decided the structure before the tags existed, so it modelled group's particle chain (#562's pair) and assign's reading of a run the capitals settle, and assign re-decided #296's positional read with a second count; all three went. S2 is the third (#614): the trailing run of a main segment is read once at group's head, before any join, and split off, so the joins that had re-asked the peel before and after themselves -- the particle chain's stop and its rollback, the bound-given reserve's two views, the frozen link's bound -- answer from that one reading, and assign places the run off it rather than reading again. The one join that must come first is the one the run itself depends on: a particle takes the words straight behind it that are both particles and suffix vocabulary, as its run always did, or the read takes them as credentials. What the bind cost is the same kind as M2's: the run now wins over a join that used to reach into it, so garbage with a connective beside the run reads differently, and a pinned P5 shape the joined view had decided moved (decisions.md#S2, 2026-10-07). Two limits, both measured on #601. Read with the TRAILING rule, not the full parse: a head the full parse reads differently (H1's lone title, a P5 join) would otherwise move where the construct ends. And binding means the run can read differently from the same words with no construct (`JANE Q. DOE GEB. LE DO DO` gives suffix `DO DO`, while the full parse of `JANE Q. DOE DO DO` chains them into the family), which rules.md#M2 accepts in so many words. Lives in. nameparser/_pipeline/_group.py (`_maiden_take`: the view, `tail_reading` over it, and the role-tagged pieces `group()` applies); nameparser/_pipeline/_pieces.py (`tail_reading`, H5's chain); the P3 own-word test in `_group.py`; nameparser/_pipeline/_comma.py (C1's read-and-bind); `_pieces.read_trailing_run` and the head of `_group._group_segment` (S2's). Reach for it when. A rule is growing stops, exceptions or a release check whose job is to keep one construct from disturbing the rest of the name, and above all when that check has started to model another stage. Ask what the name reads as without the construct, and make that the rule.
50+
Problem shape. An optional construct sits inside a name — a maiden clause, a trailing title — and the requirement is that writing it changes nothing else about the name. Checked after the fact, that invariant has to be defended at every place the construct can end, and each defense has to model what later stages would do to the words it gives up. Contract statement. Read the name as if the construct were not written, using the rule that reads that position, not the whole parse. The construct takes only what that reading leaves. Where later stages could still move what the reading decided, give those words their roles at the take so no later join can reach them. The invariant then holds by construction and is not a property each stop must defend. How it works. M2 before #601 scanned forward, stopped at suffix-looking words, and kept a release check asking at each stop whether the name left standing would read what was given up as post-nominals. That check modelled the P2 chain, the P5 bound join and P6's attachment, and #548 was it failing at the one stop that never asked. #601 replaced the stops and the check with the clause-free reading plus the bind: a net 689 lines out of `_group.py` (+210/−899 in the #601 commit), and #535's grids went from 15,586 and 8,713 invariant violations to 0 (prototype measurement, 2026-10-03, #601's comments). The first half already appears twice. H5's trailing title is TRANSPARENT: "what stands once the chain is taken reads exactly as it would read written without the title, plus the title". P3 asks its word count and its one-case question of the name's OWN words, so a clause beside the name changes neither. M2 is the first to bind as well, because its construct sits in front of the run it reads and the joins come later. C1 is the second (#613): the part after the comma is read once, after classify and before any join, by the reading a part holding no name word gets, and a postnominal part's roles are set there. segment had decided the structure before the tags existed, so it modelled group's particle chain (#562's pair) and assign's reading of a run the capitals settle, and assign re-decided #296's positional read with a second count; all three went. S2 is the third (#614): the trailing run of a main segment is read once, between P3's connective joins and the particle chain, and split off, so the joins that had re-asked the peel before and after themselves -- the chain's stop and its rollback, the bound-given reserve's two views -- answer from that one reading, and assign places the run off it. Where the read sits is the lesson. Joins the run does not depend on but whose units it counts come FIRST: read before P3's connectives, the H5 chain took the second title of a bare `Mr. and Mrs.` and left 'and' as the family name, and a joined pair counted as two words (#614's review). Joins the run does depend on come after, and the read counts what they will build instead -- the particle chain's run and the plain words after it as one unit, in a copy -- with every word the reading weighs left a word of its own; a shape-only roman numeral folded into that unit was the one more miss (decisions.md#S2, 2026-10-07). Two limits, both measured on #601. Read with the TRAILING rule, not the full parse: a head the full parse reads differently (H1's lone title, a P5 join) would otherwise move where the construct ends. And binding means the run can read differently from the same words with no construct (`JANE Q. DOE GEB. LE DO DO` gives suffix `DO DO`, while the full parse of `JANE Q. DOE DO DO` chains them into the family), which rules.md#M2 accepts in so many words. Lives in. nameparser/_pipeline/_group.py (`_maiden_take`: the view, `tail_reading` over it, and the role-tagged pieces `group()` applies); nameparser/_pipeline/_pieces.py (`tail_reading`, H5's chain); the P3 own-word test in `_group.py`; nameparser/_pipeline/_comma.py (C1's read-and-bind); `_pieces.read_trailing_run` and `_counting_view`, and the read after P3's joins in `_group._group_segment` (S2's). Reach for it when. A rule is growing stops, exceptions or a release check whose job is to keep one construct from disturbing the rest of the name, and above all when that check has started to model another stage. Ask what the name reads as without the construct, and make that the rule.
5151

5252
## LICENSED-BY-NEIGHBOR — a structural word is structural only beside the right neighbor
5353

‎docs/design/rules.md‎

Lines changed: 20 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -329,13 +329,13 @@ H5. Rationale: a word abbreviated with a period at the END of a name
329329
real it is the word left STANDING that carries it, which is H4's
330330
report and not this rule's.
331331
Accepted: the chain is read together with the trailing suffix
332-
run (S2), over the words as written and before any of them is
333-
joined, so no join reaches a word it took: a particle chain (P2)
334-
stops in front of a trailing title rather than taking it into
335-
the family name. After a family comma the given part is read
336-
once its words are joined, and a title word a connective joined
337-
into a longer unit there is no single word for the chain to
338-
take. A maiden clause ends where the run the end of the name
332+
run (S2), once the connectives have joined (P3) and before the
333+
particle chain is made, so a particle chain (P2) stops in front
334+
of a trailing title rather than taking it into the family name.
335+
A title word a connective joined into a longer unit is no single
336+
word for the chain to take. After a family comma the given part
337+
is read once all its words are joined, so there a particle chain
338+
takes a trailing title as before. A maiden clause ends where the run the end of the name
339339
reads without it begins (M2), so where a trailing rule reads the
340340
part a trailing title ends the clause and is a title.
341341
"John van der Berg Prof." → title="Prof."
@@ -451,9 +451,10 @@ P2. Rationale: a particle is written as part of the surname it
451451
precedes, and a title stands outside the name entirely.
452452
A particle joins the words after it into one name part, the
453453
join running until the next particle starts a group of its own,
454-
a trailing suffix begins — the trailing run S2 reads, once, over
455-
the words as written before any of them is joined, so the chain
456-
cannot change what it counts: a trailing roman numeral, or a
454+
a trailing suffix begins — the trailing run S2 reads, once,
455+
before the chain is made and counting the chain's own run as the
456+
one name word it will be, so the chain cannot change what it
457+
counts: a trailing roman numeral, or a
457458
bare acronym with words to spare, ends the chain as a suffix word
458459
does, and so does a bare ambiguous acronym written in capitals in
459460
a mixed-case name, which needs no words to spare; the same
@@ -723,9 +724,8 @@ P5. Rationale: some given-name words are incomplete alone — "abdul"
723724
there is to spare is what the name's trailing reading leaves:
724725
the trailing suffix run (S2) and the trailing title run (H5),
725726
each read over what the other leaves until neither takes
726-
anything more, are read once, over the words as written and
727-
before the join is tried, and the name words they leave are the
728-
words to spare. A trailing
727+
anything more, are read once, before the join is tried, and the
728+
name words they leave are the words to spare. A trailing
729729
roman numeral, or a bare acronym the run takes, or a trailing
730730
title word the run takes, is no
731731
word to spare. The join joins two name words into one and, read
@@ -1026,12 +1026,13 @@ S2. Rationale: generational suffixes and credentials are recognized
10261026
letter or one after each of two or more letter chunks, counts
10271027
unambiguously; a single trailing period is the abbreviation
10281028
shape any word can wear and does not. The run at a name's
1029-
trailing slot is read ONCE, over the words as written, before any
1030-
of them is joined into a longer part — a particle first taking
1031-
the words straight behind it that are both particles and suffix
1032-
vocabulary, which are its run's (P2) — and no join reaches a word
1033-
it took: a connective, a particle chain or a bound given name
1034-
joins only the words in front of it (P3, P2, P5). A
1029+
trailing slot is read ONCE, once the connectives have joined (P3)
1030+
and before the particle chain (P2) and the bound given-name join
1031+
(P5) are made, counting a particle run and the name words the
1032+
chain will take with it as the one name word it will make of them
1033+
— the words both particles and suffix vocabulary standing straight
1034+
behind a particle among them — and neither join reaches a word
1035+
the run took. A
10351036
BARE ambiguous acronym is consumed only when the name has words
10361037
to spare — as the second of two words it stays the family
10371038
name — and at the slots that report, either reading carries the

‎docs/release_log.rst‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,7 @@ Release Log
2222

2323
- **Fix a credential run after a comma losing the name in front of it when two surname particles stand side by side in the run.** ``HumanName("John Smith, PhD DO DO")`` gives first ``John``, last ``Smith``, suffix ``PhD DO DO``, where 2.2 and 2.3 gave first ``PhD``, last ``DO DO John Smith`` and 2.0 and 2.1 gave title ``PhD``, first ``DO DO`` -- 1.4.0's reading, restored. ``DO``, ``MC`` and ``VD`` are credentials and surname particles at once, and two particles next to each other had joined into one particle run that took the credentials apart; the part after the comma is now read as credentials before any particle can join it. ``John Smith, PhD vd DO`` gives suffix ``PhD vd DO`` the same way, and ``John Smith, MD DO DO`` gives suffix ``MD DO DO`` where 2.3 gave title ``MD``, first ``DO``, middle ``DO``. A single ``DO`` is still left to its capitals (``John Smith, PhD DO`` gives suffix ``PhD DO``), and with one name word before the comma a part of credentials is all suffixes (``Smith, PhD DO DO`` gives last ``Smith``, suffix ``PhD DO DO``, and ``Smith, MA DO DO`` last ``Smith``, suffix ``MA DO DO``, where 2.3 gave first ``PhD`` and first ``MA``, last ``DO DO Smith``). See the #562 entry under ``C1`` in ``docs/design/decisions.md`` (closes #562)
2424

25-
- **Fix a trailing title being taken into a surname that begins with a particle.** ``HumanName("John van der Berg Prof.")`` gives title ``Prof.``, last ``van der Berg``, where every release from 1.4.0 through 2.3.0 gave last ``van der Berg Prof.``. The words at the end of a name -- its suffixes and the titles behind them -- are now read once, before any particle or connective joins the words in front of them, so no join can take one of them into the name. See the #614 entry under ``S2`` in ``docs/design/decisions.md`` (closes #614)
25+
- **Fix a trailing title being taken into a surname that begins with a particle.** ``HumanName("John van der Berg Prof.")`` gives title ``Prof.``, last ``van der Berg``, where every release from 1.4.0 through 2.3.0 gave last ``van der Berg Prof.``. The suffixes and titles at the end of a name are now read before the surname's particles join the words after them, so the particles no longer take a title in. Behind a family comma the given part is read as before (``HumanName("Smith, John van der Berg Prof.")`` keeps middle ``van der Berg Prof.``). The same reading order brings back 1.4.0's ``HumanName("abdul J. V")``: first ``abdul J.``, last ``V``, where 2.2 and 2.3 gave first ``abdul``, middle ``J.``. See the #614 entry under ``S2`` in ``docs/design/decisions.md`` (closes #614)
2626

2727
- **New Policy field unlisted_dotted_suffixes, on by default: a dotted acronym nobody has listed is read by position.** ``HumanName("John Smith X.Y.Z.")`` gives suffix ``X.Y.Z.`` where every release gave last ``X.Y.Z.``, while ``Jack X.Y.Z.`` keeps its surname, the same words-to-spare rule a listed acronym takes -- and both readings are reported. After a comma the count is of the words before it, and two dotted single letters are the exception: they are how a person's initials are written, and two words before a comma may be one surname, so ``García Márquez, G.J.`` keeps first ``G.J.`` and last ``García Márquez`` and reports the fork, unless an unambiguous post-nominal in front of the initials that is not also a title, or another unlisted dotted word beside them, says otherwise (``John Smith, PhD X.Y.`` gives suffix ``PhD X.Y.``, while ``García Márquez, Ms G.J.`` keeps title ``Ms``, first ``G.J.``). Three letters or more read by the count, so ``John Smith, X.Y.Z.`` gives suffix ``X.Y.Z.`` -- and so does ``García Márquez, G.J.R.``, the accepted cost of the line, since initials are conventionally written apart (``García Márquez, G. J. R.``), as separate words this rule does not read (#563). Case is irrelevant here: the periods are the signal, so ``john smith x.y.z.`` reads the same way. Words the vocabulary does know are untouched (``M.A.``, ``Ph.D.``, ``A.B.C.``), a single trailing period is still not this shape (``John Smith Xyz.`` keeps last ``Xyz.``), and a dotted run at the FRONT of a name is untouched (``J.R.R. Tolkien``). One accident retires with it: a dotted word whose only vocabulary matches were SINGLE ASCII CHARACTERS -- the roman numerals the suffix list holds, and the lone digit ``2`` -- was reading as a generational suffix, so ``Jack X.Y.I.`` gives last ``X.Y.I.`` again, as 1.4.0 read it, while ``Msc.Ed.``, ``JD.CPA`` and ``Lt.Gov.`` are unchanged. The digit is why a dotted VERSION STRING moves with them and moves SILENTLY: ``John Smith 1.4.2`` gives last ``1.4.2`` where 2.3 gave suffix ``1.4.2``, and ``John Smith, 1.4.2`` gives first ``1.4.2``, last ``John Smith``. Such a token reports nothing at any policy -- it is no acronym either, the shape reading wanting every chunk alphabetic -- and a version string read as a credential was the same accident this retirement removes. That retirement is NOT behind this switch and stands either way -- setting it to ``False`` reads an unlisted dotted word as name material by position instead (``John Smith X.Y.Z.`` keeps last ``X.Y.Z.``), the pre-2.4 reading for THAT half alone. See the ``S2`` and ``suffix-acronym-collisions`` entries of ``docs/design/decisions.md`` (closes #516)
2828

‎nameparser/_pipeline/_assign.py‎

Lines changed: 8 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -2,8 +2,9 @@
22
33
Consumes: pieces + piece_tags (grouped), segments, structure, tokens,
44
one_case, and tail_reads -- rules.md#S2's trailing run as group read it
5-
once before its joins (#614), which the main segment's placement takes
6-
rather than peeling again.
5+
once, after P3's joins and before the particle chain (#614), which the
6+
main segment's placement takes rather than peeling again, save where
7+
group's joins left no name word in front of it.
78
Produces: tokens with roles set on every main-stream token, and the
89
folded-middle tag on the particles P6's attachment joins to the family
910
after a family comma (#613).
@@ -371,7 +372,11 @@ def _assign_main(seg_idx: int, state: ParseState,
371372
# group's joins then left none past the leading titles -- a join
372373
# that made the name a title ('Freiherr von Berg Dr. and Ed.
373374
# Prof.') -- the reading is taken here over what stands, as it was
374-
# everywhere before #614, so the name keeps a word.
375+
# taken everywhere before #614, so the name keeps a word. The same
376+
# procedure and not always the same answer: the run group read
377+
# split off was kept out of group's joins, so what stands differs
378+
# from what stood before #614 (that name read family 'von Berg Dr.
379+
# and Ed. Prof.' then).
375380
read = (state.tail_reads[seg_idx]
376381
if seg_idx < len(state.tail_reads) else None)
377382
if read is not None:

0 commit comments

Comments
 (0)