diff --git a/docs/design/decisions/vocabulary-model.md b/docs/design/decisions/vocabulary-model.md index abde56723..c2bc4247e 100644 --- a/docs/design/decisions/vocabulary-model.md +++ b/docs/design/decisions/vocabulary-model.md @@ -454,7 +454,7 @@ Open: #913 (the title words phase 2j left, read as titles though borne as names) - **2026-10-31 #753, phase 3j (its built clause that a caller's listing of a dropped word is dropped by the delta superseded by 2026-11-02 #754, the `Lexicon` surface, in effect since 4i-c, 2026-11-02 #754, sub-phase 4i-c; its head's "ONLY THE CHOSEN FILES AND NATIVE-SCRIPT VOCABULARY", item 4's "every Latin-script reading leaving" and the `UNCHOOSABLE` bullet's "a strict one drops both" superseded by 2026-11-02 #1005) — FILE-STRICT IS BUILT: `parser_for(..., strict=True)` COMPILES ONLY THE CHOSEN FILES AND NATIVE-SCRIPT VOCABULARY; A STRICT CALL CHOOSING NO LANGUAGE RAISES; NO `locales.choose`; THE WORDS STRICT UNGATES ARE DERIVED AND HELD BY A TEST; A FILE HOLDING NO READING IS A STRICT CHOICE ONLY; THE DEFAULT DOES NOT MOVE.** SUPERSEDES: in the 2026-10-30 #753, file-strict entry, its verdict's "the rest of its API is proposed, not decided, and nothing is built"; its "Proposed, not decided" bullet, every item decided as proposed save `locales.choose(*tags)`, which is left out; and the files-with-no-readings bullet's "strict `vi` needs vi.toml readings first". Carries out 3j of item 11 of the 2026-10-27 #753 entry, with the definition of the 2026-10-30 entry. The four API points below were decided on 2026-10-31; the rest of the 2026-10-30 entry's proposed bullet is built as proposed (no `Locale.strict` field; the chosen files' `default = false` readings turned on and no other file's; patches and fragments applied after the strict compile, untouched; `Lexicon.given_names` and `Lexicon.surnames` outside it; packs stacked as a union; a caller's listing of a dropped word dropped by the delta). Built on master `8438ccbb`. 1. **A strict call that chooses no language raises `ValueError`.** The message names the packs given and offers a pack that chooses a language, the first pack's code where a strict choice accepts it as one (`parser_for(locales.ZH, strict=True)` offers `Locale(code='zh', lexicon=Lexicon.empty(), languages=frozenset({'zh'}))`), else `en`; tests/v2/test_parser.py runs the offered code. 2. **No public `locales.choose`.** A bare choice is a `Locale` with an empty lexicon and the tags, the spelling docs/locales.rst teaches. - 3. **The words strict ungates are derived**, `_reading_index.strict_ungated`, each strict choice against the plain choice of the same tags and never against the default's markers, which miss a reading the choice turns on; tests/v2/test_reading_index.py holds every one-tag strict choice to a per-tag table, which is the 2026-10-30 entry's 16 words and 27 word-and-tag pairs over 13 tags exactly, so a word that joins or leaves is a reviewed diff. + 3. **The words strict ungates are derived**, `_reading_index.strict_ungated`, each strict choice against the plain choice of the same tags and never against the default's markers, which miss a reading the choice turns on; tests/v2/test_reading_index.py holds every one-tag strict choice to a per-tag table, which was the 2026-10-30 entry's 16 words and 27 word-and-tag pairs over 13 tags exactly on 2026-10-31, the strict-gap batches of 2026-11-03 moving it, so a word that joins or leaves is a reviewed diff. 4. **A strict choice accepts a tag whose file holds no reading** (hu, kk, kok, vi and zu today): under strict it is not inert, every Latin-script reading leaving. A plain choice of one is still refused, its message now saying a strict one accepts it. The reading index records every file's tag for this (`FILES` in nameparser/config/_index.py, read by `_reading_index.files`), a file holding only exclusions writing no row of `READINGS`. - The build refuses `prefer` and `strict` as data, saying a strength is the caller's (`parser_for(..., strict=True)`), where it had said each was reserved for phase 3; the RFC's phase 3 API section had decided the message. - `_reading_index`'s `UNCHOOSABLE` comment and `selected`'s refusal text, which said no choice removes a reading of `mul` or `und`, now say a plain choice never does and a strict one drops both. @@ -546,3 +546,6 @@ Open: #913 (the title words phase 2j left, read as titles though borne as names) - **2026-11-03 #1005, the strict-gap batches (Derek) — FIVE BATCHES OF WORDS A STRICT CHOICE LOST, EACH ITS OWN CHANGE: `jr`, `junior` AND `jnr` MOVE TO `mul.toml` (BATCH A, THIS ENTRY, WHICH MOVES NO DEFAULT OR PLAIN-CHOICE PARSE); fil.toml TAKES `de`, `del`, `san`, `la` AND `los`; ca.toml `de`, `del` AND `la`; si.toml `de`; ar.toml A `given` READING OF `sultan`, EACH MEASURED IN ITS OWN ENTRY.** Decided in chat on 2026-11-03 ("All five, here in order"), on the re-ranked strict-gap report (`tools/differential/config_diff.py gaps` on master f48c88a3, a broken record laid to the word whose own token moved; saved with its simulation in the session notes, not in a tree). Each batch is its own PR, measured on master; this entry records the decision and each batch's measurement as it lands, beside the change. - **`jr`, `junior`, `jnr` → `mul.toml`.** Written after names in every country's data, generations with no name reading in any file, so check 13 admits them: strict choices broke 1,036 records on `jr` across 15 languages' countries (fil 408, pt-BR 183, de 48, fr 48, zh 36, ta 36, ms 34, es 27), 246 on `junior` (pt-BR 153, pt 25, es 18) and 13 on `jnr` (ak), and every one falls to 0 with the move (simulated). It costs the records where a holder typed `Junior` into the family box or as a given name and strict, reading it as a name, agreed: pt-BR 57, fr 11, es 5, pt 3, fil 1, ak 1 (the review's re-run, 2026-11-03); and a given name `Junior` after a family comma reads as the generation, as the default reads it, with a report (`Silva, Junior João` under strict `pt-BR` → family 'Silva', suffix 'Junior João', suffix-or-name, where strict read given 'Junior'). On the change, over a grid of 21 shapes (each comma shape, cased, upper, lower and dotted spellings, a credential behind it) under the three orders: the default and the plain `pt-BR`, `fil` and `en` choices move no cell, nor strict `en`; strict `pt-BR`, `fil`, `ko` and `de` move 57 cells each, every one onto the default's reading of the generation (`Silva, Junior` → suffix 'Junior', where strict read given 'Junior'), save the parts of those names another file's word decides (`MD`'s report, en's `do`, pt's `da` and `de`). `junior` is also a given name (ORCID's given boxes 480 times, Brazil, Ghana and Cameroon most), which no file records: a vocabulary-collisions question, not a `mul.toml` one. - **Recompute.** `tools/differential/config_diff.py gaps` on the tree before and after (`gaps fil pt-BR de fr zh ta ms es pt ak` reaches every row above); the grid is the 21 names `Paulo Silva Junior`, `Paulo Silva Jr`, `Paulo Silva Jr.`, `Paulo Silva, Jr.`, `Silva, Paulo, Jr.`, `Silva Jr, Paulo`, `Silva Jr., Paulo`, `, Jr.`, `Silva, Jr.`, `Junior Silva`, `Silva, Junior`, `Paulo Junior Silva`, `PAULO SILVA JUNIOR`, `paulo silva junior`, `Paulo Silva JNR`, `Paulo Silva Jnr.`, `Paulo Silva Jr., MD`, `Dr. Paulo Silva Jr`, `Silva, Paulo Jr`, `Paulo de Souza Junior` and `Jr`, each parsed in both trees under the three orders and `default`, `plain:PT_BR`, `plain:fil`, `plain:en`, `strict:PT_BR`, `strict:fil`, `strict:en`, `strict:ko` and `strict:de` (`config_diff.build` names each), and the cells compared; the review's own grid of 10 spellings by 21 shapes over 15 plain and 17 strict choices moved 0 default, plain or strict `en` cells and 438 strict ones, every one onto the default's reading save another file's word. +- **2026-11-03 #1005, strict-gap batch B, fil (Derek) — fil.toml TAKES THE SPANISH-DERIVED PARTICLES OF FILIPINO SURNAMES, `de`, `del`, `san`, `la` AND `los`; NO DEFAULT OR PLAIN-CHOICE PARSE MOVES; UNDER STRICT `fil`, `del`, `la` AND `san` READ AS NEVER-GIVEN.** One of the five strict-gap batches decided in chat on 2026-11-03 ("All five, here in order"), on the re-ranked report (`tools/differential/config_diff.py gaps` on master f48c88a3; the batches' own entry is the 2026-11-03 #1005, the strict-gap batches entry). fil.toml held `dela` alone of these, so strict `fil` read every other one as a name word: it broke 473 records on `de`, 94 on `del`, 70 on `san`, 46 on `la` and 19 on `los` (`Helbert De La Cruz`, `Joshua Del Prado`, `Raul San Diego`, Philippine ORCID, PatentsView and PSC records), and with batch A's `jr` the language's 1,066 broken records fall to 28 (simulated with all five batches). Each entry copies es.toml's kind and properties (`de` and `los` state `may_be_given = false`, no file holding a given reading of either). + - **What moves.** The compiled default does not (`tools/vocab/build.py` reports `_compiled.py` current). Over a grid of 21 Filipino names (each comma shape, cased, upper and lower spellings, a title and a generation beside them) under the three orders, the default and plain `fil` move no cell; strict `fil` and strict `fil` + `en` move 50 each, toward the default's reading save one class: `del`, `la` and `san` hold a `given` reading in es.toml alone, which strict `fil` drops while it keeps fil.toml's particle reading, so there they read as never-given particles, and a bare `Del Rosario` reads family 'Del Rosario' where the default reads given 'Del' and reports particle-or-given. That reads the Filipino surname as the surname it is; `_reading_index.strict_ungated`'s reviewed table gains `fil: del, la, san`, as it holds `it`'s for the same three. The same class after a family comma loses the given name: `Santos, Del` reads family 'Santos Del' where the default and plain `fil` read given 'Del', as the default already reads `Santos, De`; 26 names of a 1-in-4 sample of the local corpora take that shape (`BITTNER, DEL`, mostly FEC). On Philippine records the class costs 2 (`La Reina Eunice Santos` and `La Trang` read all-family, `la` broke 2 under strict `fil` on the change), `del` and `san` none (the review's run, 2026-11-03). Not added: `las` (`Las Piñas`, 3 Philippine records under strict `fil`), left for a later batch. A plain choice reads `Reading.languages` of `de` and `la` with `fil` among the files (the default too; plain `fil` names `fil` alone). + - **Recompute.** `tools/differential/config_diff.py gaps fil` on the tree before and after; the grid is the 21 names `Juan De La Cruz`, `Juan dela Cruz`, `Maria Del Rosario`, `Jose San Juan`, `Juan De Los Santos`, `De La Cruz, Juan`, `Del Rosario, Maria Luisa`, `, De La Cruz`, `Cruz, Juan de la`, `JUAN DE LA CRUZ`, `juan de la cruz`, `Juan De Guzman Jr.`, `Del Rosario`, `La Cruz`, `San Juan`, `Juan Del`, `Maria La Paz Reyes`, `Dr. Jose De Leon`, `Del Pilar Santos`, `Juan Santos, De La` and `Ma. Teresa De Los Reyes`, each parsed in both trees under the three orders and `default`, `plain:fil`, `strict:fil` and `strict:fil+en` (`config_diff.build` names each), and the cells compared; the review's own grid of 39 names agreed in direction, and `config_diff.py diff default plain:fil --local all --match '(?i)\b(de|del|la|san|los)\b' --sample 4` read 0 of 61,947 names differently in either tree. diff --git a/docs/release_log.rst b/docs/release_log.rst index bc05df069..2eea013a8 100644 --- a/docs/release_log.rst +++ b/docs/release_log.rst @@ -16,7 +16,7 @@ Release Log - **Add a switch for names with a compound given name.** ``Parser(policy=Policy(two_given_names=True)).parse("Juan Carlos Pérez")`` gives given ``Juan Carlos``, where 1.4.0 through 2.8.0 gave given ``Juan``, middle ``Carlos``. Off by default: under it a middle name beside the given name joins it (``John Quincy Adams`` gives given ``John Quincy``). With ``two_family_names`` on too, the family takes its word first, so ``María José García López`` reads given ``María José``, family ``García López``. A ``Policy`` pickled by 2.8 loads with it off. :issue:`704` - **Add Lexicon.given_names, given names two_family_names leaves out of the family.** ``Parser(policy=Policy(two_family_names=True), lexicon=Lexicon.default() | Lexicon(given_names={"luis"})).parse("José Luis Pérez")`` gives middle ``Luis``, family ``Pérez``, where the switch alone, in 2.8.0 as now, gives family ``Luis Pérez``. Empty by default, so the switch alone reads as before; an entry without accents matches the word with them. :issue:`959` - **Add Token.reading, the reading the parse took for each word.** ``parse("John Smith PhD Jones").tokens[-1].reading`` is ``Reading('credential' as 'suffix', 'position')``, the reading 2.7's ``Ambiguity.alternative`` gave only for the other side of a fork, and ``ReadingKind`` gains ``patronymic``. The field is not compared, and a ``ParsedName`` pickled by 2.x loads with no readings. :issue:`754` - - **Add Reading.languages, the language files that list a word the way it was read.** ``parse("Dr. Juan de la Vega III").tokens[2].reading.languages`` is ``('es', 'fr', 'it', 'nl', 'pt')`` for the particle ``de``, and ``Ambiguity.alternative`` names its files too; a reading by shape or position names none. Under a choice of languages it names the files the choice reads: ``parser_for(locales.PT)`` names ``('pt',)`` for that ``de``. It is not compared, and a ``Reading`` pickled by 2.7 or 2.8 loads with none. The first parse in a process that meets a listed word loads the reading index once: about 1 ms, or about 16 ms where Python writes no bytecode cache. :issue:`754`, :issue:`753` + - **Add Reading.languages, the language files that list a word the way it was read.** ``parse("Dr. Juan de la Vega III").tokens[2].reading.languages`` is ``('es', 'fil', 'fr', 'it', 'nl', 'pt')`` for the particle ``de``, and ``Ambiguity.alternative`` names its files too; a reading by shape or position names none. Under a choice of languages it names the files the choice reads: ``parser_for(locales.PT)`` names ``('pt',)`` for that ``de``. It is not compared, and a ``Reading`` pickled by 2.7 or 2.8 loads with none. The first parse in a process that meets a listed word loads the reading index once: about 1 ms, or about 16 ms where Python writes no bytecode cache. :issue:`754`, :issue:`753` - **Add Czech, Slovak, Slovenian and Croatian packs, locales.CS, SK, SL and HR.** ``parser_for(locales.CS).parse("Jan Novák, MUDr")`` gives title ``MUDr``, given ``Jan``, family ``Novák``, where 1.4.0 through 2.8.0 gave given ``MUDr``, family ``Jan Novák``. Each pack chooses its one language and turns on words the default does not read: ``ml.`` and ``st.`` after the name in all four (``Jiří Dienstbier ml.``), the Czech and Slovak academic titles and ``DrSc.``, and Czech ``Pan`` and ``roz.``. Choosing the language drops ``st``'s Saint readings (``St Clair Smith`` reads given ``St``), which choosing ``fr`` as well keeps. ``CS`` and ``SK`` also read two family names, so ``Alena Pešová Řehořová`` gives family ``Pešová Řehořová`` and two given names read as a given name and a family name (``Petr Pavel Novák`` gives family ``Pavel Novák``); a family comma keeps both. :issue:`753` - **Add Hindi and Bengali packs, locales.HI and locales.BN, for the romanized Indic titles.** ``parser_for(locales.HI).parse("Smt Nirmala Sitharaman")`` gives title ``Smt``, given ``Nirmala``, where 1.4.0 through 2.8.0 gave given ``Smt``, middle ``Nirmala``. ``HI`` reads ``Smt``, ``Shrimati`` and ``Shriyut`` as titles and ``BN`` ``Smt``, ``Shrimati`` and ``Adhyapak``; ``Shri`` and ``Sushri`` stay name words under both, since each opens given names (``Shri Kant``, ``Sushri Sangita``). See :doc:`locales`. :issue:`345`, :issue:`753` - **Add parser_for(..., strict=True), to read only the chosen languages' words.** ``parser_for(Locale(code="ko", lexicon=Lexicon.empty(), languages=frozenset({"ko"})), strict=True).parse("Kim Do Hyun")`` gives middle ``Do``, family ``Hyun``, where the default, as in 1.4.0 through 2.8.0, and a plain ``ko`` choice give family ``Do Hyun``, ``Do`` read as the Portuguese particle. Only the chosen languages' vocabulary files are read, with the words every language writes alike (``Dr``, ``PhD``, ``Mr``, the numerals) and the words written in other scripts; every other language's words leave, the rest of the English credentials and salutations included, so ``Kim Min Jun, MD`` gives given ``MD`` unless ``en`` is chosen too. A strict call that chooses no language raises ``ValueError``. ``python -m nameparser`` gains ``--strict``, and ``--locale`` takes a language tag no pack has (``--locale ko``). See :doc:`locales`. :issue:`753`, :issue:`1005` diff --git a/docs/usage.rst b/docs/usage.rst index 41c338dbb..64514a686 100644 --- a/docs/usage.rst +++ b/docs/usage.rst @@ -1270,8 +1270,8 @@ file does hold turns that file's reading on, and names it: ... print(f"{tok.text}: {tok.reading.languages}") Dr.: ('mul',) Juan: () - de: ('es', 'fr', 'it', 'nl', 'pt') - la: ('es', 'fr', 'it') + de: ('es', 'fil', 'fr', 'it', 'nl', 'pt') + la: ('es', 'fil', 'fr', 'it') Vega: () III: ('mul',) diff --git a/nameparser/_types.py b/nameparser/_types.py index 9e4ce819c..04beb05b5 100644 --- a/nameparser/_types.py +++ b/nameparser/_types.py @@ -958,7 +958,7 @@ class Reading: source: ReadingSource #: The BCP 47 tags of the shipped vocabulary files that hold this #: reading of the word, on in the parse's own ``Lexicon`` - #: (``("es", "fr", "it", "nl", "pt")`` for the particle ``de`` by + #: (``("es", "fil", "fr", "it", "nl", "pt")`` for the particle ``de`` by #: default, ``("pt",)`` under a ``pt`` choice, which drops the #: others' readings of it; ``("mul",)`` for the title ``Dr.``), so #: a choice of languages names the files it turns on and not those diff --git a/nameparser/config/_index.py b/nameparser/config/_index.py index deef21946..5078c1a13 100644 --- a/nameparser/config/_index.py +++ b/nameparser/config/_index.py @@ -573,14 +573,14 @@ ('dcvo', (('en', 'credential', ()),)), ('dd', (('en', 'credential', ()),)), ('dds', (('en', 'credential', ()),)), - ('de', (('es', 'preposition', ()), ('fr', 'preposition', ()), ('it', 'preposition', ()), ('nl', 'article', ()), ('pt', 'preposition', ()))), + ('de', (('es', 'preposition', ()), ('fil', 'preposition', ()), ('fr', 'preposition', ()), ('it', 'preposition', ()), ('nl', 'article', ()), ('pt', 'preposition', ()))), ("de'", (('it', 'preposition', ()),)), ('deacon', (('en', 'religious', ()), ('en', 'given', ()), ('en', 'family', ()))), ('dean', (('en', 'office', ()), ('en', 'given', ()), ('en', 'family', ()))), ('ded', (('en', 'credential', ()),)), ('degli', (('it', 'preposition', ()),)), ('dei', (('it', 'preposition', ()),)), - ('del', (('es', 'preposition', ()), ('es', 'given', ()), ('it', 'preposition', ()))), + ('del', (('es', 'preposition', ()), ('es', 'given', ()), ('fil', 'preposition', ()), ('it', 'preposition', ()))), ('dela', (('fil', 'preposition', ()), ('fil', 'given', ()))), ('delegate', (('en', 'office', ()),)), ("dell'", (('it', 'preposition', ()),)), @@ -975,7 +975,7 @@ ('kreivi', (('fi', 'nobility', ()),)), ('kt', (('en', 'credential', (('cased', 'Kt'),)),)), ("l'", (('fr', 'article', ()), ('it', 'article', ()))), - ('la', (('es', 'article', ()), ('es', 'given', ()), ('fr', 'article', ()), ('it', 'article', ()))), + ('la', (('es', 'article', ()), ('es', 'given', ()), ('fil', 'article', ()), ('fr', 'article', ()), ('it', 'article', ()))), ('lac', (('en', 'credential', ()), ('en', 'family', ()))), ('ladc', (('en', 'credential', ()),)), ('lady', (('en', 'nobility', (('attaches', ('justice',)),)),)), @@ -1035,7 +1035,7 @@ ('lmsw', (('en', 'credential', ()),)), ('lmt', (('en', 'credential', ()),)), ('lord', (('en', 'nobility', (('attaches', ('chancellor', 'justice')),)),)), - ('los', (('es', 'article', ()),)), + ('los', (('es', 'article', ()), ('fil', 'article', ()))), ('lp', (('en', 'credential', ()),)), ('lpa', (('en', 'credential', ()),)), ('lpc', (('en', 'credential', ()),)), @@ -1531,7 +1531,7 @@ ('sainte', (('fr', 'religious', ()),)), ('same', (('en', 'credential', ()),)), ('saml', (('en', 'given', (('same_as', 'samuel'),)),)), - ('san', (('es', 'saint', ()), ('es', 'given', ()), ('it', 'saint', ()))), + ('san', (('es', 'saint', ()), ('es', 'given', ()), ('fil', 'saint', ()), ('it', 'saint', ()))), ('santa', (('es', 'saint', ()), ('es', 'given', ()), ('it', 'saint', ()), ('pt', 'saint', ()))), ('saoshyant', (('en', 'religious', ()),)), ('sasm', (('en', 'credential', ()),)), diff --git a/nameparser/vocab/fil.toml b/nameparser/vocab/fil.toml index c80829399..167b17e20 100644 --- a/nameparser/vocab/fil.toml +++ b/nameparser/vocab/fil.toml @@ -12,6 +12,14 @@ salutation = { issue = 606, gender = "f", note = "borne: ~500 surnames, in Phili text = "ginoo" salutation = { issue = 606, gender = "m", note = "borne: ~500 surnames, in Philippine lists writing Surname, Given with a comma, forebears.io, worldwide, rounded, 2026-10-04; shipped under the graf class (decisions/salutation-titles)", after_given = true } +[[word]] +text = "de" +preposition = { may_be_given = false, expands = "of: Spanish-derived Filipino surnames, \"De Guzman\", \"De La Cruz\"", note = "strict fil broke 473 records on it while only es.toml and others held it (config_diff.py gaps on f48c88a3, decisions/vocabulary-model, 2026-11-03)", issue = 1005 } + +[[word]] +text = "del" +preposition = { expands = "of the: \"Del Rosario\"", note = "strict fil broke 94 records (config_diff.py gaps on f48c88a3, decisions/vocabulary-model, 2026-11-03)", issue = 1005 } + [[word]] text = "dela" given = { us_ssa = 565, as_of = 2026-10-25, note = "US SSA births summed over both sexes; nam_dict band 1 in Germany, 1 in Bulgaria" } @@ -19,6 +27,18 @@ given = { us_ssa = 565, as_of = 2026-10-25, note = "US SSA births summed over bo expands = "of the, de la contracted: \"Dela Cruz\"" note = "Dela is a short form of Adela, and an African name meaning saviour/redeemer -- so the Filipino surname particle is not the only reading" +[[word]] +text = "la" +article = { expands = "the, fem. sg.: \"De La Cruz\"", note = "strict fil broke 46 records (config_diff.py gaps on f48c88a3, decisions/vocabulary-model, 2026-11-03)", issue = 1005 } + +[[word]] +text = "los" +article = { may_be_given = false, expands = "the, masc. pl.: \"De Los Santos\"", note = "strict fil broke 19 records (config_diff.py gaps on f48c88a3, decisions/vocabulary-model, 2026-11-03)", issue = 1005 } + +[[word]] +text = "san" +saint = { expands = "Saint: \"San Juan\", \"San Jose\"", note = "strict fil broke 70 records (config_diff.py gaps on f48c88a3, decisions/vocabulary-model, 2026-11-03)", issue = 1005 } + [[word]] text = "jo" given = { us_ssa = 180911, wikidata = 974, as_of = 2026-10-26, same_as = "josé", note = "José written short, in Philippine usage; the counts are of Jo, a given name of its own (US SSA births summed over both sexes, F 179,057, M 1,854; Wikidata a lower bound) and a Korean surname, so a period-marked Jo. opening the given part is a given name on every reading (#789)" } diff --git a/tests/v2/pipeline/test_assemble.py b/tests/v2/pipeline/test_assemble.py index 864b944a7..9530dec0b 100644 --- a/tests/v2/pipeline/test_assemble.py +++ b/tests/v2/pipeline/test_assemble.py @@ -482,8 +482,8 @@ def test_every_suffix_words_reading_alternative_and_index_kind_agree( _EXPECTED_LANGUAGES = [ # a particle, by its particle rows only ('la' is a given name in es # too, a reading the parse did not take) - ("Dr. Juan de la Vega III", "de", ("es", "fr", "it", "nl", "pt")), - ("Dr. Juan de la Vega III", "la", ("es", "fr", "it")), + ("Dr. Juan de la Vega III", "de", ("es", "fil", "fr", "it", "nl", "pt")), + ("Dr. Juan de la Vega III", "la", ("es", "fil", "fr", "it")), ("Dr. Juan de la Vega III", "Dr.", ("mul",)), ("Dr. Juan de la Vega III", "III", ("mul",)), ("John Smith PhD", "PhD", ("mul",)), diff --git a/tests/v2/test_parser.py b/tests/v2/test_parser.py index c4fed7a40..c6c01017e 100644 --- a/tests/v2/test_parser.py +++ b/tests/v2/test_parser.py @@ -2972,7 +2972,7 @@ def test_reading_languages_follow_the_choice() -> None: assert _languages_of(pt, "Vincent van Gogh", "van") == ("nl",) # with no choice, the files on by default, as before assert _languages_of(Parser(), "Ana de Souza", "de") == ( - "es", "fr", "it", "nl", "pt") + "es", "fil", "fr", "it", "nl", "pt") def test_reading_languages_under_a_strict_choice_name_the_chosen_files() -> None: diff --git a/tests/v2/test_reading_index.py b/tests/v2/test_reading_index.py index 08eb54fd6..c05d0289a 100644 --- a/tests/v2/test_reading_index.py +++ b/tests/v2/test_reading_index.py @@ -340,13 +340,15 @@ def test_a_file_holding_no_reading_is_chosen_only_strictly() -> None: #: The words each strict choice of one tag ungates against the plain #: choice of the same tag (decisions/vocabulary-model's 2026-10-30 #753, -#: file-strict entry: 16 words, 27 word-and-tag pairs over 13 tags). A +#: file-strict entry measured 16 words, 27 word-and-tag pairs over 13 +#: tags on 2026-10-31; the strict-gap batches of 2026-11-03 moved it). A #: word leaves this table when a name reading is written in the chosen #: file whose names carry it, and joins it when a chosen file's #: vocabulary reading loses its gate: review each move. STRICT_UNGATED = { "ar": {"cheikh", "sultan"}, "cs": {"st"}, + "fil": {"del", "la", "san"}, "fr": {"la", "marquis", "marquise", "st"}, "ga": {"mac"}, "gd": {"mac", "nic"},