diff --git a/docs/design/decisions/vocabulary-model.md b/docs/design/decisions/vocabulary-model.md index 2f8c08312..d398cd5c7 100644 --- a/docs/design/decisions/vocabulary-model.md +++ b/docs/design/decisions/vocabulary-model.md @@ -555,3 +555,6 @@ Open: #913 (the title words phase 2j left, read as titles though borne as names) - **2026-11-03 #1005, strict-gap batch D, si (Derek) — si.toml TAKES `de`, THE PARTICLE OF SRI LANKAN SURNAMES OF PORTUGUESE ORIGIN, ITS FIRST LATIN-SCRIPT WORD; NO DEFAULT OR PLAIN-CHOICE PARSE MOVES.** One of the five strict-gap batches decided in chat on 2026-11-03 ("All five, here in order"), on the re-ranked report (`tools/differential/config_diff.py gaps` on master f48c88a3; the batches' own entry is the 2026-11-03 #1005, the strict-gap batches entry). Sri Lankan names write `de` before a surname (`de Silva`, `de Mel`, `de Alwis`) in Latin letters, and strict `si` broke 334 of the language's 340 broken records on it (`Sampath Damika De Silva`); the 340 fall to 6 on this change alone. The review read all 330 Sri Lankan ORCID names holding `de` and found every one Sinhalese (`De Silva`, `de Alwis`, `De Zoysa`), so ta.toml, whose tag also stands under LK, is owed nothing; strict `ta`'s other `de` losses are Goan names of India, pt's (2026-11-03). The entry copies es.toml's (`may_be_given = false`, no file holding a given reading). - **What moves.** The compiled default does not (`tools/vocab/build.py` reports `_compiled.py` current). Over a grid of 12 Sri Lankan names (each comma shape, cased, upper and lower spellings, a title and a credential beside them) under the three orders, the default and plain `si` move no cell; strict `si` and strict `si` + `en` move 28 each, every one onto the default's reading. The reviewed table of the words strict ungates does not move. `Reading.languages` of `de` gains `si` beside es, fr, it, nl and pt under the default and every other plain choice, and plain `si` names `si` alone, its own file's reading replacing the others' (each holds `de` as a never-given particle, so no role moves). - **Recompute.** `tools/differential/config_diff.py gaps si ta` on the tree before and after; the grid is the 12 names `Sampath Damika De Silva`, `Nimal de Silva`, `de Silva, Nimal`, `De Silva, Nimal Anura`, `, de Silva`, `NIMAL DE SILVA`, `nimal de silva`, `De Silva`, `Anura De Mel`, `Dr. Nimal de Silva`, `Nimal de Silva, PhD` and `Ruwan Kumara de Alwis`, each parsed in both trees under the three orders and `default`, `plain:si`, `strict:si` and `strict:si+en` (`config_diff.build` names each), and the cells compared; the review's own grid of 915 cells agreed, and `config_diff.py diff default plain:si strict:si --local all --match '(?i)(^|[\s,.])de([\s,.]|$)' --sample 4 --readings` read the default and plain `si` alike in both trees over 59,321 names. +- **2026-11-03 #1005, strict-gap batch F, ar (Derek) — ar.toml GIVES `sultan` A `given` READING, THE ARABIC GIVEN NAME, AND A `family` READING, SO STRICT `ar` KEEPS ITS GATE; NO ROLE MOVES UNDER THE DEFAULT OR A PLAIN CHOICE.** One of the five strict-gap batches decided in chat on 2026-11-03 ("All five, here in order"), on the re-ranked report (`tools/differential/config_diff.py gaps` on master f48c88a3; the batches' own entry is the 2026-11-03 #1005, the strict-gap batches entry). `sultan`'s `given` and `family` readings stood in tr.toml alone, so strict `ar` kept ar.toml's nobility reading and dropped the gate (decisions/vocabulary-model's derived-marker clause; rules/H9), reading the given name as a title: 226 records broken over the records of Arab countries (`Sultan Sameea`, Bahrain), all of them recovered on this change alone (strict `ar` 623 broken to 397). The readings carry their own counts, taken 2026-11-03 with `tools/name_usage.py --boxes`: given, ORCID's Saudi given boxes 265 (0.48%) and the UAE's 32, PatentsView's Saudi 46, nam_dict band 3 in Arabia/Persia; family, ORCID's family boxes in Pakistan 141, Iraq 84, Egypt 59, Saudi Arabia 46 and the UAE 13 (ar.toml's `sultana` carries one too). The `family` reading moves no role in the grid; it names `ar` in `Reading.languages` where strict `ar` read a family `Sultan` with no file. + - **What moves.** The compiled default does not (`tools/vocab/build.py` reports `_compiled.py` current). Over a grid of 15 names (each comma shape, cased, upper and lower spellings, `bin`, a title, a title phrase) under the three orders, the default and plain `ar` move no cell; strict `ar` moves 36 and strict `ar` + `en` 33, onto the default's reading, save `Sultan of Oman` under strict `ar` alone, whose phrase needs en.toml's `of` (with `en` chosen it reads title 'Sultan of Oman', as the default does). `_reading_index.strict_ungated`'s reviewed table drops `sultan` from `ar`, keeping it under `ms`, whose file holds the nobility reading and no name one: strict `ms` still reads `Sultan` as a title in 721 of the 2,842 local names holding it (`ASHRAF, SULTAN` → title), the next case of this gap. No role moves under the default or a plain choice, but `Reading.languages` of a given or family `Sultan` gains `ar` beside `tr`. + - **Recompute.** `tools/differential/config_diff.py gaps ar ms tr` on the tree before and after; the grid is the 15 names `Sultan Sameea`, `Sultan Al Qasimi`, `Mr Sultan Khan`, `Al Qasimi, Sultan`, `Sultan, Ahmed`, `Ahmed Sultan`, `, Sultan`, `SULTAN SAMEEA`, `sultan sameea`, `Sultan`, `Sultan bin Zayed`, `Sultan Qaboos bin Said`, `Dr. Sultan Sameea`, `Hurrem Sultan` and `Sultan of Oman`, each parsed in both trees under the three orders and `default`, `plain:ar`, `strict:ar` and `strict:ar+en` (`config_diff.build` names each), and the cells compared; `config_diff.py diff default plain:ar strict:ar plain:ms strict:ms --local all --match '(?i)sultan' --readings` moved only strict `ar` (724 of 2,842 names) in the review's run. diff --git a/nameparser/config/_index.py b/nameparser/config/_index.py index 5b8d9e8b1..e2eaffb4b 100644 --- a/nameparser/config/_index.py +++ b/nameparser/config/_index.py @@ -1645,7 +1645,7 @@ ('subaltern', (('en', 'military', ()),)), ('subedar', (('hi', 'military', ()),)), ('suffragist', (('en', 'occupation', ()),)), - ('sultan', (('ar', 'nobility', ()), ('ms', 'nobility', ()), ('tr', 'nobility', ()), ('tr', 'given', ()), ('tr', 'family', ()))), + ('sultan', (('ar', 'nobility', ()), ('ar', 'given', ()), ('ar', 'family', ()), ('ms', 'nobility', ()), ('tr', 'nobility', ()), ('tr', 'given', ()), ('tr', 'family', ()))), ('sultana', (('ar', 'nobility', ()), ('ar', 'given', ()), ('ar', 'family', ()))), ('superior', (('en', 'modifier', ()),)), ('supreme', (('en', 'modifier', ()),)), diff --git a/nameparser/vocab/ar.toml b/nameparser/vocab/ar.toml index 0e85ecf3c..56b344529 100644 --- a/nameparser/vocab/ar.toml +++ b/nameparser/vocab/ar.toml @@ -130,6 +130,8 @@ religious = { addresses = "given", gender = "f", issue = 269, expands = "the haj [[word]] text = "sultan" nobility = {} +given = { note = "the Arabic given name, counted 2026-11-03: ORCID given boxes Saudi Arabia 265 (0.48%), UAE 32; PatentsView given boxes Saudi Arabia 46 (0.41%); nam_dict band 3 in Arabia/Persia (tools/name_usage.py --boxes); tr.toml's reading carries the Wikidata total. Strict ar, which drops tr.toml's given reading, read it as a title in 226 records of Arab countries (config_diff.py gaps on f48c88a3, decisions/vocabulary-model, 2026-11-03)" } +family = { note = "an Arabic surname, counted 2026-11-03: ORCID family boxes Pakistan 141, Iraq 84, Egypt 59, Saudi Arabia 46, UAE 13 (tools/name_usage.py --boxes)" } [[word]] text = "sultana" diff --git a/tests/v2/test_reading_index.py b/tests/v2/test_reading_index.py index 4eb2d6278..e0033cbb8 100644 --- a/tests/v2/test_reading_index.py +++ b/tests/v2/test_reading_index.py @@ -346,7 +346,7 @@ def test_a_file_holding_no_reading_is_chosen_only_strictly() -> None: #: file whose names carry it, and joins it when a chosen file's #: vocabulary reading loses its gate: review each move. STRICT_UNGATED = { - "ar": {"cheikh", "sultan"}, + "ar": {"cheikh"}, "ca": {"del", "la"}, "cs": {"st"}, "fil": {"del", "la", "san"},