Skip to content

Provide constants in non-Latin scripts (Cyrillic, Greek, Arabic, Hebrew) #269

Description

@derek73

The parsing machinery is fully Unicode-clean — \w, .lower(), and the piece logic all handle non-Latin scripts — but the constant sets (titles, prefixes, conjunctions, suffixes) are almost entirely Latin-script. Verified on v1.3.0 that runtime additions work end-to-end with no code changes, e.g.:

>>> C = Constants()
>>> C.titles.add('г-н')
>>> C.prefixes.add('بن')
>>> HumanName("г-н Иван Петров", constants=C).title
'г-н'
>>> HumanName("محمد بن سلمان", constants=C).last
'بن سلمان'

Unlike Latin transliterations (which collide across cultures and need opt-in flags), native-script constants cannot collide with Latin-script names, so they are viable as defaults. Only within-script ambiguity needs vetting.

Candidates:

  • Cyrillic titles: г-н, г-жа, д-р, проф, акад; Ukrainian пан, пані. (Abbreviated forms like проф. currently work only by accident via the leading-position period_abbreviation fallback; hyphenated г-н doesn't.)
  • Cyrillic conjunctions: и; Ukrainian і/та (interacts with the initial-regex question, see Should a bare 'И' parse as an initial or the conjunction 'and'? (e.g. 'Хосе И Мария Сантос') #267)
  • Cyrillic suffixes: мл/ст (junior/senior) — needs within-script collision vetting
  • Greek: titles κ, κα, κος, δρ, καθ; conjunction και
  • Arabic script prefixes: بن (bin), بنت (bint), ابن (ibn), أبو/ابو (abu), آل (Āl); title الشيخ. Excluded: standalone ال (normally attached, too ambiguous), conjunction و (attaches without a space)
  • Hebrew prefixes: בן (ben), בת (bat); title מר. Titles containing geresh/gershayim (ד"ר, גב') need testing against the nickname-extraction regexes first.

Out of scope: CJK (family-first order and unspaced segmentation — see #83), Cyrillic patronymic suffixes (already covered by the 1.3.0 regexes).

Coverage should be parsing-behavior tests per script, not assertions on set contents. Related: #265, #266.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions