Skip to content

Add industry vocabulary support - #81

Merged
IchenDEV merged 2 commits into
mainfrom
codex/industry-vocabulary
Aug 24, 2026
Merged

Add industry vocabulary support#81
IchenDEV merged 2 commits into
mainfrom
codex/industry-vocabulary

Conversation

@IchenDEV

@IchenDEV IchenDEV commented Aug 24, 2026

Copy link
Copy Markdown
Owner

Summary

  • add versioned zh-CN vocabulary packs for medical, legal, finance/accounting, and software technology, with reference-only source and redistribution metadata
  • route the selected pack through Apple Speech and Whisper contextual biasing, conservative output normalization, and correction-only LLM prompt guidance while keeping personal terms at higher priority
  • add a native macOS settings picker with searchable term previews, localization, catalog validation, deterministic evaluation, and primary-source research artifacts

Chinese industry lexicon landscape

  • map 18 mainland-Chinese industry areas spanning IT, cybersecurity, medicine/TCM, law, finance, manufacturing, energy, construction, transport, agriculture, education, government service, e-commerce, telecom, aerospace, publishing, environment/weather, and tourism/catering
  • classify sources as bundled, reference-only, or import-only; public access is not treated as commercial redistribution permission
  • identify THUOCL/MIT as the strongest bundled candidate (157,171 normalized rows / 156,668 unique at pinned commit), subject to age, format, quality, and real-audio gates
  • keep national standards, TermOnline, and ministry/agency terminology as reference-only unless separately licensed; recommend local user import for Sogou, Baidu, and mixed/proprietary input-method ecosystems

Evaluation

The deterministic text corpus validates catalog loading, selection, priority, correction, and non-target preservation:

  • baseline target-term recall: 20.59%
  • treatment target-term recall: 100.00%
  • absolute improvement: 79.41 percentage points
  • non-target preservation: 100.00%
  • recognition context: at most 44 phrases per bundled pack, below the 100-phrase limit

This is not a real-audio ASR quality claim. The research artifacts define the frozen paired-audio release gates; that corpus has not yet been recorded and run.

Tests

  • bash scripts/ci-basic-checks.sh
  • PYTHONDONTWRITEBYTECODE=1 python3 -m unittest scripts/tests/test_evaluate_voice_quality.py (6 passed)
  • Swift frontend parse for all source and XCTest Swift files
  • jq empty Sources/Resources/IndustryLexicons.json
  • git diff --check origin/main...HEAD

swift test --filter IndustryLexiconTests is currently blocked before application tests compile because this machine is configured with Command Line Tools only and the MLX dependency cannot find the metal compiler. The standalone evaluator compiles the production catalog and replacement-engine sources directly and passes.

@IchenDEV
IchenDEV marked this pull request as ready for review August 24, 2026 07:03
@IchenDEV
IchenDEV force-pushed the codex/industry-vocabulary branch from 638ea26 to dc6768d Compare August 24, 2026 07:30
@IchenDEV
IchenDEV merged commit de45ea8 into main Aug 24, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant