Repository navigation
fix(metrics): size the WER cost matrix for long inputs - #3784
Merged
Merged
Conversation
compute_wer_by_line allocated the edit-distance matrix as int16. An edit distance can be as large as max(len_hyp, len_ref), so the first row/column initialisation raises OverflowError as soon as either input reaches 32768 tokens, e.g. one 32768-token hypothesis against a one-token reference. Use int32, which covers any input this O(len_hyp * len_ref) loop can realistically reach. Results for existing inputs are unchanged. Fixes modelscope#3781 Signed-off-by: Batianxieshen1 <280706568+Batianxieshen1@users.noreply.github.com>
LauraGPT
approved these changes
Oct 20, 2026
LauraGPT
left a comment
Collaborator
There was a problem hiding this comment.
Validated against current main 879a219c717123fdec6024e8e56c734a7aace2ff and exact head abc71f8019882a1ff598493858b60ed0cddadb34.
The submitted boundary tests reproduce the NumPy int16 OverflowError on the base (2 failures / 4 tests) and pass on both the exact head and current synthetic merge (4/4 each). The adjacent RapidFuzz metric tests pass 2/2 on both trees, changed-file compilation passes, and recursive tree reconciliation shows only funasr/metrics/wer.py and tests/test_metrics_wer.py differ from current main. The int32 change is appropriately scoped for this CPU-only edit-distance helper; no model or GPU runtime behavior is changed.
2 of 9 tasks
LauraGPT
pushed a commit
that referenced
this pull request
Oct 21, 2026
…3785) Four standalone copies of compute_wer_by_line still allocated the edit-distance matrix as int16: the runtime bundles (runtime/python/utils and runtime/python/libtorch/funasr_torch) and the two Aishell examples. They raise the same OverflowError once either input reaches 32768 tokens, which #3784 fixed for the library copy in funasr/metrics/wer.py. Apply the same int32 sizing, and cover every copy from tests/test_metrics_wer.py so the duplicates cannot silently regress: one boundary case per file plus an agreement check against the library helper. Signed-off-by: Batianxieshen1 <280706568+Batianxieshen1@users.noreply.github.com> Co-authored-by: Batianxieshen1 <280706568+Batianxieshen1@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
compute_wer_by_lineallocated its edit-distance matrix asint16. An editdistance can be as large as
max(len_hyp, len_ref), so initialising the firstrow/column raises
OverflowErroras soon as either input reaches 32768 tokens:This reproduces with one 32768-token hypothesis and a one-token reference — the
matrix is only 65538 cells, so it does not take two long inputs to hit. All
tokens can be identical, and the resulting distance (32767) still fits in int16;
it is the initialisation storing the index 32768 that overflows.
Fix: use
int32, which covers any input thisO(len_hyp * len_ref)loop canrealistically reach. Results for existing inputs are unchanged.
Fixes #3781
Type of change
Validation
python -m compileall funasr examples tests(exit 0; only pre-existingSyntaxWarnings inexamples/industrial_data_pretraining/fun_asr_nano/tools/format5res.py)Manual verification on NumPy 2.3.5:
compute_wer_by_line(["a"] * 32768, ["a"])raisedOverflowError: Python integer 32768 out of bounds for int16.{'nwords': 1, 'cor': 1, 'wrong': 32767, 'ins': 32767, 'del': 0, 'sub': 0}.compute_wer_by_line(["a"], ["a"] * 32768)returns{'nwords': 32768, 'cor': 1, 'wrong': 32767, 'ins': 0, 'del': 32767, 'sub': 0}.identical
compute_wer_by_lineresults before and after the change, sonormal-scoring behaviour is untouched.
Two regression tests were added to
tests/test_metrics_wer.pycovering bothdirections of the boundary:
User impact
Anyone scoring long transcripts or concatenated utterances with
compute_wer_by_line(includingcompute_werand the CER detail report) nolonger crashes once a line reaches 32768 tokens; such inputs now produce the
correct edit distance instead of an exception.
Note
The same
dtype=np.int16pattern exists in four standalone copies of thishelper —
runtime/python/utils/compute_wer.py:99,runtime/python/libtorch/funasr_torch/utils/compute_wer.py:99,examples/aishell/paraformer/utils/compute_wer.py:103andexamples/aishell/e_paraformer/utils/compute_wer.py:103. I kept this PR limitedto the library function and its tests; happy to send a follow-up for those if
you want them aligned.