Repository navigation
Fix stale start timestamp in timestamp_sentence_en when timestamp list is shorter than punctuation list - #3741
Conversation
When punc_id_list is longer than timestamp_postprocessed (a mismatch the function already logs a warning for), the entries past the end of timestamp_postprocessed get timestamp=None from zip_longest. For the sentence that starts on one of those entries, the is_sentence_start flag was consumed unconditionally on the first iteration regardless of whether timestamp was None, so sentence_start kept whatever value was left over from the previous, unrelated sentence instead of being reported as unknown. The sibling function timestamp_sentence already handles this correctly by resetting sentence_start to None after each flush and only setting it once a real timestamp shows up. Mirror that logic here instead of the one-shot is_sentence_start flag. Before: a trailing sentence whose first word has no timestamp gets start reused from the previous sentence (wrong, looks valid). After: start is None for that sentence, matching timestamp_sentence.
LauraGPT
left a comment
There was a problem hiding this comment.
Independently verified the regression at 12817ec using the real FunASR module in the native CPU test environment.
- Ran the new three-test file against its parent 7f85ddd: 1 failed, 2 passed. The new case fails specifically with
assert 0 is None. - Ran
python -m pytest -q tests/test_timestamp_tools.pyon this PR head: 3 passed. - The reset/retry logic now agrees with
timestamp_sentencefor the missing trailing timestamp, while the existing normal-output and whitespace regressions still pass. I found no blocking issue in this scoped change.
This verification is limited to these timestamp unit tests. I have not independently reproduced or classified the four full-suite failures mentioned in the PR description. No model download, inference, or hardware validation was performed.
|
Thanks for the independent verification. Re-ran On the failures: the PR description's "Full local suite" line is this same run, not all 113 test files. It is 68 files, grep-selected for not needing network access or model download, then trimmed by 5 more that fail to import in this environment (three need The three tests are |
LauraGPT
left a comment
There was a problem hiding this comment.
Rechecked exact head 12817ecdbb1037394e28e9474aab55897cf84637 against current main 7b098acefc22999f1be483b7b6fca62db8c8ad39.
The focused timestamp suite passes 3/3 on both the submitted head and a clean synthetic merge; compileall and git diff --check also pass. The previously recorded parent control fails only the new regression (1 failed, 2 passed), confirming that the test discriminates the stale-start bug. The reset-and-retry behavior is scoped and matches the sibling timestamp_sentence fallback.
Approving.
Summary
timestamp_sentence_eninfunasr/utils/timestamp_tools.pyuses a one-shotis_sentence_startflag to decide when to record a new sentence's start timestamp. The flag fires exactly once right after a sentence is flushed, regardless of whether the current timestamp isNone. Whenpunc_id_listandtimestamp_postprocesseddiffer in length (a case the code already detects and logs a warning for),itertools.zip_longestpads the missing trailing entries withtimestamp=None. If the first token of a new sentence lands on one of these paddedNoneentries,is_sentence_startis cleared without ever storing a real start value, sosentence_startkeeps whatever value was left from the previous, unrelated sentence. The sibling functiontimestamp_sentence(used for Chinese/default text) avoids this: it resetssentence_start = Noneafter each flush and only sets it once a non-Nonetimestamp actually appears, retrying on every iteration until one does.This PR replaces the one-shot flag in
timestamp_sentence_enwith the same reset-and-retry pattern already used bytimestamp_sentence. No other behavior changes.Repro before the fix, with
punc_id_list = [1, 3, 3],timestamp_postprocessed = [[0, 100], [100, 200]](one entry short),text_postprocessed = "hello world bar":startis copied from the first sentence ("hello world.", alsostart=0, end=200), so the two unrelated sentences get identical timestamps instead of the second one being flagged unknown.timestamp_sentenceon the same shape of input correctly returnsstart=None.After the fix,
timestamp_sentence_enreturnsstart=Nonefor that sentence, matching the Chinese sibling.Type of change
Validation
python -m compileall funasr examples tests-- exit 0python -m pytest tests/test_timestamp_tools.py -q-- 3 passed (2 pre-existing + 1 new regression test)funasr/utils/timestamp_tools.pyto the pre-fix commit and reran the same test file: the new test fails withassert 0 is None, confirming it catches the bug. The two pre-existing tests still pass on the old code. Restored the fix afterward.uvicorndependency in the local environment). None touchfunasr/utils/timestamp_tools.pyor any file this PR changes.Added
tests/test_timestamp_tools.py::test_timestamp_sentence_en_missing_trailing_timestamp_matches_zh_fallback, which feeds the same repro inputs to bothtimestamp_sentence_enandtimestamp_sentenceand asserts the second sentence'sstartisNonein both, so they agree.User impact
Anyone calling
generate()withsentence_timestamp=Trueanden_post_proc=True(English/latin sentence-level timestamps, e.g. subtitle or caption generation infunasr/auto/auto_model.py) hits this whenever the punctuation model's token count and the ASR timestamp count diverge for a batch. That mismatch is already logged as a warning elsewhere in the code, so it's a known, occurring condition rather than a contrived edge case. Before this fix the output silently carried a wrong, duplicated start time for the affected sentence instead ofNone, which is worse than a crash since it looks like valid data downstream.Notes for reviewers
Change is scoped to
timestamp_sentence_en;timestamp_sentenceis untouched. Behavior for the common case (matching-length lists) is unchanged, verified by the two pre-existing tests still passing.