fix(desktop): preserve huddle speech boundaries - #6397
Merged
Conversation
added 2 commits
August 20, 2026 10:37
Signed-off-by: Wren <5217c5c2f7bfb4333e46d17c98a9255a52dadee18dcd43a43536b95e6776dfa0@buzz.block.builderlab.xyz> Co-authored-by: Wren <5217c5c2f7bfb4333e46d17c98a9255a52dadee18dcd43a43536b95e6776dfa0@buzz.block.builderlab.xyz>
Co-authored-by: Wren <5217c5c2f7bfb4333e46d17c98a9255a52dadee18dcd43a43536b95e6776dfa0@buzz.block.builderlab.xyz> Signed-off-by: Wren <5217c5c2f7bfb4333e46d17c98a9255a52dadee18dcd43a43536b95e6776dfa0@buzz.block.builderlab.xyz>
added 2 commits
August 20, 2026 13:03
Co-authored-by: Wren <5217c5c2f7bfb4333e46d17c98a9255a52dadee18dcd43a43536b95e6776dfa0@buzz.block.builderlab.xyz> Signed-off-by: Wren <5217c5c2f7bfb4333e46d17c98a9255a52dadee18dcd43a43536b95e6776dfa0@buzz.block.builderlab.xyz>
Co-authored-by: Wren <5217c5c2f7bfb4333e46d17c98a9255a52dadee18dcd43a43536b95e6776dfa0@buzz.block.builderlab.xyz> Signed-off-by: Wren <5217c5c2f7bfb4333e46d17c98a9255a52dadee18dcd43a43536b95e6776dfa0@buzz.block.builderlab.xyz>
wpfleger96
added a commit
that referenced
this pull request
Aug 20, 2026
…-scoping * origin/main: perf(desktop): resolve references without directory scans (#6328) feat(llm): stamp thinking effort on call-completed log line (#6424) Fix cross-owner relay agent mentions in owner-only builds (#6338) feat(cli): accept Buzz message links for thread reads (#6359) feat(workflows): add workflow editor (#6248) fix(desktop): preserve huddle speech boundaries (#6397) test(desktop): use a wordlist-safe separator in passphrase word-count test (#6356) fix(models): curate Databricks alias-aware labels for 5 missing endpoints (#6360) fix(acp): guard against unrequested public relay skills (#6394) Signed-off-by: Duncan <dcfd242e557282d7a1e2cf2e6877522682f1e5c6156dc92ca7d90eaedd3b0f95@buzz.block.builderlab.xyz> Co-authored-by: Will Pfleger <pfleger.will@gmail.com> Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
atishpatel
added a commit
that referenced
this pull request
Aug 20, 2026
…ounting-and-solo * origin/main: (48 commits) fix(hooks): scope pre-push lanes to branch merge-base diff (#6423) Enforce a three-day dependency cooldown (#6426) perf(desktop): resolve references without directory scans (#6328) feat(llm): stamp thinking effort on call-completed log line (#6424) Fix cross-owner relay agent mentions in owner-only builds (#6338) feat(cli): accept Buzz message links for thread reads (#6359) feat(workflows): add workflow editor (#6248) fix(desktop): preserve huddle speech boundaries (#6397) test(desktop): use a wordlist-safe separator in passphrase word-count test (#6356) fix(models): curate Databricks alias-aware labels for 5 missing endpoints (#6360) fix(acp): guard against unrequested public relay skills (#6394) feat(desktop): refine context-aware Projects collaboration (#6396) fix(desktop): distinguish duplicate agent devices (#6337) feat(desktop): close Buzz window with Cmd+W (#6314) refactor(prompt): simplify Buzz agent guidance (#6340) feat(desktop): make Projects workspaces selectable (#6368) Add Buzz-native collaboration benchmarks (#6264) Polish mobile timeline and emoji interactions (#6297) feat(desktop): make the Projects overview follow the selected section (#6335) refactor(desktop): coordinate TTS playback (#6341) ... Signed-off-by: Atish Patel <atishpatel2012@gmail.com>
brow
added a commit
that referenced
this pull request
Aug 20, 2026
…ifications-pr * origin/main: (33 commits) fix(hooks): scope pre-push lanes to branch merge-base diff (#6423) Enforce a three-day dependency cooldown (#6426) perf(desktop): resolve references without directory scans (#6328) feat(llm): stamp thinking effort on call-completed log line (#6424) Fix cross-owner relay agent mentions in owner-only builds (#6338) feat(cli): accept Buzz message links for thread reads (#6359) feat(workflows): add workflow editor (#6248) fix(desktop): preserve huddle speech boundaries (#6397) test(desktop): use a wordlist-safe separator in passphrase word-count test (#6356) fix(models): curate Databricks alias-aware labels for 5 missing endpoints (#6360) fix(acp): guard against unrequested public relay skills (#6394) feat(desktop): refine context-aware Projects collaboration (#6396) fix(desktop): distinguish duplicate agent devices (#6337) feat(desktop): close Buzz window with Cmd+W (#6314) refactor(prompt): simplify Buzz agent guidance (#6340) feat(desktop): make Projects workspaces selectable (#6368) Add Buzz-native collaboration benchmarks (#6264) Polish mobile timeline and emoji interactions (#6297) feat(desktop): make the Projects overview follow the selected section (#6335) refactor(desktop): coordinate TTS playback (#6341) ... Signed-off-by: Tom Brow <tomb@block.xyz>
TheSentinel454
added a commit
that referenced
this pull request
Aug 21, 2026
* origin/main: (64 commits) chore(deps): pin earshot below 1.2.0 pending a VAD threshold re-pick (#6392) polish(desktop): finish Projects navigation and context chrome (#6429) fix(desktop): clarify add agents channel action (#6374) Repair stale large channel roster snapshots (#6251) feat(desktop-messages): show compact Buzz link metadata (#6252) feat(workflows): reply in-thread from send_message action (#6178) perf(desktop): split discover_acp_providers into cheap and forced paths (#6330) fix(desktop): restore recent channel sorting (#6402) fix(desktop): isolate main timeline stacking context from focus drawer (#6398) fix(desktop): make reconnect repair lossless (#6415) fix(hooks): scope pre-push lanes to branch merge-base diff (#6423) Enforce a three-day dependency cooldown (#6426) perf(desktop): resolve references without directory scans (#6328) feat(llm): stamp thinking effort on call-completed log line (#6424) Fix cross-owner relay agent mentions in owner-only builds (#6338) feat(cli): accept Buzz message links for thread reads (#6359) feat(workflows): add workflow editor (#6248) fix(desktop): preserve huddle speech boundaries (#6397) test(desktop): use a wordlist-safe separator in passphrase word-count test (#6356) fix(models): curate Databricks alias-aware labels for 5 missing endpoints (#6360) ... Signed-off-by: Luke Tornquist <tornquist@squareup.com>
TheSentinel454
added a commit
that referenced
this pull request
Aug 21, 2026
…kflow-management * origin/main: (27 commits) chore(deps): pin earshot below 1.2.0 pending a VAD threshold re-pick (#6392) polish(desktop): finish Projects navigation and context chrome (#6429) fix(desktop): clarify add agents channel action (#6374) Repair stale large channel roster snapshots (#6251) feat(desktop-messages): show compact Buzz link metadata (#6252) feat(workflows): reply in-thread from send_message action (#6178) perf(desktop): split discover_acp_providers into cheap and forced paths (#6330) fix(desktop): restore recent channel sorting (#6402) fix(desktop): isolate main timeline stacking context from focus drawer (#6398) fix(desktop): make reconnect repair lossless (#6415) fix(hooks): scope pre-push lanes to branch merge-base diff (#6423) Enforce a three-day dependency cooldown (#6426) perf(desktop): resolve references without directory scans (#6328) feat(llm): stamp thinking effort on call-completed log line (#6424) Fix cross-owner relay agent mentions in owner-only builds (#6338) feat(cli): accept Buzz message links for thread reads (#6359) feat(workflows): add workflow editor (#6248) fix(desktop): preserve huddle speech boundaries (#6397) test(desktop): use a wordlist-safe separator in passphrase word-count test (#6356) fix(models): curate Databricks alias-aware labels for 5 missing endpoints (#6360) ... Signed-off-by: Luke Tornquist <tornquist@squareup.com> # Conflicts: # desktop/src/features/workflows/ui/WorkflowDeleteDialog.tsx # desktop/src/features/workflows/ui/WorkflowsView.tsx
cursor Bot
pushed a commit
to Nuncio-hq/crew
that referenced
this pull request
Aug 23, 2026
Upstream sync of block#5825 and block#6397 plus the always-on part of block#5671. block#5825 — 20 Hz speaker levels and the ~30 Hz mic analyser lived in the same context value as huddle lifecycle state, so every tick re-rendered every `useHuddle()` consumer (and their subtrees) across the whole app. Levels now live in a separate `HuddleLevelsContext` (`useHuddleLevels()`), the mic analyser moves to `useMicLevelAnalyser`, and the lifecycle value is memoized. Meters still see every tick; ordinary consumers see none (20 -> 0 re-renders per second of huddle audio). block#6397 — endpointing was a single threshold plus a silence counter, so a one-frame blip opened a segment, a dipping vowel started the countdown, and word onsets/offsets were clipped. Speech boundaries now live in a pure `VadEndpoint`: onset needs consecutive above-threshold frames, confirmation prepends bounded pre-roll, offset uses a lower threshold (hysteresis), flush keeps only a hangover tail, and reset drops buffered audio so nothing leaks between segments. Short segments now log why they were dropped. block#5671 — only the push-to-talk grouping fix is portable here: a held shortcut no longer silence-flushes mid-hold when the mic is also manually open (`vad_flush_allowed`). The env-gated STT/TTS levers live in TTS internals this branch does not touch; documented as such in docs/crew/HERMES.md, and BUZZ_STT_FLUSH_MS stays removed. Fixes #275 Signed-off-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
cursor Bot
pushed a commit
to Nuncio-hq/crew
that referenced
this pull request
Aug 23, 2026
Upstream sync of block#5825 and block#6397 plus the always-on part of block#5671. block#5825 — 20 Hz speaker levels and the ~30 Hz mic analyser lived in the same context value as huddle lifecycle state, so every tick re-rendered every `useHuddle()` consumer (and their subtrees) across the whole app. Levels now live in a separate `HuddleLevelsContext` (`useHuddleLevels()`), the mic analyser moves to `useMicLevelAnalyser`, and the lifecycle value is memoized. Meters still see every tick; ordinary consumers see none (20 -> 0 re-renders per second of huddle audio). block#6397 — endpointing was a single threshold plus a silence counter, so a one-frame blip opened a segment, a dipping vowel started the countdown, and word onsets/offsets were clipped. Speech boundaries now live in a pure `VadEndpoint`: onset needs consecutive above-threshold frames, confirmation prepends bounded pre-roll, offset uses a lower threshold (hysteresis), flush keeps only a hangover tail, and reset drops buffered audio so nothing leaks between segments. Short segments now log why they were dropped. block#5671 — only the push-to-talk grouping fix is portable here: a held shortcut no longer silence-flushes mid-hold when the mic is also manually open (`vad_flush_allowed`). The env-gated STT/TTS levers live in TTS internals this branch does not touch; documented as such in docs/crew/HERMES.md, and BUZZ_STT_FLUSH_MS stays removed. Fixes #275 Signed-off-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
BradGroux
pushed a commit
to BradGroux/buzz
that referenced
this pull request
Aug 23, 2026
## Problem and intent Buzz huddles have been clipping speech into fragments: one live turn arrived as the single letter “M,” and another ended mid-sentence. The detector was making each frame decision independently, but the surrounding endpointing policy had no onset confirmation or pre-roll, used one threshold for both entering and leaving speech, and silently discarded short segments. This PR keeps Earshot 1.1.0 and fixes that policy around it. It is intentionally model-independent so the separate Earshot/Silero bake-off can evaluate detectors under the same segmentation behavior. ## What changed One production file changed: `desktop/src-tauri/src/huddle/stt.rs` (**+367/−79** versus `main`). - Add a pure `VadEndpoint` state machine around Earshot probabilities. - Preserve 256 ms (16 frames) of pre-roll before confirmed onset. - Require three consecutive frames above 0.50 to enter speech. - Use 0.35 to leave speech, preserving hysteresis-band audio. - Retain 96 ms (6 frames) of hangover while keeping the existing 304 ms silence-flush window. - Make short-segment drops visible in logs instead of silent. - Add boundary, onset, hysteresis, hangover, drop-path, and PTT policy tests. - Bind the thresholds to Earshot 1.1.0 in source. Earshot 1.2.x is deliberately parked pending a matched-policy bake-off; block#6392 separately prevents Renovate from silently crossing that boundary. ## Boundary behavior and known tradeoffs A hard message boundary clears pre-roll. That prevents segment N audio from reaching segment N+1, but a fast follow-up turn may receive less than the full 256 ms onset window. On the 121-clip corpus, **12 of 25 non-first segments had truncated pre-roll, with 32 ms worst observed at the shipping constants**. This result **holds at both 208 ms and 256 ms on this corpus**; it is not independent of pre-roll. The observed follow-ups were bimodal—fast cases clustered at 2–10 frames and the next case was 19—so no larger pre-roll budget could reach those fast cases in this corpus. If a future refactor makes pre-roll survive a boundary, segment N reaches segment N+1 under the strict predicate: ```text gap < VAD_PRE_ROLL_FRAMES - VAD_ONSET_FRAMES ``` At the shipped values that means gaps 0–12 leak and gap 13 is the first clean case. Hangover and the silence-flush window do not enter this bound because leakage rides the pre-roll deque. Marker-origin tests exercise the real flush/reset and next-onset drain; a rolling-buffer mutant leaks 3,328 samples and fails them. Other known behavior: - `voiced_frames` now counts frames above the 0.35 exit threshold once speech begins, not only frames above 0.50. Corpus drop counts measured under `> 0.50` counting must be re-baselined rather than compared directly; the old “19 silent drops” figure is not a valid before/after baseline. - If PTT releases while the manually-open microphone flips or remains on, there is no combined-transmit falling edge. The uninterrupted utterance correctly closes on normal VAD timing instead of an edge flush. - The GUI-to-`push_audio_pcm` wiring is unchanged. The final acoustic gate exercises the checked-in release-profile latency harness against the real production `SttPipeline` and `TtsPipeline`; Tyler’s hand-test covers the live GUI leg by design. ## Verification ledger All gates attest exact head `2dad6bb0e2e2a0af115a9a06c26cf6183fedb633`. - **Implementation:** complete pre-push gate green; remote SHA matched local HEAD. - **Exact-SHA verification:** `cargo fmt --check`; `cargo clippy --all-targets -- -D warnings`; full `desktop/src-tauri` tests: **2,702 passed, 0 failed, 18 ignored**; HEAD re-confirmed unchanged afterward. - **Mutation review:** engine region SHA-256 `3c161d8f6e4d22c02dff3c76498430964beda51a034d34651dbdb03e8ff69aa0`; 5/5 policy mutants killed, including deleted-clear and rolling-buffer regressions. Direct gap sweep leaked at 0–12 and was clean from 13. - **Release acoustic gate:** negative control deliberately overstated the expected segment count and failed with exit 101. Soft/short onset produced 1/1 segment (`“I'm happy.”`). Natural-pause fixture used two exact 700 ms pauses (>304 ms) and produced the pre-registered 3/3 intact segments at the scripted boundaries. - **Release CPU beside Pocket TTS:** soft fixture median 32.1%, p95/max 56.8% (5 sparse samples); natural fixture median 14.55%, p95/max 17.2% (8 sparse samples). These are scoped process samples from synthesis start through append acceptance, not a general desktop CPU benchmark. Local review receipts (not committed to the repository): - `.scratch/vad-live-release-2dad6bb0e/` - `RESEARCH/VAD_ARM_B_MUTATION_RIG_2026_08_20/` - `RESEARCH/EARSHOT_1_1_0_TO_1_2_2_MEASUREMENT_2026_08_20.md` ## Hand-test focus Before merge, exercise the two live shapes that originally failed: 1. Soft short openers such as “M” and “yes.” 2. Natural mid-sentence pauses. Also try a reply immediately after the previous message commits; that is intentionally the least-protected onset case because hard boundaries clear pre-roll. --- Authorship disclosure: this change was implemented and the PR opened by **Wren**, Tyler’s Buzz agent, using Tyler’s GitHub identity after prior disclosure and authorization in the originating Buzz thread. Tyler remains the accountable human reviewer/operator. --------- Signed-off-by: Wren <5217c5c2f7bfb4333e46d17c98a9255a52dadee18dcd43a43536b95e6776dfa0@buzz.block.builderlab.xyz> Co-authored-by: Wren <5217c5c2f7bfb4333e46d17c98a9255a52dadee18dcd43a43536b95e6776dfa0@buzz.block.builderlab.xyz>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem and intent
Buzz huddles have been clipping speech into fragments: one live turn arrived as the single letter “M,” and another ended mid-sentence. The detector was making each frame decision independently, but the surrounding endpointing policy had no onset confirmation or pre-roll, used one threshold for both entering and leaving speech, and silently discarded short segments.
This PR keeps Earshot 1.1.0 and fixes that policy around it. It is intentionally model-independent so the separate Earshot/Silero bake-off can evaluate detectors under the same segmentation behavior.
What changed
One production file changed:
desktop/src-tauri/src/huddle/stt.rs(+367/−79 versusmain).VadEndpointstate machine around Earshot probabilities.Boundary behavior and known tradeoffs
A hard message boundary clears pre-roll. That prevents segment N audio from reaching segment N+1, but a fast follow-up turn may receive less than the full 256 ms onset window.
On the 121-clip corpus, 12 of 25 non-first segments had truncated pre-roll, with 32 ms worst observed at the shipping constants. This result holds at both 208 ms and 256 ms on this corpus; it is not independent of pre-roll. The observed follow-ups were bimodal—fast cases clustered at 2–10 frames and the next case was 19—so no larger pre-roll budget could reach those fast cases in this corpus.
If a future refactor makes pre-roll survive a boundary, segment N reaches segment N+1 under the strict predicate:
At the shipped values that means gaps 0–12 leak and gap 13 is the first clean case. Hangover and the silence-flush window do not enter this bound because leakage rides the pre-roll deque. Marker-origin tests exercise the real flush/reset and next-onset drain; a rolling-buffer mutant leaks 3,328 samples and fails them.
Other known behavior:
voiced_framesnow counts frames above the 0.35 exit threshold once speech begins, not only frames above 0.50. Corpus drop counts measured under> 0.50counting must be re-baselined rather than compared directly; the old “19 silent drops” figure is not a valid before/after baseline.push_audio_pcmwiring is unchanged. The final acoustic gate exercises the checked-in release-profile latency harness against the real productionSttPipelineandTtsPipeline; Tyler’s hand-test covers the live GUI leg by design.Verification ledger
All gates attest exact head
2dad6bb0e2e2a0af115a9a06c26cf6183fedb633.cargo fmt --check;cargo clippy --all-targets -- -D warnings; fulldesktop/src-tauritests: 2,702 passed, 0 failed, 18 ignored; HEAD re-confirmed unchanged afterward.3c161d8f6e4d22c02dff3c76498430964beda51a034d34651dbdb03e8ff69aa0; 5/5 policy mutants killed, including deleted-clear and rolling-buffer regressions. Direct gap sweep leaked at 0–12 and was clean from 13.“I'm happy.”). Natural-pause fixture used two exact 700 ms pauses (>304 ms) and produced the pre-registered 3/3 intact segments at the scripted boundaries.Local review receipts (not committed to the repository):
.scratch/vad-live-release-2dad6bb0e/RESEARCH/VAD_ARM_B_MUTATION_RIG_2026_08_20/RESEARCH/EARSHOT_1_1_0_TO_1_2_2_MEASUREMENT_2026_08_20.mdHand-test focus
Before merge, exercise the two live shapes that originally failed:
Also try a reply immediately after the previous message commits; that is intentionally the least-protected onset case because hard boundaries clear pre-roll.
Authorship disclosure: this change was implemented and the PR opened by Wren, Tyler’s Buzz agent, using Tyler’s GitHub identity after prior disclosure and authorization in the originating Buzz thread. Tyler remains the accountable human reviewer/operator.