Live dictation falls behind real time: streaming re-decodes a 450 ms overlap and Auto picks the slower iGPU #32050

What happened

Live dictation on this laptop takes about as long to finish as the recording: roughly 30s of speech leaves about 30s of waiting after the key is released, before cleanup even starts. The machine is an i5-8350U with an Intel UHD 620 iGPU and 15 GB RAM.

Diagnosis

Batch transcription is fast; the streaming path dictation uses is not. Measured with the bundled transcribe-cpp and the default handy-computer/parakeet-unified-en-0.6b-gguf (Q8_0) on this machine:

• 38.7s audio, batch: 8.6s CPU (0.22x real time), 11.4s Vulkan (0.29x).
• 38.7s audio, streaming with 0.5s chunks: 49.2s CPU (1.27x) plus 1.6s finalize; 76.1s Vulkan (1.97x) plus 2.3s finalize.
• 16.6s audio, streaming: 16.6s CPU (1.00x); 27.0s Vulkan (1.63x).

Two causes:

1. apps/server/src/speech/audio.ts:11 keeps a 450ms prefill, and line 137 sets streamAt = Math.max(lastKept, Math.max(0, samples - prefill * frame)). Each 0.5s chunk therefore re-decodes about 0.45s of already-heard audio, so the stream processes ~1.9x the audio length plus per-chunk overhead. The deficit grows with recording length (1.00x at 16.6s to 1.27x at 38.7s on CPU).
2. speechAcceleration: "auto" resolves to Vulkan0 (UHD 620), which is about 55% slower than CPU for this workload and never falls back because the backend loads successfully. Auto could prefer CPU when the only GPU is integrated, or measure and fall back.

Net: the backlog at key release is roughly (real-time factor - 1) * duration, which is ~30s for a 30s clip on Auto. That matches the reported wait.

Steps to reproduce

1. Use the default model and set Speech -> Hardware to Auto on a machine with an integrated GPU.
2. Dictate for about 30s in the composer.
3. Watch the transcript keep updating for roughly another 30s after release.
4. Repeat with Hardware = CPU: on short clips the wait drops to a few seconds.

Version

0.1.8-nightly.20261106.4262 (server commit c55db3e); still present on main as of 2026-11-06

Environment

Fedora Linux 44 (KDE Plasma), T3 Code desktop app with a local server, Intel i5-8350U, Intel UHD Graphics 620 (Vulkan), 15 GB RAM, parakeet-unified-en-0.6b-Q8_0

Evidence

Benchmarks run against the same native library and model the desktop app loads, feeding 0.5s PCM chunks through the transcribe-cpp streaming session:


38.7s audio: batch CPU 8.6s / Vulkan 11.4s
              streaming CPU 49.2s / Vulkan 76.1s
16.6s audio: streaming CPU 16.6s / Vulkan 27.0s


`getAvailableBackends()` on this machine lists Vulkan0 (Intel UHD Graphics 620, iGPU) and CPU (i5-8350U); `auto` selects Vulkan0.

Related issues

#31956 changes which benchmark number the settings page shows; #30961 added live chunk transcription. No existing report covers the streaming throughput gap or the Auto device choice.

Fix applied or workaround

Set Speech -> Hardware to CPU (roughly halves the backlog on this machine) and keep dictations under ~15-20s.

Filed by

opencode/deepseek-v4.1-flash-free via the OpenCode harness (T3 Code triage)