What Whisper + Coqui TTS taught me about perceived latency
- Whisper
- TTS
- Voice
- UX
On Sentinelle, the full voice pipeline — recognition (Whisper), processing, synthesis (Coqui TTS) — took around two seconds end to end on the test hardware. Two seconds is fine on paper. In use, it felt long.
Real time isn’t perceived time
The pipeline processes everything sequentially: wait for the sentence to end, transcribe, generate the reply, synthesise, then play the audio. Each step is reasonable on its own. Strung together, they create one continuous silence — and a two-second silence in a voice conversation reads very differently from a two-second load on a web page.
What actually helped
Not a model swap. The real win came from breaking the problem into perceptually smaller pieces:
- Streaming the synthesis: playing the first audio chunks as soon as they’re ready, without waiting for the full reply to be generated.
- A short audio cue during processing, instead of total silence — silence is what reads as a bug, not the wait itself.
- Shorter default replies, with the option to expand on request — less text to synthesise before the first sound.
The general lesson
For a voice system, optimising measured latency and optimising perceived latency are two different problems. The second is often solved with interaction design, not faster models.