The Speaker Decides When to Stop

# labs/vad_listen.py
SAMPLE_RATE = 16000    # Hz, the capture rate the whole pipeline uses
FRAME_MS = 30          # ms per frame; webrtcvad accepts 10, 20 or 30
FRAME_SAMPLES = SAMPLE_RATE * FRAME_MS // 1000
FRAME_BYTES = FRAME_SAMPLES * 2        # int16 audio: two bytes per sample

if __name__ == "__main__":
    print(f"samples per frame: {FRAME_SAMPLES}")
    print(f"bytes per frame:   {FRAME_BYTES}")
    print(f"frames per second: {1000 / FRAME_MS:.1f}")

$ uv run python labs/vad_listen.py
samples per frame: 480
bytes per frame:   960
frames per second: 33.3

Follow the arithmetic once and you will never be confused by a frame again. One second of this audio is 16,000 samples, two bytes each, so 32,000 bytes a second. Thirty milliseconds is three hundredths of that: 480 samples, 960 bytes, and a shade over 33 of them per second. Every one of those numbers is derived, not typed. Change FRAME_MS to 20 and the samples become 320, the bytes 640, and the timeout you meet in a moment recounts itself. Two constants at the top, and no stray 480 buried in a stream call waiting to disagree with a stray 960 in a slice.

import math
import struct

import webrtcvad

AGGRESSIVENESS = 2     # 0 passes almost everything, 3 discards hardest

vad = webrtcvad.Vad(AGGRESSIVENESS)

silence = b"\x00\x00" * FRAME_SAMPLES
tone = b"".join(
    struct.pack("<h", int(20000 * math.sin(2 * math.pi * 440 * n / SAMPLE_RATE)))
    for n in range(FRAME_SAMPLES)
)

for name, frame in (("silence", silence), ("440 Hz tone", tone)):
    print(f"{name:12s} {len(frame)} bytes  speech={vad.is_speech(frame, SAMPLE_RATE)}")

$ uv run python labs/vad_listen.py
silence      960 bytes  speech=False
440 Hz tone  960 bytes  speech=True

Two synthetic frames, built by hand so this stage runs on a machine with no microphone attached. The silent one is 480 zero samples and comes back False, as it should. The second is a pure 440 Hz tone, a tuning-fork A, and it comes back True, which is the most useful wrong answer in this chapter. Nobody sang that. A voice detector detects voice-like signal in the bands where speech has energy, and a strong steady tone sitting in the middle of those bands passes for one. Take the lesson now, before it costs you: this classifier goes behind the wake word from chapter 13, never in front of it. It tells you a person may be talking. Only Whisper can tell you they were talking to her.

AGGRESSIVENESS runs 0 to 3 and tunes how eager the detector is to throw a frame away. At 0 it keeps nearly everything, so a noisy room reads as one long utterance. At 3 it is severe enough to drop the quiet leading consonant of a word, and you get recordings that begin at "-emind me". Level 2 is a reasonable place to start in a normal room, and you will tune it in this chapter's exercises against the room you actually live in.

import numpy as np
import sounddevice as sd

SILENCE_THRESHOLD = 0.005    # the RMS gate from chapter 13, unchanged

def frame_rms(frame: bytes) -> float:
    samples = np.frombuffer(frame, dtype=np.int16).astype(np.float32) / 32768.0
    return float(np.sqrt(np.mean(samples ** 2)))

def trace(seconds: float = 3.0) -> None:
    vad = webrtcvad.Vad(AGGRESSIVENESS)
    with sd.RawInputStream(samplerate=SAMPLE_RATE, blocksize=FRAME_SAMPLES,
                           dtype="int16", channels=1) as stream:
        for i in range(int(seconds * 1000 / FRAME_MS)):
            frame = bytes(stream.read(FRAME_SAMPLES)[0])
            rms = frame_rms(frame)
            gate = "open" if rms >= SILENCE_THRESHOLD else "closed"
            speech = vad.is_speech(frame, SAMPLE_RATE)
            print(f"{i:3d}  rms={rms:.4f}  gate={gate:6s} vad={speech}")

$ uv run python labs/vad_listen.py   # fan running, then a short sentence; your numbers will vary
  0  rms=0.0121  gate=open   vad=False
  1  rms=0.0118  gate=open   vad=False
  2  rms=0.0134  gate=open   vad=False
  3  rms=0.0402  gate=open   vad=True
  4  rms=0.0761  gate=open   vad=True
  5  rms=0.0693  gate=open   vad=True
  6  rms=0.0044  gate=closed vad=True
  7  rms=0.0038  gate=closed vad=True
  8  rms=0.0126  gate=open   vad=False
  9  rms=0.0119  gate=open   vad=False

There is the whole argument, printed. Your figures will not match these, because they are a capture of one room with one fan and one microphone gain, but the pattern will reproduce anywhere. Frames 0 to 2 are an empty room with a fan in it: the RMS gate calls them audio and opens, the detector calls them noise and stays shut. Frames 6 and 7 are the tail of the last word, a fricative quiet enough to drop under 0.005, so the gate slams while the detector still hears a person. The loudness meter is wrong in both directions on the same three seconds, and no threshold you pick moves both errors the way you want.

import collections

RING_FRAMES = 10                                     # 300 ms of pre-roll and vote window
MIN_SPEECH_FRAMES = 5                                # 5 of those 10 must be voiced to start
SILENCE_TIMEOUT_MS = 1500
SILENCE_TIMEOUT_FRAMES = SILENCE_TIMEOUT_MS // FRAME_MS

def record_utterance() -> bytes:
    vad = webrtcvad.Vad(AGGRESSIVENESS)
    ring: collections.deque[tuple[bytes, bool]] = collections.deque(maxlen=RING_FRAMES)
    voiced: list[bytes] = []
    triggered = False
    silent_run = 0

    print("Listening...")
    with sd.RawInputStream(samplerate=SAMPLE_RATE, blocksize=FRAME_SAMPLES,
                           dtype="int16", channels=1) as stream:
        while True:
            frame = bytes(stream.read(FRAME_SAMPLES)[0])
            speech = vad.is_speech(frame, SAMPLE_RATE)

            if not triggered:
                ring.append((frame, speech))
                if sum(1 for _, s in ring if s) >= MIN_SPEECH_FRAMES:
                    triggered = True
                    voiced.extend(f for f, _ in ring)
                    ring.clear()
                    print("Speech detected, recording...")
                continue

            voiced.append(frame)
            if speech:
                silent_run = 0
            else:
                silent_run += 1
                if silent_run >= SILENCE_TIMEOUT_FRAMES:
                    print(f"Quiet for {SILENCE_TIMEOUT_MS} ms, stopping.")
                    break

    return b"".join(voiced)

if __name__ == "__main__":
    pcm = record_utterance()
    frames = len(pcm) // FRAME_BYTES
    print(f"captured {frames} frames ({frames * FRAME_MS / 1000:.2f} s)")

$ uv run python labs/vad_listen.py   # a live microphone; your frame count will differ
Listening...
Speech detected, recording...
Quiet for 1500 ms, stopping.
captured 141 frames (4.23 s)

One deque does two jobs, and that is the design worth studying. Before the trigger it is a voting window: ten frames wide, and five of them voiced is the entry condition. At the instant it fires, those same ten frames are the audio that arrived while the detector was still making up its mind, so they get poured into voiced before the live frames start arriving. Without that, every recording would open on the second syllable, because proving speech has started takes five frames and those five frames are your first consonant.

After the trigger the ring is finished and the code takes the simpler path: append every frame, count consecutive quiet ones, reset on any voice. Note what is not in this function. No duration, no deadline, no stopwatch. It returns when you stop talking and not before, and the 141 frames in that capture are 4.23 seconds because the sentence was 4.23 seconds long, minus the fixed second and a half of proof at the end.

import wave

UTTERANCE = "glados/data/utterance.wav"

def trim_trailing_silence(pcm: bytes, keep_ms: int = 300) -> bytes:
    drop = (SILENCE_TIMEOUT_MS - keep_ms) // FRAME_MS * FRAME_BYTES
    return pcm[:-drop] if len(pcm) > drop else pcm

def save_wav(pcm: bytes, path: str = UTTERANCE) -> None:
    with wave.open(path, "wb") as wf:
        wf.setnchannels(1)
        wf.setsampwidth(2)             # 16-bit, matching the capture
        wf.setframerate(SAMPLE_RATE)
        wf.writeframes(pcm)

if __name__ == "__main__":                  # prove the writer with no microphone
    save_wav(b"\x00\x00" * (SAMPLE_RATE // 2), "glados/data/writer_check.wav")
    with wave.open("glados/data/writer_check.wav") as wf:
        print(f"channels={wf.getnchannels()} width={wf.getsampwidth()} "
              f"rate={wf.getframerate()} frames={wf.getnframes()}")

$ uv run python labs/vad_listen.py
channels=1 width=2 rate=16000 frames=8000

Half a second of zeros in, 8,000 frames back out at 16 kHz mono 16-bit: the writer is told the same three facts the capture used, so nothing is reinterpreted on the way to disk. Get any of the three wrong and the file plays at the wrong pitch or as static, the mistake chapter 4's last exercise had you produce on purpose.

trim_trailing_silence is the small repair that pays for section one's complaint. The recorder had to hear 1.5 seconds of quiet to know you were finished, and that proof is now sitting in the buffer as 50 frames of nothing, which is exactly the fuel Whisper hallucinates on. Dropping 40 of them leaves 300 milliseconds of natural tail, enough that the final consonant is not clipped and little enough that the model has nothing to invent from.

# labs/wake_word.py — the command window stops being a duration
from labs.vad_listen import (FRAME_BYTES, UTTERANCE, record_utterance,
                             save_wav, trim_trailing_silence)

if __name__ == "__main__":
    model = WhisperModel("base", device="cpu", compute_type="int8")
    print("Waiting for wake word. Say 'Hey GLaDOS'...")
    while True:
        woke, heard = detect_wake_word(model)
        if not woke:
            continue
        raw = record_utterance()
        pcm = trim_trailing_silence(raw)
        save_wav(pcm, UTTERANCE)
        print(f"{len(raw) // FRAME_BYTES} frames captured, "
              f"{len(pcm) // FRAME_BYTES} after trim")
        segments, _ = model.transcribe(UTTERANCE, language="en")
        print("You:", " ".join(s.text.strip() for s in segments))
        break

$ uv run python labs/wake_word.py   # live mic and model; your transcript will read differently
Waiting for wake word. Say 'Hey GLaDOS'...
Wake word detected in: 'hey glados'
Listening...
Speech detected, recording...
Quiet for 1500 ms, stopping.
187 frames captured, 147 after trim
You: remind me to move the laundry before the printer in the hall eats another envelope

187 frames is 5.61 seconds of talking, past the window this replaced, and she waited for all of it. The clip that reached Whisper was 147 frames, because the 40 frames of silence that proved the sentence had ended were never the sentence. The loop from chapter 13 changed in one place, where a fixed duration became a call. Everything else about the cascade holds: the wake word still guards the expensive stage, and the transcriber still receives a 16 kHz mono WAV it can read without resampling, since the detector demanded the same rate the transcriber was trained on.