Hidden Assumptions in whisper

30-second audio chunks with 1-second overlap provide sufficient context for seamless transcription, and word boundaries never span across the overlap regions

If this fails: Long words, sentences, or phrases that cross chunk boundaries get incorrectly segmented or duplicated, producing fragmented transcripts with timing gaps or repeated text segments

Audio processing uses hardcoded constants N_SAMPLES=480000 (30 seconds * 16kHz) and N_FRAMES=3000, but never validates that input audio length matches these expectations

If this fails: Audio shorter than 30 seconds gets zero-padded while longer audio gets truncated, but downstream components expecting exactly 3000 frames crash if the padding/trimming logic changes or fails

FFmpeg executable exists in system PATH with support for the specific input audio format, and the subprocess call will complete successfully without timeout or resource limits

If this fails: If FFmpeg is missing, has different command-line arguments, or hits memory/CPU limits on large files, the subprocess raises CalledProcessError but this isn't handled, crashing the entire transcription pipeline

The tokenizer's sot_sequence (start-of-transcript tokens) contains language and task tokens in a specific order that matches what the model was trained to expect

If this fails: If the tokenizer produces a different token sequence format or the model expects different special tokens, the decoder generates garbage text because the conditioning tokens don't match the training distribution

Input tensor x has its time dimension as the last axis, and F.pad with 'reflect' mode is applied along the last dimension for temporal smoothing

If this fails: If cross-attention weights have a different axis ordering (e.g., time on axis 1), the median filtering operates on the wrong dimension, producing meaningless smoothed attention patterns and incorrect word alignments

The tiktoken tokenizer vocabulary exactly matches the model's training vocabulary, with identical token IDs for all special tokens including language identifiers, task tokens, and timestamp tokens

If this fails: If there's a version mismatch between tiktoken and the model's expected vocabulary, token IDs map to wrong words or cause out-of-vocabulary errors, producing completely incorrect transcriptions

PyTorch version supports scaled_dot_product_attention for memory-efficient attention, and falls back gracefully to manual attention computation when SDPA is unavailable

If this fails: On older PyTorch versions or specific hardware where SDPA fails unexpectedly, the fallback manual attention may have different numerical precision or memory usage patterns, causing subtle differences in transcription quality or OOM errors

Maximum sequence length n_text_ctx from ModelDimensions provides sufficient context for any reasonable transcription, and the model never needs to generate longer sequences

If this fails: For very long continuous speech without natural breaks, the fixed context window truncates the attention history, causing the model to lose track of earlier context and produce repetitive or incoherent text

Base85-encoded attention head indices for word-level timing remain accurate for the specific model checkpoints, and these hardcoded values correspond to heads that learned word-audio alignment during training

If this fails: If models are retrained or fine-tuned with different attention patterns, the hardcoded alignment heads no longer correlate with word boundaries, producing incorrect timestamp alignments that don't match the actual spoken words

Language detection returns language codes that exist in the LANGUAGES dictionary, and the detected language matches one of the 99 supported languages in the tokenizer vocabulary

If this fails: If the model detects a language variant or dialect not in the predefined list, the language lookup fails and defaults to English, potentially producing poor transcription quality for non-English speech