Building a Practical Speech-to-Text Pipeline with Wav2Vec2 and FFmpeg

Automatic speech recognition works best when the entire media workflow is planned end-to-end. Built as a Python workflow in Google Colab, the Speech-to-Text System accepts both audio and video files, generating structured transcripts and final videos with burned-in subtitles.

The project focuses on the practical pipeline: extracting audio from video with FFmpeg, standardizing the signal, handling long audio through chunking, and routing the resulting text into multiple useful formats.

Acoustic neural networks require consistent audio input. Using Librosa and FFmpeg, audio streams are downmixed to single-channel mono and resampled to 16 kHz, matching the exact format expected by the Wav2Vec2 feature extractor.

For long recordings, dividing the audio into sequential chunks prevents memory errors in Google Colab and allows each segment to be processed in order while preserving accurate timing.

Transcription is handled by the pretrained facebook/wav2vec2-base-960h model loaded through Hugging Face Transformers. The model processes the audio chunks and decodes the spoken words into text.

The pipeline compiles the transcriptions into three practical file formats: plain TXT for quick reading, tabular CSV structured with Pandas for chunk-level inspection, and structured JSON for easy integration with other software tools.

In addition to plain text, the pipeline converts chunk timestamps into standard SubRip (SRT) subtitle files with accurate start and end times.

In the final step, FFmpeg burns the subtitle text directly onto the video frames. This creates a finished video with permanent open captions that can be played on any device or browser without needing external subtitle files.