07 // MACHINE LEARNING
Speech-to-Text System
An end-to-end English speech-to-text prototype that converts audio and video into reusable transcripts, SRT subtitles, and video with burned-in captions.
[01]
Project Gallery
[02]
Project Overview
Built in Google Colab, the workflow accepts WAV and MP3 audio or MP4 video. Video audio is extracted with FFmpeg, converted to mono at 16 kHz, and divided into chunks before transcription.
Each audio chunk is processed with the pretrained facebook/wav2vec2-base-960h model through Hugging Face Transformers. The results can be exported as TXT, CSV, JSON, and SRT files or embedded directly into the final video.
[03]
My Contribution
- Built the ingestion workflow for WAV and MP3 audio and MP4 video, including audio extraction from video with FFmpeg.
- Implemented mono 16 kHz conversion and chunk-based processing to prepare longer audio for sequential inference.
- Integrated the pretrained Wav2Vec2 model through Hugging Face Transformers and structured the transcription results as TXT, CSV, and JSON files.
- Generated timestamp-based SRT subtitles and used FFmpeg to embed them into the final video.
This project gave me experience combining a pretrained machine learning model with an end-to-end audio and video processing workflow.
Tech Stack
- Python
- Wav2Vec2
- Hugging Face Transformers
- Librosa
- FFmpeg
- Google Colab
[04]
System Scope
■01 // Media Preparation
- WAV and MP3 audio input
- MP4 video input
- FFmpeg audio extraction from video
- Mono 16 kHz conversion
- Chunk-based processing for longer audio
■02 // ASR Inference
- English transcription only
- Pretrained facebook/wav2vec2-base-960h
- Hugging Face Transformers model execution
- Sequential inference for audio chunks
- No custom fine-tuning
- No WER or CER benchmark
■03 // Transcript & Subtitle Outputs
- TXT transcript
- CSV transcript
- JSON transcript
- Timestamp-based SRT subtitle
- Video with burned-in subtitles
- Before-and-after visual evidence