Using FFMPEG and Whisper to add subtitles to a video

I used whisper and ffmpeg at the command line to add subtitles to a video. It worked really well and this is the process I will use for this.

This is James / experimenting

I want start creating short videos for sharing on LinkedIn. Apparently a lot of people watch videos on LinkedIn with sound turned off, and so it’s important for them to have subtitles. I know I can use the Swiss army knife of video FFMPEG to add subtitles from a SRT (SubRip file format) file, and I can create one of those from a video file using the wonderful Whisper tool from OpenAI. I already have both installed on OSX so this should be simple.

I’m going to use the video I created for the page I wrote about my personal knowledge management system, which you can see here: Tiro, a personal note-processing system.

First I will create a .wav audio file from the video file I have. I think some versions of whisper can read video files, but I installed a version that is optimized for my Macbook Pro processor and this version can’t read video files:

ffmpeg -i tiro-video.mp4 -vn -acodec pcm_s16le -ar 16000 tiro-video.wav

Now I will create the .srt file using Whisper:

whisper -m /Users/jamesjohnson/tools/whisper/whisper.cpp/models/ggml-base.en.bin --output-srt tiro-video.wav

Wow it is so fast. It took five seconds to transcribe a five and half minute audio file.

Finally, let’s add the subtitles using FFMPEG:

ffmpeg -i tiro-video.mp4 -vf subtitles=tiro-video.wav.srt tiro-video-subs.mp4

That was pretty quick. And this is our result:

That’s pretty good right? It’s a little bit ugly, but very readable, and whisper has done a great job on the transcription. This will be my standard process for adding subtitles to videos from now on.