Processing a podcast transcript with the ChatGPT API

artificial intelligence
chatgpt
python
Published

February 21, 2024

Summary

I was going to use Python and the ChatGPT API to cut up a podcast transcript and process it. But I discovered I could just cut and paste the whole transcript (over 9k words) into ChatGPT, so I didn’t need to do anything more complex than that.

I listened to a great podcast on William Blake. I want to:

  1. Extract the structure of the podcast.
  2. Extract the most important facts.
  3. Extract the most interesting facts.

I generated a transcript of the podcast using Whisper, which I have installed locally.

whisper william-blake-podcast.mp3 --model medium --language English --output_format txt > blake.txt

It took Whisper a few hours to do the processing (I need to install a version that uses the Apple M1 Pro chip properly) and the transcript is perfect as far as I can see. It even manages to transcribe Blake’s obscure made-up names fairly well. Very impressive. It generated timestamps, which I didn’t want - I thought the txt output format didn’t have timestamps. I removed the timestamps in VS Code, then did a search and replace to put everything on one line, then searched and replaced on full-stops so that each sentence is on a separate line.

The transcript is about 9200 words, or about 12k tokens (according to the OpenAI tokenizer). In theory this is about within the maximum number of tokens for a prompt for gpt-4-turbo-preview. However it returns a maximum of 4,096 output tokens, so it would be better to split the transcript up into blocks of say, 2500 words. I don’t want to split up mid-sentence and I could do that automatically with Python just by splitting at full-stops, but I also don’t want to split it up in the middle of a paragraph or section. So I split it up by hand. It only took a moment and sometimes doing stuff by hand is best.

I’ve just discovered that I can cut and paste the entire transcript into a single ChatGPT window, so I don’t need to do anything more complex than that to achieve what I am trying to do. So experiment ended!