Automatic Subtitles with faster-whisper on a CPU Server: Batch SRT Files, Simplified Chinese and ffmpeg

Subtitles make short videos easier to watch without sound, help platforms understand your content, and turn recorded calls, meetings and live streams into searchable text. Whisper is an open-source speech recognition model that handles Chinese, English and dozens of other languages well, and faster-whisper runs it efficiently on an ordinary CPU server. This guide installs faster-whisper, chooses a model size, converts audio and video into SRT subtitle files in bulk, gets clean Simplified Chinese output, and burns subtitles into videos with ffmpeg.

Step 1: Install faster-whisper

# Debian / Ubuntu
apt install -y python3-venv ffmpeg
python3 -m venv /opt/whisper
/opt/whisper/bin/pip install --upgrade pip
/opt/whisper/bin/pip install faster-whisper "av<19"    # PyAV 19 is not yet compatible with faster-whisper 1.2

A separate virtual environment keeps the libraries away from the system Python. ffmpeg is used for extracting audio and burning subtitles.

Step 2: Choose a Model Size

Start with small, transcribe a 5-minute sample and time it, then decide whether the accuracy of medium or large-v3 is worth the extra time for your content.

Step 3: Create the Transcription Script

cat > /opt/whisper/transcribe.py <<'EOF2'
import os, sys
from faster_whisper import WhisperModel

def ts(t):
    ms = int(round(t * 1000))
    h, ms = divmod(ms, 3600000)
    m, ms = divmod(ms, 60000)
    s, ms = divmod(ms, 1000)
    return f"{h:02d}:{m:02d}:{s:02d},{ms:03d}"

size = os.environ.get("WHISPER_MODEL", "small")
lang = os.environ.get("WHISPER_LANG") or None          # e.g. zh, en; empty = auto-detect
prompt = os.environ.get("WHISPER_PROMPT") or None
model = WhisperModel(size, device="cpu", compute_type="int8", cpu_threads=os.cpu_count())

for path in sys.argv[1:]:
    segments, info = model.transcribe(path, language=lang, initial_prompt=prompt,
                                      beam_size=5, vad_filter=True)
    out = os.path.splitext(path)[0] + ".srt"
    with open(out, "w", encoding="utf-8") as f:
        for i, seg in enumerate(segments, 1):
            f.write(f"{i}\n{ts(seg.start)} --> {ts(seg.end)}\n{seg.text.strip()}\n\n")
    print(f"{path}: language={info.language} -> {out}")
EOF2

vad_filter skips silence and music without speech, which speeds things up and avoids invented text in silent parts.

Step 4: Transcribe Files and Folders

# One file, automatic language detection
/opt/whisper/bin/python /opt/whisper/transcribe.py /data/videos/demo.mp4

# A whole folder with the medium model, in the background, with a log
cd /data/videos
WHISPER_MODEL=medium nohup /opt/whisper/bin/python /opt/whisper/transcribe.py *.mp4 > transcribe.log 2>&1 &
tail -f transcribe.log

Each input gets an .srt file with the same name next to it. faster-whisper reads most audio and video formats directly; for very large files you can extract the audio first with ffmpeg -i in.mp4 -vn -ac 1 -ar 16000 in.wav.

Step 5: Get Clean Simplified Chinese

For Mandarin, Whisper sometimes writes Traditional characters or leaves out punctuation. Fix the language and give it a short Simplified Chinese example sentence as the initial prompt; the model follows its style:

WHISPER_LANG=zh WHISPER_PROMPT="以下是普通话的句子，请使用简体中文和标点符号。" \
  /opt/whisper/bin/python /opt/whisper/transcribe.py livestream-0612.mp4

The same trick works for product names and jargon: put the correct spellings into the prompt so the model uses them.

Step 6: Burn Subtitles into a Video

# Hard subtitles (always visible), keeping the original audio
ffmpeg -i demo.mp4 -vf "subtitles=demo.srt:force_style='FontSize=20,Outline=1'" -c:a copy demo-sub.mp4

# Or add them as a selectable subtitle track (MP4)
ffmpeg -i demo.mp4 -i demo.srt -c copy -c:s mov_text demo-softsub.mp4

For Chinese subtitles, make sure a Chinese font is installed on the server (for example apt install fonts-noto-cjk), otherwise the characters appear as boxes. Most platforms, including YouTube, also accept the .srt file as an upload next to the video.

FAQ

How long does it take?

It depends on the model and the CPU. On a modern multi-core server the small model typically processes audio faster than real time; medium and large-v3 are several times slower. Measure with a short sample before processing large archives.

The text repeats or contains phrases that were never said

This usually happens in long silent or music-only parts. Keep vad_filter enabled, try a larger model, and check that the audio is not too quiet or distorted.

Can I translate the subtitles?

Whisper can translate speech directly into English (task="translate" in model.transcribe). For other target languages, translate the SRT text with a language model, for example the local one from running a local LLM with Ollama.

Still stuck after following these steps? Open a support ticket and the IMIDC 24/7 technical team will help. Include the server IP, OS version, CPU and RAM, the command you ran and the full error output so we can pinpoint the issue faster.