AI Voiceover for Short Videos on Your Own Server: Kokoro Text-to-Speech in Chinese, English, Japanese and More

Short videos for TikTok, Reels, YouTube Shorts and marketplace product pages need narration, and recording every script yourself takes time. Online text-to-speech services charge per character and require you to upload your scripts. Kokoro is an open-weight speech model with natural-sounding voices that is small enough to run on an ordinary CPU server. This guide installs it, helps you choose voices, turns script files into WAV voiceovers with matching SRT subtitles, handles English words inside Chinese narration, and prepares the audio for upload.

Step 1: Install Kokoro

# Debian 12 / Ubuntu 22.04 or 24.04 (Kokoro needs Python 3.10 to 3.12)
apt install -y python3-venv espeak-ng ffmpeg
python3 -m venv /opt/tts
/opt/tts/bin/pip install --upgrade pip
/opt/tts/bin/pip install torch --index-url https://download.pytorch.org/whl/cpu   # CPU-only build, much smaller
/opt/tts/bin/pip install kokoro soundfile "misaki[zh]"

# Japanese voices additionally need:
/opt/tts/bin/pip install "misaki[ja]"
/opt/tts/bin/python -m unidic download          # Japanese dictionary, about 500 MB

If your system Python is 3.13 or newer (for example Debian 13), create the environment with Python 3.12 through uv instead; --seed adds pip, so all later commands stay the same:

curl -LsSf https://astral.sh/uv/install.sh | sh
~/.local/bin/uv venv --seed -p 3.12 /opt/tts
/opt/tts/bin/pip install torch --index-url https://download.pytorch.org/whl/cpu
/opt/tts/bin/pip install kokoro soundfile "misaki[zh]"

The model (about 330 MB) and each voice are downloaded from Hugging Face the first time they are used.

Step 2: Choose a Language and Voice

The first letter of a voice name is its language and the second is female (f) or male (m). Kokoro does not speak Korean, Russian, Thai, Vietnamese or Malay; for those markets, use subtitles or another speech engine.

Step 3: Create the Voiceover Script

cat > /opt/tts/tts.py <<'EOF2'
import os, sys
import numpy as np
import soundfile as sf
from kokoro import KPipeline

LANG = os.environ.get("TTS_LANG", "a")            # a = US English, z = Mandarin Chinese, j = Japanese ...
VOICE = os.environ.get("TTS_VOICE", "af_heart")
SPEED = float(os.environ.get("TTS_SPEED", "1.0"))
PAUSE = float(os.environ.get("TTS_PAUSE", "0.3"))  # seconds of silence after each line
RATE = 24000

def ts(t):
    ms = int(round(t * 1000))
    h, ms = divmod(ms, 3600000)
    m, ms = divmod(ms, 60000)
    s, ms = divmod(ms, 1000)
    return f"{h:02d}:{m:02d}:{s:02d},{ms:03d}"

pipeline = KPipeline(lang_code=LANG, repo_id="hexgrad/Kokoro-82M")
for path in sys.argv[1:]:
    lines = [l.strip() for l in open(path, encoding="utf-8") if l.strip()]
    parts, srt, pos = [], [], 0
    for i, line in enumerate(lines, 1):
        show, _, say = line.partition("|")            # "subtitle text|text to speak" (optional)
        chunks = [np.asarray(a, dtype=np.float32) for _, _, a in pipeline(say or show, voice=VOICE, speed=SPEED)]
        audio = np.concatenate(chunks) if chunks else np.zeros(RATE // 2, dtype=np.float32)
        srt.append(f"{i}\n{ts(pos / RATE)} --> {ts((pos + len(audio)) / RATE)}\n{show.strip()}\n")
        silence = np.zeros(int(PAUSE * RATE), dtype=np.float32)
        parts += [audio, silence]
        pos += len(audio) + len(silence)
    base = os.path.splitext(path)[0]
    sf.write(base + ".wav", np.concatenate(parts), RATE)
    with open(base + ".srt", "w", encoding="utf-8") as f:
        f.write("\n".join(srt))
    print(f"{path}: {len(lines)} lines, {pos / RATE:.1f}s -> {base}.wav, {base}.srt", flush=True)
EOF2

Each line of a text file becomes one sentence of narration and one subtitle. The script writes a WAV file and an SRT file with exactly matching timings next to the text file. Language, voice, speed and the pause between lines are set with environment variables.

Step 4: Generate Your First Voiceover

mkdir -p /data/voice && cd /data/voice
cat > demo.txt <<'EOF2'
这款竹制笔记本支架有六档高度可调。
支持 USB-C 接口和 iPhone。|支持 优艾斯比西 接口和爱疯。
折叠后可以放进背包，重量只有一点一公斤。
EOF2
TTS_LANG=z TTS_VOICE=zf_xiaobei /opt/tts/bin/python /opt/tts/tts.py demo.txt

# English:  TTS_LANG=a TTS_VOICE=af_heart     Japanese:  TTS_LANG=j TTS_VOICE=jf_alpha

In our tests the Chinese voices read numbers such as 39.9 or 2026 correctly, but skip or garble Latin words such as USB-C or iPhone. For such lines, write the subtitle text first, then | and the text to be spoken, as in the second line above: the subtitle shows the correct product name while the voice reads the phonetic version. The same trick helps with brand names in Japanese (write them in katakana after the |).

Step 5: Find the Right Voice and Pace

# Render the same text with several voices to compare them
for v in zf_xiaobei zf_xiaoni zm_yunxi zm_yunjian; do
  cp demo.txt "try-$v.txt"
  TTS_LANG=z TTS_VOICE=$v /opt/tts/bin/python /opt/tts/tts.py "try-$v.txt"
done

# Slightly faster speech and shorter pauses, typical for short videos
TTS_LANG=z TTS_VOICE=zm_yunxi TTS_SPEED=1.1 TTS_PAUSE=0.2 /opt/tts/bin/python /opt/tts/tts.py demo.txt

Download the WAV files and listen on a phone, where most viewers will hear them. Short sentences with commas in natural places sound much better than long, complex ones.

Step 6: Prepare the Audio for Upload

# Normalise loudness to about -14 LUFS (common for short-video platforms) and make an MP3
ffmpeg -i demo.wav -af loudnorm=I=-14:TP=-1.5 -ar 48000 -ac 2 demo-final.wav
ffmpeg -i demo-final.wav -c:a libmp3lame -q:a 2 demo.mp3

Import the audio and demo.srt into your video editor, or let a script build the whole video from photos, voice and subtitles as described in making product short videos automatically. To turn an existing video into another language, see dubbing videos into another language with AI.

FAQ

How fast is it on a CPU server?

Kokoro is a small model. On a modern multi-core server it usually produces speech faster than real time, so a one-minute voiceover takes well under a minute after the model has loaded. Speed depends on the CPU and the length of the text.

Can I clone my own voice?

Kokoro only offers its built-in voices. Voice cloning needs other models, usually with a GPU. Only clone voices with the explicit consent of the speaker, and follow each platform's rules on synthetic content.

Can I use the voices commercially?

At the time of writing, Kokoro's code and model are released under the Apache 2.0 licence, which allows commercial use. Check the current licence on the project page before you rely on it, and follow the platforms' rules for AI-generated content.

English text fails with an espeak-ng error

English and several other languages use espeak-ng for words that are not in the dictionary. Make sure the espeak-ng package from Step 1 is installed, then run the script again.

Still stuck after following these steps? Open a support ticket and the IMIDC 24/7 technical team will help. Include the server IP, OS and Python version, the command you ran and the full error output so we can pinpoint the issue faster.