Find What Customers Really Talk About: Group Reviews and Tickets into Topics with Embeddings

Predefined categories only find the problems you already expect. A new production batch with a weak hinge, a courier that loses parcels in one region, or a confusing setup step shows up as many differently worded complaints, often in several languages. Embeddings turn every text into a list of numbers that represents its meaning, so texts about the same thing end up close together whatever their wording or language. This guide groups reviews, tickets or survey answers into topics with a multilingual embedding model and k-means clustering, lets a local language model name each topic, and writes a report with topic sizes, average ratings, example quotes and monthly trends.

Step 1: Install the Tools

apt install -y python3-venv
python3 -m venv /opt/reviews                         # skip if you created it for the review analysis guide
/opt/reviews/bin/pip install ollama numpy scikit-learn
ollama pull bge-m3          # multilingual embedding model
ollama pull qwen2.5:7b      # names the topics

Ollama setup is covered in running a local LLM with Ollama.

Step 2: Prepare the Texts

Use a UTF-8 CSV file with a text column. Optional columns are used when present: rating adds the average rating per topic, and date (YYYY-MM-DD) adds a table of topics per month. The review file from analysing customer reviews with a local LLM works unchanged. You need at least 50 texts; several hundred or more give clearer topics.

Step 3: Create the Topic Script

cat > /opt/reviews/topics.py <<'EOF2'
import csv, json, os, sys
from collections import Counter
import numpy as np
import ollama
from sklearn.cluster import KMeans
from sklearn.metrics import silhouette_score

EMBED = os.environ.get("EMBED_MODEL", "bge-m3")          # multilingual embedding model
MODEL = os.environ.get("MODEL", "qwen2.5:7b")
LANGUAGE = os.environ.get("REPORT_LANG", "English")
LABEL = {"type": "object", "properties": {"label": {"type": "string"}, "description": {"type": "string"}},
         "required": ["label", "description"]}

with open(sys.argv[1], encoding="utf-8-sig") as f:
    rows = [r for r in csv.DictReader(f) if r.get("text", "").strip()]
if len(rows) < 50:
    sys.exit("Need at least 50 texts to find meaningful topics.")
texts = [r["text"].strip()[:2000] for r in rows]

vectors = []
for i in range(0, len(texts), 64):                        # embed in batches of 64
    vectors += ollama.embed(model=EMBED, input=texts[i:i + 64]).embeddings
    print(f"embedded {min(i + 64, len(texts))}/{len(texts)}", flush=True)
X = np.array(vectors, dtype=np.float32)
X /= np.linalg.norm(X, axis=1, keepdims=True)

if os.environ.get("K"):
    k = int(os.environ["K"])
else:                                                     # try 3..15 topics, keep the best-separated result
    scores = {}
    for n in range(3, max(4, min(15, len(texts) // 10)) + 1):
        scores[n] = silhouette_score(X, KMeans(n_clusters=n, n_init=10, random_state=0).fit_predict(X))
    k = max(scores, key=scores.get)
km = KMeans(n_clusters=k, n_init=10, random_state=0).fit(X)

topics = {}
for c in range(k):
    idx = np.where(km.labels_ == c)[0]
    nearest = idx[np.argsort(np.linalg.norm(X[idx] - km.cluster_centers_[c], axis=1))[:12]]
    r = ollama.chat(model=MODEL, format=LABEL, options={"temperature": 0}, messages=[
        {"role": "system", "content": f"These customer texts belong to one topic. Give the topic a short label "
         f"(max 6 words) and a one-sentence description, both in {LANGUAGE}."},
        {"role": "user", "content": "\n---\n".join(texts[j] for j in nearest)}])
    t = json.loads(r.message.content)
    ratings = [float(rows[j]["rating"]) for j in idx if rows[j].get("rating", "").replace(".", "", 1).isdigit()]
    topics[c] = {"label": f"T{c + 1} {t['label']}", "description": t["description"], "count": len(idx),
                 "rating": round(sum(ratings) / len(ratings), 2) if ratings else "",
                 "examples": [texts[j][:200] for j in nearest[:3]]}

with open("texts_with_topics.csv", "w", newline="", encoding="utf-8-sig") as f:
    w = csv.writer(f)
    w.writerow(list(rows[0].keys()) + ["topic"])
    for r, c in zip(rows, km.labels_):
        w.writerow(list(r.values()) + [topics[c]["label"]])

with open("topics.md", "w", encoding="utf-8") as f:
    for c, t in sorted(topics.items(), key=lambda x: -x[1]["count"]):
        f.write(f"## {t['label']}: {t['count']} texts ({t['count'] * 100 // len(rows)}%)"
                + (f", average rating {t['rating']}" if t["rating"] != "" else "") + "\n\n")
        f.write(t["description"] + "\n\n" + "".join(f"> {e}\n>\n" for e in t["examples"]) + "\n")
    if "date" in rows[0]:                                 # topic share per month, to spot rising problems
        months = sorted({r["date"][:7] for r in rows if r.get("date")})
        f.write("## Topics per month\n\n| Topic | " + " | ".join(months) + " |\n|---|" + "---|" * len(months) + "\n")
        for c, t in topics.items():
            counts = Counter(r["date"][:7] for r, l in zip(rows, km.labels_) if l == c and r.get("date"))
            f.write(f"| {t['label']} | " + " | ".join(str(counts.get(m, 0)) for m in months) + " |\n")
print(f"done: {k} topics -> topics.md, texts_with_topics.csv")
EOF2

The script embeds the texts in batches, tries between 3 and 15 topics (fewer for small files) and keeps the number that separates the groups best (the silhouette score), then sends the 12 texts closest to the centre of each topic to the language model to name it.

Step 4: Run It

cd /data/reviews
REPORT_LANG="Simplified Chinese" /opt/reviews/bin/python /opt/reviews/topics.py reviews.csv

# Force a fixed number of topics, for example 8
K=8 REPORT_LANG="Simplified Chinese" /opt/reviews/bin/python /opt/reviews/topics.py reviews.csv

The results are topics.md, which lists the topics from largest to smallest with their share, average rating, description, example quotes and monthly counts, and texts_with_topics.csv, your original file with a topic column added.

Step 5: Interpret the Topics

• Large topics with a low average rating are your biggest pain points.
• A topic that grows month by month points to something that changed: a new batch, supplier, courier or listing text.
• A topic that mixes two subjects means there are too few topics; run again with a higher K. Many tiny, similar topics mean K is too high.
• Filter texts_with_topics.csv by a topic to read all of its texts before you decide what to do.

Step 6: Put the Results to Work

• Combine topics with the per-review sentiment and aspects from the review analysis guide for a complete picture.
• Turn frequent questions into FAQ entries for your website or support chatbot, see adding an AI customer service chatbot with Dify.
• Prepare consistent answers for recurring complaints, see drafting replies to customer reviews with AI.

FAQ

How long does it take?

Creating embeddings is much faster than generating text: a few thousand short reviews usually take minutes on a multi-core CPU. Naming the topics takes one model answer per topic.

Will I get the same topics every time?

With the same data and settings, yes: the clustering uses a fixed random seed and the model runs at temperature 0. New data or a different K changes the topics.

Does it work with mixed languages?

Yes. bge-m3 places texts with the same meaning close together across languages, so Chinese, English and Thai complaints about the same problem can end up in one topic. The topic names are written in REPORT_LANG.

Still stuck after following these steps? Open a support ticket and the IMIDC 24/7 technical team will help. Include the server IP, the number of texts, the models you use and the error output so we can pinpoint the issue faster.