Embeddings & Meaning

Turn text into numbers that capture meaning, so a computer can find what you meant — not just the words you typed.

Building with AI · Lesson 13 · 10 min read

A customer types “my card got declined” into your help search. The article that answers it is titled “Payment failures.” They share zero words, so old-fashioned keyword search returns nothing and the customer leaves angry. To build RAG, you first need a way for a computer to tell that two phrases mean the same thing even when they look nothing alike. That trick is called an embedding.

An embedding is text turned into a list of numbers

An embedding is a model that converts a piece of text into a fixed-length list of numbers — a vector (think hundreds or thousands of numbers). It’s the same learning-from-examples trick from Module 1: the model was trained on huge amounts of text until it learned to place text with similar meaning at nearby positions, and text with different meaning far apart. The individual numbers aren’t human-readable — what matters is where a piece of text lands relative to everything else.

Imagine a tiny 2-number embedding (real ones are much bigger):
• “my card got declined” → [0.91, 0.10]
• “payment failure”      → [0.88, 0.13]   ← almost the same spot
• “sunny beach vacation” → [0.05, 0.97]   ← far away
No shared words, but the first two land right next to each other because they mean the same thing.

Closeness = similarity in meaning

Once text is a point in space, “do these mean the same thing?” becomes a math question: how close are the two points? The common measure is cosine similarity — roughly, do the two vectors point in the same direction (≈1 = very similar) or different directions (≈0 = unrelated). Semantic search falls right out of this: embed the user’s query, embed every document once in advance, and return the documents whose vectors are closest to the query’s. That nearest-neighbor lookup is the retrieval engine inside RAG.

What this buys you — and its honest limits

Embeddings let you match by meaning, handle synonyms and paraphrases for free, and even work across languages with the right model. But keep three things straight: (1) you must embed the query and the documents with the same model — vectors from different models live in different spaces and aren’t comparable; (2) embeddings reflect their training, so a niche domain may need a domain-tuned model; and (3) “close in meaning” is not “correct” — retrieval finds relevant text, and the language model still has to use it well. Embeddings are a fast, fuzzy meaning-matcher, not a fact-checker.

Semantic search end-to-end for the help-desk problem:
1. Ahead of time, embed every help article. “Payment failures” → some vector v1.
2. A user searches “my card got declined.” Embed the query → vector q.
3. Compute cosine similarity of q against every article vector. q vs v1 ≈ 0.92 (very close); q vs the “reset your password” article ≈ 0.11 (far).
4. Return the top matches → the “Payment failures” article surfaces first, despite sharing no words with the query. Keyword search would have returned nothing.