Query Rewriting & Expansion
You’ve been improving the documents. But the user’s raw question is often the weakest link — so rewrite it into something retrieval can actually match.
Everything so far — chunking, embeddings — has improved the documents side of retrieval. But there are two sides to a match, and we’ve ignored the other: the query itself. Real users type “it’s broken, help” or “that thing from the meeting,” and retrieval, which matches meaning-to-meaning (lesson 45), has nothing solid to grab onto. The document might be perfectly chunked and embedded and still not get found, because the query was too vague, too short, or used different words than the docs. The fix is to improve the query before searching. Hold the question: if the answer exists in your docs but the user’s wording can’t find it, what could you do to the query to bridge the gap?
Rewriting: turn a bad query into a good one
Query rewriting uses a quick model call to transform the user’s raw input into a cleaner search query before retrieval runs. It fixes several classic problems: it resolves references (“that thing from the meeting” → the actual topic, using conversation context from Module 2), adds missing specifics, and translates casual phrasing into the vocabulary your documents actually use. Crucially, in a back-and-forth chat, a follow-up like “what about the second one?” is meaningless to a fresh retrieval — rewriting folds in the context to produce a standalone query (“the pricing of the Pro plan”). One small model call turns an unsearchable fragment into something retrieval can match.
Rewriting a chat follow-up: • Conversation so far: user asked about “the Starter and Pro plans,” got an answer. • Raw follow-up: “and the second one’s limits?” → retrieval sees “second one’s limits,” retrieves garbage. • Rewritten (context folded in): “What are the usage limits of the Pro plan?” → now retrieval has real, standalone terms to match. Same intent, findable query.
Expansion and the hypothetical-answer trick
Query expansion widens the net: generate a few variations of the query — synonyms, related terms, rephrasings — and retrieve for all of them, then merge the results. This catches documents that used different words for the same idea (a user’s “laptop won’t turn on” vs a doc’s “device fails to boot”). A clever cousin is the hypothetical-answer trick (often called HyDE): instead of embedding the short question, ask the model to write a plausible answer to it first, then embed that and search with it. Why it helps: a full hypothetical answer looks much more like the real answer document than a terse question does, so it lands closer to it in meaning-space. You’re not trusting the made-up answer — you’re using it purely as a better-shaped search key.
Split multi-part queries (decomposition, again)
The last move you already know. A question like “How do refunds work and what’s the SLA for enterprise?” is really two questions whose answers live in different documents — a single embedding of the whole thing lands between both topics and retrieves a muddled mix (the averaging failure from lesson 45, now on the query side). The fix is decomposition (lesson 40): split the compound query into its parts, retrieve for each separately, and combine the found context. This is the same principle running through the whole module — align the unit you search with to one coherent idea, whether that unit is a chunk (document side) or a sub-query (query side). Rewriting cleans the query, expansion widens it, HyDE reshapes it, and splitting divides it — four ways to stop making retrieval guess.
Picture asking a librarian for help. If you mumble “you know, that blue book about the thing,” they’re stuck. A good librarian doesn’t just search your exact words — they reframe your request: “Ah, you mean the finance textbook with the blue cover — let me also check under ‘accounting’ and ‘bookkeeping’” (rewriting + expansion). If you ask a two-part question, they split it and fetch from two sections. And the hypothetical-answer trick is like the librarian imagining what the right book’s back-cover blurb would say, then finding the book that matches that — because a full description matches a book far better than a vague question does. The search got better without touching a single book on the shelves.
Four queries, four moves: 1. “it crashed again” (in a support chat) → rewrite using context: “The dashboard crashes on export in version 2.3.” 2. “laptop won’t turn on” → expand: also search “device won’t boot,” “won’t power on.” 3. A one-line obscure question → HyDE: draft a plausible paragraph answer, embed that, search with it. 4. “refund policy and enterprise SLA?” → split into two sub-queries, retrieve each, combine. Each technique targets a different reason the raw query failed.
This is the reading. The interactive version — active-recall quiz, a hands-on experiment you run in your own AI, and an earned mastery check — is free in the app.
Start this lesson free →