Choosing a Model

There is no “best” model — only the right trade-off of quality, speed, and cost for your job.

Building with AI · Lesson 3 · 8 min read

You’re building a feature and there are a dozen models to pick from — some cost 30× more than others, some are twice as fast, some you can run on your own server. A teammate says “just use the most powerful one.” That advice will quietly bankrupt a high-traffic feature or make it painfully slow. How do you actually choose? Hold that — by the end you’ll have a repeatable way to decide.

Frontier vs. open-weight

Frontier models are the largest, most capable models, served through an API by a provider. You don’t host them; you call them and pay per token. Open-weight models have published weights you can download and run yourself — more control and privacy, no per-call fee, but you handle the hardware and the quality is often (not always) a step behind the frontier. Neither is “better” in the abstract; they trade control for capability.

The three dials: quality, latency, cost

Almost every model choice is a balance of three things:

• Quality — how good the answers are on your task.
• Latency — how fast it responds.
• Cost — price per token (and whether you pay at all).

You usually can’t max all three. A frontier model maximizes quality but costs more and is often slower; a small model is cheap and fast but less capable. The skill is matching the dial that matters for this task — not reflexively buying maximum quality you don’t need.

Two features, two right answers:
• A legal-contract analyzer: errors are expensive, volume is low → pay for top quality.
• An autocomplete that fires on every keystroke for millions of users: needs low latency + low cost, and a smaller model is plenty. Same company, opposite choices — driven by the job, not by prestige.

What models can’t do natively

A bare model only predicts text. By itself it can’t browse the web, run code, look up today’s date, or read your private database — it only knows patterns from its training data (which has a cutoff) plus whatever you put in the context window. Those abilities come from tools you wire up around the model (later modules). Knowing the bare model’s limits tells you when you need prompting vs. when you actually need retrieval or tools.

Choosing for a customer-support summarizer:
1. Task: condense a support chat into 3 bullet points. Volume: high. Stakes: medium.
2. Quality bar: moderate — bullets just need to be accurate, not brilliant.
3. Latency: matters (agents are waiting). Cost: matters (thousands/day).
4. Decision: a fast, cheaper mid-tier model, and reserve the frontier model only for the rare escalations. You reasoned from the job to the dials to the pick — repeatable for any feature.