How the Model Chooses a Tool

The model picks a tool by matching the situation to each tool’s name and description — so those words are the real decision logic.

Building with AI · Lesson 38 · 9 min read

In Module 5 you gave the model one tool and it called it. But a real agent has many tools — search, calendar, email, database — and at each “decide” step it must pick the right one and the right arguments. There’s no if statement doing this; the model just… chooses. So what is it actually choosing from, and why does an agent so often call the wrong tool or none at all? The answer reveals that a tool’s description is its selection logic — and that more tools isn’t always better. Hold the question: with no hard-coded router, how does the model decide which tool fits?

The model matches the situation to each tool’s description

When you give the model tools (Module 5), each comes with a name, a description, and an argument schema. At decide-time, the model reads the current situation (goal + state) and, in effect, compares it against every tool’s name and description to find the best match — then extracts the arguments from the situation. There is no separate routing code; the descriptions ARE the routing logic. A tool described as “search_flights: find available flights between two cities on a date” gets chosen when the situation is about finding flights. Vague or overlapping descriptions → the model guesses wrong, exactly like an under-specified prompt (Module 2) produces vague output.

Two failure modes: wrong tool, or wrong arguments

Selection can go wrong in two distinct ways, and telling them apart is half the debugging (recall the four failure categories from Module 2). Wrong tool: two tools sound similar (“get_user” vs “get_account”) so the model picks the wrong one — a description problem. Wrong arguments: the model picks the right tool but fills a field badly (wrong date format, a made-up id) — a schema/clarity problem. A quick test isolates it: does it reach for the correct tool but botch the inputs, or grab the wrong tool entirely? Each points to a different fix (clearer description vs clearer schema).

Same failure, two causes:
• Wrong tool: user asks to “cancel my order,” agent calls refund_order because both descriptions say “handle order money.” Fix the descriptions so each owns a distinct job.
• Wrong args: agent correctly calls book_flight but passes date: "next Friday" when the schema wants YYYY-MM-DD. Fix the schema/description to demand the exact format.

Fewer, well-separated tools beat a giant toolbox

It’s tempting to hand the agent 40 tools “just in case.” Don’t. The more tools — especially ones with overlapping purposes — the harder the matching problem, and the more often the model picks wrong or stalls. A tight set of tools with crisp, non-overlapping descriptions dramatically improves selection. If two tools could plausibly answer the same situation, that ambiguity is a bug you introduced. The best-performing agents usually have a small, clearly-carved toolset — which is exactly why the next lesson is about designing individual tools well.

Diagnosing a mis-selecting agent:
1. Symptom: the agent keeps emailing customers when it was supposed to log a note.
2. Look at the tools: send_email (“notify the customer”) and add_note (“record an internal note”). Their descriptions overlap on “about the customer.”
3. Root cause: wrong tool, driven by fuzzy descriptions — not a bad model.
4. Fix: sharpen the descriptions (“send_email: sends a message TO the customer’s inbox”; “add_note: saves an internal-only note, never seen by the customer”). Selection snaps into place because the routing logic — the words — is now unambiguous.