Designing a Tool Catalog
A colleague once wrapped our whole orders API behind a single tool. He called it orders, gave it an action field (get, search, refund, cancel) and a params object whose shape changed depending on the action. One tool, four jobs, very DRY. The model could not keep it straight.
Ask it to pull up an order and maybe one time in six it would stuff a search filter into params, or set action to cancel when the user only wanted to look. Nothing was wrong with the code behind the tool. The problem was that to the model, orders was a single instruction that meant four contradictory things, plus a bag of parameters whose shape it had to reverse-engineer from the conversation. We split it into getOrder, searchOrders, refundOrder, and cancelOrder, changed nothing underneath, and the misfires basically stopped.
That is the whole subject of this article, in one story. You already know what a tool is: a name, a description, and a schema, wired to a function you own. This is about designing the set of them so a model actually picks the right one and calls it correctly. It is less like writing code and more like writing an API, for a consumer that reads your docs literally, cannot ask a follow-up question, and gets exactly one shot per turn.
The docs are the interface
Here is the reframing that makes the rest of this click. The model never sees your implementation. It cannot read the function body, step through it, or check what it returns by trying it. All it ever gets is the declaration: the tool’s name, its description, and the JSON Schema for its arguments. That text is the entire interface. The function is just what happens after the model has already committed to a call.
So a tool declaration is documentation, and its reader is a fast, literal-minded new hire who has your toolbox in front of them and no way to walk over and ask what the label means. If two drawers are both labeled “stuff”, they open the wrong one. If a drawer is labeled flag, they guess. If the description says “manage orders” and nothing about when to use it, they reach for it at the wrong moment. Everything that follows is about writing labels this reader can act on.
Granularity: not too coarse, not too fine
The orders mega-tool failed because it was too coarse. One entry point, a mode flag, and a shape-shifting payload is the most common tool-design mistake, and it fails in a specific way: the model has to make two guesses (which mode, what payload shape) before it has done anything, and it gets one of them wrong often enough to hurt.
The fix is not to swing to the opposite extreme. Twenty micro-tools, one per database column, is its own failure. Now the model drowns: it has to read a long menu, hold it all in context, and stitch three calls together to do one obvious thing. list_users, then list_events, then create_event, when what the task wanted was “book a meeting”. Every extra hop is another chance to misfire and another round trip you pay for.
The target is a tool shaped like a task the user wants done, not like one row of your schema or one endpoint of your REST API. If a job almost always takes three of your tools in the same order, that is a hint the three should be one tool that does the whole job. A good scheduleMeeting tool finds a free slot and books it in one call, hiding the three API requests behind it. The model asked to book a meeting, so give it a tool that books a meeting.
Names and descriptions do the steering
Once the granularity is right, the description is where you win or lose the model’s attention. This is the most consequential text in the whole system, because it is the only thing the model reads when deciding whether this tool fits the moment. A good description answers four questions in order: what the tool does, when to reach for it, when not to, and what it hands back.
A few habits do most of the work here, and they cost you sentences, not sprints.
Say when not to use it. The single most useful line in a description is often the exclusion. refundOrder that says “not for cancelling an unpaid order, use cancelOrder” stops the model at the exact fork where it would otherwise flip a coin. Whenever two tools live near each other, name the other one in both descriptions. Overlap you do not disambiguate is overlap the model resolves at random.
Name things the way you would explain them. A parameter called user_id tells the model it wants an id; a parameter called user invites it to pass a name, an email, or a whole object, and now half your calls arrive with the wrong thing in the field. startDate beats date. maxResults beats n. The name is documentation the model reads before it reads anything else.
Write the description like an onboarding doc. Imagine handing this tool to someone on their first day with zero context on your product. What would confuse them? That confusion is exactly what trips the model. Spell out the units (amountCents, not “amount”), the format (ISO 8601 date), the thing that is obvious to you and invisible to a stranger.
import { tool } from "ai"; // provider-neutral shape; the ideas are portable
import { z } from "zod";
// vague: the model has to guess when this fits and what "query" accepts
const search = tool({
description: "Search.",
inputSchema: z.object({ query: z.string() }),
execute: runSearch,
});
// sharp: it says when to use it, when not, and constrains the inputs
const searchDocs = tool({
description:
"Full-text search over the customer's own help articles. " +
"Use when the user asks how to do something or where a setting lives. " +
"Not for live account data (use getAccount) and not for web results.",
inputSchema: z.object({
query: z.string().describe("A few keywords, not a full sentence."),
section: z.enum(["billing", "security", "api", "general"]).optional(),
limit: z.number().int().min(1).max(20).default(5),
}),
execute: runSearch,
});Tight schemas mean less to get wrong
The schema is the other half of the interface, and the same principle from structured output applies: every free-form field is a place the model can wander, and every constraint is a rail it cannot leave. You do not re-teach schema design here, it carries straight over. What changes is the emphasis.
Prefer an enum to a free string any time the answer is one of a known set. reason: enum(["defective", "wrong_item", "late"]) gives the model three legal moves; reason: string gives it a blank page and you get “item was broken”, “broken item”, and “defective” as three spellings of one thing. Give optional fields a sane default or make them genuinely optional, so the model can leave out what it does not know instead of inventing a value to satisfy a required field. Keep the argument object flat and small. Fewer fields, tighter types, and clear names mean fewer independent things the model has to get right in a single shot.
The “too many tools” problem
Every tool you declare is spent twice. It costs tokens, because the full name, description, and schema of every tool is sent to the model on every single turn. And it costs attention, because the model has to read that whole menu and pick from it before it can act. Both costs grow with the size of the catalog, and past a point they bite.
The attention cost is the one that surprises people. Selection accuracy is not flat across catalog sizes. A 2025 benchmark that varied only the number of tools available found accuracy dropping anywhere from a few points to more than half as the catalog grew, and this held even on models with a 128k context window, so “it all fits” is not the same as “it all gets used”. As of 2026, independent tests tend to land frontier models somewhere around the high 80s to mid 90s at roughly fifty tools, and down into the 40s-to-80s range by a couple hundred, depending heavily on the model. Treat the exact numbers as a moving target. The shape is the durable lesson: high and flat for a small set, then a slide.
Some of the decline is a “lost in the middle” effect: a tool sitting halfway down a long list is chosen less reliably than one near the top or bottom, purely because of where it sits. You cannot fix that with better descriptions. You fix it by not showing the model a long list in the first place.
Three moves, roughly in order of effort:
Scope the catalog to the task. The simplest and most underused option. A customer-support agent does not need your deployment tools. If your app has distinct modes, hand the model only the tools that mode needs. A catalog of six beats a catalog of sixty for the same job.
Group and namespace. When you genuinely have many tools, prefix them so their structure is legible: docs_search, docs_get, billing_refund, billing_invoice. The model reads the namespace as a hint about which family a task belongs to, which cuts down cross-family confusion. Naming by service and by resource both help; which prefix scheme wins is worth a quick test rather than a guess.
Retrieve a subset per request. The move that scales furthest. Instead of declaring every tool up front, keep them in an index and, for each user request, pull the handful that are relevant and declare only those. Concretely, you embed each tool’s description, embed the request, and pick the top few by similarity, the same retrieval you would build for RAG. A related pattern gives the model a searchTools function it calls first, gets back the two it needs, and loads only those. Either way the model is always choosing from about five tools, so accuracy stays in the flat part of the curve no matter how big the underlying catalog grows.
Errors are feedback, not failures
Here is a design surface most people forget exists: what your tool returns when it cannot do the thing. The instinct from ordinary code is to throw, or to let a 500 bubble up and crash the loop. That is exactly wrong for a tool, because the model is sitting right there, mid-conversation, ready to try again if you tell it something useful.
A tool result is just data the model reads on its next turn. An error is a perfectly good result. The question is whether it is a result the model can act on.
The difference is entirely in the text you return. "city not found" on its own is barely better than a 500. "no city named 'Sprngfield'; check the spelling or add a country code, for example Springfield, US" tells the model precisely what to change, and a capable model will fix the argument and call again without any help from you. You are writing a prompt, in the error path, aimed at the one reader who can act on it immediately.
async function runTool(name, args) {
if (name === "getWeather") {
const city = CITIES.find((c) => c.matches(args.city));
if (!city) {
// return the miss as data, phrased as an instruction
return {
error: "city_not_found",
message:
`No city matched "${args.city}". Check the spelling, ` +
`or disambiguate with a country code, e.g. "Springfield, US".`,
};
}
return { tempC: city.tempC, condition: city.condition };
}
throw new Error(`unknown tool: ${name}`); // a real bug, not a recoverable miss
}Two boundaries are worth drawing. A tool miss (bad argument, not found, upstream returned nothing useful) should come back as a result the model can recover from, and this pairs with the ordinary retries and backoff you would build for any flaky upstream, with the model as a surprisingly good recovery layer on top. A genuine bug in your own code is different and should throw and be caught by your loop, not fed to the model as if it were a normal outcome. And whatever you return, do not leak internals: a stack trace or a raw SQL error handed to the model is both useless to it and a small information-disclosure hole, since the model may repeat it to the user or to whatever read your context next.
Side effects need more care than reads
A read tool that runs twice is a wasted call. A write tool that runs twice can be a duplicate charge, two emails, a double refund. This matters more with models than with ordinary clients for a specific reason: the loop retries. A network blip, a timeout, a defensively re-run request, and your mutating tool fires again.
So design mutating tools to survive a repeat. The cleanest way is an idempotency key: the tool takes or derives a key, and a second call with the same key returns the first call’s result instead of doing the work again. Same discipline you would use for any payment endpoint, pointed at a caller that is even more likely to retry.
The stakes scale with reversibility. Reading a row is cheap to get wrong. Sending money, deleting data, or emailing a customer is not, and those tools deserve a harder gate than a schema check. Put a ceiling on anything that spends (a refundOrder with no maximum is a prompt away from a bad afternoon), and for the truly irreversible actions put a person in the path. That is human in the loop: the model proposes the call, your system pauses, a human approves, and only then does it run. The model’s request is never the authorization.
Log every call, then evaluate the set
Two closing practices turn a catalog from “works in the demo” into something you can trust and improve.
Log every tool call and its result. Which tool ran, with which arguments, what came back, how long it took. Tool calls are the most interesting and least visible thing your app does, and when the model behaves bafflingly, the trace is how you find out whether it picked the wrong tool, called the right one wrong, or got a bad result and recovered. This is plain observability pointed at the part of the system that improvises, and there is a whole lesson on doing it for AI specifically in tracing AI.
Evaluate the catalog, do not eyeball it. The honest way to know whether a tool’s description is good is to measure how often the model picks it correctly on real tasks, then change one word and measure again. Small refinements to descriptions produce outsized swings in selection accuracy, which is wonderful news (cheap wins) and a warning (you cannot tell by reading). Build a handful of tools for your highest-value workflows, put them behind a small eval set of realistic requests, and scale the catalog up only as the numbers hold. That same eval is how you catch the day a catalog crossed the size where accuracy started to slide.
Watch the descriptions steer the pick
Below is a tiny stand-in for the selection step, with no model and no network in the page. Four tools, each with a description you can edit. Type a request, or pick one, and a crude keyword matcher (playing the part of the model) scores every tool by how well its description overlaps the request, and picks the winner. The point is not that this is how a real model chooses. It is that the only thing it reads is the description, and so the same is effectively true of a real model.
Try the “what’s it like outside in Paris” request first. With the default weather description it may miss, because the words do not overlap. Now edit getWeather’s description to mention “temperature, conditions, whether it is hot, cold, raining, what it is like outside” and run it again. The pick fixes itself. You just did prompt engineering on a tool, which is the entire job.
The lesson underneath the toy matcher holds for the real thing. The model programs against your descriptions. Vague ones get missed calls and wrong picks; sharp ones that say what a tool is for and when to reach for it get the right call at the right moment. The whole discipline is that you are writing an interface for a reader who only has the docs.
Summary
- The model never sees your code. It reads only the tool’s name, description, and schema, so that text is the entire interface. Design it like an API for a literal reader who cannot ask a follow-up question.
- Get granularity right. One
do_everythingtool with a mode flag makes the model guess the mode; twenty micro-tools make it chain and drown. Size tools to tasks the user wants done, not to your database rows or REST endpoints. - Descriptions do the steering. Say what a tool is for, when to use it, when not to (naming the neighbouring tool), and what it returns. Name parameters like
user_id, notuser, and write for someone on their first day. - Tighten schemas: enums over free strings, sane defaults, flat and small. The same discipline as structured output, and it still does not make a valid shape a correct value, so validate in your code.
- Past a couple dozen tools, selection accuracy slides and every declaration costs tokens on every turn. Scope the catalog to the task, namespace it, or retrieve a relevant subset per request so the model always chooses from a handful.
- Return errors as actionable sentences the model can recover from (“city not found, add a country code”), not opaque codes or stack traces. A tool miss is data; a real bug should throw. Never leak internals.
- Mutating tools will be retried by the loop, so make them idempotent, cap anything that spends, and gate irreversible actions behind a human. Split read from write and apply least privilege.
- Log every tool call and result, then evaluate the catalog on realistic requests. Description tweaks swing accuracy hard, so measure instead of eyeballing.