The Agent Loop
The first agent I put in production cost us about 240 dollars in a single night and never finished the job. It was supposed to reconcile eighty invoices against our payment records, one at a time. Somewhere around invoice nine it hit a record it could not match, called the lookup tool, got nothing useful back, decided to try the same lookup again, got the same nothing, and kept going. All night. Same tool, same arguments, a fresh model call every few seconds, until morning and the bill.
Nothing was wrong with the tool. The tool did exactly what I wrote. The bug was that I had built a while loop with no clear idea of when to stop, handed it a spending limit, and gone to bed.
That is the honest version of what an agent is. Not a mind, not a planner with a soul, just a loop around tool calling that runs until something tells it to quit. Once you see the loop, the mystery drains out and the real work shows up: deciding when to stop, keeping the transcript from exploding, and knowing when you should not have reached for an agent at all. This article is about the loop and everything that goes wrong around it.
It is a while loop, and you have already seen it
You met the engine in tool calling. One round trip: the model asks for a tool, your code runs it, you feed the result back, the model answers. The whole thing rides on the ordinary messages array and a while that keeps going as long as the model keeps requesting tools.
An agent is that exact loop, unchanged. You give it more than one tool, you give it a goal big enough to need several turns, and you let the while run more than once. Here it is, stripped to the shape you already know:
const messages = [
{ role: "system", content: "You reconcile invoices against payment records." },
{ role: "user", content: "Reconcile the 80 invoices in batch B-19." },
];
let reply = await callModel(messages, { tools });
while (reply.toolCalls?.length) {
messages.push(reply); // the assistant turn that asked
for (const call of reply.toolCalls) {
const result = await runTool(call.name, call.arguments);
messages.push({ role: "tool", toolCallId: call.id, content: JSON.stringify(result) });
}
reply = await callModel(messages, { tools }); // let it see the results, decide again
}
return reply.content; // a text answer, finallyRead the loop condition again: reply.toolCalls?.length. That is the model’s choice, not yours. It keeps requesting tools until it decides it has enough, then returns text and the while falls through. You are not driving. You wrote a while whose exit is controlled by a text predictor, which is a genuinely strange thing to do and the source of every problem in the back half of this article.
So what makes it feel intelligent, if the control flow is one line you could have written in week two? The adaptation. Each turn the model sees everything so far, the goal plus every tool result, and picks the next move based on what came back. It reads a “payment not found” and tries a different account. It reads three matches and asks which one. That per-turn choosing, across many turns, is the whole illusion of an agent. The loop is dumb. The thing inside the loop is where the capability lives.
People give this cycle a name, perceive, decide, act, and the three beats map cleanly onto the code. The model perceives the transcript so far. It decides on a tool call or a final answer. Your code acts by running the tool and appending the result. Then round again. The name is useful because it tells you where each responsibility lives: perceiving and deciding are the model’s, acting is yours, and the two sides never blur. The model never runs anything. You never decide anything. That boundary is the same one from tool calling, just looped.
Nothing here says when to stop
Look back at that while and find the line that ends it on a bad day. There isn’t one. The loop exits when the model stops asking for tools, and a confused model asks forever. My invoice agent was not broken. It was working precisely as written, and what I wrote was “keep calling the model until it feels finished,” with no floor under “feels.”
So the first thing you add to any real agent, before anything clever, is a set of hard limits it cannot argue its way past. Not one limit. A confused loop can burn you along several different axes at once, and you want a guard on each:
- Steps. Cap the number of iterations. This is the blunt one and the one you must never skip.
- Tokens. Every turn re-sends the whole transcript, so a loop of five turns each pulling a 50k-token document is as expensive as two hundred tiny turns. Count tokens, not just steps.
- Wall-clock. A deadline. If the task is not done in sixty seconds, something is wrong and the user is already gone.
- Cost. The one that actually hurts. Track the running spend and stop at a ceiling.
In code the guards are boring, which is the point. Boring is what you want between a text predictor and your invoice:
const budget = {
steps: 0, maxSteps: 20,
tokens: 0, maxTokens: 200_000,
deadline: Date.now() + 60_000,
};
let reply = await callModel(messages, { tools });
let stopped = null;
while (reply.toolCalls?.length) {
if (++budget.steps > budget.maxSteps) { stopped = "step cap"; break; }
if (budget.tokens > budget.maxTokens) { stopped = "token budget"; break; }
if (Date.now() > budget.deadline) { stopped = "deadline"; break; }
messages.push(reply);
for (const call of reply.toolCalls) {
const result = await runTool(call.name, call.arguments);
messages.push({ role: "tool", toolCallId: call.id, content: JSON.stringify(result) });
}
reply = await callModel(messages, { tools });
budget.tokens += reply.usage?.totalTokens ?? 0;
}
if (stopped) return handleStop(stopped, messages); // surface partial work, do not just die
return reply.content;Notice the loop no longer trusts the model to end itself. It ends on the model’s choice or on any tripped guard, whichever comes first. And when a guard trips you do not throw the whole run in the bin. You return what progress there is and a clear “I stopped because I hit the step cap,” which is a far better user experience than a silent hang or a stack trace.
Most toolkits know this and ship a default cap so a first-time loop cannot run forever. Around twenty steps is a common default. They also give you a proper stop hook instead of a hand-rolled counter: you pass a set of conditions (a step count, “this specific tool was called,” or a custom function that reads the accumulated token and cost usage), and the loop halts when any of them is true.
Noticing when it is stuck
A step cap of twenty saves your wallet. It does not save your latency or your dignity, because twenty identical failing calls is still twenty wasted calls and a user watching a spinner. The cap is a backstop. The better move is to notice the agent is stuck and cut it off early, long before the cap.
The clearest signal is repetition. When the model calls the same tool with the same arguments for the third time in a row, it is not working, it is spinning. A tiny detector catches it:
const seen = new Map();
function isStalling(call) {
const key = call.name + ":" + JSON.stringify(call.arguments);
const n = (seen.get(key) ?? 0) + 1;
seen.set(key, n);
return n >= 3; // same tool, same args, third time: stuck
}Exact repeats are the easy case. The sneakier one is the semantic loop, where the arguments differ but mean the same thing: search("auth error"), then search("login failure"), then search("cannot sign in"), three swings at one idea, none landing. You catch those by watching the results instead of the calls. If several turns go by and no new information entered the transcript, the progress delta is zero and the agent is stuck regardless of how it phrased things.
Catching the stall is half of it. What you do next matters just as much, because “detected a loop, threw an error” is a worse outcome than the loop. The move that works is blunt: take the option to keep looping away and force a decision. Inject a message telling the model it has repeated itself with no new information and must now either answer with what it has or say plainly that it cannot. Most stuck agents produce a perfectly reasonable answer the moment you close the door on trying again.
if (isStalling(call)) {
messages.push({
role: "user",
content:
"You have already run that exact call with no new result. " +
"Do not call it again. Answer with what you have, or tell the user you cannot complete this.",
});
continue; // skip the repeat, let the model respond to the nudge
}That nudge is the difference between an agent that gives up gracefully and one that grinds until the cap. The cap still sits behind it as insurance. You want both, the smart early exit and the dumb backstop, because each covers a case the other misses.
The transcript only grows
Here is the constraint that shapes every long-running agent, and it is the one tutorials skip. The model has no memory between calls. Each turn you send the entire conversation again, from the system prompt to the latest tool result, because that array is the model’s only working memory. There is no session on the other end holding your place. Every call is stateless, and “state” is a thing you carry and resend.
Which means the transcript grows every single turn, and it grows fastest exactly where agents do their work: tool results. A search that returns forty rows, an API response with a hundred fields, a file you read into context. Ten turns of that and you are re-sending a small book on every call. Two things break as it grows. First, you march toward the hard edge of the context window and the call eventually fails outright. Second, and sooner, quality sags. Models get measurably worse at using a long context well before they run out of room, an effect people now call context rot: the important detail from turn two gets buried under nine turns of noise and the model stops paying attention to it.
There are three defenses, cheapest first. Trim results before appending. If the model needs three fields, do not paste back the whole hundred-field response. Shape tool outputs to what the next decision requires. This alone buys most agents all the room they need. Compact old turns. When the transcript crosses a threshold, replace the older middle of the conversation with a short summary the model writes: “checked invoices 1 to 40, matched all but 9 and 14.” The recent turns stay verbatim, the ancient history collapses to a sentence. Externalize memory. For genuinely long tasks, let the agent write notes to a scratchpad outside the window, a file or a row in a store, and read them back on demand, so the durable state lives in storage instead of in every request. That last pattern shades into a topic of its own, context engineering, which is really the discipline of deciding what deserves a place in the window at all.
One caution on compaction, since it is easy to do badly. Do not summarize blindly on a timer. If you compact in the middle of an active sub-task you can throw away the exact detail the model was about to use, and now it is confused and has less to work with. Compact when the window is genuinely filling, prefer to keep the recent turns intact, and treat the summary as lossy, because it is.
A failed tool is a message, not a crash
Tools fail. A record is missing, an upstream times out, a rate limit bites. The instinct from ordinary code is to let the exception propagate, but in an agent that instinct kills the whole run over one bad call. The better default is to catch the failure and hand it back to the model as a tool result, because the model is a surprisingly good recovery layer when you let it see what went wrong.
async function runTool(name, args) {
try {
const input = validate(name, args); // schema-check the model's arguments first
return await tools[name].execute(input);
} catch (err) {
// hand the failure back as data, not as a thrown exception
return { error: err.code ?? "tool_failed", message: String(err.message).slice(0, 200) };
}
}Feed the model { error: "not_found", message: "no payment for invoice 9" } and it can do something intelligent with it: ask the user to confirm the invoice number, try a broader search, or flag that one invoice and move on to the rest. That is the whole batch saved instead of lost. Validating the arguments before you run anything is the same untrusted-input discipline from designing a tool catalog and validating input, and returning failures as data is the tool-loop version of the error handling and retries and backoff you already build for any flaky upstream.
You cannot unit-test a plan
Here is what unsettles engineers coming from normal systems. Run the same agent on the same input twice and it can take two different paths. It might check populations in a different order, phrase a search differently, or solve a task in five steps one time and seven the next. Sampling makes the model’s choices non-deterministic by default, and even pinned low it is not a pure function. Your familiar test, “given this input, assert this exact output,” does not hold.
So you stop testing the path and start testing the outcome. Do not assert the sequence of tool calls, because that sequence is allowed to vary and a passing-then-failing test that only checks the route is noise. Assert what the user actually cares about: did it reach a correct answer, did it stay inside budget, did it never call the delete tool it had no business calling.
// flaky: pins the exact route the model took, which it is free to change
expect(toolCalls).toEqual(["listCountries", "getPopulation", "getCapital"]);
// durable: pins the outcome and the guardrails, and lets the path vary
expect(result.answer).toContain("New Delhi");
expect(result.steps).toBeLessThanOrEqual(10);
expect(result.toolsUsed).not.toContain("deleteRecord");Because any single run can get lucky or unlucky, one pass proves little. You judge an agent the way you judge a flaky integration, over many runs against a set of graded cases, which is the subject of evals. And you make the baffling runs debuggable by logging every turn, every tool call with its arguments and its result, so that when an agent does something inexplicable in production you can replay the exact transcript instead of guessing. That is ordinary observability pointed at the one part of your system that improvises.
When you do not need an agent at all
This is the most valuable judgment in the whole topic, and it runs against the hype, so I will be blunt. Most tasks people reach for an agent to do should not be an agent. If you already know the steps, do not let a model decide the control flow. Write the steps.
An agent buys you exactly one thing: the ability to decide the path at runtime, when you genuinely cannot know it in advance. You pay for that flexibility with latency, cost, and a system you cannot fully test. When the path is fixed, you are paying that price for nothing. My invoice reconciliation is the perfect example of the mistake. The steps never actually varied: fetch the invoice, look up the payment, compare, record the result. That is a for loop over eighty invoices with three function calls in the body. I reached for an agent because “AI” felt like the answer, and I turned a boring, testable, cheap pipeline into an expensive one that hung overnight.
// the steps are known, so just write them: no model deciding control flow
for (const invoice of batch) {
const record = await fetchInvoice(invoice.id);
const payment = await findPayment(record);
results.push(reconcile(record, payment));
}That runs in a second, costs nothing, and cannot loop forever. If a step needs a model, a fuzzy match or a summary, you call the model inside the pipeline for that one step. The model does the judgement it is good at, and your code keeps the control flow it is good at. Reserve the loop for when the shape of the work is genuinely unknown ahead of time: the number of steps depends on what you find, the next move depends on the last result, the tools you need vary per request. That is when handing control to the model earns its cost.
See a loop run, with no model
Below is an agent loop you can drive, with no model and no network anywhere on the page. The task is fixed: find the capital of the country with the largest population, from a small hardcoded list. A pretend agent works it out over several turns, one tool call per turn, exactly like the real loop. It lists the countries, looks up each population one at a time, picks the largest, then asks for that country’s capital and answers. Watch the step counter climb and the message list grow with every turn.
The important control is the max steps slider. The full solve needs seven turns. Drop the cap below that and hit run: the loop stops early with no answer, because the model ran out of iterations before it finished. That is the difference between a cap that lets the work complete and a cap set too tight, and it is the exact knob that stood between me and a 240-dollar night.
Run it once at ten and watch it finish in seven steps. Then slide the cap to four and run again. It never gets past collecting populations, stops with partial progress, and tells you why. Same agent, same tools, same task. The only thing that changed is the number you were willing to pay for, which is the whole lesson of this article in one slider.
Summary
- An agent is a
whileloop around tool calling: send the messages, run any tool the model asks for, append the result, loop, and exit when the model returns text instead of a tool call. - The loop is trivial. The intelligence is the model choosing tools and adapting to results across turns. Perceive, decide, act: the model perceives and decides, your code acts, and the boundary never blurs.
- The loop condition is the model’s choice, so it has no natural stop. Add hard limits it cannot argue past: steps, tokens, wall-clock, and cost. A runaway loop is a runaway bill.
- Budget tokens and dollars, not just steps. Five turns each dragging a huge document is as expensive as two hundred tiny ones. Toolkits ship a default cap (around twenty steps is common), but always keep your own.
- Detect a stuck agent early: the same call three times, or several turns with no new information, means it is spinning. Break the loop and force it to answer or admit it cannot, rather than grinding to the cap.
- The transcript grows every turn because the model is stateless and you resend everything. Long tasks strain the context window and quality sags before the hard limit. Trim big tool results, compact old turns into a summary, and push durable state to a scratchpad.
- A failed tool should come back as a result the model can react to, not a thrown exception that kills the run. But cap the recoveries, or “try again” becomes its own loop.
- You cannot pin the path, so test the outcome, not the sequence of calls. Judge agents over many runs with evals, and log every turn for replay via observability.
- If you already know the steps, write a pipeline. It is cheaper, faster, testable, and cannot loop forever. Reach for the loop only when the path is genuinely dynamic.
- Durability across steps and human checkpoints for irreversible actions build on this same loop, in multi-step workflows and human in the loop.