“Agent” has become one of those words that means whatever the speaker needs it to mean. Strip away the marketing and most agents come down to a small loop. You give a language model a goal and a list of tools. It either answers or asks for a tool. Your code runs the tool, appends the result to the conversation and asks again. It repeats until the model answers or you run out of patience.
This lab is that loop with nothing hidden. You ask a question about the writing on this site. The model can search about 260 passages taken from the articles here, read one in full, or use a calculator. Every model call and tool run appears in the trace in the order it happened, with its latency, its token count and, for Claude, its cost.
There are two ways to run it. With your own Anthropic API key, the browser calls Claude directly. The key lives only in this tab’s memory. Or, in a recent desktop Chrome, it can use Gemini Nano, a small model that Chrome runs on your own device. Neither way needs a server of mine. The tools are ordinary TypeScript running in your browser.
The loop, in about twenty lines #
Here is the core of it, simplified from the lab’s code:
let messages = [{ role: 'user', content: question }];
for (let step = 1; step <= maxSteps; step++) {
const response = await client.messages.create({ model, system, tools, messages });
messages.push({ role: 'assistant', content: response.content });
if (response.stop_reason !== 'tool_use') return finalText(response);
const results = [];
for (const call of response.content.filter((b) => b.type === 'tool_use')) {
results.push({ type: 'tool_result', tool_use_id: call.id, content: runTool(call) });
}
messages.push({ role: 'user', content: results });
}A few details in there are easy to get wrong.
The model never runs anything. It returns a structured request, such as search_site with {"query": "ravine learning rate"}. Your code decides whether and how to run it. That is where permissions, validation and sandboxing belong. The calculator here parses arithmetic itself and never calls eval, because the model’s output should be treated as untrusted input.
The model has no memory between calls. Each request sends the whole conversation again: the question, every earlier reply, every tool call and every tool result. Watch the “tokens in” number grow from step to step. In one run I recorded, it went from 989 to 2,586 to 2,893 tokens. Long agent runs get slower and more expensive for this reason alone. Techniques like prompt caching and clearing old tool results exist to manage that growth.
Errors are results too. If a tool fails, for example when the model asks to read a passage id that doesn’t exist, the lab doesn’t crash. It sends the error message back as the tool result, flagged as an error. Models are good at reading “No passage with id X. Use an id from search_site.” and trying again. A loop that throws away errors takes away the information the model needs to correct itself.
Reading a trace #
Ask the default question, How much KV cache does Llama 3.1 8B need for one 32K-token conversation?, and look at what a good trace does:
- It searches, sometimes with two queries at once. Claude can request several tool calls in one turn, and the lab sends all their results back together.
- It reads the most promising passage in full, instead of trusting a 280-character snippet.
- It calculates with the numbers it just read.
- It answers and cites the passage ids it used.
Search results are not always useful. In one run on the gradient descent example question, a second, more abstract query about “curvature” and “stability” returned a passage about black holes. The model ignored it and read the right article. That is normal. Retrieval gets things wrong, and part of an agent’s job is to recognise an unhelpful result and try something else.
Where it goes wrong #
The most useful comparison I found came from asking two Claude models for Llama 3.1 8B’s KV cache size per token, with the layer count, KV heads and head size in the question.
Claude Sonnet 5 searched first, found the formula in the transformer notes, read that passage and computed bytes. That is correct.
Claude Haiku 4.5 skipped the search. The question seemed to contain everything it needed, so it went straight to the calculator with 32 * 8 * 128 * 2. The calculator correctly returned 65,536. The answer was still wrong by a factor of two, because the cache stores both a key and a value. The tool worked. The model used the wrong formula.
That is one run of each model, not a benchmark, but the pattern is common. Tools make the steps a model chooses reliable. They do not make its choices correct. A calculator removes arithmetic mistakes and does nothing for a mistake in the setup. This is also why the system prompt says to search before relying on details. It is a small nudge toward checking sources, and the stronger model followed it.
Two ways to call a tool #
The Claude and Gemini Nano versions look similar in the trace, but they work differently.
Claude supports native tool use. Each tool is described with a JSON Schema, and the model returns structured tool_use blocks. The lab also marks the tools strict, so the API guarantees the arguments match the schema.
Chrome’s built-in model has no tool API in this setup. Instead, the lab uses the older ReAct pattern: reason, then act. The system prompt lists the tools in plain text. Every reply must be a JSON object with a short thought, an action and an input. Chrome’s responseConstraint option enforces that shape during generation, so the output is always parseable, even if the choice inside it is poor. After each action, the lab sends back an Observation: message with the tool’s result.
Both approaches run the same loop. The difference is who enforces the format. With native tool use, the model and API were trained and built for it. With ReAct, you describe the format in the prompt and add a decoding constraint. Small on-device models have far less capacity than frontier models, so expect Nano to pick odd search terms, repeat itself or stop early. Seeing those failures in the trace is part of what the lab is for.
Why the step limit matters #
Every agent needs a stopping rule that doesn’t depend on the model. Without one, a confused model can search forever, and each lap costs time and money. This lab stops after the step limit you choose and says so in the trace.
In production you would usually add more guards: a token or cost budget, a wall-clock timeout, and a rule for tools with side effects. A tool that only reads, like the ones here, is safe to call in a loop. A tool that sends an email or deletes a file needs a person to confirm it, or at least an idempotency key, so that a retry doesn’t repeat the action.
What to take from it #
Most of the engineering in an agent is outside the model: the tool descriptions, what each result returns and how long it is, how errors are reported, when the loop stops, and what gets logged. A trace like this one is the most useful debugging tool you have. When an agent gives a wrong answer, the trace usually shows the step where it went wrong. It might be a bad search query, a skipped read, a misread result or, as with Haiku above, a confident shortcut.
Open What the model was sent at the bottom after a run. It shows the complete context for the last call, exactly as the model received it. Most surprises with agents turn out to be something in that context, or something missing from it.