How does an AI agent work: think, call a tool, look at the result, repeat?

It works as a loop: the model decides what to do next, a tool does it, and the result goes back to the model. A language model on its own only produces text. Tool use is the mechanism that lets it interact with external systems, for example fetching real-time information from an API or executing code. A program separate from the model watches its output for a special tool-calling syntax, calls the tool when it appears, and feeds the output back into the model's input.

Why bother? Researchers behind a paper called Toolformer pointed out a paradox: language models struggle with basic functions such as arithmetic or factual lookup, where much simpler and smaller models excel. Their answer was that models can teach themselves to use external tools, a calculator among them.

One recipe for putting thinking and doing together is ReAct, described by Shunyu Yao and colleagues in an October 2022 paper. The model writes out reasoning and actions in turn: the reasoning helps it track and update its plan and handle exceptions, while the actions fetch more information from external sources. In the paper's tests, a model that consulted a simple Wikipedia API overcame hallucination and error propagation seen in plain chain-of-thought reasoning.

Engineers at Anthropic summarize the result: agents are typically just language models using tools based on environmental feedback in a loop. At each step the agent should get ground truth from its environment, such as a tool result or the output of running code, to judge its progress.