What is an AI agent (and when you don't need one)
An LLM takes a prompt and gives one answer. An AI agent uses an LLM in a loop: it reasons about what to do, takes an action by calling a tool, observes the result, and reasons again, until the task is done. That lets it do work, like looking up an order, searching the web, writing and running code or booking something, instead of only answering. Agents are powerful for tasks where the steps can't be fixed in advance, and they are slower, more expensive and harder to test than code, so if a script or a fixed workflow can do the job, use that instead.
LLMs predict, agents execute
Ask an LLM what the weather is in Indore right now. It can't know, so it either says it doesn't know or guesses from its training data.
Ask an agent the same question. It recognises that it needs live data, calls a weather API, gets the actual temperature and tells you.
That is the whole difference in one example: an LLM predicts text, an agent executes actions. One call and one answer, versus a loop that keeps going until a job is done.
Why LLMs need tools
Take a simple question: what is 65 multiplied by 85? A language model has no calculator inside it. It predicts the answer as text, from patterns in its training data, and for many calculations that prediction is wrong. Chat apps like ChatGPT get this right today because they are already agents behind the scenes, with tools like code execution.
People don't calculate 65 × 85 in their heads either. We invented calculators, and we know that to use one, we only need to pass 65, the multiply sign and 85, not the whole sentence. An agent does the same thing: it understands the request with the LLM, works out which tool it needs and what to pass to it, calls the tool, and uses the result.
How an agent works
Most agents follow the same loop, often called ReAct (reason and act):
- Thought: the model reasons about the goal and what to do next.
- Action: it chooses a tool and its arguments, like
calculator("65 * 85")orget_order("ORD-123456"). - Observation: your code runs the tool and gives the result back to the model.
- Repeat until the model decides the task is done and gives the final answer.
The parts that make this possible:
- A model that can reason and choose tools.
- Tools: functions, APIs, databases, search, code execution, other agents.
- Instructions: the goal, the rules, when to use which tool, when to stop.
- Memory: what has happened so far in this task, and sometimes what it learned in earlier tasks.
- A loop with limits: how many steps it can take, how much it can spend, when it must ask a person.
Anthropic's description of an agent is a good one to keep in mind: a model using tools in a loop, based on feedback from its environment. The environment is wherever the agent works, a browser, a codebase, a database, a set of APIs.
Agents compared with things you already know
| What it does | Example | |
|---|---|---|
| LLM | Answers from what it learned | "Explain our refund policy" (it can't know yours) |
| RAG | Answers from your retrieved documents | "What is our refund policy?" (from your policy PDF) |
| Script or automation | Runs fixed steps written in code | Every night, export orders to a spreadsheet |
| Workflow | Fixed steps, some using an LLM | Classify each email, then route it to a team |
| Agent | Decides its own steps with tools | "Find why this customer was charged twice and fix it" |
You can combine them: an agent can use RAG as one of its tools, and a workflow can contain a step that is an agent.
When you don't need an agent
This is the most important question on this page.
If a Python script, a normal workflow or an existing product solves the problem, don't add an agent. Scripts handle exactly the cases you wrote them for, but they do it the same way every time, for free, in milliseconds, and they're easy to test. An agent needs API calls, token tracking, an LLM in the loop, error handling for its decisions, evaluation and monitoring, and every one of those is cost and maintenance. If you solve the problem with less software and less maintenance, that is also good engineering.
Use an agent when:
- The steps can't be known in advance and depend on what the agent finds along the way.
- Inputs vary too much for fixed rules, like free-text requests from customers.
- The task needs judgement between many possible actions.
- A wrong step can be detected and corrected, or approved by a person before it matters.
Agents in production
Recruiters and clients expect people who build agents to understand the topics, like tools, planning and memory, and above all to have production experience. A business only cares about an agent that keeps working in production, and that is where most agents fail: tools time out, models choose the wrong tool, loops never end, costs climb, and nobody can explain what happened in a run.
That is why the rest of this section is as much about control as about capability: workflows vs agents, tool design, human in the loop, state and recovery and cost control.
Start simple
Agents work best when each one is simple and solves one specific problem. It is the same divide and conquer idea as differentiation and integration in maths: take a big problem, break it into small parts you can solve, then put the solutions together. One agent that reliably handles refunds is worth more than one "assistant" that tries to do everything and fails unpredictably.
What this section covers
| Page | The question it answers |
|---|---|
| Workflows vs agents | Should this be fixed steps or an agent? |
| Tool design | How do I design tools agents use correctly? |
| Model Context Protocol | How do agents connect to tools and data in a standard way? |
| Planning patterns | ReAct, plan-and-execute, reflection: which to use? |
| Agent memory | What should an agent remember, and where? |
| Multi-agent systems | When do several agents help? |
| Agent frameworks | LangGraph, CrewAI and others: do I need one? |
| Human in the loop | Where should a person approve or review? |
| State and recovery | How do long-running agents survive failures? |
| Computer-use agents | Agents that use browsers and screens |
| Coding agents | How coding agents work and how to work with them |
| Voice agents | Real-time voice: latency and interruptions |
| Cost control | How to stop agents from running up the bill |
From the floor
In the AI agents bootcamp orientation, I made the same point I make here: you can use scripts for automation, and a script only handles the specific task and the combinations you code for, while an agent can handle much more variety. But if the work can be done through scripts, use scripts, because an agent uses resources, needs API calls, tracking and an LLM in the loop, and all of that has to be configured and maintained.
The first tool every agent gets in our sessions is a calculator, with the question about multiplying 65 and 85, because it shows the gap between an LLM and an agent in a way nobody forgets: the LLM has to predict the answer, and the agent just uses the calculator.