An AI agent is a software system that uses a language model to pursue a goal over several steps — it decides what to do, uses tools or data to do it, checks the result, and adapts — rather than answering a single prompt in one shot. The difference that matters in practice is autonomy with a purpose: an agent plans, acts and reacts towards an outcome, where a chatbot simply responds. And the hard part is not building one that demos well; it is building one that behaves reliably in production.
What an AI agent actually is
A single call to a language model is a transaction: prompt in, answer out. An agent wraps that model in a loop with a goal. Given an outcome — “find the tenders we could credibly bid for this week”, “research this prospect and draft an opening message” — it works out the steps, executes them with tools (search, databases, APIs, documents), inspects what came back, and decides what to do next. It keeps going until the goal is met, a limit is reached, or the work is handed to a person.
Three properties separate an agent from other software that happens to use a language model:
- It pursues a goal, not a prompt. The unit of work is an outcome, and the agent is judged on whether the outcome was reached — not on whether one answer read well.
- It acts on real systems. An agent earns its keep by calling tools: querying a database, searching a corpus, filing a record, drafting into a workflow. Text that never touches a system is commentary, not work.
- It adapts as it goes. It reads the result of each step and adjusts — retries a failed call, reformulates a search that returned nothing, escalates when it is out of its depth.
None of this needs the word “autonomous” to do heavy lifting. In a production agent the autonomy is bounded on purpose: the goal is scoped, the tools are typed and permissioned, the iterations are capped, and the expensive decisions carry a human handoff. An agent with no bounds is not more capable; it is just unaccountable.
Agent vs chatbot vs RPA
The three get conflated because all three automate knowledge work. They are different tools for different problems.
| Chatbot | RPA | AI agent | |
|---|---|---|---|
| How it works | Responds to each message in turn | Follows fixed, pre-defined rules | Plans and executes steps towards a goal |
| Path through the work | None — one exchange at a time | Identical every run | Reasoned at run time, varies case by case |
| Handles unstructured input | Partially — it can discuss it | No — it breaks | Yes — that is the point of it |
| Fails by | Giving a poor answer | Stopping when the process deviates | Drifting, if unmeasured and unwatched |
| Right for | Conversation and Q&A | Stable, repetitive processes | Work that needs judgement over several steps |
The chatbot comparison is about depth: a chatbot’s intelligence ends at the reply, an agent’s continues into action. The RPA comparison is about flexibility, and it matters enough for a decision that we have written it up separately — see AI agents vs RPA: which do you need? In practice the strongest systems often combine them: the agent supplies the judgement, RPA or plain code executes the deterministic steps.
What an agent is made of
Strip the branding off any serious agent and you find the same five parts.
The model. The reasoning engine — it interprets the goal, plans steps, and makes the judgement calls. Different steps often warrant different models; routing each step to the smallest model that clears the quality bar is one of the main levers on what an agent costs to run.
Tools. The agent’s hands: typed, validated calls into search, databases, APIs, and documents. The discipline is in the typing — an agent that free-forms text at your systems is a liability, whereas a tool layer that validates inputs and outputs makes the agent’s actions checkable.
Memory and context. What the agent knows while it works — the retrieved documents, the results of earlier steps, the rules it must respect. Most of the craft here is retrieval: getting the right material in front of the model at the right step, which is a design problem in its own right.
The planning loop. The machinery that turns a goal into steps and decides what happens after each one: continue, retry, re-plan, or stop. Loops are where agents quietly multiply cost, so production loops are capped and budgeted.
Guardrails and handoff. The bounds: what the agent may touch, how many times it may try, what it must never do, and when a person takes over. In a production agent these are explicit design artefacts, not hopes.
Where agents genuinely help — and where they don’t
Agents are the right tool when the work has judgement in it: the inputs are unstructured, the path varies case by case, and each step depends on what the last one found. Multi-source research is the canonical shape — read widely, extract what matters, cross-reference, draft a grounded output. The tender-intelligence agent we run in production for Magellan Circle is exactly this: it ingests EU tenders from multiple public sources, retrieves against the client’s own material, extracts structured criteria, and drafts fully-cited first-pass bid responses — work that consumed weeks of analyst time, compressed to hours.
Agents are the wrong tool when the process is stable and deterministic — a fixed sequence of steps over structured data wants a script or RPA, which is cheaper, faster, and easier to verify. They are also the wrong tool when a single wrong answer is intolerable and no human review is planned; the honest design there keeps a person in the loop, or does not use an agent at all. Part of our Scope stage exists to say exactly that when it is true.
Why production is the real test
Everything above describes what an agent is. What decides whether one is worth having is different: a demo proves an agent can work once, and production means it works every day, on inputs nobody hand-picked, at a cost the business case survives.
That gap has a shape, and it is mostly engineering. An agent headed for production needs evaluation — a measure agreed up front and a harness that scores the agent against it on real data, repeatably. It needs observability — every run traced step by step, so failures are diagnosed from evidence rather than guessed at. It needs a cost-per-task budget, because a demo runs once and production runs thousands of times. And it needs failure paths designed in advance: timeouts, retries, fallbacks, human handoff. We have written the fuller version of this as how to deploy AI agents in production.
Most agents that fail do not fail loudly. They demo well, ship unmeasured, and quietly get switched off six months later because nobody could say whether they still worked. The definition of an agent is the easy part; the discipline around it is the job.
How we build them
Agent Foundry Labs is a production AI agent studio — we design, build, and run agents through a three-stage engagement: Scope, where we define the agent and agree the measure it must move; Build, where it ships with evaluation, observability, and cost discipline in from day one; and Run, where we keep it healthy and report against the number while it is live.
If there is a workflow in your business you suspect an agent could carry, the first step is not a build — it is agreeing what “working” would mean, with a number attached. Book a 30-minute call and we will start there.