The thing most people point to when they say “AI” — the chat window on a website, where you type a question and read an answer — is the least agentic technology in the entire field. Yet “AI” and “AI agent” have collapsed into a single idea in most conversations, and the confusion is causing real mistakes: boards approving “agent” strategies that are actually chatbots, vendors selling “agents” that are actually scripts, and teams wildly over- or under-estimating what they’ve deployed.

The distinction is simple once you see it, and worth getting right, because it changes everything downstream: what the system can do, how it fails, and how you have to govern it.

Here’s the cleanest version. When you use a chatbot, you are the agent. You decide what to ask, you read the answer, you decide the next step, you carry the work forward. The model is a very smart tool you operate one turn at a time. A real agent removes you from that loop: it takes a goal, then decides its own steps and acts on them, with tools, over a long stretch of time, without you driving each move.

A chatbot answers. An agent acts. That’s the whole distinction — and almost nobody is actually running the second thing yet.

For the rest of us: chauffeur vs. handing over the keys

Two analogies make this concrete.

A chatbot is like asking a brilliant expert questions through a hatch. You ask, they answer, you decide what to do with the answer. They never leave the room. They have no idea whether you acted on what they said. Every step of the actual work is yours. This is what a large language model — an LLM, the technology behind ChatGPT and Claude — does in its basic form: it takes your text in, gives text back, and stops. It has no memory of yesterday and takes no action in the world.

An agent is like handing someone the car keys and an address. You don’t dictate the turns. They start the engine, read the road, decide the route, handle the detours, and arrive — checking in only when they’re done or genuinely stuck. In software terms, an agent is an LLM put in a loop: it’s given a goal and a set of tools (the ability to search, run code, send an email, update a record), and it repeatedly decides what to do next, does it, looks at the result, and decides again — until the job is finished.

The difference isn’t intelligence. It’s the same underlying model. The difference is autonomy over time: who decides the next step, and for how long the system runs without you.

When you use a chatbot, you’re the one doing the work

It’s worth being precise about what the familiar chat window actually is, because the familiarity hides how limited it is as an autonomous system.

A chat interface is request-and-response. You send a message, the model generates a reply, and the interaction ends there. Nothing happens until you type again. The model doesn’t pursue a goal across turns; you hold the goal in your head and feed it the work one prompt at a time. You are the planner, the memory, the error-checker, and the hands. The model is an extraordinarily capable component, but it’s a component you operate.

That’s not a criticism — it’s a fantastically useful tool, and for most of what people do today it’s exactly the right one. But calling it an “agent” is like calling a power drill a carpenter. The drill is doing none of the deciding.

What makes something an agent

Anthropic, in its widely-cited guide on building agents, settled on a refreshingly simple definition: an agent is an LLM autonomously using tools in a loop. Three words carry the weight — autonomously, tools, loop.

Tools mean the model can do more than talk: it can take actions in the world — query a database, run code, call an API, send a message. Loop means it doesn’t stop after one response; it acts, observes the result, and decides the next action, over and over. Autonomously means it makes those decisions, dynamically, rather than following steps a human pre-wrote. The building block underneath is an LLM augmented with tools, memory, and retrieval — but the thing that makes it an agent is that it’s been handed the wheel.

This is what I run at home with Hermes. I don’t prompt it step by step. I give it an objective; it plans, uses its tools, works across a long horizon, and comes back when it’s done. Some days it runs for a long time on a single instruction. That “long time, little instruction” quality is the tell. If you’re typing every step, you don’t have an agent. You have a chatbot and a very busy human.

Most things sold as “agents” are workflows

There’s a middle category that muddies the water, and honesty about it matters. Between the chatbot and the true agent sits the workflow: an LLM wired into a predefined sequence of steps that a developer wrote in code. Step one calls the model to classify an email, step two routes it, step three drafts a reply. It feels autonomous because no human is in the loop, but the path is fixed. The model isn’t deciding the route; it’s filling in blanks along a track someone laid down in advance.

Anthropic draws exactly this line: workflows orchestrate models through predetermined code paths; agents let the model dynamically direct its own process. Both are useful — workflows are often the better, safer choice for well-defined tasks. But they are not the same thing, and a lot of what’s marketed as “agentic AI” today is a workflow with good branding. Knowing which one you have is the difference between accurate expectations and an expensive surprise.

Why the distinction actually matters

This isn’t pedantry. Where a system sits on the chatbot → workflow → agent spectrum determines three things you can’t afford to get wrong.

Capability. A chatbot can only help you think. A true agent can complete multi-step work end to end. Promise a board “agents” and deliver a chatbot, and the gap is brutal.

Failure modes. A chatbot’s mistake is a wrong answer you can see and ignore. An agent’s mistake compounds: it acts on a bad decision, observes a distorted result, and decides the next step on top of the error. Over a long-running loop, small mistakes can cascade into large, real-world ones — emails sent, records changed, money moved. Autonomy multiplies both value and risk.

Governance. You can review every answer a chatbot gives because you’re there for each turn. You cannot review every step of an agent that runs autonomously for an hour. So governing an agent is a different discipline entirely: setting boundaries, limiting tool access, defining the blast radius, deciding where it must check in. (Which is exactly why “human in the loop” needs to evolve into something more precise — a topic for its own piece.)

What this means

The next time someone says “we’re deploying an AI agent,” treat it as the start of a conversation, not the end. Ask the questions that actually separate the categories:

Does it act without you, or does it wait for your next prompt? If you drive every step, it’s a chatbot — useful, but not an agent.

Does it decide its own steps, or follow a fixed script? If the path is pre-written in code, it’s a workflow. Fine — often preferable — but be honest about it, because it can’t handle the open-ended work a true agent can.

Over what time horizon, with what tools, and what’s the blast radius? This is the real risk question. The more autonomy and the more powerful the tools, the more your attention belongs on boundaries rather than capabilities.

We are, overwhelmingly, still in the chatbot era — operating extraordinarily capable models one turn at a time. The agent era is coming and partly here, but it’s narrower and earlier than the marketing suggests. Getting the words right is the first step to getting the expectations, the risks, and the governance right. A chatbot answers. An agent acts. Knowing which one you have is not a detail. It’s the whole game.

References

  • Anthropic, Building Effective Agents (December 2024) — agents as “LLMs autonomously using tools in a loop”; the workflow-vs-agent distinction (predefined code paths vs dynamic self-direction); the augmented-LLM building block (tools, memory, retrieval).
  • keller-ai — Hermes, the Agent.