
On this page
- What is an AI agent, and how is it different from automation?
- What can AI agents actually do for a small business today?
- What can AI agents not do reliably yet?
- Where do agents go wrong in a small business?
- What do the consent rules say when an agent contacts customers?
- Workflow, agent or person: how do you choose?
- How do you try an agent without risking the business?
- What should you be skeptical of when buying an agent?
- Frequently asked questions
- Sources
Most pages on this topic list ten things AI agents can do and stop there. The more useful question for an owner is narrower: which of your jobs can an agent do reliably today, which can it not, and what should still have a person's name on it? This guide answers that with the published evidence, including the parts vendors rarely quote.
What is an AI agent, and how is it different from automation?
An AI agent is a system where an AI model chooses its own steps and tools to reach a goal, rather than following a script. Anthropic's engineering guide separates two designs. In workflows, "LLMs and tools are orchestrated through predefined code paths". In agents, "LLMs dynamically direct their own processes and tool usage, maintaining control over how they accomplish tasks."
That difference decides almost everything else in this guide:
| Fixed automation (workflow) | AI agent | |
|---|---|---|
| Who chooses the steps | You, in advance | The model, as it goes |
| Same input, same result | Yes | Not always |
| Handles open-ended input | Poorly | Better |
| Easy to test | Yes | Harder |
| Cost and speed | Low, fast | Higher, slower |
The last row follows Anthropic's own advice: agentic systems "trade latency and cost for better task performance", and the guide recommends starting with the simplest option because an agent may not be needed at all. The label is also used loosely. A website chat window, a lead-reply sequence and a genuine agent are three different products, and our guides to what an AI chatbot costs and automating lead follow-up cover the first two.
What can AI agents actually do for a small business today?
Agents are dependable when the job is narrow, the inputs are predictable and success can be checked. The research points to four kinds of work that fit:
- Executing a defined workflow. In the CRMArena-Pro benchmark (Salesforce researchers, 2025), "Workflow Execution" was the most tractable skill, with top agents above 83% success on single-turn tasks.
- Sorting and extracting. Reading an enquiry, tagging its type and copying the details into a record.
- Drafting for approval. First replies, summaries of a call or a thread, with a person sending.
- Booking inside fixed rules. Offering free slots from a calendar you control, as an AI receptionist does for calls.
None of those requires the agent to improvise, which is why they work. If you are unsure where to begin, what a small business should automate first gives a scoring method that applies equally to agents.
What can AI agents not do reliably yet?
Open-ended, multi-step work with consequences is where agents still fail often. Three pieces of primary evidence show it, and each comes with its own caveat.
Realistic office tasks: about a third completed
TheAgentCompany (Xu and 20 co-authors, first posted December 2024, revised September 2025) simulates a small software company with internal websites and data. Agents work like a digital employee: browsing, writing code, running programs and messaging coworkers. The authors report that "the most competitive agent can complete 30% of tasks autonomously."
Back-and-forth with customers: success falls
CRMArena-Pro (Huang and eight co-authors, May 2025) tests agents on business scenarios such as sales and customer service. Leading agents reached about 58% success on single-turn tasks, which dropped to roughly 35% when the task needed several turns, as a real customer conversation does. The paper also reports "near-zero inherent confidentiality awareness": left alone, an agent does not know which information it should not share. Prompting for it helped but often cost task performance.
A real small shop: it lost money
Anthropic and Andon Labs let Claude Sonnet 3.7 run a small shop in Anthropic's San Francisco office for about a month (Project Vend, 2025). The agent chose stock, set prices and answered customers on Slack. It sold items below cost, was talked into discounts and giveaways, turned down a very profitable offer for a drink that costs far less online, and for a time gave customers a payment account it had made up. Anthropic's conclusion was plain: "If Anthropic were deciding today to expand into the in-office vending market, we would not hire Claudius." The same write-up argues many of the failures could be fixed with better instructions and tools, and that the experiment showed some real strengths, such as finding suppliers for unusual requests.
How to read these results honestly
These are snapshots. The models tested were the ones available when each study ran, and newer ones may score higher. Benchmarks are also not your business: your tasks may be easier, or your data messier. What the results do establish is the pattern. Agents are strongest when following a defined process and weakest when they must use judgment across many steps or hold a line under pressure, which is what a customer asking for a discount does.
Where do agents go wrong in a small business?
The failures in the research map onto familiar small-business risks:
- Confident wrong actions. An invented account number, a price below cost. A person reviewing outputs catches these; a fully autonomous agent does not.
- Being persuaded. Project Vend's agent gave way to requests for discounts. An agent that can issue refunds or credits needs a hard limit set outside the model, not a polite instruction.
- Leaking information. With near-zero confidentiality awareness, an agent that can read your whole inbox may repeat what it should not. Give it access to the data the task needs and nothing else.
- Errors that compound. Anthropic warns that costs rise with autonomy and errors can compound across steps, and recommends testing in a sandbox with guardrails.
- Messaging without consent. Covered next.
What do the consent rules say when an agent contacts customers?
Treat an agent that calls or texts like any other marketing channel. In the US, 47 CFR 64.1200(a)(2) bars telemarketing calls made "using an automatic telephone dialing system or an artificial or prerecorded voice" unless the call is made with "the prior express written consent of the called party". The rule's text does not exempt a call because an AI placed it, so a voice agent dialing leads needs that written consent first. Carriers expect registration for business texting, and WhatsApp expects opt-in. UK and EU rules differ again; the GoHighLevel setup checklist walks through the registration steps, and our lead follow-up guide sets out the consent rules by market. This is general information, not legal advice.
Workflow, agent or person: how do you choose?
Choose by the shape of the input and the cost of a mistake. Use this table for each task you are considering:
| If the task is... | Use | Example |
|---|---|---|
| Same steps every time, data already digital | A fixed workflow | New enquiry creates a record and sends an acknowledgement |
| Input varies in wording, but the action set is small | A workflow with an AI step | Classify an enquiry, then route it by fixed rules |
| Open-ended, many possible steps, low harm if wrong | An agent, with logging | Researching a supplier and drafting a comparison |
| Money, a promise, a complaint or a decision about a person | A person, assisted by AI drafts | Refunds, quotes, disputes, hiring |
Most small-business wins sit in the first two rows. A tool such as n8n, Make or Zapier handles them; n8n vs Make vs Zapier compares how each bills. Reserve the agent row for the tasks where a fixed workflow genuinely cannot cope.
How do you try an agent without risking the business?
Start small, in suggest-only mode, and let evidence widen its permissions:
- Pick one task your team repeats often, with a clear test of a correct result.
- Run it in draft mode. The agent proposes; a person approves. Do this for several weeks.
- Log every correction. Count how often a person changed the output and why.
- Set limits outside the model. Spending caps, a list of actions it may take, and no access to data it does not need.
- Widen permissions only when corrections are rare, and keep approval on anything involving money, promises or customer-facing messages.
- Name an owner who reads the log weekly. Unowned automations fail quietly.
If you would rather have the task mapped and the guardrails built for you, our AI and automation service starts by mapping the job before quoting; 1920 Agency is based in Islamabad and quotes a fixed price after that mapping. For market rates and what to ask before you buy, see what an automation agency charges.
What should you be skeptical of when buying an agent?
Be skeptical of any claim that arrives without a test you can repeat. A few checks help:
- Ask what the agent is allowed to do on its own. If the answer is "anything", ask for the approval and spending controls.
- Ask for the failure rate on tasks like yours, not an average across customers. A vendor that cannot show one has not measured it.
- Treat unsourced percentages as marketing. Pages that quote a figure for time saved or customers helped without naming the study or the sample give you nothing to verify.
- Check the label. If the product follows fixed steps, it is a workflow, which may be exactly what you need and should cost accordingly.
Frequently asked questions
What can AI agents do for a small business today?
They do best at narrow jobs with clear success tests: sorting and tagging enquiries, drafting replies for a person to approve, pulling details from emails or documents into a record, and executing a defined workflow. Benchmark results are strongest for that last category and weaker for open-ended, multi-step work.
What is the difference between an AI agent and a chatbot?
A chatbot answers in a conversation. An agent is given a goal and tools, such as your calendar or CRM, and decides which steps to take. Anthropic describes agents as systems where the model dynamically directs its own process and tool use. Many products sold as agents are chatbots or fixed automations with an AI step.
Is an AI agent the same as an automation?
No. An automation (a workflow) follows steps you defined in advance, so it behaves the same way each time. An agent chooses its own steps, which suits open-ended input but makes behavior harder to predict and test. For a known process, a workflow is usually cheaper and safer.
Can an AI agent replace an employee?
The evidence does not support that yet. In the TheAgentCompany benchmark, which simulates a small software company, the most competitive agent completed 30% of tasks autonomously. Agents can take over parts of a role, such as first drafts and data entry, with a person responsible for the outcome.
What are the risks of using an AI agent?
Wrong actions taken confidently, invented details, leaking information the agent should not share, being talked into discounts, and sending messages without valid consent. The research cited below documents each of these in some form. Limit what the agent can do, log its actions and require approval for anything that spends money or contacts a customer.
Do I need consent before an AI agent calls or texts customers?
Usually yes for marketing. US federal rules at 47 CFR 64.1200(a)(2) bar telemarketing calls made with an autodialer or an artificial or prerecorded voice unless the called party gave prior express written consent. UK, EU and messaging-app rules differ, so check yours.
How should a small business start with AI agents?
Pick one repeated task with a clear success test, run the agent in suggest-only mode for a few weeks, measure how often a person had to correct it, then widen its permissions only if the corrections are rare. Start with the task your team repeats most.
Sources
- Building effective agents, Anthropic, checked 12 November 2026
- TheAgentCompany: Benchmarking LLM Agents on Consequential Real World Tasks, Xu et al., arXiv, checked 12 November 2026
- CRMArena-Pro: Holistic Assessment of LLM Agents Across Diverse Business Scenarios and Interactions, Huang et al., arXiv, checked 12 November 2026
- Project Vend: Can Claude run a small shop?, Anthropic, checked 12 November 2026
- 47 CFR 64.1200, Delivery restrictions, Legal Information Institute, checked 12 November 2026



