What is an AI agent? How it works, with an office example

Say an enquiry holds a postcode. A fixed workflow can look it up and send it to the right team. When the route is clear, that is the simple answer. Adding a model only adds one more decision to check.

Now say the enquiry has a long story, 4 attachments and an unclear request. A model can help read it. You can still keep it inside a fixed workflow: sort it, summarise it and pass it to a person.

An agent earns its place when the next step depends on what it finds. It might open a policy, see that it points to a schedule, read that schedule, then ask for a missing detail. That freedom also means more paths to test.

Anthropic's guide to workflows and agents splits fixed steps from model-directed action. Treat that as a design idea, not a rule every vendor's labels follow.

A large language model is software trained on patterns in text. Inside an agent, it reads a request, plans, picks a tool and drafts a reply. The app around it decides which of those actions actually run.

Say the model asks for a calendar lookup. The app checks that the lookup is allowed and returns the result. A smooth answer and a real lookup are 2 different pieces of proof. Check what the model said apart from what the system confirmed.

Tools are the actions the app lets the model take. A tool might search a folder, open a customer record or save an email draft. Each tool takes an input, such as a booking ID, and returns a result, such as 3 free times.

A narrow tool is easier to control. "Create a draft in the enquiries inbox" has a clear limit. Full access to a computer opens far more actions. Match access to the one job you want done.

An agent can read the current message, chosen files and what its tools return. That working information is called context. Old context gives old answers, even when the agent follows its rules.

Put a date beside each key rule. If 2 files disagree, give the agent a route to a reviewer. "These 2 files conflict" is a more useful result than a confident reply that hides the clash.

Memory is information kept for later. It might be a list of finished steps, a chat summary or a saved preference. Many agents work inside one session only; lasting memory is a design choice.

Decide who can read, fix and delete what it keeps. A note that says "client prefers Tuesday" can go stale. A date and a source help a reviewer decide whether to trust it.

Guardrails are limits on what the agent can do. Examples: only 2 folders it can read, approval before any send, and a stop after 10 steps. Instructions explain the rules; enforced permissions decide what the software can really do.

For a first trial, split making work from releasing it. Let the agent create a draft. Name one person who checks it. Early mistakes then stay inside the office.

A good test case has 3 parts written before the run: the input, the expected output and the reason. For example: a fictional email asking to move a meeting to a day the office is closed. Expected output: a draft that offers 2 open days instead. Reason: it checks that the agent reads the opening hours.

Write one case for each of the 5 kinds in the trial diagram. Keep them in a shared document, never only inside the tool, so you can re-run them after any change to the instructions, the model or the connected systems.

When a real job goes wrong, turn it into a new fictional test case.

Leave out any task where a wrong answer is costly and hard to spot. Advice on tax, law or health falls here, as do payments, refunds and anything that commits the business to a price.

Also wait on tasks with no clear owner. If 3 people each assume another will check the draft, nobody does. Give each task one named owner before it goes near an agent.

Finally, skip tasks that happen once a month. You will not see enough runs to judge the result, and the setup time will rarely pay back. A task that comes up 10 times a week gives you real evidence within 2 weeks.

A useful test: could a new staff member check the agent's output in under 5 minutes with the documents you already have? If the answer is no, the task needs a clearer source or a narrower output before any software is involved. Some failures are plain, like a tool error; others look fine until someone checks the record.

Ask for proof with each answer. For a policy question, that means the policy title and section. For a booking, it means the real calendar entry. A link that opens is only the start: a reviewer still checks that the source backs the claim.

Use test cases with known answers. Include one old file, one unclear name and one missing attachment. Write the expected result before each run, so a polished reply cannot quietly change what counts as success.

Prompt injection is text that tries to make an AI follow orders hidden in what it reads. An email or web page might tell the agent to drop its task or reveal data. That text is input to check, never an order that changes its access.

In a trial, put one obvious hostile line in a fictional email. Check that the agent stays inside its task. A warning in the prompt is one layer; tight tools and a person approving key actions add 2 more.

Decide what the agent does when email, a calendar or a file system is down. A good failure names the step that failed and keeps finished work safe. It should tell "no matching record" apart from "the lookup did not run".

Also test a repeat. If an agent saves a booking and then drops its connection, a full retry can make a duplicate. A task reference and a check of the saved record stop that.

We ran a controlled check of an editing workflow on 16 October 2026. In an isolated copy of a 20-page website, we left an old section address in 20 links and an old guide address in 2 homepage links. The pages still loaded, but those 22 links went nowhere.

The article SEO check passed all 17 articles in that copy. A separate link check then found the 22 broken links. The 2 checks measured different things: page structure on one side, real link targets on the other.

This is a real test, unlike the fictional office examples above. Its lesson is narrow: a passed check proves only what that check looked at. An agent's edit still needs its own target and click checks before the task is accepted.

Use one row per job and 6 columns: the date, the task reference, the result (accepted, fixed or failed), the minutes a person spent checking it, the usage charge if the tool shows one, and a short note on what went wrong.

At the end of each week, add up 3 numbers: accepted jobs, total review minutes and total charges. Divide the charges and the minutes by the accepted jobs. That gives you a cost per useful result you can compare with doing the job by hand.

Keep failed jobs on the sheet. They cost time even when they produce nothing, and a rising failure count is often the first sign that a connected system or a source document has changed. Check the sheet every Friday for the first 4 weeks, then monthly once the numbers settle. Share the totals with whoever approved the trial, so the decision to continue rests on the same numbers.

Over a few months, your test cases and this sheet become the best record of what the agent can and cannot do.

Ask the supplier to run your task with fictional data. Watch one normal run and one failed run. Ask to see the activity log, the approval step and where an admin removes access.

Ask for the data path and the charges in writing. Do logs keep message text? Does the model provider get attachments? Can admins set a usage cap? Keep the answers beside the contract.

Ask who owns it after the demo. Someone must handle changed systems, expired access, new policies and error reports. A working demo is useful proof, but running it every week is part of the buying decision.