AI agents & tool use

1. An agent is not just a longer prompt

A useful convention distinguishes workflows, whose paths are largely specified in code, from agents, where a model chooses more of the path and tool sequence. Real applications combine both. This is a practical vocabulary, not a universally agreed boundary. Anthropic’s workflow–agent distinction.

“Agentic” describes where discretion sits. It does not establish competence, unrestricted access or a particular model architecture. For this guide, an agent can be small and tightly constrained: even an adaptive research loop can qualify without running continuously or controlling a computer.

The practical design question is where flexibility helps. A known sequence can remain a workflow; uncertain intermediate steps may justify model-directed choices. Start with the simplest arrangement that meets the task. Anthropic’s implementation guidance.

2. Inside the execution loop

ReAct demonstrated a pattern of interleaving reasoning, actions and observations. An observation can change the next step: a search result may expose a missing fact; an error may require a different request. Reasoning text changes the model’s context, while an executed action can change its environment. The ReAct paper.

The surrounding software is often called a harness or runtime. It supplies tool definitions, invokes services and returns observations. The model does not send an email merely by generating the words “email sent.” A tool must run, and its result must be interpreted. Toolformer’s inference procedure makes this separation explicit: generation pauses for execution, then resumes with the returned result. Toolformer’s execution mechanism.

For readers evaluating a system, the important trace is the observable record: proposed calls, approved actions, returned evidence and unresolved errors. Fluent intermediate commentary is not a substitute for checking those records.

3. Planning is useful only if the plan can change

In a useful plan, steps have a purpose and a completion test. “Investigate the issue” is vague. “Find the affected version, reproduce the failure, propose a patch and run the relevant tests” exposes dependencies. This is an illustrative decomposition: the appropriate tests and stopping rules depend on the task.

Planning can happen incrementally. ReAct’s model uses observations to update its approach rather than committing to an uninterrupted sequence before seeing the environment. That flexibility is valuable when a tool returns an unexpected result. It also creates a need for explicit stopping conditions. ReAct’s reasoning–action formulation.

Three different ways behaviour can improve

Training: Toolformer sampled candidate API calls, executed them, retained useful examples according to a language-modelling loss criterion and fine-tuned the model. This teaches tool-use behaviour through weight updates. The original study did not solve arbitrary interactive, chained tool use; its training calls were independent. Toolformer’s method and limitations.

Feedback in context: Reflexion stores text-based reflections from previous attempts and uses them on later trials without updating model weights. Its evaluators can include tests, heuristics or models. A faulty test or mistaken reflection can therefore misdirect the next attempt; “self-reflection” is not self-certification. Reflexion’s architecture and limitations.

Task organisation: an orchestrator can assign distinct questions to workers and combine their results. This differs from a fixed parallel workflow when the subtasks are chosen dynamically. Multiple agents are a coordination pattern, not an automatic quality improvement. Anthropic’s orchestrator–workers pattern.

For a research briefing, independent evidence checks may help. Having several workers repeat the same search may merely duplicate effort. Treat that comparison as a design judgement to test, not a universal rule that more agents are better.

4. A tool call is a proposal with a contract

A tool definition describes an operation and its inputs. The model selects a tool and supplies arguments; the application handles execution and returns a result or error. The Model Context Protocol (MCP) standardises parts of this interaction, including tool discovery, input schemas and optional output schemas. It is an integration protocol, not a model or a safety certificate. MCP’s tools specification, version 2025-11-25.

MCP requires server-side input validation and appropriate access controls. It also warns that tool annotations should be treated as untrusted unless they come from trusted servers. A tool describing itself as harmless is not independent evidence that it is. MCP validation and security considerations.

The interface affects performance

SWE-agent studied an agent–computer interface designed around clear actions, compact observations and useful feedback. Its editing tool could reject selected syntax or lint failures and return diagnostic information. Such a guard catches a class of bad edits; it does not establish that the program behaves correctly. SWE-agent’s interface design.

This suggests a practical distinction: improving a model and improving the interface it uses are separate levers. A narrower operation with a precise result can make the task easier to inspect than a broad instruction whose effects are difficult to establish.

5. Retrieval supplies evidence, not guaranteed truth

Retrieval-augmented generation connects a generator to an external information store. The original RAG research combined learned generation with retrieved Wikipedia passages; it also investigated changing the retrieval index without retraining the generator. Its particular probabilistic formulation is not the definition of every modern retrieval pipeline. Lewis et al.’s RAG paper.

A useful mental model is: formulate a query, find candidates, select evidence and generate an answer conditioned on it. A vector database is one possible component, not a prerequisite for searching documents.

These are separate choices: combining rankings is not the same operation as running a learned semantic reranker. And a reranker cannot rescue a document that never entered its candidate set. Elastic’s ranking and reranking documentation; the candidate-set limit follows from that pipeline.

When retrieval becomes adaptive

An agent can search again when evidence is missing, follow a reference or compare conflicting documents. Self-RAG explores a trained version of adaptive retrieval and critique using special tokens to judge retrieval need and aspects of the resulting passages and answer. Those learned judgements are still model outputs, not proofs of factual correctness. Self-RAG’s method.

Our editorial recommendation is to preserve enough provenance to inspect a claim: the original passage, document identity, date or version, and the reason it supports the answer. Similarity establishes a retrieval relationship; it does not by itself establish that a source is current, authoritative or consistent with another source.

6. Storing information is not the same as remembering reliably

“Memory” can refer to information in the current prompt, state persisted by the application or knowledge encoded in trained weights. These are not interchangeable. A database write can affect a later conversation without performing a model-training update.

The terminology describes stored content, not a particular storage engine. Semantic memory does not mean semantic search, and changing a playbook need not change model weights. LangChain’s memory overview. The questions in the table are our design checklist.

Context, persistence and forgetting

MemGPT separates limited prompt space from external recall and archival storage. Its queue manager can remove older messages from the prompt while retaining them in recall storage and keeping a recursive summary in context. Evicted from context therefore does not mean deleted from storage. This architecture makes additional information accessible; it does not promise infinite, perfectly reliable recall. MemGPT’s memory hierarchy and queue manager.

Compression introduces a trade-off. LongMemEval found that replacing stored conversations with extracted summaries or facts could lose detail and harm overall question answering in its tested setups; fact decomposition helped its multi-session reasoning category. The lesson is conditional, not “never summarise.” Its experiments separate indexing, retrieval and reading because failure at any stage can break the answer. LongMemEval’s memory-design experiments.

Generative Agents used relevance, recency and importance to retrieve remembered observations and built reflections linked to supporting memories. Its evaluation also documented missed or embellished recollections. The study concerned simulated social behaviour, not proof that a production assistant’s memory is trustworthy. Generative Agents’ memory and evaluation sections.

For an implementation, we would distinguish not currently loaded, not retrieved, superseded and deleted. They imply different remedies. A shorter prompt alone does not demonstrate deletion from a durable store, and an old preference should not silently overrule a newer instruction.

7. Permissions belong outside the model’s persuasion loop

There are several boundaries between “connected” and “allowed.” Authentication establishes an identity; authorisation constrains its access; task-specific approval permits a particular consequential action. A connection to a mail service should not be interpreted as blanket permission to send any message.

MCP’s HTTP authorisation specification uses access tokens and scopes, including resource-specific token validation. Its scope guidance favours the minimum required access. These provisions are not automatic properties of every MCP connection: authorisation is optional in the protocol, and local stdio integrations have different arrangements. MCP authorisation, version 2025-11-25.

Approval should bind to the action being reviewed: the actor, tool, target and normalised parameters, with an appropriate expiry. Changing a recipient or payload after approval changes what was authorised. When a required policy or approval check is unavailable, consequential actions should fail closed. OWASP’s approval and enforcement recommendations.

In our briefing example, access to approved research documents, permission to create a draft and approval to send that specific draft to specified recipients are three different grants. Remembering that the user “likes weekly updates” supplies none of them automatically.

8. Read-only tools can still leak information

Indirect prompt injection occurs when instructions embedded in lower-trust material—such as a retrieved page or message—attempt to redirect an assistant. The material is supposed to be evidence or task data, not a new source of authority. Greshake and colleagues demonstrated this problem in synthetic applications and then-existing integrations. The original indirect prompt-injection study.

Crucially, a nominal read operation can transmit information. A search query or requested URL carries arguments outside the system. Blocking database writes therefore does not, by itself, prevent disclosure through outbound requests. This is a demonstrated attack class, not a claim that every current tool is vulnerable in the same way. The study’s information-gathering analysis.

Why one defence is not enough

AgentDojo evaluates attacks and defences in simulated tool-using environments. Its analysis shows limits to filtering available tools: a tool necessary for the legitimate task may also enable an attacker’s objective. Attacks can also contaminate an answer without causing a forbidden write. These benchmark cases are controlled evaluations, not a census of real-world incidents. AgentDojo’s defence analysis and limitations.

MCP’s security guidance addresses additional system boundaries, including restricted filesystem and network access for local servers, visible consent before one-click configuration executes commands, and rejection of token passthrough. Installing a local tool server can itself execute software; this is distinct from asking a model to use an already configured tool. MCP security best practices.

Our synthesis is to check both capability and information flow: which operations are reachable, which data they may access, which destinations may receive it, and which outputs require review. Clear instructions help express intent, but they are not a replacement for those enforceable boundaries.

9. Autonomy needs a budget and a stop rule

“Keep trying until you succeed” leaves important decisions unspecified. For a concrete design, set bounds on elapsed time, tool calls, model usage, retries and consequential actions. Decide which uncertainties require a human and which incomplete outcomes are acceptable. These are application design choices, not a universal set of magic thresholds.

We would separate three kinds of stopping condition:

• Completion: the requested result has passed its defined checks.
• Pause: further progress needs evidence, approval or a user choice.
• Termination: a safety boundary or hard resource limit has been reached.

A useful handoff explains what is complete, what is uncertain, which external actions occurred and what decision is needed. “I got stuck” hides the state the next person needs.

Human oversight must be usable

Not every harmless intermediate step needs an identical warning. Excessive approval prompts risk habituation, while vague prompts can conceal the actual consequence. CoSAI’s MCP security report discusses approval fatigue and favours constrained, purpose-built operations and meaningful control boundaries. This is design guidance in a January 2026 draft, not experimental proof of a universally optimal approval interface. CoSAI’s draft MCP security report.

For example, reviewing a named draft and its recipients offers more useful information than approving “continue work.” The right threshold depends on the task’s consequences and the authority already granted. A lower-friction interface should not silently broaden that authority.

Retries also consume shared capacity. Distributed-systems guidance warns that retries can amplify overload, especially when several layers retry independently. An agent’s willingness to continue does not make unlimited attempts operationally sound. AWS on retry amplification.

10. Recovery means more than trying again

A failure message describes an observation, not necessarily the complete external state. Recovery should depend on what is known about the request, the service contract and any effects already produced.

This classification combines AWS’s retry guidance, Azure’s compensation pattern and the permission boundaries above. Backoff and jitter spread retries; they do not make an unsafe write safe to repeat.

Idempotency addresses duplicate effects

An idempotency key lets a service recognise repeated submissions of the same intended operation. Safe handling depends on the server’s contract: recording the request identity and applying the effect must be coordinated. A client-side log alone cannot guarantee deduplication across a crash. The key should represent the same intent; identical parameters might also describe two legitimate, separate requests. AWS on idempotent APIs.

Checkpoints do not rewind the outside world

A checkpoint records application execution state. LangGraph’s checkpointers support resuming saved threads; explicit replay from an earlier checkpoint can re-execute later nodes, including API requests. Restoring state is therefore not a guarantee against duplicated external effects. LangGraph’s checkpoint and replay semantics.

Compensation is a separate, application-specific action intended to address completed work. It need not restore an identical earlier world, especially when other users have made changes. Compensation can itself fail, and some actions cannot be reversed. A robust process records progress and has an escalation path. Azure’s compensating transaction pattern.

In short: retry repeats a request; reconcile establishes its actual status; resume continues saved work; compensate performs an allowed corrective action. Choosing between them is part of the system’s design.

11. Measure reliable completion, not persuasive activity

A useful evaluation asks whether the requested outcome was achieved, whether the process respected its constraints and what it cost. Tool-call count or an impressive transcript cannot answer all three.

τ-bench evaluates conversations with simulated users, domain tools and policies, including checks against the final database state. Its paper explicitly notes that a matching database outcome can be necessary but insufficient: a refund could reach the expected state without obtaining required approval. This is why outcome checks and policy checks should be separated. τ-bench’s evaluation design and caveat.

Test the failure paths deliberately

Our recommended evaluation set includes missing evidence, conflicting dates, stale memory, malformed tool results, denied permissions, timeouts after writes and interrupted runs. Report unsuccessful and escalated cases alongside successful ones. A system that pauses correctly should not be confused with one that silently invents completion.

Separate subsystem checks are informative. LongMemEval distinguishes retrieving the relevant memory from correctly using it in an answer. SWE-agent distinguishes an editing interface’s guards from the task’s functional tests. An apparent success in an intermediate component should not be promoted into end-to-end correctness. LongMemEval’s evaluation stages; SWE-agent’s interface and task evaluation.

Finally, record the model, prompts, tool versions, permissions, task distribution and resource limits. Otherwise a change in measured performance may reflect a different environment rather than a better agent. Historical benchmark results remain evidence about their reported setups, not current product rankings.

12. Put the pieces together: a briefing agent

Hypothetical task: “Prepare a weekly product briefing from approved documents. Create a draft. Send it to the named team only after I approve the exact version.” This is an explanatory design, not a deployed system or an additional action being taken on this website.

1. Define the boundary. Record the permitted document collection, draft destination, recipient list and stopping budget. Sending remains gated.
2. Plan the evidence work. Identify the questions, search the approved collection and investigate material gaps. Preserve document dates and source passages.
3. Use memory selectively. Retrieve the user’s current formatting preference. Keep it distinct from source evidence and from permission to distribute.
4. Create and verify the draft. Check factual support, save the artifact and inspect the actual saved result. A plausible model response is not the saved document.
5. Request the specified approval. Show the exact draft and recipients. If either changes, reassess approval rather than reusing it blindly.
6. Execute and reconcile. After authorised sending, establish the service’s actual result. If delivery status is uncertain, follow its reconciliation contract or hand off; do not assume another send is harmless.

This example combines the permission, retrieval and recovery principles above. Its central discipline is to keep a visible distinction between planned, authorised, attempted, confirmed and unresolved work.

The questions worth asking of any agent system

What can it observe? What can it change? What leaves the system? Who can grant additional authority? What counts as verified completion? What persists after interruption? And what happens when the external state is uncertain?

These questions expose the system behind the demo. Model capability matters, but so do the tools, evidence, state management and control boundaries through which that capability reaches the world.

Glossary

Sources and research notes

This guide synthesises original studies, official protocol specifications and first-party engineering documentation, accessed on 6 September 2026. Descriptive links sit beside the relevant claims. The catalogue records the versions reviewed; living documentation may subsequently change.

Research results are bounded by their datasets, models and environments. Simulated-agent benchmarks are not field incident rates, and older experiments are not present-day leaderboards. “Agent” has competing definitions; this guide uses the practical workflow–agent distinction stated at the start. CoSAI’s cited report is a draft.

The diagrams, checklists and hypothetical examples are explanatory synthesis, not measurements, production-ready security designs or a claim that any particular proprietary system follows this exact architecture.