Multi-Agent Systems Explained: Architecture, Patterns and When to Use Them

A multi-agent system (MAS) is a setup where several autonomous AI agents work together - or compete - to complete tasks that would be brittle, slow, or impossible for a single agent. Each agent has its own role, tools, memory, and decision loop. They coordinate through messages, shared state, or an orchestrator.

The idea is not new. Multi-agent systems have been studied in distributed AI since the 1980s, with foundational work by Wooldridge, Jennings and others on agent communication languages and coordination protocols. What changed in 2023-2024 is that large language models became capable enough to act as the reasoning core of an agent, which made the architecture practical for general business problems rather than narrow research benchmarks.

This article explains what a multi-agent system actually is, the architectural patterns you will encounter, where they earn their keep over a single-agent or pipeline approach, and what to watch for if you are considering one in production.

What a multi-agent system actually is

An agent, in the modern LLM sense, is a loop: a model receives a goal, decides on an action (call a tool, ask a question, write a file, hand off to another agent), observes the result, and decides again. A single agent runs that loop until it thinks the goal is met or it gives up. A multi-agent system runs several of those loops in parallel or in sequence, with some mechanism for coordination.

The minimum components of a useful MAS:

• Agents with distinct roles, prompts, and tool access. A "researcher" agent might have web search and a vector store. A "writer" agent might have a document editor and a style guide. A "critic" agent might have only the output and an evaluation rubric.
• A communication channel. Usually structured messages, sometimes a shared scratchpad or blackboard, sometimes a typed event bus.
• An orchestration layer that decides which agent runs when. This can be hard-coded (a graph), model-driven (a supervisor agent routes work), or emergent (agents call each other directly).
• Shared memory or state. Conversation history, intermediate artefacts, retrieved documents. Anthropic, OpenAI and the LangChain team all stress that state management is where most multi-agent systems fail in production.
• Termination conditions. Without these, agents loop forever or burn through token budgets.

If you strip those down, a multi-agent system is just a structured way of decomposing a problem so different specialised reasoning processes can work on different parts of it. The complexity comes from coordination, not from the agents themselves.

The main architectural patterns

There are four patterns you will see in real systems. Most production deployments combine two or three.

Supervisor / orchestrator-worker

One agent acts as the supervisor. It receives the original task, breaks it into subtasks, delegates each subtask to a specialised worker agent, and assembles the results. The workers do not talk to each other directly - the supervisor mediates.

This is the pattern Anthropic described in its June 2025 engineering post on building Claude's research feature, and the one most teams should start with. It is easy to debug because the supervisor's reasoning trace is a single thread. It scales by adding more worker types. It fails predictably - if the supervisor mis-plans, the whole run is wrong, but you can see why.

Hierarchical / nested teams

An extension of supervisor-worker where workers can themselves be supervisors of sub-teams. A top-level research supervisor delegates to a market-research supervisor and a technical-research supervisor, each of which has its own pool of workers.

This is how you handle genuinely large problems - a 200-page due-diligence report, a multi-week migration plan - without one supervisor losing the plot in a context window. The cost is coordination overhead and harder debugging.

Network / peer-to-peer

Agents talk to each other directly with no central orchestrator. Each agent decides who to hand off to based on its own reasoning. This is closer to the classical multi-agent systems literature, and to frameworks like AutoGen and CrewAI in their less-structured modes.

It is flexible and can produce emergent problem-solving behaviour. It is also much harder to predict, debug, and bound. Most production teams avoid pure peer-to-peer for anything customer-facing.

Sequential pipeline with critique

Not quite full multi-agent, but worth naming because it is the highest ROI pattern for many use cases. A producer agent generates output, a critic agent evaluates it against a rubric, and a revisor agent applies the feedback. You can chain several critic-revisor cycles before a final output.

This pattern - sometimes called "reflection" or "self-refine" after the 2023 paper by Madaan et al. - reliably improves output quality on tasks with clear success criteria (code, structured documents, classification). It is cheap to implement, easy to evaluate, and rarely goes off the rails.

When a multi-agent system beats a single agent

Multi-agent systems are not free. They cost more tokens, take longer to run, are harder to evaluate, and introduce new failure modes (agents disagreeing, infinite loops, context fragmentation). You should only reach for one when the problem genuinely requires it.

Anthropic's own data on its multi-agent research system is instructive: it used roughly 15x more tokens than a single Claude chat for the same query, but produced materially better results on open-ended research tasks. That trade is worth it for high-value, low-frequency work. It is not worth it for a customer support reply.

The cases where multi-agent earns its cost:

• Genuinely parallel subtasks. Researching ten competitors, summarising fifty documents, checking a contract against twenty policy clauses. A single agent does these in sequence and runs out of context. Multiple agents do them in parallel and merge.
• Distinct skills with different tool requirements. A SQL-writing agent needs database access and a schema. A chart-making agent needs a plotting library. Forcing one agent to hold both contexts plus the user goal degrades performance.
• Adversarial or evaluative loops. Generation-critique pairs, red-team / blue-team setups, multi-perspective analysis. Asking the same model to both produce and critique in one prompt is weaker than separating the roles.
• Long-horizon tasks with checkpoints. Anything that runs for more than a few minutes benefits from a supervisor that can pause, assess progress, and re-plan.

If your problem is none of these - it is a single well-scoped task with one set of tools - a single agent with good prompts and a tight tool set will beat a multi-agent system on cost, latency, and reliability. Start there. Add agents when you have a concrete reason.

The frameworks worth knowing

You do not have to pick a framework to build a multi-agent system - a few hundred lines of Python around an LLM API will do it - but the major frameworks encode useful patterns and save weeks of plumbing.

LangGraph (from the LangChain team) models agents as a directed graph where nodes are agents or tools and edges are transitions. Good for supervisor patterns and explicit control flow. Strong production tooling via LangSmith for tracing.

Microsoft AutoGen pioneered the conversational multi-agent pattern - agents exchange messages in a chat-like protocol. The v0.4 release in early 2025 added an event-driven actor model that addressed earlier scaling complaints. Good for research and prototyping.

CrewAI takes a role-based approach: you define agents as personas with goals and backstories, assign them to tasks, and a crew object handles orchestration. Friendly for non-research teams, popular for business automation.

OpenAI Swarm / Agents SDK. OpenAI released Swarm as an educational reference in late 2024, then shipped the production Agents SDK in 2025 with handoffs, guardrails, and tracing built in. Lightweight, opinionated, well-suited to OpenAI-only stacks.

Anthropic's Claude Agent SDK (formerly Claude Code SDK) focuses on the tool-use loop and file-system primitives. Particularly strong for coding agents and tasks that touch real artefacts.

For most mid-market production work, LangGraph and the OpenAI Agents SDK are the two we reach for first. LangGraph when control flow is complex or the model needs to be swappable; Agents SDK when the team is already OpenAI-heavy and wants minimal scaffolding.

Where multi-agent systems are being used in production

Some concrete categories where multi-agent architectures are now in commercial use:

• Deep research and analysis. Anthropic's research feature, OpenAI's Deep Research, and Perplexity Pro all use multi-agent approaches under the hood to plan, parallelise, and synthesise.
• Software engineering. Cognition's Devin, GitHub's coding agent, and Cursor's background agents coordinate planner, coder, and verifier roles. The SWE-bench leaderboard has been dominated by multi-agent approaches since mid-2024.
• Customer operations. Routing-and-resolution patterns where a triage agent classifies the request, specialist agents handle billing, technical, or retention queues, and a quality agent reviews before send. Klarna, Intercom Fin, and Sierra have all published variants.
• Document-heavy professional services. Contract review, due diligence, audit support. Harvey, Hebbia, and others run multi-agent pipelines over large document sets where one agent indexes, another retrieves, another synthesises, another cites.
• Marketing and content operations. Editorial systems where a researcher, writer, editor, and SEO checker each handle a stage. The content engine running this site is one example.

The pattern across all of these is the same: high-value knowledge work, structured enough to decompose, where the cost of multiple agents is small compared with the cost of a human doing the same work.

What goes wrong in production

The failure modes are well understood now, and worth naming so you can design around them.

Context fragmentation. Each agent has a partial view. If the supervisor briefs workers badly, they make locally sensible but globally wrong decisions. Anthropic's post-mortems on early multi-agent prototypes repeatedly point to brief quality as the single biggest predictor of output quality.

Token cost blowout. Multi-agent systems can use 10-20x the tokens of a single agent. If you do not have a clear ROI hypothesis and a per-task budget, costs spiral fast. Instrument every run.

Latency. Even with parallel workers, supervisor planning and synthesis add minutes. Anything user-facing in real time is a poor fit unless you stream intermediate progress.

Evaluation difficulty. A single-agent system has one output you can grade. A multi-agent system has dozens of intermediate steps, any of which can be the problem. You need traces (LangSmith, Langfuse, Arize, or built-in framework tracing) from day one.

Coordination failures. Agents disagree, loop, or hand off in circles. Hard limits on turn counts, explicit termination conditions, and a supervisor that can break ties are non-negotiable.

Security and prompt injection. Each agent is an attack surface. An agent that reads untrusted web content and can also call internal tools is a classic confused-deputy problem. The UK's National Cyber Security Centre and the OWASP LLM Top 10 both flag agentic systems as a heightened risk category. Tool access should be least-privilege, and any agent touching external content should be sandboxed from agents with privileged tools.

A pragmatic build sequence

If you are building your first multi-agent system, the sequence that works:

1. Solve the problem with one agent first. Get a baseline. Measure quality, cost, and latency. Many problems stop here.
2. Identify the specific failure modes of the single agent. Is it running out of context? Mixing skills badly? Failing to self-critique? Each maps to a different multi-agent pattern.
3. Add the minimum number of agents that addresses the failure. If a critic loop fixes it, do not build a five-agent hierarchy.
4. Instrument from day one. Tracing, per-step cost, per-step latency, per-step success rate. Without this you cannot improve the system.
5. Define hard termination conditions. Max turns, max tokens, max wall-clock, max retries. Every framework supports these. Use them.
6. Build the evaluation harness before scaling. A small set of representative tasks with expected outputs, run on every change. This is the difference between a demo and a production system.

The teams who succeed with multi-agent systems treat them as software engineering with a probabilistic component, not as magic. Boring discipline beats clever architecture every time.

Frequently asked questions

What is the difference between a multi-agent system and an agentic workflow?

An agentic workflow is any process where an LLM makes decisions about what to do next, including calling tools and looping. A multi-agent system is a specific kind of agentic workflow where two or more distinct agents - each with its own role, prompt, and often its own tools - coordinate to complete the task. A single agent calling five tools in a loop is an agentic workflow but not a multi-agent system. A researcher agent handing off to a writer agent who hands off to an editor agent is a multi-agent system. The distinction matters because the engineering challenges - coordination, shared state, termination - only appear once you have multiple agents.

Do I need a framework like LangGraph or CrewAI to build one?

No, but you probably want one once you go beyond a prototype. The core of a multi-agent system is a few hundred lines of Python: agents are functions that call an LLM and return structured output, coordination is a loop, shared state is a dictionary. Frameworks earn their keep by providing tracing, durable state, retry logic, streaming, and standard patterns for handoffs and termination. For a one-off internal tool, plain Python with the OpenAI or Anthropic SDK is fine. For anything that runs unattended in production, the observability and durability features of a framework like LangGraph save weeks of work.

How much does a multi-agent system cost to run compared with a single agent?

Expect 5-20x the token cost of an equivalent single-agent run, depending on architecture. Anthropic has reported its own multi-agent research system at roughly 15x. Costs come from supervisor planning, parallel worker exploration (some of which is discarded), inter-agent messages, and synthesis. For high-value tasks - research that would take a human analyst a day, contract review that would take a paralegal hours - this is trivial. For high-volume low-value tasks like routine customer replies, the economics rarely work. Always calculate per-task cost against the human-time equivalent before committing to a multi-agent architecture.

How do I stop agents from looping forever or going off-task?

Three layers. First, hard limits at the framework level: max turns per agent, max total turns, max tokens, max wall-clock. Every production framework supports these and you should set them aggressively. Second, explicit termination conditions in the supervisor's prompt - a clear definition of "done" and instructions to stop when met. Third, a watchdog or evaluator agent that periodically checks whether progress is being made and can force termination if not. Loops usually happen because the success criteria are vague or the supervisor cannot tell when a worker has failed. Tighten both.

Are multi-agent systems safe to deploy with access to internal data?

They can be, but the attack surface is larger than a single agent. The main risk is prompt injection via agent inputs - if any agent in the chain reads untrusted content (a web page, an email, a user-uploaded file) and any other agent has access to privileged tools (databases, internal APIs, the ability to send messages), you have a confused-deputy vulnerability. The OWASP LLM Top 10 and NCSC guidance both call this out. Mitigations: least-privilege tool access per agent, sandbox agents that handle external content away from agents with internal tool access, validate all inter-agent messages against a schema, and log everything. Treat each agent as a separate security principal.

When should I use a single agent with tools instead of multiple agents?

Default to a single agent whenever the task fits in one context window, requires one coherent set of tools, and has a clear linear structure. A customer support reply, a SQL query, a code change in one file, a summary of one document - all single-agent jobs. Reach for multiple agents when the task has genuinely parallel parts (research many sources at once), distinct skills that benefit from separate prompts and tool sets (code-write vs code-review), or requires adversarial loops (generate-critique-revise). The honest answer is that maybe 70% of "AI agent" problems are best solved with one well-prompted agent and a tight tool set, and you should be sceptical of any vendor pushing you to multi-agent by default.

How do I evaluate whether a multi-agent system is actually working?

Build the evaluation harness before you scale the system. You need three layers: end-to-end task success on a held-out set of representative inputs (does the system actually solve the problem); per-step quality on intermediate outputs (is the researcher returning useful sources, is the critic catching real issues); and operational metrics (cost per task, latency, error rate, human-intervention rate). Tools like LangSmith, Langfuse, Braintrust, and Arize Phoenix all support this kind of multi-level evaluation. Without it, you cannot tell whether a change made things better or worse, and "vibes-based" tuning of multi-agent systems is how teams end up with expensive, slow, mediocre systems.

How long does it take to build a production multi-agent system?

For a focused use case with a clear baseline, expect 6-12 weeks from kickoff to a system you would put in front of users. The first two weeks go to scoping, single-agent baseline, and evaluation harness. Weeks three to eight cover the multi-agent build, including the inevitable rework when the first architecture turns out to be wrong. Weeks nine to twelve cover hardening: termination conditions, error handling, monitoring, cost controls, security review. Ambitious projects that try to ship a multi-agent system in two weeks almost always end up with something that demos well and fails in production. The hard parts are not the agents - they are the coordination, evaluation, and operations.

Where to go from here

If you have a problem that genuinely looks like a multi-agent fit - high-value knowledge work, parallel structure, distinct skills - the path is: prove a single-agent baseline, identify the specific failure, add the minimum coordination needed, instrument heavily, and harden before scaling. Skip steps and you will build an expensive demo. AI Advisory designs and builds these systems for UK mid-market businesses, from initial scoping through to running them in production.

Ready to put this into production? book a discovery call.