What is a multi-agent system?
A multi-agent system is one in which several AI agents, each a language model using tools in a loop, work together to complete a task. Orchestration is how you decide which agent does what, in what order, what information they share and when they stop.
The short answer for businesses: most problems should start with a single agent and a clear set of tools. Move to multiple agents only for a specific reason, such as work that can run in parallel, information that exceeds one context window, or a need for separate security boundaries per piece of work.
If you need the basics first, read what an AI agent is.
First principle: start with one agent
Both Anthropic and Microsoft recommend starting at the simplest level. In "Building effective agents", Anthropic advises finding the simplest solution possible and only increasing complexity when needed, which might mean not building agentic systems at all. Microsoft Learn's architecture guide defines three levels: a direct model call, a single agent with tools, and multi-agent orchestration. It recommends the lowest level that reliably meets requirements and calls a single agent with tools often the right default for enterprise use cases.
Anthropic also observes that multi-agent systems are today often applied where a single agent would perform better, and that it has seen teams spend months building elaborate multi-agent architectures only to find that better prompting on a single agent achieved equivalent results.
| Level | Description | When to use |
|---|---|---|
| Direct model call | One call with a good prompt, no tools | Classification, summarization, translation: single-step tasks |
| Single agent with tools | One agent chooses tools and loops over several steps | Varied queries in one domain, such as order lookup or data queries |
| Multi-agent | Several specialized agents with a coordination mechanism | Cross-functional problems, separate security boundaries, or parallelizable work |
Tests before adding an agent
- Can one agent with clear instructions and the right tools do the job? If yes, stop there.
- Does the work genuinely split into independent branches that can run in parallel?
- Is a single agent choosing among too many easily confused tools?
- Do security requirements force you to separate access by piece of work?
- Is the task valuable enough to pay for the extra tokens?
Comparing multi-agent orchestration patterns
The names below come from Anthropic and Microsoft Learn documentation; many frameworks use different names for the same idea.
| Pattern | How it works | Fits when | Risks to watch |
|---|---|---|---|
| Orchestrator-worker | A central agent breaks the task down dynamically, delegates to workers and synthesizes results | You cannot predict the subtasks in advance, such as multi-source research | Orchestrator spawns too many workers; faulty synthesis |
| Sequential pipeline | Agents chained in a fixed order; each processes the previous output | Multi-step processes with clear dependencies: draft, review, polish | Early errors propagate downstream; no parallelism |
| Concurrent (fan-out) | Several agents process the same input independently, then results are merged | You need independent perspectives or lower latency | Contradictory results need merge rules; simultaneous resource spikes |
| Handoff | One agent active at a time; agents decide to transfer to another specialist or a person | The right specialist only emerges during processing, such as multi-area customer support | Endless bouncing; unpredictable routing |
| Group chat / debate | Agents discuss in one thread with a manager choosing who speaks; includes maker-checker loops | You need critique, consensus or quality review | Conversation loops; hard to control with many agents |
| Hierarchical | A lead agent manages sub-agents, possibly over several levels; a variant uses a manager that plans dynamically (Microsoft calls it magentic) | Large, open-ended problems that need a plan built and revised | Slow convergence; unpredictable cost; hard to trace |
A note on group chat
Microsoft Learn recommends limiting group chat to three or fewer agents to keep control, and for maker-checker loops it requires clear acceptance criteria, an iteration cap and fallback behavior when the cap is hit, such as escalation to a human reviewer.
Combining patterns
Real systems often combine patterns: a pipeline for preprocessing, concurrency for independent analysis, handoff when a person must take over. Do not force every stage into a single pattern.
Lessons from a production system: Anthropic's multi-agent research
Anthropic has published how it built its Research feature: a lead agent plans the research and spawns subagents that search in parallel, and a separate agent handles citations at the end. This is the orchestrator-worker pattern.
In Anthropic's internal evaluation, a multi-agent setup (Claude Opus 4 as lead, Claude Sonnet 4 as subagents) outperformed a single Claude Opus 4 agent by 90.2% on its research eval. This is the vendor's own measurement, for one type of work: breadth-first search.
The limits matter more: Anthropic says multi-agent systems suit valuable tasks with heavy parallelization, information that exceeds a single context window and many complex tools, while domains where all agents must share the same context or that involve many dependencies between agents, such as most coding tasks, are not a good fit today.
- Early on, agents spawned 50 subagents for simple queries, searched endlessly for nonexistent sources and distracted each other with excessive updates.
- The fix included explicit effort-scaling rules, for example simple fact-finding needs just one agent with 3–10 tool calls.
- Every delegation to a subagent needs a clear objective, output format, allowed tools and task boundaries.
Common failure modes
Microsoft Learn notes that multi-agent systems bring back classic distributed-systems problems: component failures, lost messages, cascading errors. Anthropic stresses that agents are stateful and errors compound: a minor issue for traditional software can derail an agent entirely.
- Duplicated or dropped work: two agents do the same part, or no agent picks up the hard part.
- Error propagation: weak output from one agent becomes another's input without a check.
- Loops: endless handoffs, debates that never conclude, or repeated tool calls.
- Context bloat: each agent adds reasoning and tool results, so context grows and quality drops.
- Inconsistent shared state: concurrent agents editing the same data.
- Over-broad permissions: sub-agents get the lead agent's rights, or return data the user may not see.
Mitigations
- Hand over work with a structured record: task, inputs, constraints, expected output, acceptance criteria.
- Validate output before passing it on; halt or retry when it is poor.
- Set hard per-task limits: tool calls, token budget, run time, number of rounds.
- Grant least privilege per agent and apply the user's permissions in every agent.
- Persist state to a durable store so work can resume after a fault instead of restarting.
- Put human approval before hard-to-reverse actions. See human-in-the-loop.
Cost: why multi-agent burns tokens
In Anthropic's data, agents typically use about 4× more tokens than chat interactions and multi-agent systems about 15×. Anthropic concludes that multi-agent systems are economically viable only when the task's value is high enough.
Microsoft Learn adds that sequential and handoff patterns call agents one at a time, so cost accumulates step by step; concurrent patterns can spike resource use; and dynamic-planning orchestrations make total cost the hardest to predict.
- Match the model to each agent's task: classification, extraction and formatting can often use smaller models.
- Track tokens per agent and per run to find the expensive parts.
- Compact context between agents with summaries or pruning.
- Set a budget per task and alert when it is exceeded.
Illustrative example
An operations team caps each task at six tool calls and 15 seconds; when a limit is reached, the system stops, saves state and passes the task to a person. These numbers are illustrative only; the right limits must be measured on real workloads.
Observability and testing for multi-agent systems
Anthropic reports that adding full production tracing let it diagnose why agents failed, whether from bad search queries, poor sources or tool failures. It also uses rainbow deployments, shifting traffic gradually between versions, so updates do not disrupt running agents.
Microsoft Learn recommends instrumenting every agent operation and handoff, tracking performance and resource use per agent, and, because outputs are nondeterministic, testing integrations with scoring rubrics or model-as-judge evaluations rather than exact-match assertions.
| Area | What to record | Used for |
|---|---|---|
| Flow | Which agents ran, in what order, who delegated to whom | Reproducing and diagnosing failures |
| Tools | Tool called, parameters, result, errors | Permission checks and misuse detection |
| Cost | Tokens and time per agent | Cost optimization and budgets |
| Quality | Output scores, retries, escalations to people | Tracking quality trends |
| People | Who approved what, and when | Accountability |
A suggested path for businesses
EYC is building toward AI-Agent-supported operations and advises on and implements AI automation for businesses; the exact scope is agreed under contract.
- Start with one agent and a narrow toolset; measure quality on real tasks. See the AI agent operating model.
- When you hit a clear limit (too many tools, a need for parallelism, a need to separate permissions), split exactly one piece of work into a second agent, usually as a pipeline or orchestrator-worker.
- Put structured handoffs, hard limits and tracing in place before adding a third agent.
- Set permissions and logs for every agent. See access control and audit logs for AI agents.
- Scale only when measurements show several agents beat one agent on the same evaluation.
Frequently asked questions
How is multi-agent different from one AI agent with many tools?
One agent with many tools still has a single reasoning loop and a single context. A multi-agent system has several agents, each with its own context, instructions and permissions, plus a coordination mechanism. Microsoft Learn calls a single agent with tools often the right default for enterprises.
When should you use multiple AI agents?
When work can run in parallel, information exceeds one context window, a single agent must choose among too many confusable tools, or separate security boundaries are required, and the task is valuable enough to pay for higher token use.
How much more do multi-agent systems cost?
In Anthropic's data, agents use about 4× the tokens of chat and multi-agent systems about 15×. Actual numbers depend on the orchestration pattern, the models and how context is managed.
Which orchestration pattern should we try first?
A sequential pipeline or orchestrator-worker is usually easiest to control. Microsoft Learn warns that choosing a complex pattern when basic sequential or concurrent orchestration would suffice is a common mistake.
How do you stop agents from looping forever?
Set hard limits on tool calls, rounds, token budget and time; define clear completion criteria; and specify what happens at the limit, such as stopping, saving state and escalating to a person.
Is MCP the same as multi-agent orchestration?
No. The Model Context Protocol standardizes how agents discover and call tools; it helps control tool access but does not decide which agent does what. See what MCP is.