You start with a single prompt. It works for the demo. Then the task gets bigger, the edge cases pile up, and one call can no longer hold the whole job in its head.
The instinct at that point is to build architecture, a graph of agents talking to agents. Usually that’s the wrong instinct. But when a single well-prompted call genuinely falls short, there’s a small catalogue of compositions worth knowing by name.
This is that catalogue. It’s framework-agnostic, the shapes are the same whether you build with LangGraph, Google ADK, the Vercel AI SDK… It’s grounded in Anthropic’s “Building Effective Agents”, whose five workflows plus the ReAct agent cover almost everything you’ll actually reach for.
Workflows vs. Agents
Before anything else, keep one distinction straight:
- Workflows — systems where LLM calls and tools are orchestrated through predefined code paths. You, the engineer, wrote the control flow.
- Agents — systems where the LLM itself directs its own process: it decides which tools to call, in what order, and when the task is done.
Patterns 1–5 below are mostly workflows, deterministic wiring around LLM calls. ReAct is the canonical agent. Neither is “better.” Workflows give you predictability and cheaper debugging; agents give you flexibility at the cost of control.
💡 Pro tip: Prefer the least agency that solves the problem. Every degree of autonomy you hand the model is a degree of behaviour you now have to test.
The Basic Unit: a Node
Every pattern is a composition of one primitive. Call it a node: an LLM plus its scaffolding.
flowchart LR
In[Input] --> Node["LLM + system prompt<br/>(+ retrieval, + tools/MCP, + memory)"]
Node --> Out[Output]
A single node with a sharp prompt solves more than people expect. The augmentations like retrieval, tools, memory are the foundation of this approach and not the patterns. Get a node reliable first and only then start composing.
⚠️ Warning: Over-engineering is the default failure mode. The same instinct that produces gratuitous microservices produces gratuitous multi-agent graphs. Reach for a pattern because a single well-prompted call demonstrably fell short not because the diagram looks impressive. 😉
1. Prompt Chaining (Pipeline)
Decompose a task into a fixed sequence of steps; each node’s output is the next node’s input.
flowchart LR
In[Input] --> A["Node A"] --> Gate{"Gate?"} --> B["Node B"] --> C["Node C"] --> Out[Output]
Gate -->|fail| Stop[Exit / correct]
- When: the task splits cleanly into ordered subtasks, and trading latency for accuracy per step is worth it.
- Each node can run a different model, think that a cheap extractor can feed a strong reasoner.
- Insert programmatic gates between steps (a length check, a schema validation) to catch a bad intermediate result before it poisons the rest of the chain or cost you tokens/computation.
- Examples: outline → draft → polish for content; extract-entities → answer for support chat; the design → spec → tasks → code shape of spec-driven development.
Trade-off: as reasoning models get stronger, a single prompt with explicit “first X, then Y, then Z” steps often replaces the chain. Split into calls only when one long prompt becomes slow, unreliable, or hard to evaluate.
2. Routing
A classifier node inspects the input and delegates to one specialized downstream handler. The router never answers. It only decides the destination.
flowchart LR
In[Input] --> R{Router}
R -->|class A| A1["Handler A"]
R -->|class B| A2["Handler B"]
R -->|class N| AN["Handler N"]
R -->|low confidence| H["Human fallback"]
A1 --> Out[Output]
A2 --> Out
AN --> Out
H --> Out
- When: distinct input categories are better served by separate prompts/tools, and lumping them into one prompt would degrade all of them.
- Two ways to build the router:
- LLM classifier: flexible, costs a call and latency.
- Embedding similarity: embed the input, compare to per-class reference vectors; cheaper, no extra LLM call, weaker on nuance.
💡 Always wire a confidence threshold + human fallback. If the router is below (say) > 70% sure, escalate rather than guess.
- Classic case: customer support splitting into billing, technical, and general-question handlers, each grounded in its own docs.
3. Parallelization
Run independent LLM calls at the same time, then combine. Two flavours:
- Sectioning — split the work into independent subtasks that run concurrently.
- Voting — run the same task multiple times to get diverse answers and aggregate (majority vote, best-of-N).
flowchart LR
In[Input] --> A1["Worker: security"]
In --> A2["Worker: maintainability"]
In --> A3["Worker: performance"]
A1 --> Agg["Aggregator"]
A2 --> Agg
A3 --> Agg
Agg --> Out[Output]
- When: subtasks don’t need each other’s output, and speed matters or when multiple independent attempts raise confidence.
- The aggregator waits for all branches, then consolidates. It’s its own paid call — each branch plus the aggregator is a separate cost.
- Sectioning also improves focus: a code review split into security / maintainability / performance workers each concentrates fully on one concern instead of diluting attention across all three.
4. Orchestrator–Workers
A central orchestrator LLM dynamically decomposes the task, decides which subset of workers to invoke, dispatches them, and synthesizes their results.
flowchart LR
In[Input] --> O["Orchestrator<br/>(plans + picks workers)"]
O -->|selected| W1["Worker 1"]
O -.->|skipped| W2["Worker 2"]
O -->|selected| W3["Worker 3"]
W1 --> Agg["Synthesizer"]
W3 --> Agg
Agg --> Out[Output]
- When: you can’t predict the subtasks in advance — the decomposition depends on the input itself. This is the key difference from Parallelization, where the split is fixed by you.
- Combines routing (choose who) with parallelization (run the chosen concurrently). Sometimes it picks one worker (collapsing to a chain); sometimes many.
- Also called Planner–Executor: the planner produces a plan but executes nothing itself; workers do the work. A fan-out/fan-in shape, one node to N and back to one.
- Precedent: Microsoft’s HuggingGPT (2023) already had this exact structure, an LLM planner selecting which specialist models to run, then a response step aggregating. It’s also the backbone of agent-to-agent (A2A) systems.
5. Evaluator–Optimizer (LLM-as-Judge Loop)
One node generates; a second node evaluates against explicit criteria and either approves or returns structured feedback for another attempt. Loop until it passes or hits a round cap.
flowchart LR
In[Input] --> Gen["Generator"]
Gen --> Eval{"Evaluator<br/>pass?"}
Eval -->|pass| Out[Output]
Eval -->|fail + feedback| Gen
- When: quality matters more than speed/cost, “good” is checkable against clear criteria, and iterative feedback measurably improves the output — mirroring how a human refines a draft with a reviewer’s notes.
- The evaluator returns structured output (pass/fail or a score) plus a feedback string. The feedback is what makes the retry productive, a bare rejection teaches the generator nothing.
- Adversarial variant: a critic that only scores against a rubric and a rewriter that sees only the flagged problems (not the critic’s reasoning). Decoupling them tends to beat a single self-correcting agent.
⚠️ Warning: Infinite-loop risk. If the bar is unreachable (too strict, contradictory, impossible), the loop never exits. Always cap rounds, a hard stop at 5–6, and force an exit.
Bonus: ReAct (Reason + Act), the Autonomous Agent
Not one of Anthropic’s five workflows — this is the true agent. A single LLM with tools runs a loop: reason → act (call a tool) → observe → decide whether it’s done. The model controls the number of iterations.
flowchart LR
In[Input] --> LLM["LLM + tools"]
LLM --> Tool["Call tool / observe result"]
Tool --> Decide{"Task solved?"}
Decide -->|no, iterate| LLM
Decide -->|need input| Ask["Ask user"]
Ask --> LLM
Decide -->|yes| Out[Final answer]
- Collapses orchestration and evaluation into one LLM. Simpler to reason about than a multi-agent graph, and often “good enough.”
- When: the problem is open-ended, the number of steps is unpredictable, and you can trust the model with autonomy inside a well-scoped tool set.
- Guardrails matter more here than anywhere. Autonomy compounds errors — cap iterations, sandbox tools, log the trajectory, and add human checkpoints on irreversible actions. Test extensively in a sandbox before trusting it in production.
- Before reaching for Orchestrator–Workers or an Evaluator loop, check whether a single ReAct agent with good tools already solves it.
Patterns Compose and Nest
Real systems rarely use one pattern in isolation. Any node can itself be a subgraph. A routing destination might internally be an Evaluator–Optimizer loop; a worker in an orchestrator might run its own chain.
flowchart LR
In[Input] --> R{Router}
R --> A["Agent A<br/>(plain chain)"]
A --> Out[Output]
R --> B["Agent B"]
Out2[Output]
subgraph Internal["Agent B — its own Evaluator-Optimizer loop"]
direction LR
Gen["Generate"] --> Ev{"Evaluate"}
Ev -->|fail| Gen
end
B -.-> Gen
Ev -->|pass| Out2
From the outside, Agent B is one routing target. Inside, it has its own architecture. Compose deliberately — every layer of nesting is a layer of debugging.
Choosing a Pattern
| Situation | Reach for |
|---|---|
| A single well-prompted call would do | Just a node — skip the architecture |
| Sequential, ordered, decomposable steps | Prompt Chaining |
| One-of-many known specialized destinations | Routing |
| Independent subtasks, or best-of-N for confidence | Parallelization |
| Subtasks can’t be predicted ahead of time | Orchestrator–Workers |
| Quality > speed, “good” is checkable, feedback helps | Evaluator–Optimizer |
| Open-ended task, unpredictable steps, trusted autonomy | ReAct |
Three principles cut across all of them, straight from Anthropic:
- Simplicity — start with the simplest thing that works; add complexity only when it demonstrably improves outcomes.
- Transparency — make the agent’s planning steps explicit and inspectable.
- Craft the tool interface — document and test your tools as carefully as your prompts. The agent–computer interface is the new human–computer interface.
Every agent framework documents these same shapes under its own naming. Learn the patterns and the framework becomes an implementation detail.
Sources and Further Reading
- Anthropic: Building Effective Agents
- Microsoft: HuggingGPT
- Harness Engineering — these patterns are the “orchestration” piece of a harness; guides, sensors, and verification loops apply within each node.