The orchestrator pattern is the simplest design that works for multi-agent systems, and the one I keep coming back to.
Routing heuristics
The orchestrator routes by intent. User asks about well pressure: route to the well agent. User asks for a SQL query: route to the Text-to-SQL agent. User asks something ambiguous: ask a clarifying question.
The routing logic lives in the orchestrator's system prompt. Don't overcomplicate it. A bulleted list of specialists with one-line descriptions of when to use each is enough for the model to do good routing.
Fallback chains
Specialists fail. Sometimes the Text-to-SQL agent can't generate a valid query. Sometimes the well agent doesn't have data for the well in question. The orchestrator needs a fallback chain: try specialist A, if it fails try specialist B, if that fails respond with the failure transparently.
The mistake to avoid: silent fallback. If the agent couldn't actually answer, the user needs to know, with enough detail to refine their question.
Latency budgets
Multi-agent systems multiply latency. Every hop is a model call, often with its own tool calls. Without budgets, a 4-step workflow becomes a 30-second wait.
Budget per agent. Budget per hop. Stream tokens as they're generated. If you can't hit the latency budget, simplify the agent graph or use a faster model for routing.
Specialization vs generalization
Narrower specialists are better. A "well data agent" that does five well-data tasks well beats an "operations agent" that does fifty tasks acceptably.
The reason: prompts. A narrow agent has a tight, specific prompt with high quality. A broad agent has a sprawling prompt that drifts over time. The narrow agent's quality stays consistent. The broad agent's quality decays.
When not to use it
If your system has one job, don't build an orchestrator. The pattern is overhead that pays for itself only when you have at least three specialists.
The orchestrator pattern is for systems that grow. If yours won't grow, you don't need it.