Orchestrator–workers: when one model should manage others

The previous two posts covered prompt chaining and parallelization — patterns where you, the developer, decide the structure: which steps run in sequence, which run in parallel, what each one does. Orchestrator–workers is the step where the model starts making those decisions itself. A supervisor LLM looks at the task, decides what subtasks are needed, delegates them to workers, reviews the results, and synthesizes a final output. The structure emerges from the input rather than being baked into the code.

That's the thing that separates it from the prior patterns, and the reason I built it as its own module in effective-agents-lab: the subtasks are not predefined. The supervisor figures them out at runtime from the brief. I wanted to see what "dynamic decomposition" actually looks like when it works and when it doesn't, so I gave it two very different inputs: a B2B SaaS campaign with a $180,000 budget across ten weeks, and a local hardware store promotion with $1,200 for one weekend. The orchestrator produces structurally different plans for each. That's the point.

What the supervisor actually does

The supervisor's first job is planning. It reads the brief and produces a list of Assignment objects — each one naming a specialist (market researcher, copywriter, media planner, etc.) along with a directive and a list of assignments it depends on. The plan is a DAG: some workers can start immediately, others need an earlier worker's output first.

Here's where I added something that's easy to skip but worth the effort: before a single worker runs, the plan goes through validate_plan(), a pure Python function that checks for structural problems — cycles, dangling dependency references, zero assignments, plans too large to be useful. If it finds anything wrong, the supervisor gets one retry with the error message attached. If the second attempt also fails, the whole run raises PlanInvalid and stops. The point of this gate is that a plan that fails structural checks will either produce garbage or loop forever, and you'd rather know that before spending API budget on workers. Deterministic checks before probabilistic execution is a theme across the whole codebase, and it applies here too.

Dependency waves, not just concurrency

Once the plan is valid, execute_plan() runs the assignments. It doesn't serialize them, and it doesn't naively parallelize everything either. It groups them into waves: each wave is all the assignments whose dependencies are already satisfied. The first wave is every assignment with no dependencies — these run concurrently via ThreadPoolExecutor. When a wave finishes, the next wave of newly-unblocked assignments starts.

This matters because full serialization ignores real independence (you're slow for no reason) and full parallelization breaks assignments that genuinely need a prior one's output. The wave approach respects both. Workers that depend on each other stay ordered; workers that don't, overlap. If the cycle check in validation somehow passed a cyclic plan through — defense in depth — execute_plan() detects that no assignment is ever ready and raises rather than looping forever.

Context discipline

The part of orchestrator–workers that bites people if they don't think about it: when a worker depends on an earlier assignment, the earlier assignment's output ends up in the worker's prompt. If you have a long chain where each step's full output flows into the next, the downstream workers' prompts balloon to the point where they either hit context limits or just get hard to reason about.

The implementation handles this by truncating the upstream context each worker receives to 8,000 characters per dependency, not the full raw output. The worker gets enough to do its job; it doesn't get everything. The truncation limit is an explicit constant in the code, not a silent trim, which matters when you're debugging a worker that seems to be missing information — you know exactly where to look.

Not every worker needs the same model

One practical thing the implementation makes visible: the five specialist types don't all use the same model. Market researchers, creative directors, and copywriters use the balanced tier — they're doing generative, judgment-heavy work. Media planners and budget analysts use the fast tier — their work is more structured, less open-ended, and accuracy at speed matters more than nuance. The supervisor itself uses the balanced tier for planning, reviewing, and synthesis.

This is the same cost logic that routing uses: right model for the task, not default model for everything. In a long orchestrated run with several assignments, the model-tier choices compound.

When it earns its cost, and when it doesn't

Orchestrator–workers adds overhead: a planning call, a validation step, a review pass after each worker, a synthesis call. That's at minimum four model calls before the actual workers even count. The Anthropic playbook puts multi-agent token costs at roughly 10–15x a single-model equivalent, and while I haven't run the full eval comparison against live models yet (no API key in the sandbox), the token math is real. You pay for flexibility.

Whether that flexibility is worth it comes down to one question: could you have written the worker list yourself before running anything? If yes — if the decomposition is predictable from the task type — then use chaining or parallelization. The orchestrator will just reproduce a plan you already know, at extra cost. But if the decomposition genuinely depends on the specific input — a campaign for a $1,200 weekend promotion needs fundamentally different structure than one for a ten-week, $180k B2B launch — then static structures can't serve you, and the supervisor earns its overhead by adapting.

That's the tell I actually use: write out the worker list on paper before running. If it's the same list every time regardless of what you put in, you don't need a supervisor model to produce it.