Prompt chaining vs one big prompt: when decomposition wins
The default move when you want an LLM to do something complex is to write one very long prompt and hope the model holds it all together. Sometimes that's fine. But there's a class of tasks where breaking the work into sequential steps — each smaller, each verifiable before you pay for the next — is clearly the better engineering choice. The harder question is knowing which situation you're in.
This is my working answer, built from implementing the prompt chaining pattern in effective-agents-lab. The patterns/chaining module is a six-stage content pipeline — outline, gate, draft, style, optional translate — and the design choices in it make the tradeoffs legible in a way that theory doesn't.
What chaining actually is
Prompt chaining is sequential: each LLM call takes the output of the previous one as input, and you move through defined stages in one direction. The playbook definition is "decomposing a task into a sequence of steps, where each LLM call processes the output of the previous one." That sounds obvious, but it undersells the key design choice: what you put between the steps.
A chain without gates is just a pipeline with extra API calls. The part that makes chaining worth choosing is the deterministic check inserted between stages — a piece of code (not another LLM call) that inspects the current step's output before allowing the next step to start. In patterns/chaining, the outline stage produces a structured Pydantic model, not free text, specifically so the gate can programmatically inspect it: section count, required-topic coverage, whether the structure is actually usable. If the gate fails, it feeds the failure reasons back into a second outline attempt. Two failed attempts raises GateFailed rather than looping forever. That's it — that's the pattern.
The gate is the whole point
Consider the alternative: ask for a finished draft in one prompt and hope it's structured well, covers the right topics, and is appropriately scoped. If it isn't, you've spent draft-sized money to learn that. With chaining, you spend outline-sized money first, verify, and only advance to draft if the structure passes. The outline costs a fraction of the draft — both in tokens and in model tier. In patterns/chaining, the outline and gate steps run on the FAST model; the draft and style steps run on BALANCED. That difference shows up in the trace as a cost difference, which is model-matching made visible rather than just theoretical.
There's also an auditability argument. The pipeline's docstring cites "clear audit trails and deterministic behavior... well-suited for regulatory environments" as a property of sequential workflows with programmatic gates. When you can see exactly what the gate rejected, on what grounds, and what retry produced — that's a different class of system than one where you send a big prompt and interpret a big response.
When a monolith beats chaining
Chaining adds latency. Each step is a round-trip, and a six-stage pipeline is six sequential calls where a single prompt is one. If the task is simple enough that one call handles it well, or if you can't define a cheap verification point between stages, you're paying latency costs without getting the benefit. The docs put it directly: "avoid it when steps don't have a clear, cheap verification point in between — at that point you're just adding latency and cost for a chain that can't actually catch its own mistakes."
The version of this I find most useful: chaining is worth it when the cost of a bad stage-N output, if uncaught, is larger than the cost of an extra round-trip. For draft-quality content, that's usually true — a poorly structured outline poisons everything downstream. For a simple classification or a short factual question, it's usually not true — one call is fine.
The other case where chaining degrades is when stages need to collaborate rather than hand off. Sequential workflows have linear dependencies: step N produces something, step N+1 consumes it, done. If your problem requires backtracking — "given the draft, revise the outline, then re-draft" — a sequential chain can implement that, but you're working against the grain of the pattern. That's the shape of an evaluator-optimizer loop (chapter 8), not a linear chain.
Honest notes on the eval numbers
The calendar row for this article says "use EAL's eval harness results." I'm going to be honest about what those are: evals/ exists and has real benchmark infrastructure, but no suite has run against actual models yet — the repo was built without a live API key. The module status table says so explicitly. So I have code-level evidence of the design (the pipeline structure, the cost model, the gate mechanics) but not live measurements of, say, how often the gate fires on real inputs or how much a --cheap run diverges in quality from a BALANCED run on the same topic.
What I can say from the offline tests: the pipeline is structured to make those measurements easy to take. Run it yourself with RUN_LIVE_TESTS=1 python -m pytest -m live once you have an API key, compare --cheap traces against BALANCED ones on the same topic, and the cost difference will be real and visible in the trace's total_cost_usd.
The pattern in practice
Running the chaining module looks like this:
python -m patterns.chaining "the tradeoffs between REST and gRPC for a new internal service" # Expected: DONE ($0.03xx, 6 stages) # Two extra stage events appear if the first outline fails the gate
The $0.03xx figure is from the README's expected output for a full BALANCED run. The six stages are: outline, gate, draft, style, and optionally translate — two light FAST calls followed by two heavier BALANCED calls. That ordering isn't arbitrary; you want the cheap verification to happen before you spend on the expensive work.
Whether to chain a given task comes down to one question: can I define a cheap, deterministic check between the first step and the expensive second step? If yes, chaining is probably worth it. If the check would itself require another LLM call to perform, you've negated half the benefit. If there's no natural break point at all, one well-constructed prompt is probably cleaner.