An Amsterdam AI-native SaaS team stands at the whiteboard on a Monday morning. Their production agent runs Claude Fable 5 for every subtask — research, extraction, drafting, judgment, verification. Their monthly LLM bill is €22,000. Their median request latency is 4.2 seconds. Their tail latency is worse. Yesterday morning they read Rakesh Gohel's LinkedIn thread on the three-tier agent loop — orchestrator plans, workers execute in parallel, advisor critiques at commitment boundaries. They spent two hours at the whiteboard. They came away with a plan that would cut the bill by roughly two-thirds and the tail latency by roughly half, if it worked.
This piece is the cmdev honest read on that pattern — what it gets right, where the framing is thinner than the thread implied, the tiers that most posts leave out of the picture, and the real cost math that decides whether it earns the operational tax. Part 2 covers the three-tier attack surface — because a three-tier architecture is also a three-tier attack surface, and that half of the conversation is missing from every post we have read.
Key takeaways
- Role separation is the durable idea. Orchestrator plans, workers execute, advisor critiques — the tier logic outlives the specific models. Sol, Flash, Fable will be replaced; the shape stays.
- The cost architecture works when the workload has real parallel volume and real judgment-worth-paying-for at commitment points. On a workload that is small and sequential, the three-tier pattern adds overhead without earning it.
- The biggest weakness the LinkedIn framing skips is self-blind verification — an orchestrator that plans work and grades it against its own rubric will pass its own blind spots every time. The verifier has to be independent of the planner, or the loop has one signal fewer than it looks.
- Real production stacks have tiers below the LLM the three-tier framework does not show — a retrieval tier, a cache tier, and a deterministic-rules tier that should intercept requests before any model runs. Not naming them makes the LLM-only stack look complete when it is not.
- The pattern is worth the tax when you have three or more concurrent subtasks per request and a decision surface where an obvious-in-retrospect mistake would matter. Below that scale, one-model-plus-good-context is usually cheaper end-to-end.
What the Design Gets Right
The durable idea is role separation. Orchestrator plans and verifies. Workers execute in parallel with self-contained briefs. Advisor critiques at commitment boundaries and never executes. The specific models mapped to each role — the LinkedIn framing uses invented names like "GPT-5.6 Sol" alongside the real Claude Fable 5 — will churn in six months. The tier logic outlives them.
Role separation earns its place for three reasons. Cost architecture — cheap parallel labour handles the bounded execution where volume wins, premium judgment stays scarce for decisions where it changes the outcome. Getting this inverse (flagship model everywhere) is where the €22,000 monthly bill above came from. Latency — parallel dispatch of self-contained briefs collapses sequential subtask time into wall-clock time at the slowest brief. On a workload with five parallel subtasks, this is often the single biggest latency lever. Failure isolation — a bad brief fails in its own worker without corrupting the others, and the orchestrator handles the re-dispatch. A single monolithic agent conflates its own reasoning failures with tool failures with subtask failures in a way that is genuinely harder to debug.
The three failure modes Gohel names — context leaks, silent partial failures, judgment applied too late — are real and worth the discipline. Every brief carries its full inputs and acceptance criteria inline; no worker depends on another. Every result gets judged against its own criteria before merge. Advisor consultation happens at commitment boundaries, not after the deliverable is already wrong. These are the disciplines that separate a demo from a production loop.
What the Design Gets Thin
Three specific weaknesses that the LinkedIn thread skips.
The first and biggest is self-blind verification. If the orchestrator both plans the work and grades the workers against its own rubric, it will pass its own blind spots every time. The rubric was written by the same model whose reasoning produced the plan; the same reasoning failures that miss a case in the plan will miss the same case in the grading. This is exactly the pattern the least-privilege secure code review architecture addresses — the verifier's job is to refute, not confirm, and the verifier's independence from the planner is what makes refutation work. The three-tier framework as commonly drawn does not enforce that independence.
The second is advisor-called-twice. "Consulted before start and before ship" catches bookend failures. It does not catch the failure that happens in the middle — the plan was fine, the delivery was fine, but the third worker returned garbage that the orchestrator failed to catch because the failure was subtle enough to survive the rubric. The Sirehs pharmacist-in-the-loop pattern is stronger — every prescription passes the pharmacist, not just the first. For high-consequence workflows, the advisor should run at every commitment boundary, not just two.
The third is the missing tiers below the LLM. Real production stacks intercept requests before any model runs. A cache tier returns the deterministic answer to a repeat query in milliseconds. A retrieval tier fetches the ground truth from a known-good source. A deterministic-rules tier answers policy questions with code, not with reasoning. The three-tier LLM framework treats every request as an LLM call, which is fine on a fresh problem and expensive on a repeat one. On production agents we have seen, the cache tier alone often handles 30-60% of traffic before any LLM cost is incurred.
The Missing Tiers Below the LLM
The order of precedence for a well-designed agent request path is roughly this — cache first, retrieval-plus-deterministic-rules second, LLM-based tiers third. Every layer above LLM is cheaper and more predictable than every layer at LLM. Skipping the cheaper layers because "the LLM can handle it" is where most teams overspend on tokens.
Cache tier. Hash the normalised request, look up an exact match, return the previous answer with a timestamp. Works well for FAQs, boilerplate transformations, deterministic transformations of stable inputs. The cache invalidation policy is the design work; the cache itself is trivial.
Retrieval tier. Hybrid search (BM25 + embeddings) against a known-good corpus. If the answer exists verbatim or near-verbatim in the corpus, return the corpus answer with the citation. This is not the same as RAG feeding an LLM — this is retrieval as the terminal step. On documentation-Q&A workloads, this handles a meaningful share of traffic before the LLM ever sees the request.
Deterministic-rules tier. Structured tools, calculations, policy checks. Any request that boils down to is this within the envelope, apply this transformation, look up this reference is a rules call, not an LLM call. This is the same discipline as the policy layer for AI agents — rules are code, code is deterministic, deterministic is auditable.
Only the requests that survive all three layers reach the LLM tiers. When they do, the three-tier loop is the right shape. Before they do, the cheaper layers should have absorbed as much as they can.
When to Reach for the Three-Tier Loop
The pattern earns its operational tax when three conditions hold. The workload has real parallel volume — at least three concurrent subtasks per request that can genuinely run independently. The commitment decisions have real cost of being wrong — the wrong answer visible to a customer, a downstream data commit, a public-facing artefact. The token spend at the current architecture is meaningful enough that halving it moves the P&L needle.
The pattern is overkill when the workload is small and sequential — a two-step research-then-summarise task on a single query does not benefit from parallel workers and does not have a commitment decision worth an advisor's time. It is also overkill when the token spend is small enough that the operational tax of running three tiers exceeds the savings from routing cheap work to cheap models.
The Amsterdam team above had three or four subtasks per request on their main workload, €22,000 per month spend, and customer-visible summaries where an obvious-in-retrospect miss would land on the wrong side of the trust question. They were the right candidate. A weekend-project agent talking to one API is not.
The Cost Math That Actually Justifies It
Rough numbers for a workload of 100,000 requests per month, three parallel subtasks per request, one advisor consultation per commitment (roughly one per five requests), and current-generation model pricing:
- Flagship-only — 100,000 × 3 subtask calls at ~$0.020 per call plus 100,000 × 2 orchestration calls at ~$0.020 = $10,000/month
- Three-tier — 100,000 × 3 worker calls at ~$0.002 (cheap-parallel tier) + 100,000 × 2 orchestrator calls at ~$0.020 + 20,000 × 1 advisor calls at ~$0.030 = $5,200/month
Roughly half the cost, all else equal. The tail latency picture improves further because parallel worker dispatch beats sequential flagship-tier calls at wall-clock time. These are direction-of-travel numbers; substitute your own model pricing and your own subtask counts before quoting them internally.
The number is meaningful. The number is also not the point. The point is that at production scale, the three-tier loop moves the P&L needle by enough to justify the design work, and it moves the latency needle by enough to matter to the on-call SRE.
If your team's LLM bill is closer to €22,000 a month than to €1,000, which of the three weaknesses above would show up in the redesign — the self-blind verifier, the middle-of-the-loop advisor gap, or the missing tiers below the LLM?
FAQs
Does role separation stop mattering when a single model is good enough at every subtask?
No, because the role separation is about cost architecture and failure isolation as much as it is about capability. Even if the same model could do every role, running the worker tier as parallel stateless calls with self-contained briefs is faster (wall-clock time) and easier to debug (failure isolation) than a single monolithic loop. The model-tiering optimisation stacks on top of the role-separation architecture; skipping either leaves value on the table.
Is fixing the self-blind verifier as simple as using a different model?
Necessary but not sufficient. Using Fable 5 as the planner and a Gemini model as the verifier reduces the reasoning-chain overlap and catches a share of the shared-blind-spot cases. It does not fix the deeper problem — the verifier is still evaluating against a rubric written by the planner. The stronger fix is to have the advisor tier write the rubric before dispatch, so the verifier is grading against a target it did not construct. That is the pattern we recommend for high-consequence workflows.
How do you decide what belongs in the cache, retrieval, or deterministic-rules tier versus the LLM tier?
The test is straightforward. Cache the requests that are byte-for-byte repeats over some window (typically 24 hours for user-facing content, longer for internal workflows). Retrieve on questions where the answer exists in a known corpus and the wording tolerance is high. Route to deterministic rules any question that boils down to a calculation, a policy lookup, or a structured transformation. Everything else falls through to the LLM tiers. The design work is not choosing which requests go where; it is instrumenting the request stream to see which categories dominate the traffic, then routing them.
What is the smallest workload where the three-tier architecture actually pays off?
Rough rule of thumb — 10,000 requests per month with three or more parallel subtasks per request, or €1,000 per month in LLM spend, or a commitment surface where a customer-visible miss would matter. Below any of those thresholds, one-model-plus-good-context is usually cleaner and the operational tax of three tiers is not justified. Above them, the three-tier pattern earns the design work.
Companion content
- The Three-Tier Agent Loop as a Three-Tier Attack Surface — Part 2, the security surface nobody names
- Coordinating the Agent — The Working Vocabulary, Part 2 — the coordination vocabulary this piece assumes
- Instructions Are Not a Boundary: Least-Privilege for AI-Assisted Code Review — the verifier-independence pattern this piece references
- LLM Resilience Patterns, Part 1: Design — the reliability primitives the three-tier loop sits on
How to engage
If your team is redesigning an agent architecture and you want a vendor-neutral read on where the three-tier pattern earns its tax and where the cheaper-tiers-below-LLM would carry more of the load, talk to us at creativeminds.dev/contact.