A month after the Friday review in Part 2, the same Lagos fintech's agent is in production. It processes 4,200 transactions on a Tuesday. Transaction #3,847 is refunded to the wrong customer, in the wrong currency, for ten times the correct amount. The engineer on-call opens the audit log. She can see the LLM's chain of thought. She cannot immediately answer three questions. Was that action approved by policy before it executed? Which permissions did the agent have when it decided? Did the eval suite catch this pattern in staging?
Governing an agent in production is the third register of the vocabulary. The first register (Part 1) named the shape of the agent. The second (Part 2) named how it reaches the world. The third names the discipline that decides whether the agent's mistakes are visible, contained, and recoverable. Seven terms, each named as a discipline rather than a checkbox — and each with the same honest treatment as the earlier parts: when to reach for it, what it costs, and what breaks when it is missing.
Key takeaways
- Guardrails are hard safety boundaries enforced in the runtime, not soft best-practice instructions in the prompt. Prompt-side rules are polite requests to a workload that will sometimes ignore them.
- The policy layer is where the rules, permissions, and approvals for agent action live as code — outside the model, evaluated on every action, with a structured decision log.
- Sandboxing is the disposable environment where the agent can test actions before they touch live systems. Every action agent needs one; the size and cost of the sandbox scales with the action's blast radius.
- Human-in-the-Loop is the pause on critical actions for human review. Design it as a first-class flow, not as an interrupt — an HITL flow that people ignore in practice is not an HITL flow.
- Agent identity is which agent acted, under which role, with which permissions, on whose behalf. Without agent identity, observability is a wall of undifferentiated LLM traces and no audit chain survives contact with the auditor.
The Boundaries
Three terms cover where the agent is technically prevented from doing things that would surprise you.
Guardrails
Hard safety boundaries enforced by the runtime — a workload cap, a filesystem sandbox, a network egress allow-list, a maximum action rate. Guardrails are what the runtime enforces regardless of what the model asks to do.
- Reach for them on every agent that takes real actions. Guardrails are the ones nobody argues about after an incident and nobody remembers to build before one.
- Trade-offs — a guardrail written into the system prompt is not a guardrail. It is a request. Real guardrails are enforced at the runtime layer where the agent cannot reach to disable them; see OPA for agent action approval and the least-privilege architecture.
- Cost and effort — the guardrails themselves are a few weeks of engineering. The discipline of keeping them in force as the agent gains new tools is the ongoing tax that most teams under-budget.
Policy Layer
The rules, permissions, and approvals for agent action, expressed as code, evaluated at runtime, outside the model. Every tool call passes through the policy layer and receives an allow or deny, with reasons.
- Reach for it the moment the agent has any tool that touches production state. Financial actions, customer communications, infrastructure changes, external API calls — all of these belong on a policy layer, not on a prompt.
- Trade-offs — the policy layer is where the compliance officer's Friday review either has structured answers or does not. Policies expressed in the prompt fail an audit; policies expressed as code with a decision log survive one. The trade-off is upfront engineering cost against ongoing audit cost.
- Cost and effort — the initial policy layer is a two-to-four-week project (Open Policy Agent, Cedar, or a custom evaluator). The ongoing work is keeping the policy set in sync with the tools the agent gains, which requires a review cadence and a policy owner.
Sandboxing
A disposable environment where the agent can test actions before they touch live systems. Filesystem writes, database mutations, network requests, code execution — all happen in a snapshot that is destroyed at the end of the task.
- Reach for it on every agent that executes code, mutates data, or takes actions with side effects the user did not explicitly authorise. Reach for it especially on agents doing security review, code changes, or research against untrusted sources.
- Trade-offs — sandboxing adds latency and infrastructure cost. A hardened container is milliseconds; a microVM is a hundred milliseconds; a fresh cloud environment is minutes. The tier decision matters. Cheaper sandboxes are faster and less isolated; more expensive ones are slower and safer.
- Cost and effort — the runtime cost scales with agent volume. The engineering cost is a one-time investment in the sandbox spin-up, tear-down, and result-marshalling machinery.
The Humans
Two terms cover the parts of the flow where a person is still the one deciding.
Human-in-the-Loop
Critical actions pause for human review, approval, or intervention before executing. The agent proposes; the human disposes.
- Reach for it on any action whose blast radius exceeds what the agent's autonomy justifies — payments above a threshold, actions in a regulated context, decisions that touch a customer relationship the business cares about.
- Trade-offs — an HITL flow that people ignore in practice is not an HITL flow. Design it as a first-class part of the workflow, not as an interrupt that arrives via email at 3 AM. Reviewers need context, not just a yes/no button; batching pending actions into a work queue is usually stronger than routing each action individually.
- Cost and effort — the technical implementation is a few days. The organisational implementation — who reviews, what SLA, what happens on escalation — is where the real work sits. Design the approval gate as a scaling asset, not as a bottleneck.
Agent Identity
Which agent acted, under which role, with which permissions, on whose behalf. Not just a service account — the full delegation chain from the originating human through the agents that carried the action.
- Reach for it as a schema decision from the first line of code. Retrofitting agent identity into an existing agent stack is meaningfully harder than designing it in from the start.
- Trade-offs — without agent identity, observability is a wall of undifferentiated LLM traces and no audit chain survives contact with the auditor. With agent identity properly modelled, the compliance officer's questions become straightforward — this agent, this role, this delegation, this action, this outcome. The principal-of-record chain is the design reference.
- Cost and effort — modest engineering cost if done from day one, meaningful cost if retrofitted. The value shows up in every audit, every incident, and every architecture review from month three onward.
The Visibility
Two terms cover how you know what happened.
Agent Observability
Traces, logs, and monitoring that let you audit what the agent did, why, and with what input. Not the model's chain of thought alone — the full timeline from user input through tool calls through response, joined on trace IDs.
- Reach for it as infrastructure alongside the agent, not as an afterthought. An agent without observability cannot be debugged, cannot be audited, and cannot be improved because you have no signal.
- Trade-offs — observability data grows fast, especially with long context windows and heavy tool use. Retention policy, cost of storage, and query performance are real operational concerns. Under-instrument and you cannot debug; over-instrument and the storage bill becomes the largest line item.
- Cost and effort — modern platforms (Langfuse, Arize, Datadog, cloud-native equivalents) make the instrumentation itself cheap. The discipline of naming spans, propagating trace IDs across the orchestrator hops, and modelling agent identity into the trace context is the engineering work.
Evaluations (Evals)
Measures of whether the agent is accurate, reliable, and production-ready — against a curated dataset, on a schedule, with pass/fail thresholds that gate deploys.
- Reach for them before the first production release. Evals are the safety net that catches the regressions the model releases will otherwise put on your production doorstep quietly.
- Trade-offs — evals are only as good as the dataset they run against. A weak eval suite gives false confidence; a good one takes ongoing curation effort. Evals also take compute — a full suite against a full dataset is minutes to hours per run, and the cadence has to be tuned.
- Cost and effort — the initial eval suite is a two-to-four-week project. The ongoing curation — adding new cases from production failures, retiring stale ones, calibrating against updated models — is a quarter of one engineer's ongoing time. Most teams under-invest here until an incident makes the investment obvious.
If Tuesday's refund incident had happened at your team next week, which of the seven terms above is the one whose absence would have made the audit-log answer hardest to write?
FAQs
Do we need all seven, or can a small agent skip some?
Every agent that takes real actions needs guardrails, some kind of policy layer, agent identity, observability, and at least a minimal eval suite. Sandboxing is skippable for read-only agents; HITL is skippable if the blast radius is truly small. The rule of thumb is that the discipline scales with the consequences of a wrong action, not with the size of the agent.
Is Open Policy Agent the right choice for the policy layer, or should we build in-house?
Open Policy Agent is the right choice for most teams. It has the ecosystem, the maturity, and the audit-friendly decision-log format. In-house is right when the policies are so business-specific that Rego becomes awkward, or when the existing platform already has a policy engine you would rather standardise on. Do not build in-house because OPA feels heavy; that feeling passes after the second policy update.
How do we handle evals when the model version changes underneath us?
Pin the model version for every eval run and record it alongside the results. When a new model version arrives, run the full suite against it before promoting. Do not assume the new model is better on your specific eval set; it usually is not, and when it is not, you want to catch it before your customers do. Model-version-pinning also gives you the reproducibility that the audit chain requires.
What is the single most under-invested discipline on this list?
Agent identity, followed closely by evals. Both are hard to add retroactively and easy to defer at the start of the project. Both are also the ones that decide whether the agent's mistakes are recoverable — because a mistake without an identity trail is a mistake nobody can attribute, and a mistake without an eval suite is a mistake that will happen again.
Companion content
- Building the Agent — The Working Vocabulary, Part 1: Foundations — the seven foundation terms that sit under this part
- Coordinating the Agent — The Working Vocabulary, Part 2: How Agents Reach the World — the six coordination terms this part relies on
- OPA for AI Agent Action Approval — the concrete policy-layer implementation pattern
- IAM for AI Agents: Delegation and the Principal-of-Record Chain — the agent-identity design reference
- Hardened Containers for AI Workloads: The Tier Decision — the sandbox-tier choice
- Agent Action Approval Gates — the HITL approval-gate pattern at scale
How to engage
If your agent is heading into production and the discipline above is behind the deployment schedule, talk to us at creativeminds.dev/contact. The cmdev diagnostic covers guardrails, policy layer, identity design, observability instrumentation, and eval-suite bootstrap — mapped to your specific stack, timed to your release calendar.