An insurance company deployed a claims processing agent that could handle 80% of routine claims without human intervention. The remaining 20% involved ambiguous situations: overlapping policy clauses, disputed damage assessments, claims touching multiple coverage types. The team’s first instinct was to keep improving the agent until it could handle 100%.
That was the wrong approach. The right approach was designing the agent to recognize the 20% it couldn’t handle and escalate those cases to human reviewers with full context. Six months later, the agent processed routine claims in 90 seconds while flagging complex cases with pre-assembled documentation that cut human review time in half.
Human-in-the-loop isn’t a failure of automation. It’s a design pattern that makes the overall system more reliable than either humans or agents working alone. This guide covers four HITL patterns, when to use each, and how to implement them without creating bottlenecks.
Interactive Concept Map
Click any node to expand or collapse. Use the controls to zoom, fit to view, or go fullscreen.
Why agents need human fallbacks
The core problem is confidence calibration. Current LLMs are frequently confident when they’re wrong. An agent that’s 95% accurate sounds impressive until you realize the 5% of incorrect outputs come with the same confidence level as the 95% of correct ones. Users can’t tell the difference.
Human-in-the-loop solves this by building explicit checkpoints where the system pauses, presents its reasoning, and waits for human confirmation before proceeding. This catches the high-confidence errors that users and automated checks miss.
Three situations always warrant HITL:
Irreversible actions. Sending an email, executing a financial transaction, deleting data, publishing content. Once done, these can’t be undone. The cost of a wrong action far exceeds the cost of a 30-second human review.
Ambiguous inputs. Queries with multiple valid interpretations, requests that touch edge cases in your business logic, inputs in domains where the agent has limited training data. When the agent isn’t sure what the user means, asking is better than guessing.
High-stakes decisions. Medical triage, legal recommendations, security incident response, financial approvals above a threshold. The consequences of errors in these domains are severe enough that human oversight is a requirement, not an option.
Pattern 1: Approval gate
The simplest HITL pattern. The agent prepares an action, presents it to a human for approval, and executes only after receiving explicit confirmation.
When to use
- Before any irreversible action (email sends, database writes, financial transactions)
- When the agent’s action will be visible to external parties
- For any action with a cost above a defined threshold
How it works
Agent receives request → Agent plans action → Agent presents plan to human
→ Human approves → Agent executes
→ Human modifies → Agent executes modified version
→ Human rejects → Agent acknowledges and offers alternatives
Implementation considerations
The approval interface needs to show three things clearly: what the agent intends to do, why it chose this action, and what the consequences will be. A vague “Approve this action?” prompt forces the human to reverse-engineer the agent’s reasoning, which defeats the purpose.
Good approval prompt:
Action: Send refund of $247.50 to customer@email.com
Reason: Order #89234 was returned within the 30-day window.
Product was defective (customer provided photos).
Impact: Account balance will be debited. Customer will receive
funds in 3-5 business days.
[Approve] [Modify Amount] [Reject]
Bad approval prompt:
Process refund? [Yes] [No]
Avoiding bottlenecks
Approval gates create latency. Minimize it by batching non-urgent approvals, setting auto-approval rules for low-risk actions below a threshold, and routing approvals to the team member with the shortest response time.
Pattern 2: Confidence-based escalation
The agent self-assesses its confidence in each response. When confidence falls below a threshold, it escalates to a human rather than presenting an uncertain answer.
When to use
- Customer-facing agents where incorrect responses damage trust
- Knowledge retrieval systems where accuracy is critical
- Any system where the agent can meaningfully estimate its own uncertainty
How it works
Agent processes query → Agent generates response + confidence score
→ Confidence >= threshold → Deliver response to user
→ Confidence < threshold → Route to human with context
Measuring confidence
Raw LLM token probabilities are unreliable confidence indicators. Better approaches:
Source coverage. If the agent’s answer is grounded in retrieved documents, measure what percentage of the answer is supported by sources. Low source coverage suggests the agent is generating from parametric knowledge rather than verified facts.
Self-consistency. Run the query three times with temperature > 0. If the answers diverge significantly, confidence is low. If they converge, confidence is higher. This adds latency and cost but catches cases where the model is unstable on a particular input.
Domain classifiers. Train a lightweight classifier to flag queries that fall outside the agent’s training distribution. Out-of-distribution queries get escalated regardless of the agent’s self-reported confidence.
Setting thresholds
Start conservative. Set the threshold high so most ambiguous cases escalate to humans. Track what humans do with escalated cases. If humans approve the agent’s response 90%+ of the time for a particular category, lower the threshold for that category. This data-driven approach avoids both over-escalation (too many human reviews) and under-escalation (wrong answers reaching users).
Pattern 3: Supervised autonomy
The agent operates autonomously within defined boundaries. When it encounters a situation outside those boundaries, it pauses and requests human guidance before continuing.
When to use
- Long-running agent workflows with multiple steps
- Agents that need to adapt to novel situations
- Workflows where the first few steps are predictable but later steps depend on intermediate results
How it works
Agent starts workflow → Executes Step 1 (within boundaries)
→ Executes Step 2 (within boundaries)
→ Step 3 encounters situation outside boundaries
→ Agent pauses, presents situation to human
→ Human provides guidance
→ Agent incorporates guidance and continues
Defining boundaries
Boundaries can be explicit rules or learned patterns:
Explicit rules: “Never modify a customer’s subscription tier without approval.” “If total order value exceeds $10,000, pause for review.” “If the agent needs to access data from a system it hasn’t been authorized for, escalate.”
Learned boundaries: Track cases where the agent made decisions that were later overridden by humans. Build a classifier from these override cases to identify similar situations in the future.
The most effective boundary systems combine both. Explicit rules catch known edge cases. Learned boundaries catch patterns that emerge from real usage.
Preserving context during pauses
When the agent pauses for human input, it must preserve its full state: what it’s done so far, what it was about to do, and why it paused. Without this context, the human reviewer can’t make an informed decision, and the agent can’t resume smoothly.
Store the agent’s state as a structured checkpoint:
{
"workflow_id": "claim-2847",
"completed_steps": [
{"step": "verify_policy", "result": "active", "timestamp": "..."},
{"step": "assess_damage", "result": "estimated $4,200", "timestamp": "..."}
],
"current_step": "determine_coverage",
"pause_reason": "Claim involves both property damage and liability. Multiple coverage clauses may apply. Need human guidance on which clause takes precedence.",
"agent_recommendation": "Apply clause 7.2 (property damage) as primary coverage",
"alternatives_considered": ["Clause 9.1 (liability)", "Combined coverage under clause 12"]
}
Pattern 4: Review and learn
The agent operates fully autonomously but a sample of its outputs are reviewed by humans after the fact. Review findings feed back into the system to improve future performance.
When to use
- High-volume, lower-stakes operations where real-time review isn’t feasible
- Systems where you want to gradually expand agent autonomy based on demonstrated performance
- Initial deployment phases where you’re building confidence in the agent’s capabilities
How it works
Agent processes all requests autonomously → System samples N% of outputs
→ Human reviews sampled outputs
→ Correct outputs: no action
→ Incorrect outputs: flagged, corrected, added to training/eval data
→ Review findings update agent prompts, tools, or guardrails
Sampling strategies
Random sampling: Review a fixed percentage (5-10%) of all outputs. Simple and unbiased, but may miss rare failure modes.
Stratified sampling: Over-sample categories with historically higher error rates. If the agent struggles with multi-step tasks, review 20% of those and 5% of simple queries.
Anomaly-triggered sampling: Flag outputs that look unusual, including unusually long responses, unusual tool call patterns, or responses that diverge from typical patterns for that query type. Review all flagged outputs plus a random baseline.
Closing the feedback loop
Reviews only matter if they change the system. Build a pipeline that converts review findings into concrete improvements:
- Weekly review summary: Categorize errors by type (accuracy, tone, safety, completeness)
- Root cause analysis: For each error category, identify whether the fix is a prompt change, tool modification, guardrail addition, or training data update
- Implement and measure: Make the change, re-evaluate on the error cases, and track whether the error rate decreases
Teams that skip the feedback loop end up reviewing the same mistakes over and over.
Choosing the right pattern
| Situation | Pattern | Why |
|---|---|---|
| Agent sends emails | Approval gate | Irreversible action, external visibility |
| Customer support bot | Confidence-based escalation | Trust is critical, real-time interaction |
| Multi-step data pipeline | Supervised autonomy | Long workflow, novel situations possible |
| Content moderation | Review and learn | High volume, continuous improvement needed |
| Financial transactions > $500 | Approval gate | High stakes, cost threshold |
| Research assistant | Confidence-based escalation | Accuracy critical, grounding verifiable |
| Code generation | Supervised autonomy | Complex workflow, output needs validation |
| Chatbot for FAQs | Review and learn | Low stakes, gradual quality improvement |
Most production systems use multiple patterns. A customer service agent might use confidence-based escalation for answering questions, approval gates for issuing refunds, and review-and-learn for tracking quality over time.
Common mistakes
Making escalation feel like failure. If the agent apologizes when it escalates (“I’m sorry, I can’t help with that”), users lose confidence. Instead, frame escalation positively: “This question needs a specialist. I’m connecting you with one now, and I’ve prepared a summary of your situation so they can help quickly.”
Forgetting to measure escalation rates. If 60% of queries escalate to humans, you don’t have an agent; you have a triage system. Track escalation rates by category and set targets for reducing them through agent improvements.
Not providing context to human reviewers. Escalating a bare query without the agent’s reasoning, attempted responses, and retrieved context forces the human to start from scratch. Always pass the full agent state when escalating.
Setting static thresholds. A confidence threshold that worked at launch may be wrong six months later as the agent’s capabilities change. Review and adjust thresholds quarterly based on escalation outcome data.
Frequently asked questions
What escalation rate should I target?
Start with 20-30% escalation for new deployments and reduce over time. Mature systems typically reach 5-15% escalation rates. The target depends on your domain; medical systems will always have higher escalation rates than FAQ bots.
How do I prevent human reviewers from becoming bottlenecks?
Three strategies: batch non-urgent reviews into scheduled review sessions, auto-approve categories where human agreement with the agent exceeds 95%, and staff the review queue based on measured demand rather than estimates.
Can I combine multiple patterns in one agent?
Yes, and you should. Most production agents use approval gates for high-risk actions, confidence-based escalation for uncertain responses, and review-and-learn for ongoing quality improvement. The patterns complement each other.
For the verification methodology behind these patterns, read our complete guide to AI agent verification. For design pattern fundamentals, see our agent design patterns guide. Subscribe to the newsletter for agent design patterns, deployment guides, and production engineering tutorials.