How to Design Human-in-the-Loop AI Workflows

Most AI projects that stall don't fail because the model is bad. They fail because nobody decided what happens when the model is unsure, wrong, or about to do something expensive. "Human-in-the-loop" is the answer people reach for, but it's often left as a vague promise that someone will "check the output."

That isn't a design. A useful human-in-the-loop workflow says exactly where a person steps in, what they see when they do, what they're allowed to change, and what happens to the work if they don't respond. This is how we approach that when we build AI systems for clients.

Start with the decision, not the model

Before choosing a model or writing a prompt, write down the decision the workflow makes. Not "process invoices," but something like: "Approve, query, or reject a supplier invoice against its purchase order."

Then ask three questions about that decision:

1. What does a wrong answer cost? A typo in a product description is cheap to fix. Paying a duplicate invoice, or rejecting a valid insurance claim, isn't.
2. Can it be undone? Drafting an email is reversible until it's sent. Sending it isn't.
3. Who is accountable today? Someone in the business already owns this decision. They should own it after AI arrives, too.

The answers tell you how much human involvement the step needs. Cheap, reversible decisions usually run automatically with sampling. Expensive or irreversible ones need a person to approve them, at least until the system has earned trust.

Place approval points deliberately

An approval point is a place in the workflow where the system stops and waits for a person. Too few and you're trusting the model with decisions it shouldn't make. Too many and you've rebuilt the manual process with extra steps.

We usually sort each step into one of four modes:

Mode	What the AI does	What the person does	Good for
Assist	Drafts or suggests	Makes the decision	High-stakes, judgement-heavy work
Approve	Proposes an action	Confirms or edits before it happens	Irreversible actions: payments, sending, publishing
Exception	Acts on clear cases, routes the rest	Handles only what's flagged	High-volume work with a clear "normal"
Audit	Acts automatically	Reviews a sample after the fact	Low-cost, reversible, well-measured steps

Most workflows mix modes. Reading a document might run in exception mode, while the payment it leads to stays in approve mode permanently.

Two rules make this easier:

• Put approval before the irreversible step, not after it. Review the email before it's sent, the payment before it's released, the record before it's overwritten.
• Approve the action, not the reasoning. Show the reviewer exactly what will happen ("Pay invoice 2291 to Supplier X, matched to PO 1182") rather than asking them to read the model's explanation and infer it.

Decide what the AI is allowed to touch

Permissions are where human-in-the-loop design meets ordinary software security. They're also where teams most often cut corners. The model should have the narrowest access that still lets it do the job.

• Read and write separately. A step that extracts data from a document needs to read it. It doesn't need write access to your ERP. Let a separate, deterministic step write the approved result.
• Give the AI the same limits as a junior employee. If a new staff member can't approve spend over a threshold, neither can the model. Reuse the approval limits the business already has.
• Keep actions behind your own code. The model proposes; your application validates the proposal against business rules, and only then calls the system that acts. This stops a confused or manipulated model from doing something your rules wouldn't allow.
• Treat inputs as untrusted. Emails, uploaded documents, and web pages can contain text written to mislead a model. Never let content in a document grant the system new permissions.

Design for exceptions from day one

The value of an AI workflow is in how cleanly it separates the routine from the unusual. That only works if exceptions are designed as carefully as the happy path.

Define what "unsure" means. Confidence scores from a model are a starting point, not a rule. Combine them with checks you control: does the extracted total match the line items? Does the supplier exist? Is the amount within the usual range for this customer? A case that fails any check goes to a person, however confident the model claims to be.

Route with context. When something is flagged, the reviewer should see the original document, what the system extracted, which check failed, and what it recommends. If they have to open three other systems to understand the case, the exception queue becomes the new bottleneck.

Give every exception an owner and a deadline. Decide who handles each type of exception, and what happens if nobody does. "Sits in a queue forever" isn't an acceptable answer. Escalate, fall back to the manual process, or hold the transaction, but decide upfront.

Make corrections count. When a reviewer fixes an extraction or overrides a decision, record what they changed and why. Those corrections are your best source of test cases, and over time they show you where the system can be trusted with more.

Log everything a person would need to explain later

Six months from now, someone will ask why a claim was rejected or a payment was made. You need to be able to answer without guessing. For every case, keep:

• The input the system received
• What the model produced, and which model and prompt version produced it
• Which checks passed or failed
• Who approved, edited, or rejected it, and when
• The final action taken

This is also what makes the system auditable for compliance teams, and it's what lets you measure accuracy honestly rather than relying on how the demo felt. If you're working out which of your AI uses the EU AI Act treats as high-risk, our seven-step guide for small businesses is a good place to start.

A worked example: checking claims against contracts

Here's how these pieces fit together in a workflow like one we built for a logistics client, where claims had to be checked against contract terms and policies before they were paid. Before AI, every claim was checked by hand. It was slow, and revenue slipped through the gaps.

1. Intake (automatic). A claim arrives by email or upload. The system stores the original and creates a case.
2. Extraction (exception mode). Document AI reads the claim: claimant, shipment reference, dates, amounts, and the reason given. Fields that fail validation, or that the model is unsure of, are flagged.
3. Contract matching (automatic, deterministic). The system looks up the shipment and the contract that applies to it. This step is ordinary code, not AI. It either finds the contract or it doesn't.
4. Assessment (assist mode). The AI compares the claim with the relevant contract clauses and liability limits, and writes a short recommendation with the clauses it relied on.
5. Routing. Claims that match the contract cleanly and fall under a value threshold go to a fast-track queue. Anything that conflicts with the contract, exceeds the threshold, or failed an earlier check goes to an assessor, with the document, extracted fields, matched clauses, and recommendation on one screen.
6. Decision (approve mode, always). A person approves, adjusts, or rejects every payout. The AI never releases money.
7. Feedback. Every edit and override is logged and fed back into the test set used to evaluate changes to the system.

As a result, assessors focus on claims that need judgement instead of re-reading the ones that don't. The business resolves claims faster with less leakage. The human decision didn't disappear. It moved to where it matters.

Earn autonomy, don't assume it

The right amount of human involvement changes over time. Start with more checkpoints than you think you need, measure how often people agree with the system, and relax a step only when the data says it's safe. Moving a step from "approve" to "exception," or from "exception" to "audit," should be a deliberate decision someone signs off on, not a byproduct of reviewers clicking "approve" without looking.

If a reviewer approves 500 cases a day, they've stopped reviewing. That's a sign the step should either be automated with sampling, or redesigned so the person sees fewer, harder cases.

Where to start

Pick one workflow with clear volume and a clear owner. Write down the decision it makes, map each step to a mode, and define your exception rules before you write a prompt. Then prove it on a few weeks of real data before building around it.

That's the shape of how we run AI consulting engagements: an assessment to find the workflows worth automating, a proof of concept on your own data, and a production build with the approval points, permissions, and audit trail designed in from the start. If you'd like to talk through a workflow you're considering, get in touch.