What can the human in the loop actually do?
In my previous article, I asked what judgement comes with the work we delegate to AI. A questionnaire can leave out an important issue. A report can give a finding too little attention. Both can look ready to use.
The reassuring answer is that a human will check the work. I want to stay with that answer for a moment, because it leaves quite a lot unresolved.
We might mean someone will read the output, check it against evidence, authorise its use or remain able to stop the system. Those are different responsibilities. Agreeing that there will be “a human in the loop” tells us very little about which of them we have actually provided for.
Start with the work someone has to do
Imagine a team using generative AI to translate a notice about a service. A colleague reads the translation, makes a few changes and sends it back. The manager records that it has been checked and approves publication.
Everyone has done something. But what did “checked” mean?
The colleague may have checked whether the translation reads naturally. The manager may have understood that to mean the eligibility conditions were compared with the approved source. Nobody may have established whether the source notice was current, or whether the wording would make sense to the people expected to act on it.
For someone reading that notice, the consequence could be a wasted trip, an unexpected document requirement or missing a service they were entitled to use. A completed review field tells us little about whether those risks were examined.
Language specialists already have ways of checking meaning and comprehension. AI-assisted translation needs to fit into that work, with access to the source, appropriate language and local knowledge, and opportunities for intended readers to identify misunderstandings. Fluency alone can't establish that a notice is safe to send.
Four terms that help us be more specific
This is one reason I've been building the Humanitarian AI Wiki alongside AidGPT. In teaching and discussions, we can agree on a phrase while carrying quite different ideas about what it requires.
The Wiki offers working definitions that separate four responsibilities.
Human review means examining an AI output or process against stated criteria. For our notice, the reviewer needs to know whether they're checking language, accuracy, suitability or all three.
Verification means checking a consequential claim or output against evidence independent of that output. The eligibility conditions need comparison with the current approved source. Asking the AI whether its translation is correct doesn't provide that independent evidence.
Human approval means recorded authorisation by someone with the authority to let a specified use or action proceed. The person checking the translation may have the expertise to identify a problem without having authority to release the notice.
Human oversight concerns the arrangements that let responsible people observe, challenge, intervene in or stop the use while it operates. For the notice, that includes responding when readers report a problem and correcting or withdrawing what has been published.
Review and verification establish what was checked. Approval records permission to proceed. Oversight keeps intervention possible afterwards. A small team may share these responsibilities among very few people, but it still needs to distinguish them.
Having a safeguard and being able to use it
This question also came up in our work for the Digital Convergence Initiative's AI Hub for Social Protection.
In the global evidence review that Avril James and I prepared, descriptions of human oversight often told us little about how it worked in practice. Published accounts rarely reported when reviewers overrode a result or what enabled them to challenge it.
One case was Samagra Vedika in Telangana, India, which linked government records to support welfare eligibility checks. Amnesty's technical explainer describes reports of officials relying on algorithmic decisions and refusing to overturn them even when people presented contrary evidence.
The companion risk assessment framework, which I co-authored with Simona Dobre and Laura van der Erve, examines the gap between planned safeguards and their application. Its discussion of this case identifies institutional pressures and capacity constraints alongside failures of meaningful review.
That leaves a further question about whether oversight remains workable as use grows.
If a pilot depends on careful checking by a small team, what happens when the volume grows and that checking becomes another responsibility alongside everything else?
A person can remain in the approval process while having less time to examine evidence, investigate a disagreement or explain why a result should change. The system can keep recording human approval while the conditions for meaningful review become weaker.
The person needs the means to disagree
Time is one part of this. Confidence in the system can also affect what a reviewer notices. Article 14 of the EU AI Act, addressing high-risk AI systems, explicitly recognises automation bias: the tendency to rely automatically or excessively on the output. It connects oversight with the ability to understand, interpret, override and stop a system.
For our translation example, I would want the reviewer to compare consequential statements with the source before accepting the draft as a whole. They need a way to record unresolved concerns and prevent release while those concerns are examined. A confident answer from the model shouldn't settle a disagreement about the evidence.
Authority matters too. A community interpreter or national colleague may recognise a problem that the approving manager cannot see. If their concern can be dismissed without examination, their involvement gives us little protection. The review process needs to explain how a challenge changes what happens next.
The people expected to use the service also bring knowledge the team may lack. The Wiki's affected-population entry draws on the Humanitarian Encyclopedia's account of different uses of the term, including people as responders and decision-makers. For this task, that means involving intended readers in checking comprehension and keeping an accessible route for feedback and correction after publication.
In a one-person team, the same person may carry out several checks. They can still record the source used, the comparison made and the uncertainty remaining. Where the task needs expertise or separate authorisation they don't have, that is a reason to obtain support or pause the use.
Someone has to pay for the checking
I don't think we can discuss human oversight honestly without discussing its cost.
Review takes time. Verification needs accessible evidence and relevant expertise. Investigating a complaint takes work that won't necessarily appear in the estimate of how quickly the AI can produce an answer.
If an organisation counts the time saved in drafting but leaves checking as an extra task for already stretched staff, it has left part of the operating cost out of the calculation. Some of that cost may then fall on the people trying to correct a decision or understand a notice.
I don't have a universal answer for how much review every task needs. The consequences of error, the available evidence and the ability to correct a mistake all matter. But I think we should test the work arriving, the time proper checking takes and what happens when the queue becomes unmanageable. That needs revisiting as use grows.
The definitions need to be open to challenge too
The Wiki is a working draft for discussion. Its method page distinguishes quoted definitions, adaptations and AidGPT working wording, drawing on terminology methods including ISO 704. A source citation shows where an idea came from. It doesn't mean the source organisation approved our wording.
There is already useful work to build on. CDAC Network's SAFE AI glossary provides plain-language definitions to help humanitarian and technical teams work together. The Wiki's oversight and approval entries draw on the SAFE AI framework alongside other sources. A full comparison of the two collections remains to be done, and I'd welcome collaboration on that.
What I'd like the Wiki to contribute is a way to follow these terms through an actual task. It should help a team say what it checked, what remains uncertain, who can approve the use and how people can challenge what follows. If the wording fails that test, it needs improving.
Try it on one piece of work
Take a notice, assessment, report or decision your team has used AI to help produce. Write down what the person reviewing it actually did. Include the evidence they checked, the concerns they raised, who authorised its use and how a problem could be corrected afterwards.
Then ask whether the person had the knowledge, time and authority to do what the process expected of them.
That's a practical place to begin. It's also the kind of judgement we practise through AidGPT, where the task and the evidence give us something concrete to discuss.
Please read the Wiki, test it against your own work and tell me where the distinctions help or break down. I'd particularly welcome experience from people doing this checking under real operational pressure.
Before we reassure ourselves that a human is in the loop, we should be able to explain what that person can actually do.
Tom
Thomas Byrnes leads AidGPT and MarketImpact Digital Solutions, which provides training and advice on responsible AI use in humanitarian and development work.
How this was written: I used AI tools to help research public sources, develop the structure, draft and revise this article, and check references. I set the direction and directed revisions, and I am responsible for the argument. Source and link checks were AI-assisted. The service-notice example is illustrative. The Telangana discussion draws on the linked public sources. No confidential client material or beneficiary-level data was used to draft this article. If you identify an error, tell me and I will correct it publicly. Read how we use AI at MarketImpact.