Having a human involved in an AI-supported decision doesn't guarantee meaningful human oversight. If AI makes the first recommendation, it can influence the person's judgement before they form their own view. The important question is therefore not simply whether a human is involved, but when they enter the process and whether they can make an independent judgement.
When organisations talk about responsible AI, one principle comes up again and again: keep a human in the loop. The idea makes sense. AI can analyse, recommend and automate, but when a decision really matters, a person remains responsible for the final call.
Problem solved? Perhaps not.
A discussion about AI and recruitment at Arendalsuka, Norways annual gathering for public debate and discussion on issues shaping business and society,raised a more uncomfortable possibility: what if the human is in the loop, but the AI has already influenced their judgement? That question matters far beyond recruitment. As AI becomes part of more decisions at work, organisations may need to think differently about what meaningful human oversight actually looks like.
The problem with going second
Imagine you're reviewing candidates for a role. Before opening an application, an AI system tells you that the candidate is a poor match. You then review the application yourself. Technically, you're still making the decision. You're the human in the loop. But are you approaching the application in the same way you would have if you hadn't seen the AI recommendation? That's where things get interesting.
In his introduction at Arendalsuka, Morten Goodwin, professor of artificial intelligence at the University of Agder, challenged one of the most common assumptions about responsible AI: that keeping a human in the loop is enough. The presence of a human doesn't necessarily make an AI-supported process independent or safe.
The order matters. Once we've been given a recommendation, it can become an anchor for our own judgement. Instead of independently assessing the information, we risk evaluating whether we agree with the AI. That's a very different task.
Human oversight can become human confirmation
This is particularly relevant because AI systems often present themselves with an appearance of authority. A colleague can say, “I have a feeling this candidate isn't right.” We know that's a subjective judgement. We might ask why. An AI system can produce a score, ranking or neatly written summary. That output can feel more objective because it appears to be based on data. But a confident-looking output isn't the same thing as an objective one. AI systems can reproduce patterns and biases found in the data and environments they learn from. That's increasingly well understood.
The less discussed risk is what happens after the AI gives us its answer. If people place too much confidence in that answer, “human oversight” can quietly turn into human confirmation. And that's particularly concerning when AI is being used to support decisions about people.
Recruitment makes the problem easy to see
Recruitment provides a useful example because AI is increasingly present on both sides of the process. Candidates can use generative AI to write and refine applications. Recruiters can use it to create job adverts and summarise candidate information. Employers can also use AI-supported tools to match, screen or rank applicants.
At Arendalsuka, one recruiter described receiving four applications for the same role that appeared to have been generated in almost exactly the same way – right down to the final sentence and punctuation. Meanwhile, employers are experimenting with technology that can take on increasingly large parts of the recruitment process. There are clear efficiency benefits. But the further AI moves from helping us process information towards helping us decide, the more important the design of human involvement becomes. Simply adding a recruiter at the end of the process doesn't answer that question.
What does meaningful human oversight look like?
One suggestion from the Arendalsuka discussion turns the usual model around. Let the human make the initial assessment. Then bring in AI. A recruiter could evaluate a candidate before asking AI to identify information they may have missed, challenge assumptions or suggest alternative interpretations. The AI becomes a second opinion rather than the first opinion. That sounds like a small change. It isn't. In the traditional “human in the loop” model, the process can look something like this:
AI recommendation → human review → decision
A different model might be:
Human assessment → AI challenge → human reconsideration → decision
The human isn't there simply to approve or reject what the system has suggested. AI is being used to challenge human thinking rather than frame it. That could be a much more meaningful form of human oversight.
This isn't only a recruitment question
The principle becomes more important when we look beyond recruitment. AI is increasingly being used to summarise information, recommend actions, identify patterns and support decisions across organisations.
Imagine an AI system suggesting:
- which employees might leave;
- who should be considered for promotion;
- which skills an organisation lacks;
- how performance information should be interpreted;
- where resources should be allocated.
A manager can technically remain responsible for every decision. But if the AI recommendation always comes first, how independent is the manager's judgement? This is why “there's always a human involved” may be too low a bar. The better question is:
What role is the human actually playing?
From human in the loop to meaningful human judgement
None of this means AI shouldn't support decisions. AI can help people work through large amounts of information, spot things they might otherwise miss and question assumptions. Those are potentially valuable capabilities. But organisations need to think beyond simply inserting a human somewhere into an automated process. For HR leaders, that means asking more specific questions:
Does the person form an independent view before seeing the AI recommendation?
Do they understand what information the recommendation is based on?
Are they expected to challenge the system – or simply check its output?
Can they disagree with it confidently?
Can they explain why the final decision was made?
Those questions get much closer to meaningful oversight than asking whether a human was technically involved.
Keeping people in control
“Human in the loop” is an appealing principle because it sounds like a clear dividing line. AI provides the efficiency. Humans provide the judgement. In practice, that line is much less clear. If AI shapes the information we see, frames the options and gives us a recommendation before we've formed our own view, human involvement alone may not be enough to keep people meaningfully in control. Perhaps the next stage of responsible AI therefore isn't about making sure a human remains somewhere in the process. It's about designing the process so that human judgement still matters. And sometimes, that may mean letting the human go first.
About this article
This article is part of our The human side of AI series ,inspired by sessions at Arendalsuke, Norways annual gathering for public debate and discussion on issues shaping business and society.. It draws on the session “KI i rekruttering: Et mer likestilt arbeidsliv?”, with Morten Goodwin, professor of artificial intelligence at the University of Agder (UiA); Thomas Jahren, senior adviser at the Equality and Anti-Discrimination Ombud; Karoline Sandsbråten Scheide, Head of Insights and Communication at HR Norge; and Safina de Klerk, Vice President at NITO. The session was moderated by Rasha Abdallah from UiA's Centre for Gender Equality.
Source: Arendalsuka – KI i rekruttering: Et mer likestilt arbeidsliv?
