The most underrated AI investment: Building the humans in the loop

Most AI regulations assume that a capable human is overseeing the system. Yet few organisations  are investing in that human. At the same time, AI is removing much of the work through which such judgment was traditionally built. In my conversations with leadership teams, the AI agenda usually focuses on use cases, ROI and productivity. Human-in-the-loop often appears at the end, as an assurance that someone will review the output. But regulations and governance frameworks, including the EU AI Act and the RBI's FREE-AI framework, depend on that oversight being effective. Far less attention is being given to whether the people expected to provide it are ready.

Why the Human Layer Is Under Strain

The apprenticeship model is weakening: Credit analysts and claims specialists built judgment by doing the work, making mistakes and being corrected. This junior work is often the first to be automated. We are therefore creating a real paradox: AI needs experienced people to oversee it, while removing the pathway that produced that experience.

AI multiplies decision volumes: A team that reviewed 50 decisions a day may soon face 500. At that volume, review can quietly become approval, without anyone consciously deciding that it should.

Better AI can reduce reviewer attention: As systems become more accurate, reviewers naturally become less vigilant. The rare error can then arrive precisely when people are least prepared to spot it.

AI errors are harder to detect: AI errors are often well written and confidently presented. They do not carry the hesitation or inconsistency that might signal a junior colleague's mistake.

The reviewer role has limited appeal: Reviewer roles are often seen as offering limited career value. That makes it harder to attract the people whose judgment the role depends on.

What Effective Human Oversight Requires

Human-in-the-loop cannot compensate for weak system design. Nor can reviewers catch errors they have not been trained to recognise. Effective oversight requires clear authority to pause, challenge or escalate, supported by documented reasoning. It should be one layer of a broader oversight architecture, not the architecture itself.

Building the Humans in the Loop

Organisations should focus on five areas when designing an effective human-in-the-loop model.

• Design the oversight architecture before staffing it -- First, identify where regulation requires human review and where decisions carry the highest stakes or are hardest to reverse. Put the most experienced reviewers and strongest escalation paths around those decisions. As volumes grow, automated monitoring for drift and anomalies can help direct their attention.
• Define what a good AI reviewer looks like -- Domain expertise is necessary, but not sufficient. Reviewers must understand how the system is likely to fail and examine each output for the hidden error, rather than simply confirm the obvious answer. These capabilities should be defined, assessed and developed like any other competency.
• Rebuild the apprenticeship deliberately -- Learning that once happened naturally must now be designed. This can include rotations into hands-on work, shadow reviews alongside AI, and case libraries with seeded errors for safe practice. Experienced professionals whose roles are being reshaped by AI are a natural first pool of reviewers. Redeploying them also preserves institutional knowledge that might otherwise be lost.
• Make reviewing a career, not a checkpoint -- Credentials, visibility and a clear path into AI governance, risk, product and architecture roles can change who applies. People who understand both the business domain and the behaviour of AI systems will be valuable leaders in an AI-led organisation. The reviewer role can be a credible place to develop them.
• Measure how oversight works in practice -- Alongside model accuracy, track how often reviewers intervene and whether similar cases are handled consistently. Used developmentally rather than punitively, these insights can show where reviewers need more support, training or authority.

The Investment Worth Making

Organisations are investing heavily in models, platforms and use cases, but far less in the people expected to keep those systems accountable. The gap may not appear in an ROI calculation. It becomes visible when a confidently presented error passes through a review step that was never designed to catch it. The organisations that scale AI with confidence will be those that invest early in building the humans in the loop. A useful starting point for any leadership team is simple: for each AI system in use today, who is the human in the loop, how were they prepared for the role, and who will replace them?

{Views are personal; the author is Partner and Leader, Data & AI Practice (Digital Lighthouse) at KPMG in India}