Where in the Loop? Testing AI Across 120 Compliance Tasks to Find Out Where Humans Are Most Needed
Oversight should sit at decision points where errors carry the highest ethical or legal weight, not on every output As compliance teams experiment with AI for everything from risk assessments to policy interpretation, a practical question emerges: Which tasks can be automated reliably, and which still require human judgment? Steph Holmes, director of compliance and ethics strategy at EQS Group, dives into her organization’s research on six AI models, finding that it excels at rule-driven work but struggles in the gray zone where data meets intent and culture intersects with language. The findings suggest oversight should be strategic, not universal, but there’s no doubt the loop isn’t complete without humans. When people talk about responsible AI, the concept of “human in the loop” tends to surface quickly, and for good reason. It sounds reassuring, almost self-evident. But as compliance teams start experimenting with AI, a practical question emerges: Where in the loo...