The Interview Is Broken. And Most AI Is Making It Worse.

2026-04-20 · Source: Towards AI - Medium · Field: Business & Management — Human Resources & Workforce Development, Operations & Process Management, AI in Human Resources · Depth: Intermediate, medium

Summary

The traditional unstructured job interview is a structurally flawed instrument, with its validity coefficient for predicting job performance dropping from 0.38 in 1998 to 0.19 by 2022, making it barely better than a coin toss. Inter-rater reliability is low, with interviewers disagreeing 25% of the time. Studies, such as a 2013 one by Dana, Dawes, and Peterson, show that interviews often add no predictive information, yet interviewers remain confident in their subjective judgments. While structured interviews consistently demonstrate higher validity (0.42) and inter-rater agreement, their adoption is low due to a perceived rigidity. AI-powered hiring tools, often trained on biased historical data, have compounded these issues by automating subjective pattern-matching rather than objective measurement, as seen with Amazon's biased algorithm and HireVue's discontinued facial analysis. Regulators are responding with laws like NYC's Local Law 144, requiring independent bias audits.

Key takeaway

For CTOs and HR leaders evaluating new hiring technologies, you should prioritize solutions that enforce structured measurement frameworks over those that automate subjective pattern-matching. Insist on tools that provide auditable, explainable scoring traceable to specific interview evidence, ensuring compliance with emerging regulations like NYC Local Law 144 and mitigating the significant costs of bad hires.

Key insights

Unstructured interviews are inherently unreliable, and AI often amplifies existing biases rather than improving hiring validity.

Principles

• Human judgment in interviews is inherently noisy.
• Structured interviews consistently outperform unstructured ones.
• AI trained on historical data inherits historical biases.

Method

To improve hiring, structure measurement before conversation: define criteria, ask consistent questions, score against rubrics, and calibrate scores over time.

In practice

• Implement structured interview questions.
• Use consistent scoring rubrics for all candidates.
• Conduct independent bias audits for AI hiring tools.

Topics

• Interview Validity
• Unstructured Interviews
• Structured Interviews
• AI Hiring Tools
• Algorithmic Bias

Best for: Investor, Entrepreneur, CTO, HR Professional, AI Product Manager, AI Ethicist

Related on AIssential

• AI Has Broken Hiring. Here’s How to Fix It. — Generative AI has commoditized interview performance, making authentic judgment the scarcest and most critical hiring signal.
• Karat: 71% of Hiring Leaders Say AI Is Breaking Technical Interviews — AI is fundamentally changing technical interviews, forcing companies to rethink how they assess candidate skills.
• I Built a Job-Matching Algorithm. Now I Understand Why LinkedIn Struggles. — Effective job matching requires deep psychological understanding of work and careers, not just advanced NLP or embeddings.
• When Traditional Hiring Fails, AI Solutions Become the Lifeline — AI-driven platforms can resolve hiring inefficiencies by focusing on precise candidate-employer matching rather than broad access.
• ‘Awkward and humiliating’: UK job hunters share frustration with AI interviews — AI interviews are widely adopted but frequently lead to negative candidate experiences and high abandonment rates.
• 7 Top AI Recruiting Tools to Streamline Your Hiring Process — AI recruiting tools enhance hiring efficiency and quality by automating tasks and providing data-driven insights.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Towards AI - Medium.