Most HR leaders know AI hiring tools "can be biased." Far fewer can explain how — and that gap is exactly what a vendor relies on in a demo. Understanding the mechanism is the difference between buying a liability and buying a tool.
Bias doesn't need a biased field
You can remove gender, age, and ethnicity from a model entirely and still get a biased system. The model learns proxies — a postcode, a university, a gap in employment, even phrasing in a CV — that correlate with the protected attribute. Strip the label, keep the signal.
The feedback loop that makes it worse
If a model is trained on who you hired and promoted historically, it learns to reproduce your past — including your past mistakes. Every cycle reinforces the pattern, and the system gets more confident, not more fair.
Questions that expose a biased tool
- What features does the model use, and which correlate with protected attributes?
- How was it tested for disparate impact, and can we see the results by group?
- What happens to candidates the model scores low — are they ever seen by a human?
- Who is accountable if a hiring decision is challenged on discrimination grounds?
If a vendor can't answer these clearly, that's your answer.