Amazon built an AI hiring tool, trained it on a decade of hiring decisions, and scrapped it when it systematically downgraded CVs from women. Most agencies don’t have Amazon’s audit resources. The Equality Act does not care about resources — it cares about outcomes.
Amazon built an AI recruitment tool, trained it on a decade of hiring decisions, and eventually scrapped it because it systematically downgraded CVs from women. The bias was not in the algorithm. It was in the historical data.
Amazon had the resources to audit the tool, identify the bias, and make the decision to discard it. Most recruitment agencies and HR teams deploying AI hiring tools do not have those resources. And the Equality Act does not care about resources — it cares about outcomes.
If your AI screening tool is producing systematically different outcomes for candidates from protected groups, you have an indirect discrimination problem. The fact that a third-party AI made the initial decision is not a defence. The responsibility sits with the employer or agency deploying the tool.
The most widely deployed AI in recruitment — tools that score, rank, or filter CVs before a human sees them — are often the least governed. Agencies know the tool saves time. They often do not know what factors it weights, what data it was trained on, or whether it has been audited for bias across protected characteristics.
Video interview analysis tools that assess facial expressions, tone of voice, word choice, and body language are in active use. They are also deeply problematic. Facial analysis AI performs worse on darker skin tones. Tone and vocabulary analysis disadvantages non-native speakers and neurodivergent candidates.
Candidate data is personal data. AI tools that process it must have a lawful basis. Automated decision-making that significantly affects individuals (including shortlisting) is restricted under GDPR Article 22: candidates have the right not to be subject to solely automated decisions and the right to an explanation. Most agencies are not compliant.
Agency-used AI platforms that aggregate data across many clients may be trained on patterns that encode sector-wide hiring biases. When you use that tool, you inherit those biases. Understanding this is part of responsible procurement.
Equality Act 2010 and AI: direct and indirect discrimination, the group disadvantage analysis. GDPR Article 22 and automated decision-making in hiring. ICO enforcement priorities in employment AI. Where agencies have already faced action.
We work through your current AI hiring tools — applicant tracking systems with AI features, CV screening, video interview analysis, assessment tools — and assess each for bias risk, GDPR compliance, and Equality Act exposure.
What questions to ask AI vendors about bias testing, training data, and audit capability. How to conduct a basic adverse impact analysis on your own shortlisting outcomes. What a contractual commitment on bias auditing should look like.
Lawful basis for AI processing of candidate data. Automated decision-making obligations. Privacy notice obligations. Data retention in AI recruitment platforms. Subject access requests in an AI context.
Drafting your AI recruitment policy. Human oversight requirements. Candidate transparency obligations. Staff training for recruiters using AI tools.
| Use case | Efficiency gain | Primary risk |
|---|---|---|
| CV screening and scoring | Volume handling, speed | Equality Act bias, GDPR Art 22 |
| Video interview AI analysis | Shortlisting speed | Facial analysis bias |
| Job description generation | Admin efficiency | Gendered language, exclusionary framing |
| Psychometric and assessment AI | Standardisation | Neurodivergent disadvantage, validity |
| Candidate matching algorithms | Reach, speed | Bias encoding, filter bubble |
| Reference checking AI | Efficiency | Accuracy, data handling |
We assess where you stand against your sector’s regulatory floor and identify your highest-priority governance gaps.
One hour with leadership. Sector-specific regulatory framework, immediate priority actions, the language to take this to the wider team.
Half or full day. Full risk mapping for your sector, governance protocols, AI use policy draft, and 30 days advisory access. The flagship engagement.
Multi-site, network, and group pricing available on request.
Specialist vendors have often done more work on bias than general-purpose AI vendors. But “done more work” is not the same as “resolved the problem.” You should ask your vendor specifically: what protected characteristics has the tool been audited against, what was the methodology, when was the last audit, what were the results? If they cannot answer those specifically, your governance gap is at the procurement level.
It depends on how the human is involved. Article 22 restricts solely automated decisions, but if a human is rubber-stamping an AI shortlist without meaningful review of the underlying scoring, the ICO may take the view that the decision is effectively automated. The human must have the information and the mandate to override the AI outcome, and this must be documented.
You can, and increasingly agencies are. But disclosure policies work both ways: if you require candidates to disclose AI use, candidates can reasonably ask which parts of your process use AI, how it affects their application, and what rights they have regarding automated processing. Having a governance framework in place before you implement a disclosure policy is the right order.
Tell us which AI tools you’re using in your hiring process and we’ll give you an honest read on your Equality Act and GDPR exposure. No sales pressure. If your governance is sound, we’ll tell you.