When AI Hiring Goes Wrong

A Qualified Candidate Who Never Got Seen

You are a 56-year-old teacher with decades of experience when you apply for a job in one of the online education platform where you have complete qualifications as stated by the website no human has ever read your application at all but the system has automatically rejected you just because you are a woman over 55 years old this is the real incident that happened to iTutorGroup in 2023 and is considered the lightest case in this article The content will show the consequences when the promise of AI hiring tools promised to remove human bias from recruitment face the reality is mentioned through 4 cases from 2018 to 2024 which has been escalating since the cases escalate from internal discovery to regulatory enforcement to vendor liability.

Amazon's Recruiting Tool (2018)

The company has developed an in-house artificial intelligence tool trained on a decade of resumes. Because the majority of engineering workers are men, the algorithm learned that "male" predicted success, which is not due to superior skills, but because of such past employment history data. This issue could have been prevented by a pre-deployment demographic parity test, which is required under the EU AI Act for high-risk employment AI. The failure occurred because the training data reflected historical hiring patterns rather than actual candidate quality, lacking both human oversight and bias testing. When engineers attempted to fix the bias, they were unable to ensure neutrality and ultimately abandoned the project before it was launched. The system penalized resumes containing the word "women's", such as "women's chess club captain". In addition, the system also downgraded graduates from two all-women's colleges.

HireVue's Video Assessments (2019)

HireVue has built platforms at more than 700 companies such as Goldman Sachs and Unilever, used to evaluate job applicants through video interviews by system that will analyze facial expressions, word choice and speech patterns. The Electronic Privacy Information Center filed a complaint reporting that results were biased and not replicable. Research found accuracy gaps that disadvantaged non-white applicants and deaf candidates. The study found that although facial identification was dropped in 2021, there is still ongoing speech pattern analysis. There was no fairness review from outside agencies prior to deployment at scale. Speech scoring penalized accents, speech impediments, and non-native speakers, and the model was entirely opaque. The EU AI Act requires employment screening tools to have documented risk assessment, human oversight, and post-deployment monitoring. The OECD AI Principles would have required transparency so candidates understand how they are evaluated.

iTutorGroup - The First EEOC AI Settlement (2023)

In August 2023, the Equal Employment Opportunity Commission reached a settlement with iTutorGroup because the application scoring program had rejected applicants who were women aged 55 and over and men aged 60 and over, with more than 200 highly qualified applicants being disqualified solely because of the age criterion, which is an automatic rejection rule, not age consideration along with other factors. This issue could have been prevented by a simple input audit to review the system's filtering rules, which would have immediately identified the age-based cutoffs. The settlement totaled $365,000, establishing a precedent that the EEOC views AI tools as subject to the same anti-discrimination laws as any other practice, meaning 'the algorithm did it' is not a valid defense. Candidates who met every qualification had never had a chance to talk to a human agent at all.

Workday - When the Vendor Is Liable (2024)

In February 2023 class action was filed against Workday alleging its automated resume screening discriminated on race, age, and disability. Later in July 2024 a California federal court allowed the case to proceed, which was considered a significant change since liability previously sat with the employer - the company that chose to use the tool, but this ruling suggests AI vendors carry employer-equivalent liability if you create a faulty system. You can't deny responsibility by claiming that we just sold the software. To prevent this, third-party algorithmic audits could identify discriminatory patterns before deployment; had Workday published its audit results, the bias would have been clear. The EEOC has filed an amicus brief in support of the plaintiff, signaling a move toward federal enforcement that holds vendors accountable.

The Pattern

The escalation is unmistakable:

1. Amazon (2018) - a company that caught its own bias, resulting in the project being terminated
2. HireVue (2019) - an external advocacy group files a complaint, causing the company to drop some features but retaining others
3. iTutorGroup (2023) - a federal regulator settles the first-ever AI discrimination case
4. Workday (2024) - a court ruling the AI vendor can be held directly liable, not just the employer

Enforcement is tightening. Liability is expanding. "We didn't know the algorithm was biased" is becoming legally indefensible.

The Audit Checklist

All four cases share the same root failure: deployment without governance. Here is what a basic AI hiring audit covers:

Pre-Deployment

Post-Deployment

Governance, Not Just Engineering

Human hiring is biased too, the interviewer will often go in the direction of someone similar to themselves, graduate from the same institution or have the same cultural background, which artificial intelligence should help solve the problem, but automation doesn't equal objectivity, algorithms trained on biased data can't eliminate bias, but it scales it through a system that caters to a large number of applicants at once, For those new to governance, check out Getting Started in AI Ethics Policy, and for a deeper look at how AI works technically, read Running AI Locally Changed How I Think. If you are using AI for hiring, you must have an audit framework in place before deployment to avoid lawsuits. These four cases show where that lack of policy leads. Building the model is engineering, but deciding how to deploy it is policy, and most companies are doing the first without the second. so the answer is not to eliminate artificial intelligence in the recruitment process, but to control, monitor and govern, examine, and audit it, and hold someone accountable when it fails.

---

Next in this series: an AI ethics policy audit template you can apply to any AI system making decisions about people.

References

• Why Amazon's Automated Hiring Tool Discriminated Against Women - ACLU
• EPIC Complaint Against HireVue - Electronic Privacy Information Center
• iTutorGroup EEOC Settlement - U.S. Equal Employment Opportunity Commission
• Mobley v. Workday: AI Vendors and Employment Discrimination Liability - Fisher Phillips
• EU AI Act Implementation Timeline - EU Artificial Intelligence Act
• OECD AI Principles - OECD