Amazon has recently abandoned an AI recruiting tool that was biased towards men seeking technical jobs (Snyder 2018). Since 2014, an Amazon team had been developing an automated system that reviewed job applicant resumes. Amazon’s system successfully taught itself to demote resumes with the word “women’s” in them and to give lower scores to graduates of various women’s colleges. Meanwhile, it decided that words such as “executed” and “captured,” which are apparently used more often in the resumes of male engineers, suggested that a candidate should be ranked more highly (ibid). The system’s gender bias failures originate from the biased training data set that it was given to learn from: Amazon’s workforce is unmistakably male-dominated, and thus the algorithm sought after resumes with characteristically male attributes. The team attempted to remove the gender bias from the system but ultimately decided to ditch the program entirely in 2017 (ibid.).
