Preventing Algorithmic Bias

Algorithmic bias occurs when an information system is skewed for or against certain groups. Typically, this stems from erroneous or unrepresentative data being fed into an algorithm. 

For example, when one technology company programmed an artificial intelligence (AI) recruiting tool to give preference to resumes that resembled those of the company’s current employees, the tool did as it was told, seeking out and favoring the patterns, words, and profiles that occurred in those resumes. But, because a majority of current employees were males, the tool unintentionally produced an algorithmic bias against women and other underrepresented groups, reinforcing imbalanced power structures. 

Not all instances of algorithmic bias are cause for serious concern (consider the robot that recognizes only sneakers as being shoes), but in situations where the stakes are high, it can result in significant harm. Used in criminal sentencing, flawed AI can perpetuate in racial bias. In fraud detection, biased AI can associate first names from other cultures with fraudulent accounts. Or, as above, biased AI recruiting tools can fail to consider entire segments of the population for certain jobs. 

With lives, livelihoods, and reputations on the line, and issues of data integrity, consent, and security in play, algorithmic bias represents a serious threat to personal privacy. 

Considering all of the above, it’s frightening to imagine the negative impact that algorithmic bias could have as AI is more broadly adopted in the government space. For example, AI used to secure the United States power grid could potentially blackout certain demographics, or AI used to detect fraud in federal assistance programs could potentially exclude valid recipients.   

Unfortunately, it’s vexingly difficult to know that a data set is biased until you notice the AI is failing certain groups. For example, by the time banks or regulators realize a rate-calculation algorithm is biased against certain races or other groups, individual consumers may already be facing emotional, financial, and reputational harms based on their unfairly determined high-interest mortgage.

Although some international efforts, such as European Union (EU) General Data Protection Regulation (GDPR) Article 22, regulate and add consumer control to automated decision making, few official controls have been implemented here in the United States. However, at the time of this writing, there is a new bill in the U.S. Senate that would require companies to audit algorithms for bias. 

The National Security Telecommunications Advisory Committee (NSTAC) Report to the President on a Cybersecurity Moonshot recommends solving society’s toughest cybersecurity issues through "all-American-hands-on-deck-style" calls to action called “Grand Challenges.” With its high stakes for privacy and society, algorithmic bias could be tackled through this lens of collective ingenuity.

From a privacy perspective, the following components should comprise the toolkit used to reduce algorithmic bias:

1. Data set criteria: Criteria for ensuring accuracy, integrity, and representation in data sets
2. Production outputs: Markers that can detect algorithmic bias in production environments
3. Transparency: Greater transparency into AI decision-making processes to be able to detect biases earlier on
4. Notice and choice: Rules for providing notice and choice to people when they are subject to automatic decision making, such as GDPR Article 22
5. Security: Requirements for protecting and securing personally identifiable information in data sets
6. Augmented intelligence: Recommendations for human oversight without introducing author bias (control over augmented intelligence)
7. Official standards: Official standards for AI development, including self-regulatory codes and standards by the National Institute of Standards and Technology (NIST).

Using privacy metrics, we can gauge whether bias is removed from algorithms and whether algorithms are working for our privacy instead of against it. A dynamic strategy must include proactive governance as well as checks and balances. Here are four key privacy metrics your organization should keep in mind:

Equal outcomes.

Algorithms produce equal outcomes for all genders, races, nationalities, and other groups that may be adversely impacted by algorithmic bias.

Data protection.

AI developers receive algorithmic bias training and, where applicable, certification. Breaches are reduced by a measurable percentage.

Transparency.

Americans know how their data is used and where it resides. Privacy statements include details about AI and automated decision making. Where applicable, consumers can consent to automated decision making or request evidence that algorithms are free from bias. 

Trust.

When polled, Americans from diverse backgrounds say they are confident in their privacy and security in information systems that use AI.