How Do We Navigate Algorithm Bias in Decision-Making Systems?

Continuing my reading on System Error: Where Big Tech Went Wrong and How We Can Reboot, the chapter, “Disaggregating the Technologies,” dives into the world of algorithmic bias with algorithmic decision-making technologies.

Currently, technologists have been employing algorithmic decision-making in various fields, such as Amazon’s hiring process, where algorithms review applicant resumes and deem whether they are fit for interviews and consideration for jobs, or even in criminal justice fields, where algorithms evaluate whether or not defendants are high-risk or low-risk of reoffending. It can be understood that these processes tend to favor certain majority groups over other minority groups to a disproportionate extent, with Amazon hiring algorithms favoring white male candidates and COMPAS algorithms (risk assessors) giving low ratings to Black defendants who are deemed unlikely to show up on their court date and likely to reoffend, affecting their plea for probation.

These algorithms bring up questions of how these algorithms should be implemented and regulated: what is deemed a “fair decision” for algorithmic decision-making, and how do we ensure validity in algorithms for users? These questions are imperative to creating a society where the separation of optimization and innovation does not overshadow the importance of equity, especially for marginalized communities.

In order to understand fairness, we must think about the difference between equality versus equity and how automated algorithms play a role in this. Through the years, we have furthered our understanding on the importance of equality and equity, especially in the sectors that algorithms are employed in. One example of this debate of equity versus equality in the Amazon hiring process is explained by the authors — do we ensure fairness for individuals by giving all applicants with “identical skills and experiences” the same predictive scores? Do we ensure fairness for groups by ensuring similar percentages of minority-group applicants as majority-group applicants to start off hiring from a level playing field? This consideration of equity and equality also complicates the technical mechanisms of these algorithms as technologists must consciously code for these controls. In order to ensure equity, these algorithms must include a sort of measurement to support some groups more than others, and even that discounts certain sociocultural perspectives and systemic structures. When understanding both equality and equity in the context of algorithmic decision-making, we must be aware of these perspectives that cause the need for equity in the first place as oftentimes, technologists lack the ability to convert these controls into code or even lack awareness of these perspectives overall.

In order to ensure validity and trust in these algorithms by humans, the authors pose three “key ingredients” for governance in algorithmic decision-making. The first, transparency, would inform users that algorithmic decision-making systems are in place and also how they are being used in the specific sector. Humans would be fully aware of the use and be able to deem whether or not results are valid. The second, auditability, would encourage for the physical code used in an algorithm to be made public. While many people would be unable to decipher the code, it will lead to solidified trust that these companies are transparent with the technologies used for those who are able to decipher it and find any issues necessary. This leads to the last “ingredient,” the commitment to due process. Individuals must be able to protest any results of these algorithms and call for evaluation on the validity of its results.

Algorithmic decision-making is a new technology that has proven to be beneficial in efficiency and optimization in countless sectors, predicting health concerns or allowing for helpful targeted ads. However, when they are used in sectors that inevitably have a large impact on one’s life trajectory, such as in the hiring process or criminal justice progress, many issues are brought to the light regarding equity and sociocultural perspective’s role in determining validity. As these technologies continue to develop and innovate further, technologists must collaborate with more humanistic experts to find a middle ground on optimization and equity.

Cover image by Kasia Bojanowska on Dribbble.