AI systems initially trained on images of young white males, skew searches
BUFFALO, N.Y. – Artificial intelligence is increasingly integrated into our daily lives — from performing manufacturing tasks to aiding physicians in surgery. Despite many upsides, AI has a disturbing element: bias in facial recognition technology that seeps into the criminal justice system, health care and other areas of daily life.
For example, a number of people of color have been falsely identified as suspects in criminal investigations that used facial recognition software. In some cases, this resulted in wrongful arrests and charges, including murder. And in December 2022, Apple was sued over allegations that the Apple Watch’s blood oxygen sensor was racially biased against those with a darker skin tones. Researchers found that patients of color were almost three times more likely to have dangerously low blood oxygen levels go undetected by pulse oximetry compared to that of white patients.
Ifeoma Nwogu, associate professor of computer science and co-director of graduate studies in the University at Buffalo’s Department of Computer Science and Engineering in the School of Engineering and Applied Sciences began studying AI in the early ‘90s when she was completing her coursework toward a master’s degree at the University of Pennsylvania.
Nwogu recently shared her insights on how AI has evolved through the years and her perspective on bias in facial recognition programs.
When did facial recognition technology come into the mainstream?
Facial recognition technology dates back to the 1960s but began to gain popularity in the early 1990s when government agencies like the Defense Advanced Research Projects Agency (DARPA) and the National Institute of Standards and Technology (NIST) launched a program called Face Recognition Technology. The goal was to promote the commercial facial recognition market by creating a database of facial images. Defense contractor National Security Systems became a leading proponent of such biometrics technology.
However, it wasn’t until around 2010 when Facebook took off that you really started to see facial recognition as a consumer product. With this social platform, everybody was very willing to put their faces on there. While Facebook supposedly protects your privacy, it also owns your data.
Although other biometrics signals are more accurate, face images are still the most popular as they’re easier to collect in a nonintrusive manner.
What are some factors influencing facial recognition bias?
Many face recognition algorithms boast of classification accuracy scores over 90%, but these outcomes are not universal due to a problem called unbalanced datasets. Much of standard training databases are composed predominantly of white males between the ages of 18 and 35, while many of the mugshot pictures that are being tested on are predominantly from Blacks and Latinos. So, the technology learns how to extract features from a face using large datasets of white, young males and a few females, but is then required to use them to analyze darker-skinned individuals.
When researchers in the 2018 Gender Shades study for IBM and Microsoft dug deeper into the behaviors of these algorithms across various systems, they found the lowest accuracy scores were obtained for Black female subjects between 18 and 30 years of age. NIST also conducted its own independent investigation and confirmed that face recognition technologies across 189 algorithms were indeed erroneous, especially on women of color.
What other factors contribute to the bias?
Devices have been created in ignorance of the population they are going to interact with, intentionally or not. Even the cameras we use aren’t set up to measure and identify darker skin tones. I, personally, have to edit my photos and increase the exposure so that I’m not lost to the background.
Has the technology improved in recent years?
The technology has definitely improved. First, there is now awareness of the potential dangers of such technologies. Many top-level AI conferences and journals now require their authors to explain the potential negative consequences of their proposed algorithms and how these can be mitigated.
Facial recognition technology continues to get better with time, with improved camera technologies, more diverse datasets, better feature abstraction methods, better performing machine learning techniques and processing speeds. NIST provides significantly larger and more diverse sets of datasets in various categories for researchers, both in academia and industry. They run various competitions and publicize the winners of the different recognition categories. In 2022, the biometrics and cryptography company, Idemia, correctly matched 99.88% of 12 million faces in the mugshot category tested by NIST. This represents a 0.02% error rate compared with 4% in 2014.
What can be done to ensure facial recognition is used correctly?
The work of oversight really has to be done at the government, policymaking level. The National Science Foundation and the Computing Research Association work like a guardian angel for the computer science profession and step in when things go off the rails.
Additionally, groups like the White House Office of Science and Technology Policy need to put together policies for AI and society. Until it’s written in law, it’s essentially a free-for-all. It’s really very complex making these policies for society. How do you make privacy a priority with AI, when all of us are gladly signing over our data to Facebook and Google?
Adding to the complication is that many representatives in Congress don’t fully understand how algorithms work. Even academics are running to catch up in some cases. As computer scientists we know how to write code, but we’re not necessarily policymakers. Sometimes you just write the code where you don’t know the consequences. Some of the bad consequences we observe today with AI are unintended consequences, but still, we can’t give up. We have to keep trying to make this work in society.
