A new study from the National Institute of Standards and Technology (NIST) has tested how accurate commercial facial recognition algorithms are at identifying people wearing protective face masks. The study reveals some commercially used systems fail at authenticating masked faces up to 50 percent of the time.
Over the last few years the growing use of nascent facial recognition technology by law enforcement agencies has led to significant public debate over the efficacy of the technology and the lack of oversight into its uses. Several cities in the United States have outright banned the use of facial recognition technology, while the American Civil Liberties Union (ACLU) is pushing for a greater transparency into where and how it is deployed.
With the COVID-19 pandemic reality meaning most people around the world are now wearing face masks in public, new questions have been raised over the effect this will have on facial recognition technology. A recent report in The Intercept revealed a leaked Homeland Security memo from late May expressing concern over the effect of face mask wearing on facial recognition accuracy.
“We assess face recognition systems used to support security operations in public spaces will be less effective while widespread public use of facemasks, including partial and full face covering, is practiced by the public to limit the spread of Covid-19,” the leaked intelligence document stated, raising concerns "violent adversaries" could subversively be using protective face masks as a way to evade law enforcement detection.
Now a new and publicly released study is offering the first in a series of investigations into the efficacy of current facial recognition systems on people wearing face masks. The NIST program is called the Face Recognition Vendor Test (FRVT) and this initial study investigated 89 commercially available facial recognition systems in one-to-one match tests using digitally masked faces.
“With the arrival of the pandemic, we need to understand how face recognition technology deals with masked faces,” says Mei Ngan, a computer scientist working at NIST. “We have begun by focusing on how an algorithm developed before the pandemic might be affected by subjects wearing face masks. Later this summer, we plan to test the accuracy of algorithms that were intentionally developed with masked faces in mind.”
The one-to-one matching tests conducted in the study involved directly matching a target face image with other images of the same person on a database. It is one of the easier tests for a facial recognition system, and generally less error-prone than larger systems that scan faces in big crowds.
The study found even the most accurate facial recognition algorithm failed at significantly high rates when confronted with a masked face. The best system tested in the study still resulted in a 5-percent failure rate tracing masked faces, compared to its regular 0.3-percent failure rate. The study also noted that “otherwise competent” algorithms failed to trace masked faces between 20 and 50 percent of the time.
More specifically, the study found the higher a face covering is on the bridge of a person’s nose, the less accurate a system is in identifying the person. Mask color was also relevant, with black masks negatively affecting accuracy more than surgical blue masks.
Interestingly, the study also found face masks significantly increased rates of false negative results but not false positive results. So this means the facial recognition algorithms were not incorrectly identifying masked faces as different people, but instead the systems were simply unable to make an effective determination in the first place.
Ngan says future NIST studies will investigate other variables and newer facial recognition systems designed to work around masked faces.
“With respect to accuracy with face masks, we expect the technology to continue to improve,” says Ngan. “But the data we’ve taken so far underscores one of the ideas common to previous FRVT tests: Individual algorithms perform differently. Users should get to know the algorithm they are using thoroughly and test its performance in their own work environment.”
But activists against facial recognition technology shouldn’t get too excited, tech companies are already racing to adapt their algorithms to our new masked world. Back in March one of China’s key facial recognition start-ups announced it had already upgraded its systems to work with masked faces. Other US-based companies are reportedly doing the same, rapidly testing and tuning their algorithms to better detect faces based on characteristics that are left uncovered.
The complete NIST facial recognition report is available here (PDF).
Source: NIST