How facial recognition systems are adapting to masks
What is facial recognition?
Facial recognitioninvolves using computing to identify human faces in images or videos, and then measuring specific facial characteristics. This can include the distance between eyes, and the relative positions of the nose, chin and mouth.
This information is combined to create afacial signature, or profile. When used for individual recognition – such as to unlock your phone – an image from the camera is compared to a recorded profile. This process of facial “verification” is relatively simple.
However, when facial recognition is used to identify faces in a crowd, it requires a significant database of profiles against which to compare the main image.
These profiles can be legally collected by enrolling large numbers of usersinto systems. But they’re sometimes collected throughcovert means.
The problem with face masks
As facial signatures are based on mathematical models of the relative positions of facial features, anything that reduced the visibility of key characteristics (such as the nose, mouth and chin) interferes with facial recognition.
There are already many ways toevade or interferewith facial recognition technologies. Some of these evolved from techniques designed to evade number plate recognition systems.
Although the coronavirus pandemic has escalated concerns around the evasion of facial recognition systems,leaked US documentsshow thesediscussionstaking place back in 2018 and 2019, too.
And while the debate on theuseandlegalityof facial recognition continues, the focus has recently shifted to the challenges presented by mask-wearing in public.
On this front, the US National Institute of Standards and Technology (NIST) coordinated amajor research projectto evaluate how masks impacted the performance of various facial recognition systems used across the globe.
Itsreport, published in July, found some algorithms struggled to correctly identify mask-wearing individualsup to 50% of the time. This was a significant error rate compared to when the same algorithms analysed unmasked faces.
Some algorithms evenstruggled to locate a facewhen a mask was covering too much of it.
Finding ways around the problem
There are currently no usable photo data sets of mask-wearing people that can be used to train and evaluate facial recognition systems.
The NIST study addressed this problem bysuperimposingmasks (of various colors, sizes and positions) over images of faces, as seen here:
It’s possible images of real masked people would allow more details to be extracted to improve recognition systems – perhaps by estimating the nose’s position based on visible protrusions in the mask.
Many facial recognition technology vendors are alreadypreparing fora future where mask use will continue, or even increase.One US companyoffers masks with customers’ faces printed on them, so they can unlock their smartphones without having to remove it.
Growing incentives for wearing masks
Evenbefore the coronavirus pandemic, masks were a common defense against air pollution and viral infection in countries including China and Japan.
Politicalactivistsalso wear masks to evade detection on the streets. Both theHong KongandBlack Lives Matterprotests have reinforced protesters’ desire to dodge facial recognition byauthorities and government agencies.
As experts forecast a future with morepandemics,rising levelsofair pollution, persistingauthoritarian regimesand a projectedincreaseinbushfiresproducing dangerous smoke – it’s likely mask-wearing will become the norm for at least a proportion of us.
Facial recognition systems will need to adapt. Detection will be based on features that remain visible such as the eyes, eyebrows, hairline and general shape of the face.
Such technologies are already under development. Several suppliers are offeringupgradesandsolutionsthat claim to deliver reliable results with mask-wearing subjects.
For those who oppose the use of facial recognition and wish to go undetected, a plain mask may suffice for now. But in the future they might have to consider alternatives, such as a mask printed with a fakecomputer-generated face.
This article is republished fromThe ConversationbyPaul Haskell-Dowland, Associate Dean (Computing and Security),Edith Cowan Universityunder a Creative Commons license. Read theoriginal article.
Story byThe Conversation
An independent news and commentary website produced by academics and journalists.An independent news and commentary website produced by academics and journalists.
Get the TNW newsletter
Get the most important tech news in your inbox each week.
