Big Brother Knows What You Look Like, and That’s OK?

Computers that can recognize faces have made great strides in the last decade, and are only getting more accurate.

That's partially because of a shift to 3-D facial recognition. Currently, most facial recognition algorithms rely on 2D techniques. Dr. Lyndon Smith, professor of computer science and machine vision at the University of the West of England, Bristol, explains that 2D technology is susceptible to light conditions and viewing angles. Comparatively, 3D facial recognition provides higher-resolution data.

“[3D facial recognition] captures very detailed data from a human face, rather like a 3D fingerprint of the face,” Smith says. “This can provide very good reliability for recognition, thereby opening up a vastly increased range of potential applications.”

The concept of an error-free algorithm is enough to inspire visions of 1984, and in fact, even today facial recognition tech is being put to some unsettling uses. A dating app that matches you to people who supposedly look like your celebrity crush? In development at the New Jersey Institute of Technology. Delta Airlines is testing a system in which facial scans replace boarding passes. And malls, casinos and stores are using facial recognition software to track who is in their building, sometimes targeting advertisements toward individuals based on the software's characterization of a person's demographics.

A few uses of facial recognition, however, are less scary. New products that help students study, find lost pets and assist blind people are on the market now or coming soon. And there's surely more to come.

Track students’ attendance and attentiveness.

Despite its friendly-sounding name, Nestor is poised to become inattentive students’ worst nightmare. The software, an artificial intelligence created by the French company LCA Learning, debuted this May. It is currently being tested in two online classes offered by the ESG Management School in Paris.

As students watch recorded lectures, Nestor uses their webcams to analyze eye movement and facial expressions. The AI notes when students appear distracted, and at the end of the lecture, quizzes them on material covered during these daydreaming periods. Nestor can also track patterns of inattentiveness and alert students when it senses they are about to lose focus.

LCA founder Marcel Saucet says that Nestor also helps teachers revise their lesson plans. If the majority of students grow distracted at the same point in a lecture, for example, the professor may want to find a new angle on the topic.

While privacy advocates have raised the usual questions about whether the technology is invasive and how the recordings will be used, Saucet has said that all data is encrypted and no video footage of students will be stored.

Help blind individuals recognize their friends and family.

In 2015, students at Birmingham City University developed the XploR cane, a device that helps the visually impaired “see” their surroundings. This ability is especially helpful at large social gatherings, where one inevitably encounters a continuous flow of individuals.

XploR operates in conjunction with its owner’s smartphone and relies on GPS, Bluetooth and facial recognition capabilities. The cane scans the faces of individuals within a 32-foot range, and if it identifies them as a friend or family member, alerts its owner. XploR then guides the blind individual to their loved one through instructions delivered via earpiece.

Earlier this year, two of XploR’s creators, Asim Majeed and Said Baadel, presented their invention at a global security conference. They hope to expand the cane’s capabilities by incorporating social media facial recognition data and––eventually––developing machine-to-machine exchange of data (for example, communicating an impaired person’s location to the driverless car sent to pick them up).

A spokesperson for the National Federation of the Blind, an advocacy group for blind people in the United States, told Wired in 2015 that an app on a smartphone might be "more cost-effective" than a technologically advanced cane, but that facial recognition technology "has potential to solve a real problem experienced by blind people."

Find a missing pet.

Facial recognition isn't just for humans. The Finding Rover app uses facial recognition to help owners reunite with lost pets.

Users preemptively upload photos of their pups, and if Fido is lost, Finding Rover alerts its extensive network of local animal shelters and app users. Those within a 10-mile radius of the animal’s last known location receive a push notification, and if they see a similar-looking pet, they can submit a photograph of it through the app. Once Finding Rover identifies a match, it notifies the pet’s owner.

The system, which took two years to create, was developed in conjunction with researchers at the University of Utah.

Today, most pets have microchips, an embedded chip that holds an identification number. If a lost animal ends up at a veterinarian’s office or animal shelter, officials check for a microchip and use the ID number to reunite pet and owner. But not everyone has access to chip scanning equipment, and not all pets are microchipped. An employee at the Wisconsin Humane Society told a local news station that "It's great to know that if your animal went missing … you have something as convenient and close as your phone to get the word out immediately," but added that the app shouldn't replace collars or microchips.

Whether facial recognition technology will be used more for good or for ill is an open question. In a 2014 study, Carnegie Mellon professor Alessandro Acquisti identified individuals walking around a college campus by comparing Facebook profile pictures to webcam images––thanks to facial recognition technology, he was successful one-third of the time. It’s been three years since Acquisti’s study, and as he cautioned in an interview with The Atlantic, “From a technological perspective, the ability to successfully conduct mass-scale facial recognition in the wild seems inevitable. Whether as a society we will accept that technology, however, is another story.”

Meilan Solly | | READ MORE

Meilan Solly is Smithsonian magazine's associate digital editor, history.