Scenario One: Imagine you open your social media account and find a message from someone you’ve never even heard of. They tell you that they saw you on the street and traced you to your account. They wonder if they could meet you. They tell you they know where you live so they can visit you at your home, if you’d prefer. You’re confused. How did they find me from just seeing me on the street?
Scenario Two: You participate in a big demonstration against a government policy that you don’t agree with. Two days later you get a visit from the police. They tell you they know you’ve been participating in anti-government demonstrations and they would like you to come with them to answer a few questions. How did they pick me out of hundreds of other people?
Scenario Three: You are at a club and have had one drink too many. You do a few things you shouldn’t. The next day you get a message on your social media site. In it are some photos of you having a good time with someone you don’t even remember. The person contacting you says he/she knows you are in a relationship and, if you pay them a certain amount of money, they will not send the photos to your partner and all your social media contacts. How did they know who you were?
These are three possible scenarios of what can happen if the new app, FindFace, becomes mainstream. FindFace is an app developed by Russian-based N-Tech Lab. It is based on an award-winning algorithm called FaceN. This algorithm uses neural network architecture which means it can learn to identify a face even when some changes are made to it. As the company puts it,
“To identify a person by his photo correctly, the algorithm must be able to extract the inherent facial features. These features are to be distinguished from the ones that can vary depending on the different temporal factors (such as age, facial expression, make-up, glasses and the particular camerawork angle, lights, background.) The unchanging characteristic features are called invariant, and due to them it is possible to tell people apart.”
The algorithm is, apparently, very powerful, if we are to believe the information they give on their website.
“Provided with just one picture, FaceN can identify a person in a split second and find his photographs among billions of other portraits in the database. The system can be used at planetary scale due to the efficient search algorithm and to the specially developed technology that allows to store the extracted features in a compact way.”
Planetary scale? If this is true, every person on planet Earth who has a photograph in any database (not only a social media site) could be identified from a photo that any random stranger takes on the street. Imagine if a program similar to FindFace had access to driver license photos, school yearbooks, and passport photos. Imagine how much they could learn about you from a simple, random photo. It’s a scary prospect.
FindFace is marketed as a dating app. If, for example, you see someone on the street you would like to meet, you could snap a quick photo of them and use the app to find out who they are. Having identified them, you could, then, contact them and possibly begin to build a relationship.
In another approach, you could use the picture of a person you like, such as a movie star, and use the app to identify people who look similar to them. Sure, a lot of people would ignore a random contact from a stranger, but some may be curious enough to let you into their cyber world. It’s really not so much different from how most dating sites operate.
The problem with FindFace is its power. You could find the identity of the person sitting nearby in a restaurant and contact them in seconds. They may be intrigued or they may just be creeped out. Kaspersky Labs tested the app and concluded that it does, indeed, work. It had some problems identifying people who turned their head at an angle or who were at a distance, but it managed to find a photo of someone on their staff who never posted a photo online. This happened because another person had posted a photo of him.
But don’t worry too much for now. FindFace currently only works with the Russian version of Facebook, Vkontakte. Facebook cannot be accessed by FindFace, at least in its current incarnation. However, Facebook already has a facial recognition program which it uses to help you tag photos that include your friends. But it doesn’t end there. They are developing a new program called, DeepFace, which has the potential to search the entire internet for photos. The end of anonymity is but a click away.
Facebook has not announced what it plans to do with this new technology, but my guess is that it’s all about marketing. It’s not only about finding every site that has a photo of you, but extending this technology to the real world. Connecting real world shoppers to their social media profiles would be a marketer’s dream. A store could, for example, use its cameras to identify shoppers and see what they were looking at while they shopped. Later, they could target their social media sites with tailored ads. In its most powerful incarnation, they could see what the shopper was looking at, identify the shopper, and send him or her an instant smartphone message. The message could even make them a special offer on what they are currently looking at. They could, for example, give them a special price if they purchased the product within a limited timeframe. Now, that’s marketing.
In fact, this is just one step further than a number of marketing firms already offer. Companies like Aislelabs use GPS, WiFi, and beacon technology to show where shoppers are located within the store, see what they are looking at, and target them for ads.
It seems in the example above that they are targeting the company’s own customer database. They don’t necessarily need your photo but your phone’s unique address (MAC address) which they can retrieve through their Wi-Fi network. If you then purchased something with a credit card, your name and other personal data could be linked with the MAC address. They know your shopping habits and how to target you. If all of this can be matched with video surveillance, they have a pretty good profile of you in their database. They could then use facial recognition technology to see if you return at a later date. Just imagine how tempting such databases would be for hackers. Despite the power of these in-store marketing systems, they still lack the ability to connect all of this information to a first-time visitor like FindFace does.
FindFace is not so coy about what it wants to do with its technology. They readily admit that they will make it available to retailers and law enforcement agencies. They are currently talking with the Moscow government about integrating their technology with the 150,000 CCTV cameras in the city. Yes, it could help solve crimes, but it doesn’t take much to imagine a sinister angle that could be exploited.
Some oppressive regimes aren’t as understanding of protests. Using a program that could identify protesting individuals would go a long way towards removing potential ‘troublemakers’ from the system. Even societies that consider themselves to be open would like to keep an eye on individuals supporting more extreme causes, free speech or not. After all, the NSA has an extensive facial recognition system which it purportedly uses to identify terrorists. That could be, but, in the process, how many law-abiding citizens are identified.
Criminals could use such apps for a number of nefarious purposes. They could use them for extortion, as in scenario 3 above. If they could identify you on the street, find your profile, and possibly your address, they would know you were not at home and take advantage of that situation. They could follow you, learn your habits, and use any information they find to launch a phishing attack. To take this further, if cyber criminals could learn what company a person works for during their investigations, they could use the information they find to launch a phishing attack that compromises the company. And let’s not forget the unwanted attention one may receive from unwelcomed strangers that could lead to stalking behavior. In short, FindFace and its relatives are like marketing apps for criminals.
In the final analysis, these sort of apps are taking us all further down the path to a loss of privacy. Many people don’t care and assume this evolution is inevitable. Others, however, are fighting back. A whole new fashion line has developed to thwart facial recognition systems and surveillance systems in general. For $250 you can buy these Japanese-made glasses that will confuse facial recognition apps…and normal people looking at you.
Not your style? Well, maybe you’d consider one of these changes in your hairstyle and makeup.
Other techniques offer fashions that often feature hoods and visors.
And if all else fails, you could always resort to a mask or a burka.
Unfortunately, using any of the privacy, anti-surveillance techniques stated above will likely get you flagged by security. Instead of making you anonymous, it will make you a person of interest. True, the facial recognition system may not be able to identify you, but it won’t stop law enforcement agencies from pulling you over for questioning. It seems that trying to remain anonymous only makes you conspicuous. It’s almost as if there’s a law about trying to be anonymous. Well, that may be next. Who knows?
Update: After this post was released yesterday, the leak site, Cryptome, released information on an IBM patent that is attempting to do the same thing that FindFace and DeepFace are doing.