Days of Past Future: Ads that Determine Your Emotional State to Design Themselves

When I visited the Museum of Future Technology in Japan, I expected to be amazed. Instead, I found that many of the exhibits were past their sell-by dates. What may have seemed futuristic when the museum was built had become somewhat routine. That’s the problem with technology nowadays. It just moves too fast. I experienced this first hand when I wrote a short story about the implications of interactive advertising. It was meant to be science fiction. I am now finding that science fiction is becoming science fact at an alarming rate.

I can’t say I was completely surprised when I heard about the new type of interactive advertising recently on display at a London bus stop. This test ad tries to make an emotional connection with the person viewing it. Nothing new in that; after all, advertisers have always tried to reach potential customers by manipulating them emotionally. Perhaps my favorite among these ads is the one seen on British TV which tries to get people to contribute money to an organization that helps homeless dogs. This adopt-a-dog ad promises that if the viewer pays a small amount of money each month, a dog, like the shivering, sad one you see in the ad, “will write you a letter”. Really? Let me tell you something. If you have a dog that can write letters, that dog can support you and a small town. But the point is, once the emotional button is pushed, logic falls by the wayside and, so the organization hopes, money will follow. That’s much of the point behind such advertising.

However, this new ad, developed by M&C Saatchi, ratchets this emotional approach up a notch. It actually tries to interpret the facial expression of the person watching it and, through trial and error, determine which ad it presents to that person gains the most interest. Over time, the ad, or more precisely the algorithm behind the ad, analyzes more and more people and begins to learn what approach works the best. In short, the ad evolves on its own.

The algorithm behind this ad is similar to programs that were used to see if certain types of computers could learn English on their own. These parallel distributed processing (PDP) programs are used to find patterns in what, at first, may seem to be nothing more than chaotic data. Through a process of trial and error, and the ability to compare its hypotheses with ideal patterns (feedback), the computer begins to make sense of nonsense and learn basic English patterns, such as the appropriate word order of sentences.

The Saatchi ad uses a Microsoft Kinect sensor camera with facial recognition software to determine the facial expressions of the person viewing the ad. The ad, which is for a fake brand of coffee, comes with over 1,000 images and components which are randomly presented to a viewer. According to how the viewer responds, the algorithm either keeps or rejects components. It, thus, learns which combinations are more effective and, in a sense, ‘writes’ its own ad. So far, according to Chief Innovation Officer David Cox, the ad has determined that people like short sentences in capital letters. Expect to hear more about what the ad discovered in the near future.

Whatever useful results the ad uncovers, it is clear they can be applied to any media. On the internet, for example, these factors could be used to design ads that are more eye-catching and which may lead people to purchase a particular product. Studies show that most ads in any format go unnoticed. Estimates vary from 88% to 97%. Advertising is expensive so anything that helps improve ad attraction would be welcomed. Combine this with normal internet data mining practices, such as which websites a person visits, and you have a very powerful way to influence a consumer.

But is it possible for an ad on the internet to recognize your facial expressions? Unfortunately, yes, the possibility exists. Affdex from Affectiva will let you participate in a facial recognition test by analyzing your own facial expressions. You can try it here. I chose not to try it because it asked me to make sure my webcam was turned on. I didn’t want to contribute to their database by giving them images that could be used in any number of ways… but at least they asked my permission. Here’s the sticking point. The company sells an SDK (software development kit) that users can design to suit their needs. What if their needs were to gather facial expressions of people who looked at their ads and what if the developers slipped in a little program (malware) to make sure that the viewers’ cameras were turned on. After all, that’s the best way to gather a lot of data for advertisers, right?

Tech marketing firm, redpepper, is taking facial-recognition-based marketing to the next, nearly sci-fi, level. They have an idea in the works called, Facedeals, that will use Facebook’s check-in. Not too many people or businesses use this Facebook app but it mainly tells your friends or others where you are or what places you like. If you enable your location on Facebook and select a photo that you think best represents you, then businesses, that also are on check-in, can have cameras with facial recognition capabilities installed that will know when you are walking by their stores. Once they recognize you, they can look through your Facebook Likes and send a specially designed offer to your smartphone. If they connected this with some emotion recognition software, they could send you a number of offers until they get the emotional response they need. This would give them data that they can use to entice you the next time you walk by. Although this is personalized marketing at its best, you may want to consider if you want to give away so much personal information. Remember, letting everyone know where you are at a particular time tells them that you are not at home, which could make life easier for anyone wanting to break into it. It could also enable cyber and regular stalking. This marketing idea is a sound one, and probably one that may be developed even more in the future, but it doesn’t come without risks.

Recent developments by Google (FaceNet) and Facebook (DeepFace) have increased facial recognition to the point where it even surpasses the facial recognition ability of humans. Google goes so far as to be able to attach a name to a face. If you feel your anonymity is rapidly disappearing, it probably is. The FBI had developed Trapwire years before these improvements in facial and emotion recognition. Trapwire makes use of the network of available CC cameras to help identify potential terrorists. It not only uses facial recognition but other behaviors to determine if someone may be dangerous. If emotion recognition is thrown into the mix, then they have a powerful tool at their disposal. Put everything together, and it may soon be possible to know what everyone is doing, feeling, and thinking at anytime we’re interested. Welcome to the future.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s