The use of facial recognition in advertising research. As published in the June-July 2018 edition of the Australian Market and Social Research Society’s publication Research News. Our facial expressions, unconsciously or not, reveal more than we think or we say. Even if a person tries hard to hide their real feelings, our countenance will often uncover the truth. A frown, wink, grimace, snarl, laugh, wince, smile… these are just a number of our – often subconscious – micro expressions, which reveal our inner emotions. They often only last a split second, yet these small facial movements uncover whether someone is angry, sad, scared, disgusted, surprised, happy or full of contempt.
This brings an interesting perspective to marketing research, where the core lies in understanding what people think and feel. Recent evidence in neuroscience has shown that people are not always aware of their own behaviour. We might generally agree to like something, yet the implicit and automatic behaviour such as our emotional expressions may well reveal otherwise. This duality brings an interesting viewpoint when wanting to understand consumer perceptions, for example of commercials. Many advertisements are centred on emotions, as research has shown that if you want people to act – whether by voting or by buying a product – you need to appeal to their emotions. Hence, emotions are a key metric when evaluating advertisement effectiveness.
Keeping in mind the fact that people aren’t always aware of their expressions, the use of eye tracking and facial coding brings many opportunities for marketing research, especially since research shows that humans are less effective in registering and analysing these reactions. Quantum Lab research, for example, demonstrates how the human eye succeeded in capturing emotional responses with a 46.33 per cent accuracy while technology was able to recognise emotions with 83.28 per cent accuracy.

Yet the question remains whether these new techniques match their promises and provide the necessary insights for decision-making. Are implicit measures the new ad-testing reality and should they replace the more traditional, explicit research approaches?
In order to gain insights into the added value of these techniques, we conducted a phased research project together with TUI Group, the leading tour operator in Europe. In addition to more classic conditions, where participants are invited to give their opinion on an ad’s execution, message and impact, another condition was added, exploring the integration of physiological data in survey research through facial coding using Quantum Sense Technology.

The use of this facial recognition technique uncovered that next to dominant expressions of (mainly positive) emotions, people also showed signs of anger or sadness, which did not become apparent through explicit questioning. Combining the facial expression analysis with the actual scene in the commercial allowed us to pinpoint the emotions and give clear guidelines for optimisation. Yet the true value comes from combining the explicit questioning with the analysis of the facial expressions. Although the facial coding provided insights about which scenes generated more negative reactions, it did not explain why. Feeding back the results of the facial recognition to the participants allowed us not only to validate their emotions but moreover to gain an understanding of the reason behind them.
This clearly demonstrates that explicit questioning still has its value and is extremely powerful when combined with other data streams. More than ever, it is the role of researchers to smartly combine emerging techniques with existing ones, in order to gain augmented understanding. These new technologies also provide new opportunities for cross-cultural research. The expression patterns of the six basic emotions are intercultural and universal, yet there are studies showing that Japanese, for example, generally reveal emotions less than those living in other cultures. Facial coding could therefore be a good technique to break through the cultural barriers.
Faces can speak louder than words, yet words will always play an essential role in gathering a comprehensive understanding of how people think and act.