What do people feel when they see a commercial? How do they react to individual scenes or messages? For a long time it was not an easy task for market researchers to decipher this scientifically. An innovative software developed by the Gesellschaft für Konsumforschung (GfK) opens up new possibilities for better understanding the purchasing behavior of consumers by means of machine learning.

At the card table and in negotiations, a poker face is used to keep your opponent in the dark about your hand and your goals and tactics. However, even absolute professionals cannot suppress every impulse. Even minimal changes in facial expressions are sufficient to be able to draw conclusions about the emotional state of a person. GfK's market research experts use this fact to find out more about the effectiveness of advertising on the recipients. This is aided by image recognition methods based on artificial intelligence. 

I know what you like!

EMO Scan is the name of the application developed by GfK in cooperation with the Fraunhofer Institute for Integrated Circuits IIS in Erlangen and emotion psychologists at the University of Geneva. It enables companies to find out in real time to what extent people find their products appealing. A camera captures the biometric facial data of potential customers and software evaluates the recorded reactions based on comparison patterns.

EMO Scan can infer from the facial expressions of the viewers the valence of an emotional reaction - i.e. the extent to which a stimulus is perceived as pleasant or unpleasant. More than 12,000 different positive and negative facial images, which are stored in a database, serve as reference values. Especially basic emotions such as joy, surprise, disgust, fear, anger or sadness are universally understood - this has been scientifically proven. The Emo Scan can therefore not only be used on the German market.

The handling of the application also allows a comparatively high degree of flexibility. The emotional reactions of the test persons are not measured in a time-consuming way by electroencephalography (EEG). All that is needed for the analysis is an Internet-capable computer with a webcam. A further advantage of this method: The results can be easily interpreted even by laypersons.

Treacherous movements

The camera and the sensors record the smallest facial movements and emotions such as laughter lines or frowning at pixel level. The system primarily evaluates those regions of the face in which positive and negative emotions can be distinguished most accurately. By means of motion tracking, it recognizes in real time whether the viewer is interested in a product - or not. The result is clear, in contrast to the usual surveys of test subjects, where inaccurate answers can distort the results. The AI-based procedure thus provides market research with relevant information on the type and intensity of the advertising effect.

The EMO Scan has already been used successfully in various customer studies, for example in testing advertising films for car brands or cosmetics, as well as in testing new TV formats. In 2012, GfK received the Innovation Award from Deutsche Marktforschung for its AI-based system. The cooperation partners are currently working on further development of the instrument to include the human voice in the analysis. After all, this is the best way to derive the emotional excitement of the test subjects. This can be relevant if visual channels are missing, for example in telephone interviews.

At least for the poker table, however, this does not play a role - there, people usually remain silent.