Neuromanagement and

• What is the neuromanagement • What are the applications • What are the basic assumptions • Neuromanagement and Neuromarketing • Neurogatrophisic Faculty Vincenzo Russo

Associate Professor of Consumer and Neuromarketing at IULM University in

Director of the Center of Research for Neuromarketing, “Behavior and Lab – IULM University.

Author of several papers and books about the Neuromarketing, Social Communication, and about the change in the Not for Profit Organization. He has widely published in recognized international journals including the Journal of Consumer Behavior, Journal of Research, Food Quality and Preference; Journal of Global Information Management, European Journal of Information Systems, in the Italian journal of organizational psychology Risorsa Uomo Faculty Giorgio Gabrielli Professor of Neuromanagement and Self Management

Laureato con lode in filosofia a Pavia con una tesi in filosofia morale, ha conseguito nell’aprile 2013 il titolo di Dottore di Ricerca in Interazioni Umane: psicologia di consumi, comportamento e comunicazione, presso l’Università IULM di Milano, con il profilo in psicologia dei processi organizzativi.

Ha frequentato corsi di formazione manageriale presso le maggiori realtà didattiche e consulenziali italiane e internazionali (SDA Bocconi, Harvard Business School ed altre). Attualmente è Amministratore Delegato di News 3.0 e Consigliere di Amministrazione di Tinaba. Ha una sua attività di Consulenza Aziendale e Formazione ed è Professional Coach e Group & Team Coach. Dal 2014 al 2016 ha ricoperto il ruolo di Deputy General Manager di RCS Pubblicità. Dal 2011 a tutto il 2013 è stato Country Manager Advertising & Online di Microsoft Italia, dopo esserne stato Sales Director Consumer & Online. Dal 1990 a tutto il 2010 ha lavorato nel Gruppo Fininvest. Prima in Publitalia ’80 e poi in Mondadori Pubblicita', fino a ricoprire il ruolo di Chief Operating Officer and Business Development. Dal 1985 al 1990 ha prestato servizio nell’Esercito Italiano come Ufficiale in servizio permanente effettivo presso la scuola di Fanteria e Cavalleria di Cesano con qualifica di Comandante di Plotone e Vice Comandante di Compagnia, con mansioni di comando e di insegnamento. È stato Consigliere di Amministrazione di Fedoweb e Audiweb. Tra i principali pilastri della sua vita manageriale e del suo percorso di studi spicca quello di mantenere elevata la motivazione e lo spirito di squadra attraverso gli strumenti e le competenze del management moderno. Tutte le attività vengono svolte in funzione di una nuova visione aziendale, impostata su learning organization, sentimento di efficacia collettiva, passaggio dal knowledge al knowing, arricchimento dei ruoli e strategia collaborativa. The origin of Neuromanagement

Prof. Qingguo Ma, della Zhejiang University e Founder of the first laboratory

Prof. Qingguo Ma and the team of the Ningbo University Neuromanagement and Neuromarketing

Editor of a single issue dedicated to the most important magazine in the world (open source) Frontiers of Psyhology dedicated to Neuromanagement and Neuromarketing

The volume is under construction and will be released by spring 2021

https://www.frontiersin.org/research-topics/14747/ neuromanagement-and-neuromarketing Neuromarketing, Consumer behavior and Consumer Research Center Temi e Obiettivi

• Modulo II: NEUROMARKETING E CUSTOMER EXPERIENCE • La Misura delle Emozioni: le principali Tecnologie a confronto - EEC; fMRI, HRV, EMG, l’Eye Tracking e i Tempi di Latenza (IAT) • Neuromarketing, User Experience e possibili applicazioni • Le principali applicazioni pratiche • nel campo della Comunicazione audiovisiva e della Pubblicità • nel campo del Digitale e dell’User experience nel mondo digitale • nel mondo del Retail, della Shopping Experience e dei luoghi fieristici e museali • nel campo della produzione creativa del Packaging, del Branding e del Labeling CHI SIAMO BEHAVIOR & BRAIN LAB NATO NEL 2008 IN COLLABORAZIONE CON

Prof. Riccardo Barbieri Prof. Luca Mainardi

© 2018 BTO All Rights Reserved Neuromarketing and Sciences The basic principles of the neuromarketing

Behavioral

Psychology Marketing

Communication Neuroscience

NEUROMARKETING Technology Consumer Behavior development

Bioengineering

Neuromarketing Applications

Consumer neuroscience

• is a type of market research that uses the techniques of brain science to understand the implicit side of decision making and its subsequent behavior

• consumer neuroscience is the foundational piece to implicit testing.

TECNICHE E METODOLOGIA DELL’ANALISI DI NEUROMARKETING Presupposti fondamentali allo sviluppo della disciplina

The Behavior and BrainLab IULM was born in 2009 at the IULM as a Consumer Behavior laboratory. Within a few years it introduces the most advanced techniques EXPERIENCE of neuroscientific analysis. The laboratory, today, is one of the main reference points for Consumer behavior in experiential environment Neuromarketing in thanks to the multidisciplinary skills of the researchers who work there. IG: Fairs, events, human performance The main aim of the lab is to integrate traditional survey techniques with the behavioural and neurophysiological aspects of consumers, in order to improve the experience, communication STORE AND RETAIL and marketing of companies. The possibility to study the , cognitive and behavioural In store experience and analysis of retail world processes of consumers requires constant updating of the latest technology and scientific IG: Markets, Stores, Shopping experience discoveries. The possibility of integrating consultancy with the academic world and publications in international scientific journals has allowed the laboratory to grow exponentially over the last decade, guaranteeing universally recognized quality and professionalism. DIGITAL Experience and Usabiliy in the digital world 11 PEOPLE including professors, researchers and experts IG: Website, E-commerce, Apps, Platforms,

In: ADVERTISING ➢ Psychology Fruition of online and offline advertising contents 82 research reports IG: Out of home, Online advertising, Spot, Television ➢ Marketing 5 Accademic books ➢ Neuroscience 30 International pubblications ➢ Engineering 37 International conferences TASTING ➢ Bioengineering Emotions in immersive tasting experience IG: Wine, Beverages, Food, Edible products

Prof. Vincenzo Russo, scientific coordinator of the research center 18 GLI STRUMENTI

EEG EYE TRACKER Per la rilevazione dei segnali elettrici del cervello mediante elettrodi posizionati sullo scalpo. L’EEG È capace di monitorare il movimento degli presenta oscillazioni a diverse frequenze associabili a occhi, restituendo come output un diversi stati attentivi e cognitivi legati ai processi tracciamento oculare che determina con decisionali. I dati registrati permettono di ricavare assoluta precisione l’intero percorso indicazioni sul grado di attivazione delle diverse aree effettuato dall’occhio durante la visione. cerebrali su una Scalp Map.

SKIN CONDUCTANCE RESPIRATION La Skin Conductance è una misura del livello La Respirazione viene rilevata attraverso un sensore di arousal, ossia del livello di attivazione diaframmatico e uno addominale. Insieme agli altri dell’organismo, caratterizzato da un segnali biologici la respirazione è un ulteriore maggiore stato attentivo-cognitivo di parametro di misurazione dell’attivazione vigilanza e/o di emozione. emozionale ed è correlata alle emozioni di paura, rabbia, gioia e tristezza.

FACE READER HEART RATE ll Face Reader è un software in grado di riconoscere Heart Rate (HR) e Heart Rate Variability (HRV). La Variabilità le emozioni provate da un soggetto mediante il cardiaca comprendente le misure del battito cardiaco per riconoscimento delle espressioni facciali non minuto e fornisce una misura dell’attivazione del sistema controllabili dalla persona. nervoso autonomo associato agli stati emotivi.

EMG TEMPI DI REAZIONE Misura il livello di contrazione dei muscoli come le Sono misurati grazie al Test implicito (IAT), un software spalle, oppure i muscoli zigomatici e corrugatori del utile per studiare gli atteggiamenti e le preferenze delle volto. La contrazione di questi muscoli è strettamente persone attraverso la misurazione dei tempi di risposta correlata con gli stati emotivi. in compiti di associazione.

Neuromarketing TECNICHE E METODOLOGIA DELL’ANALISI DI NEUROMARKETING Presupposti fondamentali allo sviluppo della disciplina

Russel model – Two assets Two key concept in our analysis…

AROUSAL

VALENCE 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020 PUBBLICAZIONI SU RIVISTE SCIENTIFICHE INTERNAZIONALI O PROCEEDING CON IF DAL 2010

1. Bilucaglia, M., Laureanti, R., Zito, M., Circi, R., Fici, A., Rivetti, F. Russo, V. (2019). Looking through blue glasses: bioelectrical measures to assess the awakening after a calm situation. In 2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) (pp. 526-529). IEEE. 2. Russo V., Songa G., Ciceri A. Mauri M., Gabrielli G., Clement J., (2019) “A Neuroscientific Method for Assessing Effectiveness of Digital vs. Print Ads Using Biometric Techniques to Measure Cross-Media Ad Experience and ” in Journal of Advertising Research pp. 1-16 April 2019 DOI: 10.2501/JAR-2019-015 3. Songa G., Slabbinck H., Vermeir I. Russo V., (2019). “How do implicit/explicit attitudes and emotional reactions to sustainable logo relate? a neurophysiological study” Food Quality and Preference 71, pp. 485-496. DOI: 10.1016/j.foodqual.2018.04.008 4. Cecilia Conti, Annamaria Costa, Claudia Maria Balzaretti, Vincenzo Russo, Doriana Eurosia Angela Tedesco (2018). “Survey on food preferences of university students: from tradition to new” in Agriculture 2018, 8(10), 155; doi:10.3390/agriculture8100155 5. Stasi A.1, Songa G.2, Mauri M.2, Ciceri A.2, Diotallevi F.3, Nardone G.1, Russo V.2 (2018) “Neuromarketing Empirical Approaches And Food Choice: A Systematic Review” Food Research International ISSN:0963-9969, doi: 10.1016/j.foodres.2017.11.049 6. Songa G. Russo V. (2017). “IAT, and the moderating role of decision-making style: an empirical study on food products. Food Quality and Preference DOI information: 10.1016/j.foodqual.2017.09.0 7. Missaglia A.L., Oppo, Mauri M. Ghiringhelli B. Ciceri A. Russo V. (2017) “The impact of emotions on recall: An empirical study on social ads” Journal of Consumer Behavior. 2017; 1– 8. Russo V., Re A., Angelini A., Jabes D. (2016). “An Analysis and Comparison of Expectations and Perceptions on Sustainable Tourism among Italian and German Tourists”. In Journal of Tourism and Leisure Studies, Vol. 1 (1), pp.1-1 9. Ciceri A., Stasi A., Nardone G., Songa G., Mauri M., Russo V. (2015). “Effect of information on food evaluation and willingness to buy: a study from a neuromarketing perspective”. In Neuromarketing: Theory and Practice (14), pp.12 10. Milani Marin L., Russo V. (2015). “Re-localizing ‘legal’ food: a social psychology perspective on community resilience, individual empowerment and citizen adaptations in food in Southern Italy”. In Agriculture And Human Values, Vol. 32 - ISSN:0889- 11. Bracale R., Milani Marin L., Russo V., Zavarrone E., Ferrara E., Balzaretti C.,Valerio A., Pasanisi F., Nisoli E., Carruba M. O. (2015). “Family lifestyle and childhood obesity in an urban city of Northern Italy”. In Eating and weight disorders, ISSN:1124-4909 12. Onorati F., Mainardi L., Sirca F., Russo V., Barbieri R., (2015). “Nonlinear analysis of pupillary dynamics”. In Biomed. Eng.-Biomed. Tech., pp. 1-12. DOI 10.1515/bmt-2015-00 13. Russo V. (2015). “La scelta tra razionalità ed emozione”. In Le Nuove Frontiere Della Scuola, Vol 37. ISSN:2281-9681 14. Russo V., Mauri M., Sirca F., Onorati F. (2013). “Emozioni, decisioni e comportamento di consumo alimentare: Le soluzioni offerte dal neuromarketing”. Poli-Femo n. 5-6, pp. 45-80, Nuova Serie ISSN: 2037-684 15. Sirca F., Onorati F., Mainardi L., Russo V. (2014). “Time-Varying Spectral Analysis of a Single EEG Channel: Application in an Affective Protocol”. In Journal of Medical and Biological Engineering, doi:10.5405/jmbe.1930 16. Bustreo M., Milani Marin L., Ghiringhelli B., Russo V. (2013). “Comportamenti di consumo, culture alimentari e dieta mediterranea: incontri e confronti tra le famiglie straniere nell’area metropolitana di Milano. Poli-Femo n.5-6, pp 81-103, Nuova Serie ISSN: 2037-6847 17. Laureati M., Jabes D., Russo V., Pagliarini E. (2013). “Sustainability and Organic Production: How Information Influences Consumer’s Expectation and Preference for Yogurt”. In Food Quality and Preference, Vol 30 (1-8 18. Onorati F., Barbieri R., Mauri M., RussoV., Mainardi L. (2013). “Characterization of Affective States by Pupillary Dynamics and Autonomic Correlates”. In Frontiers in Neuroengineering ,Vol 6 (9). 19. M. Mauri, F. Onorati, V. Russo, L. Mainardi, R. Barbieri, "Autobiographical recall of emotions correlating with different psychophysiological patterns", Proceedings of 5th International Conference on Multidisciplinary Perspectives on Child and Teen Consumption, Milan, December 11 – 14, 2012. 20. M. Mauri, V. Russo, R. Manzotti, P. Moderato. "Proposta di un modello applicato per lo studio dei fenomeni emotivi e attentivi durante l’ esposizione a stimoli audiovisivi", Atti dell’ Ottavo Convegno Nazionale AISC (Associazione Italiana di Scienze Cognitive); Milano, Italia; 1 – 2 Dicembre 2011. 21. M. Mauri, V. Russo, R. Manzotti, P. Moderato. "Proposta di un modello applicato per lo studio dei fenomeni emotivi e attentivi durante l’ esposizione a stimoli audiovisivi", Atti dell’ Ottavo Convegno Nazionale AISC (Associazione Italiana di Scienze Cognitive); Milano, Italia; 1 – 2 Dicembre 2011. 22. M. Mauri, F. Onorati, V. Russo. "Eye-Tracking recordings and psychophysiological reactions versus expressed opinions about advertising spots", Proceedings of 23rd Sinergie Annual Congress on Corporate Governance and Strategic Communication, November 10 – 11, 2011; Milan, Italy. 23. M. Mauri, F. Onorati, V. Russo. "Eye-Tracking e reazioni psicofisiologiche combinate con opinioni espresse in merito a spot pubblicitari”, Atti del Convegno Nazionale AIP (Associazione Italiana di Psicologia) Sezione Psicologia delle Organizzazioni: Senso e Prospettive dell’ Applicare, Nuove Declinazioni della Psicologia del Lavoro e delle Organizzazioni"; Milano, Italia; 13 – 14 Ottobre, 2011. 24. M. Mauri, F. Onorati, V. Russo. "Eye-Tracking recordings and psychophysiological reactions versus expressed opinions about advertising spots", Proceedings of 23rd Sinergie Annual Congress on Corporate Governance and Strategic Communication, November 10 – 11, 2011; Milan, Italy. 25. M. Mauri, F. Onorati, V. Russo. "Eye-Tracking e reazioni psicofisiologiche combinate con opinioni espresse in merito a spot pubblicitari”, Atti del Convegno Nazionale AIP (Associazione Italiana di Psicologia) Sezione Psicologia delle Organizzazioni: Senso e Prospettive dell’ Applicare, Nuove Declinazioni della Psicologia del Lavoro e delle Organizzazioni"; Milano, Italia; 13 – 14 Ottobre, 2011. 26. M. Mauri, F. Onorati, V. Russo. "Psychophysiological and Eye-Tracking recordings while watching flyers: a Neuromarketing Experiment", Poster presentation at IAREP (International Association for Research in Economic Psychology) Conference; Exeter, United Kingdom; July 12 – 16, 2011. 27. Y. Ostrovsky, M. Mauri, L. Citi, C. E. Preda, P. Cipresso, R. Barbieri, M. Saccani and C. Lenti; “Gaze and emotional response in visual research tasks involving faces within scenes in autistic individuals: a study combining psychophysiological and eye-tracking measures”, 27th Annual Meeting of American Academy of Child and Adolescent ; New York, USA; October 28 - 31, 2010. (Download pdf) 28. V. Magagnin, M. Mauri, P. Cipresso, L. Mainardi, E. N. Brown, S. Cerutti, M. Villamira and R. Barbieri; “Heart Rate Variability and respiratory sinus arrhythmia assessment of affective states by bivariate autoregressive spectral analysis”, Computing in Cardiology Conference; Belfast, Northern Ireland; September 26 - 29, 2010 (http://www.cinc.org/archives/2010/pdf/0145.pdf). 29. M. Mauri, V. Magagnin, P. Cipresso, L. Mainardi, E. N. Brown, S. Cerutti, M. Villamira and R. Barbieri; “Psychophysiological signals associated with affective states”, 32nd Annual International Conference of the IEEE Engineering in Medicine and Biology Society; Buenos Aires, Argentina; August 31 - September 4, 2010. (Download pdf) 30. M. Mauri, P. Cipresso, M. Villamira; “Psychophysiological reactions and pupil dilation during stress and relaxation”, 14th International Biofeedback Conference 2010; , Italy; April 13 - 17, 2010. © 2018 BTO All Rights Reserved

Neuromarketing and Applications Neuromarketing and Applications ASSUMPTIONS Presupposti teorici e sviluppo della disciplina Neuromarketing and Applications

There are two main reasons for this trend

First, the possibility that neuromarketing will become cheaper and faster than other marketing methods;

Second, the hope that neuromarketing will provide marketers with information that is not obtainable through conventional marketing methods. Consumers don’t think how they feel. “ David Ogilvy “ They don’t say what they think and they don’t do what they say The emotional brain 95% OF OUR PURCHASING DECISION- MAKING PROCESS TAKES PLACE AT THE NONCONSCIUS LEVEL

TECNICHE E METODOLOGIA DELL’ANALISI DI NEUROMARKETING Presupposti fondamentali allo sviluppo della disciplina The Iceberg of consumer’s mind

Consciousness processes ▪ Driven by : - Rationality - Knowledge - Pondering - ……

Unconsciousness processes ▪ Driven by : - Emotions - Unconsciousness

TECNICHE E METODOLOGIA DELL’ANALISI DI NEUROMARKETING Presupposti fondamentali allo sviluppo della disciplina

Herbert Simon, 1980 Theory of Bounded Rationality

The figure of the rational consumer, who is always able to evaluate all the information coming from reality in a correct way, is abandoned.

➢ I can always elaborate all the ➢ I cannot always elaborate all the information I need to take a information. Emotions, decision irrationality and achieving personal satisfaction are the drivers of my behavior.

I’M MOSTLY RATIONAL I’M MOSTLY IRRATIONAL TECNICHE E METODOLOGIA DELL’ANALISI DI NEUROMARKETING Presupposti fondamentali allo sviluppo della disciplina

LIMBIC SYSTEM

The subcortical structures are responsible for the immediate processing of information and emotions. They play a fundamental role in the regulation of vital parameters and can be considered as the oldest part of our brain.

CORTEX

The cerebral cortex is responsible for information processing and integration processes. The cortical areas are the most recent on an evolutionary level. They allow to produce thought, language and mediate with a more rational approach the information coming from reality. TECNICHE E METODOLOGIA DELL’ANALISI DI NEUROMARKETING Presupposti fondamentali allo sviluppo della disciplina

THE HIGH ROAD Le Doux – Two way theory (1996) Sensory Cortex

THE LOW ROAD Sensory Thalamus Amigdala

Stimuli Emotional Responses TECNICHE E METODOLOGIA DELL’ANALISI DI NEUROMARKETING Presupposti fondamentali allo sviluppo della disciplina

Change of paradigm!

“We are not thinking machines that feel, we are feeling machines that think” (Damasio, 1994) Consumers don’t think how they feel. “ David Ogilvy “ They don’t say what they think and they don’t do what they say Social Desirability Effect

Why does people not tell what they think?

• People, sometimes, are a little liars!

• People provide answers in order to preserve their self-image

(Paulhus 1984) or to manage the image of themselves conveyed to

others (Crosby, Bromley and Saxe 1980; De Maio 1984; Maass, Castelli and Arcuri 2000, Camerer, Loewenstein, and Prelec 2005).

• People (Marlowe and Crowne 1961; Edwards 1957; Roccato 2003) offer a socially desirable responding, also unintentionally, due to the lack of introspection abilities (Boca 1996; Greenwald and Banaji 1995; Kitawaki and Nagabuchi 1998; Banaji 2001) and to the difficulty to verbalizing emotions (Penn 2006). Consumers don’t think how they feel. “ David Ogilvy “ They don’t say what they think and they don’t do what they say

Brain Systems Involved in the Psychology of

CONCEPTS RECAP

Eye tracking

But SMI was bought by Apple and no longer gives support TECHNIQUES AND METHODOLOGY OF NEUROMARKETING ANALYSIS Eye Tracking System

The large number of cortical and subcortical areas, crossed by visual information, makes sight one of the most developed and important sense. ¼ of our brain contains areas involved in processing of visual information.

The interconnection of these structures reveals a strong link between perceptual data and cognitive-attentive processes. At the bottom of the Eye Tracker operation there is MIND-EYE HYPOTHESIS (Hoffman, 1998)

MIND-EYE HYPOTHESIS

Theory based on Neuroscience. It declares that what we observe reflects what we are thinking about or what we are interested in. TECHNIQUES AND METHODOLOGY OF NEUROMARKETING ANALYSIS Eye Tracking System

Three different types of Eye Tracker

DISPLAY BAR GLASSES VR HEADSET

In lab On Field In lab

• Static tracking • Dynamic tracking • Dynamic tracking • Display Analysis • Analysis of the surrounding • Virtual reality environment TECHNIQUES AND METHODOLOGY OF NEUROMARKETING ANALYSIS Eye Tracking System

The Eye Tracker works with infrared light technology. For this reason, it is extremely sensitive to sunlight and is not suitable for outdoor ABOUT GLASSES environments without adequate protections.

The Eye Tracker glasses (field and dynamic acquisition version) is a very useful tool when you want to analyze environmental contexts.

• Retail and Store analysis • Dynamic Human Performance • App Analysis • Product Placement • Real Packaging Analysis • …. TECHNIQUES AND METHODOLOGY OF NEUROMARKETING ANALYSIS Eye Tracking System When you have collected all data, the Eye Tracker provides different types of output that differ according to the kind of information sought on the stimulus. Among the various outputs, we can find a distinction between those that offer comparable metrics (quantitative) and those that provide visual feedback on exploration (qualitative).

QUALITATIVE OUTPUT • Heat Maps • Focus Maps

QUALI-QUANTITATIVE OUTPUT • Scan Path • Gridded A.O.I.

• Area of Interest QUANTITATIVE OUTPUT • Fixation metrics • Blinking Analysis • Pupillometry TECHNIQUES AND METHODOLOGY OF NEUROMARKETING ANALYSIS Eye Tracking System

HEATMAPS

The heatmap is specific device that uses the amount of fixations on a stimulus to give a very intuitive output.

The areas marked by red color are the most displayed, those in green the least displayed.

Shorter transition Longer transition

NUDGE E EMOZIONI

Digital Community © 2018 BTO All Rights Reserved TECHNIQUES AND METHODOLOGY OF NEUROMARKETING ANALYSIS Eye Tracking System

HEATMAPS DAY EVENING

The heatmap can also be obtained on environmental contexts, projecting on a static image (snapshot) the video portions recorded through the eye tracking glasses. The software, using the image recognition based on A.I., projects the fixations on the snapshot. TECHNIQUES AND METHODOLOGY OF NEUROMARKETING ANALYSIS Eye Tracking System

HEATMAPS SET UP NOT SET UP TECNICHE E METODOLOGIA DELL’ANALISI DI NEUROMARKETING Eye Tracking System

HEATMAPS LED SPOGLIATOI TECNICHE E METODOLOGIA DELL’ANALISI DI NEUROMARKETING Eye Tracking System

HEATMAPS TECHNIQUES AND METHODOLOGY OF NEUROMARKETING ANALYSIS Eye Tracking System

FOCUSMAPS

The Focusmap is based on the opposite concept of Heatmap.

It uses fixations to create a map of shadows (areas where there has been no fixation) highlighting the areas displayed only. TECHNIQUES AND METHODOLOGY OF NEUROMARKETING ANALYSIS Eye Tracking System

SCANPATH

The Scanpath faithfully reproduces the ocular path of a subject on the visualized stimulus. It’s a qualitative output. It allows you to sequentially reproduce the visual path. However, it is composed of circle and connecting lines which provide information about fixations and saccades.

➢ Circle: represent eye fixations. That is, the moments when the eye is stable and the information is really perceived and reworked. The larger the circle, the longer the duration of the fixation.

➢ The connecting lines: represent instead the saccades.

The example shows the overlapping ScanPath of 3 users (characterized by the different color of the route) on an e-commerce. ✓ Packaging Analisys (Duchowski, 2002)

✓ Eye Tracking does not require active cooperation of the viewer this technique can be used with children and infants. The gaze to the eye are similar to adults (Jones and Klin, 20013)

NEURO TRACKER:

EYE TRACKING:

BIO TRACKER: EYETRACKING SYSTEM : ANALISI DELL’ATTENZIONE VISIVA RISULTATI DISEGNO DI RICERCA

Web pages with moderate complexity are more popular (Lindgaard et al. 2006), pages trigger negative emotions (Tuch et al. 2009)

71 Le limitazioni TECHNIQUES AND METHODOLOGY OF NEUROMARKETING ANALYSIS Eye Tracking System

AREA OF INTEREST

The Area of Interest is more practical. This is designed by the researcher according to his goals and the elements that he wants to analyze. Once the area of interest has been selected and drawn, it is possible to extract indicative measures on performance. Among the indicators of performance we can find:

• TIME SPENT : time spent on area • ENTRY TIME : entry time on area • SEQUENCE : sequence of visualization of area • HIT RATIO : percentage of the sample that displayed the area. • REVISITS : average revisits on area • REVISITORS: number of subjects that has revisited the area

• FIXATION COUNT : number of fixation on area FIXATION AVERAGE: average fixations on area ESEMPIO ETICHETTA: RETRO

SOMMELIER INESPERTI

Capienza della bottiglia Certificazione biologica

Annata Grado alcolico

© 2018 BTO All Rights Reserved ESEMPIO ETICHETTA: RETRO

Gli uomini non prendono in considerazione tutte le informazioni disponibili, ma ne selezionano solo alcune.

UOMO DONNA

Digital Community © 2018 BTO All Rights Reserved MAPPA Di RILEVANZA sommelier VS. Non Experts TECHNIQUES AND METHODOLOGY OF NEUROMARKETING ANALYSIS Eye Tracking System

OTHER OUTPUT

Blinking is the frequency with the eyes physiologically close for short ranges of time. This seems to indicate the level of satisfaction of a stimulus. The greater the time span BLINKING between blinks, the greater the level of involvement in stimulus (Dunning et al., 2010)

Pupilometry or pupillometric analysis focuses attention on the mean amplitude and on pupil variations. This type of analysis, already widely used since the 1970s, is very PUPILLOMETRY complex because, besides emotions and cognitive load, pupil amplitude is subject to exogenous variables such as brightness and endogenous variables such as cognitive stress level.

TECHNIQUES AND METHODOLOGY OF NEUROMARKETING ANALYSIS Neuroanalysis: what is EEG?

The technological development has allowed the creation of different brain imaging techniques. These instruments are able to map the cerebral activity and to visualize the areas with more activation. This type of analysis allows us to understand “which” areas are activated and with which intensity. However, brain imaging techniques are very expensive, difficult to use and data analysis takes a lot of time.

All Brain Imaging techniques are based on different assumptions and operations:

▪ PET: positron emission tomography ▪ fMRI : functional magnetic resonance ▪ MRI: magnetic resonance imaging

Due to their difficult used, laboratories prefer

EEG TECHNIQUES AND METHODOLOGY OF NEUROMARKETING ANALYSIS Neuroanalysis: what is EEG?

Why EEG?

Cost-effective

• It is portable. How can you think to transport an fMRI or a PET Faster in an on-field experiment?

Best temporal analysis • It is economic in terms of time spent to put the sensors

• It is economic in terms of money you need to use to do a study Portable • It is economic in terms of analysis you need to do on data Non-invasive technique • It has the best temporal resolution compared to other techniques (but it has the worst spatial resolution)

TECHNIQUES AND METHODOLOGY OF NEUROMARKETING ANALYSIS Neuroanalysis: what is EEG?

Open BCI – Channels Tbd Emotiv Epoch – Channels 14 B-Alert X10 – Channels 8 Muse – Channels 2

What are the differences?

➢ Density: is represented by the number of sensors. More sensors ! bigger area in which the data can be collected. A large number of sensors takes a long time for the set up, but ensures a high quality data.

➢ Sampling rate: is determined by the amount of data that the helmet is able to collect in a second. A 128 Hz sampling indicates that data is sent from all sensors 128 times in a second. If we have a higher sample rate, we will have a better data quality. TECHNIQUES AND METHODOLOGY OF NEUROMARKETING ANALYSIS Neuroanalysis: what is EEG?

What are the characteristics of the EEG helmet? How does it work?

EEG allows us to have:

➢ High temporal resolution: Live data collection

➢ Low spatial resolution: It is not able to distinguish the activation of specific cortical areas, but it gives us a global figure of the brain lobes’ activity. TECHNIQUES AND METHODOLOGY OF NEUROMARKETING ANALYSIS Neuroanalysis: what is EEG?

How can you see the brain by EEG analysis?

EEG analysis doesn’t give us data about subcortical structures and we can’t visualize or map which are the most active areas.

EEG gives us information about electrical activity of the cortex where the sensors are positioned. EEG can describe which kind of brain activity there is.

How? With a difference in the calculation of electrical activity between an active point, (on the scalp) and a point close to a neutral zone (e.g.: lobe of the ear) ACTIVE POINT REFERENCE / GROUND where there is no brain activity. Electrical activity of an Electrical activity of neutral active electrode - zone (es: lobe of the ear) TECNICHE E METODOLOGIA DELL’ANALISI DI NEUROMARKETING Neuroanalisi: cosa è l’EEG? L’output RAW: la matrice dati DO NOT PANIC!!!

Ogni sensore produce un output numerico per ogni istante di tempo t (il cui valore complessivo è legato al concetto già visto di «campionamento». Questo valore numerico è il risultato della differenza vista pocanzi tra l’elettrodo attivo e quello neutro (Reference). TECHNIQUES AND METHODOLOGY OF NEUROMARKETING ANALYSIS Neuroanalysis: what is EEG?

Raw output should be carefully filtered of any movement or determined by the same instrumentation artefacts.

AND THEN…WHAT CAN WE DO?

SPECTRAL ANALYSIS SPECIFIC INDEX quantifies the amount of rhythmic (or oscillatory) Through algorithms able to work on the matrix, it is activity of different frequency in EEG. possible to find indicators known in literature that are connected with specific brain activities.

▪ Rhythm Alpha ▪ Rhythm Beta ▪ AWI ▪ Rhythm Theta ▪ Cognitive Engagement ▪ Rhythm Delta ▪ Index ▪ Rhythm Gamma ▪ Attention Index ▪ Etc… TECNICHE E METODOLOGIA DELL’ANALISI DI NEUROMARKETING Neuroanalisi: cosa è l’EEG?

Analisi spettrale L’analisi spettrale lavora sui ritmi cerebrali, ovvero sulle differenti forme e tipologie che le onde cerebrali (frutto dell’attività elettrochimica delle sinapsi) possono produrre. Ogni ‘forma’ di onda cerebrale è connessa ad un ritmo, che a sua volta è riconducibile ad una certa attività del nostro cervello.

Onde molto lente (0,5 – 4 Hz) caratteristiche degli stati di sonno profondo e rigenerazione fisica.

Onde lente (4 – 7 Hz) connesse a momenti di alta creatività e produzione mentale, ma anche a stati di sonno leggero

(7 – 14 Hz) è definito ritmo «bloccante». Connesso a stati di rilassamento (veglia ad occhi chiusi), sogno e sogno lucido.

(14 – 30 Hz) caratteristico di stati attentivi, presenti nelle attività quotidiane. Tipico della veglia.

(> 30 Hz) è connesso a stati meditativi, e di grande energia psichica NEUROANALYSIS (EEG): WHAT ARE WE MEASURING? VALENCE EEG allows to calculate the index of Frontal Alpha Asymmetry (Briesemeister et al., 2013; Coan et al., 2003; Davidson et al., 1979;; Vecchiato at al., 2012 (Davidson, 2004 e 2012; Tomarken et al. 1990; Ohme et al. 2009 e 2010 Cacioppo et al., 2000; Briesemeister et al., 2013; Vecchiato et al., 2011, Venkatraman et al. 2015)

A positive value indicates greater A negative value indicates a activation of the left hemisphere at the predominance of the right hemisphere at prefrontal level the prefrontal level

«Approach» towards stimulus, INTEREST «Avoidance» of the stimulus, (Harmon-Jones et al., 2010) DETACHMENT (Harmon-Jones et al., 2010) INSTRUMENTS / METHODOLOGY RESEARCH DISEGNODESIGN NEURO TRACKER: ENGAGEMENT INDICATOR (ASIMMETRY INDEX - AI) DI RICERCA APPROACH-WITHDRAWAL INDEX (CEREBRAL ASYMMETRY (ALPHA BAND): FAA (FRONTAL ALPHA ASYMMETRY))

• Asymmetry of Pre-Frontal Cortex Activity is closely linked to the feeling of pleasantness experienced by the subject during sensorial stimulation (Di Flumeri et al. 2017)

• In this regard, from the electroencephalographic (EEG) signal it is possible to estimate the Approach/Withdrawal (AW) index, which has been largely investigated and validated in scientific literature, regarding visual, acoustic and olfactory stimuli. (Harmon-Jones, Gable & Peterson, 2010; Di Flumeri et al. 2017).

• Positive FAA indicates a tendency to approach or engage with a stimulus, and a (Davidson, Ekman, Saron, Senulis & Friesen, 1990; Harmon- negative FAA WHICH indicates a tendency to withdraw or disengage from a stimulus. Jones, Gable & Peterson, 2010; Price, Peterson & Harmon- Jones, 2012; Rutherford & Lindell, 2011) (Astolfi, et al. 2009; Ohm et al., 2011; Ohme, Matukin & Szczurko, 2010; Ohme, • The neural signals was measured through electrodes placed Reykowska, Wiener & Choromanska, 2009, 2010; Vecchiato et al., 2010; Vecchiato et al., 2014; Vecchiato et al., 2011). over the F3 and F4 positions, corresponding to the right and left part of the frontal lobe according to the SI 10-20.

• This index reflects the subject’s tendency to approach or avoid the stimulus Davidson, Ekman, Saron, Senulis & Friesen, 1990; Harmon-Jones, Gable & Peterson, 2010\12222 Gli indicatori

• Memorization Index (Onde Theta su area prefrontale sinistra) (Summerfield e Mangels, 2005; Werkle_Bergner et al., 2006);

• Attention Index Modifica ritmo delle Onde Alpha sulla zona prefrontale (Klimesh 1999; Petersen e Posner, 2012) Le attività alpha nelle regioni occipitali è maggiore quando le persone perseguono un compito di immaginazione (Cooper et al. 2003) e minore quando prestano attenzione agli stimoli

EEG – specific detection/indices: Indices B) EEG ABM INDICES

HIGH ENGAGEMENT: Characterized by processing visual stimuli using comprehensive motor skills and executive resources. When engagement results as high, this index reveals high levels of attention and concentration.

LOW ENGAGEMENT: Characterized by decreased sensory scanning and processing. Typically connected with boring, monotonous or not engaging tasks.

DISTRACTION: Characterized by disengagement from sensory processing resulting from boredom tasks.

DROWSY: Delayed responses to stimuli caused by fatigue and sporadic spindles. it reflects the subject’s level of drowsiness, hence his risk of making sleep-related errors.

WORKLOAD: Connected with cognitive effort on a stimulus. The workload indicator refers to higher cognitive processes, such as analytical reasoning and decision making. High value result when the subject is involved in an active process of information integration and analysis, such as during problem solving or the performance of a demanding task. EEG AND EYE TRACKING DATA INTEGRATION

The Eye Tracker can be combined with the EEG thus allowing to study frame by frame the level of interest (or even cognitive effort – cognitive workload) BIOANALISI Arousal e valenza fisiologica

➢Skin Conductance ➢Heart Rate Variability ➢Respiration ➢Electromiography TECNICHE E METODOLOGIA DELL’ANALISI DI NEUROMARKETING Bionalisi: la conduttanza cutanea

Come si manifesta e come è possibile misurare l’Arousal?

SKIN RESPIRATION CONDUCTANCE

MUSCLE HEART RATE ACTIVITY TECHNIQUES AND METHODOLOGY OF NEUROMARKETING ANALYSIS Bioanalysis: skin conductance

SKIN CONDUCTANCE Increase Production of Skin becomes a good PERSPIRATION SALINE SOLUTION CONDUCTOR Skin conductance consists in perspiration of the skin. It is a phenomenon managed by sympathetic To measure skin conductance, a small electrical potential is applied (increasing production) and parasympathetic between two electrodes strapped or taped to the palmar side of the (decreasing production) systems in order to hand, and the amount of current conducted between the electrodes regulate and cool our organism. is measured.

> conductance > perspiration > arousal > emotional activation

Direct connection between and micro/ macro changes in sweating. TECNICHE E METODOLOGIA DELL’ANALISI DI NEUROMARKETING Bionalisi: la conduttanza cutanea

SKIN CONDUCTANCE SENSORS

HEART RATE SENSOR Riconoscimento del picco vocale VIRTUAL REALTY + EEG + EYE TRACKING FACIAL EXPRESSION Le emozioni attraverso il volto

➢Le teorizzazioni di P.Ekman ➢Face Reader ➢F.A.C.S.

TECHNIQUES AND METHODOLOGY OF NEUROMARKETING ANALYSIS Face Reader for analysis of facial expressions

Automatic software that can create a 3D mask on the face and provide a FACE READER feedback on the emotion felt or the overall value of the emotion (positive vs Automatic Facial Emotion Recognition negative) FACE READER GLI STRUMENTI EEG RISULTATI VIDEO BOLDRINI Quale ispira più fiducia e sorride veramente?

In 250 millisecondi siamo in grado di valutare istintivamente 104

Le attrezzature Le attrezzature Le attrezzature PERCHE’ USARLO

Il Marketing Emozionale e le Neuroscienze e il cambio di paradigma Exemples of outputs

Advertising

Emotions (facial expression analysis + eye tracking) The emotional trend

Valenza Espressioni del volto 3

2.25

1.5

0.75

0

-0.75

-1.5

-2.25

-3 0 8 15 23 30 The emotional trend

Espressioni del volto

3

2.25

1.5

0.75

0

-0.75

-1.5

-2.25

-3 0 8 15 23 30 L’andamento emotivo – Spot BAP – Artista della birra

Espressioni del volto

3 2.25 1.5 0.75 0 -0.75 -1.5 -2.25 -3 0 8 15 23 30 L’andamento emotivo – Spot BAP – Artista della birra

Espressioni del volto

“300 luppoli”

3 2.25 1.5 0.75 0 -0.75 -1.5 -2.25 -3 0 8 15 23 30

“selezionare e combinare tra oltre…” Espressioni del volto

“quelli che danno vita a ricette speciali”

3 2.25 1.5 0.75 0 -0.75 -1.5 -2.25 -3 0 8 15 23 30 Espressioni del volto

“l’arte di produrre birre straordinarie”

3 2.25 1.5 0.75 0 -0.75 -1.5 -2.25 -3 0 8 15 23 30

➢ La valenza è elevata nella sequenza che mostra l’artigiano e le vasche di fermentazione della birra, con la voce narrante che parla dell’arte della produzione (comunicazione coerente ed efficace) Espressioni del volto

3 2.25 1.5 0.75 0 -0.75 -1.5 -2.25 -3 0 8 15 23 30

➢ Si riscontra un abbassamento della valenza (che rimane comunque positiva) in corrispondenza del secondo 16, in cui si mostra la birra che viene versata nel bicchiere. Espressioni del volto

3 2.25 1.5 0.75 0 -0.75 -1.5 -2.25 -3 0 8 15 23 30 Espressioni del volto

3 2.25 1.5 0.75 0 -0.75 -1.5 -2.25 -3 0 8 15 23 30 L’andamento emotivo – Spot BAP – Artista della birra

3

2.25

1.5

0.75

0

-0.75

-1.5

-2.25

-3 0 8 15 23 30 I neuroni specchio – G. Rizzolatti (1994)

• Il ruolo dei neuroni specchio (Rizzolatti, 1994) • solco superiore temporale • Esistono popolazioni neurali premotorie e somatosensoriali coinvolte sia nell’esecuzione che nell’osservazione delle azioni finalizzate compiute da altri, con un ruolo cruciale nell’imitazione, nell’immedesimazione e nell’apprendimento G. Rizzolatti, C. Sinigaglia

So quel che fai. Il cervello che agisce e i neuroni specchio • osservare il movimento altrui aiuta ad apprenderne i movimenti e a ripeterli con minore possibilità di errore (cfr. Raffaello Cortina, Milano, 2006 la performance sportiva)

123 I neuroni a specchio – G. Rizzolatti (1994)

• La visione di immagini di persone che provano disgusto provoca l’attivazione dell’insula

• L’insula si attiva anche se il disgusto lo proviamo in prima persona

Grado di sovrapposizione di attivazione dell’insula alla Both of Us Disgusted in My Insula: The Common Neural Basis of Seeing and Feeling Disgust di Bruno Wicker, Christian Keysers, Jane Plailly, Jean-Pierre visione e all’esperienza di disgusto. Le macchie bianche sono Royet, Vittorio Gallese, and Giacomo Rizzolatti (2003) i neuroni specchio 124 L’andamento emotivo – Spot BAP – Artista della birra

3

2.25

1.5

0.75

0

-0.75

-1.5

-2.25

-3 0 8 15 23 30 L’andamento emotivo – Spot BAP – Artista della birra

“birrificio Angelo Poretti”

3 2.25 1.5 0.75 0 -0.75 -1.5 -2.25 -3 0 8 15 23 30 Espressioni del volto

“maestri del luppolo”

3

2.25

1.5

0.75

0

-0.75

-1.5

-2.25

-3 0 8 15 23 30 Finale birra poretti NEUROSCIENCE CONSUMER NEUROSCIENCE GASTROPHYSICS

It is a multidisciplinary science IT is the combination of consumer (gastronomical physics) is an emerging that combines physiology, research with modern interdisciplinary science that employs anatomy, molecular biology, neuroscience. principles from physics and chemistry to attain a fundamental understanding of developmental biology, the worlds of gastronomy and cooking. cytology, mathematical The goal of the field is to find modeling, and psychology to neural explanations for consumer Gastrophysical topics of interest include understand the fundamental behaviors investigations of the raw materials of and emergent properties of food, the effects of food preparation, and quantitative aspects of the physical basis for food quality, flavour, appreciation and absorption in the human body What is Gastrophysic?

•Many factors influence our experience of food and drink

•Sensory science tell us about people’s perception of the sensory attributes of what they eat and drink, how sweeter the taste, how intense the flavors, how much they like a dish.

•Gastrophysic builds on the strengths of a number of disciplines including psychology, , neurogastronomy, marketing, design and behavioral economics. Charles Spence Oxford University Winner of the Ig-Nobel Prize for Nutrition 2008

“Changing what a persone sees can radically alter what they hear, changing what they hear may influence what they feel, and altering what they feel can change what they taste” The expectations create by the marketing can change the consumer experience and the perception of the reality •If we change the color of a fruit juice with an odorless and tasteless dye such as oenocyanin which makes the color of the drink redder, •consumers perceive that drink more strongly odorous than colorless ones

(Engen, 1975, Zellner et al 2005, Hidaka and Shimoda, 2014).

•Moreover, if an acid solution (typical of fruit) is added, the drink is perceived even more fruity and "natural”

(Zampini et al., 2008)

•A red colored drink is perceived sweeter than 10% like the equivalent addition of sugar (Johnson e Clydesdale 1982) Expectations of smells and cerebral activation

Knowing that a product is very bitter activates the posterior part of the medial and posterior insula and of the opeculum

However, if the expectation of less bitterness is created before the tasting, the insula and the operculum are activated less and the product is perceived less bitter (although it is always the same)

Nitschke et al. 2006 Altering expectancy dampens neural response to aversive taste in primary taste cortex. a label with predominant colors in green and yellow makes a lemon beer tastier than the same label with different colors

Furthermore, the label with colors coerent with the content improves the perception of product quality

Barnett, and Spence (2016) Assessing the Effect of Changing a Bottled Beer Label on Taste Ratings Nutrition and Food Technology: Open Access The relationship between taste and view The launch of Lemon 7Up with 15% more yellow in the can has made the taste of the drink with more lemon (Esteri, 2011)

CHESKIN, L. 1957. How to Predict What People Will Buy, Liveright, New York, the change in color has changed consumers' perception of quality NY. "The brain uses visual information in order to build expectations about the food, its texture, creaminess, freshness, softness, odour and taste” a chocolate mousse is perceived as sweeter if presented in a white plate, compared to the same mousse, but served on a blue or black plate.

The chocolate mousse plate’s color also impacts the brain activity at early stages of visual processing. 7% more sweet, 13% more fragant 95% pleasurable

(C. SPENCE, 2017) The sound is an ingredient for the chef

Bittersweet symphonies Low-frequency background music makes a food product (chocolate or wine) more bitter

Charles Spence cordinator of the Crossmodal Laboratory at Oxford University e Fat Duck restaurant 2015

http://condimentjunkie.co.uk/blog/2015/4/27/bittersweet-symphony • Spence won the Ig Nobel Prize for Nutrition 2008 • Spence has shown that the crunchiness and freshness of potato crips can increase by amplifying the sound of the bite

• With high-frequency background music the crunchiness and the pleasantness of the potato grows

• (Zampini e Spence, 2004 - The role of auditory cues in modulating the perceived crispness and staleness of potato chips) For six weeks in a cafeteria, 6 types of products were presented with different labels (one articulated and one standard)

- New York Style Cheesecake with chocolate sauce

- cheesecake

The descriptive labels were chosen in 27% more cases than the others and the products were more appreciated.

In addition, consumers claimed not only that they were willing to pay almost 10% more for each descriptive menu item, but there was also a positive effect on the perceived quality of the cafeteria. Does food and drink taste better if you pay more for it? Some investigated what happened in the brain of wine drinkers when given different and sometime misleading, information about the price of a red wine.

A 5$ bottle of wine was either correctly described or else mislabeled as a 45$ bottle. The price was displayed on a monitor, whenever a small amount of wine was squirted in to the participant's mouth. During the experiment the participants have to rating the perception of the quality of the wine with self-report (Plassmann, et al. 2008)

The result was expected.

The wine presented as more expensive was rated as more good Everyone reported liking the expensive wine more than cheap wine. Crucially , analysis of the brain scans revealed increases in blood flow in the reward center of the brain associated with the price cue. More activation of the OPC

Telling people that the wine was more expensive led to an increase in activation in the medial orbital- frontal cortex, a small part of the brain located just behind the eyes

By contrast, no change in blood flow was observed in the primary taste cortex the part of the brain that processes the sensory-discriminative attributes of taste

When the same wines were presented eight weeks later now without any indication as to their price no significant difference in pleasantness were reported Neurogastrophisic