Modelling and Analysing Behaviours and Emotions via Complex User Interactions Mohamed Mostafa Mohamed Sayed Ahmed May 2018 arXiv:1902.07683v1 [cs.HC] 20 Feb 2019 Abstract Over the past 15 years, the volume, richness and quality of data collected from the combined social networking platforms has increased beyond all expectation, provid- ing researchers from a variety of disciplines to use it in their research. Perhaps more impactfully, it has provided the foundation for a range of new products and services, transforming industries such as advertising and marketing, as well as bringing the chal- lenges of sharing personal data into the public consciousness. But how to make sense of the ever-increasing volume of big social data so that we can better understand and improve the user experience in increasingly complex, data-driven digital systems. This link with usability and the user experience of data-driven system bridges into the wider field of human-computer interaction (HCI), attracting interdisciplinary researchers as we see the demand for consumer technologies, software and systems, as well as the in- tegration of social networks into our everyday lives. The fact that the data largely posted on social networks tends to be textual, provides a further link to linguistics, psychology and psycholinguistics to better understand the relationship between human behaviours offline and online. In this thesis, we present a novel conceptual framework based on a complex digital system using collected longitudinal datasets to predict system status based on the per- sonality traits and emotions extracted from text posted by users. The system framework was built using a dataset collected from an online scholarship system in which 2000 stu- dents had their digital behaviour and social network behaviour collected for this study. We contextualise this research project with a wider review and critical analysis of the current psycholinguistics, artificial intelligence and human-computer interaction literat- ure, which reveals a gap of mapping and understanding digital profiling against system status. Through developing and applying a hybrid approach of data science and data analysis techniques to the datasets which ultimately led to the development of the novel concep- tual model and PMSys system. The empirical foundation and validation is underpinned by a chain of experiments exploring the association and interrelations between the key parameters, linking back to the wider literature, which is used to improve the response of the intelligent agents based on the reported errors, as well as predicting the emotions raised by the user and selecting the appropriate answer. By extracting the user’s beha- viour (personality traits and emotions), the proposed conceptual model predicted 68% of the system statuses (idle, down, slow and error). Furthermore, a web-based applica- tion was developed to simulate events to users and to verify the framework; this model predicted 61% of the system statuses. Alongside the wider academic dissemination of this work, features of this novel model and system are currently being commercialised as part of an intelligent chatbot engine to provides a customer services support to a range of commercial clients across a variety of industrial sectors. ii Acknowledgements Firstly, I would like to express my sincere gratitude to my advisor Prof. Tom Crick MBE for his continuous support during my PhD study and related academic work – for his patience, motivation, and immense knowledge. His guidance helped me throughout the research and writing of this thesis; I could not have imagined having a better advisor and mentor during this journey. Besides my advisor, I would like to thank my director of studies Dr Jason Williams for always jumping in to help anytime and through my academic career and for being a great head of a department. I would like to express my special appreciation to Dr Giles Oatley for his massive support through the early stages of my research and his guidance in my academic life. I would also like to thank Dr Ana Calderon for always being there when needed for her insightful comments and encouragement and continuous support on this journey at all levels. My sincere thanks also go to Dr Yasser Elshayeb and Dr Ehab Abdelrahmen, for their support during my early career start and for always being there, I am grateful for their advice and support. I would like to dedicate this thesis to my father, who always believed in me and pushed me forward towards my goals, his guidance and encouragement were always invaluable, I am forever grateful to him (may his soul rest in peace). My sincere thanks to my mother, her prayer to me is what sustained me thus far. My brothers (Hossam and Ahmed) and sisters (Hala and Heba) and all my family members for their continuing support during my PhD journey. Last but not least, words cannot express how grateful I am to my beloved wife Marwa and my two darling daughters, I cannot thank her enough for her sacrifices, companionship, love, support and encouragement you have provided in every minute of this journey. I am beyond grateful to you – without your precious support it would not be possible to finally complete this chapter of my life. Contents Abstract i List of Figures vi List of Tables viii List of Code Listings 1 1 Introduction 2 1.1 Overview . .2 1.2 Contribution to Knowledge . .5 1.3 Publications . .5 1.4 Ethics Approval . .6 1.5 Thesis Outline . .6 2 Personality, Behaviour and Emotions 8 2.1 Introduction . .8 2.2 Psycholinguistics . .8 2.3 Personality Theories . .9 2.3.1 Cattell’s 16 Personality Factors . 10 2.3.2 The Enneagram of Personality . 11 2.3.3 Analytical Psychology (Jungian) . 12 2.3.4 Myers-Briggs Personality Types . 12 2.4 The “Big Five” Personality Traits . 12 2.4.1 Linking Online Social Networks and Personality . 15 2.5 Language Analysis . 16 2.5.1 Open Vocabulary Approaches . 17 2.5.2 Closed Vocabulary Approaches . 17 2.5.3 The Linguistic Inquiry and Word Count (LIWC) Tool . 17 2.6 Cognitive Science . 19 2.6.1 Emotional Intelligence . 19 i 2.6.2 Self Assessment of Emotions . 21 2.6.3 Temporal Behaviour . 22 2.6.4 Applications of Cognitive Science . 23 2.7 Summary . 25 3 Human-Computer Interaction 26 3.1 Introduction . 26 3.2 Applications . 27 3.3 User Experience and Usability . 27 3.4 Usability of Complex Information Systems . 29 3.4.1 System Events . 29 3.4.2 Response Times and Human Perceptions . 32 3.5 Summary . 33 4 Artificial Intelligence 34 4.1 Introduction . 34 4.2 Computational Intelligence . 35 4.2.1 Neural Networks . 36 4.2.2 Natural Language Processing . 38 4.2.3 IBM Watson Tone Analyzer . 39 4.3 Machine Learning . 40 4.4 Classifiers and Regressions . 41 4.4.1 Linear Regression . 42 4.4.2 Multiple Linear Regression . 43 4.4.3 Ordinal Regression . 44 4.4.4 Multinomial Logistics Regression . 44 4.4.5 Binomial Logistic Regression . 44 4.4.6 Mahalanobis Distance . 44 4.4.7 Naive Bayes Classifier . 45 4.5 Sentiment Analysis . 45 4.6 Summary . 46 5 Methodology 48 5.1 Introduction . 48 5.2 System Overview . 48 5.3 Data Sources and Workflows . 50 5.3.1 Motivation Letters . 51 5.3.2 Role of the Facebook Page . 51 5.3.3 Help Desk Platform and Ticketing System . 53 5.4 Identifying Computer System Status and Events . 53 5.4.1 Identifying System Status . 54 ii 5.5 Extracting Personality Traits . 58 5.5.1 Using the Mairesse Approach . 59 5.5.2 Rationale of Using the Big Five Personality Theory . 59 5.6 Extracting Emotions from Text . 59 5.7 Mapping Facebook User Profiles . 60 5.8 Verifying Accuracy using the IBM Watson Tone Analyzer . 63 5.8.1 Introduction . 63 5.8.2 Comparing Statistical Differences Between Traits . 64 5.8.3 The Mann-Whitney U Test . 65 5.8.4 Summary . 66 5.9 Summary . 66 6 Empirical Grounding for the PMsys Engine 68 6.1 Introduction . 68 6.2 Profiling Complex Online Interactions . 68 6.2.1 What Behaviour Can You Infer From a Digital Footprint? . 68 6.2.2 Parameters and Feature Extraction . 69 6.2.3 Relating a User’s Digital Behaviour and Personality Traits . 70 6.2.4 Extracting LIWC Data Features . 72 6.2.5 Discussion . 76 6.3 Mapping User Behaviour to System Stages . 77 6.3.1 Introduction . 77 6.3.2 Binomial Logistic Regression (Logistic Regression) . 83 6.3.3 Key Findings and Discussion . 84 6.4 Relationship Between Personality Traits and Emotion . 85 6.4.1 Introduction . 85 6.4.2 Personality Traits and Temporal Behaviour . 86 6.4.3 Association between Personality Traits and Six Basic Emotions 88 6.4.4 Discussion . 89 6.5 Investigating Behavioural and Emotional Change . 90 6.5.1 Introduction . 90 6.5.2 Ordinal Regression Analysis . 90 6.5.3 Multinomial Logistics Regression . 95 6.5.4 Key Findings and Discussion . 97 6.6 Incorporating Emotion and Personality-Based Analysis in User-centered Modelling . 98 6.6.1 Introduction . 98 6.6.2 Analysis . 100 6.6.3 Key Findings . 101 6.6.4 Model Evaluation . 106 6.7 Summary . 106 iii 7 Developing the Conceptual Framework for the PMSys Engine 108 7.1 Introduction . 108 7.2 Personality Traits vs. Emotions, Gender and Age . 108 7.2.1 Introduction . 108 7.2.2 Binomial Logistic Regression . 109 7.2.3 Pearson’s Partial Correlation . 111 7.2.4 Rationale of using Random Forest Tree . 116 7.2.5 Key Findings . 117 7.3 Model Verification: Observing Emotions in Real Time .