Continuous User Understanding in Software Evolution
Total Page:16
File Type:pdf, Size:1020Kb
FAKULTAT¨ FUR¨ INFORMATIK DER TECHNISCHEN UNIVERSITAT¨ MUNCHEN¨ Dissertation Continuous User Understanding in Software Evolution Jan Ole Johanßen FAKULTAT¨ FUR¨ INFORMATIK DER TECHNISCHEN UNIVERSITAT¨ MUNCHEN¨ Forschungs- und Lehreinheit 1 Angewandte Softwaretechnik Continuous User Understanding in Software Evolution Jan Ole Johanßen Vollstandiger¨ Abdruck der von der Fakultat¨ fur¨ Informatik der Technischen Univer- sitat¨ Munchen¨ zur Erlangung des akademischen Grades eines Doktors der Naturwissenschaften (Dr. rer. nat.) genehmigten Dissertation. Vorsitzender: Prof. Dr. Florian Matthes Prufende/-r¨ der Dissertation: 1. Prof. Dr. Bernd Brugge¨ 2. Prof. Dr. Barbara Paech Die Dissertation wurde am 05.09.2019 bei der Technischen Universitat¨ Munchen¨ ein- gereicht und durch die Fakultat¨ fur¨ Informatik am 31.10.2019 angenommen. Abstract Explicit user feedback represents an important source of knowledge for software developers. However, there are deficiencies in current practices of user feedback capture and utilization as they do not address the users’ tacit knowledge. Our research goal is to establish continuous user understanding so that devel- opers can extract tacit knowledge of users for utilization in software evolution. We build on the foundations of continuous software engineering (CSE), which com- prises the frequent and rapid delivery of software increments to obtain instant and diverse user feedback. We argue that in particular implicit user feedback serves as a suitable means that contributes to the extraction of users’ tacit knowledge. We conducted a semi-structured interview study with 24 practitioners from 17 companies to explore how practitioners apply CSE during software evolution and how the availability of user feedback supports CSE. We identified five recommen- dations for continuous user feedback capture and utilization of requirements. The main contribution of this dissertation is CuuSE, a framework for continu- ous user understanding in software evolution. It collects user feedback through so- called Cuu kits, which unobtrusively capture and automatically process user feed- back. CuuSE relies on feature crumbs to map user feedback to a feature under de- velopment. Another contribution is the development of a reference platform for Cu- uSE that visualizes user feedback. The platform’s feasibility and extensibility were demonstrated by implementing four Cuu kits, namely EmotionKit, ThinkingAloud- Kit, BehaviorKit, and PersonaKit. CuuSE also includes a workflow that guides de- velopers in applying the framework to extract users’ tacit knowledge. CuuSE has been validated in three cycles. In a laboratory experiment with 12 par- ticipants, we validated the applicability and feasibility of EmotionKit. The results indicate that emotional responses derived from facial expressions enable the detec- tion of usability problems. In a quasi-experiment, we introduced the Cuu syllabus as an instrument to disseminate the framework and confirmed its applicability with 9 developers. With the help of a survey, we investigated developers’ perceived use- fulness and ease of use, as well as their intend to use CuuSE. The majority of 10 de- velopers exhibited a positive attitude toward the CuuSE framework and provided helpful suggestions for future improvements. v Zusammenfassung Nutzerreaktionen sind eine wichtige Wissensquelle fur¨ die Anwendungsentwick- lung. Aktuelle Praktiken zur Erfassung und Verwendung von Nutzerreaktionen berucksichtigen¨ allerdings nicht die unausgesprochenen Eindrucke¨ von Nutzenden. Das Forschungsziel dieser Dissertation ist ein kontinuierliches Verstandnis¨ von Nutzenden, sodass Entwickelnde unausgesprochene Eindrucke¨ verstehen und fur¨ die Verbesserung ihrer Anwendung verwenden konnen.¨ Unsere Hypothese ist, dass sich insbesondere implizite Nutzerreaktionen eignen, um unausgesprochene Ein- drucke¨ von Nutzenden extrahieren zu konnen.¨ Um eine Vielzahl an Nutzerreaktio- nen einzuholen, verwenden wir einen kontinuierlichen Anwendungsentwicklungs- prozess, der auf haufigen¨ und schnellen Entwicklungszyklen basiert. Im Rahmen einer Interviewstudie mit 24 Praktizierenden aus 17 Unternehmen haben wir zunachst¨ untersucht, wie diese kontinuierlich Anwendungen entwickeln. Daraus haben wir funf¨ Empfehlungen fur¨ die kontinuierliche Erfassung und Ver- wendung von Nutzerreaktionen identifiziert. CuuSE ist ein Framework fur¨ das kontinuierliche Verstandnis¨ von Nutzerre- aktionen. Es umfasst einen Arbeitsablauf, der Entwickelnde bei dem Verstandnis¨ von Nutzerreaktionen unterstutzt.¨ Es sammelt Nutzungswissen mithilfe von Kom- ponenten fur¨ die Erfassung und Verarbeitung von Nutzerreaktionen, sogenannten Cuu Kits. CuuSE baut auf Feature Crumbs auf, die eine Verbindung zwischen Nut- zerreaktionen und Features herstellen. Die Machbarkeit und Erweiterbarkeit von CuuSE wurden durch die Entwicklung einer Referenzimplementierung und den Cuu Kits EmotionKit, ThinkingAloudKit, BehaviorKit, und PersonaKit demonstriert. Zur Validierung von CuuSE wurden drei empirische Zyklen durchgefuhrt.¨ In einem Experiment mit 12 Teilnehmenden wurde unter Laborbedingungen die Ver- wendbarkeit von EmotionKit untersucht. Die Resultate zeigen, dass emotionale Re- aktionen aus den Gesichtszugen¨ von Nutzern abgeleitet und fur¨ Ruckschl¨ usse¨ auf die Nutzerfreundlichkeit herangezogen werden konnen.¨ Zur Verbreitung und Lehre von CuuSE wurde ein Syllabus entwickelt. In einem Quasi-Experiment mit 9 Ent- wickelnden wurde dessen Anwendbarkeit bestatigt.¨ Eine Umfrage mit 10 Entwi- ckelnden bestatigte¨ die Nutzlichkeit¨ und Nutzerfreundlichkeit von CuuSE, sowie die Bereitschaft, CuuSE auch in zukunftigen¨ Projekten zu verwenden. vii Acknowledgements First and foremost, I would like to express my deepest gratitude to Professor Bernd Bruegge for the opportunity to join the Chair for Applied Software Engineering and to conduct research. I am thankful for his endless enthusiasm, countless new ideas that emerged from inspiring discussions, his trust in my work, his constant feed- back, as well as his openness and willingness to share his experiences. I am grate- ful for the knowledge that I gained from Professor Barbara Paech and thank her for the precious and helpful feedback that she provided over the last years. The research collaboration with the Chair for Software Engineering at Heidelberg Uni- versity is truly unique: I thank Anja Kleebaum, who became an indispensable col- league to discuss any aspect of our project and research work, which is reflected in the accomplishment of this dissertation. I thank the German Research Foundation DFG (Deutsche Forschungsgemeinschaft) for the support of this dissertation with the CURES project as part of the SPP1593 priority program. I am grateful to be part of the LS1 family, a chair full of trust and cheerfulness. I thank all my colleagues, in particular Monika Markl, Helma Schneider, and Uta Weber for their support with any kind of organizational matter; Matthias Linhuber, Florian Angermeier, and Florian Bodlee´ for their help as system administrators and student assistants; Sebastian Jonas for his trust and support as a mentor; Stephan Kr- usche for his scientific guidance and the discussions about soccer; Lukas Alperowitz and Nadine von Frankenberg und Ludwigsdorff for not only sharing an office, but also ideas and laughs with me; Rana Alkadhi for the great collaboration and her scientific advises; Dominic Henze and Dora Dzvonyar for their unlimited supply with positive energy; Andreas Seitz, Constantin Scheuermann, and Marian Avezum for the enjoyable chats during coffee breaks and the afternoon walks; as well as Jan Philip Bernius and Lara Marie Reimer for their excellent work as research assistants. I thank all students that participated in my research projects, in particular Thomas Gunzel,¨ Florian Schaule, Michael Frohlich,¨ Jan Philip Bernius, Lara Marie Reimer, Florian Fittschen, Nicolas Vorwerg, and Nityananda Zbil, as well as all participants of the interviews, the laboratory experiment, the quasi-experiment, and the survey. I thank my friends for their continuous support, even when this meant to skip one or another board game night. Finally, I am deeply grateful for the endless and unconditional support of my family that laid the foundation for this dissertation. ix To My Family. Contents Abstract v Zusammenfassung vii Conventions xix I Prelude 1 1 Introduction 3 1.1 Motivation . .4 1.2 Problem Context . .6 1.3 Goals . .8 1.4 Contributions . 10 1.4.1 Design Problems . 11 1.4.2 Knowledge Questions . 12 1.4.3 Research Cycles . 14 1.5 Research Scope . 16 1.6 Dissertation Structure . 18 2 Foundations 23 2.1 Concepts . 24 2.1.1 Tacit Knowledge . 24 2.1.2 Conceptual Models . 26 2.1.3 User Feedback . 27 2.1.4 Engineering and Management Processes . 30 2.2 Methodologies . 33 2.2.1 Design Science and Technical Action Research . 33 2.2.2 Empirical Research Strategies . 34 2.2.3 Measurement Strategies . 38 xiii II Problem Investigation 41 3 Problem Context Analysis and Research Setup 43 3.1 Continuous Software Engineering . 44 3.2 CURES Project . 45 3.3 Case Study Design . 48 3.3.1 Refinement of Knowledge Questions . 49 3.3.2 Case and Subject Selection . 53 3.3.3 Data Collection Procedure . 54 3.3.4 Analysis Procedure . 54 3.4 Descriptive Case Study Data . 57 3.5 Validity of Case Study Results . 59 4 Current Practice in Continuous Software Engineering 63 4.1 Results on How Companies Apply CSE . 64 4.1.1 Definitions of CSE . 65 4.1.2 Relevant Elements of CSE . 68 4.1.3 Experiences with CSE . 70 4.1.4 Future