Master Thesis Software Engineering Thesis no: MSE-2011:78 Month Year Usability and user experience: measurement model Eriks Klotins School of Computing Blekinge Institute of Technology SE-371 79 Karlskrona 1 Sweden This thesis is submitted to the School of Engineering at Blekinge Institute of Technology in partial fulfillment of the requirements for the degree of Master of Science in Software Engineering. The thesis is equivalent to XXX weeks of full time studies. Contact Information: Eriks Klotins Address: 123-46 Lubanas St. LV-1021, Riga, Latvia E-mail: [email protected] University advisor: Dr. Tony Gorschek Department of Systems and Software Engineering School of Computing Internet : Blekinge Institute of www.bth.se/com Technology Phone : +46 455 38 50 00 2 SE -371 79 Karlskrona Fax : +46 455 38 50 57 Sweden Abstract Software quality is one of success factors in software development. Usability and user experience (U&UX) as a part of software quality is becoming more and more important. Although, there has been successful attempts to formalize specific parts of U&UX there is still a need for a systematic framework of U&UX evaluation. This thesis is aimed to study the state of the art in U&UX evaluation in order to develop a single framework that comprises existing knowledge on the topic. Furthermore, the U&UX evaluation framework is aimed to support product development in industry and provide a versatile guide for U&UX practitioners. The study is based on reference based systematic review. The literature review covers both scientific publications and industrial grade papers. The papers to be reviewed were selected by their relevance to the study goals and credibility of the source. The result of this is three layer U&UX evaluation framework. First layer of the Model features breakdown structure of usability and user experience. Total number of usability and context of use attributes is 217. Second layer of the model contains guidelines of how to perform usability evaluation. Third layer features validation strategies and guidelines on how to expand the Model. In order to enable practical use of the Model both static and dynamic validation should take place. There are many models in place attempting to formalize U&UX evaluation. However, most of them focuses on particular branch of usability or are too broad to be applied practically without adaption. Furthermore, there are many resources offering practical usability and user experience checklists or guidelines. However, most of them lack connection with industry standards such as ISO/IEC 9126. The Model presented in this thesis attempts to fill the gap between high level industry standards and cook book style U&UX guidelines. Keywords: Usability, evaluation, model, guidelines 3 Table of contents 1 Introduction ........................................................................................................................... 8 1.1 Structure of the thesis .................................................................................................... 8 2 Background ............................................................................................................................ 9 2.1 Goals of usability and user experience evaluation ........................................................ 9 2.2 Usability evolution methods .......................................................................................... 9 2.3 Studies on user experience ............................................................................................ 11 3 Research methodology .......................................................................................................... 14 3.1 Problem statement ......................................................................................................... 14 3.2 Research questions ........................................................................................................ 14 3.3 Research scope .............................................................................................................. 15 3.4 Validity evaluation ........................................................................................................ 16 3.4.1 Internal validity ....................................................................................................... 16 3.4.2 Conclusion validity ................................................................................................. 16 3.4.3 External validity ...................................................................................................... 16 3.4.4 Construct validity .................................................................................................... 17 3.5 Research methods .......................................................................................................... 17 3.5.1 Literature review ..................................................................................................... 17 3.5.2 Model development ................................................................................................. 21 4 Model development ............................................................................................................... 25 4.1 Level I – structure and definition of usability and user experience ............................... 25 4.1.1 Product characteristics ............................................................................................. 26 4.1.2 Context of use characteristics .................................................................................. 27 4.1.3 User characteristics ................................................................................................. 27 4.1.4 Connection between usability, context of use and user experience ........................ 27 4.2 Level II – usability and user experience evaluation guidelines ..................................... 28 4 4.2.1 System definition ..................................................................................................... 30 4.2.2 Context of use, user and product definition ............................................................ 30 4.2.3 Definition of evaluation goals ................................................................................. 31 4.2.4 Development of tests and preparations for test execution ....................................... 31 4.2.5 Test execution ......................................................................................................... 32 4.2.6 Analysis and interpretation ...................................................................................... 33 4.2.7 Presentation and package ........................................................................................ 33 4.3 Layer III – model validation guidelines ........................................................................ 33 4.4 Static validation design – an example ........................................................................... 34 4.4.1 Step 1 - System description ..................................................................................... 34 4.4.2 Step 2 – context of use definition ............................................................................ 34 4.4.3 Step 3 – user definition ............................................................................................ 35 4.4.4 Step 4 – product definition ...................................................................................... 35 4.4.5 Definition of evaluation goals ................................................................................. 35 4.4.6 Development of test plan ......................................................................................... 35 4.4.7 Test execution ......................................................................................................... 37 4.4.8 Analysis and interpretation ...................................................................................... 37 4.4.9 Reporting the results ................................................................................................ 37 4.5 Skipped usability and user experience evaluation methods .......................................... 38 5 Results ................................................................................................................................... 39 5.1 Main results ................................................................................................................... 39 5.1.1 Usability and user experience attributes .................................................................. 39 5.1.2 Usability and user experience evaluation methodology .......................................... 39 5.1.3 Validation guidelines ............................................................................................... 40 5.2 Lessons learned Error! Bookmark not defined. 6 Conclusion ............................................................................................................................. 41 6.1 Discussion ...................................................................................................................... 41 5 6.2 Further work .................................................................................................................. 41 Appendix A - Guidelines on how to expand the Model ........................................................ 43 Appendix B – example of evaluation documentation Error! Bookmark not defined. Appendix B – criteria for