University of Southampton Research Repository ePrints Soton Copyright © and Moral Rights for this thesis are retained by the author and/or other copyright owners. A copy can be downloaded for personal non-commercial research or study, without prior permission or charge. This thesis cannot be reproduced or quoted extensively from without first obtaining permission in writing from the copyright holder/s. The content must not be changed in any way or sold commercially in any format or medium without the formal permission of the copyright holders. When referring to this work, full bibliographic details including the author, title, awarding institution and date of the thesis must be given e.g. AUTHOR (year of submission) "Full thesis title", University of Southampton, name of the University School or Department, PhD Thesis, pagination http://eprints.soton.ac.uk UNIVERSITY OF SOUTHAMPTON FACULTY OF BUSINESS, LAW AND ART School of Business Repertory Grid Technique: A Pragmatic Approach to Evaluating User Experience in Visualisation Navigation by Azira Ab Aziz Thesis for the degree of Doctor of Philosophy September 2016 UNIVERSITY OF SOUTHAMPTON ABSTRACT FACULTY OF BUSINESS, LAW AND ART School of Business Thesis for the degree of Doctor of Philosophy REPERTORY GRID TECHNIQUE: A PRAGMATIC APPROACH TO EVALUATING USER EXPERIENCE IN VISUALISATION NAVIGATION Azira Ab Aziz The aim of this study is to evaluate user experience of visualisation navigation by adopting a pragmatic approach. A pragmatic approach with the adoption of Repertory Grid Technique (RGT) reveals a different side of visualisation evaluation. Visualisation in past research has been studied through different theoretical lenses, discussed here under four headings: human cognition; technology interventions; data and information; and evaluation behaviours. From these four headings emerge objective and subjective user experience measurements. These insights have been used to demonstrate the implications of RGT and how it can generate valid data for analysing user experience. The findings elicited from 48 users demonstrate the contributions of the study specifically to evaluation approaches. The implications of RGT explored uncommon research paradigms in visualisation research, improvement in the elicitation method and extensions to the RGT, enhancing the research credibility. The outcomes derived from RGT were used as indicators to uncover user experience. These outcomes were: (1) the list of the hierarchical factors to evaluate user experience; (2) the potential implications for practice when designing visualisation navigation; and (3) the richness of the classification of visualisation navigation features generated by user experience. A comparison between the generated data and previous studies was used to demonstrate the impact on the research context. The main contributions of this research are fourfold. First, the research followed a pragmatic paradigm and adopted RGT, which is uncommon for visualisation research exploring users’ perceptions towards visualisation, and constitutes a methodological innovation. Second, it fills a gap in the theoretical basis of evaluating visualisation navigation based on user experience. It also articulates four lenses of past research: human cognition, technology interventions, data and information, and evaluation behaviours. Third, it determines the important elements of user experience towards visualisation. Fourth, the research bridges the gap between designers, users and academicians by exploring the visualisation phenomenon. Keywords: visualisation, navigation, pragmatic, repertory grid techniques, mixed method. Table of Contents ABSTRACT .......................................................................................................................... i Table of Contents .............................................................................................................. i List of tables ...................................................................................................................... v List of figures................................................................................................................... vii DECLARATION OF AUTHORSHIP...................................................................................... ix Acknowledgements ......................................................................................................... xi Definitions and Abbreviations .......................................................................................xiii Chapter 1: Introduction .......................................................................................... 1 1.1 Visualisation, Evaluation and User Experience ........................................................1 1.2 Focus of the Research .................................................................................................2 1.3 Thesis Outline ...............................................................................................................3 Chapter 2: Literature Review ................................................................................. 5 2.1 Introduction ..................................................................................................................5 2.2 Visualisation ..................................................................................................................6 2.2.1 Features of visualisation methods .............................................................8 2.2.2 Prior studies in Visualisation .................................................................... 25 2.3 Visualisation evaluation ........................................................................................... 34 2.4 Users’ Experience (UE) in Visualisation ................................................................. 40 2.4.1 Theories related to users’ experience .................................................... 40 2.4.2 Objective and subjective user experience measurements .................. 44 2.5 Research Gap (shortcoming in existing evaluation) ............................................. 47 Chapter 3: Methodology ...................................................................................... 49 3.1 Introduction ............................................................................................................... 49 3.2 Current practice of research paradigm .................................................................. 49 3.3 Pragmatic Research Design ..................................................................................... 52 3.3.1 Philosophical assumptions ....................................................................... 52 3.3.2 Theoretical background for Pragmatic (Personal Construct Theory) . 53 3.3.3 Mixed Method Approach ......................................................................... 54 i 3.4 Repertory Grid Technique (RGT) ............................................................................. 56 3.4.1 Research Context ....................................................................................... 58 3.4.2 The Characteristics of the Sample ........................................................... 58 3.4.3 Preliminary Stages (Pilot Study) ............................................................... 60 3.4.4 The Experiment and Tasks involved in RGT ............................................ 62 3.5 Research Credibility .................................................................................................. 73 3.5.1 Validity ......................................................................................................... 73 3.5.2 Reliability ..................................................................................................... 75 3.6 Ethical Consideration ................................................................................................ 77 3.7 Limitations .................................................................................................................. 78 Chapter 4: Data Analysis ...................................................................................... 79 4.1 Introduction ............................................................................................................... 79 4.2 Data Analysis Techniques for RGT ........................................................................... 80 4.3 Quantitative: Principal Component Analysis (PCA) .............................................. 84 4.3.1 Construct Correlations .............................................................................. 85 4.3.2 Relationship between Constructs and Elements ................................... 90 4.4 Qualitative: Aggregation of Thematic Analysis and Three Layer Classification Scheme ........................................................................................................................ 92 4.4.1 Identify factors elicited from respondents ............................................. 92 4.4.2 Relationship of elicited factors: Cluster Analysis ................................... 97 4.4.3 Gender Distribution ................................................................................... 99 4.4.4 Age Distribution........................................................................................ 100 4.5 Mixed method matrix (quantitative and qualitative) ......................................... 101 4.6 Summary of results ................................................................................................