On the Visualisation of Large User Models in Web Based Systems James B. Uther S I O D T E RE·ME ·MUT A N M S E · E A D A Thesis Submitted for the Degree of Doctor of Philosophy The University of Sydney November 2001 ii c James B. Uther 2001 iii ABSTRACT This thesis describes the creation and refinement of a new tool for visualising large user models, that can be made available to users on the World Wide Web. User models are the set of beliefs a (software) system holds about a user. User-adapted applications, and increasingly, web sites, use a user model to help the interaction with a user. As these models start to contain more personal and sensitive information, and affect the experience of the software user, it becomes important for the user to be able to inspect and control that data. This thesis presents work that aims to help users see an overview of the data and beliefs contained in their user model. While there has been work on scrutable user models that support exploration and user control [Kay99, ZRNG99], they have been focused on the inspection of individual model components. This thesis helps users quickly search for interesting features in models of several hundreds of components. This thesis presents the design and implementation of three iterations of the tool, and user tests of each design. The final implementation is evaluated in trial with more than 50 users. Much recent work on user-adapted systems has involved adaptive hypertext and ser- vices on the World Wide Web. An important feature of this work is its ability to work as a natural part of a web site. Furthermore, the user model format presented here leverages an Internet standard for complex metadata, allowing for inter-operation with a broad range of web services. iv ACKNOWLEDGEMENTS This thesis has been in gestation for longer than most. Therefore, the list of people to whom I owe thanks is nearly endless. I’ll try and pick out some examples here: During the course of this PhD Maria and I have seen a lot of what life has to offer. Throughout it all she has been at my side with encouragement, advice, friendship and patience. I couldn’t have kept going without her, and I know of no words that can truly express my gratitude for her support. She truly is my best friend. By the time you read this she’ll also be my wife! My supervisor A/Prof Judy Kay has, once again, shown patience and courage above and beyond the call of any academic duty. A/Prof Alan Fekete, Prof. Ann Sefton and A/Prof Bob Kummerfeld also did their share of keeping me somewhat on track through some difficult times. And of course my family were always there when I needed help. They always are, which is wonderful. For the most part I was not enrolled full time. I’d like to thank my various employers during this period for their patience in allowing me to be distracted by, and even supporting, this work, which was not always related to what they really wanted me to do. The Faculty of Medicine at the University of Sydney were always more than generous offering me space and resources to apply my work there to my research for this thesis. In particular Prof. Ann Sefton (again), Dr Jill Gordon, Mr Stewart Barnet, Prof. Simon Carlile and Mr Wayne Davies were always helpful and encouraging. My cow-orkers, Vicki & Chris, should also be mentioned for their stimulating political analysis and coffee. My more recent employers, F-Secure, have also been quite understanding in these last few months of writing. My productivity at work has dropped alarmingly and I’ve been trying to tell anyone who’ll listen that virus scanners are not complete without an integrated scrutable user model. With any luck I’ll get better now (although I’m convinced that IPSec configuration tools could well do with some user modelling help ...). v DEDICATION This thesis is dedicated to Maria. Love is putting up with several years of ruined week- ends and evenings, and still encouraging me to ruin more! Thank you. Contents 1 Introduction 1 1.1 Scrutable User Models ............................ 5 1.1.1 User Model Servers .......................... 7 1.1.2 Differing Interpretations of Data ................... 8 1.2 Summary and Thesis Overview ........................ 8 2 Background 11 2.1 User Models .................................. 11 2.1.1 Scrutable User Models ........................ 12 2.1.2 Visualisation of User Models ..................... 14 2.2 Information Visualisation ........................... 19 2.2.1 Visual Properties and Coding ..................... 20 2.2.2 Dynamic Queries ........................... 21 2.2.3 Focus + Context and Distortion ................... 25 2.2.4 Animation .............................. 30 2.3 The World Wide Web ............................. 31 2.3.1 Data .................................. 31 2.3.2 Transfer ................................ 34 2.3.3 Names ................................ 35 2.3.4 Summary ............................... 36 2.4 Knowledge Representation on the Web .................... 36 2.4.1 Metadata ............................... 36 2.4.2 Ontologies .............................. 37 2.4.3 Resource Description Format .................... 38 2.5 The Java Programming Environment ..................... 43 vii viii CONTENTS 3 Domains and Models 45 3.1 Medical Knowledge .............................. 45 3.1.1 Learning Topics ........................... 46 3.1.2 Online Assessment .......................... 46 3.1.3 The Generated Online Assessment Model .............. 51 3.2 The Movie Preferences Domain ....................... 54 4 Design Constraints and Early Design Experiments 59 4.1 Design Constraints for a Visualisation of Large User Models ........ 59 4.2 Description of a VlUM Model ........................ 61 4.2.1 The Component Data ......................... 61 4.2.2 The Graph .............................. 62 4.3 Version One .................................. 62 4.3.1 Display ................................ 63 4.3.2 Implementation ............................ 65 4.3.3 First Formative Evaluation ...................... 69 4.4 Version Two .................................. 71 4.4.1 Second Evaluation .......................... 71 5 VlUM 2.0 77 5.1 Appearances .................................. 77 5.1.1 Menu Bar ............................... 79 5.1.2 Slider ................................. 80 5.1.3 The Display .............................. 81 5.1.4 Selection ............................... 84 5.1.5 Status Bar ............................... 85 5.1.6 Experiments in Anti-Aliasing and Transparency ........... 85 5.2 File Format .................................. 86 5.2.1 Startup ................................ 88 5.3 A Software Environment for Managing and Monitoring Experiments .... 89 5.3.1 Asking Questions ........................... 90 5.3.2 Logging ................................ 94 5.3.3 Marking Questions .......................... 96 5.3.4 Benefits ................................ 97 CONTENTS ix 6 Evaluation 99 6.1Aim...................................... 99 6.2 Method ....................................100 6.3 Analysis ....................................104 6.4 Results .....................................106 6.4.1 Time to Answer ............................106 6.4.2 Percentage of Correct Answers ....................109 6.4.3 Steps Taken to Answer ........................109 6.4.4 Results by Task ............................111 6.4.5 Effect of Participant Age .......................122 6.4.6 Other Participant Differences .....................125 6.5 Summary ...................................128 7 Conclusions 131 7.1 Future Directions ...............................132 7.1.1 User Model Representation ......................132 7.1.2 Visualisation .............................133 7.1.3 Further Uses .............................135 7.2 Contributions .................................137 A Learning Topic Example 139 B Online Assessment User Surveys 141 B.1 First User Survey ...............................141 B.1.1 Mail to Students ...........................141 B.1.2 Student Responses ..........................142 B.2 Second User Survey ..............................145 C Movie Domain Question File 149 D Graphs from the Evaluation 159 D.1 Tutorial ....................................160 D.2 Experiment ..................................165 E RDF File of a Movie Recommendation Model 183 x CONTENTS F RDF File of an Online Assessment Model 185 G Movie Experiment Logs 187 G.1 Tasks Done Well ...............................187 G.2 An Average Session ..............................199 G.3 A Poor Session ................................209 H Worked Examples of Tasks 221 I Dot Graphs 229 List of Figures 2.1 The QV tool showing a user model for a user of the SAM text editor. Image courtesy of J. Kay ................................ 15 2.2 The QV tool showing a user model for a user of the SAM text editor with all branches expanded. Image courtesy of J. Kay .................. 16 2.3 An example of a VISNET visualisation of a Bayesian Belief Network.Figure from [ZRNG99]. ................................ 17 2.4 The ‘viewer control panel’ from a viewer in TAGUS. Image from [PS95]. .. 18 2.5 The ‘partition editor’ from the BGP-MS system. Image from [KP95]. .... 19 2.6 A screenshot of SEESOFT visualisation. In this example, darker lines have been executed more times in an execution of the program. ......... 21 2.7 A screenshot of a dynamic query tool for exploring the periodic table of ele- ments. Image courtesy of B. Shneiderman, http://www.cs.umd.edu/hcil/spotfire/. .......................................