ABSTRACT Title of dissertation: INTERACTIVE VISUALIZATION TECHNIQUES FOR SEARCHING TEMPORAL CATEGORICAL DATA Taowei David Wang, Doctor of Philosophy, 2010 Dissertation directed by: Professor Ben Shneiderman Department of Computer Science Temporal data has always captured people's imagination. Large databases of temporal data contain temporal patterns that can lead to the discovery of important cause-and-effect phenomena. Since discovering these patterns is a difficult task, there is a great opportunity to improve support for searching. Temporal analysis of, for example, medical records, web server logs, legal, academic, or criminal records can benefit from more effective search strategies. This dissertation describes several interactive visualization techniques designed to enhance analysts' experience in performing search, exploration, and summariza- tion of multiple sets of temporal categorical data. These techniques are imple- mented in the software Lifelines2 (http://www.cs.umd.edu/hcil/lifelines2/). Lifelines2 is an interactive visualization system that enables analysts to dynamically change their focus in order to expose temporal ordering of event sequences and study the prevalence of such orderings. This dissertation makes four main contributions. The first three are technical contributions, and the last is a process model that generalizes user behavior. First, the Align-Rank-Filter framework is presented to help analysts perform visual search and exploration. It enables analysts to center their attention on temporal events that are the focus of their inquiry. Through a controlled experiment, alignment alone is shown to improve user performance speed by up to 60% in tasks that require under- standing of temporal ordering of events. The initial successful exploration on the alignment operator led to its fuller exploitation. Further enhancements to filtering are presented to better incorporate alignment. Second, I designed and implemented the Temporal Pattern Search (TPS) algorithm for filtering to support the common, but difficult-to-specify absence of operator in a temporal pattern. TPS exploits the data structure of the visualization system, and it compares favorably to exist- ing common approaches. Third, I present the temporal summaries technique as an overview to support grouping and comparison features in Lifelines2. They support higher-level tasks such as hypothesis generation. These features take advantage of alignment, and the entirety of the system is evaluated in several long-term case studies with domain experts working on their own problems. Fourth, from these long-term case studies, I generalize a process model that describes analyst behavior in searching and interacting with temporal categorical data. Gleaning from obser- vations in the case studies, collaborators' interviews and commentaries, and logs of Lifelines2 usage, I recommend feature design guidelines for future visualization designers for temporal categorical data. The enthusiasm of the domain experts who used Lifelines2, the changing strate- gies for problem-solving, and their initial successes suggest these interactive visual- ization techniques are a valuable addition to search capabilities. INTERACTIVE VISUALIZATION TECHNIQUES FOR SEARCHING TEMPORAL CATEGORICAL DATA by Taowei David Wang Dissertation submitted to the Faculty of the Graduate School of the University of Maryland, College Park in partial fulfillment of the requirements for the degree of Doctor of Philosophy 2010 Advisory Committee: Professor Ben Shneiderman, Chair/Advisor Dr. Catherine Plaisant Professor Ben Bederson Professor Amol Deshpande Professor Galit Shmu´eli c Copyright by Taowei David Wang 2010 Dedication To my parents r{. and f_ ii Acknowledgments The path to a Ph.D. is never easy. I have had the fortune to be surrounded by incredibly capable and supportive people that made this difficult journey of growth and learning seem almost... enjoyable :) I would like to first thank my advisor Ben Shneiderman, whose tireless en- thusiasm and passion for all things HCI and, in particular, our work continue to inspire me. From writing papers with him, I learned how to be a researcher. From his communicating, engaging, and working with others, I learned how to be a better researcher. From the enthusiasm he displayed when communicating, engaging, and working with others, I learned what it means to be a great researcher. As I sit here and write these lessons, I realize that these principles do not just apply to research, but they apply in life in general. Ben, thank you for your lessons in leadership, your intellectual challenges, your inspiring enthusiasm, all of which have contributed to my growth. I would like to also thank Catherine Plaisant, whose enthusiasm rivals only Ben's. I have learned a tremendous amount from her on the procedures of working with users and many design principles. She is not only a close collaborator of my publications, she has also helped a lot in shaping my dissertation. I thank you for all your teaching and suggestions to my work along the way. I also thank you for being a stout supporter for me. My journey to Ph.D. had its highs and lows. There was a time when I was not sure if I would continue. In those most desperate days, if it were not for my iii incredibly supportive friends, I would not be writing this lengthy document (believe it or not, there is some joy in writing it). I would especially like to thank first Jennifer Golbeck, for her guidance and encouragements in those times of uncertainly and despair. If it were not for Jen, I would not have gotten in contact with Ben and Catherine and continued on my journey this far. Secondly, I would like to thank Bijan Parsia. In those darkest hours, he had also tirelessly taken upon himself to advise and to encourage me to continue on the journey to Ph.D. I would like to thank James Hendler, my previous advisor, for letting me become part of MINDSWAP, where I first began to learn how to do research and write papers, and where I met important friends. I also thank Jim for his generous financial support after he has left the university. I thank the members of the HCIL for their support, challenges, and suggestions along the way. I have enjoyed the intellectual and the not-so intellectual discussions with the students, faculty, and staff. I like the collective open-mindedness to new and strange ideas, and I appreciate very much everyone's constructive and supportive attitudes toward one another. That made the lab such an idea-fostering and friendly environment. If \Standing on the shoulders of giants" characterizes the building of knowledge from the endeavors of others, I am pleased to say that I have been fortunate to be working with the giants directly in the HCIL day-in and day-out. I have learned a lot by listening and watching the seasoned researchers at work { with devices, with kids, with students. Finally, I thank the fellow HCIL students { the giants of tomorrow {, whose mutual support, critiques and suggestions have made our work that much better. iv I thank Mark Smith and his team from the Washington Hospital Center and MedStar Health: David Roseman, Greg Marchand, Phuong Ho, and Vikramjit Mukherjee. This research would not have materialized without their generous fi- nancial support and close collaboration. I would also like to thank the following collaborators and supporters: Shawn Murphy from Harvard Medical School and Partners HealthCare, and Neil Spring from the Department of Computer Science, University of Maryland. I thank my friends, particularly Margaret Tsai, James Phongsuwan, and Holly Martinson for putting up with my neglect and keeping me sane by reminding me that there is a world outside of computer science. I suppose I should also thank my friends on Facebook, but perhaps a status update would be appropriate. Finally, I would like to thank my family { for every phone call in a holiday to remind me that I am in their thoughts, and for every phone call just because. I would like to especially thank my parents for supporting my decision of going to graduate school and live my adventure. Lastly, I would like to thank my 1993 Buick. Even though you finished before I did, I appreciate your faithful 16 years of service. v Table of Contents List of Figures x List of Abbreviations xvi 1 Introduction 1 1.1 Overview of the Dissertation . .1 1.2 Contributions . .2 1.3 Dissertation Organization . .4 2 Background and Related Work 6 2.1 Temporal Data Visualization . .7 2.1.1 Temporal Categorical Data . .7 2.1.1.1 Classical Designs . .7 2.1.1.2 Modern Innovations . 11 2.1.1.3 Multiple Record Analysis . 14 2.1.1.4 Aggregating Temporal Categorical Data . 16 2.1.1.5 Temporal Numerical Data . 18 2.1.2 Other Representation of Time . 21 2.2 Searching for Temporally-Ordered Data . 22 2.2.1 Sequential Approaches . 23 2.2.2 Novel Query Interfaces . 28 2.3 Summary . 31 3 Supporting Visual Comparisons: Alignment and Intervals of Validity 34 3.1 Overview . 34 3.2 Background and Motivation . 35 3.3 Lifelines2 . 37 3.3.1 Interface Description . 37 3.3.2 Design Discussion of Alignment . 40 3.3.3 Design Discussion of Intervals of Validity . 42 3.4 Evaluation . 43 3.4.1 Controlled Experiment . 45 3.4.1.1 Experimental Procedure . 45 3.4.1.2 Experimental Results . 50 3.4.2 Domain Expert Interviews . 55 3.4.2.1 Interview Procedure . 55 3.4.2.2 Interview Results . 57 3.5 Discussion . 61 3.5.1 Post-Experiment Design Improvements . 61 3.5.2 Generalizability and Limitations . 64 3.5.3 Impact . 64 3.6 Summary . 66 vi 4 Supporting Temporal Pattern Search: An Algorithm 67 4.1 Overview . 67 4.2 Background and Motivation . 68 4.3 Problem Description . 73 4.4 TPS Algorithm Description . 74 4.4.1 Algorithm Overview .