ADOPTING A GRAPHICAL PERSPECTIVE IN INTERACTIVE INFORMATION RETRIEVAL RESEARCH by MATTHEW MITSUI A dissertation submitted to the School of Graduate Studies Rutgers, The State University of New Jersey In partial fulfillment of the requirements For the degree of Doctor of Philosophy Graduate Program in Computer Science Written under the direction of Chirag Shah and approved by New Brunswick, New Jersey OCTOBER, 2018 ABSTRACT OF THE DISSERTATION Adopting a Graphical Perspective in Interactive Information Retrieval Research by Matthew Mitsui Dissertation Director: Chirag Shah Previous work in interactive information retrieval (IIR) has explored the relationships between individuals' search behavior, the characteristics of their search tasks, and their perceptions of their tasks, such as perceived topic familiarity and task difficulty. This work ultimately serves goals like personalization and search satisfaction. It is believed that predictions of task characteristics or searcher characteristics from observed behav- ior can help tailor search experiences to support task completion and search satisfaction. Often, research examines changes in behaviors when one or two characteristics change at a time. It applies methods such as t-tests, ANOVAs, and multivariate regression. This dissertation shows the limitations of this empirical framework. The contribution of this dissertation is in demonstrating that task characteristics, user characteristics, and behaviors should be empirically studied as a network of dependencies. It expands empirical work using graphical modeling, which can uniquely capture phenomena such as mediation and conditional independence. Research questions regarding mediation and conditional independence can hence now be answered with this different frame- work. This dissertation empirically shows when knowledge about behavior and certain task characteristics can be used to learn about other aspects of the task. It shows how task and user characteristics simultaneously affect behavior while potentially affecting ii each other. Specifically applying path analysis and Bayesian structure learning, results are shown to agree well with past literature and to also extend our understanding of the information seeking process. This dissertation discusses and shows the benefits and challenges of this modeling approach. iii Acknowledgements I'd like to think that life { albeit brief for me thus far { has imparted some pieces of wisdom to me over the years. The first is that you sometimes meet people who eventu- ally know you better than you know yourself. The second is that even with incredible perseverance and individual effort, luck and circumstance can be bigger influences in our lives than we realize. I'd like to take the time to thank the good people I've been lucky to meet over the years, some of whom know me better than I know myself and all of whom have pushed me to be a better person. And without them, this PhD certainly would not have been possible. First and foremost, I would like to sincerely thank my dissertation advisor, Prof. Chirag Shah, for his guidance and rigorous support. \Like a hawk" would do well to describe his acuity, perception, and diligence, but like all fledglings, PhD students must eventually leave the nest. He is the first listed here who has perhaps come to know me better than I know myself, and he helped push me toward success. I am grateful to him for opening me to the world of information retrieval research that is such a large part of my life today. I am grateful for his encouragement throughout all stages of the PhD process, as this dissertation took many shapes and forms and as my path went through its ups and downs. I am grateful to him for encouraging me to ask the right questions, in order to enable me to succeed in the world of research. I would like to thank my committee members, Prof. Am´elieMarian, Prof. Yongfeng Zhang, and Prof. Emine Yilmaz for their invaluable guidance. All of them have provided useful and encouraging advice which helped to inform and solidify the dissertation presented here. Thank you. I would like to thank the various professors with whom I've interacted extensively in the Department of Computer Science. This includes our past and present Graduate iv and Department chairs, including Prof. William Steiger, Prof. Matthew Stone, and Prof. Thu Nguyen. As 2 years of my life were also dedicated to teaching, I would like to thank Prof. Martin Farach-Colton, Prof. Kostas Bekris, and Prof. Brian Russell, under whom I was a Teaching Assistant. I can only hope to have fulfilled a fraction of my role here as an effective teaching assistant to the hundreds of students I've encountered. I would also like to thank the professors who encouraged me to even consider graduate school, which would not have even been an afterthought for a young undergraduate student from Southern New Jersey. Specifically, I would like to thank Prof. Alexander Schliep and the late Prof. Michael D. Grigoriadis in this regard. I would likewise like to thank all sources of funding during my PhD. These in- clude the Department of Computer Science, my advisor, the Rutgers Perceptual Science IGERT, the Institute of Museum and Library Services, the National Science Founda- tion, and various conference and summer school travel grants. Without these, the PhD would certainly have been impossible. I would also like to thank various members of the InfoSeeking laboratory for their assistance throughout the years. Many people have come through this laboratory during my stay, and I apologize for missing anyone: Dr. Chathra Hendahewa, Dr. Roberto Gonz´alez-Iba~nez,Dr. Dongho Choi, Dr. Erik Choi, Dr. Ziad Matni, Dr. Long Le, Ruoyuan Gao, Souvick Ghosh, Jiqun Liu, Soumik Mandal, Jonathan Pulliza, Manasa Rath, Shawon Sarkar, Shannon Tomlinson, Yiwei Wang, Abdurahman Sherif, SeoYoon Sung, Kevin Albertson, Anastasia Ryssiouk, SaraAnn Stanway, and Diana Soltani. I would like to thank people who have allowed me to use datasets to incorporate into my dissertation, including Souvick Ghosh, Jiqun Liu, and Yiwei Wang. I would like to give special thanks to Chathra Hendahewa for taking me under her wing during my first year in the InfoSeeking laboratory and assisting me with my transition process. I would like to thank the administrative staff at both the School of Communication and Information and the Department of Computer Science for their assistance in various administrative matters, allowing me to basically have a home in two departments. Friends have come and gone over the years. But I'd like to thank those whom I've met at the various summer schools and conferences for great conversations, tips, v advice, commiserating, and generally making the whole PhD process more fun and more memorable. I'd also like to thank my friends outside of my field who have helped give me perspective. Without them, I could neither see the forest nor the trees. If I were to list everyone, that would take several pages, and going into adequate detail would merit another dissertation. But thank you for all the great conversations at Rutgers, Pino's, various conferences, summer schools, the ever omnipresent social media, in my travels domestic and abroad, and beyond. I look forward to seeing everyone at some point in life beyond the PhD. I would like to thank my parents, who have also given me much unconditional support throughout the years, especially during the toughest times. I'd like to thank them for raising me and encouraging me to go beyond my limits. And lastly, heartfelt gratitude goes toward my partner and fellow researcher, Dr. Stephanie Anglin, the last (but certainly not least) listed here who has come to know me better than I know myself. She has taught me about patience and love and has perhaps also taught me to how to be a better person. I enjoy and appreciate our candid but energized talks about research, and I cherish the love and support we share. Thank you for being my best friend. vi Table of Contents Abstract :::::::::::::::::::::::::::::::::::::::: ii Acknowledgements ::::::::::::::::::::::::::::::::: iv 1. Introduction ::::::::::::::::::::::::::::::::::: 1 1.1. Task and Interactive Information Retrieval . 3 1.2. Problem Definition and Contribution . 4 1.3. Limitations . 6 1.4. Methods . 7 1.5. Research Questions . 7 1.6. Structure of the Dissertation . 8 2. Background :::::::::::::::::::::::::::::::::::: 10 2.1. Definitions of Task . 10 2.1.1. Tasks in non-IR vs. IR settings . 10 2.1.2. A Holistic Taxonomy for Task . 12 2.2. Theory: Putting the User in Task . 14 2.3. Characterizing Task from Behavior . 17 2.3.1. Task Type . 17 2.3.2. Complexity . 19 2.3.3. Task/Topic Familiarity and Expertise . 21 2.3.4. Task Difficulty . 23 2.3.5. Time Pressure and Other Affective and Intentional Factors . 24 2.4. Excursion: Log Task Extraction . 26 2.5. Summary . 28 vii 3. Framework :::::::::::::::::::::::::::::::::::: 34 3.1. Overview of Prior Statistical Techniques . 34 3.1.1. Independent sample tests: Usages . 35 3.1.2. Independent sample tests: Equation and Graphical Forms . 35 3.1.3. Machine Learning: Usages and Forms . 36 3.1.4. Dependent sample tests: Usages . 38 3.1.5. Dependent sample tests: Equation and Graphical Forms . 38 3.2. A Case for Graphical Modeling . 38 3.2.1. Argument 1: Conditional Independence and Mediation . 40 3.2.2. Argument 2: Bridging the Theoretical and Practical . 43 3.3. Graphical Modeling (and Structural Equation Models) in IR . 43 3.3.1. Building Graphical Models - Path Models and Bayesian Networks 45 Model Specification through Literature Review . 45 Data-Driven Model Specification . 46 3.4. Determining Sample Size . 47 4. Experimental Design :::::::::::::::::::::::::::::: 49 4.1. Experiment 1: Confirmatory Analysis . 50 4.1.1. Experimental Design . 50 4.1.2. Evaluation Methodology . 53 4.2. Experiment 2: Structure Learning . 54 4.2.1. Experimental Design . 54 4.2.2.