A Cognitive Game for Teaching Policy Argument Matthew W. Easterday August 2010 CMU-HCII-10-106 Human-Computer Interaction Institute School of Computer Science Carnegie Mellon University Pittsburgh, PA 15213 esis Committee: Richard Scheines (Co-chair) Vincent Aleven (Co-chair) Sharon Carver (Co-chair) Gautam Biswas (Vanderbilt University) Submitted in partial fulfillment of the requirements for the Degree of Doctor of Philosophy and the Program for Interdisciplinary Educational Research Copyright © 2010 Matthew W. Easterday. All rights reserved. 1 2 Abstract Our democracy depends upon the creation of an active engaged citizenry. e purpose of this dissertation is to provide the foundational research necessary for constructing an intelligent tutoring system to teach policy deliberation. e dissertation makes five use-inspired basic research contributions to the knowledge and technology of Intelligent Tutoring Systems and Artificial Intelligence in Education. Specifically it: (a) develops a cognitive framework for deliberation, (b) localizes reasoning difficulties within the synthesis stage of the framework, (c) shows that causal diagrams can improve reasoning, (d) demonstrates that we can design intelligent tutoring systems that teach deliberation, and (e) shows that educational games can increase learning and interest by using intelligent tutoring approaches to providing assistance. 3 4 Acknowledgements is work was supported in part by a graduate training grant awarded to Carnegie Mellon University by the U.S. Department of Education (# R305B040063), the Siebel Scholars Foundation, and the Pittsburgh Science of Learning Center, which is funded by the National Science Foundation (# SBE-0836012). e opinions expressed are those of the author and do not represent the views of the U.S. Department of Education, the Siebel Scholars Foundation, or the National Science Foundation. 5 6 Table of contents 1. Democratizing deliberation through intelligent tutoring 9 2. Research on learning environments for deliberation 15 3. Localizing reasoning difficulties in synthesis 33 4. Using causal diagrams to improve policy reasoning 45 5. An instructional system for teaching deliberation 63 6. Combining games with intelligent tutors to improve learning and motivation 101 7. Conclusion: Towards a curriculum for engaged citizenship 135 Appendix A: Inquiry Environments 157 Appendix B: Representation Tools 161 Appendix C: Policy games 165 Appendix D. Argument moves 171 Appendix E. Tutoring rules 175 Appendix F: Intrinsic Motivation Inventory 181 Bibliography 183 Glossary 191 7 1. Democratizing deliberation through intelligent tutoring Summary. Our democracy depends on the creation of an active engaged citizenry. e purpose of this dissertation is to provide the foundational research necessary for constructing and an intelligent tutoring system for policy deliberation. is chapter describes five use-inspired basic research contributions to the knowledge and technology of Intelligent Tutoring Systems and Artificial Intelligence in Education. ese contributions are: (a) to develop a cognitive framework for deliberation, (b) to localize reasoning difficulties within the framework, (c) to show that causal diagrams can improve reasoning, (d) to demonstrate that deliberation can be tutored, and (e) to show that games can better increase learning and interest by using intelligent tutoring approaches to providing assistance. We are continually beset by political problems. As this dissertation is being written, we are facing the worst economic crisis since the Great Depression, we are experiencing the worst oil spill in the nation's history as the threat of global warming continues unabated; and, we are fighting two wars in Iraq and Afghanistan the latter of which is now the longest war ever waged by the United States. Our political system seems unable to prevent these catastrophes or to act decisively after crises emerge. Underlying these problems is a weakened democracy. If we were to try to define the "operating system" of our democracy: its ability to detect problems (journalism), its ability to make decisions about problems (an active engaged citizenry), and its ability to act on these decisions (representatives uncorrupted by financial influence), we can see that it does not function as well as one might hope. e institution of commercial journalism is slowly imploding, only about half the population is willing to vote (the most minimal form of political participation), and our elected officials spend the bulk of their time fundraising with a clear effect on their behavior. But we cannot even hope to repair these elements if we lack a basic foundation of civic education–a civic education that teaches students how to become active engaged citizens who can deliberate, persuade, and act. e research goal of this dissertation is to provide the foundational research necessary to construct an intelligent tutoring system for democratizing civic knowledge; that is, to help citizens learn to deliberate using software that provides automated assistance to learners (VanLehn, 2006; Woolf, 2009). is research and development question holds relevance to several fields of knowledge yet remains virtually unaddressed. Learning Science Psychologists have for the most part ignored one of the most pressing educational imperatives of our time. is is despite the fact that creating effective instruction in policy deliberation relies heavily on issues central to cognitive science including: confirmation bias, external representations, feedback, and problem solving in ill-defined domains. Having shown that citizens do not deliberate rationally on the basis of evidence, some psychologists have even argued that leaders should appeal more to emotion or better frame the issues (Westen, 2007; Lakoff, 2002). While more persuasion might win elections, it will do little to promote an active, engaged citizenry that demands better policy. e alternative, ignored by cognitive science, is civic education (e Center for Information and Research on Civic Learning and Engagement & e Carnegie Corporation of New York, 2003). 9 It is not surprising that psychologists have ignored this research problem. e development of a deliberation tutor primarily concerns the design of instructional dynamics that is, interactions between students, teachers, content, and environment. Even schools of education for which the design of instructional dynamics should be the primary focus, frequently ignore these issues (Ball & Forzani, 2007). In addition, the problem of deliberation tutoring requires resources in artificial intelligence and software engineering that are not always available to schools of education. Sciences of the Artificial, HCI, and Intelligent Tutoring research e question of developing an intelligent deliberation tutor also falls squarely within the purview of Human-Computer Interaction (HCI). HCI has been defined as the design and study of the artificial, i.e., man-made artifacts (Simon, 1996). HCI can also be thought of as the set of problems that involve the interaction of audience with technology to achieve a purpose (Buchanan, 1999). From Simon's perspective, we see that research on intelligent tutoring systems (ITS) clearly concerns the design and study of a particular form of the artificial. From Buchanan's perspective, we see that ITS concerns the interaction between an audience (learners), and a technology (intelligent tutoring systems) to achieve a purpose (expertise). In fact, the focus of this particular research question on policy and intelligent tutoring is one of the central themes in Simon's Sciences of the Artificial, which devotes a great deal of time to the design problems of social planning. is research question also falls under one of the grand challenges of information science: to provide a teacher for every student (Computing Research Association, 2003; 2005). Sadly, the intelligent tutoring community has produced little work on deliberation tutoring. is is most likely for two reasons. First, while deficits in civic skill may be the undoing of our republic, the importance of deliberation is not reflected in research spending, which emphasizes reading and STEM (science, technology, engineering and mathematics). Second, the ill-structured nature of deliberation has made the development of a viable deliberative tutoring system difficult given our current capabilities in artificial intelligence. So while the ITS community is well poised to address this research question, practical considerations push ITS research in other directions. Pasteur's Quadrant: Use-inspired, basic research e Post-WWII, U.S. model of government-sponsored research advocated by Vannever Bush holds that our nation's health, prosperity, and security depend on the technological advances of applied research, which in turn depends on basic research (Bush, 1945). is model led to the creation of the National Science Foundation. However, other studies examining the design of technological systems find that basic research may play a limited role in the development of a specific technology (Sherwin & Isenson, 1967). e more modern Pasteur's Quadrant model of research argues that for the nation to capture the technological return on its investment in basic science, we must embrace a third type of research: use-inspired, basic-research, "… work that locates the center of research in an area of basic scientific ignorance that lies at the heart of a social problem" (Stokes 1997). ere are an increasingly large number of examples of Pasteur's Quadrant research.