Tools for Tutoring Theoretical Computer Science Topics
Total Page:16
File Type:pdf, Size:1020Kb
University of Massachusetts Amherst ScholarWorks@UMass Amherst Doctoral Dissertations Dissertations and Theses November 2019 Tools for Tutoring Theoretical Computer Science Topics Mark McCartin-Lim University of Massachusetts Amherst Follow this and additional works at: https://scholarworks.umass.edu/dissertations_2 Part of the Artificial Intelligence and Robotics Commons, Graphics and Human Computer Interfaces Commons, Other Computer Sciences Commons, and the Theory and Algorithms Commons Recommended Citation McCartin-Lim, Mark, "Tools for Tutoring Theoretical Computer Science Topics" (2019). Doctoral Dissertations. 1797. https://doi.org/10.7275/15233091 https://scholarworks.umass.edu/dissertations_2/1797 This Open Access Dissertation is brought to you for free and open access by the Dissertations and Theses at ScholarWorks@UMass Amherst. It has been accepted for inclusion in Doctoral Dissertations by an authorized administrator of ScholarWorks@UMass Amherst. For more information, please contact [email protected]. TOOLS FOR TUTORING THEORETICAL COMPUTER SCIENCE TOPICS A Dissertation Presented by MARK MCCARTIN-LIM Submitted to the Graduate School of the University of Massachusetts Amherst in partial fulfillment of the requirements for the degree of DOCTOR OF PHILOSOPHY September 2019 College of Information and Computer Sciences c Copyright by Mark McCartin-Lim 2019 All Rights Reserved TOOLS FOR TUTORING THEORETICAL COMPUTER SCIENCE TOPICS A Dissertation Presented by MARK MCCARTIN-LIM Approved as to style and content by: Andrew McGregor, Co-chair Beverly Woolf, Co-chair David Mix Barrington, Member Siman Wong, Member James Allan, Department Chair College of Information and Computer Sciences DEDICATION To the students who faithfully came to my office hours, whose struggles and perseverance inspired this dissertation. To my good friend Lucas, who provided much needed moral support during the hardest times, and who gave me the courage to choose this dissertation topic. To my parents, who patiently waited for me to finish my Ph.D., and supported me, often financially, during that time. To Marina, who convinced me to keep going when I was almost about to quit. ACKNOWLEDGMENTS I acknowledge my long and challenging journey to produce this dissertation re- quired assistance and encouragement from many people. I want to thank them. My sincere thanks goes to my co-advisors Beverly Woolf and Andrew McGregor for their guidance, help, and forbearance with my journey. Professor McGregor was my advisor when I started my Ph.D. program in 2009 working on research projects under his supervision. When I decided to do my own research on intelligent tutoring systems in 2012, Professor McGregor was willing to remain a co-advisor to me even though his experience with intelligent tutoring systems was limited. He always found time from his busy schedule, even during his sabbatical leave, to help me when I needed it. He generously gave his vast knowledge, wisdom, and guidance to enable me to complete my journey. My interest in intelligent tutoring systems developed with my attendance of Professor Woolf's course on the subject. Her course inspired me to create the computerized tutoring system that is in my dissertation. Professor Woolf became my co-advisor in 2012. She used her expertise to help me develop the cumbersome protocol required by the IRB (Institutional Review Board) for the experimental testing of my tutoring system. Professor Woolf also advised me how to deal with the many time consuming issues imposed by the IRB. My journey could not be completed without her help and guidance. I greatly appreciate the participation of Professors David Barrington, Robert Moll, and Simon Wong, in addition to Andrew McGregor and Beverly Woolf, as members of my review committee. Professor Barrington pointed me to the work of Gries and Schneider which I referenced in my dissertation. Professor Moll loaned me a book he co-authored which I cited in my dissertation. Professor Moll is no longer on the v committee because he has retired. The guidance provided by the committee has facilitated my journey. I thank Professors Barrington, McGregor, and Krishnamurthy for providing the opportunities and assistance for me to conduct my experimental testing in conjunction with portions of the courses they taught. The experimental testing is a critical element of my dissertation. I am grateful to Alistair Sinclair for motivating me to start my journey. Professor Sinclair took me under his wing when I was a visiting undergraduate student at the University of California, Berkeley in 2005. Professor Sinclair gave me one on one mentoring sessions every week for more than a year. He exposed me to the joys of making discoveries through research, and convinced me to pursue a Ph.D. degree in computer science. I am also grateful to many colleagues, friends, family members, and my parents for their advice and encouraging messages. They gave me the morale boosting sustenance I needed to complete my long and challenging journey. vi ABSTRACT TOOLS FOR TUTORING THEORETICAL COMPUTER SCIENCE TOPICS SEPTEMBER 2019 MARK MCCARTIN-LIM B.S., UNIVERSITY OF CALIFORNIA, SANTA BARBARA M.S., UNIVERSITY OF MASSACHUSETTS AMHERST Ph.D., UNIVERSITY OF MASSACHUSETTS AMHERST Directed by: Professor Andrew McGregor and Professor Beverly Woolf This thesis introduces Complexity Tutor, a tutoring system to assist in learn- ing abstract proof-based topics, which has been specifically targeted towards the pop- ulation of computer science students studying theoretical computer science. Existing literature has shown tremendous educational benefits produced by active learning techniques, student-centered pedagogy, gamification and intelligent tutoring systems. However, previously, there had been almost no research on adapting these ideas to the domain of theoretical computer science. As a population, computer science students receive immediate feedback from compilers and debuggers, but receive no similar level of guidance for theoretical coursework. One hypothesis of this thesis is that immediate feedback while working on theoretical problems would be particularly well-received by students, and this hypothesis has been supported by the feedback of students who used the system. vii This thesis makes several contributions to the field. It provides assistance for teaching proof construction in theoretical computer science. A second contribution is a framework that can be readily adapted to many other domains with abstract math- ematical content. Exercises can be constructed in natural language and instructors with limited programming knowledge can quickly develop new subject material for Complexity Tutor. A third contribution is a platform for writing algorithms in Python code that has been integrated into this framework, for constructive proofs in computer science. A fourth contribution is development of an interactive environment that uses a novel graphical puzzle-like platform and gamification ideas to teach proof concepts. The learning curve for students is reduced, in comparison to other systems that use a formal language or complex interface. A multi-semester evaluation of 101 computer science students using Complexity Tutor was conducted. An additional 98 students participated in the study as part of control groups. Complexity Tutor was used to help students learn the topics of NP-completeness in algorithms classes and prepositional logic proofs in discrete math classes. Since this is the first significant study of using a computerized tutoring system in theoretical computer science, results from the study not only provide evidence to support the suitability of using tutoring systems in theoretical computer science, but also provide insights for future research directions. viii TABLE OF CONTENTS Page ACKNOWLEDGMENTS ............................................. v ABSTRACT ......................................................... vii LIST OF TABLES ................................................... xv LIST OF FIGURES.................................................. xvi CHAPTER 1. INTRODUCTION ................................................. 1 1.1 Challenges in teaching theoretical topics . .3 1.1.1 Student-centered pedagogy versus traditional lecture formats . .3 1.1.2 Abstract concepts disconnected from other computer science courses . .6 1.2 Insights on meta-skills of students who successfully learn theoretical computer science concepts . .7 1.2.1 Acquiring and applying factual knowledge . .7 1.2.2 Translating informal intuition into formal reasoning . .8 1.2.3 Understanding notation and semantics . .9 1.2.4 Applying analogy / proof schema . 11 1.2.5 Generating examples / inductive reasoning . 11 1.3 Components of Complexity Tutor .............................. 14 1.4 Potential advantages of Complexity Tutor ....................... 15 2. RELATED WORK USING COMPUTERS TO AID IN TEACHING THEOREM PROVING ........................... 17 2.1 Pedagogical issues with automated proof verifiers . 17 ix 2.1.1 Formal logical systems . 18 2.1.2 Differences between proofs constructed from formal logical systems and informal narrative proofs . 20 2.1.3 Can informal narrative proofs be substituted by those constructed from a formal logical system? . 21 2.1.3.1 Assertion level reasoning . 23 2.1.4 Consideration of proof granularity . 25 2.2 A brief introduction to automated theorem proving . 27 2.2.1 Resolution-based theorem provers . 29 2.2.2 Peter B. Andrews' theory of matings . 32 2.2.3 More references on automated theorem