An Intelligent Debugging Tutor for Novice Computer Science Students Elizabeth Emily Carter Lehigh University
Total Page:16
File Type:pdf, Size:1020Kb
Lehigh University Lehigh Preserve Theses and Dissertations 2014 An Intelligent Debugging Tutor For Novice Computer Science Students Elizabeth Emily Carter Lehigh University Follow this and additional works at: http://preserve.lehigh.edu/etd Part of the Computer Sciences Commons Recommended Citation Carter, Elizabeth Emily, "An Intelligent Debugging Tutor For Novice Computer Science Students" (2014). Theses and Dissertations. Paper 1447. This Dissertation is brought to you for free and open access by Lehigh Preserve. It has been accepted for inclusion in Theses and Dissertations by an authorized administrator of Lehigh Preserve. For more information, please contact [email protected]. An Intelligent Debugging Tutor For Novice Computer Science Students By Elizabeth Emily Carter Presented to the Graduate and Research Committee Of Lehigh University In Candidacy for the Degree of Doctor of Philosophy In Computer Science Lehigh University May 2014 © Copyright by Elizabeth Carter All rights reserved ii Approved and recommended for acceptance as a dissertation in partial fulfillment of the requirements for the degree of Doctor of Philosophy __________________ Date ________________________ Dissertation Advisor (Dr. Gang Tan) ___________________ Accepted Date Committee Members: _______________________ Dr. Gang Tan ________________________ Dr. Jeff Heflin ________________________ Dr. H. Lynn Columba ________________________ Dr. Glenn Blank iii Acknowledgements: First and foremost I would like to thank my de facto advisor Dr. Glenn Blank. He was the first faculty member I spoke to when I applied to Lehigh’s PhD program and he has guided me through the entire process. His guidance and editing assistance have been absolutely invaluable. I would like to thank Dr. Blank, Dr. Henry Odi, Dr. H. Lynn Columba, Dr. Susan Szczepanski, Dr. Jennifer Swann, and Dr. Terry Hart for giving me the opportunity to work on the LVSTEM project, which funded most of my time at Lehigh. I would also like to thank my official advisor Dr. Gang Tan and my committee members (Dr. Blank, Dr. Tan, Dr. Jeff Heflin, and Dr. H. Lynn Columba) for their advice and for serving on my committee. For their advice on my experimental design and the formal evaluation process I would like to thank Dr. Edwin Kay and Dr. Diane Bates from TCNJ. Professor Sharon Kalafut, Dr. Brian D. Davison, and Dr. Brian Y. Chen for their advice, feedback, and for allowing me into their classrooms to interact with their students. I would like to thank the following people from Phillipsburg High School, Cranford High School, and Warren Hills High School for assisting me in the IRB approval process for their schools and for welcoming me into their classrooms so that I could present my system to their students. From Phillipsburg High School: Mr. George Chando, Dr. Marianne Trapani, and Mrs. Laura Wojick; from Cranford High School: Dr. Paul Ward and Mr. Richard Bell; and from Warren Hills High School: Mr. Earl Clymer, Mr. Daryl Detrick, and Mr. John Hajdu. I’d also like to thank Mr. Detrick for allotting time for me to present my system to high school faculty at two of the CSTA meetings. I would also like to thank all of the students who participated in the evaluation – this dissertation would not have been possible without you! I would also like to thank Heidi Wegrzyn, Jeanne Steinberg, and Brianne Lisk for all of their assistance with the required paperwork and other administrative issues I encountered. iv This dissertation is dedicated to my Mother and Father, Nancy and Randy Carter, who have stood by me through everything and always encouraged me to pursue all of my dreams - even when those dreams ended up being more difficult to pursue than I had ever imagined. I am so grateful to have you as parents. Thank you so much for being my best friends. v Table of Contents ABSTRACT: .................................................................................................................................. 1 1 CHAPTER 1: WHY A TUTORING SYSTEM FOR DEBUGGING? .............................. 2 1.1 INTRODUCTION................................................................................................................... 2 1.2 RESEARCH QUESTIONS ...................................................................................................... 3 1.3 CONTRIBUTIONS................................................................................................................. 6 1.4 WHAT IS AN INTELLIGENT TUTORING SYSTEM? ............................................................... 9 1.4.1 Architecture ................................................................................................................ 9 1.4.2 Methods..................................................................................................................... 13 1.5 NATURE OF THE PROBLEM AND DEFECT REPRESENTATION ........................................... 15 2 CHAPTER 2: TEACHING DEBUGGING ........................................................................ 19 2.1 EXPLICITLY TEACHING DEBUGGING SKILLS ................................................................... 19 3 CHAPTER 3: SYSTEMS FOR TEACHING DEBUGGING, DEFECT DETECTION AND SUPPORTING THE NOVICE DEBUGGER ................................................................. 27 3.1 DEFECT DETECTION SYSTEMS ......................................................................................... 27 3.2 WHYLINE.......................................................................................................................... 29 3.3 TOOLKITS ......................................................................................................................... 36 3.4 SPECIALIZED IDES AND LIBRARIES ................................................................................. 37 3.5 INSTEP ............................................................................................................................ 41 3.6 FLINT .............................................................................................................................. 43 3.7 BACKSTOP........................................................................................................................ 44 3.8 INTELLIGENT TUTORING SYSTEMS FOR PROGRAMMING................................................. 48 3.8.1 Anderson Lisp Tutor ................................................................................................. 48 vi 3.8.2 ELM-PE .................................................................................................................... 49 3.8.3 ELM-ART.................................................................................................................. 50 3.8.4 CBRMETAL .............................................................................................................. 50 3.9 INTELLIGENT TUTORING SYSTEMS FOR DEBUGGING ...................................................... 59 3.9.1 PROUST.................................................................................................................... 60 3.9.2 DebugIt ..................................................................................................................... 62 3.9.3 Problets and its Precursor........................................................................................ 63 4 CHAPTER 4: IMPLEMENTATION.................................................................................. 67 4.1 HIGH LEVEL SYSTEM OVERVIEW: ................................................................................... 68 4.1.1 Analyzer/Connector module ..................................................................................... 71 4.2 PHASE 1: HANDWRITTEN EXERCISES............................................................................... 74 4.2.1 Domain Module ........................................................................................................ 74 4.2.2 Student Module ......................................................................................................... 81 4.2.3 Pedagogical Module................................................................................................. 81 4.3 PHASE 2: DYNAMIC EXERCISE GENERATION .................................................................. 92 4.3.1 Exercise Generator................................................................................................... 93 4.3.2 Case Acquisition ..................................................................................................... 100 4.4 PHASE 3: DEBUG-TIME SUPPORT FOR NOVICE PROGRAMMERS.................................... 103 4.5 RESEARCH QUESTIONS REVISITED ................................................................................ 106 4.5.1 How should the domain be represented and reasoned about?............................... 106 4.5.2 How can the expert knowledge base be kept tractable? How can the system acquire domain knowledge?............................................................................................................. 109 4.5.3 Could the system (ITS-Debug) generate exercises dynamically?........................... 110 4.5.4 Should the system support other languages? How would supporting other languages change the system? ............................................................................................ 112 vii 4.5.5 How should the system model student knowledge? ................................................ 113 4.5.6 How should the system reason about student