Improving Debugging Skills in the Classroom – The Effects of Teaching a Systematic Debugging Process Tilman Michaeli Ralf Romeike Computing Education Research Group Computing Education Research Group Friedrich-Alexander-Universität Erlangen-Nürnberg Freie Universität Berlin Martensstraße 3, 91058 Erlangen, Germany Königin-Luise-Str. 24-26, 14195 Berlin, Germany [email protected] [email protected] ABSTRACT 1 INTRODUCTION Debugging code is a central skill for students but also a consid- Programming requires a multitude of competencies and their teach- erable challenge when learning to program: helplessness and, in ing represents a central challenge in computer science education. consequence, frustration when confronted with errors is a common Learners do not only have to understand programming concepts, phenomenon in the K12 classroom. Debugging is distinct from gen- but they also need to be able to independently find solutions when eral programming abilities, therefore it should be taught explicitly. being confronted with errors – an essential skill in the context of Despite this, debugging is an underrepresented topic in the class- programming: Systematically examining programs for bugs, finding room as well as in computer science education research, as only and fixing them is a core competence of professional developers. few studies, materials and concepts discuss the explicit teaching of They spend between 20 and 40 percent of their working time debug- debugging. Consequently, novices are often left on their own in de- ging [27]. However, dealing with errors is particularly difficult for veloping debugging skills. This paper analyses the effectiveness of programming novices and poses a major obstacle when learning to explicitly teaching a systematic debugging process, especially with program [22]. As programming novices tend to make more errors, regard to the students’ self-efficacy and the resulting debugging this often is a major source of frustration [26]. Effective debugging performance. To this end, we developed an intervention, piloted it therefore facilitates the process of learning to program. and then examined it in a pre-post-control-group-test-design: Both Moreover, debugging skills do not only play a major role in experimental and control groups were surveyed using a question- the programming domain: debugging is also ubiquitous in our ev- naire and given debugging exercises as a pre-test. Afterward, the eryday life and research findings indicate that the explicit teach- intervention was carried out in the experimental group, while the ing of debugging can result in a transfer of debugging skills to a control group continued to work on debugging exercises. During non-programming domain [8]. Recently, debugging has gained an the post-test, the students once more worked on debugging exer- increasing amount of attention in the context of computational cises and were surveyed. The results show a significant increase thinking [35]. Thus, debugging is prominently found in recent cur- in both self-efficacy expectations and debugging performance in ricula that build upon computational thinking such as the British the experimental group in contrast to the control group. Therefore, “Computing Curriculum” [7]. our study provides empirical arguments for explicitly teaching de- Despite the importance of teaching debugging, there are surpris- bugging and simultaneously offers a hands-on approach for the ingly few studies dealing with the explicit teaching of debugging classroom. skills. At the same time, teachers lack concepts and material for teaching debugging. Instead, they focus on content such as algo- CCS CONCEPTS rithmic concepts or language syntax when teaching programming • Social and professional topics → K-12 education; [24]. Therefore, novices are often left alone with their errors and, consequently, are forced to learn debugging “the hard way”. KEYWORDS The aim of this intervention study is to examine the influence of the explicit teaching of a systematic process to debugging. With the debugging, teaching practice, CS education, intervention study focus being on the influence of this intervention on students’ self- ACM Reference format: efficacy and debugging performance. For this purpose, we developed Tilman Michaeli and Ralf Romeike. 2019. Improving Debugging Skills in the and piloted an intervention and examined it in a pre-post-control- Classroom – The Effects of Teaching a Systematic Debugging Process. In group-test-design. Proceedings of WiPSCE ’19, Glasgow, Scotland, October 23-25, 2019, 7 pages. This paper is structured as follows: In section 2, related research https://doi.org/xxx regarding a systematic debugging process and debugging in the classroom is discussed. In section 3, we outline the methodology of Permission to make digital or hard copies of part or all of this work for personal or our intervention study and the pre-post-control-group-test-design. classroom use is granted without fee provided that copies are not made or distributed Afterward, we present our results regarding the effects on self- for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. efficacy (RQ1) and debugging performance (RQ2). In section 5,the For all other uses, contact the owner/author(s). results are discussed and in section 6, we present our conclusions. WiPSCE ’19, October 23-25, 2019, Glasgow, Scotland © 2019 Copyright held by the owner/author(s). ACM ISBN xxx. https://doi.org/xxx WiPSCE ’19, October 23-25, 2019, Glasgow, Scotland Tilman Michaeli and Ralf Romeike 2 RELATED WORK Studies on error frequency for novices commonly focus either 2.1 Debugging skills on specific error types (such as errors detectable by tools orthe compiler), exclusively consider the final versions of programs (and Debugging describes the process of finding and fixing errors. De- therefore omitting interim versions), or feature only small sample bugging skills differ from general programming skills,1 as[ ] or [12] sizes. For this reason, it is difficult to make quantitative statements show. They found, that, while pronounced debugging skills usually based on literature alone. Altadmri and Brown [4] give an overview indicate corresponding skills in programming, the reverse is not of bugs commonly encountered by novices in Java, based on the necessarily true: Good programmers are not always good debuggers. analysis of the BlueJ-Blackbox data set. Their study shows that This raises the question: What makes a “good” debugger? overall semantic errors occur more frequently than syntactic ones, The use of debugging strategies – sometimes referred to as particularly once users show a certain degree of programming profi- tactics – plays an important role in the debugging process [11]: ciency – although a syntax error was the most common error found. strategies such as tracing the control flow using print-f debugging, When they do occur, these syntactic errors are fixed very quickly, commenting out code or slicing can provide information that helps due to their low error severity (typically measured based on the time to localize the error [31]. One of the main differences between ex- needed to fix an error). In a study with a limited data set, Hall etal. perts and novices is the efficiency of applying debugging strategies [16] showed that logical errors seem to be the most widespread type. [23]. Murphy et al. [25] give an overview of strategies commonly Spohrer and Soloway [32] add to this that non-construct-related employed by novices. errors occur more frequently than construct-related ones, contrary Furthermore, the proficient usage of tools – most prominently, to popular belief. Errors the compiler does not identify are con- but not limited to the debugger – can support the debugging process sidered much more difficult – and therefore time-consuming –to in a similar way as strategies and is, therefore, also seen as an find and fix [4, 25]. Nevertheless, teachers report that compile-time integral debugging skill. errors also pose a major problem for novices, especially in the K12 Often, there is no need to apply advanced debugging strategies classroom and therefore need to be addressed in teaching materials or tools in order to find and fix errors: With experience, and there- [24]. fore the application of heuristics and patterns, typical errors and their possible causes are more easily found. To support this 2.3 Teaching Debugging “learning from previous mistakes”, many professional developers maintain a debugging “diary” to document their debugging experi- Murphy et al. [25] as well as Kessler and Anderson [20] argue that ence [27]. debugging skills should be explicitly taught. Despite this, there are The application of a systematic debugging process – some- surprisingly few studies, both on university teaching and in K12, times referred to as strategy – i.e. the structured pursuit of a high- which deal with the explicit teaching of debugging. level plan, represents the fundamental skill upon which the afore- 2.3.1 Debugging in Higher Education. Chmiel and Loui [9] used mentioned skills are built. For professional debuggers, a large series voluntary debugging tasks and debugging logs to promote students’ of process models can be found (such as [13, 14, 19, 31, 34, 36]). debugging skills. It turned out that students who had completed the These models agree on the following aspects: After testing the pro- voluntary