An Exploration of the Effects of Enhanced Compiler Error Messages for Computer Programming Novices
Total Page:16
File Type:pdf, Size:1020Kb
Technological University Dublin ARROW@TU Dublin Theses LTTC Programme Outputs 2015-11 An Exploration Of The Effects Of Enhanced Compiler Error Messages For Computer Programming Novices Brett A. Becker Technological University Dublin Follow this and additional works at: https://arrow.tudublin.ie/ltcdis Part of the Computer and Systems Architecture Commons, and the Educational Methods Commons Recommended Citation Becker, B. (2015) An exploration of the effects of enhanced compiler error messages for computer programming novices. Thesis submitted to Technologicl University Dublin in part fulfilment of the requirements for the award of Masters (M.A.) in Higher Education, November 2015. This Theses, Masters is brought to you for free and open access by the LTTC Programme Outputs at ARROW@TU Dublin. It has been accepted for inclusion in Theses by an authorized administrator of ARROW@TU Dublin. For more information, please contact [email protected], [email protected]. This work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 4.0 License An Exploration of the Effects of Enhanced Compiler Error Messages for Computer Programming Novices A thesis submitted to Dublin Institute of Technology in part fulfilment of the requirements for the award of Masters (M.A.) in Higher Education by Brett A. Becker November 2015 Supervisor: Dr Claire McDonnell Learning Teaching and Technology Centre, Dublin Institute of Technology Declaration I certify that this thesis which I now submit for examination for the award of Masters (M.A.) in Higher Education is entirely my own work and has not been taken from the work of others, save and to the extent that such work has been cited and acknowledged within the text of my own work. This thesis has not been submitted in whole or in part for another award in any other third level institution other than part-fulfilment of the award named above. The work reported on in this thesis conforms to the principles and requirements of Dublin Institute of Technology's guidelines for ethics in research. Dublin Institute of Technology has permission to keep, lend or copy this thesis in whole or in part, on condition that any such use of the material of the thesis be duly acknowledged. Signed.................................................. Date...................................................... Abstract Computer programming is an essential skill that all computing students must master and is increasingly important in many diverse disciplines. It is also difficult to learn. One of the many challenges novice programmers face from the start are notoriously cryptic compiler error messages. These report details on errors made by students and are essential as the primary source of information used to rectify those errors. However these difficult to understand messages are often a barrier to progress and a source of discouragement. A high number of student errors, and in particular a high frequency of repeated errors – when a student makes the same error consecutively – have been shown to be indicators of students who are struggling with learning to program. This instrumental case study research investigates the student experience with, and the effects of, software that has been specifically written to help students overcome their challenges with compiler error messages. This software provides help by enhancing error messages, presenting them in a straightforward, informative manner. Two cohorts of first year computing students at an Irish higher education institution participated over two academic years; a control group in 2014-15 that did not experience enhanced error messages, and an intervention group in 2013-14 that did. This thesis lays out a comprehensive view of the student experience starting with a quantitative analysis of the student errors themselves. It then views the students as groups, revealing interesting differences in error profiles. Following this, some individual student profiles and behaviours are investigated. Finally, the student experience is discovered through their own words and opinions by means of a survey that incorporated closed and open-ended questions. In addition to reductions in errors overall, errors per student, and the key metric of repeated error frequency, the intervention group is shown to behave more cohesively with fewer indications of struggling students. A positive learning experience using the software is reported by the students and the lecturer. These results are of interest to educators who have witnessed students struggle with learning to program, and who are looking to help remove the barrier presented by compiler error messages. This work is important for two reasons. First, the effects of error message enhancement have been debated in the literature – this work provides evidence that there can be positive effects. Second, these results should be generalisable at least in part, to other languages, students and institutions. Acknowledgements I would like to acknowledge the professional guidance, support and amazing patience of my supervisor Dr. Claire McDonnell, and Dr. Jen Harvey for serving as second reader. I would also like to thank all of the staff of the Dublin Institute of Technology Learning, Teaching and Technology Centre for their constructive suggestions and enthusiastic encouragement during the past two very enjoyable years. I am very appreciative for the support of the president of the College of Computing Technology, Neil Gallagher, and the extremely helpful assistance of Ricardo Iwashima and (soon to be Dr.) Graham Glanville. I am also indebted to Dr. Fiona O’Riordan for proofreading and many enjoyable conversations, not only limited to this work. Finally I am very thankful to my mother Sharon, who has read every word in now four theses, and who had to endure way too much talk about them. This is a similar plight faced by my fiancée Dr. Catherine Mooney, who I would like to thank for everything, but will only cite here her amazing patience and skill in data analysis. Without her help this work would only be a shadow of what it has become. Contents List of Figures ................................................................................................................. i List of Tables .................................................................................................................. i Abbreviations and Definitions ....................................................................................... ii Chapter 1 Introduction .............................................................................................. 1 1.1 Context ................................................................................................................. 3 1.1.1 Impact of Compiler Error Messages on Novice Programmers ..................... 3 1.1.2 My Own Context........................................................................................... 5 1.2 Rationale and Motivation ..................................................................................... 5 1.3 Research Aims & Objectives ............................................................................... 8 1.4 Thesis Layout ....................................................................................................... 9 Chapter 2 Related Work ......................................................................................... 10 2.1 Compiler Error Messages: Importance and Difficulties .................................... 10 2.3 Patterns of Behaviour and Links to Performance .............................................. 13 2.2 Frequency, Categorisation and Databases of Errors .......................................... 15 2.3 Compiler Error Enhancement ............................................................................ 18 2.3.1 Systems for Teaching Novices.................................................................... 18 2.3.2 Industrial Systems ....................................................................................... 23 2.4 The Current Challenge ....................................................................................... 23 Chapter 3 Research Methodology and Methods..................................................... 25 3.1 Theoretical Perspective ...................................................................................... 25 3.2 Research Methodology ...................................................................................... 26 3.3 Research Methods .............................................................................................. 28 3.3.1 Quantitative ................................................................................................. 29 3.3.2 Qualitative ................................................................................................... 30 3.4 Ethical Considerations ....................................................................................... 31 3.5 Technical Implementation Details ..................................................................... 32 Chapter 4 Findings and Analysis ............................................................................ 36 4.1 Do enhanced compiler error messages reduce the overall number of errors? ... 39 4.1.1 High-Frequency Errors ............................................................................... 41 4.1.2 Summary ..................................................................................................... 45 4.2 Do enhanced compiler error messages