Master Thesis 60 Credits
Total Page:16
File Type:pdf, Size:1020Kb
View metadata, citation and similar papers at core.ac.uk brought to you by CORE provided by NORA - Norwegian Open Research Archives UNIVERSITY OF OSLO Department of Informatics Effects of Expertise and Strategies on Program Comprehension in Maintenance of Object-Oriented Systems: A Controlled Experiment with Professional Developers Master thesis 60 credits Kaja Kværn [email protected] May 1st 2006 2 Abstract The research presented in this thesis identifies the comprehension strategies used by professional programmers with different expertise during maintenance tasks on a previously unknown application in Java. Furthermore the effects of the strategies on performance and the effects of expertise on strategies and performance are investigated. The skill to comprehend a program is useful, especially in relation to software maintenance. In order to improve software maintenance, knowledge of how professional programmers comprehend a program is important. This knowledge can be used to develop techniques to support the process, or to improve education. We conducted a controlled experiment with 24 professional developers from five companies. The participants were introduced to a 3600 LOC Java application on which to perform three maintenance tasks. Their actions were logged and we have developed a data analysis tool to get a detailed picture of the participants’ actions. Our further development of GRUMPS made it possible to see the participants’ use of documentation, classes visited, when they compiled and how they executed the application. The data analysis tool can be reused by researchers in similar studies and contribute to improve the understanding of developers’ program comprehension strategies. In addition to the GRUMPS data, feedback was collected through the feedback-collection method. Two different approaches to the task solving were identified. In the first one, called systematic strategy, the participants tried to get an overview of the application before performing the maintenance tasks, by reading the system documentation or running the application. 75% of the participants applied this strategy. In the second one, called as-needed strategy, the participants went straight to the task solving, without trying to acquire any knowledge of the system on which to perform the tasks. This strategy was applied by 25% of the participants. We compared the performance of the participants applying the two strategies with respect to the correctness of solution, use of documentation, use of compilation and execution facilities and the solution time. The assessment of the solutions showed that there was a correlation between modifying Task 2 successfully and using the systematic strategy. However, in Task 3 and Task 4 those who used the as-needed strategy also performed well. The participants who used the systematic strategy spent more time solving Task 2, but for Task 3 and Task 4 the results were reversed. Three levels of programming expertise were identified and from the total number of 24 participants, eight participants were object-oriented novice , four participants were procedural expert and twelve participants were object-oriented expert . It was a clear difference in the comprehension approaches between the three groups of participants. The as-needed strategy was applied most by the object-oriented novices who also compiled more. These participants showed weaker results both in Task 2 and Task 4. Thus, novices should learn from the experts in order to maintain a system successfully. 3 4 Acknowledgements First of all, I would like to thank my supervisor Amela Karahasanovi ć for her encouragement, guidance and support with this research. Her feedback and advices have been very valuable to me. I am grateful to Simula Research Laboratory for support and great facilities. I thank Richard Thomas for providing database code and documentation of great value and Gunnar J. Carelius for his technical support. I am thankful to the participants in the experiment and Rolf Vassdokken and Johan Almqvist for preparing and testing the experiment material. I also thank the Student Association at Simula Research Laboratory and Nina Elisabeth Holt for social activities and lunches during the research period. I specially thank Gøril Tømmerberg for being such a fantastic co-worker during our graduate studies and with this research. I also thank Jørgen Busvold for research collaboration and Line Joakimsen and Kristrún Arnarsdottir for inspiration and excellent teamwork in several courses. Finally, I am deeply grateful to my parents for always supporting me and believing in me throughout my studies. Oslo, May 2006 Kaja Kværn 5 6 Table of Contents 1 INTRODUCTION ..................................................................................................................................... 11 1.1 MOTIVATION ...................................................................................................................................... 11 1.2 OBJECTIVE .......................................................................................................................................... 12 1.3 RESEARCH METHOD ........................................................................................................................... 13 1.4 RESEARCH CONTEXT .......................................................................................................................... 13 1.5 CONTRIBUTION ................................................................................................................................... 13 1.6 CHAPTER OVERVIEW .......................................................................................................................... 14 2 RELATED WORK.................................................................................................................................... 17 2.1 IDENTIFICATION OF RELATED WORK .................................................................................................. 17 2.2 PROGRAM COMPREHENSION STRATEGIES ........................................................................................... 17 2.2.1 Direction of Comprehension ......................................................................................................... 18 2.2.2 Breadth of Comprehension............................................................................................................ 18 2.3 EMPIRICAL STUDIES OF COMPREHENSION STRATEGIES ...................................................................... 19 2.4 EMPIRICAL STUDIES OF EFFECT OF EXPERTISE ON PROGRAM COMPREHENSION ................................ 25 2.5 SUMMARY .......................................................................................................................................... 29 3 METHOD................................................................................................................................................... 31 3.1 PARTICIPANTS .................................................................................................................................... 31 3.2 EXPERIMENTAL PROCEDURE .............................................................................................................. 31 3.3 TOOLS AND DATA COLLECTION .......................................................................................................... 32 3.3.1 GRUMPS....................................................................................................................................... 32 3.4 TASKS ................................................................................................................................................. 36 3.5 THE APPLICATION .............................................................................................................................. 37 3.6 DOCUMENTATION ............................................................................................................................... 37 4 ANALYSIS................................................................................................................................................. 39 4.1 ANALYSIS MODEL .............................................................................................................................. 39 4.1.1 Data Preparation for the Analysis ................................................................................................ 39 4.1.2 Correctness ................................................................................................................................... 40 4.1.3 Solution Time ................................................................................................................................ 40 4.1.4 Classification of Developers ......................................................................................................... 41 4.2 IDENTIFICATION OF COMPREHENSION STRATEGIES ............................................................................ 44 4.2.1 Measures Used in Previous Studies .............................................................................................. 44 4.2.2 Definitions..................................................................................................................................... 45 4.2.3 Data from GRUMPS ..................................................................................................................... 46 4.2.4 Data from Feedback-Collection...................................................................................................