AUTOMATIC GENERATION AND ASSESSMENT OF SOURCE CODE METHOD SUMMARIES A dissertation submitted to Kent State University in partial fulfillment of the requirements for the degree of Doctor of Philosophy by Nahla Abid April 2017 Dissertation written by Nahla Abid B.Sc., king abdulaziz university, Jeddah, Saudi Arabia, 2008 M.S., Bowling Green State University, USA, 2012 Ph.D., Kent State University, USA, 2017 Approved by _______________________________________ Dr. Jonathan Maletic, Doctoral Advisor, Department of Computer Science _______________________________________ Dr. Mikhail Nesterenko, Department of Computer Science _______________________________________ Dr. Gwenn L. Volkert, Department of Computer Science _______________________________________ Dr. Bonita Sharif, Department of Computer Science and Information Systems, Youngstown State University _______________________________________ Dr. Jocelyn R. Folk, Department of Psychological Sciences _______________________________________ Dr. Christopher Was, Department of Psychological Sciences Accepted by ______________________________________ Dr. Javed I. Khan, Professor, Chair, Department of Computer Science ______________________________________ Dr. James L. Blank, Professor, Dean, College of Arts and Sciences ii TABLE OF CONTENTS TABLE OF CONTENTS ............................................................................................... III LIST OF FIGURES ........................................................................................................ IX LIST OF TABLES .......................................................................................................... XI ACKNOWLEDGEMENTS ......................................................................................... XV CHAPTER 1 INTRODUCTION ..................................................................................... 1 1.1 Research Focus .......................................................................................................... 4 1.2 Research Goal ........................................................................................................... 5 1.3 Contributions ............................................................................................................. 5 1.4 Organization .............................................................................................................. 6 1.5 Publication Notes ...................................................................................................... 7 CHAPTER 2 BACKGROUND AND RELATED WORK ........................................... 8 2.1 Stereotype Description and Identification ................................................................. 8 2.2 Source-Code Summarization ................................................................................... 10 2.2.1 Natural-Language Processing (NLP) ............................................................ 11 2.2.2 Generating Summaries for Java Methods ..................................................... 12 2.2.3 Describing High-Level Action of Java Methods ........................................... 15 2.2.4 Describing the Formal Parameters of Java Methods ..................................... 18 2.2.5 Including Context of Methods Using Call Graph ......................................... 20 2.2.6 Determining Loop Actions ............................................................................ 22 2.2.7 Text Retrieval (TR) ....................................................................................... 25 2.2.7.1VSM and LSI .......................................................................................... 25 iii 2.2.7.2 Improving VSM Approach by Eye Tracking Study .............................. 28 2.2.8 Mining Human Descriptions ......................................................................... 30 2.2.9 Static Analysis and NLP ............................................................................... 31 2.3 Source-code Comprehension ................................................................................... 34 2.4 Eye Tracking in Software Engineering ................................................................... 38 2.4.1 Eye Tracking in Code Comprehension ................................................................... 38 2.4.2 Eye Tracking in Code Summarization .......................................................... 41 2.4.2.1 At Statement Level ................................................................................ 42 2.4.2.2 At Term Level ........................................................................................ 43 2.4.2.3 Reading Pattern ...................................................................................... 44 CHAPTER 3 METHOD DOCUMENTATION USING STREOTYPES .................. 46 3.1 The Approach .......................................................................................................... 46 3.2 Templates ................................................................................................................ 50 3.2.1 Short Description .......................................................................................... 50 3.2.2 Templates for Accessors ............................................................................... 51 3.2.3 Templates for Mutators ................................................................................. 55 3.2.4 Templates for Creational Methods. ............................................................... 58 3.2.5 External Objects ............................................................................................ 60 3.2.6 Data Members and Reference Parameters .................................................... 60 3.2.7 List of Calls and Stereotypes ......................................................................... 61 3.3 Conclusions ............................................................................................................. 62 CHAPTER 4 EVALUATION STUDY ......................................................................... 63 iv 4.1 Study Participants .................................................................................................... 64 4.2 Experimental Design and Procedure ....................................................................... 64 4.3 Data Collected ......................................................................................................... 66 4.4 Results and Discussion ............................................................................................ 67 4.4.1 Per System ..................................................................................................... 68 4.4.2 Accuracy ....................................................................................................... 69 4.4.3 Missing/Containing Important Information .................................................. 71 4.4.4 Including Unnecessary Information .............................................................. 74 4.5 Threats to Validity ................................................................................................... 75 4.6 Conclusions ............................................................................................................. 76 CHAPTER 5 IMPROVING AND EVALUATING THE IMPORTANCE OF THE FOUR DOCUMENTATION SECTIONS ........................................................ 77 5.1 Improving The Automated Approach ..................................................................... 77 5.2 Second Study ........................................................................................................... 78 5.2.1 Participants ...................................................................................................... 78 5.2.2 Experimental Design and Procedure ............................................................... 79 5.3 Results and Discussion ............................................................................................ 80 5.3.1 No short description. ....................................................................................... 82 5.3.2 No calls and stereotypes .................................................................................. 83 5.3.3 No data members read..................................................................................... 84 5.3.4 No list of objects ............................................................................................. 85 5.4 Threats to Validity ................................................................................................... 86 v 5.5 Conclusions ............................................................................................................. 86 CHAPTER 6 EYE TRACKING STUDY OF SUMMARIZING SOURCE CODE . 87 6.1 Study Participants .................................................................................................... 89 6.2 Study Systems and Method Selection ..................................................................... 89 6.3 Study Procedure and Instrumentation ..................................................................... 91 6.4 Eye Tracking Apparatus and Measures ................................................................... 92 6.5 Correctness .............................................................................................................. 93 6.6 Gaze Behavior Inside the Method ........................................................................... 93 6.6.1 Term locations Identification