Computer-Assisted Translation: An Empirical Investigation of Cognitive Effort A dissertation submitted to Kent State University in partial fulfillment of the requirements for the degree of Doctor of Philosophy by Christopher Davey Mellinger May, 2014 © Copyright by Christopher Davey Mellinger 2014 All Rights Reserved ii Dissertation written by Christopher D. Mellinger B.A., Wake Forest University, 2007 M.A., Kent State University, 2009 Ph.D., Kent State University, 2014 Approved by Keiran Dunne, Chair, Doctoral Dissertation Committee Gregory Shreve, Members, Doctoral Dissertation Committee Erik Angelone Jonathan Maletic William Merriman Accepted by Keiran Dunne, Chair, Department of Modern and Classical Language Studies Raymond Craig, Dean, College of Arts and Sciences iii TABLE OF CONTENTS LIST OF FIGURES ......................................................................................................... ix LIST OF TABLES ............................................................................................................ x ACKNOWLEDGMENTS ............................................................................................... xi CHAPTER 1 ...................................................................................................................... 1 Introduction ....................................................................................................................... 1 Statement of the Problem ........................................................................................... 4 Purpose of the Study .................................................................................................. 6 Hypotheses ................................................................................................................. 8 Significance of the Study .......................................................................................... 10 Overview of the Dissertation ................................................................................... 14 CHAPTER 2 .................................................................................................................... 16 Conceptual Framework .................................................................................................. 16 Translation Technology ........................................................................................... 17 Terminology Management Systems ............................................................... 18 Terminology Extraction Tools ........................................................................ 20 Corpus Tools ................................................................................................... 21 Translation Memory Systems and Segmentation ........................................... 24 iv Machine Translation ....................................................................................... 28 Role of Translation Technology ..................................................................... 29 Post-editing .............................................................................................................. 30 Types and Levels of Post-editing of Machine Translation ............................. 31 Editing of Human Translation vs. Post-editing of Machine Translation ........ 33 Working Memory ..................................................................................................... 36 Cognitive Effort in Translation ................................................................................ 39 Perception of Effort......................................................................................... 40 CHAPTER 3 .................................................................................................................... 42 Methodology .................................................................................................................... 42 Experimental Design ................................................................................................ 43 Data Collection ............................................................................................... 44 Keystroke Logging.......................................................................................... 49 Pause Metrics .................................................................................................. 51 Segment Scoring ............................................................................................. 54 Participant Selection ................................................................................................ 56 Description of Participants .............................................................................. 57 Pre-selection Survey ....................................................................................... 59 Experimental Task ................................................................................................... 60 v Analysis........................................................................................................... 63 CHAPTER 4 .................................................................................................................... 64 Results .............................................................................................................................. 64 Data Preparation.............................................................................................. 64 General Observations ...................................................................................... 68 Exerted Effort by Match Type ........................................................................ 69 Perception of Effort vs. Keystrokes ................................................................ 73 Perception of Effort vs. Mouse Clicks ............................................................ 75 Perception of Effort vs. Time in Segment ...................................................... 76 Perception of Effort vs. Exerted Effort ........................................................... 77 Additional Statistical Information................................................................... 78 CHAPTER 5 .................................................................................................................... 83 Discussion of the Results ................................................................................................ 83 Effect of Match Levels on Cognitive Effort .............................................................. 84 General Observations and Initial Pauses......................................................... 84 Editing Behavior ............................................................................................. 87 Exerted Effort by Match Type ........................................................................ 91 Perception of Effort vs. Keystrokes ................................................................ 99 Perception of Effort vs. Mouseclicks ............................................................ 101 vi Perception of Effort vs. Time per Segment .................................................. 102 Perception of Effort vs. Exerted Effort ......................................................... 104 CHAPTER 6 .................................................................................................................. 107 CONCLUSION ............................................................................................................. 107 Implications............................................................................................................ 109 Pedagogy ....................................................................................................... 109 Research Design and Methodology .............................................................. 114 Tool Design ................................................................................................... 116 Economics of Translation ............................................................................. 117 Cognitive Model ........................................................................................... 120 Limitations of the Study ......................................................................................... 122 Future Directions ................................................................................................... 125 REFERENCES .............................................................................................................. 127 APPENDIX A: Consent Form – Pre-Screening Survey .............................................140 APPENDIX B: Pre-Screening Survey ..........................................................................141 APPENDIX C: Survey Recruitment Script .................................................................144 APPENDIX D: Experimental Instrument – Source Text ..........................................145 APPENDIX E: Experimental Instrument – Target Text Versions with Markup ...147 APPENDIX F: Experimental Instrument – Version Setup .......................................154 vii APPENDIX G: Experiment Recruitment Script ........................................................156 APPENDIX H: Consent Form – Experiment ..............................................................157 APPENDIX I: Summary of Means (SD) and one-way, repeated measures ANOVA statistics ...........................................................................................................................158