HARNESSING COLLECTIVE INTELLIGENCE FOR TRANSLATION: AN ASSESSMENT OF CROWDSOURCING AS A MEANS OF BRIDGING THE CANADIAN LINGUISTIC DIGITAL DIVIDE Steven O’Brien Supervisor: Dr. Ajax Persaud Thesis submitted to the Faculty of Graduate and Postdoctoral Studies in partial fulfillment of the requirements for the degree of MASTER OF SCIENCE IN ELECTRONIC BUSINESS TECHNOLOGIES Telfer School of Management Faculty of Graduate and Postdoctoral Studies University of Ottawa © Steven O‟Brien, Ottawa, Canada, 2011 Table of Contents List of Tables .................................................................................................................................. vii List of Figures ................................................................................................................................... x List of Equations ............................................................................................................................. xii Acknowledgments......................................................................................................................... xiii Abstract ......................................................................................................................................... xiv Chapter 1: Introduction .................................................................................................................. 1 1.1 Rationale and Significance .................................................................................................... 3 1.2 Anticipated Contribution ...................................................................................................... 6 Chapter 2: Theoretical Framework ................................................................................................. 7 2.1 Collaborative Online Language Transfer ............................................................................... 8 2.2 Performance Criteria ............................................................................................................. 9 2.3 Lightweight Peer Production .............................................................................................. 14 2.4 The Landscape of Document Engineering .......................................................................... 18 2.5 Problem Statement ............................................................................................................. 19 Chapter 3: Phase I (Translation Creation) - Methodology ............................................................ 21 3.1 Overview ............................................................................................................................. 21 3.2 Source Text.......................................................................................................................... 23 3.3 Qualitative Translation Goals (Translation Brief) ............................................................... 25 3.4 Amazon Mechanical Turk .................................................................................................... 25 3.5 Confluence Software ........................................................................................................... 27 3.6 Paragraph Translation Process ........................................................................................... 27 3.7 Paragraph Evaluation Process ............................................................................................. 27 3.8 Revision Creation Process ................................................................................................... 28 3.9 Revision Evaluation Process ................................................................................................ 29 Chapter 4: Phase I (Translation Creation) - Analysis ..................................................................... 30 4.1 Key Performance Indicators ................................................................................................ 30 4.2 Participation ........................................................................................................................ 30 4.3 Non-Participation ................................................................................................................ 31 4.4 Time .................................................................................................................................... 32 ii 4.5 Cost ..................................................................................................................................... 32 4.6 Evaluation Selection ............................................................................................................ 32 4.7 Diversity .............................................................................................................................. 33 4.8 Approximate Translation Quality ........................................................................................ 36 Chapter 5: Phase II (Translation Evaluation) - Methodology ........................................................ 37 5.1 Overall Translation Quality ................................................................................................. 37 5.2 Task Repetition ................................................................................................................... 38 5.3 Knowledge of Translation Methods .................................................................................... 38 5.4 General Translation Errors .................................................................................................. 38 5.5 Overt Errors ......................................................................................................................... 39 5.6 Usage Errors ........................................................................................................................ 39 5.7 Author Errors ...................................................................................................................... 40 5.8 Contextual Translation Errors ............................................................................................. 40 5.9 Translation Diversity ........................................................................................................... 40 5.10 Machine Translation ......................................................................................................... 41 5.11 Professional Translation.................................................................................................... 41 5.12 Crowdsourced Translation ................................................................................................ 41 5.13 Crowdsourced Translation Acceptance ............................................................................ 41 5.14 Research Hypotheses ........................................................................................................ 42 5.15 Hypothesis Testing ............................................................................................................ 45 5.16 Partial Least Squares – Path Modeling with Smart PLS 2.0 .............................................. 45 5.17 Structural Equation Modeling using PLS-PM .................................................................... 46 5.18 Study Participants ............................................................................................................. 48 5.19 Instrument and Measurement ......................................................................................... 51 Chapter 6: Phase II (Translation Evaluation) - Analysis ................................................................ 53 6.1 Overview ............................................................................................................................. 53 6.2 Hypothesis (H1) ................................................................................................................... 55 6.3 Hypothesis (H2) ................................................................................................................... 58 6.4 Hypothesis (H3) ................................................................................................................... 61 iii 6.5 Hypothesis (H4) ................................................................................................................... 64 6.6 Hypothesis (H5) ................................................................................................................... 66 6.7 Hypothesis (H6) ................................................................................................................... 69 6.8 Hypothesis (H7) ................................................................................................................... 73 6.9 Hypothesis (H8) ................................................................................................................... 76 6.9 Crowdsourced Translation Acceptance .............................................................................. 79 6.10 Rank Order Positively Influences Overall Translation Quality .......................................... 84 6.11 Multivariate Model of Crowdsourced Translation ........................................................... 86 Chapter 7: Discussion .................................................................................................................... 94 7.1 Limitations and Future Research .......................................................................................