A Mixed Methods Investigation on Perceived Quality in Crowdsourced Localisation
Total Page:16
File Type:pdf, Size:1020Kb
A Mixed Methods Investigation on Perceived Quality in Crowdsourced Localisation Thesis Submitted for the Degree of Doctor of Philosophy by Tabea Margret De Wille Department of Computer Science and Information Systems, University of Limerick Supervisors: Dr Chris Exton and Reinhard Schäler Submitted to the University of Limerick, April 2017 2 Abstract With thousands of language communities worldwide and ever growing access to technology and internet communication, the demand for translation of content has been growing steadily. Traditional forms of language translation cannot cope with the future needs and demands of humanity. One of the solutions described and practiced has been localisation crowdsourcing. However, one of the concerns that have been expressed is that crowdsourced localisation might not provide sufficiently high quality. One perspective on quality is that of perceived quality, which can be influenced by extrinsic cues rather than the actual, objectively observable attributes of a product. This research asks the following questions: 1. How is quality in localisation defined and to what extent do different groups agree on which aspects of quality are important? 2. What are prevalent attitudes, 'naive theories' and opinions on crowdsourcing and quality within the localisation field? 3. What is the impact of 'naive theories' on perceived quality in localisation? This research uses a mixed methods approach that includes a survey as well as a set of experiments. It targets a range of groups in localisation, from translators and buyers to students and lecturers of translation studies. As a special focus, it includes volunteers and organisations that collaborate with the non-profit organisation The Rosetta Foundation1 (TRF) to provide voluntary, crowdsourced localisation for not-for-profit projects via the platform Trommons2. Based on the data collected and analysed, this research concluded that accurate representation of meaning was of primary concern for all groups. In addition, specific aspects were raised for each group that were congruent with their main interests and perspectives. Overall, attitudes towards crowdsourced localisation were positive. However, 1 https://www.therosettafoundation.org/ 2 https://trommons.org/ 3 crowdsourcing also introduced quality uncertainty, which was especially prevalent in respondents with lower levels of training, experience and specialisation. 4 Declaration I hereby declare that this thesis is entirely my own work except for collaborations as outlined in the acknowledgements section of this thesis, and that it has not been submitted as an exercise for a degree at any other university. Parts of this thesis have been presented at conferences and published in conference proceedings. A full list of the publications and conference presentations can be consulted in appendix A. 5 Acknowledgements I am grateful and indebted to many people who have supported me over the years. To name just a few: First and foremost, I would like to thank my supervisors Dr. Chris Exton and Reinhard Schäler for their invaluable guidance, kind support and generous mentorship over the past years. I would also like to thank Dr Montserrat Bermudez Bausela at the University Alfonso X el Sabio in Madrid (UAX) for the opportunity to collaborate on design and execution of Experiment 1 and the associated questionnaire. Questionnaires 1 and 2 were conducted in collaboration with staff at The Rosetta Foundation and researchers at the Localisation Researcher Centre3. I would like to thank Paulina Abzieher, Alba Pi Barris, Rubén Escobar Capilla, Angela Daly, Geraldine Exton, Konstantinos Koutsikouris, Alison Nolan, and Soroush Saadatfar for the collaboration. Many thanks to Konstantinos Koutsikouris for conducting the coding that served as foundation for the inter coder analysis calculations, as well as for extensively testing the questionnaire. I would also like to thank Geraldine Exton for helping with proofreading this thesis. Special thanks go to my parents Eberhard and Martina De Wille who have over many years enabled me to pursue my education. I would also like to thank my siblings and friends for their support over the years. Especially Susanne De Wille, Geraldine Exton, Konstantinos Koutsikouris, and my partner Paul Broe have been my rocks and cheer leaders along this process. Thank you! Finally, I would like to thank all those that have agreed to be interviewed, fill in questionnaires, participate in experiments, provide translations/reviews and pilot questionnaires. Without you, this research would have not been possible and I am very grateful for your time and your generosity. This research is supported by Science Foundation Ireland through the CNGL Programme (Grant 12/CE/I2267) in the ADAPT Centre (www.adaptcentre.ie) at the University of Limerick 3 http://www.localisation.ie/ 6 Dedication To Emilia, Johanna, Katharina, Magdalena, Sarah-Marie and all those yet to come. 7 Table of Contents Abstract ............................................................................................................................................... 3 Declaration .......................................................................................................................................... 5 Acknowledgements ............................................................................................................................. 6 Dedication............................................................................................................................................ 7 Table of Contents ................................................................................................................................ 8 List of Tables ...................................................................................................................................... 11 List of Figures ..................................................................................................................................... 14 List of Abbreviations .......................................................................................................................... 21 Part I: Research Context .................................................................................................................... 22 Chapter 1: Introduction ................................................................................................................. 22 1.1. Introduction and Overview ................................................................................................. 22 1.2. Research Questions ............................................................................................................ 30 1.3. Thesis Structure .................................................................................................................. 33 Chapter 2: Theoretical Framework ................................................................................................ 34 2.1. The Concept and Study of Quality ...................................................................................... 34 2.2. Core Assumptions of this Thesis ......................................................................................... 40 2.3. The Quality Production Process.......................................................................................... 45 2.4. The Quality Experience Process .......................................................................................... 48 2.5. The Quality Evaluation Process .......................................................................................... 53 2.6. Summary ............................................................................................................................. 59 Chapter 3: Quality in Localisation .................................................................................................. 60 3.1. Introduction ........................................................................................................................ 60 3.2. The Theoretician's Perspective ........................................................................................... 61 3.3. The Business of Localisation ............................................................................................... 64 3.4 Applying the Integrative Quality Framework ...................................................................... 79 3.5. Summary ............................................................................................................................ 92 Chapter 4: Quality in Crowdsourced Localisation ......................................................................... 94 4.1. Crowdsourced and Volunteer Localisation ........................................................................ 94 4.2. Summary .......................................................................................................................... 127 Part II: Methodological Approach ................................................................................................... 128 Chapter 5: Methodology and Mixed Methods Research Design ................................................ 128 5.1. Introduction ..................................................................................................................... 128 5.2. Overview and Overall Study Design ................................................................................. 131 5.3. Outline of the Research Design .......................................................................................