This work is made freely available under open access. AUTHOR: TITLE: YEAR: OpenAIR citation: This work was submitted to- and approved by Robert Gordon University in partial fulfilment of the following degree: _______________________________________________________________________________________________ OpenAIR takedown statement: Section 6 of the “Repository policy for OpenAIR @ RGU” (available from http://www.rgu.ac.uk/staff-and-current- students/library/library-policies/repository-policies) provides guidance on the criteria under which RGU will consider withdrawing material from OpenAIR. If you believe that this item is subject to any of these criteria, or for any other reason should not be held on OpenAIR, then please contact [email protected] with the details of the item and the nature of your complaint. This thesis is distributed under a CC ____________ license. ____________________________________________________ CAWIML: A Computer Assisted Web Interviewing Mark-up Language Jose Miguel Lloret Perez A thesis submitted in partial fulfilment of the requirements of the Robert Gordon University for the degree of Master of Research This research programme was carried out in collaboration with Pexel Research Services October 2016 Abstract Computer-Assisted Web Interviewing (CAWI) is the new mode of conducting surveys through web browsers. This on-line solution extends the traditional paper questionnaire with functionality to inform the order of questions, the logic to guide question relevance and inconsistency checks to validate responses. Large scale international surveys are typically conducted by research agencies in multiple countries using CAWI systems. However, these demand for non-proprietary and platform independent questionnaire definitions that work throughout multiple survey systems. In this thesis, we conduct a comparative analysis at two levels: one for the different Extensible Markup Language (XML) authoring solutions that capture questionnaire features; and another to explore the architecture styles for the most popular CAWI solutions. The popular hierarchical model, employed to manage the questionnaire flow, is not semantically intuitive to domain experts and lacks flexibility to allow for ques- tionnaire design refinements. An analysis of system architectures suggests that the commonly adopted multi-page paradigm to build web pages, neither reduces the server burden nor addresses the responsiveness requirements expected from survey systems. Accordingly to address the language shortcomings we introduce a Computer Assisted Web Interviewing Markup Language (CAWIML) that uses two schema languages to validate vocabulary, structures and relationships among XML constructs and adopts a state-transition model to manage the routing and flow of questions. CAWIML serves our Representational State Transfer (REST) system to drive the design and collection stages through a single-page web build. We present our language results from testing CAWIML on a comprehensive set of real-world surveys from Pexel Research Services and use the distribution of CAWIML's vocabulary on this sample to demonstrate its coverage of questionnaire features and effective routing support. In order to evaluate our platform, we computationally simulate both the stress test for parallel processing of requests and interviewee behaviour in terms of different user interaction response configuration levels. Results suggest that both the parallelism and variation in user behaviour can be handled within acceptable levels of usability thresholds. ii Keywords: survey, questionnaire, xml, mark-up, xml schema, cai, cawi, state- transition, rest, single-page. iii Declaration of Authorship I declare that I am the sole author of this thesis and that all verbatim extracts contained in the thesis have been identified as such and all sources of information have been specifically acknowledged in the bibliography. Parts of the work presented in this thesis have appeared in the following publication: • Lloret, J. and Wiratunga, N. (2015). Survey state model (SSM). XML Authoring of Electronic Questionnaires. In XML Prague. A conference on XML, pages 159-177. (Chapter4) iv Acknowledgements Firstly, I would like to express my sincere gratitude to my supervisor Dr. Nirmalie Wiratunga for the continuous support of my MRes study and related research, for her patience, motivation, and immense knowledge. Her guidance has helped me in all the time of research and writing of this thesis. Besides my supervisor, I would like to thank Pexel Research Services and Robert Gor- don University for giving me the opportunity to undertake a Knowledge Transfer Part- nerships (KTP) project with the possibility to study for a postgraduate qualification. In particular, I am grateful to Bruce Leslie for his insightful comments and expertise at every stage of this research, but also for his task of selecting and testing the real-world survey samples through my experiments. On a more personal level, I must thank my patient and understanding girlfriend Daria for all her love and support. She has not only accepted me on all my bad days but also has encouraged me many times to put all my best for this research. I have also to mention perhaps the most faithful companion, my dog Hugo, for all the days and nights that he has been sat next to me. Last but not least, I thank my family, and in particular my mum, who has been always proud of all the efforts I have made in order to look for better career opportunities. v Contents Abstract iii Declaration of Authorship iv Acknowledgementsv 1 Introduction1 1.1 Research Motivation.............................2 1.2 Research Aim and Objectives........................3 1.3 Significance of Research Contributions...................4 1.4 Ethical Issues.................................5 1.5 Thesis Overview...............................5 2 Background7 2.1 Electronic Questionnaires..........................7 2.2 Extensible Mark-up Language (XML)................... 10 2.3 Schemas Languages.............................. 11 2.3.1 Document Type Definition (DTD)................. 12 2.3.2 XML Schema Definition (XSD)................... 13 2.3.3 Regular Expression Language for XML New Generation (RELAX NG).................................. 14 2.3.4 Schematron (SCH).......................... 14 2.4 XPath Query Language........................... 16 2.5 Conclusion.................................. 16 3 Literature 18 3.1 The XML Languages............................. 18 3.1.1 Survey Interchange Standard (Triple-S).............. 18 3.1.2 Questionnaire Definition Language (QDL)............. 19 3.1.3 Data Documentation Initiative (DDI)............... 19 vi 3.1.4 Simple Survey System (SSS).................... 19 3.2 Comparative Analysis of XML Languages................. 20 3.2.1 Routing Constructs......................... 20 3.2.2 Personalisation Constructs..................... 20 3.2.3 Routing Flow Paradigms...................... 22 3.2.4 Schema Languages.......................... 24 3.2.5 Expressions Notation......................... 25 3.2.6 Survey Stages............................. 26 3.3 CAWI Systems................................ 27 3.3.1 CAWI System Architectures..................... 27 3.3.2 Performance Testing......................... 29 3.4 Conclusions.................................. 30 4 CAWI Mark-up Language 31 4.1 The State-Transition Modelling Solution.................. 31 4.2 The RPN Notation.............................. 34 4.3 The XML Language Solution........................ 35 4.4 CAWIML Language Details......................... 35 4.4.1 The Content Constructs....................... 38 4.4.2 The Routing Constructs....................... 40 4.4.3 The Personalisation Constructs................... 45 4.5 Conclusions.................................. 49 5 CAWI System 50 5.1 Architecture of the System......................... 50 5.1.1 RESTful API............................. 51 5.1.2 Business Layer............................ 52 5.1.3 Database Solution.......................... 54 5.1.4 Single Page Application....................... 54 5.2 Implementation Details........................... 55 5.2.1 The Server Side............................ 55 5.2.2 The Client Side............................ 59 5.3 Conclusions.................................. 62 6 Evaluation 63 6.1 XML Language Evaluation......................... 63 6.2 Collection Stage Evaluation......................... 64 6.2.1 Methodology............................. 64 6.2.2 Metrics................................ 67 vii 6.2.3 Results................................ 67 6.3 Conclusions.................................. 70 7 Conclusion 72 7.1 Objectives Revisited............................. 72 7.2 Future Work................................. 73 Bibliography 75 A Appendix 78 A.1 Real Questionnaires Sample......................... 78 A.2 CAWIML Instance.............................. 78 viii List of Tables 2.1 Validation levels supported by the most popular XML schema languages 12 3.1 Comparison of XML languages for electronic questionnaires....... 21 4.1 Unary Operators of CAWIML........................ 34 4.2 Binary Operators of CAWIML....................... 36 6.1 Frequency of questionnaire constructs separated by survey........ 65 6.2 Stress test results varying No. Users.................... 69 6.3 Stress test totals for each configuration level................ 70 ix List of Figures 2.1 Fully functional paper questionnaire instance...............9 2.2 XML validation process........................... 11 2.3 Schematron