Improving Validity and Reliability in Children’s Self Reports of Technology Use by Matthew Horton A thesis submitted in partial fulfilment for the requirements of the degree of Doctor of Philosophy at the University of Central Lancashire July 2013 Student Declaration Concurrent registration for two or more academic awards I declare that while registered as a candidate for the research degree, I have not been a registered candidate or enrolled student for another award of the University or other academic or professional institution. Material submitted for another award I declare that no material contained in the thesis has been used in any other submission for an academic award and is solely my own work. Collaboration Where a candidate’s research programme is part of a collaborative project, the thesis must indicate in addition clearly the candidate’s individual contribution and the extent of the collaboration. Please state below: No work was carried out as part of a collaborative project Signature of Candidate Type of Award Doctor of Philosophy School Computing, Engineering and Physical Sciences ii Abstract Researchers working in child computer interaction are constantly seeking new methods and new techniques that will enable them to carry out more valid and more reliable research. Much of this research typically considers the design and development of new products and of new interactive techniques and researchers seek to understand how easy such innovations are for children, how much fun they are to use and how attractive they may be for use. The impact of prior technology use on the children’s responses in those contexts is the core concern of this thesis. The thesis provides a set of tools (survey instruments and guidelines) that can be used by the CCI research community to ascertain the prior experience of children with any technology and with any task. These tools are generated using theory, experience and literature and are validated through user studies. The PETT survey tool comprises three questionnaires, CTEQ, CTUQ and CTHQ and an associated user guide that clearly articulates how to use PETT and demonstrates the flexibility of PETT to be used in many contexts. The guidelines (RWC, SWC and SRT) can be applied on three levels, for general use in research with children, in the design of surveys and in the specifics of designing self-report tools for prior technology experience. iii Table of Contents Improving Validity and Reliability in Children’s Self Reports of Technology Use ................................................................................................................. i Student Declaration ........................................................................................................ ii Abstract .............................................................................................................................. iii Table of Contents ..............................................................................................................iv List of Tables ...................................................................................................................... xi List of Figures ................................................................................................................. xiii Acknowledgments......................................................................................................... xvi 1 CHAPTER ONE: INTRODUCTION .......................................................................... 1 1.1 Research Motivation ........................................................................................................ 1 1.2 Research Context .............................................................................................................. 2 1.3 Research Aim and Objectives ....................................................................................... 3 1.4 Contributions ..................................................................................................................... 4 1.5 Methodology ....................................................................................................................... 4 1.5.1 Ethics ......................................................................................................................................... 6 1.6 Thesis Structure ................................................................................................................ 6 1.7 Summary and Conclusions ............................................................................................ 8 2 CHAPTER TWO: CHILD RESEARCH ..................................................................... 9 2.1 Research with Children .................................................................................................. 9 2.1.1 What are Children? ............................................................................................................ 10 2.1.2 Known Issues ....................................................................................................................... 10 2.1.3 Child Development ............................................................................................................ 14 2.1.3.1 Jean Piaget ................................................................................................................................... 15 2.1.3.2 Lev Vygotsky .............................................................................................................................. 18 2.1.4 Ethics ....................................................................................................................................... 21 2.1.4.1 Informed Consent .................................................................................................................... 22 2.1.4.2 Confidentiality/Privacy ......................................................................................................... 23 2.1.4.3 Vulnerability .............................................................................................................................. 26 2.1.4.4 Ethics Boards/Committees.................................................................................................. 27 2.2 Child Computer Interaction ....................................................................................... 27 2.2.1 The Emergence of CCI and IDC ..................................................................................... 28 iv 2.2.2 Major Themes of CCI ......................................................................................................... 29 2.2.3 Unknowns of CCI Research ............................................................................................ 30 2.2.4 The Future of CCI ............................................................................................................... 32 2.3 Summary and Conclusions ......................................................................................... 33 3 CHAPTER 3: SURVEY METHODS ....................................................................... 37 3.1 Prior Experience ............................................................................................................ 38 3.2 Introduction to Survey Methods ............................................................................... 39 3.3 Survey Instruments ....................................................................................................... 40 3.3.1 Questionnaires .................................................................................................................... 40 3.3.2 Interviews ............................................................................................................................. 42 3.4 Populations and Samples ............................................................................................ 42 3.5 Variables ........................................................................................................................... 44 3.6 Question Types ............................................................................................................... 44 3.6.1 Open-ended Questions ..................................................................................................... 44 3.6.2 Closed Questions ................................................................................................................ 45 3.7 Pretesting ......................................................................................................................... 46 3.8 Validity............................................................................................................................... 47 3.8.1 Content Validity .................................................................................................................. 47 3.8.2 Criterion Validity ................................................................................................................ 48 3.8.3 Construct Validity .............................................................................................................. 48 3.9 Reliability ......................................................................................................................... 48 3.10 Using Survey Methods with Children ...................................................................... 49 3.10.1 Thumbs-Up Scale and Frequency of Use Scale ................................................. 50 3.10.2 How Children Answer Questions