Final Report, 20 July 2009 Usage and Impact Study of JISC‐funded Phase 1 Digitisation Projects & the Toolkit for the Impact of Digitised Scholarly Resources (TIDSR) Eric T. Meyer Kathryn Eccles Michael Thelwall Christine Madsen Oxford Internet Institute University of Oxford Report available online at http://microsites.oii.ox.ac.uk/tidsr/ Page 1 of 176 Meyer, Eccles, Thelwall, Madsen: JISC Phase 1 Usage & Impact This study was funded by the Joint Information Systems Committee (JISC) as part of the JISC Usage and Impact Study of JISC‐funded Phase 1 Digitisation Projects & Toolkit for the Impact of Digitised Scholarly Resources. For more information about the project, or for contact information: http://microsites.oii.ox.ac.uk/tidsr/welcome [Cover image: Census of Great Britain 1841, History of the Census of Great Britain, 1841, p. 85, TNA RG 27/1, available at http://www.histpop.org] Page 2 of 176 Meyer, Eccles, Thelwall, Madsen: JISC Phase 1 Usage & Impact JISC Phase 1 Usage and Impact / TIDSR Final Report Table of Contents 1 Summary .......................................................................................................................................... 7 2 Background ...................................................................................................................................... 8 2.1 Measuring Impact: Previous Studies ....................................................................................... 8 2.1.1 UCL LAIRAH ......................................................................................................................... 8 2.1.2 Berkeley Digital Resources .................................................................................................. 9 2.1.3 MIT OpenCourseWare ...................................................................................................... 10 2.1.4 Rice Impact of Digital Resources on Humanities .............................................................. 11 2.1.5 Wolverhampton‐Loughborough Digital Repository Management Practices ................... 11 2.2 Gaps in Previous Studies ....................................................................................................... 12 3 Aims and Objectives ...................................................................................................................... 13 4 Methodology and Implementation ............................................................................................... 14 4.1 Quantitative Measures ......................................................................................................... 14 4.1.1 Webometrics ..................................................................................................................... 14 4.1.2 Analytics ............................................................................................................................ 15 4.1.3 Log files ............................................................................................................................. 15 4.1.4 Scientometric/Bibliometric analysis ................................................................................. 16 4.1.5 Content analysis ................................................................................................................ 16 4.1.6 Quantitative Methods: General Limitations ..................................................................... 17 4.2 Qualitative Measures ............................................................................................................ 17 4.2.1 Stakeholder Interviews ..................................................................................................... 17 4.2.2 Resource surveys/Referrer Analysis ................................................................................. 18 4.2.3 User feedback ................................................................................................................... 18 4.2.4 Focus groups ..................................................................................................................... 19 4.2.5 Questionnaires/Surveys .................................................................................................... 19 4.2.6 Qualitative Methods: General Limitations ........................................................................ 19 5 Case studies ................................................................................................................................... 21 5.1 Histpop: The Online Historical Population Reports .............................................................. 21 5.1.1 About the project .............................................................................................................. 21 5.1.2 What we learned ............................................................................................................... 21 5.2 19th Century British Library Newspapers .............................................................................. 24 5.2.1 About the project .............................................................................................................. 24 5.2.2 What we learned ............................................................................................................... 24 5.3 British Library Archival Sound Recordings ............................................................................ 27 5.3.1 About the project .............................................................................................................. 27 5.3.2 What we learned ............................................................................................................... 28 5.4 British Official Publications Collaborative Reader Information Service (BOPCRIS): 18th Century Official Parliamentary Publications Portal .......................................................................... 30 5.4.1 About the project .............................................................................................................. 30 5.4.2 What we learned ............................................................................................................... 30 5.5 Wellcome Medical Journals Backfiles ................................................................................... 32 5.5.1 About the project .............................................................................................................. 32 5.5.2 What we learned ............................................................................................................... 32 6 Toolkit for the Impact of Digitised Scholarly Resources (TIDSR) ................................................... 34 7 Overall lessons learned and conclusions ....................................................................................... 35 8 References cited ............................................................................................................................ 37 Appendixes: Detailed Reports............................................................................................................... 38 9 Webometrics Report ..................................................................................................................... 39 Page 3 of 176 Meyer, Eccles, Thelwall, Madsen: JISC Phase 1 Usage & Impact 9.1 Introduction .......................................................................................................................... 39 9.2 Summary of methodology .................................................................................................... 39 9.2.1 Selection of project URLs .................................................................................................. 40 9.2.2 Problems in running the analysis ...................................................................................... 40 9.2.3 Selection of comparator sites ........................................................................................... 40 9.2.4 Note about limitations ...................................................................................................... 41 9.2.5 Definition of terms ............................................................................................................ 41 9.3 Project 1 – Histpop ................................................................................................................ 42 9.3.1 Project 1 Comparator sites ............................................................................................... 42 9.3.2 Overview of results ........................................................................................................... 43 9.4 Project 2 – 19th Century British Library Newspapers ............................................................ 44 9.4.1 Project 2 Comparator sites ............................................................................................... 44 9.4.2 Overview of results ........................................................................................................... 45 9.5 Project 3 – British Library Archival Sound Recordings .......................................................... 47 9.5.1 Project 3 Comparator sites ............................................................................................... 47 9.5.2 Overview of results ........................................................................................................... 48 9.6 Project 4 – BOPCRIS: 18th Century Official Parliamentary Publications Portal ....................