Librarians' Perceptions of Quality Digital Reference Services
Total Page：16
File Type：pdf, Size：1020Kb
LIBRARIANS' PERCEPTIONS OF QUALITY DIGITAL REFERENCE SERVICES BY MEANS OF CRITICAL INCIDENTS by Eylem Ozkaramanl³Ä B.A., Hacettepe University, 1998 M.A., Hacettepe University, 2000 Submitted to the Graduate Faculty of the Department of Library and Information Science School of Information Sciences in partial ful¯llment of the requirements for the degree of Doctor of Philosophy University of Pittsburgh February, 2005 UNIVERSITY OF PITTSBURGH DEPARTMENT OF LIBRARY AND INFORMATION SCIENCE SCHOOL OF INFORMATION SCIENCES This dissertation was presented by Eylem Ozkaramanl³Ä It was defended on February 25, 2005 and approved by Edie M. Rasmussen, Ph.D., Advisor Jose-Marie Gri±ths, Ph.D. David Robins, Ph.D. Gloriana St. Clair, Ph.D. Dissertation Director: Edie M. Rasmussen, Ph.D., Advisor ii Copyright °c by Eylem Ozkaramanl³Ä February, 2005 iii LIBRARIANS' PERCEPTIONS OF QUALITY DIGITAL REFERENCE SERVICES BY MEANS OF CRITICAL INCIDENTS Eylem Ozkaramanl³,Ä Ph.D. University of Pittsburgh, February, 2005 This research is an e®ort to understand chat reference services through librarians' perceptions of successful and less successful chat reference service. Ten Academic libraries in Ohio and Pennsylvania which o®ered chat reference services were identi¯ed and 40 librarians were interviewed in order to address the research questions of this study. The main methodology used was the Critical Incident Technique (CIT) that is based on collecting and analyzing the most memorable experiences of human behavior in order to evaluate and identify ways to increase e®ectiveness of service. On-site, personal interviews were conducted with librarians who provide chat reference services. The subjects were initially asked to de¯ne chat reference service and compare it with traditional reference services. Following CIT procedures, they were then asked to recall and describe successful and less successful chat reference interactions and to make suggestions for better chat reference services. The interviews were transcribed and entered into a database for coding and content analysis of the collected data using qualitative data analysis software (MAXqda). Coded data were transformed into categories to determine and describe librarians' perceptions of chat reference services. The six major themes that emerged from this study were: 1-Characteristics of chat reference, 2-Attitudes of librarians and users, 3-E±ciency of reference interview and question negotiation, 4-Service improvement and management issues, 5-Training and review, and 6-Publicity and user awareness. These themes were discussed throughout the study. Findings from the recorded critical incidents indicate the importance of \attitudes" of librarians and users, the role of question \negotiation" and \type", and the availability of \resources" in successful chat iv reference service. The de¯ning characteristics of chat reference, that it is online, remotely available and delivered through software raised issues of \technology", \chat software", \service location" and \service hours" which were also signi¯cant in de¯ning service success. Furthermore, investigation of reference service quality criteria, evaluation measures and methods were explored by comparing literature on traditional and chat reference services and study ¯ndings. This study provides practical evaluation criteria for providing successful chat reference services in three categories based on: librarians' performance, chat software, and marketing issues. Further research for developing comprehensive digital reference evaluation criteria is recommended. v TABLE OF CONTENTS 1.0 INTRODUCTION AND OVERVIEW .................... 1 1.1 Background and Purpose of the Study ..................... 1 1.2 Research Questions and Assumptions ...................... 3 1.3 Structure and Outline .............................. 4 1.4 Limitations .................................... 5 2.0 LITERATURE REVIEW ............................. 6 2.1 Introduction ................................... 6 2.2 The Reference Interview ............................. 7 2.3 Digital Reference Services (DRS) : De¯nitions and Development ....... 9 2.3.1 E-mail and Web Form Based Reference Services ............ 10 2.3.2 Real-Time Reference Services ....................... 13 2.4 Quality Evaluation and Measurement: Traditional vs. Digital Reference Services ................... 17 2.4.1 Evaluation for Quality ........................... 17 2.4.2 Evaluation in Reference Services ..................... 18 2.4.2.1 User vs. Librarian Perspectives on Evaluation ......... 21 2.4.3 Establishing Quality Standards to Reference Services ......... 22 2.4.3.1 Traditional Reference Services .................. 22 2.4.3.2 Digital Reference Services .................... 23 2.5 Summary ..................................... 27 3.0 RESEARCH DESIGN AND METHODOLOGY .............. 28 3.1 Introduction ................................... 28 vi 3.2 Research Questions ................................ 28 3.3 Pilot Study .................................... 30 3.3.1 Interview Questions ............................ 30 3.3.2 Subjects .................................. 31 3.3.3 Pilot Data Analysis ............................ 31 3.3.3.1 Pilot Study Critical Incidents Findings ............. 36 3.3.3.2 Summary of Pilot Study ..................... 38 3.4 Methodological Framework of the Main Study ................. 39 3.4.1 Selection of Sites .............................. 39 3.4.2 Selection of Subjects ............................ 40 3.4.3 Data Collection Methods ......................... 41 3.4.3.1 Critical Incident Technique (CIT) ................ 41 3.4.3.2 Design of the Study Using CIT ................. 42 3.4.3.3 Content Analysis ......................... 43 4.0 RESULTS OF THE STUDY ........................... 45 4.1 Development of the Coding Scheme ....................... 45 4.1.1 Theme I: Characteristics of Chat Reference ............... 47 4.1.2 Theme II: Attitudes of Librarians' and Users .............. 49 4.1.3 Theme III: E±ciency of Reference Interview and Question Negotiation 52 4.1.4 Theme IV: Service Improvement and Management Suggestions .... 54 4.1.5 Theme V: Training and Review ..................... 57 4.1.6 Theme VI: Publicity and User Awareness ................ 59 4.2 Analysis of Critical Incidents .......................... 60 4.2.1 Themes of Critical Incidents ....................... 60 4.2.1.1 Attitudes ............................. 63 4.2.1.2 Question Negotiation and Resources .............. 65 4.2.1.3 Technical and Software ...................... 68 4.2.1.4 Location and Service Hours ................... 70 4.3 Overall Summary of Critical Incidents ..................... 71 4.4 Reliability Test .................................. 75 vii 5.0 DISCUSSION .................................... 78 5.1 Major Findings .................................. 78 5.1.1 Results Related to Research Questions One, Two and Five ...... 84 5.1.1.1 Chat Reference Evaluation Criteria and Measurement: .... 86 5.1.2 Results Related to Research Questions Three and Four ........ 91 5.2 Conclusions .................................... 93 5.2.1 Limitations of the Study ......................... 95 5.2.2 Suggestions for Further Research ..................... 96 BIBLIOGRAPHY .................................... 98 APPENDIX A. EXAMPLE TRANSCRIPT FOR CHAT REFERENCE .. 105 APPENDIX B. EXAMPLE WEB BASED QUERY FORM ........... 109 APPENDIX C. INFORMED CONSENT FORM ................ 111 APPENDIX D. LIBRARIAN PROFILE FORM ................. 112 APPENDIX E. INTERVIEW QUESTIONS ................... 113 APPENDIX F. MAIN STUDY CATEGORY SCHEME ............ 114 APPENDIX G. CRITICAL INCIDENTS CATEGORY SCHEME ..... 119 APPENDIX H. CHAT REFERENCE EVALUATION CRITERIA ..... 122 APPENDIX I. SCREEN SHOT PSU-1 ...................... 125 APPENDIX J. SCREEN SHOT: PSU-2 ...................... 126 APPENDIX K. SCREEN SHOT: OHIO STATE UNIVERSITY ....... 127 APPENDIX L. SCREEN SHOT: OHIO STATE UNIV.- OHIOLINK .... 128 APPENDIX M. SCREEN SHOT: OHIO STATE UNIV.- OHIOLINK-2 .. 129 viii LIST OF TABLES 1 Librarian Pro¯les and Experience ........................ 46 2 Typical Chat Reference Characteristics ..................... 50 3 Typical Librarian and User Attitudes ...................... 53 4 Typical Question Negotiation ........................... 55 5 Typical Service Improvement and Management ................. 58 6 Typical Training and Review ........................... 61 7 Typical Publicity and User Awareness ...................... 61 8 Distribution of Critical Incidents ......................... 62 9 Distribution of Critical Incidents Between Schools and Subjects ....... 62 10 Illustration of Reviewers and Researcher Agreement Level ........... 77 11 Extract from Categories as Advices ....................... 81 12 Literature on Reference Services Evaluation Measures ............. 85 13 Evaluation Techniques ............................... 90 14 DRS Guidelines vs. Librarians' Perceptions ................... 94 ix LIST OF FIGURES 1 Sample screen from MAXqda. .......................... 34 2 Pilot study ¯ndings ................................ 36 3 Illustration of the overlap of critical incident themes. .............. 64 x ACKNOWLEDGEMENTS I am extremely indebted to all of my research participants | 40 remarkable librarians| for your enthusiasm and willingness to take part on my study. Your understanding and openness in sharing your experience is deeply appreciated. Without your interest, this dissertation would not have been possible. I will forever be