CENTRE FOR CLINICAL GOVERNANCE RESEARCH Peer review in medicine: a comprehensive review of the literature The Centre for Clinical Governance Research in Health, University of New South Wales The Centre for Clinical Governance Research in Health undertakes strategic research, evaluations and research-based projects of national and international standing with a core interest to investigate health sector issues of policy, culture, systems, governance and leadership. Peer review in medicine: a comprehensive review of the literature First produced 2009 by the Centre for Clinical Governance Research in Health, Faculty of Medicine, University of New South Wales, Sydney, NSW 2052. © Travaglia J, Debono D. 2009 This report is copyright. Apart from fair dealing for the purpose of private study, research, criticism or review, as permitted under the Copyright Act, 1968, no part of this publication may be reproduced by any process without the written permission of the copyright owners and the publisher. National Library of Australia Cataloguing-in-Publication data: Title: Peer review in medicine: a comprehensive review of the literature 1. Literature review method. 2. I. Travaglia J, II. Debono, D. V. University of New South Wales. Centre for Clinical Governance Research in Health. Centre for Clinical Governance Research, University of New South Wales, Sydney Australia http://clingov.med.unsw.edu.au Centre for Clinical Governance Research, University of New South Wales 2009 ii Peer review in medicine: a comprehensive review of the literature Peer review in medicine: a comprehensive review of the literature Duration of project February to April 2009 Search period 1950 to April 2009 Key words searched Peer Review Appraisal Assessment Evaluation Multi-source 360 degree Doctor Physician (various disciplines identified) Databases searched Medline from 1950 Embase from 1980 CINAHL from 1982 Criteria applied Peer review Peer appraisal Peer assessment Peer evaluation Doctor or physician (various disciplines identified) Contact details Contact: Jo Travaglia Email: [email protected] Telephone: 02 9385 2594 Centre for Clinical Governance Research in Health Faculty of Medicine University of New South Wales Sydney NSW 2052 Australia Centre for Clinical Governance Research, University of New South Wales 2009 iii Peer review in medicine: a comprehensive review of the literature CONTENTS 1. Introduction....................................................................................................4 2. Background....................................................................................................4 2.1 Definition of peer review ............................................................................................. 5 3. Method ..........................................................................................................8 3.1 Overview of method and research question ............................................................... 8 3.2 Review process........................................................................................................... 8 3.3 Search strategies ...................................................................................................... 10 3.3.1 Search of databases ...................................................................................... 10 3.3.2 Hand search of journals ................................................................................. 10 3.3.3 Search of grey literature................................................................................. 10 3.3.4 Snowball and citation tracking ....................................................................... 10 3.4 Search findings ......................................................................................................... 11 3.5 Analysis..................................................................................................................... 13 3.5.1 Triangulated reviewer analysis ...................................................................... 13 3.5.2 Concept analysis............................................................................................ 13 4. Findings and discussion.............................................................................14 4.1 Overview of concepts emerging from the peer review literature............................... 14 4.2 Thematic analysis of peer review literature............................................................... 19 4.2.1 The purpose of peer review ........................................................................... 19 4.2.2 The peer review method ................................................................................ 21 4.2.3 The role of reviewers ..................................................................................... 22 4.2.4 Peer review instruments and tools................................................................. 23 4.2.5 The feedback process.................................................................................... 26 4.2.6 Barriers to peer review and strategies to increase participation .................... 28 4.3 The evidence base for peer review........................................................................... 29 4.4 Limitations of peer review ......................................................................................... 31 5. Conclusion ...................................................................................................34 6. References ...................................................................................................35 7. Appendices ..................................................................................................50 Appendix A: Evidence sheet ............................................................................................ 51 Appendix B: Examples of peer review instruments.......................................................... 52 Appendix C: Advice given by HcAT to assessors providing feedback............................. 57 Appendix D: Evidence base for selected validated peer review instruments................... 60 Appendix E: Bibliography and abstracts .......................................................................... 87 Centre for Clinical Governance Research, University of New South Wales 2009 4 Peer review in medicine: a comprehensive review of the literature 1. INTRODUCTION The Centre for Clinical Governance Research (CCGR) was asked by the Statewide Quality Branch in March 2009 to identify, review and synthesise evidence on a range of topics intended to support the Understanding clinical practice toolkit. This review is a comprehensive analysis of the literature on the use of peer review with and by doctors. Following reviews address issues of: clinical audit; morbidity and mortality reviews; case review; limited adverse occurrence screening; clinical indicators; and complaints and patient satisfaction. The review uses the protocol for the rapid assessment, conceptualisation, and timely concise analysis of the literature [PRACTICAL],a developed by the CCGR. PRACTICAL emerged from CCGR’s research in the fields of clinical governance, patient safety, interprofessionalism and accreditation amongst other areas. In this review we present the results of a comprehensive review of the literature on peer review in medicine. The literature was identified using a combination of data searching, hand searching of journals and snowball technique. At the end of the review we provide abstracts and citations, arranged alphabetically by author, for the articles identified using the outlined search strategy. 2. BACKGROUND Peer review is one of numerous performance measures and quality and safety strategies which can be undertaken alone, or as part of a wider framework, such as performance appraisal,1 certification, validation, recertification or revalidation,1 continuous professional development,2 3 quality improvement and or clinical governance.4 5 A relatively new phenomenon in healthcare,3 6 interest in peer review as a way of evaluating doctors’ performance has grown in recent years. This has been partly in response to high profile patient safety inquiries,7 8 but also as a result of the growing recognition of the changing role of, and demands on, doctors.9 10 Peer review has also come to be seen as a useful tool in the formalisation and systematisation of requirements for annual assessments of doctors, which in the past had been undertaken in “... an ad hoc basis with wide variation in practice both between and within categories.”1: 1089 In practice, certain types and elements of peer review, notability morbidity and mortality reviews (considered in a subsequent monograph in this series),11 and clinical supervision,12 have traditionally formed part of a medical practitioner’s development program. Work on peer review measures at least in the undergraduate classroom has been undertaken in the US since the mid 1970s.13 14 The earliest reference to peer review identified through this literature review was a 1978 paper in the Australian and New Zealand Journal of Medicine reporting on a study in Royal Adelaide Hospital which had used peer review to a Jeffrey Braithwaite had the idea of labelling the Centre’s mode for reviewing literature ‘PRACTICAL’. This monograph was written by Joanne Travaglia, Jeffrey Braithwaite and Deborah Debono Centre for Clinical Governance Research, University of New South Wales 2009 5 Peer review in medicine: a comprehensive