How to Assess Students and Trainees in Medicine and Health Companion website Visit the companion website at: www.wiley.com/go/assesshealth On the website, you will find: • Links to useful online resources • Practical guidance • on running assessments and mark sheets • on standard setting • on running OSCEs • on simulation scenarios This title is is also available as an e-book. For more details, please see www.wiley.com/buy/9780470670897 or scan this QR code: How to Assess Students and Trainees in Medicine and Health Edited by Olwyn M. R. Westwood Professor of Medical Education Barts and the London School of Medicine and Dentistry Queen Mary, University of London Ann Griffin Clinical Senior Lecturer in Medical Education and Deputy Director UCL Medical School Frank C. Hay Emeritus Professor of Immunology St George’s University of London A John Wiley & Sons, Ltd., Publication This edition first published 2013 © 2013 by John Wiley & Sons, Ltd Wiley-Blackwell is an imprint of John Wiley & Sons, formed by the merger of Wiley’s global Scientific, Technical and Medical business with Blackwell Publishing. Registered office: John Wiley & Sons, Ltd, The Atrium, Southern Gate, Chichester, West Sussex, PO19 8SQ, UK Editorial offices: 9600 Garsington Road, Oxford, OX4 2DQ, UK The Atrium, Southern Gate, Chichester, West Sussex, PO19 8SQ, UK 111 River Street, Hoboken, NJ 07030-5774, USA For details of our global editorial offices, for customer services and for information about how to apply for permission to reuse the copyright material in this book please see our website at www.wiley.com/wiley-blackwell. The right of the author to be identified as the author of this work has been asserted in accordance with the UK Copyright, Designs and Patents Act 1988. All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopying, recording or otherwise, except as permitted by the UK Copyright, Designs and Patents Act 1988, without the prior permission of the publisher. Designations used by companies to distinguish their products are often claimed as trademarks. All brand names and product names used in this book are trade names, service marks, trademarks or registered trademarks of their respective owners. The publisher is not associated with any product or vendor mentioned in this book. This publication is designed to provide accurate and authoritative information in regard to the subject matter covered. It is sold on the understanding that the publisher is not engaged in rendering professional services. If professional advice or other expert assistance is required, the services of a competent professional should be sought. Library of Congress Cataloging-in-Publication Data is available. A catalogue record for this book is available from the British Library. Wiley also publishes its books in a variety of electronic formats. Some content that appears in print may not be available in electronic books. Cover design by Meaden Creative Set in 9.5/12 pt Minion by Toppan Best-set Premedia Limited 1 2013 Contents Contributors, vii Foreword, ix Preface, xi Chapter 1 Principles of assessment, 1 Olwyn M. R. Westwood and Ann Griffin Chapter 2 Assessment of knowledge, 16 Steve Capey Chapter 3 Development of skills competence assessment, 38 Alison Sturrock Chapter 4 Assessment of performance, 59 Gavin Johnson and Allison Wiseman Chapter 5 Assessment of professionalism, 76 Deborah Gill Chapter 6 Setting the standard in assessments, 94 Steve Capey and Frank C. Hay Chapter 7 Feedback and assessment, 114 Ann Griffin, Sophie Park and Catherine O’Keeffe Chapter 8 Psychometrics in medical education assessment, 141 Katherine Woolf Chapter 9 Examiners and examination boards, 167 Olwyn M. R. Westwood Chapter 10 Future developments in assessment and feedback, 185 Jane Dacre Index, 201 v Contributors Steve Capey Sophie Park Director of Assessment GP and Principal Teaching Fellow Swansea Medical School The Research Department of University of Swansea Primary Care and Population Health Jane Dacre University College London Professor of Medical Education Director Alison Sturrock UCL Medical School Clinical Senior Lecturer in Medical Education Deborah Gill UCL Medical School Clinical Senior Lecturer in Medical Education and Deputy Director Olwyn M. R. Westwood UCL Medical School Professor of Medical Education Barts and the London School of Ann Griffin Medicine and Dentistry Clinical Senior Lecturer in Medical Queen Mary, University of London Education and Deputy Director UCL Medical School Katherine Woolf Lecturer in Medical Education Frank C. Hay UCL Medical School Emeritus Professor of Immunology St George’s University of London Allison Wiseman Senior Tutor Gavin Johnson School of Health and Social Care Clinical Senior Lecturer in Medical University of Surrey Education UCL Medical School Catherine O’Keeffe Senior Lecturer in Clinical Education Institute of Education University of London vii Foreword A historical perspective Almost exactly 100 years ago Abraham Flexner published his famous analysis of medical education in the United States which generated a seismic change in the way doctors were trained internationally. Training in medicine has evolved from an apprenticeship model of teaching, where ‘wisdom’ was gained based on centuries of folk practice, to a regulated academic discipline, firmly founded on university-based scientific training. The twentieth century saw an unprecedented expansion in our under- standing of the basic sciences underlying medicine, and the location of medical education firmly in scholarly universities. To be sure, this may have led the ‘pendulum’ to swing too far away from the ‘art’ of medicine. However, recent developments in medical education have begun to redress that balance, such that there is now a widespread appreciation of the value of an integrated approach to care. The equilibrium between the art and science of medicine, although not yet completely established, undoubtedly offers today’s students a much better opportunity to develop into more effective and empathetic practitioners. Much of what is meant by the term ‘art’ in this context is actu- ally no less scientific than modern genomics. What is meant by this term is the understanding of the complex interactions in a human being that encompasses not just the physical and natural but also the social sciences. The watchword of many an attentive practitioner when confronted with a clinical problem is to ask for the evidence base on which this decision is to be made. While there are clearly problems with the over-zealous quest for the application of evidence, for example, in rare diseases. Nonetheless evi- dence-based medicine has undoubtedly improved clinical practice. The evidence to support our practices in medical education needs to be better. This volume brings together some of those data to summarise our current understanding of medical assessment. The ability to select and assess healthcare professionals is a complex area and we need to continue to strive for better tools for assessing individuals and their performance. This will ix x Foreword only be achieved if we pay close attention to what we already know and, rather more importantly, what we do not know, so that we can fill these gaps. The editors have brought together a significant body of talent to sum- marise the state-of-the-art methodologies in medical assessment and feed- back. If we are to see the quality of medical education improve on a platform of a sound scientific evidence base, it is important that we take heed of this set of data and look forward to another century of growth and development – just as that which was built on the work of Abraham Flexner. Professor Anthony Warrens Dean for Education Barts and The London School of Medicine and Dentistry Queen Mary, University of London Preface Medical education as world leader in assessment Medical education leads the world in its pioneering work on assessment. The international movement of medical graduates has certainly contributed to the perceived need for testing. The regulation of medical education and medical practice has likewise given rise to serious reflection on the assess- ment of progress and of attainment. Equally as significant, perhaps, is the complexity of medicine itself. The combination of knowledge, skills, problem solving and values has demanded a sophisticated approach to both curricu- lum design and assessment. Added to this educational salmagundi is the diversity of sites of learning, the extensive range of teachers and the unpredictability of the clinical context, especially at postgraduate level. Scant surprise, then, that assessment has been seen as the dependable foundation on which to allow a learner to progress, to enter the profession and, in some countries, to stay within the profession. So assessment in medicine is a serious business. Reassuringly, throughout all the consequent research and technical devel- opment work that has been undertaken, the leading assessment theoreticians and practitioners have never disregarded their responsibility to communi- cate with medical teachers and students. And perhaps, even, with the public. In more recent times, that communication has largely taken the form of technical academic papers, conference presentations and training courses. But there are few places where what is said and what is done are presented as one accessible argument. This book addresses the deficit. One of its many strengths is that it repre- sents a collaboration between two leading, and rather different, London medical schools. The practical and theoretical wisdom of the authors is apparent at every turn of the page. Not only have they presented the theoreti- cal basis, but they have also set this within the context of practice. Not only do they know the literature, but they are also able to cast a light on the xi xii Preface experience of examiners and of the candidates. Their presentation of the field is accessible precisely because of their ability to reflect on both theory and practice.