© BMJ Publishing Group 2001 Chapter 4 © Crown copyright 2000 Chapter 24 © Crown copyright 1995, 2000 Chapters 25 and 26 © The Cochrane Collaboration 2000 All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopy- ing, recording and/or otherwise, without the prior written permission of the publishers. First published in 1995 by the BMJ Publishing Group, BMA House, Tavistock Square, London WC1H 9JR www.bmjbooks.com First edition 1995 Second impression 1997 Second edition 2001 British Library Cataloguing in Publication Data A catalogue record for this book is available from the British Library ISBN 0-7279-1488–X Typeset by Phoenix Photosetting, Chatham, Kent Printed and bound by MPG Books, Bodmin, Cornwall Contents Contributors viii Foreword xiii Introduction 1 Rationale, potentials, and promise of systematic reviews 3 MATTHIAS EGGER, GEORGE DAVEY SMITH, KEITH O’ROURKE Part I: Systematic reviews of controlled trials 2 Principles of and procedures for systematic reviews 23 MATTHIAS EGGER, GEORGE DAVEY SMITH 3 Problems and limitations in conducting systematic reviews 43 MATTHIAS EGGER, KAY DICKERSIN, GEORGE DAVEY SMITH 4 Identifying randomised trials 69 CAROL LEFEBVRE, MICHAEL JCLARKE 5 Assessing the quality of randomised controlled trials 87 PETER JÜNI, DOUGLAS G ALTMAN, MATTHIAS EGGER 6 Obtaining individual patient data from randomised controlled trials 109 MICHAEL J CLARKE, LESLEY A STEWART 7 Assessing the quality of reports of systematic reviews: the QUOROM statement compared to other tools 122 BEVERLEY SHEA, CATHERINE DUBÉ, DAVID MOHER Part II: Investigating variability within and between studies 8 Going beyond the grand mean: subgroup analysis in meta-analysis of randomised trials 143 GEORGE DAVEY SMITH, MATTHIAS EGGER v CONTENTS 9 Why and how sources of heterogeneity should be investigated 157 SIMON G THOMPSON 10 Analysing the relationship between treatment benefit and underlying risk: precautions and recommendations 176 STEPHEN J SHARP 11 Investigating and dealing with publication and other biases 189 JONATHAN A C STERNE, MATTHIAS EGGER, GEORGE DAVEY SMITH Part III: Systematic reviews of observational studies 12 Systematic reviews of observational studies 211 MATTHIAS EGGER, GEORGE DAVEY SMITH, MARTIN SCHNEIDER 13 Systematic reviews of evaluations of prognostic variables 228 DOUGLAS G ALTMAN 14 Systematic reviews of evaluations of diagnostic and screening tests 248 JONATHAN JDEEKS Part IV: Statistical methods and computer software 15 Statistical methods for examining heterogeneity and combining results from several studies in meta-analysis 285 JONATHAN J DEEKS, DOUGLAS G ALTMAN, MICHAEL J BRADBURN 16 Effect measures for meta-analysis of trials with binary outcomes 313 JONATHAN J DEEKS, DOUGLAS G ALTMAN 17 Meta-analysis software 336 JONATHAN A C STERNE, MATTHIAS EGGER, ALEXANDER J SUTTON 18 Meta-analysis in StataTM 347 JONATHAN A C STERNE, MICHAEL J BRADBURN, MATTHIAS EGGER Part V: Using systematic reviews in practice 19 Applying the results of systematic reviews at the bedside 373 FINLAY A MCALISTER 20 Numbers needed to treat derived from meta-analyses: pitfalls and cautions 386 SHAH EBRAHIM vi CONTENTS 21 Using systematic reviews in clinical guideline development 400 MARTIN ECCLES, NICK FREEMANTLE, JAMES MASON 22 Using systematic reviews for evidence based policy making 410 J A MUIR GRAY 23 Using systematic reviews for economic evaluation 419 MIRANDA MUGFORD 24 Using systematic reviews and registers of ongoing trials for scientific and ethical trial design, monitoring, and reporting 429 IAIN CHALMERS Part VI: The Cochrane Collaboration 25 The Cochrane Collaboration in the 20th century 447 GERD ANTES, ANDREW D OXMAN FOR THE COCHRANE COLLABORATION 26 The Cochrane Collaboration in the 21st century: ten challenges and one reason why they must be met 459 ANDREW D OXMAN Index 475 vii Contributors Douglas G Altman Professor of Statistics in Medicine ICRF Medical Statistics Group Centre for Statistics in Medicine Institute of Health Sciences University of Oxford Oxford, UK Gerd Antes Director German Cochrane Centre Institut für Medizinische Biometrie und Medizinische Informatik University of Freiburg Freiburg i.B., Germany Michael J Bradburn Medical Statistician ICRF Medical Statistics Group Centre for Statistics in Medicine Institute of Health Sciences University of Oxford, Oxford, UK Iain Chalmers Director UK Cochrane Centre NHS Research and Development Programme Oxford, UK Michael J Clarke Associate Director (Research) UK Cochrane Centre NHS Research and Development Programme and Overviews’ Co-ordinator Clinical Trials Service Unit Oxford, UK viii CONTRIBUTORS George Davey Smith Professor of Clinical Epidemiology Division of Epidemiology and MRC Health Services Research Collaboration Department of Social Medicine University of Bristol Bristol, UK Jonathan J Deeks Senior Medical Statistician Systematic Review Development Programme Centre for Statistics in Medicine Institute of Health Sciences University of Oxford Oxford, UK Kay Dickersin Associate Professor Department of Community Health Brown University Rhode Island, USA Catherine Dubé Department of Medicine Division of Gastro-enterology University of Ottawa Ottawa, Canada Shah Ebrahim Professor in Epidemiology of Ageing Division of Epidemiology and MRC Health Services Research Collaboration Department of Social Medicine University of Bristol Bristol, UK Martin Eccles Professor of Clinical Effectiveness Centre for Health Services Research University of Newcastle upon Tyne Newcastle upon Tyne, UK ix CONTRIBUTORS Matthias Egger Senior Lecturer in Epidemiology and Public Health Medicine Division of Health Services Research and MRC Health Services Research Collaboration Department of Social Medicine University of Bristol Bristol, UK Nick Freemantle Reader in Epidemiology and Biostatistics Medicines Evaluation Group Centre for Health Economics University of York York, UK J A Muir Gray Director Institute of Health Sciences University of Oxford Oxford, UK Peter Jüni Research Fellow MRC Health Services Research Collaboration Department of Social Medicine University of Bristol Bristol, UK Carol Lefebvre Information Specialist UK Cochrane Centre NHS Research and Development Programme Oxford, UK James Mason Senior Research Fellow Medicines Evaluation Group Centre for Health Economics University of York York, UK Finlay A McAlister Assistant Professor Division of General Internal Medicine University of Alberta Hospital Edmonton, Canada x CONTRIBUTORS David Moher Director Thomas C Chalmers Centre for Systematic Reviews Children’s Hospital of Eastern Ontario Research Institute University of Ottawa Ottawa, Canada Miranda Mugford Professor of Health Economics School of Health Policy and Practice University of East Anglia Norwich, UK Keith O’Rourke Statistician Clinical Epidemiology Unit Loeb Research Institute Ottawa Hospital Ottawa, Canada Andrew D Oxman Director Health Services Research Unit National Institute of Public Health Oslo, Norway Martin Schneider Specialist Registrar in Internal Medicine Department of Medicine University Hospitals Geneva, Switzerland Stephen J Sharp Medical Statistician GlaxoWellcome Research and Development London, UK Beverley Shea Loeb Health Research Institute Clinical Epidemiology Unit Ottawa Hospital University of Ottawa Ottawa, Canada xi CONTRIBUTORS Jonathan A C Sterne Senior Lecturer in Medical Statistics Division of Epidemiology and MRC Health Services Research Collaboration Department of Social Medicine University of Bristol Bristol, UK Lesley A Stewart Head Meta-Analysis Group MRC Clinical Trials Unit London, UK Alexander J Sutton Lecturer in Medical Statistics Department of Epidemiology and Public Health University of Leicester Leicester, UK Simon G Thompson Director MRC Biostatistics Unit Institute of Public Health University of Cambridge Cambridge, UK xii Foreword “If, as is sometimes supposed, science consisted in nothing but the laborious accumulation of facts, it would soon come to a standstill, crushed, as it were, under its own weight. The suggestion of a new idea, or the detection of a law, supersedes much that has previously been a burden on the memory, and by introducing order and coherence facilitates the retention of the remainder in an available form.. Two processes are thus at work side by side, the reception of new material and the digestion and assimilation of the old; and as both are essential we may spare ourselves the discussion of their relative importance. One remark, however, should be made. The work which deserves, but I am afraid does not always receive, the most credit is that in which discovery and explanation go hand in hand, in which not only are new facts presented, but their relation to old ones is pointed out.”1 The above quotation is from the presidential address given by Lord Rayleigh, Professor of Physics at Cambridge University, at the meeting of the British Association for the Advancement of Science held in Montreal in 1884. More than a century later, research funding agencies, research ethics committees, researchers and journal editors in the field of health research have only just begun to take Lord Rayleigh’s injunction seriously. Research synthesis has a long history and has been developed in many spheres of scientific activity.2 Social scientists in the United States, in particular, have been actively discussing, developing and applying methods for this kind of research for more than quarter