Evidence to Inform the Development of QUADAS-2
Total Page:16
File Type:pdf, Size:1020Kb
Updating QUADAS: Evidence to inform the development of QUADAS-2 Penny Whiting, Anne Rutjes, Marie Westwood, Susan Mallett, Mariska Leeflang, Hans Reitsma, Jon Deeks, Jonathan Sterne, Patrick Bossuyt 1 QUADAS Steering Group members: Penny Whiting, Department of Social Medicine, University of Bristol Anne Rutjes, Departments of Social and Preventive Medicine and Rheumatology, University of Berne Jonathan Sterne, Department of Social Medicine, University of Bristol Jon Deeks, Unit of Public Health, Epidemiology & Biostatistics, University of Birmingham Mariska Leeflang, Department of Clinical Epidemiology, Biostatistics and Informatics, AMC, University of Amsterdam Patrick Bossuyt, Department of Clinical Epidemiology, Biostatistics and Informatics, AMC, University of Amsterdam Hans Reitsma, Department of Clinical Epidemiology, Biostatistics and Informatics, AMC, University of Amsterdam Marie Westwood, Kleijnen Systematic Reviews, York Susan Mallett, Centre for Statistics in Medicine, University of Oxford 2 Contents Chapter 1: Background .......................................................................................................................... 5 Items in italics are those removed from the Cochrane version of QUADAS.Chapter 2: Approach and Scope of QUADAS-2 ............................................................................................................................... 6 Chapter 2: Approach and Scope of QUADAS-2 ...................................................................................... 7 2.1 Rationale for QUADAS-2 ............................................................................................................. 9 2.2 Scope of QUADAS-2 .................................................................................................................... 9 2.2 Develop the evidence base ....................................................................................................... 12 2.3 Generate a list of items for consideration for inclusion in QUADAS-2 (Chapter 7) .................. 13 Chapter 3: Diagnostic test accuracy (DTA) reviews: Conduct and reporting of quality assessment .. 14 3.1 Objectives .................................................................................................................................. 15 3.2 Methods .................................................................................................................................... 15 3.3 Results ....................................................................................................................................... 16 3.4 Summary ................................................................................................................................... 24 Reported problems with applying QUADAS items .......................................................................... 26 3.5 Implications for QUADAS-2 ....................................................................................................... 27 Chapter 4: Feedback from Reviewers .................................................................................................. 28 4.1 Objective .................................................................................................................................... 29 4.2 Methods ..................................................................................................................................... 29 4.3 Results ........................................................................................................................................ 29 4.4 Summary ................................................................................................................................... 43 4.5 Implications for QUADAS-2 ................................................................................................... 44 Chapter 5: Sources of Variation and Bias in Studies of Diagnostic Accuracy: an updated systematic review ............................................................................................................................ 46 5.1 Objectives .................................................................................................................................. 47 5.2 Methods ..................................................................................................................................... 47 5.3 Results ........................................................................................................................................ 49 5.4 Summary of results .................................................................................................................... 70 5.5 Implications for QUADAS-2 ........................................................................................................ 70 3 Chapter 6: Review of studies that have evaluated QUADAS ............................................................... 72 6.1 Objectives .................................................................................................................................. 73 6.2 Methods .................................................................................................................................... 73 6.3 Results ....................................................................................................................................... 73 6.3 Summary ................................................................................................................................... 76 6.4 Implications for QUADAS-2 ........................................................................................................ 76 Chapter 7: Generating a list of items .................................................................................................. 78 7.1 Recommendations from the evidence base .............................................................................. 78 8. References ....................................................................................................................................... 84 Appendix 3.1: General Details of Included Reviews .......................................................................... 102 Appendix 3.2a: Details of Quality Assessment: Reviews that used QUADAS .................................... 106 Appendix 3.2b: Details of Quality Assessment: Reviews that did not use QUADAS ......................... 111 Appendix 3.3: Details on how quality assessment was incorporated into the review ..................... 116 Appendix 3.4: Data extraction form .................................................................................................. 120 Appendix 4: QUADAS questionnaire and detailed summary of responses ....................................... 122 Appendix 5.1: Search strategies ........................................................................................................ 151 Appendix 5.2: Detailed data extraction tables from original review ................................................ 154 Appendix 6: Summary of studies that have evaluated QUADAS ....................................................... 190 4 Chapter 1: Background QUADAS is a quality assessment tool for use in systematic reviews of diagnostic test accuracy (DTA) studies that we developed in 2003 (Table 1).(1)The steps we employed to develop QUADAS are outlined in Figure 1. We developed an initial list of possible items for inclusion through reviewing sources of bias and variation in DTA studies, reviewing existing quality assessment tools for DTA studies and examining how quality was incorporated into DTA reviews. We then conducted a Delphi procedure to refine this initial list of items to produce QUADAS. Members of the Delphi panel were experts in the area of diagnostic research. The process also included a preliminary evaluation of QUADAS which involved assessing inter-rater agreement in the rating of a set of 30 studies and gathering feedback from 20 reviewers who had used QUADAS in their reviews.(2) Based on this, some modifications were proposed for the scoring of two of the QUADAS items: interpretation of uninterpretable/intermediate test results and withdrawals. Since its development QUADAS has been used in a large number of systematic reviews: it has been cited over 300 times and searching the DARE database using the term “QUADAS” identified 96 reviews. A modified version of QUADAS, with items related to the quality of reporting removed, has been adopted for use by the Cochrane Collaboration and is recommended for use in all Cochrane DTA reviews.(3) QUADAS has also been recommended for use by NICE. Our own experience, anecdotal reports, and feedback via Cochrane suggest some problems with the current version of QUADAS. These include problems in scoring certain items (in particular items on spectrum, uninterpretable/intermediate test results and withdrawals), possible overlap between items (for example partial verification bias and withdrawals), and certain situations where it is difficult to use QUADAS (for example in topics in which the reference standard involves an element of follow-up). We therefore decided to revisit QUADAS with the aim of using the experience gathered through its use and new evidence regarding sources of bias and variation to update QUADAS to produce “QUADAS-2”. 5 Figure 1: Development of original QUADAS tool Review of sources Review of diagnostic Review of use of quality of bias and quality assessment assessment in diagnostics variation tools