Critical Appraisal of Medical Literature – Short Course
Total Page:16
File Type:pdf, Size:1020Kb
Critical Appraisal of Medical Literature – Short course What is Critical appraisal? Critical appraisal is the systematic evaluation of clinical research papers in order to establish: If the study addresses a clearly focused question? Are valid methods used to address this question? If the study results are important? Are these results applicable to my patient or population? Resources for critical appraisal and reporting guidelines are available in The Equator Network, JAMA, Oxford Evidence based medicine, SQUIRE and TREND. How to do Critical appraisal? The steps involved in critical appraisal are Choose a journal article relevant to your research question Read the article in detail, especially the section on methods Answer the critical appraisal questions Make a presentation on critical appraisal of the article Quick references for each study design: The critical appraisal questions for each study design have been enclosed. The questions are based on JAMA Users guide to Medical Literature. However, we strongly recommend reading the guide in detail prior to critical appraisal. DIAGNOSTIC TEST How serious is the risk of bias? Did participating patients constitute a representative sample of those presenting with a diagnostic dilemma? Did investigators compare the test to an appropriate, independent reference standard? Were those interpreting the test and reference standard blind to the other result? Did all patients receive the same reference standard irrespective of the test results? What are the results? What likelihood ratios were associated with the range of possible test results? How can i apply the results to patient care? Will the reproducibility of the test results and their interpretation be satisfactory in my clinical setting? Are the study results applicable to the patients in my practice? Will the test results change my management strategy? Will patients be better off as a result of the test? CASP Checklist: 12 questions to help you make sense of a Diagnostic Test study How to use this appraisal tool: Three broad issues need to be considered when appraising a trial: Are the results of the study valid? (Section A) What are the results? (Section B) Will the results help locally? (Section C) The 12 questions on the following pages are designed to help you think about these issues systematically. The first three questions are screening questions and can be answered quickly. If the answer to both is “yes”, it is worth proceeding with the remaining questions. There is some degree of overlap between the questions, you are asked to record a “yes”, “no” or “can’t tell” to most of the questions. A number of italicised prompts are given after each question. These are designed to remind you why the question is important. Record your reasons for your answers in the spaces provided. About: These checklists were designed to be used as educational pedagogic tools, as part of a workshop setting, therefore we do not suggest a scoring system. The core CASP checklists (randomised controlled trial & systematic review) were based on JAMA 'Users’ guides to the medical literature 1994 (adapted from Guyatt GH, Sackett DL, and Cook DJ), and piloted with health care practitioners. For each new checklist, a group of experts were assembled to develop and pilot the checklist and the workshop format with which it would be used. Over the years overall adjustments have been made to the format, but a recent survey of checklist users reiterated that the basic format continues to be useful and appropriate. Referencing: we recommend using the Harvard style citation, i.e.: Critical Appraisal Skills Programme (2018). CASP (insert name of checklist i.e. Diagnostic Test Study) Checklist. [online] Available at: URL. Accessed: Date Accessed. ©CASP this work is licensed under the Creative Commons Attribution – Non-Commercial- Share A like. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc- sa/3.0/ www.casp-uk.net Critical Appraisal Skills Programme (CASP) part of Oxford Centre for Triple Value Healthcare Ltd www.casp-uk.net Paper for appraisal and reference: Section A: Are the results of the trial valid? 1. Was there a clear question Yes HINT: A question should include for the study to address? information about Can’t Tell • the population • the test No • the setting • the outcomes Comments: 2. Was there a comparison Yes HINT: Is this reference test(s) the best with an appropriate available indicator in the circumstances reference standard? Can’t Tell No Comments: Is it worth continuing? 3. Did all patients get the Yes HINT: Consider diagnostic test and • were both received regardless of the reference standard? Can’t Tell results of the test of interest • Check the 2x2 table (verification No bias) Comments: 2 4. Could the results of the test Yes HINT: Consider have been influenced by the • was there blinding results of the reference Can’t Tell • were the tests performed standard? independently No • review bias Comments: 5. Is the disease status of the Yes HINT: Consider tested population clearly • presenting symptoms described? Can’t Tell • disease stage of severity • co-morbidity No • differential diagnoses (spectrum bias) Comments: 6. Were the methods for Yes HINT: Consider performing the test described in • was a protocol followed sufficient detail? Can’t Tell No Comments: Section B: What are the results? 3 7. What are the results? HINT: Consider • are the sensitivity and specificity and/or likelihood ratios presented • are the results presented in such a way that we can work them out Comments: 8. How sure are we about the results? HINT: Consider Consequences and cost of alternatives • could they have occurred by chance performed? • are there confidence limits • what are they Comments: Section C: Will the results help locally? Consider whether you are primarily interested in the impact on a population or individual level 9. Can the results be applied to Yes HINT: Do you think your your patients/the population patients/population are so different from of interest? Can’t Tell those in the study that the results cannot be applied, such as age, sex, ethnicity and No spectrum bias Comments: 10. Can the test be applied to Yes HINT: Consider your patient or population of • resources and opportunity costs interest? Can’t Tell • level and availability of expertise required to interpret the tests No • current practice and availability of services 4 Comments: 11. Were all outcomes Yes HINT: Consider important to the individual • will the knowledge of the test result or population considered? Can’t Tell improve patient wellbeing • will the knowledge of the test result No lead to a change in patient management Comments: 12. What would be the impact of using this test on your patients/population? Comments: 5 See discussions, stats, and author profiles for this publication at: https://www.researchgate.net/publication/292612112 Critical Appraisal of a Diagnostic Test Study Article · February 2016 CITATIONS READS 0 1,749 1 author: Leonardo Roever (BRAMETIS) Brazilian Network of Research in Meta-analysis 554 PUBLICATIONS 2,115 CITATIONS SEE PROFILE Some of the authors of this publication are also working on these related projects: Rivaroxaban or Apixaban in Mechanical Valves: RAMV Study View project IMCSC group ( Section: Diagnosis ) View project All content following this page was uploaded by Leonardo Roever on 02 February 2016. The user has requested enhancement of the downloaded file. Roever, Evidence Based Medicine and Practice 2015, 1:1 Evidence Based Medicine and Practice http://dx.doi.org/10.4172/EBMP.1000e104 Editorial Open Access Journal Critical Appraisal of a Diagnostic Test Study Leonardo Roever* Department of Clinical Research, Federal University of Uberlândia, Uberlândia, Brazil *Corresponding author: Leonardo Roever, Department of Clinical Research, Av Pará, 1720 - Bairro Umuarama, Uberlândia-MG-CEP 38400-902, Brazil, Tel: +553488039878; E-mail: [email protected] Rec Date: 30 November, 2015; Acc Date: 07 December, 2015; Pub Date: 14 December, 2015 Copyright: © 2015 Roever L. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. Introduction Diagnostic Odds Ratio: LR+/LR– = (TP/FN) / (FP/TN) Be able to evaluate a diagnostic test is not an easy task. Diagnostic Did all patients get the diagnostic test and reference standard? tests are invaluable tools used to distinguish between patients having a disease and those who have not. It is essential to be able to critically Were both received regardless of the results of the test of interest? Check the 2 × 2 table (verification bias). appraise published articles on a diagnostic test. The list of questions below can help you better appreciate and understand the diagnostic Was the reference standard applied regardless of the index test result? Is the studies better. The Table 1 shows the checklists needed to make a diagnostic test available, affordable, accurate, and precise in your setting? critical analysis of a diagnostic test study [1-12]. The index test results interpreted without knowledge of the results of the reference standard. If a threshold is used, it is pre-specified. The index test, its Appraisal questions conduct, and its interpretation are similar to that used in practice with the target population of the guideline. Was there a clear question for the study to address? A question