DOCSLIB.ORG

Identifying Approaches for Assessing Methodological and Reporting

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Identifying Approaches for Assessing Methodological and Reporting Pussegoda et al. Systematic Reviews (2017) 6:117 DOI 10.1186/s13643-017-0507-6 RESEARCH Open Access Identifying approaches for assessing methodological and reporting quality of systematic reviews: a descriptive study Kusala Pussegoda1, Lucy Turner1, Chantelle Garritty1,2, Alain Mayhew1, Becky Skidmore1, Adrienne Stevens1,2, Isabelle Boutron3, Rafael Sarkis-Onofre4, Lise M. Bjerre5,6,7, Asbjørn Hróbjartsson8, Douglas G. Altman9 and David Moher10* Abstract Background: The methodological quality and completeness of reporting of the systematic reviews (SRs) is fundamental to optimal implementation of evidence-based health care and the reduction of research waste. Methods exist to appraise SRs yet little is known about how they are used in SRs or where there are potential gaps in research best-practice guidance materials. The aims of this study are to identify reports assessing the methodological quality (MQ) and/or reporting quality (RQ) of a cohort of SRs and to assess their number, general characteristics, and approaches to ‘quality’ assessment over time. Methods: The Cochrane Library, MEDLINE®, and EMBASE® were searched from January 1990 to October 16, 2014, for reports assessing MQ and/or RQ of SRs. Title, abstract, and full-text screening of all reports were conducted independently by two reviewers. Reports assessing the MQ and/or RQ of a cohort of ten or more SRs of interventions were included. All results are reported as frequencies and percentages of reports. Results: Of 20,765 unique records retrieved, 1189 of them were reviewed for full-text review, of which 76 reports were included. Eight previously published approaches to assessing MQ or reporting guidelines used as proxy to assess RQ were used in 80% (61/76) of identified reports. These included two reporting guidelines (PRISMA and QUOROM) and five quality assessment tools (AMSTAR, R-AMSTAR, OQAQ, Mulrow, Sacks) and GRADE criteria. The remaining 24% (18/76) of reports developed their own criteria. PRISMA, OQAQ, and AMSTAR were the most commonly used published tools to assess MQ or RQ. In conjunction with other approaches, published tools were used in 29% (22/76) of reports, with 36% (8/22) assessing adherence to both PRISMA and AMSTAR criteria and 26% (6/22) using QUOROM and OQAQ. Conclusions: The methods used to assess quality of SRs are diverse, and none has become universally accepted. The most commonly used quality assessment tools are AMSTAR, OQAQ, and PRISMA. As new tools and guidelines are developed to improve both the MQ and RQ of SRs, authors of methodological studies are encouraged to put thoughtful consideration into the use of appropriate tools to assess quality and reporting. Keywords: Reporting quality, Methodological quality, Systematic reviews, Guideline adherence * Correspondence: [email protected] 10Centre for Journalology, Canadian EQUATOR Centre, Clinical Epidemiology Program, Centre for Practice-Changing Research, Ottawa Hospital Research Institute, Ottawa, Ontario, Canada Full list of author information is available at the end of the article © The Author(s). 2017 Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated. Pussegoda et al. Systematic Reviews (2017) 6:117 Page 2 of 12 Background address several conceptual and methodological advances With the global annual expenditure of biomedical re- in the conduct and reporting of SRs. The development search estimated to be in excess of 100 billion USD [1], and adoption of SR MQ and RQ tools aim to assess, and it is no surprise that the extent of published literature is hopefully improve, the design, conduct, and reporting of growing each year, with PubMed® housing over 24 mil- SRs. Which tools are accepted and used by SR authors to lion citations, for example [2]. Researchers and decision assess MQ and completeness of reporting was unclear. makers have recognized that although there are hun- We set out to identify methodological evaluations asses- dreds of thousands of studies of healthcare interven- sing the MQ and/or RQ of SRs published from 1990 to tions, the quality of research and reporting is variable. 2014 in order to determine the approaches that were used. Evidence indicates that unless research is adequately de- signed and reported, the resources invested in research are not used effectively [1]. One estimate suggests that Methods at least 50% of published research studies were poorly Definitions and important concepts conducted making them difficult to interpret and use to We defined SRs and meta-analyses in line with that pro- inform best practice [1]. vided by the Cochrane Collaboration and the PRISMA ‘ ’ Systematic reviews (SRs) are considered the gold statement [12, 13]. We adopted the term overview,to standard for healthcare decision-making as they evaluate mean a summary of evidence from more than one SR, in- the quality and confidence of all of the available evidence cluding the combination of different populations, different addressing specific questions, such as the benefits and interventions, different outcomes (both favourable ones harms of specific health care interventions. When SR and adverse events), or different conditions [14, 15]. It is ‘ ’ conduct is optimal, that is, when best practices are synonymous with systematic review of systematic reviews, ‘ ’ ‘ ’ employed to minimize biases in the process of collecting, reviews of reviews,oran umbrella review .Wehavein- ‘ ’ appraising, and synthesizing the evidence, researchers cluded publications of methodological overviews, meaning can best understand whether or not they can be research that has assessed the MQ and/or RQ of a cohort ‘ ’ confident in the findings [3, 4]. Further, when SR report- of SRs and refer to these publications simply as reports . ing is optimal, the essential information is presented for practice guideline developers and other stakeholders, Methodological quality and completeness of reporting such as policy makers and clinicians to facilitate transla- It is necessary to make clear the distinction between MQ tion into guidance and improved patient care. and RQ. MQ addresses how well a SR was designed and Criteria for assessing the quality of primary research conducted (e.g. literature search, selection criteria, pooling emerged in the late 1980s with the rise of evidence- of data) [8]. RQ evaluates the description of the method- based medicine. This set the stage for guidelines asses- ology and findings [11]. Moreover, to distinguish from MQ, sing quality of SR conduct to be developed. Several sets the concept of risk of bias to assess primary studies is used of criteria had been developed early on including to refer to systematic flaws or limitations in the design, con- Mulrow [5] and Sacks criteria [6]. It was not until duct, or analysis of research that distort the findings [16]. Oxman and Guyatt developed the Overview Quality As- Examples are the Cochrane Risk of Bias tool for random- sessment Questionnaire (OQAQ) [7] in 1991, that a vali- ized controlled trials [17], ROBINS-I for non-randomized dated tool for assessing methodological quality (MQ) studies [18], QUADAS-2 [19] for diagnostic studies, and existed for SRs of intervention studies. More than a dec- ROBIS for SRs [16]). ade after OQAQ, A Measurement Tool to Assess Sys- tematic Reviews (AMSTAR) [8] was developed and validated in 2007 to address additional SR quality criteria Objectives including potential sources of bias that were not in- The objectives of this study are to identify reports asses- cluded in the OQAQ tool. In 2010, AMSTAR was re- sing the MQ and/or RQ of SRs and to assess their general vised (R-AMSTAR) to provide a quantitative scoring characteristics and approaches used. method to assess quality [9]. With criteria available for assessing SR conduct, it was apparent that SR authors address the standards for improving reporting quality Eligibility criteria (RQ) as well. In 1999, the Quality of Reporting of Meta- Inclusion criteria analyses (QUOROM) statement [10] was created to evalu- We included any methodological report published be- ate the completeness of reporting of meta-analysis of tween January 1990 and October 2014 whose stated pri- randomized trials. Subsequently, in 2009, QUOROM was mary intent was to assess the quality of methodology, updated as the Preferred Reporting Items of Systematic reporting, or other self-identified quality indicator(s) of a reviews and Meta-Analyses (PRISMA) statement [11] to cohort of SRs of interventions. Pussegoda et al. Systematic Reviews (2017) 6:117 Page 3 of 12 Exclusion criteria two reviewers in duplicate; a 10% random sample of re- We excluded reports of clinical interventions, whose pri- ports was assessed for accuracy. A pre-extraction meet- mary intent was not to look at methodological quality or ing was held for each extraction stage along with pilot reporting and rather to summarise SR evidence for use testing to ensure consistency across reviewers. The fol- in healthcare decision-making; reports assessing the lowing general characteristics were extracted: year of quality of SRs of diagnostic, screening, etiological, or publication; number of included SRs; specified medical prognostic studies only; and evaluations of SRs that in- area; databases searched; language restrictions; SR defin- clude study designs other than randomized controlled ition; reporting of availability of study protocol; and trials such as, narrative reviews, rapid reviews, network source of funding. The method of assessing MQ or RQ meta-analyses, and editorials. Reports in languages other of SRs was extracted. Additional items pertaining to the than English were excluded due to budget constraints evaluated reviews were extracted including the following: (Additional file 1) [20–31].
Load more

Recommended publications
  • Health Economics & Decision Science
    School of Health And Related Research Health Economics & Decision Science (HEDS) Discussion Paper Series Factors influencing the use of research for policy-making: an umbrella review protocol Authors: Erika E. Atienzo, Eva Kaltenthaler, Susan K. Baxter Corresponding author: Erika E. Atienzo ScHARR, University of Sheffield, Regent Court, 30 Regent Street, Sheffield, S1 4DA Fax: (+44) (0)114 222 0749 Email: [email protected] No. 16.07 Disclaimer: This series is intended to promote discussion and to provide information about work in progress. The views expressed in this series are those of the authors. Comments are welcome, and should be sent to the corresponding author. This paper is also hosted on the White Rose Repository: http://eprints.whiterose.ac.uk/ FACTORS INFLUENCING THE USE OF RESEARCH FOR POLICY-MAKING: AN UMBRELLA REVIEW PROTOCOL Erika E. Atienzoa*, Eva Kaltenthalera, Susan K. Baxterb a Health Economics and Decision Science, School of Health and Related Research, University of Sheffield b Public Health, School of Health and Related Research, University of Sheffield *Corresponding author: Erika E. Atienzo ScHARR, University of Sheffield Regent Court, 30 Regent Street Sheffield, S1 4DA (UK) [email protected] March 2016 ABSTRACT Over the last two decades, there has been an emphasis on the concept of evidence-based policy. However, evidence-based policy remains a major challenge and a gap exists in the systematic translation of scientific knowledge into policies. The awareness of this evidence- policy gap has led to a proliferation of research. As the demand for evidence-informed policy-making escalates, so does the need to unveil the mechanisms by which we can influence the process of research uptake.
    [Show full text]
  • A Meta-Review of Transparency and Reproducibility-Related Reporting Practices in Published Meta- Analyses on Clinical Psychological Interventions (2000-2020)
    A meta-review of transparency and reproducibility-related reporting practices in published meta- analyses on clinical psychological interventions (2000-2020). Rubén López-Nicolás1, José Antonio López-López1, María Rubio-Aparicio2 & Julio Sánchez- Meca1 1 Universidad de Murcia (Spain) 2 Universidad de Alicante (Spain) Author note: This paper has been published at Behavior Research Methods. The published version is available at: https://doi.org/10.3758/s13428-021-01644-z. All materials, data, and analysis script coded have been made publicly available on the Open Science Framework: https://osf.io/xg97b/. Correspondence should be addressed to Rubén López-Nicolás, Facultad de Psicología, Campus de Espinardo, Universidad de Murcia, edificio nº 31, 30100 Murcia, España. E-mail: [email protected] 1 Abstract Meta-analysis is a powerful and important tool to synthesize the literature about a research topic. Like other kinds of research, meta-analyses must be reproducible to be compliant with the principles of the scientific method. Furthermore, reproducible meta-analyses can be easily updated with new data and reanalysed applying new and more refined analysis techniques. We attempted to empirically assess the prevalence of transparency and reproducibility-related reporting practices in published meta-analyses from clinical psychology by examining a random sample of 100 meta-analyses. Our purpose was to identify the key points that could be improved with the aim to provide some recommendations to carry out reproducible meta-analyses. We conducted a meta-review of meta-analyses of psychological interventions published between 2000 and 2020. We searched PubMed, PsycInfo and Web of Science databases. A structured coding form to assess transparency indicators was created based on previous studies and existing meta-analysis guidelines.
    [Show full text]
  • Therapeutic Benefits of Glutamine: an Umbrella Review of Meta-Analyses
    576 BIOMEDICAL REPORTS 6: 576-584, 2017 Therapeutic benefits of glutamine: An umbrella review of meta-analyses MARC P. McRAE Department of Basic Science, National University of Health Sciences, Lombard, IL 60148, USA Received February 8, 2017; Accepted March 9, 2017 DOI: 10.3892/br.2017.885 Abstract. Glutamine may be an essential amino acid in and shortening of the length of stay in hospital. Furthermore, patients with catabolic disease, as it has been demonstrated glutamine supplementation appeared to reduce the rate of that circulating glutamine levels drop during critical illness in-patient mortality, but the majority of meta-analyses did and following major surgery; this may result in an increase in not reach statistical significance. However, researchers must secondary infection risk, recovery time and mortality rates. appreciate the positive results with caution in light of the However, there is much discrepancy in the literature with fact that there exists statistically significant heterogeneity for regards to randomized controlled studies, and therefore, the the majority of meta-analyses, and statistically significant present study is an umbrella review of published meta-analyses, publication bias in almost half. conducted to examine the effectiveness of glutamine's role as a therapeutic agent. A search using PubMed, Cochrane Library Introduction and CINAHL from January 1st, 1980 to December 31st, 2016 was conducted using the following strategy: ‘Glutamine AND Glutamine may be a conditionally essential amino acid in (meta-analysis OR systematic review)’ and publications were patients with catabolic disease as it has been demonstrated retrieved, which provided quantitative statistical analysis of that circulating plasma glutamine concentrations drop during pooled treatment effects on the relative risks of infectious critical illness and following major surgery (1).
    [Show full text]
  • Ozone Exposure and Health Effects: a Protocol for an Umbrella Review and Effect-­Specific Systematic Maps
    Open access Protocol BMJ Open: first published as 10.1136/bmjopen-2019-034854 on 13 August 2020. Downloaded from Ozone exposure and health effects: a protocol for an umbrella review and effect-­specific­systematic­maps Tianyu Zhao ,1,2,3,4 Iana Markevych,1,2,3,5 Christian Janßen,4 Dennis Nowak,1,2 Nadine Steckling- Muschack,1,6 Joachim Heinrich1,2,3,7 To cite: Zhao T, Markevych I, ABSTRACT Strengths and limitations of this study Janßen C, et al. Ozone Introduction Ambient ozone exposure may be adverse exposure and health effects: to health. Since the reported associations between ozone ► To the best of our knowledge, this protocol is the a protocol for an umbrella and health effects are heterogeneous and the underlying review and effect- specific first one for an umbrella review or a systematic map pathways are indistinct, the overall relationship remains systematic maps. BMJ Open focusing on ozone exposure and health outcomes. unclear. Only a few overall syntheses of the evidence 2020;10:e034854. doi:10.1136/ ► The umbrella review will provide a comprehensive regarding ozone and health effects are available to date. bmjopen-2019-034854 outline of the currently available systematic reviews Methods and analysis We plan to summarise the current on ambient ozone exposure and health outcomes, ► Prepublication history and evidence on ozone- related health effects systematically. and an epidemiological evidence reservoir accessi- additional material for this First, to identify the possible associations between paper are available online. To ble to the public. ambient ozone exposure and health outcomes, we will view these files, please visit ► The systematic map of a specific ozone-rela ted conduct an umbrella review.
    [Show full text]
  • Summarizing Systematic Reviews: Methodological Development
    METHODOLOGY PAPER Summarizing systematic reviews: methodological development, conduct and reporting of an umbrella review approach Edoardo Aromataris,1 Ritin Fernandez,2 Christina M. Godfrey,3 Cheryl Holly,4 Hanan Khalil5 and Patraporn Tungpunkom6 1The Joanna Briggs Institute, University of Adelaide, Adelaide, South Australia, 2School of Nursing, University of Wollongong; Centre for Research in 05/21/2018 on BhDMf5ePHKav1zEoum1tQfN4a+kJLhEZgbsIHo4XMi0hCywCX1AWnYQp/IlQrHD3gy8G8VagVy1noQVV3wH2du7UHDCprP5m8mBdFQ69S0Y= by https://journals.lww.com/ijebh from Downloaded 3 Downloaded Nursing and Health, St George Hospital, Sydney, Australia, Queen’s Joanna Briggs Collaboration, Queen’s University School of Nursing, Kingston, Ontario, Canada, 4Northeast Institute for Evidence Synthesis and Translation Rutgers, University School of Nursing Newark, New Jersey, USA, 5 6 from Monash University, School of Rural Health, Victoria, Australia, and Thailand Centre for Evidence Based Nursing, Midwifery, and Health Science: https://journals.lww.com/ijebh a Collaborating Centre of JBI, Faculty of Nursing, Chiang Mai University, Chiang Mai, Thailand ABSTRACT by BhDMf5ePHKav1zEoum1tQfN4a+kJLhEZgbsIHo4XMi0hCywCX1AWnYQp/IlQrHD3gy8G8VagVy1noQVV3wH2du7UHDCprP5m8mBdFQ69S0Y= Aims: With the increase in the number of systematic reviews available, a logical next step to provide decision makers in healthcare with the evidence they require has been the conduct of reviews of existing systematic reviews. Syntheses of existing systematic reviews are referred to by many different
    [Show full text]
  • An Umbrella Review and Narrative Synthesis of the Effectiveness Of
    International Journal of Environmental Research and Public Health Review An Umbrella Review and Narrative Synthesis of the Effectiveness of Interventions Aimed at Decreasing Food Prices to Increase Food Quality Chiara Milani 1, Chiara Lorini 2,* , Alberto Baldasseroni 3, Claudia Dellisanti 3 and Guglielmo Bonaccorsi 2 1 School of Specialization in Hygiene and Preventive Medicine, University of Florence, 50134 Florence, Italy 2 Department of Health Sciences, University of Florence, 50134 Florence, Italy 3 Department of Prevention, Local Health Unit Toscana Centro, 50135 Florence, Italy * Correspondence: chiara.lorini@unifi.it; Tel.: +39-0552751065 Received: 12 June 2019; Accepted: 1 July 2019; Published: 2 July 2019 Abstract: Background: sustainability of population diet is a public health concern: the high price of healthy food is one of the main causes of diet-related health problems. The aim of this study is to synthesize the evidence produced by systematic reviews that evaluated the effectiveness of decreasing healthy food prices to improve accessibility in order to positively modify the dietary pattern. Methods: We carried out a review of systematic reviews that examined the effects of the interventions, by exploring the online databases PubMed, Embase, Web of Science, Cochrane Library and hand-searching the reference lists. Results: after screening by titles and abstracts, we selected 11 systematic reviews that met the inclusion criteria, plus one that was hand-searched. The review generally presented a good quality. Studies concluded that measures aimed at modifying the prices of targeted healthy food were effective in improving population diet by modifying what people buy. Conclusions: the complexity of the outcome—population diet—as well as the poor transferability of data across populations and geographical areas makes it obligatory to provide clear and universal conclusions.
    [Show full text]
  • Analysis CMAJ
    Analysis CMAJ Integration of evidence from multiple meta-analyses: a primer on umbrella reviews, treatment networks and multiple treatments meta-analyses John P.A. Ioannidis MD Previously published at www.cmaj.ca eta-analysis is an important research design for appraising evidence and guiding medical practice Key points and health policy.1 Meta-analyses draw strength from • A single meta-analysis of a treatment comparison for a M single outcome offers a limited view if there are many combining data from many studies. However, even if perfectly treatments or many important outcomes to consider. done with perfect data, a single meta-analysis that addresses • Umbrella reviews assemble together several systematic 1 treatment comparison for 1 outcome may offer a short- reviews on the same condition. sighted view of the evidence. This may suffice for decision- • Treatment networks quantitatively analyze data for all making if there is only 1 treatment choice for this condition, treatment comparisons on the same disease. only 1 outcome of interest and research results are perfect. • Multiple treatments meta-analysis can rank the However, usually there are many treatments to choose from, effectiveness of many treatments in a network. many outcomes to consider and research is imperfect. For • Integration of evidence from multiple meta-analyses may example, there are 68 antidepressant drugs to choose from,2 be extended across many diseases. dozens of scales to measure depression outcomes, and biases abound in research about antidepressants.3,4 Given this complexity, one has to consider what alternative ples behind these types of analysis. I also briefly discuss treatments are available and what their effects are on various methods that analyze together data about several diseases and beneficial and harmful outcomes.
    [Show full text]
  • Improving Peer Review of Systematic Reviews by Involving Librarians and Information 2 Specialists: Protocol for a Randomized Controlled Trial
    1 Improving peer review of systematic reviews by involving librarians and information 2 specialists: protocol for a randomized controlled trial 3 Melissa L. Rethlefsen1, Sara Schroter2, Lex M. Bouter3, David Moher4, Ana Patricia Ayala5, 4 Jamie J. Kirkham6, Maurice P. Zeegers7 5 1. Health Sciences Library & Informatics Center, University of New Mexico, MSC 09 5100, 1 University of New Mexico, Albuquerque, 6 NM 87131-0001, [email protected], https://orcid.org/0000-0001-5322-9368 7 2. BMJ Publishing Group, London, United Kingdom, https://orcid.org/0000-0002-8791-8564 8 3. Department of Philosophy, Faculty of Humanities, Vrije Universiteit Amsterdam, De Boelelaan 1105, 1081 HV, Amsterdam, The 9 Netherlands; Department of Epidemiology and Data Science, Amsterdam UMC, Vrije Universiteit Amsterdam, De Boelelaan 1089a, 10 1081 HV, Amsterdam, The Netherlands, https://orcid.org/0000-0002-2659-5482 11 4. Centre for Journalology, Clinical Epidemiology Program, Ottawa Hospital Research Institute, The Ottawa Hospital, General Campus, 12 Centre for Practice Changing Research Building, 501 Smyth Road, PO BOX 201B, Ottawa, Ontario, K1H 8L6, Canada, https://orcid.org/ 13 0000-0003-2434-4206 14 5. Gerstein Science Information Centre, University of Toronto, Toronto, Canada, https://orcid.org/0000-0002-3613-2270 15 6. Centre for Biostatistics, University of Manchester, Manchester, UK, https://orcid.org/0000-0003-2579-9325 16 7. Care and Public Health Research Institute (CAPHRI), Maastricht University Medical Center+, Universiteitssingel 40, 6229, ER, 17 Maastricht, the Netherlands, https://orcid.org/0000-0002-2387-083X 18Abstract 19Background: Problems continue to exist with the reporting of and risk of bias in search methods and 20strategies in systematic reviews and related review types.
    [Show full text]
  • Ambient Air Pollution, Extreme Temperatures and Birth Outcomes: a Protocol for an Umbrella Review, Systematic Review and Meta-Analysis
    International Journal of Environmental Research and Public Health Study Protocol Ambient Air Pollution, Extreme Temperatures and Birth Outcomes: A Protocol for an Umbrella Review, Systematic Review and Meta-Analysis Sylvester Dodzi Nyadanu 1,2,*, Gizachew Assefa Tessema 2,3 , Ben Mullins 2, Bernard Kumi-Boateng 4 , Michelle Lee Bell 5 and Gavin Pereira 2,6,7 1 Education, Culture and Health Opportunities (ECHO) Ghana, ECHO Research Group International, P. O. Box 424, Aflao, Ghana 2 School of Public Health, Curtin University, Perth, Kent Street, Bentley, Western Australia 6102, Australia; [email protected] (G.A.T.); [email protected] (B.M.); [email protected] (G.P.) 3 School of Public Health, University of Adelaide, Adelaide, South Australia 5000, Australia 4 Department of Geomatic Engineering, University of Mines and Technology, P. O. Box 237, Tarkwa, Ghana; [email protected] 5 School of the Environment, Yale University, New Haven, CT 06511, USA; [email protected] 6 Telethon Kids Institute, Northern Entrance, Perth Children’s Hospital, Nedlands, Western Australia 6009, Australia 7 Centre for Fertility and Health (CeFH), Norwegian Institute of Public Health, 0473 Oslo, Norway * Correspondence: [email protected] Received: 1 October 2020; Accepted: 18 November 2020; Published: 21 November 2020 Abstract: Prenatal exposure to ambient air pollution and extreme temperatures are among the major risk factors of adverse birth outcomes and with potential long-term effects during the life course. Although low- and middle-income countries (LMICs) are most vulnerable, there is limited synthesis of evidence in such settings. This document describes a protocol for both an umbrella review (Systematic Review 1) and a focused systematic review and meta-analysis of studies from LMICs (Systematic Review 2).
    [Show full text]
  • Violence Prevention Interventions: an Umbrella Review Protocol
    Violence prevention interventions: An Umbrella Review protocol Achim Wolf, John P. A. Ioannidis, Seena Fazel Protocol v 1.0 – June 29, 2015 University of Oxford, Department of Psychiatry, Warneford Hospital, Warneford Lane, Headington, Oxford, United Kingdom Funding: Wellcome Trust (095806) Background Violence is among the largest causes of disability-adjusted life years worldwide according to the World Health Organization.[1] In the US alone, the yearly impact of violence on health services is in excess of $5 billion.[2] A wide range of studies on violence prevention interventions have been reported, and a synthesis of this evidence would be important for public policy and planning and the development of preventive efforts.[3, 4] Therefore, we propose to conduct an umbrella review on the effectiveness of violence prevention interventions worldwide, and use this review as a basis for recommendations for future prevention programs and research. Review objectives “What are effective interventions for the prevention of violence?” Methods Inclusion criteria For inclusion, reviews have to evaluate the effectiveness of primary prevention (aimed at groups regardless of risk), secondary prevention interventions (aimed at high risk groups), or tertiary prevention (aimed at previously violent groups). Types of participants Participants include general population groups, those considered at high-risk, and previous offenders or previously violent people. Interventions All types of interventions will be included if they examine effects on violence, defined below. Outcomes The outcome to be examined in this umbrella review is violence. The definition of the World Health Organization will be used: “The intentional use of physical force or power, threatened or actual, against oneself, another person, or against a group or community, that either results in or has a high likelihood of resulting in injury, death, psychological harm, maldevelopment or deprivation.”[5] Reviews focusing on self-directed violence (i.e.
    [Show full text]
  • Validity of Observational Evidence on Putative Risk and Protective Factors
    Janiaud et al. BMC Medicine (2021) 19:157 https://doi.org/10.1186/s12916-021-02020-6 RESEARCH ARTICLE Open Access Validity of observational evidence on putative risk and protective factors: appraisal of 3744 meta-analyses on 57 topics Perrine Janiaud1,2, Arnav Agarwal3, Ioanna Tzoulaki4,5, Evropi Theodoratou6,7, Konstantinos K. Tsilidis4,5, Evangelos Evangelou4,5 and John P. A. Ioannidis1,8,9,10,11* Abstract Background: The validity of observational studies and their meta-analyses is contested. Here, we aimed to appraise thousands of meta-analyses of observational studies using a pre-specified set of quantitative criteria that assess the significance, amount, consistency, and bias of the evidence. We also aimed to compare results from meta-analyses of observational studies against meta-analyses of randomized controlled trials (RCTs) and Mendelian randomization (MR) studies. Methods: We retrieved from PubMed (last update, November 19, 2020) umbrella reviews including meta-analyses of observational studies assessing putative risk or protective factors, regardless of the nature of the exposure and health outcome. We extracted information on 7 quantitative criteria that reflect the level of statistical support, the amount of data, the consistency across different studies, and hints pointing to potential bias. These criteria were level of statistical significance (pre-categorized according to 10−6, 0.001, and 0.05 p-value thresholds), sample size, statistical significance for the largest study, 95% prediction intervals, between-study heterogeneity, and the results of tests for small study effects and for excess significance. * Correspondence: [email protected] 1Meta-Research Innovation Center at Stanford (METRICS), Stanford, CA 94305, USA 8Department of Epidemiology and Population Health, Stanford University School of Medicine, Stanford, CA 94305, USA Full list of author information is available at the end of the article © The Author(s).
    [Show full text]
  • 'Umbrella' Reviews Or 'Review of Reviews'
    Leeds Institute of Health Sciences Literature search methods for an overview of reviews (‘umbrella’ reviews or ‘review of reviews’) Judy Wright Senior Information Specialist Leeds Institute of Health Sciences, University of Leeds Dr Rebecca Walwyn Principal Statistician Leeds Institute of Clinical Trials Research, University of Leeds Judy Wright © University of Leeds 2016. This work is made available for reuse under the terms of the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International Licence A simple search? Searches for systematic reviews can be straightforward But, if reviewer also wants to ‘back track’ to get the primary studies out of reviews, it can get complicated…. Image: Complexity by Mark Skipper https://www.flickr.com/photos/bitterjug/7670055210 (CC BY 2.0) Session overview • Definitions • Attractions of overviews of systematic reviews • Search methods for finding SRs – DEMO • Overview methods unravelling • Example of CBT reviews • Search methods for tracking primary studies • Comparison of search methods for overviews of reviews Terms and definitions • Cochrane Overviews of reviews (Cochrane Overviews) are Cochrane reviews designed to compile evidence from multiple systematic reviews of interventions into one accessible and usable document.(1) • Umbrella review specifically refers to review compiling evidence from multiple reviews into one accessible and usable document. Focuses on broad condition or problem for which there are competing interventions and highlights reviews that address these interventions and their results. (2) Overview of reviews terms CBT examples • Effects of school-based interventions targeting obesity-related behaviors and body weight change: a systematic umbrella review. (Behav Med. 37(1):15-25, 2011 Jan.) • Standardised mindfulness-based interventions in healthcare: an overview of systematic reviews and meta-analyses of RCTs.
    [Show full text]