DOCSLIB.ORG

Is It Worthwhile Including Observational Studies in Systematic Reviews of Effectiveness?

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Is It Worthwhile Including Observational Studies in Systematic Reviews of Effectiveness? CRD_mcdaid05_Poster.qxd 13/6/05 5:12 pm Page 1 Is it worthwhile including observational studies in systematic reviews of effectiveness? The experience from a review of treatments for childhood retinoblastoma Catriona Mc Daid, Suzanne Hartley, Anne-Marie Bagnall, Gill Ritchie, Kate Light, Rob Riemsma Centre for Reviews and Dissemination, University of York, UK Background • Overall there were considerable problems with quality Figure 1: Mapping of included studies (see Table). Without randomised allocation there was a Retinoblastoma is a rare malignant tumour of the retina and high risk of selection bias in all studies. Studies were also usually occurs in children under two years old. It is an susceptible to detection and performance bias, with the aggressive tumour that can lead to loss of vision and, in retrospective studies particularly susceptible as they were extreme cases, death. The prognoses for vision and survival less likely to have a study protocol specifying the have significantly improved with the development of more intervention and outcome assessments. timely diagnosis and improved treatment methods. Important • Due to the considerable limitations of the evidence clinical factors associated with prognosis are age and stage of identified, it was not possible to make meaningful and disease at diagnosis. Patients with the hereditary form of robust conclusions about the relative effectiveness of retinoblastoma may be predisposed to significant long-term different treatment approaches for childhood complications. retinoblastoma. Historically, enucleation was the standard treatment for unilateral retinoblastoma. In bilateral retinoblastoma, the eye ■ ■ ■ with the most advanced tumour was commonly removed and EBRT Chemotherapy EBRT with Chemotherapy the contralateral eye treated with external beam radiotherapy ■ Enucleation ■ Local Treatments (EBRT). However, EBRT may be associated with the risk of Table 1: Summary of study quality serious long-term complications. There has been an increasing trend in the use of chemotherapy and local Quality criteria Studies meeting Comments conservative treatments such as cryotherapy, criteria photocoagulation and thermotherapy and a decrease in the Description of assignment to n=14 Allocation was on the basis of disease severity, time period in which use of radiotherapy and enucleation, where possible. intervention treatment was given, the clinic which provided the treatment or generally the treatment protocol in use. Objective Groups balanced by design n=2 Identification of relevant prognostic n=21 Generally the only variable reported was Reese-Ellsworth •To conduct a systematic review of the evidence-base for variables classification or type of retinoblastoma. treatments for retinoblastoma. Matched for relevant prognostic n=4 Some studies also reported outcomes by Reese-Ellsworth •To consider the problems and benefits of including variables or effect of any group classification difference evaluated in a valid observational studies in a systematic review of the analysis clinical effectiveness of treatments for retinoblastoma. Intervention groups comparable at n=4 In five studies there was evidence that groups were not comparable baseline and in the remaining studies it was unclear or the relevant Methods information was not reported • Seventeen electronic databases were searched from Number of patients lost to follow-up n=2 Twenty-four studies included only patients on whom follow-up reported and rates similar across information was available. In four studies loss to follow-up was inception to April 2004. groups reported only for both treatment groups combined. In one study • Studies of participants diagnosed with childhood rates were reported for both groups but loss to follow-up was not similar across groups. retinoblastoma were eligible for inclusion. Any Follow-up period reported for both n=13 Mainly through use of Kaplan-Meier analysis intervention, or combination of interventions, and all treatment groups clinical outcomes were eligible. Where controlled trials Analyses adjusted for different n=10 were not available, prospective and retrospective cohort lengths of follow-up studies with clear comparisons between treatment Follow-up long enough for the n=20 groups were eligible. outcomes to occur Treatment clearly specified n=12 •Two reviewers independently assessed titles and Clearly defined criteria for measuring n=15 abstracts and full papers. outcomes • One reviewer carried out data extraction and quality assessment and this was checked by a second reviewer. Review conclusions Was it worthwhile including •Anarrative synthesis was conducted. • The conclusion of the review was that good quality observational studies? randomised controlled trials assessing the • Stricter study design inclusion criteria would have led Results effectiveness of different treatment options for to the same conclusions in a more efficient way. childhood retinoblastoma are required. • Thirty-one studies met the inclusion criteria. Apart from • It could be argued that it was inappropriate to include one study where some of the participants were • Where controlled trials are not feasible, only high observational studies of treatment interventions in a randomised, only observational comparative studies were quality prospective, non-randomised studies should systematic review due to the potential for bias. be given consideration, due to the generally higher identified. Twenty-seven studies were retrospective. • However the review would have been less risk of bias in retrospective studies. • The studies were diverse. comprehensive. Specific problems with the study designs used in this field were identified. We were • Most of the studies investigated EBRT or chemotherapy able to make specific recommendations for (see Figure). There were very few studies available on This project was funded by the UK National Health improvement. Hopefully this will be useful for local treatments with only plaque radiotherapy or Service Health Technology Assessment (HTA) clinicians evaluating treatments for retinoblastoma. brachytherapy and photocoagulation being assessed Programme (project number 03/63). The full results of • This review is currently being updated as a Cochrane as individual treatments. There were no comparative this systematic review will be published as a HTA review. Only controlled studies will be eligible for studies assessing the effectiveness of cryotherapy, monograph. The views and opinions expressed here inclusion. It is unlikely that any studies will be found thermotherapy or chemothermotherapy. No studies are those of the authors and do not necessarily reflect that meet these stricter inclusion criteria. compared different local treatments. those of the Department of Health. For further details contact Dr. Catriona Mc Daid [email protected] Promoting the use of research based knowledge Centre for Reviews and Dissemination.
Load more

Recommended publications
  • Adaptive Clinical Trials: an Introduction
    Adaptive clinical trials: an introduction What are the advantages and disadvantages of adaptive clinical trial designs? How and why were Introduction adaptive clinical Adaptive clinical trial design is trials developed? becoming a hot topic in healthcare research, with some researchers In 2004, the FDA published a report arguing that adaptive trials have the on the problems faced by the potential to get new drugs to market scientific community in developing quicker. In this article, we explain new medical treatments.2 The what adaptive trials are and why they report highlighted that the pace of were developed, and we explore both innovation in biomedical science is the advantages of adaptive designs outstripping the rate of advances and the concerns being raised by in the available technologies and some in the healthcare community. tools for evaluating new treatments. Outdated tools are being used to assess new treatments and there What are adaptive is a critical need to improve the effectiveness and efficiency of clinical trials? clinical trials.2 The current process for developing Adaptive clinical trials enable new treatments is expensive, takes researchers to change an aspect a long time and in some cases, the of a trial design at an interim development process has to be assessment, while controlling stopped after significant amounts the rate of type 1 errors.1 Interim of time and resources have been assessments can help to determine invested.2 In 2006, the FDA published whether a trial design is the most a “Critical Path Opportunities
    [Show full text]
  • Quasi-Experimental Studies in the Fields of Infection Control and Antibiotic Resistance, Ten Years Later: a Systematic Review
    HHS Public Access Author manuscript Author ManuscriptAuthor Manuscript Author Infect Control Manuscript Author Hosp Epidemiol Manuscript Author . Author manuscript; available in PMC 2019 November 12. Published in final edited form as: Infect Control Hosp Epidemiol. 2018 February ; 39(2): 170–176. doi:10.1017/ice.2017.296. Quasi-experimental Studies in the Fields of Infection Control and Antibiotic Resistance, Ten Years Later: A Systematic Review Rotana Alsaggaf, MS, Lyndsay M. O’Hara, PhD, MPH, Kristen A. Stafford, PhD, MPH, Surbhi Leekha, MBBS, MPH, Anthony D. Harris, MD, MPH, CDC Prevention Epicenters Program Department of Epidemiology and Public Health, University of Maryland School of Medicine, Baltimore, Maryland. Abstract OBJECTIVE.—A systematic review of quasi-experimental studies in the field of infectious diseases was published in 2005. The aim of this study was to assess improvements in the design and reporting of quasi-experiments 10 years after the initial review. We also aimed to report the statistical methods used to analyze quasi-experimental data. DESIGN.—Systematic review of articles published from January 1, 2013, to December 31, 2014, in 4 major infectious disease journals. METHODS.—Quasi-experimental studies focused on infection control and antibiotic resistance were identified and classified based on 4 criteria: (1) type of quasi-experimental design used, (2) justification of the use of the design, (3) use of correct nomenclature to describe the design, and (4) statistical methods used. RESULTS.—Of 2,600 articles, 173 (7%) featured a quasi-experimental design, compared to 73 of 2,320 articles (3%) in the previous review (P<.01). Moreover, 21 articles (12%) utilized a study design with a control group; 6 (3.5%) justified the use of a quasi-experimental design; and 68 (39%) identified their design using the correct nomenclature.
    [Show full text]
  • A Guide to Systematic Review and Meta-Analysis of Prediction Model Performance
    RESEARCH METHODS AND REPORTING A guide to systematic review and meta-analysis of prediction model performance Thomas P A Debray,1,2 Johanna A A G Damen,1,2 Kym I E Snell,3 Joie Ensor,3 Lotty Hooft,1,2 Johannes B Reitsma,1,2 Richard D Riley,3 Karel G M Moons1,2 1Cochrane Netherlands, Validation of prediction models is diagnostic test accuracy studies. Compared to therapeu- University Medical Center tic intervention and diagnostic test accuracy studies, Utrecht, PO Box 85500 Str highly recommended and increasingly there is limited guidance on the conduct of systematic 6.131, 3508 GA Utrecht, Netherlands common in the literature. A systematic reviews and meta-analysis of primary prognosis studies. 2Julius Center for Health review of validation studies is therefore A common aim of primary prognostic studies con- Sciences and Primary Care, cerns the development of so-called prognostic predic- University Medical Center helpful, with meta-analysis needed to tion models or indices. These models estimate the Utrecht, PO Box 85500 Str 6.131, 3508 GA Utrecht, summarise the predictive performance individualised probability or risk that a certain condi- Netherlands of the model being validated across tion will occur in the future by combining information 3Research Institute for Primary from multiple prognostic factors from an individual. Care and Health Sciences, Keele different settings and populations. This Unfortunately, there is often conflicting evidence about University, Staffordshire, UK article provides guidance for the predictive performance of developed prognostic Correspondence to: T P A Debray [email protected] researchers systematically reviewing prediction models. For this reason, there is a growing Additional material is published demand for evidence synthesis of (external validation) online only.
    [Show full text]
  • How to Get Started with a Systematic Review in Epidemiology: an Introductory Guide for Early Career Researchers
    Denison et al. Archives of Public Health 2013, 71:21 http://www.archpublichealth.com/content/71/1/21 ARCHIVES OF PUBLIC HEALTH METHODOLOGY Open Access How to get started with a systematic review in epidemiology: an introductory guide for early career researchers Hayley J Denison1*†, Richard M Dodds1,2†, Georgia Ntani1†, Rachel Cooper3†, Cyrus Cooper1†, Avan Aihie Sayer1,2† and Janis Baird1† Abstract Background: Systematic review is a powerful research tool which aims to identify and synthesize all evidence relevant to a research question. The approach taken is much like that used in a scientific experiment, with high priority given to the transparency and reproducibility of the methods used and to handling all evidence in a consistent manner. Early career researchers may find themselves in a position where they decide to undertake a systematic review, for example it may form part or all of a PhD thesis. Those with no prior experience of systematic review may need considerable support and direction getting started with such a project. Here we set out in simple terms how to get started with a systematic review. Discussion: Advice is given on matters such as developing a review protocol, searching using databases and other methods, data extraction, risk of bias assessment and data synthesis including meta-analysis. Signposts to further information and useful resources are also given. Conclusion: A well-conducted systematic review benefits the scientific field by providing a summary of existing evidence and highlighting unanswered questions. For the individual, undertaking a systematic review is also a great opportunity to improve skills in critical appraisal and in synthesising evidence.
    [Show full text]
  • Guidelines for Reporting Meta-Epidemiological Methodology Research
    EBM Primer Evid Based Med: first published as 10.1136/ebmed-2017-110713 on 12 July 2017. Downloaded from Guidelines for reporting meta-epidemiological methodology research Mohammad Hassan Murad, Zhen Wang 10.1136/ebmed-2017-110713 Abstract The goal is generally broad but often focuses on exam- Published research should be reported to evidence users ining the impact of certain characteristics of clinical studies on the observed effect, describing the distribu- Evidence-Based Practice with clarity and transparency that facilitate optimal tion of research evidence in a specific setting, exam- Center, Mayo Clinic, Rochester, appraisal and use of evidence and allow replication Minnesota, USA by other researchers. Guidelines for such reporting ining heterogeneity and exploring its causes, identifying are available for several types of studies but not for and describing plausible biases and providing empirical meta-epidemiological methodology studies. Meta- evidence for hypothesised associations. Unlike classic Correspondence to: epidemiological studies adopt a systematic review epidemiology, the unit of analysis for meta-epidemio- Dr Mohammad Hassan Murad, or meta-analysis approach to examine the impact logical studies is a study, not a patient. The outcomes Evidence-based Practice Center, of certain characteristics of clinical studies on the of meta-epidemiological studies are usually not clinical Mayo Clinic, 200 First Street 6–8 observed effect and provide empirical evidence for outcomes. SW, Rochester, MN 55905, USA; hypothesised associations. The unit of analysis in meta- In this guide, we adapt the items used in the PRISMA murad. mohammad@ mayo. edu 9 epidemiological studies is a study, not a patient. The statement for reporting systematic reviews and outcomes of meta-epidemiological studies are usually meta-analysis to fit the setting of meta- epidemiological not clinical outcomes.
    [Show full text]
  • Adaptive Designs in Clinical Trials: Why Use Them, and How to Run and Report Them Philip Pallmann1* , Alun W
    Pallmann et al. BMC Medicine (2018) 16:29 https://doi.org/10.1186/s12916-018-1017-7 CORRESPONDENCE Open Access Adaptive designs in clinical trials: why use them, and how to run and report them Philip Pallmann1* , Alun W. Bedding2, Babak Choodari-Oskooei3, Munyaradzi Dimairo4,LauraFlight5, Lisa V. Hampson1,6, Jane Holmes7, Adrian P. Mander8, Lang’o Odondi7, Matthew R. Sydes3,SofíaS.Villar8, James M. S. Wason8,9, Christopher J. Weir10, Graham M. Wheeler8,11, Christina Yap12 and Thomas Jaki1 Abstract Adaptive designs can make clinical trials more flexible by utilising results accumulating in the trial to modify the trial’s course in accordance with pre-specified rules. Trials with an adaptive design are often more efficient, informative and ethical than trials with a traditional fixed design since they often make better use of resources such as time and money, and might require fewer participants. Adaptive designs can be applied across all phases of clinical research, from early-phase dose escalation to confirmatory trials. The pace of the uptake of adaptive designs in clinical research, however, has remained well behind that of the statistical literature introducing new methods and highlighting their potential advantages. We speculate that one factor contributing to this is that the full range of adaptations available to trial designs, as well as their goals, advantages and limitations, remains unfamiliar to many parts of the clinical community. Additionally, the term adaptive design has been misleadingly used as an all-encompassing label to refer to certain methods that could be deemed controversial or that have been inadequately implemented. We believe that even if the planning and analysis of a trial is undertaken by an expert statistician, it is essential that the investigators understand the implications of using an adaptive design, for example, what the practical challenges are, what can (and cannot) be inferred from the results of such a trial, and how to report and communicate the results.
    [Show full text]
  • A Systematic Review of Quasi-Experimental Study Designs in the Fields of Infection Control and Antibiotic Resistance
    INVITED ARTICLE ANTIMICROBIAL RESISTANCE George M. Eliopoulos, Section Editor A Systematic Review of Quasi-Experimental Study Designs in the Fields of Infection Control and Antibiotic Resistance Anthony D. Harris,1,2 Ebbing Lautenbach,3,4 and Eli Perencevich1,2 1 2 Division of Health Care Outcomes Research, Department of Epidemiology and Preventive Medicine, University of Maryland School of Medicine, and Veterans Downloaded from https://academic.oup.com/cid/article/41/1/77/325424 by guest on 24 September 2021 Affairs Maryland Health Care System, Baltimore, Maryland; and 3Division of Infectious Diseases, Department of Medicine, and 4Department of Biostatistics and Epidemiology, Center for Clinical Epidemiology and Biostatistics, University of Pennsylvania School of Medicine, Philadelphia, Pennsylvania We performed a systematic review of articles published during a 2-year period in 4 journals in the field of infectious diseases to determine the extent to which the quasi-experimental study design is used to evaluate infection control and antibiotic resistance. We evaluated studies on the basis of the following criteria: type of quasi-experimental study design used, justification of the use of the design, use of correct nomenclature to describe the design, and recognition of potential limitations of the design. A total of 73 articles featured a quasi-experimental study design. Twelve (16%) were associated with a quasi-exper- imental design involving a control group. Three (4%) provided justification for the use of the quasi-experimental study design. Sixteen (22%) used correct nomenclature to describe the study. Seventeen (23%) mentioned at least 1 of the potential limitations of the use of a quasi-experimental study design.
    [Show full text]
  • A Systematic Review of COVID-19 Epidemiology Based on Current Evidence
    Journal of Clinical Medicine Review A Systematic Review of COVID-19 Epidemiology Based on Current Evidence Minah Park, Alex R. Cook *, Jue Tao Lim , Yinxiaohe Sun and Borame L. Dickens Saw Swee Hock School of Public Health, National Health Systems, National University of Singapore, Singapore 117549, Singapore; [email protected] (M.P.); [email protected] (J.T.L.); [email protected] (Y.S.); [email protected] (B.L.D.) * Correspondence: [email protected]; Tel.: +65-8569-9949 Received: 18 March 2020; Accepted: 27 March 2020; Published: 31 March 2020 Abstract: As the novel coronavirus (SARS-CoV-2) continues to spread rapidly across the globe, we aimed to identify and summarize the existing evidence on epidemiological characteristics of SARS-CoV-2 and the effectiveness of control measures to inform policymakers and leaders in formulating management guidelines, and to provide directions for future research. We conducted a systematic review of the published literature and preprints on the coronavirus disease (COVID-19) outbreak following predefined eligibility criteria. Of 317 research articles generated from our initial search on PubMed and preprint archives on 21 February 2020, 41 met our inclusion criteria and were included in the review. Current evidence suggests that it takes about 3-7 days for the epidemic to double in size. Of 21 estimates for the basic reproduction number ranging from 1.9 to 6.5, 13 were between 2.0 and 3.0. The incubation period was estimated to be 4-6 days, whereas the serial interval was estimated to be 4-8 days. Though the true case fatality risk is yet unknown, current model-based estimates ranged from 0.3% to 1.4% for outside China.
    [Show full text]
  • Protocol for a Systematic Review and Meta- Analysis of Quasi
    Open access Protocol BMJ Open: first published as 10.1136/bmjopen-2020-038258 on 9 September 2020. Downloaded from Risk factors for disruptive behaviours: protocol for a systematic review and meta- analysis of quasi- experimental evidence Lucy Karwatowska ,1 Simon Russell,1 Francesca Solmi ,2 Bianca Lucia De Stavola,1 Sara Jaffee,3 Jean- Baptiste Pingault,4,5 Essi Viding4 To cite: Karwatowska L, ABSTRACT Strengths and limitations of this study Russell S, Solmi F, et al. Introduction Disruptive behaviour disorders, including Risk factors for disruptive oppositional defiant disorder and conduct disorder, are a behaviours: protocol for a ► The strengths of the current study include systemat- common set of diagnoses in childhood and adolescence, systematic review and meta- ically evaluating quasi- experimental research, which analysis of quasi- experimental with global estimates of 5.7%, 3.6% and 2.1% for any aims to estimate causal effects, of a wide range of evidence. BMJ Open disruptive disorder, oppositional defiant disorder and risk factors for a variety of disruptive behaviours. 2020;10:e038258. doi:10.1136/ conduct disorder, respectively. There are high economic ► The risk factors will not be selected a priori allowing bmjopen-2020-038258 and social costs associated with disruptive behaviours any risk factor to be included in the review and the and the prevalence of these disorders has increased in ► Prepublication history and quality of evidence will be assessed to provide con- additional material for this recent years. As such, disruptive behaviours represent an fidence in the effect estimates. paper are available online. To escalating major public health concern and it is important ► The present study will be the first to synthesise view these files, please visit to understand what factors may influence the risk of these quasi- experimental evidence for disruptive be- the journal online (http:// dx.
    [Show full text]
  • A Systematic Review and Meta-Analysis of Clinical Trials on Physical Interventions for Lateral Epicondylalgia
    411 REVIEW A systematic review and meta-analysis of clinical trials on physical interventions for lateral epicondylalgia L Bisset, A Paungmali, B Vicenzino, E Beller ............................................................................................................................... Br J Sports Med 2005;39:411–422. doi: 10.1136/bjsm.2004.016170 A systematic review of the literature on the effectiveness of to May 1999), and non-steroidal anti-inflamma- tory drugs (NSAIDs; databases searched to June physical interventions for lateral epicondylalgia (tennis 2001).11–14 Another systematic review (databases elbow) was carried out. Seventy six randomised controlled searched to June 2002) within the Cochrane trials were identified, 28 of which satisfied the minimum Library has looked at the effectiveness of transverse friction massage on general tendino- criteria for meta-analysis. The evidence suggests that pathy conditions.15 The prevailing conclusion extracorporeal shock wave therapy is not beneficial in the drawn by many of these reviews was that of treatment of tennis elbow. There is a lack of evidence for insufficient evidence to support any one treat- ment over another, predominantly because of the the long term benefit of physical interventions in general. low statistical power, inadequate internal validity, However, further research with long term follow up into and insufficient data reported in most of the trials. manipulation and exercise as treatments is indicated. Before the systematic review of Smidt et al,10 there was
    [Show full text]
  • Has a Systematic Review Been Done on Your Topic?
    U T S CHOOL OF P UBLIC H EALTH L IBRARY Has a Systematic Review Been Done on Your Topic? Before you begin the process of developing a systematic review, you need to determine if a systematic review been done on your topic. Is it identical to your proposed topic? How current is it? The number of completed SRs has increased dramatically over the past few years so don’t be discouraged as you begin searching the literature for prior SRs. You may very well find a systematic review that is similar to yours, but your focus might include a different aspect of the topic. The prior SR may have included only clinical trials and you intend to review case-control studies or it may have included subjects over 50 but you want to look at populations younger than 30. Another key consideration is the currency of prior SRs you find as even those that are only a few years old can be out of date. In some cases, an SR can be out of date in only a year, especially if a new treatment or diagnostic method for a disease or condition has come into use. Therefore, even if you find SRs on your topic, be sure to look at them to make certain your topic really isn’t taken. If your topic is taken, look at their conclusions. What do they feel is missing from their review? What are their recommendations for follow-up reviews? Is it something that you can do? For this handbook, the starting topic will be: How can patient compliance for treatment of latent tuberculosis be improved? Inclusion criteria include: English language, and multiple research methodologies including clinical trials, evaluation studies, pre-post design, and case-control studies.
    [Show full text]
  • Checklist for Quasi-Experimental Studies (Non-Randomized Experimental Studies)
    The Joanna Briggs Institute Critical Appraisal tools for use in JBI Systematic Reviews Checklist for Quasi-Experimental Studies (non-randomized experimental studies) http://joannabriggs.org/research/critical-appraisal-tools.html www.joannabriggs.org The Joanna Briggs Institute Introduction The Joanna Briggs Institute (JBI) is an international, membership based research and development organization within the Faculty of Health Sciences at the University of Adelaide. The Institute specializes in promoting and supporting evidence-based healthcare by providing access to resources for professionals in nursing, midwifery, medicine, and allied health. With over 80 collaborating centres and entities, servicing over 90 countries, the Institute is a recognized global leader in evidence-based healthcare. JBI Systematic Reviews The core of evidence synthesis is the systematic review of literature of a particular intervention, condition or issue. The systematic review is essentially an analysis of the available literature (that is, evidence) and a judgment of the effectiveness or otherwise of a practice, involving a series of complex steps. The JBI takes a particular view on what counts as evidence and the methods utilized to synthesize those different types of evidence. In line with this broader view of evidence, the Institute has developed theories, methodologies and rigorous processes for the critical appraisal and synthesis of these diverse forms of evidence in order to aid in clinical decision-making in health care. There now exists JBI guidance for conducting reviews of effectiveness research, qualitative research, prevalence/incidence, etiology/risk, economic evaluations, text/opinion, diagnostic test accuracy, mixed-methods, umbrella reviews and scoping reviews. Further information regarding JBI systematic reviews can be found in the JBI Reviewer’s Manual on our website.
    [Show full text]