Quality Assessment of Randomized Controlled Trials in Shiraz University of Medical Sciences Bahmani Jahromi Maryam1, Salehi Alireza2,*, Marzban Maryam3, Habibi Amin1
Total Page:16
File Type:pdf, Size:1020Kb
Journal of Scientometric Res. 2019; 8(2):102-108 http://www.jscires.org Research Note Quality Assessment of Randomized Controlled Trials in Shiraz University of Medical Sciences Bahmani Jahromi Maryam1, Salehi Alireza2,*, Marzban Maryam3, Habibi Amin1 1Department of MPH, Student Research Committee, Shiraz Medical School, Shiraz University of Medical Sciences, Shiraz, IRAN. 2Research Center for Traditional Medicine and History of Medicine, Shiraz University of Medical Sciences, Shiraz, IRAN. 3Department of Public Health, School of Public Health, Bushehr University of Medical Science, Bushehr, IRAN. Correspondence Alireza Salehi, ABSTRACT Randomized Controlled Trial (RCT) is scientific investigations that evaluate the safety Research Center for Traditional Medicine and efficacy of new drugs or therapeutic procedures by using human subjects. This and History of Medicine, Shiraz Univer- study aimed to assess the quality of RCT reports at Shiraz University of Medical sity of Medical Sciences, Shiraz, IRAN. Sciences (SUMS) from 2014 to 2016. A systematic search was done in international Email: [email protected] databases with the keyword “SUMS” covering the three years yielding 9124 articles. Eventually, 120 articles were selected out of 540 RCT-based articles through propor- Received: 26-12-2018 tional stratified sampling. We used the 2010 Consolidated Standards of Reporting Revised: 29-01-2019 Trials (CONSORT) statement beside the Jadad scale to assess the quality of reports. We used the Pearson Correlation to investigate probable correlations between the Accepted: 12-07-2019 CONSORT and Jadad mean scores and the Analysis of Variance (ANOVA) test to DOI: 10.5530/jscires.8.2.16 find the difference in the quality assessment tools in other studied variables. Among the selected articles, the average number of applicable reported items was 71.4 (59.5%) in the CONSORT 2010 checklist and 64 (53.4%) in Jadad. Among these RCTs, 55 (45.8%) had a high quality with Jadad scores ≥ 3 points. In addition, CONSORT and Jadad mean scores were significantly correlated P( <0.001, r: 0.808). We found no change in the mean score of CONSORT (P=0.788) or Jadad scale (P=0.492) over these three years. While the number of RCTs has gradually increased in these years, the quality of these reports has remained unchanged. Thus, national medical academics should make more efforts to conduct high-quality studies to ensure an appropriate study design. Keywords: Randomizes controlled trial, CONSORT, Jadad, Quality assessment, SUMS, Iran. INTRODUCTION and balancing, both known and unknown prognostic factors.[4] This will help the researchers to explore any effects in the RCTs are scientific investigations that evaluate the safety and treatment group versus the control group while adjusting for efficacy of new drugs or therapeutic procedures by using other variables, leading to a correct decision regarding the human subjects. The results generated by these studies are adequacy and competence of the intervention.[5] considered the most valuable data in the area of evidence- based medicine.[1,2] Adequate quality in the study design, Therefore, a number of scientists and editors attempted to implementation and reporting is substantial in obtaining develop a package to help researchers to improve their trial [6] reliable results for application in clinical interventions. RCTs design. The outcome was the CONSORT statement, which are often known as the gold standard for clinical trials.[3] includes a checklist of essential items that should be consid- ered in reporting RCTs and a diagram to certificate the flow Proper randomization, after assessing the participants for of participants through the trial in order to improve its quality. eligibility and inclusion criteria and prior to the intervention, It was first published in 1996 and updated in 2001 and 2010, will yield meaningful benefits such as reducing allocation bias enabling the readers to understand a trial’s design, conduct, analysis and interpretation and also helping them to assess the Copyright validity of the results.[5] © The Author(s). 2019 This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/li- In addition, the Jadad scale sometimes referred to as the censes/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the Oxford quality scoring system, is the most popular instru- source, provide a link to the Creative Commons license, and indicate if changes ment worldwide to facilitate the independent assessment of were made. the methodological quality of a clinical trial.[7,8] Journal of Scientometric Research, Vol 8, Issue 2, May-Aug 2019 102 Maryam, et al.: Quality Assessment of Randomized Controlled Trials Nevertheless, the literature contains many articles with poor design, methodology and analysis, even from academic sources, which will compromise scientific evidence.[9] Ascertaining the accuracy of the methodology is an indispensable factor in choosing studies for publication. Thus, evaluation of the quality of RCTs is essential, as the suboptimal quality of trials will negatively influence the researchers’ understanding of evidence and consequently affect the scientific ranking of universities.[10] Besides, Iran has had a good scientific growth in the recent decade and has raised the number of Iranian RCTs published in prestigious journals. By the year 2014, there has been an ordinary growth in Iranian’s research. Since 2014, rapid growth in scientific products, including RCTs, which were mainly conducted in medical universities of science, has been observed.[10,11] It is important to investigate that rather than the quantitative increase of RCTs, whether the quality of the reports has increased or not. Therefore, we decided to investigate the quality and degree of confidence in the RCTs published throughout 3-years (2014 to 2016), using the CONSORT 2010 and Jadad scales to find out how the items of these two scales have been used. Figure 1: Flow diagram for selection of articles for inclusion in systematic MATERIALS AND METHODS review. * Other types of the article include experimental studies, in vitro and non- Search strategy: A systematic search was performed in the RCT studies. international databases including Scopus, Web of Science, Embase, PubMed, Cochrane Library and national databases such as Science Information Database (SID), Iran Medex and the checklist items.[14] Any uncertainty regarding the correct Magiran with the keyword “Shiraz University of Medical assignment was clarified with a third researcher (M.M). The Sciences” covering three years from 2014 to 2016. inter-rater reliability was assessed for these random samples Selection of RCTs with a mean intraclass correlation coefficient of 90% [95% CI: 67.20, 97.50] and -value < 0.001. After achieving acceptable Out of 9124 articles returned from the search, 4827 duplicate P concordance in appraising, the articles were assessed based on reports were excluded from the study. The design of all the both CONSORT and Jadad scores. To minimize the selection articles was checked by two independent reviewers who were blinded to the authors and the journals in which they were bias, the articles without free access were purchased. In addition, published. They identified all eligible articles with randomly the researchers were blinded to the authors and journals assigned participants in an experimental or a control group to during the assessment process (Figure 1). conduct RCTs.[3] The following article types were considered ineligible for the current study: trials other than RCTs CONSORT statement 2010 (non-RCT studies, experimental studies, in vitro studies, etc.); The latest version of the CONSORT statement includes a case-control, cohort, cross-sectional, qualitative and method- checklist of essential items in five sections: Title and abstract, ological studies; editorials, short communications; case reports, introduction, method, discussion and other information. case series; and all types of reviews. Considering the similar Ambiguous items have been clearly explained in updated studies and comparing proportions formula. Eventually, 120 guidelines for reporting parallel group randomized trials– articles were selected out of the remaining 540 RCT-based CONSORT 2010 Explanation and Elaboration.[4] All articles through proportional stratified random sampling with CONSORT items were weighted equally. If the item was a design effect of 1.5, a confidence level of 95% and Number of Error of 10%.[12,13] Each selected article was assigned an mentioned in the proper section of the article, one point was identification code. The selected RCTs were assessed by two assigned; if the item was not mentioned, zero was assigned. researchers (M.B and M.H) independently. First, a random Items which were not applied were recorded as missing. The sample of 12 RCTs (10% of all included RCTs) was cross- mean and frequency were calculated for each CONSORT checked to reach an agreement for unambiguous allocation of item. The average score was calculated for applicable items Journal of Scientometric Research, Vol 8, Issue 2, May-Aug 2019 103 Maryam, et al.: Quality Assessment of Randomized Controlled Trials in the studied articles. Also, we calculated the mean score for Limitation: One of the limitations of this study was that our each studied articles. analysis was limited to published