Use of Clinical Trial Registries in Otolaryngology Systematic Reviews

Letters

Corresponding Author: Jesse D. Pasternak, MD, MPH, FRCSC, Division of Figure 2. Cumulative Risk of Being Diagnosed With Thyroid Cancer General Surgery, Department of General Surgery, University Health Network, After Benign Index Biopsy Results 200 Elizabeth St, Toronto, ON M5G 2C4, Canada ([email protected]).

10 Published Online: April 18, 2019. doi:10.1001/jamaoto.2019.0386 Author Contributions: Drs Urbach and Pasternak had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy 8 of the data analysis. Drs Urbach and Pasternak are co-principle investigators. Concept and design: Devon, Rotstein, Urbach, Pasternak. Acquisition, analysis, or interpretation of data: Bongers, Kluijfhout, Rotstein, 6 Vriens, Urbach, Pasternak. Drafting of the manuscript: Bongers, Kluijfhout, Pasternak. 4 Critical revision of the manuscript for important intellectual content: All authors. Statistical analysis: Bongers, Urbach. Cumulative Risk, % Cumulative Administrative, technical, or material support: Bongers, Kluijfhout, Urbach, 2 Pasternak. Supervision: Rotstein, Vriens, Urbach, Pasternak. Conflict of Interest Disclosures: None reported. 0 0 2 4 6 8 10 12 14 16 18 20 22 24 Funding/Support: This study was supported by the Ontario Institute for Cancer Years After Benign Index Biopsy Research and Cancer Care Ontario (CCO) through funding provided by the Government of Ontario. This study was supported through provision of data by Vertical axis has been rescaled from 0% to 10% to better demonstrate the the Institute for Clinical Evaluative Sciences (ICES) and CCO and through cumulative risk curve. funding support to ICES from an annual grant by the Ministry of Health and Long-Term Care. Role of the Funder/Sponsor: The funding sources had no role in the design and conduct of the study; collection, management, analysis, and interpretation of The study design strengthens the validity of these re- the data; preparation, review, or approval of the manuscript; and decision to sults. The administrative health databases from Ontario are submit the manuscript for publication. 6 more than 95% complete for cancer diagnosis and procedures. Disclaimer: The opinions, results, and conclusions reported in this article are By capturing an entire population, our study was less suscep- those of the authors and are independent from the funding sources. No tible to the types of selection biases and confounding that may endorsement by the Institute for Clinical Evaluative Sciences, Cancer Care Ontario, or the Government of Ontario is intended or should be inferred. have influenced other studies. 1. Hegedüs L. Clinical practice: the thyroid nodule. N Engl J Med. 2004;351(17): Limitations to this study include the lack of patient- 1764-1771. doi:10.1056/NEJMcp031436 specific clinical information such as results of ultrasonogra- 2. Durante C, Costante G, Lucisano G, et al. The natural history of benign phy and pathologic tests. In addition, changes in specimen thyroid nodules. JAMA. 2015;313(9):926-935. doi:10.1001/jama.2015.0956 management and diagnostic criteria over time may have in- 3. Zevallos JP, Hartman CM, Kramer JR, Sturgis EM, Chiao EY. Increased thyroid fluenced the rate of carcinoma because of more incidentally cancer incidence corresponds to increased use of thyroid ultrasound and found microcarcinomas. Although the rate of carcinoma may fine-needle aspiration: a study of the Veterans Affairs health care system. Cancer. 2015;121(5):741-746. doi:10.1002/cncr.29122 have increased, these diagnoses are likely predominantly due 4. Becker-Weidman DJ, Malhotra N, Reilly DF, Selvam N, Parker L, Nazarian LN. to a clinically irrelevant entity. Imaging surveillance in patients after a benign fine-needle aspiration biopsy of Based on a large provincial population followed long- the thyroid: associated cost and incidence of subsequent cancer. AJRAmJ term after initially benign results of thyroid biopsy, the rate of Roentgenol. 2017;208(2):358-361. doi:10.2214/AJR.16.16691 doi:10.2214/AJR.16. malignant neoplasms was low, which questions the need of 16691 follow-up biopsies for all patients. Because cumulative risk 5. Xing M. Molecular pathogenesis and mechanisms of thyroid cancer. Nat Rev Cancer. 2013;13(3):184-199. doi:10.1038/nrc3431 of thyroid cancer in these patients is higher than the baseline 6. Hall S, Schulze K, Groome P, Mackillop W, Holowaty E. Using cancer registry lifetime risk of the population, further large risk stratification data for survival studies: the example of the Ontario Cancer Registry. J Clin studies incorporating standard ultrasound biopsy data are Epidemiol. 2006;59(1):67-76. doi:10.1016/j.jclinepi.2005.05.001 needed to identify those requiring long-term follow-up. Use of Clinical Trial Registries in Otolaryngology Pim J. Bongers, MD Systematic Reviews Wouter P. Kluijfhout, MD, PhD Systematic reviews are foundational to evidence-based Karen Devon, MD, FRCSC patient care, yet these reviews are criticized for including Lorne E. Rotstein, MD, FRCSC only statistically significant findings to estimate summary Menno R. Vriens, MD, PhD effects, which may lead to publication bias.1,2 The exclusion David R. Urbach, MSc, MD, FRCSC of nonsignificant and/or unpublished results can lead to an Jesse D. Pasternak, MD, MPH, FRCSC overrepresentation of an intervention’s actual effect and 3,4 Author Affiliations: Division of General Surgery, Department of Surgery, potentially imprecise clinical decision making. Although University Health Network, Toronto, Ontario, Canada (Bongers, Rotstein, many methods exist to minimize publication bias in system- Pasternak); Department of Surgical Oncology and Endocrine Surgery, University atic reviews, the use of clinical trial registries (CTRs) is par- Medical Center, Utrecht, the Netherlands (Bongers, Kluijfhout, Vriens); ticularly encouraging. Department of Surgery, Women’s College Hospital, Toronto, Ontario, Canada (Devon, Urbach); Institute of Health Policy, Management and Evaluation, Clinical trial registries index new and ongoing studies, and University of Toronto, Toronto, Ontario, Canada (Urbach). may provide access to ongoing trial data prior to a study’s print Accepted for Publication: February 13, 2019. release, even if nonsignificant findings arise.5 Given the

580 JAMA Otolaryngology–Head & Neck Surgery June 2019 Volume 145, Number 6 (Reprinted) jamaotolaryngology.com

Downloaded From: https://jamanetwork.com/ on 09/28/2021 Letters

potential of CTRs to identify unpublished studies for mitigat- relevant studies for further consideration. Given that CTRs are ing the potential of publication bias, we investigated the use freely available in the public domain and are not time con- of CTRs in systematic reviews published in the top otolaryn- suming to search, we recommend that they be routinely in- gology journals and evaluated whether additional studies could cluded as part of a larger gray literature search strategy and have been located for systematic reviews that omitted represent a promising solution to address publication bias. CTR searches. Kaleb Fuller, BS Methods | Using Google Scholar h5-index scores, the following Wade Arthur, BS journals were identified: American Journal of Otolaryngology– Jonathan Riddle, MS Head and Neck Medicine and Surgery, Clinical Otolaryngology, Matt Vassar, PhD Current Opinions in Otolaryngology & Head and Neck Surgery, International Journal of Otolaryngology, JAMA Otolaryngology– Author Affiliations: Center for Health Sciences, Department of Psychiatry and Behavioral Sciences, Oklahoma State University, Tulsa. Head & Neck Surgery, Journal of the Association for Research Corresponding Author: Kaleb Fuller, BS, Center for Health Sciences, Oklahoma in Otolaryngology, Journal of Otolaryngology–Head & Neck State University, 1111 W 17th St, Tulsa, OK 74107 ([email protected]). Surgery, The Laryngoscope, and Otolaryngology–Head and Accepted for Publication: February 23, 2019. Neck Surgery. A PubMed search was performed for relevant Conflict of Interest Disclosures: None reported. systematic reviews from January 1, 2008, to December 8, 2017. Published Online: April 25, 2019. doi:10.1001/jamaoto.2019.0521 On December 19, 2017, we searched for systematic reviews pub- Author Contributions: Kaleb Fuller had full access to all of the data in the study lished by the Cochrane Ear, Nose and Throat Disorders Group and takes responsibility for the integrity of the data and the accuracy of the using the Cochrane Database of Systematic Reviews with the data analysis. same date limiter. Eligible reviews were examined to identify Study concept and design: Vassar. whether a CTR was searched, and if so, which CTR(s) were re- Acquisition, analysis, or interpretation of data: All authors. Drafting of the manuscript: All authors. ported. We also extracted title, author(s), journal, country of Critical revision of the manuscript for important intellectual content: origin, and source (PubMed or Cochrane). We used the World Fuller, Vassar. Health Organization’s International Clinical Trials Registry Statistical analysis: Fuller, Riddle. Administrative, technical, or material support: Arthur, Riddle, Vassar. Platform (ICTRP) Version 2.1 list of eligible CTRs and also Study supervision: Vassar. included ClinicalTrials.gov.6 1. Rosenfeld RM, Shiffman RN. Clinical practice guideline development manual: We next selected a random sample of 30 journal- a quality-driven approach for translating evidence into action. Otolaryngol Head published systematic reviews that excluded trial registry Neck Surg. 2009;140(6)(suppl 1):S1-S43. doi:10.1016/j.otohns.2009.04.015 searches, and searched ClinicalTrials.gov for completed pri- 2. Institute of Medicine, Board on Health Care Services, Committee on mary studies that could have been further considered by sys- Standards for Developing Trustworthy Clinical Practice Guidelines. Clinical Practice Guidelines We Can Trust. National Academies Press; Washington, DC 2011. tematic reviewers had they conducted CTR searches. For each review, we sought to identify the number of clinical trials that 3. Dwan K, Gamble C, Williamson PR, Kirkham JJ; Reporting Bias Group. Systematic review of the empirical evidence of study publication bias and were possibilities for inclusion. ClinicalTrials.gov searches were outcome reporting bias: an updated review. PLoS One. 2013;8(7):e66844. doi: conducted from December 2 to 9, 2018. 10.1371/journal.pone.0066844 4. Montori VM, Smieja M, Guyatt GH. Publication bias: a brief review for Results | After exclusions, 515 PubMed journal-published sys- clinicians. Mayo Clin Proc. 2000;75(12):1284-1288. doi:10.4065/75.12.1284 tematic reviews and 39 Cochrane reviews were included. Of 5. Committee on Clinical Trial Registries, Board on Health Sciences Policy, the 515 journal-published reviews, 26 (5%) reported a search Institute of Medicine. Content of a Clinical Trial Registry. US: National Academies Press; Washington, DC 2006. of 1 or more CTRs. ClinicalTrials.gov was cited most 6. WHO. Primary Registries. August 2018. http://www.who.int/ictrp/network/ frequently (n = 24), followed by the ICTRP (n = 8), the primary/en/. Accessed September 23, 2018. European Clinical Trials Register (n = 2), and the Chinese Clinical Trial Registry (n = 1). In comparison, of the 39 Cochrane reviews, 35 (90%) reported searching a CTR. The OBSERVATION Cochrane reviews reported searches of 3 registries: ICTRP (n = 35), ClinicalTrials.gov (n = 31), and ISRCTN/IRCT Follicular Thyroid Carcinoma Presenting (n=28). as a Manubrium Mass From our random sample of 30 systematic reviews, We describe a case of an elderly man who presented with swell- 14 (46.7%) could have found at least 1 relevant study for ing over the manubrium 12 years after substernal left hemithy- inclusion in the systematic reviews had a CTR search been roidectomy for benign goiter. Subsequent manubrium resec- performed. tion showed follicular cell thyroid carcinoma. Revision of the previously resected goiter specimen revealed no malignant Discussion | Only a minority of systematic reviews published in tumor. To our knowledge, this is the first case of follicular top-ranked otolaryngology journals reported searching a CTR cancer presenting in a prior sternotomy following resection of (26 of 515 [5%]). By comparison, systematic reviews pub- a benign goiter. lished by the Cochrane Collaboration almost always reported searching trial registries. Furthermore, almost 50% of system- Report of a Case | A man in his 70s presented with swelling of atic reviews in our second analysis would have located the manubrium 12 years after resection of a large left subster-

jamaotolaryngology.com (Reprinted) JAMA Otolaryngology–Head & Neck Surgery June 2019 Volume 145, Number 6 581

Downloaded From: https://jamanetwork.com/ on 09/28/2021