This article was downloaded by: [Department Of Fisheries] On: 17 November 2014, At: 00:12 Publisher: Taylor & Francis Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK Journal of Aquatic Animal Health Publication details, including instructions for authors and subscription information: http://www.tandfonline.com/loi/uahh20 Improvements are Needed in Reporting of Accuracy Studies for Diagnostic Tests Used for Detection of Finfish Pathogens Ian A. Gardnera, Timothy Burnleyb & Charles Caraguelc a Atlantic Veterinary College, University of Prince Edward Island, 550 University Avenue, Charlottetown, Prince Edward Island C1A 4P3, Canada b Eastern Epidemiological Services, 23 Karen Drive, Cornwall, Prince Edward Island C0A 1H8, Canada c School of Animal and Veterinary Sciences, Roseworthy Campus, University of Adelaide, Roseworthy, South Australia 5371, Australia Published online: 17 Sep 2014. To cite this article: Ian A. Gardner, Timothy Burnley & Charles Caraguel (2014) Improvements are Needed in Reporting of Accuracy Studies for Diagnostic Tests Used for Detection of Finfish Pathogens, Journal of Aquatic Animal Health, 26:4, 203-209, DOI: 10.1080/08997659.2014.938867 To link to this article: http://dx.doi.org/10.1080/08997659.2014.938867 PLEASE SCROLL DOWN FOR ARTICLE Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”) contained in the publications on our platform. However, Taylor & Francis, our agents, and our licensors make no representations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of the Content. Any opinions and views expressed in this publication are the opinions and views of the authors, and are not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be relied upon and should be independently verified with primary sources of information. Taylor and Francis shall not be liable for any losses, actions, claims, proceedings, demands, costs, expenses, damages, and other liabilities whatsoever or howsoever caused arising directly or indirectly in connection with, in relation to or arising out of the use of the Content. This article may be used for research, teaching, and private study purposes. Any substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any form to anyone is expressly forbidden. Terms & Conditions of access and use can be found at http:// www.tandfonline.com/page/terms-and-conditions Journal of Aquatic Animal Health 26:203–209, 2014 C American Fisheries Society 2014 ISSN: 0899-7659 print / 1548-8667 online DOI: 10.1080/08997659.2014.938867 COMMUNICATION Improvements are Needed in Reporting of Accuracy Studies for Diagnostic Tests Used for Detection of Finfish Pathogens Ian A. Gardner* Atlantic Veterinary College, University of Prince Edward Island, 550 University Avenue, Charlottetown, Prince Edward Island C1A 4P3, Canada Timothy Burnley Eastern Epidemiological Services, 23 Karen Drive, Cornwall, Prince Edward Island C0A 1H8, Canada Charles Caraguel School of Animal and Veterinary Sciences, Roseworthy Campus, University of Adelaide, Roseworthy, South Australia 5371, Australia Accurate diagnostic tests are central to the detection, control, Abstract and prevention of pathogen spread in aquaculture. The World Indices of test accuracy, such as diagnostic sensitivity and speci- Organization for Animal Health (OIE) specifies that test accu- ficity, are important considerations in test selection for a defined racy in aquatic species should be estimated to assess a test’s purpose (e.g., screening or confirmation) and affect the interpreta- tion of test results. Many biomedical journals recommend that au- “fitness for the intended purpose(s).” Such purposes include the thors clearly and transparently report test accuracy studies follow- demonstration or certification of freedom from disease, the con- ing the Standards for Reporting of Diagnostic Accuracy (STARD) firmation of clinically suspected cases, and the estimation of guidelines (www.stard-statement.org). This allows readers to eval- disease prevalence in a fish population, to list a few. Indices uate overall study validity and assess potential bias in diagnostic of test accuracy, such as diagnostic sensitivity and specificity, sensitivity and specificity estimates. The purpose of the present study was to evaluate the reporting quality of studies evaluating are important considerations when selecting a test for a defined test accuracy for finfish diseases using the 25 items in the STARD purpose and interpreting its results. Regardless of the intended checklist. Based on a database search, 11 studies that included purpose of the test, complete and transparent reporting of the estimates of diagnostic accuracy were identified for independent design elements and the results of a test accuracy study are evaluation by three reviewers. For each study, STARD checklist essential so that readers can appraise the validity of the test, items were scored as “yes,” “no,” or “not applicable.” Only 10 of the 25 items were consistently reported in most (≥80%) pa- including the potential for bias in diagnostic sensitivity and pers, and reporting of the other items was highly variable (mostly specificity estimates. Downloaded by [Department Of Fisheries] at 00:12 17 November 2014 between 30% and 60%). Three items (“number, training, and ex- The Standards for Reporting of Diagnostic Accuracy pertise of readers and testers”; “time interval between index tests (STARD) statement (www.stard-statement.org) was published and reference standard”; and “handling of indeterminate results, in 2003 to encourage improved reporting of the important fea- missing data, and outliers of the index tests”) were reported in less than 10% of papers. Two items (“time interval between index tests tures of test accuracy studies in human medicine (Bossuyt et al. and reference standard” and “adverse effects from testing”) were 2003). The statement was motivated by widespread evidence of considered minimally relevant to fish health because test samples bias in test accuracy studies and the finding that incomplete re- usually are collected postmortem. Modification of STARD to fit porting or a lack of reporting on items in the STARD checklist finfish studies should increase use by authors and thereby improve was associated with overly optimistic estimates of diagnostic the overall reporting quality regardless of how the study was de- signed. Furthermore, the use of STARD may lead to the improved sensitivity and specificity (Lijmer et al. 1999). The STARD design of future studies. does not prescribe directly how to design a test accuracy study. Rather, it focuses on clear and transparent reporting of its key *Corresponding author: [email protected] Received September 24, 2013; accepted May 15, 2014 203 204 GARDNER ET AL. TABLE 1. Reviewer scoring of STARD checklist items for 11 test accuracy studies in finfish that were published between 2006 and 2012. The percentage of yes responses was calculated based on a denominator of the total number of yes and no responses. Abbreviations are as follows: NA = item considered not applicable to the reviewed study and NC = not calculated because of a zero denominator. Reviewer 1 Reviewer 2 Reviewer 3a Section, topic, Description Yes Yes Yes Median % and total Item of item (%) No NA (%) No NA (%) No NA yes scores TITLE, ABSTRACT, 1 Identify the article as a study 7 (64) 4 0 7 (64) 4 0 8 (80) 2 0 64 KEYWORDS of diagnostic accuracy (recommend medical subject headings “sensitivity” and “specificity”). INTRODUCTION 2 State the research questions or 11 (100) 0 0 11 (100) 0 0 10 (100) 0 0 100 study aims, such as estimating diagnostic accuracy or comparing accuracy between tests or across participant groups. METHODS Participants 3 The study population: the 11 (100) 0 0 11 (100) 0 0 10 (100) 0 0 100 inclusion and exclusion criteria, setting, and locations where data were collected. 4 Participant recruitment: was 10 (91) 1 0 10 (91) 1 0 9 (90) 1 0 91 recruitment based on the presenting symptoms, the results from previous tests, or the fact that the participants had received the index tests or the reference standard? 5 Participant sampling: was the 9 (82) 2 0 9 (82) 2 0 9 (90) 1 0 82 study population a consecutive series of participants defined by the selection criteria in items 3 and 4? If not, specify how participants were further selected. 6 Data collection: was data 11 (100) 0 0 11 (100) 0 0 10 (100) 0 0 100 Downloaded by [Department Of Fisheries] at 00:12 17 November 2014 collection planned before the index test and reference standard were performed (prospective study) or after (retrospective study)? Test methods 7 The reference standard and its 7 (100) 0 4 9 (82) 0 2 4 (40) 6 0 82 rationale. 8 Technical specifications of 10 (91) 1 0 10 (91) 1 0 10 (100) 0 0 91 materials and methods involved, including how and when measurements were taken, or cite references for index tests and reference standard. (Continued on next page) COMMUNICATION 205 TABLE 1. Continued. Reviewer 1 Reviewer 2 Reviewer 3a Section, topic, Description Yes Yes Yes Median % and total Item of item (%) No NA (%) No NA (%) No NA yes scores 9 Definition of and rationale for 6 (55) 5 0 6 (55) 5 0 5 (50) 5 0 55 the units, cutoffs, and categories of the results of the index tests and the reference standard. 10 The number, training, and 0 (0) 11 0 1 (10) 10 0 0 (0) 10 0 0 expertise of the persons executing and reading the index tests and the reference standard. 11 Whether or not the readers of 2 (22) 9 0 2 (22) 9 0 2 (20) 8 0 22 the index tests and reference standard were blind (masked) to the results of the other test and describe any other clinical information available to the readers. Statistical methods 12 Methods for calculating or 6 (55) 5 0 6 (55) 5 0 7 (70) 3 0 55 comparing measures of diagnostic accuracy and the statistical methods used to quantify uncertainty (e.g., 95% confidence intervals).