Education review Br J Sports Med: first published as 10.1136/bjsports-2020-102607 on 19 August 2020. Downloaded from may possess considerable statistical exper- Call to increase statistical collaboration tise. When we use the term ‘statistician’ in this paper, we broadly include individuals in sports science, sport and exercise from other methods- focussed disciplines if they have extensive statistical training and medicine and sports physiotherapy experience. The shortage of statisticians working Kristin L Sainani ,1 David N Borg,2 Aaron R Caldwell ,3 in the field means that sports science and 4 5 6 medicine researchers are often designing Michael L Butson, Matthew S Tenan, Andrew J Vickers, studies and running analyses by them- Andrew D Vigotsky,7 John Warmenhoven,8,9 Robert Nguyen,10 selves. Some of these researchers under- Keith R. Lohse,11 Emma J Knight,12 Norma Bargary13 take in-depth training in statistics and are well-equipped to handle these tasks. However—as with other applied disci- plines—sports science and medicine Statistical errors are common in many Though some academic sports science researchers often lack adequate training in biomedical fields.1–5 We believe the nature and medicine studies employ statisticians, study design and statistics, which can lead 22–24 and impact of these errors to be great such collaborations are an exception rather to errors. This is especially problem- enough in sports science and medicine to than the norm. To determine the extent of atic as study designs and data sets become warrant special attention.6–14 Poor meth- collaboration, we performed a systematic more complex. odological and statistical practices have review of articles published in quartile one We are also concerned by a phenom- led to calls for change in other fields, such sports science journals in 2019 (see online enon in sports science and medicine. as psychology.15–18 We believe that a supplementary file 1 for methods and online Scientists in these fields are developing similar call to action is needed in sports supplementary file 2 for data). The initial statistical methods and introducing them science and medicine. Specifically, we see extraction included 8970 articles; of the into the literature without adequate peer 25–27 two pressing needs: (1) to increase collab- 400 articles selected at random, 299 were review from the statistics community. oration between researchers and statisti- deemed eligible and included in the review Many of these methods are statistically cians, and (2) to increase statistical training (figure 1). We found that only 13.3% (95% and mathematically flawed.28 29 While within the exercise science/medicine/phys- CI: 9.5% to 17.2%) of papers had at least advances in statistics sometimes have iotherapy (PT) discipline. Our call to one coauthor affiliated with a biostatis- come from applied disciplines (eg, work action extends the work of those who tics, statistics, data science, data analytics, on measurement done in education have previously called for increased statis- epidemiology, maths, computer science or and psychology), these novel statistical tical collaboration in sports medicine and economics department (figure 2). It should methods were presented, critiqued and sports injury research.19–21 be noted that we included a broad set of evaluated in the statistical literature methodological departments because we before they were introduced and used in 1Epidemiology and Population Health, Stanford recognise that individuals from these fields an applied context. University, Stanford, California, USA http://bjsm.bmj.com/ 2Menzies Health Institute Queensland, Griffith University, Nathan, Queensland, Australia 3Thermal and Mountain Medicine Division, US Army Research Institute of Environmental Medicine, Natick, Massachusetts, USA 4Deptartment of Health & Medical Sciences, Swinburne University of Technology, Hawthorn, Victoria, Australia 5Optimum Performance Analytics Associates LLC, Apex, North Carolina, USA on September 28, 2021 by guest. Protected copyright. 6Department of Epidemiology and Biostatistics, Memorial Sloan Kettering Cancer Center, New York, New York, USA 7Departments of Biomedical Engineering and Statistics, Northwestern University, Evanston, Illinois, USA 8Exercise & Sport Science, Faculty of Health Sciences, University of Sydney, Sydney, New South Wales, Australia 9Australian Institute of Sport, Canberra, Australian Capital Territory, Australia 10Department of Mathematics and Statistics, University of New South Wales, Sydney, New South Wales, Australia 11Health, Kinesiology, and Recreation; Department of Physical Therapy and Athletic Training, University of Utah Health, Salt Lake City, Utah, USA 12School of Public Health, University of Adelaide, Adelaide, South Australia, Australia 13Department of Mathematics and Statistics, University of Limerick, Limerick, Ireland Correspondence to Dr Kristin L Sainani, Epidemiology and Population Health, Stanford University, Stanford, California, USA; kcobb@ stanford. edu Figure 1 Flowchart of the article search and inclusion for the systematic review. Sainani KL, et al. Br J Sports Med Month 2020 Vol 0 No 0 1 Education review Br J Sports Med: first published as 10.1136/bjsports-2020-102607 on 19 August 2020. Downloaded from be caused entirely by strong confounding by body mass. The authors should have undertaken a multivariable analysis that accounted for body mass. Errors in statistical reporting—sports medicine/orthopedics/rehabilitation A large study was undertaken to under- stand factors that predict athlete recovery 2 years after an ACL reconstruction.32 The manuscript reports that: ‘Multivari- able regression analyses were constructed to examine which baseline risk factors were independently associated with each Figure 2 Percentage of data- containing articles in quartile one sports science journals that outcome variable…primary outcome include at least one coauthor affiliated with a statistics or other methodologically- oriented variables were all treated as contin- department (from our systematic review, n=299). Statistics includes biostatistics, statistics, data uous’, but the manuscript reports ORs. science and data analytics departments; epidemiology includes authors from departments of ORs are typically reported for binary, community health, population health, health or public health if they are trained as epidemiologists not continuous outcomes. This discrep- or statisticians; computer science includes information technology department. ancy caught the eye of an author in the present commentary, and a series of 33 34 In this commentary, we present two a more appropriate measure of reliability letters to the editor determined that series of case studies that illustrate the and (2) when we extracted data from a highly nuanced, thoughtful and appro- importance of effective collaboration figure 3 of that paper to roughly estimate priate analysis was performed on the data. between sports science and medicine the ICC (2,1), we found a value and 95% However, the modelling approach was researchers, and statisticians. We discuss CI of 0.72 (0.32 to 0.89). This estimate is poorly described—which makes it difficult barriers that have prevented collabora- too imprecise to draw useful conclusions; to judge the validity of the study and also tion. We recommend next steps forward. reliability may plausibly be anywhere hampers reproducibility. In this case, the from insufficient to excellent. In this case, research team included individuals with CASE STUDIES: AVOIDABLE the authors failed to perform an a priori statistical expertise who were involved in STATISTICAL ERRORS sample size calculation, leading to a study study planning and data analysis; however, Statistical errors can occur during study that was too small to adequately answer these individuals may have been insuffi- design, data analysis or reporting. The case the question of interest. ciently involved in drafting the paper. studies described below do not provide an exhaustive list of possible errors. Rather, Errors in data analysis—nutrition and CASE STUDIES: INVENTING NEW we highlight several instances where an endocrinology in exercise STATISTICS http://bjsm.bmj.com/ error may have been avoided with more A study of vitamin D levels and menstrual Introducing new statistical methods into statistical knowledge or greater collabo- status in 77 college-aged women concluded the literature typically involves several ration with statisticians. Other references that ‘Women who did not meet the recom- steps: (1) writing down mathematical provide further examples of common mended level of 30 ng/mL of 25(OH)D equations that explicitly formulate the statistical errors in sports science and had almost five times the odds of having method; (2) establishing the empirical 6–11 medicine. menstrual cycle disorders as women who behaviour of the method through mathe- were above the recommended vitamin D matical proofs, simulations or both; and on September 28, 2021 by guest. Protected copyright. Errors in study design—exercise level’.31 The study has important implica- (3) publishing in a statistics journal or in physiology tions for women athletes, who frequently a general interest journal following peer A study of 14 active men aimed to estab- experience menstrual cycle irregularities. review by statisticians. Given the technical lish the reliability of a biomarker test used A closer inspection of the analysis expertise required, statisticians or mathe- to measure gastrointestinal (GI)