1 Forward teaching: Recommendations for improving statistical in psychological 2 and behavioral sciences

3 Kai Ruggeri12

4 1 Department of Psychology, University of Cambridge, Cambridge, United Kingdom

5 2 Department of Health Policy and Management, Mailman School of Public Health, Columbia 6 University in the City of New York, USA

7 Corresponding Author:

8 Kai Ruggeri

9

10 Email address: [email protected]

1

PeerJ Preprints | https://doi.org/10.7287/peerj.preprints.3265v1 | CC BY 4.0 Open Access | rec: 19 Sep 2017, publ: 19 Sep 2017 11 Abstract 12 To improve outcomes in education - namely the acquisition of statistical literacy - we 13 need a new approach to how the subject is delivered in psychology and behavioral sciences. To do 14 this, key indicators and impacts of barriers to learning statistics can be utilized from existing 15 evidence from recent findings on statistical literacy. Using these, this paper proposes nine elements 16 seen as critical for improving the delivery and associated outcomes for statistics teaching in higher 17 education. Each of the nine elements may be systematically assessed and translated for wider use 18 if effective.

2

PeerJ Preprints | https://doi.org/10.7287/peerj.preprints.3265v1 | CC BY 4.0 Open Access | rec: 19 Sep 2017, publ: 19 Sep 2017 19 Introduction 20 Students do not like the way statistics are taught. This may not come as a surprise to many, but the 21 consequences of this negative experience are significant for students (Chiesi & Primi, 2010) as 22 well as for professors (Uttl & Smibert, 2017) and populations (Calzada Prado & Marzal, 2013). 23 These implications have been highlighted in a recent paper in this journal, which indicates that 24 teaching in quantitative subjects is a strong predictor of lower student evaluations when compared 25 to other topics Uttl & Smibert, 2017). These findings were tied to negative outcomes for academic 26 career progression, which likely has unstudied consequences on the experience level of those 27 responsible for teaching the course. 28 Those who attempt statistical teaching interventions should be commended for recognizing that 29 traditional approaches to teaching statistics may not be as effective or as desirable. The perpetuity 30 of similar findings implies that not everyone acknowledges an issue even exists, though this may 31 also be linked to high turnover in who teaches. While some academics attempt various efforts and 32 even build interventions through a solid theoretical base, this approach has been generally limited. 33 While some techniques may appear obvious, even the cursory scanning of teaching guidance for 34 statistics will reveal much advice has little evidence base and resorts to anecdotes (e.g. ‘In my 35 experience…’), blogs, or untested pieces whose review is limited to the idea rather than empirical 36 evidence – an odd result considering the sources. 37 There is a growing body of evidence for understanding what hinders acquisition of statistical 38 thinking. As those become better identified, academics should be encouraged to attempt 39 scientifically validated improvements to how we engage our classrooms (Kottemann & Salimian, 40 2008), especially when it comes to deep understanding of statistics anxiety (Chew & Dillon, 2014). 41 That educators are now working in earnest to contribute to this presents a positive outlook for a 42 potential future of long-awaited statistically literate populations (Wallman, 1993). Anecdotes, 43 however, have yet to produce a sustained improvement. We need more than this.

44 Take the example of classroom activities used early in the teaching of statistics as a way to improve 45 outcomes (Chiou et al., 2014). This method employs a previously validated one-minute exercise 46 (Stead, 2005) meant to engage students in such a way that statistics anxiety should be reduced and 47 lead to improved test scores. The result: grades increase marginally, a contribution is added to the 48 ‘bag of tricks’ (Gelman & Nolan, 2002), and another anecdote goes into circulation. Without 49 seeking to discredit interventions that show some benefit, the following piece is an argument – 50 even a plea – that if we are truly devoted to gainful change from teaching statistics, it will require 51 more than a minute, clever lecture anecdotes, or new approaches to assessment. 52 We require a new way of thinking about how to deliver the skills, techniques, and mindset 53 necessary for populations that can think statistically. Such abilities enable more people to make 54 better decisions based on information provided, whether in scientific study, government policy, or 55 everyday living (Chew & Dillon, 2014). Unfortunately, the same studies that generate these 56 insights indicate that statistics anxiety is a major barrier to realizing them, largely due to statistics 57 anxiety (Chew & Dillon, 2014). The necessary, substantial change to overcome this begins with 58 the person at the front of the lecture hall. However, this requires implementing evidence-backed 59 approaches to teaching, not simply more assessment of interventions and the students enrolled in 60 them.

3

PeerJ Preprints | https://doi.org/10.7287/peerj.preprints.3265v1 | CC BY 4.0 Open Access | rec: 19 Sep 2017, publ: 19 Sep 2017 61 To respond to this, we propose a way of moving forward with our teaching (or ‘teaching forward’). 62 To do so, this piece offers a brief review of relevant barriers to acquiring statistical literacy, namely 63 statistics anxiety, followed by a set of evidence-based proposals for how to reduce those challenges 64 through better teaching. Though some elements will seem common sense, evidence indicates there 65 are many instances where they are not in place, and it is hoped that this will at least spur meaningful 66 debate toward that end. 67 What we already know 68 Statistics anxiety, its precursors, and its impacts, are well understood. This has largely been 69 through work using the Statistics Anxiety Rating Scale (STARS; Cruise, Cash, & Bolton, 1980). 70 STARS has been the prevailing measure for study, and recent validation work confirms most of 71 its properties continue to hold (DeVaney, 2016). Statistics anxiety is not limited to the social 72 sciences or to English-speaking regions, which has been confirmed by recent work in medical 73 students (Stanisavljevic et al., 2014; Gaudet, et al. 2014) as well as on several continents (Papousek 74 et al. 2012; Liu et al., 2011). To briefly summarize: 75 1. Students in the behavioral, social, and biological sciences are pretty anxious about statistics 76 as a class, assignment, and examination (Field, 2014; Baloglu & Zelhart, 2004). 77 2. Some anxiety is not a bad thing; a lot of anxiety is (Macher et al., 2015). 78 3. Performance in statistics is a subjective measure – we can do better as educators at 79 assessing this (Macher et al., 2013) but should be careful about teaching to the test. 80 4. Teaching is the ultimate moderator: do it right and you are much more likely to get a 81 positive result with a larger number of students; do it wrong and you lose people (Ruggeri 82 et al., 2008). The latter is much easier and more common. 83 5. The people who teach statistics are not always in love with the subject, either (Martins et 84 al., 2012). 85 6. Introducing innovative activities, materials, and teaching styles in the classroom may lead 86 to improved outcomes, but many proposed approaches have not been rigorously evaluated 87 or followed up (Chiou et al., 2014; Gould, 2010; Field, 2009). 88 7. Statistics anxiety is not merely a classroom and performance issue, but one that may impact 89 student well-being, particularly of more vulnerable groups (Jordan, et al., 2014). 90 What can we take from the evidence so far? 91 No first-day activity, no single joke, no vivid anecdote is likely to address all issues faced in 92 ensuring statistical literacy is gained by the greatest number of students. This holds true even if 93 lecturers are able to deliver them. Why is this a problem? If the people who are trained specifically 94 to be analytical (i.e. scientists) struggle to grasp statistical inference, then it is unlikely the general 95 population can easily understand quantitative information presented from any number of sources 96 (Ruggeri et al., 2011). Such arguments are backed up by any number of examples from mainstream 97 media and scientific papers about misunderstanding topics such as risk (e.g. Spiegelhalter, 2013). 98 Instead of a simple response, we need to rethink our approach to how the subject is taught from 99 the outset. One-off activities may offer some impact, but even so, the potential gain from going 100 further than this cannot be missed based on small increases in exam scores. The same may be said 101 of using humor (Field, 2009), guest speakers from professions of interest to students (Ruggeri,

4

PeerJ Preprints | https://doi.org/10.7287/peerj.preprints.3265v1 | CC BY 4.0 Open Access | rec: 19 Sep 2017, publ: 19 Sep 2017 102 20091) and other ‘tricks’ (Gelman & Nolan, 2002). It is unsurprising that these may offer little 103 impact when not utilized within a wider strategy but instead simply called upon for the occasional 104 change of pace. 105 The evidence ultimately indicates that we need systematically assessed approaches to teaching that 106 demonstrably and reliably reduce the legitimate impacts created by mathematical and statistical 107 anxiety. These barriers currently result in low retention of students and knowledge acquisition as 108 well as the avoidance of quantitative information in some more severe cases (Alexander & 109 Onwuegbuzie, 2007). Any new approach should extend beyond simply new forms of presenting 110 the same information for improved test performance, but instead to aim for advances in statistical 111 thinking relevant to modern influences and demands (Cumming, 2014; Gould, 2010). 112 What we need now (the opinion part) 113 In the classroom, there are myriad options for approaching essentially any topic taught in statistics. 114 As such, one could reasonably argue it may be equal parts art and science. This is complicated 115 further given the unlikelihood of engaging all students equally with one style or activity. However, 116 amongst the worst approaches is to present merely theoretical or formulaic knowledge without a 117 deliberate effort to provide context, examples or other applications. Furthermore, superficial 118 approaches to context whereby a passive reference is presented following a complex description 119 of a skill will do little to enhance understanding. 120 Telling a joke or a story at the beginning of the lecture will simply not be enough to make students 121 learn. The effort to reach more students comes long before the classroom, in committed preparation 122 beyond pulling pages from a textbook but rather stories from experience or significant exposure 123 (Chew & Dillon, 2014). In this way, the approach to developing teaching materials is very much 124 a science (Wainer, 2011) and should be evaluated as such. 125 Without question, developing this requires tangible support from the institution as many lecturers 126 are simply too overwhelmed with other obligations to devote time to improved teaching. This is 127 further problematic when balancing between advancing personal statistical skills and teaching 128 material that is only new for students. Beyond that, while there is good evidence in support of 129 humorous approaches, a mixture of styles may be more useful. For example, utilizing entertaining 130 topics prior to referring to serious matters of statistical relevance such as the Challenger crash or 131 social debates about airport screening and human approaches to error. This is certainly not an 132 exhaustive list; the key is to leave an impression through keeping minds engaged (Kottemann & 133 Salimian, 2008; Wainer, 2005), and empirical support for broadly effective approaches is no doubt 134 lacking. Given the low availability, simply continuing to study statistics anxiety further or promote 135 anecdotal teaching improvements risks redundant confirmation of issues matched with negligible 136 improvements in grades. 137 If not abundantly clear from the general tone of this , this in no way implies that statistics 138 is somehow unique from other university-level courses. Nobel laureate Ronald Coase long argued, 139 for example, that economics is simply not taught in such a way to ‘enlighten the public’ on the use 140 of resources, but instead is approached on what academic economists need to know to publish 141 papers in academia. Much of the same could be said about statistics and a wide body of literature

1 Author highlights this is not a peer-reviewed reference. Findings from this study showed no significant difference and thus received no interest from journals.

5

PeerJ Preprints | https://doi.org/10.7287/peerj.preprints.3265v1 | CC BY 4.0 Open Access | rec: 19 Sep 2017, publ: 19 Sep 2017 142 suggests similar thinking in many other fields as well. When we know more about ‘economics 143 anxiety’, expect another paper. 144 Teaching forward 145 To address these challenges, nine approaches are proposed to implement into teaching. Where 146 utilized, they should be studied for effectiveness in the acquisition of statistical literacy beyond 147 grades. While some may have the appearance of common sense, a recent review confirms that 148 many are simply not applied in practice (Chew & Dillon, 2014). 149 1. Focus on inspiring students (who will form part of the general population) through the 150 benefits of statistical understanding and application far ahead of the necessary calculations. 151 If they understand the language, meaning, and importance, they are better prepared to apply 152 it. 153 2. Constantly emphasize applications through examples. If you are teaching it in a university, 154 you should have used it in practice. 155 3. Balance the emphasis on formulae and decision-making at least until understanding of the 156 context is evident. Anyone can refer to a textbook for guidance on how-to; knowing when 157 and appreciating why are much more relevant battles in the classroom. 158 4. Do not start out by teaching statistics within the context of a calculator or software 159 packages. Teach the fundamentals and reduce anxiety through demonstrating the value of 160 statistical knowledge. In other words, get rid of the black boxes. 161 5. Be engaging. There are more approaches than just adding humor that can captivate a 162 classroom with the importance of statistical thinking – students will respond to the most 163 sincere, whether it is with laughter2, tears or personal experience. 164 6. Never let students lose sight of the principle that statistics is a tool to be used in answering 165 questions – research or otherwise – not vice-versa. We do not conduct research to use 166 statistics; we use statistics when necessary to answer questions when we have appropriate 167 data. The statistics are not the boss, but rather they are the tool. 168 7. Even if well-intentioned, do not simply develop a plan to assist those struggling to learn. 169 Implementing the previous six points would build toward a forward shift in classroom (i.e. 170 population) outcomes, offering gains for all students. 171 8. Focus on critical thinking instead of covering more chapters. Teach ‘The New Statistics’ 172 (Cumming, 2014) before jumping to Bayesian cognitive modeling (Lee & Wagenmakers, 173 2014), no matter how much the latter will be more likely to draw a keynote invitation. Such 174 topics should come when students have a firm grasp of the fundamentals relevant to the 175 skill as well as considerable practice. Remember: just because we don’t teach it doesn’t 176 mean they won’t learn it. 177 9. Do not focus merely on teaching how to generate results. Teach how to identify patterns, 178 infer from partial information, interpret ethically, and to illuminate insights. 179 180 Collectively the idea is simple: if the mindset and fluent statistical thinking are delivered, students 181 can go much further on their own than if topics, theory, and calculations are simply thrown at them 182 (Gould, 2010). These points are meant as means to engage all students, not only to cover required

2 Statistics lecturers may want to have someone check those stories before using in class. What we find interesting is not always what students find interesting.

6

PeerJ Preprints | https://doi.org/10.7287/peerj.preprints.3265v1 | CC BY 4.0 Open Access | rec: 19 Sep 2017, publ: 19 Sep 2017 183 topics or attempt to help those struggling. Together, they would provide the ability to understand, 184 to be critical, and perhaps most sustaining, for students to be able to go much further on their own, 185 far beyond their time in the classroom. 186 Delivering this level of statistical literacy would be nothing short of empowering, and current 187 trends in data usage will offer new ways to assess if this has been addressed. For example, could 188 students independently explain why data privacy matters? Could they generate data from scratch 189 to show how online behaviors can be used to derive a large amount of information about 190 individuals? Even without teaching these specific skills, such thinking is a better indication of 191 literacy than simply being able to recite formulae or produce an essay on Item Response Theory 192 algorithms. 193 Ultimately, the purpose of teaching is to inspire: an inspired mind will go far further with 194 knowledge than simply recalling what is presented in a class. No evidence has been presented to 195 have challenged this conclusion and the present argument suggests that such an approach is 196 possible and measurable. It is now time for us as educators to put to practice – both the teaching 197 and the analysis. Should anyone disagree, please note all nine points above are testable hypotheses. 198 Please hold your rejection until appropriate tests have been carried out.

199 Acknowledgements 200 The author would like to thank Sir David Spiegelhalter, Michael Aitken-Deakin, and Andy Field 201 for various comments and suggestions regarding this paper as well as general teaching advice. 202 Additionally, Karl Kelley should be acknowledged for general inspiration on this topic. Students 203 – my own and those of others – who actually fill in the feedback forms at the end of the semester 204 are also thanked on behalf of the author.

205 206

7

PeerJ Preprints | https://doi.org/10.7287/peerj.preprints.3265v1 | CC BY 4.0 Open Access | rec: 19 Sep 2017, publ: 19 Sep 2017 207 References

208 Alexander, E. S., & Onwuegbuzie, A. J. (2007). Academic procrastination and the role of hope as 209 a coping strategy. Personality and Individual Differences, 42(7), 1301-1310.

210 Baloglu, M., & Zelhart, P. F. (2004). Statistics anxiety: a detailed review of the literature. 211 Psychology and Education, 40, 1-15.

212 Calzada Prado, J., & Marzal, M. Á. (2013). Incorporating into 213 programs: Core competencies and contents. Libri, 63(2), 123-134.

214 Chew, P. K., & Dillon, D. B. (2014). Statistics anxiety update refining the construct and 215 recommendations for a new research agenda. Perspectives on Psychological Science, 9(2), 196- 216 208.

217 Chiesi, F., & Primi, C. (2010). Cognitive and Non-Cognitive Factors Related to Students' Statistics 218 Achievement. Research Journal, 9(1), 6-26.

219 Chiou, C-C., Wang, Y-M., & Lee, L-T. (2014) Reducing statistics anxiety and enhancing statistics 220 learning achievement: effectiveness of a one-minute strategy. Psychological Reports, 115, 297- 221 310. DOI: 10.2466/11.04.PR0.115c12z3

222 Cruise, R. J., Cash, R. W., & Bolton, D. L. (1980). Statistical Anxiety Rating Scale Manual. 223 Unpublished manuscript. Andrews University; Berrien Springs, MI.

224 Cumming, G. (2014). The new statistics: Why and how. Psychological science, 25(1), 7-29.

225 DeVaney, T. A. (2016). Confirmatory Factor Analysis of the Statistical Anxiety Rating Scale With 226 Online Graduate Students. Psychological reports, 118(2), 565-586.

227 Field, A. (2009). Can humour make students love statistics? The Psychologist, 22(3), 210-213.

228 Field, A. (2014). Skills in Mathematics and Statistics in Psychology and Tackling Transition. 229 Report for the Higher Education Academy Division of Science, Technology, Engineering and 230 Mathematics.

231 Gaudet, J., Singh, M. D., Epstein, I., Santa Mina, E., & Gula, T. (2014). Learn the game but don't 232 play it: Nurses' perspectives on learning and applying statistics in practice. Nurse education today, 233 34(7), 1080-1086.

234 Gelman, A., & Nolan, D. (2002). Teaching statistics: A bag of tricks. Oxford University Press.

235 Gould, R. (2010). Statistics and the modern student. International Statistical Review, 78(2), 297- 236 315.

237 Jordan, J. A., McGladdery, G., & Dyer, K. (2014). in Higher Education: Implications for 238 Maths Anxiety, Statistics Anxiety and Psychological Well‐being. Dyslexia, 20(3), 225-240.

8

PeerJ Preprints | https://doi.org/10.7287/peerj.preprints.3265v1 | CC BY 4.0 Open Access | rec: 19 Sep 2017, publ: 19 Sep 2017 239 Kottemann, J. E., & Salimian, F. (2008). Engaging students in statistics. Decision Sciences Journal 240 of Innovative Education, 6(2), 247-250.

241 Lee, M. D., & Wagenmakers, E. J. (2014). Bayesian cognitive modeling: A practical course. 242 Cambridge University Press.

243 Liu, S., Onwuegbuzie, A. J., & Meng, L. (2011). Examination of the score reliability and validity 244 of the statistics anxiety rating scale in a Chinese population: Comparisons of statistics anxiety 245 between Chinese college students and their Western counterparts. Journal of Educational Enquiry, 246 11(1), 29-42.

247 Macher, D., Paechter, M., Papousek, I., Ruggeri, K., Freudenthaler, H. H., & Arendasy, M. (2013). 248 Statistics anxiety, state anxiety during an examination, and academic achievement. British Journal 249 of Educational Psychology, 83(4), 535-549.

250 Macher, D., Papousek, I., Ruggeri, K. & Pächter, M., (2015). Statistics anxiety and performance: 251 Blessings in disguise. Frontiers in Psychology: Cognition. DOI: 10.3389/fpsyg.2015.01116

252 Martins, J. A., Nascimento, M. M., & Estrada Roca, M. A. (2012). Looking back over their 253 shoulders: a qualitative analysis of Portuguese teachers' attitudes towards statistics. Statistics 254 Education Research Journal, 11(2), 26-44.

255 Papousek, I., Ruggeri, K., Macher, D., Paechter, M., Heene, M., Weiss, E. M., Schulter, G. & 256 Freudenthaler, H. H. (2012). Psychometric evaluation and experimental validation of the Statistics 257 Anxiety Rating Scale. Journal of Personality Assessment, 94(1).

258 Ruggeri, K., Dempster, M., & Hanna, D. (2011). The Impact of Misunderstanding the Nature of 259 Statistics. Psychology Teaching Review, 17(1), 35-40.

260 Ruggeri, K., Dempster, M., Hanna, D., & Cleary, C. (2008). Experiences and Expectations: The 261 Real Reason Nobody Likes Stats. Psychology Teaching Review, 14(2), 75-83.

262 Ruggeri, K., Dempster, M., & Hanna, D. (2009). An Empirically Based Classroom Intervention 263 for Statistics. Paper presented at the British Psychological Society’s Evidence Based Education 264 Conference. Milton Keynes, England.

265 Spiegelhalter, D. (2013). Are you 45% more likely to die in a UK hospital rather than a US 266 hospital? British Medical Journal, 347. Accessed on 16 February, 2015.

267 Stanisavljevic, D., Trajkovic, G., Marinkovic, J., Bukumiric, Z., Cirkovic, A., & Milic, N. (2014). 268 Assessing Attitudes towards Statistics among Medical Students: Psychometric Properties of the 269 Serbian Version of the Survey of Attitudes Towards Statistics (SATS). PLoS one, 9(11), e112567.

270 Stead, D. R. (2005). A review of the one-minute paper. Active Learning in Higher Education, 6(2), 271 118-131.

272 Uttl, B., & Smibert, D. (2017). Student evaluations of teaching: Teaching quantitative courses can 273 be hazardous to one’s career. PeerJ, 5, e3299.

9

PeerJ Preprints | https://doi.org/10.7287/peerj.preprints.3265v1 | CC BY 4.0 Open Access | rec: 19 Sep 2017, publ: 19 Sep 2017 274 Wainer, H. (2005). Graphic discovery: A trout in the milk and other visual adventures. Princeton 275 University Press.

276 Wainer, H. (2011). Uneducated guesses: Using evidence to uncover misguided education policies. 277 Princeton University Press.

278 Wallman, K. K. (1993). Enhancing statistical literacy: Enriching our society. Journal of the 279 American Statistical Association, 88(421), 1-8.

10

PeerJ Preprints | https://doi.org/10.7287/peerj.preprints.3265v1 | CC BY 4.0 Open Access | rec: 19 Sep 2017, publ: 19 Sep 2017