)ORULGD6WDWH8QLYHUVLW\/LEUDULHV
2020 Reproducible Social Work Research Daniel J. Dunleavy
Unfortunately, the conference was postponed due to the Coronavirus pandemic. The proposed syllabus is presented here for feedback and discussion. If you share or use otherwise use this material, please acknowledge its developers and the RR2020 conference.
Follow this and additional works at DigiNole: FSU's Digital Repository. For more information, please contact [email protected] REPRODUCIBLE SW RESEARCH
About:
This syllabus was created as a discussion piece as part of the Research Reproducibility 2020 Conference (Topic: Educating for Reproducibility: Pathways to Research Integrity) at the University of Florida, Gainesville, FL, USA on March 17, 2020.
Unfortunately, the conference was postponed due to the Coronavirus pandemic. The proposed syllabus is presented here for feedback and discussion. If you share or use otherwise use this material, please acknowledge its developers and the RR2020 conference.
DJD & JRL 03/12/2020
Suggested citation:
Dunleavy, D. J., & Lacasse, J. R. (2020). Reproducible social work research [Course Syllabus]. DOI 10.17605/OSF.IO/PX62B
1
REPRODUCIBLE SW RESEARCH
Research Reproducibility 2020 Educating for Reproducibility: Pathways to Research Integrity University of Florida, Gainesville, FL, USA
Towards Reproducible Social Work Research: A Graduate Course on Reproducibility, Rigor, and Meta-Science
Daniel J. Dunleavy(1), Jeffrey R. Lacasse(2) (1)Florida State University, College of Social Work, [email protected] (2)Florida State University, College of Social Work, [email protected]
ABSTRACT
Introduction. The veracity of scientific findings in the biomedical and social sciences has come under intense scrutiny over the last two decades in what has been called the “replication crisis”. Though a key contributor to the development of social welfare policy and a primary creator/consumer of social welfare research, social workers have been relatively silent (or unaware) about many of these issues. Like other social science disciplines, social work is susceptible to many of the underlying causes of this crisis (e.g. flexible data analysis, Questionable Research Practices, publication bias, etc.). As concerned social work scholars and educators, we believe the discipline requires a renewed focus on research reproducibility and rigor.
Objectives. We aim to: (1) increase the knowledge of graduate social work students on the topics of replicability and optimal research practices, (2) educate social work students about the field of meta-research, and (3) prepare social work students to engage in reproducible research projects, by using an interdisciplinary approach, which integrates scholarly works from social work with adjacent fields (e.g. medicine, epidemiology, social psychology, economics, and statistics).
Methods. Our poster outlines a novel semester-long, graduate-level research course, which is split across eight topic areas: (1) Critical Thinking, Epistemology, and the Philosophy of Science, (2) Cognitive Errors and Biases, (3) The Replication Crisis, (4) Statistical Inference, (5) Questionable Research Practices, (6) The Sociology of Science and Knowledge Dissemination, (7) Meta-Research, and (8) Reproducibility and Research Integrity. Students are exposed to a mixture of articles and texts from across the sciences, including philosophy and statistics. Assignments include: Critiquing published research articles (in the student’s area of interest), creating and utilizing a public repository for future research projects, evaluating and critiquing the preregistration of a published study, and a term paper on a topic of their choosing (building upon a topic area from the course).
Conclusion. This semester-long course provides social work students with a much- needed overview of current issues in scientific research. As educators, we believe this is but one step towards a more comprehensive effort to reform our field, and scientific practice more broadly.
2
REPRODUCIBLE SW RESEARCH
[Version 1.0]
FLORIDA STATE UNIVERSITY COLLEGE OF SOCIAL WORK
Reproducible Social Work Research SOWXXXX-XX YEAR Proposed 3 Credits
Instructors: Daniel Dunleavy Class Time: X/Y 00:00AM-00:00PM Jeffrey Lacasse E-Mail: [email protected] Class Location: XXXX [email protected] Office Hours: X/Y 00:00AM-00:00PM Office Location: XXXX [or by appointment]
COURSE DESCRIPTION Reproducible Social Work Research is a semester-long course intended to complement foundational, graduate coursework on social work research. This course introduces students to topics and issues related to research reproducibility, data management, meta-research, the rigorous and ethical practice of social work research, and other contemporary research topics. The course content prepares students to understand current and historical controversies in biomedical and social science research, with the aim of improving student skills in conducting reproducible social work research. Students learn to critique and appraise contemporary research practices, evaluate their own beliefs about scientific practice, and will become better prepared to engage with open and reproducible research practices. The course will be split across eight topic areas, including: (1) Critical Thinking, Epistemology, and the Philosophy of Science, (2) Cognitive Errors and Biases, (3) The Replication Crisis, (4) Statistical Inference, (5) Questionable Research Practices,(6) The Sociology of Science and Knowledge Dissemination, (7) Meta-Research, and (8) Reproducibility and Research Integrity.
The prerequisites for this course are a basic research methods class, a course in inferential statistics taken at the graduate level, and a course in the philosophy of science or on theories and models in social science research.
FORMAT A variety of instructional approaches will be utilized in this course. In addition to lecture material, students will be expected to involve themselves in class discussions and small group exercises. Additionally, this course will utilize technology to enhance learning. A
3
REPRODUCIBLE SW RESEARCH
course website is available where class documents, Powerpoint slides, etc. will be posted. Additional resources such as sample assignments and writing style guides will also be posted to the course website.
COURSE OBJECTIVES - Upon completion of the course, the student will:
Expected Student Performance Outcomes As Measured By
Papers, Article 1. Increase knowledge and understanding of research Critique, Class replicability and optimal research practices. Discussion/Lecture
Papers, Article 2. Increase knowledge and methods of meta-research, including Critique, Class methods used for detecting Questionable Research Practices Discussion/Lecture (QRPs).
Papers, Article 3. Increase knowledge and understanding of data management. Critique, Class This includes accessing and utilizing an identified research Discussion/Lecture, repository. In-Class Activities
Papers, Article 4. Increase knowledge and understanding the importance of Critique, Class research reproducibility and rigor, within the context of Discussion/Lecture contemporary social work research.
COURSE TEXTBOOK:
Chambers, C. (2017). The 7 deadly sins of psychology: A manifesto for reforming the culture of scientific practice. Princeton, NJ: Princeton University Press
Dienes, Z. (2008). Understanding psychology as a science: An introduction to scientific and statistical inference. London, UK: Palgrave Macmillan.
Readings: A tentative reading list can be found below and in the course schedule. Supplemental reading materials will be uploaded to the courses website.
4
REPRODUCIBLE SW RESEARCH
Assignments & Grading
(1) BRIEF PAPER ON SCIENCE (20%): No more than 10 pages (excluding title page and references)
A. Describe your current views about science and scientists (use citations as appropriate): • What is science? • What are its methods? • Who should be considered a scientist? • What makes science rigorous?
B. What distinction, if any, do you make between scientifically derived information and other types of information derived from such sources as religion, tradition, authority or gossip? Why do you hold the position you do hold?
C. What impact in your opinion do scientific procedures have on current professional social work thinking and practice? Where possible provide an example along with references.
D. Present a clear, concise description of a substantive issue (a problem or issue that is of interest to you). Explain why you consider that problem/issue as important (if possible by appropriate references from the literature) within an area of social work to which you intend to contribute through your scholarship and research. First describe your general area of interest, and then describe your specific interests/problem area/interests within that arena (where appropriate utilize the research literature). Be prepared to present your ideas to the class (10 minutes presenting & 10 minutes for questions). [This section will be used as a starting point for the Term Paper]
(2) PARTICIPATION/ATTENDANCE/PROFESSIONALISM (10%): Attendance and participation are key components to your successful completion of the course. You are expected to attend all classes (unless you have an excused absence) and to participate in classroom activities (i.e. group/classroom discussions; lecture; videos; experiential exercises; etc.). You will be graded on quality and consistency of participation in addition to attendance.
(3) ARTICLE CRITIQUES (40%): There will be 4 opportunities during the semester (see schedule for dates) for students to bring in a published study on a topic of interest. The student should be able to provide a brief oral overview of the study’s background, design/methods, analyses, and conclusions. Additionally, each article critique will focus on a particular aspect of the study (e.g. the design/methods, reporting of statistical analyses), which will be critiqued by the student and discussed as a group. Criticisms and discussion should reflect issues discussed during the semester.
(4) TERM PAPER (30%): No more than 15 pages (excluding title page and references)
A. Describe your specific interests/problem area/interests within that arena. [This will likely be informed by Part D of your Brief Paper on Science]
5
REPRODUCIBLE SW RESEARCH
B. Drawing on the course content, please provide 2 examples of what you deem to be rigorous, reproducible research in this area. • Explain the study and its methods • What makes it, in your opinion a rigorous study? • In what ways is it reproducible? • What other strengths/limitations do you note?
C. Drawing on the course content, please provide 2 examples of what you deem to be poor or irreproducible research in this area. • Explain the study and its methods • What makes it, in your opinion a poorly conducted study? • In what ways is it irreproducible? • What other strengths/limitations do you note?
D. Noting the above, discuss what you deem to be ways your topic area could be more rigorously explored in the future by researchers. Again, here you’ll want to draw on the topics, readings, and discussions we’ve had during the course.
Grades will be distributed as follows:
94 - 100 = A 80 – 83.9 = B- 67 – 69.9 = D+
90 – 93.9 = A- 77 – 79.9 = C+ 64 – 66.9 = D
87 – 89.9 = B+ 74 – 76.9 = C 60 – 63.9 = D-
84 – 86.9 = B 70 – 73.9 = C- 0 – 59.9 = F
6
REPRODUCIBLE SW RESEARCH
COURSE OUTLINE WEEK 1 MM/DD-MM/DD • Topic: Critical Thinking, Epistemology, and the Philosophy of Science • Subtopic: Knowledge in the Human Sciences
• Readings:
Critical Thinking
o Gambrill, E. (2000). The role of critical thinking in evidence based social work. In P. Allen-Meares & C. Garvin (Eds.), The handbook of social work direct practice (pp. 43–63). Thousand Oaks, CA: SAGE.
o Sagan, C. (1987). The burden of skepticism. Skeptical Inquirer, 12, 1-13.
Epistemology/Philosophy of Science
o Dienes, Z. (2008). CHs 1-2.
o Sober, E. (1999). Testability. Proceedings and Addresses of the American Philosophical Association, 73(2), 47-76.
o Hendricks, V. F. (2013). Why knowledge is not democratic. Grasp [online magazine].
o Muthukrishna, M., & Henrich, J. (2019). A problem in theory. Nature Human Behavior, 3(3), 221-229.
• Recommended Readings:
o Meehl, P. E. (1976). Theory-testing in psychology and physics: A methodological paradox. Philosophy of Science, 34(2), 103-115.
• Assignment/Activity:
o Discussion of Readings, Importance of the Course, and Student Views on Science/Research
WEEK 2 MM/DD-MM/DD • Topic: Cognitive Errors and Biases • Subtopic: Echo-Chambers
• Readings:
7
REPRODUCIBLE SW RESEARCH
o Nickerson, R. S. (1998). Confirmation bias: A ubiquitous phenomenon in many guises. Review of General Psychology, 2(2), 175–220.
o Gambrill, E. D. (2012). CHs 12-13 from Propaganda in the Helping Professions. New York, NY: Oxford University Press.
o Hendricks, V. F., & Vestergaard, M. (2019). Alternative facts, misinformation, and fake news. In Reality lost: Markets of attention, misinformation and manipulation (pp. 49-77). SpringerOpen.
• Recommended Readings:
o Gambrill, E. D. (1997). Social work practice: A critical thinker’s guide. New York, NY: Oxford University Press.
o Hendricks, V. F., & Vestergaard, M. (2019). Reality lost: Markets of attention, misinformation and manipulation. SpringerOpen.
• Assignment/Activity:
o Brief Paper Due (MM/DD)
o Discussion of Readings and Real-Word Examples of Bias/Misinformation; Student Presentations for Paper #1
WEEK 3 MM/DD-MM/DD • Topic: The Replication Crisis • Subtopic: Causes of the Crisis
• Readings:
o Chambers, C. (2017). CHs 2-4.
o Ioannidis, J. P. A. (2005). Why most published research findings are false. PLoS Medicine, 2(8), e124.
o Yaffe, J. (2019). From the editor — Do we have a replication crisis in social work research? Journal of Social Work Education, 55(1), 1-4.
• Recommended Readings:
8
REPRODUCIBLE SW RESEARCH
o Button, K. S., Ioannidis, J. P. A., Mokrysz, C., Nosek, B. A., Flint, J., Robinson, E. S. J., & Munafò, M. R. (2013). Power failure: Why small sample size undermines the reliability of neuroscience. Nature Reviews Neuroscience, 14, 365-376.
o Colling, L. J., & Sz cs, D. (2018). Statistical inference and the replication crisis. Review of Philosophy and Psychology. https://doi.org/10.1007/s13164-018-0421-4
o Taubes, G. (1995). Epidemiology faces its limits. Science, 269(5221), 164-169.
• Assignment/Activity:
o Article Critique #1 Due [Emphasis: Theory, Argument, or Discussion section]
o Discussion of Readings and Article Critique
WEEK 4 MM/DD-MM/DD • Topic: The Replication Crisis • Subtopic: Remedies
• Readings:
o Chambers, C. (2017). CH 8.
o Chambers, C. (2013). Registered Reports: A new publishing initiative at Cortex. Cortex, 49, 609-610.
o Wagenmakers, E-J., Wetzels, R., Borsboom, D., van der Maas, H. L. J., & Kievit, R. A. (2012). An agenda for purely confirmatory research. Perspectives on Psychological Science, 7(6), 632-638.
o Nuijten, M. B. (2018). Research on research. A meta-scientific study of problems and solutions in psychological science. s.l.: Gildeprint. Retrieved from: https://research.tilburguniversity.edu/files/26158177/Nuijten_Res earch_30_05_2018.pdf
o McNeeley, S., & Warner, J. J. (2015). Replication in criminology: A necessary practice. European Journal of Criminology, 12(5), 581-597.
• Recommended Readings:
9
REPRODUCIBLE SW RESEARCH
o Chartier, C. R., Kline, M., McCarthy, R. J., Nuijten, M. B., Dunleavy, D. J., & Ledgerwood, A. (2018, December). The cooperative revolution is making psychological science better. APS Observer, 31(10), 23-25.
• Assignment/Activity:
o Discussion of Readings
WEEK 5 MM/DD-MM/DD • Topic: Statistical Inference • Subtopic: Schools of Inference and Foundational Issues
• Readings:
o Dienes, Z. (2008). CHs 3-4.
o Lewin-Koh, N., Taper, M. L., & Lele, S. R. (2004). A brief tour of statistical concepts. In M. L. Taper & S. R. Lee (Eds.) The nature of scientific evidence: Statistical, philosophical, and empirical considerations (3-16). Chicago, IL: The University of Chicago Press.
o Wagenmakers, E-J., Lee, M., Lodeqtckx, T., & Iverson, G. J. (2008). Bayesian versus frequentist inference. In H. Hoijtink, I. Klugkist, & P. A. Boelen (Eds.), Statistics for social and behavioral sciences. Bayesian evaluation of informative hypotheses (pp. 181-207). New York, NY, US: Springer.
• Recommended Readings:
o Dienes, Z. (2008). CH 5.
o Oakes, M. (1986). Statistical inference: A commentary for the social and behavioural sciences. New York: John Wiley.
o Hacking, I. (1976). Logic of statistical inference. London, UK: Cambridge University Press.
o Royall, R. (1997). Statistical evidence: A likelihood paradigm. Boca Raton, FL: Chapman & Hall/CRC.
• Assignment/Activity:
o Discussion of Readings
10
REPRODUCIBLE SW RESEARCH
WEEK 6 MM/DD-MM/DD • Topic: Statistical Inference • Subtopic: Current Issues and Controversies
• Readings:
o Benjamin, D. J., Berger, J. O., Johannesson, M., Nosek, B. A., Wagenmakers, E-J., Berk, R., Bollen, K. A., … Johnson, V. E. (2018). Redefine statistical significance. Nature Human Behavior, 2, 6-10.
o Lakens, D., Adolfi, F. G., Albers, C. J., Anvari, F., Apps, M. A. J., Argamon, S. E., … Zwaan, R. A. (2018). Justify your alpha: A response to “Redefine statistical significance”. Nature Human Behavior, 2, 168-171. https://www.nature.com/articles/s41562- 018-0311-x
o Greenland, S., Senn, S. J., Rothman, K. J., Carlin, J. B., Poole, C., Goodman, S. N. & Altman, D. G. (2016). Statistical tests, p values, confidence intervals, and power: A guide to misinterpretations. European Journal of Epidemiology, 31(4), 337-350.
o Silberzahn, R., Uhlmann, E. L., Martin, D. P., Anselmi, P., Aust, F., Awtrey, E., ... Nosek, B. A. (2018). Many analysts, one dataset: Making transparent how variations in analytical choices affect results. Advances in Methods and Practices in Psychological Science, 1(3), 337-356. https://doi.org/10.1177/2515245917747646
o Wagenmakers, E-J. (2004). A practical solution to the pervasive problem of p-values. Psychonomic Bulletin & Review, 14(5), 779- 804.
• Recommended Readings:
• Mayo, D. G. (2018). Statistical inference as severe testing. New York, NY: Cambridge University Press.
• Meehl, P. E. (1978). Theoretical risks and tabular asterisks: Sir Karl, Sir Ronald, and the slow progress of soft psychology. Journal of Consulting and Clinical Psychology, 46, 806- 834.
o McShane, B. B., Gal, D., Gelman, A., Robert, C., & Tackett, J. L. (2019). Abandon statistical significance. The American Statistician, 73(S1), 235-245.
11
REPRODUCIBLE SW RESEARCH
o Morey, R. D. (2017). When the statistical tail wags the scientific dog. Should we ‘redefine’ statistical significance? Medium [Blog] Retrieved from: https://medium.com/@richarddmorey/when-the- statistical-tailwags-the-scientific-dogd09a9f1a7c63
o Morey, R. D. (2018). Redefining statistical significance: The statistical arguments. Medium [Blog] Retrieved from: https://medium.com/@richarddmorey/redefining- statisticalsignificance-the-statistical-arguments-ae9007bc1f91
o Wasserstein, R. L., Schirm, A. L., & Lazar, N. A. (2019). Moving to a world beyond “p<0.05”. The American Statistician, 73(supp1), 1-19.
o Lakens, D. Improving your statistical inferences. [Recommended course]
• Assignment/Activity:
o Article Critique #2 Due [Emphasis: Methods/Statistics]
o Discussion of Readings and Article Critique
WEEK 7 MM/DD-MM/DD • Topic: Questionable Research Practices • Subtopic: HARKing, P-hacking, and Selective Reporting
• Readings:
o Fiedler, K., & Schwarz, N. (2016). Questionable research practices revisited. Social Psychological and Personality Science, 7(1), 45-52.
Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011). False- positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science, 22(11), 1359-1366.
o Kerr, N. L. (1998). HARKing: Hypothesizing after the results are known. Personality and Social Psychology Review, 2(3), 196- 217.
o Murphy, K. R., & Aguinis, H. (2019). HARKing: How badly can cherry-picking and question trolling produce bias in published results? Journal of Business and Psychology, 34(1), 1-17.
12
REPRODUCIBLE SW RESEARCH
o Wansink, B. (2016, November 21). The grad student who never said “no”. Healthier & Happier [Blog]. Retrieved from: https://web.archive.org/web/20170312041524/http:/www.brianwa nsink.com/phdadvice/the-grad-student-who-never-said-no
• Recommended Readings: N/A
• Assignment/Activity:
o Discussion of Readings and Implications of QRP for Social Work Research/Policies
WEEK 8 MM/DD-MM/DD • Topic: Questionable Research Practices • Subtopic: Methods for Diagnosing QRPs and Potential Solutions
• Readings:
o Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2012). A 21- word solution. Dialogue: The Official Newsletter of the Society for Personality and Social Psychology, 26, 4-7.
o Simonsohn, U., Nelson, L. D., & Simmons, J. P. (2014). P-curve: A key to the file-drawer. Journal of Experimental Psychology: General. 143(2), 534-547.
o Smaldino, P. E., Turner, M. A., & Kallens, P. A. C. (2019). Open science and modified funding lotteries can impede the natural selection of bad science. Royal Society Open Science, 6(7), https://doi.org/10.1098/rsos.190194
• Recommended Readings:
o Schelbe, L., & Thyer, B. A. (2019). Child and Adolescent Social Work Journal editorial policy: Guidelines for authors. Child and Adolescent Social Work Journal, 36, 75-80
• Assignment/Activity:
o Discussion of Readings and:
. Declaration on Research Assessment. https://sfdora.org/
13
REPRODUCIBLE SW RESEARCH
. Schönbrodt, F. D., Maier, M., Heene, M., & Zehetleitner, M. (2015). Voluntary commitment to research transparency. Retrieved from: http://www.researchtransparency.org/ and https://osf.io/4dvkw/
WEEK 9 MM/DD-MM/DD • Topic: The Sociology of Science and Knowledge Dissemination • Subtopic: On The Development and Spread of Misinformation
• Readings:
o Nissen, S. B., Magidson, T., Gross, K., & Bergstrom, C. T. (2016). Publication bias and the canonization of false facts. eLife, 5, e21451.
o O’Connor, C., & Weatherall, J. O. (2018). Scientific polarization. European Journal for Philosophy of Science, 8, 855-875.
o O’Connor, C., & Weatherall, J. O. (2019). False beliefs and the social structure of science: Some models and case studies. Retrieved from: http://cailinoconnor.com/wp- content/uploads/2019/08/Misinformation_Groupthink.pdf
o Hansen, P. G., Hendricks, V. F., & Rensvig, R. K. (2013). Infostorms. Metaphilosophy, 44(3), 1-26.
o Ioannidis, J. P. A. (2012). Why science is not necessarily self- correcting. Perspective on Psychological Science, 7(6), 645-654.
• Recommended Readings: N/A
• Assignment/Activity:
o Article Critique #3 Due [Emphasis: QRPs]
o Discussion of Readings and Article Critique
WEEK 10 MM/DD-MM/DD • Topic: The Sociology of Science and Knowledge Dissemination • Subtopic: Scientific Values and Antidotes to Misinformation
• Readings:
14
REPRODUCIBLE SW RESEARCH
o Merton, R. K. (1973). The normative structure of science. In R. K. Merton (Ed.) The sociology of science: Theoretical and empirical investigations (pp. 267-278). Chicago, IL: The University of Chicago Press.
o Anderson, M. S., Ronning, E. A., DeVries, R., & Martinson, B. C. (2010). Extending the Mertonian norms: Scientists’ subscription to norms of research. Journal of Higher Education, 81(2), 366- 393.
o Nosek, B. A., Spies, J. R., & Motyl, M. (2012). Scientific utopia II: Restructuring incentives and practices to promote truth over publishability. Perspectives on Psychological Science, 7(6), 615- 631.
o Vazire, S. (2017, October 24). Criticizing a scientist’s work isn’t bullying. It’s science. Slate. Retrieved from: https://slate.com/technology/2017/10/criticizing-a-scientists-work- isntbullying.html
• Recommended Readings: N/A
• Assignment/Activity:
o Discussion of Readings
WEEK 11 MM/DD-MM/DD • Topic: Meta-Research • Subtopic: N/A
• Readings:
o Hardwicke, T. E., Serghou, S., Janiaud, P., Danchev, V., Crüwell, S., Goodman, S. N., & Ioannidis, J. P. A. (in press). Calibrating the scientific ecosystem through meta-research. Annual Review of Statistics and Its Application. Retrieved from: https://osf.io/krb58/.
o Ioannidis, J. P. A., Fanelli, D., Dunne, D. D., & Goodman, S. N. (2015). Meta-research: Evaluation and improvement of research methods and practices. PLoS Biology, 13(10), e1002264.
• Recommended Readings: N/A
• Assignment/Activity:
15
REPRODUCIBLE SW RESEARCH
o Discussion of Readings and Value/Types of Meta-Research
WEEK 12 MM/DD-MM/DD • Topic: Meta-Research • Subtopic: Social Work Research
• Readings:
o Dunleavy, D. J. (2020). Appraising contemporary social work research: Meta-research on statistical reporting, statistical power, and evidential value. Unpublished doctoral dissertation.
o Epstein, W. M. (1990). Confirmational response bias among social work journals. Science, Technology and Human Values, 15(1), 9–38.
o Orme, J. G., & Combs-Orme, T. D. (1986). Statistical power and Type II error in social work research. Social Work Research & Abstracts, 22(3), 3-10.
• Recommended Readings:
o Perron, B. E., Victor, B. G., Hodge, D. R., Salas-Wright, C. P., Vaughn, M. G., & Taylor, R. J. (2017). Laying the foundation for scientometric research: A data science approach. Research on Social Work Practice, 27, 802–812
• Assignment/Activity:
o Article Critique #4 Due [Student Choice]
o Discussion of Readings and Article Critique
WEEK 13 MM/DD-MM/DD • Topic: Reproducibility and Research Integrity • Subtopic: Resources for Reproducible Research
• Readings:
o Klein, O., Hardwicke, T. E., Aust, F., Breuer, J., Danielsson, H., Mohr, A. H., Ijzerman, H., … Frank, M. C.. (2018). A practical guide for transparency in psychological science. Collabra: Psychology, 4(1): 20. DOI: https://doi.org/10.1525/collabra.158
16
REPRODUCIBLE SW RESEARCH
o Krypotos, A-M., Klugkist, I., Mertens, G., & Engelhard, I. M. (2019). A step-by-step guide on preregistration and effective data sharing for psychopathology research. Journal of Abnormal Psychology, 128(6), 517-527.
o http://www.shinyapps.org/apps/TransparencyChecklist/
• Recommended Readings: N/A
• Assignment/Activity:
o Discussion of Readings and Creating an Open Science Framework (OSF) Profile; Discuss Plans for Uploading and Detailing a Project for Class Peer-Review.
WEEK 14 MM/DD-MM/DD • Topic: Reproducibility and Research Integrity • Subtopic: Producing Useful and Reproducible Social Work Research
• Readings:
o Howard, M. H., & Garland, E. L. (2015). Social work research: 2044. Journal of the Society for Social Work and Research, 6(2), 173-200.
• Recommended Readings:
o Forscher, P. S., Aczel, B., Chartier, C. R., Musser, E. D., Horstmann, K. T., Grahe, J. E., … Flake, J. K. (2019, March 20). Psychological Science Accelerator data management bylaws. https://doi.org/10.31234/osf.io/buqyc
• Assignment/Activity:
o Discussion of Readings and In-Class Review of OSF Project Pages. : Week 15 MM/DD-MM/DD • Topic: On The Duty of Social Workers to Conduct Rigorous, Open Research • Subtopic: Research Ethics, Open Access
• Readings:
17
REPRODUCIBLE SW RESEARCH
o Dunleavy, D. J. (2020, February). Coronavirus as impetus for a lasting change in research culture. SocArXiv: https://osf.io/preprints/socarxiv/2ryt3/
o Vazire, S. (2017). Quality uncertainty erodes trust in science. Collabra: Psychology, 3(1), DOI: https://doi.org/10.1525/collabra.74
o Altman, D. G. (1980). Misuse of statistics is unethical. British Medical Journal, 281, 1182-1184.
o Altman, D. G. (1982). Statistics in medical journals. Statistics in Medicine, 1(1), 59-71.
• Recommended Readings:
o Elliott, K. C., & McKaughan, D. J. (2014). Nonepistemic values and the multiple goals of science. Philosophy of Science, 81, 1- 21.
o Tripodi, T. (1974). Uses and abuses of social research in social work. New York, NY: Columbia University Press.
o Ziliak, S. T., & McCloskey, D. N. (2008). The cult of statistical significance: How the standard error costs us jobs, justice, and lives. Ann Arbor, MI: University of Michigan.
• Assignment/Activity:
o Discussion of Readings and Guided Reflection Upon Course Content, Purpose, and Future Student Research, Etc.
o Term Paper Due (MM/DD) Week 16 MM/DD-MM/DD • No Class (University Exam Week) GOOD LUCK!
18