Resources for Mini Meta-Analysis and Related Things Updated on June 25, 2018

Here are some resources that I and others find useful. I know my mini meta SPPC article and Excel were not exhaustive and I didn’t cover many things, so I hope this resource page will give you some answers. I’ll update this page whenever I find something new or receive suggestions.

Meta-Analysis Guides & Workshops

• David Wilson, who wrote Practical Meta-Analysis, has a website containing PowerPoints and macros (for SPSS, , and SAS) as well as some useful links on meta-analysis o I created an Excel template based on his slides (read the Excel page to see how this template differs from my previous SPPC template) • McShane and Böckenholt (2017) wrote an article in the Journal of Consumer Research similar to my mini meta paper. It comes with a very cool website that calculates meta- analysis for you: https://blakemcshane.shinyapps.io/spmetacase1/ • Courtney Soderberg from the Center for Open Science gave a mini meta-analysis workshop at SIPS 2017. All materials are publicly available: https://osf.io/nmdtq/ • Adam Pegler wrote a tutorial blog on using to calculate mini meta based on my article • The founders of MetaLab at Stanford made video tutorials on meta-analysis in general • Geoff Cumming has a video tutorial on meta-analysis and other “new ” or read his Psychological Science article on the “new statistics” • Patrick Forscher’s meta-analysis syllabus and class materials are available online • A blog post containing 5 tips to understand and carefully interpret meta-analytic data • Emily Kothe has meta-analysis templates using R metaphor package: osf.io/6bk7b/

Specialized Software/ Web App for Doing Meta-Analysis

• R metafor package: http://www.metafor-project.org/doku.php o There is a corresponding great website with very useful FAQ • R psychmeta package: https://cran.r-project.org/web/packages/psychmeta/index.html • JASP: https://jasp-stats.org/download/ • Meta-Analysis via Shiny (MAVIS): http://kylehamilton.net/shiny/MAVIS/ • JAMOVI with meta-analysis add-on: https://github.com/kylehamilton/MAJOR • Comprehensive Meta-Analysis (expensive): https://www.meta-analysis.com/ • metaBUS: http://metabus.org/ • MetaLab: http://metalab.stanford.edu/

Some Books on Meta-Analysis

• Borenstein et al. (2009): Introduction to Meta-Analysis • Cooper et al. (2009): Handbook of Research Synthesis and Meta-Analysis • Lipsey & Wilson (2000): Practical Meta-Analysis • Rosenthal (1991): Meta-Analytic Procedures for Social Research

Syllabi with Recommended Readings

• A collaborative syllabus and resources in this Google Doc • Katie Corker’s syllabus • Gilad Feldman’s syllabus • Patrick Forscher’s syllabus and class materials • Judith Hall’s syllabus • John Sakaluk’s syllabus

Effect Size, , & Visualization

• Shiny web app for calculating effect size, power, sample size, AND confidence interval • Website for converting effect sizes from various sources • I personally use Daniël Lakens’ spreadsheet to calculate effect size: https://osf.io/ixgcd/ o Read Daniël Lakens’ effect size primer in Frontiers in Psychology o Katherine Wood made a Shiny online app of this effect size spreadsheet • Effect size and confidence interval Shiny web app calculator • Effect size calculator that includes ICC and MLM • Rosnow & Rosenthal (2009)’s effect size primer that I personally find easy to read • For confidence interval, see Daniël Lakens’ website

• Neyeloff, Fuchs, & Moreira (2012) wrote an article on using Excel to calculate meta-analysis and to create Forest plots. The Excel is in the supplementary section at the end of page.

Power Analysis

• Guide on how to do power analysis for meta-analysis • Website for various forms of power analysis • Website for power analysis in mediations • PANGEA power analysis for general ANOVA designs

Repeated, Within, or Mixed Design

• Bonett (2008): https://www.ncbi.nlm.nih.gov/pubmed/18557680 • Dunlap et al. (1996): http://psycnet.apa.org/journals/met/1/2/170/ • Morris & DeShon (2002): http://psycnet.apa.org/journals/met/7/1/105/ • See Jake Westfall’s blog on effect size in within design

Other Forms of Meta-Analysis

• Equivalence testing meta-analysis • Multilevel meta-analysis • Structural Equation Modeling meta-analysis • Instead of doing a meta-analysis, you can also do a multilevel model on all your data. Basically, you would nest your raw data by study number. MLM allows you to test for moderation and mediation, so it can be more useful than doing a traditional meta-analysis.

Random Effects vs Fixed Effects in Mini Meta-Analyses

• Jackson & Turner (2017) in Research Synthesis Methods • Bender et al. (2018) in Research Synthesis Methods • A short blog post about the differences between random and fixed models

Issues with Meta-Analysis Generally & Some Recommendations

• Katie Corker has slides that summarized some issues: osf.io/7bdks/ and osf.io/gt4zp/ • John Ioannidis wrote a short summary • Ueno, Fastrich, & Murayama (2016) wrote about misuse and Type I error • Lakens, Hilgard, & Staaks (2016) wrote an article with recommendations • PRISMA guideline for transparent reporting of meta-analysis

Social Media In my experience, I find that researchers are very willing to help others on social media.

• Facebook groups: PsychMAP or Psychological Methods Discussion Group • Twitter: Some of the people I mentioned above are very active on Twitter, and here is a short list of some brilliant people I follow to get advice and news in statistics and research methods (most of them have expertise in meta-analysis). I came across many of these resources on Twitter and this resource page is a group effort (#OpenScience)! In no particular order:

Adam Pegler: @pegleraj Alex Danvers: @Alex_Danvers Alexander Etz: @AlxEtz Calvin Lai: @CalvinKLai Christina Bergmann: @chbergma Cog Tales (MetaLab founders): @CogTalesTweet Courtney Soderberg: @cksoderberg Daniël Lakens: @lakens Elizabeth Page-Gould: @page_gould Emily Kothe: @emilyandthelime Gilad Feldman: @mgto_org Heather Urry: @HeatherUrry Jake Westfall: @CookieSci JK Flake: @JkayFlake Joe Hilgard: @JoeHilgard John Sakaluk: @JohnSakaluk

Katie Corker: @katiecorker metaBUS: @MetaBUSproject Michael Frank: @mcxfrank Mike Cheung: @mikewlcheung Patrick Forscher: @psforscher Sanjay Srivastava:@hardsci Simine Vazire: @siminevazire Tal Yarkoni: @talyarkoni Tessa West: @TessaWestNYU The 100% CI: @the100ci Vivian Ta: @VivianTa22 W. Kyle Hamilton: @kylehamilton Wolfgang Viechtbauer: @wviechtb

If you have more questions, you can contact me via: Email: [email protected] Twitter: @jinxgoh