Topics in Testing Mediation Models: Power, Confounding, and Bias DISSERTATION Presented in Partial Fulfillment of the Requirements for the Degree Doctor of Philosophy in the Graduate School of The Ohio State University By Robert Arthur Agler Graduate Program in Psychology The Ohio State University 2015 Dissertation Committee: Dr. Paulus De Boeck, Advisor Dr. Robert Cudeck Dr. Andrew Hayes Dr. Duane Wegener Copyrighted by Robert Arthur Agler 2015 Abstract In this dissertation we consider different statistical methodologies to be employed at all stages of testing mediation claims. We begin by examining the relative performance of various methods of testing direct and indirect effects, both in terms of statistical power and the risk of Type I errors. Specifically, we compare a normal-theory approach to testing direct and indirect effects (Sobel, 1982) using either regression or structural equation models with different estimations to bootstrapping techniques (Efron, 2003). We then discuss factor models as an alternative model to mediation models for cases where they make conceptual sense, and as a method of examining worst-case confounding scenarios. We present formulae that discuss their relationships, and investigate the use of structural equation modeling as a way to distinguish between these two models. Finally, we investigate the utility of fungible weights (Waller, 2008) for examining parameter sensitivity in mediation. Fungible weights provide almost equal description of the dependent variable as do the optimal weights, yet may be quite discrepant with the optimal weights and suggest alternative interpretations. We also provide a function to facilitate their use. ii Acknowledgments I do not believe that I can adequately express my gratitude for the opportunities and support I have been given by my friends, family, and colleagues. Specifically, I wish to thank my advisor Dr. Paulus De Boeck for the opportunity to study quantitative psychology, my parents for always believing in me, and my girlfriend for being by my side through this process. I have come further and overcome far more than I could have ever believed before I began my schooling, and it is because of the many chances and words of encouragement that my friends and family have given me. iii Vita 2010................................................................B.S. Psychology, James Madison University 2012................................................................M.A. Social Psychology, The Ohio State ........................................................................University 2010-2011 2010-2011 ......................................................Graduate Fellow, Department of ........................................................................Psychology, The Ohio State University 2011- present 2011- present .................................................Graduate Teaching Associate, Department ........................................................................of Psychology, The Ohio State University Publications Arkin, R.M, & Agler, R.A. (2012). Focus on individual differences: A throwback and a throw down. PsycCRITIQUES, 57(23). Carroll, P. J., Agler, R. A., & Newhart, D. W. (2015). Beyond Cause to Consequence: The Road from Possible to Core Self-Revision. Self and Identity, 14(4), 482-498. Hayes, A. F., & Agler, R. A. (2014). On the standard error of the difference between independent regression coefficients in moderation analysis. Multiple Linear Regression Viewpoints, 40 (2), 16-27. iv Fields of Study Major Field: Psychology v Table of Contents Abstract ............................................................................................................................... ii Acknowledgments.............................................................................................................. iii Vita ..................................................................................................................................... iv List of Tables ................................................................................................................... viii Chapter 1: Introduction ........................................................................................................1 Chapter 2: Relative Performance of Methods of Testing Mediation Effects .....................14 Chapter 3: Factor Model as an Alternative Explanation ....................................................57 Chapter 4: Testing a Factor Model against a Mediation Model ........................................86 Chapter 5: Fungible Weights in Mediation ......................................................................103 Chapter 6: General Discussion.........................................................................................141 References ........................................................................................................................146 Appendix A: Full Results for Chapter 3 ..........................................................................154 Appendix B: Formulas for Converting Correlations to Factor Loadings, One Factor ....156 Appendix C Formulas for Converting Regression Weights to Factor Loadings, vi One Factor ........................................................................................................................157 Appendix D: Formulas for Converting Correlations to Factor Loadings, Two Factors ..159 Appendix E: Fungible Mediation Function .....................................................................161 Appendix F: Fungible Mediation Example......................................................................167 vii List of Tables Table 1. Power for testing the direct effect for all methods, collapsed across all effect size combinations. ...................................................................................................................31 Table 2. Type I error rates when testing the direct effect for all methods, collapsed across all effect size combinations. .............................................................................................32 Table 3. Power for testing the indirect effect for all methods, collapsed across all effect size combinations. ............................................................................................................39 Table 4. Type I error rates when testing the indirect effect for all methods, collapsed across all effect size combinations.. .................................................................................40 Table 5. Sample correlation and regression coefficients based on vector angles and lengths, for four select cases, and vector lengths of 0.8 and 0.5. .....................................79 Table 6. Comparison of model fit statistics for the models we estimate here. EL = equal loadings. CE = correlated errors. LL = lag-lag from the latent variable at t0 to the one at t2. ......................................................................................................................................97 Table 7. Regression results for predictors of fungible interval of the direct and indirect effects in the single mediator case. ..................................................................................118 Table 8. Regression results for the predictors of the fungible interval of the direct and indirect effects. The results for the indirect effect ab2 are not shown, but the results are of viii the same nature for ab2, excepting that the effects are related to , , and rather than , , and . ..............................................................126 Table 9. Results presented in Chapter 3, based on possible factor space angles and vector lengths of .8 for a given combination of mediation results. .............................................154 Table 10. Results presented in Chapter 3, based on possible factor space angles and vector lengths of .5 for a given combination of mediation results...................................155 ix List of Figures Figure 1. ROC curve for the direct effect and N = 50, collapsed across all effect size combinations. Full plot comparing the specificity (1 – observed Type I error rate) and sensitivity (1 – observed Type II error rate) is on the right, and the plot on the left shows a limited range of nominal α levels for comparison. .......................................................28 Figure 2. ROC curve for the direct effect and N = 100, collapsed across all effect size combinations. Full plot comparing specificity (1 – observed Type I error rate) and sensitivity (1 – observed Type II error rate) is on the right, and the plot on the left shows a limited range of nominal α levels for comparison. .......................................................29 Figure 3. ROC curve for the direct effect and N = 200, collapsed across all effect size combinations. Full plot comparing specificity (1 – observed Type I error rate) and sensitivity (1 – observed Type II error rate) is on the right, and the plot on the left shows a limited range of nominal α levels for comparison. .......................................................30 Figure 4. ROC curve for the indirect effect and N = 50, collapsed across all effect size combinations. Full plot comparing specificity (1 – observed Type I error rate) and sensitivity (1 – observed Type II error rate) is on the right, and the plot on the left shows a limited range of nominal α levels for comparison. .......................................................36 xi Figure 5. ROC curve