Florida State University Libraries Electronic Theses, Treatises and Dissertations The Graduate School 2014 A Comparison of Three Approaches to Confidence Interval Estimation for Coefficient Omega Jie Xu Follow this and additional works at the FSU Digital Library. For more information, please contact [email protected] FLORIDA STATE UNIVERSITY COLLEGE OF EDUCATION A COMPARISON OF THREE APPROACHES TO CONFIDENCE INTERVAL ESTIMATION FOR COEFFICIENT OMEGA By JIE XU A Thesis submitted to the Department of Educational Psychology and Learning Systems in partial fulfillment of the requirements for the degree of Master of Science Degree Awarded: Fall Semester, 2014 Jie Xu defended this thesis on August 11, 2014. The members of the supervisory committee were: Yanyun Yang Professor Directing Thesis Betsy Becker Committee Member Russell Almond Committee Member The Graduate School has verified and approved the above-named committee members, and certifies that the thesis has been approved in accordance with university requirements. ii TABLE OF CONTENTS List of Tables ...................................................................................................................................v List of Figures ................................................................................................................................ vi Abstract ......................................................................................................................................... vii INTRODUCTION ...........................................................................................................................1 LITERATURE REVIEW ................................................................................................................6 Reliability ....................................................................................................................................6 Reliability Estimation from CTT ................................................................................................6 Test-retest ................................................................................................................................7 Alternative-forms ....................................................................................................................7 Internal Consistency ................................................................................................................8 Coefficient ............................................................................................................................9 Violation of the homogeneity assumption ..........................................................................9 Violation of the essential tau equivalence assumption .....................................................10 Violation of the uncorrelated errors assumption ...............................................................11 Misinterpretation of as an index of homogeneity ..........................................................12 Reliability Estimation Based on CFA within SEM Framework ...............................................12 Confirmatory Factor Analysis ...............................................................................................13 An illustration of the one-factor CFA Model ....................................................................13 The link between CTT and CFA for reliability estimation ...............................................15 Coefficient .........................................................................................................................16 Definition and formula for computing coefficient ........................................................16 Relationship between coefficient and coefficient .......................................................17 Confidence Interval ...................................................................................................................18 Null Hypothesis Significance Testing ...................................................................................18 Interval Estimation ................................................................................................................19 Three Approaches to Interval Estimation for Coefficient .....................................................21 Wald Method .........................................................................................................................21 Wald CI for an individual parameter ................................................................................21 Wald CI for a function of parameters ...............................................................................22 Wald CI for coefficient based on the delta method .......................................................25 Likelihood Method ................................................................................................................26 Likelihood ratio statistic ....................................................................................................26 Likelihood-based CI computed via the likelihood function of a single parameter ...........27 Likelihood-based CI computed via likelihood function of multiple parameters ..............28 Likelihood-based CI for coefficient ..............................................................................30 Bias-corrected and Accelerated Bootstrap Method ...............................................................30 A brief introduction to bootstrap technique ......................................................................31 Construction of bias-corrected and accelerated bootstrap CI ...........................................32 BCa bootstrap CI for coefficient ...................................................................................34 A Comparison among Three Interval Estimation Methods ......................................................34 Statistical Test .......................................................................................................................34 Statistics Applied ..................................................................................................................35 Assumption of Multivariate Normality .................................................................................36 iii Symmetry ..............................................................................................................................36 Sample Size ...........................................................................................................................37 Variance/Invariance to Parameter Transformation ...............................................................37 Previous Research .....................................................................................................................38 The Rationale and Purpose of the Proposed Study ...................................................................40 METHODS ....................................................................................................................................42 Design Factors ...........................................................................................................................43 Data Generation Procedure .......................................................................................................44 Data Analysis ............................................................................................................................46 RESULTS ......................................................................................................................................51 Non-convergence ......................................................................................................................51 Coverage Probability .................................................................................................................51 Interval Width ...........................................................................................................................55 Relative Bias of Point Estimates ...............................................................................................57 DISCUSSION AND CONCLUSIONS .........................................................................................61 Major Findings from Simulation Study ....................................................................................62 Discussion and Suggestions ......................................................................................................64 Limitations and Future Research ..............................................................................................65 APPENDICES .............................................................................................................................68 A. R-CODES FOR DATA GENERATION ..............................................................................68 B. CODES FOR CONFDENCE INTERVAL ESTIMATION IN R .........................................70 C. NONNORMAL DISTRIBUTIONS OF OBSERVED SCORES ..........................................71 D. EFFECTS OF DESIGN FACTORS ON COVERAGE PROBABILITY .............................77 E. EFFECTS OF DESIGN FACTORS ON INTERVAL WIDTH ............................................80 F. MEANS OF CI BOUNDS AND POPULATION COEFFICIENT OMEGA .......................91 REFERENCES ............................................................................................................................101