A Factor Analysis of K-12 Student Feedback Items and the Association
Total Page:16
File Type:pdf, Size:1020Kb
Iowa State University Capstones, Theses and Retrospective Theses and Dissertations Dissertations 1999 A factor analysis of K-12 student feedback items and the association of these items to criterion- referenced tests in reading, language arts, and mathematics Frances Ann Kayona Iowa State University Follow this and additional works at: https://lib.dr.iastate.edu/rtd Part of the Educational Administration and Supervision Commons, Other Education Commons, Reading and Language Commons, and the Science and Mathematics Education Commons Recommended Citation Kayona, Frances Ann, "A factor analysis of K-12 student feedback items and the association of these items to criterion-referenced tests in reading, language arts, and mathematics " (1999). Retrospective Theses and Dissertations. 12575. https://lib.dr.iastate.edu/rtd/12575 This Dissertation is brought to you for free and open access by the Iowa State University Capstones, Theses and Dissertations at Iowa State University Digital Repository. It has been accepted for inclusion in Retrospective Theses and Dissertations by an authorized administrator of Iowa State University Digital Repository. For more information, please contact [email protected]. INFORMATION TO USERS This manuscript has been reproduced from the microfilm master. UMI films the t»ct directly fi'om the original or copy submitted. Thus, some thesis and dissertation copies are in typewriter face, while others may be fi'om any type of computer printer. The quality of this reproduction is dependent upon the quality of the copy submitted. Broken or indistinct print, colored or poor quality illustrations and photographs, print bleedthrough, substandard margins, and improper alignment can adversely affect reproduction. In the unlikely event that the author did not send UME a complete manuscript and there are missing pages, these will be noted. Also, if unauthorized copyright material had to be removed, a note will indicate the deletion. Oversize materials (e.g., maps, drawings, charts) are reproduced by sectioning the original, beginning at the upper left-hand comer and continuing fi'om left to right in equal sections with small overlaps. Each original is also photographed in one exposure and is included in reduced form at the back of the book. Photographs included in the original manuscript have been reproduced xerographically in this copy. Higher quality 6" x 9" black and white photographic prints are available for any photographs or illustrations appearing in this copy for an additional charge. Contact UME directly to order. UMI A Bell & Howell Infonnation Company 300 North Zed) Road, Ann Aibor MI 48106-1346 USA 313/761-4700 800/521-0600 A factor analysis of K-12 student feedback items and the association of these items to criterion-referenced tests in reading, language arts, and mathematics by Frances Ann Kayona A dissertation submitted to the graduate faculty in partial fulfillment of the requirements for the degree of DOCTOR OF PHILOSOPHY Major: Education Major Professor: Richard P. Manatt Iowa State University Ames, Iowa 1999 Copyright © Frances Arm Kayona, 1999. All rights reserved. TJMX Number: 9924729 Copyriglit 1999 by Kayona, Frcinces Ann All rights reserved. UMI Microform 9924729 Copyright 1999, by UMI Company. All rights reserved. This microform edition is protected against unauthorized copying under Title 17, United States Code. UMI 300 North Zeeb Road Ann Arbor, MI 48103 ii Graduate College Iowa State University This is to certify that the doctoral dissertation of Frances Ann Kayona has met the dissertation requirements of Iowa State University Signature was redacted for privacy. Major Professor Signature was redacted for privacy. For the Major Program Signature was redacted for privacy. Lte College iii TABLE OF CONTENTS UST OF TABLES vii CHAPTER I. INTRODUCTION 1 Introduction 1 National Board for Professional Teaching Standards 1 Statement of the Problem 2 Purposes of the Study 3 Objectives of the Study 4 Research Questions 4 Hypotheses to be Tested 6 Assumptions for the Study 6 Definitions of Terms 7 Delimitations of the Study 8 Himian Subjects Approval 9 CHAPTER n. REVIEW OF LITERATURE 10 School Reform: Trends and Issues 11 Three waves of educational reform 12 Reform in Teacher Education 15 Smdent teacher supervision 16 Student Feedback: A Controversial Issue 17 Issues of bias at the K-12 level 18 Concerns of validity at the postsecondary level 19 School Improvement Model Projects 21 Smdent feedback for prospective teachers ' 22 The Case for Self-Evaluation 23 Center for Research on Educational Accountability and Teacher Evalxiation (CREATE) 24 The Multiple-Appraiser Approach 27 Leniency bias 28 Smdent teacher evaluation: Multiple-appraisers 30 Effective Teaching and Smdent Learning 32 Stanford Teacher Assessment Project 33 A fiamework for teaching 33 A call for accoimtability 34 Dimensions or domains of teaching 35 Language arts/reading teaching strategies 36 Mathematics teaching strategies 37 Literature Summary 39 CHAPTER m. METHODOLOGY 47 The Questionnaire 47 Questionnaire development 47 iv Instrument validity 50 Instrument reliability; Cronbach alpha coefficient 51 Item discrimination: Menne-Tolsma formula 51 Securing Participation in the Smdy 52 The Criterion-Referenced Tests 52 The Sample 53 Collection of Data 56 Analysis and Treatment of Data 57 Research Questions 58 Hypotheses to be Tested 60 CHAPTER IV. FINDINGS 62 Descriptive Results 63 Lower Elementary (K-2) Questionnaire 63 Upper Elementary (3-5) Questionnaire 66 Middle School (6-8) Questionnaire 66 Senior High School (9-12) Questionnaire 68 Factor Analysis Results 71 Factor loadings 71 Research Question la (K-2) 72 Research Question 2a (K-2) 74 Research Question lb (3-5) 76 Research C^estion 2b (3-5) 78 Research Question Ic (6-8) 79 Research Question 2c (6-8) 83 Research Question Id (9-12) 86 Research Question 2d (9-12) 87 Correlation Results 91 Research Question 3a (K-2) 91 Research Question 3b (3-5) 93 Research Question 3c (6-8) 95 Research Question 3d (9-12) 97 Item Discrimination/ReliabiliQ^ Results 100 Research Question 4a (K-2) 101 Research Question 4b (3-5) 101 Research Question 4c (6-8: Language arts 106 Research Question 4d (6-8): Mathematics 106 Research Question 4e (9-12): Language arts 106 Research Question 4f (9-12): Mathematics 111 CHAPTER V. SUMMARY, CONCLUSIONS. DISCUSSION, LIMITATIONS, AND RECOMMENDATIONS 116 Summary 116 Methodology summarized 117 Findings summarized: Descriptives 117 Findings summarized: Factor analysis and reliability 118 V Findings smmnarized: Correlations 120 Findings summarized: Reliability and discrimination for piloted items 121 Conclusions 124 Research Question la 125 Research Question lb 125 Research Question Ic 125 Research Question Id 126 Research Question 2a 126 Research Question 2b 127 Research Question 2c 127 Research Question 2d 127 Research Question 3a 127 Research Question 3b 128 Research Question 3c 128 Research Question 3d 128 Research Question 4a 128 Research Question 4b 129 Research Question 4c 129 Research Question 4d 129 Research Question 4e 130 Research Question 4f 130 Limitations 130 Discussion 131 Components of teaching practice 132 K-2 smdent feedback instrument 134 3-5 student feedback instrument 134 6-8 student feedback instrument 135 9-12 smdent feedback instrument 136 Recommendations for Research 137 Recommendations for Practice 138 APPENDDCA. HUMAN SUBJECTS APPROVAL FORM 140 APPENDDC B. GRADE LEVEL K-2 FEEDBACK QUESTIONNAIRE 142 APPENDIX C. GRADE LEVEL 3-5 FEEDBACK QUESTIONNAIRE 145 APPENDIX D. GRADE LEVEL 6-8 FEEDBACK QUESTIONNAIRE 148 APPENDIX E. GRADE LEVEL 9-12 FEEDBACK QUESTIONNAIRE 151 APPENDIX F. LETTERS TO BUILDING ADMINISTRATORS AND TEACHERS 154 APPENDIX G. COVER LETTER TO ALL PARTICIPANTS 157 APPENDIX H. EXPLORATORY FACTOR ANALYSIS RESULTS (K-2) 159 vi APPENDIX I. EXPLORATORY FACTOR ANALYSIS RESULTS (3-5) 161 APPENDIX J. EXPLORATORY FACTOR ANALYSIS RESULTS (6-8) 163 APPENDIX K. EXPLORATORY FACTOR ANALYSIS RESULTS (9-12) 165 APPENDIX L. REVISED K-2 STUDENT FEEDBACK INSTRUMENTS 167 BIBUOGRAPHY 181 ACKNOWLEDGMENTS 188 vii LIST OF TABLES Table 1. A summary of the research literature opposing K-12 student feedback of teacher classroom instruction 41 Table 2. A summary of the research literature supporting K-12 student feedback of teacher classroom instruction 42 Table 3. A summary of the research literature for K-12 student feedback of student teacher classroom instruction 43 Table 4. A simmiary of the research literature for self-evaluation of teacher classroom instruction 44 Table 5. A summary of the research literature for a multiple-appraiser approach to teacher classroom instruction 45 Table 6. A siunmary of the research literature for a multiple-appraiser approach to student teacher classroom instruction 46 Table 7. Aggregate profile of participating teachers and students per grade level 54 Table 8. Aggregate profile of participating teachers and smdents per grade level in mathematics 54 Table 9. Aggregate profile of participating teachers and students per grade level in language arts 54 Table 10. Aggregate profile of participating teachers and smdents per grade level in reading 55 Table 11. Aggregate profile of participating students per grade level 55 Table 12. Aggregate profile of participating students per grade level in three subjects 56 Table 13. Smdent feedback to teachers (lower elementary, K-2) 20-item questioimaire: Ranked mean scores 64 Table 14. Student feedback to teachers (upper elementary, 3-5) 20-item questioimaire: Ranked mean scores 65 Table