Final Report
Total Page:16
File Type:pdf, Size:1020Kb
Phase-I Study of the E ects of Professional Development and Long-Term Support on Program Implementation and Scaling Up: Final Report Paul R. Brandon, Alice K. H. Taum, Carlos C. Ayala, Donald B. Young, Mary E. Gray, Thomas W. Speitel, Thanh Truc T. Nguyen, and Francis M. Pottenger III Curriculum Research & Development Group College of Education University of Hawai‘i at Mānoa Honolulu, Hawai‘i May 2007 Phase-I Study of the Effects of Professional Development and Long-Term Support on Program Implementation and Scaling Up: Final Report Paul R. Brandon, Alice K. H. Taum, Carlos C. Ayala, Donald B. Young, Mary E. Gray, Thomas W. Speitel, Thanh Truc T. Nguyen, and Francis M. Pottenger III Curriculum Research & Development Group College of Education University of Hawai‘i at Mānoa Honolulu, Hawai‘i May 2007 This is the final report of a project funded by the National Science Foundation (Grant No. REC0228158), Paul R. Brandon, Principal Investigator (PI), Donald B. Young, co-PI, and Alice K. H. Taum, Project Manager. Paul Brandon oversaw the project, with Donald Young’s assistance, and was in charge of developing the teacher questionnaire and the instrument for judging program quality. In addition to managing the project, Alice Taum was in charge of developing the observation coding protocol. Carlos Ayala of Sonoma State University developed the student assessment. Thomas Speitel and Mary Gray oversaw the development of the alternative version of professional development (PD) that is described in the study. Thanh Truc Nguyen contributed significantly to the development of the multimedia DVD-ROM that was developed as part of the alternative PD package. Francis M. Pottenger served as a representative, with Donald Young, of the developers of Foundational Approaches in Science Teaching program. Any opinions, findings, and conclusions or recommendations expressed in this report are those of the authors and do not necessarily reflect the views of the National Science Foundation. An electronic copy of the report, given in one file for the body of the report and another for the appendixes, is available at http://www.hawaii. edu/crdg/programs/pre/scup.html . CONTENTS Tables.......................................................................v Figures..................................................................... vi Acknowledgments ........................................................... vii Executive Summary........................................................... ix Chapter I. Purpose, Rationale, Background, and Theoretical Model ......................1 Purpose...................................................................1 Rationale for the Project .....................................................2 The Duration of PD......................................................3 Background ...............................................................4 Inquiry Learning in Science Education .......................................4 Overview of FAST.......................................................6 The Theoretical Model Underlying the Instrument Development Component of the Project ............................................................13 Program Implementation.................................................15 Teacher Variables ......................................................18 School and Community Variables ..........................................18 Chapter II. Development and Trial of FASTPro, the Alternative Version of FAST Professional Development...................................................19 Comparison of Traditional FAST PD and FASTPro...............................19 Development of FASTPro ..................................................22 Development of FASTStart...............................................22 Development of FASTeR ................................................23 Chapter III. Development of the Inquiry Science Implementation and Outcome Measures ....28 Development of the Inquiry Science Observation Code Sheet .......................28 Rationale and Focus.....................................................29 Collecting Observation Data on Videotape...................................30 ISOCS Development....................................................30 Development of the Inquiry Science Teacher Questionnaire ........................33 Preparing a Draft List of the Variables That Address Teaching Science with Inquiry Methods ...........................................................34 Identifying the Variables That Address the Context Within Which Inquiry Science Is Taught ............................................................34 Item Selection, Item Development, and Cognitive Interviews ....................37 Development of the Inquiry Science Questioning Quality Method ...................39 Development of the Description of the Criteria ...............................40 Preparation of the Facilities, Equipment, Materials, and Procedures ...............40 iii Development of the Inquiry Science Student Assessment ..........................41 Content Knowledge-Type and Science Inquiry Frameworks .....................42 Content Knowledge and Science Inquiry Outcome Measures Development .........44 Student Attitudinal Measures Development ..................................50 Chapter IV. Validity Analyses of Data Collected With the Implementation and Outcome Measures ................................................................55 Inquiry Science Observation Code Sheet Validity Study ...........................55 Data Collection ........................................................55 Validity Analyses.......................................................57 Inquiry Science Teacher Questionnaire Validity Study ............................62 Data Collection ........................................................62 Validity Analyses.......................................................66 Inquiry Science Questioning Quality Method Validity Study ........................75 Data Collection ........................................................75 Validity Analyses.......................................................81 Inquiry Science Student Assessment Validity Study ...............................85 Data Collection ........................................................85 Validity Analyses.......................................................86 References..................................................................90 Appendix A. Summary of Nine Studies Examining the Effects of the Foundational Approaches in Science Teaching Program .....................................103 Appendix B. Inquiry Science Observation Guide ...................................111 Appendix C. Variables Covered on the Inquiry Science Teacher Questionnaire, Inquiry Science Observation Code Sheet, and Inquiry Science Student Assessment and the Items Addressing the Variables .......................................129 Appendix D. Inquiry Science Teacher Questionnaire ................................149 Appendix E. Tables of Results on the Inquiry Science Teacher Questionnaire Items .......167 Appendix F. Inquiry Science Student Assessment ..................................189 iv TABLES I-1. Schedule of Traditional Two-Week FAST Professional Development Institute ..........9 II-1. Professional Development Features That Are Identified in the Literature and How They Are Addressed in Traditional FAST Professional Development and in FASTPro ...20 II-2. How Traditional FAST Professional Development (PD) and FastPro Address the National Science Education Standards (NSES) for Professional Development (PD) in Science Inquiry, Grades 5–8. .......................................................21 II-3. Schedule for FastStart, the One-Week FAST Professional Development Institute ......22 II-4. Numbers of Videos and Slide Shows Developed for FASTeR Student Investigations, by Investigation Phase (Introduction, Procedures, Data Collection and Analysis, and Summary) . 26 II-5. Teachers’ (N = 25) Responses to Five Likert-scale Items about the Usefulness of FASTeR.................................................................27 III-1. Variables Addressed on the Inquiry Science Teacher Questionnaire, the Type of Variable (Adherence, Exposure, or Context) Addressed by the Variables, and Corresponding Item Numbers in Appendix C ....................................................37 III-2. Partial List of Assessment Targets Identified During First Review of FAST 1 Materials by Potential Assessment Method ................................................46 III-3. Student Assessment Content Knowledge Test ..................................47 III-4. The Evidence Explanation Continuum Transformations and Reformulation and the Conceptual, Epistemic and Social Process Domains...............................49 III-5. Motivation Constructs and Sample Items Addressing Them .......................51 III-6. Nature of Science Domains and Sample Items .................................54 IV-1. Population of 100%-Usable Videotaped FAST Physical Science Investigations and the Sample (Shaded) Examined in Validity Studies ..................................56 IV-2. Number of Reconciled Codes for Each Code and Percentage on Which the Two Coders Agreed on Their First Coding ................................................58 IV-3. Comparison of ISTQ Sample with K–12 Teachers Nationwide ....................63 IV-4. Principal Component Analysis Results for the Collaboration Frequency Scale ........68 IV-5. Principal Component