State-wide Evaluation of the New Hampshire ESEA Title II, Part D Grant Program Final Report New Hampshire Department of Education October 20, 2011 Hezel Associates, LLC 731 James Street, Suite 300 Syracuse, NY 13203 315-422-3512 [email protected] State-wide Evaluation of the New Hampshire ESEA Title II, Part D Grant Program State-wide Evaluation of the New Hampshire ESEA Title II, Part D Grant Program August 30, 2011 New Hampshire Department of Education 101 Pleasant Street Concord, NH 03301-3494 New Hampshire Department of Education Cathy Higgins, Ed.D. State Educational Technology Director Hezel Associates, LLC 731 James Street, Suite 300 Syracuse, NY 13203 Report Preparers: Kirk Knestis, Director of Research and Evaluation, Hezel Associates Naomi Smoke-Zur, Senior Research Analyst, Hezel Associates Rachel Gibson, Research Analyst, Hezel Associates Jessica Yusaitis Pike, Research Analyst, Hezel Associates Cathy Higgins, State Educational Technology Director, New Hampshire Department of Education [Signature Line] Hezel Associates, LLC ii State-wide Evaluation of the New Hampshire ESEA Title II, Part D Grant Program TABLE OF CONTENTS Executive Summary .......................................................................................................... vii I. Introduction ....................................................................................................................15 II. State Ed Tech Program Context ....................................................................................18 A. Summary: State Ed Tech Allocations and Awards ............................................... 18 1. Total amount of FY 2009 funding for the State .............................................. 18 2. The number, percent and amounts of FY 2009 grants awarded competitively and by formula based on the FY 2009 appropriation (the combined total of ARRA and regular funds). .............................................................................. 18 B. Competitive Grant Program Description .............................................................. 18 III. The Activity .................................................................................................................19 A. Features ................................................................................................................. 19 1. ARRA 21st Century Classrooms .................................................................... 19 2. Classroom Technology Mini-Grants ............................................................... 25 3. Tech Leader Cohort (TLC) Program .............................................................. 27 B. Resources Allocated.............................................................................................. 29 C. Scale and Complexity ........................................................................................... 30 IV. The Evaluation .............................................................................................................37 A. Scope ..................................................................................................................... 37 B. Objectives and Questions ...................................................................................... 37 C. Evaluation Methods .............................................................................................. 38 V. Results .....................................................................................................................53 A. ARRA 21st Century Classrooms Grant ................................................................ 55 1. Findings By Research Question ...................................................................... 55 RQ1: How well are school staff members turning classrooms into technology- rich learning environments, fully equipped with hardware, software, and rich digital resources for learning? ......................................................................... 55 RQ2: To what degree are these technology-rich settings encouraging mediating outcomes for students, including interactive learning, higher-level thinking skills, and student engagement? ...................................................................... 80 RQ3: To what degree does the provision of technology tools translate into real opportunities for students to collaborate and connect with new content? ...... 90 RQ4: How are new technologies and resources serving students of various groups, including those with the highest need? .............................................. 95 RQ5: How are grantees doing in terms of training teachers not only how to use technology but also how to translate their new skills into practice in their teaching? ....................................................................................................... 103 Sustainability for Continued Technology Integration......................................... 110 2. Conclusions ................................................................................................... 110 B. Classroom Technology Mini-Grants ................................................................... 113 1. Findings By Research Question .................................................................... 113 RQ1: How well are school staff members turning classrooms into technology- rich learning environments, fully equipped with hardware, software, and rich digital resources for learning? ....................................................................... 113 Hezel Associates, LLC i State-wide Evaluation of the New Hampshire ESEA Title II, Part D Grant Program RQ2: To what degree are these technology-rich settings encouraging mediating outcomes for students, including interactive learning, higher-level thinking skills, and student engagement? .................................................................... 124 RQ3: To what degree does the provision of technology tools translate into real opportunities for students to collaborate and connect with new content? .... 129 RQ4: How are new technologies and resources serving students of various groups, including those with the highest need? ............................................ 131 RQ5: How are grantees doing in terms of training teachers not only how to use technology but also how to translate their new skills into practice in their teaching? ....................................................................................................... 133 Sustainability for Continued Technology Integration......................................... 139 2. Conclusions ................................................................................................... 140 C. Tech Leader Cohort (TLC) Program .................................................................. 141 1. Findings By Research Question .................................................................... 141 RQ1: How well are school staff members turning classrooms into technology- rich learning environments, fully equipped with hardware, software, and rich digital resources for learning? ....................................................................... 141 RQ2: To what degree are these technology-rich settings encouraging mediating outcomes for students, including interactive learning, higher-level thinking skills, and student engagement? .................................................................... 153 RQ3: To what degree does the provision of technology tools translate into real opportunities for students to collaborate and connect with new content? .... 157 RQ4: How are new technologies and resources serving students of various groups, including those with the highest need? ............................................ 158 RQ5: How are grantees doing in terms of training teachers not only how to use technology but also how to translate their new skills into practice in their teaching? ....................................................................................................... 159 Sustainability for Continued Technology Integration......................................... 166 2. Conclusions ................................................................................................... 166 D. Discussion of Exemplary Sites ........................................................................... 168 VI. Recommendations and Lessons Learned ...................................................................172 Appendices .................................................................................................................. A-1 Appendix 1: List of Acronyms and Abbreviations ......................................................... A-2 Appendix 2: Evaluation Timetable ................................................................................. A-4 Appendix 3: Bibliography of References ....................................................................... A-6 Appendix 4: Evaluation Instruments .............................................................................. A-8 Appendix 5: NH Title II-D Logic Model ...................................................................... A-81 Appendix 6: Tables of Findings .................................................................................... A-83 Table and Figure References Table 1. ARRA 21st Century Classrooms Timetable ...............................................17 Table 2. TLC Program/Classroom Technology Mini-Grants Timetable ..................17 Table 3. Scale and Complexity (ARRA/Title II-D