Tornado Alley Film Summative Evaluation Report September 15, 2014 Tornado Alley Film Summative Evaluation Report Submitted to: Deborah Raksany Giant Screen Films 500 Davis Street Suite 1005 Evanston, IL 60201 Submitted by: Elizabeth Goldman Kim Streitburger RMC Research Corporation 1000 Market Street, Bldg. #2 Portsmouth, NH 03801 September 15, 2014 CONTENTS Executive Summary ................................................................................................................1 Evaluation Methodology ........................................................................................................6 Section I. Tornado Alley Film Evaluation ...............................................................................8 Methodology ..............................................................................................................8 Survey Respondents...................................................................................................9 Survey Findings ..........................................................................................................10 Focus Group Participants ...........................................................................................22 Focus Group Findings .................................................................................................23 Survey Summary ........................................................................................................34 Focus Group Summary ...............................................................................................36 Section II. Educator Guide Evaluation ....................................................................................38 Introduction ...............................................................................................................38 Survey Findings ..........................................................................................................38 Summary ....................................................................................................................44 Section III. Professional Development Workshop and Grant-Funded ...................................46 Community Projects Introduction ...............................................................................................................46 Survey Findings ..........................................................................................................47 Grant-Funded Community Projects ...........................................................................51 Summary ....................................................................................................................60 Section IV. Educational Outreach and Vehicle Tour Evaluation ............................................64 Methodology ..............................................................................................................65 Institutional Outreach ................................................................................................65 Remote and Underserved Outreach ..........................................................................74 Summary ....................................................................................................................85 Institutional Outreach ................................................................................................85 Remote and Underserved Outreach ..........................................................................86 i CONTENTS Appendices Appendix A. Tornado Alley PreViewing Survey Appendix B. Tornado Alley PostViewing Survey Appendix C. TA Focus Group Survey Appendix D. Tornado Alley Focus Group Questions Appendix E. Tornado-Educators Guide Appendix F. Educator Survey Appendix G. Franklin Institute Evaluation Form Appendix H. Franklin Institute Facilitator Questions Appendix I. Local Project Planning Application Appendix J. Community Project Interview Appendix K. Tornado Outreach Log Appendix L. Tornado Outreach Museum Host Survey Appendix M. Tornado Outreach Site Survey Appendix N. Outreach Staff Quests Appendix O. Outreach in Joplin Missouri ii Executive Summary Tornados can occur anywhere in the U.S., but most occur in “tornado alley”—a broad swath of the country’s heartland that stretches from Georgia in the East to the Rocky Mountains in the West. And despite decades of research, tornados remain scientific mysteries. The VORTEX2 research project, funded by NSF and NOAA, is the most ambitious attempt yet to understand tornados—their origins, structure, and evolution. Directed by Sean Casey of TV tornado chasing fame and featuring the scientists and missions of VORTEX2, the Tornado Alley film and the surrounding project elements were designed to “bring science to life”, engaging audiences in the excitement of scientific research and the stupendous technologies involved in investigating these awe- inspiring and terrible weather phenomena, and model science as a “cool” career for young people. The Tornado Alley project contained five elements: • The 3-D, large format Tornado Alley film; • A Tornado Alley Educator’s Guide, website, and resources; • Professional development workshops utilizing cybertechnology to allow museum-based informal science educators and formal educators interact remotely with VORTEX2 researchers and experience visualizations of weather data, and to subsequently conduct related educational programs in their own communities; • Outreach using the tornado intercept vehicles (TIV) and Doppler on Wheels (DOW) to bring scientists and weather-monitoring technology into direct contact with audiences at museums and science centers; and • Outreach using scientists, the TIV and DOW and a traveling 3D projection system to present elements of the Tornado Alley film and featured weather research technology to tornado- affected and underserved communities. Each element of the project was evaluated independently using a mix of quantitative and qualitative data gathered through surveys, interviews, and focus groups. Findings are presented here in terms of project goals; individual descriptions of each Tornado Alley element appear in following sections. Within the project’s overarching goal of communicating weather science research and information to public and professional audiences, eight specific goals were enunciated: • to increase audience members’ factual knowledge of science • to increase audience members’ understanding of the scientific process and the work scientists do • to increase audience members’ knowledge of technologies used in weather science • to increase audience members’ interest in participating in scientific endeavors • to increase student interest in weather science and careers • to increase teacher confidence in teaching weather science and related STEM topics • to increase scientists’ confidence and interest in sharing their work with wider audiences • to incorporate project resources in educational curricula or programming The Tornado Alley project addressed the overarching goal through all five elements, most directly through the film itself and the outreach efforts. Introduced in 2011, the film was seen by more than 5 million viewers as of the date of this report, and is anticipated to reach over 7 million during its theatrical life. Educator materials, including an educator guide, posters, and a website, have reached thousands. (Nearly 600,000 visitors have accessed the website to date.) The museum-based outreach efforts significantly supported the film in the majority of US venues: 67 museums or science centers, or over 80% of exhibitors, hosted the TIV and/or the DOW vehicles, and over 200 unique events were held at the 79 institutional theaters. A very conservative estimate for the audience directly reached through these activities (those who physically toured a vehicle, had a conversation with a scientist, etc.) is 150,000 across the US, although at least 250,000 likely experienced at least a passive interaction. Outreach to tornado-affected communities included activities at 7 sites in 2012, considered a pilot phase, and at 20 sites in 2013. Venues ranged from small museums without theaters to community centers and school gymnasiums to challenging "untraditional" sites—such as a warehouse space located adjacent to trailers being used as temporary classrooms in a town that had been completely leveled by a recent tornado, or a special section of a Lowe's Home improvement store which hosted the screenings indoors and provided a community resource fair featuring researchers, first responders, etc. outdoors in its parking lot. The program’s second phase in 2013 emphasized outreach to economically underserved areas—both urban and rural. Programs were held in areas of Missouri, with high minority populations, such as Ferguson, MO, as well as Indian Reservations in South Dakota, located in the nation’s poorest county (Buffalo County, SD). The professional development program enabled the development of at least fourteen new models for informal science practitioners who seek to engage audiences (teachers, students, or families) with tornado science, the nature of scientific research, or data analysis content. Factual Knowledge of Science Surveys from potential and actual viewers and focus groups with viewers of the Tornado Alley film highlight some key knowledge gains reported by audience members. Primary among them were a greater understanding of the scientific process and more, and new knowledge about how tornados form. Following the film