M ulti-Site Public Engagement with Science Synthetic Biology Final Evaluation Report prepared by Rockman et al Research & Evaluation January 2018 Evaluation & Report Credits: Camellia Sanford-Dolly, Lead Researcher Claire Quimby, Data Collection, Analysis, and Reporting Rockman et al Research & Evaluation 201 Mission Street, Suite 1320 San Francisco, CA 94105 Project Credits: This study is supported by the National Science Foundation (Award No. 1421179). Any opinions, findings, conclusions, or recommendations expressed in this report are those of the evaluation team and do not necessarily reflect the views of the National Science Foundation. We are grateful to the internal evaluation team at the Museum of Science Boston for their collaboration and support throughout the project in designing instruments, recruiting participants, providing access to data, and devising methods for dissemination. About Rockman et al: Rockman et al is an innovative research, evaluation, and consulting company that specializes in examining critical issues in formal and informal education. The Rockman team includes evaluators with diverse backgrounds and skill sets who help clients answer critical questions in clear, direct, and honest ways. Rockman et al has served as the lead evaluation firm for numerous projects funded by the National Science Foundation as well as several other public and private funding agencies. Learn more at www.rockman.com. 2 Table of Contents Introduction .............................................................................................................................. 4 The MSPES Synthetic Biology Project ................................................................................................................................ 4 Project Activities and Timeline .......................................................................................................................................... 4 Key Findings .......................................................................................................................................................................... 8 External Evaluation ................................................................................................................... 9 Year 1 Evaluation Findings ................................................................................................................................................. 9 Year 2 Evaluation Methods ..............................................................................................................................................10 Overview of Participants and Building with Biology Events ................................................ 12 A Range of Events .............................................................................................................................................................12 Profile of Host Sites .............................................................................................................................................................13 Profile of Facilitators ..........................................................................................................................................................14 Host Site Spotlights.................................................................................................................. 17 Museum of Life and Science, Durham, North Carolina ..............................................................................................17 The Children’s Museum, Indianapolis and the Dow Science Ambassadors ..........................................................18 Montana State University, Bozeman, Montana ...........................................................................................................20 The Tech Museum of Innovation, San Jose, California ...............................................................................................23 The Buffalo Museum of Science, Buffalo, New York ....................................................................................................24 Oregon Museum of Science and Industry, Portland, Oregon ..................................................................................26 Perot Museum of Nature and Science, Dallas, Texas .................................................................................................28 Outcomes for Host Sites ......................................................................................................... 30 Public Engagement with Science ..................................................................................................................................30 Understanding of Synthetic Biology ...............................................................................................................................34 Collaborating with Scientists............................................................................................................................................37 Building Capacity for Evaluation ....................................................................................................................................42 Outcomes for Facilitators ....................................................................................................... 45 A New Type of Outreach Experience ............................................................................................................................45 Understanding of Synthetic Biology ...............................................................................................................................46 Training in PES .....................................................................................................................................................................48 Communicating with the Public .....................................................................................................................................52 Learning from Scientists, Learning from the Public ......................................................................................................53 Promoting PES on a Nationwide Scale – Lessons Learned .................................................. 57 The Importance of PES and the Role of ISEs .................................................................................................................57 Tackling a Tricky Subject ..................................................................................................................................................58 Providing Support for a Nationwide Effort.....................................................................................................................59 Developing Effective Hands-on Activities .....................................................................................................................62 Developing Effective Forums...........................................................................................................................................63 Project Management and Decision-Making................................................................................................................64 Conclusion and Next Steps ................................................................................................... 66 Ongoing Building with Biology Events ............................................................................................................................66 Sustainability and Dissemination .....................................................................................................................................66 Future Projects and Lasting Influence ............................................................................................................................68 Addendum: Editing Our Evolution ......................................................................................... 69 Introduction ........................................................................................................................................................................69 Supplementary Project Activities ....................................................................................................................................69 Evaluation Questions and Methods ...............................................................................................................................70 General Nature and Success of the Forums.................................................................................................................71 Outcomes for Hosts ...........................................................................................................................................................74 Outcomes for Scientists ....................................................................................................................................................81 Benefits for All .....................................................................................................................................................................88 3 Introduction The MSPES Synthetic Biology Project The Multi-Site Public Engagement with Science – Synthetic Biology project (MSPES) was a three-year effort funded by the National Science Foundation and spearheaded by museum professionals and scientists dedicated to public outreach. The core goal of MSPES was to promote meaningful conversations and interactions