COMMUNITIES OF TRANSFORMATION AND THEIR WORK SCALING STEM REFORM ADRIANNA KEZAR SEAN GEHRKE PULLIAS CENTER FOR HIGHER EDUCATION ROSSIER SCHOOL OF EDUCATION UNIVERSITY OF SOUTHERN CALIFORNIA Th anks to the leaders of four communities of practice (CoPs) - Project Kaleidoscope (PKAL), SENCER, BioQUEST, and the POGIL Project for allowing us to participate and learn from their communities. Pullias Center for Higher Education Rossier School of Education University of Southern California December 2015 Th is monograph is based upon work supported by the National Science Foundation under Grant No. (NSF DUE-1226242). Any opinions, fi ndings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily refl ect the views of the National Science Foundation. Table of Contents I. Background and Overview of Project 1 Th e Four Communities of Transformation II. Framing: Social Networks and Communities of Practice 6 Benefi ts of Networks Design of Networks Defi ning Characteristics of Communities of Practice Designing Communities of Practice Lifecycle of Communities of Practice: Formation and Sustaining Localized and Distributed Communities of Practice Summary III. Overview of Methods 13 Data Collection Data Analysis Trustworthiness and Validity Summary IV. What are These Entities? Communities of Transformation 16 STEM Reform Communities in Relation to the Communities of Practice Literature Communities of Transformation Summary V. Outcomes and Benefi ts from Participation 22 Individual Benefi ts Organizational Outcomes Multiple Benefi ts of Ongoing Involvement Who Benefi ts? Gains for Women and Faculty of Color Team Involvement for Organizational Change and Leadership Development Additional Strategies for Broader Impact/Outcomes Summary VI. Design for Engagement & Outcomes 31 Communities of Transformation as Unique and Important Professional Development Opportunities Designing for Engagement Philosophy as Design Personal Interactions: Peer-to-Peer Learning, Brainstorming with Others, and Mentoring Trends in Quantitative Analyses Related to Design Reinforcing the Importance of Philosophy and Interactions Deeper Engagement Matters for STEM Reform Community Engagement and Design Matters more than Institutional, Professional, and Personal Characteristics Organizational Outcomes Related to STEM Reform Deeper Engagement Matters for STEM Reform Collective Eff ort for STEM Reform Community Design for Departmental and Institutional Change Cultures of Personal Support Summary VII. Formation and Lifecycle of Communities of Transformation 45 Potential Phase Coalescing Maturing Stewardship Transformation Summary VIII. Common Challenges in Evolution 54 Funding Shifting Focus Community Leadership Too Much Identifi ed with an Individual Leader Project-Focused versus Community-Focused Decisions Staleness Legitimacy Th e Dominant Culture of Science Education Maintaining Community Integrity Focus on General Faculty Improvement versus a Specifi c Pedagogical Approach Increasing and Changing Demands on Faculty Summary IX. Expansion Strategies: More on the Maturing Phase 61 Disciplinary Focus Institutional Focus Sector-wide Focus Constituent-based Focus National Focus International Focus Summary X. Sustainability Model: Understanding the Stewardship Phase 69 Leadership Development, Distribution, and Succession Planning A Viable Financial Model A Professional Staff Formal Feedback and Advice Mechanism Assessment/Research A Community-Derived Strategy that is Articulated, yet Evolving Summary XI. Future Considerations for STEM Reform Communities of Transformation 74 Capitalize More on Disciplinary Work Explore Complementary Online/Virtual Ways to Foster Community Focus More on Network Development Work with Graduate Students/Align with Graduate Initiatives Work with Centers for Teaching and Learning on Campus Work with Postdoctoral Organizations Expand Consultation Work Create On-Campus Learning Communities/Communities of Transformation Supporting Innovators on their Home Campuses Consider Approaches that Deeply Embed STEM Reform Capitalize on Other Reform Initiatives Consider Becoming a Membership Organization Consider a Networked Improvement Community Additional Mechanisms to Support Systemic Change XII. Areas for Further Research 78 Th e Continuum from Community of Practice to Social Networks Communities of Transformation Social Networks within Communities of Transformation Non-organizationally Located Communities of Practice Broader Impacts Other Approaches to Examining Outcomes Comparison of Diff erent Models of STEM Reform Eff orts Th e Tension between Stability/Sustainability and Appeal of Informal Community Th e Challenges of Expansion Th e Eff ect of the Changing Faculty on STEM Reform XIII. Conclusion 82 XIV. Bibliography 83 Executive Summary Th is mixed-methods study examined four STEM communities (BioQUEST, Project Kaleidoscope, the POGIL Project, and SENCER) in order to better understand the roles of these communities in advancing the goals of scal- ing STEM education reform. Th e project explored three key questions: • How do members and leaders of communities of practice (CoPs1) perceive CoP design (membership, structure, communication, activities, and organization to support new knowledge development and ac- tion) shapes the ability to achieve goals (around undergraduate STEM pedagogical change and diff u- sion)? • What are the perceived benefi ts of participation in a STEM reform community of practice or network for the individual participants and for their campuses? • How do communities of practice and networks form, and how are they sustained in ways that help them to achieve their goals? Th e following are the key fi ndings related to these questions: The study identifi ed a novel approach to improving STEM education, which we have called communities of transformation. Th is study identifi ed a unique variant of communities of practice, called communities of transformation (CoTs) that are present in the STEM reform area. Th e defi ning feature of these newly identifi ed entities is their focus on explor- ing philosophically, in deep and fundamental ways, how science is taught. Th is can lead to more substantive changes that have the potential to address the problems described in national reports around underrepresentation of women and underserved minorities, persistence rates, and success among students. Th ese communities of transformation create innovative spaces that have the potential to shift institutional and disciplinary norms. We identify how these diff er from more traditional professional development models, including campus-based professional development and disciplinary meetings. Communities of transformation address both individual faculty and broader systemic change. Much of the early work to improve STEM education has focused on altering individual faculty behavior through faculty development and dissemination of best practices. Some more recent eff orts (e.g., Association of American Universities’ STEM reform initiative) focus on changing broader systemic and institutional norms. Communities of transformation provide support for individual faculty change, but they also simultaneously work (to varying degrees) to shift departmental cultures, institutional norms, and disciplinary values. In this report, we describe work that these communities conducted to alter the conversation around teaching within disciplines, as well as evidence of broader impact achieved through service on national committees and task forces aimed at improving STEM education. Th eir unique work to address both the individual faculty and the broader system is a compelling strategy for change. We found that the strength of these eff orts lies in working from the ground up, with individual faculty buy-in, motiva- tion, and support for improving practice. Additionally, in general, institutional type, discipline, and rank/appointment status were not signifi cantly associated with the outcomes we measured, when accounting for our other engagement, design, and motivation variables. We think this points to the potential for CoTs resembling those in our study to contribute to overcoming typical barriers to reform such as reward structures, disciplinary cultures, and a lack of institutional leadership. 1 A community of practice is a group of people who share a concern or a passion for something they do and learn how to do it as they interact regularly (Allee, 2000; Lave, 1988; Wenger, 1998 and 2007). i | Communities of Transformation Benefi ts of these communities accrue to both individual faculty and to their institutions. Participants reported that the greatest benefi ts of involvement in these communities came in the form of learning and improving in their teaching, reenergizing them in their sense of satisfaction and fulfi llment in their work, and gaining credibility for their work related to STEM reform. Additionally, nearly 35% of participants indicated that engagement in these communities contributed to changes related to STEM reform in their departments, while more than one in fi ve participants indicated that some sort of institutional change had come about as a result of involvement in these communities. We also identifi ed how involvement of several individuals from a single institu- tion increased departmental and institutional benefi ts. In fact, one of the largest eff ect sizes we observed to predict departmental and institutional benefi ts came from having more peers from the same institution involved