build IT Scaling and Sustaining an Afterschool Computer Science Program for Girls SPECIAL FOCUS ON STEM LEARNING by Melissa Koch, Torie Gorges, and William R. Penuel

“I want to be a software engineer because I want to County (GIAC) designed the materials and professional development to teach technology and computer science be involved with computers.” –Build IT participant skills not only to girls but also to afterschool facilitators— “I would like to create software because I would make who are primarily young women—while building facili-

a lot of money, and people in these jobs are intelligent.” MELISSA KOCH is senior educational developer at SRI Interna- tional’s Center for Technology in Learning and the director of Build –Build IT participant IT and InnovaTE3, two afterschool programs funded by the National Science Foundation that encourage girls to pursue STEM careers. She “I have been so inspired working with this curriculum designs and implements educational learning environments for edu- cators and youth in and out of school. She specializes in developing and with the whole Build IT team that I have applied to a engineering, computer science, and technology-based curricula. graduate program...in learning, media and technology.” TORIE GORGES is a research social scientist at SRI International’s Center for Technology in Learning. She evaluates educational pro- –Build IT facilitator grams ranging from technology-rich early childhood and afterschool curricula to high school reform initiatives. She led the evaluation of The program that elicited these statements is Build IT, a the Build IT program. She holds an M.A. from Stanford University’s two-year afterschool and summer curriculum designed School of Education. help middle school girls develop fluency in information WILLIAM R. PENUEL is professor of educational psychology and technology (IT), interest in mathematics and computer learning sciences in the School of Education at the University of Colorado Boulder. He studies the implementation and effective- science, and knowledge of IT careers. Build IT is a ness of technology-supported programs in science, engineering, and problem-based curriculum consisting of six units that mathematics programs both in and out of school. He is a former capitalize on girls’ interest in design and communica- co-principal investigator on Build IT and InnovaTE3, where he helped tion. SRI International’s Center for Technology and to develop embedded assessments of girls’ design skills. His current Learning (SRI) and Girls Incorporated of Alameda research focuses on developing interest in science. tators’ capacity to provide this programming. To date, ed to grow at a rate faster than the average rate for all Build IT has been implemented at 33 sites and has occupations (Lacey & Wright, 2009; National Science reached more than 2,000 girls and 50 afterschool educa- Board, 2010). The fundamental obstacles to girls enter- tors in the U.S. and Canada through the Girls Inc. net- ing the workforce in science, technology, engineering, work of affiliates. Co-developed and co-owned by SRI and mathematics (STEM) today are the value girls place and GIAC, with funding from the National Science on STEM careers, their interest in STEM topics, and their Foundation and the Noyce Foundation, Build IT is now expectations of success in STEM fields (Barman, 1996; managed by the Girls Inc. national organization, which Brickhouse, Lowery, & Schultz, 2000; Chambers, 1983; provides professional development Eccles, 1994, 2007; Eccles, Wong, for all its affiliates. & Peck, 2006). To be convinced of This paper outlines the need Nationally, women make the value of STEM careers and their for sustainable, scalable after- up half of the workforce potential success, girls need to see school computer science programs but hold one-quarter or their interests reflected in STEM targeting girls and describes the fewer of the positions in courses and informal learning op- development of one such curricu- portunities so that science and lum. Evaluation research on girls’ engineering and computer- math become a central part of the learning of computer science and related fields. “girls they are” (Brickhouse et al., on the capacity of afterschool staff 2000). Girls should participate in and organizations to provide computer science program- tasks that are relevant to their lives and have social im- ming leads to our description of a research-based approach pact; they should also connect with role models in STEM to sustaining and scaling the program nationally—an professions and receive feedback and encouragement approach that other programs might use to expand their (Eccles, 1994; Halpern et al., 2007; National Center for reach and impact. Women and Information Technology, 2007). Afterschool settings show promise as places for youth from all back- The Need for Sustainable and Scalable grounds to gain confidence and interest in STEM careers IT Afterschool Programming (National Research Council, 2009). Policymakers, educators, and industry professionals ad- vocate for teaching technology fluency and computer sci- Reaching Girls and Young Women: ence in and out of school, especially for underserved Build IT Participants populations including girls, Latinos/as, and African Programs like Build IT are needed to overcome these ob- Americans. Unfortunately, “computer science programs stacles and change the statistics on numbers of women are often overlooked and underfunded, leading to insuf- and minorities in STEM careers. More than 65 percent of ficient curricula, a lack of teacher training in computer the girls participating in Build IT are African American or science, and decreased gender and ethnic diversity in Latina and from low socioeconomic status homes. For computer science programs and careers” (Coalition for many participants, Build IT is one of the few venues that Science After School, 2010). Each year, afterschool edu- gives them regular access to technology, opportunities to cators and learning science researchers create numerous design technological solutions, and exposure to IT ca- afterschool programs, but many of these programs end reers. The program also uses educative curriculum mate- with the initial funding. Starting with an important na- rials and a train-the-trainer approach to target staff learn- tional need, such as the one that Build IT addresses—in- ing, since afterschool staff often see computer science creasing the number of girls interested in pursuing com- content as daunting. All of the 31 staff members current- puter science learning and possibly careers—is an ly facilitating Build IT are women, and 55 percent are important first step toward building a sustainable and women of color. The majority are in their 20s and 30s scalable program. and were not familiar with computer science concepts Nationally, women make up half of the workforce when they began the program. but hold one-quarter or fewer of the positions in engi- neering and computer-related fields. Fewer than seven Developing for Scaling and Sustainability percent of Latina or African-American women have de- Education research has articulated the features for scal- grees or careers in these fields (National Academy of ing and sustaining innovations in schools (Coburn, 2003; Sciences et al., 2010). Yet these occupations are predict- Schneider & McDonald, 2007; St. John, 2003), includ-

Koch, Gorges, & Penuel BUILD IT 59 ing school science programs (Blumenfeld, Fishman, ward technology or to be interested in IT careers. Krajcik, Marx, & Soloway, 2000; Fishman & Krajcik, Researchers also interviewed and observed girls in the pro- 2003). In developing this program, we adapted this re- gram, capturing qualitative data on girls’ interest and en- search base for afterschool learning. Just as the absence gagement in IT. of a clear plan for implementation and scaling hampers The evaluation team interviewed, observed, and col- efforts to scale STEM innovations in schools (Confrey, lected implementation reports from staff. Staff also com- Lemke, Marshall, & Sabelli, 2002; McLaughlin & Mitra, pleted online surveys to document their impressions of 2001), so too does the absence of such plans hinder how well the program met the needs of the girls and of afterschool programs. the organization, how well the program addressed pro- To anticipate the challenges of building a scalable, fessional development needs, their plans to continue or sustained program, developers designed Build IT to un- discontinue the program, and their own IT learning and fold in multiple stages. Rather than waiting to think career interests. about sustainability and dissemination until after the Research questions for the evaluation included: program design had been articulated, scale and sustain- • Are girls engaged, achieving IT fluency, and interested in ability plans were integral to the concept. pursuing IT careers, including taking the necessary high The co-design process played a key role in these school mathematics and computer science courses? plans. In co-design, researchers and developers lead a • Is staff capacity at each site increased and supported to highly facilitated, team-based process with practitioners offer this IT fluency programming? to design and implement prototypes of the innovation. • Is this curriculum sustainable in different settings? To this process, SRI team members brought their exper- tise in research and development in the learning scienc- Evaluation results, outlined below, show that Build es, and the GIAC team brought its IT is achieving its goals. Girls’ atti- expertise in implementing youth tudes toward IT and understand- development programs for girls. Rather than waiting to ing of IT concepts improved. This Build IT team worked for think about sustainability Afterschool staff members in- three years to develop, implement, and dissemination until creased their capacity to offer the and refine the program. In later program and developed interest in years, other Girls Inc. affiliates im- after the program design IT education and careers for them- plemented the program, with the had been articulated, selves. These findings provide evi- national organization leading the scale and sustainability dence for the sustainability and professional development. plans were integral to scalability of the program.

Evaluation Findings the concept. Growth in Girls Throughout the life of Build IT, in- The data show that Build IT moti- ternal and external evaluators have used a mixed-methods vates girls to explore IT and pursue IT careers. Girls who approach to document changes both in girls’ attitudes to- saw IT careers as solitary and boring began to see them ward and understanding of IT and in staff members’ ca- as collaborative, fun, and intellectually stimulating; pacity to sustain and scale computer science program- many participants started to see IT as a possible career. ming, examining changes at both individual and Their attitudes toward math also changed. In the pilot organizational levels. Researchers surveyed girls about scale-up, we saw statistically significant improvement in their perceptions of and interest in IT fields and about girls’ confidence in math and belief in its usefulness. We their computer usage and skills. The evaluators also as- saw modest (but not statistically significant) improve- sessed participants’ understanding of IT concepts. In the ments in girls’ confidence with computers, attitudes first three years of the program, a comparison group from toward IT careers, and gender-neutral views of careers. the same schools and communities as program partici- Excerpts from interviews with girls illustrate these pants responded to the surveys and assessments. In most changes: of its settings, Build IT is part of a larger afterschool and I might be able to do that. summer program rather than an independent program for You could do amazing things. which participants sign up. Attendees are thus no more I thought [the jobs] were hard but seemed kind of fun. likely than other similar girls to have positive attitudes to-

60 Afterschool Matters Fall 2012 Girls in Build IT strengthen their technology fluency. noted they would need to find funding to continue. In the pilot scale-up, girls reported an increase in their Leaders also said the program was rewarding for staff and technology skills, and assessments showed improved IT girls. One said: knowledge. We saw statistically significant improvements At our site, we serve a large majority of girls from in girls’ frequency of computer use, self-reported com- very low-income, single-parent/guardian house- puter skills, perception of the usefulness of mathematics, holds who do not have the economic resources to and confidence in using math. Similarly, in initial imple- expose their daughters to IT equipment, programs, mentation at one affiliate, we saw a statistically signifi- or mentors. Without a program like [this one], their cant change in participants’ conceptual understanding, daughters would have minimal or no exposure to IT as compared to that of a similar group of girls not par- fields, careers, and information. ticipating in the program. In addition, girls who partici- pated in two years of the program scored higher on as- Additionally, the majority of facilitators—73 per- sessments of IT conceptual understanding than girls with cent—said they were comfortable implementing the pro- one year or less of participation. Finally, data from the gram; the remaining 27 percent report reported that they initial implementation with one affiliate indicate that were comfortable “to some extent.” Build IT participants with multiple years of exposure to the curriculum increasingly planned to take computer- Growth in Afterschool Staff related courses and college-track math courses. The Build IT curriculum is designed to teach staff as well as girls. Data show that staff who implemented the pro- Growth in Organizational Capacity gram often became comfortable troubleshooting technical To achieve scale and sustainability, a program must not problems and doing computer programming using HTML only meet its goals for youth participants, but also build or object-oriented programming tools. It was not uncom- organizational capacity. During the mon to see a staff member rooting first three years of Build IT imple- Staff who implemented in the organization’s server closet. mentation and subsequent two One said, “My Internet went down years of pilot scaling, all staff and the program often became the other day and it said ISP and organization leaders reported that comfortable LAN and all that stuff…and I was, the program was a good fit with the troubleshooting technical like, ‘Wow, I know what these needs of their organization, com- problems and doing things mean.’” Staff members’ com- munity, and girls; they said that fort with curriculum concepts also they would implement the program computer programming grew: they began to successfully again. Affiliate executive directors using HTML or object- incorporate and teach important found that they could secure local oriented programming concepts such as the engineering funding for Build IT and similar tools. It was not design process of defining the programs. Of the seven affiliates problem, brainstorming, sketch- that participated in the pilot scale- uncommon to see a ing, researching, developing, test- up, six are continuing to implement staff member rooting in ing, and using the new technology. the program. The national organi- the organization’s Researchers also found evi- zation hopes to scale Build IT to all server closet. dence that staff gained more than of its affiliates. the capacity to teach the curricu- Preliminary data from the re- lum. In a survey on staff capacity cent (2010–2012) scale-up of the Build IT program to 21 and IT learning, more than 60 percent of responding fa- affiliates (33 program sites) reinforced the pilot scale-up cilitators said the program influenced their career and findings, showing that the program is sustainable and education plans: 58 percent said they were thinking scalable. Ninety-five percent of organization leaders sur- about or pursuing a career involving STEM and 47 per- veyed said that the program met the needs of the com- cent were thinking about or pursuing further education munity and aligned with their organization’s goals. in STEM. One facilitator, for example, has moved on to a Leaders said that the program had support from their technology job, and another entered an educational tech- funders and was not expensive to implement; all but one nology graduate program. Two others have added a com- planned to continue offering the program, though a few puter science or technology focus to their postsecondary

Koch, Gorges, & Penuel BUILD IT 61 education. Others have created To achieve “deep and professional development, and roles in their organizations as coor- consequential change” future implementations. dinators of the Build IT curricu- (Coburn, 2003) in • Sustainability means maintaining lum, in effect building a career lad- the depth of the program—and der for STEM-focused educators afterschool STEM learning, allowing for acceptable adapta- and built-in support for the pro- our experience and tions—over time, under less than gram. Finally, staff members at the research led us to a co- ideal conditions. site that co-developed the curricu- design process, in which • Evolution of the innovation for lum have taken on leadership roles sustainability involves three types by becoming trainers for affiliates developers from the of innovators: developers, re- new to the curriculum. learning sciences and searchers, and practitioners. Encouraging the facilitators— youth development fields Practitioners’ implementation in- nearly all of whom are women and collaborated to develop a fluences future research and de- many of whom are women of col- rich, usable curriculum velopment. Evaluations and as- or—to pursue IT careers was not sessment tools that informed the an original goal of the program, that meets the needs original innovation can help but it certainly addresses the na- of youth and practitioners to adapt the innova- tional need for more women and their communities. tion and can provide data for particularly women of color in IT. funders of the sustained program. It may seem counterproductive to Cutting across all five of these facilitate staff members’ leaving the program; however, dimensions, researchers developing science curricula at from the start, the program development team planned the University of Michigan (Blumenfeld et al., 2000; for the high staff turnover that is common in afterschool Fishman & Krajcik, 2003) have identifiedusability —by organizations. In order to promote organizational memo- students, teachers, and administrators—as key to the ry of the program, Girls Inc. affiliate leaders as well as fa- sustainability of an innovation in schools: cilitators attend Build IT professional development. If an innovation is “usable,” this means three things: Additionally, the curriculum materials themselves are de- (1) that the innovation is adaptable to the organiza- signed to educate new staff members as they prepare ac- tion’s context, (2) that the organization is able to en- tivities and use them with the girls. act the innovation successfully, and (3) that the orga- nization is able to sustain the innovation. (Fishman Research-based Framework for Sustainability & Krajcik, 2003, p. 565) and Scalability of Afterschool STEM Frameworks for scaling and sustaining school-based in- These researchers note that the innovation is more novations provided insights to the program development than the curriculum materials; it includes planning for team for planning the stages of Build IT. Coburn (2003) ongoing support of the organization’s capacity to imple- outlined four interrelated dimensions for scaling and ment effective science curricula. Not only must teachers sustaining education innovations: depth, spread, shift, and students be able to use the materials, but also the and sustainability. Dede and Rockman (2007) added a organization must have the capacity to use the program. fifth dimension, evolution. Developers can think about Other researchers of in-school science learning have not- these five dimensions both sequentially and collectively, ed the importance and interplay of the usability of the as they reinforce one another. curriculum and the building of the organization’s capac- • Depth refers to the effect of the innovation on youth ity to offer the curriculum (Cohen & Ball, 1999; St. John, learning and educators’ practice. Coburn (2003) states 2003), a capacity that includes alignment with the orga- that “reform must effect deep and consequential nization’s culture, policy, and management initiatives change” (p. 4). (Blumenfeld et al., 2000; Fishman & Krajcik, 2003). • Spread is the traditional notion of scale: the spread of a reform to a greater number of sites. Achieving Depth through Co-Design • Shift in ownership requires that the practitioners re- To achieve “deep and consequential change” (Coburn, sponsible for implementation, not the developers, 2003) in afterschool STEM learning, our experience and have full authority, including over ongoing support, research led us to a co-design process, in which developers

62 Afterschool Matters Fall 2012 from the learning sciences and youth development fields designed to teach afterschool educators as much as the collaborated to develop a rich, usable curriculum that girls, so that the staff can understand and implement the meets the needs of youth and their communities. Penuel, curriculum. Educative curriculum materials increase Roschelle, and Schectman (2007) define co-design as a educators’ knowledge in specific instances of instruc- “highly facilitated, team-based process in which educa- tional decision making and help them develop more tors, researchers, and developers work together in defined general knowledge that they can apply flexibly in new roles to design an educational innovation, implement the situations (Ball & Cohen, 1996; Davis & Krajik, 2005). innovation with educators and students as a prototype, Build IT’s educative elements include computer science and evaluate each prototype’s significance for addressing a and IT concepts along with research-based practices for concrete educational need” (p. 51). engaging girls in these concepts. These elements reveal The Build IT team used philosophies and pedagogical the developers’ pedagogical judgments and help staff to approaches from the learning sciences and youth develop- access information, learn subject matter, anticipate and ment fields to develop a constructivist, problem-based cur- interpret what learners may think or do, and relate units riculum. The program’s hands-on experiences are not and big ideas. solely computer based; they enable youth to use their bod- Figure 1 outlines the Build IT team’s co-design ap- ies, creativity, energy, and visual representations to act out proach. To begin, the Build IT team developed a shared computational approaches to solving problems. The co- understanding of co-design, the afterschool learning en- design process allowed constant checking of the program’s vironment at Girls Inc., and the role of each contributor. usability for youth and youth development leaders. In ad- Next, the team conducted one or two brainstorming dition to iterative co-design, we incorporated the meetings using the Understanding by Design approach. Understanding by Design approach (Wiggins & McTighe, The team identified the “enduring understandings” (big 1998) to identify learning goals and ways of achieving them. ideas) in computer science, discussed products youth Learning goals, assessments, and activities were articulated could produce or actions they could do to demonstrate in a language consistent with youth development. their understanding of the concepts, and shared activity Throughout development, the Build IT team incor- ideas that could elicit these products or actions. porated educative elements in the curriculum that were Once an outline was agreed on, SRI team members drafted the curriculum, based on their com- Figure 1. Co-Design Approach puter science and mathematics expertise, and Girls Inc. team members reviewed it. The groups revised until both teams deemed Shared Understanding drafts to be ready for implementation. The team then prepared selected activities to pilot Brainstorm with UbD as guide with a few girls and shared the curriculum draft with advisors. The team revised the cur- Develop riculum again based on this feedback. Another round of revisions came after staff gave feed- back on initial professional development ses- Revise sions. Next, the unit was fully implemented with 15 or more girls. Formative evaluation of Pilot the implementation and feedback from girls and staff informed the next phase of revisions. Revise Each unit went through about three rounds of drafting, piloting, and revising. Professional Development At first glance, co-design may seem overly difficult: agreeing on curricular goals and fol- Revise lowing a structured iteration process are time- consuming. Yet co-design can help develop greater ownership over designs, strengthen Implement STEM content, and make designs more usable in real settings (Penuel et al., 2007).

Koch, Gorges, & Penuel BUILD IT 63 Achieving Spread by Building Partnerships the responsibility of the learning sciences organization, In order to spread and achieve scale, an innovation must with an articulated plan for transferring ownership to the influence the organization’s norms and principles, in- youth development organization. cluding policies, curriculum implementation, and pro- fessional development (Coburn, 2003). Proven impact, Sustaining Programs through Professional ease of use, and fit with the organization are critical fac- Development Infrastructure tors in achieving scale. Professional development plays a key role in sustaining a Partnerships can support an innovation’s spread. A program. As programs move toward sustainability, re- report on the sustainability of 21st Century Community sources for professional development and other assis- Learning Centers by The Finance Project (2006) finds tance often dissipate, especially for programs attempting partnerships to be essential for long-term sustainability. to achieve scale as well as sustainability (Coburn, 2003). Specifically, partners should have shared goals, clear In youth organizations, staff turnover is high. roles in program development and refinement, and cred- Organizations may train staff to implement a program ibility with funders. Partnerships one year, only to lose those staff the also have the potential to expand The youth development next year. A process for inducting the capacity of programs to coordi- organization led new staff to support the program nate educational and social servic- and providing for ongoing profes- es for children living in need, so implementation from Day sional learning can help maintain that afterschool programming can One of the project. The capacity. be as effective as possible (de Build IT team used the Build IT addressed this issue Kanter, Adair, Chung, & Stonehill, implementation of the by sharing professional develop- 2003). ment responsibilities with sites For Build IT, the work began curriculum by girls from the beginning. A program with key partnerships among the and facilitators as a source manager worked side-by-side with two developers, SRI and GIAC, and of information for learning sciences researchers and the Girls Inc. national office, which making refinements. program developers to design and would provide professional devel- deliver professional development. opment and scaling support for its SRI staff led the initial professional network of more than 150 affiliates. Each affiliate has de- development for each unit; for the second implementa- veloped further partnerships with local tech organiza- tion, both organizations co-led the professional develop- tions, since the curriculum includes connecting girls with ment. By the third implementation, Girls Inc. staff led women STEM professionals. This strategy of establishing the professional development. ongoing partnerships with the local STEM community Build IT is successful in part because ongoing pro- has the potential to keep the program current with STEM fessional development is part of the infrastructure of changes and to attract new funders. Girls Inc. at each affiliate and nationwide. Like many other youth-serving organizations, affiliates experience Developing Ownership from the Beginning frequent staff turnover but have a relatively stable core of Rather Than Shifting program managers. The national organization provides During the initial stages of design and pilot implementa- professional development on many of its programs; its tion, curriculum developers and researchers typically professional development staff includes Build IT in a drive the process. For the Build IT program, the co-design suite of STEM programs offered to affiliates. Having a process facilitated a partnership that capitalized on the professional development staff and a training infrastruc- skills of both organizations. It also anticipated the end of ture for face-to-face sessions, webinars, and online sup- grant funding, so that design decisions were based on port makes Girls Inc. capable of sustaining innovations. how to support ongoing implementation within the larger afterschool program. The youth development organiza- Developing and Aligning Frames That Allow tion led implementation from Day One of the project. The a Program to Evolve Build IT team used the implementation of the curriculum A single project that initiates a cycle of program develop- by girls and facilitators as a source of information for ment typically presents a single “frame” to a potential making refinements. Professional development began as funder. The term frame (Goffman, 1974; Snow & Benford,

64 Afterschool Matters Fall 2012 1988) refers to a specific definition of a problem, a path engaging youth development and learning sciences experts to its solution, and a rationale that makes the solution in a co-design process, using professional development to compelling. continually support the program and transfer ownership to A proposal frame—the initial rationale for winning practitioners, and working within an established network funding—is rarely enough to sustain a program across of affiliates. As a result, Build IT is having a positive effect multiple projects or to convince new groups to fund new on girls and afterschool staff throughout the nation. development or implementation in new settings. A key task for sustainability is to develop multiple frames for References defining problems and to establish congruence among Ball, D. L., & Cohen, D. K. (1996). Reform by the book: them. This activity of aligning frames cannot be simply What is—or might be—the role of curriculum materials “chasing the money,” but rather must be a genuine bridg- in teacher learning and instructional reform? Educational ing or extension of activity in ways that allow the pro- Researcher, 25(9), 6–8. gram to adapt, grow, and even transform as it moves to Barman, C. (1996). How do students really view science new contexts. and scientists? Science & Children, The frame for funding Build IT 34, 30–33. has varied according to the needs A key task for sustainability Blumenfeld, P., Fishman, B. J., and resources of the affiliates and is to develop multiple Krajcik, J., Marx, R. W., & their communities. For example, frames for defining Soloway, E. (2000). Creating one affiliate’s search for women usable innovations in systemic STEM professionals led to the dis- problems and to establish reform: Scaling-up technology- covery that the city had an initia- congruence among them. embedded project-based science tive to attract IT businesses. This activity of aligning in urban schools. Educational Through collaborations with the frames cannot be simply Psychologist, 35(3), 149–164. city and a local university, this af- “chasing the money,” but filiate secured funding for Build IT, Brickhouse, N. W., Lowery, P., & identified field trip opportunities, rather must be a genuine Schultz, K. (2000). What kind of a and established relationships with bridging or extension of girl does science? The construction STEM professional role models activity in ways that allow of school science identities. Journal who regularly participate in the the program to adapt, of Research in Science Teaching, program. This affiliate’s frame com- 37(5), 441–458. bined the need for funds with the grow, and even Chambers, D. W. (1983). need for role models. The level of transform as it moves to Stereotypic images of the scientist: local interest in IT jobs enables this new contexts. The draw-a-scientist test. Science affiliate and others to use Build IT Education, 67, 255–265. as a marketing tool to fund not Coalition for Science After School. only Build IT but other programs as well. (2010). Computer Science Education Week: December Build IT started with framing a need to encourage 5th–11th, 2010. Retrieved from http://scienceafterschool. girls to pursue computer science and IT careers. At the blogspot.com/2010/12/computer-science-education- national level, Build IT’s success has made it part of the week.html frame on using evaluation data to show how Girls Inc. programs affect girls. The national organization uses Coburn, C. E. (2003). Rethinking scale: Moving beyond multiple frames of funding, professional development, numbers to deep and lasting change. Educational scale, research, and evaluation to achieve its goals, which Researcher, 32(6), 3–12. include making sure all affiliates can implement Build IT. Cohen, D. K., & Ball, D. L. (1999). Instruction, capacity, and improvement (CPRE Research Report Series RR-043). Conclusion Philadelphia, PA: University of Pennsylvania Planning for sustainability and scalability from the Consortium for Policy Research in Education. beginning is an important means of ensuring that Confrey, J., Lemke, J. L., Marshall, J., & Sabelli, N. programs continue beyond their initial grant funding. The (2002). A final report on a conference on models of Build IT development team successfully achieved scale by implementation research within science and mathematics

Koch, Gorges, & Penuel BUILD IT 65 instruction in urban schools. Austin, TX: University of McLaughlin, M. W., & Mitra, D. (2001). Theory-based Texas, Systemic Research Collaborative for Education in change and change-based theory: Going deeper, going Mathematics, Science, and Technology. broader. Journal of Educational Change, 2, 301–323. Davis, E. A., & Krajcik, J. (2005). Designing educative National Academy of Sciences, National Academy of curriculum materials to promote teacher learning. Engineering, Institute of Medicine, & National Research Educational Researcher, 34(3), 3–14. Council. (2010). Expanding underrepresented minority Dede, C., & Rockman, S. (2007, Spring). Lessons participation: America’s science and technology talent at the learned from studying how innovation can achieve crossroads. Washington, DC: National Academies Press. scale. Threshold, 4–10. Washington, DC: Cable in the National Center for Women & Information Technology. Classroom. (2007). Guide to promising practices in informal informa- de Kanter, A., Adair, J. K., Chung, A.-M., & Stonehill, tion technology education for girls. Boulder, CO: NCWIT R. M. (2003). Ensuring quality and sustainability in and Girl Scouts. Retrieved from http://www.ncwit.org/ after-school programs: How partnerships play a key pdf/Practices_Guide_FINAL.pdf role. Yearbook of the National Society for the Study of National Research Council. (2009). Learning science in Education, 102(2), 201–220. informal environments: People, places, and pursuits. Eccles, J. S. (1994). Understanding women’s educational Washington, DC: National Academies Press. and occupational choices: Applying the Eccles et al. National Science Board. (2010). Preparing the next model of achievement-related choices. Psychology of generation of STEM innovators: Identifying and developing Women Quarterly, 18, 585–609. our nation’s human capital. Arlington, VA: National Eccles, J. S. (2007). Where are all the women? Gender Science Foundation. differences in participation in physical science and Penuel, W. R., Roschelle, J. & Shechtman, N. (2007). engineering. In S. J. Ceci & W. M. Williams (Eds.), Why Designing formative assessment software with teachers: Aren’t More Women in Science? Top Researchers Debate the An analysis of the co-design process. Research and Evidence (pp. 199–210). Washington, DC: American Practice in Technology Enhanced Learning, 2(1), 51–74. Psychological Association. Schneider, B., & McDonald, S. K. (Eds.). (2007). Eccles, J. S., Wong, C. A., & Peck, S. C. (2006). Scale-up in education: Issues in practice (Vol. 2). New Ethnicity as a social context for the development of York, NY: Rowman & Littlefield. African-American adolescents. Journal of School Snow, D. A., & Benford, R. D. (1988). Ideology, frame Psychology, 44, 407–426. resonance, and participant mobilization. International Finance Project. (2006). Sustaining 21st Century Social Movement Research, 1, 197–217. Community Learning Centers: What works for programs St. John, M. (2003). The legacies of the LSCs. Panel at the and how policymakers can help. Washington, DC: Author. Third Annual Conference on Sustainability of Systemic Fishman, B., & Krajcik, J. (2003). What does it mean to Reform. Retrieved from http://sustainability2003.terc. create sustainable science curriculum innovations? A edu/do/247/show.html commentary. Science Education, 87(4), 564–573. Wiggins, G., & McTighe, J. (1998). Understanding by Goffman, E. (1974). Frame analysis: An essay on the design. Alexandria, VA: Association for Supervision and organization of experience. Boston, MA: Northeastern Curriculum Development. University Press. Halpern, D., Aronson, J., Reimer, N., Simpkins, S., Star, J., & Wentzel, K. (2007). Encouraging girls in math and science (NCER 2007-2003). Washington, DC: National Center for Education Research, Institute of Education Sciences, U.S. Department of Education. Available from http://ncer.ed.gov Lacey, T. A., & Wright, B. (2009, November). Occupational employment projections to 2018. Monthly Labor Review, 132(11), 82–123.

66 Afterschool Matters Fall 2012