Capacity Building to Diversify STEM Realizing Potential among HBCUs SCIENCE, TECHNOLOGY, ENGINEERING, AND MATHEM ATICS Beatriz Chu Clewell Clemencia Cosentino de Cohen Lisa Tsui Capacity Building to Diversify STEM Realizing Potential among HBCUs FINDINGS FROM THE N ATIONAL E VALUATION OF THE HISTORICALLY B LACK C OLLEGES AND U NIVERSITIES UNDERGRADUATE P ROGRAM Beatriz Chu Clewell Clemencia Cosentino de Cohen Lisa Tsui Contributors to this study: Alla Myrvoda Erin Mudd Gilfenbaum Nicole Deterding Prog ram for Evaluation and Equity Resea rch (PEER) 2010 This report was prepared for the National Science Foundation, Directorate for Education and Human Resources (EHR), Division of Human Resource Development (HRD). The data collection, analysis, and reporting of this ma- terial was conducted in accordance with OMB Control No. 3145-0204 and supported by REC Contract # GS-23F- 8198H D050597. This report is based on the full technical report (270 pages) submitted to, and approved by, the National Science Foundation. © 2010, the Urban Institute The Urban Institute is a nonprofit, nonpartisan policy research and educational organization that examines so- cial, economic, and governance problems confronting the nation and evaluates the public and private means to alle- viate them. The Urban Institute’s Program for Evaluation and Equity Research (PEER) focuses on education research. PEER staff, who carried out this evaluation, conduct studies of educational attainment, educational access, minori- ties in mathematics and science, teacher education, teacher recruitment and retention, and educational assessments. The opinions, conclusions, and recommendations expressed in this document are those of the authors and do not necessarily reflect the views of the National Science Foundation or the Urban Institute. CONTENTS Acknowledgments . iv Executive Summary . v Introduction . 1 Role of HBCUs in Producing Scientists and Engineers. 1 Historically Black Colleges and Universities Undergraduate Program . 2 Evaluation Design. 4 Research Questions . 4 Methodology . 4 Process Evaluation Findings: Characteristics of HBCU-UP Projects . 6 Project Components: How is HBCU-UP Being Implemented?. 6 Why Is the HBCU-UP Core Model Successful? . 7 Characteristics of Successful Projects: Insights from the Case Studies . 8 Summary. 9 Summative Evaluation Findings: Outcomes of HBCU-UP. 10 Characteristics of HBCU-UP Graduates . 10 Educational Outcomes of HBCU-UP Graduates. 10 Workforce Outcomes of HBCU-UP Graduates. 13 Faculty Opinions and Perceptions . 14 Institutional Capacity Building: Curricular Enhancements . 17 Knowledge Base: HBCU-UP Model Core . 18 Summary. 20 Conclusions and Recommendations. 22 Notes . 24 References . 25 C APACITY B UILDING TO D IVERSIFY STEM: REALIZING P OTENTIAL AMONG HBCUS iii Acknowledgments We wish to thank the great many people who extended View A&M University; Dr. Aliakbar Haghighi, Professor cooperation and assistance to us in completing this com- at Prairie View A&M University, Dr. Danielle Gray- prehensive study. We are much indebted to the case Singh, HBCU-UP Director at Tuskegee University; Dr. study sites—Bennett College, Hampton University, Connie Leggett, Codirector at Albany State University; North Carolina A&T State University, and Tougaloo and Dr. Luther Williams, Provost at Tuskegee University. College—and the project directors, faculty members, administrators, and students who participated in our case Our work was much facilitated by the support and study interviews and focus groups. We also thank the guidance of the staff at the National Science Foundation. HBCU-UP project directors for so generously giving In particular, this study benefitted greatly from the ded- their time to participate in the telephone interviews and ication and leadership of the NSF contracting officer to submit course revision information. We are grateful to Elmima Johnson, NSF Senior Advisor on Evaluation the faculty at the HBCU-UP sites for participating in the Bernice Anderson, and the HBCU-UP program officers faculty survey and the numerous HBCU-UP graduates Jesse DeAro, Camille McKayle, Marilyn Suiter, and from all over the nation who responded to our survey. Claudia Rankins. Special thanks are in order for those who provided We gratefully acknowledge the contribution of Rob feedback, information, and assistance in a number of Santos, senior methodologist at the Urban Institute, for areas, including facilitating field testing of instruments, his expert guidance with the study design and statistical gaining access to graduates and faculty, and verifying data. methods and for his feedback on the draft report. We These individuals include Dr. Shirley McBay, President also received comments from a number of internal and of QEM (Quality Education for Minorities) Network; external reviewers, which resulted in improvements to Dr. Jason Kim, President of Systemic Research Inc.; the final report. Finally, we would like to acknowledge Linda Crasco, Executive Director of Systemic Research the staff at NuStats for their unrelenting efforts to in- Inc.; Dr. Kelvin Kirby, HBCU-UP Director at Prairie crease the survey response rates. iv C APACITY B UILDING TO D IVERSIFY STEM: REALIZING P OTENTIAL AMONG HBCUS Executive Summary The Historically Black Colleges and Universities career outcomes of graduates of HBCU-UP projects, Undergraduate Program (HBCU-UP), administered by and, most importantly, the efficacy of the HBCU-UP the National Science Foundation (NSF) Directorate for model. Through a quasi-experimental design that com- Education and Human Resources, Division of Human pared HBCU-UP faculty and graduates to nationally Resource Development, began in 1998. The goal of representative samples of each—the analysis led to five HBCU-UP is to enhance the quality of undergraduate conclusions and recommendations.1 education and research in science, technology, engineer- ing and mathematics (STEM) at HBCUs as a means to broaden participation in the nation’s STEM workforce. Conclusions Between 1998 and 2009, the program made 139 insti- 1. HBCU-UP grantees succeeded in building an institu- tutional awards for a total of over $200 million. tional infrastructure that supports the education of STEM The external evaluation, commissioned by NSF in majors. Institutions carried out curricular and instruc- 2006, focused on the implementation projects funded tional reforms, provided faculty professional develop- under the HBCU-UP program, which are five-year, in- ment, established academic support services for stu- stitutionwide STEM education and research capacity- dents, engaged in collaborative relationships with other building projects. Given great flexibility to design and institutions and entities, and upgraded their laboratory implement strategies that address each institution’s and STEM instructional equipment. Some succeeded STEM needs and long-term goals, projects tended to in institutionalizing key project components. focus on institutional capacity-building activities such 2. The HBCU-UP program yielded an intervention as curriculum development and faculty professional de- model characterized by a core set of strategies associated velopment. NSF expected projects to produce “signifi- with successful student outcomes. Core strategies include cant improvements in undergraduate STEM education curricular reform, faculty professional development, and research programs” (National Science Foundation and summer bridge programs. Alumni from institu- 2006). The evaluation of HBCU-UP measured changes tions that employed all core strategies were more likely carried out by grantees, and whether those changes were to stay in the STEM education pipeline, and those em- associated with the ultimate program outcome of con- ployed in STEM were more likely to have earned a tributing to the education and retention of minority graduate degree. students in STEM (as a means to broaden participation in the STEM labor force). 3. Successful HBCU-UP projects shared elements that sug- gest that effective projects (a) design interventions to ad- The Urban Institute evaluation of the HBCU-UP dress well-defined problems; (b) provide a comprehen- program included both process and summative compo- sive array of strategies that span institutional nents, seeking to understand the implementation of the infrastructure improvement, faculty development, and program and to measure its ultimate outcomes. The student support services; (c) tailor their strategies and process component relied mostly on qualitative meth- activities to their institutional mission and characteris- ods (interviews and case studies) to study the character- tics; and (d) institutionalize the key components of their istics of funded projects as well as the factors that may projects. promote or inhibit the attainment of project goals. Results guided the definition of the HBCU-UP model 4. HBCU-UP graduates outperform national samples of of institutional capacity building and informed the STEM baccalaureate degree holders in degree completion thrust of this evaluation: the summative component. and in participation in the STEM workforce with a grad- The summative evaluation relied on qualitative and uate degree. Compared to STEM graduates nationally, quantitative methods to analyze the course revisions HBCU-UP alumni were more likely to have sought and carried out by institutions, the experiences and opinions obtained graduate degrees (overall and in STEM), of participating faculty, the educational progression and equally likely