Annie E. Casey Foundation, 701 St. Paul Street, Baltimore, MD 21202
Total Page：16
File Type：pdf, Size：1020Kb
DOCUMENT RESUME ED 448 236 UD 033 893 TITLE Transforming Neighborhoods into Family-Supporting Environments: Evaluation Issues and Challenges. Report of the Annie E. Casey Foundation's Research and Evaluation Conference (Baltimore, Maryland, March 1999) INSTITUTION Annie E. Casey Foundation, Baltimore, MD. PUB DATE 2000-09-00 NOTE 79p. AVAILABLE FROM Annie E. Casey Foundation, 701 St. Paul Street, Baltimore, MD 21202. Tel: 410-223-2890; Fax: 410-547-6624; Web site: http://www.aecf.org. PUB TYPE Collected Works Proceedings (021) EDRS PRICE MF01/PC04 Plus Postage. DESCRIPTORS *Community Development; *Evaluation Methods; *Family Programs; Family Status; *Neighborhood Improvement; *Program Evaluation; Technical Assistance ABSTRACT The Annie E. Casey Foundation's fourth conference occurred as the Foundation launched a new neighborhood transformation and family development strategy centered on strengthening families. This conference summary presents discussions of the major themes from the conference. Section 1, "Understanding the Connection between Families and Communities," explores how neighborhood resources and characteristics can impact a family's efforts to raise its children. Section 2,"Evaluating Initiatives That Strengthen Families and Build Communities," describes the challenges facing evaluators of neighborhood- and family-building initiatives, the issues involved in providing technical assistance on evaluation, and tactics for presenting data in useful and compelling ways. Section 3, "Lessons for Future Initiatives and Evaluations," concludes the summary with lessons and recommendations proposed by the participants. Appendixes contain the conference agenda, transcripts of two keynote addresses, and a list of participants. The keynote addresses (not titled) were by Angela Blackwell and Felton Earls.(SLD) Reproductions supplied by EDRS are the best that can be made from the original document. transforming neighborhoods into family-supporting environments: EVALUATION ISSUES AND CHALLENGES U.S. DEPARTMENT OF EDUCATION ONNA 01E0.01100W f1111111211 and Imoraement EDUCATIONAL RESOURCES INFORMATION CENTER (ERIC) This document has been reproduced as Waived from the parson or otqananuon Originating 4. O Minor changes nave been made to improve reproduction gustily. Points of view or opinions stated 111 INS document do not necessarily represent BEST COPY AVAILABLE; official OERI position cr policy. Ce) Report of the PERMISSION TO REPRODUCE AND DISSEMINATE THIS MATERIAL HAS Annie E. Casey Foundation's BEEN GRANTED BY March 1999 Research and Evaluation Conference twit V., rzeCet TO THE EDUCATIONAL RESOURCES INFORMATION CENTER (ERIC) 4 transforming neighborhoods into family-supporting environments:EVALUATION ISSUES AND CHALLENGES BEST COPY AVAILABLE Report of the Annie E. Casey Foundation's March 1999 Research and Evaluation Conference 3 Additional copies of the report are available from: The Annie E. Casey Foundation 701 St. Paul Street Baltimore, MD 21202 410-223-2890 Publications Order Line 410-547-6600 Phone 410-547-6624 Fax www.aecf.org printed on recycled paper September 2000 4 Research and Evaluation at Thennie E. Casey Foundation The Annie E. Casey Foundation's mission,motivation, and message to the world is its commitment to changing and improving life outcomes for our most disadvantagedchildren and families. That belief is rooted in the conviction that outcomes for children will notimprove without fundamental, comprehensive, and durable changes in many service and support systems.Current health, education, juvenile justice, and other delivery systems for disadvantagedchildren and families too often are fragmented, inaccessible, expensive, and irrelevant. Theyfrequently fail to deliver essential services until it is too late, contributing to an overall level ofineffectiveness and to an intergenerational cycle of poverty. The Foundation operates on the premise that these conditions canbe reversedthat communities can prosper, families can thrive, and children candevelop when neighborhoods are supportive, sustaining, and served by systems that are relevant, respectful, androoted in the communities that they serve. The Foundation believes that strategic investments in awarenessbuilding, capacity development, program demonstrations, and research and evaluation can help movedysfunctional service systems toward greater collaboration, coordination, and flexibility. In addition to leadership, funding, and other key factors, thesechanges require accurate, relevant, and compelling information. Research and evaluation are a conduit forinformationand information is power. With information on resultsprovided by evaluation, community stakeholders can make better decisions about organizational practices. Similarly, states informedby research and evaluation can make better decisions about the allocation of resources and policiesthat affect children and families. Evaluation plays a major role in the Foundation's theory of change, as atool for: Improving accountability: contributing to understanding about thedegree to which interventions represent good judgments about the organizations,communities, and people in which the Foundation places its confidence and resources. Revealing the soundness of theories, the practicality of policies, theappropriateness of planning timelines, the relevance of technical assistance, and the extent towhich the Foundation has established effective partnerships with grantees. Informing funders about the viability of working with states, cities,community-based organizations, and child- and family-serving systems to achieve realtransformation and reform. For these reasons, the Casey Foundation believes thatresearch and evaluation can, should, and must be a critical and integral component ofcomprehensive reform strategies. 5 C ntents CONFERENCE OVERVIEW 1 CROSS-CUTTING THEMES 2 UNDERSTANDING THE CONNECTION BETWEEN FAMILIES AND COMMUNITIES 4 DOES NEIGHBORHOOD MATTER? 4 HOW DOES NEIGHBORHOOD MATTER? 4 Social Capital 4 Social Networks and Embeddedness 5 Collective Efficacy 7 Mechanisms by which Neighborhoods Affect Child and Family Development and Well-Being 8 The Role of Community Context 8 COMMUNITY CAPACITY FOR SOCIAL CHANGE 10 EVALUATING INITIATIVES THAT STRENGTHEN FAMILIES AND BUILD COMMUNITIES 14 CHALLENGES FOR EVALUATION DESIGN AND METHODOLOGY 14 Clarifying the Unit of Analysis 14 Clarifying the Purpose and Magnitude of an Evaluation 15 Defining Outcomes, Indicators, and Measures 16 Measuring Change 22 Conducting Comparative Research 25 Timing Evaluation Activities 26 Integrating Qualitative and Quantitative Data 28 Using a Two-Tiered Approach to Combine National and Local Evaluations 29 TECHNICAL ASSISTANCE ISSUES 33 PRESENTING EVALUATION DATA IN USEFUL AND COMPELLING WAYS 38 Documenting the Change Process 38 Providing Communities with Feedback 39 Understanding and Using Data as an Advocacy Tool 41 Building Local Capacity to Appreciate and Use Data 42 LESSONS FOR FUTURE RESEARCH AND EVALUATION 45 CREATE NEW ROLES AND RELATIONSHIPS 45 IMPROVE DYNAMICS BETWEEN FUNDERS, EVALUATORS, AND COMMUNITIES 45 CREATE ENVIRONMENTS AND OPPORTUNITIES THAT SUPPORT ONGOING REFLECTION 46 NEW DIRECTIONS FOR RESEARCH 47 APPENDIX A: CONFERENCE AGENDA A-1 APPENDIX B: KEYNOTE ADDRESS BY ANGELA BLACKWELL B-1 APPENDIX C: KEYNOTE ADDRESS BY DR. FELTON EARLS C-1 APPENDIX D: CONFERENCE PARTICIPANTS D-1 6 Confereiiuceverview The Annie E. Casey Foundation transforming neighborhoods into places continued its dialogue about difficult that nurture and support families and research issues with leaders and reinforce children's chances for success. evaluators of social interventions at Neighborhood transformation a conference in March 1999. The community buildinginvolves providing conference, held in Baltimore, was the the incentives, investments, and fourth in a series of forums offered opportunities needed to connect families periodically by the Foundation since 1994 with jobs, friends and neighbors, faith- to help identify emerging challenges and based institutions, and other social explore promising solutions. relationships, networks, and supports. The topics of the conferences reflect the It is clear to us that when families ... live in Foundation's evolving strategies for places that exacerbate their struggles improving outcomes for disadvantaged places that are perceived to be or are children and families, as well as new unsafe, places that are prone to extensive developments in the art and science of drug use and drug commerce, where understanding social change. The first economic investments are weak, where conference focused on issues related to the social supports are fragile ... and where evaluation of complex public system public systems are disproportionately reform. The second conference addressed weakit is easy for even the strongest ways of using evaluation results more families to feel compromised and effectively to advance public policy and overwhelmed. build public will. The third conference Tony Cipollone explored the complexities of evaluating The Annie E. Casey Foundation comprehensive community change. The fourth conference occurred as the The Foundation's investments under this Foundation launched a new neighborhood strategy, including the Making Connections transformation and family development demonstration initiative, will seek to build strategy, based on the following premises: public will, advance public policy, reform public systems, build family-supporting Although good interventions and neighborhoods, and strengthen