'Learning Together' YK Process Evaluation
Total Page：16
File Type：pdf, Size：1020Kb
Learning Together Young Knocknaheeny Process Evaluation, 2015 to 2017 Documenting and evaluating the implementation processes of the Young Knocknaheeny Programme during its first three years. Young Knocknaheeny, HSE Building, Harbourview Road, Knocknaheeny, Cork 021 601 0656 [email protected] youngknocknaheeny www.youngknocknaheeny.ie Young Knocknaheeny Area Based Childhood Programme Learning Together Young Knocknaheeny Process Evaluation, 2015 to 2017 2018 Publication The authors of this report are: Ms. Lynn Buckley and Dr Margaret Curtin How to cite this report Any citation of this report should use the following reference: Buckley, L., Curtin, M. (2018) Learning Together: YK Process Evaluation, 2015 to 2017. Cork: Young Knocknaheeny (YK). Young Knocknaheeny, HSE Building, Harbourview Road, Knocknaheeny, Cork YK Research Advisory Group Dr Margaret Curtin (YK Chairperson; School of Nursing and Midwifery, UCC), Dr Shirley Martin (Applied Social Studies, UCC), Dr Siobhan O’Sullivan (Applied Social Studies, UCC), Dr Maria Dempsey (Applied Psychology, UCC), Ms. Anne Horgan (HSE; YK Management Group), Dr Pat Corbett (YK Management Group) and Ms. Katherine Harford (YK Programme Manager). Contributors Ms. Catherine Maguire, Ms. Grace Walsh, Ms. Sally O’Sullivan, Dr Shirley Martin, Ms. Jenna Russell, Dr Katrina Collins, and Ms. Emma Ní Neill. Ms. Braedon Donald, previous YK Research Officer, contributed to the development, design and data collection phases of the evaluation. Acknowledgements YK Community YK would like to acknowledge all of the children and parents in the YK community and thank them for their engagement with the programme and for their participation in interviews and focus groups. We would also like to thank the many local practitioners who gave so readily of their time, taking part in the various capacity building trainings and participating in interviews and focus groups. Service Delivery Partners YK would like to acknowledge the contribution of its many service delivery partners who committed their time, resources, skills and expertise to the implementation of the YK Programme; HSE, UCC, Cork City Council, Tusla, CUMH, Cork City Childcare Commitee, as well as local education, childcare and community services. YK Team YK would like to acknowledge the contribution of all YK staff members and thank them for their support throughout all phases of the evaluation. Lead Agency YK would like to thank NICHE for acting as lead agency for the programme. Funders YK would also like to acknowledge the funding and support offered by its main funding bodies: The Department of Children and Youth Affairs and Atlantic Philanthropies. YK would also like to thank the Area Based Childhood (ABC) Programme Managers, the Centre for Effective Services and Pobal. 4 Table of Contents Acknowledgements 4 Preface 10 Executive Summary 11 Glossary of Terms 13 Definitions 14 Chapter 1: Introduction 15 1.1 The YK Programme 17 1.2 History of Young Knocknaheeny 18 1.3 The YK Catchment Area 20 1.4 The Science 21 1.5 YK Development Process 23 Preparation 23 Planning 23 Implementation 27 Maintenance 29 1.6 The Evaluation Report 31 Chapter 2: Evaluation Methodology 33 2.1 Evaluation Rationale & Focus 35 2.2 Research Frameworks 35 2.3 Evaluation Purpose, Aims and Objectives 37 2.4 Research Questions 37 2.5 Research Design 38 2.6 Data Management 41 2.7 Ethical Considerations 42 2.8 Report Limitations 42 Chapter 3: Evaluation Results 43 3.1 Strategy 1: Infant Health and Well-being 45 3.1.1 Overview of Strategy 47 3.1.2 Evaluation of Strategy 1 48 3.1.3 IMH Practitioner Training and Coaching Programme 48 3.1.4 YK Inter-disciplinary IMH Home Visiting Programme 59 3.1.5 Engagement with Strategy 1 Programmes 75 3.1.6 Implementation Process of Strategy 1 77 3.1.7 Key Findings and Learning 81 3.2 Strategy 2: Speech, Language and Literacy 83 3.2.1 Overview of Strategy 85 3.2.2 Evaluation of Strategy 2 85 3.2.3 Babbling Babies 85 3.2.4 Hanen Programmes 95 3.2.5 Implementation Process of Strategy 2 109 3.2.6 Key Findings and Learning 112 3.3 Strategy 3: Early Years Care and Education 115 3.3.1 Overview of Strategy 117 3.3.2 Evaluation of Strategy 3 118 3.3.3 Environment Rating Scales Assessments 121 3.3.4 HighScope Training Programme 127 3.3.5 Mentoring Programme 135 3.3.6 Environmental Enhancement Funding 139 3.3.7 Implementation Process of Strategy 3 141 3.3.8 Key Findings and Learning 144 3.4 Strategy 4: Pro-social Behaviour and Self-regulation 147 3.4.1 Overview of Strategy 149 3.4.2 Evaluation of Strategy 4 149 3.4.3 Incredible Years Parenting Programme 150 3.4.4 IY Teacher Classroom Management Programme 160 3.4.5 Implementation Process of Strategy 4 166 3.4.6 Key Findings and Learning 169 3.5 Programme Level Results 173 3.5.1 Overview 175 3.5.2 YK Programme Reach 176 3.5.3 Process Evaluation 177 3.5.4 Key Findings and Learning 181 Chapter 4: Discussion and Recommendations 183 Why these Findings Matter 185 Recommendations 187 References 191 Appendices 195 Further Information 210 List of Tables Table 1.1: YK catchment area demographic profile (Total population 12,000) 20 Table 1.2 YK Strategies Summary 26 Table 1.3: YK Logic Model 2015 27 Table 1.4: Pro-social Behaviour Strategy Feedback 17 Table 2.1: YK Evaluation Stages and Processes 37 Table 2.2: Stage 3 and 4 Evaluation Questions 39 Table 3.1.1: Respondent characteristics, IMH masterclass questionnaires 50 Table 3.1.2: Strategy 1 Implementation Process, 2015 to 2018 78 Table 3.1.3: Strategy 1 Key Findings 81 Table 3.1.4: Strategy 1 Key Learning 81 Table 3.2.1: Parent awareness of communication development milestones 88 Table 3.2.2: General SLCN promotion strategies provided during BB clinics 88 Table 3.2.3: OLO concerns re child development 89 Table 3.2.4: OLO concerns re risk factors for communication development 89 Table 3.2.5: Advice offered by OLOs following the identity of risk factors 90 Table 3.2.6: Strategy 2 Implementation Process, 2015 to 2018 110 Table 3.2.7: Strategy 2 Key Findings 112 Table 3.2.8: Strategy 2 Key Learning 113 Table 3.3.1: Overall pre and post ECERS-3 results by subscale 122 Table 3.3.2: Overall pre and post ITERS-R results by subscale 123 Table 3.3.3: Overall Childcare Quality Improvement Results 124 Table 3.3.4: Summary of quality improvements in EY centres 125 Table 3.3.5: Strategy 3 Implementation Process, 2015 to 2018 142 Table 3.3.6: Strategy 3 Key Findings 144 Table 3.3.7: Strategy 3 Key Learning 144 Table 3.4.1: Duration of Consultation 152 Table 3.4.2: Type of support offered 153 Table 3.4.3: Type of IY strategy reports as used by parents at each consultation 153 Table 3.4.4: Intensity level at time of consultation 154 Table 3.4.5: Strategy 4 Implementation Process, 2015 to 2018 167 Table 3.4.6: Strategy 4 Key Findings 169 Table 3.4.7: Strategy 4 Key Learning 170 Table 3.5.1: Programme Level Findings 181 Table 3.5.2: Programme Level Learning 182 List of Figures Figure 1.1: YK Integrated Strategies 25 Figure 1.2: YK Spend by Year by Category, Jan 2015 to Dec 2017 29 Figure 1.3: YK spend per year versus total allocation, Jan 2015 – Dec 2017 30 Figure 1.4: YK spend per year versus total spend, Jan 2015 – Dec 2017 30 Figure 2.1: Overview of ‘The Framework for Programme Evaluation’: Elements of the framework [CDC, 2013] 36 Figure 3.1.1: Practitioner rates of encountering issues related to IMH 51 Figure 3.1.2: Practitioners’ interpretation of social and emotional development stages 51 Figure 3.1.3: Practitioners’ ability to identify lack of progress in children’s social and emotional development 52 Figure 3.1.4: Duration of home visit 62 Figure 3.1.5: Team members conducting home visits 63 Figure 3.1.6: Type of support offered via HVP home visits or phone calls 63 Figure 3.1.7 Intensity of visit by duration 64 Figure 3.1.8: Intensity of programme involvement by families, Jan ’15 to Dec ‘17 75 Figure 3.1.9: Geographical area of participants, Jan ‘15 to Dec ‘17 75 Figure 3.1.10: Nature of Referrals to YK, Jan ‘15 to Dec ‘17 76 Figure 3.1.11: Registration Pathways, Jan ’15 to Dec ‘17 76 Figure 3.2.1: ECERS-3 & ITERS-R Pre and Post Results 97 Figure 3.3.1: No. of participants in EY suite of quality improvement measures 120 Figure 3.3.2: Overall pre and post ECERS-3 results by subscale 122 Figure 3.3.3: Overall pre and post ITERS-R Results by subscale 123 Figure 3.4.1: IY Parenting Programme Participation, Apr ’15 to Dec ‘17 152 Figure 3.4.2: IY TCM Practitioner Participation, Jan ’15 to Dec ‘17 161 Figure 3.4.3: IY TCM participation breakdown by profession, Jan ’15 to Dec ‘17 162 Figure 3.5.1: Total family and children participation rates, Jan ’15 to Dec ‘17 176 Figure 3.5.2: Total practitioner participation rates, Jan ‘15 to Dec ‘17 176 Preface The Young Knocknaheeny Consortium, and later the Young Knocknaheeny Area Based Childhood Programme (YK), grew from previously existing collaborative relationships among community-based organisations and statutory agencies and services. It was grown by people active in the community, and people working in those organisations.