Final Evaluation of AWASAR Project
CARE Nepal-NURTURE Programme, Nepal
Submitted to:
CARE Nepal
Dhobighat, Lalitpur, Nepal
Submitted by:
Dr. Ritu Prasad Gartoulla on behalf of Evaluation team
Research Centre for Integrated Development (RECID) Nepal
2nd floor, House No. 543, Block No. 2 GA
Kathmandu-7, Chabahil, Chhuchepati
January 2019
ACKNOWLEDGEMENTS
RECID Nepal is grateful to CARE Nepal for offering the opportunity to carry out the final evaluation of AWASAR project. The evaluation team would like to express sincere gratitude to Mr. Jagadishwor Ghimire (Program Coordinator, Health), Mr. Ram Thagunna (Project Manager, AWASAR), Mr. Yogesh Chapagain (Procurement Associate), Mr. Binaya Shrestha from finance, Project Coordinators and other relevant officials associated with the AWASAR project for their support in finalizing the evaluation tools and for their valuable inputs to improve the final evaluation report. Special gratitude goes to the implementing partners of CARE Nepal, RRDC-Mugu and GIFT-Bajura, for their support in coordinating the fieldwork.
The evaluation team would like to express heartfelt appreciations to a wide range of individuals who supported the evaluation. We are grateful to the different stakeholders from Mugu and Bajura districts including District Public Health Office, District Education Office, District Agriculture Development Office, Civil society organizations, women farmers' groups, school children, cooperatives, community-based organizations, local level government (Rural Municipalities) and (former) VDC Secretaries who have contributed to make the study successful in different ways.
Special gratitude goes to entire participants and respondents including the migrated people and children of Mugu and Bajura Districts for their patience, cooperation and valuable time by sharing their insights, experiences and providing information to us to be incorporated in the final report. We express our deep appreciation to all the district supervisors and enumerators for the hard work and timely completion of field work.
Dr. Ritu Prasad Gartoulla on behalf of the evaluation team
Research Centre for Integrated Development (RECID) Nepal
January 2019
i
TABLE OF CONTENTS
Acknowledgements ...... i
Table of Contents ...... ii
List of Tables and Figures ...... vi
List of abbreviations ...... viii
Executive Summary ...... x
CHAPTER I ...... 1
BACKGROUND AND EVALUATION OBJECTIVES ...... 1
1.1 Background ...... 1
1.2 Brief description of the project ...... 1
1.3 Objectives of the evaluation ...... 2
1.4 Structure of the report...... 2
CHAPTER II ...... 3
EVALUATION METHODOLOGY ...... 3
2.1 Scope and areas of evaluation ...... 3
2.2 Study approach ...... 4
2.3 Study Sites...... 4
2.3 Study participants ...... 5
2.4 Study methods ...... 5
2.4.1 Document (literature) review/ secondary data analysis ...... 6
2.4.2 Household Survey ...... 6
2.4.3 Focus group discussion ...... 6
2.4.4 Key informant interviews (KIIs) ...... 7
2.4.5 Institutional assessment (Health facility, agriculture and livestock service center) ...... 7
2.4.6 Observation and case study ...... 7 ii
2.5 Sampling ...... 7
2.6 Research ethics ...... 9
2.7 Quality Assurance ...... 10
2.8 Limitation of the evaluation ...... 10
CHAPTER III ...... 11
FINDINGS ON MAJOR PROJECT ACTIVITIES ...... 11
3.1 Improved access to flexible and better-quality education for seasonal migrant children...... 11
3.2 Improved nutrition and food security of families in the target districts ...... 13
3.3 Strengthened links between communities and service providers to ensure quality services ...... 15
CHAPTER IV...... 17
FINDINGS of the EVALUATION STUDY ...... 17
4.1 Socio-demographic characteristics of the respondents ...... 17
4.1.1 General socio-demographics ...... 17
4.1.2 Household temporary migration pattern ...... 18
4.2 Improved education of the children ...... 18
4.2.1 Regularity in attending school ...... 19
4.2.2 Attitude of parents towards education of children ...... 21
4.2.3 School facilities upgraded to improve access ...... 22
4.2.4 Support for formal and Non-formal education program ...... 23
4.3 Food security ...... 24
4.3.1 Source of income ...... 24
4.3.2 Agricultural practice ...... 25
4.3.3 Involvement in agriculture related groups ...... 28
4.3.4 Source of technical advice and information related to agriculture ...... 29
4.4 Increased awareness about nutrition and healthy food habits in community...... 30
4.4.1 Food items consumed during last 1 week ...... 30
4.4.2 Food servings to pregnant and lactating women ...... 32
iii
4.5 Strengthened links between communities and service providers ...... 33
4.5.1 Capacity building of services/CBOs/NGOs ...... 33
4.5.1 Increased service utilization ...... 34
Maternal health service utilization ...... 35
CHAPTER V ...... 37
DISCUSSION AND ANALYSIS ...... 37
5.1 Adequacy and relevance ...... 37
5.2 Effectiveness (Where the right things done) ...... 38
5.3 Management Efficiency (Were things done right?) ...... 39
5.4 Outcome and Impact ...... 40
5.5 Accountability and transparency ...... 41
5.6 Sustainability ...... 41
5.7 Gaps and challenges of the project ...... 42
5.8 Best exercises ...... 44
5.9 Gender Equality and Social Inclusion ...... 44
CHAPTER VI...... 45
LESSON LEARNED, RECOMMENDATIONS AND CONCLUSIONS ...... 45
6.1 Lessons learned ...... 45
6.2 Conclusions ...... 46
6.2.1 General conclusion ...... 46
6.2.2 Overall conclusion of the evaluation ...... 47
6.3 Recommendations ...... 49
Annexes ...... 50
Annex I. Evaluation methodology summary ...... 50
Annex II: Achievement of project log-frame indicators ...... 52
Annex III: Additional results from household survey ...... 53
iv
Annex IV: Evaluation team...... 56
Annex V. Secondary data from DEO, DAO and Health Facilities ...... 57
Annex VI. Evaluation tools ...... 58
Annex VII. List of persons contacted for KII...... 59
Annex VIII: Household survey database ...... 60
Annex IX: Detail qualitative data (transcribes and analysis) ...... 61
Annex X: Successful Case Studies ...... 62
v
LIST OF TABLES AND FIGURES
List of tables
Table 1. Sample summary...... 9
Table 2. Progress of activities supporting formal and non-formal education program ...... 12
Table 3. Progress of activities in relation to improving and increasing the production of nutritious food ...... 13
Table 4. Progress on increased awareness about nutrition and health food habits ...... 14
Table 5. Socio-demographic characteristics of the respondents ...... 17
Table 6. Temporary migration ...... 18
Table 7. Children’s regularity to school ...... 19
Table 8. Attitude towards children’s education ...... 22
Table 9. Agricultural practice ...... 26
Table 10. Main crops in the study area ...... 26
Table 11. Involvement in agricultural groups ...... 29
Table 12. Source of agricultural advice ...... 29
Table 13. Food items consumed during last 1 week ...... 30
Table 14. Food servings to pregnant and lactating women ...... 32
Table 15. Service utilization ...... 35
Table 16. ANC and delivery care utilization ...... 35
Table 17. Gaps and challenges in relation to education, agriculture and nutrition programs .. 43
Table 18: The summary and overall conclusion of project evaluation ...... 47
Table 19. Sample size by study district and (former) VDCs ...... 53
Table 20. Drinking water ...... 53
Table 21. Sanitation facilities ...... 54
Table 22. Hand washing with soap and water ...... 54
vi
Table 23. Method of household waste disposal ...... 55
List of figures
Figure 1. AWASAR Project implemented VDCs of Mugu District ...... 4
Figure 2. AWASAR Project implemented VDCs of Bajura District...... 5
Figure 3. Progress of school facilities ...... 11
Figure 4. Progress on strengthening capacity of community-based organizations ...... 16
Figure 5. School days (per year) lost by children due to migration ...... 21
Figure 6. Sources of HH income (Multiple responses)...... 24
Figure 7. Annual income and expenditure of households...... 25
Figure 8. Food sufficiency by district ...... 27
Figure 9. Trend of agricultural production in recent years ...... 28
Figure 10. PNC visit ...... 36
Figure 11: Overall evaluation of the project in five evaluation criteria ...... 48
vii
LIST OF ABBREVIATIONS
ANC: Antenatal care
CSB: Community Score Board
CSO: Civil Society Organization
DADO: District Agriculture Development Office
DDC: District Development Committee
DEO: District Education Office
DAO: District Agriculture Office
DHO: District Health Office
ECA: Extra Curricular Activities
FCHV: Female Community Health Volunteer
FGD: Focus Group Discussion
GoN: Government of Nepal
HF: Health Facility
HH: Household
I/NGO: International/Non-Governmental Organizations
KII: Key Informant Interview
NGO: Non-Governmental Organizations
NTFP: Non-timber Forest Product
PD: Positive Deviance
PE: Peer Educators
PNC: Postnatal Care
PTA: Parent Teacher Association
RECID: Research Center for Integrated Development
RM: Rural Municipality
viii
RRDC: Reconstruction and Research Development Centre
SATH: Self-Applied Technique for Health
SD: Standard Deviation
SMC: School Management Committee
SIP: School Improvement Plan
ToR: Terms of Reference
VDC: Village Development Committee
WASH: Water, Sanitation and Hygiene
ix
EXECUTIVE SUMMARY
Introduction
CARE Nepal, in partnership with local partners Reconstruction and Research Development Center (RRDC) in Mugu and Generating Income for Foster Transformation (GIFT) in Bajura, implemented a 3-year DANIDA-funded AWASAR project from January 2016 to December 2018. The project aimed to reach the unreached children and their families in 10 former VDCs of Bajura (located in three Rural Municipalities) and 10 former VDCs of Mugu (located in five Rural Municipalities) districts. This project supported hard to reach mountain children and their families to improve their educational and food security status of hard to reach children and their families. The project was designed to meet their basic needs in education and food security and equip community organizations for strengthened service delivery. This report is the summary of the evaluation conducted by Research Centre for Integrated Development (RECID/N) Nepal to assess the effectiveness of project strategies and interventions in achieving the desired outcomes and outputs,
Evaluation methodology
The evaluation employed descriptive cross-sectional study design using both qualitative and quantitative methods. Study was conducted in December 2018 and the reports was finalized in January 2019 in two Rural Municipalities (RMs) in Mugu and three RMs in Bajura covering at least 19 of the original 20 VDCs in two districts. The study method primarily consisted of a desk review of available literatures and project documents, household questionnaire survey, interaction with project/partner staff, key informant interviews and focus group discussions. A total of 389 questionnaires were filled with head of household or adult member of the household and 87 women having children below two years of age. The qualitative study included 12 focus group discussions and 35 key informant interviews (with project personnel, health workers, school personnel, school children, RMs, district level government stakeholders, local leaders, FCHV, and social mobilizer). Project documents, including the periodic reports of district level stakeholders in agriculture, health and education were also reviewed. Quantitative data were analysed using descriptive statistical tools by SPSS 20. Qualitative findings were primarily analysed by using thematic analysis, and conclusions were drawn about the outcomes of the project. Findings are compared with the baseline study conducted in December 2016.
x
Major findings of the study
One-third (32.4%) of the households have family members who are involved in temporary migration, which is fore frequently observed in Mugu.
Improved education
Project supported different activities in the school such as construction of gender friendly toilets, refurbishment of furniture, solar lights, sports materials and mini library support to improve the environment in the school. Scholarships to the disadvantaged and needy students helped to retain the students from the poor family in the school. Further, addressing the needs of migrant students helped to retain students. Household survey showed that the mean number of days lost at school in a year was 70.9 (SD=43.3) days, which was 85.2 (SD=69.5) in baseline. Proportion of households with children who regularly attend schools has been increased from 45.2% at baseline to 65.7% during the final evaluation. Support to formal and nor-formal education program resulted in awareness of community leaders, parents on need of education and helped school enrollment and retention of school going children in the community. The evaluation showed that parents attitude towards education of children improved in the end line compared to the baseline.
Improved food security
Various activities were conducted to increase the food security in the project intervention districts. Only small fractions of land in these two districts are cultivable and furthermore, due to lack of modern techniques, crop yield and food security was very poor. Also, community awareness activities ensured people understand the importance of nutritious food and that of kitchen garden in nutrition.
Around 81% of the respondents mentioned agriculture as the major source of the household income. There was a reduction on number of households relying on agriculture and livestock with consequent increment in other sources such as business, service and foreign employment. Less than one in six households (14.2%) had food sufficiency for the whole year. However, this figure has increased from the baseline value, which stood at 10.4%. Number of households which did not had food security even for three months reduced by half from 38% at baseline to 19% at end line. About 13.1% respondents reported increasing trend of crop production in recent years; an improvement from 9.2% in the baseline. Households consuming food items from four or more food groups (food diversity) was increased from
xi
60.1% to 84.1%. The average number of food servings provided to the pregnant women in 24 hours preceding the survey increased from 2.9 (SD ± 0.7) at baseline to 3.5 (SD±1.3) at end line.
Strengthened service delivery
The project supported in capacity building of government service delivery mechanisms as well as Civil Society Organizations (CSOs) NGOs in the project area. NGO partner staff were trained on leadership, case-story writing and local level planning process. Project trained the members of CSOs (Dalit, Janajati and members of women federations) on leadership, advocacy and local level planning process based on local government operation guideline 2074. Community Score Board (CSB) approach initiated by the project helped to make the service delivery mechanisms more accountable and transparent. These efforts have fostered active participation of disadvantaged groups in community level interventions and more responsive service delivery. The household survey showed that visits to Agriculture Service Center and Livestock Service Center increased markedly at end line in comparison to baseline. Improvements are observed in maternal health service utilization pattern.
Conclusion
AWASAR project succeeded in achieving the objectives of the project. The project was able to implement its activities in a timely manner and met the target interventions. The project has made an effort to improve overall food security and educational status of communities and children in the project intervention area. Capacity building efforts to local level government and civil society organizations improved service delivery. The efforts made by CARE Nepal and implementing partners together with community beneficiaries largely resulted to the successful completion of the project achieving its aims and targets and delivering benefits to the targeted groups as envisioned by the project. The evaluation team believe that the learning from the project could be an important asset for CARE Nepal and other relevant organizations to further improve the implementation aspects in similar other project endeavors in the future.
xii
CHAPTER I
BACKGROUND AND EVALUATION OBJECTIVES
1.1 Background
The mountain children and their families in the Far Western and Western mountains of Nepal are the poorest people in Nepal with unacceptably low access to quality education and nutritional status among the children. Improving basic education, nutrition and health plays a key role in reducing poverty and meeting the Sustainable Development Goals. Ensuring the access to education and food security for the children is also crucial enabling them to enjoy their right to a better life. CARE Nepal implemented “AWASAR Project” in Bajura and Mugu districts of Far and Mid-Western hills of Nepal with the aim of improving the nutritional and educational status of hard to reach children in the project areas.
1.2 Brief description of the project
AWASAR project was a 3-year DANIDA-funded project running from July 2015 to December 2018 in Mugu and Bajura districts of Nepal. This project supported mountain children and their families to improve the nutritional and educational status of children. The project was designed to meet their basic needs in education and food security and equip their organisations with the tools for claiming the progressive realization of human rights. The project was implemented by local partners Reconstruction and Research Development Center (RRDC) in Mugu and Generating Income for Foster Transformation (GIFT) in Bajura. The project aimed to reach the unreached children and their families in 10 former VDCs of Bajura (located in three Rural Municipalities) and 10 former VDCs of Mugu (located in five Rural Municipalities) districts.
The overall objective of the AWASAR project was to improve the nutritional and educational status of hard to reach children in the project area. Following were the specific objectives:
Improve access to flexible and quality education for seasonal migrant children (high hills to lower elevation).
Improve nutrition and food security of families in the target districts.
1
Strengthen links between communities and service providers to ensure quality services.
Major areas of program intervention were education, food security and nutrition and health promotion.
1.3 Objectives of the evaluation
The overall objective of the evaluation was to conduct project level final evaluation to assess the outcomes and impact of project. The evaluation assessed project achievements at the end of the implementation period. The evaluation has also identified good practices, problems and challenges faced during implementation and how they were overcome and addressed. The survey measured and updated the overall achievements of the project based on log frame indicators (for specific objective indicators and result indicators), which were compared with the baseline survey conducted in 2016.
By finding and triangulating project's inputs, outputs and outcomes the study explored how the projects had maintained relevance, effectiveness, efficiency, sustainability and transparency, value for money, whether the project represents aforesaid things or not. Additionally, the evaluation team explored if the approach had been gender and social inclusive and accountable for people of the project areas. The overall final assessment of the project exercise focused more on how the project, its activities, engagement and approach contributed to education, agriculture/livelihood, health and nutrition with capacity enhancement part of local government establishing good rapport of service seekers and service providers through accountability tools.
1.4 Structure of the report
This report is divided into six Chapters. Chapter 1 provides background and evaluation objectives, Chapter 2 include evaluation methodology, Chapter 3 comprises findings summarised from project progress reports and Chapter 4 presents results of quantitative and qualitative study. Discussion in relation to evaluation criteria is presented in Chapter 5. Finally, Chapter 6 includes lessons learned, recommendations and conclusion, which is followed by annexes.
2
CHAPTER II
EVALUATION METHODOLOGY
This chapter describes the approach, methodology and processes adopted while carrying out the evaluation study. Prior to implementing end line evaluation, CARE Nepal engaged the services of local research firm, Research Centre for Integrated Development, Nepal (RECID/N) to carry out the field research required for the end line study. Systematic steps were adopted to conduct the end line survey of country program of CARE Nepal as the standard procedure. Two Districts Supervisors were assigned in the respective districts. Prior engaging them, training was provided on 19 December 2019 in relation to study methodology and rendering them acquainted with the questionnaire, importance of voluntary participation, research ethics and entire data collection process. Fieldwork of the evaluation started immediately after training to Supervisors and Enumerators. Most of the KIIs and FGDs were conducted by District Supervisors, while household (HH) survey was conducted by trained data Enumerators.
2.1 Scope and areas of evaluation
The evaluation team collected, analyzed and reported different information at the household, community, district and central levels. Following information were collected, analyzed and reported at different levels to find adequate information relevant with the requirements of the project.
Socio-economic (including demographic) and migration status of the households Educational status of the school going children of the household Existing non-formal education in the project area Food security and food sufficiency Sources of livelihood among migrant households Average annual income (farm and off-farm products, including livestock) Extent of agricultural/vegetables and goat production Farmers utilizing the services of service providers Farmers groups affiliated into DLSO/DADO and able to tap their resources
3
2.2 Study approach
Study approaches to conduct the final evaluation were both quantitative and qualitative. Quantitative study established the end line value of the project indicators. The valuation also collected qualitative information based on participatory framework, with the participation of local communities of different strata, ethnicity, gender and communities as well as various interest groups and concerned stakeholders (I/NGOs, CSOs, Government Organizations, communities, private sectors, etc).
2.3 Study Sites
The field work for the evaluation was carried out in 2 Rural Municipalities (RMs) in Mugu and 3 RMs in Bajura covering 19 of the original 20 VDCs in two districts. Within the sample households, head of household or any adult member of the households was targeted. Figure 1 and Figure 2 shows the Mugu and Bajura district map highlighting the AWASAR project implemented VDCs.
Figure 1. AWASAR Project implemented VDCs of Mugu District
4
Figure 2. AWASAR Project implemented VDCs of Bajura District
2.3 Study participants
The project targeted three different types of beneficiaries: school and school going children, members of mothers’ groups and members of farmer’s groups. The evaluation primarily included methods and tools targeting these three types of beneficiaries. In addition to the target beneficiaries, the study also included methods to collect data from project staff and relevant stakeholders.
2.4 Study methods
The evaluation information was collected as such that gathered information supported analysis of all relevant indicators and cross cutting issues as defined in the project log frame
5
and indicated in project documents. The evaluation employed both qualitative and quantitative methods for data collection and included review of secondary information. The evaluation utilized a wide range of appropriate tools and methodologies to measure status of social, economic, physical, natural, educational and human assets of targeted communities/households within the scope of AWASAR project and as spelled out in the project document. The study methods specifically consisted of questionnaire survey at household level as well as desk review of available literatures and documents, key informant interview (KII), focus group discussion (FGD), and observation. Please find detailed attached in Annex 1 for Summary of the evaluation methodology.
2.4.1 Document (literature) review/ secondary data analysis
The study team gathered and reviewed the related documents (secondary information) from different sources. The project proposals, project documents and project log frame, baseline report, project progress reports, project factsheets, training reports, case/success stories, were reviewed. Secondary information was collected from district/Rural Municipality profiles. Other relevant studies and literature were used as reference from published and unpublished reports and documents.
2.4.2 Household Survey
Questionnaire survey at household level was conducted among representative samples from the project districts. The questionnaire surveys were administered by trained enumerators via face to face interview. The questionnaire was developed in English and translated in Nepali for final use. The translated questionnaire was pre-tested and improved. Quantitative indicators of the project outputs and outcomes were generated by household questionnaire survey such as household socio- demography, service (health, education, agriculture) utilization pattern, education of the children, migration status, sources of livelihood of the migrant and non-migrant households, food security, household food habits, WASH, etc.
2.4.3 Focus group discussion
Focus group discussions (FGDs) were conducted with members of women’s groups, farmer’s groups and school children. Two FGDs in each with women’s groups, farmer’s groups and school children in each of the two districts (a total of 12 FGDs: 7 in Bajura and 5 in Mugu) were carried out.
6
2.4.4 Key informant interviews (KIIs)
Key informant interviews (KII) was carried out with persons from related authorities and stakeholders which included VDC secretaries, members of farmers group, school authorities, extension agents, lead farmers, community groups, FCHVs, service providers (resource centers), District Health Office, Education Office, Agricultural Office, and Rural Municipality authorities. KII checklist was prepared following the project log frame and administered by the consultant(s) individually. Altogether around 30 KIIs (15 per district) were conducted.
2.4.5 Institutional assessment (Health facility, agriculture and livestock service center)
Major indicators associated with project indicators around health, education and nutrition was collected from Health facility, District Agricultural and Education Office and Agriculture Service Centers.
2.4.6 Observation and case study
Field observation was conducted to observe the actual situation of the client and assesses information related to livelihood practices, income and employment related information. Critical cases were identified and developed reflecting success or challenges in attaining the project objectives. Cases with positive changes owing to the project interventions were also considered. Case studies were recorded by observation and in-depth interviews and later transcribed/edited. Three case studies from each of the study district was collected and incorporated in the report.
The tools for evaluation study were developed based on the baseline tools and review of project documents. The evaluation methodology, including the study sites and evaluation tools were finalized together with the CARE project team from CARE Nepal.
2.5 Sampling
Household survey was conducted among the representative samples of the list of primary beneficiaries of the project (i.e. sampling frame for the study has been taken as the beneficiary list). Household survey was carried out in 9 VDCS of Bajura District. Sample size for household survey has been calculated applying population survey formula as below:
Sample size (n) = z2 × pq
d2+ z2pq/N
7
Where, n = required sample size z =1.96 for 95% confidence level p = prevalence of main outcome indicators d = precision or error allowed in the study = 0.05 q = (1-p) = 1-0.739 = 0.261
N= total study population (2800, as the ToR mentioned that around 2800 primary level children will be benefitted from the project interventions)
Sample size (n) = (1.96)2 × 0.739 × 0.261 / (0.05)2 + (1.96) 2 x 0.739 × 0.261 / 2800 =278
Taking the design effect of 1.3, the sample size will be 278 X 1.3 = 361
Taking non-response rate as 5% the total sample size will be 361 + 361 X 5% = 380.
The samples were collected from respective VDCs of both Districts. If the respondents in the selected household could not be contacted, the household was replaced by immediate next household in the VDC. One eligible adult (preferable head of household) in each household was selected for an interview. Women with children below two years of age, if present in the sample household, were also interviewed.
Fieldwork was completed within a week, from 2075/9/5 and 2075/9/12. The summaries of data collected with dates of completion from various stakeholders are presented in
8
Table 1.
9
Table 1. Sample summary
Study technique Stakeholders Mugu Bajura Total School children 2 2 4 FGD Women’s group 1 3 4 Farmer’s group 2 2 4 Nutrition focal person of 1 1 2 DPHO Focal person of DPHO 1 1 2 Focal person of DEO 1 1 2 Focal person of Agriculture Service 1 1 2 Centre KII Focal person of District 1 1 2 Agriculture Office VDC Secretary or Ward 10 10 20 Chief FCHV 1 1 2 Teacher 2 1 3 Case studies 3 3 6 Observation checklist Local level – HP/PHC 2 1 3 health facility District PHO 1 1 2 DEO 1 1 2 DPHO 1 1 2 Secondary Data Review Agriculture Service center 1 1 2 DAO 1 1 2 Total HH Survey Household 221 168 389
2.6 Research ethics
Research teams involved in the study are committed to respect the ethical principles of the research. The session on research ethics was included during training to Field Supervisors Enumerators responsible for data collection. RECID/N provided all data collectors with consent forms to be provided to eligible participants prior to starting any interview. Oral informed consent was ensured before each interview. Consent forms were incorporated as part of each tool included in the annexes. Privacy and confidentiality surrounding the participants and data collectors involved in the study, in addition, the careful management of data being collected was maintained.
10
2.7 Quality Assurance
Efforts were made to improve the validity and reliability of the study. During the design phase, sample was selected proportionately from sampled (previous) VDC, to identify the respondents in case of household survey. The questionnaire was developed and finalized in close consultation with CARE Nepal. Supervisors and enumerators were adequately trained for data collection. During field work phase, Enumerators were closely supervised during data collection by field supervisors as well as core study team. Any error or confusion in data collection was recorded and was resolved early. During the data analysis, filter errors, appropriate coding of non-response and missing values, data cleaning, out-of- range values, and other logical check and balance were performed.
2.8 Limitation of the evaluation
Time constraint and snow fall (winter season) in the study area hindered the movement for data collection. The findings and analysis presented in this report are largely based on the discussions and observations made in the field study. The evaluation did not attempt to assess ‘Management efficiency’ of the project in detail.
11
CHAPTER III
FINDINGS ON MAJOR PROJECT ACTIVITIES
This chapter presents the progress of the activities associated with the project outputs up to October 2018. Most of the findings presented in this chapter resulted from the review of the mid-term and annual progress reports of the project.
3.1 Improved access to flexible and better-quality education for seasonal migrant children
Output 3.1.1: School facilities upgraded to improve access
The project supported to improve the available facilities at the school with activities such as construction of toilet, basic furniture (including floor setting) support, installation of solar lamps and cookers, computer and learning material support, mini library support, sports material support, project board installation, etc. Most of the targets were met. While white board and computer set installation achieved in more than the set target, WASH (toilet and water supply) package support in 15 schools of target project area was not provided at all. Students and teachers of both districts benefited from these activities.
Total Target Total achieved % achieved
800 750
700
600
500
400
300
200 100 103 100 100 100 100 100 75 30 30 30 31 30 30 30 30 30 30 30 30 30 15 2 4 3 0 0 0 Construct Support Install solar Mini library Sports Project White board Computer WASH separate schools with lamps and support material Board and and learning Package toilets and floor setting cookers support installation computer set material support in 15 water installation support in School of facilities Gadekhola project area
Figure 3. Progress of school facilities
12
Output 3.1.2: Support for formal and Non-formal education program support
Majority of the activities to support for formal and nor-formal education program had achieved the set target, except for activities, such as formulation and approval of accelerated learning curriculum, fully literate VDC and district declaration campaign, providing training to teachers on child centric teaching learning methods, monitoring visits from local and district stakeholders.
Table 2. Progress of activities supporting formal and non-formal education program
S. Activities Unit Tar Achi % N. get eved achiev ed 1 Accelerated learning curriculum formulated and approved No 1 0 0 2 Orientation to the adult literacy facilitators (3 days) Facilita 19 19 100 tors 3 Support and facilitate adult literacy class in two VDCs VDC 2 2 100 (Bajura and Mugu) 4 Fully literate VDC and district declaration campaign VDC 2 2 100 5 Provide training for teachers on child centric teaching Teache 88 83 94 learning methods rs 6 Orientation to the traditional healers to support formal and VDCs 20 20 100 non-formal education 7 Follow up Orientation with traditional healers focusing on Events 20 20 100 Education, Livelihood & Health/Nutrition. 8 Orientation to School management committee and PTA to SMC & 30 30 100 increase enrolment and continuity to school PTA 9 Felicitation of teachers/students/parents as an School 3 3 100 acknowledgement of their contribution of active participation in ensuring good quality education and enrolment campaign 10 Education support for poor and marginalized children. Student 450 525 117 s 11 Monitoring visit from Stakeholders (Education, Health, Stakeh 8 8 100 Agriculture, Rural /Municipalities representatives etc.) olders 12 Support Active teaching learning materials (ATML) to Schools 9 9 100 schools 13 Selection of Peer Educators by gathering mass in community PE 180 178 99 14 Orientation of Peer Educators (PEs) PEs 180 178 99 15 Refresher of PE PEs 180 176 98 16 Quarterly review meeting of PE PEs 130 130 100 17 Mobilization of local PEs PEs 180 180 100 18 Peer Educators facilitation cost monthly basis PEs 180 178 99 19 PE promotional materials (Cap, T-shirt and Bag) PEs 180 178 99
13
20 Stationery support to Health mother group (MG) and Peer MG & 180 180 100 Educators (PEs) PEs 21 30 child clubs oriented in child rights Schools 30 30 100 22 One day orientation to child club members Schools 30 30 100 23 Bi-monthly meeting of child club (CC) CC 90 90 100
3.2 Improved nutrition and food security of families in the target districts
Output 3.2.1: Improving and increasing the production of nutritious food
Entire activities in relation to improving and increasing the production of nutritious food had achieved its set target except production of project documentary and demonstrating on model irrigation water tank as shown in Table 3.
Table 3. Progress of activities in relation to improving and increasing the production of nutritious food
S.N Activities Unit Total Target Total % . Achieved Achie ved 1 Extension Agent (EA) and SM Persons 2 Trainings 55 100 trainings and 53 extension agents 2 Refresher training of extension agents Persons 40 / 2 event 40 / 2 100 and SM /district event /district 3 Training of 180 lead farmers (At least Lead farmer 180 180 100 50% female) 4 Refresher training of 180 lead farmers Lead farmer 180 168 93 5 Training of farmer group members FG Member 3000 3117 104 6 Introduce kitchen gardens on school Schools 30 30 100 grounds to teach children about agriculture and nutrition 7 Vegetable seed support for farmers Farmer 2000 3117 156 8 Model irrigation water tank Water tank 4 4 100 9 Establish demonstration plots and Plot 60 63 100 seeds; multiplication on farms (incl. re- introducing indigenous crops) 10 Training for farmers on demonstration Lead farmer 60 52 87 plot 11 Refresher Training for farmers on Lead farmer 60 60 100 demonstration plot 12 Exposure visit and overall learning and Event 2 2 100 documentation 13 Learning visit to Demonstration plot Farmer 60 52 87 by farmers Group
14
14 Production of IEC/BCC materials Flier IEC/BCC 6 4 67 (radio jingle, hoarding board and wall painting, flier) 15 Radio messages production and Radio 13 13 100 broadcasting Massage 16 Wall Painting and Hoarding Board Events 13 13 100 17 Street Drama focusing on awareness Events 20 18 90 regarding health, education and livelihood. 18 Prepare Project Documentary Events 2 2 100 19 Training to Teacher and students for Events 15 15 100 Kitchen garden at supported schools 20 Training to Smart Package Community Events 78 75 96 in Karmashi, Ama and Bumnidika 21 Agriculture equipment package Household 78 75 96 support to 60 farmers 22 Follow up refresher of farmer group Farmers 2000 2092 105 member during seed distribution 24 Agriculture equipment support to Farmers 41 82 200 farmers and School Output 3.2.2: Increased awareness about nutrition and healthy food habits in the community
As shown in Table 4 regarding the progress on increased awareness about nutrition and health food habits, majority of the intended activities had met the set target except providing equipment support to health facilities as per need.
Table 4. Progress on increased awareness about nutrition and health food habits
S.N. Activities Unit Total Total % target achieved achieved 1 Training for female community health FCHV 180 180 100 volunteers (FCHV) on (Health and Nutrition, BEHAVE, SATHE, PD HEARTH) 2 Refresher training for FCHVs FCHV 186 184 99 3 Bag support to the FCHVs 186 186 100 4 Support on Full Immunization VDC VDC 20 12 60 declaration campaign 5 Cultural programs Events 20 20 100 6 FCHVs facilitate sessions in mothers’ groups FCHV 90 90 100 (Health and Nutrition, BEHAVE, SATH, PD session Hearth) (Target number -20/60 VDC/events for FY17) 7 Sensitization of parents for child education, VDCs 10 10 100 nutrition and health
15
8 Organize food day, sanitation and nutrition Schools 1 1 100 week activities in schools to increase awareness about nutrition and other health and nutrition related days celebration 9 Training of 180 mothers group (MG) MG 180 180 100 10 Training of 30 Schools Management SMC 30 30 100 Committee (SMC) 11 PD Hearth training and demonstrations VDC 50 40 80 12 MUAC tape and weighing machine for DHO & 3 3 100 Health mother group HP 13 Weighing machine for health facilities and DHO & 20 20 100 district HP 14 Equipment support to HFs as per need DHO & 20 23 115 HP 15 On site coaching with mother group on Events 180 180 100 Health & Sanitation 16 Training of 30 SMCs about Health, Hygiene Events 30 30 100 & Sanitation 17 Health Equipment Support to health HP 13 13 100 facilities/birthing centres 18 Equipment support to health facilities as per Health 60 60 100 need (Stretcher Support) Mothers group
3.3 Strengthened links between communities and service providers to ensure quality services
Output 3.3.1: Strengthened capacity of local NGOs and indigenous peoples' organizations to raise voices on rights
Support was provided to the 30 partner staffs on leadership, case story writing, photography and planning process of local body. Due to transitional phase of state reformation, this event was not carried out for Civil Society Organizations (CSOs) in 2017. In 2018, training was provided to the CSOs members on leadership, advocacy and local level planning process on the basis of local government operation guideline 2074.
16
Output 3.3.2: Strengthened capacity of community-based organizations to engage in local planning and monitoring
Entire activities in order to strengthen capacity of community-based organisations to engage in local planning and monitoring were completed as planned, with an exception of conducting wrap-up meeting at Rural/Municipality level, because this was yet to be thought out.
100 100 100 100 100 100 85 90 80 80 Total target 80 Total achieved 70 60 50 40 30 20 20 20 20 20 17 20 8 8 8 4 4 10 0 0 0 District level FGDs with FCHVs, Orientation of CSB Consultation with Follow up/review Coordination Wrap up meeting planning and mothers' groups process and service providers semiannual Meeting with at orientation of CSB and community outcome and analysis of meeting on CHSB Rural Rural/Municipality leaders indicators progress Municipalities and level Municipalities
Figure 4. Progress on strengthening capacity of community-based organizations
17
CHAPTER IV
FINDINGS OF THE EVALUATION STUDY
This chapter incorporates findings from both quantitative and qualitative study. The study was centered on exploring the situation of the project with respect to the three expected outcomes of the project. This chapter is divided into three major sections which subsequently describe the findings under each of the expected outcomes of the project, as identified in log- frame matrix: Improved education, improved food security and Strengthened service delivery. First the general socio-demographic characteristics of the sampled households are described which is followed by findings related to each of the project components. Findings from household survey are compared against baseline data from baseline report provided by CARE Nepal. Qualitative information is incorporated in parallel with the quantitative findings as relevant.
4.1 Socio-demographic characteristics of the respondents
4.1.1 General socio-demographics
More than half (56.8%) of the household survey samples were from Mugu district. The male respondents (68.9%) outnumbered the female respondents (31.1%) participated in the survey. Majority of the respondents (57.8%) belonged to the ‘Brahmin/Chhetri/Thakuri’ ethnic group, with almost equal proportion (21%) of ‘indigenous’ and ‘Dalits’.
Table 5. Socio-demographic characteristics of the respondents
Mugu Bajura Total
n % n % n %
Sample size 221 56.8 168 43.2 389 100.0
Sex of the respondent Female 52 23.5 69 41.1 121 31.1 Male 169 76.5 99 58.9 268 68.9
Ethnicity
Brahmin/Chhetri/Thakuri 86 38.9 139 82.7 225 57.8 Indigenous (Janajati) 78 35.3 2 1.2 80 20.6
Dalit 57 25.8 27 16.1 84 21.6 Age of the respondent mean 40.8 (12.1) 41.5 (12.2) 41.1 (12.1) (SD) The mean age of the respondents was found to be 41.1 years (range: 18-77 years, SD: 12.1 years).
18
4.1.2 Household temporary migration pattern
Out of total 389 households surveyed, one-third (32.4%) of the households have family members who are involved in temporary migration. Such type of migration was more frequently observed in Mugu (53.4%) compared to Bajura (4.8%).
Table 6. Temporary migration
End line Baseline
n % n % Temporary migration of family members n=389 n=344 Yes 126 32.4 166 48.3 No 263 67.6 178 51.7 Reasons for migration (Multiple response) n=126 n=166 Escape from cold 68 54.0 127 76.5 Tradition 45 35.7 101 60.8 Livestock rearing 58 46.0 45 27.1 Business purpose 23 18.3 29 17.5 Other (e.g. herbs collection, study, employment, 14 11.1 97 58.4 etc.) Temporary migration of children n=126 N=166 Yes 97 77.0 142 89.3 No 29 23.0 17 10.7
The major reasons for temporary migration included escape from the cold (54.0%) and livestock rearing (46.0%). Around 36% of the respondents also mentioned ‘tradition’ as the reason behind migration. People in the study area have long tradition of migrating low-land during winter together with their livestock. Children also migrate together with the household members in 77% of the migrant households.
4.2 Improved education of the children
One of the reasons behind poor school attendance is the lack of gender friendly toilets with proper hygiene in the school. Lack of separate toilets is a de-motivating and preventive factor for girls to go to school as they feel humiliated and teased by their counterparts. CARE Nepal, through AWASAR project implemented various school activities including construction of separate toilets at school premises with an aim to encourage girls to attend school. The project also provided scholarships to the underserved child, and marginalized children.
It was found that the project met the required objectives. Scholarships provided to the students helped to improve the school attendance. Furthermore, gender friendly toilets helped
19
children retain in the school. Specially, the female students were more comfortable to join the school. A school teacher from Mugu district was well aware about the scholarships provided through the project. He admitted that education materials and computers had been provided to the school and scholarships to poor students. District Education Officer from Mugu believes that the scholarship schemes, construction of toilets and various awareness programs has helped to improve the quality of education in the district. He further emphasized on improving the collaboration from the project in the future.
“Education level of our district is medium. For last three years, we have started scholarship for poor and talented students. We have some programs like we celebrate children’s day and Education day. AWASAR also gets involved in those programs” (DEO Mugu)
In addition to above mentioned activities, informal education program, quiz contests were conducted in different occasions including celebration of the national education day. Admission campaign was also organized to increase the school enrollment among those children who were out of school during that period. All these activities in together, helped to increase school enrollment, retention and completion.
4.2.1 Regularity in attending school
Around two-third (65.7%) of the households mentioned that their children go to the school regularly. Only 3% of the households have children, who do not go to school at all. Table 7 shows that the proportion of households with children who regularly attended schools has been increased from 45.2% at baseline to 65.7% during the final evaluation. A decrement in the households with children ‘do not go to school’ at all has also observed.
The major reasons for not going to the school reported by the respondents include temporary migration of the children together with their parents (73.9%), children go to collect herbs (59.6%), agricultural work (21.3%), livestock rearing (30.3%), and children do not want to go school (25.0%), look after smaller children (16.9%) and others.
Table 7. Children’s regularity to school
End line Baseline
n % n % Children go to school n=268 n=330
Do not go to school 8 3.0 26 7.9
Go regularly 176 65.7 149 45.2
Do not go regularly 84 31.3 155 47.0
Reasons for not going school regularly n=92 n=178
20
(Multiple response) Do not want to go 23 25.0 57 32.0
Temporary migration of children 68 73.9 85 47.8
Collect herbs 53 59.6 45 25.3
Livestock rearing/grazing 27 30.3 33 18.5
Look after small children 15 16.9 30 16.9
Agricultural work 19 21.3 64 36.0
Water fetching 4 4.5 9 5.1
Other 2 2.2 8 4.5 School dropout in last 2 years Yes 23 8.6 n/a No 245 91.4 n/a Sex of the child dropped (n=23) Female 7 30.4 Male 5 21.7 Both 11 47.8 School enrollment rate n=533 Enrolled 512 96.1 Not enrolled 21 3.49 When asked whether any children dropped out from the school during last two years, 8.6% of the households had children dropped-out, with almost half of the households having both male and female child stopped going to the school.
School enrollment ratio in the end line survey (as per the household survey findings) stands at 96.1%, which is similar to a mini-survey conducted by the project which showed a marginal increase 1% (from 94.2% in 2016 and 95.5% in 2018) (Table 7).
According to the District Education Office (DEO) Mugu, total school enrollment rate was stagnant at 93% in 2016 and 2017, with a slight decrease to 91% in 2018. The total school enrollment rate was in increasing trend at in Bajura with 89%, 94% and 95% in the years 2016, 2017 and 2018 respectively.
Interviews conducted with different stakeholders also enforced that migration was a common scenario in these districts. Various interventions targeted towards migrating population helped to bring back these children into the school. DEO from Mugu revealed that students migrated for a period of 2-3 months; although migration within the district was routine, some of them even migrated out of the district. Schools in the district have provided special attention to the migrant students and provided extra classes for those students on their return to the school. Awareness raising activities were organized among the guardians to make them understand the importance of these extra classes.
21
“We give more attention to the migrant students and provide extra class when they come back to school. We also aware guardian of those migrant student to attend school regularly for extra class” (School Teacher, Mugu)
In addition, DEO from Bajura believed that project helped to address the needs of the migrant school children and address their needs towards education.
“This project helps us in very critical condition. We appreciate AWASAR project especially in the field of child education for migrant” (DEO, BAJURA)
The evaluation compared the average number of loss of school days for school going children due to seasonal migration. The mean number of days lost at school was 70.9 (SD=43.3) days, which was 85.2 (SD=69.5) in baseline.
30 days or less 30 to 60 days 60 to 90 days More than 90 days 100% 90% 23.7 25.0 80% 70% 60% 30.9 50% 55.0 40% 12.4 30% 20% 33.0 12.5 10% 7.5 0% End line Baseline
Figure 5. School days (per year) lost by children due to migration
Compared to the baseline, the proportion of children reporting less number of days lost (30 days or less) has been increased in the end line, which was largely because of decrease in ‘60 to 90 days’ from the baseline (Figure 5).
4.2.2 Attitude of parents towards education of children
Nine different statements were presented to the respondents and asked whether they agree or disagree to the statements (Table 8). The proportion of parents, who agreed on the statements ‘Children should go to school regularly’; ‘Girls also need education to contribute the
22
development of country’; ‘Son and Daughter are equally able to achieve quality education’; ‘Girls have intellectual ability to utilize the education they have gained’ and ‘Educated sons and educated daughters are equally important’ was more than 98%. Around 10% of the parents believed that there should be separate schools for sons and daughter and 15.5% mentioned that investment in girl’s education is waste of resources. The findings showed that parents attitude towards education of children was relatively positive.
Table 8. Attitude towards children’s education
End line Baseline Education affects the life of children. 76.1% 64.4% Son and Daughter are equally able to achieve quality 94.5% 94.4% education. There should be separate schools for sons and daughter. 9.5% 16.2% Children should go to school regularly. 98.9% 97.6% Educated sons and educated daughters are equally important. 99.2% 91.8% Girls have intellectual ability to utilize the education they 99.5% 93.2% have gained. Girls also can get job if they are educated. 98.4% 80.3% Investment in girl’s education is waste of resources. 15.5% 13.5% Girls also need education to contribute the development of 98.9% 97.1% country.
Table 8 also shows that the parents’ attitude towards children’s education has been improved in end line as compared to the baseline findings, with higher proportion of parents reporting ‘Education affects the life of children’, ‘Educated sons and daughters are equally important’, and ‘Girls also can get job if they are educated’.
4.2.3 School facilities upgraded to improve access
The infrastructures built by AWASAR Project, as shown in Figure 3, had been very expedient to increase the quality of education. The information, education and communication materials provided in school had helped children to adapt healthy behaviors and avoid unhealthy diets. School children had become more disciplined and the teachers have become very positive. Teaching and learning environment had become participatory due to use of picture, pamphlet and adopting demonstration technique. Games liked by boys and girls children, such as football, skipping and ring games had been available at school. These activities had led to motivate and encourage parents and children to continue schooling for children. This has resulted in increased admission rate and success rate in SEE examination. 23
“AWASAR provide Scholarship program to poor students, computer classes for better education and education material to promote learning process.” (School Teacher, Mugu)
4.2.4 Support for formal and Non-formal education program
Table 2 indicated that the majority of the activities to support for formal and nor-formal education program had achieved the set target, except for formulation and approval of accelerated learning curriculum, fully literate VDC and district declaration campaign, providing training to teachers on child centric teaching learning methods, monitoring visits from local and district stakeholders.
It was found that in Gaubhul and Swami Karthik VDCs in Bajura district, the project activities including informal education helped parents without any formal education to realize the importance of education and encouraged them to send their children to school. In addition, scholarships and free distribution of books and clothing helped families under the line of poverty to send their children to the school. The effect of these interventions was observed in the form of reduction in the absenteeism and dropout from the school.
“AWASAR project had been working since last three years to bring the changes in the field of education, health and nutrition and agriculture. It decreases the school absenteeism rate among students” (Secretary, Gaubul VDC-3, Bajura)
Suryadaya Adharbhut School in Bajura succeeded to increase the pass percentage of students due to arrangement of temporary school for seasonal students.
“This project brings training for teachers and child club for the improvement of their skills. For the control of migration among students, parents took care of their children and send them regularly at school.” (Teacher, Shree Suryadaya Adharbhut Bidhyalay, Bajura)
Not only the key stakeholders, Principle and school teachers but also the students from the project intervention areas had realized that the project has provided them opportunity to flourish their education.
“We love to attend school regularly because school provides education material, sports material and computer class.” (FGD-Student, Khatyad, Mugu)
24
4.3 Food security
4.3.1 Source of income
Around 81% of the respondents mentioned agriculture as the major source of the household income, followed by livestock (50.6%), labour/daily wages, herbs collection, foreign employment and others. When compared to the baseline figure, households relying on agriculture and livestock have been decreased with consequent increment in other sources such as business, service and foreign employment.
Endline Baseline 94.2%
80.7%
56.7% 50.6% 43.4% 43.7%
32.3% 29.8% 27.2% 25.9% 19.0% 13.1% 9.0% 9.3% 5.5% .5%
Agriculture Livestock Business Daily Service Herbs Foreign Other wage/labour collection employment
Figure 6. Sources of HH income (Multiple responses)
Figure 7 shows the annual income and expenditure of the families. More than one-third of the families were earning 2 to 5 lakhs NPR per year and 11.9% had income more than 5 lakhs. Majority of the households had expenditure between 1 to 5 lakhs.
25
Figure 7. Annual income and expenditure of households
4.3.2 Agricultural practice
Only 39% of field is appropriate for farming in Mugu. The condition is similar in another mountain district of Bajura. In addition, crop yield is poor with the traditional methods of farming. AWASAR project indeed provided opportunities the local communities through their various interventions and helped them to become agriculturally dependent. Distribution of the hybrid crops suitable for the high altitudes and capacity building on modern technologies has indeed increased crop production in these districts. A review of available secondary data provided by Agriculture Service Centres showed that in the past three years, there is a steady increase in production and productivity of indigenous and nutrient rich crops on farm plots. The production of such crops for the years 2016-18 was 40%, 43% and 56.5% respectively. Furthermore, 23% crops species was adopted by families in 2016 which increased to 33% in 2017 and 40% in 2018. An Agriculture service center in Mugu further emphasized that the collaboration with the project has helped people to grow agricultural crops and reduce food insecurity.
“Before AWASAR project local people did not know about modern farming technology. The project brought changes in farming technique. Nowadays, they produce seasonal vegetables and fruits as well as off-seasonal vegetables by using Green house and other modern technology” (Service provider, Agriculture Service Center, Mugu)
Training and capacity building on modern farming technology, use of hybrid crops and greenhouses resulted in better yield of the crops.
“We suffered from food crisis for certain months but AWASAR project provided training to us after that we are able to produce unseasonal vegetables and foods by using modern technique” (FGD- Farmers’ group, Kamarung, Mugu)
Survey also showed that there is increasing trend in food security, agricultural crop yield in the project areas from the baseline survey. Although the proportionate increment is low, it has established the base for improvement in the future.
Table 9 depicted that out of the total households surveyed, 95.4% had agricultural land and the remaining 4.6% did not have any agricultural land. Around 29% of the households reported to have 3 to 5 Ropani of land. The proportion of households operating own agricultural land by themselves was 93.7 %, with 1.3% not involved in agriculture.
26
Table 9. Agricultural practice
End line Baseline Availability of agricultural land
Yes 95.4% 91.6% No 4.6% 8.4% Size of land available
No land available 4.6% 8.4% 1 to 2 Ropani 19.3% 15.4% 3 to 5 Ropani 29.3% 34.3% 6 to 10 Ropani 27.8% 29.1% More than 10 Ropani 19.0% 12.5% Agricultural practice Self-farming on own land 93.7 % 91.3% Give land (Contract) to others 0.3% 2.0% Contract other's land 4.7% 4.4% No farming 1.3% 2.3% Food sufficiency from own farm land Yes 14.3% 10.4% No 85.7% 89.6% Number of food sufficient months Less than 3 months 19.0% 38.0% 3 to 6 months 32.6% 37.6% 6 to 9 months 34.4% 14.2% 9 to 12 months 8.7% 3.8% More than 12 months 14.2% 6.2% The proportion of households reporting food sufficiency around the year from their own production is quite low (14.2%), however the figure was an increment from the baseline value of 10.4% (Table 9). Around 19.0% of the households had food sufficient for only less than 3 months, which was decreased from 38% in the baseline.
Major crops producedTable 10 shows the major corps (cereals, vegetables and fruits) produced in the project area. Potato, wheat, barley, millet and maize are the main cereals planted by the households.
Table 10. Main crops in the study area
Winter Rainy season
Cereals Barley, wheat, potato, Fapar, Potato, barley, Chinu, Kaguni, fapar, maize, millet, pulses paddy, maize, beans, pluses, soybean, junelo Vegetables Cabbage, cauliflower, reddish, Cauliflower, cabbage, tomato, pumpkin, green leafy spinach, tomato coriander, vegetables, spinach, brinjal, onion, garlic, beans, chilly reddish, tomato, chilli, kangkung
27
Fruits Banana, lime, orange Apple, plum, Aaru, Naspati, Khanu, Chuli, guava, walnut, peach, cucumber, pumpkin, banana
Most of the KII and FGD participants agreed that the produced food from agriculture is not enough for the food need of family members for a whole year. In an average it last no more than six months. Animal husbandry, herbs collection, unskilled work (labor work) and foreign employment are other major sources of livelihood subsistence.
Figure 8 shows that food sufficient households are found higher in Mugu as compared to Bajura, which is consistent with the baseline findings. The improvement in the end line is largely represented by Bajura districts where the proportion increased from 6.6% to 10.2%.
17.4% 18.0% Endline Baseline 14.3%
10.2% 10.4%
6.6%
Mugu Bajura Total
Figure 8. Food sufficiency by district
Figure 9 shows that 13.1% respondents reported increasing trend of crop production in recent years; an improvement from 9.2% in the baseline. Consequently, lower proportion of households reported decreasing trend in crop production in recent years.
28
Increasing Same Decreasing 100% 90% 30.0 80% 40.1 70% 54.1 53.3 68.2 60% 75.2 50% 40% 62.2 46.6 26.3 30% 34.2 20% 22.6 16.8 10% 20.4 13.1 9.2 7.8 11.7 8.0 0% End line Baseline End line Baseline End line Baseline Total Mugu Bajura
Figure 9. Trend of agricultural production in recent years
Higher proportion of respondents from Bajura reported decreasing trend in agricultural production both in baseline and end line, though the overall improvements are also seen in Bajura.
Only 9% of the households (7.5% in Mugu and 10.8% in Bajura) reporting involving in cash crops for selling purpose, with major cash crops being potato, tomato, beans, spinach, millet and maize.
4.3.3 Involvement in agriculture related groups
More than half (51%) of the respondents or any of their family members were involved in group(s) related to agriculture. As opposed to the baseline figure, higher proportion of households from Bajura district reported involving in such groups compared to Mugu. Among them, around one-fourth (23.7%) of the respondents reported that the agricultural group they were involved in was not registered in any office/organization. While, more than half (60.6%) of the respondents mentioned the registration of the groups in District Agricultural Development Office. More than half (59.2%) of such agricultural groups were women’s groups.
29
Table 11. Involvement in agricultural groups
End-line Baseline
n % n % Involved in agricultural groups n=384 n=340
Yes 196 51.0 98 28.8
No 188 49.0 242 71.1
Type of agricultural groups n=196 n=98
Female 116 59.2 53 54.1
Male 5 2.6 8 8.2
Mixed 75 38.3 37 37.8
Is the group registered somewhere? n=196 n=98
No 46 23.7 22 22.4
Yes, in Agriculture Development Office 132 60.6 51 52.0
Yes, in cooperative 31 16.0 4 4.1
Yes, in Gharelu office 2 1.0 2 2.0
Yes, other 6 3.1 0 0.0
Don't know 1 0.5 19 19.4
4.3.4 Source of technical advice and information related to agriculture
There was a significant improvement in households receiving agriculture related information and advice compared to the baseline. Higher proportion of households in the final evaluation reported receiving necessary agricultural advice from community organizations (NGOs), agricultural resource center and agricultural groups. (Table 12)
Table 12. Source of agricultural advice
End line Baseline
n % n % Source of agricultural advice (Multiple response)
Don't know 3 0.8 52 15.6
No where 19 4.9 165 49.5
Agricultural groups 165 43.0 71 21.3
Agricultural resource center 226 58.9 62 18.6
Neighbour/friends 167 43.5 44 13.2
NGOs 333 86.7 36 10.8
Community resource persons 91 23.7 8 2.4
Other 28 7.3 4 1.2
30
4.4 Increased awareness about nutrition and healthy food habits in community.
Due to poor diet among pregnant women lead to malnutrition among newborn baby and children. Nutrition Focal Person from District Health Office, Mugu emphasized that malnutrition is common in Mugu where almost half of the students in few of the project intervention areas were malnourished. Malnutrition not only affected the school going children, but the cultural and religious taboos and feeding behavior engulfed pregnant, lactating mothers and the newborn babies. BEHAVE framework was used to conduct behavior mapping exercise to develop BCC strategies on existing food habits. 180 FCHVs and 270 peer educators were briefed on this strategy. After their orientation, they facilitated members of mothers’ groups on health sanitation and nutrition. In addition, food day and nutrition week activities were organized in school to increase awareness about nutrition. These activities increased awareness about nutrition and healthy food habits in the community.
4.4.1 Food items consumed during last 1 week
Respondents were asked about the food items the members of the family consumed during the previous one week prior to the survey. Table 13 shows that most of the households (92.1%) had consumed food items made up of grains and 83.3% mentioned consuming legumes/pulses. Compared to the baseline, the proportion of households consuming legumes, vegetables, fruits and sweet foods increased.
Households consuming food items from four or more food groups (food diversity) was increased from 60.1% to 84.1%.
Table 13. Food items consumed during last 1 week
End line Baseline Food items consumed during last 1 week
Food made from grains 92.1% 99.4% Food made from legumes 83.3% 75.8% Green leafy vegetables 59.3% 38.8% Food made from roots and tuber 71.2% 68.7% Other vegetables 54.8% 33.7% Pumpkin/ Sweet potato 25.2% 18.7% Fruit 20.1% 14.9% yogurt /cheese 36.0% 50.4% Meat, fish, egg 33.3% 37.0% 31
food made from oil fat, or butter 42.6% 55.4% Sugar and chocolate 29.6% 19.5% Food diversity (4 or more food groups)
Yes 84.1% 60.1% No 15.9% 39.9% After the implementation of this project women focused on their nutrition that ultimately leads to their good health. Women improved knowledge on the importance of food and nutrition during pregnancy, delivery and also for the newborn. They knew on diversity of nutrition (different types of vitamin, protein, nutrients) that is required for development of bone, brain and physical development. Women and their family had started avoiding a diet that is high in sugar, fat and salt, which will help them reduce weight gain and non- communicable diseases.
“Regarding food hygiene, entire women have formed kitchen garden at their own home and have been advocating for the same, relating to nutritional advantages” (FGD-Women’s group, Pandusain, Bajura)
Not only in Bajura, but also in Mugu, people were aware on the importance of healthy and balanced diet.
“After training we know the importance of nutritious food during pregnancy and advantage of healthy food among children” (FGD-Women’s group, Mugu)
Further, there is change in the food habits among the school children. Students who used to take junk foods have started to take home made foods.
“After AWASAR project was launched, people are aware, and their habit has changed; students now eat locally produce nutritional foods rather than junk foods” (DEO Mugu)
Project helped in improving and increasing the production of nutritious food in different communities in the target areas of the intervention districts. In Bajura, project successfully introduced different training and demonstration among the target population for preparing the different food for different age group. Firstly, this project enhanced the people’s (women’s and farmer’s group) knowledge on importance of kitchen garden and nutritious value of green leafy vegetables and the whole grain. Regarding food hygiene, women have formed kitchen garden at their own home and have been advocating for the same, relating to nutritional advantages. Women not only had increased level of behavior change
32
communication on food hygiene/nutrition/breast milk, but also are more aware on different types of food required during pregnancy and postpartum for both mother and baby.
“They also knew that different foods contain different types of vitamin, protein, nutrients that are required for development of bone, brain and physical development” (FGD, Women’s group, Bajura)
4.4.2 Food servings to pregnant and lactating women
Most of the respondents in the qualitative interviews were familiar about the importance of nutritious food on infant, child, pregnant and lactating mother. For instance, they prepared nutritious food and provided pregnant and lactating women with extra food. Likewise, the project delivered knowledge on importance of colostrum (thick, first milk that comes out of mother’s breast soon after delivery), exclusive breast feeding and complementary food for their child. Women not only improved the practices of complementary feeding after six months of birth of baby, but also gained knowledge about routine immunization to their newly born to 15-month child. In addition, post-natal check-up was also improved. It can be summarized that due to project interventions, people from the community have increased knowledge on safe and institutional delivery, exclusive breastfeeding and immunization.
Out of total households surveyed, around 25% of the households had pregnant or lactating women. Respondents were asked about the number of food servings to the pregnant and lactating women during last 24 hours. The average number of food serving increased from 2.9 (SD ±0.7) at baseline to 3.5 (SD±1.3) at end line (Table 14). More than two-third of the respondents mentioned food servings of 4 or more during last 24 hours, which is almost two- fold compared to the baseline. 46% of the households reported consuming more than usual foods by lactating and pregnant women.
Table 14. Food servings to pregnant and lactating women
End line Baseline
n % n % Presence of pregnant or lactating women in the n=352 n=344 house Yes 87 24.7 113 32.8
No 265 75.3 231 67.2
Number of food servings to pregnant or lactating n=87 n=113 women during last 24 hours <=2 18 20.7 33 29.2
3 24 27.6 54 47.8
33
>=4 16 35.6 20 17.7
Don't know 14 16.1 6 5.3
Pregnant and lactating women feeding practice
As usual 28 32.2
More than usual 40 46.0
Less than usual 12 13.8
Other 7 8.0 The qualitative findings also observed that the AWASAR project brought changes in traditional thinking towards pregnant women, improving knowledge and practice of maternal health care. They now have knowledge that mother should have extra food, routine ANC checkup, take iron/folic acid tablet supplementation, plenty of rest and do less work than before. They also knew about the importance colostrum milk to the newborn.
4.5 Strengthened links between communities and service providers
Several organizations from both government and non-government sectors are engaged in community-based program in the project districts. AWASAR project had a good collaboration with other stakeholders such as Seto Guraas, KRIDAC, Good Neighbors, Phase Nepal, SAHAS and HIRNDEC and others who worked to complement each other in various programs in education, agriculture and health. AWASAR project implemented a number of capacity development activities for the local government and CSOs.
4.5.1 Capacity building of services/CBOs/NGOs
Entire activities to enhance capacity building of services/CBOs/NGOs as planned (see Figure 4) were completed in order to strengthen capacity of community-based organisations to engage in local planning and monitoring, only except conducting wrap up meeting at Municipality or Rural-Municipality level based on review of the project progress reports.
The project implemented the Community Score Board (CSB), a governance and accountability tool, developed by CARE Nepal. The different trainings provided by the project to service providers and community users committees included contents related to accountability whereby they can be more responsive towards the needs of community. The project initiated and supported the institutionalization of CSB approach which was a proved accountability tool to bridge the gap between service demand and supply. Evaluation participants indicated that because of CSB quality of services has been improved. CSB has provided a negotiated space between right holders (community people) and duty bearers (service providers), whereby they could have a meaningful dialogue with the aim of
34
improving quality of services, and hence strengthened good governance. A study conducted by the project on Community Score Board showed positive changes in perception of both service providers and seekers.
“CSB previously included the indicators of hygiene and sanitation only; but after the project intervention, the community score board included indicators of nutrition and agriculture too. That is the reason for improved use of Community score board than past.” (CBO, Mugu)
The project provided local level planning and health roll out training to the elected representatives of the local government in 8 RMs. Participants of the evaluation including informants from RMs informed that fund leverage and allocation of fund in health issues increased in recent years. Meaningful participation of implementing organizations, government line agencies and CBOs in local planning resulted in changes, which were observed through community score board. “IP organizations and CBOs are participating meaningfully in local level planning and implementing the project and the progresses were observed through Community Score Board.” (Informant, RMs, Mugu)
“AWASAR coordinates with other government and non-government organization before conducting the program to strengthen on the livelihood of people.” (A VDC secretary, in KII, Bajura)
4.5.1 Increased service utilization
One of the objectives of the AWASAR project was strengthen service delivery and improve linkages with community people, thereby improving available service utilization. The survey asked about the different types of government services available at the community and the visits made by the respondents during last one year. Error! Reference source not found. shows that there was a sharp improvement in service utilization pattern as higher proportion of respondents reported visiting health facility, agriculture and livestock service center.
35
Table 15. Service utilization
End line Baseline
n % n % Visited service center during last one year
No 31 10.0 39 11.5
Health facility 208 67.3 204 60.0
Agriculture service center 177 57.3 65 19.1
Livestock service center 101 32.7 70 20.6
Municipality office/DDC 213 68.9 257 75.6
School 153 49.5 141 46.7
Other 20 6.5 5 1.5
Maternal health service utilization
The survey asked questions related to maternal health service utilization among the women who have children below 2 years of age. The proportion of women, who received at least one ANC during the pregnancy of most recent birth, was 92%, which was an increment from 73.3% in baseline. Women making recommended four or more ANC visits also increased from 41.6% to 52.2%. The proportion of women delivering at health facility almost doubled from 34.3% to 65.8% and similar improvements could be seen in PNC service utilization (Error! Reference source not found.).
According to the health facility record, visits to the service providers increased from 55% in 2016 to 75% in 2018.
Table 16. ANC and delivery care utilization
End line Baseline
n % n % ANC n=75 n=105
Yes 69 92.0 77 73.3
No 6 8.0 28 26.7
Number of ANC n=69 n=77
1 3 4.3 11 14.3
2 15 21.7 13 16.9
3 15 21.7 21 27.3
4 or more 36 52.2 32 41.6
Number of ANC visit n=75 n=105
Less than 4 39 52.0 73 69.5
4 or more 36 48.0 32 30.5
Place of delivery
Home 26 34.2 69 65.7
Health facility 50 65.8 36 34.3
36
73.7
Endline Baseline
42.9
% 32.8
4.8
At least one PNC 3 or more PNC
Figure 10. PNC visit1
“The project supported service providers for capacity enhancement and community people for demand creation. Community people are now better able to get benefit from the available services. The project also supported to improve harmony in the relation between local organization and community.” (A KII participant, DHO Mugu)
1 Women who delivered at health facility, were considered receiving the first PNC visit at health facility after birth. 37
CHAPTER V
DISCUSSION AND ANALYSIS
This chapter presents the analysis of the findings and its interpretations. Discussions on findings against different evaluation criteria that include relevance/appropriateness, effectiveness, impact and sustainability are provided separately for core programs of AWASAR Project, i.e. Education, Agriculture and Nutrition.
5.1 Adequacy and relevance
Project implemented activities were in line with the government strategy and policies. The project jointly worked together with government line agencies at district and local level to address the issues of education, nutrition and agriculture.
Education
Project succeeded in enhancing school enrollment, reduced absenteeism and dropout rates. Interviews and discussion with the different stake holders in the district and communities revealed that the engagement of stakeholders in planning and implementation was fair enough and they were aware of the activities and actively engaged in these activities. Awareness among the parents, especially those who are seasonal migrants on the importance of education during migration and this helped retaining students in school during migration. Scholarships helped retaining poor and marginalized students in the school.
Agriculture
The project supported to improve food security and crop yield in the intervention area. Through training and capacity building of the farmers on off season farming through the use of green house and cultivation of hybrid crops, it succeeded in increasing production. This project resulted in behavioral change in agriculture by developing the appropriate strategy (campaign, training on seed production, plantation of seed and distribution of seed) and met the identified needs of the agriculture.
Nutrition
In nutrition too, the project provided important supports in succeeding the government’s aim of reducing malnutrition among pregnant, lactating women, newborn and under 5 children. This project brought the change in behavior by providing training and orientation on
38
preparing the super flour (sarbottam pitho), important of ANC/PNC, awareness about exclusive breast feeding and complementary feeding, etc.
Strengthen service deliver and linkages
The project has delivered a number of capacity building activities to the representatives at local government, service providers, CSOs, partner NGOs. The qualitative findings of the evaluation revealed increased fund leverage, improved coordination and monitoring of the service programs from government and CSOs. CSB seems to be a vital tool for strengthening the accountability parts both the service providers and service receivers.
5.2 Effectiveness (Where the right things done)
Overall, the AWASAR Project had been smoothly implemented in the project areas of both districts. This program was successful in bringing positive changes in the lives of women, men, children and marginalized communities.
Education
The project targeted towards improving the facilities in the school, community awareness on importance of formal education, provision of temporary provision for migrant students by conducting awareness sessions with their parents. In addition, they provided scholarships to the needy children who were out of school due to financial constraints. Project activity incorporate gender, governance and resilience by advocating the gender issues in education; increase the campaign especially for the girls, information education and counseling for promoting girls from the community levels.
Agriculture
Project enforced on crop yielding by the use of modern technologies that could be implemented in the project areas. Uses of greenhouse concept for farming, compost manure production from locally available resources were cost effective intervention to improve crop production and food security. It further resulted in financial sustainability of the community people and improved the livelihood by introducing the concept of cash crops.
Nutrition
Project supported in kitchen garden concept; which helped people to produce nutritious fresh vegetables necessary for balanced diet. Furthermore, awareness activities on balanced and nutritious food, importance of colostrum feeding, and breast feeding helped to reduce
39
malnutrition among mother and newborn child. Additionally, immunization, and hygienic diet for the men and women, training and demonstration about how to make super flour (Surbottom Pitho) were the best strategy for achieving the impact of the project among the women’s groups.
5.3 Management Efficiency (Were things done right?)
Most of the project activities had been completed as planned in coordination with partner organizations and relevant stakeholder in project areas in both districts. The performance had been satisfactory as evident from survey, interviews and review of progress reports. It was deemed that the budget was inadequate for health and nutrition programs as opposed to education and agricultural sector. It was also thought that improvement could be made to increase coordination with newly elected local level representatives of Municipalities and Rural Municipalities in current federal context.
Education
There was coordination and collaboration between the implementing partner and the concerned stakeholders in education, agriculture and nutrition programs. Monitoring and supervision of the program along with effective feedback to the concerned agency was swift. Quality of supervision, efficiency in financial administration, ability to anticipate problems and extend implementation support, adequacy of reporting, recommendations and effectiveness of follow-up on recommendations were done very effectively and conveniently in the field of education. As a result of such collaboration, activities were managed rapidly, at some instances there was even 700% achievement in some interventions.
Agriculture
There was coordination and collaboration between the implementing partner and the concerned stakeholders. Monitoring and evaluation was conducted by partner NGOs in relation to the techniques of food production and farmers were supervised as to minimize the impact of climate change. Food security shelter was built by CARE and distributed seeds, provided training on modern technology. Financial support for agriculture sector was good enough.
Farmers were glad that through this project, they could avoid food crisis. For example, due to the capability of staffs all farmers could produce unseasonal foods adopting modern technology.
40
Nutrition
Local NGOs updated the progress of activities on CSB. They also recommended community people to utilize local nutrient foods and provide feedback to the CARE. Service providers and community people were motivated and encouraged to adapt health habit. For example, people are consuming nutritious food because of counseling received on nutrition by project staffs. It was deemed that the budget was inadequate in health and nutrition.
5.4 Outcome and Impact
Measuring impact of the project may require long-term follow up studies. The evaluation showed substantial impact in terms of demand creation as well as service delivery. Improvements in the project level impact indicators such as proportion of food secure households, school enrollment rate, maternal health service utilization and improvements in quality of service delivery mechanisms by local government are observed. However, long term impact may be impeded with questions about the sustainability of the initiations such as community score board.
The large majority from project areas of both districts were satisfied towards the AWASAR Project. Overall, there was improvement in physical infrastructure of schools, health and nutrition support and food and livelihoods of the poor people, and capacity building of mother’s groups, farmers group, and migrant population. Implementation of CSB improved governance, accountability, public participation in the agriculture and health programs for joint goal setting, planning and regular monitoring.
Education
Schools were upgraded to improve all facilities, and enough support was provided to build infrastructure of library and school. Pre-education and mobile school campaigns were conducted. In addition, in-service training for teachers was provided. Educational status of targeted area was upgraded. Migrant students were benefited to complete their study.
From the view of stakeholders of RMs, School teacher, mothers group and students, project had achieved their objectives very well. Overall, the rate of school enrollment had increased, dropout rate has decreased, and low dropout was observed in early school. Knowledge was increased regarding the importance of schooling in the community.
41
Agriculture
There was increased awareness and knowledge about farming technique to eliminate food crisis. Training was provided to lead farmers. Support was provided to build kitchen garden and green house. Training on demonstration plots and seeds multiplication of farms was provided. Overall, food security system was secured and the increased level of independence regarding agriculture aspect and livelihood was observed.
Nutrition
Community people were able to know about healthy and nutritious foods. Community score board was established and well implemented. Various food day and nutrition week in schools were celebrated. There was an involvement of FCHV and mothers group on health sanitation and nutrition programs. Overall, nutrition and food security of families are improved in the target district.
5.5 Accountability and transparency
CARE Nepal initiated the Community Score Board (CSB) approach to improve the governance and public participation in the services delivered at local level. CSB has also helped to improve accountability and transparency of the health service. People are now more informed about the present status of their communities and the services available at local level. Community participation and involvement of relevant stakeholders was sought at different phases of the project. Relevant stakeholders at district and local level were duly consulted throughout the project.
CARE organized regular public hearing and public audit events to present the progress update of the implemented projects. The general public was informed about the project details such as total budget allocated, total amount spent and total amount remaining and work progress status and timeline. Additional approaches such as displaying information boards have been maintained. However, by the time of this evaluation, project phase out event and handover of the project activities was yet to be planned and conducted.
5.6 Sustainability
The project has trained FCHVs and local peer educators for delivering a range of messages related to food security, nutrition, education, health and WASH in the community. The best part of AWASAR project was that the activities were implemented in coordination with the local government stakeholders adopting national policies and programs on education, 42
agriculture and nutrition. The project strengthened capacity of local government to plan and implement more responsive service delivery mechanisms towards the need of poor and marginalized groups. With the knowledge and skills gained by local facilitators and service providers, they could continue to deliver the message and raise awareness in the community even beyond project duration period. CSB approach has also played an important role as an advocacy tools for prioritizing issues and allocating resources.
In the context of newly changed political environment, locally elected representatives from local government have direct role of planning and allocating resources for community development. The project oriented the representatives from the local government on local level planning process. There is some evidence of increase in fund leverage at local government in the project districts.
The evaluation team did not witness project phase out event and wrap-up meeting to hand- over the project initiatives and activities to the local government. Initiatives of the project such as CSB are questionable for its sustainability. In addition, the project could have made additional efforts to make linkages of the farmers’ and mothers’ group with other projects and programs from government as well as private sector. A strong ownership among the stakeholders and community people is required for sustainability.
5.7 Gaps and challenges of the project
While the programs were implemented to have concrete output and impact, there were few rooms for improvement. The major gaps identified were due to programs being implemented in selected VDCs of districts, and the project areas were scattered. Marginalized and vulnerable groups were still not captured in the programs due to their instable residence and poverty. At some point, there was high turnover of the staffs that delayed the project implementation. There is lack of support received from Government of Nepal in terms of adopting accelerated learning curriculum, specifically targeted for Himalayan regions, where schools remain closed due to cold weather and seasonal migration. Regular monitoring, evaluation and supervision of the project activities would have been helpful, although only few were conducted. Overall, poverty, illiteracy, geographical challenge and migration were the major challenges to the project.
43
Table 17. Gaps and challenges in relation to education, agriculture and nutrition programs
Education Agriculture Nutrition Although success of the Although success was Although success was observed project was observed in terms substantial in terms of: in terms of: of: Increased productivity of Increased knowledge on safe Increased number of crops, motherhood. enrollment in school, Increased knowledge of Decreased maternal mortality Decreased in number of alternate production of rate irregularity in school crops, Decreased neonatal mortality attendance, Increased livelihood of rate. Decreased dropout rate of farmer’s group and the school students, community. The major issue was that there Provision of facilities for was less interest in health sector migrant school children. Few issues considered were: in comparison to agriculture and The duration of this education sectors. The other There were few issues to be project was not enough to challenge was that due to considered as of below: achieve some of the illiteracy, people found it AWASAR Project was not goals. difficult to get acquainted with able to integrate whole Insufficient field for CSB. district, only the selected production of food, VDCs were chosen. hybrid seed was unable to Possible solution: No provision of building for grow because of climate, Promote participation of migrant children in and there was lack of vulnerable group and FCHV in Himalayan areas. market for business the project activities. Conduct Scholarship was not enough purpose. demonstration program on the for the underserved school Project was unable to topic of nutrition and healthy children. provide seed to all foods in the community to raise Poverty, illiteracy, community (farmer awareness on condition resulted geographical challenge and group). due to lack of nutrients. migration were the major challenges to the project. Possible solution: Proper counseling to all RMs Possible solution: and Farmers, motivate to Proper counseling to all farmers by providing guardians and teachers should incentives, provide seeds, be provided. Biannual in-service coordinate with farmers about training for teachers should be the land plot and provide them provided with regular feedback regularly. monitoring and evaluation adopting standard monitoring checklist.
44
5.8 Best exercises
Some of the best practices of the project which could be replicable in other similar projects are described. Regarding education, extra classes for migrant children were conducted, scholarships were provided to poor students, school facilities improved by supporting mini libraries, sport materials, construction of separate toilets were built for boys and girls, and computer supports. School kitchen garden is one of the innovative approach (in the project area) implemented by the project. A food demonstration event at community gatherings was found effective. As a result of the trainings on modern farming technology, farmers were able to produce unseasonal food employing their gained skill on their own. Farmers and community people had increased knowledge to understand community issues, improved use of local resources and became self-confident. They had developed positive attitude towards nutrient rich food during pregnancy, and postpartum period.
CSB was considered most effective in improving the accountability and responsiveness of the delivery of health services. CSB has helped to make public aware about the current status of various health indicators and it has become an important monitoring as well as accountability tool for quality of service, behaviour of health workers, regularity, timely open of health posts, including others. It also demonstrates good practice for maintaining accountability and transparency of the activities conducted by development agencies. Joint monitoring visits of project activities were made involving different project stakeholders and journalist.
5.9 Gender Equality and Social Inclusion
The evaluation did not directly focus on exploring the gender component of the project in detail. Gender Equality and Social Inclusion (GESI) policy was developed by CARE Nepal and partner organizations and was implemented in the project. Issues around gender equality and social inclusion were considered to engage and empower women and marginalized people to have their participation in decision making process. High level of female participation was observed in various project activities, and the gender stereotype was non- existent. Dalit women were included in the women’s and/ farmer groups to reduce caste- based discrimination.
45
CHAPTER VI
LESSON LEARNED, RECOMMENDATIONS AND CONCLUSIONS
6.1 Lessons learned
Overall, the AWASAR Project was successfully accomplished up to meeting the target due to the clear vision of the project itself, clarity among entire AWASAR Project staffs and partner organizations, and working in close coordination with local and district level stakeholders for education, agriculture and nutrition activities. The integrated efforts of district education office, school management committees, local community and parents, and the local leaders is very vital for the regular classes and quality education in hard to reach areas. The kitchen garden at the school has played a crucial role to increase knowledge among students on the need of kitchen garden for good nutrition; this helped in adopting the kitchen garden at the household level, and to reduce junk food consumption. The provision of new techniques and materials to the farmers has increased their motivation to produce vegetables in their own farm to supplement nutritious food and sell it for earnings.
Regular monitoring of the program activities and support provided was lacking and was deemed necessary vital to implement the program activities according to the set plans. The transparency maintained supported in increasing ownership by the community members and supported in generating local support for the implementation of the project.
Education
It was essential to know how to encourage students and guardian for better education to live quality of life in long run. The teacher at school should be at adequate numbers. Privacy is essential to ensure uniformity and address need of boys and girls separately. Agriculture To increase production of foods, it would be better to encourage and provide training on various modern technologies. In addition, the trainings should be provided in timely manner.
Nutrition Awareness program and orientation program should be adequately provided.
46
6.2 Conclusions
6.2.1 General conclusion
The final evaluation of the AWASAR project was targeted to determine and explore its outputs, outcome and impact over the last three years. The blends of qualitative and qualitative methods were employed adopted to collect the information from field including local and district level line agencies using semi-structured questionnaire, guidelines and checklists.
Regarding education component, the construction of toilets at schools of both districts were completed; gender friendly toilets were built, wooden furniture was supported, lunch box was provided, solar lamps and cookers in schools were provided, mini-library was built, sports materials provided, and actions was commenced to enhance accelerated learning. In addition, orientation was provided to the adult literacy facilitators, helped to form fully literate VDC, training was provided to teachers on child centric teaching learning methods, scholarship provision was granted for poor and talented students and promotional materials were provided to peer educators in the community.
Regarding agriculture component, trainings were provided to extension agent and refresher trainings were provided in both district. Training was provided to lead farmers and farmer group members. Kitchen garden at school grounds was introduced to teach children about agriculture and nutrition. Vegetable and seed support was provided to the community people. Furthermore, demonstration plots and seeds multiplication of farms were established, along with training provided to farmer on demonstration plots.
Regarding health and nutrition component, basic training and refresher training was provided to FCHV on health and nutrition. Full support was provided to have full immunization, cultural programs were implemented to spread message on education, health and agricultural component. Various days were celebrated, such as food day, sanitation and nutrition week. Trainings were provided to mothers’ groups on enhancing the health of pregnant and postpartum women. They were encouraged to utilize the services from health facilities. Health facilities were supported with suction machine, fetal dropper and other equipment as per need.
47
6.2.2 Overall conclusion of the evaluation
The summary of the above analysis and discussion has been presented in Table 18 and Figure 11 which shows that in the range of 0 to 4, the project scored highest in relevance and effectiveness (score 4), while the three criteria impact, accountability and transparency, and sustainability each scored 3.
Table 18: The summary and overall conclusion of project evaluation
Evaluation Evaluation Main findings/Major facts identified Results Relevance Highly Relevant The project was highly relevant to meet the livelihood needs of the people, (A) especially the children from migrant households. Integrated activities in food security, education and health, as well as major focus on strengthening on the delivery of services were highly relevant to improve service delivery to the remote and marginalized villages focusing migrant children. The communities targeted were poor and marginalized groups living in remote parts of the country. Moreover, the project activities were in line with the government strategy and policies. Effectiveness Highly Majority of the project objectives were satisfactorily met. The project has Effective (A) contributed towards improved food security, quality education and service utilization by stimulating accountable service delivery as well as mobilizing community people for demand creation. The evaluation team witnessed more responsive service delivery. Household survey and FGDs with community people showed increased visit to service centers such as Health Post, and Agriculture and Livestock Service Centers. Management NA The evaluation did not focus on quantitative efficiency analysis of the project. No Efficiency major issues in financial administration. Strengthening government service delivery mechanisms, local capacity building, kitchen gardening, greenhouse concept for farming, compost manure production from locally available resources were cost effective intervention at the long-run. However, the evaluation findings may not be able to judge the management efficiency of the project. Impact High (B) Measuring impact of the project may require long-term follow up studies. The evaluation showed substantial impact in terms of demand creation as well as service delivery. Improvements in the project level impact indicators such as proportion of food secure households, school enrollment rate, maternal health service utilization and improvements in quality of service delivery mechanisms by local government are observed. However, long term impact may be impeded with questions about the sustainability of the initiations such as community score board. Accountability High (B) Community score board approach has made service providers more accountable and and transparency transparent towards the community people, with improved quality of services. CARE organized regular public hearing and public audit events to present the progress update of the implemented projects. The public were informed about the project details. Additional approaches such as displaying information boards have been maintained. However, by the time of this evaluation, project phase out event and handover of the project activities was not conducted. The project did not have clear exit strategy. Sustainability/ Sustainable (B) Local capacity building, strengthening linkages between service delivery and Connectedness demand, initiation of CSB approach, and promotion of kitchen gardening and modern farming are some of the activities of the project likely to sustain. The evaluation team felt that the project could have made additional efforts to make linkages of the farmers’ and mothers’ group with other projects and programs from government as well as private sector. A strong ownership among the stakeholders and community people on initiatives such as CSB is required for sustainability.
48
Overall Very Good (B)2 Overall, the project is found in “very good” performance position. conclusion
The scale of outcomes evaluation results (A= 4, B = 3, C = 3, D = 1, E = 0). Relevance: Highly Relevant (A), Relevant (B), Moderately Relevant (C), Fairly Relevant (D) Not Relevant (E) Effectiveness: Highly Effective (A), Effective (B), Moderately Effective (C), Fairly Effective (D) Not Effective (E) Impact: Excellent (A), High (B), Moderate (C), Fair (D), Low (E) Accountability: Excellent (A), High (B), Moderate (C), Fair (D), Low (E) Sustainability: Highly sustainable (A), Sustainable, (B), Moderately Sustainable (C), Fairly Sustainable (D), Not Capacitate (E) Overall Evaluation: Excellent (A), Very Good (B), Good (C), Fair (D), Poor (E)
Relevance 4
3
2 Sustainability/ Effectiveness Connectedness 1
0
Accountability and Impact transparency
Figure 11: Overall evaluation of the project in five evaluation criteria
Overall, the project is found to be to be ‘very good’ with the average score of 3 (in the range of 0 to 4). The project was able to meet its target as proposed in education, agriculture, nutrition components and supported to strengthen link between communities and service providers and to increase coordination and linkage with district level stakeholders including government line agencies, to ensure quality services. The efforts made by the CARE Nepal and the partner organizations largely resulted to the successful completion of the project achieving its aims and targets and delivering benefits to the targeted beneficiaries as envisioned by the project. The evaluation team believe that the learning from the project
2 The average score of all the five evaluation criteria is 3.4 = 3 (i.e. B).
49
could be an important asset for CARE Nepal and other relevant organizations to further improve the implementation aspects in similar other project endeavors in the future
6.3 Recommendations
Based on the evaluation of the project, following recommendations have been made:
CSB approach is effective for ensuring accountability in terms of improving food security, health and education. CSB is effective in terms of mobilizing communities as well as fostering accountability of service providers. Such approaches could be replicated in other similar projects. Proper handover of project to the local government with plan on post-project support by CARE Nepal. The evaluation team felt that due to short duration of project, the capacity of local stakeholders has not developed to the extent that could have been done. The newly elected local government may need continuous capacity building support. It was observed from qualitative interviews that the program coverage should be expanded to other VDCs of selected districts and program planned according to current federal context. Provide business scope for farmers in the community to sell hybrid crops. Increase linkage and coordination with local and district level stakeholders, i.e. Rural Municipalities and Municipalities, Department of Education, Department of Agriculture, Agricultural Service Centers, Family Welfare Division and other relevant stakeholders from the early stage of the project planning, implementation, monitoring, evaluation and dissemination.
50
ANNEXES
Annex I. Evaluation methodology summary
Indicators Methodology/Tools Overall objective: Improve the Improved nutritional status of pregnant & lactating mothers, and HH survey nutritional and educational status of children under 5 years (measures: reduced protein energy FGD-mothers‟ group hard to reach children in Bajura and malnutrition rates for children under 5, improved anemia level KII with health worker, FCHVs Mugu districts of the mid- western among pregnant and lactating mothers) KII – nutrition focal person at DPHO high hills of Nepal KII- District Agricultural Office Increase in the food sufficient months of the households of HH survey targeted children FGD (Farmers‟ and mothers‟ group) KII with Agricultural Service Center District agriculture office (DAO) reports Increase the educational level of children under 15 years Household Survey (increase school enrollment, completion rates, reduced drop-out FGD-Community and school children rates for boys and girls) KII – school authority, resource person, members of SMC and PTA, DEO District Education Offices and the individual school records Immediate objective: 1: Improved % adult literacy increased and assisted government national Household Survey access to flexible and better quality campaign on full literate VDC/RM ward. FGD-Community education for seasonal migrant KII – school authority, resource person, children (high hills to members of SMC and PTA, DEO lower elevation) District Education Offices # adult become aware about their migrant children' education Household Survey quality (enrollment, retention, school facilities, nutrition, FGD-Community upgraded) District Education Offices
51 Indicators Methodology/Tools Positive change in attitude of parents towards education of Household Survey children (especially girls and Dalits) (as observed by a percentage FGD-Community increase in the number of parents who insist on children's education and send the children to school every morning) # Or (%) the children of marginalized people from vulnerable Project report communities accessed/increased in school education (enrollment, HH survey retention, upgraded, school facilities, nutrition etc.) #schools…..% students Analysis of results increased quality education via formal education. 2: Improved nutrition and food % increase in production and productivity of indigenous and District Agriculture Office records security nutrient rich crops on farm plots (improved varieties) # of crop species adopted by families (increased diversity) District Agriculture Office records
Evidence of improved food preparation and consumption of more Household survey will include a rapid nutritious food at family level assessment of food habits
Evidence of improved hygiene and sanitation practice Observation and household survey
3: Strengthened links between % Increased visits by communities to service providers Household survey communities and service providers (extension/service centers, health posts) Service delivery records to ensure service quality
Evidence of IP organisations and CBOs participating KII with CBOs, RMs, government officials meaningfully in local planning and review meetings at VDC and at various levels VDC and DDC records DDC levels % Increase in DDC, VDC (DEO, DAO, OHO) budget allocations in the area of accelerated education, nutrition, hygiene and sanitation related services that would benefit the target communities
Trend of improvement in indicators on community score boards Community Score Boards – analysis for health (hygiene & sanitation), nutrition and agriculture Household survey extension services
52 Annex II: Achievement of project log-frame indicators
Project Achievement.docx
53 Annex III: Additional results from household survey
Table 19. Sample size by study district and (former) VDCs Sample Size District/VDC n % within district Mugu 221
Bhiyi 22 10.0 Hangyalu 23 10.4 Jima 22 10.0 Kimri 21 9.5 Mugu 23 10.4 Natharpu 22 10.0 Pulu 21 9.5 Seri 23 10.4 Siranchaur 22 10.0 Sukdhik 22 10.0 Bajura 168
Badhu 17 10.1 Dahakot 22 13.1 Gotri 17 10.1 Jukot 15 8.9 Kolti 22 13.1 Manakot 22 13.1 Pandusen 22 13.1 Rugin 15 8.9 Sappata 16 9.5 Total 389
Table 20. Drinking water End line Baseline
n % n % Main source of drinking water
Pipe water 383 99.7 286 83.1
Stream/river 1 0.3 36 10.5
Spring 19 5.5
Other 3 0.9
Location of tap (in case of pipe water)
Inside house 6 1.6 2 0.7
House yard 97 25.3 13 4.5
Community tap 280 73.1 271 94.8
Water fetching time
Less than 5 minutes 84 24.6 22 6.4
5 to 15 minutes 188 55 215 62.5
15 to 30 minutes 55 16.1 80 23.3
More than 30 minutes 15 4.4 27 7.8
Is water sufficient for drinking around the year?
Yes 283 75.9
No 90 24.1
Water storage in the household
Covered container 347 90.6 182 52.9
Not covered container 34 8.9 155 45.1
No water at house 2 0.5 7 2.0
54
Table 21. Sanitation facilities End line Baseline
n % n % Toilet available at house Yes 354 92.4 321 93.3 No 29 7.6 23 6.7 Type of toilet Water-sealed 309 87.8 206 64.2 Traditional pit latrine 43 12.2 115 35.8 Water sufficient to use in toilet Yes 260 82 206 64.2 No 57 14.7 115 35.8 Water available at toilet Yes 252 72.8 167 52 No 94 27.2 154 48 Toilet cleanliness Not clean (Foul smell, flies, dirty) 11 3.4 61 19 Somewhat clean 279 86.6 230 71.7 Clean 32 9.9 30 9.3 Does every member of the household use toilet? Yes 328 88.2 n/a No 44 11.8 n/a
Table 22. Hand washing with soap and water End line Baseline Hand washing with soap and water (Multiple response)
No handwashing with soap and water 9.7% 11.6% After defecation 82.2% 73.0% Before eating 78.5% 75.0% After working in field/dirt 75.3% 60.2% After cleaning children defecated 68.8% 46.2% Before breastfeeding/feeding child 60.8% 29.9% Before cooking/preparing food 54.2% 39.5% Hand washing station at house (Multiple response)
No handwashing station at home 6.1% 19.5% Yes, running water 28.3% 15.1% Yes, water in container 65.6% 47.1% Yes, Soap available 60.1% 43.0% Yes, Ash, sand available 16.7% 24.7%
55 Table 23. Method of household waste disposal End line Baseline n % n % Solid waste management
Dumping 29 7.7 29 8.5
Burning 47 12.5 33 9.6
Throwing in manure 221 58.8 134 39.1
Throw elsewhere 79 21 147 42.9
Liquid waste management
Throw in kitchen garden 122 32.4 86 25.1
Throw in land 23 6.1 26 7.6
Throw elsewhere 62 16.4 106 31.0
Throw in pit 118 31.3 98 28.7
Other (NALI) 52 13.8 26 7.6
56 Annex IV: Evaluation team
SN Name Position Qualification Major Roles Field of experience 1.Dr. Govinda Principal PhD Ag. Plan, 35+ years long Prasad Investigator (PI), Economics coordinate and experience in the Koirala Livelihood/ monitor the field of agriculture Agriculture/ study. Value Chain expert 2.Prof. Dr. Co-PI Ph.D. Medical Assist Team 42+ years long Ritu Prasad Anthropology, Leader for experience in the Gartoulla MA, MPH Plan, field of public coordinate and health, education, monitor the nutrition, WASH, whole study. DRR, environment management,
strategic plan development, GESI, project monitoring and evaluation. 3.Dr. Research PhD Support to entire 10+ years long Pragya Consultant Epidemiology evaluation experience in in Gartoulla activities entire project including management. analysis. 4. Dr. Qualitative Data MBBS, MDGP, Support the 8 years + Sandesh Analyst MPH study team in experience in Pantha conducting theoretical and
qualitative practical aspects (PhD fieldwork and of research Candidate) supervise the methodology and enumerators. project management. 5. Mr. Public Health MSc Research Analyse 12 years + Deependra Expert/Data methodology, household experience in both Kaji Thapa Analyst MA sociology, survey data, theoretical and (PhD (Consultant) MPH support the practical aspects of candidate) team in report research writing and methodology, finalization, epidemiology, ensure quality proposal writing, of evaluation project evaluation, study. data analysis (both quantitative and qualitative). 6.Supervisors From RECID As required. Supervisor HH Relevant experience and Roster Survey and Data Enumerators Entry
57 Annex V. Secondary data from DEO, DAO and Health Facilities
Bajura and Mugu_E_Checklist_DistrictAgriOff.docx
Bajura_Mugu_E_Che cklist_DEO.docx
Bajura_Mugu_secon dary_HF.docx
Bajura_Mugu_ E_School Checklist 15.docx
Bajura_DEO_second ary data.docx
58 Annex VI. Evaluation tools
AWASR HH Questionnaire_Nepali.docx
59 Annex VII. List of persons contacted for KII
Stakeholders Contacted person at BAJURA Contacted person at MUGU
Nutrition focal person of Ram Chandra Yadav Parsuram Nepali DPHO Focal person of DPHO Sushila Phing Suraj Gurau Focal person of DEO Gopal Joshi Ramesh Malla Focal person of Agriculture Man Bdr Rawol Sokiram Nepali Service Centre Focal person of District Man Bdr Rawol Dhan Bahadur Budha Agriculture Office VDC Secretary or ward chief Jaya Bdr Adhikari Mer Bhadur Budha Bir jit Kattel Pema Snagmu Lama Amma lal Karki Tilak Rawal Ganesh Rokaya Karjan Shahi Sangita Bista Man Prasad Bhatta Bishnu Bdr Buda Bir Malla Man Bdr Bohora Ram Banadur Rawal Prakesh Shahi Chirring Gyango Lama Ganesh Bdr Rokaya Lakpa Chhiring Prem Bdr Rawot FCHV Pragati Adhikari Karma Fulging Lama Teacher Pradeip Raj Regmi Gauri Devi Tamata Rang Bir Shahi Hira Lal Shahi
60 Annex VIII: Household survey database
61 Annex IX: Detail qualitative data (transcribes and analysis)
62 Annex X: Successful Case Studies
AWASAR_case-stud y_1_Bajura.docx
AWASAR_case-stud y_2_Bajura.docx
AWASAR_case-stud y-3_Bajura.docx
AWASAR_Case Study_Mugu.docx
63