TEARFUND UK IN SIERRA LEONE

Water and Sanitation Programme February-December 2002

Jaluahun Chiefdom, Kailahun District Eastern Province, Sierra Leone

Evaluation Report

Period of Evaluation: December 9-13 2002

Submitted by: 1. Sahr Ngegba, EDRA Consultancy, 50 Bathurst Street, Freetown

December 2002 TABLE OF CONTENTS

List of Acronyms...... i Executive Summary...... i 1.0 Chapter ONE: Introduction...... 1 1.1 Background...... 1 1.2 Terms of Reference (TOR) ...... 2 1.3 Methodology...... 4 1.4 Limitations and Constraints...... 5 1.5 Structure of the Report...... 6 2.0 Chapter TWO: Discussion Of Findings ...... 7 2.1 Original Programme Aims and Objectives...... 7 2.2 Changes in objectives...... 7 2.3 Programme activities and implementation...... 7 2.3.1 Water and Sanitation Unit...... 8 2.3.2 Public Health Education ...... 10 2.4 Perception and Impressions of beneficiaries, Partners, other NGO’s and CBO’s....13 2.41 Beneficiaries ...... 13 2.42 NGO’s and CBOs ...... 15 2.5 Project Impact ...... 16 3.0 Chapter THREE: Conclusion and recommendations ...... 21 3.1 Appropriateness ...... 21 3.2 Effectiveness...... 23 3.3 Efficiency...... 26 3.4 Impact on Communities...... 27 3.5 Coverage and External Linkages ...... 28 3.6 Sustainability...... 29 3.7 Coordination ...... 30 3.8 Peace Building and Conflict Transformation ...... 32 3.9 Spill Over Effect of Programme Benefits...... 32 Appendix I: Proposed Strategy and work plan ...... 35 Appendix II-Key questions Used...... 36 Questionnaire for Partners ...... 36 Questionnaire for project personnel...... 37 Discussion Guide for Communities...... 38 Discussion Guide for Heads of Programme...... 39 Appendix III: Schedule of Community Discussions...... 40 Apendix IV: Project Communities visited and composition of focus group....41 Appendix V: List of Interviewees...... 42 Appendix VI: Schedule of Interviews and Activities...... 43 Appendix VII: Project Performance Indicator...... 44 Appendix VIII: Activities undertaken and status ...... 45 Appendix IX: Communities and their locations ...... 46 Appendix-X: Original and Present Population of communities ...... 47 Appendix XI: Structural and Water Quality Data ...... 48 Appendix XII: Logical Framework Matrix ...... 49 LIST OF ACRONYMS

CHVs Community Health Volunteers

CBO Community Based Organisation

DRT Disaster Response Team

ECHO European Commission Humanitarian Aid Office

EDRA Evaluation, Development and Research Associates

EFSL Evangelical Fellowship of Sierra Leone

IDPs Internally Displaced Persons

KAP Knowledge Attitude and Practice

MOH Ministry of Health

MOU Memorandum of Understanding

NGO Non Governmental Organisation

ORS Oral Re-hydration Solution

PHE Public Health Educators

PIM Participatory Impact Monitoring

RUF Revolutionary United Front

TBAs Traditional Birth Attendants

TOR Terms of Reference

UNAMSIL United Nations Mission in Sierra Leone

i UNICEF United Nations International Children’s Emergency Fund

UNOCHA United Nations Office for the Coordination of Humanitarian Affairs

VDCs Village Development Committees

WFP World Food Programme

Watsan Water and Sanitation

ii EXECUTIVE SUMMARY

Introduction This assessment was to examine the effectiveness and impact of the ECHO funded, water and sanitation project in Jaluahun Chiefdom, Kailahun District, Sierra Leone. The assignment was undertaken by an external consultant to provide an independent assessment of the extent to which the projects purposes and results, as laid out in the project proposal, have been achieved. The joint purpose of the evaluation was to fulfill the external evaluation requirements for the project as agreed with ECHO and Tearfund’s internal accountability obligations under the Disaster Management System. The exercise was conducted in December 2002. Terms of Reference To guide this exercise, the following terms of reference (TOR) were developed: • Assess the programmes appropriateness • Measure its effectiveness and efficiency • Assess project impact on beneficiaries • Determine programmes coverage and external linkages • Assess sustainability and coordination • Assess the impact of Peace building and conflict transformation activities

Methodology The assessment was participatory. Various instruments were prepared and used. One was used to collect information from beneficiary communities in Jaluahun Chiefdom, another to collect information from other NGOs, partners and project personnel etc.

Findings Beneficiaries tend to have a positive perception of Tearfund and hold them in very high esteem in the region, especially as a wellspring of development activities in their communities. The agency has provided Watsan services to 13 communities in the chiefdom. These facilities are seen to be benefiting virtually every one in these locations. The work of Tearfund in the communities has not only provided facilities but has inspired and motivated people to return to these communities. The agency has worked especially hard in the area of motivating people to work in groups rather than as individuals. This has helped the beneficiaries to be more united and promote a peaceful co-existence in their communities. Communities record a positive impact in their localities. They also agree that Tearfund is addressing their priority needs but in future, in addition to the Watsan provision, the priority they

i see is helping replace their thatched roofs with C.I. sheets, constructing more toilets and providing health facilities. The agency provided training to Community Health Volunteers building their capacities to take up maintenance and care of the facilities provided in their communities. The involvement of beneficiaries in the administration and management of community activities has greatly empowered the communities to understand and cope with problems and difficulties that will be envisaged in future sustainability. There has been improvement in the, knowledge attitude and practice to public health and sanitation in the target communities. The incidence of diseases (e.g. water-borne diseases, malaria etc) that were prevalent in these communities has dropped to a relatively low percentage as reflected by clinical reports in the region and interviews. Although the beneficiaries have a positive perception of Tearfund, and the fact that more facilities than planned have been provided to the communities, this positive image of adequacy, quality and satisfaction towards Tearfund’s Watsan interventions could slowly be replaced with a negative impression particularly as more people return to these villages. The facilities will be over burdened and hence liable to damage in a short time if provisions are not made for the new arrivals. Tearfund collaborates with other NGOs in Sierra Leone. It is well known and has an excellent reputation with efficient staff. All local NGOs that were asked to rate Tearfund answered with ‘above average’. They consider Tearfund to have a very relevant programme and an appropriate community development concept. It is seen to be contributing to the rebuilding and development of the war torn communities in the Jaluahun Chiefdom. The agency maintains good relations with government departments to ensure that the agency is in line with the policies of government to facilitate continuity should Tearfund pull out of these communities.

Conclusions and Recommendations Tearfund’s interventions are greatly appreciated by their beneficiaries. On the whole, Tearfund is seen in a positive light by beneficiaries and partners and their services are seen to be the priority need of the target communities, which to them are perceived as primary to their survival. There are fewer incidences of the common ailments in the region. The communities are cleaner with access to clean water and have a clear understanding of the causes of common ailments, the preparation of ORS and are better prepared to respond to cases of diarrhoea. Future challenges considered as priority by the communities, include shelter, medical facilities, micro credit for women groups and input (seeds, tools and food for work) for men’s groups.

ii This report proposes a number of recommendations meant to facilitate achievement of Tearfund’s objectives. These recommendations also include staff capacity building and perfecting participation to strengthen the beneficiary community so as to ensure sustainability of the services provided.

iii 1.0 CHAPTER ONE:

INTRODUCTION

1.1 Background

Tearfund UK is an international NGO operating in Sierra Leone. The organisation is a Christian organisation working in relief and development, which focuses on working through partners but with an operational response when required. The agency presently operates directly with communities and through local partners in the Northern and Eastern regions of Sierra Leone.

Sierra Leone is a country just emerging from a ten-year brutal and destructive war, which started in March 1991 and was declared officially ended in 2002. The war was characterized by widespread destruction of property, water and sanitation facilities including Community infrastructure and the destruction of public services such as health and education. These challenges are realistically beyond the capacity of the government or the local communities to provide the services required and facilitate an effective return of IDPs and refugees.

Tearfund Disaster Response Team (DRT) worked in Sierra Leone from 1996 until 1997 when the situation worsened rendering the country ungovernable. Within this period, the team implemented a relief and rehabilitation programme in Freetown and Bo. The organization’s work was suspended following the invasion of Freetown in May 1997 by the Armed Forces of Sierra Leone and the RUF. Against a background of a ten-year civil war, the signing of the Lome Peace Agreement in June 1999, the deployment of UNAMSIL troops, and British military support , Tearfund DRT returned in October 2000. With ECHO’s assistance, the DRT established a pilot water and sanitation project in two communities (Yendema and Tongoma) in Jaluahun chiefdom, between September 2001 and January 2002. Jaluahun chiefdom is located approximately 80km North East of Kenema in the Eastern Province.

Tearfund’s beneficiaries are primarily returnees, resettling in their original communities following their displacement to camps within and outside Sierra Leone as a result of the

1 conflict. Following an assessment between November and December 2001, six communities were identified. In March three more village communities were identified one of which had reported incidences of bloody diarrhoea out break. By July 2002 Tearfund identified and served four more communities using savings made from the ECHO budget. Work in these additional communities started in October 2002. The project operations are coordinated from the organizations’ regional field office in Daru town. The project period is due to end in December 2002.

The evaluation took place in Jaluahun chiefdom, Kailahun district, Sierra Leone between the period 9-13 December, 2002. This intervention is considered to provide an important mechanism for the organization’s work, transparency and accountability to ECHO, her donor. In this regard, Mr. Sahr Ngegba of Evaluation, Development and Research Associates (EDRA) Consultancy was contacted. The consultant and Tearfund agreed to the Terms Of Reference (TOR) after extensive processes of discussing the whole process of the evaluation.

The Tearfund senior staff including the Programme Director and Programme Support Manager had the opportunity to meet with the proposed evaluation personnel and discussed the draft TOR in November 2002. The evaluation was an end of project evaluation.

1.2 Terms of Reference (TOR)

The aim of the evaluation is to assess the extent to which the project purposes and results as laid out in the project proposals have been achieved. It is also intended to fulfill Tearfund accountability obligations under Tearfund Disaster Management System.

The evaluation exercise covered the periods February to December 2002. The investigation concentrated on the ECHO-funded (Phase II) project – ECHO Tearfund Water and Sanitation Assistance, Jaluahun chiefdom.

The TORs agreed for the evaluation was:

2 1. Appropriateness − To what extent were the objectives of the project relevant to the situation and the humanitarian needs? − Were the activities appropriate in terms of the identified needs of beneficiaries, customs and practices of affected population? − Were the activities compatible with the goals and objectives of ECHO, Tearfund and other implementing NGO’s? − Was the chosen means of intervention likely to significantly reduce the problem for the target population? − How did the situation causing the problem evolve and how has the intervention responded to the changing situation? − Should the programme have been continued for longer or should it have been concluded earlier?

2. Effectiveness − To what extent did the project outputs lead to the achievement of its objectives? − What were the main issues influencing the achievement or non-achievement of the objectives? − Did the intervention reach its target population? − Was the programme implemented as planned? − Was the project cost effective in comparison to other comparable projects, and in relation to appropriate alternative approaches?

3. Efficiency − What were the costs of inputs relative to outputs? − Were the activities cost efficient? − Were the objectives achieved at the least cost? − Was the project implemented in the most efficient way compared to alternative approaches? − Were the objectives achieved in a timely manner? − What proportion of inputs were locally purchased / imported?

4. Impact on communities − What has happened as a result of the project? − To what extent has the project assisted / adversely affected the beneficiaries and other communities? − What impact has the intervention had on the context and underlying causes of the situation? − What would have happened if the project had not been implemented? − The extent to which project components have strengthened or impeded existing coping mechanisms of beneficiary communities and local organizations? − To what extent has the contribution of other implementing NGO’s/organizations affected the impact of this project and contributed to the achievement / non-achievement of its objectives?

5. Coverage & External Linkages

3 − Identify ways in which Tearfund’s profile and relationship with ECHO has been either positively or negatively affected through the implementation of this project. − Did the intervention reach the groups in most need / intended groups? − What were the differences? Were they external or a product of the project design and implementation? − Where any unmet needs addressed and if so by whom?

6. Sustainability − What measures were taken in project design and implementation to improve connectedness between short-term activities and longer-term issues? − To what extent will the benefits of project components be continued after the life of the project and who will be responsible for them? − Is there a clear plan for the hand-over of responsibilities? − Will the project be culturally acceptable and how does it link to the local capacities and power structures? − How will recurrent costs and future expenditures be covered? What is the effect on other economic activities?

7. Co-ordination − How did the agency co-ordinate its activities with other NGO’s and local authorities? − Was it regarded as a constructive and reliable partner by those organizations responsible for co-ordination? − What effects did the level of co-ordination have on the agency’s project and those of other agencies? − How adequately / inadequately did the agency co-ordinate its activities with ECHO? What was the impact if this?

8. Peace building and conflict transformation: • How has peace building helped communities overcome/restore relations among the communities?

9. Spillover of project benefits • To what extent has the presence of the project resulted in spill over benefits e.g. roads constructed from food for work of WFP?

1.3 Methodology

The evaluation was carried out between 9-13 December 2002. A mix of methods (structured and unstructured interviews) was used throughout the evaluation. It started with a task analysis, document review and development of survey instruments (see Appendix I-Proposed strategy and work plan). In order to obtain relevant data, various documents were examined. Interview guides to guide the discussion with beneficiaries and community people were developed. Other instruments were directed towards the

4 programme heads and personnel focusing mainly on themes relating to the implementation, management and partners perception (see Appendix II-Key questions used). Visits were made to eleven project communities (see Appendix III-Schedule of community discussion). In each community, a focus group discussion was held with representatives of the community. The size of the groups varied from one community to the other despite the fact that they were informed of our visits (see appendix IV– Project communities visited and composition of focus groups). Unstructured discussions were also held with individuals including decision makers and programme staff (see appendix V– list of interviewees).

1.4 Limitations and Constraints

During the field visits, Tearfund project staff were very supportive and willing to provide assistance where necessary. The staff also assisted in identifying and contacting the necessary personnel and institutions.

Tearfund Freetown staff were very instrumental in the preparation, making travel arrangements and communication. During the Evaluation exercise, the field office was equally helpful in providing information and facilitating movement to meet project beneficiaries and see facilities provided. Ms. Carol Nyamu and Mrs. Janet Ross-Jordan, the Field Coordinator and Watsan engineer respectively, were especially supportive.

Transportation or availability of vehicles, which is also a primary constraint affecting the work of the organization, was one of the major limitations to the evaluation as the field office has only three vehicles. The exercise took the form of three days visit (see appendix VI–scheduled of interview and activities). The timing of the visit also affected the process; the pressure of work was high especially when the project should officially end that month so there was pressure to complete planned work at different locations. The assessment also took place when community people were engaged in clearing/brushing and harvesting crops (cocoa, coffee, kola nuts, etc) at which time many community dwellers leave their homes as early as 7.00 a.m. and return to town late in the evenings. Only a few people including the aged remained in the towns during the day. Community people who were present made themselves available and were willing to discuss once the purpose of the visit had been explained to the leadership.

5 1.5 Structure of the Report

This part is the introduction and has presented a background to Tearfund’s water and sanitation programme and background to the Evaluation process including the TOR, methodology and limitations to the process.

Part II presents the findings and discussion; perception and impressions of beneficiaries and other NGOs and partners and project impact. While Part III presents the conclusions and recommendations.

6 2.0 CHAPTER TWO:

DISCUSSION OF FINDINGS

2.1 Original Programme Aims and Objectives

In October 2001, when the proposal was submitted to and approved by ECHO, the overall and specific objectives of the programme were:

Overall objective: Tearfund has initiated to enable resettlement and recovery of communities in newly accessible areas of Sierra Leone, which have been devastated following ten years of conflict.

Specific objectives: Tearfund initiated to provide basic water and sanitation services to six communities in Jaluahun chiefdom in order to facilitate resettlement. The extent of achievement of the objectives will be monitored and measured against performance indicators (see Appendix VIII-Project performance indicators).

2.2 Changes in objectives

There has been no major shift from the original objectives stated above. However, to further enforce the achievement of original programme aims and objectives Tearfund has initiated activities intended to facilitate peace building, reconciliation and sustain services provided

2.3 Programme activities and implementation

Tearfund UK has been working in Jaluahun chiefdom since October 2001. In February 2002, a second phase of the programme followed. Public health has been the main focus comprising provision of safe drinking water alongside hygiene promotion in targeted communities.

The programme facilitated digging of wells, protection of water sources, rehabilitation of old wells, the construction of squat-hole family pit latrines in post conflict village communities to support and enable resettlement. A health and hygiene education programme by the PHEs in each village supports the provision of these facilities.

Construction work on wells, latrines and spring boxes were completed in the first nine villages ahead of schedule and under the approved budget cost. Following this, and with approval of the donor partner, ECHO, four more communities were taken on to rehabilitate three wells, construct three spring boxes and a hundred pit latrines (see Appendix VIII-Activities undertaken and status). Presently, Tearfund has provided water and sanitation services in thirteen communities in five sections (see Appendix IX- Communities and location).

Community intervention starts with an assessment and then submission of proposal(s). Community sensitization through meetings with community leaders and their communities follow this stage. Then a memorandum of understanding (MOU) is agreed and signed between the community and Tearfund. Before actual implementation, a baseline data collection through a Knowledge, Attitude and Practice (KAP) survey is conducted among community individuals, households and groups. This survey assesses the general details of population, distribution, and morbidity issues, public health awareness within the community, water and sanitation issues, vector borne diseases prevalence, solid waste disposal, etc.

There are two main components within Tearfund’s water and sanitation programme: water and sanitation facilities and the health education. Other activities include peace building

2.3.1 Water and Sanitation Unit

This component seeks to help selected village communities by constructing new wells, spring boxes and latrines and rehabilitating old wells. With reference to the population size, Tearfund takes a decision on the number of the facilities to be provided per project community. The ratio is used of one water point to 500 beneficiaries (a minimum standard in the district set by the Ministry of Health and Sanitation) and one pit latrine allocated per family situated within their compound.

8 To date, the project has serviced thirteen communities. Nine were earlier intended but following early completion of construction work in the nine planned communities with surplus funds at hand from the approved amount, Tearfund proposed to include four more new communities (Magbema, Bandajuma, Pelewahun and Godie) to benefit from the funds. This was approved by ECHO and work started in October leaving only three months at the disposal of the organization to complete the work.

So far eight new wells have been dug, six rehabilitated, and ten spring boxes, eight laundry areas, and 652 pit latrines have been constructed (see Appendix XIII-activities undertaken and their status). This means that in addition to the original proposal two new wells have been dug, four old ones rehabilitated and six spring boxes, 282 pit latrines and eight laundry areas constructed. The wells are hand dug by the community at the peak of the dry season. The rehabilitated wells are cleared, dewatered, maintenanced and headwall constructed. The water is chlorinated, dewatered a second time and covered. The wells and spring boxes are fitted with a hand pump to access the water, an access hatch and a pulley frame which could be used to collect water with a bucket should the pump break down and to enhance subsequent chlorination of the water in the well or spring box. The spring boxes are located down hill, each well or spring box is fenced to protect the area.

The Clerks of Works and Technicians do the construction and rehabilitation work on facilities, which is supervised by the Watsan Supervisor and monitored by the Watsan Engineer. Tearfund provides the technical expertise and non-local materials including iron rods, cement, CI sheets, nails, etc and also provide transport assistance to ferry sand and gravel where the source is far from the work site. The community’s responsibilities include the provision of the manual labour, i.e. volunteering all the labour to haul the gravel and sand to the site of work, the provision bush sticks, and storage facilities for storing the materials (cement, CI sheets, nails, etc) at the work site, and feeding for the workforce. The community also selects local expertise including masons, and carpenters to work along side Tearfund team. This approach is discussed in each community and facilities are chosen in preference to the available community population and facilities.

9 To facilitate the transfer of skills, 3 community members have received training in pump maintenance as pump mechanics. They will remain in the communities and take responsibility to repair and service the pumps when necessary. In the four new communities, at least four others including women representatives have been identified for training in pump maintenance and repair. For general care of the water source, at least one member has been trained per community. The agency perceives women to be more stable within communities. They are also expected to be more concerned about the said facilities than the male folks.

2.3.2 Public Health Education

This aspect seeks to promote general sanitation practices including in and out of home causes and management of diarrhoea, vector control, worm infestation, cleanliness, safe water uses, preparation of Oral Re-hydration Solution (ORS), home management of fever, HIV/AIDS, and common community diseases such as Malaria and Cholera. The Public Health Educators (PHEs) have also been involved in Child-to-Child health education in schools, and Peace Building. This component is given high priority by Tearfund and complements the Watsan component.

The PHEs are the key facilitators of this unit. These educators move into the desired community before hand and live with the community to enhance understanding of the culture, community behavior and knowledge. The team of educators encourages the community to select individuals through discussion with the village elders, decision makers and group (men, women and youth) leaders based on certain criteria such as gender balance, section or zone, previous experience and willingness to serve their community.

The individuals selected are then trained as Community Health Volunteers (CHVs). This is central to the functions of the PHE component. The number of CHVs trained per community depends on the population size and number of facilities. These volunteers once trained help sensitize and train other community people and monitor disease outbreak in their respective communities. The PHEs while in the community spend time with the CHVs and the community leaders, encouraging them to motivate the community to implement good hygiene practices such as erecting washing lines, plate racks and compost fences or proper refuse disposal. To date, 64 males and 72

10 females (a ratio of 47% men to 52% women) totaling 136 in all have been trained in 13 villages within the 11 months of operation. The government of Sierra Leone and the United Nations Children’s Fund (UNICEF) initiated this idea in 1996, to focus on hygiene promotion messages around diarrhoea disease.

The training focuses on the importance of good hygiene practices in and out of homes. This includes hand washing, the importance of safe water and its protection, vector control and safe disposal of excreta (particularly infants) for which potties are distributed, rubbish disposal and compound cleanliness, home management of fever, HIV/AIDS and Peace Building. There is an important component on the recognition of diarrhoea, particularly in infants and children, and its treatment with ORS. Trainees are taught how to prepare ORS, and are given supplies with which to do it. These supplies are part of the hygiene kit that trainees receive: one bucket with lid, 2 pieces of soap, a jug and mug for pouring water, a spoon and salt and sugar for making ORS.

The training adopts an active learning approach, and highlights a participatory way of encouraging communities. CHVs are key community mobilisers, who have been the primary support to the construction of facilities. Follow up support is provided by Tearfund employed PHEs who carried out training. This sustained support helps to ensure that the impact of CHV training is maximized and that communities receive constant inputs on appropriate hygiene behavior. It also means that CHVs are important points of contact for Tearfund in monitoring and learning from all public health work.

To support the process of handing over responsibility for facilities to communities, three pump technicians were trained to support the care and maintenance of pumps. Tools for the pump technicians have been given to them. Communities visited were however, concern about the fact that Tearfund has supported only 14 communities so far while there are over 100 villages communities in the chiefdom with the same problems. Despite this, people have reacted relatively positively overall to the need to organize themselves to support the maintenance of facilities provided by Tearfund. Time will tell whether the structures in place i.e. CHVs supporting PHE, and Community pump technicians will be able to do this.

11 Consultation is a major strategy used by Tearfund in their working communities. The Clerks of Work and the PHEs are in constant contact with the community and encourage them to volunteer their labour and employ better health techniques. The PHEs also provide health education messages to children in schools. Recently, the PHE Unit has been involved in sensitization on issues of gender equality in the communities.

Peace Building and reconciliation activities have also been implemented in the communities by the PHE unit. To enforce this, Tearfund and its partner, Evangelical Fellowship of Sierra Leone (EFSL) facilitated a workshop on Peace Building with emphasis on causes of conflict, mediation and reconciliation. Over eighty participants attended the training including Tearfund staff, Christian and Muslim leaders, chiefs, women and youth leaders from communities served by Tearfund. Some participants have organized workshops on peace building in their respective communities and facilitated conflict resolution among relations or community people. Interviews revealed testimonies of reconciliation and acceptance of former combatants into their communities – for example:

• “We are now united, formerly we (ex-combatants) were rejected even by our own friends and close relatives but since that peace workshop we are now living together peacefully”.

Findings show that the population of inhabitants in the target communities has increased. In some cases the population was reported to have doubled and continues to increase. Although there are no clear indicators to confirm that the return of people is as a result of Tearfund’s interventions, it is perceived by respondents that this increase is associated with the work of Tearfund in the respective communities and to a greater extent housing and accommodation facilities. The number of houses, built from the inception of the intervention of Tearfund is one of the major indicators of increase in the population of the communities (See Appendix X–Original and present population of operational communities).

To facilitate sustainability, Tearfund is experimenting a cost recovery strategy in addition to the training of CHVs and pump mechanics. This strategy is under trial and is very participatory. It involves community meetings that discuss various issues such as ownership and maintenance of the facilities when Tearfund exits. The whole process involves questioning, discussion and suggestions. How can the facility be best

12 maintained? Who will pay for such services? And how? Managing resources and developing by-laws for the group are among issues discussed. The discussion most often takes place in groups sometimes determined by age or sex. The groups report back in the plenary and through facilitation by Tearfund staff, the findings are refined and a common consensus reached and agreed by the whole community.

2.4 Perception and Impressions of Beneficiaries, Partners, other NGO’s and CBO’s.

2.4.1 Beneficiaries Of the 13 communities served by Tearfund, 11 (85%) of these communities were visited and focus group discussions held with beneficiaries. The groups varied in number from one community to another with male and female representation who participated in the discussion (see Appendix X). Responses (not edited) are summarized below:

Beneficiary’s views on Tearfund UK’s Activities Tearfund • Promotes environmental hygiene and cleanliness • Provides safe drinking water, toilets and latrines • Helps the poor • Drives away various diseases • Helps people to survive by providing cleaner environment, with less mosquitoes and flies • Promotes long life • Trains community volunteers in sanitation to keep the environment clean • Organize workshops on peace building and community development

Beneficiary’s views on things done or are being done well • The provision of safe drinking water • The construction of compost fences and toilets • The provision of kettles, rubber buckets and cups for drinking • The provision of toilet facilities which has made our environment cleaner • Training in hygiene and sanitation as well as teaching us to prepare ORS • The provision of washing points • Organizing workshops on peace building

Beneficiary’s views on things not done well • Washing point has not been provided in some communities while in others it is • Not provided materials for some toilets • Spring water is smelly • Tearfund promised to assist us to repair our roads especially the bridges but yet to come • Not providing enough toilets • Some communities have received drinking and others house hold utensils while others did not • Tearfund did not keep to their promise of providing more toilets because of more people arriving

Beneficiary’s views on priority needs Interviewees were requested to give their view on whether Tearfund is meeting their priority needs? Of the eleven communities 9 (80%) responded yes while 2 (20%) responded No 13

Yes No 80% 20%

Beneficiary’s examples of priority needs met • Tearfund has provided us clean drinking water that is assured through the year • Provided toilets so our environment is cleaner and health is wealth

Beneficiary’s views on priority needs not met • Not provided shelter as they promised • Tearfund has not provided health facilities

Beneficiary’s views on whether “the poorest are served” Are the poorest and most vulnerable served Of the 11 communities interviewed, 10% responded no while 90% responded in the affirmative

Yes No 90 10

Beneficiary’s views on the extent of community participation Respondents were requested to rate the extent of community participation using a scale of 1-5, i.e. 1.Poor; 2. Fair; 3. Average; 4. Good and 5. Very Good

Rating/Scale 1. Poor 2. Fair 3.Average 4. Good 5. V. good 0% 0% 15% 25% 60%

Beneficiary’s views on community involvement • The community provides the local materials (sand, stones, sticks) and the manual labour • We undergo training to do it ourselves like teaching others on hygiene • We the community make our laws for the facilities

Beneficiary’s views on Tearfund’s Impact on the community • We now have cleaner and airy environment • Clean and pure water is always available even mid night you don’t have to go far in the forest like before • Now we dry our clothes on lines and we know its value • There are lesser mosquitoes and flies around • Now it is possible to conveniently get into and clean our plantations on the periphery of the township because it’s cleaner. It is no longer used as toilets, we can even collect fruits that drop and sell or eat them with out fear • Lesser disease incidence e.g. diarrhoea and malaria were rampant in their communities and killed many people. • People spend less on medication now. We can prepare ORS and take care of diarrhoea, fever attack etc

Beneficiaries’ views on what has changed • We no longer dry our clothes on the ground or dispose of waste just like that • Waste is disposed in a confined area (compost fence) off the house 14 • We have a cleaner environment and less disease outbreak • Has clean drinking water which we drink without fear of getting sick • There are lesser incidences of snake bites, lesser mosquitoes and driver ant invasions in our homes • We have fresh air- used to find the town edge stinking with human waste, and children no longer defecate near the houses

Beneficiary’s views on Impact of Tearfund’s intervention on the return and resettlement of their communities • Very high almost 100% • Contributed greatly • Tearfund has provided all the basic things we need especially after the war except that they have not helped with shelter

Beneficiary’s views on what would have happened if Tearfund had not intervened into their communities • Most people would not have returned yet • Many people would have died especially of diarrhoea and vomiting • Defecating around houses by especially the children would have been rampant making the environment very smelly • Many disputes and quarrels, no unity, grudge and no meaningful development

Beneficiary’s views on future interventions Tearfund • Help provide medical facilities with facilities for delivery • to help provide shelter • to help the community construct schools and places of worship • to identify and train, retrain and equip TBAs • logistic support like tools and seeds to self help groups • to help restock their animals • to provide chemicals for control of mosquitoes in pit latrines

2.4.2 NGO’s and CBOs Over 20 NGO’s operate in Kailahun district nine of which (NGO’s and CBOs) have offices in Daru town. Five of the nine and one based in Freetown responded to the survey.

NGO’s and CBO’s views on Tearfund’s mission and business • To provide relief services • Rendering assistance to affected villages • Providing water and sanitation services • Supporting social facilities such as football field preparation • Supporting group activities e.g. cassava farming • Digging wells • Promotes HIV/AIDS preventing campaign

15

NGO’s and CBO’s View on Tearfund’s programme strengthens • Providing drinking water and sanitation to remote communities • Creating and facilitating unity among communities • Bringing the youths together • Helping the poor • Having very good reputation within the community • Doing a good job • Open to share ideas with other partners and beneficiaries

NGO’s and CBO’s View on weakness of Tearfund programme • Not working in the headquarter towns (Segbwema and Daru) where there is also a serious problem of clean water which is causing problems for the people • Needs to step up co-operation and collaboration with local NGOs and CBOs • Very seldom attend inter-agency meetings-Kenema • Do not work in the community they live

NGO’s and CBO’s Views on Tearfund’s approach • Good: uses bottom to top approach • Develop their programme and work with the beneficiaries

NGO’s and CBO’s Concern about Tearfund’s work • There is need to work in the headquarter towns which will send signals throughout the entire community • They seem to be under-staffed

2.5 Project Impact

Interviews with residents of Daru Township and several client villages visited showed that Tearfund is held in very high esteem in the area. It appeared to be the wellspring of development activities in Jaluahun chiefdom.

Those who have benefited directly from the project activities are the entire inhabitants of the 13 villages in which the project operates. Interviews with the communities and evaluators observations showed that the agency has served more than the intended target population of. 9,400 (see appendix XII). Realistically the project has served not only people in the designated villages in which the project operates but also returnees who chose to use these villages as transit points before moving to their own villages. In the event, valuable skills are being transferred to non-targeted communities where Tearfund is not implementing.

In the absence of data on new arrivals and up to date census figures, it would be invalid to calculate the total number of individuals served by extrapolation. However, there are

16 evidences of more new houses being constructed or completed and families who have just returned to resettle in the communities. In material terms, these beneficiaries have gained from clean and protected water facilities, water storage and carrying containers, cups, soap, kettles, toilet facilities and training in the Watsan programme of Tearfund.

Each component has contributed towards the project’s total impact. It is commendable that some of its activities are collaborative efforts with the government, Ministry of Health in the Kailahun district, OXFAM, etc.

The beneficiaries are a mix of both Muslims and Christians. The agency has served several thousand people. Findings showed that the programme targets everyone within the project village.

Social change: One very important aspect that beneficiaries are learning is the value of social cohesion i.e. the awareness of the benefits of community work such as working in groups, managing group activities as well as developing group rules and procedures. The agency has worked especially hard in motivating people to work in groups rather than as individuals, which was the usual practice before Tearfund’s intervention in the area. This has helped villagers to be more united and promote a peaceful co-existence in the target communities.

There was an impressive level of understanding amongst those asked of the transmission route of diarrhoea, amongst all age groups and in the villages. 50% of people interviewed demonstrated knowledge of this subject – transmission by flies to food and water, water contaminated by faeces and the risk of defecating in the bush. There has been considerable change in the people’s attitudes. In the past it was a common attitude of village communities to use the bushes, backyards and town edges for defecation and they were reluctant to construct or declined to use toilet facilities. However, from evidence obtained in the evaluation visits, it was obvious that there has been a change in their attitude since they are now very enthusiastic about having and using toilets facilities and those that have not been able to secure one are now demanding one. Observation of family latrines provided by Tearfund support show that 70% of those inspected are well kept and do not give out bad odour. Most people said they no longer went to the bush to defecate instead they prefer to come back to the town 17 to use the facility available. They stressed various ways of taking care of the surrounding of wells, and mentioned the use of potties by young children.

Further suggestion of the impact of PHE’s ability to influence communities is given by the fact that some members interviewed said that they kept their drinking water in a bucket or in jerry cans with lid. In addition, members of all the communities visited 15% said they carried out daily cleaning of their latrines. The reminder said that they carried out brushing with hard brush without specifying how often they did this.

The evaluation finding shows that the peace building and reconciliation awareness raising activities have contributed immensely to reconciliation and a peaceful co- existence among inhabitants of the project villages. It has also contributed to the understanding of and community participation in the well being of the society.

Prevalence of disease: Interviews with the beneficiaries revealed that before the intervention of Tearfund in the Jaluahun Chiefdom in 2001, the prevalence of diseases such as diarrhoea and vomiting, lassa fever, waterborne diseases, malaria, worm infestation etc was very high and accounted for an estimated 50% of deaths in the chiefdom. However, since the inception of Tearfund Watsan project in 2001, the situation has improved immensely. According to the clinical data from MOH and respondents, there was a significant reduction in diarrhoea from 50% to 5% in communities where Tearfund have completed their activities. It can be assumed that effective hygiene promotion and the provision of a safe water supply played an important role in this. However, there were frequent requests from the community members for improved medical facilities, and comments on the difficulty of referring serious cases of diarrhoea when the clinic or health centre is often several miles away and patients have to be ferried in hammocks to Daru Township. The only hospital (Nixon Memorial Hospital) was vandalised when over run by the RUF during the war and is presently not operating.

Quality and, quantity of services provided: All protected water sources provided by Tearfund tested safe according to faecal coliform counts. The surrounding of all the 7 water points observed was clean and fenced. 80% of the communities contacted said the taste of their well was either good or very good. The remainder said it was either rusty 18 or reddish in colour in two villages, which is caused by iron. 60% of the people interviewed collected water from a protected well or spring and queues were not observed by the wells.

Quantity of water provided in the villages that were visited.

Community Population Total quantity of Quantity of water Remark water available (litres) available for 12 hours per person per day Feijia 601 30,400 50.58 This quantity will drop to half in the dry season. Mamboma 120 15,200 126 From 1 spring Yekandor 229 88,920 388.30 From I spring Sembehun 771 104,120 135.5 From 1 well plus spring Taninahun 400 88,920 222.30 From 1 spring Gbeika 520 104, 920 201.77 From 1 well plus spring

The importance of family latrines was generally recognised, and there were requests for support for more latrines with reference to the addition of population in future because most people have still not returned to their homes. Some CHVs mentioned however, the need for particular skills in sensitisation of communities about latrines, in a context where many people are more concern about finding enough food to eat. There was complaint of slow or unsatisfactory completion of the latrine squat plates in certain villages visited. The delay in completion has also been attributed to the communities delay in providing the local resources such as sand and stones.

Sustainability and self-reliance: The training provided to community members and volunteers for the maintenance of the wells, pumps and latrines has definitely built up their capacity to take up maintenance and care of the facilities provided. With the support of Tearfund’s PHE staff, the community sets by-laws for the care and use of hand pumps and their surroundings, and CHVs and community leaders are responsible for ensuring that community members observe them. The good condition of the wells suggests a sense of communal ownership and responsibility, and the key importance of CHVs as promoters of these kinds of positive attitudes.

Encouraging group activities has been very instrumental in curbing labour shortage within communities, which has further provided assistance to pilot communities in a cost effective way. Before now, women in the said communities did not partake in decision making or talk in gatherings rather than observing. However, with the 19 intervention of Tearfund, the community has discovered the advantages of allowing women to co-operate with the men on community issues. The involvement of beneficiaries in the administration and management of community activities has greatly empowered the communities to understand and cope with problems and difficulties that will be envisaged in future.

This evaluation indicate that the majority of the people in the villages where Tearfund has worked benefit from a clean and sufficient water supply (see Appendix XI-structural and water quality data), and understand the importance of safe water for good health. Health statistics that are available support this, showing the level of diarrhoea disease is relatively low, at 5% all cases reported in clinics.

There appears to be a good understanding of the importance of latrines, which are used and well kept. There is evidence that people understand the value of environmental hygiene, reflected in the use of, for example, plate racks and compost fences, and keeping the surroundings of wells clean.

There is an evidence of good co-operation within the community to construct and support facilities, including the involvement of women in the construction activities and the collection of fees to support the maintenance of pumps.

20 3.0 CHAPTER THREE:

CONCLUSION AND RECOMMENDATIONS

3.1 Appropriateness

It is clear that Tearfund interventions are greatly appreciated by the beneficiaries. On the whole, Tearfund is seen in a positive light by beneficiaries and partners. Although 80% of the beneficiary communities visited reported that their priority need has been addressed a small majority (20%) disagrees. For this group, shelter and health centre facilities were revealed to be most important. Discussions (focus group discussion) reveal that at least one community accepted Tearfund services basically because everyone is looking for survival and there is no option. The beneficiaries were not informed about Tearfund’s purpose, duration of its support and expect Tearfund to stay with the community endlessly doing everything.

The communities revealed that the services provided by Tearfund are the ‘life-line’ to their survival. ‘Providing us with clean water especially after a bloody war during which people died and were not properly buried is a priority’. People who died or were killed during the war were simply thrown into pit latrines, streams or dug-out wells. Early 2000 when people actually started returning many people died from drinking contaminated water and food. However, at present, deaths due to these causes are reportedly negligible. People, especially children, no longer use back yards or bushes to defecate or dispose waste as it used to be the practice in the target communities.

The activities of Tearfund in the Jaluahun community have definitely contributed to the return and resettlement of returnees into the area. The approach used by Tearfund is participatory. It is a consultative process involving all stakeholders including the beneficiaries in deciding on their own development priorities. In the partnership, Tearfund and the beneficiary community share control over development initiatives, decision-making and the provision of resources which affect the beneficiaries or the implementation of the programme. This has led to capacity building and empowerment of the local communities. The beneficiary communities are now aware of the implications of improper sanitation and poor hygiene conditions. Generally, their knowledge, attitudes and practices towards public health has improved. This

21 knowledge has significantly reduced the prevalence of health problems in their communities. However there is need to:

• Maintain planning strategies by involving the community to participate in the process from day one

• In particular Tearfund staff need to consider the best way to approach communities where Tearfund is starting work for the first time to ensure that the length or duration of Tearfund’s support is clear from the beginning, to avoid raising unrealistic expectations.

• Sensitize or define Tearfund to the target community: what does Tearfund stand for? What does Tearfund do? How do they it? Who will benefit? How? How long is Tearfund with them? Who will do what, own what? What will Tearfund be doing and when?

• Get the community to contribute more and be prepared to take ownership of facilities on exit

Although the beneficiaries have a positive perception of Tearfund, this image is slowly being replaced with a negative impression particularly as more people return. The facilities will be over burdened and hence liable to damage in a short time. Toilet facilities in most communities, are no longer adequate, most houses have no toilets. In view of this, it is recommended that:

• More time and facilities particularly toilet facilities be provided per community to enforce the impact and make it more meaningful to the beneficiaries

• Empower and encourage communities to find ways of providing these facilities locally with minimal or no external support

All water supplies provided by Tearfund are safe in terms of faecal coliform contamination, and people are happy with the taste. People in most villages are receiving up to 20 litres of water per person per day, though in the most populous villages people complain of still having insufficient water supply. People in most of the villages supported by Tearfund had a clear understanding of the causes of diarrhoea, and many were able to explain how to make oral re-hydration solution (sugar/salt solution) to treat diarrhoea

22 • There is need to empower the community through training or otherwise to be able to take on for themselves the construction of such facilities for their communities

• Training communities to cast slabs and construct latrines for themselves in future will make Tearfund’ s communities continue after the life of the project.

The pumps fitted on the wells look good, however, technically they are proving not to be ideal. In particular, the spring boxes are shallow while the pumps used (Indian Mark II Pumps) are more suitable for deep wells. Even the effectiveness of the pumps is diminished with decrease in depth. Another known implication encountered with this make is that it could suck up mud, the weak pressure could cause the handle of the pump to hang up and require several pumping actions before water is sucked up, as already the case in one of the communities visited which could become worse in the dry season.

• Identify pumps adapted or suitable for shallow wells more durable, have available spares, are easy to operate and could be much easily managed by the community.

• Design spring boxes on hill sides where possible and fit with taps (gravity flow)

3.2 Effectiveness

At the early stages of the project, co-operation from beneficiaries was restrained as a result of the malice, grudge and traumatic status of the beneficiary communities caused by the brutal war and displacement. During the evaluation, it was discovered that detailed investigation was not done in the identification of springs for protection, like colour and the amount of iron content before construction, similarly no technological standards or bill of quantities were realistically investigated before project inception. This may have had a de-motivating effect on the communities after realising that the facility for which they have worked so hard is not useful to them. It is therefore recommended that:

• Thorough investigations and bill of quantities per facility be determined for realistic costing of facilities and monitoring of activities during planning

23 • Ensure technical standards and designs are available to work out bill of quantities for all structures and training before implementation. This will also help field staff to request for the actual quantity of materials they need and to develop their skills to interpret plans in future for cost effectiveness

• Water should be tested for suitability for consumption or presence of impurities before investing into its construction

Children, particularly those under 5 years, are barred from collecting water from hand pumps in some communities. This is said to be because they are believed to damage them, and appears to come from a sense of ownership and responsibility for the care of the pump. The effect this has on the workload of their mothers could be investigated further for future work.

• Tearfund and its project management need to rethink the issue of ownership and sustainability and promote it more vigorously.

The organization has done more than it intended to do within the set time frame of 12 months (see Appendix XII – Logical Framework Matrix). The management has undertaken to introduce the idea of cost recovery by introducing a management system manned by the community itself.

Over 600 family latrines, 10 spring boxes and 8 new wells were constructed, while 6 old ones were rehabilitated. Washing points, which were also not initially included in the proposal, have been constructed (see appendix VIII: Activities undertaken and their status). More villages were served than planned. However this decision has had somewhat negative impact on the work of the PHEs. Taking on extra communities implies that the time spent on the communities actually planned for is limited hence limiting the anticipated impact

The following is therefore recommended:

• The scope of activities be restricted per target community to what the agency verifies as the most pressing and realistic needs of that target group and that which the agency could do best within the time period

• Reduce operational areas to match the capacity of the agency in terms of staff and resources as well as time at the disposal of the field office

24 • Concentrate activities over a manageable area for a reasonable period to allow for the implementation of all the steps so as to ensure as well as maximise the expected impact before eventually moving out to other communities of choice

PHEs also indicated that their work was more difficult because the communities who were just returning to resettle had the burden of providing feeding for them in their communities particularly so when people do not have enough food to eat themselves. The aspect of PHEs could be covered more fully, in terms of discussing how best to work with communities in such a context in future training courses.

• Feeding for field staff should be included in the financial request for donors to cover the cost.

Tearfund has done less in the area of gender. The evaluator found that the women did not speak or participate fully. It was also clear that women did not believe that their opinion could work among their male counterparts and preferred the men to talk for them although they made good suggestions and contributions during discussions. Such behaviour could prevent decisions taken from reflecting the needs of both sexes. It was also found that the younger men were more opposed to the women than the aged men. In most communities visited, the men made the decisions, which is considered counter to the social development of the women. Speaking to some community members, it was mentioned that the women role is to prepare food at meetings. It is therefore recommended that:

• Initiate programmes that affect or concern women directly i.e. gender sensitive programmes such as human rights and civic education, the right of the girl child, and women’s empowerment through micro credit schemes so that they have more say in their communities etc

• Tearfund field staff investigate what women and men in their target communities see as appropriate participation by women and how it could be improved

25

3.3 Efficiency

In terms of efficiency, Tearfund intervention in Jaluahun chiefdom can be described as excellent. In relation to the project proposal, more facilities were provided from the same funds than was planned and more communities were served. This gave the impression that the project was efficient and costed less.

Strategies were effective and acceptable to the communities. It took the form of partnership in development with all sides contributing to the achievement of the objectives. While Tearfund contributed the technical and financial support, the beneficiaries provided the resources locally available. This approach contributed to the timely achievement of the set objectives. There were indications of important positive results of Tearfund’s work in Jaluahun. The public approach used by Tearfund in the provision of safe water and sanitation facilities alongside hygiene promotion has been demonstrated to show positive result.

The Community Health Volunteers trained by Tearfund to promote hygienic behaviour and administer ORS, are well respected by their communities. Though women seem to prefer the back seat, they are seen to be active and effective. They are actively involved as CHVs and appreciate the ability to respond to diarrhoea. They have participated in construction activities and in some communities women are responsible for collecting money for the pump maintenance. There are women members in each group of CHVs identified, in some communities they are said to participate actively in discussion, while in others their main role was said to be food preparation.

The management committee at village level is ready to take responsibility for the maintenance and care of facilities. Money has started to be collected to support the maintenance of facilities and pump caretakers have been trained. The training of community health volunteers, supported by the management committees seems to be particularly effective in encouraging and sustaining positive hygiene practices. Since the evaluation, this has been complemented by the training of community pump technicians who carry out basic repairs on the hand pumps.

26 3.4 Impact on Communities

It was revealed by beneficiary communities during interviews that should Tearfund not intervene into their communities in these areas, these areas would have been prone to disasters resulting from diseases such as diarrhoea, lassa fever, malaria and water borne diseases to name a few. Some respondents also revealed that they would not have dared to return so soon. Other impact on community as observed by the evaluator includes the following:

• Communities are much more stable and safer, population has increased with more people moving in to resettle from larger cities and camp centres.

• The community people’s health is improving from poor to healthier conditions

• Initially the communities were not aware of the causes of common diseases like diarrhoea

• The communities realize that their activities are greatly responsible for certain common ailments in their communities

• Sanitation facilities have improved. While the incidence of common diseases (malaria, water born diseases, diarrhoea etc) considered fatal before the intervention of Tearfund has declined drastically

• Beneficiaries have access to basic services including clean and protected water and do not have to walk long distances to fetch water

• There is attitudinal change in communities. Through health education sensitization ventures by Tearfund, community people can now identify the need for certain facilities e.g. toilet, proper refuse disposal, etc

• They are able to manage diarrhoea or provide first aid support to diarrhoea patients until they reach medical help

The project has contributed immensely to the state of affairs in their operational communities. In the early period of return to these communities the state of affairs was characterized by suspicions, hatred, grudge and grievances especially against perpetrators/ex-combatants and to some extent chiefs, some of who actually abandoned their people on the eve of attack on their communities. However, the peace building activities of Tearfund has facilitated reconciliation and unity among such people. There are now more co-operations with authority, former combatants are accepted, etc. No other NGOs were found to be working in the project villages. These achievements can therefore be regarded as the sole property of Tearfund. In view of the above, the team recommends that:

27 • Tearfund encourages peace-building activities in target communities and support peace initiatives by the communities themselves

• Provide training in participatory methods for PHE staff such training should include developing and using PIM tools

• Tearfund together with beneficiaries, develop a participatory monitoring scheme to facilitate monitoring of impact on community and the achievement of the over all intentions of the programme in their communities.

3.5 Coverage and External Linkages

ECHO has been committed to funding Tearfund DRT. Tearfund’s profile and relationship with ECHO has remained positive. This is manifested by the confidence and certainty Tearfund personnel exhibit – ‘we are confident that ECHO will approve funding for our activities for 2003’ a senior official claimed. Project activities have also locally raised the profile of Tearfund in the Watsan sector.

The field office has limited direct contact with the donor ECHO. The majority of the discussions and contact is through the Tearfund office in Freetown. During the period under review, ECHO made only one visit to the field office. Such visitors have limited knowledge of what the actual situation in the project area is. It is therefore recommended that in future:

• ECHO to make more frequent visits than previous years, such visits could be included in the programmes monitoring scheme

• ECHO makes direct contact with and visit the field office to see and feel the situation on the ground

Tearfund operates in the Eastern and Northern regions (i.e. Kailahun in the East and Port Loko district in the North). In these regions, Tearfund targets whole communities especially the returnee population. More facilities were provided in more communities than planned with no additional resources. In addition to the Watsan activities, other unmet needs were also addressed including peace building and promotion of peaceful co-existence among project beneficiaries.

All the pilot communities have enthusiastically received the agency. Its interventions are reportedly considered to meet the primary needs of their communities and are very timely. Although considered geographically widespread in the regions, its activities are

28 however perceived by beneficiary communities as sparsely distributed assisting less than a third of the communities in the Jaluahun chiefdom so far. The population of the target communities that were very low at the time of intervention is increasing rapidly. More families are arriving and houses been constructed. In view of the foregoing discussions, the evaluation team recommends that:

• The project avoids rapid expansion with respect to activities, the size of clients and area of coverage.

• The extent of project area should be confined to a realistic radius from the project base

3.6 Sustainability

Since the planning stage of the programme, sustainability has been an important ingredient in the strategies though low key. Findings revealed that Tearfund earlier discussed the issue of further funding for continuation and sustainability with ECHO, the donor who suggested Tearfund to seek further funding from Sierra Leone Resettlement and Rehabilitation Programme (ECSLRRP) and the discussions are still on going.

To facilitate continuity, Tearfund has put in place structures including training of community health volunteers who will train others, educating volunteers to monitor and manage the facilities provided, and organizing management committees at village level to manage and organize the community to earn funds to facilitate repair and maintenance work on facilities. In addition, the volunteers and community has been sensitized in the use of various items including steel and rubber buckets, kettles, potties and soap.

The facilities provided, in addition to the sensitization of communities on hygiene and cleanliness, were found to have impacted on economic activities of the beneficiaries. The beneficiaries are healthier and stronger and can work to earn more while they spend relatively less on diseases and other implications associated with poor health and hygiene conditions, which were common. In this respect, the following are recommended:

29 • To enforce sustainability, Tearfund needs to introduce after project monitoring systems. Introduce a budget line in the programme for follow-up after project activities. The duration decided realistically and enforced. This set-up will see whether the systems put in place are working, and provide supervisory support and advice to promote continuity. Such structures will also help Tearfund to go back to previous communities of work and correct certain structures in place, which are not working well.

• Enforce use of locally available resources, including human. This will promote sustainability and encourage the community to take on the provision of such facilities hence enforcing the capacity of Tearfund’s target communities rather than maintaining Tearfund’s physical presence.

• Provide minimal logistical support in the form of basic tools to local experts such as carpenters and masons in target community, nominated by the concern community beneficiaries to facilitate take over from Tearfund. They will take over the construction of the facilities in their communities

To facilitate or promote sustainability, Tearfund field staff undertook to maximize involvement of the beneficiaries from the inception stage. The community is encouraged to participate in identifying the community needs, what to do about them and who will do what. They are encouraged to make more input than just physical things. They have been encouraged to organize closing ceremonies, which mark the end of Tearfund intervention in their communities.

It has also involved the government by consulting and working within the set policy guidelines in as much as this gives the feeling of government’s presence. Government also appreciates that what the agency is doing is what is needed which will also ensure continuity by the government should the agency exit.

3.7 Coordination

Tearfund takes part in meetings and functions with other NGOs (both national as well as international), government and UN Agencies. Tearfund’s collaboration with other NGOs in Sierra Leone takes place via inter agency fora coordinated by UNOCHA.

Tearfund is well known in Daru Town. The agency has an excellent reputation among other NGOs, the Ministry of Health and the Ministry of Energy and Power. They are considered as a leading NGO. Their staff are known to be very efficient, delivering

30 their promises and good quality work. All local NGOs that were asked to rate Tearfund answered with ‘above average’.

Among locally based NGOs, Tearfund is known and there are links between some of them. Some know Tearfund from the inter-agency meetings or have heard about them and their work in the chiefdom while others like WFP have personal contacts with staff. With some international NGOs, Tearfund has established good contacts and relationships.

Local NGOs consider Tearfund to have a very relevant programme and an appropriate community development concept. It is seen to contribute to the rebuilding and development of the war torn communities in Jaluahun Chiefdom of Sierra Leone.

Tearfund maintains good relationships with government departments. This is demonstrated by the invitation to meetings by the ministry and consultation with such departments to ensure that the agency is in line with the policies of government so as to facilitate continuity on the departure of Tearfund in the said communities.

Despite all the aforementioned positive points, one of the NGOs interviewed alleged that Tearfund’s participation in inter-agency meetings and collaboration are limited. This is however perceived by the evaluator as a personal opinion. In view of the above conclusions, the following recommendations are made:

• Tearfund to maintain and further strengthen their relationship with other NGOs generally operating in the area of Watsan activities in Sierra Leone and specifically agencies implementing water and sanitation projects in their regions of operation

• Network with other humanitarian and development agencies in the Eastern and Northern regions to facilitate information sharing and experience exchange

• Encourage information sharing with other NGOs so that activities or services, which cannot be provided for by Tearfund can be communicated to such agencies for possible intervention. In return others might make available information about communities most in need of Watsan facilities, which could be discussed by Tearfund for possible intervention.

31 3.8 Peace Building and Conflict Transformation

Some community people including ex-combatants who were rejected by their communities were hesitant to return while others identified hideouts where they were based due to the fear of reprisals. However, the evaluation exercise revealed that following the peace building and conflict transformation workshop held in Daru by Tearfund in collaboration with the Evangelical Fellowship of Sierra Leone, community representatives deemed it necessary to organize similar workshops for their people and have fostered reconciliation and forgiveness. Such individuals (ex-combatants) are now living in their communities and peacefully co-existing.

3.9 Spill Over Effect of Programme Benefits

Tearfund UK’s impact is not restricted to direct material benefits and the provision of water and sanitation services, limited though this may be. From interviews it was revealed that the formation of Community Management Committees and training Community Health Volunteers in target villages, through which community members engage in communal activities and help one another in times of need, is considered a positive result of Tearfund’s activities in the region.

It was apparent from interviews and discussions that repairs and maintenance on the road network within Tearfund’s operational communities have been made possible by Food-For-Work which Tearfund obtained from WFP. Though this was intended to facilitate Tearfund’s access to the said communities, it has resulted in a spill over benefit facilitating vehicular traffic within the chiefdom.

Discussion with the clients and other beneficiaries continually stressed the benefits brought them by Tearfund. However, a number of respondents indicated that should the project be closed or terminated for any reason, it would be very difficult to continue especially when people are just returning to start life. The evaluation team interprets this as evidence of dependence. This calls into question whether in fact Tearfund’s intervention has been able to promote the notion of self-reliance in the target communities.

32 Much more therefore must be done to promote self-reliance. This is not to say that the project has done nothing in this direction. It must be remembered that for the period under review even though the war had ended and the project area was accessible, Tearfund is the only major agency that has been operating in the thirteen communities. The people have grown accustomed to the organization as an institution in their communities and might find it difficult to divorce themselves from Tearfund which has been so much a part of their lives and regarded as a saving grace in these communities helping them in restarting their lives. Consequently, these communities may not want to imagine their communities without Tearfund let alone see it close or exit their communities. This attitude of dependence or handout system as already mentioned is one of the legacies of the over ten-year civil war characterized by relief services.

• Get the community more involved in action planning, after identifying what is to be done. What will their contribution and involvement be?

• Initiate plans to facilitate community projects through -Identification of at least one local carpenter and one mason per target community who will be provided basic tools to facilitate the provision of more toilet facilities for families not provided for, purely on the community’s own accord -Create a communication link between the beneficiary community and Tearfund’s field office to facilitate technical support to such group should need be

• Greater attempts must be made to help beneficiaries take up more responsibilities or ownership for their projects and for themselves.

At present the field office has 35 staff, 30 local staff, 2 expatriates and 3 security guards. Tearfund is considered well staffed by other NGOs, which is a source of pride to the agency. Additional communities and activities were added without taking on new staff. This suggests that part of future growth may be possible with current staff composition at office and field levels. In addition, observations of staff reveal that a lot of time of programme staff goes to record keeping. Possibly, this could be streamlined. It was also revealed that a majority of the technicians are recruited and treated as casual workers with no contract. This is not only against government policy but it also implies that such workers do not get any identification cards, feel part of Tearfund, enjoy medical benefits and that their safety is not treated with much seriousness. Hence, as manifested, the morale of such technical staff is low despite the high esteem in which the beneficiaries hold it. As a result of the above, the following are recommended: 33 • Recruit technical staff and confirm or disapprove appointment after a reasonable probational period of at most 3 months and review every year as necessary

• Issue contracts to technical staff detailing conditions of service and benefits due included in staff package.

• It is therefore recommended that staff be trained/empowered either through in- service courses, exchange visits or refresher courses.

• Review the conditions of service taking into consideration, the additional cost of staff benefits and other emoluments, which could be borne, by ECHO, the donor partner or Tearfund.

• Technical staff in particular should be protected from danger and be provided with safety equipment when they are at work.

34

APPENDIX I: PROPOSED STRATEGY AND WORK PLAN

EDRA Consultancy proposes the following strategies: 1. Agree on the terms of reference for the Evaluation 2. Develop data collection instruments, collect and analyze data 3. Present draft report 4. Revise and present final evaluation report Work plan December 2002 Activity 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23

Document X review Dev. Survey X instrument Travel to project X X area & back Data collection X X X X

Data analysis & X X report writing Submit draft X report Present findings X

Submit final X report

35 APPENDIX II-KEY QUESTIONS USED

Evaluation of ECHO/Tearfund Water and Sanitation Programme

Questionnaire for Partners

1. In your view what is Tearfund’s mission in Sierra Leone

2. What is Tearfund’s main business in Sierra Leone?

3. On a scale of 1 (the least) to 5 (the highest), with which of these Tearfund programmes are you familiar Programme 1 2 3 4 5

4. To the best of your knowledge, please state the strengths and Weaknesses of each of Tearfund programmes Programme Strength Weaknesses

5. What are Tearfund’s other strengths?

6. What are Tearfund’s other Weaknesses?

7. What is your view of the approach used by Tearfund?

8. What comments can you make with regards your experience and perception of Tearfund?

36

Evaluation of ECHO/Tearfund Water and Sanitation Programme

Questionnaire for project personnel

(Please use additional paper if necessary)

1. What changes have there been in the programmes original objectives and why?

2. Please list the different components of your programme in the region

3. What activities have been undertaken in each component?

4. Who identifies those who benefit from each programme component?

5. How are they selected to be beneficiaries of these components?

6. Who has been the main beneficiary (e.g. Children, single parents, women, men, family units, whole communities) of each component of the project? Component Beneficiary Target Population Actual

7. What impact has your intervention had on the resettlement and living conditions of those who are beneficiaries? Component Impact

8. In your view, what has been the impact of the overall programme on the target community?

9. What lessons have you learned?

10. What new activities that were not initially intended have been incorporated?

11. What has been the constraints or hindrances to effectiveness?

12. In your view did the intervention reach the intended groups/groups most in need? Yes...... No...... (If No in 12 above, please give reasons)

13. What measures are in place to sustain the benefits of the project when you exit, and who will be in charge?

14. Please give reasons why you think this programme should continue to exist, or be move to other community

37

Evaluation of ECHO/Tearfund Water and Sanitation Programme

Discussion Guide for Communities

(Please use additional paper if necessary)

1. In your understanding what does Tearfund do? 2. Tell us 2 or 3 things Tearfund is doing well or has done well in this community? 3. Tell us 2 or 3 things Tearfund is not doing well or has not done well in this community? 4. Has Tearfund been addressing your priority needs and concerns? Yes...... or No...... 5. If yes to 4 above, give two or three examples 6. If no to 4 above, give 2 or 3 examples as to why you feel Tearfund has not been addressing your community’s needs and concerns 7. Using a scale of 1 (the least) to 5 (the highest), to what extent has Tearfund involved your community in the implementation of activities in the community Activity 1 2 3 4 5

8. Please explain how your community is involved in the implementation of activities 9. Who else is working here, and what do they do? 10. Do community prefer other sources of water? Yes ...... or No ...... If yes name 2 sources 11. What is the effect of the project on other economic activities of the community? 12. What has changed for you? 14. Do you think Tearfund has been able to help the poorest and most vulnerable in your community? Yes...... or No..... 13. Highlight 2 or 3 things you consider as being achievement due to Tearfund’s intervention in your community. 14. How has it affected you as an individual or your family? 15. What impact has the intervention had on the return and resettlement of the community? 16. What would have happened if the project had not been implemented in this community? 17. What other comments can you make with regards your experience and perception of Tearfund?

38

Evaluation of ECHO/Tearfund Water and Sanitation Assistance in

Discussion Guide for Heads of Programme

(Please use additional paper if necessary)

1. How has your profile and relationship with ECHO been affected through the implementation of this project?

2. What measures did you put in place to improve the linkage between short-term activities and long-term issues?

3. What measures are in place to sustain the benefits of the project components when you exit? Who will be responsible?

4. How will recurrent costs and future expenditures be financed?

5. In your view, how far has the project output lead to the achievements of your objectives?

6. In your view, what issues (i.e. negative or positive) influenced the achievements or non-achievements of the objectives?

7. Was the programme implemented as planned? Yes ...... or No..... If no, Why?

39

APPENDIX III: SCHEDULE OF COMMUNITY DISCUSSIONS

Date Destination Headquarters/Section Monday December 9 Travel to Daru Town Kailahun district

Tearfund headquarters Daru Town

Tuesday December 10 Godie Gbomo Town Jonga section

Pelewahun Upper Nyawa section

Gboo section

Wednesday December 11 Feijei Upper Nyawa section

Mamboma Keimaya section

Yeikandor Keimaya section

Sembehun Jonga section

Taninahun Ngeyubla Jonga section

Gbeika Jonga section

Maloma Upper Nyawa section

Thursday December 12 Magbema Gboo section

Gbaama Upper Nyama section

40 APENDIX IV: PROJECT COMMUNITIES VISITED AND COMPOSITION

OF FOCUS GROUP

Section Community Number of Women Men Interviewees Gboo Pelewahun 22 13 9

Jonga Godie 14 6 8

Upper Nyawa Feijia 51 24 27

Keimaya Mamboma 9 5 4

Keimaya Yeikandor 17 9 8

Jonga Sembehun 16 8 8

Jonga Taninahun 26 8 18

Jonga Gbeika 16 11 5

Upper Nyawa Maloma 34 18 16

Gboo Magbema 18 9 9

Upper Nyawa Gbaama 25 11 14

Total 248 122 126

41 APPENDIX V: LIST OF INTERVIEWEES

Name Designation A. M. Swaray Field Worker, Save the Children UK Angela Y. Amara Public Health Educator Augustus Simbo Programme Officer for local NGO Baindu Lansana Public Health Educator Bockarie J. Bunduca Senior Technician Brima Essa Town Chief, Mamboma Buakai Sambu II Town Chief, Magbema Carol Nyamu Field Coordinator Cecilia Kallon Public Health Educator Daniel Menjor Clerk of Work, Edward C. Fayia Assistant Education Manager, Kailahun District, IRC Gladys Ngegba Public Health Educator Hawa Samba I Headwoman J. M. Samu Deputy Town Chief Janet Ross-Jordan Watsan Engineer Joseph Johnny Public Health Educator Kinni Vaibu Section Chief Koroma Gawa Chief Gbeika Mohamed Lansanah Headman, Magbema Town Moinina Boima Section Chief Momoh Vandi Technician Morris Coomber Clerk Of Works Morrison Brima Public Health Educator Mr. A. A. Jusu Field Animator, Friends of Africa Mr. Abel Sannoh Administrative Officer, Relief and Development Agency, 12 Okonkwo Street Mr. Musa Sannoh Town Chief, Maloma Musa Buakai-Wa Town Chief, Taninahun Pa Buake Essa Section Chief Peter Jogua Public Health Educator Sahr Mansa Musa Technician Samba Williams Public Health Educator

42 APPENDIX VI: SCHEDULE OF INTERVIEWS AND ACTIVITIES

Date Activity 6 December Literature review, instrument development and pre-testing, contact field office 9 December Tearfund Field Officer, Daru Review of documents and field reports 10 December Project communities CHVs in various communities 11 & 12 December Project communities Technicians and field staff Community leaders 13 December IRC Save the Children UK WFP, Daru office FARDA IOM Africare Sierra Leone Travel: Freetown to Daru 16-18 December Data entry, analysis and report writing

43 APPENDIX VII: PROJECT PERFORMANCE INDICATOR

a. Population in target area has access to: - clean water at a ratio of 500 people to 1 water point - the location of water points in some communities have traditional relevance and TEARFUND respects the local context wherever possible whilst ensuring adequate water can be provided - 1 latrine per compound

b. Facilities provision facilitates increase in return rates.

These indicators will be verified through regular site visits by project management staff. The Programme Director as well as Government and NGO statistics on people movement from place to place.

Results/Expected Outcome Provision of water, appropriate sanitation and improved knowledge, attitude and practice to public health for an estimated 9,400 beneficiaries over a twelve months period

1. Drinking water is provided in sufficient quantities for an estimated 9,400 people

2. Latrines are provided in each of the compounds

3. Knowledge, attitudes and practices to public health is improved

c. Completion of three spring boxes, three well rehabilitation, nine new wells and 484 family latrines

d. Delivery of public health education and training of 40 community-based educators to government Blue Flag standard

e. Change in knowledge, attitude and practice in local community

f. Reduction in cases of water related diseases identified by TEARFUND and IMC’s monthly morbidity and mortality reports

The procurement and distribution records and the signed completion certificates will confirm these indicators. The hygiene education will be verified by the analysis of baseline and post intervention knowledge attitude and practice (KAP) surveys and the public health training schedule, certificates and reports.

44 APPENDIX VIII: ACTIVITIES UNDERTAKEN AND STATUS

Original Present Wells New SSB Laundry Latrines Status/Comments Village Pop. Pop. Rehab. Wells Feijia 728 601 2 1 1 88 SB extra and under construction Mamboma 124 120 1 16 Finished Yeikandor 349 229 1 1 36 Finished Gbama 671 836 1 1 1 72 LA not started Gbeika 585 520 1 1 1 54 LA not started Maloma 909 778 1 1 1 1 78 LA not started Taninahun -50 400 1 1 1 58 WR extra work from reserve Not in money original Sembehun 327 771 1 1 1 70 LA no started proposal Kpandahun 818 948 2 80 Bandajuma 790 1 1 1? 40 LA not started – depends on progress – may not be built Magbema 687 1 1 1 40 All extra work Godie 155 1 10 Pelewahun 174 1 1 10

Note: Original populations are based on census data from February/march 2002. Taninahun has 2 houses when first assessment was carried out in January 2002.

We are trying to find out when villagers returned to the villages by asking which month/year they returned, during the present census.

45 APPENDIX IX: COMMUNITIES AND THEIR LOCATIONS

Community Section Feijia Upper Nyawa

Gbaama Upper Nyawa

Gbondahun Upper Nyawa

Gbomotor Upper Nyawa

Gbeika Jonga

Sembehun Jonga

Taninahun Ngegbla Jonga

Godie Jonga

Yeikandor Keimaya

Mamboma Keimaya

Bandajuma Dansei

Magbema Gboo

Pelewahun Gboo

46 APPENDIX-X: ORIGINAL AND PRESENT POPULATION OF

COMMUNITIES

Village Original Population. Present Population. Feijia 728 601

Mamboma 124 120

Yeikandor 349 229

Gbama 671 836

Gbeika 585 520

Maloma 909 778

Taninahun -50 400

Sembehun 327 771

Kpandahun 818 948

Bandajuma - 790

Magbema - 687

Godie - 155

Pelewahun - 174

47 APPENDIX XI: STRUCTURAL AND WATER QUALITY DATA

Source Date Depth of Turbidity Flow Faecal Coliforms Village Type Completed Base (M) GPS (N) GPS (W) (NTU) Colour Taste Odour rate (l/s) (no./100ml) Feijia Well 1 29/05/2002 19.73 07 58.85 010 52.35 0 None None None 0.23 9 Feijia Well 2 29/05/2002 19.83 07 58.86 010 52.31 0 None None None 0.31 8 Mamboma Well 15/06/2002 11.23 07 58.67 010 53.08 0 None None None 0.25 0 Maloma Well 1 10/07/2002 8.03 08 02.34 010 51.71 0 None None None 0.31 0 Maloma Well 2 07/03/2002 16.01 08 02.34 010 51.71 0 None None None 0.35 0 Maloma Spring 15/03/2002 2.41 08 02.40 010 51.64 0 None None None 0.35 0 Gbeika Well 10/06/2002 6.28 08 02.98 010 50.30 0 None None None 0.31 3 Gbeika Spring 10/06/200 1.89 08 03.00 010 50.30 15 Red SomeSome 0.35 0 Yeikandor Spring 15/03/2002 2.58 07 59.28 010 54.56 0 None None None 0.23 0 Taninahun Spring 01/05/2002 2.05 08 02.27 010 49.05 0 None None None 0.22 0 Sembehun Well 10/06/2002 4.38 08 03.94 010 49.62 0 None None None 0.28 0 Sembehun Spring 07/04/2002 2.05 08 03.84 010 49.57 25 Red Some Some 0.23 0 Kpondahun Well 1 28/06/2002 11.11 07 59.92 010 51.59 0 None None None 0.20 0 Kpondahun Well 2 28/06/2002 9.2 07 59.95 010 51.45 0 None None None 0.22 0 Gbaama Well 28/06/2002 18.26 07 59.94 010 51.53 0 None None None 0.25 0 Gbaama Spring 15/03/2002 2.03 08 00.16 010 51.98 0 None None None 0.28 0

48 APPENDIX XII: LOGICAL FRAMERORK MATRIX

Intervention Logic Objectively Verifiable Sources of Verification Risks and Assumptions Indicators To enable resettlement and ƒ Rise in population of ƒ Statistics on people movement as Overall recovery of communities in resettled communities collected by Government and Objectives newly accessible areas of ƒ Reduction of number of other NGO’s Sierra Leone which have people living in sub- ƒ Statistics from refugee/IDP been devastated following standard conditions camps and host communities ten years of conflict

To ensure that resettlers in ƒ Provision of facilities ƒ Site visits ƒ Medium/high risk of Specific six communities in Jaluahun increases current return ƒ Tearfund and community increased influx of Project Chiefdom have access to rates records of population figures displaced people into Purpose basic water and sanitation ƒ Analysis of current and return rates target area causing strain services in order to facilitate population and return rates upon existing resettlement. infrastructure

For up to 9,400 beneficiaries ƒ Population in target area ƒ Procurement and distribution ƒ Low risk of resumption Results over a twelve month period: has access to: records: waybills signed by of widespread conflict 1. Drinking water is ƒ water at ratio of 500 beneficiaries for materials which would result in provided in sufficient people:1 water point (where received project delays/ cessation quantities appropriate) ƒ Construction records and reports 2. Latrines are provided in ƒ 1 latrine per household ƒ Signed Completion Certificates Assumptions: each of the compounds • Change in the knowledge, ƒ Baseline and post intervention ƒ Return rates and 3. Knowledge attitudes and attitudes and practice of KAP surveys population movements practices to public health target group from baseline ƒ Public health training schedule follow current trends is improved survey and reports ƒ Water supply points • A reduction in the incidence ƒ Morbidity and mortality reports provide anticipated yield of water-borne diseases from IMC Mobile Clinic

49