Evaluation Ex Post of NGO Managed Projects in Lebanon

NGO managed projects in Lebanon

• “Implementation of a research centre for agricultural dissemination and development in Marjayoun Region” – NGO Movimento Africa ’70 (Movement Africa ’70)

• “Support to local development and creation of an orientation pilot centre at Deir El Ahmar, Bekaa Valley” – NGO Ricerca e Cooperazione (Research and Cooperation)

• “Socio-economic development of the fishermen community of Batroun” – NGO Ricerca e Cooperazione (Research and Cooperation)

2013 EVALUATION

Evaluation ex post of NGO managed projects in Lebanon

The present evaluation report has been redacted by Dr Serena Saquella and Dr Lydia Fabiana Rossi. Opinions expressed in this report are those of the authors alone and do not necessarily reflect the views of the Ministry of Foreign Affairs/Directorate General for Development Cooperation, of NGO Movimento Africa ’70 (Movement Africa ’70) and of NGO Ricerca e Cooperazione (Research and Cooperation).

Evaluation ex post of NGO managed projects in Lebanon

TABLE OF CONTENTS

LIST OF ACRONYMS 5

1. EXECUTIVE SUMMARY 6 2. ORIGINS OF THE INITIATIVE 13 3. STRUCTURE OF THE PRESENT REPORT 13 4. INTRODUCTION 14 5. SITUATION IN LEBANON 15 6. METHODOLOGY OF THE EVALUATION 16

Part A Implementation of a research centre for agricultural dissemination and development in Marjayoun Region - AID No. 7776 - NGO Movimento Africa ’70 (Movement Africa ’70) A1. SYNTHESIS OF EVALUATION ...... 19 A2. GENERAL FRAMEWORK ...... 20 a.2.1 Project synthesis ...... 20 a.2.2 Mission synthesis ...... 20 a.2.3 Analysis of coherence and quality of the design of the Logical Framework ...... 21 A.3 EVALUATION ACCORDING TO THE CRITERIA OF RELEVANCE, EFFICIENCY, EFFECTIVENESS, IMPACT AND SUSTAINABILITY ...... 25 a.3.1 RELEVANCE ...... 25 a.3.1.1 Identification of needs and of beneficiaries ...... 25 a.3.1.2 Identification of the local counterpart ...... 25 a.3.1.3 Relationships with central and local authorities ...... 26 a.3.2 EFFICIENCY ...... 26 a.3.2.1 Funds and time management ...... 27 a.3.2.2 Staff management ...... 28 a.3.3 EFFECTIVENESS ...... 29 A.3.4 IMPACT ...... 32 A.3.5 SUSTAINABILITY ...... 33 A.4 CONCLUSION ...... 34 A.5 RECOMMENDATIONS ...... 34

Evaluation ex post of NGO managed projects in Lebanon

Part B

Support to local development and creation of an orientation pilot centre at Deir El Ahmar, Bekaa Valley - AID No. 8650 of NGO Ricerca e Cooperazione (Research and Cooperation);

B.1 SYNTHESIS OF EVALUATION……………………………………………………….37 B.2. GENERAL FRAMEWORK…………………………………………………………….38 b.2.1 Project synthesis b.2.2 Mission synthesis b.2.3 Analysis of coherence and quality of the design of the Logical Framework B3 EVALUATION ACCORDING TO THE CRITERIA OF REVELANCE, EFFICIENCY, EFFECTIVENESS, IMPACT AND SUSTAINABILITY b.3.1 RELEVANCE b.3.1.1 Identification of needs and of beneficiaries b.3.1.2 Identification of the local counterpart b.3.1.3 Relationships with local authorities b.3.2 EFFICIENCY b.3.2.1 Funds and time management b.3.2.2 Staff management b.3.3 EFFECTIVENESS b.3.4 IMPACT b.3.5 SUSTAINABILITY B.4 CONCLUSION B.5 RECOMMENDATIONS

Part C Socio-economic development of the fishermen community of Batroun - AID No. 8255 of NGO Ricerca e Cooperazione (Research and Cooperation). C.1 SYNTHESIS OF EVALUATION C.2 GENERAL FRAMEWORK

Evaluation ex post of NGO managed projects in Lebanon

c.2.1 Project synthesis c.2.2 Mission synthesis c.2.3 Analysis of coherence and quality of the design of the Logical Framework C.3 EVALUATION ACCORDING TO THE CRITERIA OF REVELANCE, EFFICIENCY, EFFECTIVENESS, IMPACT AND SUSTAINABILITY c.3.1 RELEVANCE c.3.1.1 Identification of needs and of beneficiaries c.3.1.2 Identification of the local counterpart c.3.1.3 Relationships with local authorities c.3.2 EFFICIENCY c.3.2.1 Funds and time management c.3.2.2 Staff management c.3.3 EFFECTIVENESS c.3.4 IMPACT c.3.5 SUSTAINABILITY C.4 CONCLUSION C.5 RECOMMENDATIONS

Evaluation ex post of NGO managed projects in Lebanon

LIST OF ACRONYMS

APS Aiuto Pubblico allo Sviluppo ODA Official Development Assistance CD Comitato Direzionale Steering Committee CCIA Chamber of Commerce, Industry and Agriculture CNRS Conseil National de la Recherche Scientifique National Council of Scientific Research CORIS Dipartimento di Comunicazione e Ricerca Sociale Department of Communication and Social Research DEA Deir El Ahmar DGCS Direzione Generale per la Cooperazione allo Sviluppo General Directorate for Development Cooperation GAL Gruppo di Azione Locale Local Action Group GO Governmental Organization IAM Istituto Agronomico del Mediterraneo Mediterranean Agronomic Institute ICU Istituto per la Cooperazione Universitaria Institute for University Cooperation LARI Lebanese Agricultural Research Institute MAE Ministero degli Affari Esteri Ministry of Foreign Affairs MoA Ministero dell’Agricoltura Ministry of Agriculture NCMS National Centre for Marine Science Organizzazione per la Cooperazione e lo Sviluppo Economico OCSE/DAC - Development Assistance Committee OECD/DAC Organization for Economic Cooperation and Development – Development Assistance Committee ODPS Organisation de Développement pour la Paix Sociale Organization of Development for Social Peace ONG Organizzazione non Governativa NGO Non-Governmental Organization ONU Organizzazione delle Nazioni Unite UN United Nations PM Project Manager QL Quadro Logico Logical Framework RC Ricerca e Cooperazione Research and Cooperation SPL Secours Populaire Libanais Lebanese People’s Rescue UCB Ufficio Centrale per il Bilancio Central Account Office UNDP United Nations Development Programme UTC Unità Tecnica Centrale Central Technical Unit UTL Unità Tecnica Locale Local Technical Unit

Evaluation ex post of NGO managed projects in Lebanon

1. EXECUTIVE SUMMARY

During the first trimester of 2013, the General Directorate for Development Cooperation of the Ministry of Foreign Affairs entrusted the Department of Communication and Social Research (CORIS) of Sapienza University of Rome, the task to carry out the ex post evaluation of three projects in Lebanon, financed by Italian Cooperation and managed by NGOs.

a. “Implementation of a research centre for agricultural dissemination and development in Marjayoun Region” of NGO Movimento Africa ’70 (Movement Africa ’70); b. “Support to local development and creation of an orientation pilot centre at Deir El Ahmar, Bekaa Valley” of NGO Ricerca e Cooperazione (Research and Cooperation); c. “Socio-economic development of the fishermen community of Batroun” of NGO Ricerca e Cooperazione (Research and Cooperation).

Italian Cooperation has been present in Lebanon from 1983 and is a privileged partner of this country. Following the conflict of 2006, the commitment of Italy has been strongly reinforced, leading to the opening of a Local Technical Unit at the Italian Embassy of Beirut in September 2007. Italy is the second European donor, after France, with another 60 programs in progress. Areas of intervention are multiple: emergency, environmental, local development, sanitarian, social, rural development and cultural and religious tourism.

The three projects, evaluated by the team, have sought to address the issue of employment in agricultural and fisheries sectors, involving Lebanese partners, such as Caritas Lebanon, Secours Populaire Libanais (Lebanese People’s Rescue, SPL), the Organisation de Développement pour la Paix Sociale (Organisation of Development for Social Peace, ODPS) and focusing on the modernization of equipment, on professional updating and on improvement of technical knowledge of farmers and fishermen.

On the level strictly methodological, the exercise has used the following tools. An evaluation grid was elaborated for each project, which, reflecting OECD/DAC-EU criteria, identified main questions and subordinate ones; identified also appropriate sources to provide answers to questions and their modalities of their use/activation (analysis of documents, bilateral meetings, focus group, semi-structured interviews etc). From the evaluation grids and from the stakeholders’ list of each project (national and local institutions, partners, NGO staff), the team elaborated some guidelines for interviews and for conducting on-site focus group. This will be discussed in more details within the evaluation report.

“REALISATION OF A RESEARCH CENTER FOR AGRICULTURAL DISSEMINATION AND DEVELOPMENT IN MARJAYOUN REGION” AID No. 7776

The initiative was identified and formulated by the local counterpart SPL (Secours Populaire Libanais, Lebanese People’s Rescue) with the support of NGO Movimento Africa ’70 (Movement Africa ’70). This project has been implemented in Marjayoun District, Department of Nabatieh (South Lebanon) and involved villages of El Khiam, Ibel El-Saki, Kfar Kila, Deir Mimas, Qlayaa, Bourj Al-Moulouk, with a strong agricultural vocation, with the objective of contributing to

Evaluation ex post of NGO managed projects in Lebanon

the improvement of South Lebanon economy, through the creation of an Agricultural Diffusion Center, aiming the transfer of appropriate production techniques among farmers living in the country. The project proposed to increase revenue of 500 farmers by about 30% in three years through three main axes: 1) Capacity building 2) Providing production services 3) Networking and inter-institutional coordination.

The initiative appeared to be responsive to beneficiaries’ needs, contributing to trigger a development process of the agricultural potential of the plain of Marjayoun, by investing particularly in training activities and technical assistance to farmers. There has been a good collaboration with local authorities and with the reference community, a good consideration of the gender component and an excellent perception of the utility of the initiative among the beneficiaries. With respect to the counterpart choice, it proved to be totally unsuccessful since SPL appeared to be unsuitable to carry out an operation of agricultural development and unable to comply with donor’s procedures. On the whole, the financial resources allocated by the Ministry of Foreign Affairs were used in an efficient manner, despite numerous changes and extensions suffered by the project. There has been, however, a low level of participation from the local counterpart, a staff turnover (four project managers) and also a low level of participation from the Ministry of Agriculture during the first two years of the project. The project effectiveness underlines the achievement of the overall number of beneficiaries and the realisation of activities, even in presence of high instability in the political context. Concerning the achievement of the specific objective “Increase revenue by 30% of 500 farmers in 7 villages through a) innovation of cultures and production techniques; b) offer of production services; c) strengthening of management capacities of local institutions”, it must be emphasized that it is an objective difficult to measure and to objectively verify. For the farmers, the mandatory management of accounting is not planned and Lebanese fiscal system appears to be completely inefficient. There was no achieved result regarding the establishment/strengthening of cooperatives, which are malfunctioning and very politicized. The initiative effects are positive on the slope of social impact (strengthening the sense of belonging to the area and the community, empowerment of women), but are very few on the slope of economic and income-generating activities. The initiative has been very appreciated by beneficiaries and considered as essential in contributing to the improvement of agricultural production. The results achieved by the project, however, should be strengthened and consolidated in order to fully achieve general and specific objectives. The scarce commercialisation of agricultural products is, for now, the project beneficiaries’ main problem, who expressed the necessity of working in consortia and finding points of sale. From this point of view, the registration and the trademark certification of products and its diffusion through an accurate marketing strategy are of fundamental importance. Regarding the choice of counterpart, it turned out to be a total failure since SPL was not suitable to implement an intervention of agricultural development and unable to comply with the donor’s procedures. Regarding other issues encountered during the project implementation (total inefficiency of cooperatives) – it can be said that, although not directly imputable to the proponent NGO and its counterpart, it could have been better planned during the identification phase.

Evaluation ex post of NGO managed projects in Lebanon

The main perplexity is the almost absolute lack of sustainability of the project, which was discussed in the previous paragraph. The functioning at full capacities of the Laboratory of the School of Khiam is of fundamental importance, as it offers long-term production services, devoted to guarantee the initiative.

Recommendations for Ministry of Foreign Affairs

There has been detected a long time lag between the project presentation, on January 23, 2003, and its approbation on June 8, 2004. It is suggested to make these times faster and leaner, in order to avoid excessive deviations from the original funding proposal, especially in a country where political and social context changes quickly, since it is characterised by severe political instability and external influences. It is advised, during the technical and economic phase, to pay more attention in the choice of the counterpart by the NGO proponent. Moreover, continuous supervision and monitoring from the UTL of projects with particular problems are suggested. The evaluation lack at the project conclusion by the UTL does not allow us to have, as useful it could be, an important element of comparison in the analysis of information collected.

Recommendations for NGO Movimento Africa ’70 (Movement Africa ’70)

Given the important and non-negligible consequences of the choice of an unsuitable partner, it is recommended that the NGO Movimento Africa ’70 (Movement Africa ’70) choose local counterparts having resources and managerial capacities, and which are able to contribute to the initiative management and to assure its sustainability through time. The NGO Movimento Africa ’70 (Movement Africa ’70) is also invited to take into consideration the lessons learned during this project. This will contribute to a more active and in-depth identification phase and future projects elaboration, focusing on the aspects that have been the most impacted (formation component) and in which the expertise of NGO Movimento Africa ’70 (Movement Africa ’70) has been shown to have an added value.

“SUPPORT TO LOCAL DEVELOPMENT AND CREATION OF AN ORIENTATION PILOT CENTER AT DEIR EL AHMAR, BEKAA VALLEY” AID No. 8650

The initiative was implemented by the NGO Ricerca e Cooperazione (Research and Cooperation) with the participation of the local counterpart Caritas Lebanon. The project offered to trigger a socio-economic development process and the diffusion of alternative crops to those illegal in the region of Deir El Ahmar (Baalbeck Caza, Lebanon), through the creation of an Orientation Pilot Center to Development aiming to the formation and the aggregation of local population. In order to strengthen local capacities, the project provided functional activities to encourage the formation of Local Action Groups (GAL). Regarding the relevance criterion, we can say that there has been an excellent collaboration with the chosen local counterpart, as well as a good collaboration with national authorities, in particular with the Ministry of Agriculture, and a good consideration in the gender component aiming to support women involvement in the activities. The allocated resources from the Ministry of Foreign Affairs appear to have been used rather efficiently. The project has been realized over a period of 38 months and 10 days compared to

Evaluation ex post of NGO managed projects in Lebanon

36 months planned. The funds have not been completely used in a consequent economy of expenditures in relation to the total value of the project. Good performance of the local counterpart who played a fundamental role in the activities realization and in the staff management. The rejection of a variant should be noted, which implied the interruption of some activities left incomplete on the field (the NGO, without waiting for the outcome of the variation, in fact negative, had already initiated some activities it had to stop). In terms of effectiveness, not all results have been completely reached (there has been few requests from beneficiaries of the rotating funds at the end of the project). There has not been sufficient evidence to evaluate the impact. Caritas Lebanon declares that around 80% of beneficiaries have increased their revenues and that the project strengthened the regional socio-economic processes and promoted also new entrepreneurial activities. At the end of the evaluation mission we did not get sufficient evidence to fully comprehend the project sustainability. However we can affirm that the documentation center is functioning and active and there is a good performance of GAL (Local Action Groups).

Recommendations for the Ministry of Foreign Affairs

It is recommended to the donor, during technical and economic evaluation, to analyse the whole project document, including the logical framework which, for the purpose of evaluating, represents a tool of great importance. The logical framework identifies some indicators which cannot be found before 2021 and 2026, some indicators are not specified or unrealistic and others are not quantifiable and not objectively verifiable. Concerning the administrative documentation provided, a certain confusion can be noticed: as reported in the paragraph related to the efficiency, the project has undergone changes in the term of expenditures after its approval. The documentation provided does not always show the same amount. If possible, a revision of this documentation is suggested. A greater coordination with the Local Technical Unit or with other offices of the headquarters is suggested, for the verification of projects in progress or concluded, in order to avoid duplication or overlap, and thus making more effective the Official Development Assistance (ODA). A final point regards the relatively long period concerning the disbursements of payment, resulting in the NGO and the counterpart advancing them.

Recommendations for NGO Ricerca e Cooperazione (Research and Cooperation)

The NGO Ricerca e Cooperazione (Research and Cooperation) is invited to take into consideration lessons learned during this project, such as developing in a better way the identification phase and considering risks of implementing activities non-predicted in the project document, which are not part of Italian Cooperation priority. Here we can mention a particular reference to the rejection of the variation, which caused problems in terms of effectiveness and efficiency. A better coordination between project partners is suggested, for the implementation of project activities, some of them (as the GAL constitution) being already realized within another project financed by Italian cooperation and implemented by IAM from Bari, in order to ensure the effectiveness of the operation and avoid duplication and overlap. A more accurate redaction of the Logical Framework is also recommended, this tool being useful to define clearly the different elements of the project intervention. Finally, NGO Ricerca e Cooperazione (Research and Cooperation) is invited to facilitate the eventual future work of evaluators by providing timely contacts and documents.

Evaluation ex post of NGO managed projects in Lebanon

“SOCIO-ECONOMIC DEVELOPMENT OF THE FISHERMEN COMMUNITY OF BATROUN” AID No. 8255

The project was realized by the NGO Ricerca e Cooperazione (Research and Cooperation), in collaboration with the local counterpart Organisation de Développement pour la Paix Sociale (Organisation of Development for Social Peace, ODPS). The project, lasting three years, aimed to revitalise the fishing industry in the coastal town of Batroun, with the objective to improve the fishermen’s quality of work (specific objective) and to contribute to the improvement of life quality of fishermen and their family, as well as the development of economic activities related to the sea (general objective). The main project activities were directed to people of the sea (informal education, introduction of new techniques and/or technologies, establishment of a loan fund, management of natural resources, raising awareness on workplace safety and improvement of embarkation safety) in order to encourage an increase in fishing profitability and, at the same time, reduce the risks while working. From the focus group, formed by a sample of beneficiaries, it emerged that the initiative has not met fully their needs. If the conditions on safety at sea (Expected Result A) have definitely improved, no result has been achieved concerning the increase of their revenues (Expected Result B) due to the fish resources scarcity of sea. The chosen local counterpart, ODPS, being newly established (founded in 2004) did not present guarantees of solid experience in cooperation activities, in particular in the proposed sector of operation. Today, the local counterpart no longer exists (ceased activities at the end of the project, in 2010). The role of the local Fishermen’s Cooperative, bearer of the fishermen’s interests inside the community and in the Lebanese context, is not really clear (perhaps because of the difficulties encountered during the first year in the relations with the Cooperative president against the project). Nevertheless, the project created a strong sense of collaboration with local authorities, in particular with the Municipality of Batroun, and the other enterprises involved in the project, including the Institute of Oceanography and Fishery of Batroun of the Ministry of Agriculture, which supported the project through all its phases. Thanks to the excellent interaction with the beneficiaries, the project contributed to unite the community, by creating a common objective, which was the rehabilitation of fishery sector. Despite the strong instabilities of the Lebanese context, a realization fairly precise and respecting the timing are found (just one extension was requested due to inclement meteorological conditions for the practical demonstration at sea, and two non-onerous variations with the consequent changes on certain items of expenditure). The main perplexity regards the collapse of the local counterpart and in consequence the collapse of the rotating loan fund (managed by ODPS), which represent important elements to affirm that, in this specific case, the use of the provided funds has been inefficient. As the rotating fund collapsed, the fishermen, who had to return the installments, have suspended their own payments. They have, however, expressed the will to reactivate them, as soon as the fund is reconstituted and assigned to another entity (such as the fishermen cooperative), in favor to other beneficiaries. Partial achievement of the expected results and of general and specific objectives for the following motivations: • The first expected result “Improvement of safety conditions at sea at Batroun” has been achieved through a series of activities which have been regularly realised, including the implementation of a training/update on boat maintenance, which was not included in the

Evaluation ex post of NGO managed projects in Lebanon

Logical Framework but was present inside narrative and reporting documents. This implementation was logically continuous with the expected result and coherent with the project finalities. The main perplexity regards the activity relative to the loan fund, for aforementioned reasons. • The second expected result, “increase of fishermen’s revenues of Batroun”, despite the realization of all activities, cannot be considered as achieved for several reasons, primarily because it is difficult to make a comparison between the revenues recorded at the beginning of the project and the revenues recorded at its conclusion.

The meeting held with fishermen pointed out that one of the main current problems is the absence of fish resources, caused by the lack of an adequate protection legislation of maritime ecosystem, the non-application of the closed season to allow the reproduction of fish stock, the illegal fishery (with the use of poisons and explosives) and the water pollution (problems already identified at the project beginning), which resulted in a worsening of their revenues in 2007. In view of the above, the specific objective “Improvement of the fishermen’s quality of work of Batroun” can be considered as partially achieved. Among the planned activities, we notice a great participation of students and of the project beneficiaries in environmental awareness campaigns and in community building activities (over 100%). We do not have sufficient and reliable information to express an opinion concerning the impact criterion (for example the difficulty in meetings organization with key stakeholders). The project contributed to strengthen the sector and to create a strong sense of sharing common problems and also supported the fishermen community empowerment, by suggesting new participatory approaches and new representative and local development dynamics. However, from the focus group emerged that since 2007, there has not been substantial change in terms of economic or social development of the beneficiary community (therefore, very scarce impact mainly due to the absence of fishery resources). The partial achievement of expected results and therefore of general and specific objectives, such as the collapse of the counterpart and of the rotating fund and all the encountered problems with the Fishermen Cooperative, are important and significant elements to say that there are no elements ensuring the project sustainability.

Recommendations for the Ministry of Foreign Affairs

It is recommended to make the approval and financing times of the initiative faster, in order to avoid excessive differences between the initial financing proposal and the needs of the referential context, which are continuously changing, above all in a country such as Lebanon (project presented to the DGCS on 28/10/2004, approved with resolution No. 142 on 25/10/2005, order of commitment on 14/12/2006). When a period longer than a year is spent between the initiative submission and its approval, it would be appropriate to require the proponent to update the financial plan submitted before the approval. It is suggested that where some initiative show particular problems, the supervision from the UTL should be more constant and in-depth, in order to ensure a greater efficiency in official development assistance. It is recommended to the donor, in phase of technical and economic evaluation, to analyse the whole project document including the logical framework. It is particularly advised in phase of submission of the project proposal to require the formulation of objectively verifiable in time indicators. The lack of such indicators reduces the possibility to evaluate, through precise

Evaluation ex post of NGO managed projects in Lebanon

data, the effects the initiative would have generated with respect to those it actually produced in the operation area.

Recommendations for the NGO Ricerca e Cooperazione (Research and Cooperation)

It is recommended to the NGO to choose more carefully the local counterpart, especially for this kind of initiative requiring experience and expertise. Its collapse at the end of the project had negative repercussions in terms of initiative sustainability and, therefore, in the lack of access to loans by the beneficiaries. A more accurate redaction of the Logical Framework is also recommended, this being a useful working tool to define clearly the different elements of the project operation and of the whole Project Cycle. It is suggested to involve in a better manner the local enterprises and other institutional subjects in the sustainability strategy of the project initiative. The NGO Ricerca e Cooperazione (Research and Cooperation) is finally invited to facilitate the future work of evaluators by providing contacts and documents in time.

Evaluation ex post of NGO managed projects in Lebanon

2. ORIGINS OF THE INITIATIVE

In the first trimester of 2013, the General Directorate for Development Cooperation of the Ministry of Foreign Affairs entrusted the Department of Communication and Social Research (CORIS) of Sapienza University of Rome the task of carrying out the evaluation of three projects managed by NGOs in Lebanon, relying on the Terms of Reference attached hereto. This Department is particularly qualified since it established as part of its educational offer a complete and articulated course of study (Bachelor degree, Master and postgraduate specialisation in “Sciences of Cooperation and Development”), in which the issue of initiative monitoring and evaluation in development cooperation is present and treated by qualified teachers from Faculties of Political Science, Economics, Literature, Communication Science and Humanistic Sciences as well as external industry experts. While the respective administrative structures provided contractually relevant documents, the office IX of the DGCS made the documentation available, the Scientific Director Prof. Marcello Fedele, assisted by Prof. Marco Cilento responsible for institutional relations with offices of the DGCS of the Ministry of Foreign Affairs, selected three consultant experts in development cooperation programmes in the persons of Maria Giuseppina Loretta Peschi, Serena Saquella and Lyda Fabiana Rossi. Dr Maria Giuseppina Loretta Peschi, assisted by Dr Serena Saquella, has participated in the launch of the evaluation operation (inception meetings, documentation analysis, interviews with staff at headquarters and on the field), while Dr Serena Saquella and Dr Lyda Fabiana Rossi have completed the work through a mission in Lebanon and the preparation of this report.

Dr Serena Saquella carried out particularly the evaluation operation of the following projects: a. “Implementation of a research center for agricultural dissemination and development in Marjayoun Region” by NGO Movimento Africa ’70 (Movement Africa ’70); b. •“Support to local development and creation of an orientation pilot center at Deir El Ahmar, Bekaa Valley” by NGO Ricerca e Cooperazione (Research and Cooperation).

In the meantime, Dr Lyda Fabiana Rossi took care of the project “Socio-economic development of the fishermen community of Batroun” by the NGO Ricerca e Cooperazione (Research and Cooperation).

3. STRUCTURE OF THE PRESENT REPORT

The present evaluation report was prepared in two distinct phases: the first part was drawn up on the basis of the analysis of the available documentation, provided by DGCS and by NGOs Movimento Africa ’70 (Movement Africa ’70) and Ricerca e Cooperazione (Research and Cooperation) and following to the mission field, a provisional outline was complied (Draft Report). Following the workshop, which will take place at the Ministry of Foreign Affairs in Rome, in which comments and feedback from stakeholders will be collected, a draft of the final report and its translation in English will be drawn up, as required in the Terms of Reference (Final Report). The Evaluation Report is structured as follow: - Executive summary; - An introductive part in which are described the socio-economic context of the beneficiary country, the stages of the evaluation and the methodology adopted by the evaluation team;

Evaluation ex post of NGO managed projects in Lebanon

- The qualitative and quantitative analysis of the consulted documentation and of the collected information in accordance with the canonical criteria adopted by OECD/DAC-EU; - Conclusions and recommendations for each project. The Final Report concludes with a series of attachments documenting the most important stages of the evaluation exercise.

4. INTRODUCTION

The present ex post evaluation took into consideration three projects implemented by Italian NGOs in Lebanon and co-financed by Italian Cooperation in a period from 2006 and 2011.

PROJECTS NGOs LEBANON Title: Implementation of a research center for agricultural dissemination and development in Marjayoun Region AID No. 7776 Resolution of the Steering Committee No. 55 of 08/06/2004 Executive Agency: NGO Movimento Africa ’70 (Movement Africa ’70) Local counterpart: Secours Populaire Libanais – SPL (Lebanese People’s Rescue) Expected duration: 36 months Implementation period: 20/01/2006 – 27/05/2010 Total value: € 1.292.411,19 Title: Support to local development and creation of an orientation pilot center at Deir El Ahmar, Bekaa Valley AID No. 8650 Resolution of the Steering Committee No. 105 of 31/08/2007 Executive Agency: NGO RC – Ricerca e Cooperazione (Research and Cooperation) Local counterpart: Caritas Libano (Caritas Lebanon) Expected duration: 36 months Implementation period: 21/04/2008 – 30/05/2011 Total value: € 1.640.174,58 Title: Socio-economic development of the fishermen community of Batroun AID No. 8255 Resolution of the Steering Committee No. 142 of 25/10/2005 Executive Agency: ONG RC – Ricerca e Cooperazione (Research and Cooperation) Local counterpart: Organisation de Développement pour la Paix Sociale - ODPS (Organisation of Development for Social Peace) Expected duration: 36 months Implementation period: 12/03/2007 – 14/04/2010 Total value: € 1.423.780,28

The three projects, evaluated by the team, have sought to tackle the issue of employment in agricultural and fishery sectors, in three different locations in Lebanon, involving Lebanese partners such as Caritas Libano (Caritas Lebanon), the Secours Populaire Libanais (Lebanese People’s Rescue, SPL), the Organisation de Développement pour la Paix Sociale (Organisation of Development for Social Peace, ODPS), by focusing on the modernisation of equipment, professional updating and improvement of technical knowledge of farmers and fishermen. The expected duration for each project was 36 months, but all three have required more time, this resulting in a number of “non-onerous extensions” and respective adjustments of expenditure specifications; the reasons of these decisions, their management modalities and their impact on the cost/benefit ratio will be discussed in more details in the following paragraphs.

Evaluation ex post of NGO managed projects in Lebanon

5. SITUATION IN LEBANON

Lebanon is a little Middle-East country with an estimated population of 4.2 million of people, a per capita income equal to $ 47.60 and a life expectancy at birth equal to 72 years. In July 2005, Lebanese government outlined a broad program of political, social and economic reforms and also started to organize a donor conference in Beirut to request international support for the reforms and for a public debt of almost 180% of GDP. The process suddenly stopped in the summer 2006 because of the conflict between the Israeli Army and Hezbollah militias. The consequences were extremely serious: one quarter of the population was displaced, 1,100 casualties, property damages for more than $ 3.6 billion and incalculable damages to economy, tourism and fragile social balance. Today Lebanon presents a high rate of youth unemployment, a low participation of women in economic and productive life (22.3% of female labour force and 71.5% of male labour force), a massive influx of foreign workers into the labour market (for example Palestinian refugees) and a large number of qualified Lebanese people looking for a job abroad1. The characteristics of the labour market in Lebanon are difficult to analyse because of the poor quality of statistics, however the rate of unemployment is estimated between 15% and 20% among young people and women (45% of unemployed are between 15 and 24 years old and the majority is looking for a first job)2. Unemployment significantly contributes to the growth of poverty indexes. According to the UNDP, around 28.5% of Lebanese population is poor and around 300,000 people live in conditions of extreme poverty3. Ranked in 2012 as 72nd on 186 countries in UN statistics on Human Development Index, Lebanon is one of the main recipient countries of Italian Official Development Assistance. Italian Cooperation is present in Lebanon since 1983 and is one of the most privileged partners of the country. Following the conflict of 2006, Italian commitment was greatly strengthened leading to the opening of a Local Technical Unit at the Embassy of Italy at Beirut in September 2007. Italy is the second European donor, after France, with over 60 ongoing programmes. Sectors of intervention are many: emergency, environment, local development, health, social services, rural development and religious and cultural tourism. In February 2007, the first projects financed through the Emergency Programme ROSS started. With this programme, Italian Cooperation reached a wide territory, assisting about one hundred Lebanese villages and municipalities and focusing particularly in the areas most affected by the conflict. The main issues addressed, in line with the “Country Strategy Paper” 2007-2013 of the European Commission, were: economic rehabilitation and restart of productive activities; restoration of damaged structures; local development; strengthening of the social fabric; development of the socio-economic role of women and environmental sanitation. In this process, the contribution of the twenty Italian NGOs present in the country, which managed over 90% of financial intervention of the Emergency Programme, was essential. Today Lebanon has to reconstruct, facing at the same time a heavy public debt, a reform process many time started but never finished, a stagnant economic growth, a political and institutional crisis which worsens tensions and internal security.

1 Country Strategy Paper 2007-2013 and National Indicative Programme 2007-2010 2 ETF, 2009-2011 3 United Nations Development Programme (UNDP, 2009)

Evaluation ex post of NGO managed projects in Lebanon

6. METHODOLOGY OF THE EVALUATION

The evaluation team conducted the operation in accordance with the instructions provided by the Terms of Reference which, in their turn, respect the evaluation criteria of OECD/DAC- EU, which are the measurement of relevance, effectiveness, efficiency, impact and of the sustainability of each project. As regards with the projects which, as mentioned, intended to upgrade and raise the technical and economic capacities of the beneficiaries, the evaluation analysed the impact and sustainability aspects, and highlighted the lessons learned (both by the beneficiaries and by the implementing agencies), by formulating some useful recommendations for possible analogous initiatives. On a strictly methodological level, the exercise used the following tools. For each project an evaluation grid was elaborated, which, respecting OECD/DAC-EU criteria, identified the main questions and the subordinate ones; identified adequate sources to provide answers to questions and how to use/activate them (documents analysis, bilateral meetings, focus group, semi-structured interviews also by phone, etc.). These attached grids were submitted to the leaders of the executive NGOs. From evaluation grids and the list of stakeholders of each project (national and local institutions, partners, NGO staff), the team elaborated:

a. Some guidelines for interviews with headquarters and project manager staff and interviews with administrative staff; b. The list of people to hear/meet; c. Some guidelines for interviews performed during on-site meetings with representative of local institutions; d. The list of people to interview on-site; e. Some guidelines to conduct focus groups with the beneficiaries taking place on-site.

The selection of people to be involved by phone and/or on-site was prepared in consultation with the leaders of executive NGOs and with the UTL in Beirut.

The evaluation was articulated in the following phases:

1. Preliminary phase, in which the reference documentation was examined, the main stakeholders of the two NGOs were interviewed and the mission programme was planned; 2. Field phase, in which interviews were conducted, as well as meetings with representative of different involved institutions and meetings with the project beneficiaries; 3. Phase of qualitative and quantitative analysis of collected information; 4. Elaboration of the report and presentation of the results through the organisation of a workshop at the Ministry of Foreign Affairs.

Evaluation ex post of NGO managed projects in Lebanon

Parte A

IMPLEMENTATION OF A RESEARCH CENTER FOR AGRICULTURAL DISSEMINATION AND DEVELOPMENT IN MARJAYOUN REGION

AID No. 7776

NGO Movimento Africa ’70 (Movement Africa ’70)

Resolution of Steering Committee No. 55 of 08/06/2004

Local counterpart Secours Populaire Libanais (Lebanese People’s Rescue) Total value € 1.292.411,19

BORNE BY DGCS: € 773.800,26 equivalent to 9,87% (plus € 72.000,00 to cover the cost of social security and insurance for Italian personnel cooperating)

BORNE BY NGO: € 193.461,49 equivalent to 14,97%

BORNE BY COUNTERPART: € 325.149,44 equivalent to 25,16% Intervention sector Agricultural development Implementation period 20/01/2006 – 27/05/2010 Type of evaluation INDEPENDENT EX POST EVALUATION Period of evaluation mission 28/11/2013 – 06/12/2013

A1. SYNTHESIS OF EVALUATION CRITERIA STRENGTHS WEAKNESSES - Good identification of the needs and of the area for its - Project identified and formulated in 2000 agricultural potential by the local counterpart. Launched in 2006 - Good collaboration with local authorities and with the without any substantial modification Relevance community of reference - Unsuitable counterpart - Good consideration of gender component - Excellent perception of the initiative utility among beneficiaries - The intervention was characterised by the submission of numerous variations and extensions - Staff turnover (four project managers in Efficiency - Good use of Ministry of Foreign Affairs’ resources, thanks to total) its flexible instruments, such as variations and non-onerous - Low participation of the counterpart extensions - Low participation of the Ministry of Agriculture during the first two years of the project - Specific objective difficult to verify objectively, as it relates to the income Effectiveness - Achievement of the total number of beneficiaries and declared by the beneficiaries implementation of activities in the project, despite the presence - No result achieved regarding the of high instability of the political context constitution/strengthening of cooperatives - Acquisition by the beneficiaries of new productive techniques - Acquisition of an « agricultural identity » by the beneficiaries - Very low commercialisation of the Impact in the area of Marjayoun, not previously entered in the agricultural products economic mapping Sustainability - Extremely little if any

A2. GENERAL FRAMEWORK a.2.1 Project synthesis

The project “Implementation of a research center for agricultural dissemination and development in Marjayoun region” has been presented to the DGCS on January 23, 2003 and approved by the Steering Committee with resolution No. 55 on June 8, 2004. The initiative identified and solicited in 2000, formulated in 2002 by the local counterpart SPL (Secours Populaire Libanais, Lebanese People’s Rescue), a socialist institution very “strong” in these years and well established on the territory, with the support of the NGO Movimento Africa ’70 (Movement Africa ’70). This project was implemented in Marjayoun District, Department of Nabatieh (South Lebanon) and involved the villages of El Khiam, Ibel El-Saki, Kfar Kila, Deir Mimas, Qlayaa, Bourj Al-Moulouk, with a strong agricultural vocation, with the objective of contributing to the improvement of South Lebanon economy, through the creation of an Agricultural Diffusion Centre aiming to transfer appropriate production techniques among farmers living in the country. The project proposed to increase revenue of 500 farmers by about 30% in three years through three main axes: 1) Capacity building 2) Providing production services 3) Networking and inter-institutional coordination

The project was the subject of numerous non-onerous variations and extensions, mainly related to external factors among which, primarily, the outbreak of hostilities between Lebanon and Israel in July 2006. Its activities officially started on January 28, 2006 with the activation of formative processes in favor of the project manager, were operationally launched on site on July 7, 2007 (date on which he assumed the duties as the project manager) and ended on May 27, 2010, with an effective duration of 52 months (22 for the first annuity, 12 for the second and 18 for the third) against the expected 36, following the request of 3 extensions (2 in the first annuity, respectively of 6 and 4 months and 1 in the third annuity of 6 months) and variations of up to 7 (3 during the first annuity, 2 in the second and 2 in the third). In reference to its total duration and the difficulties encountered, due to external factors, there was also a staff project turnover (4 project managers in total) quite significant, albeit it did not affect the normal pursuit of activities. a.2.2 Mission synthesis

Before and during the evaluation mission, an extensive documentation has been consulted including: - Financing proposal of the NGO and the relative technical and economic evaluation of the UTC; - Requests for non-onerous variations and requests for extensions and relative technical advices; - Reports on the state of progress relative to the three project annuities; - Correspondences Ministry of Foreign Affairs/NGO Movimento Africa ’70 (Movement Africa ’70)

Evaluation ex post of NGO managed projects in Lebanon

During the mission, the team performed a series of meetings and interviews. Meetings were conducted on the base of the availability of local stakeholders and of beneficiaries verified by UTL: - UTL; - Former representative of SPL, local counterpart of the initiative; - Dean of the Technical School of Agriculture of Khiam; - Beneficiaries (individuals and representative of cooperatives); - Laboratory technician currently employed by the Ministry of Agriculture; - Local agronomist.

The following structures have been visited: - The Technical School of Agriculture of Khiam, the analysis laboratory and the teaching vivarium; - Demonstration Plot (orchard). a.2.3 Analysis of coherence and quality of the design of the Logical Framework

To make easier to understand what will be illustrated in the following pages, we should quickly remind the consistency of the project. It pursued the general objective of “Improving the socio-economic conditions of South Lebanon population, affected by Israeli occupation” and the specific objective of “Increasing revenue by 30% of 500 farmers in 7 villages through: a) innovation of cultures and productive techniques; b) offer of production services; c) strengthening of management capacities of local institutions”. To this end, the project had identified 3 levels of results and a significant number of actions to achieve them, according to the outline of the Logical Framework below.

Intervention Logic Indicators Sources of Verification Conditions GENERAL OBJECTIVE Improve the socio-economic a) overall net migration Statistics from the Ministry conditions of South Lebanon b) % of young people of Agriculture and of population, affected by Israeli emigrating Health occupation c) No. years of life expectancy at birth d) GDP per capita SPECIFIC OBJECTIVE Increase revenue by 30% of 500 70% of farmers who Sample survey ad hoc in The competition of farmers in 7 villages through: a) modernised their business the 7 villages indicated in regional agricultural innovation of cultures and have increased their the project products is not favoured productive techniques; b) offer of income by 30% on by some forms of public production services; c) average in 3 years, taking subsidy that would strengthening of management as a reference per capita damage internal market capacities of local institutions income of 1999 estimated prices by Ministry of Agriculture at US$ 200 per month EXPECTED RESULTS Capacity Building 1.1 500 agricultural businesses a) 80% of farmers have a) LARI annual reports Availability of credit and modernised with the introduction introduced new b) Agronomic surveys on agricultural inputs to of new cultivation systems and cultivation systems samples of farmers economic conditions new cultures that have increased b) 80% have introduced appropriate to farmers’ their productivity new cultures potential c) 80% have increased traditional productions by 30% on average in the three years of project, with respect to the production of 1999 1.2 7 first-level and 2 second-level a) 70% of farmers have a) Constituent acts of first The different confessions

Evaluation ex post of NGO managed projects in Lebanon

associations constituted, gathered in 7 first-level and second-level do not assume a character integrating the majority of associations of producers associations of conflict and adversity agricultural producers and b) 5 first-level among the farmers groups coordinating productive and associations are associated commercial activities in 2 consortia of second- level 1.3 The 7 local associations a) All course participants a) Final reports and records Presence of a regulatory (municipality + caza) and 10 pass the final test of internships framework defining clear organisations (NGOs + verification b) Final satisfaction test responsibilities of associations) promote a more b) 75% of participants is c) Surveys of working institutions called up to cautious management of natural satisfied of the course procedures after one year, manage natural resources resources, in particular water contents with trained persons resources, through the specific c) 50% of participants use training of their staff (32 members in their work the tools of public local and regional acquired during the institutions and 30 members of training organisations acquire the specific skills to administrate and manage local natural resources) Providing production services 2.1 The Centre of agricultural a) 70% of farmers receive a) field notebooks of LARI and the Ministry of diffusion for research helps 500 support from the Centre Centre technicians Agriculture define an local agricultural businesses to b) 90% of new b) field notebooks for agreement for the improve production cultivations were sorting culture management of the Centre launched in collaboration of agricultural diffusion in with the Centre Khiam 2.2 Image and quality of local a) 70% of the 500 a) sample analysis on Quality standards are products improved through the agricultural businesses products accepted and respected by constitution of a trademark of produce and b) survey in some cities of the farmers geographical origin of products commercialise their own Lebanon products according to the trademark protocol b) the trademark is recognised by 5 consumers on 10 in big cities Networking and inter-institutional coordination 3.1 Set up a local coordination a) the Commission gathers a) minutes of the (CAZA), composed by regularly once a month Commission representatives from institutions, b) No. of agreements b) evaluation of the staff on agricultural and financial bilaterally signed between its role organisations and research centres, the different institutions to elaborate and coordinate and organisations agricultural development policies ACTIVITIES Capacity building Means (see budget) Costs (see budget) 1.1.1 Implementation of 35 The identification process a) Written profile of each demonstrating businesses (5 per of parcels is successfully selected parcel village) differentiated by agro- allowed. b) Monitoring report ecological area, availability of Successful completion of water resources and productive the programme techniques (intensive, agroforestry) 1.1.2 315 days of agricultural 400 farmers on 500 a) Participants list training (pre-seeding, on-going participate to at least 70% b) Trainers’ reports and field day) in 3 years in 7 of training days c) Evaluation form villages 1.1.3 Supply of agricultural inputs 400 farmers on 500 use a) Vivarium notebooks to extend demonstrating cultures selected material (seeds) b) Receipts and sales ledger produced by the Centre 1.1.4 Weekly technical assistance No. of farmers who Field notebooks of to each village (two technicians received assistance agricultural technicians from the Centre of agricultural No. of updating meetings diffusion) carried out in the villages 1.2.1 12 annual meetings in the 7 No. of participants of a) Workshop reports villages to promote the project and monthly meetings b) List of subscribers

Evaluation ex post of NGO managed projects in Lebanon

to create pre-associative groups of No. of pre-associative farmers groups constituted 1.2.2 120 days (4x10 months) of a) A programmatic a) List of workshop administrative, accounting and document for the 8 participants organisational training in 3 years associations; b) Evaluation reports from for the managers of b) Lobbying promotions trainers cooperatives/consortia/associations for the 8 associations; c) Reports of workshop of producers c) Networking participants coordination; d) Consultation process among the associations 1.2.3 Constitution and legalisation All the associations are a) Establishment minutes of of 7 cooperatives/associations of officially registered and the associations producers and 2 second-level operating b) Meetings minutes organisations 1.3.1 15 days of training per year a) All the persons a) Attendance records for administrative staff of participate to at least 70% b) Final evaluation form institutions/companies/local NGOs of the training modules; responsible of natural resources b) The contents and the management methodology of the training modules are satisfying 1.3.2 15 days of training per year a) All the 30 operators a) Attendance records for technicians in charge of the participate to at least 70% b) Final evaluation form implementation of actions of of training modules environmental remediation b) The contents and the (forestry, sewage management…) methodology of the training modules are satisfying 1.3.3 7 courses of 7 days each of At least 20 farmers Attendance records technical training for management participate to at least 70% and maintenance of irrigation of meetings equipment Providing services 2.1.1 Equipment of a centre of Centre equipped and a) Equipment inventory of agricultural diffusion to carry out operating the Centre researches and production services At least 150 each year b) Activity log receive assistance from the Centre 2.1.2 5 courses of 5 days each of Courses take place Programming frameworks training/updating for internal staff regularly. At least 3 of Centre activities; report experimental activities are on the research project of evaluated the Centre 2.1.3 Agronomic data collection Sufficient number of a) Verification of the system in the 7 villages at thematic maps produced; existence of the maps in determined time intervals and at least 70% of collected question preparation of cultivation plans information are in the b) Reports of information form of database brochures produced by the Centre 2.1.4 Definition of quality Number of quality Programming frameworks protocols and test of control analysis in one year of Centre activities quality (for organoleptic quality parameters and for residues of fungicide treatments) 2.2.1 Equipment of 7 centres, one Number of meetings Field notebooks of per village, of collecting and carried out in one year. At technicians advisers elaboration of products according least 50 farmers per to the protocols of the trademark village take part in the of geographical origin meetings. 2.2.2 Market research and creation Sufficient number of Meeting minutes of a database for the collecting of market analysis carried data on demand and supply of out local, domestic and international market 2.2.3 Cultivations programming Number of programming Field notebooks of and sorting in the 7 villages meetings carried out in technicians advisers

Evaluation ex post of NGO managed projects in Lebanon

according to the market one year. At least 50 requirements farmers take part in each meeting 2.2.4 Creation of the trademark of No. of promoting Meetings calendar geographical origin meetings carried out 2.2.5 Launch of 3 television No. of projected spots and TV contracts and contracts advertising and billboard published posters with promotion agencies campaigns for the promotion of the trademark of geographical origin and related products Networking and inter-institutional coordination 3.1.1 Creation of a working group The committee carries out Periodic reports on to define joint actions in favour of efficiently the functions meetings. Staff evaluation local agricultural development on its role 3.1.2 Convocation of bilateral negotiating tables to define framework agreements in the agricultural sector (amendment of legislative provisions, definition of guidelines, access to credit, circulation of agricultural inputs, commercial agreements) Security and access guaranteed in the project area PRECONDITIONS The peace process between Lebanon and Israel continues

For the purposes of this evaluation, the Logical Framework was an analysis tool quite useful in that it was sufficiently described and well structured. Regarding the general objective “Improve the socio-economic conditions of South Lebanon population, affected by Israeli occupation”, if the indicators a), b), d) appear to be too generic, the indicator c) does not seem reliable for both the expected results and the objectives, but especially for its duration. The sources of verification are also pretty generic. The specific objective “Increase revenue by 30% of 500 farmers in 7 villages through: a) innovation of cultures and productive techniques; b) offer of production services; c) strengthening of management capacities of local institutions” is in line with the general objective and is clear, well described and detailed. The chosen indicator, although being relevant, appears to be difficult to measure objectively, as it relates to the income declared by the beneficiaries. The verification sources are accessible and reliable. The expected results, with respect to the three components 1. Level of capacity building 2. Providing services 3. Networking and inter-institutional coordination are well described and contribute to the achievement of general and specific objectives. The indicators are well formulated, reliable, measurable and objectively verifiable. The verification sources are accessible and reliable. The activities are well detailed, the sequence and its mutual interdependence, including the time sequence and the duration, are well defined. The indicators are objectively verifiable and the verification sources are reliable and accessible. The necessary external conditions and pre-conditions have been well evaluated and made explicit, some of them occurred.

Evaluation ex post of NGO managed projects in Lebanon

A.3 EVALUATION ACCORDING TO THE CRITERIA OF RELEVANCE, EFFICIENCY, EFFECTIVENESS, IMPACT AND SUSTAINABILITY

a.3.1 RELEVANCE a.3.1.1 Identification of needs and of beneficiaries

The needs of farmers of the Plain of Marjayoun, predominantly of formative character, have been identified in 2000 by the local counterpart through an analysis work and a study of the local reality as well as a comparison with other local stakeholders, including the LARI and the Ministry of Agriculture, institutions deeply involved during the implementation phase of the project. In those years, the needs identified by the project appeared to be basically correct and responsive to the needs of the beneficiaries. It should be recalled, however, that the project activities started only in 2006, without making any substantial change to the document of the original project, in a moment of particular fragility of political and social context, leading to a change in needs and different priorities from those planned. The excessive amount of time which elapsed between the project presentation, its approval and its beginning has certainly had an important influence. To reorient and remodel the project activities and priorities, it is necessary to mention the active involvement and direct support from the Ministry of Agriculture and LARI, which have had a decisive role contributing to improve the project implementation by identifying the real needs of the moment. The identification of beneficiaries, carried out mainly by local municipalities, was very precise and took into consideration the gender component aiming to sustain women involvement in the activities. The meetings with a group of beneficiaries confirm the above said, as they consider the initiative was very useful and responsive to their needs. The intervention modality used, above all regarding the components of the capacity building and providing services, was very appropriate to facilitate economic development process in an area considered as rich in potential. As for the component of Inter-institutional coordination and its implementation, although the necessary activities started with periodic informal meetings during the first and second annuities, they were suspended during the third annuity because of the difficulties met in the project implementation and were consigned to UNDP which, in all countries where it intervenes, generally covers the role of institutional coordinator.

a.3.1.2 Identification of the local counterpart

The relationship with the local counterpart, the Scours Populaire Libanais (Lebanese People’s Rescue, SPL) was developed from 1999, with the planning of an intervention in the sector of transformation of agricultural products in South Lebanon. SPL reveals itself to be a very weak institution with respect to the expectations and unsuitable to the role of local counterpart for two fundamental reasons: first of all, because it lacks experience in the sector of agricultural development, since it is a NGO specialized in emergency interventions and in “Water and Sanitation”; secondly, because it has proven to be very deficient in terms of management, especially for what concerns the application of the rigorous procedures of the Ministry of Foreign Affairs/DGCS.

Evaluation ex post of NGO managed projects in Lebanon

Therefore, despite having participated actively in the initiative drafting, it failed to provide the proper support during the implementation phase, revealing itself little sensible to the activities. Its role was substantially materialised in the coordination of training activities and staff selection in the first period of project implementation. To overcome this problem, the NGO Movimento Africa ’70 (Movement Africa ’70) decided to involve more actively and directly the School of Khiam and the Ministry of Agriculture, which became the main partners of the project.

a.3.1.3 Relationships with central and local authorities

The Ministry of Agriculture was very weak during the first two years4 because of the complexity and the political situation at that time. The work with the aforementioned Minister was instead very positive in the third year, thanks to his strong presence and support. The local authorities also had a very important support role by allowing to integrate the project activities with the agricultural policies at that time.

a.3.2 EFFICIENCY

The outbreak of hostilities and the destabilisation of Lebanese internal politics have heavily conditioned and slowed down the first two annuities of the project. In the first two years of activity, the instability of Lebanese internal politics caused many managerial and logistical problems to a project which activities were located just 10 km from the border with Israeli territories. The extension of the project timelines led to the necessity to reorganise and actualise the project activities in order to keep up with a fast changing environment.

Reassuming, the project timelines were the following:

First annuity January 28, 2006 – November 27, 2007 (22 months) First request of slipping extension of six months on 16/01/2007 Second request of slipping extension of four months on 04/072007 Second annuity November 28, 2007 –November 27, 2008 (12 months) Third annuity November 28, 2008 –May 27, 2010 (18 months) Request of slipping extension of six months on 05/10/2009

Below are the variations requested during the project implementation:

First annuity January 28, 2006 – November 27, 2007 I Variation non-onerous dated on 4/11/2006 II Variation non-onerous dated on 18/04/2007 III Variation to the budget dated on 04/07/2007 Second annuity November 28, 2007 – November 27, 2008 I Variation non-onerous dated on 31/10/2007 II Variation non-onerous dated on 30/04/2007

4 The agricultural sector, primary component of the project, underwent the passivity of the previous Ministry of Agriculture and of its officials, in charge until the beginning of 2009. With the election of the new Minister a new impulse however occurred.

Evaluation ex post of NGO managed projects in Lebanon

Third annuity November 28, 2008 – May 27, 2010 I Variation non-onerous dated on 14/10/2008 II Variation non-onerous with extension request of six months dated on 05/10/2009

a.3.2.1 Funds and time management

First annuity: January 28, 2006 –November 27, 2007 The first annuity had a total duration of 22 months. The NGO Movimento Africa ’70 (Movement Africa ’70) presented its first extension request of six months on the first annuity on January 16, 2007. The reason behind this non-onerous extension request is to be attributed to the outbreak of hostilities with Israel on July 12, 2006. Following these war events, although the project was not officially suspended, it had to drastically slow down the implementation of the foreseen actions in the timing diagram. In addition, on July 17, 2006, the project manager decided to come back to Italy, suspending his functions in accordance with the Italian Embassy at Beirut and the Crisis Unit. During the project reprogramming, the project manager was no longer available to fill that position. That is why a selection of a new project manager was initiated and finalised. With the re-establishment of the minimum conditions of safety, a mission was carried out by an expatriate consultant commissioned by the NGO Movimento Africa ’70 (Movement Africa ’70). It finalised a comprehensive evaluation of the situation of the project area, on the base of which a new Operating Plan has been redacted of the first annuity’s actions, taking into account the profound changes that occurred in the beneficiaries’ needs and the nature of the intervention to implement. These changes have inevitably led to a non-onerous variation request, the first concerning the first annuity. A second variation request has been finalised at the purchase of an additional vehicle, necessary to guarantee a more efficient mobility of the project staff. A third non-onerous variation request and a further extension request of four months were submitted in July 2007, bringing the end to the first annuity to November 27, 2007. The mentioned requests intended to compensate for the delays caused by hostilities and the need for reorganisation and activities updating, in the light of the large period of time elapsing from the first draft of the project in 2000, as well as the organisational and operative weakening of the partner SPL and to the significant changes in the political context and the safety conditions in South Lebanon.

Second annuity: November 28, 2007 – November 27, 2008 The second annuity had a total duration of 12 months. Close to the end of the first annuity and following the strategic reset of the project activities, a non-onerous variation was requested in October 2007 on the second annuity (the fifth variation from the beginning of the project). This variation followed the substantial reset that occurred during the first annuity, which subsequently changed the initial setting. All the changes brought to the project during the first annuity inevitably affected the project on its entire duration. This is a rather complex variation, in which the strategies approved with the variations of the first annuity are proposed, with variations on the macro areas of expenditure items.

Evaluation ex post of NGO managed projects in Lebanon

Third annuity: November 28, 2008 – May 27, 2010 The third annuity had a total duration of 18 months, with an extension request of 6 months, which brought the project to end on May 27, 2010, and two non-onerous variations. The first variation of the third annuity (the sixth compared to the previous annuities) derives from an adjustment of the functioning costs of the project and confirms the guidelines and actions approved by the variations of the first and second annuities, which allowed the smooth execution of the project activities planned. In essence, the variation completes and consolidates a general reorientation and a coherent update of the initiative.

An extension of six months was then asked, for the completion of some activities (Training activities for local cadres – Items of expenditure 3.4 and 3.6) and was associated with a variation for the prolongation of some managerial costs of the project and of the position of leader project. Facing a request of variation of € 24,790.22 – which included some additional costs for the rental of six months of offices in Beirut and Marjayoun – the Ministry of Foreign Affairs granted approval of € 15,447.80 as the NGO was already beneficiary of a funding under the ROSS III emergency programme and thus was in measure to ensure cover of these costs.

In total, the allocated resources by the Ministry of Foreign Affairs were used in an efficient way, despite the numerous variations and extensions faced by the project. Changes made to the original document concerned more or less all the dedicated items of expenditure for the aforementioned reasons. The item of expenditure 3.4 Training of local cadres is worth mentioning, it refers above all to the training activities planned in the project and which are the main block of the entire initiative. Important changes were made on a methodological point of view, by introducing a more participatory approach to training and a better involvement of farmers. The efficient use of resources was made possible thanks to the flexible instruments (variations and extensions) the donor made available.

a.3.2.2 Staff management

Predictions of human resources employment were not well respected. A part of the local staff was reshaped after consultation, with a higher employment of part- time lecturers, technicians and students from the School of Khiam, instead of involving a smaller number at full time as it was initially planned. This choice was made taking into consideration the difficulties encountered in the recruitment of local staff, as Marjayoun region is an area difficult to reach and remote. In total there have been four Project Managers, with different bands, from band A do band C with consequent budget residues on various activities. Compared to the initial structure of the project, where more counseling from expatriated experts was planned, more space was instead given to local consultants, knowing the territory and the topics covered by the project. The local staff was selected by the counterpart SPL through job posting on websites (http://daleel-madani.org/webdirectory-ngos), CV gathering and selection, interviews and pre- selection test. During the interviews, the evaluation of the staff availability to work in such a critical area as the Plains of Marjayoun was very important.

Evaluation ex post of NGO managed projects in Lebanon

a.3.3 EFFECTIVENESS

Regarding the achievement of the specific objective “Increasing revenue by 30% of 500 farmers in 7 villages through: a) innovation of cultures and productive techniques; b) offer of production services; c) strengthening of management capacities of local institutions”, it should be underlined that it is an objective difficult to measure and to verify objectively. For the farmers, the mandatory management of accounting is not planned and tax system in Lebanon appears to be totally inefficient. The increase of revenue by 30% at the end of the project was based on the declaration of a sample of beneficiaries through a sample survey ad hoc in the 7 villages, as indicated in the verification sources of the Logical Framework. The achieved results of the project at its end with reference to the formulation from the Logical Framework are analysed below.

Level of Capacity Building

R.1.1 500 agricultural businesses modernised with the introduction of new cultivation systems and new cultures that have increased their productivity. This result has been largely achieved. All the beneficiaries, a total of 500 farmers – individuals or cooperatives – have adhered to new cultivation systems. The achievement of this result was concretised with the direct support to farmers through activities of technical assistance (services of agricultural dissemination and training), provision of inputs (mowers, providers for agro medicine, animal feed and fertilisers) to 308 farmers, cooperatives members and rehabilitation of certain machinery and facilities. The choice of inputs was made in an accurate case-by-case manner by the local staff in order to encounter the effective necessities of the beneficiaries. Thirty-five plots have been implemented (5 per village) through the selection of 35 producer/farmers who have actively participated to all the project activities, increasing their technical knowledge and managerial skills and making available to other farmers the acquired knowledge. Of the 35 demonstrative activities, five referred to women (14% against 20% of the objective). The demonstrative plots were located in different agro-ecological situations to respond to different types of environmental conditions and resources availability in the different production sectors. A particular attention in the 35 plots was dedicated to extensive and intensive farming the most representative of the agro-pastoral system of Marjayoun area, with forages production in the business for animal feeding. Regarding the training activities, they were launched with the argued delays during the first annuity and continued at their full capacity in the following ones, until the end of the project. Demonstrative days and field trainings were carried out, after preparation in classroom. The dissemination activity was implemented in experimental plots and in fields made available by farmers. A part of these training days took place in the Centre for Services of Khiam. In the course of the last annuity, the farmers had the opportunity to take part to visits outside the region to share their own experience, in particular in the context of beekeeping and natural resources management. In the three years of project, 856 visits to single producers were made, in which 273 to farmers who requested support and intervention of experts of the School of Khiam, after having participated to the field visits at the demonstrative plots. This shows the relevance of the service for the farmers, which allowed the project to achieve a higher number of beneficiaries than expected. In general, training and technical assistance were the main axes of the project and activities were continuously monitored.

Evaluation ex post of NGO managed projects in Lebanon

R.1.2 7 first-level and 2 second-level associations constituted, integrating the majority of agricultural producers and coordinating productive and commercial activities. This result was modified with a variation as the first-level associations were already registered. Therefore an attempt to strengthen the first-level associations already existing was made and to create two second-level associations through a series of trainings and legal counseling for their staff. The original purpose of these activities envisaged the institutional consolidation of certain cooperatives and the activation of a union between cooperatives, providing to a consistent group of civil servants and active members the instruments to strengthen ties with their members, to develop a working programme and to start activities of defence of their members’ interests and of lobbying in their favour. This pathway was characterised by strong problems because of the nature of cooperatives, malfunctioning and very politicised. After the war of 2006, funds started to pour and cooperatives were born to create a structured framework to aid distribution. From this point of view the result was scarce. Therefore training activities and legal counseling was often done with farmers not formally organised. The training activities initiated during the first annuity and continued until the end of the third. Contrarily to the prevision (administrative, accounting and organizational training), the formative modules were elaborated on the base of farmers’ needs with a participative spirit. Therefore it was decided to reorient a part of the training to the acquisition of technical skills (use of the received equipment, training on the transformation of milk into yogurt and cheese, especially for women of cooperative of Kfar Kila and Ibel Saki). The methodology on which the training was based had a prevalently practical character (training on the job, learning by doing, field visits and practical experimentation especially in what concerns early and late crops). Regarding the creation of two second-level associations, the project clashed several times with the individualistic interests of each cooperative and the relative political affiliations to create a consortium protecting the interests of the farmer members. There have been different meetings, both individually and with the cooperatives chairmen but they did not led to the desired result. Despite the efforts it was not possible to constitute two second-level associations.

R.1.3 The 7 local associations (municipality + caza) and 10 organisations (NGOs + associations) promote a more cautious management of natural resources, in particular water resources, through the specific training of their staff (32 members of public local and regional institutions and 30 members of organisations acquire the specific skills to administrate and manage local natural resources). The activities planned to achieve this result – modified and/or merged into variations – regarded a series of theoretical and practical trainings, launched during the second annuity, addressed to technicians and administrators in public and private sectors. The trainings dealt with the management of rivers, forest resources and green spaces external and internal to urban centres, treatment and use of sewage, pollution of water resources for domestic use, waste disposal. Managers of different municipalities took part in this training. Some visits outside Marjayoun (Bentjbeil) to two composting centres were carried out; this activity was extremely important for the Municipality of Khiam which realised subsequently composting activities of organic waste with a project financed by other donors. The visit allowed them to verify directly the issues and advantages related to the composting production from urban waste.

Evaluation ex post of NGO managed projects in Lebanon

The planned activities for the achievement of this result also concerned training on types of crops and irrigation systems. This training was initiated during the first annuity and continued in the following. The theoretical and practical courses organised resulted to be useful and enjoyed an active participation from local actors (372 presents) on key sectors of local development.

Level of Providing services

R.2.1 The Centre of agricultural diffusion for research helps 500 local agricultural businesses to improve production. This result has been largely achieved, despite the reformulation of some activities planned in the Logical Framework. The Services Centre for Rural Development of the School of Khiam proposed to ensure technical assistance to producers and cooperatives, orientation to municipalities and to provide services to famers, through the strengthening of the analysis laboratory of the School of Khiam. The Laboratory, previously created by the NGO ICU and destructed during the conflict, was rehabilitated and renovated (electrical and plumbing installations) by the NGO Movimento Africa ’70 (Movement Africa ’70) which also provided equipment and materials. At the conclusion of the project, these equipment and materials were donated to the Ministry of Agriculture. The Laboratory activities (analysis of soil, water, olive oil, compost and fertilisers) started at full capacity in 2009. During the project implementation phase, a high turnout was registered.

R.2.2 Image and quality of local products improved through the constitution of a trademark of geographical origin of products. In this case also some activities were reformulated in variations without influencing however the achievement of the expected result. The result was achieved through a series of activities, including: - Realisation, with a participative approach, of a brochure easily available for consultation by farmers, for the production of some cultivations mainly present in the area: olive, peach and nectarine and oregano. - Two market researches aiming to identify the most appropriate mode of production in terms of costs, quality and quantity of products and the promotion of agriculture as an activity and a valid source of revenue. - Creation, with a participatory approach, of the Trademark of Geographical Origin applied on agricultural products of the initiative beneficiaries. The trademark diffusion was made possible through the launch of advertising campaigns (billboards and local TV) and the direct support from producers through their participation to fairs and manifestations. Labels with the logo of the trademark of geographical origin were also made to be affixed on products. It was not possible to register the trademark (available at the office of the NGO Movimento Africa ’70, Movement Africa ’70) as it would have resulted in a longer project period.

Evaluation ex post of NGO managed projects in Lebanon

Level of Networking and inter-institutional coordination

R.3.1 Set up a local coordination (CAZA), composed by representatives from institutions, agricultural and financial organisations and research centres, to elaborate and coordinate agricultural development policies. During the two first annuities, the first coordination activities between the School of Khiam, municipalities and various NGOs were undertaken, through informal meetings of inter- institutional coordination. Due to important difficulties during the implementation phase of the project, other actions were prioritised and all the activities related to the realisation of the inter-institutional committee were suspended. The role of coordination between the various organisations working in the area was undertaken by UNDP, which in all countries in which it intervenes often plays this coordinating role in an institutional and highly professional manner. UNDP carried out periodic meetings, both general and thematic, at Tyre and Marjayoun.

A.3.4 IMPACT

The effects of the initiative in terms of impact, albeit positive, proved to be scarce. The services organised by the project were created to facilitate a developing process of the agricultural potential of Marjayoun area, with results in the medium and long term. In economic terms, the project invested a substantial fund, both for training, production and coordination activities. The investment results in the improvement of productivity in the Plain of Marjayoun, with obvious positive effects also on the population and the expansion of the production of the project’s target groups, in particular farmers. The gender component was taken into account, and women were mainly employed in food processing and honey production. Farmers received extremely important training sessions, even for those of them who were not initially farmers and who were re-employed in this sector for self-sustaining reasons.

The positive effects encountered are: - Acquisition by beneficiaries of new production techniques with consequent improvement. Compared to 2006 agricultural production has undoubtedly improved, but there are few farmers who base their income only on agriculture; - Acquisition of an “agricultural identity” by the beneficiaries of Marjayoun area, not previously included in the economic mapping. Farmers beneficiaries of the operation now appear as “conscious actors” and are perfectly aware of the importance and the potential each project activity have had on their agricultural activity; - New production techniques and new products, offering the possibility to have higher margins in agriculture, attracted also the younger ones who possess lands from their parents or relatives; - Improvement of production in certain periods of the year, when it was normally inexistent (temporary diversification and seasonal cultivations).

With respect to the points listed above, three and a half years after its conclusion, the project, although in a limited manner, continues to have a positive impact. The impact results to be more obvious and important on cooperatives which received equipment and continue to use it. For example, the cooperative of Kfar Kila commercialises its products (olive oil, soap and medicinal herbs).

Evaluation ex post of NGO managed projects in Lebanon

With regards to the women cooperative Ibel El Saki of food production, though it closed, some of the former members still achieve to commercialise their products (food products, soap, rose and orange water, cheese). At individual farmers/producers’ level, the impact was materialised with the improvement of production techniques and the use of inputs and fertilisers. The main problem at the moment consists in the lack of commercialisation of their agricultural products which determines consequently their income. The commercialisation of products is strictly related to the Trademark of Geographical Origin, its diffusion being possible through the launch of advertising campaigns, the direct support of producers through their participation to fairs and manifestations and the realisation of labels to apply on products. The trademark revealed itself to be a marketing instrument of high importance in terms of product commercialisation, contributing decisively in the increase of farmers’ income. However, the trademark was operating until the end of the project. Despite several attempts made by the NGO Movimento Africa ’70 (Movement Africa ’70), it has not been possible to register and certify the trademark, due to a lack of a legally recognised agency in Lebanon and to the bureaucratic difficulties encountered. To conclude, the limited impact of the initiative is not imputable only to the executive NGO, which, however, should have in design phase taken into account the eventual issues and weaknesses and define an alternative strategy. It is also attributable to the inability to pursue the work started for the trademark registration and the important component of marketing which would have positively influenced commercialisation of products and thus revenue increase.

A.3.5 SUSTAINABILITY

Regarding the issue related to the Capacity Building of cooperatives, the lack of association and cooperative capacities noted in the country makes the implemented activities with cooperatives less sustainable. Almost all cooperatives were created from the external necessity to give a public aspect to aid distribution, without a real interest in their proper functioning. This affected farmers themselves, who are discouraged by this form of association and prefer to work independently, without seeing the real advantages a cooperative society can offer. Regarding the Centre for Agricultural Dissemination Services at the Technical School of Agriculture of Khiam, a major objective was to make the Analysis Laboratory more functional to provide all analysis that can compete with those of the market. The equipment dotation allowed in fact to perform analysis of water and soil, proteins in aliments (cheese, eggs, beverages etc.), to analyse fertilisers composition to meet the minimum requirements for commercialisation (correspondence of analysed nitrogen content with the one declared), analysis of sewage sludge of water waste (to decide the potential disposal). The sustainability of activities from the Centre of Services and its equipment was assigned to the Ministry of Agriculture, which should have ensure qualified staff assigned to the Laboratory and its maintenance. The arrangements have ceased, perhaps because of a lack financial resources or because of political reasons. At the time of the mission, the equipment was still functioning but, because of the lack of appropriate staff, the Laboratory is not very active and all the chemical products have now expired. Moreover, the machinery maintenance was guaranteed by the firm where they were purchased until six months ago, currently no one is in charge. The risk is that this equipment will be completely dismantled and abandoned.

Evaluation ex post of NGO managed projects in Lebanon

A.4 CONCLUSION

The initiative appeared to be responsive to the beneficiaries’ needs. It contributed to trigger a development process of the agricultural potential of the plain of Marjayoun, by investing particularly in training activities and technical assistance to farmers. The investment results mainly in the improvement of productive capacities of farmers. The effects of this initiative are positive concerning the social impact (strengthening the sense of belonging to the area and the community, women empowerment) but very scarce concerning economic and income generating activities. The initiative was highly appreciated by the beneficiaries and considered essential in the contribution to the improvement of agricultural production. The results, however, should be reinforced and consolidated in order to fully achieve the general and specific objectives. Agricultural projects of this type require in fact much more time to consolidate on the ground. The scarce commercialisation of agricultural products is, at the moment, the main problem of the project beneficiaries, who have expressed the need to work in consortia and to find points of sale. From this point of view, the registration and the trademark certification of products and its diffusion through an accurate marketing strategy are of fundamental importance. Regarding the choice of counterpart, it turned out to be a total failure since SPL was not suitable to implement an intervention of agricultural development and unable to comply with the donor’s procedures. Regarding other issues encountered during the project implementation (total inefficiency of cooperatives) – it can be said that, although not directly imputable to the proponent NGO and its counterpart, it could have been better planned during the identification phase. The main perplexity is the almost absolute lack of sustainability of the project, which was discussed in the previous paragraph. The functioning at full capacities of the Laboratory of the School of Khiam is of fundamental importance, as it offers long-term production services, devoted to guarantee the initiative sustainability.

A.5 RECOMMENDATIONS

For the Ministry of Foreign Affairs

There has been detected a long time lag between the project presentation, on January 23, 2003, and its approbation on June 8, 2004. It is suggested to make these times faster and leaner, in order to avoid excessive deviations from the original funding proposal, especially in a country where political and social context changes quickly, since it is characterized by severe political instability and external influences. It is advised, during the technical and economic phase, to pay more attention in the choice of the counterpart by the NGO proponent. Moreover, continuous supervision and monitoring from the UTL of projects with particular problems are suggested. The evaluation lack at the project conclusion by the UTL does not allow to have, as useful it could be, an important element of comparison in the analysis of information collected.

For NGO Movimento Africa ’70 (Movement Africa ’70)

Given the important and non-negligible consequences of the choice of an unsuitable partner, it is recommended that the NGO Movimento Africa ’70 (Movement Africa ’70) choose local

Evaluation ex post of NGO managed projects in Lebanon

counterparts having resources and managerial capacities, and which are able to contribute to the initiative management and to assure its sustainability through time. The NGO Movimento Africa ’70 (Movement Africa ’70) is also invited to take into consideration the lessons learned during this project. This will contribute to a more active and in-depth identification phase and future projects elaboration, focusing on the aspects that have been the most impacted (formation component) and in which the expertise of NGO Movimento Africa ’70 (Movement Africa ’70) has been shown to have an added value.

Evaluation ex post of NGO managed projects in Lebanon

Part B

SUPPORT TO LOCAL DEVELOPMENT AND CREATION OF AN ORIENTATION PILOT CENTRE AT DEIR EL AHMAR, BEKAA VALLEY

AID N. 8650

ONG Ricerca e Cooperazione (NGO Research and Cooperation)

Resolution of Steering Committee n. 105 del 31.08.2007

Local counterpart Caritas Lebanon Total value € 1.640.174,58

BORNE BY DGCS: € 819.685,08 equivalent to 49,98% (plus € 54.000,00 to cover the cost of social security and insurance for Italian personnel cooperating)

BORNE BY ONG: € 247.174,00 equivalent to 15,07%

BORNE BY COUNTERPART: € 573.315,50 equivalent to 34,95% Intervention sector Agricultural development Implementation period 21.04.2008 – 30.05.2011 Type of evaluation INDEPENDENT EX POST EVALUATION Period of evaluation mission 28.11.2013 – 06.12.2013

Evaluation ex post of NGO managed projects in Lebanon

B.1 SYNTHESIS OF EVALUATION

CRITERIA STRENGHTS WEAKNESSES - Excellent collaboration with the local counterpart - Good collaboration with national - Hashish producers are authorities, particularly with the not among the project Relevance Ministry of Agriculture, and with beneficiaries community of reference - Good consideration of gender component - Good use of Ministry of Foreign Affairs’ resources. Funds have not - Rejection of a variant on been completely used in a the second and third consequent economy of annuity expenditures in relation to the total Efficiency value of the project - Fulfilment of activities in the course of the project - Good performance of the local counterpart - Not all the results have been completely achieved; - Achievement of the total number - There has been few of beneficiaries requests from beneficiaries of the rotating funds at the end Effectiveness of the project - Only about 440 persons out of 620 expected from the project document attended the technical/updating courses. There has not been sufficient Impact evidence to evaluate the impact. - The documentation center is There has not been sufficient functioning and active evidence to evaluate the Sustainability - Good performance of GAL (Local effectiveness of rotating fund. Action Groups)

Evaluation ex post of NGO managed projects in Lebanon

B.2 GENERAL FRAMEWORK b.2.1 Project synthesis

The project “Support to local development and creation of an orientation pilot center at Deir El Ahmar, Bekaa Valley” has been presented to the DGCS on July 17, 2007 and approved by the Steering Committee for Cooperation and Development with resolution No. 105 on August 31, 2007. The initiative has been implemented with the participation of the local counterpart Caritas Lebanon. Aim of the project was the triggering of a socio-economic development process and the diffusion of alternative cultivations, at the expense of the illegal ones in the region of Deir El Ahmar (CazaBaalbeck, Lebanon). This goal was meant to be achieved trough the creation of an Orientation to Development Pilot Centre, designed for training activities and for the aggregation of local population. In order to strengthen local capacities, the project provided functional activities to encourage the formation of Local Action Groups (GAL). These activities were intended to contribute to the definition of an integrated local development plan, alternative to the previous one, which was incoherent, unstructured and considerably bound to the eventual profitability of illegal cultivations. Other purposes of the project were more generic ones: poverty reduction within the region, reduction of rural exodus and elimination of illegal cultivations. Works started officially April 21, 2008 and ended May 30, 2011, lasting actually 38 months and 10 days (14 months and 10 days within the first annuity, 12 months for the second one and 12 months for the third one), compared to the 36 months planned. In effect, during the first annuity, an extension and two variations (the first one regularly approved and due to the replacement for resignation of the project manager; the second one rejected) were requested. b.2.2 Mission synthesis

An extensive documentation has been consulted before and during the evaluation mission; it included: - Financing proposal of the NGO and the relative technical and economic evaluation of the Local Technical Unit; - Requests for non-onerous variations and requests for extensions and relative technical advices; - Reports on the state of progress relative to the three project annuities; - Correspondences Ministry of Foreign Affairs/NGO Research and Cooperation; - Technical note from Local Technical Unit and request for opinion about the realization of the project goals.

During the mission, the team performed a series of meetings and interviews. Meetings were conducted on the base of the availability of local stakeholders and of beneficiaries verified by UTL: - UTL; - Caritas Lebanon; - NGO Research and Cooperation; - Meeting at Caritas Deir El Ahmar with Municipality, a representative of GAL and four beneficiaries.

Evaluation ex post of NGO managed projects in Lebanon

The following structure have been visited: - Caritas Deir El Ahmar. This evaluation identified some particular and significant limits both for expected meetings and for the gathering of information and documentation. - The organization of meetings was problematic. It was not possible to meet the fundamental actors of the project (as the Ministry of Agriculture, the Lebanese University, the Union of Cooperatives and the Episcopate of Deir El Ahmar) even if the meetings were in the project agenda and their importance was stressed many times during the mission; - The implementation of focus groups with a representative sample of beneficiaries was not possible. The local counterpart Caritas Lebanon, responsible for the organization of meetings, only involved four beneficiaries for a meeting with the representatives of Caritas Lebanon, the Mayor of the Municipality and a representative of GAL; - This evaluation is mainly based on the documentation produced for reasons aforementioned; - Despite our requests, we could not get much of the documentation filed at the counterpart’s offices; this prevents us from doing a more detailed analysis and from having a partial validation of information gathered and supplied by Caritas Lebanon; - More significant limits concern the evaluation about impact and sustainability. b.2.3 Analysis of coherence and quality of the design of the Logical Framework To make this section more comprehensible, we briefly recall the foundation of this project. It pursues the following generic aims: 1. gradual interruption of illegal cultivations in Bekaa; 2. reduction of rural exodus; 3. increasing the income of disadvantaged families; 4. new social groups’ access to the labor market and the more specific aim of “triggering a local socio-economic development, alternative to the one based on illegal cultivations”. For this purpose, the project identified four main expected results and several means to achieve them, according to the outline of the Logical Framework below.

Intervention Logic Indicators Sources of Conditions Verification GENERAL OBJECTIVE 1 Gradual interruption of illegal  the amount of  local markets cultivations in Bekaa products exported survey from the region in 10  exportations years increases by 8%  the amount of legal cultivations in the region increases by 15% 2 Reduction of rural exodus  the amount of young  Sample surveys people working in  Statistical the region in 10 surveys by the years increases by Ministery of 10% Education  the amount of  Numerical requests for surveys at the emigration visa in 10 Regional

Evaluation ex post of NGO managed projects in Lebanon

years is Maternity Wards reduced/fixed  the amount of births in loco in 10 years increases by 5%  enrolment in primaries schools increases in 15 years 3 Increasing the income of  Education among  Sample surveys disadvantaged families rural families  Ministry of increases Social Affairs  The amount of families sustained for economic diseases decreases by 10% in 10 years 4 New social groups’ access to the  The amount of  Chamber of labor market entrepreneur women Commerce increases by 5% in Zahlé. 10 years  Statistical  The amount of surveys by the women registered in National the National Assistance Fund Assistance Fund increases by 10% in 10 years SPECIFIC OBJECTIVE The triggering of a local socio-  Within the end of the  CCIA Zahlé.  Governmental policies economic development, alternative project, at least 5  Sample against illegal to the one based on illegal kinds of new interviews cultivations are cultivations economic activities  Farms implemented are born in the accounting  International region  Region’s Data organizations operate  Within the 3 years of base focusing on the the project, at least  MoA’s census. eradication of illegal 15 farms of the cultivations region improved  Political stability in the their farming area systems. EXPECTED RESULTS A Local actors are able to interact and  Implementation of 1  GAL  Local authorities do develop integrated local GAL documentation not oppose to activities development strategies throughout  Within the third year  Interactive in question the implementation of an interactive of the project, the Charts produced  There are no and multi-disciplinary summary GAL produced a  Constitutive oppositions to the use document. The latter is used to local action plan assemblies of satellite collection define available resources and to  Within the first half memoranda of data for the outline possible future scenarios for of the second year of  Local action plan territorial mapping development. activity, an Implementation of GAL integrated Chart of physical resources + urban and rural local organization (WebGIS system) was produced B Stakeholders and farmers attend:  180 workers of the  The Orientation  There are no pressures educational and technical orientation project area used Centre Book from illegal activities regarding the potentialities orientation services  Registered cultivations farmers. of their area of expertise (and/or within the 30th accesses to the alternative ones); training activities month of the project. website regarding the improvement of  At least 60% of  Reports of the cultivations. stakeholders in the project project area know  Sample about the services interviews offered by the Centre  Project database  At least 20% of  CCIA Zahlé.

Evaluation ex post of NGO managed projects in Lebanon

farmers adapt or change farming procedures after having used the services offered by the Centre C The farm managers in the region can  At least 9 new  Callings to the  Farm managers rely on rely on new and improved professions for the Centre new professions professions. agricultural industry  Book of the created are born within the workers  Farm workers actually 30th month of the attending attend retraining project updating and courses.  At least 3 existing training courses professions for the  Sample agricultural industry interviews are updated within among farm the 30th month of the managers in the project region  At least 10% of farm  Progress of the managers in the project region used agricultural professions created by the project. D Residents of the region have more  At least 4 new  CCIA Zahlé.  Support to local opportunities to work in extra professions in the  Sample development agricultural sectors field of food interviews. governmental policies processing are born have been  At least 6 new implemented by GO professions for extra and NGO agricultural sectors  Farm workers actually are updated participate to  Workers involved in recommended extra agricultural activities sectors increased by 5% ACTIVITIES A0 Setting up and organization of the project office A00 Setting up and organization of the The building of the Centre Orientation and Services Centre is not delayed Activities concerning the expected result “A” A1 Territorial mapping using a system There are no oppositions of satellite collection of data GIS and to the use of satellite elaboration of computerized and collection of data for the interactive technical papers; the latter territorial mapping are provided with legend concerning the actual condition of the territory, the condition of the planning, possible intervention strategies and sustainable development scenarios. suA Implementation of a WebGIS for 2 using the research results A3 Implementation of an integrated and  Essential information interactive database with the is easily accessible resources of the territory  Willingness to interact from people unrelated to the project A4 Implementation of a Local Action  Actual participation of Group (GAL). people to community development activities  Local authorities agree with the implementation of the GAL A5 Technical and logistical support to  Willingness of workers

Evaluation ex post of NGO managed projects in Lebanon

the meetings between the local GAL to move and cooperate. and the national and international  Easy access to visas ones. Aim of this initiative was  Local authorities agree sharing experiences and good with the operating procedures and giving a implementation of the boost to the implementation of GAL national and international network operating for the self determination in the development process. A6 Implementation of a plan of action for local development Activities concerning the expected result "B” B1 Preparation of sectorial studies about  Easy access to available resources and potentialities information/data of the regional productive section  Willingness of people to cooperate  Workers effectively cooperate B2 Entraining courses to improve  Local farm workers agricultural procedures use entraining activities offered  MoA’s willingness to send their technicians to courses B3 Entraining courses aimed at Local farm workers enjoy introducing new agricultural entraining activities procedure offered Activities concerning the expected result “C” C1 Entraining courses for skilled farm  Local farm workers workers enjoy the entraining activities offered Activities concerning the expected result “D” D1 Implementation of entraining courses  Implementation of a for the processing of agricultural sale network products  Interest in offered products D2 Activities supporting the marketing  Farm workers are of products processed by local interested in entraining cooperatives activities offered Cross activities T1 Implementation of a Credit Fund There are no interferences within the definition of the credit criteria T2 Implementation of a documentation centre

The Logical Framework turned out to be an unreliable tool for this evaluation. Concerning general objectives, indicators used cannot be observed until 2021 and 2026 - a too wide period - therefore they cannot be observed at the moment. Furthermore, some indicators are not quantified and not objectively verifiable, others are not specific and realistic. The specific aim of “triggering a local socio-economic development, alternative to the one based on illegal cultivations” its logically connected to the general ones. Indicators are objectively verifiable while recommended sources of verification are not useful and accessible. Nevertheless, indicators prove that the project wanted to vary the supply of professions extending them beyond agricultural industry, though knowing that redirecting illegal cultivations in the agricultural industry was possible. Concerning the expected result A, two results have been combined (the interactive and multidisciplinary synthesis map and the implementation of GAL), not necessarily connected though the interactive synthesis map is a useful tool for GAL’s activities. The result

Evaluation ex post of NGO managed projects in Lebanon

contributes to achieving the specific aim, indicators are objectively verifiable, easily identifiable and coherent with the expected result. The expected result B contributes to achieving the specific aim and indicators are coherent with the result. Nevertheless, it is not clear if indicators evaluate illegal farmers or all farmers in the area. The expected result C is logically connected with the specific aim, indicators are coherent with the result, objectively verifiable and easily identifiable. The expected result D, though logically connected with the specific aim, seems to extremely extend the beneficiaries of the project, from illegal farmers (which were, anyhow, not beneficiaries of the project) to citizens in general. Consequently, the 3rd indicator of this result is not easy to evaluate. All the activities are well depicted, sufficiently detailed and coherent with the corresponding expected result. All the identified sources of verification are reliable. Conditions and risks are clearly identified. Some of them occurred during the implementation of the project.

B3 EVALUATION ACCORDING TO THE CRITERIA OF REVELANCE, EFFICIENCY, EFFECTIVENESS, IMPACT AND SUSTAINABILITY b.3.1 RELEVANCE b.3.1.1 Identification of needs and of beneficiaries The project has been proposed by Caritas Lebanon and the Episcopate of DeirElAhmar which, together with the NGO Research and Cooperation, analyzed the opportunity of an intervention to foster the development recovery in the region. In 2004, NGO Research and Cooperation accomplishes the first mission for early surveys. Several further missions and meeting with local stakeholders led to the outlining of the local situation and needs. The main problems the project wanted to deal with were: 1. The absence of orientation services for both the agricultural industry and other present or potential industries; 2. The absence of informational and entraining centers for professional and technical qualification and retraining; 3. The access to credit for lower class citizens was problematic; 4. There was no coordination between local stakeholders and institutions; consequently, they were not able to develop and implement integrated territorial development strategies. Thanks to the engagement of the Union of Cooperatives, a questionnaire to reveal the needs was given to the project beneficiaries. Therefore, the choice of entraining activities was grounded on the beneficiaries’ necessities and requirements. The identification of beneficiaries, carried out with the partnership of local municipalities, was accurate and took into account the gender component for the aim of supporting women participation in activities. Territorial support Caritas centres present in the area gave to the project was essential; it quickens the identification of beneficiaries, of new necessities and possible local and

Evaluation ex post of NGO managed projects in Lebanon

territorial problems. During the mission we were able to meet only 4 beneficiaries, which considered the initiative a very useful one and responding to their needs. A so limited and not symbolic sample didn’t allow us to have a reliable feedback. b.3.1.2 Identification of the local counterpart The NGO Research and Cooperation is proud of a long-time collaboration with Caritas Lebanon, an institution that operates in the Lebanese territory, particularly in DeirElAhmar region through a department office. Caritas Lebanon has a deep experience in agricultural industry, local development and food farming processing; in the past, it already managed several projects with the Ministry of Agriculture in the areas involved in this project. Thanks to its experience in the field, it has been possible to have a detailed and updated picture of the local situation and of the several interventions already implemented in the area (due to the delay between the identification and the approval of the project). The chosen local counterpart had a fundamental role in the implementation of activities and staff management. Its support was determined and continuous; it was an actual co- management and, sometimes, Caritas filled the vacuums of NGO Research and Cooperation, due to the absence of a project manager (3 project managers succeeded). The counterpart shared all project’s execution phases, giving its support both at institutional level with local authorities and at territorial level with structures and staff in several centres in the territory. Thanks to its technical experience in the territory, the project could explain local situations, being dynamically integrated in the local development already started in the area. Furthermore, it contributed with a micro-credit component that provided an economic support to some beneficiaries for improving their activities. In this way, it guaranteed a better support for their families, affected by the economic crisis due to the 2006 war. b.3.1.3 Relationships with local authorities National authorities, the Ministry of Agriculture in particular, turned out to be helpful and positively participated to the events organized during the implementation of the project. Concerning decentralized authorities, a network with all actors (public and private) present in the territory has been created. One of the main expected results of the project was the implementation of a Local Action Group (GAL), to promote local development and involve local actors (we will discuss this later). b.3.2 EFFICIENCY b.3.2.1 Funds and time management Though the intense political instabilities and the actual logistic, climatic and environmental problems, due to the nature and the isolation of the area which were very problematic for the project planning, the project has been implemented in 38 months and 10 days instead of the 36 expected.

The timetable of the project was: 1st annuity 21 april 2008 – 30 may 2009 (14 months and 10 Request for an extension for a two months and ten days delay on the date days) 30.03.2009 2nd annuity 31 may 2009 – 30 may 2010 (12 months)

Evaluation ex post of NGO managed projects in Lebanon

3rd annuity 31 may 2010 – 30 may 2011 (12 months)

The requested variants in three annuities were:

1st annuity 21 April 2008 – 30 may 2009 I non onerous variant 2nd annuity 31may 2009 – 30 may 2010

II non onerous variant requested for 2nd and 3rd annuity on the date 30 march 2010 NOT APPROVED 3rd annuity 31 may 2010 – 30 may 2011

The project has undergone changes in the term of expenditures before the official launch, which took place on April 21, 2008. After UTC’s favorable reception of the technical and economic evaluation, the project has been officially approved within the Steering Committee with resolution n° 105 on August 31, 2007, for a total contribution of € 826.791,65 from the Directorate General for Development Cooperation (partitioned in 3 annuities). Later, the Central Account Office at MAE remarked that some items of expenditure were higher compared to the retail amounts of the addeds. Therefore, the proposer NGO reformulated a new budget for a total contribution of € 819.685,08 from the Directorate General for Development Cooperation, economizing € 7.106,57. This fact provoked remarkable confusion when producing later documentations. In some documents we can find old amounts, in other the new ones. Concerning time management, the timetable for activities proposed by the project have been followed. Indeed, only an extension of two months and 10 days on the 1st annuity has been requested, due to early resignation of the project manager. The NGO then had to choose a new personality. The early resignation was also the reason for the request for a variant. Some of the project manager’s monthly salaries have been reallocated in favor of a 30 days wage to the Italian Coordinator. He was the project supervisor during the transitional period of the registration of the contract for the new project manager. Also two entraining professional courses (Local Sustainable Tourism Management and Tailoring) have been included in the variant. The rejection of a relevant variant for the 2nd and 3rd annuities was problematic. Among several items of expenditure subject to variant there was the cancellation of the Rotating Fund for the implementation of an Integrated Touristic System5. The Rotating Fund, in the original document of the project, was one of the main activities in respect to the sustainability of the initiative. Since it was not possible to interview the administrative staff of the project, it is difficult to evaluate the efficiency on the whole. According to the documentation, MAE’s resources turned out to be very efficiently used, although not all results have been completely fulfilled; in this latter case, their use was not so efficient. The main remark regards the employ of many professions considering the Italian and local staff, which represents the 53,47% of the total value of the initiative.

5 NGO’s works for the implementation of an integrated Touristic System have been suspended due to the negative outcome of the variant, leaving activties unfinished.

Evaluation ex post of NGO managed projects in Lebanon

Compared with the estimated budget of € 1.640.174,58, the actual cost of the project was € 1.621.489,68 with an unexpended balance of € 18.684,90. b.3.2.2 Staff management The forecasts on staff employment have been mainly followed. The local staff has been duly entrained about the project goals and the organizational procedures. Three project managers succeeded. First manager’s resignation, which caused the prolonged absence of a manager within the project, didn’t provoked particular problems, thanks to the joined effort of the project staff and the Caritas counterpart. The project has been co-directed and co-managed in all its parts, including the staff employed. b.3.3 EFFECTIVENESS Shown below the project final results referring to the Logical Framework form. Result A: Local actors are able to interact and express integrated local development strategies, throughout the implementation of an interactive and multi-disciplinary synthesis map, used to identify the available resources and outline possible development scenarios. Implementation of the Local Action Group. Regarding mapping activities, other mappings had already been done in the project area by different authorities within another project (TERCOM). Therefore, the existing tools have been made available to the Orientation Centre users and to the local communities. Events and meetings for the divulgation of information gathered have been organized; the purpose was easing the territorial awareness both from an agricultural point of view (soil characteristics, cultivations, water resources, weather) and from the point of view of the supply of services in the territory. Access to the Internet for the consultation of research material on similar themes has been made available for the Orientation Centre users. Moreover, from 2009, Lebanese government implemented a computerized GIS system that can be consulted at the website http://www.gislebanon.gov.lb. All the gathered material through both studies and scientific and technical publications (handbooks, studies, soil maps) constituted a library that can be consulted with photocopiable paper material at the centre and for free. Concerning the implementation of the Local Action Group, the existing GAL Baalbek “Via Appia6” has been reinforced; it had been implemented within the project TERCOM by IAM of Bari and co-financed by Italian Cooperation. During November 2010, two GAL from Puglia (“Terra d’Arneo” GAL and “Santa Maria di Leuca” GAL) met “Via Appia” GAL. This visit was a good way to exchange knowledge and procedures to facilitate the implementation of a strategic local development plan. The group went through 5 training days and made several tours within the territory, with particular regard to the project areas. During the training, informational material and material produced by the two Italian GAL has been provided. The Baalbek GAL and the two GAL from Puglia exchanged their contacts in order to share information and support, with a view to a potential financing for implementing the strategic plan of GAL Baalbek. During the training, the possibility for the two GAL to create a reciprocal exchange in terms of projects implementation came to light.

6 There are 8 principal members (Managing Board) and Baalbek and Yammune Municipalities, local associations, cooperatives and private entities take part in it.

Evaluation ex post of NGO managed projects in Lebanon

A strategic plan for local development has been developed with the aim to appraise the cultural and traditional heritage of the area to increase tourism. The actual activities didn’t comply with the plan, except for an entraining course for tourist guides and a workshop on sustainable tourism, which took place in 2009.

Result B: Farmers and agents have access to technical and informational orientation activities on the potentialities of their area of expertise or alternative ones and to entraining activities aimed to the improvement of cultivations. This result have been successfully reached. Between 2008 and 2009 four sectorial studies: 1. Updated survey on the socio-economical characters of the population involved in the project of 2008. 2. Study in the agricultural industry. This survey implied: 1) administration of a specific questionnaire addressed to administrative authorities of 28 villages; it made the comprehension of perceived problems in the area possible. 2) visiting the main production industries within the territory (cooperatives, farms, potential tourism sites); this allowed the checking of main problems and positive examples in the field. 3) administration of a questionnaire addressed to 303 entrepreneurs in the agricultural industry with the aim to highlight the farming peculiarities in the area. 3. Socio-economic survey 4. Survey on craft.

Result C: farm managers of the region can count on new and improved professions. This result has also been reached through professional entraining courses:  21 persons trained in nursery techniques and micrografting techniques (2 cycles for an amount of 42 persons);  28 persons trained in compost techniques and fertilization of fruit trees (2 cycles for an amount of 56 persons);  42 persons entrained in compost (practical course)  24 persons entrained on different types of grafting for fruit trees

Result D: citizens of the region have more opportunities of employment in extra-agricultural sectors. Several cycles of entraining courses have been implemented for an amount of 440 beneficiaries against the 620 expected in the project document. Courses lasted for an average of 5 working days. It was not possible to check the actual repercussion of these activities in the economic context within the region.

To reach the specific and general aims, the project considered some cross activities.

T1 implementing a loan fund The fund has been regularly used by the counterpart. Caritas Lebanon created an account for this fund, proceeding with the distribution and managing agreement with the referential bank. A questionnaire for the evaluation of the proposals concerning the implementation of projects has been developed, both from an economical and a social point of view. Priority has been given to applicants who attended entraining activities within the project. The amount of applicants for the initial loan was 15 (5 in the agricultural industry for the improving of equipment and machinery; 2 in the food processing industry; 8 in professional crafts sector). Compared with the amounts of attendants to the entraining courses, the amount of applicants for the loan was very low.

Evaluation ex post of NGO managed projects in Lebanon

T2 implementation of a documentation centre The centre has been implemented and it is constantly updated with new materials. At the centre there is a database which can be consulted with the list of publications partitioned by themes and of all the materials about the training occurred within the initiative. The materials, supplied as booklet, are provided to users who ask them for. b.3.4 IMPACT This evaluation identified some particular and significant limits and we don’t have sufficient and reliable elements to give an opinion concerning the criteria of the impact. Besides many difficulties regarding the organizing of meetings and the access to produced documentation, many times requested, it was not possible to meet stakeholders of the project nor organize any focus group, despite they were set in the timetable. Caritas Lebanon affirms that about the 80% of beneficiaries increased their income; the project strengthens the region’s socio-economic processes, supporting small and medium farms in the region, also promoting new entrepreneurial activities. This processes have been a valid alternative to the local agricultural reality, which following the prohibition of illegal cultivations by the government had to redevelop the agricultural lands exploitation. Different solutions to guarantee an income have been found. b.3.5 SUSTAINABILITY At the end of the evaluation of the mission, the elements were not sufficient to fully understand the project sustainability. Concerning the Rotating Fund whose management was devolved upon the counterpart, updated data were not available. At the conclusion of the project, people who benefited from the fund were barely the 3% (15 applicants of 440 courses attendants). The data supplied to the embassy/UTL by Caritas Lebanon, a year past the conclusion of the project (may 2012), show a relevant increase of beneficiaries who achieve access to the credit (52 beneficiaries), having the budget expected by the project completely allocated. Concerning the documentation centre implemented at Caritas Lebanon, it is functioning and available for users who ask for it. The centre has a library and there is a database with the list of publications divided by themes and all materials concerning entraining courses implemented during the initiative; it can be consulted. Since it was not possible to meet a representative sample of beneficiaries or agricultural- economic professionals, we are not able to give an opinion about its utility. In closing, the work implemented through the reinforcement of the GAL Via Appia Baalbek, created in 2008 thanks to the help of IAM Bari, partner of the project, proceeds producing good results. Currently many social and economic professionals and seven municipalities (Baalbek, Iaat, Chlifa, Btedii, Deir El Ahmar, Taraya, Yammouneh) work in the GAL. Recently a networking job started with Byblos. Though the local development strategic plan aimed at the promotion of cultural and traditional heritage did not have a feedback in the project activities, it allowed the development of financing requests to several donors. Four municipalities proposed ten development projects to UNPD and to the EU, of which one has been approved and financed.

B.4 CONCLUSION

Evaluation ex post of NGO managed projects in Lebanon

This evaluation, as it has already been said before, identified relevant limits regarding both the access to documentation and the meetings. These limits did not allow us to have a clear and updated framework on the concrete situation. The evaluation is, therefore, incomplete. We do not have sufficient elements to express an opinion about the relevance of the aims and their fulfillment, about the beneficiaries’ perception of the utility of the initiative, about the impact. Concerning the project sustainability, the information gathered are not sufficient: according to the counterpart’s statements, the Rotating Fund is functioning, also for the non beneficiaries of the project, but we do not have updated data though our requests to Caritas Lebanon, both during the mission and later via mail. Regarding the documentation centre at Caritas Lebanon, we do not have sufficient elements to express an opinion about its utility, since it was not possible to meet a representative sample of beneficiaries and/or agricultural-economic professionals. We were able to interview only with one of the three project managers (dr. Jessica Uccellatori, who followed the final phase of the project) and with the representative of Caritas Lebanon. The meeting with the NGO Research and Cooperation, both in Italy and in Lebanon, did not convey relevant and sufficient information. The brief conversation had with the Mayor of the Deir El Ahmar municipality focused especially on the importance of reinforcing the tourism of the area, an element subject to a rejected variant from the MAE. The strength of the project is identified in Caritas Lebanon’s good management. The counterpart shared all the project phases, giving its support on an institutional level with the local authorities and on a territorial level, with structures and staff in several centres all-over the territory. An additional positive element is identified in the concrete work of the GAL Baalbek, of which we talked about in the paragraph concerning sustainability.

B.5 RECOMMENDATIONS For the Ministry of Foreign Affairs

We advise the donor, when evaluating the techno-economic aspects, to analyze the whole project document including the logical framework which, for the evaluation aim, is of great importance. The logical framework identifies some indicators that cannot be observed before 2021 and 2026, others not specific and realistic, others not quantified and objectively verifiable. Concerning the produced administrative documentation, there is a kind of confusion: as reported in the paragraph pertaining efficiency, the project went through variations in the expenditure specifications after its approval. Documentation does not always report the same amounts. If possible, we suggest a revision of this documentation. A deeper coordination with the Local Technical Unit or with other offices at the headquarters is advisable, to verify projects currently underway or concluded, so that it is possible to avoid duplications or overlappings and, therefore, make the ODA more efficient. A last point concerns the relatively extended times related to payments, resulting in the NGO and the counterpart paying in advance.

For the NGO Research and Cooperation

We call upon the NGO Research and Cooperation to consider what has been learned in this project, with a better development of the identification phase, and to consider the risks of implementing activities unexpected in the project document and that are not Italian

Evaluation ex post of NGO managed projects in Lebanon

Cooperation’s priorities - with a particular reference to the rejection of a variant which caused problems in terms of efficiency and effectiveness. A deeper coordination with stakeholders for the implementation of activities (some of them, as the implementation of the GAL, already implemented within another project financed by the Italian Cooperation and implemented by the IAM Bari) is advisable, to guarantee the efficiency and avoid possible duplications or overlappings. We recommend, therefore, a more careful drafting of the Framework, as a tool of work useful to clearly identify the different elements of the project. In closing, we call upon the NGO Research and Cooperation to facilitate the possible future work of evaluators, supplying contacts and documents in due time.

Evaluation ex post of NGO managed projects in Lebanon

SOCIO-ECONOMIC DEVELOPMENT OF THE FISHERMEN COMMUNITY OF BATROUN AID N. 8255

ONG Ricerca e Cooperazione (NGO Research and Cooperation)

Resolution of Steering Committee n. 142 of 25.10.2005

Local counterpart Organisation de Développement pour la Paix Sociale (ODPS) Total value € 1.423.780,28

BORNE BY DGCS: € 707.211,54 equivalent to 49,67% (plus € 54.000,00 to cover the cost of social security and insurance for Italian personnel cooperating)

BORNE BY ONG: € 214.114,78 equivalent to 15,04%

BORNE BY COUNTERPART: € 502.453,97 equivalent to 35,29% Intervention sector Socio-economic development Implementation period 12.03.2007 - 12.04.2010 Type of evaluation INDEPENDENT EX POST EVALUATION Period of evaluation mission 28.11.2013 – 06.12.2013

Evaluation ex post of NGO managed projects in Lebanon

C.1 SYNTHESIS OF EVALUATION

CRITERIA STRENGHTS WEAKNESSES - Local authorities’ positive attitude - Good and continuous collaboration with national (the - Not fully responding to Ministry of Agriculture) and local the beneficiaries’ needs Relevance authorities (Batroun Municipality) - Lacking experience of - Excellent interaction with the the local counterpart project beneficiaries - Problems relating the relationship with the fishermen cooperative - Fulfilment of activities observing Efficiency the timetable - Not fully efficient use of allocated resources - Not all the expected results nor the general and specific aims have Effectiveness - been completely achieved;

- A strong feeling of sharing aims within the Batroun community and - Lacking impact due mostly to Impact among beneficiaries scarcity of fishing resources

- The Loan Fund is not functioning anymore Sustainability - The local counterpart ODPS went out of business

Evaluation ex post of NGO managed projects in Lebanon

C.2 GENERAL FRAMEWORK c.2.1 Project synthesis The initiative called “Socio-economic development of the fishermen community of Batroun” has been presented to the DGCS on October 28, 2004 and approved by the Steering Committee with resolution No. 105 on October 25, 2005. The project has been implemented with the participation of the local counterpart Organization de Développement pour la Paix Social (ODPS), born in 2004 to answer the Lebanese society’s needs. The project, lasted three years, was aimed at the revitalizing of the fishing industry in the coastal town of Batroun, with the objective to improve the fishermen’s quality of work (specific objective) and to contribute to the improvement of life quality of fishermen and their family, as well as the development of economic activities related to the sea (general objective). The initiative was meant to support the fishermen’s empowerment through the promotion of new participative approaches and new local development dynamics. The main project activities were directed to people of the sea (informal education, introduction of new techniques and/or technologies, establishment of a loan fund, management of natural resources, raising awareness on workplace safety and improvement of embarkation safety) in order to encourage an increase in fishing profitability and, at the same time, reduce the risks while working. Activities officially started on March the 12th, 2007 and ended on April the 12th, 2010, with an effective duration of 37 months (respectively 13 months the first annuity, 12 months the second annuity, 12 months the third annuity) instead of the 36 expected, following the request of 2 non onerous variants (the first one during the first annuity and the second one during the third annuity) and one extension of 30 days. c.2.2 Mission synthesis Before and during the mission in Batroun, documents gathered at the Directorate General of Cooperation and Development offices and at NGOs offices have been analyzed; among which: - financing proposal; - request of non onerous variants and extension and related technical advice; - narrative reports (three annuities) and balance sheets (I and III annuities) regarding the progress of the project; - monitoring reports within the NGO; - report on the management of the microcredit fund; - letter to the Municipality of Batroun; - UTL’s technical note; - project closing file; - correspondence MAE/ONG RC.

During the mission in Lebanon the evaluation team carried out meetings and interviews: - UTL; - Municipality of Batroun; - Research and Cooperation Lebanon; - A sample of beneficiaries.

Evaluation ex post of NGO managed projects in Lebanon

This evaluation identified some particular and significant limits both for expected meetings and for the gathering of information and documentation: - The organization of meetings was problematic. It was not possible to meet the fundamental actors of the project, as the local counterpart ODPS, which ceased activities at the end of the project, the Ministry of Agriculture (fishing department), the Ministry of Social Affairs, the Fishermen of North Lebanon Trade Union and the National Centre for Marine Sciences (NCMS), though the meetings were in the agenda and their importance was stressed during the mission; - This evaluation is mainly based on the documentation produced for the motivations above mentioned; - Despite our requests, we did not have access to a good part of the documentation which would have allowed a more detailed analysis and a partial validation of the little info gathered.

On site visits of places concerned in the initiative: - Equipment distributed during the project - Equipped ships

After the mission, all the information gathered during the interviews and the focus group has been confronted, in order to integrate them. Later we proceeded with the drafting of the evaluation report as requested in the reference aims. c.2.3 Analysis of coherence and quality of the design of the Logical Framework To make this section more comprehensible, we briefly recall the foundation of this project. It pursues the following generic aims: 1. contribute to the improvement of life quality of fishermen and their family; 2. development of economic activities related to the sea. And the specific aim “improvement of the fishermen’s quality of work”. For this purpose, the project identified two main expected results and several means to achieve them, according to the outline of the Logical Framework below.

Intervention Logic Indicators Sources of Conditions verification GENERAL OBJECTIVES  the amount of people  Registrations in contribute to the improvement of life enrolled in the the Fishermen 1 quality of fishermen and their family Fishermen Trade Trade Union Union stabilises (Tripoli)  Tourism increases  Official statistics  The amount of of the Ministry people starting of Tourism, of economic activities the Chamber of development of economic activities related to the sea Commerce and 2 related to the sea increases of Tax Office  The already existing business related to sea activities increases SPECIFIC OBJECTIVE  Amount of young  Regional and  Implementation of improvement of the fishermen’s people between15 national statistics governmental politics quality of work and 20 years with a by the Lebanese aimed at founding a

Evaluation ex post of NGO managed projects in Lebanon

higher degree of Ministry of social, healthcare and education increases Education pension prevention by 20% in 10 years system for the artisan fishermen EXPECTED RESULTS  Amount of ships  Census of ships  More regular equipped with safety equipped with maintenance of ships equipment increases safety equipment by 30% at the end of during the Improvement of sea safety A the project project conditions in Batroun  Knowledge  Questionnaires at concerning the start and at workplace safety the end of the increased project  The average income  Economic  Reduction of of fishermen families inquests within competitiveness of Increase of Batroun fishermen’s in Batroun increases the project imported fishing B income by 20% by the end products

of the third year of the project ACTIVITIES Activities concerning the Result A RESOURCES COSTS  Workplace safety Study about the safety of the fishing specialists A1 fleet of Batroun  Social hosts in support  Workplace safety specialist  Social hosts Raising awareness of the importance A2  Local workgroup of workplace safety enjoyed by fishermen and/or members of their families A3 Implementation of a loan fund  Socio-economic study  Economy specialists  Social hosts  Microcredit specialist  Loan funds A4 Organization of entraining/updating courses about ships maintenance Activities concerning the Result B  Implementation of laws on fishing and reinforcement of controls against illegal practices by the Lebanese Government  Economy  Implementation of B1 Socio-economic study specialists environment laws to  Social hosts effectively regulate an environment preserving management of the territory  Equipment are still easy to find  Fishing specialists B2 Fishing industry study  Social hosts

Evaluation ex post of NGO managed projects in Lebanon

 Socio-economic study (B1)  Regular meetings of the workgroup with informative sessions about different sectors (law, small business management, B3 Community building activities workplace safety, sea resources management, ecc.)  Participation specialist  Economy specialist  Environment specialist  Fishing specialist  Social hosts  Fishing industry study (B2)  Implementation of theoretical B4 Fishing practices training seminars  Practice exercise afloat  Equipped pilot boats  Regular workgroup meetings Raising awareness of environment B5  Environment and resources sustainability specialist  Social hosts  Purchase and installation of on board safety (GPS) and fishing Equipping pilot boats for illustrative B6 equipment purposes  Purchase of fish traps, nets, etc., for illustrative purposes  Socio-economic study  Economy B7 Implementation of a loan fund specialists  Micro-credit specialist  Loan funds PRE-CONDITION Keeping the regional

political aspects stable

The logical framework turned out to be an unreliable tool for this evaluation. Concerning the general objectives of the project, “contribute to the improvement of life quality of fishermen and their family” and “the development of economic activities related to the sea”, the chosen indicators are rather generic or non relevant. The specific aim “improvement of the fishermen’s quality of work” is related to the general objectives from a logical point of view, nevertheless, the choice of the indicator seems only indirectly connected and too much postponed (10 years), as well as easily influenced by many

Evaluation ex post of NGO managed projects in Lebanon

other factors (access to schools, fishermen’s awareness of children education, safety conditions). Furthermore, the verification sources could be difficult to access to in a decade. The two expected results, “improvement of sea safety conditions in Batroun” and “increase of Batroun fishermen’s income”, as expressed, seem logically connected with the specific aim of the initiative. Not all activities seem coherent with the respective expected results (ex.: the community building activity (B.3) does not seem directly inherent to the result B, neither to result. A C result could have been thought, specific for this kind of activity). In closing, it is to be stressed that the activity concerning the organization of entraining/updating courses on boat maintenance (A.4) does not exist in the original framework, but it is reported in the report on the progress of the 1st annuity. Conditions, as risks, are clearly stated.

C.3 EVALUATION ACCORDING TO THE CRITERIA OF REVELANCE, EFFICIENCY, EFFECTIVENESS, IMPACT AND SUSTAINABILITY c.3.1 RELEVANCE c.3.1.1 Identification of needs and of beneficiaries The project has been identified by the NGO Research and Cooperation and implemented in collaboration with the local counterpart ODPS. The idea of this project came out following a very similar initiative concerning the socio-economic development of a fishermen community in Tiro, realized with Caritas Lebanon. Therefore, it actually is a project identified and reformulated on the base of this previous experience in Tiro. The idea of the project came out following the worsening of the environmental situation in the area, which influenced the economic conditions of the fishermen community of Batroun. In particular, from the documentation we deduce the following problems: fishermen’s poor quality of life, high risks connected with afloat safety, lack of laws on social safety, insufficient maintenance of the fleet, high costs for the equipment, problematic acccess to credit for the fishermen, competitiveness with fish imports, illegal fishing and exploitation of sea resources). Nevertheless, the very few interviews we could carry out did not supply sufficient and reliable information concerning the identification of the beneficiaries’ needs. From the only reading of the documentation, we can deduce that to identify the beneficiaries, the Batroun fishermen’s Cooperative, included between other bodies involved in the project document, supplied a long list of fishermen’s names, divided in 3 groups (full time fishermen, part-time fishermen, pastime fishermen) which included a big amount of “non fishermen” or “sometimes fishermen”. Thanks to the socio-economic study, started during the first annuity and lasted for the entire duration of the project, managed by an economist, a participation specialist and a social host, it was possible to identify who was doing fishing as a sustenance activity (totally or partially). The latter were the project beneficiaries and have been completely involved in the expected activities. From the focus group, formed by a sample of beneficiaries, it emerged that the initiative has not met fully their needs. If the conditions on safety at sea have definitely improved, no result has been achieved concerning the increase of their revenues due to the fish resources scarcity of sea. The meeting with the Municipality of Batroun, which contributed to the identification of beneficiaries, confirmed what mentioned above. c.3.1.2 Identification of the local counterpart

Evaluation ex post of NGO managed projects in Lebanon

The chosen local counterpart ODPS (Organisation de Dèveloppement pour la Paix Social) was founded in 2004 - year in which the project has been presented - by the former president of Caritas Lebanon, an organization which the NGO RC had cooperated with for the realization of other projects and went through collaborative relationships. Today, the local counterpart no longer exists. From the interview with one of the two project managers, it can be seen that ODPS cease its activities in 2010, the year when the project concluded. Following the ceasing of the activities by the counterpart, it was not possible to analyze some aspects in depth, among which the one relative to the Rotating Fund, whom management the ODPS was entrusted with. The fundamental role of the local counterpart, during the implementation of the project activities, as well as the management of the Rotating Fund, was essentially to facilitate the relationships with the referring community and with both the local and the national authorities, and also supply supporting staff and select local specialist to include in the project organogram. c.3.1.3 Relationships with local authorities The project produced a strong feeling of collaboration with local authorities, in particular with the Batroun Municipality, and other bodies involved in the project, among which the Ministry of Agriculture’s Oceanography and Fishing Institute of Batroun, which supported the project in all its phases. No collaboration and involvement, on the other hand, with the Social Affairs Ministry, except for the sharing of information about the project progress, although it has been included between several other bodies involved in the initiative. The role of the local Fishermen Cooperative7, the fishermen’s stakeholder within the community and in the Lebanese context, is not clear (maybe because of problems during the firs annuity in relationships with the Cooperative president, who was opposing the project). Since we were not able to understand the fundamental motivation of this opposition and since it was not possible to interview any spokesmen, we cannot draw a conclusion. Those who took part in the focus group stated that the Cooperative is participating, active, but they participated in the project individually. c.3.2 EFFICIENCY c.3.2.1 Funds and time management Despite the strong instabilities of the Lebanese context (in particular the outbreak of hostilities in 2006, the disastrous effects on the marine ecosystem caused by the oil runoff from the generating station in Jihye, as well as the NahrElBared crisis in May 2007, few months after the activities started), the project started its activities on March the 12th, 2007 and ended on April the 14th, 2010, with an effective duration of 37 months instead of the 36 expected in the project document.

The timetable of the project was: 1st annuity 12 march 2007 – 12 April 2008 (13 months) Request for an extension for a month delay on the date 26.02.2008 2nd annuity 13 april 2008 – 12 april 2009 (12 months) 3rd annuity 13 april 2009 – 14 april 2010

7 The role of the cooperatives consists in helping their members (not only fishermen, but also farmers) though the enrollments are often influenced by local powers games; their aims is to increase their electoral support making include their names in the lists.

Evaluation ex post of NGO managed projects in Lebanon

(12 months)

The requested non onerous variants were:

1st annuity 12 march 2007 – 12 april 2008 I non onerous variant 3rd annuity 13 april 2009 – 14 april 2010 II non onerous variant

The request for an extension of a month during the first annuity essentially derived from problems concerning the implementation of fishing courses with a practical demonstration afloat, problems due to inclement meteorological conditions. The first request for a variant referred to the enter into force of new reporting procedures, which considered the introduction of final accounting audit costs, with resulting changings in some items of expenditure. The second request for a non-onerous variant during the third annuity referred, instead, to the modification of some items of expenditures, among which the reduction of the microcredit fund, implemented at the beginning of the project and lasted for the whole third annuity; it represented an opportunity for fishermen both to stabilize their activity and for increase their competence through the learning of entrepreneurial abilities and familiarity with banking procedures, previously not accessible for them. The Rotating Fund, at the end of the project, has been legally moved from the NGO RC to the counterpart ODPS. A rate of repayment of the order amounting to 93% had been registered when the transfer was due. Following the ODPS’s bankruptcy (pointed out also for the promoted project AID 9027/RC/LBN “Integrated Local Development in seven municipalities in Tiro’s region, South Lebanon), after the end of the project, the rotating fund was included in the bankruptcy procedure. Fishermen who had to give some installments back, suspended their payments, but still expressed their will to reactivate them once the fund would have been reconstituted and managed by another entity (for example, the fishermen’s cooperative), for other beneficiaries’ benefit. We notice that the activity concerning the implementation of a loan fund is in the logical framework twice (respectively A3 and B7); therefore, it is expected both for the achievement of the expected result A and for the expected result B. Nevertheless, it is identified as a unique activity in the reports. Since we did not have adequate answers concerning the Rotating Fund, we cannot draw a conclusion. Hence, as we were not able to interview the administrative staff of the project, it is also very difficult to do an overall evaluation about the efficiency. Concerning time management, we noticed a fairly precise realization of activities. The main perplexity regards the collapse of the local counterpart and in consequence the collapse of the rotating loan fund, which represent important elements to affirm that, in this specific case, the use of the provided funds has been inefficient. The same is for the partial achievement of the expected results and the generic and specific objectives. Concerning the estimated budget, the project ended with a surplus of €2.934,22

Codice Items of expenditure Expected cost Effective cost

1 VOLUNTEERS AND COOPERANTS’ SELECTION AND

Evaluation ex post of NGO managed projects in Lebanon

TRAINING

1.1 Volunteers’ selection and training 1.2 Cooperants’ selection and training € 2.065,83 € 2.065,83 VOLUNTEERS AND COOPERANT’S MANAGEMENT AND 2 WAGES 2.1 Volunteers’ wages 2.2 Cooperants’ wages € 168.134,06 € 157.875,24 2.3 Costs of volunteers and cooperant’s management € 7.436,88 € 7.230,30

3 OTHER ITEMS OF EXPENDITURE 3.1 Volunteers and cooperant’s techno-professional enhancement 3.2 Other Italian staff € 180.257,81 € 187.520,64 3.3 Local staff € 345.562,93 € 359.633,41 3.4 Local staff training € 88.973,00 € 88.271,60 3.5 Scholarships 3.6 Equipment € 118.302,91 € 115.908,46 3.7 Constructions € 145.200,00 € 142.325,17 3.8 Lands 3.9 Extra financial resources € 191.520,00 € 186.173,67 3.10 Monitoring and verification missions € 48.034,27 € 45.549,04 3.11 General costs € 128.292,69 € 128.292,69 TOTAL AMOUNT € 1.423.780,28 € 1.420.846,06

c.3.2.2 Staff management Though some problems related to the choice of a local Economist8 and a Workplace Safety Specialist during the first and the second annuities, previsions about staff employment have been rather respected, except for some little variations on the items of expenditures. Two project managers succeeded. The staff has been chosen by the NGO RC in cooperation with the local counterpart, which supplied local professionals to include in the project organogram.

c.3.3 EFFECTIVENESS

8 In Lebanon, economist is the equivalent of Business Administrator

Evaluation ex post of NGO managed projects in Lebanon

The analysis of results achieved at the end of the project, referring to the logical framework formulation, shown below.

R.A. Improvement of safety conditions afloat in Batroun

This expected result has been achieved through a series of activities, including the implementation of a training/update course on boat maintenance, which was not included in the Logical Framework but was present inside narrative and reporting documents. This implementation was logically continuous with the expected result and coherent with the project finalities. Fishing is considered an extremely dangerous job and in most cases incidents are attributable to the absence of appropriate onboard equipment, to the lacking of experience or technical knowledge, as well as inappropriate training. To achieve this result, a series of activities which have been regularly realised was expected:

A1: During the second annuity, a study about the safety conditions of Batroun’s fleet has been conducted (though some information had been gathered during the first annuity), aimed to the gathering of information useful to the achievement of the expected result. In our opinion, it would have been better to realize this study, of which we do not have a duplicate, during the first annuity since it would have represented the foundation of the whole executive organization of the initiative.

A2: An educational and raising awareness on workplace safety campaign has been realised during the third annuity, through the distribution of instructive brochures, posters and seminars which facilitated the continuous sharing of information between staff and beneficiaries. For the fishermen’s use, a handbook on safety conditions has been produced in English and French; later, it was distributed to the competent Ministries and to the fishermen’s cooperative (at a later stage it was translated in Arabic and it was distributed as a comic strip, to have a more extensive circulation and for a better understanding).

A3: Already during the first year, a loan credit has been funded to supply fishermen with financial resources for the purchase or the renewal of new equipment, in order to increase onboard safety conditions and the efficiency of the fishing boat. The credit supply for current costs has been left out (purchase of fuel, oils) or ordinary boats maintenance. Furthermore, those who fished pastime have been not included in the microcredit. Abovementioned the list of procedures and criteria necessary to obtain access to credit: - First, the exhibition of a Microcredit application form at the project offices was necessary; it had to include the information about the applicant, the requested amount, the aim and a pro forma invoice for the purchase of good/s; once all criteria were fulfilled, the procedures for the request started, otherwise it would have been rejected. - Later, a deep analysis of the ability to return the requested amount (“Appraisal of the elegibility criteria”) was expected, through a series of information about the family members, possible additional income sources, work experience, possible other financing requests, etc. - In the third phase, a committee formed by 4 fishermen elected by the community (Representative of the Fishermen9) gathered; it gave a first opinion about the credit

9 in the project document the fishermen’s Cooperative had an active role both in the preliminary phase of financing requests and in the later phase of reimbursements. Since a lack of collaboration from the cooperative’s managers was identified, the Representatives of the Fishermen, formed by 4 fishermen, was born.

Evaluation ex post of NGO managed projects in Lebanon

supply; later, the Management Committee for Microcredit gathered to definitively validate the supply of all or part of the requested amount. - Last, once went through all the previous phases, the applicant signed a financing contract and when signing, the project manager issued the cheque.

Due to bureaucratic problems, supplying money through a financial institution was impossible. Since a joint credit line (RC/ODPS) would have taken very long time and caused many bureaucratic problems upon the handover when the project ended, the supply was direct, through a specific account at Audi Bank. Though, we can tell from the documentation that, at the end of the project, €30.000,00 were supplied to 18 fishermen, members of the Batroun community. The reimbursement rate, at the end of the project, was equal to 90%. The aim was the creation of a rotating fund so that reimbursements were once again at the community’s disposal. The official accounting was managed by the project manager who also took care of the deposit of gathered amounts of money. The Cooperative was guaranteeing the gathering of the expected quota to reimburse the Counterpart. As above-mentioned, today the local counterpart is no longer existing. This is a problem because it is responsible for the microcredit fund management. Some participants of the focus group told us that they had access to the credit, but now they don’t know who to address to for the reimbursement rates.

A4: The entraining/updating courses on boat maintenance (engines maintenance, appropriate procedures to recycle oils), expected for all the project duration, were scarcely attended, unlike the previsions (<280 participations), since some arguments have been discussed also during the informal meetings of community building. Materials for boat maintenance have been supplied to 54 fishermen and, contextually, the maintenance training occurred. During the third annuity, of 280 expected participations (6 days a year for 15 fishermen in 3 years of project) 115 have been registered. It is worth to highlight that this activity is not in the original logical framework, though it is reported in the first annuity. It is very coherent with the project objectives, and in particular, with the expected result A.

Indicators identified to achieve this result, as reported in the original logical framework, are the following: 1. Amount of boats equipped with appropriate instrumentations increased by 30% at the end of the project: to reach this aim, the supply of updated equipment was not expected (activity indeed expected for the result B - see logical framework B&). It is not clear if this equipment has been supplied thanks to the access to the loan fund or through the project funds or during the organization of entraining/updating courses on boat maintenance (A4), as abovementioned. 2. Awareness of the importance of workplace safety increased: it is difficult to measure improvements in knowledge and or behavioural changings since no test has been carried out after the entraining/updating courses on boats maintenance. Nevertheless, the interviewed beneficiaries praised how the courses have been managed, as well as the raising awareness on workplace safety campaign, the quality of the material supplied (brochures, handbook on safety conditions, posters) and the reports drafted with all the staff.

R.B: increase of Batroun fishermen’s income

Evaluation ex post of NGO managed projects in Lebanon

Shortly, this expected result cannot be considered fulfilled for several motivations, primarily because it is difficult to compare incomes when the project started and those registered at its end. The meeting held with fishermen pointed out that one of the main current problems is the absence of fish resources, caused by the lack of an adequate protection legislation of maritime ecosystem, the non-application of the closed season to allow the reproduction of fish stock, the illegal fishery (with the use of poisons and explosives) and the water pollution (problems already identified at the project beginning), which resulted in a worsening of their revenues in 2007.

To achieve this result, a series of activities which have been regularly realised was expected, though many difficulties:

B1: An economist, a participation specialist and a social host realised a socio-economic study, started during the first year of the project; it showed a clearer picture of the relationships between the Cooperative management and the fishermen, of social dynamics, of different target groups’ specific needs, as well as, thanks to the daily relationships with the community, the possibility to identify those who were fishing for their livelihood (partially or totally). The gathering of economic data continued through the whole duration of the project.

B2: A study about the fishing industry has been organised during the first annuity, illustrating the Batroun fleet and the equipment used; it was carried on in the second and third year with the gathering and the analysis of fishing data and economic indicators, as the daily gross income, daily costs, etc. The computer system for the systematisation and the analysis of gathered data (through the use of a database), suggested by the project and shared with the Lebanese Ministry of Agriculture during the second annuity, has been made more efficient thanks to the support of ministerial staff and to the sharing of opinions.

B3: The community buildings activities aimed at increasing the involvement of the workgroup formed by the community fishermen. Started during the first annuity, when a cognitive survey on community mechanisms was made and target groups were identified, these activities later outlined conflicts within the fishermen’s Cooperative (in particular, the president did not seem to be appreciated by the majority of members) and an informal committee was born. During the second year of the project, attempts to include the cooperative’s management were dropped, constantly keeping the Batroun Mayor, mr. Marcellino Hark, informed; until the half of the third annuity, indeed, following the changing concerning the cooperative’s managers (occurred in January 2010), an attempt was made to give a new impulse. Also during the third annuity, some project activities have been addressed to the Batroun’s students, age-group 16/17 years, who carried out social participation activities with the support of the whole project staff. Each class drafted two booklets concerning Batroun’s problems and their possible solutions. Of the 180 expected participations to the meetings, the documentations declare 314 participations (students and fishermen) registered for the community meetings. It is worth to highlight that these activities are not directly connected nor with the expected result A neither with the B, so an expected result C could have been included in the logical framework, which, given the problems pointed out, seemed fundamental.

B4: The project specialists carried out several sessions of training on fishing techniques at the port; there were 43 attendants (respectively 18, 11 and 14 for each session) against the 180 expected during the first annuity. The training supplied with useful information aimed at

Evaluation ex post of NGO managed projects in Lebanon

widen the fringing reef in order to reduce the current exploitation of the area, avoid the production of new techniques that can endanger some marine species, verify and eventually introduce new techniques, etc. later, other informal training sessions on fishing techniques occurred, in order to sharing information and knowledge between specialists and fishermen. During the third annuity 299 participations against the 180 expected have been registered (the sessions concerned the appropriate assembling and consequent use of fishfinders); a practical exercitation followed the theoretical one. Four new fishing techniques, never used before in the intervention area, have been introduced, among which shrimps traps, long liners and two different kind of nets.

B5: Activities concerning environmental awareness and sustainable resources expected the students’ collaboration, as for the community building activities. During the first year of the project, a meeting/discussion on environmental emergency, on the interactions between the local community and the civil protection, volunteering and networking has been organised. 129 students and some professors and fishermen participated to the meeting. The meeting between fishermen and student can be seen as a very positive attitude within the community. During the second year too, sessions on environmental sustainability, pollution especially in Lebanon have been held; information about the Mediterranean oceanography have been supplied (temperature, saltiness, nutrients). Cooperation with the National Centre of Sea Research of the Lebanese National Committee for the Scientific Research (headquarters are in Batroun) has been enhanced and a visit to the research boat CANA10 (a gift by the Italian government). During the last year 498 participations to educational/discussing sessions have been registered, against the 180 expected. As previously stated, this activity too could be considered as an expected result C.

B6: Concerning the activity of equipping pilot boats for illustrative purposes, at the end of the project 15 boats were equipped with fishfinders and 22 boats with radio communication VHS systems. Furthermore, during the second year of the project, the equipping of some boats with fishing tools for illustrative purposes (among which a specific net for fishing hakes, traps for octopuses and nets for shrimps) has been provided. Later, the majority of the port fleet has been updated; boats have been provided with fishfinders (sonar which allow the operator to receive information about the profundity, the ocean floor structure, as well as identify fishes), GPS devices, nets and other fishing equipment. Moreover, the cooperative has been provided with radio VHS systems, which make the communication between fishermen and the civil protection possible. Despite that, during the focus group, it has been pointed out that part of the equipment supplied to the fishermen is not being used because it is not accurate when locating schools of fish, so that it is no longer in use. Nets lasted only four months, also. Furthermore, participants to the focus group reported that the actual problem of the area is the absence of fish resources, caused by the lack of an adequate protection legislation of maritime ecosystem, the non-application of the closed season to allow the reproduction of fish stock, the illegal fishery (with the use of poisons and explosives) and the water pollution (problems already identified at the project beginning). The absence or the inappropriate use of the equipment available for beneficiaries negatively influenced the achievement of the expected result.

B7: In this context also the implementation of a loan fund for the upgrade of boats was expected, as abovementioned. As already told, the activity concerning the implementation of a loan fund is in the logical framework twice (respectively A3 e B7), expected both for the

10 The first boat making oceanographic researches in Lebanese history

Evaluation ex post of NGO managed projects in Lebanon

achievement of the expected result A and B; nevertheless, in reports, it appears as a unique activity. We cannot draw a conclusion since wee did not have sufficient answers about the Rotating Fund issue.

The indicators identified to achieve this result are the following, as reported in the original logical framework: - The Batroun Fishermen’s families’ average income increases by 20% by the end of the third year of the project. At the beginning of its activities, the initiative estimated an average income equal to 189.000 LL corresponding to about 125$ per month per fisherman. Nevertheless, it is still difficult to compare incomes registered at the beginning of the project and those registered at the end. On one hand, an increase of average incomes could be supposed, since there was a significant impact in investing ability; in confirmation, indeed, the very high rate of reimbursements (equal to about the 90%) of the monthly installments of those who obtained the access to the loan fund. On the other hand, as stated by the beneficiaries during the focus group, and as already previously stated, the main current problem in the area is the absence of fish resources, caused by the lack of an adequate protection legislation of maritime ecosystem, the non-application of the closed season to allow the reproduction of fish stock, the illegal fishery (with the use of poisons and explosives) and the water pollution (problems already identified at the project beginning).

In view of what abovementioned, the specific objective “Improvement of the Batroun fishermen’s quality of work” can be considered as partially achieved. c.3.4 IMPACT The present evaluation went through some particular and relevant limits and we do not have sufficient and reliable information to express an opinion concerning the impact criterion. Beyond the many difficulties in the organization of meetings and in the obtaining of the produced documentation, it was not possible to meet the key stakeholders, though they were in agenda. Data gathered during the focus group pointed out that that since 2007, there has not been substantial change in terms of economic or social development of the beneficiary community. Definitely the project contributed to strengthen the sector and to create a strong sense of sharing common problems, thanks to several community empowerment activities which have been very participated by the Batroun community. Furthermore, the project supported the fishermen’s community empowerment, by suggesting new participatory approaches and new representative and local development dynamics. Since we did not have the possibility to interview any spokesman from the Ministry of Social Affairs, which was interested in the project activities, especially for its impact on the population’s quality of life, we cannot draw further conclusions. c.3.5 SUSTAINABILITY The partial achievement of expected results and, therefore, of general and specific objectives, such as the collapse of the counterpart and of the rotating fund - of which we talked about in the efficiency paragraph - and all the encountered problems with the Fishermen Cooperative are important and significant elements to say that there are no elements ensuring the project sustainability. The local key actors have a fundamental role in guaranteeing that the processes started by the initiative will have a long-term sustainability.

Evaluation ex post of NGO managed projects in Lebanon

Thanks to the relevant interaction between the project and the community during the three years of the project, hopefully in future fishermen will promote the initiatives born during the project with new generations. In closing, since the project set a strong collaboration with local authorities and, in particular, with the Municipality of Batroun, a higher sustainability of the activities started during the project should have been achieved.

C.4 CONCLUSION This evaluation, as already told before, went through relevant limits both in the access to documentation and in meetings. These limits prevented us from having a clear and updated picture both when identifying the project and in the actual contest. We do not have sufficient elements to draw a comprehensive conclusion about the importance of needs when identifying the project and so, on the objectives, since the few interviews (with the two project managers, the NGO RC and the former president of ODPS, who answered via mails) did not supply sufficient information. This led to difficulties when trying to objectively understand the documentation we referred to, which is also incomplete and of which we talked about in previous paragraphs. Concerning the needs identified by the project, from reading the documentation it emerges that a long-lasting identification phase had started in 2001, when the NGO Research and Cooperation’s specialists started a consultation with some representatives of the Oceanographic and Fishing Institute of Batroun and with the local fishermen’s Cooperative. Problems concerning poverty, absence of safety conditions afloat, and the additional effects of the increasing level of pollution of the Lebanese coasts (for example, urban waste water drained into the sea without a having been processed, NDA Lebanon, 2003) and the extreme exploitation of sea resources terribly reduced the sea “fertility” (a process which seems to be overwhelming on an international scale). Therefore, the necessity to act in order to improve the socio-economic conditions of the Batroun fishermen’s community is born. At the time of the mission, beneficiaries pointed out other priority problems to solve and preparatory to this initiative, especially concerning the environment (the lack of an adequate protection legislation of maritime ecosystem, the non-application of the closed season to allow the reproduction of fish stock, the illegal fishery and the water pollution). An optimal environmental situation could be an important resource of wellness rather than an economic one, indeed, because it would have positive economic, social, cultural and healthcare effects for the entire community. The very brief meeting with the Mayor confirmed what abovementioned, even if it cannot be found in the documentation supplied (see the Municipality’s appreciation letter). The expected results and the general and specific objectives have been partially achieved. The impact has been extremely low and reduced except for the fishermen’s community empowerment, by suggesting new participatory approaches and new representative and local development dynamics. The project sustainability is still a critical element for the motivations abovementioned.

C.5 RECOMMENDATIONS For the Ministry of Foreign Affairs

Evaluation ex post of NGO managed projects in Lebanon

(project presented to the DGCS on 28/10/2004, approved with resolution No. 142 on 25/10/2005, order of commitment on 14/12/2006). When a period longer than a year is spent between the initiative submission and its approval, it would be appropriate to require the proponent to update the financial plan submitted before the approval. It is suggested that where some initiative show particular problems, the supervision from the UTL should be more constant and in-depth, in order to ensure a greater efficiency in official development assistance. It is recommended to the donor, in phase of technical and economic evaluation, to analyse the whole project document including the logical framework. It is particularly advised in phase of submission of the project proposal to require the formulation of objectively verifiable in time indicators. The lack of such indicators reduces the possibility to evaluate, through precise data, the effects the initiative would have generated with respect to those it actually produced in the intervantion area.

For the NGO Research and Cooperation It is recommended to the NGO to choose more carefully the local counterpart, especially for this kind of initiative requiring experience and expertise. Its collapse at the end of the project had negative repercussions in terms of initiative sustainability and, therefore, in the lack of access to loans by the beneficiaries. A more accurate redaction of the Logical Framework is also recommended, this being a useful working tool to define clearly the different elements of the project operation and of the whole Project Cycle. It is suggested to involve in a better manner the local enterprises and other institutional subjects in the sustainability strategy of the project initiative. The NGO Ricerca e Cooperazione (Research and Cooperation) is finally invited to facilitate the future work of evaluators by providing contacts and documents in time.