A Critical Analysis of the Leader Evaluation
Total Page:16
File Type:pdf, Size:1020Kb
Academic year 2015/2016 FROM GOOD-WILL TO GOOD-USE: A CRITICAL ANALYSIS OF THE LEADER EVALUATION Matteo Metta Supervisor: Dr. Barbara van Mierlo Thesis submitted in partial fulfilment of the requirements for the joint academic degree of International Master of Science in Rural Development from Ghent University (Belgium), Agrocampus Ouest (France), Humboldt University of Berlin (Germany), Slovak University of Agriculture in Nitra (Slovakia) and University of Pisa (Italy) in collaboration with Wageningen University (The Netherlands), This thesis was elaborated and defended at Wageningen University within the framework of the European Erasmus Mundus Programme “Erasmus Mundus International Master of Science in Rural Development " (Course N° 2010-0114 – R 04- 018/001 Certification This is an unpublished M.Sc. thesis and is not prepared for further distribution. The author and the promoter give the permission to use this thesis for consultation and to copy parts of it for personal use. Every other use is subject to the copyright laws, more specifically the source must be extensively specified when using results from this thesis. Dr. Barbara van Mierlo Matteo Metta _____________________ ____________________ Thesis online access release I hereby authorize the IMRD secretariat to make this thesis available on line on the IMRD website Matteo Metta _____________________ From Good-Will to Good-Use: a Critical Analysis of the LEADER evaluation 2 Abstract For the first time in the 25 years of the history of LEADER, Local Action Groups implementing this programme have been called upon to undertake the evaluation exercise at the level of their Community-Led Local Development strategies - Article 33, point 1(f) of Regulation (EU) No 1303/2013. While the EU Regulation does not specify how to respond to this requirement, there is an extensive body of literature illustrating methods, guidelines, methodologies, studies, and recommendations with regard to how to evaluate the LEADER programme at the LAG level. Despite the good will of evaluation experts or alike to contribute to the evaluation of LEADER, the literature still has considerable gaps in relation to the final utilization of these propositions. In particular, the prevailing attitude of evaluation experts, is to overlook the importance of direct collaboration (from Latin: working together) with local stakeholders throughout the design and implementation of the LEADER evaluation. In addition, the current literature seems to separate the methodological discussions from the complex and interactive arena in which the evaluation of LEADER unfolds (e.g. political pressure on the future of LEADER, strong penetration of IT solutions in the Monitoring & Evaluation business, and conventional views about rural management). In this thesis, utilization refers to the responsiveness of the evaluation to the multiple situations and intentions of stakeholders in the multi-level governance structure, and in particular, at the LAG level. Based on the utilization-focused approach of Michael Quinn Patton (2008), this applied qualitative research endeavours to provide both theoretical and practical elements to reflect on the design, implementation and use of the evaluation of LEADER at the LAG level. It extends the good will of the evaluation approaches based primarily on the conceptual analysis of the programme, with the good use (or situational responsiveness) expressed through the interviews of 13 LAG managers across different EU areas. The results show the diversity in terms of intentions and situations varying across different LAGs in the EU, which adds new elements to the complexity of the LEADER evaluation. The complexity of LEADER evaluations stems from the mutual interplay between different domains (the organizational features of the LAG, the Community, the Local Development Strategy, and the larger LEADER system) and contextual forces. This suggests that, to turn the good will into good use, the LEADER evaluators should become facilitators. Facilitation, here is defined as the intermediation among different interconnected parts of the same system, making it necessary not only for guiding the LAGs to respond systematically to complexity, but also for supporting the negotiation and collaboration among different stakeholders concerning decisions about the evaluation criteria, values, purposes, methods, and process uses. Finally, this thesis concludes by raising two research questions to understand and address the utilization of the LEADER evaluation at the LAG level. Key words: Evaluation, LEADER, Utilization, Responsiveness, Complexity From Good-Will to Good-Use: a Critical Analysis of the LEADER evaluation 3 Acknowledgment I am personally grateful with all the LAG managers that have contributed to the realization of this work. Many thanks are due to Johanna, Ph.D. student who thought me that being a student is a long life experience that can be lived together with other life experiences, the nature, the agricultural farming, the alpacas, and the family. Many thanks go to Myles, Blanca, Anne-Charlotte, Vincenzo, and all my colleagues that have supported my studies with their insights, expertise, and generosities. I would like to extend my warmest thanks to my supervisor Barbara van Mierlo for having made me thinking always through myself, for her constructive suggestions, for having supported and encouraged me during the difficult moments, and for having realized my little aspiration to work with her since my previous MSc thesis at University of Foggia. I would like also to thank all the other friends who have given their positive energy and support for realizing this work: Frode, Kirsten, Flora, Jordan, Katka, and the Prof. Dirk Roep. I could not forget to thank also my previous professors from the University of Foggia (Italy) for having given me the basic human and scientific foundations for studying and researching. Finally, I would like to thank most heartily my Family and Maria for giving to me all the energy and the inspirations to study in Belgium, Germany, Slovakia, and the Netherlands and finally graduating as International Master in Rural Development. From Good-Will to Good-Use: a Critical Analysis of the LEADER evaluation 4 Table of contents 1. INTRODUCTION .................................................................................................................................... 8 2. FROM “GOOD WILL” TO “GOOD USE” OF THE LEADER EVALUATION ................................................... 10 3. PROBLEM STATEMENT ........................................................................................................................ 15 4. AIM, OBJECTIVE, AND RESEARCH QUESTIONS ..................................................................................... 17 5. METHODOLOGY ................................................................................................................................. 20 5.1 CONCEPTUALIZING THE PROGRAMME INTERVENTION ....................................................................................... 20 5.1.1. Conceptualizing the theory of change .......................................................................................... 21 5.1.2. Conceptualizing the elements promoting the different scale of changes ..................................... 22 5.1.3. Approaches used for designing the theory of change ................................................................... 22 5.1.4. Typologies of supported activities ................................................................................................ 23 5.2. FOCUSING ON THE UTILIZATION OF THE EVALUATION ....................................................................................... 23 5.3. RESEARCH METHODS ................................................................................................................................ 27 5.3.1. Procedures .................................................................................................................................... 27 5.3.2 Sampling and data ......................................................................................................................... 28 5.3.3 Data Analysis ................................................................................................................................. 30 5.4. LIMITS OF THE STUDY ................................................................................................................................. 30 5.4.1 Theoretical limits ........................................................................................................................... 30 5.4.2 Methodological limits .................................................................................................................... 31 6. CONCEPTUALIZING THE LEADER PROGRAMME .................................................................................. 34 6.1. THE COMPLEXITY OF THE LEADER ‘S THEORY OF CHANGE ................................................................................ 34 6.2. THE LEADER ELEMENTS PROMOTING DIFFERENT SCALE OF CHANGES ................................................................ 36 6.3 THE LEADER APPROACH ............................................................................................................................ 38 6.4. TYPOLOGIES OF ACTIVITIES ......................................................................................................................... 42 7. THE LEADER EVALUATION AT LAG LEVEL: THE