Blue Ventures Conservation Level 2 Annex, Omnibus Business Centre, 39-41 North Road, London N7 9DP Tel: +44 (0) 20 7697 8598 Fax: +44 (0) 800 066 4032 [email protected] www.blueventures.org Registered charity #1098893
Terms of Reference: Evaluation of Blue Ventures project “Improving the profitability and ecological sustainability of octopus fisheries in southwest Madagascar”
1.1. Background information Blue Ventures develops transformative approaches for nurturing and sustaining locally led marine conservation. We work in places where the ocean is vital to local cultures and economies, and we are committed to protecting marine biodiversity in ways that benefit coastal people. More information about Blue Ventures’ mission and our approach to marine conservation can be found on our website.
Blue Ventures has been working with Vezo communities in the Atsimo Andrefana region, SW Madagascar, since 2003, providing support to villagelevel management associations (including Velondriake, Beandriake, Soariake and Fimihara).
Temporary fishery closures are a powerful management tool that can demonstrate the economic benefits of fisheries management rapidly, to fishers and seafood buyers alike. By producing tangible benefits to coastal communities, this approach can build robust support for broader marine management initiatives. In SW Madagascar, where fishing accounts for more than 80% of income for around 90% of households, temporary fisheries closures have focussed on the octopus fishery, one of the most economically significant fisheries in the region (BarnesMauthe et al 2013).
Importantly, these closures have provided the catalyst for encouraging community engagement in ambitious marine conservation efforts, including the creation of permanent marine reserves, now established at a number of sites by communities in parallel with temporary fishery closures within locally managed marine areas (LMMAs).
1.2. Project description The current DFID funded project focuses on the traditional fishing communities of southwest Madagascar. In particular the focus is on women within these communities.
Octopus fishing is the principal and for many women the sole source of income, with 98% of women surveyed naming octopus as their main source of revenue (Epps 2006) for many households in these communities. It is a particularly important commodity for girls, women and the elderly, because they are able to fish octopus on reef flats by foot and using simple spears.
In southern Madagascar, where community characteristics typify those of deep poverty, women have traditionally played a secondary role in decisionmaking. Their status is subordinate to men, and their inferior status has many negative consequences for their education, health, nutrition and life prospects. Poverty has forced men to migrate away to work for long periods, leaving many households headed by women (Metcalf 2007, Cripps 2009).
The current initiative builds on BV’s work communityled management of the octopus fishery that began in the region in 2003, as well as work by the Madagascar government’s Projet d'Appui aux Communauté des Pêcheurs de Toliara (PACP) project, which aimed to decrease poverty amongst coastal communities of southwest Madagascar.
The project has aimed to increase the sustainability and profitability of this crucially important fishery, increasing income for women and food security within Vezo communities. By empowering communities, and in particular women, to undertake resource management, additional lasting impacts on the lives of the beneficiaries are expected to include increased gender equity and community participation in resource management and decision making, as well as increased environmental awareness and support for fisheries management and conservation actions.
1.3. Purpose of evaluation The project evaluation is also requested by the DFID fund manager (GPAF). The independent final evaluation reports submitted to Blue Ventures will be further submitted to the fund manager and used to inform their understanding of the grantee’s performance at the project level. The report will in turn be used to inform the evaluation manager’s assessment of performance at the GPAF fund level. The outcomes of the project evaluation are also expected to be useful to both Blue Ventures and other partners in explaining the impacts of the project and developing further support for community led management of octopus fisheries in Madagascar.
The independent final evaluation report needs to be a substantial document that (a) answers all the elements of the Terms of Reference (ToR); (b) provides findings and
conclusions that are based on robust and transparent evidence; and (c) where necessary supplements the grantee’s own data with independent research.
1.4. Key objectives of evaluation 1.4.1. To independently verify (and supplement where necessary) Blue Ventures’ record of achievement as reported through their Annual Reports and as defined in the project logframe; 1.4.2. To assess the extent to which the project was good value for money, which includes considering: ● How well the project met its objectives; ● How well the project applied value for money principles of effectiveness, economy, efficiency in relation to delivery of its outcome; ● What has happened because of DFID funding that wouldn’t have otherwise happened; and ● How well the project aligns with DFID’s goals of supporting the delivery of the MDGs.
1.5. Verification of reporting The first task of the final evaluation is to verify grantee achievement. The record of achievement will be presented in past Annual Reports and progress against the project logframe. This exercise could include verifying information that was collected by the grantee for reporting purposes and possibly supplementing this data with additional information collected through primary and secondary research.
Verifying the results from the project logframe will begin to capture what the project has achieved. However, there will be other activities and results that occur outside of the logframe that may require examination in order to respond to the different evaluation questions. Verifying reporting will also necessarily include a review of the data and systems that were used to populate results.
1.6. Assessment of Value for Money The final evaluation should assess the extent to which the delivery and results of the project are good value for money. Value for money can be defined in different ways, but at minimum the evaluation report should include an assessment against:
● How well the project applied value for money principles of effectiveness, economy, efficiency in relation to delivery of its outcome;
● What has happened because of DFID funding that wouldn’t have otherwise happened.
1.7. Evaluation questions 1.7.1. Relevance ● To what extent did the grantee contribute to the MDGs, specifically offtrack MDGs? specifically to this project: ○ MDG1 incidence of extreme poverty, reducing the number people living below the poverty line ○ MDG3 Gender equality integrating women in fisheries management decision making ○ MDG7 Ensuring environmental sustainability, contributing to community led conservation action ● To what extent did the project target and reach the poor and marginalised? ● To what extent did the project mainstream gender equality in the design and delivery of activities (and or other relevant excluded groups)? ● How well did the project respond to the needs of target beneficiaries, including how these needs evolved over time?
1.7.2. Effectiveness ● To what extent are the results that are reported a fair and accurate record of achievement? ○ Do the original indicators chosen effectively assess the impact of the project? ○ How have the context/conditions changed during the project changed and are outputs/outcomes still accurate/fair? ○ Have the inputs of the projects had the desired impact? e.g. has the development of women’s (ATM) groups actually changed women’s outlook on management? ● To what extent has the project delivered results that are value for money? To include but not limited to: ○ How well the project applied value for money principles of effectiveness, economy, and efficiency ○ What has happened because of DFID funding that wouldn’t have otherwise happened; and ● What challenges has the project faced and how has the project been adapted or found solutions to these challenges?
● What are the key drivers and barriers affecting the delivery of results for the project?
1.7.3. Efficiency ● To what extent did the grantee deliver results on time and on budget against agreed plans? ● To what extent did the project understand cost drivers and manage these in relation to performance requirements?
1.7.4. Sustainability ● To what extent has the project leveraged additional resources (financial and inkind) from other sources? ● What effect has this had on the scale, delivery or sustainability of activities? ● To what extent is there evidence that the benefits delivered by the project will be sustained after the project ends?
1.7.5. Impact ● To what extent and how has the project built the capacity of civil society? Specifically to this project: ○ The community association’s management abilities ○ The management capabilities of the regional platform (CGP) ○ Women’s capacity to engage in management/decision making ○ Improved skill sets (literacy and numeracy) ● How many people are engaged in octopus fisheries management that weren’t before? ● How many people benefit from octopus fisheries management that didn’t before the project? ● To what extent and how has the project affected people in ways that were not originally intended?
2. Evaluation Methods
2.1. Approach proposed
The proposed method for the independent impact evaluation will be a mixed methods approach combining qualitative assessments and information with quantitative data collected during the project. This approach will allow triangulation of the evidence
provided allowing synthesis between the approaches, double checking of results, and a deeper and more robust evaluation.
2.2. Materials available for review The evaluator will be provided access to all qualitative and quantitative data collected during the lifetime of the project, as well as additional information that is deemed of interest.
2.3. Proposed additional research To ensure the evaluation takes into account the local context in which the project is occurring, it is proposed that the evaluator/evaluation team conduct a short mission to the project site. During this mission the evaluator(s) will be able to conduct any additional qualitative information collection and learn more about the impacts of the project by speaking with community members and project staff. Note. Any surveys to be undertaken should be approved by the project manager prior to field work to ensure questions are culturally or socially sensitive.
3. Contractual and Reporting Arrangements
3.1. Qualifications/skills required
● An evaluation specialist with a minimum of seven years’ experience in programme/project evaluation in an international development context; ● Experience of resultsbased monitoring and evaluation; ● Ability to design and plan the evaluation approaches and research methodologies, including quantitative and qualitative research methods; ● Relevant knowledge and experience working in the small scale fisheries sector; ● Ability to manage a largescale and complex evaluation and research process, including interpreting baseline data and conducting a final evaluation; ● Ability to design, manage and implement primary research in potentially challenging project environments. This may include the design of surveys, indepth interviews, focus group and other research; ● Design and manage data and information systems capable of handling large datasets for monitoring and evaluation purposes; ● Appropriate country knowledge/experience including language proficiency to conduct the research required or that resources be made available (e.g. translator or social gatekeeper) to enable the research to proceed smoothly.
3.2. Management arrangements The consultant will be required to have an initial meeting with Blue Ventures Conservation Programmes Manager and Monitoring and Evaluation Manager to fully develop the evaluation plan. The consultant will be expected to provide regular updates, by email and/ or skype, to these managers and to coordinate field plans with project staff, in particular the project manager.
3.3. Deliverables The consultant will produce an initial summary powerpoint presentation with key emerging findings. Following this the consultant will then produce a written report documenting the findings of the evaluation.
This report shall include an assessment of the degree to which the project has satisfied the original objectives and key performance indicators, and recommendation for Blue Ventures and partners for what can be done better in the next phase of the project. All deliverables must be submitted in English.
3.4. Provisional timeframe
Date Action
21st September 2015 Technical offers submitted see details below
25th September 2015 Shortlist interviews
2nd October 2015 Selection of consultant
12th October 2015 Preparation for evaluation, including agreeing with BV on approach/methodology/interview format and schedules
19th26th October 2015 Start of evaluation in field
By end of field evaluation Submission of initial powerpoint presentation summarising key emerging findings
8th January 2016 Submission of draft evaluation report to Blue Ventures
25th January 2016 Submission and presentation of final report to Blue Ventures (potentially in person at BV’s office in Toliara, Madagascar or London, UK)
Technical and financial offers A technical offer shall be submitted to Blue Ventures in English.
Technical offers must not exceed 10 pages in length and must contain the following: ● address and contact details of evaluation team ● summary of qualifications and experiences ● proposed approach and methodologies ● detailed financial offer ● CVs for the consultants as annexes
Applicants should send a the technical offer to Monitoring and Evaluation Coordinator, email: [email protected]. Offers must in the same email be sent as copy to [email protected].
Please include the reference “Octopus project evaluation” in the subject line.