Evaluation SNV

New strategy, new results?

Client: Ministry of Foreign Affairs

ECORYS Nederland BV SEOR BV

Willem Cornelissen

Maaike Oosterbaan

Rotterdam, 13 March 2006

SEOR BV ECORYS Nederland BV P.O. Box 1738 P.O. Box 4175 3000 DR Rotterdam 3006 AD Rotterdam The Netherlands Watermanweg 44

3067 GG Rotterdam T +31 (0)10 408 2220 The Netherlands F +31 (0)10 408 9650

www.seor.nl T +31 (0)10 453 88 00

F +31 (0)10 453 07 68

E [email protected] W www.ecorys.com Registration no. 24316726

ECORYS Macro & Sector Policies T +31 (0)10 453 87 53 F +31 (0)10 452 36 60

DS/AN13968

Table of contents

Preface 7

Summary 9

List of abbreviations and acronyms 19

1 Introduction 21

2 SNV 27 2.1 Description of the changes 27 2.2 Present situation 29 2.3 Current activities in three countries 30

3 Changes in structure and working methods at corporate level 35 3.1 Introduction 35 3.2 Development of the transformation components 35 3.2.1 Organisation of activities 36 3.2.2 Organisation of people 39 3.2.3 Organisation structure and processes 42 3.2.4 Organisation of funding 46 3.2.5 The role of SNV in the field 48 3.2.6 The role of SNV as knowledge organisation 50 3.2.7 Increase of efficiency 52 3.3 Assessment of the realisation of the transformation components 54

4 The field level implementation of capacity building 59 4.1 SNV intervention logic 59 4.2 Input 61 4.2.1 Client selection 62 4.2.2 Practice areas 70 4.2.3 Classification of inputs 73 4.2.4 Advisory staff and their competency 74 4.2.5 Assessment 76 4.3 Output 78 4.3.1 Assessment 85 4.4 Effectiveness (outcome and results) 86 4.4.1 Methodology used by SNV 86 4.4.2 Outcome: Capacities strengthened 88 4.4.3 Sustainability of capacities 90

DS/AN13968 4.4.4 Results: strengthened capacities lead to improved performance by clients 92 4.4.5 Assessment 95 4.5 Impact 95 4.5.1 Methodology used by SNV 95 4.5.2 Impact: improved performance by clients leads to reduced poverty and better governance 99 4.5.3 Assessment 100

5 Efficiency and learning capacity 103 5.1 Efficiency 103 5.2 Effectiveness and organisation 105 5.3 Learning Capacity and Practice areas 106 5.4 Assessment 111

6 Conclusions and points of attention for a next phase 115 6.1 Conclusions by topic 115 6.2 Points of attention for the next phase 120

Annex A List of contacts and documentation 123

Annex B Capacity Development Services 141

Preface

This report contains the findings and conclusions of the evaluation of SNV (Netherlands Development Organisation). SNV, since late 2001 an independent organisation, receives core funding from the Ministry of Foreign Affairs. The purpose of the evaluation was to assess the progress in the transformation process as agreed upon between the Ministry of Foreign Affairs and SNV, to contribute to the accountability of the use of the funds and to report on the results achieved in the field.

The evaluation assessed the realisation of the transformation process of SNV from an organisation that delivered technical assistance through projects to an organisation that supplies advisory services to meso level organisations. Furthermore, the effectiveness, efficiency of activities and learning capacity of the organisation were assessed. The evaluation covered the period 2002-2005.

The evaluation was carried out during the period November 2005 to January 2006 and had to cope with a very tight time schedule. The evaluation included a study in the Netherlands, a survey among all SNV offices in the field and field visits to Bolivia, Kenya and Mali. The team consisted of consultants from ECORYS Netherlands BV and SEOR BV of the Erasmus University Rotterdam. The composition of the team is presented below.

Willem Cornelissen Team leader Maaike Oosterbaan Project coordinator, Team Netherlands John Zevenbergen Team Netherlands Sarah Sijses Responsible for survey SNV portfolios Agnes Janszen Responsible for efficiency analysis Loet Mennes Senior advisor Karen Bakhuisen Field visit Kenya Kees van der Zanden Field visit Mali Graham Alders Local consultant Kenya Brahima Fomba Local consultant Mali Rafael Rojas L. Local consultant Bolivia

The team would like to thank all SNV staff involved in the evaluation for providing information to the evaluation team and for the well prepared field visits in Bolivia, Kenya and Mali. Without this support from SNV, the evaluation could not have been carried out in such a short period. It should also be mentioned that the response to the survey to all SNV field offices was overwhelming.

The team is also grateful for the cooperation of the many persons interviewed (see Annex A) in the Netherlands and in the three countries visited.

Evaluation SNV 7

Summary

Introduction The Netherlands Ministry of Foreign Affairs has assigned the evaluation of the implementation of the Subsidy Agreement between the Ministry and the Netherlands Development Organisation SNV over the period late 2002-2005 to ECORYS Netherlands in cooperation with SEOR of the Erasmus University Rotterdam. After approval of the methodological inception report on 28 th November 2005, the evaluative research took place during the month December 2005, and analysis and reporting during January 2006. The research has entailed a desk-study and interviews, a survey among all SNV portfolios 1 worldwide and field visits to the Santa Cruz portfolio in Bolivia, the North Rift portfolio in Kenya and the Koulikoro portfolio in Mali.

The Terms of Reference formulated the following objective for the evaluation:

To which extent is SNV capable to strengthen and promote the local capacity of meso- level organisations in developing countries? The strengthening of local capacity is defined as the process of capacity building (of an organisation) to reduce poverty and improve good governance. The evaluation has to focus on both approach and organisation of SNV and the achieved results, as well as the coherence between the two.

The Terms of Reference formulated three main questions: 1. In which manner has SNV realized changes in its structure and working methods since becoming autonomous? 2. Did the advisory services of SNV contribute to capacity building and did this contribute to sustainable poverty reduction and good governance? 3. How efficient and effective are the advisory services of SNV and what is the learning capacity of SNV?

Since 2002, the relation between SNV and the Ministry of Foreign Affairs has been settled in a Subsidy Agreement that provides funding to SNV for an amount of € 500 million over a period of five years. This represents well over 90 percent of all SNV costs. SNV has acquired a separate position in the Dutch development co-operation as a foundation which is almost fully dependent on funding, but is autonomous for its operations, namely offering advisory services free of charge in more than 30 countries.

To SNV the period under review was characterised by a multi-dimensional change process comprising:

1 SNV operates at sub-national level and operates from sub-national offices. The total of advisory activities managed by and implemented through such an office is called a ‘portfolio’. Worldwide there are 100-110 portfolio teams active.

Evaluation SNV 9 • Disentanglement from the Ministry of Foreign Affairs (2001) and transformation into an autonomous foundation; • A new intervention strategy (2001) leading to new services and activities; • These new services required both a change in the required capacities of staff and a change in the organisational structure of SNV.

The objective and research questions mentioned above imply that this evaluation neither analysed the full range of the relations between the Ministry of Foreign Affairs and SNV, nor the policy implications thereof, such as the complementarities of SNV services vis-à- vis other cooperation activities either realized by or subsidized by the Ministry of Foreign Affairs.

The evaluation has been mainly based on the statistical and financial information provided by SNV. This information has not been checked on veracity. The findings of three field mission to Bolivia, Kenya and Mali are used as case studies and cannot be hold representative for the entire organisation. However, the survey among portfolios covered almost the entire population of SNV portfolios and can be taken as statistically representative. Due to the restrictions in research time available, the effectiveness of the advisory services in the field has been studied on the basis of secondary information (existing studies and information) and interviews.

Findings The findings are structured along the lines of the three main research questions as mentioned in the Terms of Reference.

1. In which ways has SNV realized changes in its structure and working methods since becoming autonomous?

The transformation has been evaluated on the basis of the ten characteristics formulated in the Subsidy Agreement. SNV aims at complying with these characteristics by the end of the period of the Subsidy Agreement (2006). These changes can be distinguished in those in working methods and those in structure:

a. changes in working methods SNV has completed a multi-faceted change process transforming its services from the supply of long-term technical assistance by individual experts to host organisations at micro level to the provision of advisory services in teams to (mainly) organisations operating at meso level. In 2004, between 64 percent (Asia) and 88 percent (East and Southern Africa) of the clients were categorised as ‘meso level clients’.

In its working methods, SNV has completed the change process towards the provision of advisory and consultancy services (by teams). In the case of still existing narrowly marked out services, the client can still be attended by a single advisor, but not on a long- term assignment anymore. Usually, in those cases, the advisory service consists of precise and technical advice. SNV has largely discontinued the implementation and administration of projects on behalf of third parties. SNV has disentangled its service delivery from SNV administered financial streams to the client, but does collaborate actively with third parties in order to obtain additional (financial) support for its clients. Such financial support is also aimed at ensuring the sustainability of SNV strengthened

10 Evaluation SNV capacities. In some cases SNV provides advisory services at a more generic level that are funded by others (such arrangements exist with the central government in Mali, but are rather exceptional). b. changes in structure The changes in the structure (‘structure follows strategy’) of SNV formed part of the strategy towards a new kind of service delivery. The organisational structure both at headquarters and in the field was changed in 2002. At headquarters a Board of Directors was introduced (in replacement of a single director). In 2002 ten support units were established at Head Office and in December 2005 these were merged into three units (Strategy, Finance and HR). In 2002, the function of regional manager was abolished and instead clusters of countries were established. Each country was managed by a Country Director. In 2003, the role of cluster coordinator was introduced. Country directors operated on a rotating basis as coordinator. In 2004, the concept of a cluster was replaced by region and in 2005 permanent Regional Directors were installed.

The field offices became country teams, responsible for the country programme and management (as a ‘full business unit’ up to late 2005). Since 2002-2003, the programme officers in the sub national teams became responsible for portfolios in their geographical area (portfolio coordinators). New structural changes (“Optimising Support”) were set in motion late 2005.

The SNV advisors have been either (re-)selected (2002) or newly recruited based on both subject matter professional qualities and advisory capabilities. Thereto, SNV applies a modern recruitment process and sophisticated toolkits for career management, promotion, coaching and training of its staff.

SNV’s structure has not taken a definite shape yet and is in a permanent process of change. While reorganisations did enable a new way of service delivery, the structure thereto has not been consolidated yet.

2. Did the advisory services of SNV contribute to capacity building and did this contribute to sustainable poverty reduction and good governance? a. selection process The improved client selection process and the provision of services to interconnected clients have increased the probability that clients produce the desired impact (in terms of poverty reduction and good governance) as compared to the previous situation in which services were provided to individual micro level host organisations.

SNV identifies its clients through a selection process that balances demand with pro- active identification. Since 2003, the pro-active identification gained more weight. As a first step in such a process the selection of countries is critical, but SNV lacks explicit criteria for selecting countries. At country and sub-national (portfolio) level, SNV has developed a well elaborated client selection process, framed by a limited number of pre- determined themes (eight so-called ‘practice areas’, 2005). The selection process has enabled SNV to provide services to clients that are mutually interlinked. Service provision is aimed at the simultaneous strengthening of various actors in a constellation (a

Evaluation SNV 11 logic and coherent group of mutually interconnected actors), whereby not only the capacities of the individual organisations are being strengthened, but also the constellation in its complex totality. SNV has shown not to shy away whenever these relationships may imply conflict, such as in the case of conflicts on land and land use in Santa Cruz, Bolivia. It has been observed that SNV delivers well-perceived services that are framed by strategies at various levels, and underpinned by context analysis, by needs assessments and agreed upon with the client.

b. capacity strengthening In conceptual terms four dimensions compose capacity building: human capabilities, institutional organisation, networking and external systems. In its new strategy SNV has focused more on institutional development and networking. ‘Connecting people’ became the new catchword. Of a more recent date is the increased attention to the systems dimension.

SNV has defined what should be understood by “the strengthened capacities of the client organisation(s)”, being (a) the capacity to change, combined with self esteem and capacity to propose; (b) the development of knowledge (based on human resource development; monitoring and learning capacity) and (c) the capacity to sustain results (managerial and financial capacities). In order to achieve those strengthened capacities, SNV applies a participatory approach in all its assignment, based on communication, coordination, knowledge sharing and a problem solving attitude.

The client satisfaction reviews provide information on the extent to which the clients are satisfied with the services provided by SNV; the end-of-year reviews focus more on the progress made in relation to the Advisory Plan. In all three portfolios visited, SNV clients gave consistently high assessments of the quality of SNV services. While SNV counts with standardised systems to measure the output and outcome level, in the three portfolios visited a different mixture of instruments was applied to measure effectiveness. The most prevailing ones are the case studies and the self-evaluation. The local case studies assessed varied in scope, approach and quality and did not provide an overall assessment of the effectiveness of the advisory services to a constellation of clients or a specific practice area. Case studies may provide an assessment of the effectiveness in a single client.

In absence of a more generic analysis on effectiveness, in the case of the selected activities of the three portfolios visited, there were ample indications of the contributions made by SNV to the capacities of the various organisations. NGOs and civil society organisations have been strengthened in their institutional organisation and have been enabled to implement programmes related to training, technical assistance, advocacy, etc. In the public sector, strengthening of capacities can take divergent forms, such as the capacity to resolve problems by negotiation and information in stead of by the use of power; the capacity to embark on participatory processes (in planning, in monitoring, in accountability) in stead of imposing decisions; the capacity to fight corruption as a joint public-civilian action; the capacity to embark on multi-actor actions; the capacity to accept and appreciate cultural diversity. Human capacities have been strengthened by training and exposure.

12 Evaluation SNV In its present (2005) use, the Advisory Plan is mainly an instrument that formalizes the advisory relation between SNV and the client .The Advisory Plan is a potentially powerful instrument as base for subsequent result measurement, but under-utilised to that end. At corporate level, there have been remarkably few external evaluations (also due to a remarkable small budget made available to that end), while the number of result and impact evaluations at country level remained limited as well. SNV has promoted self- evaluation by clients and case studies for result measurement. However, self-evaluation does not provide any insight on the effects of actors working in constellation and may show flaws with respect to the required objectivity of the analysis. The case studies reviewed from three countries were rather descriptive and site specific. Although these instruments may contribute in getting evidence on results, they are not sufficient and not academically solid enough to generate evidence on results and impact. The thematic/regional evaluations are still in development. SNV has announced that in 2006 more attention will be paid to develop the evaluation instrument. c. expected impact As component of the methodology used, the evaluation mission had challenged three portfolios to produce evidence on impact. Even in absence of solid research, as mentioned above, the three portfolios produced strong indications that the services delivered do contribute to the envisaged results and that clients do contribute to produce the expected impact. Advisors of the three portfolios were able to describe both results and impact, but well under-pinned evidence was lacking.

The expected impact of the SNV provided advisory services is that they contribute to the overarching objectives: poverty reduction and good governance. For the analysis of contributions to poverty reduction, the poverty concept was disaggregated into its three dimensions: the income dimension, the access to social services dimension and the ‘voice’ (political) dimension. From the three portfolios visited it could be derived that the practice area ‘market access to the poor’ does contribute to the income dimension of poverty, but for other practice areas there might be a potential (such as in collaborative forest management) but exists less evidence for contributions to income. Some do contribute to either the ‘access to services’ and ‘voice’ dimensions of poverty, mostly when it concerns activities in relation to local governance. Much stronger are the indications that SNV advisory services do contribute significantly to good governance.

3. How efficient and effective are the advisory services of SNV and what is the learning capacity of SNV? a. efficiency SNV is a decentralised development organisation with permanent offices and staff at sub- national level. SNV’s rather unique features make that efficiency benchmarks from other organisations are hardly meaningful. In absence of meaningful benchmarks, SNV has determined its indicators for efficiency mainly in two ways: 1. on the basis of the ratio between the costs for its ‘primary process’ and its ‘support costs’. The ‘primary process’ costs include, next to the remuneration of advisors, the related costs made at field offices and head quarters and costs made in relation to the development of partnerships. All other costs are ‘support costs’.

Evaluation SNV 13 2. on the basis of direct and indirect costs. Direct costs include all costs of the field offices, a part of the costs of the head quarters and costs made in relation to partnerships. Indirect costs are the costs of the headquarters which could not be allocated directly.

Using the first calculation method, the support costs decreased from 45 percent in 2002 to 43 percent in 2005. The second calculation method shows that de indirect costs decreased from 18 percent in 2002 to 9 percent in 2005.

SNV has the aim to reduce the support costs (calculated on the basis of the first method) to 35 percent. This has to be achieved by a reorganisation (the 2005 Optimising Support strategy) and by revision of the share of no advisory staff in field offices (53 percent of all field level costs in 2004).

b. effectiveness If a client has been selected and there is sufficient common ground to establish a relation, SNV formalizes this through an Agreement (Memorandum of Understanding). The Agreement refers in general terms to the responsibilities of both SNV and client. The Advisory Plan is the principal instrument used to define SNV’s advisory services to a particular client. In principle, the Advisory Plan could indicate the expected results. SNV’s instruments in place to measure whether the strengthened capacities actually lead to improved performance of the clients (results) and whether this performance contributes to poverty reduction and/or good governance are less developed. SNV applies a mixture of instruments, such as self-evaluation by clients, case studies, and selected external evaluations.

At portfolio level there is sufficient hard evidence (case studies) that capacities of clients have been strengthened. Of the four dimensions distinguished, SNV has achieved most in networking and institutional organisation and to a lesser extent on the systems dimension. Although direct training and south-south capacity building is also geared to the human capability dimension, this has not been the main thrust of SNV’s activities.

c. learning capacity In the past, SNV advisors were mainly focused on their own advisory work with clients. In September 2003, a Knowledge Management Unit was established. Regional knowledge networks were initiated around thematic areas of expertise (practice areas) with the aim to motivate people to interact, to offer and ask for help, to share experiences and to develop new ideas. For day-to-day communication, virtual collaboration platforms (D-groups) have been established. As a result of the networks, cross border exchanges and collaboration between advisors have become common practice in SNV. Subject matter specialists were contracted to revitalise the knowledge development. The regional cooperation in knowledge networks also contributed to south-south knowledge sharing, although the three field visits and the world-wide survey suggest that in practice many interpretations exist about what should be understood by south-south contacts.

SNV is a learning organisation that invests in its main asset, its human resources. SNV disseminates its knowledge through internet facilities, a Journal and (inter-)national presentations and it ‘learns’ as an organisation by incorporating external knowledge and

14 Evaluation SNV experience. The interchange with private sector actors and academic knowledge could be further enhanced.

Conclusions

Changes in working methods and structure With respect to the transformation of the ten characteristics as contemplated in the Subsidy Agreement, it can be concluded that SNV has complied to seven out of the ten transformation characteristics. It has achieved approximately 80 percent of all the targets set in 2002.

Three characteristics are partly fulfilled but should be further developed. These characteristics are result measurement, the strategy of third party funding and the efficiency of the organisation.

SNV has made an enormous effort to realise a drastic transformation from an organisation focused on technical assistance and implementation of projects to an organisation that supplies advisory services aimed at strengthening the capacity of client organisations. This transformation process has been realised, but the corresponding structure has not been consolidated yet, considering the fact that a new restructuring process has started late 2005.

Contribution to capacity building and impact At portfolio level there is sufficient hard evidence (case studies) that capacities of clients have been strengthened. SNV has achieved most in networking and institutional organisation and to a lesser extent on the systems dimension.

In the Advisory Plan, the agreement with the client on inputs, outputs, outcomes and expected impact should be laid down. However, the quality of Advisory Plans reviewed in the three portfolios visited varied strongly. These plans do not provide a basis yet for the measurement of the extent to which the planned capacity building is realised. Client satisfaction forms show that the clients are satisfied with the services provided, but this is hardly an indication of progress made with respect to the realisation of the capacity strengthening objectives.

In order to maximise the probability that impact can be achieved, a proper client selection system is required. SNV has developed a well-perceived client selection system based on context analyses at various levels of aggregation (national, regional and client). Although no standard measurement system has been developed yet, in the three portfolios visited, SNV advisors were able to produce strong indications that the clients do have an impact on poverty reduction and good governance. The available information shows that the impact on good governance is more pronounced than the impact on poverty reduction (in particular the income dimension thereof).

Evaluation SNV 15 Efficiency Beyond doubt SNV staff is hard working, committed and identified with the organisation. Over the period under review, SNV faced no financial constraints and experienced a period of under-expenditure. SNV has decided to discontinue the implementation and administration of projects on behalf of third parties. In consequence, SNV is not competing on the ‘market’ for capacity building assignments. Due to its financing structure, the organisation lacks financial stimuli for efficiency.

SNV has set its own efficiency indicators and has set its own targets for realising a significant reduction in support costs. In the report ‘Optimizing Support (June 2005) it is mentioned that SNV has the aim to reduce its support costs to 35 percent of the total budget.

Learning capacity SNV has put much effort in the development of its own organisation and staff. It can be concluded that SNV is moving towards becoming an international recognised knowledge organisation. In that respect, SNV’s participatory and ‘connecting people’ approach has more appeal for international exposure than subject matter knowledge in practice areas, where (in some cases) specialised knowledge centres do exist already (water, forestry).

Points of attention for the next phase

The evaluation team recommends to pay attention to the following points for the coming period of eventual subsidy provision by the Ministry of Foreign Affairs: • to provide practical contents to the concepts “complementarity’ and ‘consistency’ between the policies and strategies of the Ministry of Foreign Affairs and those of SNV. This should comprise the establishment of sound criteria for country selection; • a review of the desirability and necessity of maintaining the existing waiver facility to embassies; • SNV’s ‘anchoring’ of activities in the PRSPs and MDGs imply both a driving force towards new subject matter areas (health and education) and the provision of services at a macro-level. The desirability and feasibility of moving into (partially) new practice areas merits careful consideration. The provision of services at macro level leads to a different kind of clients and a different context of other service delivery mechanisms. The implications to SNV in terms of manpower, delivery of services free of charge to well-endowed clients and the opportunities to make use of ‘outsourcing’ to existing capacity building delivers at macro level (delegated co- operation) merits more analysis and study as being done so far in the three countries visited; • to include an exit strategy for SNV support services at various levels (by assignment, by client [in the advisory plan], but also by practice area in a portfolio area and eventually at the portfolio level itself); • to develop effectiveness and impact measurement instruments; • to explore the opportunities (but also the threats) for inclusion of financial stimuli in both the supply of core funding to SNV and the allocation by SNV to its various country and portfolio offices.

16 Evaluation SNV As an implication of the changes produced in the structure and way of funding of SNV, both implicit and explicit accountability checks have changed over time. These changes implied that major decisions (for example on country selection or kind of activities) are made by SNV only. Although the relationship between SNV and the Ministry of Foreign Affairs was no subject of the present evaluation, the extension of the present working relationship may merit re-assessment.

Evaluation SNV 17

List of abbreviations and acronyms

ANICT Agence Nationale d'Investissement des Collectivités Territoriales BOD Board of Directors CCC Centre de Conseil Communal CD Country Director CDS Capacity Development Services CEADES Colectivo de Estudios Aplicados al Desarrollo Social CEJIS Centro de Estudios Jurídicos e Investigación Social CT Collectivité territoriale Danida Danish International Development Agency DED Deutsche Entwicklungsdienst ECDPM European Centre for Development Policy Management EUN Contractant Européen GTZ German Technical Cooperation HCN Home Country Nationals Helvétas Association Suisse pour la Coopération Internationale INRA Instituto Nacional de Reforma Agraria KIT Koninklijk Instituut voor de Tropen (Royal Tropical Institute) MATCL Ministère de l’Administration Territoriale et des Collectivités Locales MDGs Millennium Development Goals MoU Memorandum of Understanding MTR Medium Term Review NGO Non-Governmental Organisation OD Organisational Development OICH Organización Indígena Chiquitana OSA Organisational Self-Assessment PRSP Poverty Reduction Strategy (Paper) RNE Royal Netherlands Embassy SNV Netherlands Development Organisation TCN Third Country Nationals ToR Terms of Reference UNDP United Nations Development Programme VNG Vereniging van Nederlandse Gemeenten WTO World Tourism Organisation

Evaluation SNV 19

1 Introduction

Background of the evaluation The Ministry of Foreign Affairs has a twofold purpose for evaluating SNV (Netherlands Development Organisation). Firstly, the evaluation will be used for the accountability of the introduced changes in the structure and working methods, as well as for the realised results. Secondly, the evaluation should contribute to the policy development concerning the role of SNV in the fields of poverty reduction and good governance.

Because the Subsidy Agreement between the Ministry of Foreign Affairs and SNV will expire at the end of 2006, the evaluation will also be used for the discussions between the two organisations for an eventual extension of the Agreement.

The Netherlands Ministry of Foreign Affairs has assigned the evaluation of SNV to ECORYS Netherlands in cooperation with SEOR of the Erasmus University Rotterdam. The proposal for the assignment was discussed with the Steering Committee on 7 November 2005. The evaluation team submitted an inception report on 25 November 2005. This inception report was discussed with and approved by the Steering Committee on 28 November 2005. The implementation of the evaluation took place during the months December 2005 and January 2006.

Objective of the evaluation The evaluation has the following objective (translated): To which extent is SNV capable to strengthen and to promote the local capacity of meso- level organisations in developing countries? The strengthening of local capacity is defined as the process of capacity building (of an organisation) to reduce poverty and improve good governance. The evaluation has to focus on both the approach and organisation of SNV and the achieved results as well as the coherence between the two.

The approach and methodology used in the evaluation has been based on the three research questions mentioned in the Terms of Reference (ToR) and were referred to in the proposal and inception report (translated): 1. In which manner has SNV realized changes in its structure and working methods since becoming autonomous? 2. Did the advisory services of SNV contribute to capacity building and did this contribute to sustainable poverty reduction and good governance? 3. How efficient and effective are the advisory services of SNV and what is the learning capacity of SNV?

The evaluation covers the period 2002-2005.

Evaluation SNV 21 Approach of the evaluation The approach and methodology used in the evaluation has been based on the three research questions mentioned above. The time constraint for the research part (approximately one month) was of importance for the choice of the research methodology.

For the answering of the first research question a (desk) study was conducted in the Netherlands. Use was made of database information and documentation of SNV, including annual reports, strategy documents, documents on capacity development and practice area evaluations. This was complemented with several informative working sessions with SNV’s Board of Directors and policy officers on issues like the organisation’s change in strategy, the typology of its advisory services and its methods for results measurement.

Interviews were also held with the Ministry of Foreign Affairs (DSI/MY), two ex- directors of SNV and partners of SNV which are active in similar fields of capacity building, namely: UNDP, ECDPM, KIT, Cordaid, VNG and Agriterra. The study of research question one was structured along the lines of the ten characteristics mentioned in the Subsidy Agreement to which SNV was expected to comply with by the end of the Agreement period (2006). The ten characteristics are listed in section 0.

For addressing the second research question, a survey among all approximately 100 SNV portfolios worldwide was conducted. The survey among all SNV portfolios worldwide was meant to obtain quantifiable information on a number of the ten characteristics and as a frame for the interpretation of information derived from the field visits. The survey consisted of two parts: one part comprised questions to be answered at portfolio level, concerning input, output, outcome/effectiveness and impact. And a second comprised of questions on third party funding and client selection, to be answered at country level. The survey was completed by 75 percent of all SNV portfolio coordinators and 82 percent of all SNV country directors 2. The results at portfolio level were first aggregated to country level and then to regional level. In the report, results from the survey are presented interchangeably with results from the field studies and data from the SNV database.

A second component aimed at addressing the second research question was realisation of and three field studies of portfolios in Bolivia, Kenya and Mali. The portfolios were selected based on criteria like the regions where SNV is most active: West & Central Africa (Mali), East & South Africa (Kenya) and Latin America (Bolivia), while the practice areas applied was the criterion for portfolio selection within a country. “Responsive and Accountable Local Government” was selected as a cross-cutting practice area over the three case-study countries (Mali, Kenya and Bolivia). In addition, for Bolivia “Collaborative Forest Management” was selected as specific practice area, for Kenya “Market Access to the Poor” and for Mali “Collaborative Dry-land Management”. The selection of portfolios that were studied in the three countries was based on the

2 Here it should be taken into account that some of the portfolio coordinators were already interviewed during the field studies and therefore were ‘exempted’ from completing the survey. Also, in some countries the recruitment of a country coordinator was still ongoing and therefore the survey was either not answered or answered by the regional director.

22 Evaluation SNV representation of the mentioned practice areas in terms of number of advisory days, in the respective portfolios. The portfolios selected were Santa Cruz in Bolivia, North Rift in Kenya and Koulikoro in Mali. Within each of the selected portfolios, a number of clients attended in the corresponding practice areas were pre-selected (after a first argued deduction, the final selection of clients was made in coordination with the Portfilio Coordinators. In the selection, the type (Government, NGO or Private ) and level (micro- meso-macro) of the client organisation was taken into account, as well as the client’s linkages with other SNV development partners in the same country / region.

Each field visit comprised: • A one-day visit to the corresponding SNV Regional Office; • A two to three day visit to the Country portfolio, the Royal Netherlands Embassy and national level clients (for example, ministries); • A field visit of four to five days to the pre-selected portfolio office and field visits to the pre-selected clients; • File research and documentation on the pre-selected clients.

During the field studies the following instruments were used: • Semi-structured interviews (based on standardised checklists elaborated for interviews with Clients, Country Directors, Regional Directors and Royal Netherlands Embassies and Clients); • Check on application of SNV standard instruments on a the pre-selected advisory services (4-5 per portfolio); • Study of secondary material (documentation, data base). For evaluation studies a standardised registration form has been used. The standardised checklists ensured the consistency between the three field studies that were carried out by different consultants.

The evaluation of research question two was based on SNV’s own interpretation of the intervention logic in terms of input, output, outcome, effect/result and impact (see figure 4.1). Since in a very short field visit of about one week, no serious primary data for analysis can be collected, the approach on effectiveness and impact of the advisory services supplied was to challenge SNV to produce the necessary evidence on effectiveness and impact of the pre-selected activities. This challenge was communicated to the SNV portfolios in advance.

For the third research question the findings from the desk study, the survey and the field studies were brought together.

For details of the approach, methodology, indicators and use of selection criteria for the field visits, reference is made to the inception report (November 25, 2005).

The advantage of the research method chosen was that the evaluation could be realised within the serious time constraint given in the Terms of Reference. The disadvantages are that –apart from the survey data- no statistical relevance can be claimed on the results of this study. Three field visits comprising only a limited number of clients provide illustrative material and impressions, but no ‘hard’ evidence.

Evaluation SNV 23 Almost all statistical, as well as financial data used in this report (other than those derived from the survey) have been supplied by SNV. The veracity of these data has not been subject to control or audit by the evaluation team.

Subsidy Agreement The relation between SNV and the Ministry of Foreign Affairs was formalised with the signing of the Subsidy Agreement in 2001. In this agreement, ten characteristics were laid down to which the SNV should comply with in 2006. These characteristics can be perceived as ‘expected results’ and are formulated as follows:

1. In 2006 SNV is a specialised supplier of advisory services to meso-level organisations, government and non-government, in developing countries. It provides services on a demand-driven basis to hundreds of organisations in some 30 countries. These clients see SNV as a respective and effective partner, who provides relevant advice, geared to the needs and conditions of these organisations, and that makes its international network and expertise adequately available; 2. SNV has developed a distinctive advisory practice for the above, that a) combines thematic, organisational and process expertise, b) has a clear methodological basis, and c) reflects strategic visions on the strengthening of the meso-level; 3. SNV staff operates as advisors. They are supported methodologically and are coached and educated individually. Their performance is continuously monitored and assessed; 4. SNV has integrated in its management relevant methodologies to come as close as possible to effective result measurement in a demand oriented setting. A combination of up-to-date instruments and methods is applied to that end; 5. SNV has investigated, developed and tested a financing strategy. A clear choice in strategy in this area has been made; 6. On the basis of its involvement in activities at the meso-level, SNV plays an acknowledged role in the discussion in its areas of specialisation, nationally as well as internationally. This is done through participation in (inter)national debates, provision of documented analysis and experience and thorough development of methodologies; 7. SNV has effective alliances for accomplishing its strategy, especially with regard to collaboration with local parties and the contribution to international knowledge networks; 8. SNV is developing itself towards a knowledge centre of national and international significance. It makes available, disseminates and publishes knowledge and is externally recognised for that; 9. SNV operates in regional contexts and promotes in this way the development of knowledge and a South-South exchange; 10. SNV has realised a significant gain in efficiency, through a) servicing more client/partners, b) strategically better chosen clients/partners, c) the stronger knowledge function and the stronger regional out-reach of its organisational set-up.

The realisation of these ten characteristics is dealt with on corporate level in chapter tree and on field level in chapter four.

24 Evaluation SNV In the field studies to three portfolios in Kenya, Mali and Bolivia over 15 activities have been studied in more detail. For clarity of the presentation of this report, the number of different examples has been kept to a minimum.

Evaluation SNV 25

2 SNV

2.1 Description of the changes

SNV has a long history. It was founded in 1965 as the Foundation for Dutch Volunteers (Stichting Nederlandse Vrijwilligers), an organisation sending out Dutch professionals to developing countries. Gradually, SNV changed into a development organisation supplying technical assistance on capacity building through well experienced experts. SNV has established offices in about 30 countries all over the world.

During the period under review (2002-2005) SNV practically changed into a complete new organisation. SNV experienced a multi-dimensional change process comprising: • disentanglement from the Ministry of Foreign Affairs in 2001; • a change in strategy (2001) leading to changes in mission and activities; • changes in required staff capacities; • changes in the organisational structure.

Becoming autonomous Prior to 2001 SNV formed part of the Ministry of Foreign Affairs (hereafter referred to as the Ministry). The management at the headquarters were staff members from the Ministry and formed part of the pool of human resources and subject to the rotating system in force in the Ministry. During the second half of the 90s the first ideas were raised that SNV needed its own identity and a more transparent organisational structure. Several potential constructions were studied, e.g. SNV becoming an agency of the Ministry or operating as an independent organisation.

Late 2001, the new status of SNV was formalised and as from 2002 onwards, SNV would operate as an independent foundation, disentangled from the Ministry. The relation between SNV and the Ministry was laid down in the Subsidy Agreement for the period of five years (2002-2006). It was agreed upon that SNV would be financed by the Ministry for an amount of about € 500 million in five years.

Change in mission and activities Not only SNV’s status changed in 2002, but its operations changed as well. The new mission was formulated as follows: “SNV delivers capacity strengthening support to meso-level organisations and local capacity builders with the aim of improving governance and reducing poverty”. At a later moment in time, and in an effort to re- phrase the mission statement in terms of Amartya Sen’s freedom concept, this was amended to: “SNV is dedicated to a society where all people enjoy the freedom to pursue their own sustainable development. Our advisors contribute to this by strengthening the capacities of local organisations”.

Evaluation SNV 27

Over time, SNV has seen a development of its “delivery mechanisms”. Roughly three different types can be distinguished 3: • A fairly unspecified support (the old SNV “input”) by a full time advisor , over time complemented with small financial support in order to make the technical assistance more effective (1965-2001); • During the 1990s the project implementation by teams became an important delivery mechanism. A permanent team of several advisors under guidance of a project co- ordinator held complex assignments, usually in the frame of a donor-supported project or programme; • In the late 1990s a third delivery mechanism developed: the flexible advisory teams . Teams or networks of advisors dealt on a flexible basis with a portfolio of clients in a certain thematic field.

The ‘hands-on’ full time advisory services, as well as the project management type of services changed into an advisory practice. Late 2001 the following capacity building support was envisaged (SNV 2001:13): • Linking (networks of) actors (government, NGO, private), themes, and levels (micro, meso, macro); • Institutional development and organisational strengthening; • Technical and thematic expertise; • Process facilitation and change management; • Networking, platform and alliance building; • Advocacy and lobby (policy influencing); • Financial brokering and support.

The core of the activities are no longer focused on the implementation of activities at grassroots level, but were defined in terms of the provision of advisory services at meso- level to organisations in developing countries. SNV demarcated the provision of such services to three areas of competence: local governance, local private sector development and natural resource management. In the course of time, these areas of competence have been further specified into the following eight so-called ‘practice areas’: • Responsive and accountable local government; • Market access for the poor; • Collaborative forest management; • Sustainable tourism; • Collaborative water management; • Dry land management; • Renewable energy – biogas; • Gender (as cross cutting area).

The above mentioned practice areas have been formalised as such in 2005. A ninth practice was defined during 2004 and in some areas introduced in 2005: “joint social service delivery and local economic development’. However, in 2005 at corporate level it

3 SNV. Making sense of the diversity, December 2001.

28 Evaluation SNV was decided that ‘joint action’ was no practice area and that ‘basic service delivery’ should be considered as one of the two large ‘impact areas’ 4 (see also section 3.2.3).

What the changes in the activities imply in practice, is illustrated by the activities visited in the context of the present evaluation in section 2.3.

Change in human resources The reorientation in activities and working method required also a change in expertise and skills of the SNV professional staff. The technical assistance experts in the field, used to work in a single project for several years, had to be become advisors with the task to supply advisory services to several meso level organisations simultaneously. The consequence of this change was that existing staff was trained to become advisors, new staff was recruited and some persons left the organisation voluntarily or involuntarily. The number of professional staff reduced in significantly in 2002, but increased swiftly afterwards.

Change in organisational structure The organisational structure both at headquarters and in the field was changed in 2002. At the headquarters a Board of Directors (BoD) was introduced. The BoD has the final responsibility for the entire organisation. The regional units with their ‘region directors’ at headquarters were discontinued. All support bureaus were reorganised in a single strategic unit and an operational support unit.

The field offices changed into country teams, with a country director, programme officers and professionals. The country teams are responsible for the country programme and management. Later, the programme officers became responsible for portfolios in a specific geographical area and became ‘portfolio coordinators’. The country director became the ‘full business manager’ of the package of portfolios, with a strong supportive function. . The Portfolio Coordinators (PCs) are the managers of the portfolio team and ensure coordination, monitoring and review of activities, budget control, adherence to SNV gender objective, quality, etc. The Country Director, PCs and the Country Administrator form together the country management team (CMT).

Since the regional coordination had been removed from headquarters, cluster teams with a cluster co-ordinator in the field (rotating country directors) were introduced with the aim to organise horizontal co-operation (within a region of several countries). Later, the clusters were formalised into regional offices with a permanent regional director in one of the countries pertaining to the region (e.g. Kenya for Eastern and Southern Africa, Burkina Faso for Western Africa).

2.2 Present situation

By the end of 2005, one more step in the row of reorganisations was taken (SNV, Optimising Support, 2005). This reorganisation is still in its early implementation phase

4 Source: Comments SNV on draft final report, 3 rd February, 2006.

Evaluation SNV 29 and has the aim to rearrange the support services at head quarters and in the field and to reduce the overhead costs. Support services currently available at country level will be clustered at the regional offices in order to expand the ‘span of control’.

The planning is that by the end of 2006, the transformation of SNV will be completed, that means that SNV will have consolidated a new status, new activities and working methods, counting with staff providing advisory services and counting with an efficient organisational structure. The realisation of the transformation will be measured on the basis of the requirements and agreements laid down in the Subsidy Agreement (see chapter 1).

The Subsidy Agreement will expire in 2006. The discussions between SNV and the Ministry about an extension or new agreement have already started. The SNV is still fully dependent from the Ministry with respect to the funding of the organisation.

2.3 Current activities in three countries

Section 2.1 described in a generic way the various changes that have led to the current provision of advisory services. This section presents a sample of clients visited in the frame of the present evaluation as illustration of the kind of services and clients attended by SNV.

1. SNV Bolivia – Santa Cruz portfolio SNV has a history in Bolivia that goes back to the period of the decennia-long dictatorships in the country (1969). During the period that the Netherlands had no formal representation in Bolivia, SNV acted as the “Dutch presence” by supporting organisations that aimed at re-establishing democracy in the country. After the first civil government was installed (1982), SNV assisted in reconstructing the fragile democracy by aiming at specific poor areas and vulnerable groups (target group approach). During the 1990s, SNV focused on assisting incipient organisational forms initiated by grassroots groups, both in relation to good –local- governance and economic activities. Amongst these organisations figured the Economic Grassroots Organisations (producer associations). SNV provided direct support to small rural municipalities that had gained importance with the Law on Popular Participation (1994) and the Municipal Administration Law (1998).

SNV’s experience with the indigenous population in the Santa Cruz department dates back to 1986 and started with natural resource management with the Chiquitana people in Lomerío, through the NGO APCOB en the Union of indigenous people CICOL. Since then, SNV has worked uninterrupted with different NGOs and representative organisations of the various indigenous people in the region (Chiquitanía and Guarayos) as well as with the ‘colonos’ (the new inhabitants from other regions).

The country multi-annual plan 2002-2004 stated that the SNV interventions in Santa Cruz focus on two lines of action: local governance and economic development, whereby the management of natural resources would be treated a cross-cutting perspective. In addition it states that: “SNV uses gender and intercultural aspects as quality criteria”. The SNV

30 Evaluation SNV policy change of 2001 / 2002 towards support to meso-level organisations brought little change in the Santa Cruz programme, since most activities were already implemented at that level. What did change in 2002 was a further focus on social grassroots organisations at the expense of the non governmental organisations, while also the departmental public institutions became more explicitly subject to SNV assistance.

Two geographical areas within the Santa Cruz Department have been prioritised: Guarayos and the Chiquitanía. In the Chiquitanía SNV has provided advisory services to three unions of indigeneous people and has strengthened their umbrella organisation Organisation of Indigenous people of the Chiquitanía, OICH. The organisations are also demanding parties in the territorial conflict around the communal territory Monte Verde. The Monte Verde conflict is of paramount proportions and has attracted international attention. On territorial conflict, SNV has pursued the approach of inter-connectedness: it supports a NGO that provides legal advice and assistance to the target organisations, it provides assistance to the governance structure of the ‘mancomunidad de la Gran Chiquitanía’, it supports the municipal governments in the area, as well as the Prefecture in Santa Cruz (in particular on conflict management). The support to the Prefecture has a ‘systems dimension’ at the macro level, being support to the Vice-ministry for Lands in La Paz in the elaboration of a Land use policy.

2. SNV Kenya – North Rift Portfolio SNV has been operating in Kenya since 1967. SNV-Kenya aims to deliver tailor made, high quality advisory services that facilitate poverty reduction and good-governance. Support is primarily targeted at organisations that form the link between national and local actors, such as government departments, local authorities, civil society organisations and associations of private sector actors. Traditionally, SNV Kenya works mainly in the arid and semi-arid areas, with a strong focus on pastoralist societies. Geographical areas of operation are organised as follows: • South Rift Portfolio covering Kajiado and Narok Districts, and Nairobi; • North Rift Portfolio covering Baringo, Keiyo, Koibatek, Marakwet, Turkana, Trans Nzoia, and West Pokot Districts; • Northern Kenya Portfolio covering Isiolo, Laikipia, Marsabit, Moyale and Samburu Districts.

The changing profile and structure of North Rift Portfolio is to be considered an integral part of the change process within the organisation. From 2002 onwards the Turkana and Kerio clusters focused on restructuring the organisation from project implementation to capacity development by advisory services. In 2003, the client portfolio concept was introduced, and Capacity Development Services (see Annex B) were introduced. In 2004 the Turkana and Kerio clusters were merged into the North Rift Portfolio and practice areas were introduced. The portfolio team witnessed a substantial growth in advisors: from 4 at the beginning of the portfolio to 14 at the end of the year 2005. A start was made with cross-border activities. In continuation, three clients within the North Rift portfolio are being described briefly:

Honey Care Honey Care is a company with a strong social focus (“social enterprise”) aimed at reducing poverty through bringing efficient market practices to the honey sector. Its

Evaluation SNV 31 objectives matched well those of SNV’s practice area “marketing access for the poor”. It works with individual farmers to ensure that they deliver quality honey for a fair market price is being paid. It has about 30% of market share in Kenya and claims with justification that it has raised the quality of honey available and reduced the volume of imported honey. Honeycare was brought into the production chain by SNV as a result of existing services provided to the NGO PADO (see next) that encouraged bee keeping among its member organisations.

Pastoral Area Development Organisation (PADO) PADO is a relatively new NGO that serves rural and pastoral peoples in two Divisions (total pop. 85,000) in West Pokot. It is composed of 11 ‘member groups’ with 250 individual members. The main focus is on gaining access to markets in honey, livestock, handicrafts and most recently, tourism. It has been supported by SNV since 2000 (having been ‘inherited’ from earlier SNV programmes) with Capacity Development Services like developing an organisational structure, developing a constitution, a baseline survey, strategic planning, elaboration of an annual work plan and a quarterly monitoring plan. It has also facilitated market access through linking up with organisations higher in the market chain, like Honey Care. PADO is considered to have the potential to become a capacity building institution itself, able to take on the role of SNV advisers in its existing area of operation.

Cheptebo Conference and Training Development Centre Cheptebo is a community based organisation, focusing on extension/training in the area of farming (dairy production, tree planting/nursery). The organisation applied in 2004 for SNV support. SNV provided capacity development services in the following areas: support to an envisioning process (vision/mission/goals, revised constitution and policies, integration of gender), development of a five year strategic plan (resulting in an Implementation/Strategic Development Plan), financial management and resource mobilisation. The outcome of the SNV support was innovation and skills (financial and human resource management), improved focus of the organisation (mission/vision), improved ability to link and communicate with the beneficiaries/community and improved financial accountability.

3. SNV Mali – Koulikoro Portfolio SNV has celebrated its 25 anniversary in Mali in 2004. Also in the region of Koulikoro, SNV has an experience that dates back to 1978 (health sector, natural resource management and rural development). SNV has assisted technical and administrative government institutions as well as village associations.

The government of Mali, within its national decentralisation policy has put in place a system to assist the Collectivités Territoriales (CT), called the “ Programme National d’Appui aux CT”. This programme comprises two main components : • A system of financial assistance, managed by the Agence Nationale d’Investissement des CT (ANICT); • A system of technical assistance to the municipalities throughout a network of so-called Conseils Communaux de Cercle ( CCC).

32 Evaluation SNV SNV is designated, by the government of Mali, as a so-called ‘ opérateur CCC’ in three cercles (districts) in the region of Koulikoro. These CCC’s assist the municipalities in various functions. SNV has identified teams of consultants to provide the services; these consultants are not considered SNV-personnel. CCC is considered a specific implementation mechanism and not so much a separate client of SNV. Nevertheless, the function of SNV as ‘ operateur CCC’ is of importance to the operations of SNVs in terms of spin-off effects to other CTs and in the identification of clients. Similar activities exist with the Collectivités Territoriales Dioila.

Of an entire different nature is SNV Koulikoro’s support to a union of sand and gravel operators (Union des Exploitants de Sable et de gravier (UESGK) Koulikoro) that are being strengthened in their operational management in terms of internal organisation and commercial activities. A comparable programme exists with the forest operators of Dioila (Structures Rurales de Gestion de Bois (SRGB), Dioila). .

Evaluation SNV 33

3 Changes in structure and working methods at corporate level

3.1 Introduction

The first research question (ToR) to be tackled in the evaluation is: “In which ways has SNV realized changes in its structure and working methods since becoming autonomous?”

Whether SNV realised changes in its structure and working methods is addressed on the base of the ten characteristics formulated in the Subsidy Agreement (see chapter 1). SNV aims at complying with these characteristics by the end of the period (2006) which indicates the finalisation of the transformation of the organisation.

For the purpose of this evaluation, these ten characteristics have been regrouped into the following seven transformation components: a. the organisation of activities (characteristic 1 and 2); b. the organisation of people (characteristic 3); c. the organisation structure and processes (characteristic 4); d. the funding strategy (characteristic 5); e. the role of SNV in the field (characteristics 6 and 7); f. the role of SNV as knowledge organisation (characteristics 8 and 9); g. increase of efficiency (characteristic 10). The detailed description of indicators for this research question is given the inception report of November 25, 2005.

The characteristics 1,2,3,4 en 6 are relevant for both the organisation at corporate level as well as the organisation in the field. Therefore, these characteristics are both dealt with in this chapter and in the following one that presents the findings in the field.

3.2 Development of the transformation components

This section discusses the evolution in the seven components of the transformation process. A detailed list of indicators used was presented in the inception report. The findings in this section are based on documentation, interviews, survey data and additional information obtained from SNV.

Evaluation SNV 35 3.2.1 Organisation of activities

The organisation of activities is related to the transformation of SNV from an organisation which supplies technical assistance on project basis to an organisation which supplies advisory services to clients at meso level. Indicators used for this component are: 5 • the development in number and type of advisory days; • the development of the client portfolio; • the demand-driven approach.

Development in advisory services Table 3.1 shows that the number of advisors and advisory days increased significantly over the years 2003-2005. Due to the reorganisation in 2002, the number of professional staff in the field reduced from 702 in 2002 to 517 in 2003. During that period field advisory staff had to change focus from the provision of individual technical assistance to the supply of advisory services (mainly in teams). Some experts left SNV voluntarily, others were asked to leave since their profile did not match the new requirements. In 2003, a time registration system was introduced from which the number of direct advisory days could be derived. The time registration system enabled to calculate the ‘productivity rate’ of the advisors i.e. the number of days of an advisor spent on direct advisory services. The objective was to achieve a productivity rate of 65 percent. Assuming that a year has 220 working days (excl. weekends, holidays, etc.), the average ‘productivity’ rate in 2004 was 61 percent which clearly indicates the focus on the ‘production’ of advisory days in that year. The estimated productivity rates for 2005 and 2006 are 57 percent and 66 percent respectively.

Table 3.1: Development of number of advisors and advisory days

2002 2003 2004 2005 2006 Number of advisors* 702 517 569 950 1,030 (est) Number of direct advisory days* - 35,848 76,937 120,000 150,000 (est) (est) Average no. of advisory days - 69 135 126 146 per advisor Number of countries* 26 26 26 33 33 * These numbers are provided by SNV.

Figures 3.1 and 3.2 show the variance by region and the development over time. In 2005, all regions achieved a ‘productivity’ of at least 65 percent, except Albania. In correspondence, the relative share of time spent to support services declined.

5 A detailed overview of indicators used is given in the inception report.

36 Evaluation SNV Figure 3.1 Share of total available working days spent on direct advisory services by region

Share of total available working days spent on direct advisory services

80% 70% 60% 50% 2003 40% 2004 30% 2005 20% 10%

Share of working days (%) days working Share of 0% Asia W&C E&S Africa Latin Albania Africa America SNV regions

Figure 3.2 Share of total available working days spent on support processes by region

Share of total available working days spent on support processes

50% 40% 30% 2003 2004 20% 2005 10% 0%

Share of working days (%) Asia W&C E&S Latin Albania Africa Africa America SNV regions

Notes: 1. Asia: Data from the new portfolios in Cambodia have been excluded. 2. West and Central Africa: There were no data available for Guinea Bissau for 2005, so data for Guinea Bissau have also been excluded for 2003 and 2004. Data for the two new SNV portfolios in DR Congo have been excluded. 3. Albania: The sum total of working days spent on various activities is 113 percent in one of the portfolios, according to the respective PC this is due to overtime. Source data 2003, 2004: SNV Database (SUM~Data 2004-12); region average is based on country level data. Source data 2005: Survey ECORYS & SEOR, 2005. Regional average is based on country level averages, which are in turn based on portfolio data. The data are estimates since the time registration for 2005 had not yet been completed at the time the questionnaire was completed.

Evaluation SNV 37 The advisory services are focused on the strengthening of the capacity of the clients. Institutional capacity building is composed by the following components 6: • enhancement of human capabilities (human resource development); • organisational development; • networking; • systems development (the external context or legal aspects, regulations and finance).

SNV’s own concept of Capacity Development Services 7 (CDS) is not precisely the same, but roughly comprises the same elements: • Diagnosis and learning; • Organisational Development; • Network and partnership building; 8 • Influencing institutional change .

In the literature, usually funding is mentioned as an element of capacity building as well. Deliberately SNV does not include funding of organisations or activities, but advisors may assist the client in acquiring third party funding.

Client development The new strategic orientation also required a new orientation on the organisations to work with in the host countries. Until 2001, SNV worked predominantly with partner organisations in specific projects. The aim was to select strategic client organisations at meso level that were supposed to have a potential for producing an impact on good governance and poverty reduction. Initially, the client organisations were identified by both reassessing the organisations with which SNV had an existing project relation and by the selection of new organisations on the basis of a context analysis. Since more clients could be served per advisor, the number of clients could be increased in a short period of time.

Table 3.2 Development of clients

2002 2003 2004 2005 2006 Number of clients* - 917 1791 2,480 2,500 (est) Average no. of clients per advisor - 1.8 3.1 2.6 2.4 Average no. of advisory days per 55 54 50 60 client % clients at meso level* - 88% 82% - - * These numbers are provided by SNV. The year 2002 is a transition year.

Table 3.2 shows that the number of clients increased rapidly. The average number of clients per advisor was 3.1 in 2004. It is estimated that the average number of clients per advisor will be around 2.5 in 2005 and 2006. The number of advisory days per client varies widely. There are clients who request short narrowly defined assistance only, while

6 See for example: Ministry of Foreign Affairs, Policy and Operations Evaluation Department. Institutional Development. IOB- Evaluations no.284. March 2000. 7 Source: SNV presentation 15 th November 2005. 8 According to the theory on capacity building, an institutional relation is one between actors and does not necessarily requires a formalised structure. In the case the relation counts with a structure, this is called organisation.

38 Evaluation SNV others maintain long lasting working relations with SNV. The average number of advisory days per client is between 50 and 60. A large majority of the clients (82 percent in 2004) are clients at meso level.

The survey data also show a considerable increase of the number in clients in 2004 and 2005 (see Table 3.3).

Table 3.3 Number and increase of clients in 2004 and 2005 per region

Region Number and increase of clients in 2004 and 2005 No. of clients per Change 2003-2004 No. of clients per Change 2004-2005 Dec. 2004 a (%) a Dec. 2005 b (%) b Asia 227 80% 341 1 50% West & Central Africa 728 4 55% 1068 2 47% East & Southern Africa 462 95% 725 3 57% Latin America 282 76% 441 56% Albania 48 269% 87 81% Notes: 1: Including the new portfolio in Cambodia. 2: Data for the portfolio in Guinea Bissau have been excluded, both for ‘04 and ’05, due to lack of data. Data for the two new portfolios in Congo are included. 3: Excluding the new portfolio in Sudan. a: Source: SNV Database (SUM~Data 2004-12); region average is based on country level data. b: Source: Survey ECORYS-SEOR; region average is based on country level data (provided by CDs) or country level averages (provided by PCs) using the most recent data up to December 2005.

Demand driven approach The ideas about the need of a demand driven approach of SNV has changed since 2000. The demand driven approach was originally advocated because of the notion that developing organisations should focus on the demand and needs of the receiving partners with the aim to increase ownership over the activities and investments. The current thinking is that SNV should not only provide services on demand but that organisations have to be selected pro-actively given their potential for generating impact or their strategic relationship to other clients. This approach is made visible though the use of a context analysis for the selection of clients (see also chapter 4).

3.2.2 Organisation of people

The transformation of SNV from a project implementer to a supplier of advisory services required an adjustment in the organisation and qualities of the staff. Indicators used are the development of the staff employed (in numbers), the development in training and coaching and the impact of reorganisations.

Development in field staff The first priority of the transformation of staff was to teach people how to advise. Traditionally SNV advisors were accustomed to do everything by themselves and not asking others for help. It took time to change the attitude of persons and to make them client oriented advisors working in teams.

Evaluation SNV 39

SNV has a policy to recruit staff from the country or region in which its works or even from another continent. This staff is referred to as Home Country Nationals (HCN) and Third Country Nationals (TCN). In 2005, there were six HCNs and/or TCNs who hold a position of country or regional director. The salary conditions for European nationals and the HCNs or TCNs differ and is still subject of discussion.

Table 3.4 shows the percentage national advisors to total advisors. This percentage increased for all regions from 54 percent in 2002 to 63 percent in 2003 and to 68 percent in 2004. For some regions in Africa (Western and Eastern Africa), the percentage nationals was even more than 70 percent. An increase of the number of national reduces the costs, because salaries of national experts are lower (see also chapter 4).

Table 3.4 The division between European and national advisors

Region Year Total no. of No. of No. of national % national employees advisors advisors advisors Region Asia (incl. Albania) 2002 247 134 67 50% 2003 201 88 50 57% 2004 226 109 72 66%

Region East Africa 2002 214 123 67 54% 2003 225 85 53 62% 2004 220 105 76 72%

Region West Africa 2002 451 191 110 58% 2003 393 144 101 70% 2004 380 149 108 72%

Region Southern Africa 2002 132 85 44 52% 2003 146 64 39 61% 2004 141 68 45 66%

Region Latin America 2002 235 169 92 54% 2003 238 136 83 61% 2004 238 138 87 63%

Total regions (excl. 2002 1279 702 380 54% Head quarters) 2003 1203 517 326 63% 2004 1205 569 388 68%

Source: Annual reports 2002, 2003 and 2004.

40 Evaluation SNV Training and coaching In the autumn of 2004, a Learning Needs Analysis was carried out, which provided input for the development of a learning strategy. 9 As part of the learning strategy the budget for learning was increased to 8% of the salary costs for the years 2005 and 2006. Elements of the learning strategy include: • Advisory Practice Learning Programme and Virtual Campus; • Networks; • Career Management; • Coaching for teams and individuals; • Learn from and improve the quality of content production and Capacity Development Services; • Monitoring and evaluation.

The numbers of advisors that followed a training or were participant in a learning programme in the period 2001-2004 were as follows:

Table 3.5 Number of advisors trained

No. of Asia Latin West and East and Balkan Total advisors America Central Southern Africa Africa 2001 64 62 107 86 319 2002 22 ? 22 2003 89 16 80 ? 185 2004 83 26 41 43 24 217 Note 1: mid 2002 all regional learning initiatives were put on hold to start again around 2004. Note 2: the figures represent the formal training/learning initiatives; initiatives especially at country and team level are not recorded as formal trainings. Source: SNV, 2005

Examples of learning initiatives and trainings are: SNV Introductory Training Course, SNV Advisory Training Courses, Advisor skills training, Coaching, Community Development Course, English Course, Project evaluation, MBA, Agricultural process, Gender perspective to practice, Team building, Project and Office Management and Basic Management Training Skills.

In collaboration with UNDP, SNV has established an online Virtual Academy where staff is encouraged to undertake online courses to improve their skills (“SNV Campus”).

Reorganisations The transformation of SNV has been realised through the implementation of several reorganisations implemented in a row. The changes in the field were mainly focused on the transformation from technical assistance to advisory services. Existing staff had to be retrained. New staff was recruited. The consequence of the reorganisations was a significant decrease of personnel in 2003 due to a number of staff decided not to stay within the new SNV (some staff members at headquarters opted for maintaining their

9 SNV, The SNV Learning Strategy, A Learning Framework for Action, 2004.

Evaluation SNV 41 contractual relationship with the Ministry of Foreign Affairs). Of a number of staff expiring contracts were not extended. The decrease in numbers advisors in 2003 is shown in Table 3.1. The exact numbers of voluntary and involuntary resigns have not been registered.

Also at the head quarters, reorganisations were initiated. The organisational structure changed and the number of staff was brought down from 108 in 2001 to 79 positions in 2003 (Annual Report 2003). The management layer of regional directors at headquarters was discontinued and transferred to the field level (‘cluster coordinators’).

Recently (December 2005) another reorganisation was set in motion. This time it concerned the support services. The aim of this organisation is to improve the quality of the support services and to reduce the overhead costs. One of the changes concerns the concentration of support at regional level covering several countries instead of country level. Responsibilities of regional and country offices will be redefined.

3.2.3 Organisation structure and processes

To implement the transformation of SNV, changes were needed in both the structure and processes of the organisation. The characteristic pertaining to this component indicates that SNV has integrated in its management relevant methodologies to come as close as possible to effective result measurement in a demand oriented setting. The characteristic refers mainly to the monitoring and evaluation system and practices of SNV. However, the evaluation team also paid attention to other issues concerning the organisation structure and processes. 10

Organisational structure The reorganisation of structure and processes can be briefly summarised in several steps. The first step took place in 2002 and related to the reorganisation of the primary process in the field. A client oriented approach was introduced. In the field, advisory teams were formed, clients were reassessed and new clients were selected.

The second step concerned the reorganisation at headquarters. Regional units at headquarters were removed and country offices were made directly responsible to the Board of Directors (initially there was a single director, later a Board was appointed). The members of the Board of Directors are appointed by the Supervisory Board. With the aim not loosing a regional focus (supra national) clusters of countries were formed and country directors were rotating as cluster co-ordinator. The new steering mechanism is presented in the figure below.

10 See inception report, November 25, 2005.

42 Evaluation SNV Figure 3.3 Steering mechanism introduced in 2002

Board of directors

Strategic Operational Body Support

Field Director Country Country FieldCountry Director RCC Director Director Director

Shared Resource Local Support Centre Legend: Hierarchical Functional PO/ PO/ PO/ PO/ PO/ Horizontal Team Team Team Team Team Leader Leader Leader Leader Leader Rotating Cluster RCC Co-ordinator

P P P P P P Professionals

Source: SNV, Contours and Principles, Internal Organisation, 2001

The steering mechanism introduced in 2002 existed of a complex set of hierarchical, horizontal and functional steering lines. In the hierarchical lines (the arrows in Figure 3.3), the Board of Directors steers directly the support units at headquarters (strategic body and operational support) and the country directors in more than 30 countries. The country directors are directly responsible for the teams in the field.

The horizontal lines (the thick blue horizontal lines in Figure 3.3) are introduced to steer exchange, cooperation and learning between staff and teams. The functional steering (the dotted lines in Figure 3.3) are lines which are not hierarchical but supporting and facilitating.

Later on in the transformation process, the regional clusters were replaced by regional offices with a permanent regional director instead of a rotating cluster coordinator.

In 2003-2004, new structures and processes were introduced to develop the internal and external knowledge sharing. Network leaders were appointed which function as a specialist on a practice area and have to support the different portfolio teams (field teams) in the region (supra national). Mid 2003, a new planning and monitoring system was launched which included, amongst others, a time writing system and a client satisfaction measurement system.

In the new organisational structure (2005), the regional directors will become an important layer in the vertical steering line. There will be a direct steering by the Board of Directors to the regional directors on support services instead of to the country directors. Regional directors form also part of the Extended Board of Directors that discusses key strategic and policy issues three to five times per year.

Evaluation SNV 43 Figure 3.4 Steering mechanism, to be introduced in 2006 Steering mechanism SNV Head Office Field

BoD

F HR S RD RD RD RD RD

Ds Ds Ds Ds Ds

PC s PC s PC s PC s PC s

= BoD: 2x /month

= HO MT: 2x/month. Discussion of Head Office operational coordination issues

= Extended Board: 3-5/year. Discussion of key stategic and policy issues

5

Source: SNV, Optimizing Support, 2005

Development of the monitoring and evaluation system The monitoring of SNV is focussed on the operational management laid down in the management cycle, with a four-monthly reporting system. In 2003 the monitoring assessments of the Mid Term Reviews took place on country level, while for 2004 a regional basis was chosen. The assessment is made by the Planning, Monitoring and Finance Unit and the reports are presented to management. The planning and monitoring process is laid down in the Planning & Monitoring Guide (July 2003).

In 2002 and 2003, a methodology was developed for evaluations and measuring results of SNV interventions. Use was made of the ‘ripple model’ (see Figure 3.5; for a comment on the ripple model and the advisory plan see box 4.3) which is based on the following evaluation levels: • Output : the advisory services to clients: advice, knowledge brokering, advocacy; • Outcome : the strengthened capacity of the clients; • Effect : the change in performance of clients towards their beneficiaries; • Impact : the change in situation of beneficiaries/society in the fields of poverty reduction and good governance

SNV decided to focus first on the two inner circles of the ripple model, namely output and outcome, since –according to this model- the outer area ripples would be produced at a later moment in time and would be less ‘visible’ since the ripples would fade gradually and intermingle with other ripples.

44 Evaluation SNV

Figure 3.5 Ripple model

broader socio-economic context external changes in partner-client performance

capacity building process

SNV output

internal organisational changes of partner-client

Source: SNV, Planning & Monitoring Guide, 2003.

The new system implied that at the impact level, SNV would no longer produce country evaluations, but practice area evaluation in stead (3-4 per year). In 2002 country studies were still elaborated for Ethiopia, Nepal and Albania. In 2003 no country evaluation were conducted anymore, while in 2004 thematic studies were started, with a study of communal forestry in Latin America and a study on local governance in Western Africa. Late 2005 a study on local tourism was ongoing.

The SNV Annual Report 2004 raised the question whether “we are accountable for the services we deliver (how many organisations we supported, numbers of trainings, etc), or whether we are accountable for contributing to poverty reduction?” and calls this question ‘an ideological discussion’. It mentions: “We can be held accountable for our choice of clients, so we ensure we are working with those organisations that have the potential to make a difference and are representing the interests of the poor” and “through research and joint evaluations, we can make plausible claims as to how we contribute to poverty reduction. The research need not be continuous or every time we are involved, as this would disrupt the cost-benefits balance and further question the effectiveness of money spent”.

For the years 2004 and 2005, the evaluation strategy was based on the implementation of about two evaluations per year (no evaluations were executed in 2003). These evaluations had to include the following considerations: • equal attention had to be paid to the practice areas; • the evaluation should be representative for SNV; • the selection should include successful and less successful interventions; • there should be a potential multiplier effect; • the number of advisory days.

Evaluation SNV 45 In 2004 two evaluations took place about collaborative forest management in Latin America and about local governance in Western Africa (see chapter 5). Both evaluations were conducted by an external evaluation team with a mixed technical and evaluation background. According to SNV, the first evaluation appeared to be rather ambitious in scope and had a predominantly client focus. These lessons were taken into account for the preparations of the second evaluation which had a focus on strategic level and appeared to be more policy-action oriented.

For the evaluation of 2005 on the tourism sector in Asia, it was decided to focus more on impact on the basis of a selection of four more in-depth case/field studies. An evaluation on market access for the poor in West and East Africa is planned for 2006.

In 2005 a different perspective was introduced -as pilot- in the Latin American Region with the concept of ‘impact area’, meant to link SNV activities directly to the Millennium Development Goals and to national poverty reduction strategies. The ‘impact area’ makes use of regional and national macro indicators like income and employment.

Besides the evaluations planned and implemented by the head office, other result measurement instruments were introduced: • evaluation studies at local level (about 20 in 2004). These studies have a very wide variation in focus, scope and objective; 11 • a client satisfaction measurement system on the basis of regular interviews with the client; • case descriptions of interventions. See chapter 4 for the use of these instruments in practice.

The yearly budget for evaluations (about one/two per year) was about € 150.000. This budget is far below the ‘rule of thumb’ for public entities (0.5%- 1% of the total resource envelop) and far below the one used in the private sector (2%).

3.2.4 Organisation of funding

In the Subsidy Agreement it is stated that SNV should have investigated, developed and tested a financing strategy by the end of 2006. A clear choice in strategy in this area has been made in the sense that SNV came to the conclusion that opportunities for diversification are scarce and that core funding by the Ministry of Foreign Affairs is the financing modality of preference.

SNV did explore additional funding possibilities. In the beginning, even a target of mobilising € 5 million additional funding was set. This appeared not to be an easy task and the target was not reached. Additional funding possibilities were often related to technical assistance and/or programme funding within specific sectors of specific target groups. This programme funding did not fit the new strategy of SNV to supply advisory services to meso level organisations only, and disentangled from finance provision and/or

11 See KIT, Lessons learnt from a selected number of SNV evaluation reports, 2004.

46 Evaluation SNV the administration of specific projects. Therefore, a search for obtaining core funding was made.

However, to find core funding happened to be even more difficult due to the following reasons: • donors are reluctant to fund general capacity development services that do not form integral part of technical assistance programme’s with formulated objectives and expected results; • bilateral donors who are willing to consider core subsidy favour their own national organisations (DED-GTZ, Danida-MS); • the EU would only consider core funding if SNV would form a consortium with similar organisations from other EU countries.

Some additional resources had been mobilised through the up-scaling of proven SNV activities in certain sectors. An example is the development of bio-gas in Asia. This funding possibility will be explored further (for example in the water sector). In a few countries, special national circumstances allowed to obtain funding for general capacity building services, such as in Mali (see chapter 4).

Table 3.6 shows that third party funding is not a significant source for funding. In 2004, more use of third funding could be made and increased to more than one million euros. Remarkable is that especially in Western Africa the sources of other donors are relatively high in 2004.

Evaluation SNV 47 Table 3.6 Funding sources per region (euro x 1,000)

Region Year DGIS core Third party DGIS projects Other income 4 funding 1 Funding 2 – waiver 3 Region Asia (incl. Albania) 2002 7,081 502 0 1,433 2003 8,044 268 51 1,312 2004 9,046 253 0 591

Region East Africa 2002 6,386 0 0 214 2003 7,478 111 9 131 2004 7,942 60 8 166

Region West Africa 2002 13,795 0 0 424 2003 15,520 133 8 318 2004 15,077 643 197 464

Region Southern Africa 2002 9,807 0 0 743 2003 10,991 61 229 581 2004 10,742 42 347 127

Region Latin America 2002 11,875 0 0 729 2003 12,318 12 64 173 2004 11,780 136 23 83

Total 2002 48,944 0 0 3,542 2003 54,350 585 360 2,515 2004 54,587 1,134 575 1,431

Explanation of columns: 1 Core funding from the Ministry of Foreign affairs, expenditures per region and per year. 2 Funding from other organisations (e.g. EU, World Bank, bilateral donors, NGO’s). Note that the contents of the concept “third party funding” has shifted over time. In 2002 it consisted mainly of funding for project implementation and administration; in 2004 third party funding refers to the funding of SNV advisory services. 3 The Subsidy Agreement includes a waiver, which implies that Dutch Embassy can assign SNV projects for implementation. 4 Other income includes sales of assets (e.g. vehicles), administrative services and interest. Source: SNV administration

3.2.5 The role of SNV in the field

The indicators for the assessment of the role of SNV in the field (characteristic 6 and 7) are a recognisable role of SNV in national and international discussions and the initiative for effective alliances with other organisations.

Recognisable role of SNV Since 2004, more attention is paid to participation in discussions in the thematic specialisations of SNV. Examples include:

48 Evaluation SNV • Three SNV advisors with expertise in the practice area local government presented a paper on the experiences of SNV with decentralisation in Benin at the International Conference on Local Development in Washington (16-18 June 2004); • SNV advisors presented eight papers on local governance processes in East Africa at the AEGIS conference in London (29 June – 2 July 2005). The papers were the result of a learning exercise with workshops facilitated by Professor Patrick Chabal. In 2005, SNV organised five regional MDG conferences on “Localising the MDG agenda” which culminated in the international conference on “Mobilising Local Capacities for the Millennium Development Goals” in The Hague in September.

SNV has also become much more active in documenting knowledge for internal and external use. Examples include: • SNV developed 15 Reference guides on eight practice areas with reference to methodologies, knowledge institutes, people etc. The guides were originally meant for internal use, but some of them have become popular instruments outside SNV as well. However, this concept has become a bit outdated now with wide access to internet for (almost) all SNV portfolios. The guides were interesting as a reference, but many contained too much information of variable quality and now the idea is to prepare more ‘state of the art’ papers, written by lead persons of a practice area in joint collaboration with some external experts; • Since 2004, SNV advisors prepared 29 case studies that describe and analyse the advisory services, practice areas and its results. For the advisors it was new to start publishing about their experiences, since they did not always see themselves as ‘knowledge workers’; • Since 2005, SNV works together with ECDPM and UNDP to develop the periodical Capacity.org into a professional magazine on capacity development for a broad community of practitioners.

International organisations (e.g. UNDP and ECDPM) have started to see SNV as a partner in the area of capacity development respectively local governance.

Development in alliances SNV has initiated partnerships with international and national organisations which could strengthen the advisory services in capacity development, knowledge brokering and advocacy. Types of partners are: • partners to strengthen capacity development in thematic specialisations (practice areas): WTO, World Wide Fund for Nature (WWF), Fair-trade Labelling Organisation (FLO), World Business Council for Sustainable Development (WBCSD), Agriprofocus and UNDP; • partners to strengthen knowledge brokering: UNDP, Agriprofocus, ECDPM, Impact Alliance and ISS; • partners to strengthen advocacy: GCAP (Global Call to Action against Poverty) and Eurostep.

In October 2004, SNV and UNDP signed a Memorandum of Understanding (MoU) with the aim to strengthen and further deepen the overall collaboration between the two organisations. In September 2005, a letter of intent was signed by the two organisations to

Evaluation SNV 49 cooperate on an integrated package of services for supporting the development of MDG- based poverty reduction strategies.

In 2004, SNV signed a MoU with the World Tourism Organisation (WTO) formalizing the already existing working relationship with regard to the Sustainable Tourism Eliminating Poverty programme with activities in Cameroon, Ethiopia and Vietnam.

SNV became one of the founders of Agri-ProFocus, a cooperative network established to support producer organisations in developing countries. The Dutch government, representatives of the financial and business sectors, research institutions, development organisations and agricultural organisations are involved in this network.

SNV engaged in a partnership with the Fair-trade Labelling Organisation (FLO) to contribute to the access to markets on fair trade terms for produces and workers in developing countries and to the strengthening of producer organisations.

3.2.6 The role of SNV as knowledge organisation

Due to the decentralised approach and direct individual involvement in projects prior to 2001, knowledge was not widely spread and shared within or outside the organisation. The new focus of SNV comprises the intention to play an active and recognisable role as a knowledge organisation (characteristics 8 and 9). Indicators used for this transformation component are the internal and external knowledge development and the development of South-South knowledge exchange.

Knowledge development (internal) An important step was set during 2001-2002 with the change towards working in teams instead of stationing advisors in the offices of host-organisations. This laid the cornerstone of a SNV community conducive for sharing and developing knowledge. In September 2003, a Knowledge Management Unit was established.

In mid 2004, regional knowledge networks were initiated around thematic areas of expertise (practice areas) with the aim to motivate people to interact, to offer and ask for help, to share experiences and to develop new ideas. For each of the networks a network leader had been appointed to initiate and facilitate the network. Network leaders are advisors and they continue to work as advisors, but 10-20% of their time can be devoted to facilitate the networks. For day-to-day communication, virtual collaboration platforms (D-groups) have been established. As a result of the networks, cross border exchanges and collaboration between advisors have become common practice in SNV. Advisors post questions on the D-group and in the resources sections of the D-groups advisors establish their virtual libraries consisting of key documents, which they consider relevant for their particular practice area. Advisors can collaborate and consult each other while drafting papers.

The ‘knowledge improvement’ experienced a certain ‘learning fatigue’ during 2003 and 2004. At corporate level less learning programmes were offered, and knowledge interchange became an intrinsic component of the function of each advisor. Newly

50 Evaluation SNV recruited (mainly) external subject matter specialists (so denominated ‘SPARKS’) were contracted to revitalise the knowledge development.

At portfolio level most advisors participate in: 1. D-Groups; 2. Knowledge networks; 3. Thematic Groups are like knowledge networks, but deal with cross cutting themes on issues like gender. . 4. Every junior advisor is supervised/supported by a coach and the Portfolio Coordinator.

Other ways of knowledge management and sharing at portfolio level are an annual 3-days learning event, the annual general meeting, and team meetings/presentations at portfolio- level. The Portfolio Coordinator stimulates each advisor to participate in international conferences.

External knowledge development After the internal knowledge networks were initiated, SNV advisors from mid 2004 onwards started to participate more and more in international discussions. They can provide inputs on the basis of the practical experiences in the field. In the past, SNV advisors were mainly focused on their own advisory work with clients. Since 2004, it has become more common to open up and to disseminate to the outside world what SNV is doing. Now, it is seen as part of their function to participate in international conferences and it is being stimulated (see also section 3.2.5). .

Development of South-South knowledge exchange SNV also had the ambition to become a nationally recognised knowledge organisation. With ‘national’ it was meant the host countries, not the Netherlands.

The regional knowledge networks on the various practice areas have improved the ‘visibility’ of SNV in host countries. For instance, SNV participated in a conference of the Association of African municipalities and was asked to become co-organiser of next year’s conference.

The regional cooperation in knowledge networks also contributed to South-South knowledge sharing, given that SNV advisors are both international and local. However, according to SNV headquarters the development of South-South cooperation between countries has been somewhat slower than intended. The survey results reveal that portfolios in different countries do not apply a standard content of the concept ‘south- south’ contacts: some countries consider all contact at national level as ‘south-south’ contacts, other limit the concept to cross border activities or restrict it to international contacts among advisors. It is SNV’s intention to magnify the practice area networks to broader alliances not only accessible to SNV advisors, but as tools geared towards regional knowledge networks.

Evaluation SNV 51 3.2.7 Increase of efficiency

The Subsidy Agreement indicates the intention of SNV to become more efficient (characteristic 10). In this section the efficiency at corporate level had been evaluated on the basis of the development of overhead costs during 2002-2005 and efficiency improvement issues for coming years. Efficiency will also be dealt at field level (see chapter 5).

Overhead 2002-2005 In the report ‘Optimizing Support, Design of the desired support organization & framework for implementation (June 2005) it is mentioned that SNV currently spends 46% of its annual budget on overhead. Overhead is here defined in a very broad sense and includes all support costs (support personnel expenses, selection and training expenses, operating expenses, and programme support expenses made by Head Office).

Table 3.7 shows the development of the primary process costs and all support costs over the years 2002-2005. The primary process costs include advisory staff related costs made at field offices and head quarters and costs made in relation the development of partnerships. The support costs are all other costs. Both primary process costs and support costs increased over the years (except the support costs in 2002) due to an overall growth of the organisation and increase of number of advisors. The support costs had been calculated at 45 percent of all costs made in 2002. This percentage was reduced in the following years to 43 percent.

Table 3.7 Calculation of programme costs – version 1 (euro million)

2002 2003 2004 2005 Primary process costs* 36.9 38.5 39.0 46.9 Field offices 30.7 32.9 33.2 40.7 Head offices 6.2 5.6 5.8 5.2 Partnerships 0 0 0 1.0

Support costs** 30.7 28.8 29.6 35.3 Field offices 18.0 19.7 20.7 25.1 Head offices 12.7 9.1 8.9 10.2

Total costs 67.6 67.3 68.6 82.2 % primary process 55% 57% 57% 57% % support 45% 43% 43% 43% Source: SNV administration * primary process costs are advisory staff related costs made at field offices and at head office ** support costs are all other costs

The support costs at field offices include costs of staff delivering services such as driving, guarding, gardening and maintenance. The percentage non-advisory staff in the field is 46 percent, 57 percent and 53 percent for the years 2002, 2003 and 2004 respectively. So for each advisor in the field there is more than one additional staff member for support services (see also Table 3.8).

52 Evaluation SNV Table 3.8 Number employees divided by regions and head quarters

2002 2003 2004 2005 Total number of employees 1373 1282 1280 1531 Number of employees in the regions 1297 1203 1205 14665 Number of advisors 702 517 569 947 % advisors/employees in the region 54% 43% 47% 64% Number employees at head quarters 94 79 75 65 % employees at HQ/advisors 13% 15% 13% 7% Source: Annual reports 2002, 2003, 2004.

SNV has also calculated its overhead or programme costs on the basis of the division of costs in direct and indirect costs. Direct costs include all costs of the field offices (including field staff), a part of the costs of the head quarters and costs made in relation to partnerships. Indirect costs relate to overhead at head quarters (excluding the part which is indicated as direct). The development of direct and indirect costs is given in Table 3.9. The percentage indirect costs of total costs decreased from 18 percent in 2002 to 9 percent in 2005. This is mainly due to the increase of direct costs at the field offices and a reduction of indirect costs made at the head office. The number of staff at the head quarters was reduced from 94 in 2002 to 65 in 2005.

Table 3.9 Calculation of programme costs – version 2 (euro million)

2002 2003 2004 2005 Direct costs 55.7 59.0 60.6 74.5 Field offices* 48.7 52.6 53.9 65.8 Head offices** 7.0 6.4 6.7 7.7 Partnerships 0 0 0 1.0

Indirect costs 11.9 8.3 8.0 7.7 Head offices** 11.9 8.3 8.0 7.7

Total costs 67.6 67.3 68.6 82.2 % direct costs 82% 88% 88% 91% % indirect costs 18% 12% 12% 9% Source: SNV administration. * field office costs are direct costs. ** head offices costs are partly direct costs and partly indirect costs.

Efficiency improvement issues SNV has set its own benchmark for percentages of support costs (Optimising Support, 2005). According to this benchmark most professional consultancy firms have overhead costs below 25 percent of total costs. SNV mentions that, because SNV is a development organisation and has an extensive infrastructure in developing countries, a higher overhead percentage is justified.

The aim of SNV is to reduce the total support costs (see table 3.9) from more than 40 percent to less than 35 percent of the annual budget by the end of 2006. This will be realised by:

Evaluation SNV 53 • a reduction of support staff; • to outsource, if possible, support tasks such as driving, guarding, gardening and maintenance; • to keep to an average span of control for managers of 10 professionals.

In 2006, the organisational structure of the support staff will be changed. Support staff in the field will be concentrated at regional level in stead of country level. The staff at the head office will be reduced to 45 full time equivalents: 19 in Finance, 12 in Human Resources and 14 in Strategy.

3.3 Assessment of the realisation of the transformation components

In this section an assessment is made of the realisation of the transformation components as described above (including the 10 characteristics of the Subsidy Agreement). This assessment is made by the evaluators on the basis of their findings.

For visualizing the extent of realization of the seven components, use is made of a ‘time scale’. On this scale, the formulation of objectives in 2001 is the point of departure and the full realisation of the objectives in 2006 is the point of destination. The colour blue in the figure below indicates the extent to which a transformation component has been realised.

Figure 3.6 Assessment of realisation of the seven transformation components

Components Start → Transformation → Target

2001 2006

a. Dev. advisory activities

b. Dev. Personnel

c. Result measurement

d. Third party funding

e. Role SNV in the field

f. Knowledge organisation

g. Efficiency

Organisation of activities It can be concluded that SNV has accomplished its target in terms of the transformation from an organisation focused on the supply of technical assistance and project funding to an organisation that supplies advisory services to clients. All activities undertaken in the

54 Evaluation SNV field are predominantly geared towards the supply of advisory services. Traditional clients were reassessed, new clients were identified, and the number of clients increased significantly. The majority of clients are organisations at meso level. The demand driven approach was changed into a pro-active approach in which more attention is paid to the selection of clients with a potential for the realisation of impact on governance and alleviating poverty.

Organisation of people SNV has put much effort in the transformation of its human resource base, from technical assistance experts to advisors specialised in advising and networking. One consequence was that several professionals left the organisation. New staff was recruited and existing staff was extensively trained and coached. It can be concluded that the transformation with respect to personnel has been realised.

However, it should be noted that the three main reorganisations concerning the strategy change, organisational change and support change have put the organisation upside down. The change process required a long period to mature and caused unrest (and uncertainty) among staff. It also absorbed a lot of time and energy from the organisation. Outsiders, like partners in the Netherlands and abroad and embassies still consider SNV as an inward focussed organisation, very busy with its own transformation process and internal systems. Therefore, SNV needs a period of organisational consolidation that may give the opportunity to devote all time and energy to SNV’s mission.

Organisation structure and processes Planning, management and control are structured according to the primary process of SNV aiming at the supply of advisory days to clients. The input (advisors, knowledge) and output (services in the form of advisory days) have had much attention. The measurement of output takes mainly place through the client satisfaction survey, while outcome is measured by means of an end of year or end of assignment evaluation. However, a well established corporate system on results, especially with respect to effect (strengthened performance by client organisation) and to impact (contribution to poverty alleviation) has not been established yet.

The monitoring system is primarily focussed on the operational management cycles and less on outcome, results and impact indicators. SNV is still experimenting with the evaluation instrument, especially with respect to the measurement of impact. The local evaluation studies do not have a common approach and methodology.

The funding strategy No explicit agreement in the Subsidy Agreement was made between SNV and the Ministry of Foreign Affairs about additional funds. It was agreed that SNV would explore the opportunities and this was done. The strategy of SNV is now to opt for a large component of core funding by the Ministry of Foreign Affairs in order to maintain an optimal autonomy and independence in its operations. At the same time SNV envisages to develop on a small scale the possibilities to mobilise additional funds from third parties for up-scaling of SNV activities.

Evaluation SNV 55 The assessment of this fifth ‘characteristic’ of the Subsidy Agreement is rather difficult, since the characteristic did not set clear targets for 2006, but just described a process. This process has been pursued and a conclusion was drawn. However, it is not clear whether this situation is sustainable in the long run.

In practice, SNV has chosen to depend for well over 90 percent on financing from the Ministry of Foreign Affairs. This has some consequences: • Although ‘disentangled’ in its organisational structure from the Ministry of Foreign Affairs and ‘autonomous’ in setting its policies and strategies, SNV has become financially dependent on the same Ministry. The future balance between the Ministry’s responsibilities over SNV’s performance, as required by the Subsidieregeling Ministerie van Buitenlandse Zaken and SNV’s independence from public entities, as established its Articles of Association (Statuten, article 2, sub 5) merits additional attention. SNV keeps a separate financing position as recipient of a subsidy by the Ministry of Foreign Affairs. Whether it is –or will be politically and legally desirable to maintain this special position is outside the scope of this evaluation;12 • The subsidy allocation by the Ministry is the main determining variable for SNV’s growth and activities.

The role of SNV in the field From 2004 onwards, SNV invested actively in strengthening its role in the international developing platforms by attending conferences and the preparation of publications together with other organisations (ECDPM and UNDP).

Also much effort is put in realising partnerships. SNV indicated that initiating and fostering partnerships is a time consuming business and that not all contacts lead automatically to fruitful alliances. In about two years the partnerships will be evaluated by SNV.

The role of SNV as knowledge organisation SNV put much effort and resources in the development of knowledge. Staff got ample opportunity to share knowledge through networks and attending knowledge sharing meetings. This was often focused on internal use of knowledge. Since 2004, the knowledge was spread more externally through publications and participation of workshops and seminars and cooperation with other organisations. It can therefore be concluded that SNV is moving in the right direction to become an international recognised knowledge organisation, although many of the initiatives are still in an early stage.

Increase of efficiency SNV indicated that a reorganisation in the support structure and a reduction of the support costs is the next and last step in the transformation process. At the moment, about half of the staff in the field has a supporting task (driving, guarding, gardening etc).

12 Conditions set by the Ministry of Foreign Affairs to other subsidy recipients do not apply to SNV. Within the MFS subsidy framework (so far not applicable to SNV) subsidy recipients are required to either to undertake commercial activities, to generate own revenues or to find third party funding in order to avoid strong dependence.

56 Evaluation SNV

The planned concentration of support services at the regional offices could create possibilities to realise economies of scale and thus reducing support costs. On the other hand, large offices at regional level introduces an additional organisational layer and longer or more lines to the head quarters.

The transformation process of SNV in a period of five years required a lot of strategy input and support at all levels. Now, SNV has been successfully transformed from an organisations delivering technical assistance on project basis to an organisation supplying advisory services, more room will be available for a focus on organisational efficiency.

Evaluation SNV 57 Box 3.1 Perceptions by partners To get a better view on how other organisations look at SNV, we have interviewed six key partner organisations of SNV (see Annex A). In recent years, SNV has worked together with all six in various countries. The main findings of our interviews are given below.

Perceived strong points of SNV: • Almost all partner organisations mention the strong local presence of SNV as a positive point. • Some also mention the demand driven working method and the good mixture between international and local staff as positive points. • Most partner organisations believe that SNV has made the right strategic choice of focusing its activities on strengthening the capacity of local organisations at the meso-level. • The strict distinction between capacity development and providing financial resources is seen as a clear decision. However, some organisations warned that this approach does not work in all countries. In practice, it is not always possible to implement plans without bringing your own resources.

Perceived weak points of SNV: • SNV and its advisors do not work in a critical environment, because broad financing is given to SNV by DGIS. SNV provides free advisory services and therefore client organisations will not be critical. • Various partner organisations pointed to the fact that, as a result of the lack of critical environment, the culture of SNV has become very much internally oriented with a lot of internal discussions and meetings. SNV is seen as an organisation with a “strong talking/meeting culture” and not a “time is money” attitude. • The SNV strategy is not always seen as very stable. It is important that SNV now will implement the new strategy and not change its direction again. According to some of the partner organisations, SNV advisors have become tired of all the changes and internal discussions. • Some think that SNV is too much focused on the Netherlands government.

Perceived overlap with other organisations: • In most cases, the SNV activities are complementary to what other organisations are doing. However, it may occur that SNV competes with other organisations. For instance, when providing support to associations of municipalities, they interfere with a (potential) market of VNG. In addition, there is the risk that SNV competes with local consultants at the local market.

SNV as a recognised knowledge institution: • According to the partner organisations, SNV is gradually building up itself as a knowledge organisation in the area of capacity building, for instance by working together with ECDPM and UNDP in Capacity.org and taking part in international discussions on capacity building.

58 Evaluation SNV 4 The field level implementation of capacity building

In its structure, this chapter follows the sequence of the subsequent stages in the SNV intervention logic (see 4.1). The chapter is based on findings derived from a survey among all SNV portfolios worldwide, as well as field visits to three selected portfolios (Santa Cruz in Bolivia, North Rift in Kenya and Koulikoro in Mali).

4.1 SNV intervention logic

The new approach had consequences for what SNV was supposed to ‘produce’ and how that ‘product’ could contribute to the achievement of the overarching objectives of poverty reduction and good governance.

Section 3.2.3 presented the ripple model as conceptual translation of what is basically the logical framework sequence in terms of output, outcome, effect and impact (see figure 4.1). While usually the logical framework is presented in a linear manner, the ripple model takes an imagination as ripples in a pool, but it is important to note that the model itself is a variation of the logical framework and adheres to the rational actor model of thinking.

In October 2005, the SNV Extended Board adopted a new model to organize SNV’s primary process, introducing a level of standardisation of procedures, documents and vocabulary as a pre-requisite for demonstrating results.

SNV provides advisory services only and does not ‘produce’ tangible results in terms of the ‘overarching’ objectives on its own, but depends on its clients to do so. According to the SNV intervention logic, the output is the immediate product of the activity (the advisory services), the outcome is that those services have resulted in improved capacities of the client. The effect / result is that these improved capacities are actually being used by the client and lead to improved performance in relation to the clients’ objectives. Results cannot (only) be attributed to the outcome, but there is a high probability that they do so. The impact is determined by the extent to which the improved performance of the client actually contributes to the overarching objectives. Once more, this envisaged impact cannot be attributed (only) to the result, but the probability that the client contributes to the overarching objectives should be as high as possible. That probability is largely determined by the selection of clients.

Evaluation SNV 59

Figure 4.1 SNV’s logical framework sequence

Input Output Outcome Effect / Impact

Result . Direct Quality Improved Improved Poverty advisory services capabilities performance alleviation, services of client of client good

. Human . Subject governance resource matter development . Capability . Client .

selection Organisation Strategy . Networks . Funding (by . Systems others)

requires additional resources

Logical sequence and instruments In order to facilitate and underpin methodologically, SNV has developed instruments for the various steps in its intervention logic for application at field level. Not all those instruments have been introduced simultaneously, and not all instruments are fully operational yet. SNV is still in the process of constructing a consistent and all-embracing system of instruments that can be applied in all countries and under all circumstances. At the same time these instruments should provide a structured frame, but should not be a straitjacket that hinders SNV operations. Figure 4.2 summarizes the main methodological instruments that have been developed for the subsequent stages in the intervention logic.

Since 2003, the planning cycle starts with a “Framework Letter”, sent by SNV headquarters to the country offices. The framework letter invites to indicate both the strategic positioning of SNV in the respective countries and the tentative choices made with respect to the main practice areas. The framework letter is mainly used for overall broad strategic choices and does set financial ceilings beforehand.

60 Evaluation SNV Figure 4.2 Instruments and logical framework

Instrument Input Output Outcome Effect Impact Project Management cycle (various instruments, like Framework Letter, Management Agreement, country multi-annual planning, area plan) Client selection instruments (context analyses at national and portfolio level) Memorandum of Understanding, Advisory Plan Management Audit Output measurement (client satisfaction survey) Outcome measurement (end of year evaluation, peer review) Case study Local evaluation (incl self-evaluation) Practice area evaluation External evaluation

Main focus of instrument Covered in part by the instrument

Based on the framework letter, a so-called Management Agreement is drawn up between headquarters and the country offices (or at regional level, as was the case in Latin America, 2004). 13 Using the Management Agreement as a broad frame, each portfolio office develops its own ‘area plan’ along the strategic lines agreed upon at country and regional level and in accordance to the multi-annual planning (C-MAP) exercise. The Portfolio Coordinators (PCs) are responsible for the outline of the portfolio plan and budget, as well as for monitoring and review of activities. Many decisions related to the various steps in the intervention logic are taken by the Country Management Team.

4.2 Input

At the moment of signing the Subsidy Agreement in 2001, it was envisaged that by 2006 SNV would have developed an advisory practice that combines thematic, management and process expertise and that should be underpinned by a clear methodology (see section 2.3: characteristic 2). These advisory practices would be directed to strategically well chosen clients (characteristic 10). And SNV staff would operate as advisors, who are coached, individually educated and continuously monitored on their performance (characteristic 3).

13 This model fits the concept of country offices as full business units. After the introduction of the Optimising Support policy (2005) part of the support services shifted to the regional level. In consequence, the formalization of responsibilities at various levels of the organisation has become more complex. According to SNV (18th January, 2006), the Agreements will continue to be subscribed between Headquarters and the Country Director, but closely liaised with the Regional offices.

Evaluation SNV 61 These expected changes refer to a large extent to the provision of services at field level. SNV provides capacity building services to meso level organisations. That is the ‘input’. Institutional capacity building is composed by the following components: 14 • enhancement of human capabilities (human resource development); • organisational development; • networking; • systems development (the external context or legal aspects, regulations and finance).

SNV’s own concept of Capacity Development Services 15 (CDS) is not precisely the same, but roughly comprises the same elements: • Diagnosis and learning; • Organisational Development; • Network and partnership building; 16 • Influencing institutional change.

These services are delivered to organisations. SNV’s main asset thereto is its human resource, its advisors. The input is defined by the following three elements: • To whom? – the client selection; • What and how? – the practice areas and capacity services delivered; • By whom? – the advisors and their competencies.

4.2.1 Client selection

Client selection is a critical step in maximizing the probability that clients eventually contribute to the overarching objectives of poverty reduction and good governance. The selection process consists –at least in theory- of a number of steps: (a) the selection of countries, (b) the selection of geographical areas within a country, (c) the selection of themes (practice areas) and finally, (d) the selection of clients and assignments. These steps are presented in continuation:

a) Selection of countries The first step in a client selection process should be the country selection. In 2000, the Ministry of Foreign Affairs stated: “regarding the policy on subsidies, DGIS takes as point of departure its selection of countries for bilateral aid. Those who apply for subsidy by the Ministry should have matching policies”. 17 The requirement for ‘matching policies’ was possibly outbalanced by the simultaneous requirement of ‘complementarity’, a concept never made explicit by the Ministry of Foreign Affairs.

The start of a multi-faceted change process in 2002 made SNV reluctant to re-assess its presence in the various countries. An overhaul of SNV presence in countries could have

14 See for example: Ministry of Foreign Affairs, Policy and Operations Evaluation Department. Institutional Development. IOB- Evaluations no.284. March 2000. 15 Source: SNV presentation 15th November 2005. 16 According to the theory on capacity building, an institutional relation is one between actors and does not necessarily requires a formalised structure. In the case the relation counts with a structure, this is called organisation. 17 Meeting SNV – PlV DGIS, 30th May 2000. Source: Hoenderdos Consult, 2004:65. See also: Subsidiebeschikking SNV, Art 9.

62 Evaluation SNV caused turmoil among advisors and even at a political level, and SNV management postponed setting criteria for country selection to a next stage. 18 In 2001, about half of the SNV host countries did not coincide with the then existing ‘list of 19+3’ countries for bilateral aid. During the period 2002-2005, SNV discontinued its activities in Botswana (2003), and started up activities in Bangladesh, Cambodia, the Democratic Republic Congo, Sudan, Bosnia Herzegovina, Macedonia, and Montenegro. SNV’s arguments used for moving into new countries varied widely. For the Balkan, efficiency arguments were used in the sense of optimal utilisation of SNV human resources and ‘span of control’. The move into other countries was mainly justified by the SNV internal policy of ‘looking for cross-border synergy’. 19 Of the newly selected countries only two ‘match’ with the 2005 list of 36 countries that receive Dutch bilateral aid. b) Selection of geographical areas SNV applies a (national) context analysis for the selection of geographical areas (and thematic areas) within a country. Usually SNV conducts this general context analysis every two years. Both at national and portfolio level monitoring of changes in the context is done frequently with all advisors. While in Kenya this context analysis was carried out by SNV staff, in Mali it was done by an external consultant (2003) and in Bolivia by a mixed team.

Prior to 2002, the area selection was mainly based on poverty indicators linked to specific target populations. Since 2004, this has changed and the background studies apply a problem-based approach, less restricted by poverty indicators only. This approach coincides with the mainstream concept of poverty as introduced in the Ministry of Foreign Affairs in 2000. 20 Survey results indicate that since 2004 (particularly in Asia and Latin America) the Poverty Reduction Strategy of the national government is used as a frame for the identification of key areas for poverty reduction.

Geographical choice is not exclusively determined by ‘the poverty of the area’. Although poverty characteristics are an important criterion, they are not the exclusive determinant. The case of Santa Cruz in Bolivia can be taken as an example in that respect. The choice for this prosperous economic centre of Bolivia may surprise, but is explained by the particularity that the Department is characterised by the most skew distribution in land ownership of Latin America 21 , as well as by forest and agricultural exploitation that causes the deprivation of the original population of their traditional land. In the violent conflicts on land (use), the voice of the indigenous population was hardly heard, due to its lack of organisation, lack of political clout and a weak cultural identity. SNV’s start in Santa Cruz (1986) was initially based on a ‘target group approach’, but gradually evolved into a presence justified by the analysis of one of the root causes of poverty, being the access to land and the lack of cultural identity and recognition of the original population.

In Mali, the geographical areas of intervention origin from those attended 8-10 years ago and were mainly determined by the presence of clients with whom SNV had maintained a

18 Interview with Board of Directors, Dec. 2005. 19 Ecorys-SEOR survey, Ecorys & SEOR field study Mali and interviews SNV The Hague, Dec.2005. 20 The poverty approach adopted by the Ministry of Foreign Affairs in 2000 focuses on combating the structural causes of poverty and on strengthening those missing or weak elements that are pivotal for enabling development. 21 The Gini index on land ownership is 0.98.

Evaluation SNV 63 long-standing relation (so-called path dependency). In Mali, the decision to move into new geographical areas (Banamba, Asongo) was triggered by internal (SNV-efficiency) considerations: “One of the criteria for the choice of new geographical areas is the optimal use of SNV infra-structure. In 2006, SNV will start in a new region, Sikasso. One of the criteria is that Sikasso is the “bridge’ between Koulikoro and the intervention zones of SNV-Burkina and facilitates cross border activities.” 22

Like in Bolivia, a comparable ‘target group focus’ was found in the choice of the North Rift portfolio in Kenya (in itself fruit of a merger of the Kerio and the Turkana clusters in 2004), where a ‘pastoralist perspective’ guided the choice for the geographical area. Also here the initial target group focus evolved into a perspective of broader ‘inclusion’ problems of semi-nomad pastoralists.

c) Selection of thematic areas Since 2004, SNV places the interest of target groups in the context of thematic (practice area) development problems (see section 2.1). The gradual shift away from a target group approach to a selection process based on pre-determined practice areas did not result in any drastic change in the presence of SNV in geographical areas in the three countries visited. In fact, SNV has not withdrawn from any area for that reason. It did move into additional geographical areas, sometimes establishing new portfolios in order to implement better a certain practice area (Mali).

The practice areas are still broadly defined, while some are in fact a regional specialisation, such as dry land management in Western and Central Africa. Both regions and countries made efforts to be more specific, more specialised in a certain (sub-) practice area. For example, in the East and South African Region, it was decided (to continue) to focus on local governance and private sector development, peace building and conflict transformation, as well as on the developmental response to HIV-Aids. Natural resource management was embedded into local governance.

Apart from a regional or country specification, also at portfolio level, practice areas can be made more detailed. For example, in the Santa Cruz portfolio practice areas have been ‘translated’ into themes like: spatial (‘territorial’) organisation, land ownership, social control and transparency. In the North Rift a deliberate choice was made to focus on Alternative Livelihood Systems, as well as a positioning with regard to HIV/Aids issues.

In the three countries visited, these choices were made on the basis of a sound analysis of regional/national poverty trends and governance issues, an analysis of roles and strategies of potential clients and other stakeholders. For each practice area, trends at regional, national and meso level were analysed by the portfolio team (or consultants contracted to that end), including priority issues to be addressed, envisaged results (products), players (partners/clients) and the relation with other practice areas. Perspectives, challenges and demands of different types of clients were analysed, and related to possible responses by

22 Comment Country Director on draft Mali report Ecorys-SEOR evaluation, 2005. In its comments on the draft final report SNV argued that SNV Mali has started to work with 44 new clients, has incorporated three new portfolio areas and is the leading force behind cross-border initiatives (SNV, 14 th February, 2006).

64 Evaluation SNV SNV. In addition, for each practice area, results (products) were formulated as well as the envisaged capacity development services.

Internal SNV processes play a role as well. In Latin America, SNV aimed at functional connections between the various programmes. There existed basically two dynamics: one of Central-America and one for South-America (Andes-Amazon). From 2002 onwards there has been an active process of creating one single coherent, recognizable Latin American programme. d) Selection of clients and assignments To whom SNV may provide services depends on various variables, such as the relevance of clients to the SNV practice areas, the potential of clients to make a difference to their environments, the demand by clients and the distribution of (potential) clients over the public (government, non-government) and private sector. Survey respondents (portfolio coordinators) also mentioned the strategic relevance of an organisation for the implementation of the national poverty reduction strategy and its contribution to achieving the MDGs as variables that are taken into account in the selection of clients.

For client selection a second type of context analysis is carried out, of an institutional character. This institutional context is not only determined by the relations between potential clients and partners, but also by a more general ‘enabling environment’ for capacity building. It is also characterised by the presence of other suppliers of similar kinds of services. The institutional context is highly dynamic and is monitored and revised at least annually by the SNV country and portfolio managers.

There is no single set of institutional context factors that is determinant for all countries. In Bolivia, the institutional environment was mainly characterized by political instability, with frequent changes of responsible ministers (and all main civil servants). At the same time various development agencies have started to operate in a ‘privatised’ manner, using models comparable to the one applied by SNV. That is to say, these agencies have been disentangled from the financial streams, offer services (and / or projects) on the basis of demand by clients and formalize the relation with the client in agreements. This is the case for GTZ and UNDP, while the Belgian cooperation operates in Bolivia through a local consultant. Also financiers of the private sector have started to operate through local agents (Misereor). The kind of inputs delivered by SNV is comparable to those delivered by GTZ, NGOs or consultancy bureaus. 23

In Kenya, a contextual point of importance was that the Dutch bilateral aid was suspended between 1999 and 2004 for governance reasons. This made SNV to balance carefully its relations with the public sector, at least at central level. The existence of the Impact Alliance (a network of 170 capacity building organisations worldwide driven by 10 partner organisations, including SNV) enabled SNV to ‘wrap’ its services in the frame of this broader network. There are a large number of other external agencies active in the

23 SNV partners, like the NGOs CEADES and CEJIS (both ex SNV clients) provide specialized advisory services to clients. SNV contracts consultants from companies to provide services to clients on behalf of SNV.

Evaluation SNV 65 North Rift area 24 , mostly supporting community based projects. SNV is the only organisation dedicated exclusively to the provision of capacity development services.

In Mali, the relatively long history of SNV in local governance on the one hand and the establishment by the Malian authorities of technical support institutions like the “Centre de Conseil Communeaux de Cercle ”(CCC) linked to financial support (ANICT) were specifically favourable enabling conditions for further activities in local governance. The Malian government has conceived the CCC modality to invite the private sector to propose advisory services to municipalities. SNV has been awarded CCC-contracts in the districts of Koulikoro, Banamba, Gao, Ansongo and Menaka, while in other districts international organisations like Helvetas and local consultancy firms have been awarded similar contracts. The Malian government pays the CCC-holders for their services, with foreign funding (SNV receives funds as well). SNV Mali claims it has enhanced the capacities of local consultants in the region of Koulikoro in the past in such a way that they can deliver advisory services to municipalities.

At portfolio level a client (group) analysis is carried out. This is a more specific analysis that contains historical background of the (potential) clients, details the inter-relations among clients, assesses the potential impact, etc. The contact persons, the thematic leaders and the PC are responsible for this client specific analysis.

In the Koulikoro portfolio, the main determining factor of SNV-client relations was rooted in history. Most of the current clients had been selected based on either direct or indirect relationships in previous years. But in addition (and in particular since 2003) new clients were selected as well, based on strategic characteristics of the potential clients, such as: • Collectivités territoriales (municipalities, districts and regions) are considered to be essential actors (authorities) for local governance and development; • Civil society: closely positioned to populations and possible actors of countervailing power (NGO’s, associations) with a particular accent for promotion of women’s interests; • National/central and deconcentrated technical services: in particular those related to SNV’s practice areas, like the structures in charge of agricultural produce chain development (‘filières agricoles’); • Economic actors: (unions of) organisations of producers on regional/geographic or on produce level, groupings of economic interest, chambers of agriculture, economic chamber of youth.

The North Rift portfolio team in Kenya used a general checklist that takes issues into account like vision/mission of the organisation, contribution to poverty alleviation, ability to sustain change, gender sensitivity, strategic positioning and commitment. Client scans, organisational self-assessment (OSA) tools, gender scan and opinion polls are used in order to assess the needs and capacities of potential clients. In 2005, two criteria were added: clients should be able to express a clear demand and should have up-scaling

24 Including World Vision, ActionAid, JICA, OXFAM, a number of church based organisations (HIV/AIDS and other issues), UNDP, UNICEF, UNHCR.

66 Evaluation SNV potential. In 2005, the North Rift Portfolio ‘sharpened’ itself through a client review and the introduction of a multi-stakeholder approach.

Also SNV Santa Cruz applies criteria to select clients 25 , such as (amongst others): • the relative importance of the client in the national and subnational context; • the capacity of the client to produce impact and ‘roll out’ this impact (priority for ‘showcase clients’); • the ‘spread’ over civil society (private sector) organisations and public sector; • the opportunities to obtain finance from third parties; • the relations (‘network’) between the various actors: deliberate choice for mutually related actors; • and, since 2004: the relation with the Bolivian Poverty Reduction Strategy and the Millennium Development Goals.

Over the years, the identification of strategic clients has become very important. In order to operate successfully in a certain practice area, SNV wants to provide services to key actors. Some of those partners/clients are so important (for example, a ministry with decentralised agencies; an umbrella organisation of associations of producers in a production-marketing chain), that without their consent or active involvement services to other actors in the field would be less meaningful (see Figure 4.3).

Figure 4.3 Strategic and related clients

Strategic and related clients

others

Strategic client Strategic client

Next to the selection of strategic clients it is important to liaise with strategic partners (for example to ensure funding). Clients and partners can be the same (for example, a ministry can be a partner when it comes to activities at the meso level, but it can be a client at national level). On the selection of these strategic partners, SNV has elaborated Guidelines on strategic partnership .26 What calls attention of these Guidelines is that it starts from a national (macro) perspective and stresses the importance of partnerships in

25 Source: Santa Cruz presentation, 9th December 2005. 26 Strategic Partnership Guideline. The document is the result of a research on partnerships by I. Pelkmans, under supervision of K. Zevenbergen, and input from a partnership-paper written by S. Salinas and D. Arts (SNV Latin America).

Evaluation SNV 67 search of mobilising efforts to achieve the MDGs and to implement the PRS (see box 4.1).

The extent to which a client may dispose over (additional) financial resources has not been a predominant criterion in the client choice. It has however, played a role in determining the kind of services to be provided, the duration and the periodicity of the advisory services. It has also been a reason for not continuing the relationship (Hoenderdos, 2004:75).

Box: 4.1 PRSP, a windward anchorage

From 2005 onwards, SNV considers the Millennium Development Goals (MDGs) and the Poverty Reduction Strategies (PRSPs) as ‘anchor’ at macro level for its activities. At portfolio level this ‘anchoring’ has been ‘translated’ into the search for a match between the activities and these frames.

Such an ‘anchor’ is not always and by definition solid. In Bolivia for example, the PRSP is a ‘dead’ document, not recognized by the last three governments (ISS-SIDA, Nov.2005). In contrast, in Kenya, the Economic Recovery Strategy for Wealth and Employment Creation is a policy defining document that has been used to ‘frame’ the activities of the North Rift portfolio.

Whatever the situation in a country might be, from the meso level perspective, PRSPs use to be an anchorage at the windward side. One of the reasons for such a statement is that PRSPs show countries as a monolite, suggesting implicitly that government will strive for poverty reduction in an equalitarian manner, as if presidentialism, strong political polarisation, economic elites, regionalism, clientalism, ethnic discrimination, power abuse and corruption were non existent. But in many countries, poverty at meso level is exactly the consequence of existing inequalities and the presence of SNV in some areas precisely explained by those inequalities. Although the increased democratisation of the formulation process to come to a PRSP contribute to reduce the inequalities mentioned, inequalities in implementation happen to be persistent.

Source: authors.

Demand and supply To whom the input is finally been supplied also depends on the balance between demand and supply for advisory services.

The Strategy document of 2001 stressed the importance of a demand driven approach. During the years 2002 and 2003 in all three portfolios a merely demand driven approach was pursued with respect to client selection, although the kind and characteristics of the clients had been laid down in profiles beforehand. Gradually SNV became convinced that through a ‘demand fundamentalism’ it would be hard to achieve its objectives. Demand based exclusiveness would even jeopardize to ‘fill in’ the profiles of preferred clients, since thereto having some strategic clients in the portfolio was a condition: results or failures in a certain practice area may largely depend on a few critical actors only. In 2004, in both Kenya and Bolivia a more pro-active approach was pursued in addition to the ‘reactive’ selection (2004). SNV’s 2005 policies required even more pro-activeness, since clients had to be “anchored at macro level”, while activities had to adhere to the MDGs, and preference had to be given to cross-border or supra-national activities.

68 Evaluation SNV

By 2005, possibly with exception of Western and Central Africa, the majority of the SNV portfolio coordinators worldwide indicated to identify new clients mainly in a pro-active manner (see Table 4.1).

Table 4.1 Share of new clients identified pro-actively in 2005

Region total/average Share of new clients identified pro-actively in 2005 Asia 56% 1 West & Central Africa 33% 2 East & Southern Africa 56% Latin America 78% 3 Albania 79% Notes: 1 The total no. of clients added in 2005 at country level (according to the Country Director) is less than the sum of the registered clients at portfolio level (according to the Portfolio Coordinators), possibly due to a lack of timely updating at country level. 2 Based on data from the Country Directors of Angola, -Benin, -Cameroon, -Ghana and -Mali. 3 Based on data from the Country Directors of Bolivia, -Honduras, -Nicaragua and Peru. Source: Survey ECORYS & SEOR, 2005; region average is based on country level data (provided by Country Directors).

Although SNV does have a corporate policy to register rejected clients and the interpretation of ‘rejection’ differ from one portfolio to another, according to survey information supplied by the portfolios, on average 26 percent of all organisations (worldwide) that requested SNV advisory services in 2005 were not accepted as SNV- clients (see Table 4.2) for various reasons. The most prevailing reasons were that the requests did not pertain to strategic constellations identified; the organisation operated at micro-level only and lacked linkages to meso-level; or the organisation aspired for obtaining financial support.

Table 4.2 Rejection of requests for support

Region Average Share of organisations rejected out of total that applied for SNV services in 2005 Asia 24% West & Central Africa 37% 1 East & Southern Africa 34% 2 Latin America 31% Albania 4% Notes: 1 Based on data for 31 out 34 portfolios being part of SNV W&C Africa, excl. Guinea Bissau. 2 Based on data for 18 out of 23 portfolios being part of SNV E&S Africa, excl. Sudan. Source: Survey ECORYS & SEOR, 2005; region average is based on country level averages, which are in turn based on portfolio data (provided by Portfolio Coordinators).

One would assume that additional criteria such as the ‘anchoring at national level’ would make the process more pro-active. However, in both the Santa Cruz and North Rift portfolios the opposite was reported: both portfolios received much more requests, in

Evaluation SNV 69 particular from national level potential clients looking for support to implement their policies or activities at meso level. More ‘top-down’ requests were received than ‘bottom up’ requests. The North Rift Portfolio reported that in 2002 about 80 percent of the clients were identified by SNV itself, and about 20 percent by demand, while at the end of 2005, this situation had been reversed. In the Koulikoro portfolio, the team received many requests from small civil society organisations and rather weak non governmental organisations. Obtaining funding is important to those organisations. All requests pass through a first check against a predetermined checklist. For example, political parties do not pass the first check, since they are not considered relevant clients for poverty reduction and good local governance. Once the first selection has been made, there is a further examination during one or more meetings between the potential client and a competent SNV advisor. Many requests are discarded due to the fact that the first interest of the potential clients is financial support (or organisations work at micro level only).

Micro-meso-macro Coming from a situation in which most SNV ‘host organisations’ were local level actors, the target was set to achieve that by 2006, 80 percent of its clients would be organisations working at the meso-level. This target was maintained over time, but the interpretation of the destination of the remainder 20 percent suffered changes over time. Since 2004, the 20 percent provides room to both macro- and micro-level clients. Although longer time series are not available, a comparison between 2003 and 2004 suggests that in most portfolios the share of macro level clients tends to increase at the expense of micro level clients (see Table 4.3).

Table 4.3 Distribution of clients according to level

Share of Clients at micro-, meso- and macro level Region average Micro Meso Macro 2003 2004 2003 2004 2003 2004 Asia 2% 4% 60% 64% 38% 32% West & Central Africa 4% 2% 84% 84% 12% 14% East & Southern Africa 1% 1% 89% 88% 10% 11% Latin America 4% 3% 88% 83% 8% 14% Albania 0% 17% 100% 60% 0% 23% Source: SNV Database (SUM~Data 2004-12).

4.2.2 Practice areas

What kind of inputs is being delivered by SNV is largely defined by the practice areas and the kind of services supplied: the capacity development services.

Definition of practice areas for a particular portfolio is a two-way process. There is a ‘top-down’ determination of the main SNV strategies at regional and national level, where the opportunities are being studied for potential developments in the –for the region or country- relevant practice areas. At the same time there is a bottom-up analysis of the (potential) constellations of clients in a portfolio area that may contribute in a significant way to ‘fill in’ a certain practice area. For example, marketing activities for the poor might have been identified as an important practice area at a national level, but if

70 Evaluation SNV there are no organised groups of producers in a certain portfolio area, it becomes less probable that in that portfolio clients may contribute significantly to poverty reduction. In that case the portfolio can decide either to start creating the necessary network of producers at grassroots level (what was the main choice in the past) or may opt to go for another practice area with a higher potential that clients contribute to poverty reduction and good governance (what has been the predominant option over the last few years). Each portfolio has a mandate to devote time outside the agreed upon practice areas to explore new, potentially successful themes. 27 New initiatives within a practice area may emerge from the knowledge networks.

At portfolio level, the practice areas are an “organizing principle” in the first place 28 : it sets SNV’s position and margins for its service delivery (although the demarcation line is never sharp). As an organizing principle it allows a portfolio to select clients that are (or have a high probability to become) linked to each other. It also enables the complementarity of expertise within the portfolio teams. Another function of practice areas is the development of specialised tools and products that can be replicated within a portfolio or among portfolios.

For Latin America there are five practice areas, the four corporate practice areas and ‘water management’ as a regional practice area. From that five, the portfolio Santa Cruz has developed activities in three: • Collaborative forest management; • Responsive and accountable local government; • Market access for the poor (including tourism), and started to develop activities within the ‘practice area’ joint action for local economic development (2004), that subsequently (in 2005) ceased to be considered as a practice area at corporate level 29 .

Cross-cutting attention is paid to gender, while since 2005 an orientation has taken place on ‘water management’ and ‘social service delivery’, one of the two impact areas 30 (see 3.2.3).

In Kenya, SNV has identified constellations of potential clients grouped in five practice areas (and within each one a limited number of themes): • Responsible and accountable government; o Water supply and sanitation; o Primary and basic education; o Local economic development; • Market access for the poor; o Value chain development of key commodities; o Access to productive capital; • Development responses to HIV/Aids; o HIV/Aids mainstreaming in service organisations;

27 Up to 20 percent of the total time, according to the Regional Advisor, Quito. 28 Debriefing Ecorys – SEOR mission portfolio Santa Cruz. 29 SNV Comments on draft final report, 3 rd February 2006 30 Ibid.

Evaluation SNV 71 o Organisational Development in HIV/Aids organisations and capacity builders; • Sustainable Pro-Poor Tourism (Strategy under development); • Peace Building and Conflict Transformation (Strategy under development).

In Mali four main practice areas have been distinguished, whereby the focus of the Koulikoro portfolio is on local governance: • Dry land management; • Responsive and accountable local government; • Market access for the poor; • Gender.

Table 4.4 Number of clients by practice area, Bolivia, Kenya and Mali

Practice areas Number of clients Bolivia Kenya Mali Resp. and accountable local government 32 38 73 Market access for the poor 33 23 11 Collaborative forest management 17 6 Sustainable tourism 2 6 Collaborative water management 2 Collaborative dry-land management 6 17 Gender 9 9 Renewable energy/biogas 1 Other Free/innovative 9 10 Joint Action for Basic service delivery and 4 5 local economic development Institutional resp. to HIV/AIDS 3 Peace building & conflict transformation 1 Not Assigned 1 1 Total – all practice areas 84 1 102 1 106 Note: 1 The total no of clients is less than the sum of the no of clients in each practice area, since some clients are receiving services in more than one practice area.

Gender In the transition from project implementation to CDS, the SNV approach to gender changed considerably. In 2002 and 2003, the approach was mainly Women and Development, and specific projects were implemented. In 2003, efforts were made to identify women organisations and enlist them as SNV clients (mainly in Africa). More or less simultaneously, a gender mainstreaming approach was adopted. In 2004, gender mainstreaming was related to the SNV themes, and a gender taskforce was established to give gender related advice to the thematic groups. In the Santa Cruz portfolio, this was ‘translated’ in a different function of the gender specialist that formed part of the team. The gender component remained strong, but less explicitly expressed in activities. Women and leadership is an integral part of the local governance practice area in all three countries.

72 Evaluation SNV 4.2.3 Classification of inputs

Within the practice areas, the Capacity Development Services determine what kind of input is being supplied to clients.

In daily practice, SNV offers a wide array of services that not always fits exactly the CDS categories. The mix of services provided to a client depends on variables like the duration of the relation, the type of client and the maturity of organisation. In consequence, the supply of inputs is dynamic and changes with the evolution of the client and the relationship between client and SNV over time.

Based on the three portfolios studied in Bolivia, Kenya and Mali, the inputs could be classified either by nature of services or by the modality of provision of advisory service (see for classification with examples Annex B).

The classification of Capacity Development Services is as follows:

Nature of Capacity Development Service Diagnostics and learning Assistance to diagnostics, planning, and learning Training and coaching (SNV contributes to the professional formation of human resources Monitoring of the implementation of development plans Development of didactical tools Organisational Development Assistance to organisational and institutional development Organisational analysis Network and partnership building Facilitation of multi-stakeholder processes (SNV “connects” organisations and people) Material and financial support for the organisation of workshops Facilitation of south-south relationships between clients and other, similar actors or relevant service providers Influencing institutional change Narrowly defined consultancies Conflict mediation (SNV provides instruments, manpower and knowledge)

Modality of Capacity Development Service • SNV as trainer in capability enhancement of human resources • SNV as provider of manpower (Systematisation of experiences, construction of data base, developer of plans, organisation of seminars, research) • SNV accompanies organisations and ‘joint effort’ (Systematisation of experiences, construction of data base, developer of plans, organisation of seminars, research) • SNV articulates, groups actors around a certain theme, links horizontally and vertically, knowledge brokerage, conflict mediation • SNV as contractor of specialist knowledge (Consultancies) • Material and financial support for the organisation of workshops, meetings with partners in a network, facilitation of south-south contacts • Lobby and advocacy

Evaluation SNV 73 This classification is not exhaustive. Specific modalities in countries may exist that cannot be classified as above. An example is the CCC in Mali, a modality designed by the Ministry for Territorial Administration as a national instrument for technical support to the local governments ( Collectivités territoriales –CT).

The modalities cannot be strictly separated from each other and may take place simultaneously, even with the same client. Or they might be sequential in a process. For example, SNV provides the manpower to elaborate manuals, regulations and the like, while in a subsequent stage the client (has been enabled to) elaborates a plan or a marketing study.

The portfolio team Santa Cruz concluded that over the period 2002-05 the focus gradually shifted from direct services in the area of organisational development of the target population to linking, networking and provision of short-term marked out support. In general, small incipient organisations require services in a broad array of aspects, while larger and more mature and well-established organisations require more specific studies and consultancies. While networking and horizontal and vertical connectivity is important for all organisations, it are mainly the meso level organisations that reap benefits from networks. 31

4.2.4 Advisory staff and their competency

The one who supplies the input is the advisor. SNV’s most important asset is its manpower. The quality of staff determines the quality of both input and output. The changes in Strategy required a different profile of staff in the field: the long-term technical advisor and the project managers had to be ‘re-moulded’ into advisors able to work in teams.

Over the period in review, SNV faced major challenges in the area of human resource management: phasing out staff (at regional, country and portfolio level), recruitment of new staff, career planning, competency management, boosting quality of existing staff and the harmonisation of human resource policies and procedures across the region. Moreover, responsibilities regarding human resource management and development shifted between regional and country level over the years. SNV indicated that the quality of its advisory services is rooted in: 32 • the selection and recruitment system of advisory staff; • the coaching (by seniors, by special staff [SPARKS]) both in technical subjects and working methods; • the training of staff and access to ‘knowledge networks’ and virtual groups (D- groups); • well developed staff performance monitoring.

31 Interview SPARK-advisor institutional development, La Paz. 32 SNV presentation primary process, November 2005.

74 Evaluation SNV The competency and career management procedures are largely set at corporate level, but implementation does show some variance by country. With respect to recruitment, coaching and training of advisors, the following has been observed during field visits: • SNV applies modern and sophisticated systems for staff recruitment. In the three countries visited, SNV has devoted a lot of attention to the recruitment process. Newly recruited personnel had to qualify not only on professional experience and knowledge, but also on attitude and vision. An active policy was pursued to incorporate more national staff (“home country nationals”), more female staff and to achieve multicultural diversity at portfolio level. Procedures may take the form of block recruitment 33 , junior programme, head hunting, and master classes. Interview processes have been streamlined, and candidates are formally assessed during a day long interview process. All job descriptions count with clearly defined competency levels; • SNV applies modern tools for (annual) staff performance monitoring. All advisors count with a Personal Results Development Plan and the Portfolio Coordinator controls the accomplishment of that plan. Thereto exists and a formal Performance Annual Review (PAR). This focuses on competencies, and includes areas for further learning. Although the PAR is primarily undertaken by the PC, it uses a feedback informant process whereby peers and subordinates are able to contribute (triangulation process); • For staff seeking promotion, SNV applies formal competency appraisal processes. Key promotion nominations are made and championed by line supervisors, and formally considered at regional management team meetings (at least in East and Southern Africa). Promotions fall into three categories; i.e. vertical, horizontal, and changed job type; • SNV applies specific programmes aimed at individual, team and organisational learning. One example is the “advisory practice learning programme”: formal exposure to new and emerging ideas around advisory practice, management and teamwork. In all three portfolios a team induction programme is organised for new advisors, followed by the formulation of a work plan and a probation period of 6 months;34 • The coaching on technical aspects (subject matter) depends on the initiatives and demand by the individual adviser (at all levels, but mainly junior and ‘medior’). The use of D-groups varies according to the personal interest of advisors. Some are keen users, while others mentioned overkill in information. Some advisors interviewed expressed criticism with regard to (the lack of) coaching and subject matter training received by the so-called SPARKS. 35

Over the period under review, the staff overhaul has been substantial. European SNV staff may have left SNV out of own initiative, or were recommended to leave since their profile did not match the new requirements anymore. There has been a deliberate policy to change the staff composition towards more “home country nationals” replacing

33 The first stage of the recruitment process in Bolivia was subcontracted to a private firm. 34 In Kenya, a “job rotation project” is envisaged for 2006 to enable staff to gain new exposure in different portfolios. 35 SPARK are experts that come from outside SNV (with one exception). They work on 2-year contracts. Comments concerned ‘the distance between a regional advisor and the reality at portfolio level’; ‘subject matter specialists lacking experience in developing countries’.

Evaluation SNV 75 European (Dutch) advisors. In the three portfolios visited the average working experience of the advisors with SNV was less than three years.

In all three portfolios the relation between home country national (HCN), third country national (TCN) and European experts (EUN) changed considerably over the period: HCN replace EUN, whereby SNV aims at a balance of 60 percent HCN per portfolio (up from 40 percent in 2001).36 Having a larger share of HCN advisors implies special human resource management requirements. First of all, the remuneration level is set in accordance to the market, while national labour legislation has to be respected. For example, SNV Kenya reviews every three years the remuneration level against benchmarks obtained from the accountancy firm Price Waterhouse Coopers. SNV aims at paying competitive salaries, without distorting the market. In the case of North Rift Portfolio this was a point of concern, since the highly competitive environment made it difficult to retain high quality advisors. A different aspect was mentioned in Santa Cruz, where one of the clients felt insufficiently attended ‘by only Bolivians’ and observed that innovation has to come from international experts.

4.2.5 Assessment

SNV does not count with explicit criteria for country selection. During the period 2002- 2005, SNV has embarked on activities in a number of countries justified by either internal efficiency or by ‘cross-border synergy’ motivations. This choice was not based on congruence with the Dutch bilateral cooperation. To the evaluators it is not clear whether correspondence with the Ministry of Foreign Affairs bilateral programme is either expected or desired or that, alternatively SNV is expected to develop activities in those countries lacking a bilateral programme or that SNV enjoys full autonomy in its country selection.

At regional (supranational) level, SNV defines broad intervention strategies. At country and portfolio level the context analyses form the basis for the identification of both practice areas (themes) and (constellations of) clients. The quality of the country context analyses reviewed by the mission is solid. In Mali, the country context analysis was based on a broad participatory process in which many existing clients and development partners joined in. Although a participatory process is laudable, the ‘shared donor perspective’ has influenced on both the thematic choices and client selection. A common and shared vision among development professionals (‘group think’) may have prevailed over sound factual analysis.

In all three portfolios there is a tendency to identify strategic clients, as well as strategic partners in constellations, preferably by (sub-) practice area. Working with interconnected clients increases the probability that the perceived impacts may be achieved. This concept of clients in constellation has been fully implemented by the portfolios visited in Bolivia and Kenya. In Mali the final selection was path dependent in the first place.

36 In its comments on the concept draft report SNV (3 rd February 2006) stated that there are no indicators in force in the distribution between HCN and EUN. However, a relation of approximately 60-40 was mentioned in a briefing by SNV in December 2005 and happens to be the indicator in use at portfolio level.

76 Evaluation SNV

There is no quantitative evidence that selected clients are the ‘best’ selection possible as compared to any other possible selection of actors. From a theoretical perspective there is no system in place that allows comparing one constellation of clients in a practice area with any other alternative constellation. So it is hard to argue that the optimal selection has been made in terms of probable contributions to poverty reduction and good governance. This comment admits that from a practical point of view such a comparative analysis is very hard to put in practice, apart from the activities related to market access, where use can be made of cost benefit analysis and / or cost – effectiveness studies.

In all analyses reviewed, ample attention was paid to the cross-cutting criteria of gender and environmental soundness.

In all three portfolios visited there has been a gradual change in the portfolio of clients; the introduction of the 2001 Strategy was by no means a ‘shock therapy’ as far as it concerns the kind of clients selected.

In Bolivia and Kenya, the identification process has lead to a well-perceived focus on core problems (land use and ownership in Santa Cruz, pastoralist perspective and peace building in North Rift) and portfolios that consist of interconnected clients. By embarking on services to clients involved in conflicts, SNV has shown courage in its choices (conflict mediation Bolivia and Kenya).

The client selection was made more demand driven during the years 2002 and 2003, but with the definition of the practice areas, client selection became more pro-active. The SNV policy of 2005 triggered an even more pro-active approach, although at portfolio level the demand seems to have increased as well, mainly from macro-level organisations.

Overall, SNV delivers well-perceived services that are framed by strategies at various levels, underpinned by context analysis, by needs assessments and agreed upon with the client.

SNV’s major asset is the quality of its staff. SNV has paid considerable attention to improve its human capital. Advisors appreciate SNV’s active competency and career management. In 2005 SNV counted with more HCN, more female staff, sound recruitment and career management, Personal Results Development Plan, Training, “SNV-Campus”, virtual knowledge groups that all contribute to an upgrading of the quality of staff.

The larger share of HCN in the staff makes SNV’s quality more dependent on the conditions (and competitivity) for high level professionals in the local market. It is important to safeguard intercultural mix in each advisory team, also from the perspective of client expectations and innovative attitude.

Evaluation SNV 77 4.3 Output

The Subsidy Agreement between the Ministry of Foreign Affairs and SNV refers to the expected output of SNV by expecting for 2006 the development of an advisory practice (characteristic 2), the development towards a knowledge centre of national and international significance (characteristic 8) and in addition, the provision of incentives to south-south knowledge development (characteristic 9).

This section deals with: • the Advisory Plan as the hinge between input and output; • the Client Satisfaction Review that is supposed to reveal whether clients are satisfied with both input and output; • the international acknowledgement of SNV (and opinions of embassies and partners thereof), and on south-south knowledge development.

The Advisory Plan If a client has been selected and there is sufficient common ground to establish a relation, SNV formalizes this through an Agreement (Memorandum of Understanding). The Agreement refers in general terms to the responsibilities of both SNV and client. In most (but not all) Agreements reference is made to the commitments of the client in terms of its own (non-financial) contributions. SNV does not apply a corporate standard for these Agreements, due to different conditions per country. The Agreement signed between SNV and clients does not define neither the conditions that determine the end of the process of capacity building nor the end of the relationship. In practice this is determined by default (when SNV stops providing services).

In terms of operations, the hinge that links input with output is the Advisory Plan. This is the principal instrument used to define SNV’s advisory services to a particular client. A single Advisory Plan may be designed for either a smaller client or a client with only one or just a few ‘assignments’, while various Advisory Plans are formulated for clients with various assignments.

Introduced as instrument in 2003 in the portfolio, SNV advisors were not familiar with its use and its potential as a base for monitoring and performance measurement. During the first years (2003, 2004) the expected results of the Plans were (in some cases) formulated in terms of SNV objectives (in stead of the expected results of the client). Over time special attention has been paid in improving the Plan. The Santa Cruz portfolio applies a standardised form for the Advisory Plans that has been modified over time and contains information on input, outputs and outcome. 37 Inputs are expressed in advisory days and sometimes in small financial support. Most Advisory Plans have a relatively short time horizon (usually one year, but they may be revised after four months).

In principle, it is a powerful instrument, being the starter document that may provide the baseline concepts and intentions that enable subsequent monitoring, internal reviews, etc.

37 The Santa Cruz portfolio considers outcome as ‘expected results’ and deviates slightly from the corporate terminology. Source: PC comments on draft Bolivia report Ecorys & SEOR, 2005.

78 Evaluation SNV In its present (2005) use, the Advisory Plan is mainly an instrument that formalizes the advisory relation between SNV and the client.

Client satisfaction survey The main SNV instrument used at output level is the client satisfaction survey (or review). The client satisfaction survey has been designed at corporate level and is structured around five core questions (see box 4.2). The survey is carried out by the directly responsible advisor(s). In the North Rift portfolio, a client satisfaction review was conducted by an external consultant in order to obtain information not influenced by the presence of SNV. 38

The SNV website states: “Our clients are our ‘raison d’être’. Their opinion on our work is important to us. Through satisfaction reviews we ‘measure’ our clients’ satisfaction with the quality of our service, our approach, and relevance of our service. However, whether the client’s capacities have indeed been strengthened is beyond our control. At the end of all assignments we review the capacity strengthening objectives that were laid down in the initial contract, and assess how well we have reached those objectives”.

Box 4.2: Client satisfaction survey

The five core questions are: • How do you assess the quality of our advisory services? • How do you assess our focus, stile and way of communication with you? • Do you think you are in control of the collaboration with SNV? • How do you qualify the learning process of the collaboration with SNV? • How relevant are advisory services by SNV for your objectives and targets?

On each of these questions a score can be given from 1 to 4 (below expectations, almost satisfactory, satisfactory, exceeds expectations). So the maximum score is 20. Clients may give comments and explanations in the margin of each question. Both client and directly responsible advisor(s) respond to the survey questions and give their scores independently from each other. Scores by client are forwarded to the country office and to headquarters for different levels of aggregation. Satisfaction scores at corporate level are used in the Annual Report. Portfolios conduct the standardized surveys since May 2003.

Source: SNV Bolivia. Note: translated from Spanish, so not the exact formulation in English

Analysis by the portfolio team Santa Cruz 39 reveals: • After initial lower scores (2003) (in part due to dissatisfaction as a result of the change from permanent advisors to part-time advisory support), all organisations tend to give higher scores since 2004;

38 M. Were. Client Satisfaction Progress Monitoring and Evaluation, Feedback and Summary (draft). December 2005. 39 Portfolio presentation Santa Cruz, 9th December 2005.

Evaluation SNV 79 • There is no substantial difference in the scores among the five main questions. Of the maximum 20 points that can be achieved, since September 2004 the sum of scores of all clients tend to oscillate between 14 and 16; • More useful than the scores, are the clients’ comments on each of the five questions. Frequently these comments are not consistent with the ‘score’ given; • Both client and advisor give a score. Overall, the difference is not relevant, but shows a slightly lower appreciation by SNV staff.

In principle, clients could use this satisfaction review to scrutinize SNV’s performance against the Advisory Plan, for example by comparing the number of advisory days agreed upon with the real time ‘received’. Or by checking the subjects mentioned in the plan against ‘the delivery’ by SNV. However, none of the clients interviewed used the satisfaction reviews in such a way. 40

In all three portfolios visited, SNV clients gave consistently high assessments of the quality of SNV services. All clients visited appreciated the quality of SNV services, in particular the quality of the participatory approach and of the diagnosis of the context and capacities of the client. In Mali, the high quality in particular with respect to local governance was mentioned. Some clients however still have more or less explicit expectations about the possibilities of SNV to increase their access to external funding, despite the clear messages of SNV-advisors that SNV does not provide financial support. 41

This high level of appreciation does not exclude criticism: in Santa Cruz, some clients expressed doubts about the sustainability (for example in the context of the frequent changes of government), while in Kenya clients were concerned about the efficiency, i.e. that advisers could be overstretched and therefore did not always delivered as agreed upon. Critical comments were made on the reliability (in time, in frequency), and the duration (plans, case studies).

The client satisfaction survey is aimed at clients. But are SNV’s partners and other related actors equally satisfied with SNV? The following summary presents the main opinions expressed in three countries to the evaluation team by the Royal Netherlands Embassies, by authorities at local and national level and by direct partners and allies.

40 PC Santa Cruz staff indicated that some clients do control the SNV input delivery. 41 One of the interviewed clients in Mali stated that if his organisation would not access any external funding in the near future, he considered the services by SNV as not having produced the expected results.

80 Evaluation SNV Royal In Bolivia, up to 2003 SNV implemented projects on behalf of the embassy. Between 2003 Netherlands and 2005 there were some direct working relations between the embassy and SNV, but no Embassies use was made of the waiver facility up to mid 2005 (an eventual position by SNV in watershed management in Cochabamba). SNV’s choice for areas of action (such as conflict mediation) is appreciated, but the embassy has questioned SNVs complementarity to the bilateral programme. An inconsistency was observed between the sectoral approach (where technical assistance is delivered untied and out of joint donor funds) and the SNV ‘tied aid’ modality. In Mali, the Embassy considered SNV a well-managed organisation with competent advisors. SNV is a source of information and a linking pin between Dutch co-financing organisations (Cordaid, Icco, Novib) and the Malian public sector. SNV-services related to context analysis, organisational assessment, strategic and operational planning (with respect to local governance) were mentioned as fields where SNV has confirmed its competence. According to the RNE, the competencies of SNV are not yet proven in economic development and natural resource management. In order to develop gender aspects, an agreement has been signed (October 2005) between MATCL/DNCT, RNE, SNV and CCC Koulikoro, Dioila and Banamba. In Kenya there has been sporadic contact between the Embassy and SNV (the Dutch bilateral aid was suspended between 1999 and 2004). Direct funding by RNE was considered uncertain, since RNE participates in basket funding arrangements that tender competitively for services. National and In Bolivia, three vice ministers interviewed were well informed about SNV’s activities and local highly appreciated SNVs services, mainly in the area of decentralisation and land policy. In authorities Mali, the ministry of territorial administration (MATL) lauded SNV’s active contributions to the formulation and design of the Malian decentralisation process. In particular, the support to the CT is considered a significant contribution. Strategic In Bolivia, partners and allies (Misereor, DED, Participación ciudadana) were highly partners and appreciative of SNV approach and working methods. SNV does ‘connect people’. However, allies the following critical comments were made as well: ‘too many areas, a lack of technical focus’, and ‘the risk to become superficial’ and some recommended “to stick to local organisations rather than supporting ministries at central level”. In Mali (Helvetas, GTZ, Swedish co-operation) considered SNV providing complementary services. They exchange strategies and tools and make use of each other’s tools. Helvetas was confident that further cooperation would be developed in economic activities. The Swedish International Development Agency has signed a financing agreement (€ 4 million over 3 years) with a consortium, lead by SNV; that is a sign of the recognition of SNV’s expertise in local governance. In Kenya (Centre for Minority Rights Development, the Impact Alliance, the Shadow Women’s Parliament and the Municipal Development Partnership). Some of these partners are clients at the same time. The Impact Alliance is a network of 170 capacity building organisations worldwide driven by 10 partner organisations, including SNV. The South East Africa Branch considers SNV a highly quality service provider. The Municipal Development Partnership felt that SNV was competing with existing research institutes on knowledge areas and a clear distinction should be made by developing knowledge and applying knowledge in practice.

Evaluation SNV 81 International acknowledgement in thematic areas As indicator for the quality of the output, the national and international recognition of the services provided has been assessed on the base of invitations to either publish articles, to make presentations or to participate in international events.

The frequency in which Portfolio Coordinators received invitations for publishing papers on their (practice) area of expertise, or to make presentations on (inter)national conferences, seminars or workshops (not organised by SNV) was surveyed among all SNV portfolios. The results are presented in Table 4.5.

Table 4.5 Invitations to Portfolio Coordinators for (inter)national presentations

Regional average Share of portfolio coordinators 1 who were invited for presentatios/papers in 2005 Never Once 2-4 times > 4 times Asia 11% 16% 47% 26% Western & Central Africa 14% 24% 45% 17% East & Southern Africa 11% 17% 61% 11% Latin America 6% 31% 31% 31% Albania 50% 0% 25% 25% Notes: 1 Share of the total number of portfolio coordinators in the region who answered the question. Source: Survey ECORYS & SEOR, 2005; region average is based on country level averages, which are in turn based on portfolio data.

Survey data indicate that most portfolio coordinators (on average 64 percent worldwide) are invited at least 2 times a year to publish a paper or make a presentation. Many respondents added that they receive more invitations than they can handle. PCs indicated to stimulate younger advisors to expose themselves to international experience as part of their own knowledge development.

Also in the three portfolios visited, SNV is frequently invited to share its experiences to a national or international audience. A more detailed analysis of these invitations 42 revealed that most interest is shown in SNVs participatory approach, in its working methods. The relative share of invitations on technical aspects of the practice areas (possibly with exception of local governance) is small. In Mali invitations were reported in the area of planning of agricultural development, self-evaluation (of the CTs), the role of NGOs in the decentralisation process and inter-municipal management. In Bolivia, SNV has become an acknowledged partner on topics like local governance, local economic development, communal forestry management, the rights of indigenous people, social control and conflict management. The North Rift portfolio reported that 13 presentations were made in 2005, amongst them at international level, such as the partnership in the area of local governance with Kings College in London. The partnering with the Impact Alliance allows knowledge to be shared, augmented and disseminated.

42 Bolivia and Mali field visits, 2005.

82 Evaluation SNV SNV is active in publishing its experiences. Most of those publications (see for publications produced at portfolio level annex A) have a rather local character. In consequence, SNV’s publications bear titles that directly refer to specific population groups or geographical areas (i.e. Samburu District Peace Committee; Lake Turkana fish study; livestock in Samorogouan). Less site-specific publications have been elaborated on local governance and decentralisation (Bolivia and Mali). In general, few SNV publications have generalized on thematic fields, based on the case studies. One advisor commented on that issue “we all produce bricks, but SNV is reluctant in constructing the house”.

South-South knowledge development The south-south knowledge development has been taken as an indicator for SNV’s ‘brokering’ knowledge as output.

SNV is “a knowledge organisation that actively promotes knowledge development and brokering” through: • development of “cutting edge” methods and tool critical to scaling success in poverty reduction; • relevant trend analysis, forecasting, modelling reports to increase awareness; • leveraging and sharing proven local, regional ad global strategies for success/ impact; • brokering spaces for collective, high-impact learning.

Bringing these generic intentions together at portfolio level results in a complex jigsaw puzzle. What is exactly SNV’s role in topics like forecasting or modelling (of what?) or what kind of knowledge is exactly being ‘brokered’? How does ‘brokering’ knowledge relate to ‘developing knowledge centres’? Or what should be understood by “leveraging global strategies”?

At portfolio level the translation is that advisors do not pretend to have all subject matter knowledge in their practice area (say forestry) “in house”, but that they are able to connect clients with those specialist knowledge carriers that do have the required knowledge. At portfolio a lot of attention is paid to ‘connecting people’.

The Subsidy Agreement does not refer to the concept of ‘knowledge brokerage’ but mentions the south – south knowledge development that can be considered as component of the knowledge brokerage.

In the survey among all SNV portfolios the question was raised what share of the budget was destined to south-south contacts. Portfolio coordinators applied different interpretations. 43

The outcome of the survey seems to suggest that more attention is paid to south-south contacts in East and Southern Africa than elsewhere (see table 4.6). Field visits to three

43 Most portfolios interpreted the concept as ‘cross border contacts between clients on issues of common interest’ (as was the interpretation of the evaluators). Others classified all contacts between clients at a national level as ‘south-south’ contacts as well (Albania).

Evaluation SNV 83 portfolios seem to confirm this impression, but the data might be misleading due to lack of conceptual uniformity.

Table 4.6 Share of budget assigned to South-South contacts

Region average Share of budget assigned to South-South contacts (2005) Asia 1 10% Western & Central Africa 2 8% East & Southern Africa 3 24% Latin America 4 3% Albania 5 46% Notes: 1 No data available for the two new portfolios in Cambodia. 2 Data for the two new SNV portfolios in DR Congo have been excluded. The figures reported by the portfolio coordinators in Mali were very high as compared to neighbouring countries (on average 44%). 3 No data available for Mozambique and Uganda. For Ethiopia and Tanzania only national portfolio data are available. In Zambia the national portfolio reported 60%, whereas the other 2 portfolios reported 0%. 4 For Ecuador only data for 1 portfolio were available. 5 The national portfolio responded the question on the base of all contacts between national organisations. Source: Survey ECORYS & SEOR, 2005; region average is based on country level averages, which are in turn based on portfolio data.

Of the portfolios visited, the Santa Cruz portfolio argued that south-south contacts have become an integral part of all activities as consequence of the regional Latin America approach. Among the clients interviewed, the south-south contacts (perceived as cross border contact) happened to be rather exceptional. This seems to coincide with the relatively low figure for Latin America in Table 4.6.44

Capacities and gender SNV defines gender equity as an important aspect of its strategy and interventions with clients. In the Santa Cruz portfolio gender has been defined as an explicit quality indicator. Gender aspects in the contacts with the clients have become mainly a cross- cutting perspective. The Santa Cruz portfolio has only two specific gender assignments. It is noteworthy that the relevant documents between SNV and the clients (MoU, Advisory Plan) reviewed by the mission (Bolivia, Mali) do not refer (Mali) o scarcely refer (Bolivia) to gender aspects. No gender indicators formed part of the Advisory Plans revised. In absence of clear targets, it is hard for SNV to monitor or evaluate effects or impact on gender. That does not deny that effects have been registered, for example in relation to the representation of the indigenous people in Santa Cruz. SNV-Mali is aware of the flaws in dealing with gender aspects and has included a gender strategy in the action plan for 2006-7 and annual plans 2006 of all portfolios.

44 In its comments to the concept final report (3 rd February, 2006), SNV stated that all clients visited “were involved in south- south contacts” and that the low percentage of budget allocation to these contact reflects a different interpretation of the definition so that no conclusion can be drawn in this respect.

84 Evaluation SNV 4.3.1 Assessment

The change process towards the new practice of provision of advisory services has been completed. In Bolivia and Kenya the modality was not new in 2002, so it did not imply sudden changes. By 2005, in all three portfolios the clients were at least for 80 percent clients at meso level.

Agreements The Agreements signed between SNV and its clients formalize the relations between the parties. They have been instrumental to the awareness among SNV advisors that the relationship between SNV and the recipients had changed and that advisors had to think in terms of ‘clients’. The Agreements contain neither an exit-strategy nor an ex-post commitment. The duration is determined by the SNV input delivery. Since results of the advisory services are sometimes only noticeable after some time has lapsed, one would have expected an ex-post commitment for monitoring reasons.

The Advisory Plans The quality of the Advisory Plans varies. Comparison in time series 45 revealed an improvement over time of the logical sequence used in the Advisory Plans, although advisors explained to have difficulties in matching the logical framework sequence with the ripple model concept (see also box 4.3). In addition, advisors have difficulties in assessing their own inputs (for example in mandays), in formulating expected results and in indicating measurable outcome indicators. 46

Client satisfaction reviews The client satisfaction survey has served its purpose in the change process, namely making advisors more ‘client conscious, client oriented’. The reviews have proven to be a useful management information tool and can be considered as forming part of the management information system. The surveys are a suitable tool for matching the advisors’ attitude and way of operation to the expectations of the client. They are a less suitable tool for technical and contents matters of the assignment. If used for statistical registration (as SNV does) too many (cultural) factors may be of influence. The use of the satisfaction scores for accountability purposes (as evidenced in the Annual Report 2004) is not appropriate.

The reviews are carried out with a frequency of four months. This seems to be a good frequency at the start of a relationship with a new client (or a new advisory team), but it becomes a rather empty routine exercise in long standing relations. This is even worse when the survey is carried simultaneously with the end-of-year outcome evaluation.

It is noteworthy that SNV has been able to achieve a high level of satisfaction amongst its clients, even in countries (like Mali) where expectations on eventual funding had been very high (and still exist).

45 The evaluation team elaborated time series of selected activities in Bolivia and Mali. 46 Some advisors interviewed indicated that they lacked both insight and instruments for assessing correctly the advisory needs of clients (although these instruments do exist). Advisors found it hard to assess their own time input. If output is being assessed at all, it is in terms of short term ‘products’ only.

Evaluation SNV 85

International acknowledgement SNV advisors do participate in many international events. SNV has developed a real ‘outstanding profile’ on its participatory methodology of work. Donors and strategic partners associate the word SNV predominantly with ‘local organisations’ and ‘local government’ with a participatory approach and with networking.

While local level experience is a comparative advantage for building expertise in ‘local governance’, that is not necessarily the case with other practice areas. SNV experiences are generated at field level, so are typical ‘case studies’ and therefore ‘site specific’. Knowledge sharing is to a large extent restricted to ‘the inner circle’ of SNV and its direct partners. Knowledge moves laterally and vertically between the various layers of SNV and between SNV and other partners in development. There is, however, only sporadic ‘tapping from’ and ‘delivering to’ local private sector knowledge (universities, consultancy firms).

Whether south-south contacts are being encouraged by SNV cannot be assessed in generic terms. Results from the survey among portfolios suggest that some consider almost all SNV activities as south – south promotion, while other seem to pay less attention. South-south activities –understood as international contacts- among clients is not a predominant feature of the advisory practices. Results and experiences do ‘pass the borders’ as a result of the regional approach, of practice area networks and direct international contacts among advisors. For example, concepts from Bolivia were replicated in Ecuador (economic grassroots organisations) and Peru (conflict management).

4.4 Effectiveness (outcome and results)

The Subsidy Agreement expected that by 2006 SNV would use relevant methods of results measurement (characteristic 4) while the client capacities would have been strengthened in all aspects and in a sustainable manner.

This section exposes the methodology and instruments used by SNV and details whether at portfolio level (of the three portfolios visited) indications exist on the outcome and results.

4.4.1 Methodology used by SNV

The SNV Annual Report 2004 stated: “Our ultimate results are our clients’ results”, and “SNV sees capacity strengthening [..] as a means, not as a goal itself”, and “capacity building must be linked to tangible impacts”. 47 But SNV also indicated that “when embarking on capacity strengthening of local organisations [….] the focus on effect and impact received less attention” (SNV 2005:11,24).

47 Source: SNV Presentation 20th December 2005.

86 Evaluation SNV

SNV has well defined what has to be understood by ‘outcome’: the strengthened capacities of the client organisation(s). SNV distinguishes the following ‘capacities of an organisation’: 48 • the capacity to change, combined with self esteem and capacity to propose; • the development of knowledge (based on human resource development; monitoring and learning capacity); • the capacity to sustain results (managerial and financial capacities).

Equally clear is what has to be understood by ‘effects / results’: the application of these enhanced capacities by the organisation leading to improved performance (for example better service delivery of public entities, better services delivery of NGOs to their target groups, higher volume or profits). So the main question for assessing results is whether the improved capacities have been put in practice by the organisation and whether these capacities lead to better performance of the organisation in relation to its own objectives.

Instruments for outcome and effect measurement For outcome measurement an ‘end of year / end of assignment evaluation’ with a fixed format has been designed and been put in practice in 2004. It is usually the Portfolio Coordinator who carries out this evaluation.

For effect / results measurement SNV counts with instruments as well, but these have been less specific and are not applied in a standard format at corporate level. Although SNV stated in its Annual Report 2004 that “after a few years we sit down with the various stakeholders from the region or practice area [..] and conduct joint evaluations” (2005:12) in practice this has not been common practice. There is no specific evaluation instrument in place for ‘sitting down with stakeholders’.

SNV’s website adds: “We may also conduct studies that challenge or validate our most important assumptions, and analyse how poverty levels have changed and what our relative roles and contributions were” and continues “let us strengthen their [the clients’] capacities to monitor and evaluate progress […]”. This latter statement refers to one of the instruments applied: the training of clients to conduct their own effectiveness evaluations.

All three portfolios visited used the client satisfaction reviews (input-output), the ‘end of year, end of assignment’ internal evaluation (outcome) and conducted case studies (input- output-outcome-result). Only a few clients visited had realised self evaluations. In the three portfolios visited each applied a different mixture of instruments and advisors expressed different visions on how and who should carry out effect evaluation.

In Santa Cruz, the measurement of outcome was perceived as “the mirror of the Advisory Plan” and consisted (next to input – output indicators, such as direct advisory days) of measuring progress on indicators agreed upon with the client; a process indicator on the learning process and discussions by the SNV team, sometimes extended to the directly involved partners. In the Santa Cruz portfolio the outcome measurement pertains

48 Source: Presentation portfolio Santa Cruz, 9th December 2005.

Evaluation SNV 87 to the standard procedures and is being carried out annually (and forms part of the group reviews among the advisors).

To assess results, series of case studies have been elaborated, but only few evaluations were carried out. 49 Local evaluations, practice area evaluations and external evaluations (not mutually exclusive) do exist, but are limited in number and refer –in the case of Bolivia- mainly to (closed) projects (such as the Embassy Funds for local governance).

In Kenya, SNV indicated that the Capacity Development Services approach was still relatively new and that effectiveness and impact can only be measured after a certain time has lapsed. In their view, putting a lot of effort in results measurement would be premature. The North Rift portfolio stressed the point that for results / effects, the clients are supposed to evaluate their own performance. Thereto SNV trains them in using OSA tools, assess their strategic planning targets and in organising opinion surveys. The evaluation team observed that this was taking place, for example at Cheptebo and Keiyo County Council.

In Mali, in the Koulikoro portfolio the evaluation systems in place are based on the models designed by headquarters, but adapted to the needs and realities of the portfolio and clients. At outcome level, this is mainly the end of year / end of assignment review, while for the results the evaluation consists mainly of case studies. Like in Kenya, in Mali SNV has put instruments in place enabling self-evaluation by clients. 50 SNV-Mali indicated to favour additional efforts in conducting case studies and action research, aimed at obtaining a better understanding of the development processes.

4.4.2 Outcome: Capacities strengthened

Has SNV been able to strengthen the capacities of its clients?

In all three portfolios, all clients interviewed manifested that their organisational capacities had improved as a result of SNV services. The kind and character of these capacities varied by kind of organisation. Only some broad generalisations and examples are presented in continuation:

NGOs In Santa Cruz, SNV’s advisories have enabled local NGOs to implement their development programmes related to training, technical assistance, etc., offered to the target group of the NGOs, being mainly the indigenous populations. In the case of support to NGO’s the question remains whether either the capacities of the NGO have to be strengthened (so the NGO can continue providing services to the target population) or that the NGO serves as a channel to reach the target population. While in the past SNV considered NGOs frequently as ‘clients’ with their own objectives and need for capacity building, gradually these are perceived as ‘partners in development’ and allies.

49 According to the SNV website, case descriptions is an activity to which at least 10 percent of the advisory days is being used. 50 The evaluation mission has not been able to confirm the application of these tools by clients.

88 Evaluation SNV In Mali, the work with local NGOs is of a different character than in Latin America. Local NGOs in Mali are still incipient (and usually smaller) with a weak management and financial base. Here, SNVs efforts target more on institutional and organisational development of the NGO rather than on ‘channelling’ knowledge through the NGOs to the target groups. NGOs are more client than strategic partners in Mali as compared to Bolivia.

Public entities Both in Bolivia and Kenya exists the tendency to work more with public sector entities and gradually looking ‘upwards’ to the national level. In Kenya this ‘upward looking’ is a sequential step that results from an ‘upgrading’ in the process of capacity development of organisations (for example economic organisations), while in Bolivia, it is more the result of a deliberate choice for ‘anchoring’ meso level processes (the systems component of the capacity building concept) at the macro level. In Bolivia, a step was made to support ministries and vice-ministries as an integral part of SNVs support to meso level organisations.

As example of strengthened capacities, we mention the case of the support to the Prefecture Santa Cruz. Here the capacities strengthened are: • the Prefecture’s capacity to deal with spatial (territorial) organisation problems based on sound information; the capacity to deal with conflicts in the area of land use and rights (agriculture and forestry); the capacity to communicate on these subjects; 51 • the capacity of participatory planning at meso-level, with a gender and intercultural focus; • general capacities of the Departmental Delegate for Anti-corruption; • the capacities to avoid and to combat forest fires.

In Mali SNV has supported the collectivités territoriales (CTs) in establishing good or better development plans. To count with such a plan was one of the conditions for CT’s to access resources from the national investment fund (ANICT). The CT’s in the SNV intervention zone are able to elaborate these development plans autonomously, although they indicated still to prefer a continuous support by SNV (mission interviews). SNV has assisted CTs in establishing forms of collaboration between municipalities. Six municipalities (Koulikoro, Méguétan, Tienfala, Dinandougou, Tougouni and Nyamina) have together prepared and agreed on a plan to manage the borders of the river Niger and to take measures against border degradation.

Not all capacities are being put in practice automatically. In Mali, the national authorities agreed upon a (SNV proposed) tool-kit for self-evaluation of the CTs, but this instrument has never been applied independently by CTs. 52

51 The Prefecture’s EDAC had a capacity to deal with only 4 to 5 conflicts, while number of conflict had increased to 120 per year. As UDECO (in INRA) there is capacity to deal with 60 conflicts per year (in form of mediation). 52 This may be largely due to the replacements of the elected counsellors after the elections of 2004. It also shows that the (autonomous) application of these tools needs to be monitored.

Evaluation SNV 89 Civil society organisations Whether the capacities of civil organisations were strengthened cannot be contested in a generic way: the array of organisations supported, and within these organisations the array of subjects dealt with is very broad. As example we mention the services offered to the umbrella organisation of unions representing indigenous people from the Eastern part of Bolivia (OICH). Capacities strengthened were (amongst others): • The capacity of territorial organisation: administration, capacity to coordinate, capacity to emit normative regulations for use of communal territory; elaboration of projects, communication and networking; • The capacity of women representatives of their interest group (i.e.‘centrals') within the indigenous population in aspects like planning, organisation, management of information and networking; • The capacity to consider the own cultural identity as an asset: recuperation of cultural heritage by visualisation of own culture.

4.4.3 Sustainability of capacities

Only advisory services that have been effective –in the sense that the capacities have been internalised in the organisation- can be sustainable. Sustainability of the capacities strengthened is not equal to the sustainability of client’s performance that results from the strengthened capacities.

One of the most important sustainability concerns to clients is the financial sustainability of the capacities strengthened. SNV does not provide financial resources. There are basically three mechanisms on which the sustainability of capacities strengthened has to be ensured in terms of finance: • as an effect of the capacity building the client’s financial performance improves, providing the client a better position to either generate addition income (for example in economic activities) or accessing external resources (for example a municipal plan may provide access to a municipal development fund); • SNV mediates in obtaining external donor funding; • SNV mediates in obtaining commercial finance (mainly economic activities).

With exception of Asia, almost all portfolios surveyed worldwide indicated that the capacity building contributes to the client’s own capacity for resource mobilisation. All portfolios answered to also liaise with donors to obtain funding for their clients, while 40 percent of the portfolios answered to look for commercial funding for their clients. Table 4.7 provides an overview by region.

90 Evaluation SNV Table 4.7 Resource mobilisation for clients for sustaining capacities strengthened

Region average Share of portfolios 1 in which resource mobilisation for clients is facilitated by: Internal resource mobilisation of the Intermediation with Intermediation with client organisation itself third parties (donors) commercial financiers Asia 53% 100% 37% West & Central Africa 2 93% 96% 36% East & Southern Africa 83% 100% 61% Latin America 81% 100% 19% Albania 100% 100% 50% Notes: 1 Share of total number of portfolios in region for which the question was answered. Categories are not mutually exclusive, and portfolios could provide more than one answer. 2 Data for the two new SNV portfolios in DR Congo have been excluded. Source: Survey ECORYS & SEOR, 2005; region average is based on country level averages, which are in turn based on portfolio data.

In all three portfolios visited efforts have been made to embed sustainability in the CDS approach itself. The portfolio teams facilitate, but “clients drive the process”. SNV advisors do assess the sustainability of the results of their services through the end-of- year reviews, through contacts after the formal finalization of the assignment and through case studies. If advisors are convinced that capacities cannot be sustained, this might be an argument to discontinue the services to a particular client.

In both Kenya and Mali many organisations still hope for ‘project funding’. Clients may be tempted to abandon the CDS approach and shift to donors that either provide funding directly or provide access to funding if the opportunity arises.

NGOs Lack of financial sustainability was the reason to discontinue the support of the portfolio Santa Cruz to the NGO CEADES. Although the target population supported by CEADES benefited substantially from the SNV advisory services, SNV advisors filled the manpower needs of the NGO. SNV decided to discontinue its support to the NGO as client, but to continue with the NGO as ‘partner’. NGOs need financiers and simply react to (external) funding impulses. So capacities strengthened in a particular subject matter may be easily abandoned when financiers put a different focus.

Public entities The sustainability of investments in human capabilities happen to be rather low in public entities, as a result of rotation of personnel, sometime linked to political instability. In Bolivia, SNV supports the Ministry of Sustainable Development that had four different ministers in a single year (2004/5). And with ministers key officials were replaced as well and new fields of operations identified. In Bolivia, the SNV supported activities with the Delegated Office for anti-corruption (DPA) were not sustained, since the office was abandoned, while also the SNV strengthened capacities in conflict mediation of the Prefecture were transferred to a different entity (INRA).

Evaluation SNV 91 In Koulikoro and Dioila, SNV strengthened capacities of the public sector were considered to be sustained, thanks to the fact that elected counsellors gained an understanding on their key tasks (on planning, civil administration, budgeting, administrative police etc.), while the management of the municipalities had improved (regular meetings, resource mobilisation for development investments). This was supported by well-functioning mechanisms of technical and financial support in the region.

Civil society organisations Capacity strengthening by SNV of civil society organisations has been sustained as expressed by improved performance of the organisations. This applies to both the local governance activities (ACOVICRUZ in Santa Cruz) and the production and marketing organisations (North Rift).

4.4.4 Results: strengthened capacities lead to improved performance by clients

Did the strengthened capacities lead to an improved performance by clients?

SNV does not count with corporate standard instruments for result measurement, apart from the (mostly descriptive) case studies. There are basically two mechanisms in place: • SNV trains its clients so that they are able to either conduct result measurement themselves (for example by systematisations of data), or may contract such services. However, if those clients do so, SNV has no system in place to use this information generated by the client; • SNV advisors, sometimes in a joint effort with partners, assess results and elaborate reports on client performance, for example in the form of a case study. Case studies occupy up to 10% of the direct advisory time per advisor.

The ECORYS & SEOR survey among portfolios reveals a frequent application of auto- evaluations by clients for results measurement. On average in 78 percent of all portfolios worldwide it was reported that a quarter or more of all their clients carried out internal performance evaluation, as shown in Table 4.8.

Table 4.8 Internal performance monitoring and evaluation by client organisations Share of portfolios1 in which clients carry out internal performance evaluation Percentage of clients 0% 1 - 25% 26 - 50% 51 - 5% > 75% Asia 11% 5% 11% 5% 68% West & Central Africa 2 0% 17% 38% 17% 28% East & Southern Africa 0% 28% 44% 6% 22% Latin America 0% 0% 38% 63% 0% Albania 0% 50% 50% 0% 0% Notes: 1 Share of total number of portfolios in the region for which the question was answered. 2 Data for the two new SNV portfolios in Congo have been excluded. Source: Survey ECORYS & SEOR, 2005; region average is based on country level averages, which are in turn based on portfolio data.

92 Evaluation SNV Performance of a client can also be determined by external evaluations or audits (in particular the broad ‘value-for-money’ kind of audits). As far as known, SNV does not make use of external audit data for assessing ‘performance results’ of its clients. However, portfolios worldwide do register that the majority of the clients are subject to external audits. In Asia all portfolios reported that at least 75 percent of their clients were subject to external audit (Table 4.9).

Table 4.9 External performance evaluation or audits of client organisations

Share of portfolios 1 in which clients are subject to external audits Percentage of clients 0% 1 - 25% 26 - 50% 51 - 5% > 75% Asia 0% 0% 0% 0% 100% West & Central Africa 2 0% 31% 38% 17% 14% East & Southern Africa 0% 11% 11% 33% 44% Latin America 0% 0% 6% 56% 38% Albania 0% 50% 50% 0% 0% Notes: 1 Share of total number of portfolios in the region for which the question was answered. 2 Data for the two new SNV portfolios in Congo have been excluded. Source: Survey ECORYS & SEOR, 2005; region average is based on country level averages, which are in turn based on portfolio data.

Correlation tests show that the share of portfolios in which between 51-100% of the clients are subject to external audits is positively correlated with the share of government clients, both on country- and regional level.53

Contrary to what is suggested by table 4.8, the application of self-evaluation and the analysis of external audit reports for performance analysis of clients were not or hardly registered in the three portfolios visited. In the North Rift self-evaluation was promoted by the portfolio team, but hardly applied by clients; in Mali it formed part of the instruments delivered to CTs, but were not applied. In Santa Cruz self-evaluations were rare exceptions. 54

The absence of an established system for results evaluation does not imply that ‘nothing can be said’ about client performance. Based on case studies, internal reporting by SNV advisors and interviews, the following examples of improvements in performance can be mentioned:

NGOs: In Mali, the NGO-network for urban economic development 55 achieved a level of capacity that has enabled to negotiate funding independently. Several NGO’s, clients of SNV have organised themselves in two networks of service suppliers (receiving financial support from Cordaid).

53 In many countries, all public entities are subject to external audit by the Supreme Audit Institute. 54 In Santa Cruz, the umbrella organisation of transparency committees (ACOVICRUZ) conducted an internal SWOT analysis. 55 APROFEM, GUAMINA and GRIDAC.

Evaluation SNV 93 Public entities In Santa Cruz, the Prefecture (and later INRA) is now able to attend conflicts on land use in a well-informed manner, using mediation and negotiation techniques instead of power. And the Prefecture is able to plan in a participatory manner, aware of the position of vulnerable population groups in society. As a result of having applied such methods, its Plan is now acknowledged with authority by various organisations, including the municipalities who have been made co-responsible for the planning process.

In Mali, the CTs have obtained a stronger capacity in promoting local development, while elected counsellors obtained knowledge and experience in the management of municipal development programmes, procurement and contract supervision.

Civil society organisations In Santa Cruz, the association of the Comité’s de Vigilancia ACOVICRUZ is capable of carrying out its corresponding representative tasks, as well as to provide services and assistance to its member Committees. It has gained general recognition (of about 50 of the 56 municipalities in Santa Cruz).

Representative organisations of indigenous people (OICH) are now better equipped to defend their territorial rights, to solve internal and external conflicts on land use and rights. The indigenous population has re-gained proud on its cultural heritage, expressed by music, handicraft and reconstruction of the own history. This cultural heritage is now being recognised by the Bolivian national community, and exposed by the Municipal Historic Museum.

In Koulikoro, the sand and grit operators have established a Union of organisations, allowing a better organisation of the produce chain. They understand the importance of good quality products and to set their prices right. This has enabled to benefit from public works contracts as providers of raw material. The forest operators of Dioila have achieved to set the boundaries of the forests and apply better wood cutting techniques, allow regeneration of trees. They have established contacts with regional service providers and wood buyers.

Overarching results However, results are not restricted to the level of individual clients and individual advisory services, but just -as consequence of the inter-connectedness - the fruit of joint and simultaneous actions. It is at this ‘combined’ level that self-evaluations by clients do not provide information anymore and are not sufficient to produce the required insight in results.

The PC of Santa Cruz elaborated a paper (Dec.2005) pinpointing results emerging from interconnected actions and presented –amongst others- the following results: • Public recognition, acceptance and incorporation of cultural and societal diversity; a general public and political recognition of what community management of forest is all about and how that relates to the rights of indigenous people; • Consolidation of indigenous lands, managed by its original population; • Improved transparency in the management of public resources at local level (fewer cases of corruption; active provision of information by municipalities).

94 Evaluation SNV

4.4.5 Assessment

In none of the three portfolios visited a logically integrated system was operational to monitor and evaluate the output-outcome-effect-impact chain. 56

SNV has developed instruments at corporate level for outcome measurement. The end of year / end of assignment evaluation is carried out, while also ‘peer reviews’ take place. The end of year review is in fact little more than an extended interview that shows some overlap the with client satisfaction survey. In the day-to-day practice, the ‘outcome’ is hardly a point of discussion, but the interview focuses on the relationship between the client and SNV.

The peer reviews do involve ‘peers’ and hardly outsiders. The ‘peers’ are mainly other SNV staff (for example from another portfolio) and directly involved partners. Since all – or most- participants in the review have direct relations with the client (directly or indirectly) and join the same experiences, there is a considerable danger of ‘group think’.

At portfolio level there is sufficient hard evidence (case studies) that capacities of clients have been strengthened (outcome).

To measure results SNV applies a mixture of less defined instruments, such as the auto- evaluation by clients; case studies; and selected external evaluation.

While the self-evaluation by clients might be the correct instrument for the individual client, it neither deprives SNV from using this information, nor does it provide insight in results at a higher level of aggregation. It is exactly at the level of interconnected results (results of constellations) that one would have expected SNV initiated evaluations. Apart from a few exceptional studies, this kind of evaluation (for example by (sub-) practice area) has not been put in place (yet).

Most case studies reviewed by the mission were rather descriptive and site specific, lacking underpinning research.

The absence of solid (external) evaluations and research does not rule out that at field level strong indications exist for the results produced by SNV delivered services.

4.5 Impact

4.5.1 Methodology used by SNV

The SNV Annual Report 2004 raised the question whether “we are accountable for the services we deliver (how many organisations we supported, numbers of trainings, etc), or

56 In 2003/4, 8 documents/instruments concerning the renewed SNV planning and monitoring cycle and related tools were forwarded from HQ to CD level and subsequently to portfolio level, including guidelines for measuring output and outcome.

Evaluation SNV 95 whether we are accountable for contributing to poverty reduction?” and adds “through research and joint evaluations, we can make plausible claims as to how we contribute to poverty reduction. The research need not be continuous or every time we are involved, as this would disrupt the cost-benefits balance and further question the effectiveness of money spent”.

Up to 2002, SNV external evaluation aimed at measuring impact at a country level. SNV introduced a change of focus towards the practice area. It was envisaged that three till four studies would be carried out per year. In 2002 country studies were still elaborated for Ethiopia, Nepal and Albania. In 2003 no country evaluation were conducted anymore, while in 2004 the studies by practice area started, with a study of communal forestry management in Latin America and a study on local governance in Western Africa. In 2005 a study on local tourism was ongoing. In the Santa Cruz portfolio, advisors were of the opinion that the practice area impact evaluations would be replaced by so-called ‘impact area’ studies, as defined at a regional level (late 2005). These ‘impact areas’ do not refer directly to the practice areas, but relate to impacts like income or employment generation.

In a recent draft policy paper 57 SNV has described the relations between strategy, client and assignments. It states that: “A high level of consensus was achieved about a simple model of SNV’s primary process in which strategy, client accounts and assignments are logically interlinked and related to various levels at which we want to demonstrate results”. According to these documents, SNV applies three methods to ensure impact: • It optimises the probability of producing impact by sound client selection; • It disseminates positive results and experiences; • It assesses impact at assignment, client and strategy level by evaluations. So, it could be assumed that at portfolio level impact studies are being conducted, for example at the level of single assignments or clients.

Selection As indicated in section 4.2, the selection of clients is well perceived and clients are gradually more selected in a constellation, what increases the probability that impact will be produced.

Dissemination SNV publishes a large amount of materials and disseminates this information in both paper and digital forms and the content is exposed during seminars, meetings, etc. In general terms, dissemination of experiences contributes to accumulation of knowledge, and replication of successful methods does take place. Whether the dissemination of that material really contributed to the desired impact has not been studied by SNV.

Impact evaluations In the survey among all portfolios, roughly about one third of all portfolios (28 percent) indicated to conduct either always, or usually impact assessments of the contributions of clients to the overarching objectives. About half of all the portfolios in West and Central

57 SNV’’s Primary Process, Essential steps. Presented to the extended Board Meeting 3-4 October 2005.

96 Evaluation SNV Africa and well over a third of all portfolios in Latin America answered that impact studies are never carried out. See Table 4.10.

Table 4.10 Frequency of impact assessment

Share of portfolios 1 in which impact assessment is carried out: Always Usually Some times Never Asia 17% 39% 39% 6% West & Central Africa 2 24% 7% 21% 48% East & Southern Africa 0% 22% 50% 28% Latin America 0% 0% 64% 36% Albania 0% 33% 67% 0% Notes: 1 Share of total number of portfolios in the region for which the question was answered. 2 Data for the two new SNV portfolios in Congo have been excluded. Source: Survey ECORYS & SEOR, 2005; region average is based on country level averages, which are in turn based on portfolio data.

Only of those portfolios that had indicated to conduct impact evaluations, most indicated that such an exercise was a joint activity between SNV advisors, clients and external evaluators. Only in Latin America and Asia, there is always an external evaluator involved. The role of the beneficiaries in impact studies seems to be less explicit (see Table 4.11).

Table 4.11 Implementation of impact assessment

Share of portfolios 1 in which impact assessment is carried out by: SNV advisors Clients Beneficiaries of clients External evaluators Asia 71% 71% 47% 94% West & Central Africa 2 93% 73% 27% 60% East & Southern Africa 77% 77% 38% 77% Latin America 22% 44% 33% 100% Albania 67% 67% 0% 33% Notes: 1 Share of total number of portfolios for whom the question was answered and in which impact assessment is carried out. Since categories are not mutually exclusive, more than one answer was possible. 2 Data for the two new SNV portfolios in Congo have been excluded Source: Survey ECORYS & SEOR, 2005; regional average is based on country level averages, which are in turn based on portfolio data.

Of those portfolios that had indicated to conduct impact evaluations, most indicated to do so within the first two years after completion of the advisory services. Only in Western and Central Africa about a third of the portfolios answered to conduct impact evaluations when more than two years have lapsed since the end of advisory contract (Table 4.12).

Evaluation SNV 97 Table 4.12 Time lapse between completion of advisory contract and the moment of impact assessment

Share of portfolios 1 in which impact assessment is carried out after the completion of an advisory contract with a client After 1 year After 2 years After > 2 years Asia 2 24% 59% 12% West & Central Africa 3,4 27% 27% 33% East & Southern Africa 5 62% 15% 0% Latin America 6 33% 11% 0% Albania 7 67% 0% 0% Notes: 1: Share of total number of portfolios for whom the question was answered and in which impact assessment is carried out. 2 This question was answered by 16 out of the 17 portfolios in which impact assessment is carried out. 3 Data for the two new SNV portfolios in Congo have been excluded. 4 This question was answered by 13 out of the 15 portfolios in which impact assessment is carried out. 5 This question was answered by 10 out of the 13 portfolios in which impact assessment is carried out. 6 This question was answered by 4 out of the 9 portfolios in which impact assessment is carried out. 7 This question was answered by 2 out of the 4 portfolios in which impact assessment is carried out. Source: Survey ECORYS & SEOR, 2005; region average is based on country level averages, which are in turn based on portfolio data.

In the field visits, both in Bolivia and in Mali, the evaluators conducted an analysis of the planning process in relation to the subsequent stages of monitoring and evaluation. In Bolivia, the analysis focused on the form and contents of the Advisory Plans, while in Mali an analysis was made on the kind of data that may enable impact assessment, such as the basic questions “who, where, when, what (and how good), how many”). The conclusion of the study in Mali58 on the advisory services was that none of the planning documents comply fully with all the required SMART (and QQTTP) indicators. While the “who and where” questions were properly addressed, the ‘what’ (quality), “how many” (quantity) and “when” (the moment of the perceived impact) were all neglected.

Since impact assessments are demanding, fastidious and sometimes expensive exercises, these should not be carried out as a simple routine, but on a selective base only and forming part of a learning process. Well-selected progress indicators on key characteristics of the portfolio or practice area could provide meaningful insight as well. A few selective milestones in terms of poverty reduction and / or good governance could form part of an Advisory Plan and could be used at different stages in the advisory process. Setting correct milestones is a difficult task. On average 86 percent of all portfolios worldwide reported to set milestones at least some times (Table 4.13).

58 The analysis comprised the “Collectivites Territoriales Koulikoro; Collectivites Territoriales Dioila; Union des Exploitants de Sable et de gravier (UESGK) Koulikoro; Structures Rurales de Gestion de Bois (SRGB), Dioila.

98 Evaluation SNV Table 4.13 Milestones in terms of poverty reduction and/or good governance, against which progress is being measured

Share of portfolios 1 in which progress ‘milestones’ is used: Always Usually Some times Never Asia 33% 39% 28% 0% West & Central Africa 2 30% 19% 22% 30% East & Southern Africa 6% 59% 35% 0% Latin America 8% 17% 33% 42% Albania 0% 75% 25% 0% Notes: 1 Share of total number of portfolios in the region for which the question was answered. 2 Data for the two new SNV portfolios in Congo have been excluded. Source: Survey ECORYS & SEOR, 2005; region average is based on country level averages, which are in turn based on portfolio data.

4.5.2 Impact: improved performance by clients leads to reduced poverty and better governance

Did the improved performance by clients contribute to poverty reduction and good governance?

Poverty reduction The poverty concept as applied by SNV has been subject to change over time. Since 2002, the concept has been placed in the frame of Amartya Sen’s freedoms. Only the evaluation of communal forestry management made an attempt to phrase the evaluation results in terms of these ‘freedoms’. In the other studies reviewed, no clear concept of poverty was used.

For evaluation purposes, poverty is understood as consisting of three dimensions: an income dimension, an access to social services, like health and education dimension and a political ‘voice’ dimension (being active and recognised actor in society).

Regarding the selected activities of the Santa Cruz portfolio, there were no underpinned indications of impact on poverty reduction If they exist at all, they would –most likely- be the strongest in the ‘voice’ dimension. In Mali, no research based studies on impact existed. Based on interviews with advisory staff and clients, it was assessed that the impact of the selected advisory services of the Koulikoro portfolio on income poverty can only be either weak or absent. Although clients have been able to establish a common solidarity fund, these funds are hardly used for economic investments, but for social events and sometimes for community contributions to the construction of schools or health centres. The CT have realised numerous social investments (schools, classes, maternities, health centres, etc.). As far as known, these investments have lead to functioning social services and may have contributed to improve the access to social services. In the North Rift portfolio, no ex-post impact evaluations were elaborated neither. But a number of studies was carried out that comprised assessments of the potential impact (for example for the market chains honey, aloe, fish and eco-tourism). The potential impact is a structuring criterion to the North Rift portfolio, aiming at a balance of 25 percent of activities directed to income poverty (market access for the

Evaluation SNV 99 poor), 50 percent of activities directed to access to social services (local governance, aids) and 25 percent directed to voice and empowerment (local governance, peace).

A comment made by the North Rift Portfolio was that “the purchasing power of the rural population is in spite of the national economic growth not increasing, in other words they remain poor and in many cases are becoming poorer. This will definitely create tension regarding the SNV objective.” (Annual Plan, 2006).

Good governance The Santa Cruz portfolio team indicated more impact on good governance then on poverty reduction, such as: • Less denouncements regarding less transparent management of public funds in Santa Cruz municipalities as a result of the operations of the Comites de Vigilancia, supported by a functional umbrella organisation (ACOVICRUZ); • Fewer conflicts on land that end up in violence or legal action, as a result of improved negotiation power and mediation capacity, both at the end of the public sector (Prefecture, INRA) and the indigenous population groups. Less conflicts among indigenous population groups on land use; • National recognition of communal forestry management by the original population; • More participative, ‘inclusive’ and ‘diversified’ attitude of meso- and micro level governments (municipalities and Prefecture), what means “better government”.

In Mali, the participation in the municipal development plans has contributed to a commitment by the population to support and implement these plans. This is expressed, for example, by the higher willingness to pay taxes in the concerned CTs and the own efforts to take the plans as frame for new initiatives.

4.5.3 Assessment

SNV has no system in place that enables portfolios or country offices to conduct meaningful impact studies. The ‘country evaluations’ have been discontinued in 2002. These were replaced by impact studies of practice areas. But only two (or three including tourism) have been carried out at corporate level. At portfolio and country level results and impact studies are remarkably absent.

100 Evaluation SNV Box 4.3 The ripple model and the Advisory Plan

SNV’s logical framework sequence is not perceived as linear, but follows the so-called ‘ripple model’ which borrows its name from the idea of throwing a stone into the water resulting in a number of ‘ripples’ being created around the point of entry of the stone into the water.

Following the picture of the ripple model (Figure 3.5), the execution of an activity results in a number of follow-up ripples. The further these ripples radiate away from the point of entry, the less intense they get and they are finally fading away. The more distant the ripples, the more they have faded away and the more difficult they are to ‘measure’.

The ripples created from an executed activity can also be supported, or counteracted, by the ripples created through other executed activities (“whose ripples are it anyway?”). The last ripple is the impact which has been created through all these interventions. According to SNV it is the most difficult to measure and may be shown only after a couple of years.

This metaphor made some SNV advisors belief that impact measurement is close to impossible, since ripples fade and attribution to SNV intervention cannot be proven. Since proving impact is questioned, no initiatives have been taken to launch impact studies at all. The visualisation of SNV acting as a single entry is contradictory to SNV’s input delivery approach, that aims at synergy and spin off effects derived from simultaneous interventions of clients in a constellation. According to that approach it are precisely the multiple entries that reinforce each other to produce impact. So, a higher wave, instead of ripples fading away.

SNV’s ripple model should be no excuse for not conducting impact studies. SNV’s orientation on poverty reduction in the client selection process could be made more operational by adhering to a logical framework sequence in the planning, as expressed in the Advisory Plan. Advisory Plans should have a logical relation to (a dimension of) poverty reduction or good governance. If the sequence between outcome-results and impact are clearly envisaged and defined, the Plans may serve as a source document for subsequent monitoring and evaluation. Source: authors.

SNV correctly states that for proper outcome and result measurement a clear cut, concise and realistic Advisory Plan is a pre-requisite. In all three countries (and based on a more detailed assessment in Bolivia and Mali) it was concluded that the Advisory Plans lack the basic consistency and indicators to be of use for monitoring and evaluation. From a monitoring and evaluation perspective, the Advisory Plans are an under-utilised tool (see box 4.3).

In general terms, clients are appreciative and indicate that the provision of capacity development services has had a positive impact on their functioning and may contribute to poverty reduction and/or good governance. Partnership building is coming out as an area where clients have gained a lot of ground.

The absence of solid studies does not rule out that portfolio staff is able to describe impact. There are strong indications of impact on the three dimensions of poverty in Kenya and Mali. Obviously economic activities, such as honey and aloe production in

Evaluation SNV 101 Kenya contribute more to the income dimension, while public entities may have a larger impact on the access to social services (Mali).

The Santa Cruz portfolio was able to indicate impact on good governance, in particular on issues like transparency, expressed by less denouncement and by a reduction in the number of conflicts on land rights and land use. There has been an impact on legislation and thinking on forest management in general. SNV has made valuable contributions to the public and political recognition of diversity in society and therewith has contributed to security of vulnerable population groups and social stability in society.

In Kenya contributions to governance could be indicated by the increase of revenue generation of the Keiyo County Council, something similar to the tax generation by CTs, as observed in Mali.

102 Evaluation SNV 5 Efficiency and learning capacity

5.1 Efficiency

The characteristic 10 of the Subsidy Agreement expects that by 2006 SNV would have gained in efficiency, through servicing more clients, through strategically better chosen clients and partners, and by a stronger knowledge function.

The service delivery, the client selection and the knowledge function have been described in the previous chapters. The efficiency at corporate level has been dealt with in section 3.2.7. This section deals with the question whether and how those elements contribute to efficiency at field level.

From an evaluative perspective ‘efficiency’ raises the question whether the same output could have been achieved using less input, or alternatively more output could have been achieved using the same input. This generic question has been taken by SNV management 59 as factor that underpins a more macro oriented approach: • SNV’s increased involvement at national level and support to ministries would expand the reach of SNV support; • Improved regional programming, cross-country interchange and joint definition of practice areas facilitate the replication of methods and would imply less ‘inventing the wheel’.

Efficiency in service delivery At portfolio level 60 ‘efficiency’ has been translated in terms of the components of characteristic 10, reflected by: • the number of clients to which the advisory services are being provided; • the selection of strategic clients and allies; • a better performance as knowledge organisation, with a broader coverage at regional level.

a) The number of clients does not automatically leads to higher efficiency; it may imply a higher volume. More clients may contribute to a higher productivity (the number of clients per advisor) or better use of (overhead) facilities and/or lower costs per unit. In the three portfolios visited, all realised a considerable growth in the number of clients (and hence in advisory days), in particular during the years 2004 and 2005. Headquarters stimulated portfolios to assume more clients during these years, since SNV experienced

59 Presentation by SNV management, November 2005. 60 Source: Estructura y enfoque del SNV. Proceso en el logro de las 10 características establecidas en el Convenio de Subsidio con DGIS para el período 2002-2006. 7 de Noviembre, 2005.

Evaluation SNV 103 under expenditure of its core funding. In the portfolios the number of clients increased between 55 and 80 percent in a two-year period (see for regional data Table 3.3). But the increase in productivity was not equally high. First, advisory services to large clients (say a city municipality) were split into more ‘product’ focused assignments, each counting with its own Agreement and Advisory Plan (for example by department of that municipality). Second, since 2004, part of the assignments became contracted out to local consultants. This has enabled to attend more clients, but did not increase necessarily the productivity. The number of clients per advisor did increase (to 2.4 in Santa Cruz and 2.6 in Kenya) although the number remained below three per advisor (at corporate level, the average is 2.6 in 2005, see table 3.2). Being an average, there are obviously advisors who have more clients, for example in the case of ‘narrow’ assignments.

Working in multi-disciplinary teams contributes to upgrade the quality of services, but is no automatic guarantee for efficiency. In Kenya, clients commented that the matrix of ‘core’ practice area advisors with cross cutting advisors (HIV/AIDS and gender) implied frequent repetition at the side of the client (informing advisors, carrying out activities). On a day-to-day level this would require more coordination. In Santa Cruz, advisors commented to spend a lot of time waiting for colleagues attending the same client. But these disadvantages are offset by the advantages of working in teams.

The use of local advisory capacity varied between 5 percent (2004) in Kenya and 20-25 percent in Bolivia. In Kenya, this additional capacity was used only when services could not be sourced in-house, but in Santa Cruz it was frequently used for consultancy type of services to clients. A constraint for expanding this manpower is the limited budget for contracting consultants at portfolio level. A related issue is the effect that SNV -as provider of services free of charge- has on the consultancy market. On the one hand SNV can be a competitor, taking out assignments that could have been carried out by commercial parties, while on the other hand SNV may ‘crowd out’ other demand for commercial providers if it makes frequent use of consultants in a ‘thin’ market.

b) The selection of clients is described in chapter 4. Working with clients in constellations is more efficient than working with individual clients. How much more efficient is hard to express since factors like replication, synergy and the like play a role. At portfolio level, in qualitative terms, advisors indicate a higher level of efficiency as a result of sound client selection in well demarcated thematic areas. In Kenya the effect of up-scaling the potential of clients and rolling out (spread effects) were mentioned as efficiency factors. In client selection there are no financial stimuli or sanctions that may influence on efficiency. SNV’s services are not subject to market forces and SNV has no internal system of incentives (stimuli and sanctions) to enhance efficiency.

c) With respect to the development of knowledge and the relationship to efficiency there are qualitative indications as well, but no quantitative indicators. At portfolio level, in daily practice the practice areas are mainly the frame that set the themes to define areas for provision of services. The efficiency is found in replication and development of tools for more than one user. SNV stressed the importance for efficiency of cross-border and regional interchange. Although international interchange and replication have increased significantly, the ‘push’ to establish international linkages has created its own demand, such as the establishment of ‘bridging’ portfolios between two existing portfolios (for

104 Evaluation SNV example in neighbouring countries). Whether that contributes to a higher efficiency has not been studied.

The time registration in the three portfolios visited shows on average that about 10 percent of the time is devoted to knowledge development. Interviews with advisory staff revealed that part of the time destined to knowledge development, such as the elaboration of case studies or joint publications 61 (or participation in practice areas network meetings) is registered as direct advisory days as well. Each advisor uses a variable amount of time to knowledge groups and D-groups (intranet facility).

5.2 Effectiveness and organisation

Chapter 3 dealt with the SNV organisation and chapter 4 with the outcome and results. This section explores briefly the eventual relations between the two. The section starts with the relation between the Ministry of Foreign Affairs and SNV.

Ministry of Foreign Affairs - SNV Until 2000, SNV formed part of the Ministry of Foreign Affairs. In 2001 SNV and the Ministry were ‘disentangled’ from each other and the new relationship was laid down in the Subsidy Agreement . The subsidy allocation for the year 2002 was agreed upon, while for the following years SNV had to submit an Annual Plan. Under the provisions of the Subsidy Agreement, covering the period 1 January 2002 to 31 December 2006, SNV submits Annual Reports, audited financial statements and an Annual Plan to the Ministry of Foreign Affairs. There are regular meetings between the Ministry and SNV to discuss those plans, although these are limited to only one or two meetings per year. The Hoenderdos Consult report mentioned that the Ministry, after having approved the 2001 Strategy Paper pursued a “hands-off approach” with respect to content matters and focused on administrative issues. The Ministry allowed SNV a period of ‘trial and error’ (Hoenderdos Consult 2004:viii, 73).

Through a waiver in the Subsidy Agreement, the Dutch Embassies are allowed to involve SNV in service delivery in the field without passing through a tendering process. In practice, the use of this waiver has been limited. Since SNV did not form part of the ‘family’ of the Ministry of Foreign Affairs anymore SNV and embassies drifted somewhat apart, also since the introduction of the sector approach implied a different working modality with less room for direct involvement of SNV.

Although for legal reasons SNV continued to operate under the international bilateral agreements signed between the Netherlands and the host country, the embassies had no control function anymore on the activities of SNV 62 .

61 PC Santa Cruz comment on draft version. 62 Moreover, since according to SNV’s Articles of Association (Statuten) SNV is independent in its operations from public entities.

Evaluation SNV 105 In the course of the period under review, the issue of complementarity and coherence between SNV and the delegated bilateral aid programme triggered discussions with respect to the level and modus operandi of collaboration. An example of such a co- operation is the water sector, where SNV and the Ministry are working in close collaboration aimed at obtaining synergy at various levels of operation. But overall, the congruence between SNV and other Dutch development actors has not been properly sorted out.

Country level After 2001, the Country Office (with Director) remained, but part of the technical staff was ‘decentralised’ to the portfolios. Initially, the national office was seen as ‘a full business unit’, although in practice it was more a coordinating and representative linking pin between national and meso-level. In 2004, the corporate policy of ‘anchoring’ SNV’s activities at the macro level implied a revitalisation for the country offices. Country offices were re-equipped with personnel with both supranational (regional) and national functions (for example, as regional thematic advisor) and started to develop own ‘portfolios’ with national level clients (for example, ministries). The introduction of the new organisational strategy Optimising Support (2005) implied a change in mandate, lines of command and control with permanent Regional Directors and operational support services (Human Resource Development, finance/administration). Country offices have ceased to be ‘full business units’ and became first responsible for the matching of activities with national development policies.

Portfolio level Programme officers respond directly to the Country Director. The client portfolio was introduced. Working in teams required an organisational change at portfolio-level. The mandates of the practice areas groups –on subject matter issues- remained rather unclear during the first years.

Over the period 2002 – 2005, at portfolio level the positioning in relation to clients went hand in hand with a search for the best way of operation. At portfolio level the following changes were introduced aimed at a higher effectiveness of the services: • more emphasis on connectivity, alliances and networks; • service delivery to those who deliver services (for example in local government); • more attention to income generating activities; • more relation to the macro-level in order to expand coverage; • incorporation of other themes that emerge from the context analysis, such as reforms of the public sector and protected areas; • more openness and flexibility to change; • more focused on product and less on process; • more studies and systematisation of experiences.

5.3 Learning Capacity and Practice areas

An important step was set with the change towards working in teams instead of stationing advisors in the offices of host-organisations. This became the cornerstone of a SNV community conducive for sharing and developing knowledge. In mid 2004, regional

106 Evaluation SNV knowledge networks were initiated around thematic areas of expertise with the aim to motivate people to interact, to share experiences and to develop new ideas. For day-to- day communication, virtual collaboration platforms (D-groups) have been established. At portfolio level most advisors participate in these D-Groups, and in knowledge networks, and thematic groups at the local level. And every junior or ‘medior’ advisor is supervised/supported by a coach and Portfolio Coordinator.

The ‘knowledge improvement’ experienced a certain ‘learning fatigue’ during 2003 and 2004. At corporate level less learning programmes were offered, and knowledge interchange became an intrinsic component of the function of each advisor. Newly recruited subject matter specialists (so denominated ‘SPARKS’) were contracted to revitalise the knowledge development.

As there was a corporate wish to focus on a limited number of practice areas, and the practice areas are the main organising principle that set the frame for client selection and for the learning process.

In continuation, the practice areas are highlighted and assessed.

Practice areas Since 2003, no longer country evaluations would be carried out, but evaluations of the practice areas. But only few external evaluations were carried out on those practice areas: 63 • Evaluation Collaborative Forest Management in Latin America, 2005; and 64 • Evaluation of local governance in Western Africa, 2005; and • A study on Local Sustainable Tourism was ongoing at the time of the present ECORYS-SEOR evaluation. 65

Collaborative Forest Management Collaborative Forest Management is one of SNV’s practice areas in Latin America. The 2005 evaluation has tried to place the results in the frame of Amartya Sen’s freedom concept (to which SNV subscribes), disaggregated into 5 different types of freedom: (1) political freedom, (2) economic facilities, (3) social opportunities, (4) transparency guarantees and (5) protective security. The report concludes: ”Collaborative Forest Management can make a significant contribution to the freedom goals and SNV has considerable experience in this field. The inclusion of collaborative forest management into SNV’s portfolios is a sound decision” (2005:10). The evaluation was carried out in Peru (Portfolio Amazonas), in Ecuador (Portfolio Center [former Morona Santiago]) and Bolivia: Portfolios Northern Amazone and Eastern Lowlands Santa Cruz. 66

The evaluation presented a list of strength and weaknesses, as well as eleven (11) main recommendations. The list of strength refers all to the general approach of SNV. Only one refers –in very general terms- to forestry management: “SNV has a strong record in empowerment of people in their struggle for land and forest rights”. In the list of

63 Winfried Zettelmeyer, Arend Jan van Bodegom. Evaluation Collaborative Forest Management. February 17, 2005. 64 Adiza Lamien Ouando, Jean Bossuyt, Volker A. Hauck, Ousmane Sy. Strategic evaluation of the local governance programmes (SNV Benin, Burkina Faso, and Mali) May 2005. 65 No (draft) report was available by December 2005. 66 The same portfolio as selected for the present evaluation.

Evaluation SNV 107 weaknesses, most refer to general SNV processes as well, but the following refer specifically to forest management: • Context analysis (national and sub-regional) around SNV’s work on CFM is too weak (stakeholder analysis, analysis of the forest sector and other sectors with an influence on forests and the people that depend on them). It is generally restricted to the client organisations and their needs. The financial and economic viability of the clients’ projects is generally not analysed and used as a criterion to take the client on or not; • Advisory plans lack a common format and their content is incomplete. Although the Santa Cruz and Amazonia portfolios have been actively facilitating platforms on conflict management on land tenure issues, the demand orientation precludes, in some cases, a proactive stance in an increasingly conflictive society to create a commonly shared vision; • There are gaps and deficiencies in the quality control and monitoring system on all levels. Control of the quality of advisory services and of the clients’ performance is missing. Currently there is not sufficient focus on effect and the ultimate impact of SNV’s work on the target group.

Out of the 11 main recommendations, the following refer directly to collaborative forestry management: • Clients’ projects should be analysed for their cause-effect logic over the whole chain, and economically appraised in form of cost-benefit and/or cost-effectiveness considerations (of varying depth according to the case) as an input for the decision on client adoption. Training in this area is necessary and should be provided for SNV staff; • Institutional memory should be created along the lines of the Bolivian initiative and archived in a subcategory of D-group SNV (LA) Forest and Territory. All CFM advisory plans, past and actual should be archived in the corresponding subcategory. Documents put on the network, either archived or not, should contain author, title and date of creation to facilitate their use.

Although the ECORYS – SEOR present evaluation team did not comprise specific expertise in forest management, a few observations can be added: In Bolivia, SNV has contributed to successfully acquire forest certification for its partners and has had impact on legislation, on thinking on forest management, on the relation between forest management and indigenous people. SNV has contributed to peaceful mediation on conflict on land and land use and has introduced participatory methods of decision making on land use. Evidence of economic sustainability of results of SNV support in CFM is poor.

Responsive and Accountable Local Government Responsible and Accountable Local Government is a practice field at corporate level that means that most portfolios worldwide carry out advisory services in that field.

In 2005, an evaluation of the practice field was carried out in Western and Central Africa (Benin, Burkina Faso and Mali). In Western Africa (but also in many other countries) these services are offered against the backdrop of an ongoing decentralisation process of government. The evaluation report states (2005:5) “It is widely recognised that democratic decentralisation, if properly implemented, holds potential for contributing to

108 Evaluation SNV poverty reduction”. In academic literature, however, this statement has not been confirmed. The report concludes that: • “The tools and approaches used [by SNV] make it possible to achieve relevant outputs and outcomes. In some cases, however, the approach is still overly academic and technical, raising problems of ownership of the processes and their products by local organisations. [..] The pace of capacity building may not always be attuned to the rhythm of the local actors involved”; • There is a lack of funding to support (the follow up of ) the capacity building activities; • SNV lacks a sound system of strategies, processes and tools to evaluate the effects of its action in terms of improving local governance, or possible impacts on poverty alleviation.

The evaluation study distinguished the tangible from the intangible effects. It focused on the positive and expected results 67 and identified the following (Adiza Lamien Ouando and others, 2005:12):

Tangible effects Intangible effects

• better perception of decentralisation as a key local • citizens’ awareness of their rights and development issue responsibilities in local development (wakening-up • local authorities better “embedded” as public of citizenship at municipal level) institutions with a development remit • empowerment of the various local actors (realistic • local authorities take on board new roles, including expectations as regards the State or donor; with regard to regulation understanding of the need to work within the • moves towards a better organisation of civil society resources available) with a view to participate in governance • emergence of a different perception by citizens of • clarification of the roles of the various actors the public authority (reduction of the divide, citizens • new frameworks for concerted action by and less reluctant to speak, claim their rights or call for networking of the actors transparency) • better circulation of information • local authorities more receptive to civil society • new ways of managing a commune • building the capacity of local authorities to listen • new methods of dialogue and negotiation of local • emergence of a different culture of relationships development priorities between the mayor, elected politicians and local • receptive attitude towards joint actions authority officers (less top-down) • awareness that the success of decentralisation also depends on positive changes at national level and that lobbying is required • awareness of the need to better link local development plans with sectoral policies and other processes at the macro level (such as the PRSPs and the Millennium goals)

67 Methodologically sound: the relevant effects are only those that were aimed at as expected results. Negative effects are (unexpected) impacts.

Evaluation SNV 109

And the report identifies the following factors that may hinder that outcomes fully unfold in effects (or impacts): • Limited opportunities to apply the capacities acquired (outcome) due to lack of financial resources (either to sustain the capacities or to implement the products of capacities, for example the application of recommendations of a consultancy); • Too much focus on clients and insufficient on processes of social change; • Bottlenecks at macro level. The effects of fostering good local governance may be restricted by national level bottlenecks and constraints. The need to make local voices heard at national level has been acknowledged. Clients were keen to see SNV taking up a more proactive role at the “macro” level in order to optimise positive effects at the “meso” level.

The report recommends bringing coherence to SNV’s activities by putting local governance development at the centre of all of SNV’s services and activities. In addition, it should promote knowledge management with a perspective towards local governance development and poverty reduction.

The present ECORYS – SEOR evaluation in Bolivia, Mali and Kenya contributes with some elements to the above mentioned effects of local governance advisory services: At local level, SNV is one of the few actors that has achieved to foster linkages of confidence between the local public sector, the civil society and the private sector. SNV contributes actively to establish and strengthen systems of checks and balances on local governments (Bolivia) or to make people feel like citizens of a local government (tax paying), being actors in a local environment (CTs Mali). At the same time local governments have been made aware of the ethnic diversity in society (Mali, Bolivia), to act through debate instead of through power and enforcement. SNV has contributed to the concept of the CCC as an instrument to support and advise the CTs.

Pro Poor Sustainable Tourism Since the 1990s, SNV has contributed to promote pro-poor sustainable tourism (linked to poverty alleviation, local socio-economic development and biodiversity conservation).

Advisory services are provided in policy and strategy development, community mobilisation, business training and environmental conservation. The advisory services are provided to a wide array of organisations in the tourism sector like National Tourism Boards, environmental NGO’s, Government authorities at different levels, Business associations, associations of local communities, national park authorities and tourism training institutes (in countries like Tanzania, Uganda, Ghana, Vietnam, Botswana, Cambodia and Nepal).

Initially, tourism was seen as a revolutionary step for SNV but achieved to be recognised as one of the eight practice areas of SNV. Support to the different pro-poor tourism initiatives by SNV evolved over the time. During the 1990s the focus was on ‘joint’ and ‘hands-on’ support with local partner organisations. Nowadays a more regional focus emerged, replacing a country specific approach. For example, the practice area has been expanded from Nepal to Vietnam, Laos, Bhutan and now Cambodia, resulting in new

110 Evaluation SNV interventions and regional programmes. For Asia, a Regional Sustainable Tourism Knowledge Network has been established.

Within SNV, the practice area is less developed than for example local governance or collaborative forest and water management. One of the entirely different aspects is the role of the private sector. Major SNV achievements are its networking and its publications on experiences in different tourism programmes and practical guides and manual.

Collaborative water management Based on secondary SNV material only, a rapid review was carried out on the practice area “collaborative water management”. 68

Since SNV has its historic roots in marginal (mainly rural) areas, where most people rely on agriculture and its related activities, and are in need of potable water, and sometimes have to defend themselves against water, the importance of water for SNVs target population was very clear. With SNV’s shift of focus to meso level organisations, the type of client shifts. While water management at a micro level is usually the task of community based organisations, at a higher level the public sector plays a more important role. That also changes the contents of the involvement towards planning, controlling and studying water management in all its aspects and with all its users. At meso level, the key challenge is to reinforce local level support agency capacity (technical, institutional, and organisational) to guide and support communities in managing water. SNV’s role is mainly one of facilitator and broker in negotiations between stakeholder groups.

Although collaborative water management is one of the practice areas of SNV, the focus is on the ‘collaborative’ part, rather than on the ‘water’ part. SNV’s main competences rest with the communal management, with local organisation around the theme water and not so much on the technical aspects of water management.

As far as could be learned from SNV website information, SNV has not yet developed a policy on integrated water resource management (IWRM). SNV’s involvement in IWRM will need to be developed over time, based on experiences and wide consultation and networking.

SNV’s core network (international and regional partners) to that end merits further expansion next to enhancement of in-house technical knowledge.

5.4 Assessment

The formal requirements regarding planning, budgeting, reporting and policy discussion agreed upon between the Ministry of Foreign Affairs and SNV are limited (Subsidieaanvraag SNV, 2001:26,27).

68 Side study conducted by Ir. R.J. Oosterbaan. December 2005.

Evaluation SNV 111 SNV is a decentralised development organisation with permanent offices and staff at sub- national level. There are basically three elements that have contributed to efficiency: the decentralization with a relative autonomy of the portfolios (“the field”), the client selection process and the knowledge development in a limited number of practice areas.

However, efficiency is not SNV strongest point. Due to the nature of SNV’s provision of services and its financing modality, the organisation lacks financial incentives for efficiency. Such incentives would have existed if SNV had supplied services under market conditions, or would have faced scarcity of financial resources; or would have counted with internal efficiency stimuli. But it did not.

Field visits revealed that there is room for efficiency gains by having a better time requirement planning for assignments. At portfolio this leads to a strange split: advisors work very long days (expressed in the time registration) and use to put extra working days in a year. The corporate drive to ‘increase efficiency’ has pushed up the number of clients, but this did not necessarily increased productivity. The expected workload per client forms part of the Advisory Plans and is expressed in mandays. SNV advisors happen to be poor managers of their own time (and lack the skills and experience to assess their time requirements) and make poor assessments of time requirements. In consequence the sum of the allocated time in the Advisory Plans of all their clients do not correspond with reality. In consequence, advisors either penalize one client in the benefit of the other, or find the ‘solution’ in making long and many working days. 69

Both management and advisors still devote a lot of time on internal (change) processes, including internal coordination (all SNV processes are very participatory; a single advisor can be member of various working groups at the same time and internal processes are time consuming). Part of this time is not visible in the time registration system.

Intensified and expanded cross border and regional contacts do contribute to efficiency in terms of creating options for replication. Whether efficiency gains are really made in the case of the physical establishment of additional ‘bridging’ portfolios between existing portfolios in two neighbouring countries (as is the case in Mali) is still to be studied and assessed by SNV.

Knowledge interchange became an intrinsic component of the function of each advisor. SNV is a learning organisation that invests in its main asset, its human resources. SNV disseminates its knowledge through internet facilities, a Journal (Vice-Versa) and (inter-) national presentations and it ‘learns’ as an organisation by incorporating external knowledge and experience. Also at both country and portfolio level, SNV is a learning organisation: flexible, knowledge eager, ability to review and reflect on strategies. The interchange with private sector actors (such as private consultants) and academic knowledge could be further enhanced.

69 This ‘solution’ to labour productivity is rather common phenomenon among professional workers. It becomes a problem when productivity per hour decreases as a result of human exhaustion.

112 Evaluation SNV The practice areas (or themes thereof) are possibly the most important organizing principle at portfolio level. But the evaluation of these practice areas has not been developed yet. SNV’s own knowledge is not equally distributed among practice areas.

Evaluation SNV 113

6 Conclusions and points of attention for a next phase

6.1 Conclusions by topic

Transformation SNV has made an enormous effort to realise a drastic transformation from an organisation focused on technical assistance and implementation of projects to an organisation that supplies advisory services aimed at strengthening capacity of client organisations. This transformation required a multi-dimensional change process, comprising its legal status, the strategy, organisational structure, processes and foremost a change in capabilities of SNV staff. Reorganisations took place both at headquarters in The Hague and at the country offices in more than 30 developing countries. Late 2005, a new reorganisation has started.

The transformation process required a lot of time, manpower, financial resources and energy from SNV. Both Dutch partners and partners and allies abroad have described SNV as an organisation that devotes a lot of time to its internal change processes. This despite SNV’s successful efforts to ‘open up’ the organisation and to connect with many partner organisations. In the transformation process, not all aspects could be embarked upon simultaneously. For example, SNV management decided to postpone decisions on country selection. At portfolio level advisors manifested a ‘change fatigue’ (Bolivia) and “hoped for calmer waters” (Kenya) for the years to come. The reorganisations did enable the new way of service delivery, but the structure thereto has not been consolidated yet.

In the three portfolios visited the introduction of the 2001 Strategy was by no means a ‘shock therapy’ but either had started in an earlier stage or was implemented smoothly. In the three countries visited, the change process towards the new practice of provision of advisory services has been completed.

Funding In accordance to the fifth characteristic of the Subsidy Agreement, SNV has explored the funding possibilities from third parties, but was hardly successful in obtaining financial resources. Since SNV preferred to stay aloof from programme implementation and searched for core funding, it implicitly has chosen for staying nearly fully dependent on the core funding by the Ministry of Foreign Affairs. This choice has consequences: 70 • SNV keeps a separate financing position in the Dutch development co-operation;

70 This exclusivity refers to other organisations that receive subsidy by the Ministry of Foreign Affairs, for example under the co-financing system (MFS).

Evaluation SNV 115 • Although autonomous for its operations, at the country and field levels, both partners and clients consider SNV as the official development organisation of the Netherlands. And they are not far out, since SNV has continued to operate under the international agreements in force between the Ministry of Foreign Affairs and the host countries; • The requirements stated in the Subsidy Regulations of the Ministry of Foreign Affairs and SNV’s Articles of Association regarding autonomy in operations are hard to match. So far, this has not been of any problem, but may call for a further precision of the relations on policy matters between the Ministry and SNV; • The single most determining factor for all SNV’s activities, its magnitude and its rhythm of growth is the budget allocated for core funding by the Ministry.

The Subsidy Agreement created a waiver facility to embassies to contract SNV services. The use of this facility remained modest only and represents one percent of all SNV expenditures (2004). It is the evaluators’ opinion that the waiver system is not consistent with the sectoral approach in which technical assistance is offered to the host country using multi-donor funds (baskets).

Efficiency Beyond doubt SNV staff is hard working, highly committed and identified with the organisation. In absence of efficiency benchmarks or efficiency requirements imposed by either third parties or market conditions, SNV has set its own efficiency norms and indicators in two ways: 1. on the basis of the ratio between the costs for its ‘primary process’ and its ‘support costs’. The ‘primary process’ costs include, next to the remuneration of advisors, the related costs made at field offices and head quarters and costs made in relation to the development of partnerships. All other costs are ‘support costs’. 2. on the basis of direct and indirect costs. Direct costs include all costs of the field offices, a part of the costs of the head quarters and costs made in relation to partnerships. Indirect costs are the costs of the headquarters which could not be allocated directly.

Using the first calculation method, the support costs decreased from 45 percent in 2002 to 43 percent in 2005. The second calculation method shows that de indirect costs decreased from 18 percent in 2002 to 9 percent in 2005.There are basically three elements that have contributed to efficiency: the decentralization with a relative autonomy of the portfolios (“the field”), the client selection process and the knowledge development in a limited number of practice areas. It is SNV’s intention to further reduce the support costs to 35 percent. This has to be achieved by reorganisation (Optimising Support, 2005) and by revision of the share of non-advisory staff in field offices (53 percent of all field level costs in 2004).

SNV has not developed an internal system of premiums and sanctions on efficiency (or effectiveness) in the allocation of its financial resources to either countries or portfolios.

There are other aspects to efficiency. SNV has made high initial investments, necessary to trigger change and to transform both the institution and its staff. However, once consolidated the system (which has not been the case in the period under review) the necessity of maintaining all systems should be re-assessed.

116 Evaluation SNV

Field visits revealed that there is room for efficiency gains by having a better input planning for assignments (duration, components).

Inputs During the period 2002-2005, SNV has embarked on activities in six ‘new’ countries. This choice was justified by internal motivations like cross-border synergy and simply ‘expansion’. There is no relation to the selection criteria used for the (delegated) bilateral development co-operation.

At country and portfolio level the context analyses form the base for the identification of both practice areas (themes) and (constellations of) clients. The quality of the country context analysis is solid, as are the analyses at portfolio and client level. The client identification process is complex, but was well-perceived for the three countries visited. In some countries a broad participatory process is applied in which existing clients and development partners join in. Next to the merits of such an approach, a shared vision among development professionals (‘group think’) may influence the results (Mali). SNV has no system in place that is able to compare the selection of one ‘constellation’ of clients (A) with a potential other constellation of clients (B) in either the same practice area or in alternative practice areas. As a consequence, the question how relevant the selected clients are to poverty reduction and/or good governance (as compared to any other potential set of clients) cannot be answered in a quantitative way. However, in general, SNV has ensured to focus on key constraints in the national and regional development context (such as land ownership and land use in Bolivia) and to work with strategic actors and partners of both public sector and civil society that are relevant in resolving (or alleviating) these key constraints to development. In the three portfolios visited, the probability that the clients selected are relevant actors to the reduction of poverty has been assessed as high.

Working with interconnected clients increases the probability that the clients contribute to the perceived impacts. This concept of clients in constellation has been fully implemented by the portfolios visited in Bolivia and Kenya. In Mali the final selection was more path dependent in its origin and less determined by the identification of core problems to development.

The identification process has led to a well-perceived focus on core problems and SNV has shown courage in assuming the provision of services to clients in conflict (for example conflict mediation in Bolivia and Kenya). Selection was more demand driven during the years 2002 and 2003, but became more pro-active as from 2004 onwards. In accordance to the intention of the subsidy agreement, in the three portfolios visited 80 percent of the clients could be characterised as ‘meso level’ clients.

SNV does not count with a clear demarcation about the kind of services it delivers and to whom, for example as far as it concerns the provision of narrowly defined consultancy services or the balance between providing advisory services and knowledge brokerage. Nevertheless, overall SNV delivers well-perceived services that are framed by strategies at various levels, underpinned by context analysis, by needs assessments and agreed upon with the client.

Evaluation SNV 117

SNV’s major asset is the quality of its staff. SNV has paid considerable attention to improving its competency and career management. As compared to 2001, in 2005 SNV counted with more local staff (Home Country Nationals-HCN), more female staff, sound recruitment and career management, Personal Results Development Plan, formal training and personal coaching, “SNV-Campus” and (virtual) knowledge groups. In most regions the share of HCN has increased to 65-70 percent of all staff. 71 The drawback of a very high share of HCN implies both a higher dependence on the ‘market’ conditions for local professionals; a friction with existing expectations of clients on knowledge transfer, and limitation on innovative and critical strength that is supposed to be a ‘raison d’être’ of an international organisation.

Output The Agreements signed between SNV and its clients have been instrumental to the awareness among SNV advisors that the recipients are clients and should be treated as such. The Agreements contain neither an exit-strategy nor an ex-post commitment. The duration is determined by the SNV input delivery.

In addition to the Agreements, Advisory Plans are being elaborated. The quality of the Advisory Plans varies. Advisors explained to have both difficulties in applying the logical framework sequence and assessing their own inputs (for example in mandays). A proper formulation of expected results and an indication of measurable outcome indicators were missing in most plans reviewed in the three portfolios. SNV correctly states that for proper outcome and result measurement a concise and realistic Advisory Plan is a pre- requisite. In practice, the Advisory Plans lack the basic consistency and indicators to be of use for monitoring and evaluation.

SNV’s main instrument to measure output is the client satisfaction survey. This instrument has served its purpose in making advisors more ‘client conscious’. The review forms part of the management information system. It is a less suitable tool for statistical registration and accountability purposes. Its four months frequency is useful for new clients but becomes routine in the case of long term relationships.

In general terms, clients are appreciative and indicate that SNV’s Capacity Development Services have (had) positive results on their functioning. Partnership building is coming out as an area where clients have gained a lot of ground. SNV has been able to achieve a high level of satisfaction amongst its clients, even in countries where expectations on eventual funding had been (or still are) very high.

Outcome An integrated system to monitor and evaluate the output-outcome-effect-impact chain is still in development. At outcome level, the instruments in place are the end of year / end

71 According to SNV, at corporate level no written indicators have ever been set for the ratio between European and national staff. Verbally, SNV corporate management mentioned (Briefing December 2005) that a ratio of 40 percent European advisors and 60 percent home country nationals is taken as an indicator. In its comments on the draft final report SNV stated “we believe that the optimal mix of HCN/TCN [..in relation to EUN?..] staff is different in the various regions and countries” and “we believe that the statement [..on existing frictions..] does not do justice to the quality of our HCN staff and may hurt them”. 14 th February, 2006. However, interviews by the evaluation mission revealed frictions as mentioned.

118 Evaluation SNV of assignment evaluation, case studies and ‘peer reviews’. The end of year review is in fact little more than an extended interview that shows some overlap with the client satisfaction survey. Peer reviews do involve only ‘peers’ (that means other SNV advisors) and hardly outsiders. Since all –or most- participants in such a review have direct relations with the client and join the same experiences, there is a considerable danger of ‘group think’ with respect to the appreciation of the results achieved.

At portfolio level there is sufficient hard evidence (case studies) that capacities of clients have been strengthened. NGOs and civil society organisations have been strengthened in their institutional organisation and have been enabled to implement programmes related to training, technical assistance, advocacy, etc. Both in Bolivia and Kenya exists the tendency to work more with public sector entities. In the public sector, strengthening of capacities can take divergent forms, such as the capacity to resolve problems by negotiation and information in stead of by the use of power; the capacity to embark on participatory processes (in planning, in monitoring, in accountability) in stead of imposing decisions; the capacity to fight corruption as a joint public-civilian action; the capacity to embark on multi-actor actions

Of the conceptual four dimensions that compose capacity building, SNV has achieved most in networking and institutional organisation and to a lesser extent on the systems dimension. Although direct training and south-south capacity building is also geared to the human capability dimension, this has not been the main thrust of SNV’s activities. 72

Results and impact As part of the evaluation approach, the mission challenged three SNV portfolios to produce evidence on results and impact. The conclusions on results and impact are based on these portfolios plus additional secondary material produced by SNV. This may have provoked a (positive) bias in the assessment. The assessment made cannot be taken as statistically representative for all SNV activities worldwide.

SNV mentions in its Planning & Monitoring Guide (2003) that: ‘We are accountable to our partners (clients), to ourselves, and to our donors’ with respect to results. Therefore, a result measurement system is needed”. To measure results SNV applies a mixture of instruments (less defined as those at output and outcome level), such as the self- evaluation by clients; case studies; and selected external evaluation. SNV has no system in place that enables portfolios or country offices to conduct meaningful results and impact studies.

The ‘country evaluations’ that were traditionally carried out, have been discontinued in 2002. These were replaced by impact studies of practice areas. But up to late 2005 only two of those studies had been completed (communal forestry management in Latin America and local governance in Western Africa) and one was ongoing (local tourism). As compared to the total resource envelope, the allocation of resources to external evaluation (€ 150,000 per annum) has been remarkably low.

72 In some countries, like Bolivia, organisations have specialised on the strengthening of human capabilities (for example in local governments). SNV ‘connects its clients to those specialized organisations.

Evaluation SNV 119 At portfolio and country level results and impact studies are close to absent. Some advisors explained that the SNV applied ‘ripple model’ suggests (ripples fading away over time and distance) that impact measurement is impossible and hence a waste of time and resources.

While the self-evaluation by clients might be the correct instrument for the individual client, it does not provide any insight in results at a higher level of aggregation, such as constellations of clients in a (sub) practice area. In addition, self evaluation has limitations as far as it concerns the objectivity in the assessment of results.

The absence of solid (external) evaluations and research does not rule out that at field level strong indications exist for the results produced by SNV delivered services. And in addition, there exist strong indications of the impact of clients on poverty reduction and good governance. Advisors in the three portfolios visited were able to describe results and impact (for example a paper produced for this evaluation by the portfolio coordinator of Santa Cruz), although well under-pinned evidence is lacking. From the three visits to portfolios it was learned that the practice area ‘market access to the poor’ does contribute to the income dimension of poverty, but for other practice areas there is less evidence that they contribute to the income dimension of poverty. Some clients do contribute to either the ‘access to services’ or ‘voice’ dimensions of poverty, mostly through the support to local governments. Much stronger are the indications that clients make significant contributions to good governance (for example as expressed by less denouncement and by a reduction in the number of open conflicts on land use in Bolivia).

6.2 Points of attention for the next phase

In the assumption that a new subsidy agreement will be signed between the Ministry of Foreign Affairs and SNV for the period post 2006, the following issues may merit further attention during the next phase. The issues have been subdivided into those that have emerged directly from conclusions made in this evaluation and others that have emerged in the margin of the present evaluation but are not based on the present evaluation.

Based on the evaluative findings presented, it is recommended to pay attention to: • SNV has gone through an extended period of organisational change. Although the next recommendation on country selection may cause tensions in the organisation, at field level there is need to consolidate the working procedures and institutional organisation, which may provide to invest more time in SNV’s mission and less in its internal change processes; • The establishment of sound criteria for country selection, since this is the first step in a process that should ensure the highest probability that the envisaged impact is being achieved. This may imply either correspondence to (or alternatively, complementarity to) the country choice for Dutch bilateral aid; • The usefulness and desirability of the waiver facility to the Dutch embassies merits a careful assessment taking into consideration the possible flaws in consistency between this facility and the provision of technical assistance under the sectoral approach;

120 Evaluation SNV • SNV’s recent effort to ‘anchor’ its activities in the PRSPs and the MDGs has consequences both in terms of subject matter (health and education) and level of service delivery (macro). SNV has indicated that a maximum of 20 percent of its clients could be ‘macro level clients’. Most macro level clients have different characteristics than the majority of clients at meso level, in terms of resource endowment and external support received. It is recommended to determine in which cases SNV’s presence at macro level is functional for its activities at meso level (for example to cover the systems dimension of the capacity strengthening of meso level clients) and in which cases other actors (donors, organisations) already active at the macro level could act on behalf of SNV (delegated co-operation). This could be the case when presence at macro level is not motivated by the services provided to clients at meso level, but triggered by institutional objectives, such as SNV becoming directly involved in the national PRSP and MDGs processes; • It is recommended to include in the major planning documents (such as the Advisory Plan) an exit strategy by client (and eventually by constellation and practice area); • SNV’s intention to further develop effectiveness and impact measurement instruments is fully supported. It is recommended to upgrade the Advisory Plan to a base-line document counting with well formulated expected results and measurable indicators; • Since SNV is not exposed to market signals that may ask for efficiency efforts, it is recommended to explore opportunities for including financial stimuli in both the supply of the core funding and the internal allocation process of resources to countries and portfolios (see box 6.1).

Box 6.1 Exploring financial incentives

SNV is an almost fully subsidised organisation. This does not bar out the functionality of financial incentives. Options to stimulate the search for efficiency gains could be further explored. Among those options figure: • The incorporation of both fixed and flexible tranches in the Subsidy Agreement. The release of the flexible tranches could be guided by a Performance Assessment Framework, comparable to those applied for budget support in bilateral agreements. • Incorporation of a competition element among countries and portfolios by assigning a premium on efficiency performance (measured against pre-established indicators) or financial sanctions in the case of under performance.

Source: authors

In addition, some points were raised by stakeholders during the evaluation that are outside the scope of the present evaluation, but that may merit further attention in a next stage.

The relationship between the Ministry of Foreign Affairs and SNV merits further attention in terms of: • The formal requirements regarding planning, budgeting, reporting and policy discussion between the Ministry of Foreign Affairs and SNV in relation to the definition of what should be understood by ‘complementarities’ of SNV services vis- à-vis other cooperation activities realised by (or funded by) the Ministry of Foreign

Evaluation SNV 121 Affairs. This concerns complementarities in country selection, in sector choice and in the level of operations; • The effectiveness and efficiency of supply of (all) services free of charge requires further research. SNVs argument that ‘SNV clients have no capacity to pay’ was valid when SNV supplied services directly to the target population and is in part valid for its clients at micro and meso level. With a growing number of well-endowed organisations attended at both meso and macro level, this policy may merit diversification. This further research should comprise the aspects of market competition in service delivery; the effect of SNV offering ‘consultancy type’ of services to clients, provided by consultants (contracted by SNV) as well as the opportunities of linking an exit strategy with the capacity building of local service providers.

As an implicit and possibly unexpected result of the change process, the number and kind of checks and balances on SNV’s acting have diminished. Although SNV’s Board of Directors strives for full accountability, the Ministry of Foreign Affairs is not only poorly equipped (Subsidy Agreement, 2001) to have an influence on SNV decisions, it has pursued a ‘hands off approach’ over the period under review. The appropriateness of the prorogation of this approach merits further reflexion.

122 Evaluation SNV Annex A List of contacts and documentation

Evaluation SNV 123

SNV head quarters Bokhoven, J. Strategy Unit Elsen, D. Board of Directors Fransen, A. Planning, Finance & Control Unit Greijn, H. Knowledge Management Unit Jenniskens, A. Board of Directors Kampen, P. Portfolio Coordinator Macedonia Mulder, M Planning, Finance & Control Unit Ubels, J. Planning, Finance & Control Unit Wellen, L. Portfolio Coordinator Vietnam Zevenbergen, K Strategy Platform

Ministry of Foreign Affairs Nibbering, J.W. DSI, MY

Ex directors SNV Pelgröm, H. Ministry of Foreign Affairs Fierens, T. Parliament

Partner organisations of SNV Blokland, K. Agriterra Hauck, V. ECDPM Hilhorst, T. KIT Hodgkin, C. KIT Jenks, B. UNDP Swart, R. VNG Toonen, J. KIT

Bolivia Añez, María del Carmen Professional territorial management OICH Angulo A, Ana Rosa SNV Advisor Governance Sta Cruz Asturrizaga, Fernando Vice Minister for Land, La Paz Ballestaed, Esther SNV Advisor, Sta Cruz Barbero, Christian SNV Advisor Land Use, La Paz Bejerano, Javier SNV Advisor, Sta Cruz Caballero, Alfredo ex director INRA, Sta Cruz Chacón, María Representative of Central in OICH, Concepción Chavez, Lourdes ACOVICRUZ, Sta Cruz Cherbu, Margarita coordinator GTI project OXFAM phase III Chumacero, Jennifer SNV Regional Officer Human Resource Development, La Paz Cortés, Jorge AFIG, Sta Cruz Durán, Angela Central OICH, San Xavier Engel, Susan Deutsche Entwicklungsdienst, DED Flores, Elba CEJIS, Sta Cruz Flores, Guillermo Representative of district Berlin in Comité de Vigilancia, San Rafael Giné, Jeannet CEADES, Sta Cruz le Grande, Jan Willem First Secretary Environment, Royal Netherlands Embassy, La Paz

Evaluation SNV 125 Guissani, Bruno Advisor Governance Royal Netherlands Embassy, La Paz van Haeringen, Reintje SNV senior advisor, Regional Office, Quito Heijdra, Hans SNV Country Director, Bolivia van den Heuvel, Hans First Secretary Rural Private Sector, Royal Netherlands Embassy, La Paz Huamani Ober, Giselle SNV Advisor Conflict mediation, Sta Cruz Huagayo, Sefredo CIP, Concepcion Ibarra, Manuel President Comité de Vigilancia, San Rafael Krekeler, Jorge Consultant for MISEREOR, Sta Cruz Legues, Carlos Vice president Management Committee Monte Verde, Concepción Loyes, Dionicio Technical team territorial management OICH, Concepción Lutke Schipholt, Karmijn SNV Advisor Institutional Development, Sta Cruz Márquez, Alejandra Advisor Governance, Royal Netherlands Embassy, La Paz Márquez, Marcelo UDECO, Sta Cruz Martinez, Sheila Director for Planning, Prefecture, Sta Cruz Moreno, Cecilia SNV Advisor Sta Cruz Macoño, Delcey Gender Secretariat of three unions, San Xavier Montero, Adriana Representative Participación Ciudadana y Control Social, Sta Cruz Mercado, Jorge Directorate Natural Resources, Prefecture, Sta Cruz Miranda, Hugo Directorate Natural Resources, Prefecture, Sta Cruz Ortega, Fausto Member Comité de Vigilancia San Rafael Oyarzún, Lily SNV Advisor Sta Cruz Petri, Robert Head Development Co-operation, Royal Netherlands Embassy La Paz Pacheco B., Diego SNV Consultant Land Use, La Paz Pérez Bustamante, Martín Vice minister for Departemental Decentralisation, La Paz Portocarrera Martínez, Carlos Vice minister for Popular Participation, Sta Cruz Rama, Marlena OICH Candelaria, Concepción Rojas, Isaías ex president Comité de Vigilancia San Rafael; ACOVICRUZ, Sta Cruz Rosa SICAL, Concepción Salinas Mulder, Silvia SNV Advisor Local Governance, La Paz Sanchez, Oscar SNV Advisor, Sta Cruz Sandoval, Freddy AFIG, Sta Cruz Sandoval, Vania Ex Departmental Delegate for Anti-Corruption, La Paz Saucedo B., Ricardo Directorate Natural Resources, Prefecture, Sta Cruz Siriocco, Miguel technical committee CICOL, OICH, Concepción Tapia B., Tania SNV Advisor, Sta Cruz Toornstra, Conny SNV Portfolio Coordinator, Sta Cruz Valdivieso, Adriana UDECO, Sta Cruz Valencia, Samuel Member Comité de Vigilancia, San Rafael Vázques, Florencio representative of 12 communities, Comité de Vigilancia, San Rafael de Wit, Chiel SNV SPARK Institutional Development, La Paz Witteveen, Bert SNV Advisor Forestry, Sta Cruz

Kenya Bohonegne, Worku Country Director SNV Ghana, future Regional Director East and Southern Africa (as of 1.1.06) Dido, Sabdyo Coordinator North Kenya Portfolio

126 Evaluation SNV Kureiya Esintele, Allyce Coordinator South Rift Portfolio Nieuwe Weme, Gerard Regional Director, East and Southern Africa Onyango-Omondi, Susan Communication officer, SNV Nairobi Worsley, Stuart Country Director SNV Kenya and Sudan SNV Kenya, North Rift Portfolio Arunga, Tito Advisor Market Access for the Poor Glotzbach, Rudolph Coordinator North Rift Portfolio Kitala, Jechoniah Advisor Market Access for the Poor Korir, Selline Advisor Peace Building and Conflict Transformation Langat, Japhet Advisor Natural Resource Management Langat, Joseph Advisor Peace Advisor Peace Building and Conflict Transformation Building and Conflict Trandformation Maina, Mary Advisor Market Access for the Poor Muhoro, Purity North Rift portfolio secretary/administrative assistant Musamdi, Martin North Rift Portfolio administrator/financial management Odallo, Gollian North Rift Portfolio administrator Wanga, Moses Advisor Natural Resource Management Wangalwa, Kizito Advisor RALG Keiyo County Council Chebore, Vincent Chairman, Keiyo County Council Chemesis, Barnabas Ag. County Treasurer Cheruiyot, Joseph Internal Auditor Koros, Shadrack County Clerk Langat, Nathan Works, Superintendent Limo Komen, David Administrative Officer II Richard, Barnabas Accountant Pastoralist Area Development Organisation (PADO) Kapelpus, Paul PADO staff (Honey Sector) Kiranu, Stella Board Member Lokir, Stephen T.G.Os Representative Lotunyo, Luka PADO staff (Accountant Samutei, Philip Board Chairman Sangwatei, Geoffrey Programme Co-ordinator Honey Care Patel, Vipul Commercial Specialist Others Amolo, Monica Women’s Shadow Parliament Awimbo, Janet Africa Coordinator, Impact Alliance Duijfjes, Gerard Deputy Head of Mission, Royal Netherlands Embassy Ethuro, Hon Member of Parliament Karu, John Economist, Ministry of Local Government. Materu, Jossy Regional Director, Municipal Development Partnership Mboga, Hamisi Secretary General, ALGAK Singoeie, Korrir Executive Director, Centre for Minority Rights Development

Mali

Evaluation SNV 127 SNV-Régionale/Burkina Faso Labo Marafa, Hindatou SNV West & Central Africa Region, Regional Human resources Advisor Schalenbourg, Hilda SNV West & Central Africa Region, Regional Administration Manager Witte, Jan de SNV West & Central Africa Region, Regional Director SNV Mali

Baye, Sonia Le Conseillère PSE et GL Equipe Nationale Beyssac Bernard, Conihl de CT Sénior, Développement économique rural Coulibaly, Bakary Coordinateur SNV-Koulikoro Diarra, Souleymane Coordinateur Equipe Gestion des ressources naturelles SNV Mali Kluft, Bastiaan CT Equipe GL Nationale; CT AMM/Comm field officer Mayi Thioye, N’Deye Responsible financier Nelen, Joost CT équipe GRN Networkleader Dry Land management SNV west and central africa Osté, Renée Conseillère DIRO/Genre Equipe Nationale Regt, Jeanette de Directrice SNV Mali Sana, Souley CT équipe Nationale Gouvernance Locale Tadjou, Faousatou Coordinatrice Equipe Nationale, gouvernance locale SNV Mali MATCL/DNCT/CCN Adama, Sissouma Directeur National des Collectivités Territotiales Ibrahima, Sylla Conseiller Technique MATCL, Chef programme décentralisation Samake, Mohammed Cheik Coordinateur CCN CEPIA Sy, Ousmane Directeur Exécutif CEPIA Royal Netherlands Embassy Buikema, Harry Embassador Traoré, Mamadou Namory Conseiller Aide au Développement Vries, Peter de Chef de Mission Adjoint, Ambassade du Royaume des Pays Bas CAFO (Coordination des Associations et ONG Féminines) Touré, Konaré Fatoumata Secrétaire à l’Education Coordination Goundo Sissoko, Kouyaté Secrétaire aux Finances Nana, Diakaté Secrétaire à l’organisation Toure, Traoré Oumou Secrétaire Exécutif AOPP(Association des Organisations Professionnelles Paysannes) Diallo, Ousmane Barké Secrétaire/communication, Agro pasteur Mopti Boly, Faliry Secrétaire Général Diarra, Moctar Secrétaire Info-C, membre de la commission fruits et légumes Sidibé, Lassine Directeur Togo, Mamoudou Coordinateur HEVETAS Gindroz, Anne Sophie Directrice de Programme Sanou, Joseph Chef Programme promotion des filières, FENASCOM (Fédération Nationale des Associations de Santé Communautaire) Keita, Ibrahima Fadiala Président Malé, Tiémoko Trésorier Général Gouvernorat de Koulikoro Apo, El Hadj Mamadou Tissa Gouverneur de la Région de Koulikoro

128 Evaluation SNV Coulibaly,Bougouzanga Bazim Conseillers aux Affaires Economiques et Financières du Gouvernorat de Koulikoro Exploitants sable/graviers Union des Exploitants de Sable et Membres du Bureau de l’Union de Graviers de Koulikoro Exploitants de sables et de graviers Exploitants bois-Koulikoro Coulibaly, Nafa Président. Coordination de la Structure Rurale de Gestion du Bois de Koulikoro Diarra, Sékou Gestionnaire Assemblée Régionale et Conseil Baba Togola, Président de l’Assemblée Régionale de Cercle de Koulikoro Mme Tamboura Mah Keita, Présidente du Conseil de Cercle CT-Koulikoro Commune de Méguétan Maire Conseillers communaux Visite du marché de bois de Membres du Bureau de la structure rurale de gestion du bois de Dioïla Tiendo(Dioïla) exploitants de bois CT-Dioïla Cercle de Dioïla Préfet du Cercle Adjoint au Préfet CT-Dioila Conseil de Cercle de Dioïla Bounama Keta, Président du conseil de Cercle Yacouba Dowélélé Mariko, Maire de Kaladougou(Dioïla) Ousmane Bah, Secrétaire général de la Mairie Coopération allemande Betke, Dirk Coordinateur PACT-GTZ

Evaluation SNV 129

Documentation

Bolivia:

Hoenderdos Consult (2004). Beleidskader Technische Assistentie. Effecten op beleid, strategie en implementatie van PUM, PSO en SNV .

Ubels, J. (2005). Capacity development. From theory to SNV’s practice .

Toornstra C & M. Nostas (2003). Proceso de Renovación Permanente de Programa SNV- Santa Cruz: Análisis de Documentos base de programa .

Development and Policy Review Network (2005). Policy Brief. Millennium Development Goals as a Challenge for the Dutch Knowledge Community .

SNV (2001). Contours and Principles. Internal Organisation SNV .

Hartman, F., M.E.Canedo, E. Machicao, J. Blackburn (2004). Evaluación del Fondo para el Fortalecimiento de Organizaciones de la Sociedad Civil – FOSC .

Ministerio de Desarrollo Sostenible. Viceministerio de Tierras (2005). Base de Discusión para una política de Tierras para el Desarrollo Rural Sostenible.

SNV (2004). Informe Narrative Enero a Junio de 2004 e Informe Final. Proyecto “Investeringsfonds Lokaal Bestuur Bolivia ” (BOAK).

SNV (2005). Estructura y Enfoque del SNV. Una auto-reflexion en Programa Bolivia en base del documento original. Progreso en el logro de las 10 características establecidas en el Convenio de Subsidio con DGIS para el período 2002-2006 .

SNV (2005). SNV Bolivia, Portafolio Santa Cruz (presentatcion 9 de Dic.2005).

SNV (2005). Proceso de Assesoría a la Prefectura del Departemento Santa Cruz .

SNV (2005). Análisis de la Contribución del Portafolio del SNV en Santa Cruz en la Generación de los impactos a nivel Globa.l

SNV (2005). Contribución del portafolio de SNV Santa Cruz en la generación de Impacto en la Reforma del Estado.

SNV (2005). Annual Report 2004. SNV Bolivia.

SNV (2004). Annual report 2003 Bolivia.

SNV (2004). Management Agreement 2005/2006. Latin America Region .

Evaluation SNV 131 SNV, PNUD, GTZ, USEN, SIPRECOS (2004 ). Propuesta Plan de Acción Inmediata. Fortalecimiento a la Gestión Descentralizada de Conflictos.

SNV Collaborative Forest Management SNV Latin America (web).

SNV Portafolio SNV Santa Cruz (web).

Kenya:

SNV (2003). Annual Report SNV Kenya 2002 . SNV Kenya.

SNV (2004). Annual Report SNV Kenya 2003 . SNV Kenya.

SNV (2004) Planning 2005-2006 . North Rift Portfolio, SNV Kenya.

SNV (2004). Strengthening local governance. Finding quality advisory approaches. SNV East and Southern Africa region.

SNV (2005). Annual report SNV 2004 . SNV The Hague.

SNV (2005). Annual Report SNV Kenya 2004 . SNV Kenya.

SNV (2005). Bricks, mortar and capacity building 1965-2005. 40 years of SNV Netherlands Development Cooperation . SNV The Hague.

SNV (2005). Capacity Development Services. A learning approach to enhancing capacity in SNV East Africa. SNV East and Southern Africa region.

SNV (2005). Case Study of Samburu District Peace Committee . SNV Kenya North Rift Portfolio.

SNV (2005). Case Study: Keiyo County Council and Inter Council Forum . SNV Kenya North Rift Portfolio.

SNV (2005). Case Study: Samburu Integrated Development Programme . SNV Kenya North Rift Portfolio.

SNV (2005). Case Study: Samburu Traders’ SACCO . SNV Kenya North Rift Portfolio.

SNV (2005). Case Study: Samburu Women Advocacy Network . SNV Kenya North Rift Portfolio.

SNV (2005). Draft Portfolio Annual Plan 2006 . North Rift Portfolio, SNV Kenya.

SNV (2005). January-April Progress Report 2005 . North Rift Portfolio, SNV Kenya.

SNV (2005). January-August Progress Report 2005 . North Rift Portfolio, SNV Kenya.

SNV (2005). Keiyo County Council – dealing with the impacts of decentralization .

132 Evaluation SNV

SNV (2005). Opening up New Markets Through Value Chain Deve lopment. The Case of West Pokot Honey. SNV Kenya North Rift Portfolio.

SNV (2005). Practice Area Responsive and Accountable Local Governance. Education Sector Sub Area . SNV Kenya, November 2005.

SNV (2005). Practice Area Responsive and Accountable Local Governance. Water Sector Sub Area . SNV Kenya, November 2005.

SNV (2005). Private Sector Development: finding quality advisory approaches, SNV East and Southern Africa region .

SNV (2005). September-December 2004 Progress Report. North Rift Portfolio, SNV Kenya.

SNV (2005). Strategy Paper for Market Access for the Poor Practice Area in SNV East and Southern Africa .

SNV (2005). Strategy Paper for Responsive and Accountable Local Governance Practice Area in SNV East and Southern Africa .

SNV (2005). Management Agreement 2005-06 SNV Region East and Southern Africa . Version 24-01-05.

Mali:

Adiza Lamien Ouando, Ousmane Sy c.s. (May 2005). Strategic evaluation of the local governance programmes (SNV Bénin, Burkina Faso, Mali), Final report (y compris powerpoint).

AFVP (Décembre 1994). De la gestion territoriale au développement local : Analyse du projet gestion de terroirs villageois à Tambacounda et proposition pour un futur programme .

CCC BANAMBA (Décembre 2001). Dispositif d’appui technique CCC BANAMBA: Rapport annuel 2001 .

CCCB (Janvier 2005). Plan annuel 2005 Centre de Conseil Communal de Banamba .

CCCK (Avril 2002). Plan 2e trimestre 2002 .

Houinsa, D.G. (Juillet 2003). Rapport de l’analyse du contexte malien en vue d’un positionnement stratégique de la SNV-Mali .

DECLIC/SNV-PDCD (Avril 1994). Rapport final : Etude sur les opportunités économiques dans le cercle de Dioïla - Cas de la commune de Massigui .

Evaluation SNV 133 DECLIC/SNV-PDCD (Avril 1999). Rapport de synthèse : Etude sur les opportunités économiques dans le cercle de Dioïla .

DECLIC/SNV-PDCD (Avril 1999). Rapport final : Etude sur les opportunités économiques dans le cercle de Dioïla - Cas de la commune de Diébé .

DNCT (Décembre 2001). Distinction honorifique .

DNCT/CCN (Décembre 2004). Dispositif d’appui technique aux collectivités territoriales : Rapport de fin de phase .

DNCT/CCN/SNV (Octobre 2005). Plan annuel 2006-2007 des CCC .

DNCT/SNV (Décembre 2004). Convention relative à la gestion du CCC d’Ansongo .

DNCT/SNV (Décembre 2004). Convention relative à la gestion du Centre de Conseil Communal de koulikoro, Banamba et Dioïla .

DNCT/SNV (Janvier 2000). Convention relative à la gestion du Centre de Conseil Communale de Koulikoro, Banamba et Dioïla .

DNCT/SNV (Janvier 2004). Convention relative à la gestion du Centre de Conseil Communale de Koulikoro, Banamba et Dioïla .

DNCT/SNV (Janvier 2005). Convention relative à la gestion du Centre de conseil communale de koulikoro, Banamba et Dioïla .

EGLK (Janvier 2005). Plan annuel 2005 .

MATCL (Décembre 2005). Rapport de mission dans la région de Koulikoro .

MATCL/DNCT/SNV (Novembre 2003) Evaluation du Programme d’Appui à la Décentralisation dans la Région de Koulikoro.

MATCL/Gouverneur de koulikoro (Juillet 2005). DAO relatif au recrutement d’un opérateur pour les CCC de Nara et Kolokani .

SNV (July 2002). SNV’s Advisory Practice, Emerging lines, Connecting peoples capacities, final version .

SNV (June 2005). Annual report 2004 SNV .

SNV (November 2005) Bricks, mortar and capacity building 1965-2005.

SNV (October 2000). SNV Netherlands Development Organisation, Strategy Paper, final draft, approved by BoD .

134 Evaluation SNV SNV (Regional Director SNV region west and central Africa & Board of directors) Management Agreement 2005-2006 Region West and Central Africa .

SNV et CEDLO (2004). La décentralisation au Mali: du discours à la pratique .

SNV MALI (Août 2005). Offre de candidature d’opérateur CCC de Kolokani .

SNV MALI (Octobre 2002).Appréciation rapport semestriel/annuel/Plan annuel.

SNV PDCD (Août 1998). Plan 2e semestre 1998 .

SNV PDCD (Août 1998). Rapport premier semestre 1998 .

SNV PDCD (Août 2001). Dispositif d’appui technique CCC K, B, D : Rapport de synthèse premier semestre 2001 .

SNV PDCD (Juillet 2000). Rapport de révision à mi-parcours 2000 .

SNV PDCD (Mai 2002). Manuel de gestion interne PDCD II .

SNV PDCD (Mars 1998). Rapport Bilan 1998 .

SNV PDCD (Mars 1999) Plan annuel 1999.

SNV PDCD (Mars 1999). Rapport bilan 1998 .

SNV PDCD (Octobre 2000). Plan annuel 2001 .

SNV PDRK (Décembre 2004). Plan annuel 2005 .

SNV PDRK (Novembre 2002). Expérience de la SNV dans la région de Koulikoro dans l’appui à la mise en œuvre de la décentralisation .

SNV PDRK (Novembre 2002). Plan annuel 2003 .

SNV PDRK (Octobre 2002). Plan annuel 2003 .

SNV PDRK CCCK (Août 2002). Renforcer les capacités des femmes et les hommes par une équipe compétente pour qu’ils puissent fournir les services et produits à toutes les catégories de la population : Maîtrise d’ouvrage .

SNV/EGL-K (Février 2004). Analyse de contexte de la région de Koulikoro.

SNV/Mali (Dec 2005). Positionnement de la SNV Mali à propos des 10 caractéristiques sur lesquelles la SNV s’est engagée (2002-2006).

SNV/PDRK/CCC (Décembre 2003). Rapport d’auto évaluation des performances des collectivités territoriales du cercle de Koulikoro .

Evaluation SNV 135

SNV-Burkina Faso (Mai 2005). Renforcement des capacités des communes urbaines au Burkina Faso .

SNV-Mali (12/2005). Objectifs stratégiques du domaine de compétences genre 2006-2008 SNV-Mali (version provisoire).

SNV-Mali (2005) Partnership SIDA/ASDI memo.

SNV-mali (Janv.2005). SNV-Mali, orientations 2005 .

SNV-Mali (Octobre 2005). Groupes de concertation/activités SNV-Mali, Orientations .

SNV-Mali Annual Planning 2004 .

SNV-Mali Annual report 2003 (excell)

SNV-Mali Outil d’auto-évaluatin des performances des collectivités territoriales.

SNV-Task force Genre CT-Flex Genre (Juin 2003). Cadre stratégique Genre de la SNV- Mali, 2003-2006.

Website SNV/Mali (2005) www.snvmali.org.

The Netherlands :

Hoenderdos Consult (november 2004). Beleidskader Technische Assistentie: Effecten op beleid, strategie en implementatie .

Information from the SNV website.

KIT (2004).Lessons learnt from a selected number of SNV evaluation reports .

Several hand outs from presentations prepared for the evaluation team by SNV.

SNV (12-22 July 2005). Management Audit 2005, Niger .

SNV (2001) Subsidieaanvraag .

SNV (April 6th 2004) Corporate annual report 2003, Connecting people’s capacities.

SNV (December 2004). Corporate plan 2005-2006, Consolidation connected with Growth .

SNV (December 7th 2005). SNV’s structure and approach, Progress on realization of 10 characteristics laid down in subsidy agreement with DGIS covering the period 2002- 2006 , (document prepared for the evaluation team).

136 Evaluation SNV

SNV (February 17th 2005). Evaluation Collaborative Forest Management , Final Report.

SNV (July 16th 2001). Contours and Principles, Internal Organisation .

SNV (July 2002). Design Internal Organisation SNV.

SNV (July 2002). SNV’s Advisory Practice, Emerging Lines .

SNV (June 2003). Planning & Monitoring Guide .

SNV (June 27th 2005). Optimizing Support, Design of the desired support organisation & framework for implementation .

SNV (May 2005). Strategic evaluation of the local governance programmes (SNV Benin, Burkina Faso, Mali), Final Report.

SNV (October 14th 2005). Optimizing Support, Supplement to Design of the desired support organisation & framework for implementation .

SNV (October 1st 2000). Strategy Paper .

Evaluation SNV 137

Case studies produced at portfolio level

Bolivia Balza Alarcón, Robert. Tierra, territorio y territorialidad indígena. Un estudio antropológico sobre la evaluación en las formas de ocupación del espacio del pueblo indígena chiquitana de le ex-reducción jesuita de San José, 2001. CPESC, APC, SNV. Identidad cultural y desarrollo de los pueblos indígenas de Santa Cruz, 2002.

Centro Indígena Paikoneka de San Javier, SNV. Cuentame una cosa: cuentos, mitos y otros relatos del pueblo chiquitano paikoneka de San Javier, 2003.

Trujillo, Raquel; Gretel Lambertín. Palabras de mujeres indígenas Guarayas, 2003.

SNV. El control social a partir de la experiencia de los Comites de Vigilancia del Departemento de Santa Cruz, 2003.

Gerben Stegeman, SNV. Gestion forestal comunitaria: una mirada analítica de los casos Guarayos, Lomerío y Santa Monica.

Kenya The North Rift Portfolio team has recently developed a number of cases as part of the BAP exercise and for (inter)national presentations. The team also prepared a number case studies (some still in draft form) in the fourth quarter of 2005, i.e. • Keiyo County Council – dealing with the impacts of decentralization; • Opening up New Markets Through Value Chain Development. The Case of West Pokot Honey; • Case Study of Samburu District Peace Committee; • Case Study Samburu Integrated Development Programme; • Case Study Samburu Women Advocacy Network; • Case Study Samburu Traders’ SACCO; • Case Study Keiyo County Council and Inter Council Forum.

Mali SNV/Mali publications, often co-productions with specialised institutions: • « De la colonisation du vide à la concertation sur le plein : réglementation de l’exploitation d’une zone pastorale à Samorogouan, Burkina Faso, International Institut for Environment and Development,Programme zones arides- Dossier N°129, novembre 2004, Joost Nelen, Nata Traoré, Moumouni Ouatara ; • « Dynamiser la santé communale : Construire des partenariats efficaces dans l’espace communal pour améliorer la qualité des services de santé », T.Hilhorst, D. Bagayogo, D. Dao, E Lodenstein et J. Toonen, KIT, octobre 2005 ; • La décentralisation au Mali : du discours à la pratique, SNV-CEDELO, KIT, Bulletin N°358, 2004 ; • La decentralisation au Mali : Maitrise d’ouvrage (3 tomes), réalisé avec le PACT/GTZ et Helvetas en appui au MATCL/DNCT, 2002 ;

Evaluation SNV 139 • L’outil d’auto évaluation des performances des collectivités territoriales réalisé avec le PACT/GTZ et Helvetas en appui au MATCL/DNCT, 2004 ; • Guide pratique de gestion des archives des collectivités territoriales, réalisé avec le PACT/GTZ et l’AFVP en appui au MATCL/DNCT, 2004.

140 Evaluation SNV Annex B Capacity Development Services

Evaluation SNV 141

Capacity Development Services

The following tables provide examples of services provided by SNV to clients out of the list of pre-selected clients in the portfolios Santa Cruz (Bolivia), North Rift (Kenya) and Koulikoro (Mali).

Nature of Capacity Development Service Examples Diagnostics and learning Assistance to diagnostics, planning, and learning Accompaniment of Prefectural planning process (Santa Cruz; SNV ‘produces’ methodologies, procedures and skills); facilitation of the drafting of the ‘support plans’ or municipal development plans (Koulikoro, Dioila) Training and coaching Sand operators Koulikoro, foresters Dioila, indigenous (SNV contributes to the professional formation of organisations Santa Cruz human resources) Monitoring of the implementation of development Collectivités territoriale (CT) Mali plans Development of didactical tools CT Koulikoro, CT Dioila; sand operators Koulikoro, foresters Dioila Organisational Development Assistance to organisational and institutional Sand operators Koulikoro, foresters Dioila umbrella development organization of municipal transparency committees, Santa Cruz Organisational analysis Most/all relationships between SNV and clients have an organisational analysis in the initial phase, often before signing the agreement. The aim is getting a shared understanding (among stakeholders) of the SWOT of an organisation and to inditify the advisory requirements. Network and partnership building Facilitation of multi-stakeholder processes (SNV Umbrella organizations and networks like “connects” organisations and people) “Participación ciudadana” (Santa, Cruz); The Impact Alliance, and the Municipal Development Partnership (Kenya) Material and financial support for the organisation of CT Koulikoro, CT Dioila sand operators Koulikoro, workshops foresters Dioila Facilitation of south-south relationships between CT Koulikoro, CT Dioila, umbrella organization of clients and other, similar actors or relevant service municipal transparency committees Santa Cruz providers Influencing institutional change Punctual consultancies Land policy, Bolivia Conflict mediation (SNV provides instruments, Support Prefectura EDAC, later INRA (Santa Cruz) manpower and knowledge)

Evaluation SNV 143 Modality Examples SNV as trainer in capability enhancement of human resources Sand operators Koulikoro, foresters Dioila, municipal transparency committees Santa Cruz SNV as provider of manpower Assistance to Ministry for decentralization (Sistematisation of experiences, construction of data base, (Mali); assistance to NGO in Bolivia; many developer of plans, organization of seminars, research) case studies SNV accompanies organizations and ‘joint effort’ Almost all clients (Sistematisation of experiences, construction of data base, developer of plans, organization of seminars, research) SNV articulates, groups actors around a certain theme, links Almost all clients horizontally and vertically, knowledge brokerage, conflict mediation SNV as contractor of specialist knowledge Land policy Bolivia (Consultancies) Material and financial support for the organisation of workshops, International expose to official of the meetings with partners in a network, facilitation of south-south umbrella organization of municipal contacts transparency committees Lobby and advocacy Training in lobbying and advocacy for various Malian clients ; Cultural heritage (Chiquitanos, Bolivia); Union of women associations (Mali)

144 Evaluation SNV