Development of FINDETER's Prioritisation Tool

Development of FINDETER’s Prioritisation Tool - FiPT

Setting up projects selection to ensure Social Impact in Colombia

London School of Economics and Political Science Capstone Project 2021 Report

April 2021

Acknowledgements

Our team would like to thank everyone involved both at LSE and FINDETER for providing us with the amazing opportunity to work on this project.

We would like to thank everyone at the Economic Studies Department at FINDETER and especially Monica, Pablo, Luis and Beatriz, for their availability and for their always constructive feedbacks. Special thanks to the numerous staff from the four Business Lines that agreed to be part of our work.

We would also like to thank our supervisor, Professor Rafael Hortala-Vallve, as well as the Capstone course convenor, Professor Bob Babajanian, for their support and for contributing with their time and knowledge to our Capstone.

Authors

Sandra Barletti Lucie Marie Sheyla Enciso Irodotos Nikolaidis

FINDETER 2021 Capstone project

Table of Contents

Acknowledgements ...... 1 List of figures ...... 4 List of tables ...... 4 List of annexes ...... 5 Acronyms ...... 5 Executive Summary ...... 7 1. Introduction ...... 10 1.1. Objectives and research question...... 10 1.2. Policy problem and its relevance ...... 11 1.2.1. FINDETER’s role and structure ...... 11 1.2.2. The role of infrastructure in Colombia’s development ...... 12 1.2.3. The need for social impact measurement and a comprehensive decision-making process ...... 13 1.3. Research justification ...... 14 1.4. Outline ...... 15 2. Policy problem ...... 16 2.1. The importance of social impact measurement in decision-making ...... 16 2.2. FINDETER’s journey towards social impact measurement ...... 18 2.2.1. Past efforts on ex-ante social impact measurement ...... 18 2.2.2. Current tools in social impact measurement ...... 21 2.3. Motivation and objective of this Capstone project ...... 22 3. Literature review ...... 24 3.1. Diagnosis of existing methodologies for ex-ante evaluation ...... 24 3.1.1. Cost-benefit analysis ...... 25 3.1.2. Cost-effectiveness analysis ...... 29 3.1.3. Multi-criteria analysis...... 30 3.2. Best practices of MCA applied to investment prioritisation ...... 33 3.2.1. MCA studies implemented by OECD countries...... 33 3.2.2. MCA studies implemented by countries in LAC region ...... 34 4. Methodology for FINDETER’s prioritisation tool ...... 36 4.1. Analytical framework ...... 36 4.1.1. Principles for the decision process ...... 36 4.1.2. Building the decision process with FINDETER ...... 36 4.1.3. Ranking of projects as a result of the decision process ...... 37 4.2. Data collection ...... 37

FINDETER 2021 Capstone project

4.3. Design process for the development of FINDETER’s prioritisation tool ...... 39 4.3.1. Step N°1: Construction of hierarchies ...... 39 4.3.2. Step N°2: Priority setting ...... 47 4.3.3. Step N°3: Weight estimation...... 50 4.3.4. Step N°4: Logical consistency...... 53 4.2.5. Step N°5: Alternatives assessment ...... 55 5. Results & discussion ...... 57 5.1. Final decision-making process for FINDETER ...... 57 5.1.1. Hierarchic synthesis ...... 57 5.1.2. Discussion and Limitations ...... 58 5.2. Simulation on 12 projects selected by FINDETER ...... 62 5.2.1. Projects sample selection ...... 62 5.2.2. Prioritisation results ...... 64 5.2.3. Sensitivity analysis ...... 69 5.2.4. Implications and potential of FiPT ...... 71 5.3. Design of the FiPT template ...... 74 6. Policy Recommendations and Concluding Remarks...... 76 6.1. Policy Recommendations ...... 76 6.1.1. Co-construction process and socialisation of results ...... 76 6.1.2. Transparency and accountability ...... 76 6.1.3. Organizational learning and continuous improvement ...... 77 6.1.4. Scope of impact and complementary efforts towards data collection ...... 78 6.2. Concluding Remarks ...... 80 7. References ...... 83 8. Annexes ...... 87

FINDETER 2021 Capstone project

List of figures

Figure 1. Timeline of FINDETER’s past efforts and current tools ...... 18 Figure 2. Summary of methodologies for ex-ante impact assessment of projects 25 Figure 3. MCA – case studies in OECD countries ...... 33 Figure 4. MCA - case studies in LAC countries ...... 34 Figure 5. Principles for the selection of indicators ...... 38 Figure 6. Steps for the application of MCA methodology ...... 39 Figure 7. Example of matrix with the scale for the pairwise comparison ...... 49 Figure 8. Matrix with the scale for the pairwise comparison of main criterion .... 49 Figure 9. Weights estimated by FINDETER’s business lines ...... 50 Figure 10. Example of the matrix with the scale for the pairwise comparison ..... 51 Figure 11. Matrix with FINDETER’s pairwise comparison of main criterion ..... 52 Figure 12. Summary of weights calculated for the four main criteria selected for the prioritisation tool ...... 53 Figure 13. Consistency check matrix for the FiPT’s four criteria ...... 55 Figure 14. Example ranking of projects by using FiPT ...... 56 Figure 15. Project’s prioritization process - hierarchic synthesis ...... 58 Figure 16. The 12 projects selected by FINDETER ...... 63 Figure 17. Ranking the 12 projects by score ...... 64 Figure 18. Ranking the 12 projects by overall score and alignment with national targeting ...... 66 Figure 19. Ranking the 12 projects by overall score and contribution of each criterion1/ ...... 67 Figure 20. Project’s comparison of FiPT results with ConTREEbute results ...... 68 Figure 21. Projects' scores under different business lines’ weights ...... 70 Figure 22. Projects' scores under Principal Component Analysis ...... 71 Figure 23. Correlation between projects’ CAPEX and score ...... 72 Figure 24. Project’s score by Sector ...... 73 Figure 25. Project’s score by FINDETER’s business line ...... 73 Figure 26. Introduction of the FiPT template in Excel ...... 75

List of tables Table 1. Summary of comparative analysis between CBA, CEA and MCA in the context of FINDETER ...... 32 Table 2. List of indicators as a proxy for gaps across social and competitive sectors ...... 43 Table 3. Saaty scale for the pairwise comparison...... 48 Table 4. Example of the estimation of total score for the ranking of projects ...... 56

FINDETER 2021 Capstone project

List of annexes

Annex 1. Terms of Reference ...... 87 Annex 2. FINDETER’s methodology for project prioritisation ...... 95 Annex 3. MCA case studies by OECD countries ...... 97 Annex 4. MCA case studies by LAC countries ...... 98 Annex 5. Example of CBA methodology ...... 99 Annex 6. Example CEA methodology ...... 100 Annex 7. Summary of the indicators for the implementation of the FiPT ...... 101 Annex 8. Justification behind indicators of FiPT ...... 106 Annex 9. Examples of ranking projects using indicators for FiPT ...... 108 Annex 10. Results of the pairwise comparison for each subcriteria and indicators ...... 110 Annex 11. FINDETER’s operations share by each business line in 2020 ...... 112 Annex 12. Description of projects’ sample selected by FINDETER ...... 113 Annex 13. Template for the interviews on the MCA framework with representatives of FINDETER’s business lines ...... 118 Annex 14. Main findings and verbatims from the interviews with the representatives of FINDETER’s business lines ...... 121 Annex 15. Template with the Saaty Scale ...... 125 Annex 16. Template with the dummy variables tables sent to the Economic Studies Department and to the Business Lines ...... 126 Annex 17. Excel template for FiPT ...... 127

Acronyms

CAPEX Capital Expenditure CEA Cost Effectiveness Analysis CBA Cost Benefit Analysis DNP Departamento Nacional de Planeación de Colombia (Colombia’s National Planning Department) DANE Departamento Administrativo Nacional de Estadística (Colombia’s National Statistics Department) ET Entidad Territorial (Territorial Entity) FINDETER Financiera de Desarrollo Territorial (Colombia’s Development Bank for Territorial Development) FBK Formación Bruta de Capital (Gross Capital Formation) IDF Índice de Desempeño Fiscal (Fiscal Performance Index)

FINDETER 2021 Capstone project

MCA Multicriteria Analysis MDM Municipality Performance Management NDP National Development Plan OECD Organisation for Economic Co-operation and Development OPEX Operational Expenditure PDET Programas de Desarrollo con Enfoque Territorial (Development Programmes with a Territorial Focus)

FINDETER 2021 Capstone project

Executive Summary

Policy Problem and Motivation for this Capstone Project

FINDETER, as a national development bank in Colombia that aims to tackle the absence of resources from the private financial system, is involved in the planning, structuring, financing and execution of infrastructure projects to accelerate the development of local municipalities.

As part of its aspiration to become the country’s leading development bank, in recent years, FINDETER has focused on developing diagnosis tools that could assist the organisation in their goal of measuring social impact. However, FINDETER’s current decision-making process for the prioritisation of projects is fragmented and heterogeneous across the organisation’s departments (business lines).

Given that FINDETER faces resource constraints regarding the number of projects it can proceed with at any given time, this Capstone project had the principal objective of developing a standardised and comprehensive framework that would assist the decision- making process and systematise the data collection on key impact indicators. The aim of the tool is to prioritise projects with the greatest relative social impact, enhancing the homogeneity, transparency and legitimacy of the project selection process within FINDETER, as well as the efficiency of resource allocation.

Literature Review

Following rigorous research comparing the advantages and disadvantages of different methodologies for ex-ante impact measurement, Multicriteria Analysis (MCA) was identified as the optimal analytical method to be employed, given FINDETER’s goal. Compared to two commonly used alternative methodologies, namely cost-benefit analysis and cost-effectiveness analysis, MCA does not provide an absolute estimation of each intervention´s individual impact. Rather, it enables projects to be ranked according to their potential relative impact, based on their performance in a set of defined criteria.

FINDETER 2021 Capstone project

Used by many OECD and neighbouring countries to prioritise infrastructure projects, the MCA framework has several advantages for FINDETER’s objective of allocating financial resources to the relatively most impactful projects in Colombia. It makes it possible to capture a wider range of relevant criteria, to include quantitative and qualitative indicators, and to encapsulate both project- and municipality-level data, thus extending the scope of impacts that can be assessed. Finally, MCA can also be designed to be flexible enough to continuously assess new projects at different stages of their design (profile phase, pre- feasibility, feasibility, execution and operation phases), matching FINDETER’s operational processes.

Study Design and Methodology

Using MCA as our theoretical framework, we identified 4 impact dimensions to reflect the priorities of the organisation and the government: (1) Economic potential; (2) Social impact; (3) Institutional capacity and (4) Strategic importance. These 4 criteria, and their corresponding importance, were incorporated on the basis of an iterative process, including weekly workshops with the Economic Studies Department, as well as interviews with business lines’ representatives. The final selection of indicators was then revised following a thorough data analysis of their completeness, redundancy and operationality.

The pairwise comparison of each criterion, sub-criterion and indicator completed by the Economic Studies Department, using the Saaty scale, allowed our team to establish weight estimates for each of them, reflecting their relative importance to FINDETER.

Results and Discussion

The application of the MCA methodology to the context and priorities of the entity lead to the development of the FINDETER project prioritisation tool (FiPT), which enables the ranking of projects according to their relative impact, within and across sectors, territories and business lines. In the final design of FiPT, the 4 criteria include 11 sub-criteria and 18 indicators. Overall, FINDETER places relatively greater importance on projects’ economic potential (weight of 38.6%) and social impact (weight of 38.4%), followed by institutional capacity (weight of 16.7%) and strategic importance (weight of 10.6%).

FINDETER 2021 Capstone project

Data availability constraints at the project level and minor concerns regarding the representativeness of indicators’ weights led to adding features to FiPT that minimise the possibility for subjectiveness and creates space for robustness checks. This aims to foster technical deliberation when prioritising projects. The application of FiPT to a sample of 12 projects, selected by FINDETER, reflects FiPT’s tendency to favour projects according to their territorial-level data and to capture the multidimensional effects of projects. These findings were confirmed by sensitivity analysis.

Policy Recommendations

Some of the key findings that surfaced during the development of FiPT, as well as from the preliminary results of this pilot test on the 12 selected projects, guided the formulation of a number of recommendations for FINDETER. First, the entity should promote an iterative and co-construction process, notably by creating an inclusive forum for debate and ensuring the dissemination of complete information regarding the underlying mechanics of the MCA methodology to all participants. This would in turn promote organisational learning and continuous improvement.

Second, FINDETER should make transparent the decision of criteria, criteria weighting, and sensitivity analysis prior to their selection, while making publicly available and open to third-party review the data used and the final outcomes.

Finally, as this Capstone project presents what should be considered a first version of FiPT, we anticipate the tool to be improved over time with updated and more granular data. This report therefore recommends improving data collection at the project level, updating the databases of administrative indicators and considering the integration of other criteria to extend the scope of the impact measured.

FINDETER 2021 Capstone project

1. Introduction

1.1. Objectives and research question

FINDETER is a national development bank in Colombia, focused on infrastructure projects that promote sustainable growth and enhance the competitiveness of Colombia’s municipalities.

The number of municipalities reached by FINDETER increased from 339 in 2016 to 685 in 2020 (FINDETER, 2019). In light of this rapid expansion, the organisation developed a strategy to demonstrate the social impact contribution of its operations at the territorial level. Several projects have been carried out to estimate the social impacts of the entity, the most recent being the recruitment of the consulting firm ConTREEbute to estimate the social internal rate of return of FINDETER in 4 sectors (Housing, Drinking Water and Sanitation, Urban Mobility and Energy), commonly regarded as competitive sectors.

FINDETER's rapid expansion, coupled with its desire to enhance and communicate its social impact, and taking into account persistent constraints on available administrative capacity as well as financial resources, have highlighted the need for a systematised and institutionalised approach to decision-making in order to prioritise projects with the greatest social impact. Given that this approach is currently lacking, FINDETER's Economic Studies Department called upon our LSE team with the objective of formulating a methodology that would address this gap.

The aim was to identify impact dimensions that cut across all of FINDETER's operative sectors and to build a tool that would allow the entity to compare projects both within and across the competitive and social sectors. Based on the different impact dimensions identified1 the tool would enable FINDETER to prioritise projects with the greatest relative holistic impact .

1 See Annex 1 with the Terms of Reference.

FINDETER 2021 Capstone project

The construction of this project prioritisation tool developed by this Capstone team, in co- construction with FINDETER’s business lines and Economic Studies Department, had a dual objective: (1) Estimate ex-ante projects’ relative social impacts, taking into account current data and resource capacity constraints; and (2) Prioritise investments in projects with the relative greatest impact. At the same time, it would achieve two aims underlying the implementation of such a tool: (1) Improve the transparency and accountability of FINDETER’s decision-making process; and (2) Increase organisational learning by creating a systematic reflex to collect data on the impacts of its operations.

The research question that this report aims to answer is therefore: How can FINDETER develop a consistent and institutionalised decision-making tool to prioritise infrastructure projects based on their relative social impact, taking into account the resources, time and data availability constraints that FINDETER faces as a development bank?

1.2. Policy problem and its relevance

1.2.1. FINDETER’s role and structure

FINDETER was established with the aim of assisting the most vulnerable municipalities, whose socioeconomic development is hampered by market failures that are often associated with developing countries. The organisation proposes sustainable and integral solutions to development along four business lines: Planning, Structuring, Financing and Execution (FINDETER, 2020). In the Planning phase, the entity works in partnership with local municipality authorities to design a road map of short, medium and long-term strategic projects. The Structuring phase involves assessing the technical, legal and financial requirements associated with each proposed project. With regards to the Financing line, FINDETER channels funds from the Central Bank of Colombia to the proposed projects or investments via financial intermediaries. Finally, for projects that fall within the Execution line, FINDETER is responsible for providing their expertise to ensure the successful development and execution of the proposed project.

Having already covered 58% of the national territory, FINDETER envisions to realise its

FINDETER 2021 Capstone project ambitions of evolving into Colombia’s leading developing bank by 2022. Since the start of the 2018 – 2022 presidential term, the entity has disbursed USD $1,42 billion, i.e. 2,28% of Colombia’s GDP, in 13 key economic sectors2.

1.2.2. The role of infrastructure in Colombia’s development

In recent decades, the development community has seen a shift from a “traditional approach” of development, which emphasises the promotion of economic growth, to one that emphasises addressing issues of structural inequality and inclusive growth (Banerjee and Duflo, 2011).

As recommended by the OECD, of which Colombia became an official member in April 2020, infrastructure has a crucial role in addressing the persistent income inequalities and significant differences in living conditions between municipalities (OECD, 2019). The OECD thus recommends structurally reforming Colombia’s infrastructure in a way that would promote inclusive growth and boost productivity, while simultaneously improving the efficiency of public spending (OECD, 2019).

The aforementioned paradigm shift is reinforced by Colombia’s National Development Plan (NDP), which covers the current presidential term (2018 - 2022) and is based on three key pillars: (1) Equity, (2) Entrepreneurship; and (3) Legality. Each pillar has distinct objectives, which need to be taken into account in any development strategy, including infrastructure. Goals, such as the reduction of multidimensional poverty within the equity pillar and the enhancement of institutional capacity within the legality pillar, were instrumental in the development phase of our tool.

Therefore, FINDETER has a crucial role to play in ensuring that the selection of its infrastructure projects corresponds to these strategic development orientations, promoting inclusive growth and raising its fight against structural inequalities.

2 The 13 sectors include: Transportation, Energy Development, Urban Development, Housing, Health, Education, Environment, Drinking Water and Sanitation, Tourism, Telecommunications, Entertainment and Sports, Culture, and Fiscal Consolidation Requirements.

FINDETER 2021 Capstone project

1.2.3. The need for social impact measurement and a comprehensive decision-making process

This shift towards promoting infrastructure for inclusive development raises the question of how FINDETER can systematise its decision-making process to incorporate dimensions of social impact. In addition, the high degree of decentralisation that characterises the planning and implementation of infrastructure in Colombia reinforces FINDETER's role as a relevant actor for inclusive territorial growth. An institutionalised, evidence-informed and transparent decision-making process, aligned with FINDETER’s organisational priorities, is thus a necessary condition for the delivery of effective infrastructure projects with better outcomes, as well as for improving the entity’s governance and performance (UK, 2016). Moreover, and as developed in greater detail in the ‘policy problem’ section, in a context of limited resources, ex-ante impact assessment is a central tool to better inform FINDETER’s decisions. Such assessments are crucial to inform the trade-offs involved, to better align resources to projects with the greatest social impact (OECD, 2017), and thus to make the entity’s decision-making process more legitimate, transparent and efficient (Marcelo et al, 2016).

In this context, FINDETER’s strategy has transitioned towards one that places more importance on the social impact it generates. Specifically, different measurement frameworks and operational tools have been developed in recent years in an attempt to prioritise projects that generate the greatest social value, rather than solely emphasising their potential financial gains. This notion is highlighted in the Business Strategy 2019- 2022, which officially established the importance of social impact to FINDETER’s growth in the medium term and to the entity reaffirming its role as a key vehicle to facilitate local territories’ development (FINDETER, 2019).

FINDETER 2021 Capstone project

1.3. Research justification

Despite recent efforts to put social impact at the heart of its strategy and improve its impact measurement, FINDETER still lacks a comprehensive framework for systematically assessing ex-ante the social impact of projects under consideration, as well as a consistent decision-making process aligned with the entity’s social impact priorities.

Our diagnosis, which included interviews with the managers of FINDETER's business lines, revealed that each business line has divergent priorities and project selection criteria in the current decision-making process. To an extent, these independent decision-making processes are due to each business line intervening at different stages in a project’s life cycle. In addition to this heterogeneous, siloed decision-making process not being aligned with FINDETER's management priorities, our diagnosis highlighted significant constraints regarding the availability of data on both projects and their potential impacts, as well as time and administrative capacity constraints for the assessment of projects in the decision- making process.

Based on this initial diagnosis, our research proposes to explore different methodologies of ex-ante social impact assessment, in order to identify which is the most relevant and appropriate in the context of FINDETER. Our research found multi-criteria analysis (MCA) to be the optimal methodology. Moreover, our research allows the formalisation and homogenisation of data collection around the defined dimensions of social impact, aligned with government and FINDETER priorities. Finally, by applying the MCA methodology to FINDETER's specific context and needs, our Capstone team provides FINDETER with a tool for prioritising projects with the greatest relative social impact. This paves the way for an institutionalised, systematic and more transparent decision- making process to enable FINDETER to maximise the efficiency and equity of its infrastructure projects, ultimately strengthening its contribution to inclusive development- oriented infrastructure.

FINDETER 2021 Capstone project

1.4. Outline

This report is structured as follows. This chapter covered the introduction, where we discussed the objectives and research question of this capstone project, the policy problem and its relevance, as well as the justification for our research. Chapter 2 will assess in a greater detail the policy problem the project aims to address. Chapter 3 includes a literature review of existing ex-ante social impact evaluation methodologies and best practices regarding MCA, the most relevant methodology in FINDETER’s context. Chapter 4 will present the capstone project’s study design & methodology. Chapter 5 includes an analysis and discussion of our findings, as well as the results of applying the prioritisation framework to a sample of 12 projects. Finally, in Chapter 6, we present our main conclusions and our project’s policy recommendations.

FINDETER 2021 Capstone project

2. Policy problem

This section will first discuss the ever-increasing importance of social impact measurement in the sphere of local and national development. This will then be followed by a description of FINDETER’s analytic journey leading up to this Capstone project and its objective: the construction of a comprehensive and institutionalised framework for decision-making in order to prioritise projects with the greatest relative impact.

2.1. The importance of social impact measurement in decision-making

As a result of the shift towards a new development paradigm, a strong demand for evidence-based policy analysis has emerged within development institutions, which has been accompanied by the implementation of rigorous impact assessments (Dhaliwal, I. et al, 2012).

Practice in development institutions suggests that economic and strategic project selection, as well as feasibility studies, provide a sufficient basis for prioritising projects with the greatest societal value (Marcelo et al, 2016). However, in reality, development institutions in many countries lack the necessary resources to carry out extensive economic analysis and instead suffice to making decisions based on incomplete or second-best information (Marcelo et al, 2016). In practice, prioritisation may be based on politics or professional judgements, without a systematic process and clear and transparent selection criteria (Petrie, 2010), leaving room for corrupt or inefficient practices. In the face of the demand for evidence-based approaches, ex-ante impact assessments serve as a key tool for development banks to establish a systematic and institutionalised selection of projects or programmes (Marcelo et al, 2016).

First, ex-ante impact assessments enhance accountability and transparency in policy and investment decision-making. Given the constant pressures on scarce resources, public institutions face increasing pressure to present evidence to justify how their investments ensure an efficient and effective use of public resources (Münich & Psacharopoulos, 2014). Ex-ante impact evaluation offers the possibility to compare different resource allocation

FINDETER 2021 Capstone project strategies to inform institutions’ allocation choices (EU, 2017). At the same time, it enables decision-makers to prioritise the most desirable solution on the basis of the costs and impacts of the different projects, thus determining strategy planning priorities (Samset & Christensen, 2017).

Second, ex-ante impact assessments help strengthen legitimacy, by making decision- making processes more transparent and systematic, as well as by demonstrating the relevance of institutions through their performance (Marcelo et al, 2016). Finally, the use of evidence-informed approaches sheds light on the trade-offs inherent in any decision and highlight measurable priority results, thus increasing efficiency and equity in policy and investment decision-making (EU, 2017).

Such an approach provides development institutions with an opportunity to make their decision-making process more efficient and with greater transparency to their various stakeholders, who can then examine the different assumptions and evidence underlying resource allocation decisions (OECD, 2006).

FINDETER 2021 Capstone project

2.2. FINDETER’s journey towards social impact measurement

2.2.1. Past efforts on ex-ante social impact measurement

In the last 5 years, FINDETER has developed a number of tools to measure the social impact of its projects. The four most relevant tools are highlighted in figure 1.

Figure 1. Timeline of FINDETER’s past efforts and current tools

Own elaboration Source: FINDETER (2017, 2019, 2020)

FINDETER 2021 Capstone project

i. Methodology for the prioritisation of projects

The aim of the tool, developed by FINDETER’s Office of Project Structuring, was to prioritise projects based on their social impact and the commercial benefits they generate for FINDETER. The MCA methodology, which will be further examined in Chapter 3.1.1, involves a selection of criteria and sub-criteria with different weights of importance that capture the outcome of interest.

One key limitation of this tool is that the selection of projects is driven almost equally by the private benefits generated for FINDETER and social impact. The resulting prioritisation could be biased for the projects with larger commercial and financial benefits for FINDETER itself rather than the social impact generated by them. This previous use of the MCA methodology by FINDETER provides us with an opportunity to build on this approach, by shifting its focus from prioritising projects with the greatest commercial value to prioritising projects with the highest social impact.

ii. ConTREEbute’s methodology for the measurement of TIR Social

ConTREEbute, a Colombian consultancy firm, was tasked with developing a tool that could measure a project’s social return of investment. ConTREEbute opted for Cost Benefit Analysis (CBA) as their preferred methodology to calculate FINDETER’s aggregated social impact in four projects in the following sectors: Housing, Urban mobility, Energy, Drinking water and Sanitation. The four projects evaluated by ConTREEbute were selected by FINDETER, due to being “representative” of the organisation’s operations in terms of the business lines and the sectors in which FINDETER delivers services, as well as having available data for analysis.

FINDETER 2021 Capstone project

ConTREEbute’s methodology and results

ConTREEbute’s strategy included the measurement of the aggregated Internal Social Rate (TIR Social, by its acronym in Spanish) based on the individual estimations of each social return generated by four projects, one for each business line and sector of interest.

The calculation of the aggregated social impact considered the social return of each project adjusted by a specific weight (W) for each business line1/:

Project Project’s name Sector Business TIR line

Project 1 Strategic Transportation System Urban mobility Planning 170.9% in ibagué

Project 2 Housing project in Santiagomanda Housing Execution 49.7%

Project 3 Aqueduct and sewage system in Drinking water Financing 30.4% Paratabueno and Sanitation

Project 4 Solar-Diesel hybrid energy Energy Structuring 14.1% generation system in Miraflores

TIR Social Aggregated = TIR_project 1*W_planning + TIR_project 2*W_structuring + TIR_project 3*W_financing + TIR_project 4*W_execution

TIR Social Aggregated = 170.9%* 9% + 14.1%*13% + 30.4*34% + 49.7%*44%

TIR Social Aggregated = 49.316% 1/ This was estimated according to the proximity of the business line to the final impact of the project. Source: ConTREEbute (2020).

The development of this tool demonstrates FINDETER’s recognition of the importance of integrating social impact measurement as part of its decision-making process. In applying CBA, ConTREEbute aimed to calculate the social return of the projects FINDETER is involved with, as a proxy for the social impact the organisation generates. While the use of CBA is justified in the context of ConTREEbute’s work, an evaluation of this methodology and the grounds for opting for MCA instead in this Capstone project, will be further discussed in Chapter 3.1.2.

FINDETER 2021 Capstone project

2.2.2. Current tools in social impact measurement

Building on these past efforts, FINDETER has developed 2 project evaluation tools that are currently in use.

i. Strategic indicators report

FINDETER produces monthly strategy reports, aggregating project information from its business lines on four indicators: 1) Coverage, representing the number of municipalities reached by FINDETER’s projects; 2) Monthly beneficiaries; 3) Projects with impacts in more than one municipality; 4) Municipalities without structuring capacities reached by FINDETER, as part of the 102 municipalities the institution identified, based on their fiscal performance index.

ii. Achievements database

This database includes project achievements during the current presidential term (2018- 2022). Even though the database includes an extensive set of variables, the information is not complete for all the projects (e.g. benefited population, investment, status of the project).

Whilst both tools emphasise the importance of data collection for development institutions, including FINDETER, they are targeted towards monitoring a project’s progression rather than measuring its social impact. This creates a window of opportunity for this Capstone project to build on these data collection efforts, by incorporating some of their elements in the development of FiPT.

FINDETER 2021 Capstone project

2.3. Motivation and objective of this Capstone project

Whilst the aforementioned efforts shed light on FINDETER’s efforts to measure the social impact of its interventions, the organisation is still lacking a comprehensive framework.

Interviews conducted with members of staff from the four business lines, shed light on the need for a tool that can homogenise the decision-making process across the organisation, while enhancing the process’s transparency and legitimacy. Interviews also highlighted the need for the tool to help aligning FINDETER’s work with national development plans. A senior executive of the Planning department stated “the interest of such a tool is threefold: it allows FINDETER to act in a homogeneous way, to have a clear decision guide and, finally, to give credibility to the entity’s approach on how projects are selected.” Currently, each business line has its own priorities when assessing a project. For example, interviewees from the Planning line stated that their emphasis is on a project’s potential to improve a municipality’s institutional capacity, whereas senior executives from the Execution line revealed that their emphasis is on a project’s economic potential.

Another key issue to take into account regards the administrative and information availability constraints that FINDETER faces. First, the project proposals that the entity receives contain information relevant to an early stage of their design, restricting the number of indicators that is needed to evaluate the potential costs and benefits. As stated by interviewees from the Structuring line “It is important to note that the nature of products that are part of an earlier stage of the projects, as it is the case for products of the structuring business line, may not immediately reflect the impact that would be generated by the project. Hence the need for and importance of a tool to assess the socio-economic impact of projects.” It was thus important to design FiPT in a way that could harness the potential of data that is available even at a project’s early stage.

In the absence of a transparent and standardised decision-making process, FINDETER’s fund allocation across a large set of projects, proposed by either municipalities or the private sector, face the risk of falling back on unsystematic, ad hoc selection. This would

FINDETER 2021 Capstone project imply not basing the decision-making on a solid diagnosis of the territories’ needs - and therefore promoting less efficient or unnecessary projects-, and not responding to government and FINDETER’s priorities.

Therefore, this Capstone’s goal is to provide a framework for the identification, assessment and prioritisation of projects that FINDETER can fund, depending on the social impact they generate. Under a comprehensive approach that can accommodate a wider range of policy objectives when deciding the allocation of resources, this process will assess qualitative and quantitative indicators along different dimensions of projects.

FINDETER 2021 Capstone project

3. Literature review

3.1. Diagnosis of existing methodologies for ex-ante evaluation

Ex-ante impact assessment, monitoring and ex-post impact assessment are intrinsic to the policy cycle, as well as to the evidence-informed design of infrastructure projects. Ex-ante assessments, by estimating a priori the potential impacts of an intervention, allow for an optimised budget allocation and decision-making process, with the objective of selecting projects with the greatest impact (OECD, 2014). Conversely, ex-post assessments measure the proven effects of an intervention a posteriori, in order to draw lessons for the continuation of the project, its extension or for the implementation of similar interventions (OECD, 2014). Since the objective of this Capstone is to support FINDETER in improving its decision-making process prior to project implementation, this report focuses on ex-ante impact measurement.

When measuring the ex-ante impacts of infrastructure interventions, the most commonly used methodologies are: (1) Cost-benefit analysis (CBA), (2) Cost-effectiveness analysis (CEA) and (3) Multicriteria analysis (MCA). CEA and CBA are more sophisticated appraisal methods and, if systematically applied, can serve as a decision-making framework. However, when faced with administrative capacity and resource limitations, extensive assessments across all projects become unfeasible in the immediate term (Marcelo et al, 2016). Thus, MCA allows us to overcome these restrictions by expanding and diversifying the impacts that can be measured, as well as by including both quantitative and qualitative indicators (see figure 2).

FINDETER 2021 Capstone project

Figure 2. Summary of methodologies for ex-ante impact assessment of projects

Source: Contreras (2004)

3.1.1. Cost-benefit analysis

i. Introduction to CBA

Cost–benefit analysis (CBA) is a methodology that assesses the economic efficiency of proposed public policies or projects, through the systematic prediction of social costs and social benefits (Haverman and Weimer, 2001). CBA involves the identification and monetisation of the direct, indirect and intangible benefits and costs resulting from the project of interest (Haverman and Weimer, 2001). A project with positive net social benefits is considered economically efficient relative to the status quo (Haverman and Weimer, 2001). CBA is typically applied when it is possible to quantify and monetise a project’s impacts.

ii. FINDETER and CBA: ConTREEbute’s work

In the context of FINDETER, applying CBA would aim to monetise the potential positive and negative impacts associated with the implementation of a proposed project. Benefits

FINDETER 2021 Capstone project of an intervention can be defined as the positive impact of a project on the wellbeing of the communities of interest. The costs involve mainly the private costs incurred, while also considering external costs on the community of interest. Benefits and costs are then brought to present value. Subsequently, this will enable the measurement of the net present value, which subtracts the sum of present value costs from the sum of present value benefits.

As previously mentioned in chapter 2.2.1, ConTREEbute used CBA to calculate FINDETER’s impact. Specifically, ConTREEbute’s strategy involved measuring the aggregated Internal Social Rate (TIR Social, by its acronym in Spanish) based on the individual estimations of social returns generated by four indicative projects associated with the four business lines of FINDETER, each within one of the following sectors: Housing, Urban mobility, Energy, Drinking water and Sanitation.

iii. Advantages of CBA as an impact evaluation methodology

Despite its flaws for use in this Capstone project, CBA has a number of advantages. The key advantage of applying CBA for ex-ante impact evaluation is that it involves a comparison between costs and benefits of an intervention using the same monetary unit (White, 2016). This is important, as it enables decision makers to evaluate ex-ante which potential interventions may lead to a more efficient allocation of resources and the highest social returns. Furthermore, given that costs and benefits are measured using the same monetary unit, as long as the impacts of interest are objectively quantifiable, CBA can be applied to compare different projects with multiple and varying impacts across different sectors (European Union, 2017). In addition, CBA is the only methodology that clearly demonstrates whether the monetary benefits of an intervention exceed its monetary costs (Haveman, 2001). This enhances decision makers’ objectivity. Lastly, by discounting future costs and benefits to the present market value, measuring Net Present Value (NPV) enables the comparison of projects with different time horizons (World Bank, 1998).

FINDETER 2021 Capstone project

iv. Assessing CBA in the context of ConTREEbute versus this Capstone project

On the other hand, there are numerous fundamental limitations of CBA, especially when applied in the context of this Capstone project. It is thus important to address the methodology’s shortcomings and link them with ConTREEbute’s work to better understand why CBA was not the preferred methodology for our project.

The first disadvantage of CBA is that it is heavily reliant on numerous assumptions, a key one being that there are perfect markets (ConTREEbute, 2020). However, in reality, this is rarely the case, as many variables of interest are non-marketable (e.g. human life or peace). This is particularly evident when attempting to monetise specific impacts by relying on the subjectiveness of stated and revealed preferences methods. These methods often rely on an interplay of willingness to pay (WTP) and willingness to accept (WTA), which may be distorted due to information asymmetries. This issue was partially overcome by ConTREEbute, as their focus was on assessing the social impact of projects within the competitive sectors and relied on the use of variables whose economic valuation and subsequent monetisation is more feasible. However, this issue becomes more prominent when assessing impacts within the social sectors, including Health and Education, where there are information gaps regarding how individuals may value health and education.

Furthermore, CBA necessitates that all costs and benefits are monetised. This can often be challenging, as in the case of monetising benefits, some impacts are intangible and subjective. In addition, there are ethical implications of monetising impacts in the social sectors (World Bank, 2010). This was not as evident in the analysis conducted by ConTREEbute, given that they are measuring impacts within four competitive sectors where the impacts are more feasible to value. This issue becomes exacerbated when measuring impacts in the social sectors, which are of interest to this Capstone project, namely Health and Education. An independent evaluation into the use of CBA within the World Bank found the argument of unquantifiable benefits to be the most legitimate one against the methodology (World Bank, 2010). Specifically, World Bank policy mandates

FINDETER 2021 Capstone project cost-benefit analysis for all investment projects and cost-effectiveness analysis for projects that are “expected to generate benefits that cannot be measured in monetary terms” (World Bank, 2010).

Lastly, due to its narrow scope on efficiency, CBA does not account for aspects of a project’s distributional effects and sustainability (Atkinson and Mourato, 2015). The analysis conducted by ConTREEbute cannot take into account the distributional effects generated by the projects, despite the stark territorial inequalities that characterise the Colombian economy. While there have been attempts to address this issue by applying distributional weights, concerns remain (Adler, 2013). Atkinson and Mourato (2015) point to the existence of a gap between CBA and the strategic objective of sustainable development, which is likely to widen with the shift of international organisations towards a broader and multidimensional notion of development.

Therefore, while ConTREEbute was able to apply CBA to assess the four projects within the competitive sectors, it is deemed to be a sub-optimal methodology for this Capstone project on a number of levels. Firstly, the highly technical and project-specific nature of CBA presents a hurdle in terms of its ease of replication for projects across social and competitive sectors. One of the key requirements of the tool this Capstone project would develop for FINDETER was its usability by all members of staff across departments. Thus, seeking alternative methods to CBA was fundamental to this Capstone project.

Furthermore, CBA’s narrow scope falls short in terms of measuring the social impact of a project. Simply reducing a project’s social impact to the monetisation of its associated costs and benefits fails to encompass the holism of the term “social impact” in today’s development world. FINDETER requested a tool that is able to evaluate a high volume of projects across sectors, while promoting equitable and sustainable development for Colombia as a whole, as well as prioritising the municipalities that are in greatest need. Vining and Weimer (2015) state that CBA is appropriate when decision making is guided solely by the principle of efficiency.

FINDETER 2021 Capstone project

The aim of this section is not to dismiss CBA in its entirety, but rather to suggest, as per Vining and Boardman (2006), its embedding within a multi-goal analysis in which efficiency is but one of the goals. Taking into account the limitations of CBA in the context of this Capstone project, we proceeded to exploring alternative methodologies that are better suited to achieving our aims.

3.1.2. Cost-effectiveness analysis

Cost-Effectiveness Analysis is a decision-making assistance tool which enables the comparison or ranking interventions with identical objectives, in order to identify the most appropriate alternative to achieve a given result at the lowest cost (EU, 2017), i.e. the economically most efficient project. To that end, this methodology compares alternatives in terms of their cost per given effect or vice versa (Belli et al, 1998), i.e. the effect induced by incurring a certain financial cost. The result of CEA is a ratio (Ardila et al, 1998), in which the cost is expressed in monetary units and the incremental effects are expressed in non-monetary terms (Dhaliwal al, 2012).

Cost-effectiveness analysis has many advantages, making it a very useful decision-making, evaluation and communication tool (EU, 2017). Firstly, this method makes it possible to summarise and compare the outcomes of projects with similar objectives, in a transparent, simple and objective way, into a single quantifiable indicator (Dhaliwal et al, 2012). Furthermore, as opposed to CBA, CEA does not require the monetisation of benefits, allowing for greater flexibility. Moreover, it makes it possible to limit the number of assumptions made, compared to CBA (Belli, 1998). For these reasons, the CEA methodology is most commonly applied to compare projects within the Education and Health sectors (McEwan, 2012).

However, in order to lead to relevant and accurate results, CEA requires certain elements to be met, which limits its capacity to address FINDETER’s objective in this Capstone Project, i.e. prioritising projects within and across different sectors.

FINDETER 2021 Capstone project

Firstly, CEA is usually restricted to capturing only one effect and focuses on the main direct outcome of the intervention (IADB, 2020). Thus, if a project generates multiple outcomes (direct and indirect) and/or a priority outcome cannot be identified, then it is recommended to avoid CEA, as it is difficult to aggregate CEAs across multiple outcomes, and its use may be irrelevant or even counterproductive (EU, 2017).

Moreover, since the outcome or impact must be common to all the alternatives evaluated, CEA requires stable variables to support the comparison between similar interventions occurring in different contexts and vice versa (Dhaliwal et al, 2012). If projects have different outcomes and common, homogeneous and quantifiable units cannot be determined to enable comparability, then the use of CEA should be avoided (Evans and Popova, 2011).

Therefore, in the context of this Capstone project, whose aim is to compare projects within and across various municipalities and sectors with various outcomes and without common measure, it seems that CEA is not the most appropriate ex-ante evaluation method.

3.1.3. Multi-criteria analysis

Multicriteria Analysis is a methodology that simplifies the decision-making process for the selection of interventions or projects. Given the existence of several criteria that jointly affect an outcome of interest, MCA establishes a hierarchy among them to facilitate the decision-making process and estimates weights that summarise each criterion’s relative importance (Saaty, 2012). While compared to CBA and CEA, this methodology does not provide a specific approximation of each intervention´s individual impact, it allows the estimation of a final ranking that informs the relative priority among all the alternatives considered (DCLG, 2009).

In the context of FINDETER’s goal of allocating financial resources to the most impactful projects in Colombia, an MCA framework will permit considering a wider range of criteria that are relevant to the decision process. These include the projects’ characteristics and direct impacts – such as the number of beneficiaries-, as well as their indirect impact on

FINDETER 2021 Capstone project diverse dimensions at a municipal level – including economic, social and environmental-. Moreover, the criteria considered can be based on both quantitative and qualitative indicators, extending the scope of impacts that can be assessed, which would otherwise not be measured (Saaty, 2012). Finally, another strength of this framework regards its ability to be designed to be flexible enough to continuously assess new project proposals and projects in different stages of design (profile, pre-feasibility, feasibility, execution and operation) (Estevez, 2013).

Therefore, MCA can be easily implemented and adapted over time to FINDETER´s evolving process for project selection. It can deal with the large pipeline of proposals that are presented by territorial and private entities while making use of reasonably attainable information (DCLG, 2009). More importantly, it permits a comprehensive assessment that can compare multiple aspects associated with each project’s impact, that are not necessarily monetary and quantitative, and therefore, tend to be excluded by other standard approaches (Estevez, 2013).

Whilst the MCA framework provides a list of prioritised projects, this methodology does not allow their comparison according to a single metric, as is the case with CBA and CEA (Thomopoulos et al, 2009). Moreover, the inclusion of qualitative indicators, including dummy variables, makes it possible to extend the scope of the data used. Though, this also leaves more room for potential discretion in the way these variables are considered, which could negatively affect the decision-making process (Saaty, 2012).

To deal with these limitations, there are sophisticated techniques and best practices that can be incorporated in the present Capstone to rigorously and objectively prioritise the projects (Saaty, 2004). Moreover, once implemented by FINDETER, this framework can lead to a learning-by-doing process that can serve to identify essential information for more extensive and insightful economic appraisals of projects in the future.

All in all, this section has highlighted the limitations of using a CBA or CEA in the context of FINDETER and demonstrated why MCA is better suited to this Capstone project. The

FINDETER 2021 Capstone project following table 1 summarises the main findings of the comparative analysis between the CBA, CEA and MCA methodologies carried out in this section.

Table 1. Summary of comparative analysis between CBA, CEA and MCA in the context of FINDETER

Own elaboration

FINDETER 2021 Capstone project

3.2. Best practices of MCA applied to investment prioritisation

This section highlights important case studies that informed us on how governments of both OECD and Latin American countries apply the MCA methodology to improve and institutionalise their decision-making processes.

3.2.1. MCA studies implemented by OECD countries

Given Colombia’s recent approval as a member of the OECD, it is interesting to look at how other OCD countries have applied MCA. For instance, MCAs have been implemented in the UK by the Ministry of Economy and Finance to prioritise infrastructure projects; in France by the Ministry of the Environment to select the most relevant projects to combat flooding risks; and in Australia by their infrastructure organisation, Infrastructure Australia, to identify projects of national interest in the transport, energy and water and sanitation sectors3.

Figure 3. MCA – case studies in OECD countries

Own elaboration Source: HM Treasury (2004), French Ministry of Ecological and Solidarity Transition (2018), Australian Government (2021)

3 See Annex 3 for more details.

FINDETER 2021 Capstone project

3.2.2. MCA studies implemented by countries in LAC region

This methodology has also been recently used by neighbouring countries and by Colombia itself, through the DNP4. MCA was used by the DNP to select the most suitable contraceptive modality for the development of infrastructure projects; by the Ministry of Finance in Chile to select projects within the transport and sanitation sectors; and by the Ministry of Economy and Finance in Peru to identify projects with high impacts on productivity and competitiveness5.

Figure 4. MCA - case studies in LAC countries

Own elaboration Source: Marcelo, House & Raina (2018), Ministerio de Hacienda de Chile (2014), Ministerio de Economía y Finanzas de Peru (2019), DNP (2016).

4 The DNP is Colombia’s National Planning Department and is in charge of defining, recommending and promoting public policy. 5 See Annex 4 for more details.

FINDETER 2021 Capstone project

The presented examples informed the development of our MCA methodology, especially regarding the selection of criteria, sub-criteria and their indicators. The following criteria were drawn by the presented case studies:

● Strategic importance of the project for the organisation and the government, in the cases of Peru, Chile, the UK and Australia,

● Regional priority, in the case of the UK,

● Economic impact, particularly in terms of unlocking private investment and fostering innovation, in the case of the UK and Colombia.

FINDETER 2021 Capstone project

4. Methodology for FINDETER’s prioritisation tool

4.1. Analytical framework

As discussed in the previous chapter, MCA appears to be the optimal analytical method to meet FINDETER’s current objective. Prior to describing our methodology, it is important to highlight key aspects of the decision-making process for the prioritisation of projects based on publications by CEPAL (2008) and Saaty (2012).

4.1.1. Principles for the decision process

The decision process involves the identification of impact dimensions, referred to as criteria, sub-criteria and corresponding indicators, common to both the social and productive sectors of interest to FINDETER. Drawing on international standards and best practices, we have tailored the MCA approach to FINDETER’s needs. The organisation’s priorities and relevant constraints were considered when defining the scoring and weights for relevant criteria, sub-criteria and indicators for the measurement of projects’ impact. This allows the final construction of a project prioritisation tool, in the form of an Excel calculator, that is both transparent with limited space for discretionary decisions and practical to use. As a case study, a sample of 12 projects selected by FINDETER is then assessed using the calculator. The developed tool enables the ranking of the 12 projects according to their performance on the identified criteria, sub-criteria and indicators, computed into a single score of relative impact.

4.1.2. Building the decision process with FINDETER

MCA is a methodology based on a co-construction process with key actors. It was therefore necessary to thoroughly understand the current decision-making process and strategic priorities of FINDETER, both at the level of the Economic Studies Department and of the four business lines’ management teams.

FINDETER 2021 Capstone project

Understanding that this tool had to address potential mismatches in standards to prioritising projects within FINDETER and the potential lack of adherence in its use, we carried out a bottom-up diagnosis to tailor this tool. This guaranteed not only its effectiveness, but also its sustainable application, eliminating path dependence based on previous unsuccessful attempts. To this end, our Capstone team was in constant dialogue with the Economic Studies team, via weekly meetings, and conducted in-depth qualitative and quantitative interviews with the heads of the business lines6. This was our principal input, driving the definition of indicators, scoring and weighting.

4.1.3. Ranking of projects as a result of the decision process

MCA methodology will be used to construct the ranking of FINDETER’s infrastructure projects, with the main objective of identifying, prioritising, and communicating with transparency the list of projects with the highest impacts and that would drive the closing of infrastructure gaps across municipalities in Colombia.

4.2. Data collection

To build the FiPT, we used the following primary data: • Interviews with representatives of FINDETER’s business lines and the Economic Studies Department. The information collected through quantitative and qualitative questions were essential inputs for the development of the tool7. The template for the interview questions can be found in Annex 13.

Additionally, we used the following secondary data: • Data at territorial level: public statistics published by the Colombian National Administrative Statistical Department (DANE, by its acronym in Spanish), the Colombian General Accounting Office (CGN, by its acronym in Spanish), and the

6 See Annex 14 for key findings. 7 In accordance with LSE’s Research Ethics, verbal and written consent was obtained to include interviewees as references in the report. Interviewees were allowed to refuse to answer questions, drop out of the experiment, and choose what data they wished to remain anonymous. No leading questions were asked, and information attributed to each interviewee was shared with them prior to the publication of the report.

FINDETER 2021 Capstone project

Colombian National Department of Planning (DNP, by its acronym in Spanish) at both the department and municipal level. This includes for example GDP per capita by department and fiscal performance index by municipality.

• Data at project level: information about the characteristics of each project. This includes for example CAPEX and benefited population.

Drawing inspiration from recommendations in the literature, the analysis of the datasets at both territorial and project levels- highlights two main constraints that have to be taken into account when selecting appropriate indicators: i) data availability, and ii) granularity of data, meaning that project- and municipal- level data are preferred in this selection. This is illustrated in figure 5.

Figure 5. Principles for the selection of indicators

Own elaboration Source: CEPAL (2008)

FINDETER 2021 Capstone project

4.3. Design process for the development of FINDETER’s prioritisation tool

Prioritisation of projects MCA simplifies the decision-making process via the establishment of a hierarchy of multiple criteria that jointly affect an objective, which in the current Capstone is the selection of most impactful projects. This type of MCA based on the Analytic Hierarchy Process (AHP) is selected for the development of FiPT since it allows to verify the logical consistency among FINDETER’s judgements for the construction of hierarchies (CEPAL, 2008).and priorities among within FINDETER. Figure 6 illustrates five main steps for its development.

Figure 6. Steps for the application of MCA methodology

Source: CEPAL (2018)

4.3.1. Step N°1: Construction of hierarchies

The aim of this first step is to identify and justify four impact dimensions that will be taken into account for each project. Under this context, infrastructure projects can potentially

FINDETER 2021 Capstone project promote economic growth in the short and long run, while simultaneously closing social gaps and fostering municipalities’ capacity building. Moreover, the allocation of resources on the projects can also grant FINDETER advantages that strengthen the organisation’s mission to multiply its long-term impact. At the same time, as a development bank in Colombia, FINDETER has the mandate to solve credit market failures through interventions that belong to the programmatic governmental agenda, extending beyond its own mission priorities.

Consequently, guided by the available data, literature and country case studies mentioned in Chapter 3, projects will be evaluated using the following four criteria: (1) Economic Potential, (2) Social Impact, (3) Institutional Capacity and (4) Strategic Importance.

Criterion N°1. Economic potential

Under this criterion, the FiPT will prioritise projects with the greatest economic potential impact. This includes the following indicators:

Sub-criterion N°1: Economic impact in the short run of public investment on country output

This sub-criterion consists of the calculation of the multiplier effect of the public investment generated by the execution of each project, in terms of the output generated for the specific department where the project is implemented, based on Bom & Ligthart (2015). Considering the case of Peru (MEF, 2019), the indicator for the investment multiplier will capture the positive externality of investment generated in the short run, due to the execution of the project. Thus, the following formula can be used to calculate the investment multiplier effect of each project in the Colombian economy8:

8 As Bom & Ligthart (2015) suggest, considering a panel of countries including OECD economies, the elasticity of public capital supplied at regional or local government is equal to 0.106.

FINDETER 2021 Capstone project

퐶푖 퐼푀푖 = [( ) ∗ 훼 ] ∗ 푃푆푗 푃퐼푗

Where:

퐼푀푖 = 퐼푛푣푒푠푡푚푒푛푡 푀푢푙푡푖푝푙푖푒푟 표푓 푒푎푐ℎ 푝푟표푗푒푐푡 푖 퐶푖 = 퐶푎푝푖푡푎푙 퐸푥푝푒푛푑푖푡푢푟푒 (퐶퐴푃퐸푋) 표푓 푒푎푐ℎ 푝푟표푗푒푐푡 푖 푃퐼푗 = 푃푢푏푙푖푐 퐼푛푣푒푠푡푚푒푛푡 푙푒푣푒푙 표푓 푒푎푐ℎ 푑푒푝푎푟푡푚푒푛푡 푗 훼 = 푚푢푙푡푖푝푙푖푒푟 푓푎푐푡표푟

푃푆푗 = 퐺퐷푃 표푓 푒푎푐ℎ 푑푒푝푎푟푡푚푒푛푡 푗

Sub-criterion N°2: Economic impact on unemployment

This sub-criterion considers the impact of each project execution on the unemployment rate at department level, where the project is implemented. As Abiad, A., Furceri D. and Topalova P. (2015) suggest, it is relevant to prioritise projects located in areas with higher rates of unemployment since their impact in reducing this situation can be greater.

Sub-criterion N°3: Promotion of human capital, innovation and productivity

This sub-criterion considers the effect of the components involved in the execution of each project on the promotion of human capital, innovation and productivity. It has been analysed that despite the increasing positive growth, Colombia lags in terms of human capital development (World Bank, 2010). Thus, this indicator considers the impact of the project in terms of adequate skills in the labour force, innovation and productivity.

Given that the information on the components of the project is not available for every project and its availability depends on the stage of the project or the nature of the business line involved (e.g. structuring phase has more information than financing), the verification of this sub-criterion will be conducted through a survey, that would allow technical assessors from FINDETER to answer several questions. They will determine whether the project has a component for the implementation of training workshops / programmes or whether it includes technology components.

FINDETER 2021 Capstone project

Criterion N°2. Social impact

Under this criterion, the FiPT will prioritise projects whose execution generate the greatest impact on living conditions. This includes the following indicators:

Sub-criterion N°1: Investment per beneficiaries This sub-criterion prioritises projects with the greatest investment in Colombian pesos per beneficiary. Based on Marcelo, House & Raina (2018), as well as the Peruvian and Chilean case study, this indicator will be measured as follows:

퐶퐴푃퐸푋 퐷푖푟푒푐푡 푖푛푣푒푠푡푚푒푛푡 푝푒푟 푏푒푛푒푓푖푐푖푎푟푦 = 푖 퐵푒푛푒푓푖푎푟푖푒푠푖 where:

퐶퐴푃퐸푋푖 = 퐶푎푝푖푡푎푙 퐸푥푝푒푛푑푖푡푢푟푒 표푓 푒푎푐ℎ 푝푟표푗푒푐푡 푖

퐵푒푛푒푓푖푐푖푎푟푖푒푠푖 = 퐷푖푟푒푐푡 푏푒푛푒푓푖푐푖푎푟푖푒푠 표푓 푒푎푐ℎ 푝푟표푗푒푐푡 푖

Sub-criterion N°2: Reduction of sectoral gaps in projects’ area of influence

This sub-criterion prioritises projects that are located in municipalities with the largest infrastructure gaps (a value below the median across the overall municipalities in Colombia) in the corresponding sector, in accordance with HM Treasury (2004) and MEF (2019). Specifically, the available indicators are shown in the following table.

FINDETER 2021 Capstone project

Table 2. List of indicators as a proxy for gaps across social and competitive sectors

Own elaboration Source: DNP (2018), DNP (2019), INVIAS (2020)

Sub-criterion N°3: Socio-economic characteristics of the area of influence

This sub-criterion prioritises projects located in the most disadvantaged municipalities in terms of poverty and income inequality, as per Ministerio de Hacienda de Chile (2014) and MEF (2019). The indicators are the following:

• The Multidimensional Poverty Index, available at the municipal level, which consists of five dimensions: (1) household educational conditions, (2) children and youth conditions, (3) health, (4) work and housing conditions, and (5) access to public home services.

• The Gini Index, available at the municipal level, which measures the statistical dispersion intended to represent the income inequality. A higher Gini

FINDETER 2021 Capstone project

Index indicates greater inequality, with high income individuals receiving much larger percentages of the total income of the population.

Criterion N°3. Institutional capacity

Under this criterion, the FiPT will prioritise projects located in municipalities with the lowest level of fiscal and management performance and potential externalities on state capacity building at the local level. Colombian municipalities located far off the economic centres face expenditure and budget constraints. Even though they receive funding from multiple sources, such as fiscal revenues, transfers from central government and royalties, they often lack the institutional capacity to back substantial infrastructure projects required to fulfil society’s needs. It is therefore important to consider the dimensions through which infrastructure projects can enhance the municipalities’ institutional capacity (FINDETER Commercial 2017). These dimensions are captured through the following sub-criteria.

Sub-criterion N° 1. Budgeting and fiscal performance

This sub-criterion prioritises projects located in municipalities with low fiscal performance, approximated by the Fiscal Performance Index9 (IDF, 2019), that captures a wide range of indicators, including the capacity for generating revenues. The result of the IDF in a score ranging from 0-100.

Sub-criterion N° 2: Municipality management capacity

This sub-criterion prioritises projects located in municipalities with low management performance10, using the Municipality Performance Index (MDM, 2019), that includes a set of indicators, including the capacity for using territorial planning instruments for local collection. The MDM assesses municipalities based on two components: Management and Development Results. The management component, which is of interest to this sub-

9 See Annex 8. 10 See Annex 8.

FINDETER 2021 Capstone project criterion, is made up of four dimensions and 12 indicators which measure the capacity of territorial entities to: 1) mobilize its own resources; 2) execute the resources of the financing sources; 3) promote open government and transparency and 4) manage land use planning instruments11. As opposed to the IDF, which is more focused on a municipality’s fiscal performance, the MDM emphasises its management performance (e.g. planning, human resources).12 The result is a score from 0 to 100.

Sub-criterion No 3. Local capacity building

This sub-criterion prioritises projects with the potential of fostering local capacity building. The indicators are the following:

• Allocation of the funding to the public sector: This indicator will consist of a dummy variable, taking the value of 2 if the project of interest involves the allocation of funds to the public sector (municipalities), 1 if the funds are channelled through private entities operating on behalf of the public sector (e.g. energy) and 0 if the funds are allocated to private companies on a private contract basis (e.g. infrastructure). Projects with higher values for the dummy variable will be prioritised.

• Revenue generation (directly and indirectly): This indicator will consist of a dummy variable, taking the value of 1 if the project of interest will enhance the municipality’s fiscal autonomy by generating revenue either directly as a result of the project’s output or indirectly via an increase in the local tax base, and 0 otherwise. Projects with a higher value for the dummy variable will be prioritised.

11 See Annex 8. 12 Subsequent analysis showed that the IDF and MDM have a correlation of 0.4, prompting us to keep both sub-criteria.

FINDETER 2021 Capstone project

Criterion N°4. Strategic importance

Under this criterion, the FiPT will prioritise projects that are closer aligned to FINDETER’s and the Government’s current strategies.

Sub-criterion No 1. Project alignment to FINDETER’s strategy

This sub-criterion prioritises projects that are closer aligned to FINDETER’s strategy 2018- 2022 using the following three complementary and mutually exclusive indicators, which take the value of 1 if the projects meets its conditions (and 0, otherwise):

• Integrate sales model: Reflects the added value that FINDETER grants when the project is carried out by FINDETER from the beginning, with the Planning Business Line technical support, until the end, with the Execution Business Line operations. The criteria will take the value of 1 if the project is involved at least with 2 business lines.

• New municipality: Reflects FINDETER’s ability to diversify the number of impacted municipalities, which can simultaneously increase the likelihood of future interventions in that territory.

• Group of municipalities: Reflects FINDETER’s ability to amplify the number of impacted municipalities, which fosters economies of scale of the interventions.

Sub-criterion No 2. Project alignment to Government’s strategy

This sub-criterion prioritises projects that are closer aligned to Colombia’s National Development Plan 2018-2022, using the following three complementary and mutually exclusive indicators:

• Pillar of Peace Building: Reflects whether the project is implemented in any of the 170 prioritised municipalities of the Development Program with Territorial Approach

FINDETER 2021 Capstone project

(PDET, by its acronyms in Spanish), that includes areas most affected by the armed conflict, poverty, illicit economies and institutional weakness.

• Pillar of Inclusion of Ethnic, Racial, and Disabled Minorities: Reflects the project’s location in areas with concentration of indigenous groups13.

• Pillar of Orange economy: Reflects the project’s contribution to the development of creative and cultural industries.

All the indicators explained in this section need to be standardised14 with the main objective of being homogenous to enable their comparison with one another (CEPAL, 2008).

4.3.2. Step N°2: Priority setting

This section is focused on the second step of the MCA methodology, which involves assessing weights based on the relative importance of each criterion, sub-criterion and indicator. The information for this step was provided by FINDETER.

Based on the Saaty scale illustrated in Figure 3, a pairwise is conducted among criteria and sub-criteria to assess one’s relative importance over the other. The Saaty scale will help determine the intensity of preferences among criteria and sub-criteria for the assessor i.e. the Economics Department of FINDETER. The scale is described in the following table:

13 Corresponds to 70 municipalities that host any of the 196 collective territories of black communities and 282 municipalities that host 811 indigenous settlements. 14 The standardisation process of each indicator consists of subtracting its minimum value, and dividing it by the difference between its maximum value and its maximum value.

FINDETER 2021 Capstone project

Table 3. Saaty scale for the pairwise comparison

Source: Saaty (2012)

Considering this scale, it is essential to build a matrix, as the one presented in figure 7, for each criterion and sub-criterion of the hierarchy explained in Step N°1.

FINDETER 2021 Capstone project

Figure 7. Example of matrix with the scale for the pairwise comparison

Own elaboration Source: Leal, J. (2020)

As part of the co-construction, the following matrix illustrates FINDETER’s Economic Studies Department answers of this scale to compare pairs of the four main criteria15.

Figure 8. Matrix with the scale for the pairwise comparison of main criterion

Own elaboration Source: based on FINDETER’s input

From these results, we can infer the following conclusions regarding the main four criteria:

● Economic potential and Social impact are considered to be of equal importance for project prioritisation. ● Economic potential is moderately more relevant in comparison with Institutional capacity and Strategic importance. ● Social impact is slighter more important than institutional capacity.

15 See Annex 10 that includes the details of the Saaty scale results for sub-criterion and indicators for FiPT.

FINDETER 2021 Capstone project

● Social impact is moderately more relevant in comparison with Strategic importance. ● Institutional capacity is slighter more important than Strategic importance.

4.3.3. Step N°3: Weight estimation

Assessing the relative importance of each criterion analysed in step N°2 for the estimation of weights. This extends to their corresponding sub-criteria and indicators. The aim of this step is to estimate the relevance of all the elements of the hierarchy included in FiPT using a quantitative synthesis. It is important to note, that due to the significant heterogeneity the weighting performed by each business line (see figure 9), the Capstone team decided to base the final estimation of weights on the preferences of the Economic Studies Department as the central agent responsible for the FiPT’s estimations. However, the information answered by each business line will be used for sensitivity analysis in Chapter 5.

Figure 9. Weights estimated by FINDETER’s business lines

Planning Economic impact

Structuring Social impact

Institutional Financing capacity Strategic importance Execution

0% 20% 40% 60%

Own elaboration Source: Interviews with FINDETER’s business lines. See more details in Annex 14.

FINDETER 2021 Capstone project

Therefore, considering the Economic Studies Department inputs, the calculation of weights includes the following stages:

1. Estimation of a square matrix based on the pairwise comparison explained in Step N°2. This stage involves the calculation for each corresponding element of the matrix with the following formula:

푥푖푖 = 푥푗푗 = (1 ∗ 1) + (퐴 ∗ 1/퐴) + (퐵 ∗ 1/퐵) + (퐶 ∗ 1/퐶)

푥푗푖 = (1/퐴 ∗ 1) + (1 ∗ 1/퐴) + (퐷 ∗ 1/퐵) + (퐸 ∗ 1/퐶)

푥푘푖 = (1/퐵 ∗ 1) + (1/퐷 ∗ 1/퐴) + (1 ∗ 1/퐵) + (퐹 ∗ 1/퐶)

푥푙푖 = (1/퐶 ∗ 1) + (1/퐸 ∗ 1/퐴) + (1/퐹 ∗ 1/퐵) + (1 ∗ 1/퐶)

Figure 10. Example of the matrix with the scale for the pairwise comparison

Own elaboration Source: Leal, J. (2020)

Following the example explained in the Step N°2, the estimation of the square matrix for the main criteria follows the process seen in figure 11.

FINDETER 2021 Capstone project

Figure 11. Matrix with FINDETER’s pairwise comparison of main criterion

Own elaboration Source: based on FINDETER’s input

2. Estimation of local and global weights: Based on the estimates presented above, we then calculate the local weight of each indicator considering the following inputs:

- Row sum of estimated values of the matrix for each criteria:

Own elaboration Source: Leal, J. (2020)

- Total sum of row estimated values for the four criterion: 푙 푙 푙 푙

푋 = ∑ 푥푖,푦 + ∑ 푥푗,푦 + ∑ 푥푘,푦 + ∑ 푥푙,푦 푦=푖 푦=푖 푦=푖 푦=푖

FINDETER 2021 Capstone project

Therefore, for each criterion, the weight can be estimated as follows: 푙 ∑ 푥푖,푦 푤 = 푦=푖 푖 푥

Therefore, based on the results presented in figure 11 with FINDETER’s pairwise comparison, the final weights are estimated for inclusion in the FiPT.

Figure 12. Summary of weights calculated for the four main criteria selected for the prioritisation tool

Own elaboration Source: based on own estimations

4.3.4. Step N°4: Logical consistency

This step aims to conduct an iterative assessment process to confirm the consistency during the pairwise comparisons. This step aims to conduct an iterative assessment process to ensure consistency during the pairwise comparisons. Based on CEPAL (2008), the consistency check includes the following two principles for the logical consistency of the value judgments included among the Steps N°2 and N°3:

• Transitivity of the preferences: if A is more relevant than B, and B is more relevant than C, then it is expected that A is more relevant than C.

• Proportionality of the preferences: if A is three times more relevant than B, and B is two times more relevant than C, then it is expected that A is six times more relevant

FINDETER 2021 Capstone project

than C.

Using the AHP approach, the logical consistency is analysed via the estimation of the consistency relation using the following formula:

퐶퐼푖 퐶푅푖 = 푅퐼푖 where:

퐶푅푖 = 퐶표푛푠푖푠푡푒푛푐푦 푟푎푡푖표 표푓 푐푟푖푡푒푟푖표푛 푖

퐶퐼푖 = 퐶표푛푠푖푠푡푒푛푐푦 푖푛푑푒푥 표푓 푐푟푖푡푒푟푖표푛 푖

푅퐼푖 = 푅푎푛푑표푚 푖푛푑푒푥 표푓 푐푟푖푡푒푟푖표푛 푖

To ensure logical consistency, the ratio obtained cannot exceed 10% (0.1). Values below this threshold provide evidence of the absence of any judgement inconsistencies. Then, the consistency index is estimated as follows:

푀 −푛 퐶퐼 = 푖 푖 n−1 where:

푀푖 = 푀푎푥푖푚푢푚 푒푖푔푒푛푣푎푙푢푒 표푓 푡ℎ푒 푚푎푡푟푖푥 표푓 푝푎푖푟푤푖푠푒 푐표푚푝푎푟푖푠표푛 푛 = 푛푢푚푏푒푟 표푓 푖푛푑푖푐푎푡표푟푠 표푓 푡ℎ푒 푐푟푖푡푒푟푖표푛 푖

The value of 푀푖 is estimated based on the pairwise matrixes explained in steps N°2 and N°3 as follows:

푙 푙 푙 푙

푀푖 = ∑ 푥푖,푦 + ∑ 푥푗,푦 + ∑ 푥푘,푦 + ∑ 푥푙,푦 푦=푖 푦=푖 푦=푖 푦=푖

FINDETER 2021 Capstone project

Therefore, based on the inputs presented in figures 11 and 12, the consistency ratio for the four main criteria included in FiPT is the following:

Figure 13. Consistency check matrix for the FiPT’s four criteria16

Own elaboration Source: based on own estimations

4.2.5. Step N°5: Alternatives assessment

The section is focused on explaining the main output of the prioritisation tool. By the assignment of weights for each indicator, each project receives a relative score that represents its total economic, social, institutional and strategic impact.

The prioritisation table includes a summary of the weights for each indicator explained in step N°1 (see Table 4). Thus, the final score is obtained by multiplying each value of each indicator at the project level by its respective assigned weight as indicated in the following formula:

푆푖 = ∑ 퐼푗 ∗ 푊푗 where:

푆푖 = 푅푒푙푎푡푖푣푒 푠푐표푟푒 표푓 푒푎푐ℎ 푝푟표푗푒푐푡 푖 퐼푗 = 푠푡푎푛푑푎푟푖푠푒푑 푣푎푙푢푒 표푓 푒푎푐ℎ 푖푛푑푖푐푎푡표푟 푗

푊푗 = 푊푒푖푔ℎ푡 표푓 푒푎푐ℎ 푖푛푑푖푐푎푡표푟 푗

16 Considering that this example illustrates the case of four criteria, it is relevant to point out that the consistency check will vary according to the number of criteria compared.

FINDETER 2021 Capstone project

To illustrate this step, the following table summarises the estimation of the scores considering the weights for each indicator for a sample projects as a simulation:

Table 4. Example of the estimation of total score for the ranking of projects

Own elaboration

Using the example explained above, the ranking of projects obtained across sectors and territories can be illustrated as follows, considering the project with higher score the one greatest project’s impact:

Figure 14. Example ranking of projects by using FiPT

Own elaboration

FINDETER 2021 Capstone project

5. Results & discussion

5.1. Final decision-making process for FINDETER

5.1.1. Hierarchic synthesis

In order to prioritise investment for projects with the greatest impact, FINDETER will assess a set of project proposals according to their fourfold impact: economic potential, social impact, institutional capacity and strategic importance. Following the baseline model described in chapter 4, the first two criteria were found to be of greater importance to FINDETER with weights of 38.6% and 34.1% respectively. In contrast, the last two criteria were assigned weights of 16.7% and 10.6% respectively. The stated percentage breakdown among the 18 indicators in total (summing up 100%) is illustrated in figure 15.

It is worth noting that iterative processes were held in order to guarantee judgement consistency from the pairwise comparisons, according to the methodology explained above. With some indicators weighing relatively more compared to others, this hierarchical structure will adjust the projects’ scores on each indicator according to their relevance for FINDETER i.e. assigned weight.

FINDETER 2021 Capstone project

Figure 15. Project’s prioritization process - hierarchic synthesis

Own elaboration based on the methodology explained in chapter 4. This graph only exhibits the global weights, which means that all weights are aggregated to 100% at the criterion-level and then at the indicator-level.

5.1.2. Discussion and Limitations

When assessing whether the final design of FiPT provides an effective decision-making tool to prioritise infrastructure projects with the greatest impact, we should analyse the data constraints in the selection of indicators, as well as the representativeness of their corresponding weight.

FINDETER 2021 Capstone project

i. Data constraints in the selection of indicators

In designing FiPT, it was important for the criteria and their respective sub-criteria to draw the majority of the project-level information under objective measures. However, the selection of the indicators and the quality of data used were subject to data availability constraints. Regarding the selection of the indicators, compromises were made to account for incomplete or non-updated datasets. For example, this was the case for the employment generation sub-criterion within the economic potential criterion. Given the lack of information regarding estimates of the levels of employment a project could generate, this implies that FiPT would prioritise projects belonging to departments with higher unemployment rate rather than for the employment generated by the project itself.

This also leads to a second point of analysis related to the territorial level at which the data is aggregated. Unemployment rates were not available at a more disaggregated level, forcing the use of data from the department level, rather than municipal level which can better capture the localised impact of the projects. Here, the concerns regard how representative department level data is of the individual municipalities it consists of.

Moving on, it is worth noting that some of the datasets used were not updated to the same year, so while most indicators correspond to information from 2018 and 2019, some of them correspond to 2016. For instance, this is the case for the dataset used for certain sectoral gaps. Given that there are no significant changes of these variables over time, this is not a major concern that could affect FiPT’s effectiveness.

Furthermore, this data restriction led to having 3 indicators out of 18 with imperfect measures and prone to subjectiveness, which could pose a threat to the validity of the results. The most prominent example is the indicator regarding a project’s economic impact in the long run within the economic potential criterion, which constitutes 22.9% of the overall model. This indicator is a dummy variable that takes the value of 1 should the project of interest promote human capital, innovation and productivity and 0 otherwise. It is acknowledged, therefore, that the relative high importance of this dummy variable can determine the final prioritisation results. To address this concern, the FiPT has been designed with a strict definition of what should be understood by human capital, innovation

FINDETER 2021 Capstone project and productivity so that judgements on this assessment would minimise the degree of subjectiveness. The same logic applied to the indicators of Sustainability and Orange Economy promotion, although their lower weight (0.9% and 0.2%, respectively) reduces the level of concern.

ii. The representativeness of weights

After analysing the diagnosis interviews conducted with the four business lines, the existence of different standards and processes for the decision-making of project selection became apparent. This led to the Economic Studies Department becoming the central agent, whose assessment would serve as the baseline model. This is due to this department being attached to the Executive Committee of FINDETER. Therefore, from a conceptual perspective, it has the capacity to holistically overview the organisation’s priorities and to align incentives across FINDETER17. Meanwhile, from an administrative perspective, it relies on the Department’s capability to centralise the collection and assessment of data necessary when conducting the prioritisation analysis.

Moreover, it is necessary to point out that the very essence of the tool highlighted the importance of co-construction between the Capstone team and FINDETER. Time constraints meant that we could not be more involved with the 4 business lines, raising concerns about the assumption on the Economic Studies Department assigning weights that are representative of FINDETER as a single entity. Thus, despite the desire to create an inclusive tool, an argument could be made that the result was a top-down approach, omitting a bidirectional approach with equal emphasis on bottom-up inputs as well.

It is acknowledged, therefore, that the prioritisation results may change when using instead each business lines’ individual preferences, which were found to differ quite substantially. Consequently, FiPT has been designed to minimise the extent to which the prioritisation results change when using different weights. However, the tool includes sensitivity analysis options, whose results are expected to feed back into the dynamic process of decision- making within FINDETER and to make space for technical deliberation when prioritising projects, as recommended by Marcelo et al (2016). Nonetheless, recommendations for

17 Based on a principal-agent theoretical model by Besley (2006).

FINDETER 2021 Capstone project further consultation in the future will be made in chapter 6.

Additionally, the baseline model yields large heterogeneities in the weights across indicators. Considering the most extreme values, the indicator of long-term economic potential (weight of 22.9%) will be a key determinant of a project’s aggregated score. The opposite will be true of the indicator on ethnic inclusion (weight of 0.1%). While this implies that indicators with low weights, such as the latter, may not play a significant role when prioritising one project over another, it should be noted, that all of the indicators were relevant for the decision-making process of FINDETER, as judged by the Economic Studies Department. Since this does not add further complexity to FiPT’s implementation, and, on the contrary, can add more precision when having highly similar projects, it was recommended to maintain all of the indicators, despite their low weight.

Finally, the indicators that are based on territorial-level data (both municipal and departmental-level data) have a joint weight of 54.5%, with the remaining weight of 45.5% coming from indicators that draw from project-level data. It is acknowledged, therefore, that projects located in municipalities with the least advantageous prospects in the related indicators may result favoured in terms of the final prioritisation score and consequent ranking. This is consistent with the diagnosis interviews held with the Business Lines as they highlighted the need to target the most vulnerable municipalities. However, it is recognised that project-level data can better measure the projects’ direct impact. Consequently, the policy recommendation section emphasises the need to address restrictions on the availability of information in order to better assess and compare projects.

FINDETER 2021 Capstone project

5.2. Simulation on 12 projects selected by FINDETER

This section will present the results of applying FiPT on 12 projects that were selected by FINDETER.

5.2.1. Projects sample selection

FINDETER chose 12 projects to be assessed by FiPT as a pilot test. The key criteria for the selection of the sample of projects were the following:

1. Representativeness of FINDETER’s business lines: selection of at least 2 projects by each of the four business lines (Planning, Structuring, Finance, Execution) to guarantee the inclusion of all the services provided by FINDETER.

2. Relevance in FINDETER’s operation share: selection of projects that belong to the sectors that jointly represent more than the threshold of 80% of FINDETER’s operation. These are: Energy Development, Housing, Urban transportation, Education, Health, Water and Sanitation18.

A description of the 12 projects can be seen below in figure 16.

18 See Annex 12 for more details.

62 Figure 16. The 12 projects selected by FINDETER

Own elaboration Source: based on FINDETER’s input

5.2.2. Prioritisation results

The results of the prioritisation of projects based on their relative social impact as calculated by FiPT are presented in figure 17. The Solar – Diesel Hybrid Energy Generation System in Miraflores in the Guaviare department ranked first with a score of 69.6. This project is followed by the Boarding School Dormitories in Leticia in the Amazonas department with a relative score of 55.7 The lowest ranking project was the housing project in Villavicencio in the Meta district with a relative score of 28.1. In light of the discussion on the final design of FiPT in the previous section, these results illustrate a few key points.

Figure 17. Ranking the 12 projects by score

Own elaboration Source: based on own estimations

FINDETER 2021 Capstone project each municipality based on their economic potential, governmental deficiencies and structural socioeconomic needs (Contaduria, 2021). This classification sorts municipalities into 7 groups: “special” and then category 1, 2, 3, 4, 5 and 6. Municipalities in the “special” category and categories 1 and 2 tend to be more populated, have greater fiscal resources and are considered to contribute more from an economic perspective, among other criteria. The further a municipality is classified from the aforementioned categories and closer to categories 5, 6, the “less favoured” that municipality is.

First, FiPT tends to partially favour projects according to their territorial-level data. This can be tested by analysing how aligned the prioritisation results are with the territorial targeting for public policies in Colombia. The DNP has created a classification system for each municipality based on their economic potential, governmental deficiencies and structural socioeconomic needs (Contaduria, 2021). This classification sorts municipalities into 7 groups: “special” and then category 1, 2, 3, 4, 5 and 6. Municipalities in the “special” category and categories 1 and 2 tend to be more populated, have greater fiscal resources and are considered to contribute more from an economic perspective, among other criteria. The further a municipality is classified from the aforementioned categories and closer to categories 5, 6, the “less favoured” that municipality is.

Of the 12 projects we were tasked with evaluating, 5 are in municipalities that are classified as category 5 (1 project) or 6 (4 projects), whereas the other 7 projects are in either the “special” category (2 projects) or category 1 (5 projects). As seen in figure 18, our tool assigned higher relative scores to the “less favoured” municipalities, with an average relative score of 46.9 as opposed to an average relative score of 36.9 in the “more favoured” municipalities.

FINDETER 2021 Capstone project

Figure 18. Ranking the 12 projects by overall score and alignment with national targeting

Own elaboration Source: based on own estimations

Second, FiPT exploits the different effects of all criteria. This can be observed when the aggregated score of each project is break down to each criterion contribution. The great heterogeneity in each criterion’s project score contribution is presented in figure 19. This means that while certain projects can be distinguished for the economic potential, they do not necessarily alleviate social disparities when located in developed, high-income and more equalitarian municipalities. This demonstrates FiPT’s adaptability to catering for the uniquely characterised combination of each case’s project and territorial-level data. For example, we see a higher contribution of the economic potential criterion in the projects in Ibagué, Villavicencio, Manizales, Cali and Pereira, whereas the social impact criterion was a key score determinant in projects in Santiago, Arboledas and Paratebueno.

FINDETER 2021 Capstone project

Figure 19. Ranking the 12 projects by overall score and contribution of each criterion1/

1/ P1 represents priority 1 and similarly for the rest. Own elaboration Source: based on own estimations

Finally, it is interesting to compare the relative ranking of the four projects that were common to both ConTREEbute and the LSE team. The four common projects include: the installation of a photovoltaic park in Miraflores, the transportation project in Ibagué, the housing project in Santiagomanda and the aqueduct and sewage project in Paratabueno. Despite using different methodologies, the relative ranking illustrated in figure 20 yields similar results. The only difference is the relative position of the project in Miraflores, which FiPT ranked 1st, whereas in the CBA analysis conducted by ConTREEbute it ranked 4th.

FINDETER 2021 Capstone project

Figure 20. Project’s comparison of FiPT results with ConTREEbute results

Own elaboration. It is worth noting that there is not a strict comparability among both methodologies. Source: ConTREEbute (2020) and own estimations.

While the significant underlying differences between the two methodologies restrict the scope of comparative analysis we can conduct based on the obtained results, it is important to assess how each tool’s components lead to the projects’ relative ranking. According to the ConTREEbute methodology, one of the key drivers of the low project ranking was the low benefit - cost ratio, which was the lowest among the four projects they evaluated.

The case of Miraflores exemplifies the superiority of using an MCA approach over CBA in the context of social impact measurement. The narrow scope of the identified benefits in the CBA methodology are in clear juxtaposition of how FiPT measures a project’s social impact. While CBA is a methodology that is often praised for its ability to quantify and monetise projects’ impacts, it is also restrictive in the quantity and quality of benefits it can measure. FiPT is able to overcome this problem by incorporating a wider range of criteria and sub-criteria, that can capture a more holistic definition of a project’s social impact, in this case the impact of prioritising one municipality with large socioeconomic needs.

FINDETER 2021 Capstone project

5.2.3. Sensitivity analysis

i. Other subjective measures: Business Lines preferences

As mentioned in chapter 4 on our methodology, during the weighting stage, each of the four business lines was asked to assign a weight to each criterion. However, the baseline results presented above take into account only the weighting determined by the Economic Studies Department. As part of a sensitivity analysis, we evaluated whether and how project scores would differ if we replaced the baseline model with each business lines’ preferences i.e. the weights they assigned to all indicators in the diagnosis interviews.

The results are presented in figure 21. Overall, the sensitivity analysis illustrates similar prioritisation patterns across business lines compared to our baseline model. It can be seen that the projects ranked in the first places according to the baseline model remain prioritised over the others in all cases. In particular, the projects in Miraflores and in Leticia are consistently ranked first and second. A correlation of the projects’ baseline scores with each business line’s scores illustrates this pattern. The baseline results are more similar to those in the Structuring Business Line model (0.87) and the Financing Business Line model (0.86), followed by the Planning Business Line model (0.77) and the Execution Business Line model (0.64). However, it is worth noting that this pattern does not hold for projects lower in the ranking.

This sensitivity analysis implies that the projects’ overall scores are more sensitive to the value of their indicators than to the value of the weights. For instance, the overall score is more influenced by the employment that projects generate or the fiscal performance of their corresponding district, than by the weight of each of those indicators.

FINDETER 2021 Capstone project

Figure 21. Projects' scores under different business lines’ weights

Own elaboration Source: based on own estimations

ii. Objective measure: Principal Component Analysis As a second approach for sensitivity analysis, we ran a Principal Component Analysis to statistically determine the weights of all criteria. This is considered common practice in the international case studies assessed, as it is useful when there is an explicit preference for objective weights -rather than subjective as is the case for FiPT-, and a likelihood of correlation and consequent redundancies in the underlying indicators (Marcelo et al, 2016).

This method determines a linear combination of criteria that captures most of the variation of the underlying data while retaining the maximum information of the original data set (Joliffe, 2002). In this context, this would mean that the weights assigned to all the indicators will be based on how much new information they provide about the impact of the projects. The results are presented in figure 22. In this case, the results do not hold as much as the previous sensitivity analysis as only the first project remains systematically prioritised.

FINDETER 2021 Capstone project

Figure 22. Projects' scores under Principal Component Analysis

Own elaboration Source: based on own estimations

Despite this method’s advantages, its applicability is limited under the current Capstone goal. This is due to the method needing a continuous change of weights according to the new set of projects that are assessed, restricting the possibility of knowing ex-ante the importance of each indicator. Moreover, FINDETER’s preferences and judgements are of key relevance to the prioritisation of projects. Given that this design may be prone to subjective analysis, in the following sections we address this limitation and formulate policy recommendations to limit the space for discretion in the decision-making process.

5.2.4. Implications and potential of FiPT

First, for this sample of projects, FiPT enabled small-scale but high-impact projects to be prioritised over other projects. To determine whether a higher CAPEX would correspond to a project having a greater social impact or not, we conducted an analysis to see how projects ranked based on their CAPEX, the results of which are presented in figure 23. Interestingly, the results indicate that a higher CAPEX does not necessarily correspond to projects having a greater social impact. In fact, the projects with the highest CAPEX ranked

FINDETER 2021 Capstone project

3rd, 4th, 9th and 7th, while the projects ranked in the top 2 had a significantly lower CAPEX. This finding indicates that by taking into account a wide spectrum of criteria and sub- criteria, FiPT is able to inform decisions that provide opportunities for small-scale but high- impact projects, incorporating an element of cost-effectiveness when FINDETER selects and delivers projects.

Figure 23. Correlation between projects’ CAPEX and score

Notes: The Y axis is adapted to inform the CAPEX values for project of priority 7 and project of priority 3. Own elaboration. Source: based on own estimations

Second, for all the projects potentially assessed, the FiPT creates space for further comparisons on their social impact. To illustrate this point, we assessed the relative social impact of the 12 projects based on the sector or business line they operate in (see figure 24 and figure 25). In our sample we see that while projects within the Energy sector rank highest, with an average score of 56, projects within the Education and Health sectors also rank highly at 2nd and 4th place respectively with average scores of 48.7 and 36.1. Furthermore, we see that projects that are part of the Structuring business line rank the highest, followed by the Planning, Execution and Finance in decreasing order of average project scores.

FINDETER 2021 Capstone project

Figure 24. Project’s score by Sector

Own elaboration Source: based on own estimations

Figure 25. Project’s score by FINDETER’s business line

Own elaboration Source: based on own estimations

While the results presented above provide some valuable insights on the potential of FiPT as a tool for effective decision-making, we advise caution when interpreting them. This is

FINDETER 2021 Capstone project due the limited sample size of the projects assessed. Given that only 12 projects were selected by FINDETER for evaluation, it would be a mistake to draw definitive conclusions from the results and extrapolate their interpretation to all project proposals within the organisation.

For example, while the finding that the tool does not seem to favour projects with a higher CAPEX is encouraging, as this could indicate that the tool encapsulates a holistic definition of social impact, a higher number of projects evaluated is required to confirm this preliminary result. Similarly, the small number of projects per sector, as well as per business line presents a hurdle for a thorough evaluation of the relative social impact performance of each sector and business line. On the sectorial side, only 2 projects were selected for each of the 6 sectors. On the business line side, 4 projects corresponded to Execution and Financing respectively, while 2 projects corresponded to Planning and Structuring respectively. Therefore, these results should be interpreted as preliminary and a first indication of FiPT’s potential in the prioritisation of projects based on their social impact.

5.3. Design of the FiPT template

All the analysis described comes from a template in Excel that was designed to systematise all the indicators previously described. Besides providing direct estimations of the projects’ relative score and corresponding ranking, it contains additional features for statistical and sensibility analysis. Technical assessors will count on a user guide to correctly manage FiPT. In figure 26, we exhibit a piece of the template19.

19 See Annex 17 for a graphical description of the Excel template.

FINDETER 2021 Capstone project

Figure 26. Introduction of the FiPT template in Excel

Own elaboration

FINDETER 2021 Capstone project

6. Policy Recommendations and Concluding Remarks

6.1. Policy Recommendations

Some of the key findings that surfaced during the development of FiPT, as well as the preliminary results of this pilot test, guided the formulation of a number of recommendations for FINDETER. These recommendations, while tailored to the MCA designed for FINDETER, are informed by recent international best practices in MCA design, notably in the UK, Australia, Peru and Chile. 6.1.1. Co-construction process and socialisation of results

By its nature, the MCA is an open and consultative process. In order to best promote the participation of interested parties in the next steps of the presented methodology’s further development, FINDETER should create an inclusive forum, in which participants will be able to express their views. This would require the dissemination of complete information regarding the underlying mechanics of the MCA methodology to all participants. In particular, it is recommended to organise workshops dedicated to the methodology and to the discussion of the different criteria, sub-criteria and indicators, as the LSE team did with the business lines consultation and the workshop for the Economic Studies team. In addition, the Economic Studies team should communicate the results of the first design of the FiPT and its simulation on the sample of 12 projects to the directors of FINDETER and representatives of the business lines, as well as with Colombian government officials. This is absolutely crucial to bring legitimacy to the analysis resulting from the MCA, as well as to ensure the acceptance, shared ownership and use over time of this decision support tool.

6.1.2. Transparency and accountability

The design of the MCA was tailored to FINDETER needs, accommodating a broad set of policy objectives, including factors that are not easy to quantify or monetise, which have been assigned a relative importance. This latter point led to a subjective weighting of criteria, sub-criteria and indicators. As seen above, despite the fact that this weighting is intuitive and corresponds to FINDETER’s policy preferences, it can lead to threat of subjective manipulation of weights and criteria to privilege certain projects over others if the right accountability mechanisms are not in place.

FINDETER 2021 Capstone project

This report recognises that it is people who make decisions, through models. The MCA methodology assists decision making in many ways, from providing structure to debates, documenting the process of analysing a decision, making trade-offs between different policy objectives clear, to creating a common understanding of the issues. However, decision making is necessarily a human function, so to facilitate the most evidence- informed and transparent decisions possible, it is necessary to enable decision makers to make judgements that are relatable, accurate and justified. Decision criteria, criteria weighting, and sensitivity analysis should be decided and made transparent in advance of the selection, while the data used and resulting analysis should be made publicly available and open to third‐party review.

6.1.3. Organizational learning and continuous improvement

The MCA methodology should be conducted in an iterative fashion and time should be allowed for decision-makers to fully explore the model. The FiPT represents one of the first attempts to develop a robust and transparent decision-making methodology for FINDETER as a whole. This has increased the level of information available to the different stakeholders and has created a reflex for systematic data collection.

The MCA proposes an agenda for systematically tackling difficult issues in decision- making. However, these efforts have faced certain constraints in terms of access to information and time. It is therefore recommended to reopen the debates around this first version of the FiPT, in order to shed light on the potential divergent views around certain scores or weights, to reveal the trade-offs inherent in decision-making or to open new options for data collection. For instance, as FINDETER operates in a number of sectors, it would be useful to set up a group of experts on sector-specific issues to ensure that these different dimensions are incorporated. This paves the way for continuous organisational improvement, both in terms of understanding and ownership of the tool as well as in terms of relevant data collection.

FINDETER 2021 Capstone project

6.1.4. Scope of impact and complementary efforts towards data collection

The discussion process around the selection of relevant indicators to select projects in an early stage raised the question of whether FINDETER was dependent on sufficiently comparable data. The design of the FiPT considered the capacity and information constraints at an early stage of the projects, which involved defining many indicators from municipal level data. While this allowed to capture as much as possible the various dimensions of project impacts and led to improved information levels to move towards the design, it also showed that there is a potential for an ongoing improvement of the FiPT by collecting increasingly updated and granular data. It is therefore recommended to increase the measurement of impact at project level, to consider the integration of more disaggregated or granular indicators, and to regularly update the databases of administrative indicators at municipal level, so as to promote an increasingly rigorous project appraisal. As such, we advise the continuation and expansion of current data collection tools, including the Strategic Indicators Report and the Achievements Database.

Complementary indicators should be included in the FiPT to extend the scope of impacts measured, particularly with regard to the environmental and sustainability dimensions of projects. The following indicators could thus be considered:

FINDETER 2021 Capstone project

Figure 26. List of potential indicators to be included in the forthcoming years

Criteria Recommended indicators

Economic Potential Employment generation associated with the execution of the project, with disaggregated data according to the type of project and the sector

Social Impact / Sectoral Gaps Disaggregated indicators between different types of areas (urban / rural)

Strategic Importance Indicators disentangling the strategic priorities of individual sectors, rather than looking at the Government strategy as a whole

Environmental Impact Impact on biodiversity and natural environments Sustainable resource management Consideration of the circular economy

Risk Mitigation Indicators for each type of risk: - Economic risk - Social risk - Institutional risk - Natural risk

Own elaboration.

Source: Infrastructure Australia (2021); Ministry of Environment (2018); MEF (2019); World Bank (2018)

Furthermore, as noted earlier, the ex-ante assessment of the impacts of infrastructure projects is one of the steps in a broader decision-making process and project cycle. It is therefore recommended to complement this ex-ante assessment with a monitoring and tracking effort during implementation as well as with evaluations after the intervention has taken place. It would be relevant to systematically collect data on the final delivery of the project and its impacts compared to the ex-ante estimations.

FINDETER 2021 Capstone project

6.2. Concluding Remarks

As this report has shown, investment in high quality public infrastructure has the potential to shape the economy and society of the future, supporting growth, improving welfare and generating employment. However, such investment is complex and poor governance is a major reason why infrastructure projects often fail to meet their time, service delivery and budget targets. Political and corruption dynamics, a lack of systematic data collection on impacts, as well as a lack of linkage between projects and the organisation's strategic priorities can compromise the decision-making process for efficient and impactful infrastructure investments.

As a development bank focused on infrastructure projects developed by territorial entities, FINDETER faces the same challenges and has a major role to play in the development of infrastructure for economic and social impact. To date, FINDETER's decision process is prone to fragmentation and heterogeneity between the priorities of the different business lines, as well as from the lack of systematic collection of impact data in the pre-investment phase. Faced with this lack of a comprehensive, homogeneous and systematic framework for infrastructure decisions, and in a context where the institution wishes to demonstrate its contribution to social impact, the need has arisen for the development of a tool for ex- ante social impact assessment and prioritisation between infrastructure projects.

FINDETER, in its pre-investment decision process, faces important constraints of data availability, time, as well as resources to allocate. In addition, the institution has to prioritise projects with diverse objectives, both within and across competitive and social sectors. In this context, the MCA proved to be the most appropriate analytical tool for prioritisation and ex-ante impact assessment. This methodology, applied by many national and international institutions, allowed us to adapt to FINDETER's needs and objectives within the framework of this Capstone and to overcome the aforementioned constraints.

This report has presented the tailor-made application of the MCA framework to FINDETER’s context. The result of this co-construction process is a tool that allows

FINDETER 2021 Capstone project

FINDETER to prioritise infrastructure projects within and across sectors according to their relative impact, based on their performance on 4 main criteria, broken down into 11 sub- criteria and 18 indicators. This tool reflects FINDETER's priorities, takes into account various dimensions of the impact of infrastructure projects, and systematically incorporates data at project and territorial level. It is important to note that this tool is not a prescriptive process, nor is it a route to a one-size-fits-all solution. Instead, this report has shown that the MCA framework is a useful tool to open up the reflection around the investment decision and the project environment, and to raise the right questions at the pre-investment phase, a critical point in projects lifecycle.

This report has also presented a simulation of the use of the calculator for 12 projects selected by FINDETER, which has allowed a ranking of these projects according to their relative social impact. These results were then analysed in terms of different sectors and business lines, compared to the results of ConTREEbute, and a sensitivity analysis was conducted.

In the short and medium term, this tailor-made MCA methodology offers significant added value. Firstly, it allows FINDETER to homogenise its decision-making process around infrastructure projects, in order to reflect priorities. In addition, such a framework allows to increase the efficiency and transparency of the decision process and thus to improve FINDETER's accountability towards its different stakeholders, which is crucial in legitimising these decisions. The MCA also makes it possible to demonstrate and communicate effectively FINDETER's contribution to the various dimensions of social impact. Finally, following the logic of continuous learning, this tool also offers the opportunity to analyse FINDETER's past investment decisions a posteriori, to learn from them and to adapt its decision-making process accordingly.

As an inherently consultative tool, the MCA allows for the integration of the priorities and constraints of different stakeholders, while raising awareness of the importance of such an institutionalised framework, thus facilitating ownership of the tool. It also makes it possible to deviate from path dependency: by elaborating a prioritisation of projects based on their

FINDETER 2021 Capstone project relative impact, this tool signals the end to the prioritisation of projects with a low social impact. Finally, it allows FINDETER's strategic priorities to be integrated into the decisions of the different business lines.

By its construction, this tool fosters organisational resilience, which is absolutely necessary to cater for new FINDETER or governmental priorities, as well as successfully adapting to respond to unforeseen events, as is the case with the current pandemic. Such a methodology, as a result of the flexibility it allows in the integration of new indicators or in the attribution of new weights, enables the decision process to adapt with resilience. Finally, because of the transparency it involves, the MCA reduces the incentives for corruption, and thus for the promotion of ill-conceived projects that do not address the needs of the population. By defining a clear pathway for the administration in charge of the investment decision, it also creates more space for public scrutiny. Overall, such a tool contributes to the improvement of FINDETER's governance, thus helping to improve organisational leadership. This results in more informed decision-making and project prioritisation processes, which allows for the maximisation of impact generation and welfare in Colombia.

Lastly, there are also several positive spillover effects of this tool, for FINDETER and beyond. First, FiPT allows FINDETER to differentiate itself from other Colombian development banks, by demonstrating in an evidence-informed way its contribution to a social impact-oriented infrastructure. Moreover, this approach can foster a collective learning process, in that this methodology could be shared with other development banks and public institutions in Colombia. The synergy between these institutions would notably enable the improvement of the tool for FINDETER, and, overall, improve the decision- making and contribution of these different institutions to a greater social impact.

FINDETER 2021 Capstone project

7. References

Abiad, A., Furceri D. and Topalova P. (2015). The Macroeconomic Effects of Public Investment: Evidence from Advanced Economics. International Monetary Fund Working Paper

Adler, M., 2013. Cost-Benefit Analysis and Distributional Weights: An Overview. SSRN Electronic Journal.

Ardila, S. et al (1998). A Review of the Use of Contingent Valuation Methods in Project Analysis at the Inter-American Development Bank, IADB.

Atkinson, G. and S. Mourato (2015), "Cost-Benefit Analysis and the Environment", OECD Environment Working Papers, No. 97, OECD Publishing, Paris, https://doi.org/10.1787/5jrp6w76tstg-en.

Banerjee, A. and Duflo, E. (2011). Poor Economics: Rethinking Poverty and The Ways to End It.

Belli, P. et al (1998). Handbook of economic analysis of investment operations. UNDP. URL: https://www.adaptation-undp.org/sites/default/files/downloads/handbookea.pdf

Besley, T. (2006). Principled agents?: The political economy of good government. Oxford University Press on Demand.

Boardman, A. E., D. Weimer, D. Greenberg (2005) Cost Benefit Analysis: Concepts and Practice. 3rd edn Upper Saddle River, N. J.: Prentice-Hall.

CONPES (2020). Política para la reactivación y el crecimiento sostenible incluyente. Consejo Nacional de Política Económica y Social – Republica de Colombia – Departamento Nacional de Planeación.

ConTREEbute (2020). Entregable 6: Medición de Impactos. Proyecto: DELTA – Medición de Impactos.

ConTREEbute (2020). Medición de impactos FINDETER. Socialización de resultados la Célula de Impacto Positivo.

DCLG (2009). Multi‐criteria Analysis: A Manual. London: London Department for Communities and Local Governance.

FINDETER 2021 Capstone project

Dhaliwal, I. et al (2012). Comparative Cost-Effectiveness Analysis to Inform Policy in Developing Countries: A General Framework with Applications for Education for valuable discussion and feedback. Education Policy in Developing Countries, 285–314.

DNP (2016). Nota técnica 4 – manual de multicriterio para la selección de proyectos de asociación público privada. Colombia.

Draskovics, T. (2018). (Quick) guide to ex-ante policy impact assessment. January, 1–28.

European Union (2017). Cost-effectiveness analysis. Evaluation Unit DEVCO. URL: https://europa.eu/capacity4dev/evaluation_guidelines/wiki/cost-effectiveness-analysis-0

Evans, D. K., & Popova, A. (2014). Cost-Effectiveness Analysis in Development: Accounting for Local Costs and Noisy Impacts. World Development, 77(September), 262– 276. https://doi.org/10.1016/j.worlddev.2015.08.020

FINDETER (2017). Metodología para la identificación, priorización y selección de proyectos. Asistencia al desarrollo territorial - ADTI. Gerencia de Estructuración de Proyectos, Vicepresidencia técnica, Vicepresidencia de Planeación. Colombia.

FINDETER (2019). Análisis del entorno sectorial 2018.

FINDETER (2019). Informe Annual Sectorial 2019. Versión: 10. Codiao: DP-DA-009.

FINDETER (2019). Plan de vuelo. Estrategia 2019-2022.

FINDETER (2020). Database – Logros. Formato información programas en ejecución 2020.

FINDETER (2020). Matriz de Priorización Priorización.

Haveman, R. and Weimer, D. (2001). Cost–Benefit Analysis. Pergamon: International Encyclopedia of the Social & Behavioral Sciences, pp.2845-2851.

Inter-American Development Bank (2020). Economic Analysis Overview. Differences between CBA-CEA.

INVIAS (2020). Reporte sobre el estado de la red vial nacional en Colombia. Instituto Nacional de Vías. URL: https://www.invias.gov.co/index.php/informacion- institucional/2-principal/57-estado-de-la-red-vial

Jolliffe, I. (2002). Principal component analysis. John Wiley & Sons, Ltd.

FINDETER 2021 Capstone project

Leal, J. (2020). AHP-express: A simplified version of the analytical hierarchy process method. MethodsX, Volume 7. https://doi.org/10.1016/j.mex.2019.11.021

Marcelo, D. et al (2016). Prioritizing Infrastructure Investment. A Framework for Government Decision-Making. World Bank Group.

McEwan, P. (2012). Cost-effectiveness analysis of education and health interventions in developing countries. Journal of Development Effectiveness, 4(2), 189–213.

Ministerio de Economía y Finanzas - MEF (2010). Resolución Directoral Nro. 003-2011- EF/68.01. Dirección General de Inversión Pública del Ministerio de Economía y Finanzas. Lima, Perú.

Ministerio de Economía y Finanzas - MEF (2019). Plan Nacional de Infraestructura para la Competitividad. Lima, Perú.

Ministry of Environment (2018). Analyse multicritère des projets de prévention des inondations. Guide méthodologique 2018, France.

Münich, D. and Psacharopoulos, G. (2014). Mechanisms and methods for cost-benefit / cost-effectiveness analysis of specific education programmes. EENEE Analytical Report No. 19, Prepare for the European Commission.

Ocampo, J., Arias, P. & Torres, J. (2018) La banca nacional de desarrollo en Colombia. Ensayos sobre Política Económica (ESPE), núm. 88, diciembre, DOI: 10.32468/espe.88

OECD (2006). Promoting Pro-Poor Growth: Harmonizing ex-ante poverty impact assessment. OECD Publications.

OECD (2014). Assessing the Impact of State Intervention in Research - Techniques, Issues

OECD. (2017). Governing Better Through Evidence-Informed Policy Making Options for an OECD Work Agenda. OECD Conference Centre.

OECD (2019), Government at a Glance 2019, Éditions OCDE, Paris.

Petrie, M. (2010). Promoting public investment efficiency: A synthesis of country experiences. Paper presented at World Bank Preparatory Workshop, Promoting Public Investment Efficiency, Global Lessons and Resources of Strengthening World Bank Support for Client Countries.

Samset, K. & Christensen, T. (2017). Ex-Ante Project Evaluation and the Complexity of Early Decision-Making. Public Organization Review 17, 1-17.

FINDETER 2021 Capstone project

Thomopoulos, N., Grant‐Muller, S., & Tight, M. (2009). Incorporating equity considerations in transport infrastructure evaluation: Current practice and a proposed methodology. Evaluation and program planning, 32(4), 351‐359.

Vining, A. and Boardman, A., 2006. Metachoice in policy analysis. Journal of Comparative Policy Analysis: Research and Practice, 8(1), pp.77-87.

Vining, A. and Weimer, D., 2015. Policy Analysis. International Encyclopedia of the Social & Behavioral Sciences, pp.273-280.

White, D. and Silloway, T. (2016). Cost –Benefit Analysis. Evidence-Based Policymaking Collaborative.

World Bank (1998). The Handbook on Economic Analysis on Investment Operations. WBI Development Studies.

World Bank (2010). Cost-Benefit Analysis in World Bank Projects. https://openknowledge.worldbank.org/handle/10986/2561 License: CC BY 3.0 IGO.”

FINDETER 2021 Capstone project

8. Annexes

Annex 1. Terms of Reference

LSE – FINDETER Capstone Measuring Social Impact for Project Prioritisation Terms of Reference August 2020

Context FINDETER is a local development bank for sustainable infrastructure in Colombia. It grants rediscount loans through financial intermediaries to finance public and private infrastructure projects and provides technical assistance in the execution of regional programs of infrastructure. Additionally, FINDETER issues guarantees to manage public funds and provide project-structuring services. Its mission is to support the sustainable development of the territories, generating well-being in the regions. Since the beginning of the 2018 – 2022 presidential term, FINDETER has disbursed USD$1,42 billion in 11 strategic economic sectors: Transportation, Energy Development, Urban Development, Housing, Health, Education, Environment, Drinking Water and Sanitation, Tourism, Telecom, Entertainment and Sports, Culture, and Fiscal Consolidation Requirements. From those sectors, 67% of the funds disbursed by the bank have been focused on Housing, Drinking Water and Sanitation, Health and Energy. To identify new opportunities and sustainable solutions for the cities, the entity has participated in the subsequent projects: Sustainable and Competitive Cities, Caribbean and Santanderes Diamond and Emblematic Cities (Findeter Sostenibilidad, 2017). After operating ten years under an outstanding social perspective, FINDETER has been able to tackle socioeconomic issues according to the guidelines of the National and Regional Development Plans. But also, the entity has been able to benefit the society through projects owed by privates such as hydroelectric and oil infrastructure.

FINDETER 2021 Capstone project

Motivation Colombian municipalities and industries located far off the economic centres have constantly faced expenditure and budget constraints. Even though public institutions receive funding from multiple sources such as fiscal revenues, transfers from central government and royalties, they are not able to back substantial infrastructure projects required to fulfil society’s needs.

Therefore, the previous situation limits the development of underprivileged areas (Findeter Commercial 2017) and creates a poor financial profile for such municipalities. Commercial banks do not realize incentives for lending to those municipalities, thereby stagnating the development of the regions in Colombia due to their incapability for leveraging big schemes (World Bank 2014). Consequently, several governments have institutionalized development banks to tackle a well identified credit market failure, which restricts access to credit and excludes relevant development key actors. These institutions have been promoting economic growth by providing credit and a wide range of advisory and capacity building programs. Development banks usually constitute the main source of long-term credit, loan guarantees, and other financial services in the infrastructure, housing and agriculture sectors. In particular, FINDETER was created in Colombia to tackle the absence of resources from private actors in the financial system to develop sustainable infrastructure projects (World Bank 2014). FINDETER, as a second-tier bank, channels loans from the central government looking to alleviate monetary restrictions of institutions. Due to competitive interest rates and long spans of time, these loans arise as an option for municipalities and companies to solve their budget restrictions in order to have better financial conditions (Findeter Commercial, 2017). Therefore, development banks aid appears as an attractive alternative to bolster big projects that require financial assistance in the long-term. From small transportation projects to contributing in huge energy projects, the entity has provided resources to develop essential schemes in the territories. The lack of a single client profile reinforces FINDETER’s mission of being a key ally for the regions, allowing their inclusion into the financial system through boosting development projects. Thanks to its management strategies, the entity had reached 646 (58%) of Colombia’s municipalities by June 2020 (see Appendix, figure 1). It has reached them by way of projects that improve investment on intermediate cities, and tackle poverty by supporting the creation of key infrastructure for development. The number of municipalities reached by FINDETER has increased from 339 in 2016 to 685 in 2020. Today, FINDETER has a broad presence throughout the country and even though its impact seems evident and there have been some approximations to measure the entity´s social impacts, we want to strengthen this social impact measurement of our infrastructure projects and have a clear supported methodology for decision making. This is crucial for the selection of projects with the highest impact, which would signal the organisation’s scope and efforts.

FINDETER 2021 Capstone project

In 2019, FINDETER was eager to better understand the impact made by its investments by measuring Social Return on Investment (SROI) or Social Internal Rate of Return

(SROR)1; something that is not implicitly measured in the intermediation rate. Due to our entity’s commitment of long-run regional development, we are convinced of the great potential to improve the actions that lead to allocating funds in most impactful projects in economic, social and environmental aspects.

Existing methodology and background for this Capstone Project FINDETER hired ConTREEbute, a consultancy firm, with the goal of finding the true social benefit of the infrastructure projects financed, structured, planned and assisted by FINDETER. The first approach for a measure of social returns for FINDETER´s operations consisted of the selection of four projects that were highly representative of the institution´s scope. Each project covered one of four non-social subsectors: housing, drinking water and sanitation, urban mobility, and energy. After choosing which projects to measure, the authors found the Cost Benefit Analysis methodology to be the best option to capture the social benefits of such projects. The aim was to try and estimate what would be the Internal Social Rate of Return in each project (SROR in Spanish, TIR Social). The TIR Social for a project can be calculated using the economic net present value formula, as follows:

• If 휌 is greater than the Social Discount Rate2, it is convenient for the economy to conduct such project. • If 휌 is smaller than the Social Discount Rate, it is not convenient for the economy to conduct the project. • If 휌 equals the social Discount Rate, the economy should be indifferent between conducting it or not.

With this Cost Benefit Analysis Framework in mind, the next step was to identify a set of economic benefits and costs for each project. This set of benefits and costs or “impacts” are to be defined in monetary terms to calculate a flow of costs and benefits and obtain the Economic Net Present Value of the project. Those impacts fall in three categories: environmental, social, and economic. To each impact a type (positive/negative) and a magnitude (high/medium/low) were assigned. From this matrix of impacts, the most relevant and those for which data were available were chosen to calculate the Economic

FINDETER 2021 Capstone project

Net Present Value of the project and finally arrive at an Internal Social Rate of Return20. For this Capstone project we would like the advisors to take as a reference the methodology of ConTREEbute for the ex-ante social impact measurement of projects and formulate a methodology to prioritise projects according to their holistic impact. Considering that FINDETER aims to assess the impact of projects not only from non- social sectors but also from social sectors, such as Education and Health, this Capstone should enable FINDETER to estimate the ex-ante impact of any project belonging to the 6 most relevant sectors impacted by the company and to prioritise those with the highest impact. The rest of the project will consist of identifying which are the four most relevant impact dimensions for any project that falls within these 6 sectors. The idea is to generalize the methodology in a way that measuring these four impact dimensions for any project belonging to these sectors will allow us to compare them and provide a measure of relative importance. The final activity will involve creating a calculator, where the projects’ impacts are sorted into a single ranking, according to their contribution to FINDETER’s main objective as a development bank in Colombia. These final figures will allow the comparison of the estimated relative impacts of selecting one project over others. Policy relevance. National development banks exist for a reason, their main objective is to alleviate market failures that create inefficiencies on credit markets. FINDETER as a national development bank has limited resources to allocate and obtain the highest possible social benefit when a project is planned, financed, structured, executed, and advised. If properly conceived, this tool would serve as a major aid for deciding where to allocate resources when (as is very often the case) multiple projects of these 6 sectors are competing for the affordable -or free- resources FINDETER has to offer.

Activities Now that a summary of the existing ConTREEbute methodology and the aim of this Capstone have been presented, the activities proposed to develop this project are:

• Carry out a diagnosis of the methodology for ex-ante impact evaluation of projects in the context of the current Capstone’s project goal. • Using the examples of type projects, the team must identify, for these 6 sectors (housing, drinking water and sanitation, urban mobility, energy. education and health), the 4 most relevant impact dimensions that can be considered to assess projects within and across sectors. This means that any project that falls within one

20 In Spanish literature it is found as Social Return on Investment (SROI – TIR Social in Spanish) while in English literature is found as Social Internal Rate of Return (SROR). The purpose of this work is to measure social impact and for this document it will be called Social Return on Investment - SROI to unify concepts. The Social Discount Rate is a parameter that allows bringing to present value a future flow of economic costs and benefits. The rate reflects a society’s preference for present consumption versus future consumption c.f. (Moore, M. A. et al. 2004). Colombia’s National Planning Department estimates Colombia’s social discount rate to be 9%.

FINDETER 2021 Capstone project

of these sectors can be measured in terms of the four impact dimensions, in order to be compared with the rest.

• Using the type projects from the non-social sectors that have already been measured by the company ConTREEbute (which include more than 4 impacts) find the best way to select the four main impact dimensions for all the sectors. This would involve finding impact dimensions that are common to projects across both non-social sectors and social (Education and Health) sectors, i.e. two sectors of great relevance to FINDETER that were not included in ConTREEbute’s previous work. This identification of common impact dimensions is aimed at estimating the overall relative impact of each of the projects of interest, leading to their prioritisation and, hence, relative importance for FINDETER. The four impact dimensions identified by the Capstone group do not necessarily have to be included among the ones ConTREEbute identified.

• Create a calculator (spreadsheet) that, based on the necessary data to calculate the impact dimensions of a project, yields an estimate of relative importance and allows a comparison between projects and their prioritization, so that it can be identified which project is more relevant, in terms of the contribution they make to FINDETER’s main objective and in terms of their expected relative social impact .

An illustration: suppose FINDETER must choose between financing an education project and a mobility project. Should both options demand the same amount of investment, the calculator would enable FINDETER to identify which option would provide the greatest relative impact

Deliverables

1. November – December 2020 1. Carry out a diagnosis of ConTREEbute’s methodology that includes a review of new Literature to highlight the shortcomings of the existing methodology about ex-ante impact evaluation of projects through Cost-Benefit Analysis in the context of the current Capstone goal. (November 9th, 2020) 2. Identify and justify which are the 4 impact dimensions that will be measured for projects in each of the 6 sectors. (November 14th, 2020). 3. Carry out a review of the available data from FINDETER regarding the projects, as well as statistics at the project and regional level that could be useful, from different official sources in Colombia. Then, identify and justify what will be the indicators and data used to measure and estimate the 4 chosen impact dimensions for the 6 sectors. (December 11th, 2020).

FINDETER 2021 Capstone project

2. February 2021 i. Create a simple and tractable calculator (preferably in Excel) where the input will be the chosen data regarding the project (point iii.) and its corresponding location. The output will be a figure estimating the relative importance of each project. It is important to note this is an essential step, because the conception of such a calculator implies that the Capstone group will have to choose the weights and the aggregation function to convert the project data into a single figure of relative importance in projects and across sectors. When the previous steps are complete, FINDETER’s group of advisors will then use the calculator from point (2.i.) to compare the 12 projects from social and competitive sectors and demonstrate which ones have the greatest relative impact and should thus be conducted and/or invested in by FINDETER.

Notes The following information will be available to be delivered at the beginning of the capstone project:

• Data set containing the project level data for a sample of projects in each sector. • Data set containing public statistics at the municipal and regional level. • All supporting documents and files related to the consultancy project including the TIR Social methodology, its impact list called “Matriz de Indentificación de Impactos” and their respective estimation.

Dataset To conduct this project and identify the relevant variables to be used in the measurement of the economic impacts of FINDETER’s projects you will have access to data at two levels. On the one side you will have contextual data at the regional level related to demographics, poverty, labor market from the municipalities or departments where the projects are conducted. On the other side you will have data at the project level, data which is not as rich as we would like to estimate a project’s impact with the least amount of assumptions and extrapolation possible; this is due to the fact that this is a first step to identify what is the relevant information to be captured for the social impact measurement. As such, there will be information that must be carefully assumed and supported by the team. Data at the regional level from public statistics: Colombia is administratively divided by departments, which are in turn divided by municipalities. Several public statistics in Colombia are produced at these two different levels. Here is a non-exhaustive list that might be relevant for the project and that will be handed to the Capstone team. 1. Yearly GDP and GDP per capita by department. 2. Monthly labour market data by department (unemployment rate, global participation rate, active population, inactive population, etc.). 3. Multidimensional poverty index by municipality or department.

FINDETER 2021 Capstone project

4. Yearly fiscal performance index by municipality. 5. Yearly municipal categorization depending on the territories’ socioeconomic needs. 6. Monthly Consumer Price Index by department. 7. Census information from 2018 (Demographics).

Data at the project level for each sector studied: Every project conducted by FINDETER through one of the 4 lines of planning, structuring, financing, and execution has eight indicators associated with it. They are: 8. Municipality where the project is conducted. 9. Project’s CAPEX. 10. Sector of the project. 11. Benefitted population. 12. Unemployed workers from the region of the project. Finally, if one additionally variable per sector should be captured to significantly improve the functioning of the calculator (e.g. number of hospital beds created in the case of health), FINDETER can commit to improve the design of the Capstone calculator.

Appendix FINDETER operates through rediscount credit operations, which consist in delivering resources to a credit institution or organizations duly authorized by law to mediate. The previous transfer aims to finance credit operations in exchange for the transfer of sufficient guarantees. Specifically, FINDETER gives resources to the Financial Intermediary at a rediscount rate. In turn, the Financial Intermediary lends the resources to the beneficiary, which might be from the public or private sector, of the project or investment at an interest rate that is negotiated according to the risk level of the client. (Findeter Comercial, 2017) The FINDETER credits might be divided according to the following lines.

• Ordinary Resource: Entity own resources destined to finance programs or projects in the financeable sector (Findeter Comercial, 2017). • Special Line: Lines of credits created with Findeter resources to boost investment in a specific sector, by common agreement with the provisions of the National Government (Findeter Comercial, 2017). • Government Supported Rate: Common initiatives between FINDETER and the national government to strengthen a particular sector of interest. The Ministry of Finance or other public entity provides the resources to foster

investment in a specific sector. Usually, the use of these rates has a specific amount and destination (Findeter Comercial, 2017).

FINDETER 2021 Capstone project

• Dollars: Financing resources that have special conditions according to this currency. All the characteristics of these credits are in dollars (Findeter Comercial, 2017).

Figure 1: Presence of Findeter across the country. Author elaboration using entity´s own database.

According to the map, FINDETER has participated in projects among all the regions thanks to its management strategies. The entity had reached 646 (58%) of Colombia’s municipalities by June 2020, where 515 municipalities are characterized as the least developed3.

References Moore, M. A., Boardman, A. E., Vining, A. R., Weimer, D. L., & Greenberg, D. H. (2004). “Just give me a number!” Practical values for the social discount rate. Journal of Policy Analysis and Management, 23(4), 789-812. ConTREEbute. (2020) Proyecto DELTA - Medición de Impactos.

FINDETER 2021 Capstone project

Annex 2. FINDETER’s methodology for project prioritisation

In 2017 FINDETER developed a methodology for the selection and prioritisation of projects based on the following criteria and subcriteria:

Criteria/Weight Indicators/subcriteria

Size of benefited population (divided by quintiles)

Size of investment Projects’s social Impact (40%) Environmental impact

Municipal categorisation

GINI coefficient

Debt capacity of project’s sponsor

Percentage of missing resources FINDETER’s potential benefit FINDETER’s vertical integration (30%) Project’s profitability

Clarity in the business scheme

Economic contingency

Legal contingency

Project’s risks for the Reputational Government (30%) Operational

Personnel contingency

Environmental contingency

FINDETER 2021 Capstone project

The score calculated for each project ranges between 1 and 5, depending on how much it contributed to FINDETER’s mission, vision and value added. The obtained score informs the decision-making process depending on which of the following ranges it belongs to:

Project Project Project for re- accepted rejected evaluation

Total >3.5 < 2.5 2.5 < score < 3.5 score

Sources: FINDETER (2017).

FINDETER 2021 Capstone project

Annex 3. MCA case studies by OECD countries

● United Kingdom Since 2011, the UK government has set out a list with the ‘Top 40 priority infrastructure investments’. This allows the government to focus on investments which make the most significant contribution towards three objectives: 1) contribute to economic growth, 2) deliver substantial new or replacement infrastructure with enhanced quality, sustainability and capacity, 3) attract or unlock private investment. In doing so, the government used the following criteria: Strategic Importance, Capital Value, regional Priority and Unlocking Investment.

● France The French Ministry of Environment and Sustainable Development used multicriteria analysis, with the objective of analysing the potential of projects for flood prevention. The application of MCA within this context, enabled the impact analysis of projects with the aim of reducing the consequences of flooding on human health, the economy and the resilience of the territories, the environment and heritage.

● Australia The goal of Australia's National Infrastructure Plan is to identify key projects of national relevance within the framework of the strategy of each sector. By using this prioritisation mechanism, the institution is then able to estimate the potential benefits of the projects under consideration and to define a list of priority investments, based on the following criteria: Strategic fit with government priorities, Social, economic and environmental value, as well as Deliverability.

Own elaboration Sources: HM Treasury (2004), French Ministry of Ecological and Solidarity Transition (2018), Australian Government (2021)

FINDETER 2021 Capstone project

Annex 4. MCA case studies by LAC countries

● Colombia In 2016, the DNP used multicriteria analysis to select the most appropriate contracting modalities for the implementation of infrastructure development projects. To select between public private partnerships and traditional public work for specific infrastructure projects, the institution used criteria such as value for money, innovation incentives and sustainability of the project.

● Chile Chile has also institutionalised the use of MCA to support the decision-making process in investment (Marcelo, House & Raina, 2018). In 2014, the government began to use it to evaluate reservoirs, based on the following criteria: Economic, Social, Strategic, Environmental/territorial, Management.

● Peru In 2019 the Ministry of Economics and Finance published the National Plan of Infrastructure Projects for Competitiveness. The objective was to provide the Peruvian Government with an accurate tool for the prioritisation of all projects across economic and social sectors. To achieve this goal, the Ministry used an MCA approach that enabled the ranking of infrastructure projects according to the following criteria: Economic Potential, Competitiveness, Management Capacity, Social Impact, Finance Impact. As a result of the MCA, the PNIC presented a portfolio of fifty-two projects with a value of almost 100,000 million soles.

Own elaboration Sources: Marcelo, House & Raina (2018), Ministerio de Hacienda de Chile (2014), Ministerio de Economía y Finanzas de Perú (2019), DNP (2016)

FINDETER 2021 Capstone project

Annex 5. Example of CBA methodology

This example is drawn from ConTREEbute’s work with FINDETER. The project assessed is an infrastructure project in Ibagué, Colombia, that aimed to improve coverage, accessibility and connectivity between the different sectors of the city (urban, peripheral and rural) through the renewal of the fleet, construction and operation of the Strategic System of Public Transport (SETP).

Measurable impacts by dimension: 1. Environmental: lower emissions of greenhouse gasses (+) 2. Socio-political: freeing up resources due to shorter travel times (+) 3. Economic: freeing of economic resource due to lower cost of fleet maintenance (+) Use of resources: ● Adjusted OPEX (-) ● Adjusted CAPEX (-)

Measuring the benefit of shorter travel times:

This figure is projected over time and is brought to present value using the 9% national social discount rate. The same process is repeated for all identified costs and benefits.

Source: ConTREEbute (2020)

FINDETER 2021 Capstone project

Annex 6. Example CEA methodology

An example of CEA is provided by the policy and economic consultancy London Economics. The organisation conducted CEA to assess the UK Space Agency’s International Partnership Programme (IPP). The objective was to assess the cost- effectiveness of IPP (space) projects relative to alternative (non-space) options. One of the sectors of interest analysed by the report is forestry. The analysis can be broken down into three steps.

Step 1. Calculate cost-effectiveness ratio of IPP forestry projects ● Total costs: 2.3m hectares, at an average cost of £21.22/hectare of avoided deforestation. ● Total impacts: 2.3m hectares of avoided deforestation by 2021

Total costs (£) 48,201,925

Total impacts (ha) 2,271,118

CEA (£/ha) 21.22

Step 2. Calculate cost-effectiveness ratio of IPP forestry project alternatives

Total costs (£) 891,621,497

Total impacts (ha) 5,457,299

CEA (£/ha) 163.38

Step 3. Compare IPP-v-Alternatives cost-effectiveness This analysis suggests that IPP (space) solutions are on average almost 8 times more cost- effective than the non-space alternatives.

Source: London Economics (2019)

100

FINDETER 2021 Capstone project

Annex 7. Summary of the indicators for the implementation of the FiPT

I. Economic potential criterion

Sub-criteria Indicator Description Unit of Source of measurement information

Economic Multiplier effect Captures the Colombian pesos DNP, impact in the of the investment increase in the FINDETER - short-run of executed district's economic Project level public/private activity via the database investment in execution of private projects consumption

Economic Reduction in the Represents the Unemployment DANE impact in the unemployment mobilisation of rate (%) at employment rate by the labour as a result department level generation execution of the of the construction project of infrastructure.

Economic Promotion of Captures to what Dummy variable FINDETER impact in the human capital, extent the project (1= if the project long-run innovation and includes includes any productivity component that component related promote to promotion of sustainable growth human capital, in the long-run. innovation and productivity; 0 = otherwise).

Own elaboration

101

FINDETER 2021 Capstone project

II. Social impact criterion

Sub-criteria Indicators Description Unit of Source of measurement information

Direct Number of direct Represents the scope in # of FINDETER - beneficiaries beneficiaries per terms of the number of beneficiaries per Project-level thousand individuals benefiting thousand of database Colombian pesos from improvements in pesos invested invested living conditions due to access to better quality i.e. # of public services beneficiaries per CAPEX 2019

Reduction of Position of the Captures the relative Municipality DANE sectoral gaps in municipality on importance of projects’ position in the the area of the indicator sectoral contribution municipal influence representative of according to the distribution of the sector relative sectoral gaps sectoral gaps considered faced by municipalities.

Socio-economic Multidimensional Constructed from % of population DANE / DNP characteristics Poverty Index at objective non- considered to be of the area of the municipal monetised measures of living in poverty influence level (2018 data) poverty. It captures the acute deprivations in health, education, and living standards at the individual level21.

Gini Index at the Measures the Unit between 0 DANE / DNP municipal level distribution of income (full equality) (2018 data) across a population. and 1 (full inequality)

Own elaboration

21 Departamento Administrativo Nacional de Estadística – DANE (2018), Methodological note of the municipal multidimensional poverty measure with census information, CNPV 2018.

102

FINDETER 2021 Capstone project

III. Institutional capacity

Sub-criteria Indicators Description Unit of Source of measurement information

Budgeting and Fiscal Measures Ranking from 0 to DNP fiscal Performance municipalities’ 100 performance Index performance in terms of fiscal viability and sustainability.

Municipal Municipality Represents a Ranking from 0 to DNP management management performance of 100 capacity score municipalities in terms of their management capacity

Local capacity Project Allocation of the 2 = if the project FINDETER building funding project funding to the allocates resources public sector. to public sector Represents the entities 1 = if the project which receive allocates resources financial contribution. to private which delivers a public service. 0 = if the project allocates resources to the private sector on a private contract.

Generation Represents the 1 = if the project FINDETER of revenue project’s promotion of generates revenue fiscal autonomy in directly or indirectly municipalities by the 0 = otherwise generation of revenue directly and indirectly (increase in local tax base). Own elaboration

103

FINDETER 2021 Capstone project

IV. Strategic importance criterion

Sub-criteria Indicator Description Unit of Source of measurement information

Alignment to Integrate It reflects the added 1=if the project is FINDETER’s sales model value that FINDETER involved in two or FINDETER strategy can grant when the more FINDETER’s project receives the business lines; technical support of two 0=otherwise or more business lines.

New It reflects the 1 = if the project’s municipality FINDETER ability to execution involved diversify the number of a new municipality; impacted 0 = otherwise municipalities, which can increase future interventions likelihood in that territory.

Multi- It reflects the 1 = if the project’s territorial FINDETER ability to execution involves project amplify the number of more than one impacted municipalities municipality; 0 = otherwise Own elaboration

104

FINDETER 2021 Capstone project

Sub-criteria Indicator Description Unit of Source of measurement information

Alignment to Sustainability It reflects the project´s 1 = if the FINDETER Government’s pillar contribution to any of the project strategy following goals: i) contributes greenhouse gas reduction with effect, ii) deforestation sustainability; reduction effect, iii) 0 = otherwise increase in sustainable production and conservation systems, iv) adaptation to climate change.

Peace It reflects the project’s 1 = if the Database Building contribution to peace project PDET Pillar building pillar, if it’s contributes execution involved one of with peace the 170 prioritised building pillar; municipalities of 0 = otherwise Development Programme with Territorial Approach (PDET, by its acronyms in Spanish)22.

Inclusion of It reflects the project´s 1 = if the DANE minorities contribution if it’s located project in territories with contributes significant proportions of with inclusion ethnic and racial of minorities minorities. 0 = otherwise

Promotion of It reflects the project´s 1 = if the FINDETER orange alignment to promotion of project economy the orange economy contributes to through artistic and orange cultural training programs. economy; 0 = otherwise Own elaboration

22 PDET prioritized municipalities with higher rates of poverty, presence of illicit economies and institutional weakness, associated with the effects of the armed conflict.

105

FINDETER 2021 Capstone project

Annex 8. Justification behind indicators of FiPT

I. Economic potential

Elasticities of Public Capital

All Public Capital Core Public Capital

Time period Regional/local National Regional/local National

Short-run 0.106 0.083 0.154 0.131

Long-run 0.145 0.122 0.193 0.170 Source: Bom & Ligthart (2015)

II. Institutional capacity

Fiscal Performance Index - Fiscal results dimension for 80% of the qualification

Thematic Indicator Objective of indicator

Transfers Dependence on Transfers Determine whether or not national transfers are the fundamental resources to finance territorial development.

Investment Fixed FBK Relevance Quantify the magnitude of the investment in the fixed FBK that the ET executes in relation to its investment expense. (Alignment with statistical statistics).

Indebtedness Sustainability of the Debt Measure the capacity of the ET to support the balance of its debt with the income available for its financing in accordance with the provisions of Law 358 of 1997.

Short borrowing term Measures the short debt of territorial entities as a proportion of their current assets.

Measures of Current savings Determine the degree to which surpluses are deficit or released to finance the investment, after covering surplus the operation, the payment of debt interest. It is a measure of financial solvency. (Integrate accounting analysis).

106

FINDETER 2021 Capstone project

Taxes workout Measure the fiscal balance of the TNCs. If there is a high deficit, it can put the territorial fiscal consolidation at risk, but if there is an excessive surplus, it also shows the low capacity of the ET to execute. (Alignment with statistical statistics). Source: DNP

Fiscal Performance Index - Territorial Financial Management Dimension for 20% of the qualification

Thematic Indicator Objective of indicator

Programming capacity It measures the capacity of the territorial entity to and revenue execution make a correct planning of its current income Cadastral update Financial Planning Execution capacity of It measures the capacity of the territorial entity to investment expenditure execute the resources committed for investment in accordance with the provisions of Law 819 of 2003

Measure efficiency in operating expenses, taking Compliance into account that it must be covered with free-use with Law 617 Level of “slack” (+) income from the ET. The slack is qualified if the of 2000 limits of authority and councils are met. (operating efficiency approximation)

Measure the capacity of TNCs to increase their Growth resources own own resources (own effort). The bonus consists effort of the average of the growth ratio of both periods Rating bonus Encourage municipalities to carry out cadastral Cadastral update processes as mechanisms to promote the goals of the Multipurpose Cadastre. (Incentive goals PND 2018-2022) Source: DNP

Classification of municipalities by the Fiscal Performance Index

Deteriorated At risk Vulnerable Solvent Sustainable

<40 40-60 60-70 70-80 >80

Source: DNP

107

FINDETER 2021 Capstone project

Annex 9. Examples of ranking projects using indicators for FiPT

I. Economic potential

Example of ranking using unemployment indicator

FINDETER has to choose between three projects: • Project 1 in Energy in Municipality A • Project 2 in Water and Sanitation in Municipality B • Project 3 in Housing in Municipality C

The Prioritisation tool would then make it possible to compare: • The rank of municipalities on the indicator of unemployment rate

Suppose that municipality A is ranked in the top third of municipalities facing the largest unemployment rate, and municipality B is ranked in the middle third. Then Project 1 would be prioritised, since it makes a positive contribution to a department in which the percentage of population unemployment is higher.

Own elaboration

II. Social impact

Example of ranking using sectoral gaps indicator

FINDETER has to choose between two projects: • Project 1 in Health in Municipality A • Project 2 in Education in Municipality B

The calculator would then make it possible to compare: • The rank of municipality A on the representative indicator of the "Health" sector, i.e. the mortality rate for example or the vaccination rate • The rank of municipality B on the representative indicator of the "Education" sector, i.e. the primary education coverage or the illiteracy rate

Suppose that municipality A is above the municipalities’ median values of sectoral gap in health, and municipality B is above the municipalities’ median values of sectoral gap in education. Both projects would then be prioritised, since they make positive contribution to a sector in which the municipality is most disadvantaged.

Own elaboration

108

FINDETER 2021 Capstone project

Example of ranking using Social characteristics of the area of influence

• Multidimensional Poverty Index: o Among two projects, and keeping everything else constant, the project targeted at the municipality with the highest MPI will be prioritized. • Gini coefficient: o Among two projects, and keeping everything else constant, the project targeted at the municipality with the highest Gini coefficient will be prioritized.

Own elaboration

IV. Institutional capacity

Box X. Example of ranking using fiscal and management performance indicators

• Budgeting and fiscal performance: o Among two projects, and keeping everything else constant, the project targeted at the municipality with the lowest level of fiscal performance will be prioritized. • Municipality management capacity: o Among two projects, and keeping everything else constant, the project targeted at the municipality with the lowest level of management performance will be prioritized.

Own elaboration

IV. Strategic importance

Example of ranking using Strategic importance indicators

If project A is aligned with any of FINDETER’s priorities but not with any of the Government’s priorities, while the other way around happens with project B, holding the rest of MCA indicators constant, both projects would be assigned a score of 100. Therefore, both projects would be equally impactful.

Own elaboration

109

FINDETER 2021 Capstone project

Annex 10. Results of the pairwise comparison for each subcriteria and indicators

I. Economic potential

Short-run economic Direct employment Long-run economic impact generation impact Short-run economic 1 1/3 1/3 impact Direct employment 3 1 1/3 generation Long-run economic 3 3 1 impact

II. Social impact

Infrastructure gaps Socio-economic Direct beneficiaries reduction characteristics

Direct beneficiaries 1 1/2 1/3

Infrastructure gaps 2 1 1/3 reduction Socio-economic 3 3 1 characteristics

• Sub-criteria: Socioeconomic characteristics

Multidimensional Inequality Poverty Multidimensional 1 4 Poverty

Inequality 1/4 1

110

FINDETER 2021 Capstone project

III. Institutional capacity

Budgeting and fiscal Municipality Local capacity building performance management capacity Budgeting and fiscal 1 1/4 1/4 performance Municipality 4 1 1 management capacity

Local capacity building 4 1 1

• Sub-criteria: Local capacity building

Potential public Potential generation of capacity building revenues Potential public 1 1/3 capacity building Potential generation of 3 1 revenues

IV. Strategic importance

Alignment to Alignment to FINDETER´s strategy Government’s strategy Alignment to 1 5 FINDETER´s strategy Alignment to 1/5 1 Government’s strategy

• Sub-criteria: Alignment to FINDETER’s strategy

Integrate sales model New municipality

Integrate sales model 1 1/5

New municipality 5 1

Groups of 1/3 1/6 municipalities

111

FINDETER 2021 Capstone project

• Sub-criteria: Alignment to Government’s strategy

Orange economy Sustainability priority Peace Building priority priority

Sustainability priority 1 3 6

Peace Building Priority 1/3 1 6

Minorities Inclusion 1/4 1/2 1/4 Priority Orange economy 1/6 1/6 1 priority

Results of the pairwise comparison for each subcriteria and indicators

Annex 11. FINDETER’s operations share by each business line in 2020

Billions of Colombian pesos of Colombian Billions

Own elaboration Source: FINDETER’s input

112

FINDETER 2021 Capstone project

Annex 12. Description of projects’ sample selected by FINDETER

Sector: Urban Transportation

1. Strategic Transportation System of Ibague

Objective: Development of the main inputs for the operation, infrastructure and technology components, as well as the financial and legal assistance for the implementation of the strategic public transport system of Ibague.

2. Strategic Transportation System of Manizales

Objective: Development of the feasibility study that will allow the municipality to continue with the initiative to reorganise its public transport system. This project will improve the city’s connectivity and improve travel times for citizens, offering a better quality of life.

Source: based on FINDETER’s input

113

FINDETER 2021 Capstone project

Sector: Water and Sanitation

1. Construction of a new sewage collector

Objective: Increase in the hydraulic capacity system from 10 to 18 m3/s by the construction of a new collector that allows the mitigation of the risk of collapse of the existing infrastructure for the canalisation of the Egoya stream.

2. Optimisation of the water and sanitation system

Objective: Development of the studies for the design of the Solar Generation System, that justify the procurement of resources for its implementation.

Source: based on FINDETER’s input

114

FINDETER 2021 Capstone project

Sector: Energy

1. Modernisation of street lights in Cartagena

Objective: Diagnosis of the alternatives considering the technical, legal and financial structuring for the development of the public lighting modernisation project.

2. Photovoltaic Solar Park in Miraflores, Guaviare

Objective: Development of the studies for the design of the Solar Generation System, that justify the procurement of resources for its implementation.

Source: FINDETER

115

FINDETER 2021 Capstone project

Sector: Health

1. Construction of the second tower of the hospital Valle de Lili in Cali

Objective: Increase the availability of infrastructure that guarantees the provision of complex health services to users in the region.

2. Replacement of the infrastructure of San Juan de Dios hospital in Arboledas

Objective: Development of the studies for the design of the Solar Generation System, that justify the procurement of resources for its implementation.

Source: FINDETER

116

FINDETER 2021 Capstone project

Sector: Education

1. School and Child Development Centre in Villavicencio, Meta

Objective: The Child Development Centre provides an institutional service that seeks to guarantee primary education, care and nutrition for children under 5 years of age.

2. Nazareth boarding school dormitories

Objective: This project includes the construction of buildings that will dormitories of the Nazareth Boarding School, which belongs to the María Auxiliadora Educational Institution. The project also includes the construction of baths for the San Antonio de Padua educational institution (in the town of Miriti) and also the improvement of classrooms and sanitary infrastructure at the Jose Celestino Mutis educational institution (in Puerto Nariño).

Source: FINDETER.

117

FINDETER 2021 Capstone project

Annex 13. Template for the interviews on the MCA framework with representatives of FINDETER’s business lines

Date and time FINDETER’s business line Interviewed - FINDETER Full name and position Interviewers - LSE team Main findings Verbatims Part I - Questions for qualitative analysis

1. Could you describe the participation of your Business Line in the regular process of selecting/designing projects to be funded by FINDETER?

2. The MCA is a framework that will identify and assess the social impact of projects that may be potentially selected by FINDETER. The outcome is a final ranking that informs the relative project prioritisation among all the alternatives considered. How would this prioritisation tool impact the decision-making in your Business Line?

3. In the past, certain initiatives were carried out in FINDETER with a similar aim (i.e. the MCA framework developed by the Office of Project Structuring, the TIR Social Estimation by ConTREEbute). Did your Business Line participate in the design of these tools? If so, how? To what extent did your Business Line find those efforts useful for the decision-making process?

4. For the design of the MCA framework, FINDETER faces restrictions regarding access to data because of the early-stage nature of the projects and of its systematisation by other offices. This has led to complement the limited information at project level with indicators at municipal level. Moreover, the potential MCA will include criteria on 1. Economic Potential, 2. Social Impact, 3. Institutional Capacity, and 4. Strategic Importance. Do you find this design useful? To what degree do you consider certain criteria more relevant for FINDETER than others?

5. Our aim is for the MCA framework to be operationally feasible to implement in the short run and flexible enough to continue its use in the long run. What do you perceive as the main challenges that could affect this goal with relation to your Business Line? How do you think we can minimise potential risks?

6. What would be your main expectations about this tool for the prioritisation of projects?

118

FINDETER 2021 Capstone project

Part II - Questions for the quantitative analysis

1. If you were to rank the identified criteria and subcriteria, how would you score them for the selection of projects (from 1 to 4 across criteria, and within subcriteria - being 1 more important and 4 less important)?

Criteria Subcriteria Subcriteria Criteria ranking ranking

1 Economic Economic impact in the short-run Potential of public/private investment in private consumption

Direct employment generation

Economic impact in the long-run

2 Social Direct beneficiaries Impact Reduction of infrastructure gaps

Socio-economic characteristics of the area of influence

3 Institution Budgeting and fiscal performance al Capacity Municipality management capacity

Fiscal sustainability

Local capacity building

4 Strategic Alignment to FINDETER´s Importanc strategy e Alignment to Government’s strategy

119

FINDETER 2021 Capstone project

2. If you were to weigh the identified criteria and subcriteria, how would you score them for the selection of projects (0%-100%)?

Criteria Subcriteria Subcriteria Criteria weight weight

1 Economic Economic impact in the short-run of Potential public/private investment in private consumption

Direct employment generation

Economic impact in the long-run

2 Social Direct beneficiaries Impact Reduction of infrastructure gaps

Socio-economic characteristics of the area of influence

3 Institutional Budgeting and fiscal performance Capacity Municipality management capacity

Fiscal sustainability

Local capacity building

4 Strategic Alignment to FINDETER´s strategy Importance Alignment to Government’s strategy

3. Is there something you would like to see included that is currently missing in the criteria and sub-criteria?

120

FINDETER 2021 Capstone project

Annex 14. Main findings and verbatims from the interviews with the representatives of FINDETER’s business lines

I. Structuring line

Date and time 28/01/2021 - 2pm UK time

Interviewers LSE Team

Main findings ● All four factors are important from project structuring. They would be prioritised depending on the needs of the territories. ● Keys for success: ○ Generalise the application of the methodology and ground its application to the territories. ○ Analyse and take into account information on territories and territorial specificities. ● Main expectation of the tool: Identify sectoral needs that can be matched to the national government's plans ● Retrospective diagnosis: This tool can be used to diagnose which projects are being supported. ● Technical tool that will undoubtedly contribute to the decision-making process and can bring more legitimacy to this process.

Verbatims ● “It is important to note that the nature of products that are part of an earlier stage of the projects, as it is the case for products of the structuring business line, may not immediately reflect the impact that would be generated by the project. Hence the need for and importance of a tool to assess the socio-economic impact of projects.” ● “This tool would be very useful throughout the life cycle of a project, until its implementation.”

121

FINDETER 2021 Capstone project

II. Execution line

Date and time 02/02/21 - 1.30 UK time

Interviewers LSE Team

Main findings ● This tool could allow the business line to align FINDETER’s work with national development plans ● Retroactive use of the tool: It would also be interesting for the business line to use this tool to review the impact FINDETER is having in different territories and the types of projects which are generating these impacts. ● All criteria are considered suitable for the model from the execution product, with economic potential being the most important one. ● Key for success: communicating the importance of the tool, ensure the implementation across areas

Verbatims “In order to minimise potential risks, the Economic Studies Department and the LSE team should explain the importance of the tool, clearly define "how to measure" this impact and make sure the tool is effectively implemented across areas.”

122

FINDETER 2021 Capstone project

III. Planning line

Date and time 29/01/2021 - 2pm UK time

Interviewers LSE Team

Main findings ● An important work around this tool will be the socialisation of the result of the final tool. In particular, it will be necessary to succeed in selling it internally as a useful tool for estimating the impacts generated by the planning product offered by the business line. ● As a relatively recently established business line with limited demand at present, Planning does not have a project prioritisation framework. ● Such a tool for estimating the social impact of projects could be useful both internally, as a mechanism for monitoring prioritised projects, and externally, for communicating the impact of its planning products. ● The priority of the Planning business line is to provide better institutional capacity to the beneficiary territories of its product. ● Among the challenges related to the implementation of the tool, the representatives of this business line note issues related to the general acceptance of the tool. Among the keys to success, it is important to communicate about the design of the tool as well as its use, in order to develop ownership of the tool in the different areas of FINDETER.

Verbatims ● “Such a tool will allow FINDETER to move from a financial analysis of its contribution to a social impact analysis.” ● “The interest of such a tool is threefold: it allows FINDETER to act in a homogeneous way, to have a clear decision guide and, finally, to give credibility to the entity’s approach on how projects are selected.”

123

FINDETER 2021 Capstone project

IV. Finance line

Date and time 27/01/2021 - 2pm UK time

Interviewers LSE Team

Main findings ● A prioritisation tool could allow the identification of new financing strategies to promote projects with greater social impacts. ● Past impact assessment models, including the estimation of social return on investment, are ex-post models which are part of a post-credit measurement process. Unfortunately, to date, these results are not taken into account by business lines’ representatives when evaluating a project and taking an investment decision. ● From both a commercial and social point of view, the business line's priority is to reach projects from an increasing number of municipalities and territories, especially those with the most pressing needs and with the least financial and administrative capacity. ● The priority criterion of the business line is the commercial impact for the entity as well as the economic impact of the investment on the territory. ● It would be interesting for this tool to assist us in prioritising projects by sector. ● It would be relevant to make the tool applicable to all sectors and business lines. It is important to avoid keeping this tool too general. Otherwise, it will be complicated for the business lines to use it when making decisions.

Verbatims ● “I would suggest making the tool very practical and easy to use.” ● “This tool would be useful to justify our decisions in terms of the social impact they generate. This is why we wanted to measure the TIR Social in the first place. But unfortunately, these social impact measurement results are not used for investment decision-making.”

124

FINDETER 2021 Capstone project

Annex 15. Template with the Saaty Scale

This template was sent to the Economic Studies Department and to the Business Lines as part of the weighting stage – Example for the Social Impact criterion, subcriteria and indicators

Subcriteria - Social Impact

Reduction of Socio-economic Direct infrastructure characteristics of the beneficiaries gaps area of influence

Direct beneficiaries

Reduction of infrastructure gaps Socio-economic characteristics of the area of influence

Indicators – Social Impact

Multidimensional Unmet Basic Needs Gini Index Poverty Index Index

Multidimensional Poverty Index

Gini Index

Unmet Basic Needs Index

125

FINDETER 2021 Capstone project

Annex 16. Template with the dummy variables tables sent to the Economic Studies Department and to the Business Lines – Example for the Economic Potential criterion

Sub- Indicator Project Code criterion Description A1 A2 B1 B2 C1 C2 D1 D2 E1 E2 G1 G2

Direct 1 = if the employment Sector is low- generation labour intensive

2 = if the Sector is medium-labour intensive

3 = if the Sector is high- labour intensive

Economic 1 = if the impact in project includes the long-run any component related to promotion of human capital, innovation and productivity

0 = otherwise

126

FINDETER 2021 Capstone project

Annex 17. Excel template for FiPT

127