Document of The World Bank FOR OFFICIAL USE ONLY
Public Disclosure Authorized Report No: ICR00004862
IMPLEMENTATION COMPLETION AND RESULTS REPORT (No. 4882-NG)
ON A CREDIT IN THE AMOUNT OF SDR 102 MILLION (US$ 160 MILLION EQUIVALENT) Public Disclosure Authorized TO THE Federal Ministry of Finance
FOR A Nigeria - Growth and Employment (GEM)Project
April 28, 2020
Public Disclosure Authorized
Finance, Competitiveness And Innovation Global Practice Africa Region
Public Disclosure Authorized
CURRENCY EQUIVALENTS
(Exchange Rate Effective {Feb 04, 2020})
Currency Unit = = US$1 US$ = SDR 1
FISCAL YEAR July 1 - June 30
Regional Vice President: Hafez M. H. Ghanem Country Director: Shubham Chaudhuri Regional Director: Elisabeth Huybens Practice Manager: Rashmi Shankar Task Team Leader(s): Kofi-Boateng Agyen, Wenye Dong ICR Main Contributor: Natalia Agapitova and Lindsey Tan Lim
ABBREVIATIONS AND ACRONYMS
AAP Africa Action Plan AFD French Development Agency AfDB African Development Bank AIEA Association of International Education Administrators APL Adaptable Program Loan AVDD Aso Villa Demo Day BDS Business Development Services BIR Boards of Internal revenue BPP Bureau of Public Procurement CAD Commercial Agriculture Development Project CBN Central Bank of Nigeria CFAA Country Financial Accountability Assessment CEM Country Economic Memorandum CPAR Country Procurement Assessment Review CPS Country Partnership Strategy DFID Department for International Development EFCC Economic and Financial Crimes Commission ESMF Environmental and Social Management Framework ESIA Environmental and Social Impact Assessment FCT Federal Capital Territory FGN Federal Government of Nigeria FIAS Foreign Investment Advisory Services FMITI Federal Ministry of Industry, Trade & Investment FMOC Federal Ministry of Commerce FMR Financial Management Reports FPD Finance and Private Development FPFMD Federal Project Financial Management Directorate FPIU Federal Project Implementation Unit FPM Financial Procedures Manual GEMS Growth and Employment in States Project ICA Investment Climate Assessment ICOR Incremental Capital Output Ratio ICP Investment Climate Program ICR Implementation Completion and Results Report ICT Information and Communications Technology IDA International Development Association IFC International Finance Corporation IPR Intellectual Property Rights IPSAS International Public Sector Accounting Standards ISPON Institute of Software Practitioners of Nigeria ITAN Information Technology Association of Nigeria IteS Information Technology Enabled Services LGA Local Government Areas LIL Learning and Innovation Loan MCT Ministry of Culture and Tourism MDA Ministries, Departments and Agencies, not Ministries and Development Agencies M&E Monitoring and Evaluation MIGA Multilateral Investment Guarantee Agency MSME Micro Small and Medium Enterprise NEEDS National Economic Empowerment and Development Strategy NESG Nigerian Economic Summit Group NCS Nigerian Customs Service NIAF Nigeria Infrastructure Advisory Facility NIPC Nigerian Investment Promotion Commission NITDA National Information Technology Development Agency NPF Nigerian Police Force NSC National Steering Committee NTDC Nigeria Tourist Development Corporation OAGF Office of the Accountant General of the Federation ODIN Outsourcing Development Initiative of Nigeria OSIC One Stop Investment Center PAD Project Appraisal Document PEFA Public Expenditure and Financial Accountability Program PEMFAR Public Expenditure Management and Financial Accountability Review PIM Project Implementation Manual PIU Project Implementation Unit PMP Pest Management Plan PPF Project Preparation Facility PPP Public Private Partnership RAP Resettlement Action Plan RPF Resettlement Policy Framework SEEDS State Economic Empowerment and Development Strategy SGC State Growth Committee SIL Specific Investment Loan SME Small and Medium Enterprises TA Technical Assistance TRIPS Trade Related Aspects of Intellectual Property Rights TVET Technical Vocational Education Training USAID United States Agency for International Development UNCTAD United Nations Conference on Trade and Development WTO World Trade Organization
TABLE OF CONTENTS
DATA SHEET ...... 1 I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES ...... 6 A. CONTEXT AT APPRAISAL ...... 6 B. SIGNIFICANT CHANGES DURING IMPLEMENTATION (IF APPLICABLE) ...... 10 II. OUTCOME ...... 16 A. RELEVANCE OF PDOs ...... 16 B. ACHIEVEMENT OF PDOs (EFFICACY) ...... 17 C. EFFICIENCY ...... 24 D. JUSTIFICATION OF OVERALL OUTCOME RATING ...... 25 E. OTHER OUTCOMES AND IMPACTS (IF ANY) ...... 25 III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME ...... 26 A. KEY FACTORS DURING PREPARATION ...... 26 B. KEY FACTORS DURING IMPLEMENTATION ...... 28 IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME .. 29 A. QUALITY OF MONITORING AND EVALUATION (M&E) ...... 29 B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE ...... 31 C. BANK PERFORMANCE ...... 32 D. RISK TO DEVELOPMENT OUTCOME ...... 34 V. LESSONS AND RECOMMENDATIONS ...... 35 ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS ...... 39 ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION ...... 48 ANNEX 3. PROJECT COST BY COMPONENT ...... 50 ANNEX 4. EFFICIENCY ANALYSIS ...... 51 ANNEX 5. BORROWER, CO-FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS ... 52 ANNEX 6. SUPPORTING DOCUMENTS (IF ANY) ...... 54 ANNEX 7. SUMMARY OF RESULTS FROM DFID ASSESSMENT ...... 55 ANNEX 8. IMPACT EVALUATION RESULTS ...... 57 ANNEX 9. BIG PLATFORM ...... 60 ANNEX 10. DFID GEMS RESULTS AND STAKEHOLDERS ...... 62 ANNEX 11. ORGANOGRAM ...... 64 ANNEX 12. ECONOMIC ANALYSIS ...... 65
The World Bank Nigeria - Growth & Employment (P103499)
DATA SHEET
BASIC INFORMATION
Product Information Project ID Project Name
P103499 Nigeria - Growth & Employment
Country Financing Instrument
Nigeria Investment Project Financing
Original EA Category Revised EA Category
Partial Assessment (B) Partial Assessment (B)
Organizations
Borrower Implementing Agency
Federal Ministry of Finance Ministry of Trade and Investment
Project Development Objective (PDO)
Original PDO To increase growth and employment in participating States.
Revised PDO To increase firm growth and employment in participating firms in Nigeria.
Page 1 of 71
The World Bank Nigeria - Growth & Employment (P103499)
FINANCING
Original Amount (US$) Revised Amount (US$) Actual Disbursed (US$) World Bank Financing
160,000,000 83,568,552 66,853,193 IDA-48820 Total 160,000,000 83,568,552 66,853,193
Non-World Bank Financing 0 0 0 Borrower/Recipient 0 0 0 Total 0 0 0 Total Project Cost 160,000,000 83,568,552 66,853,193
KEY DATES
ApprovalFIN_TABLE_DAT Effectiveness MTR Review Original Closing Actual Closing 17-Mar-2011 29-Jul-2013 15-Oct-2014 30-Dec-2016 29-Mar-2019
RESTRUCTURING AND/OR ADDITIONAL FINANCING
Date(s) Amount Disbursed (US$M) Key Revisions 01-Aug-2017 56.26 Change in Results Framework Change in Components and Cost Change in Loan Closing Date(s) Reallocation between Disbursement Categories Change in Disbursements Arrangements Change in Implementation Schedule 07-Dec-2018 79.85 Change in Results Framework Change in Components and Cost Change in Loan Closing Date(s) Change in Implementation Schedule 29-Mar-2019 66.85 Change in Components and Cost Cancellation of Financing Reallocation between Disbursement Categories
Page 2 of 71
The World Bank Nigeria - Growth & Employment (P103499)
KEY RATINGS
Outcome Bank Performance M&E Quality Moderately Unsatisfactory Moderately Unsatisfactory Modest
RATINGS OF PROJECT PERFORMANCE IN ISRs
Actual No. Date ISR Archived DO Rating IP Rating Disbursements (US$M) 01 27-Oct-2011 Satisfactory Moderately Unsatisfactory .81
02 30-Jun-2012 Satisfactory Satisfactory 1.06 Moderately 03 04-Aug-2012 Moderately Unsatisfactory 1.06 Unsatisfactory Moderately 04 06-Mar-2013 Moderately Unsatisfactory 1.06 Unsatisfactory Moderately 05 07-Sep-2013 Moderately Unsatisfactory 1.06 Unsatisfactory Moderately 06 18-Jan-2014 Moderately Unsatisfactory 2.48 Unsatisfactory 07 05-Jul-2014 Moderately Satisfactory Moderately Satisfactory 4.42
08 31-Dec-2014 Moderately Satisfactory Moderately Satisfactory 5.66
09 08-Jul-2015 Moderately Satisfactory Moderately Satisfactory 9.26 Moderately 10 18-Nov-2015 Moderately Unsatisfactory 10.96 Unsatisfactory Moderately 11 30-May-2016 Moderately Unsatisfactory 14.64 Unsatisfactory Moderately 12 12-Dec-2016 Moderately Unsatisfactory 33.14 Unsatisfactory Moderately 13 30-Jun-2017 Moderately Unsatisfactory 56.26 Unsatisfactory Moderately 14 12-Feb-2018 Moderately Unsatisfactory 59.85 Unsatisfactory Moderately 15 01-Apr-2019 Unsatisfactory 66.85 Unsatisfactory
Page 3 of 71
The World Bank Nigeria - Growth & Employment (P103499)
SECTORS AND THEMES
Sectors Major Sector/Sector (%)
Public Administration 1 Sub-National Government 1
Information and Communications Technologies 31 Other Information and Communications Technologies 31
Education 3 Workforce Development and Vocational Education 3
Industry, Trade and Services 65 Public Administration - Industry, Trade and Services 23 Other Industry, Trade and Services 42
Themes Major Theme/ Theme (Level 2)/ Theme (Level 3) (%) Economic Policy 14
Trade 14
Trade Facilitation 14
Private Sector Development 76
Business Enabling Environment 56
Investment and Business Climate 29
Regulation and Competition Policy 9
Innovation and Technology Policy 18
Jobs 10
Job Creation 10
Public Private Partnerships 10
Page 4 of 71
The World Bank Nigeria - Growth & Employment (P103499)
Urban and Rural Development 20
Urban Development 10
Urban Infrastructure and Service Delivery 10
Rural Development 10
Rural Infrastructure and service delivery 10
ADM STAFF
Role At Approval At ICR
Regional Vice President: Obiageli Katryn Ezekwesili Hafez M. H. Ghanem
Country Director: Onno Ruhl Shubham Chaudhuri
Director: Marilou Jane D. Uy Elisabeth Huybens
Practice Manager: Paul Noumba Um Rashmi Shankar Kofi-Boateng Agyen, Wenye Task Team Leader(s): Ismail Radwan Dong ICR Contributing Author: Natalia Agapitova
Page 5 of 71
The World Bank Nigeria - Growth & Employment (P103499)
I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES
A. CONTEXT AT APPRAISAL
Context
1. At the time of appraisal, the Nigerian economy suffered from long-running structural weaknesses, including, but not limited to, Dutch disease, income inequality, instability, and weak governance, but between 2003 and 2008, the Nigerian economy boomed as a result of higher oil prices and a series of homegrown economic reforms. The oil sector accounted for 95 percent of export revenue and 80 percent of government revenues at the time of project appraisal. The 2008 fall in oil prices resulted in a budget shortfall and a significant depreciation of the Naira. However, through prudent management of oil windfalls, Nigeria accumulated over US$45 billion in foreign currency reserves, which provided a cushion for the country to weather the financial crisis.
2. The oil sector was dominant in the Nigerian economy but provided only 0.15 percent of employment, and diversification became paramount, with most of the workforce and the poor engaged in agriculture accounting for 60 percent of employment and 23 percent of GDP in 2005. The non-oil sector grew rapidly, averaging 10.2 percent during the country’s economic boom; however, it did not match this growth in terms of formal employment. Diversifying from the oil-dependent economy became a priority for the country, particularly because of the economic crisis and the depletion of the country’s Excess Crude Account to US$0.5 billion in 2010. This diversification was necessary also to reach Nigeria’s unmet vision of becoming a top twenty economy by 2020. The project was designed to address the above challenges and remove bottlenecks to private sector growth and competitiveness, with a focus on the non-oil economy.
3. The Growth and Employment in States (GEMS) project was designed to generate growth in the non- oil sectors and create employment. The GEMS adopted a targeted approach and conducted a sector study during project preparation to identify sectors with strong potential for generating growth and employment. According to the PAD, these industries were chosen based on economic analysis and assessments of the feasibility for successful reforms, potential “spillovers” into other industries, and the private sector’s willingness to invest. The selected sectors included: Information and Communications Technology (ICT)—infrastructure, skills and regulatory constraints; Hospitality—cumbersome visa process, limited use of ICT, weak management and customer service, legal challenges related to land acquisition and development; Entertainment—copyright piracy, weak distribution chains, low levels of technical skills, production equipment, investment and limited access to finance; Wholesale & Retail—poor storage, handling/transport and physical market infrastructure; underdeveloped processing and packaging; and constrained competition; Construction & Real Estate—lack of skilled labor, constrained land acquisition and development, poor quality / high cost of key inputs, limited access to finance; Meat & Leather—outdated equipment and technology, import and export delays.
Page 6 of 71
The World Bank Nigeria - Growth & Employment (P103499)
Theory of Change (Results Chain)
4. The Theory of Change (ToC) was not presented in the PAD but has been constructed in this ICR based on project documents at appraisal including the PAD (Figure 1). The theory of change relied on several assumptions formed during the project identification activities also described in the ToC including the pre- selection of clusters. The theory of change does not separate DFID-funded activities from the World Bank activities because the project was conceived as joint project, although the financing arrangements were separate (parallel financing). Figure 1: Theory of Change at Appraisal (reconstructed from the PAD by the ICR authors)
COMPONENTS/ACTIVITIES OUTPUTS OUTCOMES IMPACT
Improved Investment Climate in Kano,
Kaduna, Lagos, and Cross River States - Investment climate reforms implemented in Kano, (DFID funded) Kaduna, Lagos, and Cross - Investment promotion facilitation River States
- One Stop Investment Center at the federal - Increased area of land Improve investment Job creation level released for development climate to reduce in the key
- Legislation reforms for improving/ - Reduced time and cost of the cost and risk of sectors simplifying investment process registering property doing business and
- Taxpayer education and better sources of - Reduced time for tax provide a greater advice on taxation (e.g. walk-in tax advice Ease of doing payment compliance incentive to invest business: the centers) - Reduced number of tax time taken to - Establish One-Stop-Shops for tax payment payments start up and - TA to Presidential Enabling Business - Reduced time taken to obtain Environment Council approvals, licenses to operate register a company
Competitiveness of Strategic Clusters - Increased number of jobs Increased - ICT: investment in ICT infrastructure, business created by firms supported foreign and climate for ICT, ICT skills building, TA for ICT through the project in each Promote domestic networks; market facilitation services cluster competitiveness of investment - Entertainment: TA/training for private sector; 15,000 in ICT firms in ICT, TA and financial support for A2F, and to 50,000 in entertainment Entertainment, develop soft and hard infrastructure 10,000 in hospitality Hospitality, Trade, - Hospitality: TA to facilitate licensing; PPP to 5,000 in Wholesale and Retail IMPACT Construction and develop tourism assets; financial incentives to 20,000 Construction Real Estate, and innovation; TVET pilots; TA public sector 5,000 Meat and Leather Economic Meat and Leather - Wholesale & Retail Trade: TA and financing - 174 government Ministries, growth in industries to improve infrastructure and safeguards, Department and Agencies will Reduced cost of participating support innovation, strengthen public be linked to broadband food and quality states agencies, develop associations and standards networks standards - Construction and Real Estate: financing and - Number of workers trained Improved skills and TA to improve skills systems and support and certified working conditions sector reforms; improve A2F; innovation - Increased firm output (sector- for workers in all - Meat and Leather: investment facilitation; TA specific output targets for clusters and financial support to improve public each cluster) Increased ICT agencies, investment climate, food safety - New PPPs created in the penetration rates - Additional Cluster Development Activities: Hospitality cluster financial incentives (performance grants) to - New private investments innovation and private sector participation attracted
Page 7 of 71
The World Bank Nigeria - Growth & Employment (P103499)
Critical assumptions National Level A1 Global and national economic and political situation remain conducive for private sector growth A2 Policy failures can be addressed, and the private sector is able to address market failures A3 Sustainable partnership of donors and continuous alignment of country priorities A4 Project activities will be implemented in close coordination between DFID and the World Bank State/Sector Level A5 Ownership and capacity of state (Kano, Kaduna, Lagos, and Cross River) and national government to implement reforms Selected clusters (ICT, Entertainment, Hospitality, Light Manufacturing/Agri-business, Construction & Real A6 Estate, Meat & Leather) will generate opportunities for growth and employment
Project Development Objectives (PDOs)
5. The original PDO was to increase growth and employment in participating states. The GEMS Project was conceived as a broad multi-donor initiative of which IDA funded three main components: 1. Component A: The Investment Climate (initially parallel financing from DFID; later supplemented with IDA funds); 2. Component B: Strengthening industry competitiveness and job creation in selected states (IDA with parallel financing from DFID) 3. Component C: Project Implementation, M&E, and Communications (IDA with parallel financing from DFID).
Key Expected Outcomes and Outcome Indicators
6. The Project was designed to deliver several key outcomes that would jointly contribute to the achievement of the PDO: An improved investment climate that reduces the cost and risk of doing business thus providing a greater incentive to invest; Increased job creation by firms in key sectors that will lead to sustainable employment opportunities for the population driven by the private sector; Increased growth in key states driven by increased investment both foreign and domestic.
The project also aimed at effective monitoring, evaluation and dissemination of information to provide lessons, which, through communication and peer learning, help to leverage the project impacts.
7. The main outcomes were to be measured by three outcome indicators: Time required to start a business in Kano, Kaduna, Lagos, and Cross River Growth rate of selected enterprises within target clusters Number of direct beneficiaries from employment and income opportunities (minimum 33 percent women)
The fourth outcome would be measured through qualitative measures. There were also a number of intermediate indicators to capture results, which are discussed below.
Page 8 of 71
The World Bank Nigeria - Growth & Employment (P103499)
Components
8. The three project components comprised activities financed by IDA with parallel financing from DFID. Although there were separate sources of financing, the project components were intended to be implemented as a single unified project. Based on the OPCS guidance, DFID-funded activities are included in the ICR since DFID-funded components were included into the project design (and described in the PAD) and were critical for achieving the PDO.
9. Component A: Improved Investment Climate (US$5 million IDA). The parallel financing from DFID was approved in the PAD for US$75 million. The objective of this component was to improve the business environment through investment promotion facilitation while piloting reforms of land and tax administration in the four participating states of Kano, Kaduna, Lagos, and Cross River. This was to be completed through the DFID Investment Climate Program (ICP) that provided technical and financial assistance to implement the following reforms: (A.1.) Investment promotion activities involved setting up a One Stop Investment Center at the federal level, bringing together 16 agencies to reduce the time to process investor applications from months to days. This would be done by improving and simplifying procedures and providing TA to the participating states required to establish effective investment promotion systems. IDA funding supported the work of the Presidential Enabling Business Environment Council (PEBEC) which was established in July 2016. (A.2.) Improvement of tax administration, by (i) improving the capacity for Boards of Internal Revenue; (ii) establishing a better system of tax administration, (iii) reducing multiple taxation, and (iv) educating taxpayers. (A.3.) Establishment of a Flexible Facility to identify new opportunities for reform and deliver peer learning activities.
10. Component B: Increased Competitiveness of Strategic Clusters (US$140 million IDA funding). Additional US$32 million funding DFID was approved in the PAD. The second component (B) aimed to benefit six strategic value chains/clusters.1 Specific World Bank activities supporting these were adapted to each cluster. Relevant activities broadly included: (i) investing in relevant infrastructure, (ii) strengthening local institutions and the enforcement of necessary laws, (iii) market research and market promotion activities, (iv) training and TA, and (v) grant and financial assistance. This performance grants program intended to incentivize the private sector to develop new ideas, move up the value chain, and increase productivity and performance levels. To ensure that private sector players have a monetary stake in each initiative, the grants were to be restricted to matching funds up to 50 percent of the total budget. Component funding was allocated to the priority sectors as follows: ICT (US$50 million IDA funding) Entertainment (US25 million IDA funding) Hospitality (US$25 million IDA funding)
1 As part of the GEMS design process, a study was carried out to identify industries with high growth and employment potential and which provide fertile ground for successful intervention. Industries were judged against two criteria: (i) the potential to offer strong upside in terms of growth, employment, and spillovers (cost discovery and economic linkages) and (ii) the feasibility of successful intervention in terms of ability to bridge the competitiveness gap, likelihood that policy failures could be addressed, and the presence of a private sector able to address market failures. (Source; Identifying Growth Pole Value Chains for Cross River, Kaduna, Kano and Lagos states, Emerging Market Economics, May 2008.) Detailed analysis of how the cost structures of three of these of industries are affected by government and market failures has also been carried out. (Source: Product Value Chain Analysis, Consilium International, 2008 covers the construction, wholesale/retail and meat & leather industries.)
Page 9 of 71
The World Bank Nigeria - Growth & Employment (P103499)
Light Manufacturing/Agri-business (US$40 million IDA funding) Construction & Real Estate (US$21.34 million DFID funding) Meat & Leather (US$11.07 million DFID funding)
11. The remainder of project finances (US$10 million IDA funding) were to be used to support additional cluster development activities within Component B of the project. The remainder of the project finances was allocated to support additional cluster development activities within Component B of the project, like those described in the six value chains described above. This was introduced to allow the project to respond flexibly to a dynamic and developing agenda, and enable it to exploit opportunities to expand the project impact in a cost-effective way. Each cluster was to benefit from performance grants to incentivize the private sector to develop new ideas, move up the value chain and increase productivity and performance levels. The grants were to be used for productive purposes, comprising goods, works and services as specified in grant agreements in each case. Each value chain was allocated its own pool of funding for performance grants. The performance grants were to be managed within the IDA funded clusters by the Federal PIU with the assistance of grant committees for the relevant sectors and within the DFID funded clusters by the relevant designated service provider. All the payments for grants to the beneficiaries were to be made centrally from the designated account managed by the FPIU, based on accounting documentation submitted and duly verified.
12. Component C: Project Implementation, M&E, and Communications (US$20 million IDA funding). Additional US$2.37 million of parallel financing from DFID were included in the PAD. This component included: (i) project management: covering the cost of equipment, vehicles, minor civil works such as office rehabilitation and other operational and maintenance expenditures. It would also cover the costs of implementation studies and the costs of the PIU (ii) M&E: the project would finance the establishment and operation of a results-based M&E system including a maintenance and management information system and the associated training cost; (iii) Project Coordination: The PIU was to be tasked with ensuring information sharing and potential synergies are maximized, whilst reducing the risk of duplication or misaligned objectives among different actors; (iv) Environmental and Social Management: The project funded the costs of implementing a comprehensive Environmental and Social Management Framework (ESMF), Resettlement Policy Framework (RPF) and Pest Management Plan (PMP). The project was to use the M&E system to monitor the various aspects of the environmental and social management measures.
B. SIGNIFICANT CHANGES DURING IMPLEMENTATION (IF APPLICABLE)
Revised PDOs and Outcome Targets
13. The project was restructured three times during which the PDO and outcome targets were revised. With the 16-month delay in project effectiveness, the project underwent its first restructuring (Level-I) in June 2013 (before effectiveness on July 29, 2013).2 The Ministry of Finance decided to remove the reference to “states” in the PDO. The PDO was revised under the first restructuring (R1) from “To increase growth and employment in participating states” to “To increase firm growth and employment in participating firms in Nigeria.” After this, the PDO remained unchanged. The name was changed from Growth and Employment in
2 R1 is not reported in the Operations Portal as it was approved prior to project effectiveness. The ICR refers to these PDO and PDO indicator revisions, project name change, etc., as a restructuring, because while the World Bank-funded portion of the project had not yet become effective, the other project activities, funded by DFID, were already well underway. This is consistent with ICR assessing the entire project, both DFID and World Bank-financed portions, together as a single project.
Page 10 of 71
The World Bank Nigeria - Growth & Employment (P103499)
States to Growth and Employment (GEM) reflecting the removal of the reference to “states”. Additional changes under each of these restructurings are discussed under the relevant sections below.
Revised PDO Indicators
14. The PDO indicators during R1 changed to align the results framework more closely with the activities of the project and the revised PDO. Specifically, the original PDO-level indicator was to track “time required to start a business,” in each of the four participating states. This indicator was dropped as it was no longer relevant when the reference to the four states was removed from the PDO at R1. Funding support was allocated to consultants hired by the government for YouWin, a GoN initiative that provide training and at least US$60 million grants to 5,674 entrepreneurs. YouWin program was a standalone business plan competition for entrepreneurial youth and co-financed by the Ministry of Finance.
15. The remaining original PDO-level indicators would capture the “growth rate of selected enterprises within target clusters” and the “number of beneficiaries benefitting from employment and income opportunities,” also including percentage values to reflect the number of female beneficiaries. At R1, these indicators were replaced to track: (i) percentage growth in sales of participating firms; (ii) percentage increase in value added per worker of participating firms; and (iii) percentage increase in the average number of workers in participating firms. Each indicator was further broken down by (a) service sector and (b) the light manufacturing sub-sector. Given that the second revised indicator (value added per worker) remained difficult to measure and attribute to the project, it was later dropped, while (i) and (iii) were further disaggregated to also reflect progress for female-headed participating firms. The Table 1 below summarizes the evolution of PDO and PDO indicators. Additional changes were made to intermediate results indicators; the full list of revisions is provided in Annex 1.
Table 1. Evolution of PDO and PDO indicators during implementation.
Document Source General PDO Formulation Sub-objectives PDO Indicators Date Document 2/17/2011 PAD To increase growth and (1) Time required to start a business employment in participating (days) - Kano, Kaduna, Lagos & states. Cross River (2) a. Growth rate of selected enterprises within target clusters (2) b. Number of direct beneficiaries benefitting from employment and income opportunities (+ percentage of women beneficiaries) 6/4/2013 Restructuring #1 To increase firm growth and (1) Improved Investment Climate Employment growth and growth rate employment in participating (2) Increased Competitiveness of of participating firms within targeted firms in Nigeria. Strategic Clusters clusters: (3) Effective project (1) a. % growth in sales of Implementation, Monitoring and participating firms - Service Sector & Evaluation, and Communication Light Manufacturing sub-Sector (1) b. % increase in value added per workers of participating firms - Service Sector & Light Manufacturing sub-Sector (1) c. % increase in average no. of
Page 11 of 71
The World Bank Nigeria - Growth & Employment (P103499)
workers in participating firms - Service Sector & Light Manufacturing sub-Sector
6/24/2013 Financing To increase firm growth and (1) Improved Investment Climate Agreement employment in participating (2) Increased Competitiveness of firms in Nigeria. Strategic Clusters (3) Project Implementation, Communications and Monitoring and Evaluation 2/25/2014 Financing No changes Agreement Corrigendum 11/26/2018 Restructuring #2 No changes 3/11/2019 Restructuring #3 No changes
Revised Components
16. The restructurings resulted in several changes to the components to address considerable lapses in implementation. The project had been in problem status from mid-November 2015 to August 2017. The project had received numerous complaints from beneficiaries. One of the key issues was a delay in payments to beneficiaries who had been competitively selected and awarded grants. There were also communication gaps with all project stakeholders that were compounded by complex administrative systems. The introduction of a new payments approval processes which was initiated and operationalized by the client, and not objected to by the Bank caused disbursement delays and beneficiary dissatisfaction.
17. After the Mid-Term Review (MTR) in June 2016, the Ministry of Finance requested a restructuring to implement the MTR recommendations. The after a political transition and included structural changes made to the project components, costs, and results framework. This draft restructuring was further revised following a request of the Minister of Trade and Investment to add a finance sub-component. The management agreed to add this sub-component in the on-going restructuring. This resulted in the August 2017 RVP- approved Level II restructuring. During this restructuring, US$5 million in IDA funds was reallocated from Component B to A—previously fully funded by DFID. This funding was to support the work of the Presidential Enabling Business Environment Council (PEBEC)—established in 2016. This also included an additional one- year extension to September 30, 2019. However, given that the Borrower (Ministry of Finance) did not counter-sign the Amended Financing Agreement (AFA), this restructuring was not official. The Borrower indicated that it was unable to counter-sign the AFA due to political economy challenges related to the new access to finance sub-component introduced at restructuring. The access to finance was originally intended to be a US$30 million equity fund that would provide quasi-equity (mezzanine) financing to smaller enterprises in Nigeria with high growth potential, with investment sizes between US$250,000 and US$2 million.
18. Through high-level discussions with the Ministry of Finance in October 2018, the World Bank agreed to process a restructuring (R2). This R2 included an extension of the closing date to March 29, 2019 to compensate for delays in the project implementation due to the political factors, to ensure that the recommendations made in the MTR were implemented, and to achieve the PDO.
19. The final restructuring (R3) included the Ministry of Finance’s ’s request for a cancellation of funds from undisbursed credit proceeds. Based on the R3, the unused IDA funds are returned to Nigeria’s IDA
Page 12 of 71
The World Bank Nigeria - Growth & Employment (P103499)
envelope. Table 2 summarizes the component changes. The changes were dated as the project closing dates because although the team received the government request for cancellation on February 28, 2019, there was insufficient funds in the GEM WB Account to process the partial cancellation. An additional request from the client to first transfer US$13 million from GEM project designated account to the World Bank account was necessary in order to process the cancellation.
Table 2. Summary of project restructurings by component Component A Component B Component C An improved investment Increased competitiveness of Effective project climate through: strategic clusters through six implementation, M&E, and 1. Land development value chain interventions: communication 2. Improvements in tax administration 1. ICT (US$50 m. IDA)) 3. Investment promotion & 2. Entertainment (US$25 m. IDA facilitation 3. Hospitality (US$25 m. IDA) 4. Flexible facility including 4. Wholesale & Retail Trade peer learning (US$30 m. IDA) At Board 5. Construction & Real Estate Approval (US$21 m. DFID) (2011) 6. Meat & Leather (US$11 m. DFID) 7. Additional Cluster Activities (US$10 m. IDA) Total Funding: US$22 million Total Funding: US$75 million Total Funding: US$172 million (US$20 m. IDA-funded, US$2.37 (fully DFID funded) (US$140 m. IDA-funded, US$32 m. DFID-funded) m. DFID-funded)
General summary of changes at Restructuring #1: . PDO changed from “To increase growth and employment in participating states” to “To increase firm growth and employment in participating firms in Nigeria.” . US$2 million reallocated from Component C to Component B . Structural changes made to Component B . Funding support allocated to consultants hired for YouWin . Modifications made to the Results Framework . Extension of the closing date from 12/30/2016 to 9/7/2018 A B C Increased competitiveness of - strategic clusters through: - R.1 Sub-Component B.I: Services (2013) (US$125 million), 1. ICT (US$58 m. IDA) 2. Entertainment (US$20 m. IDA) 3. Other service clusters—incl. hospitality and construction (US$17 m. IDA) 4. Wholesale & Retail (US$30 m. DFID) Sub-Component B.II: Light Manufacturing (US$47 million)
Page 13 of 71
The World Bank Nigeria - Growth & Employment (P103499)
1. Agro-processing (US$17 m. IDA) 2. Other Manufacturing (US$30 m. IDA) Total Funding: US$20 million Total Funding: US$172 million (US$18 m. IDA-funded, US$2.37 (US$142 m. IDA-funded, US$30 m. m. DFID-funded) DFID-funded) General summary of changes at Restructuring #2: . Reallocation of funds allocated to each component, with US$3 million and US$2 million reallocated to Component A from Components B and C respectively . Structural changes made to Component B . Reduction in the number of project activities listed in the financing plan from 41 to 18 . Changes made to the Result Framework, with one PDO-level indicator dropped and targets revised for all others . Extension of closing date from 9/7/2018 to 9/30/2019 A B C Increased competitiveness of - strategic clusters through: - Proposed - Sub-Component B.I: Services restructuring (US$77 million) – Changes to (2017) Results Framework only - Sub-Component B.II: Light Manufacturing (US$ 30 million) – Changes to Results Framework only - Sub-Component B.III: SME Investment Fund (US$30 million) New project sub-component Total Funding: US$80 million added (US$75 m. DFID-funded, Total Funding: US$172 million US$5 m. IDA-funded) (US$137 m. IDA-funded, US$30 m. DFID-funded) General summary of changes at Restructuring #3: R.2 . Change of closing date to 3/29/2019 (2018) . No structural changes made to project components
General summary of changes at Restructuring #4: . Partial cancellation of undisbursed funds R.3 . The Borrower has decided not to go ahead with an SME Investment Fund proposed during (2019) the level II restructuring approved in August 2017, necessitating a revised restructuring . No structural changes made to project components
Page 14 of 71
The World Bank Nigeria - Growth & Employment (P103499)
Other Changes
20. The RF was revised twice in 2013 and 2017. The original RF included indicators relevant to both DFID and IDA-funded work, and they implied synchronized implementation. However, effectiveness and implementation delays caused a time lag between DFID and IDA-funded activities of more than three years. On the IDA side, intermediate-level indicators were added to better measure results under Component B.
21. The indicators capturing outcomes of DFID-funded activities were dropped from the RF during R1 because they were captured through the DFID reporting system. Targets for Component A were already achieved by the time the IDA-funded activities started implementation. They were tracked through indicators: (i) area of land released for development by the state, (ii) days required to register property, (iii) cost of registering property as percentage of property value, (iv) time for tax payment compliance, (v) number of tax payments, and (vi) time taken to obtain all approvals and licenses to operate. These were measured for each participating state when and where it was relevant to do so. While some of the indicators listed above were not tracked or reported in the DFID project completion report (PCR), this component was assessed to be achieved.
22. In 2017, the changes to RF and design reflected the recommendations of the MTR. The main changes were: (i) revisions to make indicators more realistic, measurable, and in line with project activities, (ii) dropping indicators that were no longer relevant as a result of changes in the objectives of the project, which included all the indicators for the initially targeted states, (iii) revised targets to adjust for the PDO changes and delays in implementation, and (iv) the addition of intermediate-level indicators to reflect project achievements more accurately and include new indicators relevant to the new sub-component B.III, the SME investment fund.
23. The restructurings extended the project closing date several times. At approval, the project was expected to close on December 31, 2016. The closing date was first extended to September 7, 2018 during R1 to take into account the initial 16-month delay in project signing and effectiveness. R2 changed the closing date to March 29, 2019. The extensions were requested to maximize the chances of achieving the PDO and to ensure that enough time would be allocated to clear any pending liabilities on the World Bank and on the client side and mitigate any potential reputational risks.
24. A partial cancellation of undisbursed IDA funds was requested in R3. This request superseded the 2017 RVP-approved Level 2 restructuring. A total amount of SDR 55,106,140.61 (approximately US$76 million) was requested to be cancelled by the client.
Rationale for Changes and Their Implication on the Original Theory of Change
25. The R1 (Level I) was proposed in response to the government’s request. Specifically, the Federal Ministry of Finance wanted to: (i) ensure that the proceeds and growth opportunities could apply to all parts of Nigeria that have the potential to develop jobs and employment for the poor—instead of focusing only on specific states; (ii) ensure that the project is aligned with recent government policies and priorities and (iii) ensure that the proceeds of the credit were primarily used for infrastructure and targeted beneficiaries while minimizing the amount for project administration. This was to be achieved by changing the project’s focus from participating states to participating firms. This change was reflected in revisions to the project Result Framework. In addition, indicators relevant to DFID project components were dropped given the time lag
Page 15 of 71
The World Bank Nigeria - Growth & Employment (P103499)
between operations and that DFID already advanced the implementation in the targeted states, as planned during the project design. R1 was formally approved and led to project Effectiveness in September 2013.
26. The attempt of the second restructuring (July 2017) was proposed to implement recommendations from the project MTR mission conducted in June 2016. At that time, only 39 percent of IDA project funds had been disbursed after four years of implementation (DFID disbursement will be discussed later). The intent was to address the causes of the implementation delays, which put the project in problem status since November 2015. The adjustment of the PDO-level indicators and targets intended to make them more realistic, which in turn would help to reflect project achievements more accurately. In addition, the number of project activities was reduced to simplify the design that was deemed overly complex, and to eliminate activities considered to be redundant and/or no longer relevant to the objectives. A pilot SME Investment Fund instrument was added to the project by government request to address industry- and firm-level constraints to growth. Finally, because DFID financing support was to phase out in 2017, US$3 million and US$2 million were respectively reallocated from Components B and C to Component A to support government efforts to improve the investment climate in Nigeria. This was requested to support the Doing Business Secretariat. The AFA was not signed due to political economy challenges surrounding the project and objections to the introductions of the SME Investment Fund from parts of the government.
27. The actual second restructuring (R2, November 2018) and third restructuring (R3, March 2019) were proposed to allow for an orderly winding down of the project. The proposal for R2 was submitted through a waiver in November 2018 and approved in December 2018 (problem projects require extension waivers from RVP and OPCS VP). This sought a short extension to the end of March 2019 to ensure time was allocated to clear any pending liabilities and avoid a reputational risk. This would allow final payments to be made by the government to grant recipients and service providers. The request for R2 and R3 repeated changes proposed in 2017 but also included the request for an extension of the closing date and for the partial cancellation of undisbursed IDA funds. The late final restructuring signals that satisfactory results vis- à-vis the PDO were expected, but the complex procurement processes underway that were not completed on time.
II. OUTCOME
A. RELEVANCE OF PDOs
Assessment of Relevance of PDOs and Rating
Original PDO: To increase growth and employment in participating states. Revised PDO: To increase firm growth and employment in participating firms in Nigeria.
Rating: Substantial
28. Though general in nature, the original PDO was substantially relevant to the government’s priorities and World Bank strategy. Growth and employment were at the heart of the NEEDS and SEEDS strategies. Nigeria’s 7-point policy agenda, the country’s poverty reduction strategy, explicitly recognized the importance of employment, diversification, and wealth creation. Nigeria’s 2020 strategy included an ambitious goal of taking
Page 16 of 71
The World Bank Nigeria - Growth & Employment (P103499)
Nigeria into the ranks of the world’s top 20 economies by 2020. The CPS, presented to the Board in July 2009, aimed to improve the enabling environment for non-oil growth. It aimed to consolidate the country’s recent, rapid growth, embed it in a more diversified economy, and broaden its distributional impact. This project directly supported the second CPS pillar, growing the private sector and focusing on non-oil growth.
29. The revision of the PDO helped maintain relevance to the World Bank’s current CPF framework and the SCD. Based on the review of priorities from the four originally targeted states, growth and employment are key development pillars for the CPF nationwide. The PDO was relevant to the 2014–2017 CPF in particular the first strategic clusters, promoting diversified growth and job creation. The GEM project supported the final focus area of the first strategic cluster, increasing financing for SMEs to create more jobs across industries beyond oil. The 2019 SCD calls for four pathways for Nigeria to its development goals. The third pathway is “promoting a private sector–led growth,” which aligns with the project’s objectives and design by providing training, consulting, and financing across multiple industries to increase productivity and employment. All stakeholders interviewed concurred the PDO, though rather general, remains relevant.
B. ACHIEVEMENT OF PDOs (EFFICACY)
Assessment of Achievement of Each Objective/Outcome
30. The ICR assesses achievement of the PDO according to a split rating because of the substantive change to the PDO. The split rating date is June 2013, which marks the revision of the PDO. At this point, £34.23 million—equivalent to US$53.79 million—had been disbursed of the actual DFID-funded portion (of a total equivalent US$126.58 million), while only US$1.06 million of the IDA funded component had been disbursed. Thus, virtually all of the activity leading up to the June 2013 break point was DFID-financed. With a total combined (DFID plus IDA) disbursement by the end of the project of US$193.45 million (US$126.58 million DFID and US$66.85 million IDA), the disbursement weight for the first period is 28.30 percent, while the weight for the second period is 71.70 percent. DFID disbursements by components is detailed in the Table 3.
Table 3. Parallel Financing from DFID: Planned and Actuals PLANNED ACTUAL COMPONENT A 75,000,000 72,725,844 COMPONENT B 32,000,000 48,181,708 COMPONENT C 2,370,000 5,674,466 TOTAL 109,370,000 126,582,017 BEFORE JUNE 2013 53,688,566 AFTER JUNE 2013 72,893,450
Period 1: March 2011–June 2013 - Substantial Achievement
31. The DFID-funded sector portfolio included sector-level activities in the Component B and markets systems activities under Component C.3 Sectors funded by DFID included: Meat & Leather, supported through GEMS 1 (phase 1) from 2010–2015—with Meat and Leather treated as individual sectors of intervention; Construction & Real Estate through GEMS 2 (phase 2) from 2010–2013; and Wholesale & Retail
3 The OPCS also ruled that the ICR will recognize the contribution of DFID in achieving the targeted outcomes by explaining the linkage with the achievement to intermediate results under each project component.
Page 17 of 71
The World Bank Nigeria - Growth & Employment (P103499)
through GEMS 4 (phase 4), which began in August 2012. Activities aimed to strengthen the performance of market systems through its “Making Markets Work for the Poor (M4P) Approach for these sectors to perform more effectively and offer improve opportunities through increased performance and income.
32. DFID-funded activities under GEMS 1 achieved systemic change, and income-related targets were exceeded in the Meat and Leather industries. The project implemented a fee-based services system to provide farmers with access to services and improved inputs. GEMS activities also supported skills development and capacity building for project staff and outside service providers who are now able to work for other stakeholders. At the impact level, indicators relate to income and employment. The income target was to reach 120,000 people (50,000 poor/2,090 women) with achievements of 237,200 people (124,650 poor/19,490 women). Total income was targeted at ₤24.7 million, with a total of ₤42 million achieved (170 percent achievement of target). The employment target was 4,400 full-time equivalents, with 4,076 achieved by the end of June 2015 (approximately 92 percent of the target). This data is based on the GEMs closing review commissioned by DFID to a third-party evaluator. There was no impact evaluation conducted, and impact data was collected based on self-assessment, beneficiary surveys, focus groups and government data.
33. DFID-funded activities under GEMS 2 were less successful in achieving its objective of strengthening the performance of market systems in construction and real estate. As explained in the DFID PCR, the team decided to close the project early (2013) based on their assessment that the project was unlikely to meet its objectives within its remaining lifetime. As a result of the DFID cancellation, the World Bank GEM project took on the funding of the construction sector activities because the sector was important for job creation and linked to housing, where the World Bank had decided to intervene (achievements discussed below).
34. Passing the tax harmonization bill and developing a proven toolkit to guide internal revenue generation through taxation helped improve the investment climate to improve growth in participating states. A key achievement under this subcomponent was to secure the adoption of these reforms at the regulatory level. Activities also targeted sensitization of officials and taxpayers, training of tax officials, public-private dialogue, and payment systems. These make up the structure used by the team to develop a systematic tax reform program. These achievements lay the groundwork for growth and increased employment and help lower the cost of doing business by streamlining processes for businesses to fulfill their taxpaying obligations and therefore promote business and employment growth: Tax-for-service (T4S) arrangements at the state level (e.g., Kano) have been put in place through a replicable state-level model to support traders in paying their taxes. Point of sale (POS) systems of collection are also in place to remove the potential for “leakage” from the system, providing confidence to taxpayers and increasing the flow of revenues at the state level. At the national level, the project worked with the Joint Tax Board (JTB) and Nigerian Governors Forum (NGF) to provide a good basis for replication in other states. To achieve this, GEMS 3 partnered with 257 agencies from the private and public sector as well as other partners at the state and federal level (DFID GEMS 3 indicator, Target = 270—Achieved).
35. Completion of a Systematic Land Title Registration (SLTR) process at the local government level in the states of Kano and Kaduna was a prerequisite for business development in those states. This reform benefits residents by providing them with title to their property, thus providing security. In Kano, beneficiaries of land titling reported an increase in value of their property of approximately 40 percent as a result of their land holding. This reform also benefits local authorities for spatial planning purposes while deepening the fiscal base. This provides a successful proof of concept for such reforms across Nigeria and
Page 18 of 71
The World Bank Nigeria - Growth & Employment (P103499)
beyond. In the state of Jigawa, support to rural land reform was also provided to develop mechanisms to support responsible agriculture investments, however, although good progress has been achieved further work is needed to demonstrate the replicability of these reforms at the rural level. With improved land titling comes improved investment in the productivity of the land and, therefore, economic growth.
36. Support to the establishment of Investment Promotion Agencies in the states of Kaduna, Kano, Lagos, and Jigawa was successful in improving credit to businesses and thus supporting growth of the private sector to grow and create employment. A significant amount of DFID funding was also used for capacity building purposes within the investment promotion agencies, both at the federal and state levels. In addition, a Collateral Registry was established at the Central Bank by the federal investment team— improving access to finance for entrepreneurs and firms. This Registry has seen a large pickup in registration, with assets worth US$102 billion registered as of 2017. In Kaduna—where investments had mobilized a total value of US$350 million by 2017—companies that worked with the local investment promotion agencies reported positive feedback.
37. The number of beneficiaries with improved access to related services reached over 3.9 million individuals in May 2017 against a target of 4.2 million for July 2017. The sustainability—measured through the percentage of new or improved services sustained in the market a year after project end—was estimated at 90 percent (Target of 90 percent). As a result, 1,338,881 people recorded a positive change in income against a target of 1,412,444. In terms of revenue, Kano state, for example, reported a related increase in revenue from N11 billion in 2011 to N30.9 billion in 2016. Tax compliance has increased (at a lower cost) with internally generated revenue rising to £990 million—equivalent to US$1.37 million4—in Kaduna, Kano, and Jigawa. These impacts are expected to continue benefitting participating states.
Period 2: June 2013—March 2019 – Modest Achievement
38. Period two marked the shift from the state-level support to firm-level support to Micro, Small and Medium Enterprises (MSMEs), to which both DFID-funded and IDA-funded activities contribute. The outcomes are assessed for both sources of funding, based on the data from their reporting systems.
39. DFID-funded Whole and Retail Market Systems (WRMS) under GEMS 4 supported a wide range of sectors and reached the end of implementation with strong performance, achieving 15 of 16 indicators targeted. Some areas of intervention include mobile money, skin salting, and micro retailing, and rice and tomato agro-service provision. The related activities covered 28 states. Impact level targets were exceeded by 27.3 percent for the number of jobs (Actual = 12,737 vs. Target = 10,003) and by 5.6 percent for the absolute number of people recording a positive change in income (Actual = 528,210 vs. Target = 500,023). However, targets for the aggregate change in income only reached 55 percent (Actual = £97,258,575 vs Target = £174,419,657).
40. Additional evidence for achievement of objectives was provided in the DFID self-assessment, which Table 2 partly summarizes. These data indicate that 2017 targets were met for three key indicators for interventions related to DFID financed activities (mainly Component A). Despite these results, the weak performance in period 2, mainly of Component B, dictates a modest efficacy rating. Annex 7 summarizes the results reported by DFID in the self-assessment.
Page 19 of 71
The World Bank Nigeria - Growth & Employment (P103499)
Table 2. Summary of Results from DFID-Financed Activities Impact Indicator Actual (2015) Actual (2016) Actual (2017) Target (2017) 1. Number of people recording 929,958 1,230,063 2,104,291 1,987,832 positive change in incomes 2. Aggregated change in 155,945,700 325,646,162 703,639,765 703,230,033 cumulative income (GBP) 3. Change in Employment (FTE 12,223 17,544 32,432 26,318 Jobs)
IDA Funding to Address Firm- and Industry-level Constraints
41. IDA-funded activities broadly cut across sectors of interest, instead of being focused and targeted at the individual sector level. While GEMS3 operated by DFID tackled cross-cutting investment climate issues, IDA funding focused at: (a) the firm-level constraints, by identifying high-potential entrepreneurs, modernizing their business practices and facilitating access to finance, and (b) industry-level constraints, by increasing labor productivity for the industry and by identifying key sectoral investment climate bottlenecks that could unleash the potential of the targeted clusters (Tourism, Hospitality, Entertainment, ICT & Light Manufacturing sectors). World Bank work supporting these sectors was fully operational in 2014.
42. Concrete project outputs under this pillar included the launch of the Business Innovation and Growth (BIG) Portal and its operationalization in January 2017 to provide capacity development services to firms and assure a transparent allocation and monitoring of subsidies (grants). The web- based platform was used to track and communicate all firm-level activities, serving as a dashboard for entrepreneurs to keep up with their capacity building and their progress on accessing a grant (tracked through a point-based system, which determined the ability of beneficiaries to apply for a grant). A total of 109,000 SMEs were registered on the platform; 5,474 received access to services, including online training, BDS, and insourcing/outsourcing. For beneficiaries of training, it was delivered through a blended approach of online and in-class training.
43. This platform was designed based on the lessons of the YouWin program that was developed using GEM technical assistance. An online platform with different access rights contributed to improved transparency of beneficiaries’ selection (based on a point system following attendance and results to the activity), provided a high level of control at different steps of the process (M&E team, beneficiaries to verify information provided by services providers, service providers to report progress, etc.) and enhanced cost effectiveness of the implementation.
44. Twenty-seven consulting firms were also trained to provide BDS to SMEs, thus contributing to firm performance, which led to their growth and job creation. An expression of interest was issued to work with BDS: 113 BDS responded. These BDS providers were screened based on their qualifications and experience. Sixty-two BDS providers were invited to a seven-day training program, and their qualifications were assessed based on a paper-based test and an observation of a consulting service provided to a MSME. The objective was to place unemployed accountants and/or marketers in SMEs that need capacity strengthening in accountancy or marketing. The GEM project was providing a grant to the SME to pay for Human Resource services and the salary of a worker for nine months. This was only piloted in Lagos and Abuja. SMEs were working with 16 selected human resources firms to determine whether they need an accountant or a marketer and to find their future employee.
Page 20 of 71
The World Bank Nigeria - Growth & Employment (P103499)
45. Outsourcing was another option for building firm-level competencies. The objective was to strengthen the capacity of SMEs by linking them with good-quality accountancy (19 firms) or marketing firms (17 firms) that will deliver a specialized service over a nine-month period. This was piloted in Lagos and Abuja. SMEs selected their accountancy or marketing firms among a pool of selected firms. The difference with the BDS activity was that outsourcing would be a consultant working in-house for a couple of hours/days a week and doing things for the entrepreneur. BDS activity was about guiding the entrepreneur in how to improve his business in four areas (facilities management, marketing, human resources, and operations).
46. A large impact evaluation was embedded in the project to measure impact from various capacity building approaches on firm’s business practices. Evidence from the Impact Evaluation results shows that the insourcing, outsourcing and consulting interventions all improved business practices in participating firms while training had a negligible impact. See details in Annex 8.
47. The ICR conducted beneficiary surveys at completion to partially compensate for weak results measurement during the project and these surveys reflect generally positive feedback on growth and job creation, though they do not measure growth and employment effects specifically. During the ICR phase, firm-level data was collected using a survey and virtual focus groups. Firms reported increases in the numbers of workers hired and growth of sales. This indicates that the project contributed toward the (revised) PDO to increase firm growth and employment in participating firms in Nigeria, but it is not clear whether the improvements were significant. Feedback was mixed on the quality and helpfulness of different capacity building services provided to the firms. The results in Table 3 reflect the views of 60 respondents. Generally, in-person training was considered most helpful, closely followed by services provided from in- house consultants (outsourcing).
Table 3. Perceptions of Beneficiaries, ICR Survey Respondents
Survey Question Yes (percent) No (percent) Did you hire more workers as a result of the GEM project? (n = 60) 83.3 16.7 Did your sales increase as a result of the GEM project? (n=59) 88.1 11.9
Survey Question Extremely A little Not at all (percent) percent) (percent) Did you find in-person training to be helpful to your business? (n=55) 70.9 20 9.1
Did you find the in-house consultant to be helpful to your business? 69.2 25 5.8 (n=52) Did you find the accountant you worked with to be helpful to your 56.6 21.7 21.7 business (n=46)
Grants Disbursement
48. The government ICR reports that a total of 1,717 firms received funding through the program, which is a 50 percent achievement of the targeted 2,500 firms. This is a result of a combination of design and implementation issues. Specifically, (i) the program’s design was inherently complex to prevent misallocation and misuse of funds, and (ii) procurement issues delayed the renewal of contracts for two main implementation
Page 21 of 71
The World Bank Nigeria - Growth & Employment (P103499)
partners (grant management and monitoring firm). Additionally, the second tranche of grants was also delayed because the grant manual required that the monitoring firm check the quality of implementation of the first tranche before disbursements of the second tranche. But the contract for this monitoring firm was also delayed because of procurement delays on the PIU and the World Bank side.
49. Concerns around the public’s attention and negative media coverage were due to payment delays and lack of communication. The GRS received and handled complaints through 20 cases. The World Bank conducted the investigation process to properly respond to these complaints according to its GRS procedures. The corruption complaints were misleading and unfounded. However, they significantly affected the implementation of related activities. As detailed in the last implementation support report, this was a direct result of communication gaps and the introduction of administrative steps worsening the initial delays. The last case was resolved in January 2020. Broadly, complaints raised were related to: economic losses due to delayed disbursement of the grants; lack of information about the program including grant awards, disbursement dates; and wrong evaluation of business leading to disqualification from the grant. The last case was resolved in January 2020 50. The project closing date was extended twice to ensure that grants allocated were all fully disbursed. All firms expecting to receive disbursements had received or were in line to receive the funds allocated by project closing. It should also be noted that some firms initially selected were disqualified as they did not meet the criteria for payment. During the virtual focus group discussions held with project beneficiaries, only issues of timeliness and poor communication were highlighted by participants. Those that mentioned grant disbursements reported frustrations relating to the complexity of requirements for entrepreneurs to withdraw the funds received, in particular related to one of the bank chains. These were put in place by the PIU and implemented in coordination with participating banks.
Industry-level support
51. As part of the Cluster sub-component activities, market analyses were conducted to identify market failures that have impeded the growth of SMEs across the 5 GEM intervention Sectors (ICT, Entertainment, Hospitality & Tourism, Construction & Light Manufacturing). The Project supported the Entertainment Industry by funding (2) online music distribution companies with a view to reduce piracy and improving the income of content developers in the music industry. Unfortunately, by ISR 14 (January 2014) it was clear that the project administration did not have the ability to develop substantive work streams in the different industry clusters. Clusters were selected and a consulting firm brought on board. Calls for proposals for ICT innovation hubs and supporting the Construction, Leather, and Light Manufacturing industries were released but no concrete project results materialized. The Project Preparation Facility was established and extended to October 2010 with the original intention of (Article II – Execution of Activities, Description of Activities): 1. Carrying out of studies, training and workshops on industrial policy, cluster development export processing zones, Private Participation in Infrastructure (PPI) and Cluster Growth. 2. Development and finalization of an Environmental and Social Management Framework (ESMF) for Lagos, Kano, Kaduna and Cross River. 3. Study tours for government counterparts on cluster and value chain development covering the major value chains under consideration for the proposed Project. 4. Proposed partnership with the Harvard University growth team. 5. Preparation of GEMS Implementation and Procurement Plans (IPPs).
Page 22 of 71
The World Bank Nigeria - Growth & Employment (P103499)
6. Training for project officials in procurement (for consultants, works and goods), financial management and disbursement as well as monitoring and evaluation and strategic planning. 7. Purchasing of office equipment as well as operational expenses for the relevant implementation units to manage and implement the proposed Project.
52. The project helped improve availability of skilled labor in key industries and promote employment of vulnerable populations including youths and women. Four hundred (400) artisans were trained in video and sound editing. To address skills deficiency in the construction industry, six hundred (600) youths were trained to promote quality of service and affordable housing development in Kaduna State. Percent increase in average number of male and female full-time employment created by participating MSMEs are over-achieved by 14% and 30%, respectively.
53. The project also helped address industry-specific policy issues. GEM solely sponsored the production of National Housing Standards that has been validated and adopted by various stakeholders in the construction value chain in Nigeria, in addition to support for the development of a robust Nigeria Housing Census to create a database that will provide reliable housing information to investors, policy makers, and end users. In collaboration with other stakeholders, GEM also led the development of the first Nigerian Leather Policy. The policy has been validated by stakeholders and approved by the Federal Executive Council.
Justification of Overall Efficacy Rating
54. Based on the evidence for efficacy presented above, the rating for the first period is substantial, which is consistent with the completion report that was prepared by DFID for the GEMS series. For this first period, the achievement of the original PDO, “to increase growth and employment in participating States,” is assessed. Reforms were achieved under DFID GEMS 3 and are now demanded in more states than the DFID program’s original four focal states. Note that the available results data for this component is from 2015 or later. Thus, while the ICR is measuring the achievement of the original PDO related to Component A (DFID-financed) activities, it relies on the only available data that are from 2015 and later.
55. The achievement of the revised PDO for the second period is modest. This rating is for achievement of the revised PDO, “to increase firm growth and employment in participating firms in Nigeria,” and covers project activities that took place during the second period (June 2013–March 2019). Based on the data from the last ISR (ISR35867), the project has achieved, and exceeded, all its PDO indicators (Summary of the outcome indicators in Table 4 below, with detailed results data in Annex 1). However, most of these results were achieved too close to the end of the project and increases in sales were probably fueled by the injections of funds to the firms through the grant program. It is therefore not possible to judge the sustainability of these achievements. Additionally, the project faced systemic implementation challenges and low disbursements. Therefore, the efficacy rating is modest.
Table 4. Summary of Outcomes from IDA-Financed Activities Impact Indicator Target Actual (2019) Actual vs Target % growth in sales of participating firms 20 141 700% % increase in average number of workers in 20 38 190% participating firms
Page 23 of 71
The World Bank Nigeria - Growth & Employment (P103499)
C. EFFICIENCY
Assessment of Efficiency and Rating 56. The economic analysis at ICR shows negative net economic benefits for the first two years and positive net economic benefits in the last two years. Due to the halting and extended nature of the project, it was difficult to conduct a proper NPV or ERR calculation. The PAD efficiency analysis provided estimates for the project’s contribution for GDP growth in the originally selected states. Given that many original activities were dropped, an update of the original analysis is not relevant. Nor is data available to conduct a partial analysis. Instead, the economic analysis of assumes that all disbursements two place in 2016-2017 and all economic benefits took place between 2016-2019. The model assumed that each MSME hired two additional people (based on the survey and focus groups) and each employee was paid the annual Lagos civil servant minimum wage. Moreover, each MSME was assumed to have increased its revenue by 20% (another common response from the focus group survey) from a base revenue assumed to be equal to the official Nigerian cut off to qualify as a micro enterprise (5 million Naira). The economic costs are incurred from the annual project disbursements which of the sake of simplicity were equally divided between 2016 and 2017. The terminal value of the cash flows was calculated using the net economic benefits from jobs and additional revenue and a 14.5% discount rate.
57. In addition to the economic analysis, qualitative information on the cost-efficiency of implementation supports a modest rating. Benefits that are expected to accrue to the project from the implemented sub-components of the project are discussed below.
58. According to the DFID project post-completion report, the efficiency of the DFID components (GEMS 3) represented good value for money. This was measured through two main indicators: (i) cost per beneficiary reached (with 15 percent additional income) and (ii) rate of return (total additional income compared with project cost). For GEMS 3—relating to investment climate work— the results achieved were £34.95 for the cost per beneficiary and 12.4 percent for the rate of return. As discussed, in the report, “the planning of intervention activities by the state and workstream reflected progress made and the scope of engagement, reflecting ‘buy in’ and local political economy considerations”. However, at approval, DFID also aimed to create 100,000 jobs at a cost per job of US$2,700 while the results from GEMS 3 reported a cost £4,069 per job as of May 2014. These values are also provided for the GEMS 1 and GEMS 4 components, with cost per beneficiary estimated at £36.23 and £28.00 respectively. This demonstrates clear improvements in efficiency and economies of scale. On the other hand, rates of return were lower for both GEMS 1 and GEMS 4 with 4.9 and 6.6. All interventions demonstrated high budget utilization, with final expenditure ranging from 0.1 to 3.5 percent away from expenditure forecast.
59. Relating to the capacity building component, BDS, insourcing and outsourcing services demonstrate evidence of cost-effectiveness—measured through participating firms’ increase in profits. The preliminary results reported by the GEM Impact Evaluation did not analyze efficiency based on actual values for the amount spent and realized costs of the interventions. However, based on the design estimate that the training, insourcing and outsourcing capacity building would cost US$2,000 per person and BDS consulting US$4,000 per person, the gains in monthly profits were estimated at US$100 for outsourcing, at US$265 from BDS outside of Lagos and Abuja. However, as noted in the preliminary report, “the insourcing and outsourcing deliver the same, or greater, improvement in business practices than BDS at half the cost, and more innovation, suggesting they perform better on a straight cost-benefit comparison basis.” There is thus some evidence for the outsourcing and BDS being cost-effective. On the other hand, training services did not appear to be cost effective.
Page 24 of 71
The World Bank Nigeria - Growth & Employment (P103499)
Despite the availability of capacity building results, no data allows for a cost-efficiency analysis of the grants component to be conducted.
60. Project extensions and cancellation of funds also contribute to a modest rating for efficiency. The combination of the need for project closing date extensions and the low disbursement rate contributed to poor project efficiency. The original project date was extended two years and three months, while final disbursements were only 41.8 percent of total project financing.
Assessment of Efficiency and Rating – Modest
D. JUSTIFICATION OF OVERALL OUTCOME RATING
61. While the project remained substantially relevant for Nigeria, both efficacy standpoint and efficiency are rated modest, primarily because a substantial portion of the project activities were not completed. There was a disconnect between DFID and IDA-funded activities. There is evidence of substantial return on the investments from DFID-funded activities (see Annex 4). IDA-funded activities produced some benefits but the sustainability of progress toward the goals of increased growth and employment is questionable. Efficiency was also modest, mainly because of delays and only partial implementation. With the split rating based on percentage of final disbursements, period 1’s satisfactory rating has a 28.30 percent weight and period 2’s unsatisfactory rating has an 71.70 percent weight (Table 4). The overall rating is, therefore, Moderately Unsatisfactory.
Table 4. Ratings for PDO and Overall Original PDO Revised PDO Overall Rating 1. Rating Satisfactory Unsatisfactory Moderately Unsatisfactory 2. Rating Value 5 2 3 3. Weight (percent disbursed: including 28.30 percent 71.70 percent 100 percent DFID-funded and IDA funded) 4. Weighted Value 1.415 1.434 2.849
E. OTHER OUTCOMES AND IMPACTS (IF ANY)
Gender
62. Although the project did not target female entrepreneurs specifically and did not record a baseline number for female-headed participating firms, it had substantial participation from female business owners. According to the SCD, over 50 percent of all Nigerian firms are woman-owned, but these firms largely involve self-employment. The project increased support to growth of women-owned SMEs, in a context where monthly profits of woman-owned SMEs are 52 percent lower than those of their male counterparts. Men are twice as likely as women to have wage employment when working: only about 4 percent of women receive wages for their work versus 8 percent of men, and women earn about 76 percent of men’s earnings for similar work. Based on 102 survey responses to the optional gender question posed by the ICR team, 65.7 percent of GEM beneficiaries identified the principal applicant as male and 34.3 percent identified the principal applicant
Page 25 of 71
The World Bank Nigeria - Growth & Employment (P103499)
as female.
Institutional Strengthening
63. The project contributed to institutional strengthening of the government at the state level. DFID- funded activities contributed to building the capacity of government agencies involved in the formulation and implementation of building regulation through training, TA, and knowledge exchange. At the national level, GEM provided TA (by financing consultants) to the presidential business environment commission, which led to reforms that improved Nigeria’s performance on the Doing Business Indicators. 64. The project also increased the capacity of private sector actors to provide BDS to SMEs and build capacity at the sectoral level. The project enhanced the availability and quality of BDS services through training and TA to private sector providers of BDS. The project also stimulated collaboration between SMEs and BDS providers by providing incentives for BDS support to firms (grants).
Mobilizing Private Sector Financing
65. The project used several MFD principles in its design by stimulating private sector solutions to SME needs. The GEM project provided matching grants, which required enterprises to come up with their own funding via bank loans or equity partner thus mobilizing private sector financing. Ventures Platform, an incubator and GEM partner, provided additional matching funding to carry out the program.
III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME
A. KEY FACTORS DURING PREPARATION
Lack of Buy-in from Key Stakeholders
63. The lack of buy-in from the states for the IDA loan required the revision of the PDO. Working across different states was a design flaw that created an excess of political stakeholders and vested interests that made the implementation more difficult, particularly for World Bank-funded activities. The borrowing plan required approval by the National Assembly, which caused a delay in the Ministry of Finance approval. The borrowing plan that went from the Ministry of Finance for Parliament approval included state-level responsibilities, which would require localized project coordination units. There were concerns about criteria and process for selecting the target state and a confusion about the money allocation to states and sectors, the budget distribution, the implementation challenges, etc. The revised PDO dropped the focus on four states, which made the project more relevant by targeting specific sectors at the national level.
Inadequate Risk Assessment and Mitigation
64. The overall assessment of risks at appraisal was moderate, despite a complex political
Page 26 of 71
The World Bank Nigeria - Growth & Employment (P103499)
environment surrounding the project and low implementation capacity. A detailed risk assessment at appraisal acknowledged several potential risks linked to the complex political system, governance challenges, precarious economic situation, and lack of capacity at the federal government for managing the project. No mitigation systems were put in place because the probability of political risks was assessed as low, and the results of the project were expected to be positive even in the case of an economic downturn. A mitigation plan was developed by the PIU with the technical design support of DfID GEMS4 to head this off, but this was not adopted. To compensate for the lack of implementation capacity, the project planned to competitively hire a project management unit, which would report to the PIU. This was a solution later adopted for the DFID-funded activities, but not the IDA-funded portion of the project. Considering that the political and governance challenges with the project implementation started shortly after appraisal, their risk level at appraisal was underestimated and mitigation measures not adequate. The overall rating was updated from moderate in ISR 4 to substantial in ISR 5 and remained substantial for all subsequent ISRs.
Design complexity
65. The original vision to create a multi-industry cluster program was too complex for the country’s level of institutional development. The sectoral focus of the project required industry expertise, which was not available in the PIU staffed by civil servants. The intention was to build synergies with DFID-funded sectoral activities, but the delays in IDA loan effectiveness and the World Bank's focus on a grant funding model also limited opportunities for synergy with the market systems- based DFID GEMS projects. Following the MTR, an agreement was reached between the World Bank and the Ministry of Trade and Industry that the PIU should be a mix of private and public sector members. Originally, the MTI supplied all the staff. A project coordinator from the private sector was hired in February 2017 and disbursements, which had been lagging picked up.
Lack of Readiness for Implementation and Weak Results framework
66. The project preparation failed to create institutional arrangements for effective implementation, and the lack of institutional capacity persisted through the span of the project. During the 2009 decision review meeting, instructions were given to finalize the recruitment of the PIU and update the RMF. However, these were not addressed before implementation. There was a lack of technical skills within the PIU, which the project tried to remedy via consultants. The PIU structure was intentionally opaque to prevent nepotism, and it created confusion in the roles and responsibilities of government officials vis-à-vis technical consultants. The project lacked relevant indicators and a means to regularly conduct effective M&E.
Implementation Delays
67. The project was delayed from approval this had lasting consequences. Initially, all stakeholders were enthusiastic about the project. However, because the project was delayed with the government, repeated consultations and no action for three years, stakeholders became less committed. The delays in effectiveness resulted in an unrealistically compressed implementation timeline. Apart from the delays in effectiveness, the PIU itself was only established and operational by the seventh ISR (July 2014) due to lags in the recruitment of technical specialists as well as FM and procurement staff who were already replaced in 2009 during project implementation. These extended delays then resulted in compressed timelines, which prevented venture capital funds from conducting their incubator programs properly
Page 27 of 71
The World Bank Nigeria - Growth & Employment (P103499)
and training was backloaded toward the end of GEM.
B. KEY FACTORS DURING IMPLEMENTATION
Factors subject to the control of the government
68. Inadequate risk management. The PIU staffed were trained in World Bank financial management, disbursement and procurement procedures. However, stakeholder risks persisted during the project. The World Bank team noted that the PIU had not undertaken similar activities in the past and this could impact financial management. A financial audit found ineligible expenses, missing payment vouchers, variances between planned and actual activities.
69. The payment process was cumbersome. The initial PIU payment process was later changed to require an additional 7–8 steps of approvals, which resulted in massive delays. The last minister changed the payment process to require all payments to go to him for approval.
70. There was a lack of communication with beneficiaries and stakeholders. Beneficiaries almost unanimously cited the lack of basic communication and prolonged silence from the implementing team. Beneficiaries were not informed of the reason and projected duration of delays. Intense negative media coverage contributed in the termination of the GEM project before its activities could be fully implemented. The project eventually began to generate outputs, but procurement issues slowed the progress, which generated negative press. The client hired a communications firm that provided the minister with media talking points and brought in INT to help mitigate negative press coverage.
71. There was poor coordination among many project players. There were challenges in coordinating between DFID and the World Bank, the PIU, and the multiple consulting firms and individual consultants. The clusters were divided up between DFID and the World Bank. Different bureaucratic processes compounded challenges caused by approval delays to synchronize activities. Because of the complex nature of the project, many implementation partners (consultant firms) were involved in implementation and some had performance issues, which slowed down the entire system. Given the lack of technical skills within the PIU, a simpler project with fewer components requiring fewer players might have been more successful.
72. Lack of government commitment. Apart from delays in receiving approvals and changing government ministers and priorities, GEM was at the center of a corruption scandal though an investigation later cleared it. This contributed to the lack of interest from the Ministry of Finance to continue the project. A financial audit found ineligible expenses, missing payment vouchers, variances between planned and actual activities. However, stakeholder risks persisted during the project.
73. The quality of supervision was negatively affected by the high TTL turnover. The project had 6 TTLs since appraisal in 2010. The turnover was particularly significant with 5 of these TTLs involved since project effectiveness in 2013. This affected the continuity of activities and quality of supervision, with limited ownership due to the turnover.
74. Overall, the project reporting, both in the aide-memoires and in the ISRs, was comprehensive. In most cases, the reporting provided a good basis for management to fully understand the project’s
Page 28 of 71
The World Bank Nigeria - Growth & Employment (P103499)
progress or lack thereof and provide appropriate guidance. However, there was limited reporting about the causes for the negative media coverage and the corruption accusations and did not provide an explanation of how the complaint letters sent by NGOs were addressed.
Factors outside of the control of the government
75. Nigeria’s macroeconomic conditions deteriorated during the project, resulting in high inflation. Between 2011 and 2018, Nigeria’s annual inflation ranged from 8.1-16.5 percent potentially making some goods unaffordable and impeding implementation. Beneficiary grant requests based on old budgets were no longer accurate and grants were not sufficient to purchase equipment/services, which had increased in prices thereby affecting their impact. The government did not act to troubleshoot this unanticipated situation. Grant budgets could have been adjusted by an inflation factor and used more of the component funds that were eventually decommitted.
IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME
A. QUALITY OF MONITORING AND EVALUATION (M&E)
M&E Design
76. At appraisal, the FPIU/PIU was tasked with setting up specific indicators and targets after project effectiveness, but this delay proved not to be well advised. The indicators identified at appraisal and the project implementation plan were to be finalized during negotiations in coordination with DFID. To improve measurement toward outcomes, the project proposed to incorporate third-party monitoring and an impact evaluation as a mitigation measure to remedy the poor quality of M&E. The original project objective focused on the participating states of Kaduna, Kano, Lagos, and Cross River and the Results Framework as originally designed built indicators intended to measure growth of targeted clusters and value chains in these states. Precise targets as well as intermediate results indicators supporting the project Theory of Change were to be set after effectiveness. The FPIU was to be staffed with an M&E specialist to carry out this work. For many of these indicators, baselines were never established due to the M&E team’s limited capacity and therefore, targets were not adequately set, and the indicators were not monitored.
77. Indicators related to IDA-funded activities were weak, while indicators for DFID-funded activities were clearly defined with baselines and targets. For IDA-funded activities, there was a lack of measurable and relevant indicators to support the theory of change. The original Results Framework was overly ambitious and without clarity on what the indicators intended to measure. In addition, indicators selected were not fully aligned with project activities. Therefore, it was problematic to attribute results directly to the project. The results chain between activities and expected outputs and outcomes was also unclear. For example, original PDO indicators aimed to measure the percentage increase in value added per for worker for participating firms. Because of the difficulty of measuring production costs across sectors to calculate this value, this indicator was dropped at the second restructuring. In many cases, even the results indicators that could provide a basis for assessment of PDO achievement were not measured. In addition, the indicators selected were unclear, difficult to measure or attribute to the project. Specifically, PDO-level indicator #2 aimed to measure the Growth rate of selected enterprises within target clusters, with no clear definition of what growth rate this referred to. This was specified during the first restructuring, defined to capture the growth in sales and employment and the value
Page 29 of 71
The World Bank Nigeria - Growth & Employment (P103499)
added per worker in participating firms. The latter was difficult to measure and was later dropped during the second restructuring.
78. The M&E framework was revised twice, but the revisions did not fully consider the changes in project design. The first M&E review happened during the restructuring in June of 2013 to reflect structural changes made to the project and its core objectives, and to improve relevance of the indicators. Specifically, PDO-level indicators were no longer to be measured at the state-level so two of these were dropped, with the remaining indicator discussed above. Intermediate result indicators were weak: (i) number of product improvements, (ii) number of sector policy reforms, (iii) number of sector assessments, (iv) completion of the business census, (v) number of business startups established and expected to survive, and (vi) increased investments in participating firms. The second set of revisions were made under the second restructuring in 2017 to reflect recommendations from the MTR that was conducted in June 2016. As a result, the team overhauled the project M&E framework for improved measurement of results. Indicators no longer relevant to the project were dropped, while targets for those retained were revised to better reflect project achievements. While the M&E framework should have been further revised at R4 to reflect the dropped SME Investment Fund component, legacy indicators remained, although they were not relevant for the revised design including: (i) indicators tracking internal rate of return of the SME investment, and (ii) all other SME Fund related indicators tracked under Component C, which after restructuring only includes project management activities. In addition, new indicators were added to track progress for the new SME investment Fund component added through this restructuring. The final results framework reflects these changes.
M&E Implementation
79. M&E implementation suffered from the initial delays in project effectiveness and implementation and the complexity of third-party monitoring. The project became effective in July 2013, however, the request for expression of interest for the monitoring and supervision firm was only published and closed in August 2015. The PIU M&E specialist was hired, and an implementation partner (firm) was contracted in February 2016— however, the implementation partner further outsourced data collection for the baseline and follow-up survey and the PIU M&E specialist unexpectedly passed away in early 2017. Several issues with the M&E work had been identified in November 2016, including (i) poor understanding of the project by the parties involved, (ii) lack of proactivity by the contractor, and (iii) performance issues of the contracted staff. The MTR from June 2016 also had suggested for the Results Framework to be revised to rationalize and realign project activities to the objectives—calling for project restructuring. As a result of these events, changes were only officially completed in the system in August 2017 (R2) with a new PIU M&E specialist recruited to populate and update the Results Framework, however, given the political economy challenges tied to the project—limited reliable data and information was collected over the course of implementation
80. Broadly, M&E arrangements were unclear, with too many actors involved in the process. The process included a major accounting firm, the BIG Platform team, independent monitors and PIU specialist—each accountable for the monitoring of different components and with no clear focal point assigned to oversee M&E more broadly at the project level. Despite several attempts by the World Bank team to bring the project back on course—for example, by instituting the BIG Platform to track progress for entrepreneurs and capture firm-level data—external factors tied political economy challenges and unexpected events made it difficult to collect and report the relevant information accurately.
Page 30 of 71
The World Bank Nigeria - Growth & Employment (P103499)
M&E Utilization
81. The challenges in M&E implementation led to limitations in the utilization of M&E. As a result of delays in setting up the M&E system and establishing the baseline and target indicators, the M&E data had a limited use during implementation. The DFID M&E systems captured results of DFID-funded activities, but the lack of a unified results framework for the project led to limitations in results measurement.
82. The impact evaluation helped to identify effective approaches but came too late in the project to help improve utilization. The evaluation helped to identify effective ways of increasing MSME business practices and capacity to stimulate growth. Specifically, the evaluation results (Annex 8) compared the impacts of capacity development, including skills outsourcing, skills insourcing, BDS, and training. The data collected included results from a one-year follow-up survey taken as the program ended and a follow-up survey (two- year follow-up).
Justification of Overall Rating of Quality of M&E
83. The M&E quality had significant shortcomings. Design was weak as were the M&E implementation and utilization within the project lifespan. While there was some useful data with which to assess PDO achievement, it was difficult to test the links in the results chain. Thus, the overall rating is Modest.
B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE
84. Environmental and Social Safeguards was rated Satisfactory throughout the ISR reporting (ISR 13–15) for the project. The safeguard policies triggered by project design did not change; no physical and economic displacements occurred. The World Bank advised the Project to establish a grievance redress mechanism (GRM). As provided in ISR 14 (dated February 13, 2018) GRM was operationalized in the project to receive numerous complaints that emerged due to operational delays in grant disbursement. ISR 15 (dated April 1, 2019) recorded about 50 individual complaints out of 109,00 individual register on the project’s portal, which had been filled under the World Bank’s grievance redress services (GRS). The complaints hinged on delays in processing applications and disbursement of approved grants. Both the GRS and the project’s GRM responded to the grievances raised.
85. Project support to individual MSMEs met all safeguards requirements. The project’s safeguards unit provided an oversight role in assessing individual business plans in compliance with set environmental and social guidelines. Only 2 percent of the assessed business plans was indicated as a high risk to the project and subsequently dropped. Business plans rated moderate to low risks were provided with adequate safeguard support to improve business plans. The Project partnered with the Kaduna State Government (KSG), the Housing Development Institute and the National Open University of Nigeria (NOUN) Kaduna Chapter to organize a Building Artisan Skills Training Program for the young and unemployed youths of Kaduna State. The program centered on providing building skills to beneficiaries in the construction sector. The Safeguard unit of the project provided occupation health and safety (OHS) support to the beneficiaries during sub-project implementation.
86. Financial management was rated satisfactory and moderately satisfactory. ISRs 1–9 ratings were satisfactory and downgraded to moderately satisfactory in ISRs 10–12 (after the MTR), 13–15.
Page 31 of 71
The World Bank Nigeria - Growth & Employment (P103499)
87. The rating for procurement declined from satisfactory to moderately unsatisfactory within the life-span of the project. Procurement was rated satisfactory in ISRs 1–6, moderately satisfactory in ISRs 7–9 and moderately unsatisfactory in ISRs 10–12 (after the MTR), 13–15. ISR 6 (dated January 18, 2014) noted that an accountant and procurement staff were put in place following restructuring #1 and first disbursement in June and September 2013. ISR 10 downgraded procurement to moderately unsatisfactory, stating that, "During the supervision mission, the Task team highlighted the multiple and repeated mistakes made in critical procurement documents over the last 4 months and for that reason, the procurement rating is Moderately Unsatisfactory. The Task team also noted the new procurement arrangements should lead to significant improvement of the documents being sent to the World Bank in the near future."
88. The team was unable to generate financial reports on time. The World Bank team met with them regularly to instruct them on what information they need to locate and accessing information to feed into the system. Due to the lack of capacity, the initial procurement team of one person plus an accountant was increased to three people plus the accountant. ISR 11 (dated May 30, 2016) reported that the new procurement consultant led to a significant improvement in the documents sent to the World Bank, but that the PIU needed technical expertise to proposals. Although procurement record keeping has improved, the PPR showed inadequate use of CQS method. The 15th and final ISR (April 1, 2019) noted that procurement performance has improved including in contract management.
C. BANK PERFORMANCE
Quality at Entry
89. The World Bank was unsuccessful in securing sufficient buy-in from the states at appraisal, and did not reach firm agreements with the key beneficiaries on project design, and budget allocation to state- level activities. The World Bank team did attempt to remedy the issue of ownership during the first restructuring and closely worked with the federal government to remedy the initial weaknesses in project preparation. The PAD indicates that extensive studies were conducted to establish the rationale for the project and the economic and financial analysis.5 Collaboration with DFID and integration of donor efforts was an innovative design feature that could have led to enhanced impact. The selection of individual clusters allowed DFID and the World Bank implementing the project though parallel financing. Although this allowed for a simpler design than co-financing arrangements, the resulting technical design was complex and not proportionate with the counterpart’s implementation capability. Several activities were dropped before implementation. As discussed under the Quality of M&E section, the quality of M&E system and particularly its design was poor.
Quality of Supervision
90. The World Bank team was proactive, trying to implement a rather ambitious project, despite a weak PIU and administrative and political roadblocks. Implementation support missions were regular, and an ongoing impact evaluation monitored a large share of beneficiaries. The partnership with DFID also helped bring technical advisors and awareness creation that came with more personnel. The World Bank team was responsive
5 Most of the report were not published, but they are available as internal documents. For example: Peter Yee and Michaela Paludetto. 2005. Nigeria: Value and Supply Chain Study. Final Draft Report Prepared for the World Bank. Concilium International Inc. Some other reports cited in the PAD are not available at the time of ICR writing.
Page 32 of 71
The World Bank Nigeria - Growth & Employment (P103499)
to changing conditions and adjusted the project design to reflect requests from the counterpart and needs of beneficiaries. There was a high degree of innovation in project design and implementation. Mission aide- memoires generally contained action plans, though ISRs did not fully reflect the severity of performance problems. The team identified solutions for compensating for initial shortcomings in the M&E systems: the baseline for the Component B was collected in 2016-2017, and program stared generating data on the target SME population. Additionally, the team leveraged trust funds and established a close collaboration with DEC to implement an impact evaluation. Although there was significant rotation in Bank TTLs, it was understandable considering the length of the project and there was a high degree of collaboration among incoming and outgoing TTLs. All the TTLs remained on the project for sufficiently long periods of time (2-5 years). One DFID staff member was based in Abuja and entirely dedicated to GEM while two World Bank Abuja staff worked on urgent client relationship matters and other issues. In total, there were three Abuja based staff working in GEM plus consultants. In January 2018, Kofi-Boateng Agyen (TTL) was moved to Abuja to focus on project outcomes and develop a relationship with the Federal Minister of Industry, Trade & Investment. The World Bank procurement team provided regular procurement training to project coordination staff, reviewed procurement challenges that cropped up and accompanied TTLs on regular non-mission visits to project unit and beneficiaries. The FM team also worked on upgrading FM software as part of efforts to ensure the seamless provision of financial data such as disbursements forecasts to aid project planning. There was significant overlap as TTLs changed with previous TTLs staying involved for short periods, joining missions with new TTLs or remaining as co-TTLs. There were multiple years where three supervision missions were conducted. There was a steady stream of video conferences and face to face meetings through to the closing date outside the ISMs. The World Bank team was able to meet with the minister with the country director present.
91. In retrospect, therefore, it would probably have been better not to have extended the project a second time. There was evidence to support the World Bank’s decision to continue the project, but in the end more than half of the project’s IDA resources had to be cancelled. During this period, the project had fallen too far behind for it feasibly to be fully implemented. One option would have been to cancel funds and scale down the project earlier. Such a move could have signaled a stronger position on the part of the World Bank and focused implementation on key activities.
Justification of Overall Rating of Bank Performance
92. The project fell well below expectations as a result of both poor design and weak implementation. As of January 31, 2019, fifty-six percent (56 percent) of the total IDA project allocation (US$160 million) has been disbursed. The project was in problem status since mid-November 2015 due to serious implementation challenges and complex political economy situations in Nigeria, including a long delay in project approval at the level of the legislature. The project also suffered from negative media exposures (allegations emanating from the legislature and a section of project beneficiaries alluding to misuse of funds by the counterpart Ministry), most of which were misleading and unfounded and yet had very negative impact on project implementation. The project has also received numerous complaints from beneficiaries and grantees whose payments under a grant sub- component had delayed even though, at the same time, they expressed satisfaction with the project, with regards to its impact on their business practices. These complaints were made to the either project’s or the World Bank’s GRS, and to OPCS. These complaints resulted in government mandating changes in staffing at the project implementation unit and the introduction of additional checks with regards to fiduciary arrangements. The end result was significant delays to implementation, unsatisfactory stakeholder engagement and project ownership challenges, and the client’s request on March 8, 2019 to cancel US$76 million of the undisbursed IDA funds (Figure 2). The decrease in disbursements at the end of the projects indicates that the unused funds were transferred back by the Nigerian government to the World Bank.
Page 33 of 71
The World Bank Nigeria - Growth & Employment (P103499)
Figure 2: Project Disbursements
Source: Operations portal, retrieved on January 31 at 13:45pm.
93. Supervision was proactive, but the World Bank could have better gauged the project’s implementation challenges at restructuring and streamlined the project. For example, the World Bank needed to properly estimate the time needed to complete the activities retained after restructuring.
The overall World Bank performance is, therefore, Moderately Unsatisfactory.
D. RISK TO DEVELOPMENT OUTCOME
94. Several measures have been taken to enhance the chances of sustaining project outcomes, especially with regard to sustaining business regulations reforms and building sustainable BDS markets. However, the project did not achieve systemic changes to SME performance and environment, and most outcomes were achieved close to the end of the project, which make the overall risk Substantial.
95. The project has had a positive effect on business environment for SMEs in the states that were targeted by the project before restructuring. DFID-funded activities aimed to create demand for reform through public sensitization and awareness raising via BMOs, traditional leaders, and lead figures in business communities. The evaluation of Component A concluded that these activities empowered the business community and brought tangible benefits (such cost savings, public works, or obtaining a certificate of occupancy) on the demand side and increasing pressure for responsible public service through greater transparency and accountability on the supply side. Component A also provided TA and training to state government and relevant policymaking agencies that improved their capacity for designing and implementing SME policy reforms.
96. From the private sector perspective, the project had a positive impact on firm-level capabilities of direct beneficiaries and increased the availability and quality of BDS to SMEs. The
Page 34 of 71
The World Bank Nigeria - Growth & Employment (P103499)
project reinforced parts of the SME ecosystem including business training school, incubators, BDS providers and consultants (in accounting, digital marketing, etc.). Local stakeholders directly benefitted from project activities (training, TA) and were actively involved in the implementation. Several BDS providers reported that they continued to use training materials developed with support from the project. Some of the business training schools including Enterprise Enrollment Center in Lagos and Kaduna Polytechnic continue to offer enterprise training after the project closed. The BDS provider also benefitted indirectly from expansion of the market for their services: the matching grants incentivized the SMEs to use, and pay for, the BDS using local providers. The project facilitated collaboration between SMEs and BDS provider through positive experience, created the demonstration effect, and contributed to expansion of demand and supply of quality BDS. Project beneficiary SMEs indicated that they are willing to pay for BDS even without GEM financial support. Although the data collected in ICR survey was too close to completion to judge the sustainability of results, it indicates that the effect of BDS services and grant support to firms might be sustained in terms of revenues and retention of jobs created through project financing.
97. Although the project was implemented through the PIU located at the Ministry of Industry, Trade and Investment , there has been little uptake of project activities by the government outside of the project funding. In October 2019, a MSME GEM EXPO was hosted by the Minister of Industry, Trade and Investment and some grantees have approached NISREAL for CBN loans. Nonetheless, the PIU was dissolved and the BIG platform taken down immediately upon project completion. All the financing and capacity building windows closed because GEM was the only source of funding and project instruments have not been institutionalized within the government’s systems. There is little indication that the government will continue to finance and implement the SME support activities financed by the project. The project implementation relying heavily on the technical capacity of international firms and consultants also reduces the likelihood of institutionalization of the project.
V. LESSONS AND RECOMMENDATIONS
98. The GEM project produced many lessons, some of which are common in “problem” projects: A very broad PDO engenders lack of commitment from stakeholders and lack of accountability for results, make it difficult keep focus of what the essential aspects during in restructurings are, and create questions on attribution of results beyond the direct beneficiaries. Political changes can lead to shifts in government priorities and commitment, so it is important to build flexibility into the PDO and the design of the project in unstable political environments. The lack of PIU readiness, and technical skills in general, slow down the implementation and must be addressed as early as possible in project preparation. The lack of M&E systems and effectiveness lead to inadequate data collection and must also be addressed during preparation to avoid management issues later on. Grant funds tend to be easier to move than borrowed funds as was the case with DFID funds compared with World Bank Group funds. Concrete plans need to be in place before closing to ensure continuity of the project’s results. These lessons are important, but were already highlighted in other ICRs, and this ICR only points out that they apply to this GEM project as well. There are also several lesson specific to GEM because the project had many innovative features, i.e., co-financing with DFID, online grant allocation platform, and impact evaluation.
Page 35 of 71
The World Bank Nigeria - Growth & Employment (P103499)
99. Multi-donor projects could effectively address diverse constraints to growth and employment in a complex environment but only if the appropriate coordination mechanisms are put in place. Once the IDA funding was effective, collaboration helped focus resources and support to critical development challenges through: (i) consolidation of development efforts (larger pool of funding, combined types of funds between IDA lending and donor grants that allow for advancing complex reforms), (ii) enhanced supervision because of additional expertise and broader technical skills by the joint team, and (iii) risk mitigation (reforms supported by DFID in Component A advanced and laid the groundwork for the implementation of Component B. However, the opportunity for a multiplier effect from this multi-donor intervention was missed because of the failure to achieve a coordinated and synchronized implementation of DFID-funded and IDA-funded components, which requires both coordination mechanisms and day to day continuous willingness to collaborate, coordinate and leverage. Although the DFID technical expert was embedded in the project implementation in an advisory role and participated in the joint supervision missions, there were no mechanisms for formal coordination and no single decision-making authority for DFID and IDA sources of funding. The lack of coordination and alignment in the log frames reflected this less effective collaboration.
100. Improved stakeholder assessment and consultations, especially in the context of a changing political environment, is essential for maintaining government commitment to the project and building partnerships that help mitigate serious political economy issues when they arise. Stakeholders’ buy-in and commitment to the PDO, project activities and implementation arrangements, is a pre-condition for successful implementation. In the case of Nigeria, a strong commitment meant inclusion of the project in the borrowing plan, which was not the case for GEM at approval. Political elections and failure to gain commitment from the new federal governments for project PDO (especially with regards to targets at the state level) and design at appraisal delayed implementation. The original PDO and design was a non-starter for the recently elected government, which had different priorities, and the project missed an opportunity for carrying out the blended DFID/IDA project in a synchronized manner. This project showed us how important it is to maintain a continuous stakeholder engagement. The equity fund was introduced after internal stakeholder consultation only with the requesting ministry. The political attacks mostly targeting the equity component. Upfront involvement of a broad range of stakeholders would mitigate the effects of the outcry going up to the senate level and two years of investigations. The resulting loss of synergies, momentum, and stakeholder trust impaired the project’s results.
101. Timely and transparent communication need to be carried out to address implementation challenges and maintain engagement with beneficiaries. Most of the delays in implementation were caused by external factors, but their negative effects could have been reduced by timely and transparent communication to partners and beneficiaries about the causes for delay and the mitigation measures. Messaging to potential beneficiaries regarding what will or will not be available, who will be eligible, and how processes will work must be unambiguously and loudly publicized before the program begins, and throughout its life, and until well after the last grant has been disbursed. The flow of communication should go both ways, with a robust and professional grievance and publicity machine must be in place before any beneficiary activity begins. At inception, a dedicated Facebook page was a popular source of information but is closed because of the World Bank corporate policy. At later stages, beneficiaries who were connected to the project through self-created WhatsApp groups remained more engaged and more positive about project performance. While the BIG Platform could have been used as a mechanism to better communicate with counterparts, its operationalization was delayed. The timing could be
Page 36 of 71
The World Bank Nigeria - Growth & Employment (P103499)
improved to maximize benefits in future operations.
102. The PIU staff needs to be accountable for the M&E, performance and results of the project, which can help with change management in the complex political context. The performance issues of the PIU stemmed from the lack of accountability for the results. Notably, the PIU performance improver towards later stages of the implementation when the accountability was related to disbursement ratio. Collaboration and collective leadership opportunities could not arise in a context where the decision-making power belonged to public sector employees whose performance was not linked to project results, who lacked expertise and experience in managing private sector contracts, and whose remuneration was much lower than the private sector consultants reporting to them. Performance contracts or performance bonuses, results agreements tied to project objectives and deliverables, and regular evaluations against agreed deliverables could improve accountability and provide incentives for performance. Sustainability of project results can be achieved by providing incentives to the implementation partners for building institutional capacity and markets for BDS. When local partners (incubators, equity firms) were engaged in the implementation, the activities continued beyond the funding of the project.
103. The design needs to match the implementation capacity of the counterpart, from the technical and governance perspective. Design of Component B activities before and after the restructuring exceeded the implementation capacity of the PIU especially in the early stages of the implementation. Ideally, the cluster component would have been managed by the PIU, but it lacked the technical expertise required to position the project well in the early stages, which damaged the credibility of the project. Valid concerns regarding governance challenges led to a complex design of the matching grants allocation system, which also exceeded the implementation capacity of the PIU.
104. The M&E system needs to be in place at effectiveness, otherwise it could lead to M&E issues, if, for example baselines for indicators are not established by then. Designing and measuring proper baseline indicators can better inform the achievability of targets set and ensure that monitoring indicators are realistic in the attribution of results. The project M&E framework was revised multiple times to better align results to activities and ensure that indicators were measurable and targets achievable. Developing the M&E framework mid-way through implementation does not provide the basis for capturing results and prevents the team from using results in project management and restructurings. The lack of M&E alignment between DFID-funded components and IDA-funded component also made it difficult to capture and attribute overreaching results for the project.
105. The use of targeted and timely impact evaluation can improve results and value-for-money of the project, but it can put an additional strain on implementation. Impact evaluation of capacity development approaches provided important lessons for future efforts to improve SME capabilities. Additionally, given significant limitations in the project’s M&E, the impact evaluation provided more information than would have otherwise been available. However, because impact evaluation findings can only be assessed after the implementation, the exercise can be perceived as cumbersome, costly and even unfair by the PIU, project stakeholders and beneficiaries who do not have direct benefits from the exercise. It is important to communicate the value from impact evaluations to beneficiaries during implementation. From the Government’s, World Bank and donors’ perspective, impact evaluations could be very valuable in programmatic development efforts where a follow-on activity could take on board the findings from the evaluations.
Page 37 of 71
The World Bank Nigeria - Growth & Employment (P103499)
106. Mechanisms to sustain development impact post-closure should be incorporated into the project design. For projects targeting SME capacity development, this could include making training materials available online to the public or establishing a network for participants to work with each other. Online platforms, such as the BIG Platform (Annex 9), should have a post-project maintenance plan to enable users to retain access and to prevent the loss of investment.
.
Page 38 of 71
The World Bank Nigeria - Growth & Employment (P103499)
ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS
A. RESULTS INDICATORS
A.1 PDO Indicators
Objective/Outcome: To increase firm growth and employment in participating firms in Nigeria. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion % growth in sales of Percentage 0.00 20.00 258.00 participating firms 02-Feb-2016 30-Sep-2019 06-Mar-2019
For firms in services Percentage 0.00 20.00 157.00
01-Oct-2016 30-Sep-2019 06-Mar-2019
For firms in industry Percentage 0.00 20.00 326.00
01-Dec-2016 30-Sep-2019 06-Mar-2019
For female-headed Percentage 0.00 20.00 99.00 participating firms 01-Nov-2016 30-Sep-2019 06-Mar-2019
Comments (achievements against targets):
Page 39 of 71
The World Bank Nigeria - Growth & Employment (P103499)
Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion % increase in average Percentage 8.00 20.00 30.00 number of workers in participating firms 03-Oct-2016 07-Sep-2018 06-Mar-2019
For firms in services Percentage 0.00 20.00 36.00
01-Oct-2016 30-Sep-2019 06-Mar-2018
For firms in industry Percentage 0.00 20.00 17.00
01-Oct-2016 30-Sep-2019 06-Mar-2019
For female-headed Percentage 0.00 20.00 0.00 participating firms 01-Oct-2016 30-Sep-2019 06-Mar-2019
Comments (achievements against targets):
Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion % growth in sales of Percentage 0.00 0.00 0.00 participating firms 02-Feb-2016 23-Jun-2017 23-Jun-2017
Page 40 of 71
The World Bank Nigeria - Growth & Employment (P103499)
For firms in services Percentage 0.00 20.00 0.00
01-Oct-2016 30-Sep-2019 23-Jun-2017
For firms in industry Percentage 0.00 20.00 0.00
01-Dec-2016 30-Sep-2019 23-Jun-2017
For female-headed Percentage 0.00 20.00 0.00 participating firms 01-Nov-2016 30-Sep-2019 23-Jun-2017
Comments (achievements against targets):
Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion % increase in average Percentage 8.00 20.00 0.00 number of workers in participating firms 03-Oct-2016 07-Sep-2018 23-Jun-2017
For firms in services Percentage 0.00 20.00 0.00
01-Oct-2016 30-Sep-2019 23-Jun-2017
For firms in industry Percentage 0.00 20.00 0.00
01-Oct-2016 30-Sep-2019 23-Jun-2017
Page 41 of 71
The World Bank Nigeria - Growth & Employment (P103499)
For female-headed Percentage 0.00 20.00 0.00 participating firms 01-Oct-2016 30-Sep-2019 23-Jun-2017
Comments (achievements against targets):
A.2 Intermediate Results Indicators
Component: Component A Increased Investment Climate
Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion No. of identified sector policy Number 0.00 19.00 19.00 reforms presented to the steering committee 01-Aug-2013 06-Mar-2019 06-Mar-2019
Comments (achievements against targets):
Component: Component B Increased Competitiveness of Strategic Clusters
Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of market Number 0.00 10.00 12.00 assessments 01-Aug-2013 30-Sep-2019 06-Mar-2019
Page 42 of 71
The World Bank Nigeria - Growth & Employment (P103499)
Comments (achievements against targets):
Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Increased investment in Percentage 0.00 10.00 33.00 participating firms 01-Aug-2013 30-Sep-2019 06-Mar-2019
Comments (achievements against targets):
Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Increase in score for business Number 0.00 20.00 14.00 practices for participating firms 02-Dec-2016 30-Sep-2019 06-Mar-2019
Comments (achievements against targets):
Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Direct project beneficiairies Number 0.00 14900.00 22633.00
04-Dec-2014 30-Sep-2019 06-Mar-2019
Page 43 of 71
The World Bank Nigeria - Growth & Employment (P103499)
of which percentage of Percentage 0.00 40.00 28.00 female beneficiairies
Comments (achievements against targets):
Component: Component C Project implementation, Monitoring and Evaluation and Communications
Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of MSMEs Number 0.00 100000.00 107910.00 registered in the BIG Platform 30-Jul-2013 30-Sep-2019 06-Mar-2019
of which outside Lagos and Number 0.00 30000.00 95907.00 FCT 30-Jul-2013 30-Sep-2019 06-Mar-2019
Of which invited to an Number 0.00 12000.00 22408.00 induction workshop 30-Jul-2013 30-Sep-2019 06-Mar-2019
Comments (achievements against targets):
Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion
Page 44 of 71
The World Bank Nigeria - Growth & Employment (P103499)
Number of MSMEs Number 0.00 8974.00 2111.00 beneficiting from capacity enhancement 05-Jul-2013 30-Sep-2019 06-Mar-2019
Of which with capacity Percentage 0.00 85.00 53.00 enhancement completed
Comments (achievements against targets):
Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Total number of trainees Number 0.00 1000.00 0.00 (cumulative) 02-Dec-2016 30-Sep-2019 23-Jun-2017
of which employed after Number 0.00 200.00 428.00 training (for individuals) 02-Dec-2016 30-Sep-2019 06-Mar-2019
Comments (achievements against targets):
Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Volume of financing to Amount(USD) 0.00 2825.00 0.00 participating firms from the SME Investment Fund 23-Jun-2017 30-Sep-2019 23-Jun-2017
Page 45 of 71
The World Bank Nigeria - Growth & Employment (P103499)
(million)
Comments (achievements against targets):
Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of participating Number 0.00 19.00 0.00 firms receiving funding from the SME Investment Fund 23-Jun-2017 30-Sep-2019 23-Jun-2017
Comments (achievements against targets):
Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion SME Investment Fund Text NO yes NO portfolio quality IRR>0 23-Jun-2017 30-Sep-2019 23-Jun-2017
Comments (achievements against targets):
Page 46 of 71
The World Bank Nigeria - Growth & Employment (P103499)
B. KEY OUTPUTS BY COMPONENT
For IDA-Funded activities based on R2: PDO: To increase firm growth and employment in participating firms in Nigeria 1. Percentage growth in sales in participating firms for (i) firms in services; (ii) firms in industry; and (iii) female-headed firms Outcome Indicators 2. Percentage increase in average number of workers in participating firms for (i) firms in services; (ii) firms in industry; and (iii) female-headed firms 1. 1. Percentage increase in investments in participating firms 2. Increase in score for business practices for participating firms 3. Total number of trainees, including value for the number employed after training (for individuals) Intermediate Results Indicators 4. Volume of financing to participating firms from the SME Investment Fund (million) 5. Number of participating firms receiving funding from the SME Investment Fund 6. SME Investment Fund portfolio quality IRR > 0 1. Number of market assessments 2. Number of identified sector policy reforms presented to the steering committee 3. Number of MSMEs registered in the BIG Platform, of which are (i) Key Outputs by Component outside Lagos and FCT; and (ii) Invited to an induction workshop (linked to the achievement of the Objective/Outcome 1) 4. Number of direct project beneficiaries, including percentage value of female beneficiaries 5. Number of MSMEs benefitting from capacity enhancement, including percentage value of those that have completed capacity enhancement
Page 47 of 71
The World Bank Nigeria - Growth & Employment (P103499)
ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION
A. TASK TEAM MEMBERS
Name Role Preparation Supervision/ICR Kofi-Boateng Agyen, Wenye Dong Task Team Leader(s) Sunday Esene Osoba, Bayo Awosemusi, Daniel Rikichi Procurement Specialist(s) Kajang Akinrinmola Oyenuga Akinyele Financial Management Specialist John Amedu Eimuhi Procurement Team Joyce Chukwuma-Nwachukwu Procurement Team Abiodun Elufioye Procurement Team Magalie Pradel Team Member David J. McKenzie Team Member Michael D. Wong Team Member (former Task Team Leader) Natalia Agapitova Team Member Guillemette Sidonie Jaffrin Team Member Joseph Ese Akpokodje Environmental Specialist Adja Mansora Dahourou Simpore Team Member (former Task Team Leader) Irene Marguerite Nnomo Ayinda-Mah Procurement Team Lindsey Tan Lim Team Member Ogochukwu Joy Medani Procurement Team Johanne Buba Team Member (former Task Team Leader) Ismail Radwan Former Task Team Leader Nneamaka Pamela Okechukwu Team Member Nadia Asgaraly Team Member Andrew Taylor Gartside Team Member Edda Mwakaselo Ivan Smith Social Specialist
Page 48 of 66
The World Bank Nigeria - Growth & Employment (P103499)
B. STAFF TIME AND COST
Staff Time and Cost Stage of Project Cycle No. of staff weeks US$ (including travel and consultant costs) Preparation FY07 7.300 46,401.81 FY08 40.424 198,384.58 FY09 18.111 152,143.99 FY10 41.047 195,151.30 FY11 14.300 94,032.37 FY12 0 - 24.01 FY19 0 -121,791.52
Total 121.18 564,298.52
Supervision/ICR FY09 0 0.00 FY11 9.900 61,607.86 FY12 34.781 239,007.66 FY13 30.142 178,663.88 FY14 34.482 196,731.04 FY15 55.183 188,773.41 FY16 49.519 167,060.36 FY17 106.877 566,679.55 FY18 160.384 784,096.20 FY19 64.092 619,763.14 FY20 23.119 126,043.27 Total 568.48 3,128,426.37
Page 49 of 66
The World Bank Nigeria - Growth & Employment (P103499)
ANNEX 3. PROJECT COST BY COMPONENT
IDA Funding: Planned and Actuals
Amount at Approval Actual at Project Percentage of Approval Components (US$M) Closing (US$M) (%) Component 1 : Improved 0 5.00 N/A Investment Climate Component 2 : Increased Competitiveness of Strategic 140.00 51.85 37.03 Custers Component 3 : Project Implementaion, Monitoring 20.00 10.00 50.00 and Evaluation and Communications Total 160.00 66.85 41.78
Parallel Financing from DFID: Planned and Actuals
Amount at Approval Actual at Project Percentage of Approval Components (US$M) Closing (US$M) (%) Component 1 : Improved 75.00 72.73 96.97 Investment Climate Component 2 : Increased 32.00 48.18 150.56 Competitiveness of Strategic Custers Component 3 : Project 2.37 5.67 239.24 Implementaion, Monitoring and Evaluation and Communications Total 109.37 126.58 115.74
Page 50 of 66
The World Bank Nigeria - Growth & Employment (P103499)
ANNEX 4. EFFICIENCY ANALYSIS
The ICR does not attempt to quantify the benefits of the small part of the project that was implemented. Specifically, as a result of the (i) project redesign and (ii) limitations in the data collected, the ICR team is unable to replicate the economic and financial analysis provided in the PAD—which provides estimates for the project’s contribution for GDP growth in the originally selected states. Given that a majority of original activities were dropped, it is unclear if updated results calculated would be attributable to the project. Nevertheless, the cost- efficiency of implementation is considered modest. Benefits that are expected to accrue to the project from the sub- components of the project that were implemented are discussed under the Efficiency section of the main text. In addition, preliminary results from the impact evaluation (page 12, 13, August 2019 Draft) measuring cost-benefit of the IDA-funded capacity building component are extracted below.
Cost-benefit analysis was carried out with the caveat that the actual spend numbers for the realized costs of the interventions were not available for all the activities. For the capacity development activities, the impact evaluation reports on spending for the training, insourcing and outsourcing, which were approximately similar in cost, at around US$2,000 per person, while the BDS consulting was twice this (US$4,000 per person). Using an exchange rate of 365 Naira to 1 USD, we have that the gain in monthly profits from outsourcing was 36,000 Naira, or around US$100. So we would need firms to experience this increase in profits for 20 months to cover the cost per person of training. However, as noted, there is considerable uncertainty in this profits estimate, and the impact on levels of profits is driven in part by a few firms with very high profit levels. Nonetheless, this gain is in the ballpark of it being cost-effective. Similarly, the gain in profits outside of Lagos and Abuja from BDS is approximately US$265 per month, which would require 15 months of gains to pay the costs of BDS. Thus, there is some evidence for the outsourcing and BDS being cost-effective. The insourcing and outsourcing deliver the same, or greater, improvements in business practices than BDS at half the cost, and more innovation, suggesting they perform better on a straight cost-benefit comparison basis. Business training does not appear cost-effective and seems to be dominated by the other interventions.
Page 51 of 66
The World Bank Nigeria - Growth & Employment (P103499)
ANNEX 5. BORROWER, CO-FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS
Government Comments
The government ICR on the GEM project was submitted to the World Bank in September 2019. Consistent with the Efficacy rating assigned in the Bank ICR, the PIU rated the project as Moderately Unsatisfactory. The content in this section attempts to summarize the information reported in the government ICR, while Table 5 was extracted from the government document.
Achievement of PDO/Efficacy
Recognizing the early implementation delays, the PIU found that a major project accomplishment was the launch of the BIG Platform, which helped channel project support to MSMEs, feeding into the key milestones and outputs reported in Table 5. As stated, the PIU considers: “the data provided on the project’s efficacy clearly confirms that despite delays and other implementation challenges, the PDO level and the intermediate level targets were achieved. However, the quantitative achievements have been undermined by severe disbursement challenges, which were manifested by slow release of funds.” Additional discussion on these results and on other project outcomes and impacts can be found in the original document.
Table 5. Key Milestones and Outputs (Source: Government ICR)
GEM PROJECT RESULT FRAMEWORK PDO To increase firm growth and employment in participating firms in Nigeria S/No Indicator Name Unit of Baseline Original End Line Actual Measurement Target Achievement Achievement (percent) Project Development Objectives 1 Growth in sales of percent 1,867,355 20 6,684,865 258 percent participating firms percent 2 Growth in sales of percent 2,342,218 20 6,007,944 157 percent participating firms: percent For firms in Services only 3 Growth in sales of percent 1,569,157 20 6,684,865 326 percent participating firms: percent For firms in Industry only 4 Growth in sales of percent 2,324,506 20 4,620,870 99 percent participating firms: percent For female-headed participating firms 5 percent Increase in percent 8.0 20 6.92 38 percent average of workers percent in participating
Page 52 of 66
The World Bank Nigeria - Growth & Employment (P103499)
firms 6 Increase in average percent 0.0 20 6.78 36 percent no. of workers in percent participating firms: For firms in Services only 7 Increase in average percent 5.9 20 6.88 17 percent no. of workers in percent participating firms: For firms in Industry only 8 Increase in average percent 5.9 20 5.77 -2 percent number of workers percent in participating firms: For female headed participating firms only
Key Factors and Lessons Learned
The government ICR also identified a number of factors that affected project implementation, which align with the lessons learned reported (extracted in Box 1). The relevant factors were listed as: (i) late commencement of project implementation; (ii) complexity of the project design; (iii) lack of proper understanding of World Bank rules and operational procedures; (iv) political economy issues; and (v) outsourcing of key project management components to third-party consultants. Again, additional discussion can be found in the original document submitted by the PIU.
Box 1. Lessons Learned and Recommendations 1. Stakeholder engagement, advocacy, and sensitization are a continuous process throughout the life of the project. 2. Complex design of developmental interventions impedes optimal achievement of results. 3. Country location of TTL helps to accelerate project implementation as challenges are quickly dealt with. 4. Clear and concise understanding of the Work Plan rules and regulation at the beginning of the project is a critical success factor for efficiency and effectiveness of the implementation process. 5. Communication is an indispensable component of program management and once there is a gap, it is capable of causing huge problems to the project. 6. Change management is important to successful project implementation and outcomes.
Co-Financier (DFID) Comments
The ICR received extensive support from the DFID secondee to the World Bank and the Coordinator for the DFID GEMS projects Andrew Gartside. It included: the background note that was prepared at the end of the secondment, an interview and detailed comments on an ICR draft.
Page 53 of 66
The World Bank Nigeria - Growth & Employment (P103499)
ANNEX 6. SUPPORTING DOCUMENTS (IF ANY)
1. Project Appraisal Document (Report No:54457-NG) – Growth and Employment in States Project (GEMS) – February 17, 2011 2. Financing Agreement between the Federal Republic of Nigeria and International Development Association (CR 4882-NG) – Growth and Employment Project – June 24, 2013 3. Corrigendum to Financing Agreement (CR 4882-NG) – Nigeria: Growth and Employment Project – February 25, 2014 4. Implementation Supervision Reports (Seq 01-15) 5. Restructuring Paper on a Proposed Project Restructuring of Nigeria Growth & Employment in States Project (Report No. 77996-NG) – June 4, 2013 6. Restructuring Paper on a Proposed Project Restructuring of Nigeria Growth & Employment (Report No. RES25557) – July 19, 2017 7. Restructuring Paper on a Proposed Project Restructuring of Nigeria Growth & Employment – (Report No. RES35150) – November 26, 2018 8. Restructuring Paper on a Proposed Project Restructuring of Nigeria Growth & Employment –March 2019 (Report No. RES35611) – March 11, 2019 9. Government Request for Cancellation of the Undisbursed Amount of US476M Under the Growth and Employment (GEM) Project NO.4882 (Ref No. 11373/S.34/C.730/T) – March 8, 2019 10. Nigeria: IDA Credit Number 4882 (Growth and Employment Project) Notice of Partial Cancellation and Transmitting Revised Withdrawal Schedule After Partial Cancellation – March 28, 2019 11. World Bank Impact Evaluation, Pre-Analysis Plan – February 2019 12. World Bank Impact Evaluation, Preliminary Results – August 2019 13. DFID: Independent Monitoring and Evaluation Project – GEMS Lesson Learning Review, Final Report – November 30, 2015 14. DFID: Cross GEMS Project Completion Review March 2018 15. Implementation and Results Report on Growth and Employment Project (GEM) submitted to the Project Coordinator, Federal Ministry of Industry, Trade and Investment, Abuja Nigeria – September 4, 2019 16. ICR Focus Group Discussions, Summary
Page 54 of 66
The World Bank Nigeria - Growth & Employment (P103499)
ANNEX 7. SUMMARY OF RESULTS FROM DFID ASSESSMENT
Excerpt from DFID Self-Assessment:
GEMS supported the raising of incomes for more than two million people and created more than 30,000 jobs, jobs whose underlying model could be replicated far wide in every state of Nigeria. On business environment reform it became the “go to” project, widely and deeply engaged with by both government and development partners, and contributing to the improvement in Nigeria’s ease of doing business rankings. It left behind proven toolkits by which states could significantly raise their internally generated revenue, and its work on land received international attention. On the business and investment side, GEMS proved that despite a difficult business environment, the private sector could still be supported to create large numbers of jobs through private sector driven opportunities, in parallel with but independent of the efforts to improve the existing regulatory context. In addition, GEMS supported work toward transformational commercial business models (such as womens microretailing models and service providers to farmers), potentially enabling such services and models to reach millions of the poor. Finally, GEMS developed various techniques for supporting investors and investment that are being adopted both within and outside of Nigeria.
Overall Outcome Assessment Based on reviewing available final data from PCRs for GEMS 1, 3, and 4, the review team developed final 2017 aggregate numbers and cumulative figures for the GEMS suite.
The overall intended impact of the GEMS suite of programs was “to increase growth, income and employment, especially for poor men and women, in target markets in selected states and nationally.” Three indicators are used to assess these impacts (Table 6).
Table 6. Indicators Used to Assess Impact Impact Indicator Actual Actual Actual (2017) Target (2015) (2016) (2017) 1. Number of people recording 929,958 1,230,063 2,104,291 1,987,832 positive change in incomes 2. Aggregated change in cumulative 155,945,700 325,646,162 703,639,765 703,230,033 income (GBP) 3. Change in Employment (FTE Jobs) 12,223 17,544 32,432 26,318
These outcome data make it clear that the GEMS suite exceeded—and in the case of employment creation, substantially outperformed—its targets. This reflects strong performance by all three of the programs that ran to completion.
This positive view is reinforced by results at the outcome level, for which the goal is “to improve the performance and inclusiveness of selected market systems that are important for poor people.” This is monitored by five indicators, of which the aggregateable four are as follows in Table 7 (indicator 7 product quality was intended to be program-specific and hence not aggregated).
Page 55 of 66
The World Bank Nigeria - Growth & Employment (P103499)
Table 7. Outcome Indicators Outcome Indicator Actual (2014) Actual (2015) Actual (2017) Target (2017) 4. Number of firms with increased 20,919 47,129 591,397 296,030 sales 5. Increase in sales amongst 135,043,954 205,131,124 356,309,021 424,876,882 targeted firms 6a. Percentage of new or 29 percent 53 percent 65 percent 58 percent improved products and services, introduced through project facilitation, that are established in the market 12 months after project support has ended 6b. Percentage of new or 29 percent 50 percent 77 percent 58 percent improved regulations or reforms, introduced through project facilitation, that are established in the market 12 months after project support has ended
The GEMS suite significantly out-performed its target for indicator 4—thanks in particular to a strong performance by GEMS 4. The suite slightly underperformed for indicator 5. However, what is perhaps more interesting, and certainly more important are the data for indicators 6a and 6b. It is these indicators, which refer to the systemic changes the GEMS suite managed to achieve, that are (arguably) the more significant indicators since they pick up on the wider legacy of the program family. Given the size and scale of the issues that the GEMS suite aimed to tackle, the direct impacts of the programs were always going to be relatively small (albeit important) in the context of a country the size of Nigeria. Therefore the true aim of the program suite was to demonstrate approaches and ways of working— dynamic impacts—that would change the wider market and regulatory systems, thus enabling the programs’ impacts more broadly than simply in the states in which GEMS operated. The overperformance on both indicators—a significant out-performance in the case of indicator 6b— demonstrates that the suite will have had a dynamic impact on the situation in Nigeria broader than the immediate activities undertaken. This conclusion on the basis of the numbers is underpinned by the observations in the PCRs of both GEMS 3 and 4. The former for example, concludes that GEMS 3 was able to create models for reform, particularly in tax and land reform, which are replicable, and which are demanded in many more states than just the program’s focal ones.
Link to the GEMS closing review: https://devtracker.dfid.gov.uk/projects/GB-1-104190/documents
Page 56 of 66
The World Bank Nigeria - Growth & Employment (P103499)
ANNEX 8. IMPACT EVALUATION RESULTS
Firms received capacity building support that reflected “treatments” for experiment groups of firms to be tested through the Impact Evaluation. These included the following groups: (i) control—receiving no capacity building support, (ii) firm owners receiving in-person training (training), (iii) firms receiving consulting services (consulting), (iv) firms linked to human resource specialists working to insource the relevant skilled needed (insourcing), and (v) firms linked to companies with professionals in business services to outsource these skills (outsourcing). Firms were scored on their initial business practices on a score out of 10. Firms that scored 5 to 8 were offered one of the programs that the study evaluated, and the very small number of firms (<3%) that scored above 8/10 on initial business practices were fast-tracked to advanced consulting or grants and were also not in the impact evaluation. For the impact evaluation of the insourcing and outsourcing pilots, further restrictions required firms to have 2 to 15 full-time employees, and not have an existing employee other than the owner doing marketing and accounting. Every firm in Lagos and Abuja that were inducted in 2016 and satisfied the eligibility criteria for the pilot programs were included in the impact evaluation. Outside of Abuja and Lagos, all firms inducted in 2016 and scored between 5 and 8 out of 10 on baseline business practices were allocated to BDS or training or control and were also in the impact evaluation. There were several smaller batches inducted after 2016, but none received the full program and were too late to be included in the impact evaluation. Since the grants were only disbursed to the firms in early 2019 there was also no time to see any effect.
Page 57 of 66
The World Bank Nigeria - Growth & Employment (P103499)
The project component assessed by the impact evaluation was only launched in 2016 (and started in 2017), compared to the project’s approval in 2011 and eventual start in 2013 which is a significant lag after the PAD. The impact evaluation was designed to assess whether this component was helping in the PDO of improving growth and employment in Nigerian firms and sought out to measure growth in sales similar to the PAD results indicator. However, the impact evaluation focused more on improving business practices/management practices in these firms as an immediate step, which is not reflected the results indicators. This was due to timing and statistical power. The delay in implementing these components of the project limited the amount of time for the impact evaluation to trace impacts through into sales and employment. Hence, improvements in business practices are a first step that can be observed within shorter term horizons, and which literature has found to be strongly associated with firm growth over time. Second, because of sample size issues and heterogeneity amongst firms, the impact evaluation has more ability to detect changes in business practices than in sales and employment.
Evidence from the Impact Evaluation results shows that the insourcing, outsourcing, and consulting interventions all improved business practices in participating firms while training had a negligible impact. This was the first impact evaluation comparing various ways of improving firm capabilities. The project was able to provide interesting lessons that could be used by other PSD projects. For instance, insourcing and outsourcing interventions were more cost-efficient than BDS consulting at approximately half the cost. In contrast, there was no measurable impact of training. Insourcing and outsourcing led to firms innovating more; this is reflected through the introduction of new products and the improvement of those already existing. The impact evaluation shows that the point estimates are for a 0.2 worker per firm increase from insourcing, 0.4 worker per firm increase from outsourcing, 0.5 worker per firm increase from training, and 0.9 worker per firm increase from BDS. But none of these coefficients are statistically significant. Indeed, the sample size of the impact evaluation was smaller than anticipated, which combined with the substantial heterogeneity in firm size among firms, makes estimating a precise effect on employment difficult.
None of the interventions had any measurable impact on employment over the two-year horizon as measured. The coefficients are positive but not statistically significant (see table below). Because the GEM project was able to induct fewer firms than originally planned, the evaluation ended up with a sample size of 753 firms as opposed to the 2,000 intended to be recruited. This affected the measurement of results relating to profits and sales, given the heterogeneity among firms and large variances in these outcomes. Despite this, there is evidence that the outsourcing intervention and BDS led to sizeable improvements in sales and profits. The picture for insourcing is less clear—the point estimates are positive, but smaller than for outsourcing.
Page 58 of 66
The World Bank Nigeria - Growth & Employment (P103499)
ImpactTable 6: Impact on Sales on Sales and and Profits Profits Inside inside Lagos/AbujaLagos/Abuja (equivalent to log specifications ) (level specifications) I.H.S. I.H.S. I.H.S. I.H.S. Sales & Real Real Monthly Yearly Monthly Yearly Profits Monthly Monthly Sales Sales Profits Profits Index Sales Profits Panel A: Impacts in First Follow-up Survey (One-year after intervention started) Assigned to Insourcing 0.378 0.547 0.206 0.620 0.084 -50.069 -58.542 (0.405) (0.446) (0.564) (0.545) (0.078) (207.678) (67.161) Assigned to Outsourcing 0.242 0.826** 0.357 0.501 0.095 -150.729 -120.107* (0.395) (0.404) (0.541) (0.582) (0.070) (211.376) (62.258) Assigned to Training -0.573 -0.233 -0.288 0.202 -0.055 -242.811 -85.447 (0.482) (0.518) (0.584) (0.572) (0.085) (204.306) (66.850) Assigned to Consulting -0.172 -0.097 0.158 -0.635 -0.047 -218.515 -106.302* (0.432) (0.493) (0.558) (0.626) (0.084) (210.385) (64.525) Mean of Control Group 13.186 15.186 11.242 13.278 0.128 1289.720 389.443 Sample Size 670 670 670 670 670 670 670 P-value: all treatments zero 0.242 0.033 0.777 0.224 0.119 0.682 0.349 P-value: all treatments equal 0.144 0.031 0.640 0.146 0.075 0.714 0.687
Panel B: Impacts in Second Follow-up Survey (Two-years after intervention started) Assigned to Insourcing 0.798 0.705 0.652 0.471 0.124 152.496 38.753 (0.613) (0.658) (0.611) (0.690) (0.119) (295.423) (54.111) Assigned to Outsourcing 1.182** 1.266** 0.775 1.393** 0.228** 591.614* 35.907 (0.590) (0.584) (0.617) (0.589) (0.110) (357.674) (52.569) Assigned to Training 0.469 0.725 0.090 0.656 0.087 251.363 47.669 (0.634) (0.634) (0.652) (0.641) (0.117) (305.619) (53.887) Assigned to Consulting 1.079* 0.935 0.760 1.194* 0.182 395.373 80.332 (0.582) (0.604) (0.613) (0.626) (0.113) (310.539) (55.381) Mean of Control Group 11.620 14.088 9.925 12.274 -0.123 1211.135 220.213 Sample Size 678 678 678 678 678 678 678 P-value: all treatments zero 0.259 0.294 0.571 0.116 0.268 0.483 0.679 P-value: all treatments equal 0.593 0.681 0.684 0.304 0.518 0.598 0.890 Notes: Regressions control for stratification batch fixed effects and baseline value of outcome. Robust standard errors in parentheses. *, **, *** denote significance at the 10, 5, and 1 percent levels respectively. I.H.S. denotes the inverse hyperbolic sine transformation. Firms that are closed are coded as having zero profits and sales. Monthly sales is real (January 2018) sales in the past month; Yearly Sales are real sales in the past year; Monthly profits and Yearly profits are real profits in the past month and past year respectively. Sales and Profits Index is the average of standardized z-scores of these first four sales and profits measures.
Page 59 of 66
The World Bank Nigeria - Growth & Employment (P103499)
ANNEX 9. BIG PLATFORM
On the BIG Platform, businesses will be able to access online resources and enroll in various activities. The BIG Platform provides: 1) Online materials (software, templates, etc.) 2) Basic generic training (accounting, marketing) 3) Generic advanced training (online and face-to-face, which is a customized version of the IFC Business Edge training) 4) Training (and other resources) tailored to the businesses in the GEM clusters, based on recommendations of the ICGs and mainly face-to-face training, e.g., food safety, intellectual property rights for the music and film industry, sound editing 5) Consulting services (coaching, mentoring, and training) provided by 30 selected BDS providers throughout the country for 8–10 months 6) Insourcing 7) Outsourcing 8) Grants. Performance grants were accessible by reaching a certain number of points. Points were accumulated by firms through attendance and performance in each activity. 9) At an induction workshop, firms were characterized based on their level of business practices sophistication (based on lessons from Bloom and Van Reenen). 4: basic training, 5–8: randomized across the four groups (training, BDS, insourcing, outsourcing), 9: BDS.
Page 60 of 66
The World Bank Nigeria - Growth & Employment (P103499)
Page 61 of 66
The World Bank Nigeria - Growth & Employment (P103499)
ANNEX 10. DFID GEMS RESULTS AND STAKEHOLDERS
Table 9. DFID GEMS Project (Source: DFID Project Completion Report) GEMS 1 – Meat and Objective: Improve the livelihood of poor men and women in the meat and Leather leather sector by applying a “Making Markets Work for the Poor” (M4P) (2010–2015) approach. Summary of Results: GEMS 1 created many opportunities for other projects and private sector actors (including service providers engaged throughout the project), to continue on the work of the project. A number of the interventions developed by GEMS 1 were handed over to other projects (both within and outside the GEMS suite) to ensure that positive market system initiatives result in sustainable change. GEMS 2 – Objective: Strengthen the performance of market systems in the Construction and construction and real estate sector for these to perform more effectively Real Estate and offer increased opportunities (jobs and income) for the poor, and for (2010–2013) women in particular in the sector. Summary of Results: DFID Nigeria decided to close the project early (2013) based on assessing it was unlikely to achieve its objectives within the remaining project lifetime. A detailed lesson learning exercise was undertaken to understand the difficulties encountered by GEMS 2, with a report produced documenting the findings. GEMS 3 – Business Objective: Work with the private and public stakeholders at national, state, Environment and local government levels to make it easier to do business in Nigeria. This (2010–2017) was to be achieved through interventions focused on addressing land, business tax, and investment. Summary of Results: GEMS 3 developed comprehensive approaches to reform. In the areas of tax and land, the models became highly developed, including a focus on relevant regulation, training, public-private dialogue, and sensitisation. The investment workstream was not quite so well advanced, reflecting that it was a newer area of activity for GEMS 3. Nevertheless, considerable advances were made too. The team also focused on ensuring the sustainability of its activities and gave thought to understanding what brings about systemic change. This led to decisions to focus on achieving proof-of-concept in their intervention areas, and in the case of tax and land, these are now well-embedded in the relevant state ministries. Work with national-level organizations, such as the Joint Tax Board (JTB) and Nigerian Governors Forum (NGF), provided a good basis for the reforms to be sustained and rolled out into other states. GEMS 4 – Objective: Apply market strengthening principles and approaches to Wholesale and achieve market-led and pro-poor impact in the wholesale and retail sectors. Retail Summary of Results: While the project achieved and even exceeded a vast (2012–2017) majority of its output and outcome targets, the project experienced a number of challenges (see 2014 MTR) that limited results at the impact level. This was the result of initial delays in broadening the scope of activity, followed by another delay in focusing back on the successful intervention areas.
Page 62 of 66
The World Bank Nigeria - Growth & Employment (P103499)
Table 10. GEM Stakeholders and their Role GEM PIU Tendered the contract for monitoring the disbursement of grants and performance of the YouWin! awardees under World Bank procedures (AM 10/2014). Enterprise Contracted to provide training for YouWin! applicants in the form of a 3- Development Center day training (ISR #7). (EDC) Kaduna Business Screened BDS providers to evaluate their knowledge and capacity to talk School to “real” business owners, with support from an international BDS expert—Tim Rose from Growbridge (03/2016). PwC Grants management firm contracted in November 2015 to ensure that management of the grants is done by a third party (AM 10/15). KPMG Monitoring and supervision firm contracted to provide independent third- party supervision of the grant windows, training, and consulting services. The scope of work included data collection for a baseline to be conducted during Induction Workshops and follow-up survey, which was outsourced to TNS. Contract with KPMG signed on 02/01/2016 (AM 03/16). TNS Subcontracted by KPMG to conduct a baseline and follow-up survey for the grant windows, training, and consulting services (AM 03/16). Greenstratos Cluster Management Advisory firm recruited in June 2015 (AM 03/16). Intermel IT firm in charge of designing and developing the BIG Platform (AM 03/16).
Additional evidence on coordination: The Task Team recommends a better structure of the PIU with focal points—both reporting to the Project Coordinator and Assistant Project Coordinator—for the two main activities, namely the BIG platform (IT firm, grant management firm, monitoring and supervision firm, training programs) and all of the cluster activities (management of cluster specialists, ICGs, all cluster related activities)—AM 10/ 15.
Page 63 of 66
The World Bank Nigeria - Growth & Employment (P103499)
ANNEX 11. ORGANOGRAM
Page 64 of 66
The World Bank Nigeria - Growth & Employment (P103499)
ANNEX 12. ECONOMIC ANALYSIS
The economic analysis of this project is built on the analyses of cash flows generated by additional employment and revenues generated by firms that benefited from the GEM project. SMEs received grants and benefits from business development service providers trained by GEM. An attempt has been made to quantify the costs and benefits that are expected to accrue from these expansions, and the Net Present Value (NPV) and the Economic Rate of Return (ERR) for this has been calculated.
Due to the halting and extended nature of the project, it was difficult to conduct a proper NPV or ERR calculation. Instead, the economic analysis assumes that all disbursements two place in 2016- 2017 and all economic benefits took place between 2016-2019. The model assumed that each MSME hired two additional people (a common response from the focus group survey) and each employee was paid the annual Lagos civil servant minimum wage. Moreover, each MSME was assumed to have increased its revenue by 20% (another common response from the focus group survey) from a base revenue assumed to be equal to the official Nigerian cut off to qualify as a micro enterprise (5 million Naira).
The economic costs are incurred from the annual project disbursements which of the sake of simplicity were equally divided between 2016 and 2017. This results in negative net economic benefits for the first two years and positive net economic benefits in the last two years. The terminal value of the cash flows was calculated using the net economic benefits from jobs and additional revenue and a 14.5% discount rate.
At Closing Total Economic Benefits $ 87,192,312 Wages from jobs created $ 4,626,312 Additional revenue generated by MSMEs $ 82,566,000 Economic Costs (project disbursements) $ 66,853,193 Net Economic Benefits $ 20,339,119
Year 1 2 3 4 2016 2017 2018 2019 Disbursements spread over four years 33,426,596 33,426,597 $ - $ - All benefits accrued in the last four years 21,798,078 21,798,078 21,798,078 21,798,078
Net Economic benefits (11,628,519) (11,628,519) 21,798,078 21,798,078 Net Economics benefits plus terminal value -11628519 -11628519 21798078 162067864
Page 65 of 66
The World Bank Nigeria - Growth & Employment (P103499)
Net Economic benefits present value (10,155,911) (11,628,519) 21,798,078 21,798,078 Net Economic benefits plus terminal value present value (10,155,911) (11,628,519) 21,798,078 162,067,864
Economic Benefits without perpetuity NPV 21,811,726 ERR 42% Economic Benefits with perpetuity
NPV 162,081,512 ERR 143%
Page 66 of 66