Document of The World Bank FOR OFFICIAL USE ONLY
Public Disclosure Authorized Report No: ICR00004743
IMPLEMENTATION COMPLETION AND RESULTS REPORT IDA-5430-KG AND IDA-H934-KG
ON A
CREDIT IN THE AMOUNT OF SDR 23. 9 MILLION
Public Disclosure Authorized (US$36.8 MILLION EQUIVALENT)
AND
GRANT IN THE AMOUNT OF SDR 5.3 MILLION
(US$8.2 MILLION EQUIVALENT)
TO THE
KYRGYZ REPUBLIC
Public Disclosure Authorized FOR THE FIRST PHASE OF THE CENTRAL ASIA ROAD LINKS PROGRAM
August 11, 2020
Transport Global Practice Europe and Central Asia Region
Public Disclosure Authorized
This document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not otherwise be disclosed without World Bank authorization.
CURRENCY EQUIVALENTS (Exchange Rate Effective November 1, 2019)
Currency Unit = Kyrgyz Som (KGS) KGS 69.95 = US$1 US$ 1.380020 = SDR 1
FISCAL YEAR July 1 – June 30
Regional Vice President: Anna M. Bjerde Country Director: Lilia Burunciuc Regional Director: Lucio Monari Practice Manager: Karla Gonzalez Carvajal Task Team Leader(s): Muhammad Zulfiqar Ahmed
ABBREVIATIONS AND ACRONYMS ADB Asian Development Bank BCP Border Crossing Point CARs Central Asia Road Links CARs-1 First Phase of the Central Asia Road Links CAREC Central Asia Regional Economic Cooperation CPF Country Partnership Framework DEP Local Maintenance Unit (Dorojno-Ekspluatatsionnoe Predpriyatie) EIRR Economic Internal Rate of Return FM Financial Management GDP Gross Domestic Product GRM Grievance Redress Mechanism HDM Highway Development and Management Model HSWIM High-Speed Weigh-in-Motion ICR Implementation Completion and Results Report IFR Interim Financial Report IPIG Investment Project Implementation Group IRI International Roughness Index ISR Implementation Status and Results Report JICA Japan International Cooperation Agency LSWIM Low-Speed Weigh-in-Motion M&E Monitoring and Evaluation MOF Ministry of Finance MOTC Ministry of Transport and Communications (changed to MOTR during implementaiton) MOTR Ministry of Transport and Roads MTR Midterm Review NRRP National Road Rehabilitation Project OBI Osh-Batken-Isfana OBI UAD Osh-Batken-Isfana Roads Department (Upravlenie Avto Dorog Osh-Batken-Isfana) PAD Project Appraisal Document PDO Project Development Objective RAMS Road Asset Management System RAP Resettlement Action Plan RMD Road Maintenance Department SLA Service-level Agreement WIMS Weigh-In-Motion System(s)
TABLE OF CONTENTS
DATA SHEET ...... i I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES ...... 1 A. CONTEXT AT APPRAISAL ...... 1 B. SIGNIFICANT CHANGES DURING IMPLEMENTATION ...... 3 II. OUTCOME ...... 5 A. RELEVANCE OF PDOs ...... 5 B. ACHIEVEMENT OF PDOs (EFFICACY) ...... 5 C. EFFICIENCY ...... 9 D. JUSTIFICATION OF OVERALL OUTCOME RATING ...... 13 E. OTHER OUTCOMES AND IMPACTS (IF ANY) ...... 13 III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME ...... 14 A. KEY FACTORS DURING PREPARATION ...... 14 B. KEY FACTORS DURING IMPLEMENTATION ...... 14 IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME ...... 15 A. QUALITY OF MONITORING AND EVALUATION (M&E) ...... 15 B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE ...... 16 C. BANK PERFORMANCE ...... 19 D. RISK TO DEVELOPMENT OUTCOME ...... 22 V. LESSONS AND RECOMMENDATIONS ...... 23 ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS ...... 25 ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION ...... 32 ANNEX 3. PROJECT COST BY COMPONENT ...... 34 ANNEX 4. EFFICIENCY ANALYSIS ...... 35 ANNEX 5. BORROWER COMMENT AND SUMMARY OF BORROWER ICR ...... 39 ANNEX 6. SUPPORTING DOCUMENTS (IN PROJECT FILE) ...... 42
The World Bank First Phase of the Central Asia Road Links Program (P132270)
DATA SHEET
BASIC INFORMATION
Product Information Project ID Project Name
P132270 First Phase of the Central Asia Road Links Program
Country Financing Instrument
Central Asia Investment Project Financing
Original EA Category Revised EA Category
Partial Assessment (B) Partial Assessment (B)
Organizations
Borrower Implementing Agency
Ministry of Finance of the Kyrgyz Republic Ministry of Transport and Roads
Project Development Objective (PDO)
Original PDO The Central Asia Road Links (CARs) program is the result of a collaborative effort initiated by respective governments in the Central Asia region which has been developed as a regional, multi-phase program considered as a regional transformational project under IDA 16 and IDA 17. The program has the overall objective to increase transport connectivity between neighboring countries in Central Asia along priority cross-border road links whilst supporting improvements in road operations and maintenance practices.
The project development objective (PDO) of the First Phase of the Central Asia Road Links Program (CARs-1) is to increase transport connectivity between the Kyrgyz Republic and Tajikistan along priority cross-border road links in Batken Oblast whilst supporting improvements in road operations and maintenance practices. The road sections to be financed under CARs-1 prioritize connectivity between the Osh and BatkenOblast in the Kyrgyz Republic and Sugd Oblast in Tajikistan as they build lateral spurs from the Osh-Isfana axis to the Tajik border.
i
The World Bank First Phase of the Central Asia Road Links Program (P132270)
FINANCING
Original Amount (US$) Revised Amount (US$) Actual Disbursed (US$) World Bank Financing
36,800,000 31,074,244 27,510,828 IDA-54300
8,200,000 8,200,000 7,384,020 IDA-H9340 Total 45,000,000 39,274,244 34,894,848
Non-World Bank Financing 0 0 0 Borrower/Recipient 9,000,000 0 0 Total 9,000,000 0 0 Total Project Cost 54,000,000 39,274,244 34,894,848
KEY DATES
Approval Effectiveness MTR Review Original Closing Actual Closing 22-Apr-2014 14-Dec-2014 15-May-2017 30-Apr-2019 15-Nov-2019
RESTRUCTURING AND/OR ADDITIONAL FINANCING
Date(s) Amount Disbursed (US$M) Key Revisions 30-May-2015 .20 28-Jan-2018 18.37 Reallocation between Disbursement Categories 06-Mar-2019 23.12 Change in Loan Closing Date(s) 26-Sep-2019 25.37 Change in Loan Closing Date(s) 13-Nov-2019 30.25 Cancellation of Financing Reallocation between Disbursement Categories
KEY RATINGS
Outcome Bank Performance M&E Quality Moderately Satisfactory Moderately Satisfactory Modest
ii
The World Bank First Phase of the Central Asia Road Links Program (P132270)
RATINGS OF PROJECT PERFORMANCE IN ISRs
Actual No. Date ISR Archived DO Rating IP Rating Disbursements (US$M) 01 23-Sep-2014 Satisfactory Satisfactory 0
02 27-Apr-2015 Satisfactory Satisfactory .20
03 23-Dec-2015 Satisfactory Moderately Satisfactory .27
04 30-Jun-2016 Satisfactory Moderately Satisfactory 4.71
05 23-Jan-2017 Satisfactory Moderately Unsatisfactory 6.83
06 20-Jun-2017 Satisfactory Moderately Satisfactory 7.62
07 03-Jan-2018 Satisfactory Moderately Satisfactory 17.79
08 15-Oct-2018 Satisfactory Moderately Satisfactory 21.09
09 21-Apr-2019 Satisfactory Moderately Satisfactory 23.50
SECTORS AND THEMES
Sectors Major Sector/Sector (%)
Transportation 100 Public Administration - Transportation 10 Rural and Inter-Urban Roads 90
Themes Major Theme/ Theme (Level 2)/ Theme (Level 3) (%)
Economic Policy 40
Trade 40
Trade Facilitation 40
Private Sector Development 20
Regional Integration 20
iii
The World Bank First Phase of the Central Asia Road Links Program (P132270)
Urban and Rural Development 40
Rural Development 40
Rural Infrastructure and service delivery 40
ADM STAFF
Role At Approval At ICR
Regional Vice President: Laura Tuck Anna M. Bjerde
Country Director: Saroj Kumar Jha Lilia Burunciuc
Director: Laszlo Lovei Lucio Monari
Practice Manager: Juan Gaviria Karla Gonzalez Carvajal
Task Team Leader(s): Cordula Rastogi Muhammad Zulfiqar Ahmed
ICR Contributing Author: Muhammad Zulfiqar Ahmed
iv
The World Bank First Phase of the Central Asia Road Links Program (P132270)
I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES
A. CONTEXT AT APPRAISAL
Context
1. Country background. At the time of project appraisal, the Kyrgyz Republic remained one of the most remote and poorest countries in the Central Asia region. The Kyrgyz economy’s rate of growth had been volatile, reflecting its heavy dependence on a few sectors of economy, such as gold production as well as re-exports. Numerous domestic and external disorders, including two revolutions since 2005, had also significantly slowed down economic growth. Gross domestic product (GDP) growth averaged just over 4 percent annually during 2003–2012, with varying year-to-year rates ranging from 8.5 percent in 2007 to −0.9 percent in 2012. Characterized by significant informal activities, the Kyrgyz economy also relied on agriculture, which accounted for about 20 percent of GDP, while services generated about 50 percent of GDP, out of which transport hads a significant share.
2. Sector background. For a landlocked country such as the Kyrgyz Republic, the road transport system has always played a major role in the national economy as 95 percent of passengers and 60 percent of freight were transported by the network. Given the dependence on road transport, improving the road link between the Kyrgyz Republic and the Tajikistan along the Osh-Batken-Isfana (OBI)-Tajikistan border road had become one of the priorities for the Kyrgyz Government. The rehabilitation of this priority border link was expected to contribute to the region’s potential to participate in trade and supply chains as well as improve regional development and integration with neighboring Tajikistan. The OBI-Tajikistan border road was one of the six strategic corridors under rehabilitation, which together carried most of the traffic and formed about 4,163 km (international roads) of the Republic’s road network. While several efforts were undertaken to rehabilitate road links, road maintenance management in the Kyrgyz Republic continued to lag. Financing road maintenance at the required level and improving current maintenance practices remained a challenge.
3. Higher-level objectives to which the project contributed. The Central Asia Road Links (CARs) Program was considered as a regional transformational project, which was developed through a collaborative effort initiated by respective governments in the Central Asia Region. The project was complementing the ongoing efforts of the Central Asian countries as outlined in the Transport and Trade Facilitation Strategy and Action Plan for 2008–2017 and endorsed by the Central Asia Regional Economic Cooperation (CAREC). The strategy envisioned the rehabilitation of six strategic transport corridors in the Central Asia Region based on their impact on economic growth and poverty reduction as highlighted by the development framework for the CAREC program 2011–2020 (CAREC 2020).
4. The First Phase of the Central Asia Road Links (CARs-1) program was also fully aligned with the country’s and the Bank’s priorities. The project was designed to support the Government’s long-term development objective of improving transport connectivity and logistics as stated in the National Sustainable Development Strategy (2013–2017). The proposed project was also fully consistent with the Country Partnership Strategy for the Kyrgyz Republic for FY14–FY17 by supporting one of the areas of engagement related to helping reduce extreme poverty and promoting shared prosperity through support for improved governance. Improving inadequate transport links in poor rural areas such as the Batken
Page 1 of 42
The World Bank First Phase of the Central Asia Road Links Program (P132270)
region was to contribute to unlocking the region’s ability to participate in trade and develop larger value chains, in particular in the field of agricultural products marketable to urban areas or the agro-industry. This was to lead to important social and economic improvements of the underserved regions and populations who earlier had limited access to markets and trade opportunities. Additionally, the road links targeted by the program were considered important trade and transit routes of further regional and local importance as indicated by the high traffic volumes (5,000–10,000 vehicles per day) experienced on the main road link in Osh and Khujand region.
Theory of Change (Results Chain)
5. The Project Appraisal Document (PAD) had no original Theory of Change. Figure 1 is the reflection of committed Project Development Objectives (PDOs), outcomes, and outputs achieved as a result of the approved project activities.
Figure 1. CARs-1 - Theory of Change
Project Development Objectives (PDOs)
6. The PDO was to increase transport connectivity between the Kyrgyz Republic and Tajikistan along priority cross-border road links in Batken Oblast, whilst supporting improvements in road operations and maintenance practices.
Page 2 of 42
The World Bank First Phase of the Central Asia Road Links Program (P132270)
Key Expected Outcomes and Outcome Indicators
7. Two expected outcomes under the project were to increase transport connectivity between the Kyrgyz Republic and Tajikistan along priority cross-border road links in Batken Oblast; and support improvements in road operations and maintenance practices.
8. Achievement of the PDO was to be measured through five PDO-level indicators: (1) volume of freight through Kyzyl-Bel/Guliston and Kairagach/Madaniyat border crossing points; (2) market accessibility index of Batken Oblast; (3) road users satisfied with maintenance of the Pulgon-Batken road section; (4) annual budget allocation for routine (summer and winter) maintenance for road sections in sustainable condition along the corridor under OBI UAD; and (5) strategic plan on transport control discussed at the regional level (CAREC) and approved by the MOTC.
Components
9. Component 1: Rehabilitation of Priority Road Links in Batken Oblast (total cost US$51.30 million, including IDA financing of US$42.30 million). The component was to finance the provision of works and consultants’ services for the rehabilitation of about 56 km of road sections in Batken Oblast along and within close vicinity of the OBI road corridor. Those road sections were expected to link the Kyrgyz Republic with its neighboring countries and include the sections; (a) the Isfana- Kairagach/Madaniyat up to the border crossing (36.08 km); (b) Batken-Tortkul (14.30 km); and (c) the Batken-Kyzyl-Bel/Guliston up to the border crossing (6.11 km).
10. Component 2: Improvement of Road Operations and Maintenance Practices (total cost US$1.70 million, including IDA financing of US$1.70 million). This component was to finance the provision of goods and consultants’ services for the improvement of road operations and maintenance practices. Improvement of maintenance practices included the establishment of a Service-level Agreement (SLA) between the Ministry of Transport and Communications (MOTC) and the Osh-Batken-Isfana Roads Department (OBI UAD). Under this agreement, OBI UAD was to become responsible for routine maintenance (summer and winter) and emergency works for the entire OBI corridor, with special focus on those sections which had been rehabilitated and/or are in sustainable condition. The component was to also support the OBI UAD and its Local Maintenance Units (DEPs) along the OBI road corridor, with a set of equipment and expose the OBI UAD management to international practices on maintenance techniques through the hiring of internationally recruited consultants.
11. Component 3: Project Management and Implementation (total cost US$1.00 million, including IDA financing of US$1.00 million). This component was expected to finance the provision of goods, consultants’ services, training, and operating costs, including financial management (FM) software and a financial audit to support project coordination, implementation, and management.
B. SIGNIFICANT CHANGES DURING IMPLEMENTATION
12. The PDO remained unchanged during the life of the project. The project was restructured five times as presented in table 1. Apart from extending the end target date, none of restructurings introduced changes to the Results Framework.
Page 3 of 42
The World Bank First Phase of the Central Asia Road Links Program (P132270)
Table 1. History of Project Restructurings Restructuring No. Date Reason for Restructuring Type 1 May 30, 2015 Level II Extension of the lapsed dated legal covenant 2 January 28, 2018 Level II Reallocation of project funds between disbursement categories 3 March 6, 2019 Level II Extension of the project closing date 4 September 26, 2019 Level II Extension of the project closing date Cancellation and reallocation of project funds between 5 November 13, 2019 Level II disbursement categories
Rationale for Changes and Their Implication on the Original Theory of Change
Rationale for Changes
13. First restructuring (May 2015). This restructuring extended the lapsed dated legal covenant in the Financing Agreement for the employment of a road engineer, a contract administration specialist, and a lawyer from March 14, 2015, to September 14, 2015. The extension of 9 months was requested to provide additional time for the Investment Project Implementation Group (IPIG) to complete recruitment for the above staff. The execution of the dated covenant was delayed due to the late initiation and delays in the selection process by the IPIG.
14. Second restructuring (January 2018). This restructuring reallocated SDR 4 million (US$5.7 million) in savings from Category 1 “Works” to Category 2 “Goods, Non-Consulting Services, Consulting Services, Training and Incremental Operating Costs, including Audit1”. The reallocated funds were to be used to extend a pilot on Weigh-In-Motion Systems (WIMS) to three (3) additional locations as well as to procure needed road maintenance equipment for UAD in Batken Oblast and other relevant consulting services to support project management and implementation.
15. Third restructuring (March 2019). The restructuring extended the original project closing date by 5 months from April 30, 2019 to September 30, 2019. The extension was to allow the contractor to complete rehabilitation works on the remaining 7 km of the total 56 km. Due to severe weather conditions in the autumn of 2018 and two landslides occurring during the same period on the road section, the contractor was unable to complete the asphalt laying works by the initially expected completion date.
16. Fourth restructuring (September 2019). This restructuring extended the closing date of the project to November 15, 2019. The extension was granted to allow for the road maintenance equipment to be delivered, inspected, and accepted before the project closure.
17. Fifth restructuring (November 2019). This restructuring partially cancelled and reallocated project funds under both credit and grant agreements in view of the closing date of the project. The restructuring cancelled credit proceeds equivalent to SDR 3,5 million from Category (1) and reallocated credit proceeds equivalent to SDR 905,739 from Category (1) to Category (2); and reallocated grant proceeds equivalent to SDR 16,000 from Category (1) to Category (2).
1 Hereinafter referred to as Category 1 and Category 2 respectively.
Page 4 of 42
The World Bank First Phase of the Central Asia Road Links Program (P132270)
Implication of Project Changes on the Original Theory of Change
18. As stated earlier, the PAD had no original Theory of Change. The changes carried out under the project restructurings had no implications on the PDOs or the Theory of Change structure of inputs and outcomes reflected in Figure 1.
II. OUTCOME
A. RELEVANCE OF PDOs
Assessment of Relevance of PDOs and Rating
19. Relevance of the PDOs is assessed as High. Increasing connectivity to improve access of people and businesses to local and global services, markets and opportunities across borders was a priority of the policies and investments of the governments of the Central Asian Republics, and it remains an active agenda todate. Therefore, the project objectives of CARs-1 continue to be relevant to the current priorities of Kyrgyz Republic for greater trade integration, regional development, and improved governance and sustainability, as defined in the long-term sustainable development strategy, embodied in Vision 2040. The project objectives: (1) to increase transport connectivity, and (2) support improvements in road operations and maintenance practices are well aligned with the priorities laid out in the Government’s mid-term development plan for 2018–2020, ‘Unity, Trust, Creation’ and the priorities defined in the Government’s Main Development Directions for the Transport Sector 2016–2025 on improving connectivity by closing infrastructure gaps and preserving and maintaining road assets.
20. The PDO to increase transport connectivity between the Kyrgyz Republic and Tajikistan along priority cross-border road links is consistent with the World Bank’s Country Partnership Framework FY19– FY22 (CPF). The project supports the CPF’s Focus Area 2: Raise productivity and build connectivity and Objective 6: Build transport connectivity, as well as, indirectly, the CPF’s Focus Area 3: Enhance economic opportunities and resilience and Objective 8: Support regional development. The first CARs project has fully contributed towards development of the subsequent series of projects in both Kyrgyz Republic and Tajikistan to increase connectivity and enhance opportunities for regional development. At a program level, the regional corridor development is continuing to show good progress. CARs-3 in Kyrgyz Republic and CARs-2 in Tajikistan are under implementation; CARs-4 in Tajikistan was recently approved, while CARs-5 (proposed to include Uzbekistan in addition to the current two countries) appears to be a prospective operation. The program has complemented well the government’s efforts to leverage other financing to support regional connectivity. Most notable mention is the Asian Development Bank (ADB) funded Regional Improvement of Border Services Project (RIBS).
B. ACHIEVEMENT OF PDOs (EFFICACY)
21. The achievement of the PDO is assessed as Substantial.
Page 5 of 42
The World Bank First Phase of the Central Asia Road Links Program (P132270)
Assessment of Achievement of Each Objective/Outcome
22. Assessment of achievement of Objective 1: Increase transport connectivity between the Kyrgyz Republic and Tajikistan along priority cross-border road links in Batken Oblast. The achievement of this objective is assessed as Substantial. According to the PAD, the connectivity part of the PDO was to be measured through the following outcome indicators: (a) volume of freight through Kyzyl-Bel/Guliston and Kairagach/Madaniyat border crossing points; (b) market accessibility index of Batken Oblast, and (c) strategic plan on transport control has been discussed at the regional level (CAREC) and approved by the MOTC. Regarding the indicator on the strategic plan, it appears to have been an inconsistency in the PAD, as this indicator is more appropriate to measure the second PDO on improving road operations and maintenance. Therefore, for the purpose of the Implementation Completion and Results Report (ICR), this indicator is considered under the second PDO.
23. Achievement of PDO-level indicators:
• Volume of freight through Kyzyl-Bel/Guliston and Kairagach/Madaniyat border crossing points. This indicator was incorrectly prepared and was not achieved. The baseline value for this indicator was presented as 500,000 tons/year with a target value of 550,000 tons/year. The actual achieved value at completion was 166,023 tons/year. This indicator was not achieved for several reasons: (a) the baseline value was substantially overestimated, which also affected the targeted end value (10 percent increase); (b) at certain times, the border crossing points were closed due to unsettled border issues, as well as due to the hot climatic conditions, resulting in a government decree where no trucks were allowed to pass the border crossing point (BCP) during daytime. In 2017, the implementing agency recommended to the World Bank to revise the target value; however, this did not take place.
• Market accessibility index of Batken Oblast. There is not enough information to assess whether this indicator was achieved. The World Bank team calculated the baseline for this index as 4.60 with a target value at project completion to be 5.50. Project documents contain no details on the formula on how the baseline was calculated, and the IPIG was not trained on how to calculate the index. The ICR attempted to calculate this index by reaching out to the DEC team, who authored the Market Accessibility Index in the Kyrgyz Republic, as referenced in the PAD, but no feedback was received.
24. Intermediate indicators. As both above PDO level indicators were considered incomplete in achievement for the reasons given above, the ICR has looked to the intermediate indicators and other evidence to evaluate this objective. The following intermediate indicators were considered in assessing the improved connectivity objective :
• Road user costs by truck along the project road sections and in project area (US$/veh-km). This indicator was fully achieved with the baseline value of 0.52 having decreased to the target value of 0.42.
• Roads rehabilitated, Non-rural (km). This indicator was fully achieved, the target value was 56 km, and the actual achieved value at completion is 56.40 km.
Page 6 of 42
The World Bank First Phase of the Central Asia Road Links Program (P132270)
25. In addition to the above formal indicators, the ICR considered the decrease in travel time on project roads (although this was not part of the Results Framework) as a proxy to measure improved transport connectivity. Tables 2 and 3 demonstrate decreases in travel time between the residential areas within the project, thus supporting the conclusion that the connectivity along the project roads has improved.
Table 2. Lot 1 Average Travel Time in 2016 versus 2019
Site Service Car (min) Pickup (min) Car (min) % % % 2016 2019 2016 2019 2016 2019 change change change Isfana-Sulukta 45 30 33.33 30 20 33.33 20 15 25.00 Sulukta-Koltso 60 45 25.00 45 35 22.22 35 30 14.29 Koltso-Kairagach 20 25 <25.00> 25 15 40.00 15 10 33.33
Average 11.11 31.85 24.21 Note:
Site Service Car (min and hours) Pickup (min and hours) Car (min and hours) % 2016 2019 % change 2016 2019 % change 2016 2019 change Batken-Kyzyl- 20 16 20.00 15 10 33.33 15 10 Bel 33.33 Batken-Tortkul 25 20 20.00 20 15 25.00 20 15 25.00 Batken-Isfana 2:40 2:15 15.63 2 2 0.00 1:45 1:30 14.29 Average 18.54 19.44 24.21
26. Thus, in the absence of available data for the objective’s PDO indicators, and considering the choice of indicators and the over/under estimation of baseline and target values, the ICR considered more traditional indicators for connectivity, such as decrease in user costs and travel time, which were fully achieved through the rehabilitation of 56.4 km of OBI-Tajikistan border road links. Based on these results, it is fair to conclude that the connectivity along the project roads has improved.
27. Assessment of achievement of Objective 2: Support improvements in road operations and maintenance practices. The achievement of this objective is assessed as Substantial. According to the PAD, the outcome indicators related to sustainability of road operations and maintenance were the following: (a) road users satisfied with maintenance of the Pulgon-Batken road section and (b) annual budget allocation for routine (summer and winter) maintenance for road sections in sustainable condition along the corridor under OBI UAD. As discussed earlier, the ICR also considered an indicator on a strategic plan on transport control under this part of the PDO. In addition, there was an important indicator to help measure the improvements in road operations and maintenance, that is, compliance by OBI UAD with service level criteria as specified in the Service-Level Agreement. This indicator was not an outcome indicator but an intermediate-level indicator. It was also utilized in the analysis of efficacy of this project objective considering its importance to the PDO.
28. PDO-level indicators
Page 7 of 42
The World Bank First Phase of the Central Asia Road Links Program (P132270)
• Road users satisfied with maintenance of the Pulgon-Batken road section. No road user data could be collected for this particular road section at the end of the project to allow the ICR to assess whether road users were satisfied with the maintenance of the Pulgon-Batken road section. The satisfaction of road users was to be assessed annually using a mobile application Zholdor.kg, which was developed by a local IT company with support from the World Bank. It was a survey tool to collect road user feedback, and a data aggregation website to generate survey results. In later years, significant effort was required to institutionalize Zholdor.kg. However, there were no financial nor human resources nor motivation from the Road Maintenance Department (RMD) of the DDH in Bishkek to respond to the feedback received from the beneficiaries in a knowledgeable and timely manner. Eventually, since it was a one-way communication channel, beneficiaries stopped submitting their feedback. The beneficiary survey and user satisfaction were not measured at the end of the project. In the absence of such specific survey data along the Pulgon-Batken section of the OBI corridor, the use of transport cost and travel time reduction along the entire road corridor (including the relevant sections) was analyzed as part of ex-ante economic evaluation. In the absence of a survey, that data was considered as a proxy for evidence of road user satisfaction since transport cost reduction and travel time reduction are an essential factor for user satisfaction.
• Annual budget allocation for routine (summer and winter) maintenance for road sections in sustainable condition along the corridor under OBI UAD. The Results Framework target shows this indicator as achieved, with the baseline value of US$80,000 increasing to US$242,489, which exceeds the end target value by 21 percent. However, analysis conducted by an international consultant, who supported the implementation of this component, with input from OBI UAD, concluded that the amount received by OBI UAD for maintenance was not sufficient to perform the needed level of maintenance as specified in the SLA. There was, however, a substantial increase in the maintenance budget allocation which is a very significant improvement, as well as providing a strong foundation for continued policy dialogue on maintenance financing in the future.
• Strategic plan on transport control has been discussed at the regional level (CAREC) and approved by the MOTR. This indicator is considered partially achieved. The project has financed the development of a comprehensive strategic plan on transport control, with the support of an international consultant. The MOTR, however, approved it as ‘Concept for a Strategic Plan on Transport Control’ in July 2019. This concept document forms the basis of formulating main directions in the axle-load control system for future work; and based on its recommendation, MOTR successfully installed eight (8) high-speed weight-in-motion systems (HSWIMS) to support the overload control mechanism in the country. This achievement was highly appreciated by the MOTR and is considered to support the transport control mandate, which has laid a foundation for sustainable road operations and maintenance. At the time of project completion, the document had not yet been discussed at the regional level (CAREC).
29. Intermediate indicators
Page 8 of 42
The World Bank First Phase of the Central Asia Road Links Program (P132270)
• Compliance by OBI UAD with service level criteria as specified in the Service-Level Agreement. This indicator was partially achieved due to multiple challenges with the implementation of the SLA. The SLA was not very clear on its definition of roles and responsibilities; financing for maintenance was not sufficient; and although the required routine maintenance equipment was procured and provided to the UADs, it was towards the end of the project. The SLA was signed for one year and was never re-signed. Despite these facts, OBI UAD did carry out limited routine maintenance on the road, using old equipment and funds allocated to the old road section (rehabilitated under the previous Bank financed National Road Rehabilitation Project (NRRP)). Notwithstanding these implementation challenges, Service Level standards have been set in the SLA; the project has laid the required institutional and resource base for sustainable maintenance; and the required equipment was ultimately provided, thereby delivering the objective of supporting improvements in road operations and maintenance practices, albeit, late in the project.
Justification of Overall Efficacy Rating
30. The overall efficacy rating is assessed as Substantial. In assessing the efficacy of each PDO, equal importance of objectives was assumed, as no indication on relative importance of either of the objectives was presented in project documents. The achievement of both PDOs is assessed as Substantial as reduction in travel time and transport cost have been achieved; and the foundations for sustainable maintenance, increasing maintenance budget, acquiring maintenance equipment for the direct labor, and installing the overloading control system have been accomplished. Although some gaps remain, overall efficacy is assessed as largely achieved and is rated Substantial.
C. EFFICIENCY
Assessment of Efficiency and Rating
31. The project’s efficiency is rated Modest. The assessment of Efficiency was borderline (approaching substantial) but ultimately was assessed as Modest due to the following reasons. In assessing the project’s efficiency, the ICR considered the economic analysis of the project and aspects of design and implementation that either contributed to or reduced efficiency. The largest, overarching activity of the project design met efficiency standards; however, there were several other issues that may have had a negative impact on efficiency during implementation. These are discussed further below.
32. Economic analysis. CARs-1 financed two contract packages of the overall OBI road corridor, totaling US$34.1 million of investments. The appraisal economic evaluation for the project had determined that the return on the investments for the road sections to be financed under CARs-1 was satisfactory with an overall EIRR of 16.6 percent. The ex post return on the investments under the CARs- 1 is satisfactory with an overall EIRR of 16.2 percent for the two road sections. The ex-ante economic analysis was done using Highway Development and Management Model (HDM-4) given the actual rehabilitation cost of the sections already rehabilitated, the road links to be financed under the project, and current road condition, traffic levels, and GDP growth projections for the Kyrgyz Republic. The ex- ante economic analysis was conducted for the entire OBI-Tajikistan border road through the analysis of 12 separate road sections.
Page 9 of 42
The World Bank First Phase of the Central Asia Road Links Program (P132270)
33. Table 4.2 in Annex 4 presents the estimated financial construction costs for the OBI corridor road sections evaluated at appraisal and the contract and actual financial construction costs for the road sections evaluated for the ex post economic evaluation, in US$ millions, for which there is a contract. On average, the contract costs are 3.7 percent higher than the appraisal estimates, the actual costs are 1.9 percent higher than the contract costs, and the actual costs are 5.3 percent higher than the appraisal estimates.
34. Table 4.3 of Annex 4 presents the traffic, in vehicles per day, collected in 2012 at appraisal, the traffic in 2019 as estimated at appraisal based on the estimated traffic growth rate defined at appraisal, and the actual traffic in 2019 based on recent traffic counts. On average, the actual 2019 traffic is 22 percent lower than estimated at appraisal.
35. Table 4.4 of Annex 4 presents the ex-ante EIRRs, presented in the PAD, and the resulting EIRRs of the ICR ex post economic evaluation for the overall OBI road corridor. The ex post EIRRs are in general lower than the appraisal EIRR estimates. The overall ex post EIRR of the OBI road corridor is 13.2 percent, which can be compared with the 24.0 percent estimated at appraisal. Overall, the OBI road corridor yielded lower economic return than the one estimated at appraisal, but nevertheless the corridor yielded a satisfactory rate of return.
36. The ex-ante economic evaluation determined that the appraisal EIRR for the road sections to be financed under CARs-1 (M13-8 and ISF-2) was satisfactory with an overall EIRR of 16.6 percent. Table 4.5 of Annex 4 presents the EIRRs estimated at appraisal and the ex post EIRRs for CARs-1 financed road sections. The ex post EIRRs under the CARs-1 are satisfactory with an overall EIRR of 16.2 percent for the two packages.
37. Aspect of design and implementation that also contributed to efficiency are described in the following paragraph.
38. Consultancy to support the institutional development component. The project’s design included a long-term engagement of an international and a local consultant to support multiple institutional development activities under Component 2: Improvement of Road Operations and Maintenance Practices. This proved to be an efficient use of resources, as the consultants supported activities that were entirely new to be implemented in the country. Specifically, this included the work on (a) road asset preservation and maintenance on the SLA between OBI UAD and the MOTC; (analysis of the legal, technical, and institutional framework; proposal of typical routine maintenance works and development of unit rates for typical works; prioritization and programming of works; and monitoring of the SLA in OBI UAD) and (b) weight and dimension control (development of a strategic plan on weight and dimension control and provision of technical support during the installation and implementation of HSWIM systems at two pilot locations). A substantial amount of groundwork has been done both at the central and local levels to continue strengthening its road operations and maintenance practices.
39. Aspects of design and implementation that reduced efficiency are described in the following paragraphs.
40. Project management. Project management is a key aspect in implementing project activities. At the start of the Project the IPIG consisted of 26 full-time experienced staff. The project management of
Page 10 of 42
The World Bank First Phase of the Central Asia Road Links Program (P132270)
the IPIG under the previous NRRP was assessed as Satisfactory; the same staff were to implement the proposed CARs-1. However, in accordance with the Order of the Minister of Transport and Roads of the Kyrgyz Republic dated May 23, 2017 the IPIG was split into the World Bank IPIG and the Asian Development Bank IPIG in order to increase the effectiveness of IPIG performance. However, the position of Financial Manager for World Bank IPIG remained vacant throughout the CARs-1 project implementation period. Key position of Director became vacant as of January 2019, in addition to this the key positions such as Road Engineer, Environmental Specialist and Contract Administration Specislists were not in place. The government adverstized several procurement notices for the positions of Road Engineer, Financial Manager and Environmental Specialist, however they remained unsucceful in finding appropriately qualified candidates. The procurement tenders for the positions of Director, Financial Manager and Environmental Specislists were conducted but were cancelled for various reasons such as participation of weaker candidates or later due to approaching project closure. Given the situation with insufficient number of IPIG staff, the project still managed to implement and complete all component activities.
41. Implementation Delays and Ownership by the MOTR. The project has seen a high turnover of the leadership of the MOTR, with five ministers leading the MOTR during the five-year project implementation. This caused significant delays in project implementation, as decision-making, strategic directions, and oversight of the IPIG, which was struggling with internal staffing issues for the most part of the project, were compromised. The main inefficiencies during the project implementation were observed in delays in procurement and non-completion of construction works that affected disbursements, impacting negatively on the contract management. For instance, the contract for the supply of road maintenance equipment, initiated at the end of the project but the equipment was deliverd late and this fact was one of the reasons for an extension of the project. The delays were primarily due to the overall lack of ownership by the MOTR that had frequent changes at the highest level, which further led to a decrease in capacity of the IPIG in last one and half year. It can be further attributed to the overall country authorizing atmosphere, which remained impacted by governance issues and stern accountability repercussions. The situation potentially led to fear of government oversight institutions and reluctance to take responsibility for decisions as several government officials at ministerial level had to resign or face criminal investigations. This often resulted in protracted evaluations, suspending decisions taken, and reluctance in contract award and signature, etc.
42. Some delays were observed with provision of government co-financing for works contract usually at the beginning of fiscal years, while the level of the co-financing was close to the required level typically at the end of the fiscal year. The project continued to suffer from extended reviews by the government and delays in progress payments to contractors and consultants for completed services or works.
43. Approval of the Strategic plan for transport control as a Concept for axle-load control. Axle-load control is an important aspect preventing road deterioration, alongside the initial quality of the construction and road maintenance. Overloaded vehicles is a big concern in the Kyrgyz Republic, and the project’s Component 2: Improvement of Road Operations and Maintenance Practices envisaged limited technical assistance to support the MOTR in reviewing regional standards, norms, and parameters on vehicle (truck) weight and axle load limits and tariff setting; developing a strategic plan for the institutional arrangements for transport control; and setting up of an axle-load control system and methods for enforcement of axle load limits. This plan was to be discussed with CAREC at regional level and adopted by the MOTR, however it was adopted as a Concept only near project closing.
Page 11 of 42
The World Bank First Phase of the Central Asia Road Links Program (P132270)
44. HSWIM pilots. Originally, the project did not envisage any goods to support axle-load control initiative, only limited technical assistance to analyze the situation as a starting point. However, before the project became effective, the MOTC requested the World Bank to use funds that were earmarked for procurement of road maintenance equipment in the amount of about US$1 million towards procuring HSWIM equipment for two pilot locations in Kemin and Sosnovka. Upon such request, the World Bank’s team considered the procurement of HSWIM equipment in line with the PDO and agreed to repurpose the project funds under Component 2: Improvement of Road Operations and Maintenance Practices toward HSWIM equipment. This left US$100,000 for road maintenance equipment, which was just enough to procure a disposal truck and a front loader, instead of 17 units of equipment as agreed during the project appraisal.
45. The implementation of the first two HSWIM pilots was challenging on many fronts. There were technical design issues related to the physical layout of the stations and drivers’ behavior; there were numerous technological challenges related to the absence of power and internet, system architecture, and software components to improve the reliability of the system and to integrate the new HSWIM systems with the existing Low-Speed WIM systems and with the licensing bodies to ensure future possibility to use the system both for manual and automatic enforcement; last but not least there were challenges of institutional capacity, requiring the restructuring of the organizational structure, with eventual involvement of the private sector, to improve responsibility, accountability, and awareness of the local staff in terms of fulfilling their mission, with the relevant training and accompanying capacity- building measures.
46. This being a pilot HSWIM installation, all these challenges were considered normal. The international consultant evaluated the pilots documenting all the lessons learned for future applications as well as developing recommendations on how to address the above-mentioned challenges. All these recommendations were fully integrated and reflected in detail in the strategic plan on transport control issued in January 2018. Following the installation of the HSWIM2 in the first two locations, the project financed additional six (6) HSWIM stations, with a total cost of EUR 3.8 million. While axle-load control mechanism played an important role and is relevant to the PDO, scaling-up of the initial pilot of two (2) stations to eight (8) was done without implementing the relevant recommendations for the required institutional setup. The project was not restructured to make the procurement and installation of HSWIM an integral part of project design to support the achievement of the PDO.
47. Road maintenance equipment. The decision early in the project to cut the financing of road maintenance equipment from US$1million to US$100,000 was not consistent with the project design. However, the MOTC assured the World Bank that it had enough maintenance equipment in other oblasts, that were receiving support from Japan International Cooperation Agency (JICA), and that it would transfer that equipment to the Batken Oblast for maintenance of the OBI corridor, which was not accomplished as planned. This equipment was essential for OBI UAD to deliver the required maintenance as agreed upon in the signed SLA between the MOTC and OBI UAD. While eventually savings under the project were used to procure nearly 50 units of road maintenance equipment costing approximately US$3 million (three times the original amount), this did not happen until the project was near closing.
2 The HSWIM were installed in Kemin, Sosnovka, Kok-Talaa, Karatai, Jany-Aryk, Chaldovar, Ak-Tilek, and Chon-Kapka.
Page 12 of 42
The World Bank First Phase of the Central Asia Road Links Program (P132270)
D. JUSTIFICATION OF OVERALL OUTCOME RATING
48. The overall outcome rating of the project is assessed as Moderately Satisfactory. This is derived by combining the following assessments: Relevance - High; Efficacy - Substantial; and Efficiency - Modest.
E. OTHER OUTCOMES AND IMPACTS (IF ANY)
Gender
49. The project was approved in 2014, when the gender tagging process was non-existent in Bank projects and was limited to gender flag selection done by the project team. This process did not require a confirmation through the Gender Tag Assessments done by the Gender Group. At appraisal, the team chose one gender flag, which was to support analysis and/or consultation on gender-related issues. The PAD briefly mentioned gender in relation to the beneficiary population, whereby the percentage of females on the total direct beneficiary population was identified at 50 percent. The PAD also stated that while women and men were to equally benefit from the project investments, citizen engagement and beneficiary consultations to be conducted during project implementation were to include evaluation of gender impacts. The PAD also planned to monitor some of the key social and economic benefits that were expected from the project, using gender-disaggregated indicators. There were no project indicators disaggregated by gender.
Institutional Strengthening
50. No institutional strengthening was carried out other than the institutional strengthening activities completed under the project. The processes relating to the development of the Service-level Agreements for road maintenance activities to be undertaken by the OBI UAD, an increased budget for road maintenance, and the pilots for axle load control by installation of HSWIMS may be attributed as aspects of supporting institutional strengthening under the project.
Mobilizing Private Sector Financing
51. No private sector financing was mobilized.
Poverty Reduction and Shared Prosperity
52. At preparation, the PAD mentioned how the CARs program would contribute to the twin goals of the World Bank Group of promoting shared prosperity and reducing extreme poverty. The increased transport connectivity was to raise prosperity in the region by making trade and production sharing more efficient, developing larger regional value chains, and tapping economies of scale, leading to important social and economic outcomes in the underserved regions where the program road links were to be located. The program was to contribute to unlocking the region’s potential in participating in commerce and supply chains, notably in the areas of producing agricultural products marketable to urban areas or in the agro-industry. The improved connectivity delivered under the project was to directly benefit about 680,000 residents of Batken and Osh Oblasts and indirectly 3 million residents of the Kyrgyz Republic and Tajikistan. At the time of project appraisal, Batken and Osh Oblasts had above-average poverty headcounts and Osh Oblast had the highest poverty density in the country. The project design did not
Page 13 of 42
The World Bank First Phase of the Central Asia Road Links Program (P132270)
include any indicators measuring the project activity contribution toward decreasing poverty levels, although the market accessibility index uses poverty data. It would be important to assess the attribution of the wider program of the CARs on poverty levels, not the individual project level.
Other Unintended Outcomes and Impacts
53. Procurement and installation of HSWIM equipment in 8 locations for transport overload control around the country was an unintended outcome. The installation of this equipment is not yet accompanied by the necessary adoption of the institutional policy, and regulatory reforms as recommended in the Strategic Plan on Transport Control prepared under the project. The Plan was adopted as a Concept only, which did not commit the MOTR to the specific reforms’ recommendation needed to support the mandate of transport control, and this may have an impact on the sustainability of outcomes.
III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME
A. KEY FACTORS DURING PREPARATION
54. Quality of detailed designs. Early on, despite preparatory work done under the previous World Bank-financed NRRP, the project started experiencing delays with the civil works component. These delays were primarily due to the multiple changes of significant nature required for the detailed designs. Based on the project documents, it appeared that when the technical designs were prepared, quality control oversight was weak as various buildings such as schools kept appearing near the road right of way, forcing the alignment to be changed.
55. Project management. The follow-up on the decisions that were discussed between the MOTC/IPIG and the World Bank on the road section design was slow and at times weak. For example, minor road alignments bypassing the disputed road segments with the Republic of Tajikistan. Such changes to the designs required additional work on realigning the road. It also required new Resettlement Action Plans (RAPs) to be prepared, agreed upon, and compensated, which all contributed substantially to the project delays during the implementation stage.
B. KEY FACTORS DURING IMPLEMENTATION
56. Project management. Project management during implementation was moderately satisfactory and affected project outcomes. The project started to experience problems with project management due to insufficient number of WB IPIG staff (absence of key staff: Financial Manager, Director, Road Engineer, Environmental Specialist, Contract Administration Specialist). This affected the implementation of activities, including the civil works and the institutional development activities. Despite having separated the IPIG and having created a dedicated Project Implementation Unit in charge of the World Bank project, project management continued to be a challenge until the project closed.
57. Despite the above challenges, the Bank team rated the PDO as Satisfactory throughout the project implementation. At MTR, the project disbursements stood at 20 percent, but the DO rating was still assessed as Satisfactory. By MTR, many of the project design flaws were known. However, the Bank team
Page 14 of 42
The World Bank First Phase of the Central Asia Road Links Program (P132270)
missed an opportunity to restructure the project at MTR or later to bring the project back on track in achieving its development objectives. With regards to Component 2: Improvement of Road Operations and Maintenance Practices, the decision to divert the financing that was earmarked for the procurement of the road maintenance equipment for OBI UAD toward the HSWIM equipment had an adverse impact on the maintenance activities. The OBI UAD was left with no equipment to conduct maintenance during most of the project, despite the commitments in the signed SLA – a consequence of the early preparation and signing of the SLA before the project hired an international consultant to support this process. It’s worth noting, however, that the equipment did become available in the final phase of the project and remains in place today providing increased capacity for maintenance.
58. Ownership Challenges from the MOTC/MOTR. During the five years of project implementation, there was a high turnover of the leadership of the MOTR, which had five ministers. This is not an unusual phenomenon in a developing country such as the Kyrgyz Republic. However, this had a significant impact on the functioning of IPIG. Details on this challenge were discussed earlier in this report under the efficiency analysis section. The IPIG had been implementing projects in the road sector for several years and was the MOTC’s institutional memory. This coupled with the absence of an IPIG Director for a significant part of the project, caused delays in timely decision-making.
IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME
A. QUALITY OF MONITORING AND EVALUATION (M&E)
M&E Design
59. The monitoring and evaluation (M&E) system designed for CARs-1 project was limited to the Results Framework and included five PDO-level indicators and four intermediate results indicators3. The project’s Theory of Change has been reformulated from the PAD, which clearly represents the logic framework and how the project activities and outputs were supposed to have contributed to the achievement of the PDO outcomes as well as the higher-level objectives. The Results Framework had clearly specified objectives. Not all results indicators were appropriately selected, some of them were not easily measurable and not all were relevant indicators:
- measuring the volume of freight through two BCP had no relevance to the project activities, it also had an overestimated baseline value, which negatively affected the targeted 10 percent increase;
- there was no explanation or detailed formula on how market accessibility index was calculated; The PAD said it measured the sum of population weighted by travel time to reach major markets without indicating what major market destinations were or what days of the year will be considered for the survey.
3 A fifth intermediate results indicator was added to the Results Framework in the project portal, Strengthen implementation capacity of the Project, however, it was not part of the original Results Framework as approved. It was also never officially added to the project Results Framework thorugh a project restructuring. Therefore, this indicator is not considered for the purposes of the ICR analysis.
Page 15 of 42
The World Bank First Phase of the Central Asia Road Links Program (P132270)
- annual budget allocation, while the indicator showed an annual increase, the amounts as defined in the Results Framework were not enough to perform required level of maintenance;
- compliance by OBI UAD with service level criteria as specified in the Service Level Agreement should have been a PDO indicator, not an intermediate indicator, considering its importance in supporting the second part of the PDO.
M&E Implementation
60. The implementation of the M&E system of the project was weak. Most of the weaknesses of the M&E design were known by the MTR. An assessment of the indicators was conducted at the MTR, and despite the challenges with the OBI UAD SLA, insufficient budget allocation for maintenance, challenges with the road user satisfaction survey tool, Zholdor.kg, indicators nor any of their values were suggested for revision.
M&E Utilization
61. The project M&E system did not fit into any wider M&E system but may have laid the ground for follow-up projects within the broader CARs initiative. Despite several shortcomings, the M&E system was used by the project team to track progress toward achieving PDOs and generate reports. An integration of the IT tool, Zholdor.kg, in M&E activities could have been greatly beneficial to institutionalize the two- way communication between the road users and road maintenance services, if the tool was properly utilized and maintained. During the first year of implementation, significant effort was invested by the developer to support the ministry, including: (a) mobile application and website were developed; (b) about 1,500,000 questionnaires from OBI road users were administered to assess winter and routine road maintenance; (c) data were entered in the system, generating reports for road maintenance performance monitoring at the central office of the MOTC. In later years, significant effort was required to institutionalize Zholdor.kg. However, there were no financial nor human resources nor motivation from the Road Maintenance Department (RMD) of the DDH in Bishkek to respond to the feedback received from the beneficiaries in a knowledgeable and timely manner. Eventually, since it was a one-way communication channel, beneficiaries stopped submitting their feedback. The beneficiary survey and user satisfaction were not measured at the end of the project.
Justification of Overall Rating of Quality of M&E
62. The M&E system was found to have several design and implementation challenges for the use of results information, absence of a clear protocol and methodology, limited capacity for analysis, and weak links between Results Framework and decision-makers. Given such limitations, the overall rating of M&E quality is assessed as Modest.
B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE
Environmental Compliance
63. The project triggered Environmental Assessment (OP/BP 4.01) and Involuntary Resettlement (OP/BP 4.12) and was assessed as Environmental Category B. The anticipated negative impacts were small
Page 16 of 42
The World Bank First Phase of the Central Asia Road Links Program (P132270)
to medium scale, easily mitigable, site specific, and predominantly confined to the construction phase. The IPIG prepared an Environmental Impact Assessment and Environmental Management Plan (EMP); the final documents were disclosed in October 2013. In 2017, the MOTR decided to make changes to the alignment of the Sulukta-Koltso section, to avoid exposure to four households and a school in the village of Koltso and to prevent an increased risk of accidents by avoiding densely populated areas. The Environmental Monitoring and Mitigation Plan was revised accordingly, and the contractor prepared a Site-specific EMP. These documents were reviewed and approved by the World Bank in July 2017 before the start of construction. The primary responsibility for monitoring compliance was assigned to the MOT IPIG.
64. Environmental safeguard compliance was moderately satisfactory during the early stage of implementation. Compliance was satisfactory between October 2018 and project closure as evidenced by the borrower’s semiannual reporting and was confirmed by World Bank’s implementation review missions and ISRs. Two significant environmental issues were identified:
• Spill compliance, noted by the supervising consultant in the completion report, which the contractor has agreed to address by replacing soil during the demobilization and defect liability period.
• The sightings of a potential endangered red list snake in the vicinity of Lot 1B during the summer of 2018. A brief biodiversity assessment was conducted. Three species that might inhabit the area were identified—two reptiles (Armour Glass Lizard and Tatary Sand Boa) and one small mammal (Brandt’s Hedgehog). As a mitigation measure, additional culvert underpasses were constructed along the route in the area where the snakes were first observed. Awareness training was also conducted with the contractor and the consultant. After the initial report, there have been no reported sightings of these species, either living or killed by traffic along the road. The assessment also identified migrating bird species as potential visitors to the general area, but none nested in the vicinity.
Social Safeguards Compliance
65. Regarding social safeguards, the most significant aspect is the management of issues related to land acquisition and resettlement, which is often considered a key constraint for timely implementation of the project. The review of safeguards documents of the project reveals that the land acquisition and resettlement issues were managed effectively and proactively. During the preparation of the feasibility study for the Isfana-Kairagach road, the social screening and preliminary survey identified 29 households on Isfana-Sulukta including an affected access road of a school in Koltso village in Sulukta-Koltso. RAPs were prepared with maps of the lots affected and followed by a series of public consultations and meetings with the Interdepartmental Commission, representatives of state authorities, and property owners to discuss the resettlement issues. The landowners and owners of properties along the road requested the MOTR and local authorities to consider changing the alignment to minimize the project impact on the land and properties As a result, the MOTR decided to prepare alternative preliminary designs for the alignment and cross-sections by changing the central line which completely excluded the above land acquisitions and was able to minimize adverse social impacts on the local communities. In the road section of Sulukta-Koltso, two land lots were found necessary for the road widening. The ministry had provided alternate land lots for these two leaseholders to continue their agriculture-based livelihood.
Page 17 of 42
The World Bank First Phase of the Central Asia Road Links Program (P132270)
The RAP completion report prepared by the client outlines all the mitigation measures adopted to mitigate impacts of land acquisition and resettlement under the project.
66. The other important aspect of social risk management under the project is the functioning of a robust Grievance Redress Mechanism (GRM) to receive and resolve complaints from the public. The project GRM consists of two levels—local and central levels. Each level had to redress the grievance and take decisions within 15 working days. The MOTR institutionalized this GRM by issuing a government order which was distributed among the local population living along the project road and respective local authorities. The GRM adopted by the project was able to address a wide range of issues related to the project activities as social and environmental problems and road safety issues arose among the local population and other stakeholders as a result of construction works. The mission notes that the GRM played a critical function to prevent public discontent and all grievances and complaints were addressed impartially on time. The World Bank mission in August 2019 reviewed the GRM log prepared by the project team and found that a total of 129 grievances were filed under the project from 2016 to 2019 and all the grievances were successfully resolved.
67. The only concerning factor related to social safeguards, however, is the lack of attention to World Bank requirements and guidelines in the labor camp management. Although most of the construction camps established under different contractors were found to be organized, systematic, and well managed, one camp was identified in the later stage of the project that was not well maintained and had poor living conditions for the resident workforce. During the final safeguards mission in August 2019, this issue had been addressed and inspection showed that all camp residents were now adequately and safely housed.
Fiduciary Compliance - Procurement
68. Procurement activities were carried out by the IPIG MOTR. Key procurement risks identified during preparation were as follows: (a) bid evaluation committee members not familiar with international procurement procedures and risk of obstruction or delay of the procurement process, especially the evaluation of bids and proposals; (b) lack of awareness of procurement opportunities available in the project for civil works, goods, and services; (c) poor quality of bidding documents, including ambiguous technical specifications; unclear and unrealistic requirements, such as delivery and completion time which bidders would be unable to meet; and frequently poor qualifications and experience of the bidders; and (d) excessive variation orders as well as poor quality construction may require remedial works.
69. The IPIG was staffed with full-time procurement specialist and contract administration specialist. However, while procurement was overall within the moderately satisfactory range, some issues occurred at times. For instance, the contract for the supply of road maintenance equipment, initiated at the end of the project, was delayed due to the delays in equipment delivery and also necessitated an extension of the project along with construction works completion. The delays observed were primarily due to decreased capacity of the IPIG in the last one year of the project.
70. Procurement of works was generally slow during the first years of project implementation, despite attempts during preparation to facilitate readiness. Procurement delays were due partly to the MOTR’s capacity constraints, which were reflected in the delayed bid evaluation. The MOTR’s procurement capacity was unstable as there were changes in the MOTR IPIG structure and multiple changes in staffing
Page 18 of 42
The World Bank First Phase of the Central Asia Road Links Program (P132270)
during the project implementation. The World Bank assisted the MOTR in addressing this by providing intensive procurement support and guidance from the World Bank’s procurement specialist located in the Bishkek Country Office.
71. The procurement risk was rated Substantial at the appraisal stage, due to both the size of the civil works contracts to be procured as well as the country and sector environment. This risk was expected to be downgraded during the project implementation phase; however, it was instead upgraded to high at the last year of the project implementation due to discovered inconsistencies and overspending of the major consultancy contract for civil works supervision, which was not prior agreed with the World Bank. This necessitated an exceptional waiver from the management of the World Bank.
Fiduciary Compliance - FM
72. The FM arrangements under the project, including planning and budgeting, accounting, financial reporting, internal controls, funds flow, and external auditing, were overall adequate and acceptable to the World Bank.
73. During the project implementation period, the FM staffing at the IPIG was reduced. This happened after the IPIG was split into two separate Project Implementation Units (one responsible for World Bank projects and the other one for Asian Development Bank projects), and after the previous financial manager left the position, the position remained vacant throughout the project implementation period. Disbursement Specialist performed the duties of Financial Manager. During the project implementation, there was an effort to fill the vacant position; however, the tender was cancelled as the sufficiently qualified candidates did not express their interest.
74. Some delays were observed with provision of government co-financing for works contract usually at the beginning of fiscal years, while the level of the co-financing was close to the required level typically at the end of the year.
75. There was a requirement for quarterly interim financial report (IFR) submission under the project. In general, the IFRs have been received on time and were overall acceptable to the World Bank, with some exceptions. The auditors issued unmodified (clean) opinion on the project’s financial statements for FY2015, 2016, 2017, and 2018, which were received by the World Bank on time. The final audit for FY2019, including also the four (4) months grace period, was submitted to the World Bank by the due date and was declared acceptable.
76. Overall, the recipient, through the World Bank IPIG, was in compliance with financial covenants of the Legal Agreement including maintenance of a minimal required level of an acceptable FM system, timely submission of audited project financial statements and unaudited IFRs of the project to the World Bank, and public disclosure of the audited project annual financial statements.
C. BANK PERFORMANCE
Quality at Entry
77. The World Bank’s quality at entry is assessed as Moderately Satisfactory.
Page 19 of 42
The World Bank First Phase of the Central Asia Road Links Program (P132270)
78. Project scope. The project design supported government’s National Sustainable Development Strategy (NSDS), 2013-17 that had focussed on investments that would further integrate the country with growing regional markets including neighboring Central Asian countries while supporting local development in the peripheral region. It complemented the works along the Osh-Batken-Isfana road corridor under the then on-going Bank financed NRRP ational Road Rehabilitation (Osh-Batken-Isfana) Project. OBI was one of the six strategic transit corridors that carry majority of the traffic in the country. Efforts to rehabilitate six strategic transport corridors in the Central Asia region had been spelled out in then on-going Transport and Trade Facilitation Strategy and Action Plan for 2008-2017, and were endorsed by Central Asia Regional Economic Cooperation (CAREC). CARs-1 design focused on the critical last links for connectivity of OBI corridor along the Fergahna valley with Tajikistan. As such the first CARs project has also contributed well towards development of subsequent series of projects in both Kyrgyz Republic and Republic of Tajikistan to increase connectivity and enhance opportunities for regional development. At a program level the regional corridor development is showing good progress. CARs-3 in Kyrgyz Republic and CARs-2 in Tajikistan are under implementation, and CARs-4 in Tajikistan was recently approved, while CARs-5 (proposed to include Uzbekistan in addition to the current two countries) appears to be a prospective operation.
79. From the regional perspective, however, the descriptive part of the CARs-1 PAD indicated that the project was to closely coordinate with CAREC and include a review of regional standards, norms, and parameters on vehicle (truck) weight and axle load limits and tariff setting as well as the development of a strategic plan for the institutional arrangements for transport control. It was not clear as to what was meant by regional standards—whether it was in relation to the Central Asia region as a whole or the Kyrgyz Republic and the Republic of Tajikistan in particular. The related PDO indicator, strategic plan on transport control to be discussed at the regional level (CAREC) and approved by the MOTC only required action from one country i.e., Kyrgyz Republic. There is however evidence that under the ongoing CARs-2 project in Tajikistan, the Bank team envisaged and have been successful to develop a strategic plan on weight and axle load control.
80. Leveraging resources for project financing. The rehabilitation of the OBI road corridor was one of the priority corridors for the Government of the Kyrgyz Republic since the country gained independence in 1991. In 2009, the World Bank leveraged financing from the EBRD and the European Commission toward parallel financing of the previous NRRP, which was the first project to rehabilitate the OBI corridor. Since then, many development partners provided financing for the rehabilitation of the OBI corridor, as shown in Table 8. The CARs-1 was approved in 2014 and its preparation facilitated government efforts to leveragefinancing from other sources such as the Exim Bank of China and the EU.
Table 4. Development Partner Financing of the OBI Corridor, 2013–2023 Co-financing by the Road Years of Project Organization Project Cost Government of the Sections (km) Implementation Kyrgyz Republic Exim Bank of China 220–232 US$91.4 million — 2013–2018 248–360 European Union 108–123 EUR 8.6 million US$3.1 million 2013–2015 Islamic Development Bank 75–103 US$21.3 million US$10.3 million 2017 to present JICA 28–75 US$133.5 million — 2018-2023 Source: IPIG.
Page 20 of 42
The World Bank First Phase of the Central Asia Road Links Program (P132270)
81. Improvement of road operations and maintenance practices. The Financing Agreement of the project had an effectiveness condition that OBI UAD had been assigned with the responsibility for routine maintenance and emergency works along the OBI-Tajikistan road corridor in accordance with the annual SLA. Having to sign the SLA as an effectiveness condition turned out to be bit early in its timing. The aim of the SLA was to move away from the traditional form of force account works, with payments based on the consumed resources and to link the payment to results / service delivered—a road in good condition. To comply with the effectiveness condition, the MOTC and OBI UAD signed the SLA for one year.
82. The design could have allowed for a different sequencing of actions. Hiring of an international consultant could have happened first to undertake the review of the institutional, policy, and regulatory aspects that needed to be developed and implemented prior to signing of the SLA. This consultant could have also helped with drafting the SLA. The signed SLA was of insufficient quality, it did not include precise duties and responsibilities, did not specify how the lump-sum amount of winter and summer maintenance was calculated, had inconsistencies in the definition of service levels, and did not express sanctions for noncompliance in monetary terms. The project could have procured the equipment for maintenance prior to having the OBI UAD sign the SLA. The financing for the road maintenance equipment was diverted toward the procurement of the HSWIM equipment. The SLA remained on paper for one year, after which it was not re-signed. Despite these facts, OBI UAD did carry out limited routine maintenance on the road, using old equipment and funds allocated to the old section (rehabilitated under the NRRP). However, those funds were not enough to undertake any serious maintenance.
Quality of Supervision
83. The World Bank’s quality of supervision is assessed as Moderately Satisfactory. The supervision effort, on average two missions per year, was sufficient. The procurement, FM, and environmental and social safeguards supervision was satisfactory. The Aide Memoires flag risks discuss potential and actual problems and have timebound action plans to catch up with the delays. However, since project management started to deteriorate from the first year of project implementation, the delays continued to accumulate, causing substantial delays in project disbursements, which were not reflected on project ratings. Despite the various challenges and delays, the team was able to identify threats to the achievement of relevant outcomes. As an example, the team was able to flexibly reallocate resources under equipment towards the installation of HSWIMS that resulted in unintended outcome in respect of ensuring sustainability. Some of the factors leading to an assessment of MS rating are as follows that could have been proactively averted after MTR.
84. This project was approved in April 2014. The preparatory work was initiated using the resources of the previous project (NRRP) as well as advance procurement under CARs-1 to ensure timely start of civil works. Despite that, one year after the Board approval the project disbursed US$0.20 million. By MTR in May/June 2017, the project disbursed US$7.62 million (about 20 percent)4 and the PDO rating was assessed as Satisfactory. By April 2019, the original closing date of the project (year 5 of the project), the disbursements stood at US$23.50 million (about 58 percent), and the PDO was kept Satisfactory until the project closure in November 2019.
4 Please refer to the ICR datasheet on page iii, table on ratings of project performance in ISRs.
Page 21 of 42
The World Bank First Phase of the Central Asia Road Links Program (P132270)
85. The MTR was an opportunity to align the project’s scope and objectives with the PDO indicators and make other improvements to the Results Framework. The PDO indicator on volume of freight through Kyzyl-Bel/Guliston and Kairagach/Madaniyat border crossing points had an overestimated baseline value. At MTR an assessment was made that freight volume was below the expected value due to the closure of one of the two BCPs. However, with the opening of the BCP, the targeted values were not met. Similarly, it was known by the MTR that the part of the PDO related to supporting improvements in road operations and maintenance needed corrective action. It was evident that the signing of the SLA was premature; significant work was needed on reviewing current practices and revising the legal framework regarding the establishment of typical routine maintenance works, unit rates and cost estimates. In addition to these policy and regulatory revisions, no equipment could be procured for OBI UAD to perform winter or summer maintenance works at the satisfactory level. The tool to annually assess the road user satisfaction with maintenance of the Pulgon-Batken road section, Zholdor.kg, was not functioning. All this mandated a course correction after MTR.
Justification of Overall Rating of Bank Performance
86. Based on the above evaluation of the quality of the World Bank’s performance at entry and at supervision, the overall World Bank performance is assessed as Moderately Satisfactory.
D. RISK TO DEVELOPMENT OUTCOME
87. The risk to the development outcome in relation to the increased transport connectivity between the Kyrgyz Republic and the Republic of Tajikistan along the priority cross-border road links as well as improvements in road operations and maintenance practices is assessed as High. Overall, the nature of the project outcomes is such that they are susceptible to both external and internal risks.
88. There is an inherent risk to achieve ‘increased transport connectivity between the Kyrgyz Republic and Tajikistan along priority cross-border road links in Batken Oblast’ as it relates to the political instability in the region and disputes over border demarcation between the Kyrgyz Republic and Tajikistan. The disputes are causing multiple conflicts over access and use of natural resources such as water for irrigation purposes and pasture grounds for grazing animals. As the project has shown, the likelihood for such an event to occur is relatively high, given that the Kairagach/Madaniyat border crossing point was closed during the first year of the project and no freight could pass through the border crossing point. From a technical standpoint, the risk remains low as the road design and construction work has been able to address all design and construction issues faced during project implementation.
89. The risk of not sustaining the ‘improvements in road operations and maintenance practices’ is assessed as high. While the project introduced OBI UAD to international standards in road operations and maintenance, the sustainable funding level to provide timely and quality maintenance especially at those road sections, which are in a poor condition, is not yet secured. The OBI UAD and its DEPs continued to follow their monthly maintenance schedules of predefined activities rather than addressing problems as they arise. This approach is rather different from the internationally recognized maintenance practices which impedes further improvements in preservation of road assets. The sustainability of operations and maintenance of eight (8) installed HSWIMS will require implementation of recommendations developed by the project, for which a political will from the central government level will be paramount.
Page 22 of 42
The World Bank First Phase of the Central Asia Road Links Program (P132270)
V. LESSONS AND RECOMMENDATIONS
90. Design. The PAD describes how the CARs program would contribute to promoting shared prosperity and reducing extreme poverty. The project design being the first in the series of CARs could have laid foundations to address the issues of unlocking the region’s potential in participating in trade and supply chains through supporting economic and social activities and creating opportunities for the populations in the region. Apart from rehabilitating a road, the project could have considered support to local communities and small businesses to improve production and marketability of agricultural produces. Likewise, a better-defined methodology and more traceability to the original baselines captured up front in M&E design could have helped improve coordination of M&E activities, increase efficiency, and streamline project communication.
91. Regional projects vs. national projects with some regional dimensions. For a project to be regional (and eligible for IDA regional financing) it needs to meet the criteria referred to in para 36 of the PAD. The PDO definition suggested as an outcome the improved connectivity between the Kyrgyz Republic and the Republic of Tajikistan; although the definition carefully mentions “priority cross-border road links in Batken Oblast”, that is a specific location in the Kyrgyz Republic. The border issues and performance between the two countries were not part of the project design, nor was there any evidence of cross- border collaboration and dialogue between the two countries. It would be important to provide a better justification for a regional project in line with the requirements set out by the Bank.
92. Project management. Effective project management is one of the key elements to project success. It allows to achieve project deliverables within the constraints of time, scope, budget and quality and avoid unnecessary scope creeps or budget overruns. As project developments have shown, the weak and understaffed IPIG frequently impeded the progress of project implementation resulted in delayed procurement processes, inadequate response to project risks, and inability to complete assigned tasks in a timely manner. Government commitment is imperative to ensure continuity and to maintain a strong implementation unit throughout the project life cycle. In addition, the change in Bank TTL shortly before project completion also posed a risk to provide continuity in project oversight and to ensure all intended activities take place before the closure. It takes time for new project staff to fully grasp the complexity of the project, build relationship with project stakeholders and ensure smooth transition in terms of processes and acceptance levels not only during the project phases, but long after the project is complete.
93. IT-based survey tools. The use of IT-based survey tools does not always justify the end means due to the capacity of the local authorities and the required level of human resources and financing to run the websites and applications. It is important to assess the appetite, capacity, responsible parties, and financial resources required to maintain IT systems and mobile applications such as Zholdor.kg before investing too much efforts in testing and implementing such systems. The application was gathering feedback from the OBI road users on the maintenance provided to the OBI corridor by OBI UAD. However, the feedback was received by the DDH in Bishkek, who was not involved in the day-to-day maintenance of the corridor, had no dedicated staff, or resources to maintain the domain and respond to the feedback. The tool would have been more functional if housed locally within the Batken Oblast. Use of standard sociological surveys and focus group discussions are more appropriate in remote areas where internet penetration is insignificant and local authorities do not have the capacity or resources to maintain high technology tools.
Page 23 of 42
The World Bank First Phase of the Central Asia Road Links Program (P132270)
94. Land acquisition and resettlement. The project has shown that the most effective way to address issues of land acquisition and resettlement is to follow the fundamental principles of the World Bank policy (OP 4.12 - Involuntary Resettlement)—avoid, minimize, and mitigate social impacts in the project planning and early stage of implementation. A local-level GRM was activated under the project with information related to technical details and potential impacts on land and properties in the vicinities of road sections. This allowed the public to share their concerns and suggestions to redesign road sections to avoid large-scale land acquisitions, relocate commercial structures, and minimize such impacts by implementing a socially responsive road project. Lessons leant through this project to minimize the risks and impacts of land acquisition of transport projects are: (a) existence of robust and active GRM systems from the beginning of the project that allow people to lodge their concerns and suggestions for road development in the areas where the road sections run through settlement areas; (b) availability of dedicated public liaison/GRM staff to bring the process under the control of the main implementing entity; (c) flexibility of RAPs, that should be updated whenever there is a change in the road alignment; and (d) local-level public consultations are conducted early on to ensure community ownership in the project.
95. Sustainability of project investments. The HSWIM and the road maintenance equipment are important and necessary to improve road operations and maintenance practices. The HSWIM and road maintenance equipment (goods) procured under Component 2: Improvement of Road Operations and Maintenance Practices cost the project nearly US$7 million. The original cost of this component as approved by the World Bank’s Board was US$1.70 million and at project completion the cost of the component was US$8.20 million, which is nearly five times the original amount. However, such spending should be accompanied by activities that support a long-term view in managing road assets, including reforms outlined in the Strategic Plan for Axle-Load Control and on building capacity to maintain a functional Road Asset Management System (RAMS). Further strengthening of the RAMS, which was financed through the World Bank’s previous project, the NRRP, with a sustainable network data collection and evaluation, could have allowed the MOTR to better monitor, plan, and program roadwork activities. Having a functional RAMS is a necessity when considering wider operations and maintenance of the roads sector considering the scarce public finances, which was amiss from the project design. .
Page 24 of 42
The World Bank First Phase of the Central Asia Road Links Program (P132270)
ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS
A. RESULTS INDICATORS
A.1 PDO Indicators
Objective/Outcome: Increase transport connectivity between Kyrgyz Republic & Tajikistan along cross-border in Batken Ob Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Volume of freight through Tones/year 500000.00 550000.00 166023.40 Kyzyl-Bel/Guliston and Kairagach/Madaniyat border 01-Nov-2013 27-Mar-2014 15-Nov-2019 crossing points
Comments (achievements against targets):
Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Market accessibility index of Number 4.60 5.50 0.00 Batken Oblast 01-Nov-2013 27-Mar-2014 15-Nov-2019
Comments (achievements against targets): No survey has been done at the end of the project.
Page 25 of 42
The World Bank First Phase of the Central Asia Road Links Program (P132270)
Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Strategic Plan on transport Yes/No N Y N control has been discussed at the regional level (CAREC) 01-Nov-2013 27-Mar-2014 15-Nov-2019 and approved by MOTC
Comments (achievements against targets):
Objective/Outcome: Supporting improvements in road operations and maintenance practices Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Road users satisfied with Percentage 0.00 80.00 0.00 maintenance of the Pulgon- Batken road section 01-Nov-2013 27-Mar-2014 15-Nov-2019
Comments (achievements against targets): No survey has been done at the end of the project.
Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion
Page 26 of 42
The World Bank First Phase of the Central Asia Road Links Program (P132270)
Annual budget allocation for Amount(USD) 80000.00 200000.00 242489.00 routine (summer and winter) maintenance for road 01-Nov-2013 27-Mar-2014 15-Nov-2019 sections in sustainable condition along the corridor under OBI UAD
Comments (achievements against targets):
A.2 Intermediate Results Indicators
Component: Rehabilitation of Priority Road Sections in Batken Oblast
Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Road user costs by truck Number 0.52 0.42 0.42 along the project road sections and in project area 01-Nov-2013 27-Mar-2014 15-Nov-2019 (US$/ veh-km)
Comments (achievements against targets):
Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Roads in good and fair Percentage 40.00 80.00 77.60
Page 27 of 42
The World Bank First Phase of the Central Asia Road Links Program (P132270)
condition as a share of total 01-Nov-2013 27-Mar-2014 15-Nov-2019 classified roads
Size of the total classified Kilometers 420.00 420.00 420.00 network
Comments (achievements against targets):
Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Roads rehabilitated, Non- Kilometers 0.00 56.00 56.40 rural 01-Nov-2013 27-Mar-2014 15-Nov-2019
Comments (achievements against targets):
Component: Improvement of Road Operations and Maintenance Practices
Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Compliance by OBI UAD with Percentage 0.00 100.00 91.40 service level criteria as specified in the Service Level 01-Nov-2013 27-Mar-2014 15-Nov-2019 Agreement
Page 28 of 42
The World Bank First Phase of the Central Asia Road Links Program (P132270)
Comments (achievements against targets):
Component: Project Management and Implementation
Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Strengthen implementation Number 0.00 0.00 0.00 capacity of the Project 01-Mar-2014 27-Mar-2014 15-Nov-2019
Comments (achievements against targets): This indicator was not part of the agreed Results Framework, therefore for the purposes of the ICR this indicator is not assessed.
Page 29 of 42
The World Bank First Phase of the Central Asia Road Links Program (P132270)
B. KEY OUTPUTS BY COMPONENT
Objective/Outcome 1. Increase transport connectivity between the Kyrgyz Republic and Tajikistan along cross-border in Batken Oblast 1. Volume of freight through Kyzyl-Bel/Guliston and Kairagach/Madaniyat border crossing points 2. Market accessibility index of Batken Oblast Outcome Indicators 3. Strategic plan on transport control has been discussed at the regional level (CAREC) and approved by MOTC 1. Road user costs by truck along the project road sections and in project area (US$/veh-km) 2. Roads rehabilitated, Non-rural Intermediate Results Indicators 3. Roads in good and fair condition as a share of total classified roads 4. Size of the total classified network 1. 56.4 km of rehabilitated road Key Outputs by Component 2. Strategic Plan on Axle Load Control (linked to the achievement of the 3. Increased number of people accessing markets in Batken and Isfana Objective/Outcome 1) 4. Reduced travel time for freight to access markets in Batken and Isfana 5. Lower road user cost by truck Objective/Outcome 2: Supporting improvements in road operations and maintenance practices 1. Road users satisfied with maintenance of the Pulgon-Batken road section Outcome Indicators 2. Annual budget allocation for routine (summer and winter) maintenance for road sections in sustainable condition along the corridor under OBI UAD Intermediate Results Indicators 1. Compliance by OBI UAD with service level criteria as specified in the Service Level Agreement 1. Procurement of road maintenance equipment for OBI UAD Key Outputs by Component 2. Establishment of SLA between MOTR and OBI UAD (linked to the achievement of the 3. Installation of eight (8) WIMS Objective/Outcome 2) 4. Two summary reports on outcomes of year 1 and year 2 pilot WIMS operations
Page 30 of 42
The World Bank First Phase of the Central Asia Road Links Program (P132270)
5. Technical assistance to OBI UAD and DDH during the implementation of SLA 6. Analysis of the legal, technical, and institutional framework of the current procedures for routine maintenance management 7.Report on prioritization and programming of works 8. Proposals for typical routine maintenance works and development of unit rates for typical works 9. Report on concepts of RAMS 10. Report on dissemination activities
Page 31 of 42
The World Bank First Phase of the Central Asia Road Links Program (P132270)
ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION
A. TASK TEAM MEMBERS
Name Role Preparation Cordula Rastogi Task Team Leader(s) Adam Shayne Lead Counsel Joseph Paul Formoso Senior Finance Officer Nagaraju Duthaluri Lead Procurement Specialist Irina Goncharova Procurement Specialist(s) Aliya Kim FM Analyst Alexei Slenzak Environmental Specialist Aly Zulficar Rahim Social Development Specialist Asli Gurkan Social Development Specialist Artan Guxho Social Development Specialist Rodrigo Archondo-Callao Senior Highway Engineer Aidai Bayalieva Operations Officer Krista Alexandra Soininen Junior Professional Associate Virginia Tanase Senior Transport Specialist Aimonchok Tashieva ET Consultant Zhanetta Baidolotova Program Assistant Funda Canli Program Assistant Shirin Imanbaeva Team Assistant Supervision/ICR Muhammad Zulfiqar Ahmed, Aidai Bayalieva Task Team Leader(s) Irina Goncharova Senior Procurement Specialist(s) Garik Sergeyan Senior FM Specialist John Bryant Collier Senior Environmental Specialist Lien Thi Bich Nguyen Program Assistant Jayne Njoki Dzowela Paralegal Jasna Mestnik Finance Officer Ma Dessirie Kalinski Finance Analyst Mohamed Ghani Razaak Senior Social Development Specialist Aidai Bayalieva Operations Officer Zhanetta Baidolotova Program Assistant Azeb Afework Program Assistant Nargiza Tynybekova Team Assistant
B. STAFF TIME AND COST
Page 32 of 42
The World Bank First Phase of the Central Asia Road Links Program (P132270)
Staff Time and Cost Stage of Project Cycle No. of staff weeks US$ (including travel and consultant costs) Preparation FY13 38.037 246,165.26 FY14 27.186 106,334.47
Total 65.22 352,499.73
Supervision/ICR FY14 4.225 18,133.20 FY15 18.143 76,067.83 FY16 31.704 97,816.38 FY17 33.794 151,780.27 FY18 26.742 144,711.66 FY19 22.977 115,568.99 FY20 25.842 115,707.46 Total 163.43 719,785.79
Page 33 of 42
The World Bank First Phase of the Central Asia Road Links Program (P132270)
ANNEX 3. PROJECT COST BY COMPONENT
Amount at Actual at Project Percentage of Components Approval Closing (US$, Approval (%) (US$, millions) millions) Rehabilitation of Priority Road 51.30 31.00 60.42 Sections in Batken Oblast Improvement of Road Operations and 1.70 8.20 482.35 Maintenance Practices Project Management and 1.00 0.90 90.00 Implementation Total 54.00 40.10 74.25
Page 34 of 42
The World Bank First Phase of the Central Asia Road Links Program (P132270)
ANNEX 4. EFFICIENCY ANALYSIS
1. The ex-ante economic analysis was done in 2013 by the World Bank for the entire program given the actual rehabilitation cost of the sections already rehabilitated, the road links financed under the program, and current road condition, traffic levels, and GDP growth projections for the Kyrgyz Republic. The economic analysis was conducted using the HDM-4, which simulates life cycle predictions of road deterioration, roadwork effects and their costs and road user costs and provides economic decision criteria for road construction and maintenance works. The HDM-4 analyzes projects by computing costs and benefits of different investment options in terms of savings in road maintenance costs, vehicle operating costs, and travel time costs. The comparison is done between the ‘do something’ scenario (project case) and the ‘do minimum’ scenario (without project case) over the analysis period.
2. The ‘do minimum’ scenario incorporates an assessment of what would happen if the project was not undertaken; therefore, it consists of the recurrent maintenance practice and reconstruction when the road reaches a very poor condition at 12 IRI (International Roughness Index), m/km. The project scenario consists of rehabilitation, improvement, or new construction works followed by proper recurrent maintenance and periodic maintenance works over the analysis period consisting of 6 cm overlays triggered at 4 IRI, m/km. The proposed investments were evaluated over 2014–2040, accounting for 25 years of project benefits at a 12 percent discount rate.
3. The economic analysis was conducted for the entire OBI-Tajikistan border road through the analysis of 12 separate road sections divided according to their characteristics, condition, and traffic covering 407 km. Table 4.1 presents the road sections evaluated at appraisal, their length at appraisal, and the actual length after the project completion.
Table 4.1. Road Sections Evaluated at Appraisal
Section Start End PAD Length Actual Length Number Location Location (km) (km) M13-1 Osh Outskirts Start Nookat Pass 7.7 7.7 M13-2 Start Nookat Pass End Nookat Pass 18.0 18.0 M13-3a End Nookat Pass Kyzyl Kiya 47.0 47.0 M13-3b Kyzyl Kiya Kok-Talaa 33.0 33.0 M13-4 Kok-Talaa Pulgon 15.0 15.0 M13-5 Pulgon Burgandy 32.0 32.0 M13-6 Burgandy km 220 65.0 65.0 M13-7 km 220 Batken Roundabout 15.0 12.0 M13-8 Batken Roundabout Tortkul 20.4 14.3 M13-9 Tortkul Ak Tatyr 23.0 23.0 M13-10 Ak Tatyr Isfana East 89.0 89.0 ISF-1 Isfana East Isfana West 6.1 6.1 ISF-2 Isfana West Kairagach Border 36.1 36.1 Total 407.3 398.2
4. Table 4.2 presents the estimated financial construction costs for the road sections evaluated at appraisal, and the contract and actual financial construction costs for the road sections evaluated for the
Page 35 of 42
The World Bank First Phase of the Central Asia Road Links Program (P132270)
ex post economic evaluation, in US$ millions, for which there is a contract. On average, the contract costs are 3.7 percent higher than the appraisal estimates, the actual costs are 1.9 percent higher than the contract costs, and the actual costs are 5.3 percent higher than the appraisal estimates.
Table 4.2. OBI Corridor Actual Financial Construction Costs Contract Financial PAD Financial Cost Actual Financial Cost Section Cost Contract/ Actual/ Actual/ Number US$, US$, US$, US$, US$, US$, PAD Contract PAD millions millions/km millions millions/km millions millions/km M13-1 6.2 0.802 ------M13-2 17.9 0.996 19.6 1.088 19.1 1.060 1.093 0.974 1.064 M13-3a 31.1 0.662 0.0 0.0 M13-3b 21.9 0.662 18.2 0.552 18.2 0.552 0.833 1.000 0.833 M13-4 10.3 0.689 7.8 0.519 7.8 0.519 0.753 1.000 0.753 M13-5 24.1 0.752 20.1 0.628 24.5 0.765 0.836 1.217 1.017 M13-6 26.6 0.410 25.0 0.385 25.0 0.385 0.939 1.000 0.939 M13-7 10.8 0.719 12.5 1.040 12.5 1.040 1.447 1.000 1.447 M13-8 13.2 0.648 9.7 0.677 9.7 0.677 1.044 1.000 1.044 M13-9 14.2 0.619 23.9 1.040 23.9 1.040 1.681 1.000 1.681 M13-10 86.6 0.973 92.6 1.040 92.6 1.040 1.070 1.000 1.070 ISF-1 9.8 1.613 0.0 0.0 ISF-2 36.0 0.997 24.4 0.677 24.4 0.677 0.679 1.000 0.679 Total 308.6 0.758 253.8 257.6 1.037 1.019 1.053
5. At appraisal, current traffic data were obtained from traffic surveys done by the units in charge of road maintenance of the project roads in 2012. Table 4.3 presents the traffic, in vehicles per day, collected in 2012 at appraisal, the traffic in 2019 as estimated at appraisal based on the estimated traffic growth rate defined at appraisal5 for the economic evaluation, and the actual traffic in 2019 based on recent traffic counts. There is high variation, among the road sections, on the difference between the estimated 2019 traffic at appraisal and the actual 2019 traffic. On average, the actual 2019 traffic is 22 percent lower than the one estimated at appraisal.
Table 4.3. OBI Corridor- Estimated at Appraisal and Actual Traffic PAD 2012 PAD 2019 Actual 2019 Actual 2019/ Section Traffic Traffic Traffic PAD 2019 Number (vpd) (vpd) (vpd) Ratio M13-1 5,185 7,008 11,582 1.65 M13-2 5,184 7,008 10,678 1.52 M13-3a 4,203 5,685 10,930 1.92 M13-3b 4,203 5,685 4,521 0.80 M13-4 4,916 6,646 1,976 0.30 M13-5 4,509 6,095 2,301 0.38 M13-6 3,260 4,407 1,391 0.32 M13-7 3,309 4,473 2,290 0.51
5 At appraisal, from 2012 to 2018, the estimated annual traffic growth rate was 5.46 percent for cars and buses and 5.20 percent for trucks.
Page 36 of 42
The World Bank First Phase of the Central Asia Road Links Program (P132270)
PAD 2012 PAD 2019 Actual 2019 Actual 2019/ Section Traffic Traffic Traffic PAD 2019 Number (vpd) (vpd) (vpd) Ratio M13-8 2,328 4,773 1,820 0.38 M13-9 3,108 4,201 1,730 0.41 M13-10 2,985 4,035 1,776 0.44 ISF-1 1,616 1,916 1,732 0.90 ISF-2 1,783 2,112 1,705 0.81 Total 3,457 4,725 3,670 0.78
6. At appraisal, it was determined that the return on the investments of the overall OBI road corridor was satisfactory with an overall EIRR of 24.0 percent based on the sensitivity analysis results . The EIRR of individual road sections along the OBI corridor had varied from 14.1 to 34.9 percent. Table 4.4 presents the EIRRs from the appraisal economic analysis, presented at the PAD, and the resulting EIRRs of the ICR ex post economic evaluation. The ICR ex post economic evaluation was done using the HDM-4 model and adopting the same assumptions and data used at the appraisal economic evaluation, however, considering the actual costs and the actual traffic in 2019.
Table 4.4. OBI Corridor - PAD Sensitivity Analysis and Ex Post EIRR Section PAD Ex Post PAD/ Number EIRR (%) EIRR (%) Ex Post EIRR M13-1 34.9 M13-2 30.3 39.7 1.31 M13-3a 29.5 M13-3b 29.5 26.0 0.88 M13-4 32.8 15.1 0.46 M13-5 28.9 12.1 0.42 M13-6 32.2 14.5 0.45 M13-7 22.5 8.5 0.38 M13-8 21.8 16.1 0.74 M13-9 23.6 6.2 0.26 M13-10 16.2 6.3 0.39 ISF-1 14.5 ISF-2 14.1 16.2 1.15 Total 24.0 13.2 0.55
7. Due to the lower actual traffic in 2019 compared with the appraisal estimates for the traffic in 2019, the ex post EIRRs are in general lower than the appraisal EIRR estimates. The overall ex post EIRR of the OBI road corridor is 13.2 percent, which can be compared with the 24.0 percent estimated at appraisal. Overall, the corridor yielded lower economic return than the one estimated at appraisal, but nevertheless the corridor yielded a satisfactory rate of return.
8. CARs-1 financed two contract packages of the overall OBI road corridor, totaling US$34.1 million of investments. The appraisal economic evaluation for the project had determined that the return on the investments for the road sections to be financed under CARs-1 was satisfactory with an overall EIRR of 16.6 percent. Table 4.5 presents the project EIRRs estimated at appraisal and the EIRRs calculated at the
Page 37 of 42
The World Bank First Phase of the Central Asia Road Links Program (P132270)
ex post economic evaluation. The ex post return on the investments under the CARs-1 is satisfactory with an overall EIRR of 16.2 percent for the two packages.
Table 4.5. CARs-1 Project Road Sections Actual Section Start End PAD Ex Post PAD/ Cost (US$, Number Location Location EIRR (%) EIRR (%) Ex Post EIRR millions) M13-8 Batken Tortkul 9.7 21.8 16.1 0.74 Roundabout ISF-2 Isfana West Kairagach Border 24.4 14.1 16.2 1.15 Total 34.1 16.6 16.2 0.97
Page 38 of 42
The World Bank First Phase of the Central Asia Road Links Program (P132270)
ANNEX 5. BORROWER COMMENT AND SUMMARY OF BORROWER ICR
The draft final version of the Bank ICR was provided to the Borrower on July 30, 2020 for comments. The Borrower responded on August 08, 2020 with following minor editorial corrections shown as text within (…) below. All corrections were appropriately incorporated into this final version of the Bank ICR.
Assessment of Efficiency and Rating. Bank ICR revised para 40: Project management. (Project management is a key aspect in implementing project activities. At the start of the Project the IPIG consisted of 26 full-time experienced staff. The project management of the IPIG under the previous NRRP was assessed as Satisfactory; the same staff were to implement the proposed CARs-1. However, in accordance with the Order of the Minister of Transport and Roads of the Kyrgyz Republic dated May 23, 2017 the IPIG was split into the World Bank IPIG and the Asian Development Bank IPIG in order to increase the effectiveness of IPIG performance. However, the position of Financial Manager for World Bank IPIG remained vacant throughout the CARs-1 project implementation period. Key position of Director became vacant as of January 2019, in addition to this the key positions such as Road Engineer, Environmental Specialist and Contract Administration Specislists were not in place. The government adverstized several procurement notices for the positions of Road Engineer, Financial Manager and Environmental Specialist, however they remained unsucceful in finding appropriately qualified candidates. The procurement tenders for the positions of Director, Financial Manager and Environmental Specislists were conducted but were cancelled for various reasons such as participation of weaker candidates or later due to approaching project closure. Given the situation with insufficient number of IPIG staff, the project still managed to implement and complete all component activities.)
Bank ICR revised para 41: Implementation Delays and Ownership by the MOTR. […in delays in procurement (and noncompletion of construction works)…]. […at the end of the project, (but the equipment was delivered late and this fact was one of the reasons for an extension)…].
Key factors during implementation. Bank ICR revised para 56: Project Management. [Project management during implementation was (moderately satisfactory) and affected…]. […(due to insufficient number of WB IPIG staff (absence of key staff: Financial Manager, Director, Road Engineer, Environmental Specialist, Contract Administration Specialist). This affected the implementation of activities, including the civil works and the institutional development activities).]
Fiduciary Compliance - Prcourement. Bank ICR revised para 69: […at the end of the project, was delayed (due to the delays in equipment delivery and also necessitated an extension of the project along with construction works completion)].
Fiduciary Compliance – FM. Bank ICR revised paras 73 and 75 respectively: […(,the position remained vacant throughout the project implementation period. Disbursement Specialist performed the duties of Financial Manager. During the project implementation, there was an effort to fill the vacant position; however, the tender was cancelled as the sufficiently qualified candidates did not express their interest.)] [(The final audit for FY2019, including also the four (4) months grace period, was submitted to the World Bank by the due date and was declared acceptable.)]
Page 39 of 42
The World Bank First Phase of the Central Asia Road Links Program (P132270)
Table 8. Development Partner Financing of the OBI Corridor, 2013–2023. [JICA, Years of Project Implementation, (2018-2023)]
Note: Below is an extract from the Borrower’s ICR duly reviewed by the Borrower as part of comments on Bank ICR draft final version. The full Borrower’s ICR is available in Project File as stated in Annex 6.
1. One of the important goals of the Project was to improve the roads operation and maintenance. Under this component road maintenance equipment was purchased for Osh-Batken-Isfana Roads Department (UAD OBI), and its staff was trained as per the best international road maintenance practices. The Project assisted the Ministry of Transport and Roads (MOTR) to study regional standards and norms on the weight size of vehicles (trucks), as well as the axial load restrictions and calculation of tariffs. The substantial outcome of the Project was the introduction of a weigh-in-motion system (WIMS) which was installed at eight posts in the south and north of the country.
2. Environmental and social impact assessment of the Project had shown that the construction would have an impact on green spaces, fences, small buildings, and commercial facilities. It was decided to revise the design of the road section. As a result of the changes, the impact on the mentioned facilities was avoided. Safeguard measures were implemented in full in order to avoid or mitigate negative social and environmental impacts of the project. During the project implementation 129 grievances were filed: grievances related to the retaining walls construction, irrigation network, access roads, removal of soil as a result of construction works. All grievances were registered and filed in order to monitor them regularly and submit the reports, including the status of the grievances and the decisions made on them to the Bank. During the construction works no incidents or accidents related to construction work involving workers or the local population was reported.
3. Project did not commence in accordance with scheduled plans. Construction works started with considerable delay due to the delayed contractor selection process. Also, there was a need to revise the design of project sites. Delay in making decisions on outstanding issues was observed from the Contractor, Consultant, and IPIG sides. There were omissions in administering the contracts for civil works and consulting services for construction supervision.
4. During the Project implementation construction works scopes were changed several times and the works completion dates were extended. But the total cost of the Project was not increased. The Bank received a request to extend the closing date from April 30, 2019, until September 30, 2019. The reason for the extension was that the construction works on Lot 1B were not completed. Then there was another extension of the Project for 1.5 months. The reason for this second extension was not related to the construction works but to allow closing of ongoing contracts for road maintenance equipment and installation of the additional overload control points.
5. Financial management of the Project was conducted appropriately. No violations have been found by Independent auditor.
6. Borrower`s performance was assessed as satisfactory. However, there were delays in the financing of civil work from the Government share.
Page 40 of 42
The World Bank First Phase of the Central Asia Road Links Program (P132270)
7. Bank`s performance was assessed as satisfactory. The Bank team regularly conducted supervision missions, worked closely with IPIG to solve problems encountered during the Project implementation. However, it was advised to strengthen the Bank`s activity in issues related to the preparation of projects, involving institutional activities. For example, when a mobile application (zholdor.kg) was introduced to assess the road user’s satisfaction, the resource potential of the Ministry (financial and human) wasn’t taken into account. To date, this application is not functioning. Another example is introduction of SLA, which was also premature due to the absence of adequate funding from the Government and the local legislation.
8. Main difficulties in the project implementation were mainly related to the need to make changes in the design of projects and obtaining permission to use borrow pits. The absence of key IPIG staff: Director since January 2019, Road Engineer since May 2019, and the Financial Manager throughout the Project implementation resulted in delays in fulfilling the tasks assigned to the Implementing Agency.
Page 41 of 42
The World Bank First Phase of the Central Asia Road Links Program (P132270)
ANNEX 6. SUPPORTING DOCUMENTS (in Project File)
(1) Project Appraisal Document (PAD) (2) Country Partnership Framework FY19-FY22 (3) Project Restructuring Papers (4) Implementation Support Mission Aide Memoires and Implementation Status & Results Reports (ISRs) (5) Environmental Impact Assessment and Environmental Management Plan (EMP) (6) Borrower’s Implementation Completion Report, July 2020.
Page 42 of 42