Document of The World Bank

Public Disclosure Authorized Report No: ICR2386

IMPLEMENTATION COMPLETION AND RESULTS REPORT (IDA-39890 IDA-39891 IDA-47180)

ON A

CREDIT

Public Disclosure Authorized IN THE AMOUNT OF SDR, 235.1 MILLION (US$348.2 MILLION EQUIVALENT)

TO THE

FEDERAL DEMOCRATIC REPUBLIC OF ETHIOPIA

FOR A

ROAD SECTOR DEVELOPMENT PHASE II PROJECT (APL2)

Public Disclosure Authorized

December 12, 2012

Transport Sector Country Department AFCE3 Africa Region

Public Disclosure Authorized

CURRENCY EQUIVALENTS

(Exchange Rate Effective June 30, 2012)

Currency Unit = Ethiopian Birr SDR 1.00 = US$1.52 US$1.00 = 17.82 Birr

FISCAL YEAR July 8 – July 7

ABBREVIATIONS AND ACRONYMS

AC Asphalt Concrete APL Adaptable Program Lending DBST Double Bituminous Surface Treatment DMOs District Maintenance Organizations ERA Ethiopian Roads Authority ERTTP Ethiopia Rural Travel and Transport Program GoE Government of Ethiopia HDM4 Highway Development and Management Model ICR Implementation Completion and Results IDA International Development Association M&E Monitoring and Evaluation MIS Management Information System NPV Net Present Value PAD Project Appraisal Document PDO Project Development Objectives PSNP Productive Safety Nets Programme RSDP Road Sector Development Program SDPRP Sustainable Development and Poverty Reduction Program SDR Special Drawing Rights VOCs Vehicle Operating Costs WIDP Wereda Integrated Development Plans

Vice President: Makhtar Diop Country Director: Guang Zhe Chen Sector Director: Jamal Saghir Sector Manager: Supee Teravaninthorn Project Team Leader: Fiona J. Collin ICR Team Leader: Bernard Aritua

FEDERAL DEMOCRATIC REPUBLIC OF ETHIOPIA

ROAD SECTOR DEVELOPMENT PHASE II PROJECT (APL2) CONTENTS

A. Basic Information ...... i B. Key Dates ...... i C. Ratings Summary ...... i D. Sector and Theme Codes ...... ii E. Bank Staff ...... ii F. Results Framework Analysis ...... ii G. Ratings of Project Performance in ISRs ...... vi H. Restructuring (if any) ...... vi I. Disbursement Profile ...... vi 1. Project Context, Development Objectives and Design ...... 1 2. Key Factors Affecting Implementation and Outcomes ...... 4 3. Assessment of Outcomes ...... 11 4. Assessment of Risk to Development Outcome...... 14 5. Assessment of Bank and Borrower Performance ...... 15 6. Lessons Learned...... 17 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners...... 18 Annex 2. Outputs by Component...... 21 Annex 3. Economic and Financial Analysis ...... 36 Annex 4 - Bank Lending and Implementation Support/Supervision Processes ...... 40 Annex 5. Summary of Borrower’s ICR and/or Comments on Draft ICR ...... 42 Annex 6. Comments of Co financiers and Other Partners/Stakeholders ...... 43 Annex 7. List of Supporting Documents ...... 45 Annex 8. Map Showing Project Roads under APL2 ...... 46

A. Basic Information

ET-Road Sector Project Country: Ethiopia Project Name: (APL2) to support the Government's RSDP IDA-39890,IDA- Project ID: P082998 L/C/TF Number(s): 39891,IDA-47180 ICR Date: 08/30/2012 ICR Type: Core ICR Lending Instrument: APL Borrower: GOE Original Total XDR 110.00M Disbursed Amount: XDR 235.09M Commitment: Revised Amount: XDR 235.10M Environmental Category: A Implementing Agencies: Ethiopian Roads Authority Co-financiers and Other External Partners:

B. Key Dates Revised / Actual Process Date Process Original Date Date(s) Concept Review: 12/04/2001 Effectiveness: 06/08/2005 06/08/2005 Appraisal: 05/25/2004 Restructuring(s): 06/04/2010 Approval: 09/22/2004 Mid-term Review: 03/31/2007 11/01/2007 Closing: 06/30/2010 06/30/2012

C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Moderately Satisfactory Risk to Development Outcome: Moderate Bank Performance: Moderately Satisfactory Borrower Performance: Moderately Satisfactory

C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings Moderately Quality at Entry: Government: Moderately Satisfactory Unsatisfactory Implementing Moderately Quality of Supervision: Moderately Satisfactory Agency/Agencies: Unsatisfactory Overall Bank Overall Borrower Moderately Satisfactory Moderately Satisfactory Performance: Performance:

i

C.3 Quality at Entry and Implementation Performance Indicators Implementation QAG Assessments Indicators Rating Performance (if any) Potential Problem Project Quality at Entry No None at any time (Yes/No): (QEA): Problem Project at any Quality of No Satisfactory time (Yes/No): Supervision (QSA): DO rating before Satisfactory Closing/Inactive status:

D. Sector and Theme Codes Original Actual Sector Code (as percent of total Bank financing) Central government administration 9 9 Roads and highways 90 90 Sub-national government administration 1 1

Theme Code (as percent of total Bank financing) Infrastructure services for private sector development 50 50 Other financial and private sector development 25 25 Rural services and infrastructure 25 25

E. Bank Staff Positions At ICR At Approval Vice President: Makhtar Diop Callisto E. Madavo Country Director: Guang Zhe Chen Ishac Diwan Sector Manager: Supee Teravaninthorn C. Sanjivi Rajasingham Project Team Leader: Fiona J Collin John D. Riverson ICR Team Leader: Bernard Aritua ICR Primary Author: Bernard Aritua

F. Results Framework Analysis

Project Development Objectives (from Project Appraisal Document) The objectives of the RSDP 2 Stage 2 (the Project) is to support the increase in road transport infrastructure and improve its reliability, strengthen the capacity for road construction, management and maintenance, and create conditions conducive to private sector participation in the road transport sector.

ii

Revised Project Development Objectives (as approved by original approving authority) The original PDOs and indicators were not revised

(a) PDO Indicator(s)

Original Target Actual Value Achieved Baseline Formally Revised Indicator Values (from approval at Completion or Target Value Target Values documents) Years Indicator 1 : Increase road density (km/1000 km2) Value quantitative or 30.80 36.24 45.7 48.10 Qualitative) Date achieved 12/31/2003 6/30/2009 4/28/2010 6/30/2012 Comments Road density across the Ethiopia classified network increased by 56 percent during the (incl. percent period of APL2 achievement) Indicator 2 : Increase proportion of roads in good condition (percentage) Value quantitative or 32 50 83 88 Qualitative) Date achieved 12/31/2003 6/30/2009 4/28/2010 6/30/2012 Comments The proportion of roads in good condition across the classified network of Ethiopia (incl. percent increased by 175percent over the life of APL2 achievement) Social, environmental, economic and planning management capacity of the clients Indicator 3 : improved Related activities under Value APL2 completed and quantitative or N/A YES N/A taken over by APL3 & Qualitative) 4 Date achieved 12/31/2003 6/30/2009 4/28/2010 6/30/2012 Comments The management of social and environmental issues has improved during the life of (incl. percent APL2. ERA has set-up a unit to deal with Environment and social issues and at the achievement) time of the ICR mission was carrying out industry wide training. ERA District Maintenance Organizations as profit centers by July 2005 and Indicator 4 : commercial center by July 2006 Value quantitative or N/A YES Indicator changed Qualitative) Date achieved 12/31/2003 7/31/2006 4/28/2010 6/30/2012 Comments At the time of second additional financing (AF) this indicator was changed as per (incl. percent indicator 5 achievement)

iii

ERA unbundled into ERA regulatory entity, and District Road Maintenance Indicator 5 : Contractors commercial enterprise Value quantitative or N/A YES N/A Qualitative) Date achieved 4/28/2010 7/31/2011 6/30/2012 Comments (incl. percent This indicator was introduced at the time of the second additional financing achievement) Share (percent) of periodic maintenance (in value terms) contracted on a competitive Indicator 6 : basis to domestic contractors Value Replaced by quantitative or 40 90 Indicator 5 Qualitative) Date achieved 12/31/2003 6/30/2009 4/28/2010 6/30/2012 Comments (incl. percent This indicator was removed at the time of second additional financing achievement) Share (percent) of routine maintenance (in value terms) contracted on a competitive Indicator 7 : basis to domestic contractors. Value Replaced by quantitative or 0 90 Indicator 5 Qualitative) Date achieved 12/31/2003 6/30/2009 4/28/2010 6/30/2012 Comments (incl. percent This indicator was removed at the time of second additional financing achievement)

(b) Intermediate Outcome Indicator(s) Actual Value Original Target Formally Revised Achieved at Indicator Baseline Value* Values (from Target Values Completion or approval documents) Target Years Indicator 1 : Roads upgraded/rehabilitated (non-rural) Value (quantitative 253 km 604 km 534 km 534km or Qualitative) Date achieved 4/28/2010 6/30/2009 4/28/2010 6/30/2012 This indictor was revised at second Additional Financing as the 70 km section of Comments Shashemene-Dodola road was dropped from IDA financing and taken over by (incl. percent Government of Ethiopia. It was completed in 2009. Consequently, the formally revised achievement) target is 534Km Indicator 2 : Road constructed (non-rural) – federal link and regional roads Value Federal link 119 Federal link 79 km Federal link 119 km Federal link 119 km (quantitative km Regional Regional roads 53 km Regional roads 173 km Regional roads 137 km or Qualitative) roads 137 km Date achieved 4/28/2010 6/30/2009 4/28/2010 6/30/2012 Comments This indicator was revised at second Additional Financing (incl. percent

iv achievement) 10 construction supervision contracts efficiently performing supervision of the above Indicator 3 : works contracts. Value (quantitative 8 10 8 Achieved or Qualitative) Date achieved 4/28/2010 6/30/2009 4/28/2010 6/30/2011 Comments (incl. percent This indicator was revised at second additional financing achievement) No of Wereda Travel and Transport Plans and Wereda Integrated Development Plans Indicator 4 : completed Value (quantitative 24 40 45 or Qualitative) Date achieved 4/28/2010 6/30/2009 6/30/2012 Comments The 45 Wereda plans financed by APL2 were completed. The Government of Ethiopia (incl. percent is in the process of implementation alongside 600 other Wereda plans. achievement) Indicator 5 : Financial Management System updated Value (quantitative N/A YES N/A Achieved or Qualitative) Date achieved 12/31/2003 7/31/2007 6/30/2012 Comments (incl. percent Installed, now part of the ERA systems achievement) Indicator 6 : Pavement Management System updated Value (quantitative N/A YES N/A Achieved or Qualitative) Date achieved 12/31/2003 7/31/2007 6/30/2012 Comments (incl. percent New system installed under remit of the Asset Management directorate achievement) Indicator 7 : ERA employees trained Value (quantitative N/A 60 N/A Achieved or Qualitative) Date achieved 12/31/2003 6/30/2009 6/30/2012 Comments Training undertaken as part of a wider effort within ERA to improve capability and (incl. percent capacity across the organization. achievement) * Target value at time of second additional financing appraisal became the revised baseline.

v

G. Ratings of Project Performance in ISRs

Actual Date ISR No. DO IP Disbursements Archived (USD millions) 1 12/15/2004 Satisfactory Satisfactory 0.00 2 05/03/2005 Satisfactory Satisfactory 0.00 3 12/16/2005 Satisfactory Satisfactory 8.00 4 06/26/2006 Satisfactory Moderately Satisfactory 8.00 5 12/26/2006 Satisfactory Moderately Satisfactory 32.42 6 06/27/2007 Satisfactory Moderately Satisfactory 34.53 7 12/11/2007 Satisfactory Satisfactory 60.67 8 06/02/2008 Satisfactory Satisfactory 78.59 9 12/20/2008 Satisfactory Satisfactory 101.85 10 06/10/2009 Satisfactory Satisfactory 130.14 11 11/30/2009 Satisfactory Satisfactory 162.70 12 06/10/2010 Satisfactory Satisfactory 195.23 13 01/04/2011 Satisfactory Moderately Satisfactory 243.05 14 06/27/2011 Satisfactory Satisfactory 294.93 15 01/09/2012 Satisfactory Satisfactory 332.18 16 07/07/2012 Satisfactory Satisfactory 360.61

H. Restructuring (if any) The project was restructured once to extend the closing date from June 30, 2010 to June 30, 2012.

I. Disbursement Profile

vi

1. Project Context, Development Objectives and Design

1.1 Context at Appraisal

1. The Government of Ethiopia (GoE) prepared the Sustainable Development and Poverty Reduction Program-2002 (SDPRP) to address the development challenges facing Ethiopia at the turn of the twenty first century. This plan, along with subsequent policy documents,1 underscores the need to make substantial investments in infrastructure as a means to create opportunities for diversified and inclusive growth.

2. Transport infrastructure plays a vital role in facilitating economic development in Ethiopia. In particular, road transport provides the predominant means for movement of people, utilization of land and natural resources, improved agricultural production and marketing, access to social services, and opportunities for sustainable growth. Road transport accounts for 95 percent of the country’s passenger and freight traffic and is the only means of access to rural communities. In spite of the relative importance of road transport, the condition of the infrastructure at appraisal was poor. Therefore, improving the quality and extent of road infrastructure was rightly considered a priority for the GoE. By the end of the twentieth century, Ethiopia’s classified road network as measured by density and quality was severely inadequate to meet the growth aspirations of the country2.

3. To address the constraints in the road sector the GoE formulated the Road Sector Development Program (RSDP) in 1997. RSDP is a multi-phased program3 supported by contributions from various development partners. The overall aim of RSDP is to restore the essential road network and create adequate capacity in the road sector4.

4. The World Bank support to RSDP was designed as a four stage Adaptable Program Loan (APL 1-4). The goal of the APL stages was to provide sustained support to the RSDP. APL1 which started in June 2003 was a US$126.8million IDA Grant designed to assist the GoE in improving the quality of priority federal roads, strengthen road sector reforms and enable private sector participation in the road sector. APL2 (US$160.9 million IDA Credit) was designed to build on APL1.

5. The rationale for World Bank’s involvement in APL2 was the continuation of support provided through APL1 and the subsequent stages (i.e. APL 3 and APL 4). The World Bank also had the potential to add value to RSDP by drawing on international/regional experience in similar programs. At the time of preparation for APL2, the World Bank was also providing road sector reform assistance in the region and

1 The subsequent plans include the Plan for Accelerated and Sustained Development to End Poverty – 2005 (PASDEP) and Growth and Transformation Plan (GTP) 2010-2015 2 At appraisal the road network was estimated to include 33,000 km - 16,000 km classified as federal and 17,000 km as regional roads. Only 12 percent of the road network was paved 3 The first RSDP was officially launched in September 1997 and ended in 2002. RSDP II started in 2002 and ended in 2007. The third phase (RSDP III) commenced in July 2007 and ended in June 2010. The fourth phase (RSDP IV) commenced in July 2010 and will end in 2015 4 Assessment of 14 Years Performance of the Road Sector Development Program, ERA Nov 2011 1

could therefore create an appropriate framework to shape the implementation and monitoring. The World Bank’s participation could therefore support long term road development in Ethiopia and contribute to economic growth aspirations of the GoE.

1.2 Original Program Objectives

6. The primary objective of the Adaptable Program Loan (APL) 5 is to provide support to restore and expand Ethiopia’s road network in order to reduce poverty and increase employment through promoting growth and access in a socially and environmentally sustainable manner.

Original Project Development Objectives (PDOs) of APL2 and Key Indicators

7. The objectives of APL2 (the project) 6 as outlined in the Project Appraisal Document were to increase the road transport infrastructure and improve its reliability; strengthen the capacity for road construction, management and maintenance; and create conditions conducive to private sector participation in the road transport sector. These objectives were to be achieved through: (a) construction, rehabilitation, upgrading and preserving the priority federal and regional road network; (b) strengthening of Ethiopia Road Authority (ERA) reform program; (c) developing the capacity in program implementation; and (d) improving rural travel and transport services.

8. According to the Project Appraisal Document (PAD), the key project indicators against which the project was to be measured were:

. Increase road density from 30.8 km/1000 sq. km in 2003 to 36.24 km/1000 sq. km by 2009.

. Increase proportion of roads in good condition from 32 percent in 2003 to 50 percent by 2009.

. Improve the social, environmental, economic and planning management capacity of the clients.

. Enable ERA District Maintenance Organizations (DMOs) to operate as profit centers by July 2005 and commercial centers by July 2006.

. Increase the share (percentage) of periodic maintenance (in value terms) contracted on a competitive basis to domestic contractors.

5 Project Appraisal Document (PAD), Federal Democratic Republic of Ethiopia, Road Sector Development Phase I Project (APL2), May 19, 2003 6 Project Appraisal Document (PAD), Federal Democratic Republic of Ethiopia, Road Sector Development Phase II Project (APL2), August 23, 2004 2

. Increase the share (percentage) of routine maintenance (in value terms) contracted on a competitive basis to domestic contractors.

1.3 Revised PDO and Key Indicators, and Reasons/Justification

9. The indicators relating to enabling the operation of ERA DMOs as profit centers and increasing the share of domestic contractors in road maintenance were revised in April 2010 at the time of the second additional financing. It was clear that since the scope of APL2 did not include periodic and routine maintenance and ERA was already reforming its approach to road maintenance these indicators could be dropped.

1.4 Main Beneficiaries

10. The main beneficiaries are not explicitly identified in the PAD for APL2. It may however be inferred from the PDOs and key indicators that APL2 was expected to benefit the general population and ERA. Successful implementation of APL2 would also benefit local consultants and contractors who would be targeted for increased participation.

1.5 Original Components

11. The original components as presented in the Project Appraisal Document for APL2 are:

. Component 1 - The upgrading of Federal Trunk and Link Roads which consists of: (a) Assela-Dodola-Goba and Shashemene - Dodola section (316 km) of the Nazareth-Dodola/Shashemene-Goba federal trunk road; (b) GobGob-Weldiya section (194 km) of the Woreta-Weldiya federal link road; (c) Shire-Adwa- Adiabun section (83 km) of the Adigrat- Shire federal link road, and (d) consultancy services for construction supervision of the road works, and implementation of the associated environmental, safety, and resettlement plans and adverse social impact mitigation.

. Component 2 - Construction of Federal Link and Regional Rural Roads which consists of: (a) the construction of Magna-Mechara section (119 km) of the Dera- Mechara federal link road to a high standard gravel road; (b) The upgrading of existing and construction of new sections of the Assosa/Sherkole-Guba regional road to a gravel road surface standard (137 km); and (c) consultancy services for construction supervision of the road works, and implementation of the associated environmental, safety, and resettlement plans and adverse social impact mitigation.

. Component 3 - continued support to the Ethiopia Rural Travel and Transport Program (ERTTP) focused on preparation of additional Wereda studies and augmentation of existing TA support.

3

. Component 4 - continued support to ERA (ERA) to enhance its capacity to plan road network maintenance and improvements; (a) by integrating Pavement Management System with Highway Development and Management Model 4 (HDM4); (b) refining ERA’s financial management and management information (MIS) systems; and (c) technology transfer to and training of ERA staff in the relevant fields to further enhance ERA’s capacity.

. Component 5 - preparation of follow up operations in APL Stages 3 or 4, including feasibility of private sector participation in financing and implementation of road programs nationally as well as in Addis Ababa.

1.6 Revised Components

12. The original components were not officially revised. However, the GoE took over funding and delivery of the Shashemene-Dodola section which was constructed by domestic contractors and supervised by domestic consultants. It was completed in 2009.

1.7 Other Significant Changes

13. In September 2004 the Bank approved an IDA credit of US$160.9 million to support the RSDP II. The IDA Board approved two packages of additional financing during the life of APL2.

14. The first additional financing was approved on June 22, 2006. Additional credit of US$87.3 million was approved to address financing shortfalls between the estimates for civil works at appraisal and contract bids7.

15. The second additional financing of US$100 million was approved on May 27, 2010. This additional financing was due to design changes and increases in quantities of various works items 8 ; principally change of pavement designs for Assela-Goba and Gobgob-Gheshena roads from Double Surface Treatment to Asphalt Concrete.

16. The scope of APL2 was also changed when the GoE took over financing of the 70 km Shashemene-Dodola road. This was eventually completed by domestic contractors in 2009.

2. Key Factors Affecting Implementation and Outcomes

17. APL2 contributed to enabling the RSDP to achieve its overall objectives. The road projects constructed under APL2 are critical to the development priorities of the GoE and the objectives of the RSDP. However, APL2 experienced significant cost increases. The project had two separate cases of additional financings - US$87.3 million in June 2006 and US$100 million in April 2010 respectively. The initial cost of APL2 at

7 Project Paper – Proposed Additional Financing in the Amount of US$87.3million, APL2, May 23, 2006 8 Project Paper – Proposed Second Additional Financing in the Amount of US$100 million, APL2, April 28, 2010 4

project appraisal was US$255.9 million. This increased to US$603.23 million at project close in June 2012.

18. In brief, the cost increases were largely associated with the engineering designs at project preparation. In some instances the designs were incomplete, and in others the initial designs did not fully take account of realities on ground. Consequently, changes had to be made during implementation leading to cost increases. Subsequent paragraphs highlight how these factors affected project implementation and outcomes. A discussion of the reasons for the cost increases is given in Annex 2.

2.1 Project Preparation, Design and Quality at Entry

19. APL2 was responsive to the needs of the GoE to significantly increase investment in road infrastructure as a means to create conditions conducive to economic growth. The drive to improve road infrastructure was steered by the GoE and hinged on the RSDP. As a result there was a strong commitment from the GoE from the start of APL2.

20. The focus of RSDP was on improving the quality and extent of road infrastructure in Ethiopia. Given the policy directions and the financing shortfalls needed to achieve road network targets, APL2 was therefore well placed to support the GoE in its growth aspirations.

21. The final packaging of APL2 resulted from changes in World Bank support to RSDP. Initially, World Bank support to RSDP was designed as a three stage Adaptable Program Loan (APL). However, the funding ceiling for stage one (APL1) could not accommodate the identified scope. As a result, stage one was divided into two phases - a grant and loan portion respectively. This led to a four stage APL with APL2 covering the five roads that could not be funded under APL1. The engineering design and feasibility studies for the respective roads had been undertaken as part of APL1. Hence in preparation for APL2, the projects prepared in June 2003 under APL1 were simply carried forward to APL2 in August 2004.

22. The cost increases in APL2 and the need for additional financing can be traced to project preparation, quality of engineering design and construction risk management at project preparation stage. Although a Quality at Entry review carried out in July 20039 (as part of APL1) rated the project’s technical soundness as Satisfactory, it became clear during implementation that the project costs in APL2 could have been controlled if the engineering designs had been advanced to a point where reliable cost estimates were obtained.

9 Projects in APL2 were originally designed as part of APL1. Due to funding constraints in APL1, some of the project roads were eventually packaged as APL2 which was appraised soon after APL1 commenced. Hence a separate Quality at Entry review was considered unnecessary. 5

2.2 Implementation

23. The principal factors that affected implementation are related to impact of pre- construction decisions, and project management and supervision of contracts.

Impact of Pre-construction Decisions

24. The project got off to a challenging start with high bids relative to the Engineer’s estimate 10 (with bids up to 200 percent higher than the Engineer’s cost estimate). Moreover, four contracts received single bids at high prices – 150 percent more. These bids were re-tendered but three of them were awarded to one contractor. To address the shortfall resulting from the disparity between the engineer’s estimate and the bids, additional financing of US$87.3 million was needed11. It was asserted in the additional financing project paper that the disparity between engineer’s estimate and the bids was related to increased prices of fuel and bitumen between project conception and project effectiveness12. While this explanation may have some merit, the lack of realism of the Engineer’s estimate cannot be explained simply by changes in commodity prices. Best practice shows that realism of prices usually results from complete designs and accurate information for developing cost estimates13. Lack of realism in cost estimates can also result from limited understanding of construction processes 14. The differences in the Engineer’s cost estimate and bids received from contractors may have been averted with rigorous project risk management and value engineering to get a better understanding of potential sources of cost increase and contingency requirements. Such a course of action would have led to review and updating of the Client’s cost and budget estimates in line with optimal engineering designs and prevailing market rates for key construction materials.

Project Management and Supervision of Contracts

25. By Mid Term Review in October 2007, most contracts were underway and the first additional financing had covered the differences between the engineer’s estimate and the bids received. The main emerging project challenge was that several of the contracts were substantially delayed due to incomplete designs and bidding documents, and issues with project supervision consultants15. Annex 2 summarizes the specific issues with respective projects and contains a detailed case study to elaborate on the highlighted issues.

26. In April 2010, a request was made for a second additional financing of US$100 million to cater for design changes in road pavement from double bituminous surface treatment (DBST) to Asphalt Concrete (AC). It was explained that the design changes were necessary to take account of increased traffic; more than was anticipated during the

10 ISR#4 11 Additional financing project paper, May 23, 2006 (Report No.36265-ET) 12 Confirmed from commodity price indices for the period 2003-2005 13 Smith N J Managing Risk in Construction Projects Blackwell Publishing 2006 14 Kerzner, H Project Management – A Systems Approach to Planning, Scheduling and Controlling, John Wiley &Sons 2009 15 Mid-term Review Aide Memoire, APL2, November 2007 (key issues in progress of civil works, page 3) 6

design stage of the Assela-Dodola-Goba and Gob Gob-Gashena road projects. In addition, increased quantities of various work items (such as changes in urban and town sections) were alleged to have contributed to the cost increases. The need for this second additional financing could have been avoided with better project management. Experience from several projects demonstrates the value of completing much of the technical engineering design and agreeing on a project execution plan before commitment to construction16. Moreover, the details of traffic growth and its impact on design of the project roads should have been taken into consideration and concluded during project appraisal17.

27. In order to get to the root causes of the cost increases relative to project award, a detailed analysis of all the projects was carried out during the ICR mission. The reasons for cost increase in APL2 are summarized in Annex 2. In brief, the cost increases resulted from variation orders and changes in design/correction of design errors and the associated impact on price adjustments 18 . The variation orders resulting from design changes amounted to US$87.78 million across the road projects and led to project delays of 2-3 years on several projects. Concomitant price adjustments and extension of time amounted to US$168.5 million19. For example in the case of Shire-Adiabun which is detailed in Annex 2, the contract award price of US$48.52 million increased to US$108.85 million. This increase was principally due to changes in design of town sections and a by-pass round the UNESCO world heritage site in Axum. The changes in design resulted in 682 days extension time and an associated price adjustment of US$36.4 million. In the case of Dodola-Goba the contract price of US$33.28 million increased to US$116.82 million due to a change from DBST to AC worth US$9.38 million and price adjustment of US$41.7 million (other design changes included widening in town sections, and new access culverts). In both cases such design changes should have been taken care of as part of the pre-construction engineering design and management. Similar observations can be made for Megna-Mechara, Geshena-Woldiya and Assela-Dodola all of which were characterized by significant cost overruns.

2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization M&E Design

28. The project indicators were agreed upon during project preparation. The expectation was that ERA would provide a point of contact for monitoring the outcomes and providing data for the World Bank team to monitor project progress. However two fundamental aspects of the M&E design were not well articulated or missed altogether. First, the design of the capacity building component could have been more thorough.

16 Halpin, D and Senior, B, Construction Management, Hamilton Publishing , 2010 17C.A. O’Flaherty, Highways – The Location, Design, Construction & Maintenance of Pavements, Butterworth Heinmann, 2005 18 Price adjustments are contractual provisions to respond to price volatility of construction materials and supplies such as asphalt, fuel, cement and steel. In many cases, prospective bidders cannot obtain firm price quotes from material suppliers for the duration of the project. This leads to price speculation and inflated bid prices to protect against possible price increases. Price adjustment allows the contracting agency (the Client) to take on a portion of these risks and in return get lower bids. 19 According to the price adjustment formula used for APL2 contracts, adjustments have to made for the cost of foreign labor, equipment, fuel, bitumen and steel. The formula and indices were agreed at contract award. Any delays in project or client initiated changes in scope would have the effect of increasing the value of price adjustment and hence cost of the project. 7

Genuine capacity building is supposed to enable the institution to diagnose the shortfalls in capacity and capability and to map-out a growth and maturity path. Second, the project indicators should have been more specific, relevant and measurable. Due to the weaknesses in M&E design, it is difficult to relate the key indicators to APL2. For example, the first and second indicators focused on increasing road density and percentage of roads in good condition across the entire 54,000 km of classified road network under ERA. This indicator is useful for measuring the progress of the entire RSDP. However, it is difficult to attribute improvements in such indicators to APL2 which is less than one percent of Ethiopia’s classified road network. ERA has an annual road budget of US$1.5 billion whereas the average annual disbursement for APL2 was US$50-70 million. Consequently, while it is possible to argue that APL2 contributed towards improvement in overall road density and condition of ERA’s classified network, it is not realistic to attribute such improvements to APL2. The same can be said of the indicators related to participation of domestic contractors in routine and periodic maintenance as APL2 never had any maintenance or indeed participation of domestic contractors.

29. In summary, it is obvious that the indicators chosen for M&E in APL2 would not be used for similar projects today. With the benefit of hindsight and improvements in Bank wide design of results indicators, it now seems that more pragmatic measures of impact such as reduction in travel time, improvement in riding quality of project roads and savings in vehicle operating costs could have been used at the program level, while measuring the delivery of physical infrastructure i.e. kilometers constructed at the project level. These would have provided measurable indicators that could be directly attributed to the investments in APL2.

M&E Implementation

30. For the chosen indicators, a baseline was established for APL2 based on the RSDP. ERA supplied the relevant data and Bank followed through on the progress through regular supervision missions.

M&E Utilization

31. The data required to produce the indicators was readily available. At the start of the project, the baseline for chosen indicators was established and the indicators were tracked throughout the project. Of course as noted above, the M&E could have been better designed.

2.4 Safeguards and Fiduciary Compliance

32. At appraisal, the project triggered a category A rating as safeguards were identified in two areas: Environmental and Involuntary Resettlement.

33. Environment and Social safeguards compliance - For the most part, overall safeguards compliance and compliance with environmental safeguards was rated Satisfactory. However in June 2008, a number of issues relating to workers health and

8

safety, and environmental protection begun to be highlighted in the implementation status and results report20. It was only in October 200921 that safeguards compliance issues were reflected in the respective ISR ratings. The environment compliance was downgraded to Moderately Satisfactory. The same rating was maintained till the end of the project. Several environmental and social safeguards issues persisted till the end of APL2 22 . Nonetheless, it was evident during the ICR mission that the implementing agency had started taking steps to address these issues. These included reinforcing of the Environment Monitoring Unit and more recently an industry wide training on environmental and social issues. These efforts are commendable and need to be supported.

34. Procurement and Financial Management – In April 2004, the World Bank carried out a procurement assessment of the capacity of the implementing agency to carryout procurement actions in compliance with World Bank requirements. As a result of this assessment, additional staff was recruited to support the procurement function and several training sessions were held to address the shortfalls. However as noted earlier in section 2.2, the quality of the client’s engineering designs affected the cost estimates and procurement of the projects. Moreover, the bid evaluation and contracts award stage were faced with challenges of limited competition. The lowest evaluated bids were much higher than the engineer’s cost estimates and the client eventually rejected four of the bids and chose to retender the contracts as the received bids were much higher than both the budget and the Engineer’s cost estimates. As highlighted earlier these procurement challenges relate to project preparation decisions and the fact that the client’s cost estimates had not been updated to reflect realistic market prices.

35. During project implementation, one of the contractors, as well as a shareholder of a second contractor were debarred. These decisions did not influence the completion of the works. However, the final variation orders for both contracts were not cleared for Bank-financing. At the time some funds associated with these variation orders were already disbursed. Government of Ethiopia will be required to refund the ineligible payments associated with contract amendments that were not approved by the Bank.

36. ERA received support from several donor organizations to implement financial management improvement in various areas such as internal controls, auditing, accounting, flow of funds, reporting and monitoring. The project seems to have benefitted from these efforts in addition to the country’s discipline in executing budget and compliance with the government’s regulations for projects. To a large extent, ERA has complied with the financial management covenants with the exception of the delays in submission of the entity audit reports.

20 ISR series #9 21 ISR series #10 22 The final ISR series #16 draws attention to protection of cut slopes and safety measures at work. However, overall safeguards compliance is rated Moderately Satisfactory 9

2.5 Post-completion Operation/Next Phase

37. APL2 was part of a four stage program with specific triggers. As a result APL3 was triggered after overall average disbursement of 30 percent of on-going civil works in APL2. As the APL program comes to an end, the post completion phase needs to refocus on areas which have exposed weaknesses in APL1 and APL2. Any future engagement of the World Bank to support the sustainability of the APLs and the RSDP needs to focus on: a) Strengthening the focus on technical engineering design in the early stages of each new project and contract management during implementation - most of the cost increases across APL2 can be traced to inadequate or flawed engineering designs. It was evident from the Implementation Completion Results (ICR) mission that ERA has several competent and dedicated technical staff. The experience of APL2 should therefore trigger a mechanism for consolidating various competencies to scrutinize individual projects and identify/mitigate risks resulting from inadequacies in engineering designs. In addition, emphasis needs to be placed on contract administration and engineering project management during implementation. Both are operational responsibilities that focus on achieving time, cost and quality objectives of selected projects. A central part of the client capacity and capability building has got to tackle the weaknesses exposed in delivery of APL2. Such efforts should necessarily seek to introduce best practice in risk management, design management, value analysis and engineering, cost control and contract management. It may even be possible to develop a specific engineering project management framework such as is common in many international public sector client organizations. b) Road asset management- In order to support the sustainability of the investments made through the APL (and more broadly the RSDP), the pavement management and asset decision support systems need to be exploited more fully. This means that use of software (such as HDM4) should be part of a broader context addressing processes, procedures, culture, values and technological enablers. All other aspects of improving overall road asset management should receive equal focus so that the system may be used for making decisions about prioritizing new road construction or rehabilitation efforts. Such an approach would influence policy decisions about contract strategy and asset management such Design & Build contracting or performance based maintenance contracts which have been piloted as part of APL3. ERA has recently introduced the new directorate of asset management which has the mandate of looking after the road assets. For the long term sustainability of RSDP, this new directorate needs to be supported and strengthened by adopting best practice from other countries that have been successful in asset management. Moreover, the three directorates (planning, operations and asset management) need to have a mechanism for integrating activities and focusing on the whole life cycle of individual projects.

38. A road map articulating how the above areas will be addressed would go a long way in supporting the next phase of World Bank engagement and the future of the RSDP after 2015.

10

3. Assessment of Outcomes

3.1 Relevance of Objectives, Design and Implementation

39. APL2 contributed to enabling the RSDP to achieve its overall objectives. The road projects constructed under APL2 are critical to the development priorities of the GoE and the objectives of the RSDP. Support to the RSDP continues to remain a priority for the partnership between the World Bank and the GoE23.

3.2 Achievement of Project Development Objectives

40. The objectives of APL224 (the project) were identified in the Project Appraisal Documents. The following paragraphs assess achievement of these objectives.

. Achievement of objective 1 - increase the road transport infrastructure and improve its reliability. The investment in physical infrastructure under APL2 contributed to increasing the road density and condition of good roads under ERA. By the end of APL2 the eight critical road construction projects were completed; adding 790 km of upgraded roads to the classified road network. However, as discussed in section 2.3, it is difficult to attribute achievements in road density and condition across the entire classified road network under ERA to APL2.

. Achievement of objective 2 - create conditions conducive to private sector participation in the road transport sector. Similar to several of the road projects under ERA, domestic consultants participated in joint venture partnerships with international consultants in APL2. However, achievement of this objective cannot be attributed to APL2. There was no participation of local contractors on any of the road projects financed under APL2 (except the 70 km Shashemene-Dodola road section which was eventually financed by the GoE). Furthermore, the project indicators for percentage of domestic contractor participation were dropped at the time of second additional financing as there was no road maintenance in APL2.

. Achievement of objective 3 - strengthen the capacity for road construction, management and maintenance. This objective was partially achieved. The investments in pavement management systems, financial management systems and management information systems have provided needed support towards efforts to reform ERA. Interviewees in the ICR mission also observed that Bank processes and the experience of APL2 contributed towards the current program by ERA to improve awareness of environmental and social safeguards among ERA staff and in the wider industry. However, the time delays and project cost overruns under APL2 indicate that there is still some way to go in fully achieving this objective. Specific

23 Identified in both the Ethiopia Growth and Transformation Plan (GTP – 2010/2015) and the draft World Bank Country Partnership Strategy FY13-16 24 Project Appraisal Document, APL2, August 23, 2004 11

strengthening is needed in areas such as engineering design, value engineering, risk management and contract administration. This will become especially critical if ERA moves towards performance based contracts and design & build type procurement strategies which require adequate knowledge of assets and more sophisticated client skills to productively engage with the private sector and to obtain value for money.

3.3 Efficiency

Efficiency measured from Outcomes

41. At project appraisal, an economic analysis was conducted for the civil works components of the project which constitute 90 percent of the total project financing. HDM3 analysis was carried out against a ‘do minimum’ case and +/- 20 percent sensitivity to cost increases. All project roads were reported in the PAD to have positive Net Present Values (NPV) and Economic Internal Rates of Return (EIRR) of more than 10.23 percent25. Table 1 below shows the appraisal costs and Economic Rate of Return (ERR) for the roads for which an ex-post economic evaluation was done26.

Table 1 - Comparison of Costs and Economic Indicators for Project Roads under APL2 1. Road Link Length Actual Project Costs Economic Indicators (km) (ETB million) Appraisal ERR Ex-Post ERR Appraisal Contract Final (percent) (percent)

Assela-Goba/ Shashemene-Dodola 318 879 949 2,068 26.7 15.7 Gobgob-Weldiya 194 498 598 1,131 23.8 17.2 Shire-adiabun 83 283 429 962 30.2 27.8

42. For comparison purposes, the original project roads evaluated at appraisal and associated assumptions have been applied to project close. The ex-post economic evaluation was done for rehabilitation and upgrading of three feeder roads for which actual traffic data was available at appraisal and end of project. Of course, as earlier indicated the final project construction packages differed from the appraisal packages. Shashemene-Dodola was funded and delivered by the GoE, Gobgob-Weldiya was delivered as two projects, Assela-Dodola and Dodola-Goba were also packaged as separate projects. Nevertheless, in order to make comparison possible, the original appraisal packaging has been maintained. The costs are given in ETB 2004 prices. Annex 3 provides details of the economic and financial analysis.

43. To compare the appraisal and final actual costs, 2011 prices were discounted using the Ethiopian price index. The ex-post evaluation was done by first replicating the

25 A discount rate of 10.23 percent is used to evaluate all infrastructure projects in Ethiopia 26 As per terms of the M&E the input data was obtained from ERA. HDM4 analysis was carried out by ERA and verified by World Bank Expert 12

appraisal economic evaluation results and then updating the project costs with the actual upgrading costs. The ex-post ERR for the first two roads declined relative to appraisal ERR, but are still above the threshold of 10.23 percent. The decline may be attributed to the cost increases and delays in completion. The ex-post ERR of the third road (Shire- Adiabun) decreased to 24.3 percent from the 30.2 percent estimated at appraisal, due to the higher upgrading costs. In sum, it is evident that while the project experienced significant cost increases, the economic indicators seem to show that the benefits are greater than the final costs. Most of the benefits relate to savings in travel time and vehicle operating costs.

Efficiency measured at Performance

44. The performance of APL2 was affected by pre-construction decisions related to engineering design of the project roads. The initial bids were much higher than the estimates on which project appraisal was based. As a result the project required additional financing. During implementation, the cost of civil works increased considerably due to changes in design and /or incomplete designs.

3.4 Justification of Overall Outcome. Rating – Moderately Satisfactory

45. APL2 contributed to increasing the essential road transport infrastructure in Ethiopia. During the project life of APL2 the road density increased from 30.8 km/1000 sq. km in 2003 to 48.10 km/1000 sq km in 2012. APL2 also contributed to the increase in proportion of roads in good condition from 32 percent in 2003 to 88 percent in 2012. However, as earlier discussed GoE has been investing approximately US$1.5 billion per year in upgrading the road infrastructure. While the improvements in road density and condition cannot be attributed to APL2, the investment certainly contributed to the achievements of the RSDP. APL2 could have made a more significant impact on RSDP and the development objectives of Ethiopia if the project costs were better managed at project preparation.

3.5 Overarching Themes, Other Outcomes and Impacts

46. Gender aspects and social development – APL2 and the associated investments in road infrastructure will have a significant impact on the affected populations. The project roads will contribute to opening up potential opportunities for the local population to access the main urban centers as RSDP reaches the goal of connecting every citizen within 2 km of an all-weather road by 2015. This may have both economic and social benefits as improvement in transport will enable youth and women to have better access to social services such as hospitals and schools. Moreover, as the GoE prioritizes modernization of agriculture, good roads will be the back bone. The Transport and Poverty Observatory Studies27 show that access to roads directly affects the activities of

27 Commissioned by the GoE to monitor poverty impacts of road project investment at local community and at household level and to assess the development impact of road infrastructure and transport operations in terms of accessibility, mobility and income 13

women - for example, by easing access to healthcare and business opportunities. The new roads were also observed to encourage production and use of intermediate modes of transport for rural to urban travel. Implementation of the Wereda Integrated Development plans will benefit from investment in roads by enabling easier access to services for women (water, grinding mills, schools, health care facilities etc.).

47. Institutional change - The overall responsibility for RSDP lies with ERA which is the implementing agency. ERA also manages and administers the Trunk and Major Link Roads Program. ERA was initially organized into three departments – Regulatory and Engineering Services, Planning & Operations and Human Resources & Financial Management. During the life of APL2, a new Asset Management directorate was created as part of the ERA modernization agenda. As a result of the various efforts under the RSDP and support from donors, ERA has evolved and attracted relatively competent staff. However, the remuneration of the staff has not kept pace with investments in physical infrastructure and growth in the economy and the domestic construction sector. The organization has therefore experienced high staff turn-over; especially at mid-level and among technical staff. Several of these individuals have been attracted to the private sector which has more competitive wages. This is an issue which will have to be addressed as the long term sustainability of investments in the road infrastructure hinge on the availability and continuity of competent staff in ERA and other public agencies.

3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops

N/A

4. Assessment of Risk to Development Outcome. Rating – Moderate

48. Given that ERA and the Road Fund are fully operational and have registered improvements in project delivery, the risks to development outcome is considered moderate. The GoE and associated development partners are committed to the growth of the road sector. APL2 has provided much needed funding for improving the road infrastructure. The main challenge in the future will be provision of funding to maintain the condition of the infrastructure and the management ability of ERA in Road Asset Management, Engineering Project Management and extracting value for money from engagements with the private sector. Addressing the future challenges hinges on ERA being able to attract, train and retain high quality staff with the relevant informed client competencies.

49. Overall at the time of ICR review, it is considered unlikely that the roads constructed under APL2 will receive less attention from the Borrower and implementing agency. The main risk to sustainability of the outcomes relates to the institutional issues.

14

5. Assessment of Bank and Borrower Performance

5.1 Bank Performance

a) Ensuring Quality at Entry. Rating – Moderately Unsatisfactory 50. The rating at entry is based on the fact that APL2 as a source of funding for the road infrastructure was responsive to the needs of the GoE to improve the extent and quality of the road network. In this sense the project was therefore timely and relevant. However, as noted earlier, most of the cost overruns can be traced to project preparation. Insufficient engineering design led to costly changes and modifications during construction. Several of the interviews during the ICR mission observed that while the Bank provided ample advice on addressing environmental, social, procurement and financial management safeguards, there was limited engineering advice during project preparation. Part of the value-added by the World Bank should have been the potential to draw on global experience in best practice of engineering design and project management. This would have led to more robust engineering designs and hence avoided the cost overruns.

51. Moreover, there was sufficient experience from the region to design the capacity building and technical support components to enable institutional growth while filling the competency gaps that led to cost overruns.

b) Quality of Supervision. Rating – Moderately Satisfactory

52. The rating is given because the Bank supervision was regular. The interviews during the ICR mission also confirmed that towards the latter part of the project, the Bank provided support in procurement and technical engineering aspects of the project due to new recruitment of appropriate Bank staff. This also led to improved response times for issues relating to the second additional financing and requirements for ‘no objection’.

c) Justification of Rating for Overall Bank Performance. Rating - Moderately Satisfactory 53. The Bank team exercised good judgment in capturing the needs and potential impact of APL2 on development agenda of Ethiopia. However, project development objectives were not properly designed, hence complicating M&E of the project. Also many of the critical problems experienced during implementation have their origin in pre- construction decisions over which the Bank had much influence.

5.2 Borrower Performance. Rating - Moderately Satisfactory

a) Government Performance. Rating – Moderately Satisfactory

54. The provision of adequate road infrastructure is seen as central to the development ambitions of Ethiopia and as such the various phases of the RSDP have continued to receive increasing support from the GoE. This commitment has given the

15

development partners an anchor to provide support when needed. However, the GoE remains the greatest contributor to the road infrastructure development program (70 percent of the road budget was funded by the GoE between 1997 and 2011). The Government was responsive to project needs and fulfilled their obligations relating to covenants and agreements. This included provision of extra funding to cover the cost overruns (150 percent more than was anticipated at appraisal). The GoE also ended up funding the 70 km Shashemene-Dodola road. It is therefore evident that the GoE supported the project. To derive the maximum benefits from the investment in the sector, the Government needs to continue the same commitment towards maintenance of the road assets and strengthening ERA and providing competitive remuneration to attract and retain high caliber staff.

b) Implementing Agency performance. Rating – Moderately Unsatisfactory

55. ERA was the executing agency for APL2. The institution has developed considerably since the formulation of the RSDP. However, the experience in both APL1 and APL2 demonstrate the need to strengthen technical and managerial capability of ERA (see section 2). Of course, building capacity must be seen in a wider context of the original base and it must be acknowledged that improving the capacity of public institutions is a long term undertaking which could not be addressed within the limited scope and timeframe of APL2. The relative improvements must be noted.

56. At an operational level, ERA was ultimately responsible for supervision of the road projects. In general, all projects in APL2 were characterized by cost overruns, costly design changes, delays in schedule, issues with quality of work and engagement with consultants and contractors. The experience demonstrates that there is still scope for ERA to improve in areas of project and design management, construction risk management, value engineering and asset management. While addressing the challenges of high staff turn-over, a more holistic approach to institutional maturity has to be developed; perhaps initially alongside Technical Assistance in specific areas. c) Justification of Rating of overall Borrower Performance Rating – Moderately Satisfactory

57. The overall performance of the Borrower is rated Moderately Satisfactory principally because the targeted roads are essential to the development of Ethiopia. All the roads in APL2 were eventually built. However this was achieved at considerably higher cost than anticipated, with major issues of quality and long after the anticipated time schedule. It may be argued that some of the factors that contributed to these challenges were beyond the scope of ERA (such as the volatile cost of materials). Nonetheless, the most significant factors were within the control of ERA as a client. One of the main reasons for the incomplete designs seems to have been the pressure from the Government to speed up preparation of projects. This may have affected the quality of designs which would have benefitted from more time for preparation.

16

6. Lessons Learned

58. Through the experience of APL2 several project-specific and more general lessons for similar undertakings may be learned:

59. Technical engineering input during project preparation – the main factors affecting implementation and outcomes in APL2 are rooted in the early project stages. Ultimately, the responsibility for this stage rests with the implementing agency. ERA has many competent staff and the reform program has adapted to the increased budget received by the road sector. Nevertheless, the experience of APL2 shows that it may be helpful to create a client hub made up of individuals from ERA (or bought in as required from external sources) to support project preparation. The ICR therefore proposes that ERA set-up a Technical Panel to focus on risk management and value engineering of projects during preparation. This panel would carry out detailed review of engineering designs for new projects and cost estimates. This would also provide an anchor for skills in areas such as: contract management, contract negotiations, risk management, value engineering and design management. Recent Bank funded projects in Ethiopia have started to support ERA in shifting their focus on technical input in the project preparation process by allocating more resources for technical design review.

60. Strategically driven and holistic capacity building, and competitive remuneration to attract and retain the best staff in ERA - It is widely accepted that an organization’s performance is closely linked to the people working within it. Hence the success or otherwise of the public sector client in delivering infrastructure necessary for supporting economic growth is closely linked to identifying, attracting, training and retaining the right individuals. Unfortunately in the case of ERA the growth in the economy has contributed to a high staff turn-over as many are attracted to better remunerated opportunities in the private sector. Organizational reforms that reflect the roles of key staff alongside market competitive salaries are needed. Perhaps even more important is the need to align the needed skills sets to the evolving role of ERA through a top-down and holistic approach to capacity building rather through ad hoc training. Ideally, an institutional maturity approach which allows the organization to diagnose competency needs alongside enablers such as processes, systems, procedures and goals would be a better approach. It would then be possible for the organization to map-out a maturity strategy aligned with current and future needs.

61. Project management and thorough investigation of the reasons for cost increase across the entire ERA portfolio of projects – at the operational level, the project management skills needed to control time, cost and quality need to be addressed. The cost overruns in APL2 and issues with delays and quality could have been alleviated. For the future, ERA needs to focus on improving cost effectiveness by better analysis of the reasons for high costs, understanding the reasons for cost variations and working collaboratively with contractors to reduce costs across the entire project portfolio. The need for this kind of approach should have been highlighted in earlier projects and included in the design of APL2.

17

62. Results chain and design of measurable M&E framework – APL2 delivered on the main project components; albeit at a higher cost. The challenge during the ICR review was attributing achievements in the results indicators to the project. In the future design of similar projects should enable the design of indicators for which direct attribution is possible. Following the results chain, indicators for the project should have included such attributes such as rate of delivery of physical infrastructure for the outputs and intermediate indicators of progress in key activities. The outcome indicators could have been factors such as the riding quality of the constructed roads, savings in travel time and reduction in vehicle operating costs. These are results oriented indicators which are specific, measurable and perhaps more importantly can be attributed to the project.

63. Need for strategic focus on infrastructure asset management – the new asset management directorate needs to work closely with the planning directorate. In order to support the sustainability of the investments made through the APL (and more broadly the RSDP), the pavement management and asset decision support systems need to be exploited more fully to support maintenance management of the entire road network. This means that use of software (such as HDM4) should be part of a broader context addressing processes, procedures, culture, values and technological enablers. Such an approach would influence policy decisions about contract strategy and asset management through performance based contracts which have been piloted as part of APL3. ERA needs to reconcile short-term priorities focused on upgrading roads and the long-term plans for whole life asset management. For this to be possible ERA needs clear focused asset management plans and better inventory of road assets. Also as part of the strategy for better asset management, the pilot projects utilizing the performance based approach to contracting and design & build contract strategies need to be expanded while taking into account the skills sets needed to make them successful.

7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners

a) Borrower/Implementing Agency

64. The official comments of the Government are in Annex 5. Most of the observations and analysis in the ICR was discussed with the implementing agency during the ICR mission. In the comments received, the implementing agency raised two issues:

(i) Reasons for the cost increase – ERA observes that in addition to design changes the high cost of projects could be attributed to turbulence in commodity prices, the global recession, high inflation and shortage of construction materials. As part of the ICR mission, each project was analyzed in detail. Annex 2 of the ICR summarizes the sources of cost increase. It is obvious from figures 2.1 and 2.2 and table 2.1 that price adjustment which made the greatest contribution to cost increases is affected by the issues highlighted by ERA. However, the need for price adjustment which contributed to the cost increase is rooted in design changes and associated delays. These issues are discussed in section 2 of the main report.

18

(ii) Reasons for ERR of project roads – ERA notes that decrease in ERR for Assela- Goba/Shashemene-Dodola and Gobgob-Weldiya is not due to low traffic but rather cost increases. As shown in Annex 3, the traffic on the two roads actually declined relative to appraisal. Nonetheless, the discussion of table 1 acknowledges the dual impact of traffic and cost of construction.

b) Cofinanciers

65. Comments were received from the European Union (EU) and Department for international Development, UK (DfID). Both organizations contributed to the Ethiopia Rural Travel and Transport Programme (ERTTP) and the Wereda studies. The studies were successfully implemented and pilot implementation is complete. The following observations were made about points raised in the report.

(i) Cost overruns – It was noted that the report highlighted a long standing shortfall of ERA. The cost overruns and delays in project schedule are wider issues. The performance appraisal mechanism of contractors and consultants and its effectiveness to date should offer great insight into the problems of cost overruns. While the impact and role of contractors and consultants cannot be underestimated, in the specific experience on APL2, the cost increases relate to preconstruction inadequacies and incomplete engineering designs. The reasons for this conclusion are discussed in section 2 and annex 2.

(ii) Competition in the Sector – It was observed that market is dominated by a limited number of contractors potentially resulting in collusion. As noted in section 2.2 APL2 had limited number of contractors. This can reasonably be expected to affect competition. However, the ICR preparation did not find evidence of collusion.

(iii) Capacity of ERA and high Staff turn-over - it was noted that ERA has limited capacity to supervise technical designs and that the issue of high staff turn-over due to disparities in remuneration relative to the private sector has been highlighted on several occasions. The ICR fully concurs with the observation. The evidence from this ICR should add to the calls for the Government of Ethiopia to revisit this issue which is central to obtaining value for money and the long term sustainability of significant investments being made by development partners and the GoE.

19

Annex 1. Project Costs and Financing a) Project Costs by Component Project Cost by Component Project Costs (USD$ million) Percent Appraisal Completion of Appraisal

1. Upgrading of Federal Trunk and Link Roads

Assela-Dodola 30.00 79.70 266% Dodola-Goba 32.04 116.83 365% Shashemene-Dodola* 17.44 - Gobgob-Gashena 20.00 48.69 243% Gashena-Woldiya 25.02 79.18 316% Shire-adiabun 25.46 108.85 428% 2. Upgrading of Federal Trunk and Link Roads Magna-Mechara 29.16 77.48 266% Sherkole-Assosa 10.00 36.30 363% Assosa-Guba 7.87 34.30 436% 3. Construction Supervision 13.79 16.5 120% 4. Rural Travel and Transport program 2.05 2.99 146% 5. TA support services for ERA 1.60 1.61 101% 6. Preparatory Activities 0.69 0.8 116% Physical Contingencies 20.60 Price Contingencies** 20.18 Total project Costs 255.90 603.23 236% *Project fully financed by GoE at a cost of US$47 million **Total price adjustments at completion - USD$ 168.5 million b) Financing Sources of Funds Appraisal Actual Cost Percent of Estimate (USD$ millions) Appraisal (USD$ millions)

International Development Association (IDA) 160.9 348.2 216% Borrower 95.00 255.03 268% Total Financing 255.90 603.23 236%

20

Annex 2. Outputs by Component

1. The objectives of the APL2 (the Project) were to increase the road transport infrastructure and improve its reliability, strengthen the capacity for road construction, management and maintenance, and create conditions conducive to private sector participation in the road transport sector. These objectives were to be achieved through: (a) construction, rehabilitation, upgrading and preserving the priority federal and regional road network; (b) strengthened ERA reform program; (c) developing the capacity in program implementation; and (d) improving rural travel and transport services. The project had five components.

Component 1 - Upgrading of Federal Trunk and Link Roads28

2. The roads under this component of APL2 were originally earmarked for rehabilitation and/or upgrading under APL1 (IDA Grant H049-ET). Hence, the feasibility studies, environmental impact assessments, and engineering designs for all road projects were carried out as part of preparation of seven road projects financed under APL1. Component 1 originally involved upgrading of 593 km of Federal trunk and link roads from gravel to bituminous paved road surface standard.

• Upgrading of the Assela-Dodola & Shashemene-Goba road (246 km). The project road lies entirely in Oromiya Region in the South Eastern part of Ethiopia and begins at Assela town, which is situated about 170 km away from Addis Ababa. The original scope consisted of the upgrading of the Assela-Dodola & Shashemene-Goba section of the Nazareth-Dodola/Shashemene-Goba gravel road to a double bituminous surface treated standard. At appraisal, the overall road condition was classified as poor. The road is classified as a trunk road and was targeted for upgrade from gravel to double bituminous surface treated road with a 7.0 m width carriageway and 1.5 m wide shoulders. At appraisal, this project was anticipated to cost US$79.48 million. • Upgrading of Shashemene-Dodola Junction (70 km). This road is classified as a trunk road and was to be upgraded from gravel to double surface dressing with a 7.0 m carriageway and 1.5 m wide shoulders. The works were expected to be packaged as 5 lots and contracted to enable increased domestic contractor participation. This project was anticipated to cost US$17.44 million. Actual achieved at completion – 70 km at a cost of US$47 million, financed by GoE.

• Upgrading of the Gob Gob-Gashena -Woldiya road (194 km). Involves upgrading of (i) the Gob Gob-Gashena (86 km) section of the Woreta-Weldiya road from gravel to a double bituminous surface treated road, and (ii) the Gashena– Woldiya (108 km) section of the Woreta-Weldiya road from gravel to an asphalt concrete surface with carriageway width of 7.0 m and 1.5 m wide shoulders. The

28 Figure 2.1 shows the cost profile of the individual roads under component 1 and component 2 at contract award and completion. It is evident that most projects were characterized by significant cost increases. Figure 2.2 and Table 2.1 outline the reasons for cost increases. The reasons for changes in cost profile for each road are subsequently discussed. 21

Woreta-Weldiya road covering a total length of about 293 km lies entirely in the Amhara Region. It is of high socio-economic importance and serves as a major traffic collector, traversing mostly difficult terrain to connect the Wollo and Gondar sub- regions. At appraisal, the road was reported to be fully engineered with a gravel- surfaced road first constructed between the 1977 and 1983. As a result of a major maintenance program, re-gravelling of this road was done under ERRP. The mountainous nature of the area traversed by the road had become a cause for erosion and surface wear; hence, its recommendation for upgrading. This project was anticipated to cost US$45.02 million.

• Upgrading of the Adwa-Shire road (83 km). The road is part of the Adigrat- Axum-Shire road that provides an important east-west link between the two primary North-South trunk roads in the northern region of Tigray. The mountainous terrain, which the road traverses, makes transportation difficult during the wet season. The existing road at project appraisal was a fully engineered gravel surface road with carriageway width of 7.0 m covering an underlying Telford base. The Telford base was exposed over some sections of the road in spite of being re-graveled under the ERRP. This road classified as a major link road and the upgrading was expected to improve it to an asphalt concrete surface. This project was anticipated to cost US$25.46 million. Actual achieved at completion – 83 km at a cost of US$79.18 million. Upgraded to an asphalt concrete surface.

Component 2- Construction of Federal/Link and Regional Rural Roads

3. The priority under the RSDP I was focused on the rehabilitation and strengthening of the highest priority trunk roads that were found in a dilapidated condition. The RSDP II focused on improving road density by opening up new potential development areas and increasing connectivity, along with completion of the rehabilitation and upgrading requirements of the road network. As a result, IDA’s support to the RSDPII also involved the construction of Federal/Link and Rural Roads namely the Magna-Mechara (119 km), and the Assosa/Sherkole-Guba (137 km) roads, respectively,

• The construction of the Magna-Mechara road (119 km). This road section is part of the Dera-Mechara road (239.10 km) project, which is located in the central part of Ethiopia in the Oromiya region. The road starts at Magna and connects to the Dera- Mechara (119 km) section financed as part of APL1. Project works on the proposed federal road link were foreseen to be constructed to a 6.0 m wide carriageway with 1.5 m shoulders and a gravel surface pavement. The Dera-Mechara road branches off at Dera town from the Nazareth-Assela trunk road 24 km from Addis Ababa, and runs in a northeasterly direction towards Mechara from where it connects to the Addis Ababa – Dire/Dawa trunk road. The population and economy of the area traversed by the road is almost totally dependent upon agriculture and livestock. The sector employs more than 90 percent of the economically active population in the project area. The construction of the road was expected to make a significant contribution to improving some of the constraints on agricultural and livestock development in the project area. This project was anticipated to cost US$29.16 million.

22

• The construction of the Assossa/Sherkole-Guba road (137 km) to gravel standard. The road is located in the western part of Ethiopia and the Benishangul Gumuz Region, interconnecting two of the most isolated zones of the Benishangul- Gumuz Region. Improving the road would provide a north-south transport backbone that is crucial for effective integration of the region. The proposed direct route was expected to reduce travel time and generate significant economic interaction between the regions. At the time of appraisal the section from Sherkole to the Abay River was impassable for motor vehicles and other sections of the road were routinely closed in the wet season. This project was anticipated to cost US$17.87 million.

• Construction Supervision. - This involved the provision of consultancy services for the supervision of civil works and the implementation of associated environmental, safety, and resettlement plans and adverse social impact mitigation. The cost of supervision of the road projects in component 1 and 2 was estimated at US$13.79 million. Eight supervision contracts were signed for the above civil works. Actual cost at completion – US$16.5 million

Component 3 - Support to the Rural Travel and Transport Program (US$2.05 million)

4. This component was expected to support the implementation of the ERTTP by financing (i) the preparation of 60 Wereda development plans for future implementation under APL3, other IDA programs and or other source of funding; and (ii) institutional strengthening of ERA in its role as the lead agency for the implementation of the ERTTP and Secretariat of the Central Program Coordinating Board.

• Preparation of 40 Wereda Integrated Development Plans – WIDP. This sub- component was to assist with the preparation of the Wereda Integrated Development plans as part of the planned expansion of the pilot implementation of the ERTTP concept and approaches in eight Weredas under joint funding with Ireland Aid and Department for International Development, UK during APL1. The plans were to be prepared using the methodologies and frameworks contained in the 'Guidelines for Preparing Wereda Plans’. The WIDPs were expected to contain interventions for (a) improvement of community/wereda road networks including mechanisms for their sustainable maintenance; (b) promotion of the use of IMTs; (c) provision of small scale infrastructure facilities (e.g., water wells, health clinics, etc.) which contribute to reducing the travel and time burdens of the community; and (d) improving the institutional and financing frameworks for sustainable service delivery.

• Support to establishment of ERTTP institutional. The ERTTP embodies many ideas and concepts, the successful implementation of which required ERA to be able to discharge its mandate as the lead agency for the implementation of the ERTTP and

23

Secretariat of the Central Program Coordinating Board. This sub-component was expected to assist in financing expansion of the ERTTP Unit within ERA so that it would eventually be able to provide technical guidance to implementers, undertake studies into specific issues arising during ERTTP implementation and the distillation of the lessons and experience emerging.

Component 4 - Institutional Strengthening of ERA (US$1.60 million)

5. The focus of this component was on supporting (i) updating the Pavement Management System and calibrating it with HDM4 for Ethiopia to enable more effective network planning and management, (ii) refining ERA’s financial management system and interface with Management Information System (MIS) at head office to effectively support, administer and manage the process of commercialization of some functions of ERA’s DMOs and head office; and (iii) organizing technology transfer and training activities with regard to the effective implementation of the above two updated systems.

Component 5 - Support to Accomplishment of Program Objectives

6. The purpose of this component was to provide consultancy services to carry out feasibility studies, and ESIA studies, as well as, design and design review for road projects to be included under follow-on operations. The possible projects were those that were planned to be executed under the APL3, 4 and thereafter. This component was anticipated to cost US$0.8 million.

24

Figure 2.1 - Cost Profile for APL2 Projects (US$ Million)

116.83 Dodola-Goba 33.28

108.85 Shire-adiabun 48.52

48.69 Gobgob-Gashena 25.68

79.18 Gashena-Woldiya 41.93

79.70 Assela-Dodola 45.12

36.30 Assosa/Sherkole-Blue Nile 22.62

77.48 Magna-Mechara 52.95

34.30 Blue Nile-Guba 27.41

- 20.00 40.00 60.00 80.00 100.00 120.00 140.00

Completion Contract award

25

Figure 2.2 - Reasons for Increase in Costs

Others 11% Changes in Design Claims 30% 2%

Legislation and policy changes 2%

Price adjustment 55%

26

Table 2.1 - Summary Analysis of Reasons for Cost Increase

Percent of Total Reason for Cost Increase Amount (US$ Million) Contract price*

Changes in Design 87.78 29.5 Widening in town sections 26.97 9.06 culvert redesign & associated costs of replacement 6.18 2.08 Resurvey due to wrong survey data 0.18 0.06 Realignment of road geometry (7.54) -2.54 Changes in quantity due to inaccurate estimate 36.72 12.34 Change of pavement design from DBST to AC 23.05 7.7 Bridge re-design 0.19 0.07 Walkways, central islands and pedestrian safety provisions 1.60 0.54 Other Design modifications - embankment protection etc 0.43 0.15 Legislation and policy changes 5.39 1.81 Tax Law changes 4.70 1.58 Interest on late payment 0.70 0.23 Price adjustment 168.51 56.64 Claims 7.36 2.4 Extension of time 5.39 1.81 Dispute resolution costs 1.39 0.47 Rebate for discount 0.81 0.27 Other minor claims - overtime, advance, interest rates etc (0.43) -0.14 Additional costs awarded after dispute resolution 0.20 0.07 *Total Contract Price for construction works US$ 297.51 million

27

Reasons for Cost Increase on Project Roads 7. Overall, the changes in design associated with widening town sections and the change in pavement structure from double surface treatment to asphalt concrete had a significant impact on the cost of the roads under APL2. Moreover these changes also affected the progress of site activities with associated impact on price adjustment for the main imported materials (fuel, bitumen and steel) and cost of construction equipment. Other modifications in the original design also led to increases in quantities of materials. For example on the Assosa-Sherkole (Blue Nile) the original cost of the contract went from US$22.62 to US$36.30. A total of US$16 million was the result of increased quantities beyond the original estimates.

8. FIDIC conditions of contract in keeping with most other conditions of contract allow for price adjustment at the time of tender. Price volatility of construction materials and supplies such as asphalt, fuel, cement and steel can often result in significant problems for contractors in preparing realistic bids. Prospective bidders may find it difficult to obtain firm price quotes from material suppliers for the duration of projects which are typically two to three years long. To hedge these risks prospective bidders may be forced into price speculation and inflated bid prices to protect against possible price increases. Price adjustment provisions allow clients to take on some of these risks associated with price volatility of major materials often resulting in lower bids.

9. In the case of APL2 most projects experienced major changes in design and delays of up to three years. Hence the impact of price adjustment was significant and greatly contributed to the cost increases. A total of US$168 million of the final cost of APL2 (57 percent of the original contract price) was the result of price adjustments on the various contracts. These are legitimate payments to contractors which are included in contract documentation. The only way to alleviate the impact of price adjustment is to control the number of client initiated changes to the original scope of the project and control the schedule parameters to ensure project delays are not significant. Uncharacteristically, the claims associated with extension of time as a result of the design changes were not significant (less than 2 percent).

10. A detailed case study of the Shire-Adiabun road which was visited during the ICR mission provides further details to elaborate on the reasons for the cost increases on a specific project to illustrate the issues highlighted above.

28

Detailed Case Study - Shire-Adiabun Project

Summary Project Description

11. The Shire – Adiabun road is located in the northwestern part of Ethiopia and is one of the oldest trunk roads in the Country. It is part of the Adigrat – Adwa – Shire road that connects the regional states of Amharic and Tigray. It also provides access to Ethiopia’s neighbors. The mountainous terrain, which the road traverses, makes transportation difficult during the wet season. The project objective was to upgrade the existing gravel road to an asphalt concrete surface.

12. At project appraisal the road consisted of gravel base 5.0 - 7.0 m wide and approximately 150 mm thick, with 0.5 m to 1.0 m earthen shoulder. The project road was designed as an upgraded 7.0 m carriage way asphalt concrete (AC) surface with 1.5 m gravel shoulder in plain and 0.5 m to 1.0 m shoulder in hilly terrain. The project preparation was based on an old design which was reviewed by another design consultant in January 1998 as part of APL1. The project package was eventually funded under APL2. As shown in Table 2.3 below, the project experienced cost overruns and variations. The cost at appraisal was 91 percent lower than the final contract award price and final project cost as at June 30, 2012 was 124 percent more than the contract price. Moreover the project was delivered 2 years later than anticipated. The aim of this case study is to provide in-depth consideration of the reasons for this outcome and to illustrate the challenges experienced by all the projects under APL2.

Table 2.3 - Basic Project Data Project Cost At appraisal (US$ Million) 25.46 Project Cost at Contract award (US$ Million) 48.52 Project Cost at Completion (US$ Million) 108.85 Type of Contract Unit Rate Contract/Re-measurement type ETB – 30 percent [covered by GOE] Currencies & Proportion of Payment EURO – 70 percent [Covered by IDA] Length of Project Road 91.8 km Date of Commencement 20th August 2007 Original Contract Period 1,095 calendar days Original completion date 19th August 2010 Extension of Time 682 calendar days Final Completion date 30th June 2012

Reasons for Variation Orders and Cost Overruns

• Design and construction of the Axum By-pass - The project road traverses a region with the historical town of Axum which is a registered UNESCO World Heritage site due to its archeological and historical value. The design review included an outline

29

description for protecting the archeologically sensitive areas from possible damages during construction. The decision to introduce a by-pass to the town was made at the procurement stage of the works, and it was indicated that design and construction of the bypass for Axum town would be undertaken during the construction phase. This was also indicated in the Contract Documents (addendum-2 of the bid documents and Volume-IV: cross section drawings); it was stipulated that the detail design of the alignment on the bypass would be carried out by the Supervision Consultant. Although the contract agreement states that provisional quantities are included for Axum by pass in the original design, it was noted during implementation that no quantities for the bypass were actually included in the original Bills of Quantities. Consequently, the Supervising Engineer had to quantify the works and estimate the costs. In the end 6.59 km long bypass with 7 m wide double carriageway (2 m wide Central Island and 4 m walkway on both sides) and pipe side drains were designed. The cost of civil works on this bypass is US$11.86 million.

• Widening of carriageway in town sections – During construction local residents in 5 towns along the project roads and their respective administrations requested for town sections to be widened in Shire, Selklaka, Wukromaray, Adiabun and Axum. The supervising consultant therefore recommended variation orders. See summary in Table 2.4 below. Of course the obvious question is why this was not carried out during project design. Consultation with local residents and completed engineering designs would have captured the need for widening in town sections and the extent of such widening. Thus averting the need for variation orders and associated disruptions to project schedule.

Table 2.4 – Summary of Variation Orders for widening carriageway in town sections along Shire-Adiabun Road Cost of Town Description variation (US$)

Provision of 4.7 km long and 10.5 m wide double Shire carriageway with 2 m central island and 4 m wide walkway 6.77 million on both side and provision of pipe side drains

Provision of 2.8 km long 14 m wide double carriageway Adiabun 2.73 million with 2 m wide median and 4 m wide walkway on both sides

Provision of 1.92 km long and 12 m wide carriageway with Seleklaka 1.5 m wide walkway on both sides and provision of 1.92 million masonry U-drains

Provision of 2.0 km long and 12 m wide carriageway with Wukromaray 1.5 m wide walkway on both sides and provision of 2.31 million masonry U-drains

30

Figure 2.3 - Widened Town Section in Seleklaka

New Side walk Central Island

Wide Carriageway

• Other changes in engineering designs - From the start of the project, the dated designs resulted in various changes and modifications that affected the progress of work and had direct and indirect impacts on project costs (see figure below). These include:

. Wrong survey data and non-existent bench marks. The entire project road had to be re-surveyed since many of the benchmarks in project documents were either erroneous or non-existent. This affected the contractors progress and added to project costs.

. Drainage structures – several culverts and bridges had to be redesigned as the engineering designs supplied to the contractor missed out over 100 culverts and 18 bridges.

. Wrong alignments – in several locations the horizontal and vertical road geometry had to be redesigned with impacts on quantities on cuts and fill sections and hard/soft excavations.

. Treatment of road bed – further investigations were needed to establish accurate extents and treatment of areas with expansive soils.

. Extent of right of way in road alignment.

. Design for safety in accident black spots and mountainous road sections.

31

Figure 2.4 – Realignment for safety

More cutting of slope than anticipated

Figure 2.5 – Wrong alignment in project design drawings that had to be changed during construction

Old New alignment alignment

Stone pitching in newly Figure 2.6 – constructed drains washed by Drainage problems rain due to inadequate design require long term solutions based on detailed hydrological analysis and design

32

Impact of changes on project costs

Table 2.5 – Summary of Project Costs

BILL NO DETAILS ETB US$

General Provisions 40,725,154.32 1 4,605,771.68

Earthworks, Subbase, Base and Shoulders 161,965,743.00 2 18,317,358.01

Bituminous Construction 120,988,192.00 3 13,683,041.78 Concrete, Steel and Structures 30,859,110.84 4 3,489,981.10

Ancillary Works 26,773,722.43 5 3,027,948.07 6 Environmental Works 4,091,697.58 462,746.55 7 Labor 322,400.00 36,461.51 8 Materials 3,609,606.00 408,224.88 9 Equipment 753,800.00 85,250.28

Contingency 38,898,942.62 4,399,238.04 Contract Price 428,988,368.79 48,516,021.89

Variation Orders 226,653,976.80 25,633,210.83

Reconciliation after Remeasurement (65,672,209.59) (7,427,134.60) Total works 589,970,136.00 66,722,098.12

Total Price Adjustment 321,808,871.51 36,394,661.00

Change in Legislation after Contract 13,727,372.25 1,552,483.80 Interest on Late Payment 3,209,823.14 363,011.82

Material Advance 9,101,834.25 1,029,363.08

Deduction in Overtime (356,775.81) (40,349.21) Total current project cost 937,461,261.34 106,021,268.61

Other Quantities Estimated in Final Certificate 25,000,000.00 2,827,350.66 Current Project Cost 962,461,261.34 108,848,619.27

33

Status of the Project at ICR Mission

13. Following the issuance of the variation order No. 1 (modification of the town section design of Shire, Seleklaka, Wukromaray, Adiabun and construction of Axum by pass) the Contractor has been granted an additional extension of time (682 Calendar days) which revised the contract completion date of the project to June 30, 2012. The contractor has now been awarded certificate of substantial completion. In accordance with the Contract Agreement, the Contractor has provided assurance to finalize outstanding project works before December 30, 2012. The Outstanding works include:

. Rectification of defective asphalt works . Junction road construction in Seleklaka and Wukromaray towns . Protection Works installation . Remaining type A and B side ditches construction . Road Marking and guardrail installation . Additional culvert provision at two locations . Shoulder treatment

Critical Review of Lessons from the Shire-Adiabun Case Study

14. The detailed case study of Shire-Adiabun highlights the impact of decisions at the early stages of project preparation on the final outcome of a project. The need to upgrade this vital link was indisputable from the start of the RSDP. The Shire Adigrat Federal road is vital for the economy of the region and the Nation as it provides connections for the regions of Tigray and Amhara. It also completes the North-South circle to the Capital Addis Ababa from both the Eastern and Western parts of the country. The roads along this circle are a priority for the GoE and the final sections are under construction. Hence it was fitting for the Shire-Adiabun road to be included as part of World Bank support to the RSDP. The project is substantially completed and will serve the national interest and the aspirations of the GoE. Nevertheless, the road has been delivered at a high cost. Some of the reasons for the cost increase can be attributed to factors outside the control of any party that participated in the identification, conception and implementation of the project. However, a significant portion of these costs could have been averted by better engineering design and project management. In this respect Shire-Adiabun road project provides vital lessons for future schemes.

• Realistic/market based cost estimates at appraisal and the tender process – the estimated cost of the Shire-Adiabun road at appraisal was US$25.46 million. A review of project documentation reveals that the estimate was based on designs carried out in 1997 – approximately 10 years before the project went out to tender. A design review was carried out in 2003, but the terms of reference where limited. As a result no investigations were carried out with regards to road geometry, hydrology, survey data, soil investigations, drainage structures, etc. Unsurprisingly, bids from contractors were 90-200 percent higher than the client’s cost estimate. This should have been a warning signal that the client’s cost estimates were grossly inadequate and should have prompted cancellation of the tender process and redesign of the

34

entire project; in spite of the political pressures to deliver the project. In the end the lowest bid was US$48.52 million.

• Detailed Engineering design and the construction phase – fundamentally the root cause of the unrealistic estimates would have a more significant impact during the construction phase. As illustrated above the flawed designs had a detrimental impact on the project – it was delayed by 2 years and the final cost is approximately US$108.85 million. This translates to more than 300 percent above the original estimate. The variation orders worth US$25 million could have been avoided if the design involved proper consultation with the towns and synchronization with the urban master plans which were already in the preliminary stages of development. It was known from the start that a bypass round the heritage sites and town of Axum would have to be designed but this was not undertaken. Moreover, during tender, contractors were not even asked to offer a priced bid for the by-pass. Many of the other design changes were avoidable. The changes and associated delays attracted price adjustments worth US$36.4 million. It was fortunate that the contractor was not overly claims oriented and in the end withdrew several claims in exchange for extension of time. The basic lesson here is that having the engineering design advanced to a point were limited changes are required and information supplied to contractors is reliable is core to the projects outcome. More complete and reliable engineering designs also reduce the risk of disputes and claims during the construction phase.

• Whole life performance of road infrastructure – From the site visit it was also evident that many of the solutions proposed during construction did not benefit from design based on analysis of whole life considerations. For example, the drains in the urban sections which were part of the variation orders have been completed covered with prefabricated slabs. Some of them have already been filled with debris from recent rains. It is unclear how these drains will be desilted and maintained after demobilization of the contractor. In several sections of the road, the choices of drainage design and road furniture also leaves questions about long term maintenance.

35

Annex 3. Economic and Financial Analysis Economic and Financial Analysis at Appraisal

a) Economic Analysis

1. For the identified national and regional road projects, the PAD explains that specific traffic counts and projections, as well as preliminary engineering designs were prepared to appraise their economic feasibility. The feasibility studies of the four main roads were undertaken in November 1999 as part of the first phase of RSDP. The feasibility studies for the rural roads under component 2 were carried out in September 2001. These studies on the economic viability of the roads were updated in December 2003. A discount rate of 10.23 percent has been used throughout the analysis.

Assumptions, Input Data and Methodology

2. Traffic Levels: Table 3.1 below shows the Average Annual Daily Traffic for the base year, for the project roads.

Table 3.1 - Average Annual Daily Traffic in PAD –Base year (2002)

Road link Length Car L/R S/Bus L/Bus S/T M/T H/T T/T Total (km) Assela-Dodola/ 316 2 229 150 148 193 163 197 150 1252 Shashemene-Goba GobGob-Woldiya 194 0 180 32 57 91 80 174 222 836 Adwa/Adiabun-Shire 83 0 58 40 30 29 54 41 69 303

3. Traffic Projections: According to the Appraisal documentation, traffic projections on project roads took into account past and future trends in GDP growth, population increase, agricultural activities, tourism, average annual daily traffic, vehicle fleet and fuel consumption. These are among the major factors influencing traffic growth rates. Two sets of growth rates were estimated for 2003-2013 and 2014-2024.

4. It was assumed that the base traffic would grow at fairly high rates but eventually tapper off as the economy stabilizes to steady growth.

5. Costs: Construction was assumed to start in 2005 and last for three years, thus the roads would be open to traffic in 2008. An analysis period of 20 years was assumed.

6. Methodology: The HDM 3 model (in HDM Manager) was used for the analysis. The proposed option was compared with a “do minimum” case. The “do minimum case” assumes that a minimum amount of maintenance of the existing road will be carried out. The economic analysis is based on benefits to road traffic (in terms of savings in VOCs & road maintenance costs) compared with the costs of rehabilitation/upgrading and 36

maintenance. Discounted benefits are then compared to discounted costs to produce measures of worth. Sensitivity analysis was done taking into account three scenarios namely a 20 percent increase in costs, a 20 percent decrease in benefits and the worst case (costs +20 percent & benefits -20 percent).

7. Benefits: The proposed road rehabilitation/upgrading would have essential benefits to the nation’s economy as well as to the environment including: • Reduced vehicle operating cost (e.g. fuel consumption, maintenance cost) • Reduced travel time • Increased road safety • Reduced noise and air pollution and • Area development.

b) Financial Analysis

8. The financial costs of construction and maintenance were estimated based on the engineer’s client estimates and converted to economic costs by applying a conversion factor of 0.83.

Economic and Financial Analysis at End of project

9. During the ICR mission preliminary economic and financial analysis was carried out in accordance with the framework established for M&E. The Borrower provided input data for economic and financial analysis. Further analysis was undertaken by the World Bank expert. Normal traffic was based on ERAs most recent classified traffic count carried out at the end of 2011.

10. The appraisal evaluation was done for rehabilitation and upgrading of four feeder roads: (i) Magna-Mechara, (ii) Gob Gob–Weldiya (iii) Adwa/Adiabun-Shire, and (iv) Assela-Goba/Shashemene-Dodola, and the construction of the Assosa-Sherkole-Goba road to gravel standard. The ex-post economic evaluation was done for the rehabilitation and upgrading of three feeder roads for which actual traffic data is available. Table 3.2 below shows the appraisal costs and Economic Rate of Return (ERR) for the roads for which an ex-post economic evaluation was done. The costs are given on ETB 2004 prices.

Table 3.2 - Appraisal Estimated Costs and Economic Rate of Return Appraisal Appraisal Appraisal Length Cost Cost ERR Road (km) (ETB million) (ETB million/km) (%) Assela-Dodola-Goba, Shasemene- Dodola 318 879.2 2.765 26.7 Gobgob-Weldiya 194 498.1 2.568 23.8 Shire-Adiabun 83 282.7 3.406 30.2 2004 Prices

37

11. According to the PAD, the three project roads were 6.0 meter width gravel roads in poor condition with daily traffic ranging from 303 to 1,252 vehicles per day in 2002. Table 3.3 below shows the project roads characteristics before the improvement works. The roads were upgraded to an asphalt concrete standard with a 50 mm asphalt concrete surface layer over a 25 mm base and 200 mm sub-base with a width of 7.0 meters.

Table 3.3 - Project Roads Characteristics 2002 Width Surface Roughness Traffic Road (m) Type (IRI, m/km) (AADT) Assela-Dodola-Goba, Shasemene-Dodola 6 Gravel 17 1,252 Gobgob-Weldiya 6 Gravel 20 836 Shire-Adiabun 6 Gravel 18 303

12. Table 3.4 below shows the current daily traffic on the project roads and the traffic composition. The roads carry a high percentage of buses and trucks, with around 85 percent of the total traffic being composed of buses and trucks. During the period 2002- 2012, the daily traffic on the first two roads decreased 54 and 24 percent respectively, while the daily traffic on the Shire-Adiabun road increased 109 percent.

Table 3.4 - 2012 Daily Traffic and Traffic Composition 2012 Small Large Small Medium Heavy Articulated Traffic Cars Pickups Bus Bus Truck Truck Truck Truck Road (AADT) (%) (%) (%) (%) (%) (%) (%) (%) Assela-Dodola- 581 1% 15% 19% 9% 14% 17% 16% 8% Goba, Shasemene- Dodola Gobgob-Weldiya 632 0% 17% 24% 6% 9% 14% 12% 17% Shire-Adiabun 633 0% 14% 27% 9% 15% 17% 15% 3%

13. Table 3.5 below shows the contract costs and the actual costs, in current prices. The contracts were signed during the 2006 to 2007 period and the road works were executed during the 2006 to 2012 period.

Table 3.5 - Contract and Actual Costs Contract Contract Actual Actual Cost Year Cost Cost Period Road (ETB million) (year) (ETB million) (years) Assela-Dodola-Goba, 949 2006 2068 2006-2012 Shasemene-Dodola Gobgob-Weldiya 598 2006 1131 2006-2012 Shire-Adiabun 429 2007 962 2007-2012

38

14. To compare the appraisal, contract and actual costs, table 3.6 below shows these costs in 2011 prices, computed using the Ethiopian price index. The actual costs for the first two roads are similar than the appraisal estimates, but the actual cost for the Shire- Adiabun road is 36 higher than the appraisal estimate.

Table 3.6 - Appraisal, Contract and Actual Costs Appraisal Contract Actual Actual per Cost Cost Cost Appraisal Road (ETB million) (ETB million) (ETB million) (ratio) Assela-Dodola-Goba, 2,919 2,516 2,978 1.02 Shasemene-Dodola Gobgob-Weldiya 1,654 1,584 1,628 0.98 Shire-Adiabun 939 1,137 1,280 1.36 2011 Prices

15. The ex-post evaluation was done first by replicating the appraisal economic evaluation results using the Roads Economic Decision Model (RED) and then updating the evaluation with: (i) actual upgrading costs and (i) actual annual traffic growth from 2002 to 2012. Table 3.7 below shows the ex-post Economic Rate of Return (ERR) for each road and the appraisal estimates. The ex-post ERR for the first two roads decreased to 15.7 and 17.2 percent respectively due to the lower actual traffic growth rate and relatively high construction costs. The ex-post ERR of the Shire-Adiabun road decreased to 27.8 percent because the increase in the upgrading costs was not compensated by the increase in the annual traffic growth rate. All roads have an ex-post ERR above 10.23 percent threshold for infrastructure projects in Ethiopia.

Table 3.7 - Ex-Post Economic Evaluation Results Appraisal Ex-Post ERR ERR Road (%) (%) Assela-Dodola-Goba, Shasemene-Dodola 26.7 15.7 Gobgob-Weldiya 23.8 17.2 Shire-Adiabun 30.2 27.8

39

Annex 4 - Bank Lending and Implementation Support/Supervision Processes a) Task Team Members Names Title Unit Bernard Aritua Transport Specialist (ICR TTL) AFTTR Petrus Benjamin Gericke Lead Transport Specialist AFTTR Haileyesus Adamtei Highway Engineer AFTTR Fiona J Collin Senior Transport Engineer AFTTR Tesfamichael Nahusenay Mitiku Senior Transport Engineer AFTTR Farida Khan Operations Analyst AFTTR Yoshimichi Kawasumi Consultant AFTTR Shalonda Robinson Program Assistant AFTTR Lemlem Workalemahu Program Assistant AFCE3 Asferachew Abate Abebe Environment Specialist AFTEN Tesfaye Ayele Senior Procurement Specialist AFTPE Lillian Namutebi Financial Management ETC AFTME Yasmin Tayyab Senior Social Development Specialist AFTCS Yeshi Gizaw Program Assistant AFCE3 Negede Lewi Senior Highway Engineer AFTTR Mesfin Wodajo Jijo Senior Transport Specialist SASDT Richard Martin Humphreys Senior Transport Economist AFTTR Antoine Lema Senior Social Development Specialist AFTCS Desta Solomon ET Consultant AFTCS

40

b) Staff Time and Costs Staff Time and Cost (Bank Budget Only) Stage of Project Cycle No. of Staff Weeks US$ Lending (P082998) FY03 0.65 1,390.34 FY04 25.99 125,553.46 FY05 5.81 15,812.74

Lending (P117644) FY10 17.12 81,971.32

Total Lending 49.57 224,727.86

Supervision/ICR FY05 14.55 36,096.50 FY06* 31.28 67,611.28 FY07 32.86 99,333.80 FY08 32.90 146,275.92 FY09 20.45 67,360.00 FY10 32.84 154,228.37 FY11 24.17 92,186.88 FY12 24.10 80,044.08 FY13 0.63 2,823.00

Total Supervision/ICR 213.78 745,959.83 *Includes time spent in preparing the First Additional Financing.

41

Annex 5. Summary of Borrower’s ICR and/or Comments on Draft ICR

Page 16, b)

1. It is known that APL2 road projects were completed with considerably higher cost than anticipated. Other than the changes in Design, this high costs were mainly due to the turbulent of external environment with commodity price surges, global recession, relatively high inflation, shortage of construction materials which forced to import from outside.

2. Given the above situation, it is not clear how the report concluded that the most significant factors for high cost were within the control of ERA as a client.

Page 39, paragraph 15

3. The report concluded that the decrease in ERR for Assela-Dodola-Goba, Shashamane-Dodola and Gobgob-Gashena to 15.7 percent and 17.2 percent respectively are due to lower actual traffic growth rate and high construction cost. However, the Bank knows that because of the low traffic forecast assumed in previous studies for these road projects, a change in the pavement wearing course from DBST to AC were proposed. Therefore, the main reason for decrease in ERR values from appraisal estimates are mainly due to the increase in costs and not low actual traffic growth rate.

42

Annex 6. Comments of Co financiers and Other Partners/Stakeholders

1) Comments from EU

1. Thank you for consulting the European Union on the Implementation Completion and Results Report (your letter WB/CD/330/11/08/2012).

2. We found many interesting information and we'll take in account the main conclusions in the formulation of our Third Road Sector Policy Support Program (SPSP 3); we would like also to discuss about it in the next Transport Sector Working Group, which should take place at the end of November.

3. I would like to add some specific comments on some points raised in your report:

Design and cost estimation Versus Works Contracts

4. This has been an issue since a long time. Almost each and every contract have had a cost and time overrun mainly because of design issues and associated claims (cost and time). ERA had started some time ago a performance appraisal mechanism of both Consultants and Contractors for its government financed projects so as to enable itself devise/formulate ways in decision making during tendering and implementation phases. But this was limited to local contractors and consultants. And its effectiveness to date is not known. So it is good to know how far this mechanism is adopted in the Bank's or other donor's procedures as there are obviously limitations to do that.

5. Moreover, it appears that since a couple years back the market is dominated by a limited number of contractors (mainly Chinese) who are well established here in the country and mainly government controlled corporations with subsidies from their government (financial support) and with high comparative advantage in tendering. There is also the issue of collusion on offers to influence/control tenders.

Capacity of ERA in Design and tendering

6. Since recent years ERA follows a procedure where by designs are checked and verified on site by its own staff but the capacity to mobilise its resources to each and every design contract is limited.

High turnover of staff

7. This has been an issue raised by most donors for decades in almost every annual stakeholder's meeting but ERA's hand is tied due to the government regulations; no matter how much it tried to improve its salaries to its staff and organisational structure it is very clear that it could not reach at a comparable level/pace as the private sector. This issue will definitely persist for some time to come. In one of the stakeholders meeting ERA director General has clearly stated that they have no other choice but simply continue recruiting new engineers when the experienced ones leave.

43

2) Comments from DfID

A. On the Relevance of the Adaptable Programme Loan (APL2) Project

1. We believe that supporting the road sector is key to growth and poverty reduction in Ethiopia and the Adaptable Programme Loan (APL2) of the World Bank has been contributing towards this end.

B. The ERTTP Project and its completion scoring

2. As highlighted in the APL2 Implementation Completion and Results report, DFID has been supporting the Ethiopian Rural Travel and Transport Programme (ERTTP) – component 3 under the APL2 programme.

3. The project was very instrumental in successfully demonstrating the impacts that could be achieved by combining road provision with non-transport projects to improve access and mobility. It has also enabled the development and dissemination of Low Volume Roads Design Manual which would be useful for road works financed by other interventions such as the Productive Safety Nets Programme (PSNP).

4. The project was successfully completed and project completion report was prepared by independent Consultants (see the attached).

5. ERA has demonstrated commitment and leadership for the effective implementation of the programme.

6. So, based on the above considerations the consultancy team put the overall rating of the project as “satisfactory”.

C. On the APL2 Implementation Completion and Results Report (ICR)

7. The report has clearly documented the project development objectives, key factors affecting implementation, assessment at outcome level, etc. That is appreciated.

8. It would have been great if the section analyzing the performance of the project component by component clearly detailed the current status of the projects (what has been achieved) compared to what is planned. For example, for the ERTTP component, the report indicated what is expected under this project and does not show what has been achieved. This would help enhance our understanding on which component work better than others and what should be done to enhance performance in less performing components.

9. We found the lesson learning a bit thin and could be expanded. It might be good if the lesson learning include some aspect of project design, planning, construction, maintenance, monitoring and reporting, etc.

44

Annex 7. List of Supporting Documents Bank Documents

. Project Appraisal Document – APL2 Aug 2004 . Project Appraisal Document – APL3 May 2007 . Assessment of 14 years Performance Road Sector Development Program, Nov 2011 . ICR Road Sector Development Phase 1 Project (APL1), Jun 2010 . APL2 Implementation Status and Results Reports – ISR Series 2-16 . APL2 proposed Additional Financing Project Paper – US$87.3 million . APL2 second Additional Financing Project Paper – US$100 million . Minutes of Decision meeting for APL2 May 2004 and Comments from Reviewers . Ethiopia - Country Assistance Strategy, 2008 – 2011 . Interim Country Assistance Strategy for Ethiopia, 2006 - 2007 . Country Assistance Strategy (CAS) for Ethiopia, 2003 - 2005 . Quarterly progress report on APL2 submitted by ERA, Jan 2012

Other . Growth and Transformation Plan (2010/2015) – Federal Democratic Republic of Ethiopia, Sep 2010 . Joint IDA-IMF Staff Advisory Note on the Growth and Transformation Plan, Jul 2011 . Sustainable Development and Poverty Reduction Program (SDPRP) . Plan for Accelerated and Sustained Development to End Poverty (PASDEP) . Morris, P. W. G. & Jamieson, A., . (2006) Linking Corporate Strategy to Project Strategy via Portfolio and Program Management. International Journal of Project Management, 25, 57-65. . Smith N J Managing Risk in Construction Projects Blackwell Publishing 2006 . Pellegrinelli, S (2010) What’s in a name: Project or Programme? International Journal of Project Management, 29, 232-240. . Winter, M. & Szczepanek, T. (2008) Projects and programmes as value creation processes: A new perspective and some practical implications. International Journal of Project Management, 26, 95-103. . Reiss, G. (1996) Programme management demystified : managing multiple projects successfully, London ; New York, E & FN Spon,.

45

32°E 36°E 40°E 42°E 44°E To ETHIOPIA Keren R ERITREA SUDAN e REP. ROAD SECTOR DEVELOPMENT PHASE II REP. d OF PROJECT (APL2) IN SUPPORT OF THE

To Adiabun Adigrat S YEMEN ETHIOPIA Gedaref Humera T ek Shire Adwa D SECOND PHASE OF THE ROAD SECTOR 14°N eze Axum e 14°N

e TIGRAY Shire-Adiabun n a DEVELOPMENT PROGRAM

Mekele a Ras Dashen k Terara (4620 m) i PROJECT ROADS A tba l ra CITIES AND TOWNS D Gonder REGION CAPITALS AMHARA e Lake AFAR s NATIONAL CAPITAL Dinder Tana e 12°N r 12°N RIVERS Gobgob Weldiya Debra t MAIN ROADS DJIBDJIBOUTI Bahir Dar Tabor Gulf B Geshena Asayita lu of REGION BOUNDARIES Guba e N ile Aden A Dese INTERNATIONAL BOUNDARIES b

a y

y Asosa-Sherkole-Guba

Gobgob-Geshena-Weldiya A Debre w r a e s g Markos h 46°E 48°E BENSHANGUL n a H Asosa E t h i o p i a n 10°N 10°N DIRE DAWA D

i P l a t e a u d Dire Dawa e To

s a Hargeysa Harer Gimbi Jijiga Nekemte ADDIS ABABA ADDIS Awash HARARI ABABA SOMALIA Magna-Mechara Mechara R Nazret a B m aro Magna i Aware Welkite s Degeh Bur Gore Gambela OROMIYA 8°N y Domo 8°N GAMBELA Asela e Asela-Dodola

A l le ko Jima l abe Shebe b Hosaina SOUTH o W Bonga a O g a d e n SUDAN ShashemeneV Goba SOMALI Sodo Awasa Dodola Warder t Kebri Dehar Wendo f W a i Dodola-Goba b Imi e R G SOUTHERN NATIONS, e s tr o W 6°N a 6°N NATIONALITES t be Sheb a ele 0 50 100 150 200 Kilometers AND PEOPLES e Genale r Negele G Ferfer 0 50 100 150 Miles To

Yavello Dawa Mogadishu

Dolo Odo This map was produced by the Map Design Unit of The World Bank. Lake The boundaries, colors, denominations and any other information AUGUST 2012 4°N Mega 4°N

shown on this map do not imply, on the part of The World Bank IBRD 39502 Turkana Group, any judgment on the legal status of any territory, or any endorsement or acceptance of such boundaries. INDIAN Moyale KENYA To OCEAN UGANDA To Wajir To 32°E 34°E 36°E 38°E Marsabit 40°E 42°E Mogadishu 44°E 46°E 48°E