Report Number ::: ICRRICRR1449614496 ICR Review IEG Independent Evaluation Group Public Disclosure Authorized

1. Project Data: Date Posted ::: 09/29/2014

Country ::: Project ID ::: P095460 Appraisal Actual Project Name ::: Integrated Project Costs (((US$M(US$MUS$M):):):): 156.2 210.9 State Highway Management Project LLL/L///CC Number::: L7411 LoanLoan////CreditCredit (((US$M(US$MUS$M):):):): 100.0 100.0 Sector Board ::: Transport Cofinancing (((US$M(US$MUS$M):):):):

Cofinanciers ::: Board Approval Date ::: 11/15/2006 Closing Date ::: 09/30/2013 10/14/2013 Public Disclosure Authorized SectorSector( (((ssss):):):): Roads and highways (87%); Sub-national government administration (10%); General agriculture fishing and forestry sector (3%) ThemeTheme( (((ssss):):):): Infrastructure services for private sector development (40% - P); Municipal governance and institution building (20% - S); Trade facilitation and market access (20% - S); Regulation and competition policy (20% - S)

Prepared by ::: Reviewed by ::: ICR Review GroupGroup:::: Coordinator ::: Jeffrey Balkind Robert Mark Lacey Christopher David IEGPS1 Nelson

2. Project Objectives and Components:

a. Objectives: Public Disclosure Authorized The Loan Agreement between the State of Bahia and the IBRD (Schedule I, page 9) states the project's objective as follows: "to increase effective use of the Borrower's road infrastructure, through : (a) rehabilitation and maintenance of key sections of the Borrower's paved road network; and (b) provision of support for institutional improvements in the Borrower's road sector, while fostering greater integration among the Borrower's regions and with the rest of the country."

According to the Project Appraisal Document (PAD, page 8): "The Project's ultimate development objective is to increase effective use of the Borrower's state road infrastructure, with an aim at stimulating higher economic growth . The objective is to be achieved through (a) rehabilitation and maintenance of key sections of the state paved road network, so as to contribute to the decrease of road transport costs on selected corridors, and to the increase in the use of the state road transport infrastructure, and (b) support to a set of key institutional improvements in the road sector to ensure the sustainability of physical achievements, as well as the fostering of a greater integration within the state’s regions, and of the state with the rest of Brazil ."

This evaluation uses the Loan Agreement version . Public Disclosure Authorized b.Were the project objectives/key associated outcome targets revised during implementation?

No

c. Components: There were two components:

Component 111:1::: Rehabilitation of State Paved Road Sections (((appraisal(appraisal cost $ 140140. ...6666 million, actual cost $$$194$194194....1111 million ).).). The component supported the introduction and execution of long -term, performance-based road maintenance and rehabilitation contracts on 2,000 km of the State of Bahia's paved road network (15 percent of the state highway network). The project listed 11 road sections initially totaling 1,177 km according to an agreed set of readiness criteria. The remaining 823 km of roads were to be identified later. All State road sections rehabilitated under the project had to be part of the road sections identified for maintenance and /or rehabilitation in the State Government's main logistics program (PELTBAHIA 2004), as suitably updated, and be part of road corridors connecting strategic cities or connecting the economically dynamic regions of the state to markets or for exports/imports from other states or countries. Investments also needed to demonstrate a minimum internal rate of return of 20%. Prior to Board presentation of the Bank loan, road sections totaling about 1,200 km for a first phase were identified, and the remainder were expected to be identified afterwards . It was these 1,200 km of works that became subject to performance-based maintenance contracting.

Component 222:2::: Institutional Strengthening (((appraisal(appraisal cost US$ 151515. 15...6666 million, actual cost $ 161616. 16...8888 millionmillion).).).). There were five sets of activities to be financed by the component :  Support the Government's efforts to optimize the role of transport services in order to de -concentrate economic growth, reduce logistics bottlenecks, and integrate the transport dimension into the planning of state -wide education and health services.  Implement the State’s logistics program (PELTBAHIA).  Strengthen the State road administration and planning system .  Strengthen the road administration’s technical capacity to support the various municipalities in maintaining their road networks and in improving transport services for the poor .  Preparation of the project works engineering designs and the carrying out of technical supervision, as well as support the road administration to coordinate implementation.

In September 2010, a Level 2 Restructuring increased the percentage of expenditures to be financed by the Bank loan from 50 to 90 percent. This addressed the issue of delays in counterpart funding . Also, a new outcome indicator was introduced: the reduction of unit vehicle operating costs on selected itineraries or road sections .

d. Comments on Project Cost, Financing, Borrower Contribution, and Dates: Project Cost ... The total cost of Phase 1 paved roads sections was US$194 million versus an appraisal estimate of $140 million. Actual investment costs on the 1,196 km that were completed under the project were 38 percent higher than estimated at appraisal. The costs of the Institutional Strengthening Program were 8 percent higher than the appraisal estimate. The principal cause of the higher cost of the investment component was the use of outdated estimates during preparation that remained uncorrected at appraisal . In addition, a year's delay in loan effectiveness led to cost increases, as did the depreciation of the US dollar against the Brazilian Real (the PAD had estimated an exchange rate of 2.16 R$/US$, but the average exchange rate over the project implementation period was 1.76 R$/US$). Due to the cost increases, a financing gap of $ 77 million emerged, leading to a significant shortfall in physical outputs -- only 1,196 km (about 60 percent) of the planned 2,000 km of road network identified for road rehabilitation and maintenance were completed under the project .

The ICR (page 8) attributes $40.7 million of the civil works cost increase to "a heavier technical solution" (4 cm of asphalt skimming and replacement, instead of 2 cm) and inflation. It attributes $33.7 million to unfavorable exchange rate changes during the project period, and $ 2.9 million to additional km in PREMAR's first phase.

Financing ... The Bank loan of US$100 million became effective on December 6, 2007 (13 months after Board approval). The loan was fully disbursed. There were no other external sources of financing .

Borrower Contribution . According to Annex 1 of the ICR, this was US$112.1 million, compared to an appraisal estimate of $86.0 million.

Key Dates . As noted above, the loan became effective in December 2007, 13 months after Board approval. The project closed on schedule on September 30, 2013,

3. Relevance of Objectives & Design:

a. Relevance of Objectives: High The project remains highly relevant to current conditions and priorities in the country and in Bahia State . The Brazil-World Bank Country Partnership Strategy (CPS) for FY2012-2015, which was approved by the Board in November 2011, focuses on three developmental pillars : equity, sustainability, and competitiveness . The project is an important part of achieving sustainability and maintaining competitiveness of the transport sector . Specifically, the CPS includes among its strategic objectives, the following sub -objective (diagram, page 18) "improving transport and logistics supply chains is critical both to increase export competitiveness and to promote economic development in lagging regions." To address these various challenges, the Government updated its National Plan for Logistics and Transport, which has among its main objectives the need to improve the efficiency of the productive sectors and help reduce regional inequality, thereby rebalancing Brazil ‘s transport matrix, in order to obtain higher efficiencies in the movement of goods and freight sector.

Bahia State's Multi-Year Plan ("Bahia 2020 Plano") attaches high priority to the road rehabilitation program . Located in the North East, Bahia State, with its important of Salvador, is key to achieving a more equitable distribution of transportation services (first pillar of the CPS) in the country as a whole.

b. Relevance of Design: RatingRating:::: Substantial The results framework had a clear causal chain, linking the investment components and the institutional strengthening activities to be financed by the project to the expected outputs and outcomes . Inputs, expected outputs, and expected outcomes were logically spelled out, using an approach that drew on state -level financed projects in large federally administered countries that also have extensive decentralization mechanisms in place . However, three exogenous factors were not sufficiently anticipated . Firstly, economic growth was occurring quite rapidly in Brazil and especially in the north -east corridor that was not a result of transport investments, but of developments in other infrastructure sectors, especially energy, Secondly, the more efficient flow of traffic which the project intended to achieve depended also on key logistical enhancements in and around the Port of Salvador, such as improved containerization, inter-modal transport, and traffic handling systems . These, in turn, required a well trained pool of skilled labor, which was outside the control of the project . Thirdly, the project design revolved critically around the introduction of performance -based incentives for road maintenance, which required a change in mentality, in both private and public sectors, and a greater speed of contract execution than was evident, especially in the first few years of the project. Planning systems were overstretched in Bahia, and this constraint contributed the shortfall of highway kilometers that were able to be rehabilitated under the project .

4. Achievement of Objectives (Efficacy): The project had two objectives: (i) to increase effective use of the Borrower's state road infrastructure, through rehabilitation and maintenance of key sections of the Borrower's paved road network; and (ii) to increase effective use of the Borrower's road infrastructure through provision of support for institutional improvements in the Borrower's road sector, while fostering greater integration among the Borrower's regions and with the rest of the country .

Objective 1:111::: Increasing effective use of the road infrastructure through rehabilitation and maintenance... ModestModest.... Outputs • 1,196 km of roads were rehabilitated under performance based maintenance contracts, against a target of 2,000 km. About 800 km of roads were not rehabilitated under the project, because they were not identified at appraisal and were not firmed up during implementation. Outcomes  Average land transport freight rates on selected itineraries fell by 10.8 percent over the life of the project versus a target reduction of 6 percent. However, this is a poor indicator as many factors can influence freight rates, such as intra and inter-modal competition (competition in the trucking industry and road freight versus rail freight), political campaigns and public pressure to keep freight rates low, and imperfect administrative/regulatory oversight at the State level .  Average annual daily traffic on the project sections increased by 9.5% per annum over the life of the project. However, not all of this increase can be attributed to the project, as economic growth was accelerating in the country, and especially in Bahia State .  Unit vehicle operating costs (VOCs) are estimated to have decreased by 12.5 percent on the project roads versus a target of 8 percent. This is a leading indicator, even though other factors besides the condition of the road surface can influence the VOCs, such as the condition of the vehicles themselves and newer technologies that were being built into vehicles.

Objective 2:222::: to increase effective use of the Borrower's road infrastructure through provision of support for institutional improvements in the Borrower's road sector... ModestModest.... Outputs  The project financed 41 consultancy contracts, of which 17 were for individual consultants  The Integrated road planning and management system for the state transport Infrastructure Department (DERBA) became operational (partially).  The number of regional offices (residencias) operating information systems on-line was 20, versus a target of 10.  The number of staff weeks of training for the highway agency's staff was 1,527, which represented a large increase over the target of 200 staff weeks.  Ten technical audits per year were completed versus a target of five .  Four feasibility studies of specific road sections were completed as per the target .  The state's multi-modal strategy was defined, including he state airport plan, the inter -municipal passenger transport plan, the transport survey for Salvador metropolitan area, and the state waterways transport infrastructure and services plan.  Four planned outputs were not achieved : (i) the data warehouse system for one of the highway agencies did not become operational; (ii) the plan to optimize logistics impact on the Salvador Metropolitan area was not developed; (iii) the geo-referenced social and economic database did not come into operation; and (iv) the regulatory body's (AGERBA) supervision capacity was not strengthened because planned training was not delivered. Outcomes • Two results were achieved through the institutional strengthening activities : (i) better transport and logistics planning (transport and logistics masterplan, inter -municipal transport master plan, and the Salvador metropolitan area transport surveys, which formed the basis for the State to set out its vision and define priorities for better intra-state connectivity, as well as Bahia’s integration within the country’s corridors and freight flows): and (ii) implementing an effective road program would not have been possible without an effective road agency whose capacity was enhanced by the project . This provided a more solid basis for planning and use of public funding for roads. • However, four planned outputs were not implemented (see above), and this inevitably impacted negatively on the achievement of the institutional strengthening objective .

5. Efficiency: The ICR calculates an ex-post internal economic rate of return (IERR) of 74 percent, well above the already high appraisal estimate of 46 percent. The major causal factor, according to the ICR, was that traffic volumes were much higher than estimated at appraisal. While it is recognized that the improved roads that were rehabilitated under the project did help to accommodate the higher traffic flows that occurred during the project period, these flows were not all on account of the project Rapid economic growth in the Bahia region was a driving factor . Rehabilitated roads did contribute to higher economic growth through more efficient distribution of freight . Annex 3 of the ICR discusses the various factors affecting the economic analysis, such as vehicle modal distribution (car and trucks) and heavier road solutions versus lighter rehabilitation works, The calculations were done for 1,177 km of the road network, or 59 percent of the estimated project network at appraisal .

Administrative and operational inefficiencies (such as inadequate cost estimates, misunderstanding of Bank procedures, and procurement-related delays) played a role in the cost overrun and physical shortfall of the investment component.

Efficiency is rated as Substantial.

aaa.a... If available, enter the Economic Rate of Return (((ERR(ERRERR))))////FinancialFinancial Rate of Return (((FRR (FRRFRR)))) at appraisal and the rerere-re---estimatedestimated value at evaluation :::

Rate Available? Point Value Coverage/Scope*

Appraisal Yes 46% 100%

ICR estimate Yes 74% 59% * Refers to percent of total project cost for which ERR/FRR was calculated.

6. Outcome: Relevance of Objectives was High, while that of Design was Substantial . Efficacy was Modest in view of moderate shortcomings, but Efficiency was Substantial . The Project Outcome is rated Moderately Satisfactory,

aaa.a... Outcome Rating ::: Moderately Satisfactory

7. Rationale for Risk to Development Outcome Rating: In this project, the key factors affecting Risk are technical, financial /economic, and political/governance.

The technical risk is moderate, Although the performance -based contracting approach has been adopted and implemented within the context of the project, nevertheless the concept had not fully taken hold by closure, and the ICR (page 22) acknowledges that some time will pass before it does so . The ICR also notes that all six road maintenance performance-based contracts supported by the project are still ongoing . They include only routine maintenance services, and it is anticipated that they will be completed by August 2014. The State of Bahia will bear the full cost of these contracts.

The financial risk is moderate. In order to receive continued budgetary allocations, an approved State Plan has to be in force. The ICR notes that the flagship study meant to lay out Bahia ’s transport and logistics master plan (PELTBAHIA) has been delayed. Several other studies (State Airports Master Plan, new regulations for waterborne transport, and re-estimation of reductions in freight transport rates, stemming from road rehabilitation ) have helped to provide stability in planning tor the sector . The 2013 state budget includes a particular allocation of about R$ 720 million in the road subsector only. However, there appears to be no soundly based estimate of annual maintenance requirements. Moreover, while the roads budget for 2013 is substantially higher than that for the previous year, the risk of falling back to pre-2010 level of investment remains. Whether, since 2010, the declining trend of transport expenditures within the total State of Bahia expenditures will continue or stabilize is an open question (ICR, page 18).

On political and governance aspects, the risk of policy reversal regarding the priority given to the transport sector among other State programs is considered to be low . In its latest mid-term planning documents (Planejamento Pluri-Anual – PPA 2012-2015), the State committed itself to the development needs of the core infrastructure required to foster development and growth . In its 2013 budget, the State allocated R$720 million to the road subsector.

Regarding institutional support, the State highway agencies have adequate technical managerial capacity, even though there have been some weaknesses as noted .

On the environmental front, the project does not pose any risks, nor on the social front, as land acquisition is not required for road rehabilitation.

aaa.a... Risk to Development Outcome Rating ::: Moderate

8. Assessment of Bank Performance:

a. Quality at entry: The project team drew on extensive Bank involvement in Brazil's transport sector, including policy dialogue and six Bank-financed public transport projects, as well as Bank support for decentralization from the federal transport agency (CBTU) to state and municipal agencies in Sao Paulo, , Belo Horizonte, and Bahia. The Bank played an important role in highlighting the strategic relevance of the project . It displayed a good understanding of the technical, financial, and economic factors . The Project Appraisal Document included an annex that analyzed the fiscal situation of the Bahia State Government, as a basis for determining the extent of counterpart funds required for project implementation .

The ICR states (page 6) that technical preparation was quick (seven months), but this probably led to the use of outdated cost estimates, which were not revised at appraisal and were a major contribution to the significant cost overrun (55 percent higher than the appraisal estimate ). The Bank team over-estimated the capacity of the agencies to execute the project within the time -line required, which also contributed to the significant shortfall in road rehabilitation output. The appraisal underestimated the time needed to overcome the cultural barriers to the adoption of Performance-Based Maintenance, especially when account is taken of the fact that this project pioneered the approach in the North East of Brazil (ICR, page 22).

There was a long delay (13 months) between Board approval and effectiveness . This was on account of unfamiliarity at the State level with Bank procedures (see Section 9 below), a factor that could have been foreseen and addressed, especially since this was a new Borrower .

There were important weaknesses in M&E design (see Section 10a below).

Quality---at -atatat----EntryEntry Rating ::: Moderately Unsatisfactory b. Quality of supervision: The team focused closely on the development impact, It displayed a broad view of the role of the transportation sector in the local economy . The team was pro-active and maintained continuity -- there was only one change in task team leadership over an 8-year period (including preparation). Although a whole year was lost before contracts could be bid out, due to the effectiveness delay, the task team got things on track after that . The QALP-2 assessment of July 2010 judged the project implementation to have been moderately satisfactory . It pointed to lack of readiness of the project on account of insufficient preparation of procurement bidding documents.

Nevertheless, the rate of contract execution was slow . The ICR (page 10) states that six civil works contracts procured through International Competitive Bidding (ICB), and the eight goods contracts (four through ICB, and four through shopping) were satisfactorily carried out. However, there were significant procurement delays, especially in the first two years of project implementation . The main factor was the re-estimation of costs, as it turned out that the appraisal estimates were based on outdated (2004) cost estimates. Also, the detailed engineering designs (completed only in 2008) specified costlier solutions (thicker asphalt overlays) than the ones anticipated in the initial road assessments, carried out at the end of 2004. Apparently, roads deteriorated substantially in the four years 2004-2008.

The task team initiated two Level-2 restructurings during implementation -- in September 2010 to raise the loan financing percentage to solve the counterpart funding constraint, and again in April 2013, which reallocated loan proceeds between categories.

Supervision reports were comprehensive, but did not firm up the cost estimation of the 800 km of road network that had been left unidentified at appraisal . This should have been a priority task of supervision . The Development Objectives rating was stated as 'Satisfactory' in all reports, except for once in 2010, and in the last two reports, in all of which it was rated Moderately Satisfactory . Implementation Progress, after being rated Moderately Unsatisfactory in 2007 and 2008, was rated Moderately Satisfactory from 2008 to 2010, and fully Satisfactory after that through the closure of the project . These ratings do not fully reflect the reality of the delays and the impact of cost overruns on the rate of physical completion .

There were significant shortcomings in M&E implementation, which the supervision reports did not fully address . The monitoring of the project focused on physical implementation and left the traffic surveys and other indicator analyses until the end (see section 10 below).

There were significant shortcomings in M&E implementation (see section 10 below).

There were no reported unresolved issues of safeguards or fiduciary compliance .

Quality of Supervision Rating ::: Moderately Unsatisfactory

Overall Bank Performance Rating ::: Moderately Unsatisfactory

9. Assessment of Borrower Performance:

a. Government Performance: Government ownership of the project was strong . The enabling environment was cohesive, with high priority attached to the freight transport sector, utilizing the road network . However, readiness of the project in the initial two years of implementation presented an issue, and the effectiveness delay contributed to subsequent cost overruns.

The Government waited until loan effectiveness before having the state road agency bid out the engineering designs. This decision proved costly – it adversely impacted the actual commencement of project implementation, as well as contributing to the cost escalation . The Government made this decision, despite the Bank ’s preparation mission having recommended the commencement of the technical work as soon as possible within the preparation phase.

Although the Bank had indicated that Additional Financing (AF) of about US$80 million to cover the cost overruns would be available, the State Government decided not to request it, because it considered that it did not have sufficient borrowing capacity for the AF, given the State's planned expenditures in other sectors in that period (2011-2013).). The global financial crisis of 2008 severely affected the fiscal position of the State of Bahia . Thus, in 2009-2010, when the project financing gap became apparent, the State indicated that it could not afford any additional financing for this project -- either from the Bank or from its own fiscal resources . Also, there was a push from the state towards social expenditures .

However, the AF could have enabled the target of 2,000 km of rehabilitation using performance based contracts to be reached. Although the State is rehabilitating an additional 2,500 km of paved road, using its own funds, this is being done under traditional, input-based contracting. The Government thus fell short of meeting the 2,000 km target for performance-based rehabilitation.

Government Performance Rating Moderately Unsatisfactory

b. Implementing Agency Performance: The outcome of the project was mainly dependent on the implementing agencies, not the Government ‘per se. ’ The Department of Transport Infrastructure (DERBA) and its Project Coordination Unit performed satisfactorily, especially given that this road rehabilitation and maintenance program was the first of its kind in Brazil ’s northeast region. The Project Coordination Unit kept things moving forward, coordinating with the various beneficiaries and providing technical contributions throughout the project phases, by supervising the development of the engineering designs and the execution of the works; and undertaking the monitoring and evaluation of activities . Semi-annual progress reports were submitted to the Bank . However, an initial misunderstanding from DERBA’s procurement office led to the insistence that bidding processes were also subject to the State of Bahia ’s procurement rules, thereby triggering additional administrative burdens and delays .

SUPET (SEINFRA’s transport studies superintendent function ) and SEI (SEPLAN’s economic and social studies superintendent function) carried out the technical assistance component . SUPET delivered more products than initially planned (the Inter-municipal Transport Plan, the Salvador Metropolitan Region Origin and Destination Survey, and the State’s Airports Master Plan).

As the state's transport, energy and communications regulatory agency, AGERBA could have benefited from training and exposure to modern regulatory concepts and techniques . However, AGERBA did not carry out its technical assistance activities. Efforts to organize a Masters in Business Administration training program did not materialize, nor did AGERBA send staff to existing MBA programs in Bahia State, or elsewhere in Brazil . The reasons for this were: (i) weak institutional capacity and numerous changes in the leadership of the agency; and (ii) the impossibility of concluding a contract between DERBA and the Rio Grande do Sul University, which had been identified to deliver the training to AGERBA. Thus, after about 2 years of unsuccessful contract negotiations, the Bank deleted this activity and reallocated the funds to other activities within the project .

Implementing Agency Performance Rating ::: Moderately Satisfactory

Overall Borrower Performance Rating ::: Moderately Satisfactory

10. M&E Design, Implementation, & Utilization:

a. M&E Design: The Monitoring and Evaluation system incorporated outcome indicators, However, the framework provided no means of attributing changes in key indicators (traffic growth, freight rates and unit vehicle operating costs ) to the project. Baseline data were sketchy (Annex 3 of the PAD, Result framework and Monitoring) and relied on federal and state-wide transport sector information. This annex did not include intermediate project outcome targets for Years 1-5, which meant that the Bank and implementing agencies teams did not have intermediate benchmarks to guide them.

The annual and semi-annual monitoring activities focused on physical progress of the project and not on implementing the extensive surveys that were intrinsic to the specification of the results framework ’s indicators (traffic flows, vehicle operating costs, etc .). Although the full benefits of the project and its resultant traffic flows are only fully apparent at completion, building the monitoring system along the way to capture this impact would have been helpful . Some of the institution building activities were insufficiently monitored (e.g. the Plan to optimize transport logistics in the Salvador Metropolitan Region was not formulated ).

b. M&E Implementation: The baseline data were augmented in the first few years of the project and are shown in the Data Sheet of the ICR . This generated summary information for supervision reports, but did not permit the monitoring and evaluation of the three outcome indicators, which were, in fact, not monitored during most of the implementation period . The calculation of land transport freight rates on ten selected roads of the project emerged from a study that was carried out only in 2013 -- during the last nine months of the project . Average reductions in freight transport rates were calculated and were extrapolated backwards to provide an annual freight charges reduction rate for the 2008-2013 period. Similarly, the average annual daily traffic increase rate was calculated at the end of the project and was also extrapolated backwards to 2008-2013 to provide an annual rate of increase . The same was done for vehicle operating costs. The monitoring and evaluation system, therefore, did not guide project implementation from year to year, but rather generated only an ex post view . DERBA monitored from year to year only the physical outputs of civil works construction.

c. M&E Utilization: M&E data generated during project implementation were adequate for monitoring what the project financed, and they have proven useful in helping design a follow -on project that, if approved, would have the necessary baseline information. The Implementation Status Results Reports made effective use of the data, as did the Mid -Term Review. The cost-benefit study carried out later on used site -specific survey results and traffic studies to supplement the aggregate data.

M&E Quality Rating ::: Modest

11. Other Issues

a. Safeguards: The project was classified as Category "B" for purposes of Environmental Assessment . Four Safeguards policies were provisionally triggered: OP/BP 4.01 (Environmental Assessment), OP 4.11 (Cultural Property), OP/BP 4.12 (Involuntary Resettlement), and OP/BP 4.10 (Indigenous Peoples).

Accordingly, the Road Agency prepared the following documents, which were found to be acceptable by the Bank :

(i) the Indigenous People Plan (ii) the Resettlement Framework; (iii) Guidelines for the Program's Environmental Evaluation .

Cultural property turned out to not be present in project -supported sections of road rehabilitation .

The ICR states that there were no involuntary resettlements, nor was there any impact on indigenous people's territories, as roads were rehabilitated on existing rights of way . Since this was a highway rehabilitation and maintenance project within the State's existing domains, the project did not involve any land acquisition .

According to the ICR, no critical safeguards issue emerged during implementation . The Bank's environmental/social specialists carried out supervision missions annually . DERBA’s environment unit visited the various road sections four times a year, including after the rehabilitation works were completed . The only environmental issue that arose related to clean-up of work sites, which according to the ICR had been addressed at all road sections by closure .

b. Fiduciary Compliance: Financial Management

Financial management was carried out in accordance with the arrangements agreed upon in the Loan Agreement and the Operational Manual. The project had an acceptable financial system in place at the outset of project implementation, relying on the Road Agency's systems . Specific financial management missions were carried out once or twice a year during implementation; no critical issues were reported .

Supervision reports, including the last one of September 24 2013, rated financial management Satisfactory, except during January-May 2010, when it was Moderately Satisfactory, due to counterpart financing issues . As noted, this issue was addressed, in mid-2010, by increasing the Bank’s share of financing for civil works and goods under the project.

The last report on Financial Management Performance of September 24, 2013 stated that the FM rating was Satisfactory.

Annual Audited Financial Statements were reviewed by the Bahia State Court of Account, which were then submitted to the Bank. Bank reviews produced no issues regarding fiduciary compliance . Auditors' opinions were unqualified.

Procurement

The Project financed six works contracts (ICBs), eight goods contracts (including four ICBs) and 41 consultancy contracts, out of which 17 were for individual consultants. Consulting services were contracted through Quality and Cost Based Selection. All procurement processes were subject to prior review, in order to maintain a tight control over selection (that it be sound and transparent).

However, delays in procurement became a major factor in the cost overrun and physical shortfall . The average procurement process duration for Quality and Cost Based Selections was 2 years and 4 months (between preparation of the requests for proposals and contract signing ). Reasons include: (i) an initial misunderstanding from DERBA’s procurement office, which led to the insistence that bidding processes were also subject to the State of Bahia’s procurement rules, thereby triggering additional administrative burden and delays; (ii) delays between the Project Coordination Unit and other State entities beneficiaries; (iii) overall poor initial quality of procurement documents, requiring back and forth between the Project Coordination Unit and the Bank; and (iv) difficulty to come up with short-lists of two consultancies maximum per country . There were no reported cases of misprocurement .

Disbursements

There were no issues, Disbursements followed eligible expenditure categories .

c. Unintended Impacts (positive or negative): .

d. Other: None

121212.12... RatingsRatings:::: ICRICRICR IEG Review Reason for Disagreement ///Comments /Comments Outcome::: Moderately Moderately Satisfactory Satisfactory Risk to Development Moderate Moderate Outcome:::

Bank Performance ::: Satisfactory Moderately There were significant shortcomings in Unsatisfactory both Quality at Entry and Quality of Supervision, including severely inaccurate cost estimates, the non-identification of 800 km of the road network, and the lack of priority given to resolving this issue during implementation. Design and implementation of the M&E system had important shortcomings.

Borrower Performance ::: Satisfactory Moderately There were significant shortcomings in Satisfactory Government performance, including the decisions to wait until effectiveness before having the State Road Agency bid out the engineering designs. Implementing Agency performance was moderately satisfactory.

Quality of ICR ::: Unsatisfactory

NOTESNOTES: - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate.

13. Lessons: Of the following lessons, the first two are taken from the ICR, and the others are drawn by IEG :  A performance-based approach to highway improvement projects takes time and requires cultural change .  Combined lending instruments and interventions across the Bank Group help to leverage sector results . The project included institutional strengthening activities, which proved conducive to fostering the sector dialogue .  Project development objectives and outcome indicators need to be closely aligned, tracing a strong causal link. This proved to be difficult in this project . Weak attribution resulted from exogenous factors driving the outcome indicators. It is important to design a results framework that links inputs -outputs-outcomes clearly, and through which outcomes can be attributed to the project . This should not be restricted to annual monitoring of only the physical progress, but needs to encompass the specified results /outcome indicators.  The full and effective financing of road rehabilitation programs needs a clear time horizon, a realistic appraisal of technical capacity, and updated estimates of the cost of civil works . The financing gap that emerged in this project, and the shortfall in physical output, reflected the absence of these elements .

14. Assessment Recommended? Yes No

Why? To verify the ratings and document lessons learned .

15. Comments on Quality of ICR:

 Quality of Evidence: the ICR presents a body of evidence that allows one to understand the history of the project and the problems it encountered. However, some of the information on key actions taken by the Bank and the State Government, including its implementing agencies, is too brief .  The quality of analysis has several major weaknesses : it lacks a critical view of the physical shortfall in output and the inability of the Bank and the Government to estimate a realistic program and time period in which to execute 2000 km of road rehabilitation (this was a design issue). The report summarizes the factors that caused the cost overrun, but over-emphasizes the effect of the exchange rate changes . The ICR could have explained more clearly that at appraisal only the first 1171 km were identified and therefore there were no cost estimates on the remaining 829 km of road network. The latter were therefore an approximation used in determining the original loan amount.  The weaknesses in the M&E system (design, implementation, and utilization) are not sufficiently analyzed.  The report is excessively narrative driven, tracing a story -line of implementation that does not come together in key parts. For example, the map shows key sections that appear to be unfinished . The project team later clarified that the segments of roads that were rehabilitated were in fact connected by an east -west national highway in good condition.  Scant information is given in the ICR on what transpired between the February 2011 supervision report and the closing date of the loan some 30 months later. aaa.a...QualityQuality of ICR Rating ::: Unsatisfactory