Document of THE WORLD BANK

FOR OFFICIAL USE ONLY

Report No. 22292 Public Disclosure Authorized

PERFORMANCE AUDIT REPORT

ARMENIA Public Disclosure Authorized

INSTITUTION BUILDING LOAN (Loan 3585-AM)

June 5, 2001 Public Disclosure Authorized

OperationsEvaluation Department Public Disclosure Authorized

This document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not otherwise be disclosed without World Bank authorization. Currency Equivalents

Currency Unit: Armenian Dram

Average Exchange Rate (Armenian Dram to US$)

1993 9.11 1997 490.85 1994 288.65 1998 504.92 1995 405.91 1999 535.06 1996 414.04 2000 539.53

Abbreviations and Acronyms

ASYCUDA UNDP Customs Software ECA Europe and Central Asia Regional Office ES Employment Service EU European Union GOA Government of IAS International Accounting Standards IBL Institution Building Loan IBRD International Bank for Reconstruction and Development ICR Implementation Completion Report IMF International Monetary Fund OED Operations Evaluation Department PAR Performance Audit Report SAC Structural Adjustment Credit SATAC Structural Adjustment Technical Assistance Credit TA Technical Assistance UNCTAD United Nations Conference on Trade and Development USAID United States Agency for International Development

Fiscal Year

January 1 - December 31

Director-General, Operations Evaluation : Mr. Robert Picciotto Director, Operations Evaluation Department Mr. Gregory K. Ingram Manager, Country Evaluation & Regional Relations : Mr. Ruben Lamdany Task Manager Mr. Elliott Hurwitz Peer Reviewer Mr. Jorge Garcia-Garcia FOR OFFICIAL USE ONLY The World Bank Washington, D.C. 20433 U.S.A.

Office of the Director-General June 5 2001 Operations Evaluation '

MEMORANDUM TO THE EXECUTIVE DIRECTORS AND THE PRESIDENT

SUBJECT: Performance Audit Report on Armenia-Institution Building Loan (IBL; Loan 3585-AM, US$10.74 million)

Attached is the Performance Audit Report (PAR) for the above loan to Armenia, made from 1993 to 1997.

The country's first loan was initiated in adverse circumstances-due to the economic transition and the war with Azerbaijan-and was the only one on 1BRD terms (all subsequent lending has been on IDA terms). The country had embarked on an extremely ambitious reform program, and requested the Bank's assistance.

The project objectives were to provide technical assistance to the government to strengthen the institutional basis for economic reform in four areas: (a) economic management; (b) resource mobilization; (c) enterprise reform (which consisted mainly of privatization support); and (d) financial sector reform. The project was well targeted, focusing on development of government capacity in critical areas including macroeconomic policy; debt management; tax and customs administration; and privatization.

The objectives were for the most part achieved because the government was substantially committed to reform and made considerable progress in a number of key areas-although the pace of advancement was variable. The government developed its own institutional capacity for independent macroeconomic policy formulation, and competency was extended to a much wider group of officials. Macroeconomic performance steadily improved during the TBL, and it seems plausible that the advice and training provided were instrumental in the improved performance. Tax and customs administration were modernized, and revenue increased substantially, although the IBL was probably not responsible for the short-term surge in revenue. After a slow start, voucher privatization progressed rapidly, with the exception that no progress was made (during the term of the JBL) on privatization of large enterprises. The IBL provided a useful foundation for later reforms, which accelerated after the May 1994 ceasefire.

The PAR rates the outcome of the project as moderately satisfactory, sustainability as likely, and institutional development as substantial. These ratings are higher for institutional development than the OED Evaluation Summary, but lower than that of the ICR, which rated outcome as satisfactory. The PAR finds that while the EBL contributed to the speed and quality of the country's reform progress, as a project that was a complement to the policy dialogue, it did not adequately address several areas which later emerged as key constraints to the country's development.

Three main lessons emerge from the project. First, given the high level of borrower commitment, this TA loan was an effective way for the Bank to engage and stimulate reform. Second, in a highly uncertain situation-the Bank in essence knew little about Armenia when the IBL was designed-flexibility is essential for a TA project. And third, an inexperienced borrower requires additional supervision. This was exemplified by the poor consultant performance and the instance in which government employees apparently worked under an IBL subcontract.

Robert Picciotto by Gregory K. Ingram This document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not otherwise be disclosed without World Bank authorization.

Contents

Preface...... Ratings and Responsibilities ...... inm....

1. Background ...... 1

2. Objectives and Design of the Loan ...... 2

Background and Description of Objectives ...... 2 Relevance of Objectives ...... 4 Assessment of Instrument ...... 55......

3. Implementation Experience ...... 5

Economic Management ...... 66...... Resource Mobilization...... 7 Enterprise Reform ...... 7 Financial Sector Reform ...... 7

4. Outcome ...... 8

5. Institutional Development Impact...... 9

6. Sustainability...... 10

7. Bank Performance ...... 10

8. Borrower Performance...... 11

9. Lessons Learned...... 1...... I

Annexes

A. Individuals Interviewed ...... 13 B. Basic Data Sheet ...... 15

The report was prepared by Elliott Hurwitz (Consultant). Mr. Jorge Garcia-Garcia was the peer reviewer. Norma Namisato provided administrative support.

Preface

This is a Performance Audit Report (PAR) on the Institution Building Loan (IBL), in the amount of US$12.0 million. Cofinanciers included USAID (US$3.4 million), and the EU (US$1.1 million), with total planned financing of US$16.5 million. Of the IBRD contribution of US$12 million, US$10.74 was disbursed and US$1.26 million was cancelled. Actual project cost was US$15.3 million. The loan was approved on March 30, 1993, effective on June 15, 1993, and closed on November 30, 1997, one year later than the original closing date.

The PAR is based on the Implementation Completion Report (ICR), the President's Report, the legal documents, project files, related economic and sector work, and discussions with Bank and Fund staff, and government documents.

An OED mission visited Armenia in March 2001, to discuss the project and several other projects with present and former officials, other donors, and World Bank Field Staff. Their cooperation and assistance in preparing this report is gratefully acknowledged.

The draft PAR was sent to the Borrower and the cofinancing agencies for comments but none were received.

111

Ratings and Responsibilities

Ratings ICR OED ES OED PAR Outcome Satisfactory Marginally Moderately Satisfactory Satisfactory Sustainability Likely Likely Likely Institutional Development Partial Modest Substantial Bank Performance Satisfactory Satisfactory Satisfactory Borrower Performance Satisfactory Satisfactory Satisfactory

Responsibilities

Staff Appraisal Completion

Task Manager Chris Hall/Onno Ruhl Onno Ruhl/Cyril Muller Division Chief Wafik Grais Wafik Grais Country Director Basil Kavalsky Judy O'Connor

1

1 Background

1.1 Extremely adverse circumstances. It is difficult to overstate the calamity that faced Armenia in 1991-93. The country, land-locked and with few natural resources, was beginning the transition from a centrally-planned to a market economy, with very limited institutional capacity. As a result of the conflict raging with Azerbaijan over Nagorno Karabakh (NK), an ethnic Armenian enclave inside Azerbaijan, refugees from Azerbaijan were streaming into Armenia. At the same time, Azerbaijan imposed a trade embargo, shutting off most supplies of energy and other critical goods (the main natural gas pipelines which supplied Armenia through Azerbaijan were closed). The border with Turkey, which was sympathetic to Azerbaijan, was closed, and the main routes into the country to the north through Georgia were subject to frequent interdiction by bandits and separatists fighting the Georgian Government. Finally, the earthquake of December, 1988 in which more than 30,000 people were killed, had destroyed a major portion of the country's productive capacity and left over 300,000 people (ten percent of the country's population) housed in primitive, unheated sheds in the mountainous north. As a precaution, the country's nuclear reactor was shut down, reducing electricity supplies by one third. While the had marshaled its resources to help Armenia repair earthquake damage, in the midst of this effort the Union was dissolved and reconstruction remained incomplete through the 1990s.

1.2 When Armenia was part of the Soviet Union, its economy was heavily interdependent with those of other Republics. The country exported sophisticated electronics and software, light industrial products, chemicals, electric equipment, and metal-working machinery. Imports consisted of energy, food, agricultural and industrial chemicals, and other intermediate goods.

1.3 Following independence in 1991, Armenia suffered a steep fall in income, with real GDP estimated to have declined by 11.7 percent in 1991 and 41.8 percent in 1992, and at its nadir GDP had fallen an aggregate of around 63 percent (since 1990).1 The rapid decline was caused by the physical and geopolitical factors noted above, as well as severe shortages of foreign exchange, an enormous increase in energy prices, and a sharp reduction in credit. The parallel fiscal shortfall was troubling, as it meant that the government had inadequate funds to deal with the country's massive needs. Table 1 below shows key macroeconomic data during the period of the IBL.

1.4 Financial disruption added to the country's woes. During 1993, the old ruble area was disintegrating without any clear prospect for Armenia to join a new ruble area or receive adequate external financing. In consequence, the government loosened financial policies through massive net lending to enterprises. The budget deficit rose to 52 percent

' Armenia's income decline of 63 percent was greater than that of any Former Soviet Union country except Georgia. The average decline for the Commonwealth Independent States (CIS, excluding the Baltics) was 51 percent. "Transition After a Decade," ECA, 2000, p. 15 (November 2000 draft). 2

Table 1. Armenia: Selected Macroeconomic Indicators 1991 1992 1993 1994 1995 1996 1997 1998 Annual GDP growth -11.7 -41.8 -8.8 5.4 6.9 5.8 3.1 7.2 GDP level (1990-=100) 88.3 51.4 46.9 49.4 52.8 55.9 57.6 61.7 Annual inflation, CP1 174.1 728.7 1823 4962 175. 18.8 13.8 8.6 Budget deficit, as%ofGDP -1.8% -7.70/ -51.90% - 16.5 0/ 9.0% -8.6% -5.7% 4.3% Tax revenues, as%GDP 4.3% 3.8% 15.00/ 13.1 12.7% 12.9% 16.3% 17.0% Source: Country Operations.

of GDP in 1993, financed by the Central Bank. In November 1993, a national currency, the dram, was introduced in the context of severe financial imbalance and negligible foreign currency reserves. In the last two months of 1993, consumer prices rose by 900 percent, compared to a monthly average of 26 percent in the first 10 months of the year.

1.5 Ambitious reform program. In this difficult economic and social context, the Armenian authorities moved forward with an ambitious reform program. The government began the privatization of agricultural land in 1991, leading to an expansion of agricultural production of 15 percent that year (Armenia was the first Former Soviet Union republic to privatize land). Privatization of small businesses began under a Presidential decree of June 1991, and many small enterprises were sold. Price decontrol began in April 1991, and by June 1992 nearly all prices were decontrolled except bread, transportation, and utilities. Coverage of the economy by the state order system was reduced in 1992 to 50 percent of its level in 1991. And initial steps were taken to privatize the distribution system.

1.6 Armenia also made a good start on developing the basic legal framework necessary for a market economy. The Property Law enacted in October 1991, provided a legal basis for companies and a basic commercial code. All legal restrictions on private entry and competition in wholesale trade were removed. And the Enterprise Privatization Law (July 1992) provided that a privatization program be developed every year to sell remaining state-owned firms, including large firms. It should be noted that reform progress slowed somewhat after mid-1992 as the war intensified, and that this slower pace lasted until May 1994, when the cease fire agreement was reached, and reform progress once again accelerated.

2 Objectives and Design of the Loan

Background and Description of Objectives

2.1 The Institution Building Loan (IBL), effective in June 1993, was the Bank's first loan to Armenia. 2 The Bank had originally intended to proceed with a Rehabilitation Loan, which would have supported policy changes and provided financing for critical

2 It was also Armenia's only IBRD loan; the IBL went to the Board in March 1993, and Armenia qualified for IDA terms later in the year. All subsequent lending has been on IDA terms. 3 imports. However, in late 1992, the IMF determined that the macroeconomic situation was too unstable, and that until there was substantial progress toward resolution of the conflict with Azerbaijan it would be unlikely to conclude a Stand-By arrangement. During this period, successive Bank missions determined that with the government's strong desire to reform and progress to date, institution building support and technical assistance could substantially expedite reform. A Country Economic Memorandum (CEM) was also prepared (March 1993) to assist the government in its reform efforts, and the CEM findings served as a foundation for the IBL. The government also needed assistance in managing the increasing flow of aid. Consequently, to provide tangible support to the reform process and in light of the country's extremely difficult situation, the Bank proceeded with the IBL.3

2.2 The IBL was initiated as a US$17.2 million project, with IBRD financing US$12.0 million, USAID US$3.7 million, the EU US$1 million, and the (GOA) US$0.5 million. USAID provided macroeconomic advice and assistance with land registration, while the EU assisted privatization. The original closing date of November 30, 1996, was later extended to November 30, 1997.

2.3 The project objectives were to assist the Armenian government to strengthen the institutional basis for economic reform through institution building programs in four areas: (a) economic management; (b) resource mobilization; (c) enterprise reform; and (d) financial sector reform. Components of the Bank project included:

(1) Economic management (subtotal: US$5.38 million) Macroeconomicpolicy formulation and aid coordination(US$1.24 million) * Strengthen MOE analytic capacity . Finance aid coordination function . Establish and develop debt reporting function in MOF Legal and regulatory reform (US$0.93 million) * Design and establish legal and institutional framework for market economy through provision of legal advisory services; draft legislation and regulations; legal training; equipment Social safety net/employment issues (US$3.21 million) * Develop social assistance strategy . Restructure and automate Employment Service (ES)

(2) Resource mobilization (subtotal: US$2.07 million) Tax administration(US$0.47 million) * Advisory services, training, and computerization support to bolster capacity of Tax Inspectorate to design and implement new tax program Customs administration(US$1.60 million) * Modernize customs laws, regulations * Restructure Customs operations, provide training in key functions (classification, valuation)

The only other lending made prior to the ceasefire with Azerbaijan (May 1994) was the US$28 million Earthquake Reconstruction Credit, approved in January 1994. 4

* Computerize clearance procedures, accounts, and statistics

(3) Enterprise reform (subtotal: US$5.21 million) Supportfor privatization (US$4.88 million) . Policy assistance for privatization program . Financing for Auction center * Printing of vouchers * Public relations for privatization program " Investment banking services for sale of "strategic enterprises" Supportfor public sector enterprise reform (US$0.33 million) . Prepare enterprises for privatization . Provide restructuring advice for 10 large loss-making enterprises

(4) Financial sector reform (subtotal: US$2.58 million) * Advice, training, and equipment to support: > Stronger bank supervision > Introduction of IAS > Development of commercial banking skills > Establishment of share registry

Relevance of Objectives

2.4 The objectiver were of substantial relevance. Given the country's dire situation as described earlier, the IBL approach of providing technical assistance in specific areas of great need was sound. Clearly, assistance with macroeconomic policy formulation was critical to helping the country stabilize a chaotic environment. Legal and regulatory advice and drafting was important to the government's developing program of new laws and regulations suiLable for a market economy. And facilitating resource mobilization- strengthening tax and customs administration-was critical in light of the government's fiscal straits and its ability to provide basic services and cope with the crisis.

2.5 Enterprise reform and privatization consisted of helping the government develop and implement its privatization policy, prepare enterprises for sale, and then implement the sale (both for mass privatization as well as case-by-case). The Bank and the government agreed that rapid divestiture of state-owned enterprises (SOEs) was the best approach, and so this component was sensible. And diagnostic studies of the large, loss- making enterprises that would need to be sold later was sound.4

2.6 In the banking sector, more than 60 new banks, most of them small and under- capitalized, were established in 1992 and 1993. The country's regulatory framework and bank supervision capabilities were clearly inadequate to deal with this new situation, and so this component clearly made sense.

4 While the approach toward privatization was later considered to have significant flaws, at the time it was in accordance with the views of the Bank's major shareholders and was considered "best practice." See discussion below. 5

2.7 However, the plan to allocate considerable resources to computerize the ES seemed dubious. Initially, it was planned to automate the service without implementing fundamental structural reforms-a sequence which did not seem sensible. Further, strengthening the ES only made sense if it was accepted that there would be sufficient jobs available into which to place the laid-off workers. Given the uncertainties of Armenia's situation in 1993, this assumption was questionable.

2.8 On balance the objectives of the IBL were substantially relevant to the development needs of Armenia at project inception.

Assessment of Instrument

2.9 The IBL was an appropriateinstrument. Given the Bank's constraints on lending to Armenia prior to the May 1994 cease-fire, the IBL was an appropriate instrument. It was the Bank's intention to support capacity development, and the rBL afforded sufficient flexibility to achieve this in light of the many uncertainties. Taking into account the magnitude of country's need for strengthened capacity, the size of the loan might seem inadequate. However, it is not clear that the country could have absorbed assistance more quickly. Initial disbursements lagged, in part because of the unfamiliarity of the Armenian authorities with Bank procedures and poor communication between the PIU and several Ministries. And, as the project progressed, grant assistance from other donors became increasingly abundant.5 The country's ability to absorb TA was probably not greater than that which the IBL, in addition to extant grant assistance, could provide.

3 Implementation Experience

3.1 The IBL became effective in June 1993. However, disbursements lagged, reaching only about 32 percent of forecast values after two years. By two years after inception, expenditure rates rose rapidly, but the initial delays required a one-year extension in the planned 3V2-year project. In retrospect, the project should probably have been designed for a longer duration.

3.2 Initial delays were partly due to the effects of the conflict and the blockade, the implications of which were underestimated by the Bank, as well as poor communication between the PIU and cognizant ministries. Until the ceasefire in May 1994, the government was somewhat distracted by the intensification of the war. Also, there was a lack of familiarity within the Ministries with Bank procedures. And the IBL's planned disbursement rates may have also been too optimistic.

3.3 Additional disbursement delays were also caused by slow progress on the employment services component. As noted earlier, a decision was made not to computerize the ES before it had been restructured. GOA delay in implementing a

5 In fact, US$1.26 million of project funds was cancelled. 6 meaningful restructuring caused further delay. Finally, collaboration with UNCTAD on purchase of software to strengthen customs administration progressed slowly.

3.4 By project conclusion, 90 percent of 1BRD funds had been disbursed, and US$1.26 million was cancelled. The main reason for the cancellation was the abundance of grant funding for TA that had become available, both from bilateral and multilateral donors.

Economic Management

3.5 Macroeconomic policy formulation and aid coordination Aside from the intellectual capital of a small group of policymakers, Armenia was starting from a low level in the area of economic management capacity. Macroeconomic policy advice and mid-level training were provided to the Ministries of Economy and Finance, and the Central Bank of Armenia. The IBL also provided key economic ministries with computers and other office automation hardware. USAID provided a full-time Resident Advisor in the Ministry of Economy. Technical advice and software was provided to establish and then gradually build the debt-reporting and aid coordination functions in the Ministry of Economy.

3.6 Key economic agencies were provided with office automation equipment: Ministries of Finance, Economy, Central Bank, Statistics Department, and National Assembly. A Foreign Aid Coordination Center was established within the Ministry of Economy, which recorded and monitored debt, and assistance was provided to train center staff.

3.7 Legal and regulatoryreform. In drafting economic legislation, one key contractor (a law firm) did not perform satisfactorily. The government felt that the draft laws and regulations produced, while suitable for a market economy, were inadequate for the Armenian context. In addition, the contractor had a weak local presence and poor coordination with the government. While all procurement in this instance was according to Bank guidelines, the government terminated the contract and reallocated the savings to computerization of several Ministry of Justice components. Another consultant produced satisfactory work in the area of investment fund legislation. So while some progress took place in this area, it was less than had been anticipated.

3.8 Social safety net/employment issues. Because of the lack of employment opportunities, it was decided not to proceed with the effort to strengthen the ES until the government had restructured it. In the event, the Ministry of Labor made relatively minor changes to the system, and just US$950,000 was used to computerize employment services functions. The remaining funds (US$2.26 million) originally intended for this application were reallocated to the completion of the tax inspectorate computer network, to development of the standardization institute, and to support for legal reform. 7

Resource Mobilization

3.9 The Tax Administration component, performed in conjunction with the IMF, made good progress: the entire State Tax Inspectorate was computerized, and an IMF resident advisor provided policy and implementation guidance (starting mid-1995). Upgrade of the Customs Administration also progressed satisfactorily after a slow start. The ASYCUDA software system, purchased from UNCTAD for US$1.6 million, was installed and improved this function considerably. By project close, the program was expanded to cover all regions of the country.

Enterprise Reform

3.10 The IBL supported the establishment of a Privatization Commission and Board, which provided policy advice and implementation assistance to the privatization program. It also financed establishment of the Yerevan Auction center, and paid for staff and training. The loan supported substantial public information efforts by the government for the privatization program, including training of local consultants.

3.11 Privatization certificates for the program years 1995-97 were printed and provided to the program. The cost of the printing was only US$144,000-compared to US$1,000,000 allocated for that task-and the remaining funds were reallocated to support other privatization tasks. Local consultants provided assistance to firms that were going through the privatization process. In addition, diagnostic studies were prepared for 10 large enterprises that were preparing to be privatized under the "strategic investor" program.6 The loan also supported financial advisory services utilized in the course of the privatization of these large firms.

Financial Sector Reform

3.12 Most of the training and technical assistance envisioned in this area did not take place. While a central registrar for securities was established, the other goals were not achieved under the IBL. Early in the project, the Bank had prepared an agenda for financial sector reform, and discussed it with the government. However, the anticipated USAID support for this component did not materialize, and so the activities were not undertaken. They were addressed, however, under later projects, and became part of the country's reform agenda.

3.13 Violation ofprocurement rules. According to the ICR, during the IBL an incident occurred in which a subcontractor retained local staff who were state employees. This is discussed further below under Borrower Performance.

6 As noted below, privatization of the strategic enterprises progressed slowly, and none of the ten were privatized during the term of the IBL. 8

4 Outcome

4.1 Overall, the IBL is rated as moderately satisfactory. The IBL was timely and well-targeted; it provided beneficial technical assistance in important areas during a key period of Armenia's reform efforts. It contributed to the speed and quality of the country's reform progress. However, it had several shortcomings, including the procurement irregularities and poor consultant performance noted, which reduced the project's benefit. More significantly, as the Bank's first technical assistance loan-and as a complement to the policy dialogue-it did not adequately address several areas which later emerged as key constraints to the country's development (discussed below in para. 4.9).

4.2 Economic management. The government developed its own institutional capacity for independent macroeconomic analysis and policy formulation, including strengthening of the statistical database. The government's economic management capabilities grew, and knowledge spread to a much wider group of officials. And, as shown in Table 1, macroeconomic performance steadily improved during the IBL. It seems likely that the advice and training provided under the IBL were instrumental in the improved performance.

4.3 In the area of legal and regulatory reform, the government made good progress in developing new laws and regulations, but the contribution of the IBL was less than had been expected. This was caused by the poor performance of a consultant (law firm), which had been contracted to provide legal advice, training, and drafting of laws and regulations suitable to a market economy. As described earlier, the government terminated the contract.

4.4 Resource mobilization. The performance of tax and customs administration improved significantly. As shown in Table 2, below, revenue as a percent of GDP rose dramatically in 1993-although it seems unlikely that this was a result of the IBL-and remained at a relatively high level in subsequent years.

Table 2. Tax Revenue as a Percent of GDP 1991 1992 1993 1994 1995 1996 1997 Tax revenues, as % GDP 4.3 3.8 15.0 13.1 12.7 12.9 16.3 Source: Armenia Country team.

4.5 Enterprisereform and privatization. In the privatization area, the IBL helped to establish a firm foundation for progress in the mass privatization and strategic privatization efforts. It was only near the end of the IBL, however, stimulated by activities under other operations, that large numbers of firms began to be privatized. And in the case of the large ("strategic") enterprises; progress was extremely slow and no privatization progress was made during the IBL although diagnostic studies were completed that were used later. 9

4.6 The approach used toward mass privatization in Armenia-voucher privatization-was mainly designed by the government, with modest input from the Bank. At the time of the IBL, this method was considered to have the benefit that it would rapidly transform the ownership of a very large number of SOEs. It was believed that this would create a critical mass of private enterprise upon which the foundation of a market economy could be built. This was the preferred approach of the Armenian Government, as well as the Bank and its major shareholders. Later, this approach was criticized on the grounds that it did not facilitate enterprise restructuring, and that it permitted workers and managers-who often had weak incentives for increased efficiency-to become dominant shareholders.

4.7 In retrospect, the prospective benefits of privatization were exaggerated. It was expected to lead to significant gains in the short- to medium-term, but institutional weakness, among other factors, severely limited the returns. And widespread appropriation of assets by managers and other well-placed individuals reduced public confidence that privatization was being undertaken in a manner that would yield substantial and widespread economic benefits.

4.8 Financialsector reform. While a share registry was established, the objectives in this area were not achieved. As noted earlier, the expected participation of USAID was not forthcoming. The country team reprogrammed the funds to other areas. Significant- and largely successful-reforms of the financial sector were carried out under subsequent projects.

4.9 Key areas neglected. The IBL was undertaken rapidly, under very difficult circumstances, and with relatively little in-depth knowledge of the country. However, during implementation it should have been possible to redirect support to areas judged as critical to development. In retrospect, the IBL did not adequately address several areas that comprised significant constraints on future Armenian development: government capacity (including effective regulation) to support a market economy; effective procedures for bankruptcy and liquidation, as well as conditions permitting the entry of new firms; and the environment for private business.

5 Institutional Development Impact

5.1 The IBL was principally an institutional development project, and in a number of key areas it made significant and timely contributions to the country's ability to make use of its resources. These were especially noteworthy as they built upon an initial condition of very low capacity.

5.2 As a result, Institutional Impact Development is rated as substantial. The shortcomings noted earlier, including its failure to address several areas that emerged as

7 See OED, "IDA Review of Private Sector Development," March 2001; "Transition After a Decade," ECA, November 2000; and "Between State and Market: Mass Privatization in Transition Economies," World Bank, September 1997. 10 serious constraints to development, lead to an Institutional Development Impact rating of moderate.

6 Sustainability

6.1 Sustainability is likely. The IBL, the Bank's first loan to Armenia, was followed by the other technical assistance and adjustment operations noted earlier. The IBL laid a foundation and built capacity that was utilized and strengthened by the follow-on operations and which benefited the country. Following the close of the IBL, Armenia's macroeconomic situation continued to improve. The government continued to implement the privatization plans initiated under the IBL, however not always with complete success. As of early 2001, most enterprises have been privatized, however, at least 75 large enterprises remain state-owned, and some nominally-privatized companies remain as much as 33 percent state-owned. Government revenues have stabilized, and the financial sector reforms planned-but not realized-under the IBL have been achieved under other operations.

7 Bank Performance

7.1 Bank performance was satisfactory. The Bank implemented the IBL under difficult conditions, with considerable urgency, and initially little knowledge of the country. Under the circumstances, the design and relevance of the project were quite satisfactory.

7.2 Increase in grant TA. The Bank underestimated how much grant TA would be available to Armenia as donors geared up-a major reason behind the IBL's lack of complete disbursal. However, Armenia had a greater degree of control over the TA from the IBL than it did over grant TA, and a number of policy officials noted that they preferred to borrow for TA on IBRD terms rather than utilize most grant facilities that provided less government control and less utility.

7.3 Supervision was satisfactory. The vast majority of tasks undertaken during the 42-year project were performed adequately. Throughout the period of the project, staff maintained an effective dialogue with the government on furthering the reform agenda. The decision to drastically downsize the Employment Services component was sound. In the case of poor performance by the consultant (law firm), all Bank procurement and contract award requirements were met, and the government-with the Bank's concurrence-took appropriate action in terminating the contract. In the instance described in the ICR in which government employees were hired by a subcontractor, the contract approved by the Bank did not contain the names of the government employees.

7.4 The IBL required intensive supervision, mainly because Armenia was an inexperienced borrower, and because the project consisted of a broad range of relatively small pieces. While records show only approximately 2 supervision missions per year, 11 and 34.5 staff-weeks over the 4V2-year project, considerable additional supervision was provided in conjunction with other projects, as well as with non-project activities.

8 Borrower Performance

8.1 Borrower performance was satisfactory. During most of the Vroject period, the government ownership was high, with many positive outcomes. Macroeconomic performance improved dramatically, and modernization of tax and customs administration was effectively carried out. Privatization of small and medium enterprises was successful, with the caveat stated earlier relating to the conceptual framework of the program. Privatization of large enterprises was less successful.

8.2 Concurrent with the project, the Bank offered advice in several key areas of reform. However, the Borrower was unwilling to pursue these recommendations, or embraced them only weakly. These included reform of the civil service, bolstering government capacity to support the private sector, and adopting and implementing effective procedures for bankruptcy and liquidation. Weaknesses in these areas later emerged as significant obstacles to development.

8.3 Violation ofprocurement rules. According to the ICR, during the IBL an incident occurred in which a subcontractor retained local staff who were state employees. These individuals were not listed in the contract approved by the Bank. The Chamber of Control of the National Assembly investigated the situation, as did the Bank's Internal Auditing Department (IAD). According to the ICR, IAD concluded that there was "evidence of non-compliance with the Bank's procurement guidelines," and the Project Implementation Unit did not follow proper procurement procedures, a project shortcoming. Intensified supervision was recommended, with missions carrying out more rigorous ex-post procurement audits, and remedial actions in accordance with these recommendations were taken.

8.4 Financial audits of the IBL were performed in accordance with the Loan Agreement, and no irregularities were noted.

9 Lessons Learned

* Given the high level of borrower commitment, this TA loan was an effective way for the Bank to engage and stimulate reform. The IBL effectively engaged the Bank in the Armenian reform program at a critical time. Armenia was seeking to become involved with the Bank, and the IBL was a successful means by which it could shape and stimulate reform. The IBL provided the means by which reforms could quickly be accelerated when the country's commitment was high. A rehabilitation loan, which would have provided direct budgetary support in exchange

As noted earlier, reform progress slowed somewhat between mid-1992 and mid-1994, due to government's preoccupation with the war. 12

for specific reform commitments, had been ruled out by unsatisfactory macroeconomic circumstances.

In a highly uncertain situation, flexibility is essential. The Bank initiated the IBL with relatively little knowledge of Armenia, and the country had very limited capacity across a broad range of areas key to the functioning of a market economy. The 1BL included sufficient flexibility to accommodate this situation. Funds were reprogrammed from areas where less was done than initially envisaged (e.g., employment services) or where cost savings were achieved (printing of privatization certificates) and applied to areas where it was determined that needs were greater (computerization of the Ministry of Justice, assessment of the judicial system, additional computerization and consultant advice on resource mobilization).

* An inexperienced borrower requires additional supervision. The IBL demonstrated again that an inexperienced Borrower requires additional supervision resources. In addition, the project consisted of a broad range of relatively small subcomponents. The poor performance by the legal consultant and the instance in which government employees worked under an IBL subcontract illustrate the need for more intensive supervision. 13

Annex A

Individuals Interviewed

Present and former GOA officials

Gagik Arzoumanian Nerses Yeritsyan Armenia Deputy Minister of Finance Central Bank of

Vardan Movsesyan Hrant Bagratyan Chairman, Energy Commission former Prime Minister

Shiraz Kirakosyan Armen Yeghizarian Commissioner, Energy Commission former Minister of Economy, and Chairman, Parliamentary Commission Vahram Nercissiantz on Finance and Economy Chief Economic Advisor to the President former World Bank Resident Levon Barkhudarian Representative former Minister of Finance

Ashot Mnatsakanyan Vahram Avanesian First Deputy Minister of Energy former Deputy Minister of Finance

Areg Galstyan Rouzanna Tarverdian Deputy Minister of Energy former Director, ACMC

World Bank Staff

Judy O'Connor Jonathan Walters Country Director former Country Economist and Task Manager, IBL, SAC 1, SAC 11, SATAC Basil KavaAsky former Country Director Martin Slough Task Manager, Enterprise Development Project Owaise Sadaat World Bank Resident Representative Ira Kichigina Country Counsel and Task Manager of Wafik Grais Legal Support Project Division Chief Cyril Muller Hafez Ghanem Task Manager, IBL, SAC II Division Chief 14

Ana Revenga Cheryl Martin former Country Economist and Task Country Officer Manager, Rehabilitation and SAC I Jacqueline Coolidge (FIAS) Lev Freinkman Specialist on Foreign Direct Investment Country Economist and Task Manager, SATAC and SAC III Aleksandra Posarac Social Sector Specialist Onno Ruhl former Country Officer Anush Bezhanyan Social Sector Specialist Chris Hall Task Manager, IBL Karen Grigorian, Resident Mission Yerevan Helen Sutch former Country Economist Gohar Gyulumyan Resident Mission Yerevan Salman Zaheer Financial and Energy Analyst David Shahzadeyan Resident Mission Yerevan Melinda Roth Alexandrowicz Private Sector Development Specialist Sussana Hayrapetyan Resident Mission Yerevan Peter Nicholas Country Officer

USAID

Barry Primm Director, Office of Economic Michael Wyzan Restructuring and Energy, USAID Economic Advisor Yerevan USAID Yerevan

Tom Delaney Michael L. Boyd Director, Program Office Senior Energy Policy Advisor USAID Yerevan USAID Yerevan

Other

Balas Horvath IMF Gerald Oberndorfer Coordinator, U.S. Assistance to the Susan Adams Caucasus IMF 15

Annex B

Basic Data Sheet

Institution Building Loan (Loan 3585-AM)

Key Project Data (amounts in US$ million) Appraisal Actual or Actual as % of estimate current estimate appraisalestimate Total project costs 17.2 15.3 89 Loan amount 12.0 10.7 89 Cofinancing 4.7 4.5 96 Cancellation (US$) 1.3 Date physical components completed 11/30/96 11/30/97

FY93 FY94 FY95 FY96 FY97 FY98 Appraisal estimate (US$M) 0.40 3.20 8.90 12.00 12.00 12.00 Actual (US$M) 0 1.03 3.28 7.33 11.25 10.74a Actual cumulative as % of Credit 0 32 37 61 94 90 aDecrease represents repayment of Special Account of $512,000. $1.26 million of loan was also cancelled.

Project Dates Original Actual Identification 07/92 Preparation 09/92 Appraisal 02/08/93

Negotiations 02/10/93 Board Presentation 03/30/93 03/30/93 Signing 04/07/93 Effectiveness 07/06/93 06/15/93 Project Completion 05/30/96 05/30/97 Loan Closing 11/30/96 11/30/97

Staff Inputs (staff weeks) Stage of Project Cycle Planned Revised Actual Weeks US$ Weeks US$ Weeks US$ Preparation through Appraisal 19.4 49.4 Negotiations through Board Approval 6.0 12.7 Supervision 44.5 95.9 19.8 50.4 108.9 238.0 Completion 5.0 6.0 5.0 6.0 1.5 2.8 Total 49.5 101.9 23.8 56.4 135.8 302.9 16

Mission Data

Date No. of Durationof Specialized staff skills Performancerating Types ofproblems Stage ofproject cycle (mm/yr.) persons mo y represented* Implement. Develop. Status objectives Through Appraisal 07/92 2 10 E n.a. n.a. 11/92 8 7 E, L, F, C Appraisal through Board 02/93 5 12 E, C n.a. n.a. approval Supervision 07/93 4 8 E, LEG, HR n.a. n.a. 09/93 5 7 E, C, F S S 07/94 1 5 E S S 08/94 1 6 E S S Economic blockade, 06/95 2 6 E S S TA delay to Labor Min., poor cons. perf. for econ. legisl. 11/95 2 7 E S S 07/96 4 7 E, F, LEG, SOCE S S 08/96 4 13 E, PROC, SOC S S 12/96 4 7 E, LEG, PROC S S 06/97 4 8 S S Completion 03/98 2 7 E, PROC S S E= Economist PROC Procurement Specialist F = Financial Analyst PSD = Private Sector Development HR = Human Resources SOC = Social Scientist LEG = Legal

Other Project Data Borrower/Executing Agency: Government of Armenia

Related Bank Credits Year of Credit title Purpose approval Status Closing date Rehabilitation Credit Economic stabilization and reform FY95 closed 06/30/96

SAC I Economic stabilization and growth FY96 closed 12/31/97 resumption SATAC I Economic stabilization and growth FY96 closed 06/30/00 resumption

SAC II Consolidate macro stability and lay FY98 closed 06/30/99 foundation for sustained medium-term growth SATAC II Technical support for SAC 11 FY98 ongoing 06/30/02

SAC III Consolidate macroeconomic stability FY99 ongoing 06/30/01 and develop a sustainable private sector I growth over the medium term