Pakistan: Balochistan Devolved Social Services Program

Completion Report

Project Number: 37218 Loan Numbers: 2202, 2203, and 2204 Grant Number: 0028 December 2011

CURRENCY EQUIVALENTS Currency Unit – Pakistan rupee/s (PRs)

At Appraisal At Program Completion 10 September 2005 31 December 2009 PRs1.00 = $0.01676 $0.01166 $1.00 = PRs 59.67 PRs 85.75

ABBREVIATIONS

ADB – Asian Development Bank ADF – Asian Development Fund BLGO – Balochistan Local Government Ordinance BPGR – Balochistan Performance Grant Rules OCR – ordinary capital resources MIS – management information system PETS – public expenditure tracking survey PFC – provincial finance commission PHE – public health engineering PHED – Public Heath Engineering Department PPO – provincial program office PPP – public–private partnership PSC – provincial steering committee QSDS – quantitative service delivery survey TA – technical assistance TMA – tehsil or town municipal administration WSS – water supply and sanitation

NOTES

(i) The fiscal year (FY) of the Government of Pakistan and its agencies ends on 30 June. FY before a calendar year denotes the year in which the fiscal year ends, e.g., FY2010 ends on 30 June 2010.

(ii) In this report, "$" refers to US dollars.

Vice President X. Zhao, Operations Group 1 Director General J. Miranda, Central and West Asia Department Country Director W. Liepach, Pakistan Resident Mission Sector Director D. Kertzman, Public Management, Financial Sector, and Trade Division, Central and West Asia Department

Team leader O. Bin Zia, Project Officer (Public Management), Pakistan Resident Mission Team members M. Abro, Senior Project Officer, Pakistan Resident Mission I. Ali Memon, Associate Project Analyst, Pakistan Resident Mission

In preparing any country program or strategy, financing any project, or by making any designation of or reference to a particular territory or geographic area in this document, the Asian Development Bank does not intend to make any judgments as to the legal or other status of any territory or area. CONTENTS

Page BASIC DATA i

I. PROGRAM DESCRIPTION 1

II. EVALUATION OF DESIGN AND IMPLEMENTATION 1 A. Relevance of Design and Formulation 1 B. Program Outputs 3 C. Program Costs 7 D. Disbursements 7 E. Program Schedule 8 F. Implementation Arrangements 8 G. Conditions and Covenants 8 H. Related Technical Assistance 9 I. Consultant Recruitment and Procurement 9 J. Performance of Consultants, Contractors, and Suppliers 10 K. Performance of the Borrower and the Executing Agency 10 L. Performance of the Asian Development Bank 11

III. EVALUATION OF PERFORMANCE 11 A. Relevance 11 B. Effectiveness in Achieving Outcome 12 C. Efficiency in Achieving Outcome and Outputs 12 D. Preliminary Assessment of Sustainability 12 E. Institutional Development and Impact 13

IV. OVERALL ASSESSMENT AND RECOMMENDATIONS 13 A. Overall Assessment 13 B. Lessons 13 C. Recommendations 14

APPENDIXES 1. Design and Monitoring Framework 15 2. Implementation Status of Policy Matrix under the Program 22 3. Release and Utilization of Performance Grants 27 4. Implementation Arrangements 29 5. Status of Compliance with Loan Covenants 30 6. Technical Assistance Completion Report of TA 4673–PAK 34 7. Overall Assessment of the Balochistan Devolved Social Services Program 36

BASIC DATA (ORDINARY CAPITAL RESOURCES LOAN)

A. Loan Identification

1. Country Pakistan 2. Loan Number 2202 3. Program Title Balochistan Devolved Social Services Program 4. Borrower Islamic Republic of Pakistan 5. Executing Agency Planning & Development Department, Balochistan 6. Amount of Loan $130 million 7. Program Completion Report PCR: PAK 1296 Number

B. Loan Data 1. Appraisal – Date Started 5 September 2005 – Date Completed 10 September 2005

2. Loan Negotiations – Date Started 24 October 2005 – Date Completed 26 October 2005

3. Date of Board Approval 08 December 2005

4. Date of Loan Agreement 24 August 2006

5. Date of Loan Effectiveness – In Loan Agreement 24 November 2006 – Actual 16 November 2006 – Number of Extensions none

6. Closing Date – In Loan Agreement 31 December 2009 – Actual 31 December 2009 – Number of Extensions none

7. Terms of Loan – Interest Rate ADB’s London interbank offered rate (LIBOR)- based lending facility – Maturity (number of years) 15 years – Grace Period (number of years) 3 years

8. Disbursements a. Dates Initial Disbursement Final Disbursement Time Interval

14 December 2006 14 December 2006 0

Effective Date Original Closing Date Time Interval

16 November 2006 31 December 2009 37 months

b. Amount ($ million) Category Last Net Amount or Original Revised Amount Available Amount Undisbursed Subloan Allocation Allocation Canceled Disbursed Balance 1. BDSSP 130 20 110 110 0

Total 130 20 110 110 0

BDSSP = Balochistan Devolved Social Services Program

C. Program Data

1. Program Cost ($ million)

Cost Appraisal Estimate Actual

Foreign Exchange Cost 130 110 Local Currency Cost 0 0 Total 130 110

2. Program Schedule Item Appraisal Estimate Actual First tranche 110 110 Second tranche 20 0

3. Program Performance Report Ratings Ratings

Development Implementation Implementation Period Objectives Progress From 31 December 2005 to 30 December 2006 Satisfactory Satisfactory From 31 December 2006 to 30 December 2007 Satisfactory Satisfactory From 31 December 2007 to 30 December 2008 Satisfactory Satisfactory From 31 December 2008 to 30 December 2009 Partly satisfactory Partly satisfactory

iii

D. Data on Asian Development Bank Missions No. of No. of Specialization of Name of Mission Date Persons Person-Days Members Loan fact-finding 4–15 Jul 2005 10 120 a, b, c, e, f, g, i, j, k, l Loan appraisal 4–10 Sep 2005 3 21 b, c, l Loan negotiations 24–26 Oct 2005 5 15 c, l, m Review 1 13–18 Mar 2006 6 21 b, c Review 2 3–8 Jul 2006 4 24 a, b, f, I Review 3 21–26 Apr 2008 2 12 a, b Review 4 12–17 May 2008 2 12 a, b a = senior social sectors specialist, b = consultant, c = health specialist, d = project analyst, e = environment specialist, f = education specialist, g = senior governance specialist, h = education advisor, i = economist, j = project economist, k = program officer (Pakistan Resident Mission), l = counsel, m = country program specialist

Note: The Pakistan Resident Mission provided support to the missions in the field.

BASIC DATA (ASIAN DEVELOPMENT FUND LOAN)

A. Loan Identification

1. Country Pakistan 2. Loan Number 2203 3. Program Title Balochistan Devolved Social Services Program 4. Borrower Islamic Republic of Pakistan 5. Executing Agency Planning & Development Department, Balochistan

6. Amount of Loan SDR45,056,000 (equivalent to $65 million) from ADB’s Special Funds resources 7. Program Completion Report PCR: PAK 1296 Number

B. Loan Data 1. Appraisal – Date Started 5 September 2005 – Date Completed 10 September 2005

2. Loan Negotiations – Date Started 24 October 2005 – Date Completed 26 October 2005

3. Date of Board Approval 08 December 2005

4. Date of Loan Agreement 24 August 2006

5. Date of Loan Effectiveness – In Loan Agreement 24 November 2006 – Actual 16 November 2006 – Number of Extensions none

6. Closing Date – In Loan Agreement 31 December 2009 – Actual 31 December 2009 – Number of Extensions none

7. Terms of Loan – Interest Rate 1% per annum during the grace period and 1.5% thereafter – Maturity (number of years) 24 years – Grace Period (number of years) 8 years

8. Disbursements a. Dates Initial Disbursement Final Disbursement Time Interval

14 December 2006 14 December 2006 0

Effective Date Original Closing Date Time Interval

16 November 2006 31 December 2009 37 months

b. Amount (SDR) Category or Last Net Amount Subloan Original Revised Amount Available Amount Undisbursed Allocation Allocation Canceled Disbursed Balance 1. BDSSP 45,056,000 29,460,000 15,596,000 15,596,000 0 a Total ($ 65,000,000 23,488,980 0 Equivalent)

BDSSP = Balochistan Devolved Social Services Program

C. Program Data

1. Program Cost (SDR) Cost Appraisal Estimate Actual

Foreign Exchange Cost 45,056,000 15,596,000 Local Currency Cost 0 0 Total 45,056,000 15,596,000

2. Program Schedule ($) Item Appraisal Estimate Actual First tranche 22,500,000 23,488,980 Second tranche 42,500,000 0

3. Program Performance Report Ratings Ratings

Development Implementation Progress Implementation Period Objectives From 31 December 2005 to 30 December 2006 Satisfactory Satisfactory From 31 December 2006 to 30 December 2007 Satisfactory Satisfactory From 31 December 2007 to 30 December 2008 Satisfactory Satisfactory From 31 December 2008 to 30 December 2009 Partly satisfactory Partly satisfactory

D. Data on Asian Development Bank Missions No. of No. of Specialization of Members Name of Mission Date Persons Person-Days Loan Fact-finding 4–15 Jul 2005 10 120 a, b, c, e, f, g, i, j, k, l Loan Appraisal 4–10 Sep 2005 3 21 b, c, l Loan Negotiations 24–26 Oct 2005 5 15 c, l, m Review 1 13–18 Mar 2006 6 12 b, c Review 2 3–8 Jul 2006 4 24 a, b, f, I Review 3 21–26 Apr 2008 2 12 a, b Review 4 12–17 May 2008 2 12 a, b a = senior social sectors specialist, b = consultant, c = health specialist, d = project analyst, e = environment specialist, f = education specialist, g = senior governance specialist, h = education advisor, i = economist, j = project economist, k = program officer (Pakistan Resident Mission), l = counsel, m = country program specialist a At historical rate. Note: The Pakistan Resident Mission provided support to the missions in the field.

BASIC DATA (ASIAN DEVELOPMENT FUND TECHNICAL ASSISTANCE LOAN)

A. Loan Identification 1. Country Pakistan 2. Loan Number 2204 3. Program Title Balochistan Devolved Social Services Program 4. Borrower Islamic Republic of Pakistan 5. Executing Agency Planning & Development Department, Balochistan 6. Amount of Loan SDR3,466,000 (equivalent to $5 million) from ADB’s Special Funds resources 7. Program Completion Report PCR: PAK 1296 Number

B. Loan Data 1. Appraisal – Date Started 5 September 2005 – Date Completed 10 September 2005

2. Loan Negotiations – Date Started 24 October 2005 – Date Completed 26 October 2005

3. Date of Board Approval 08 December 2005

4. Date of Loan Agreement 24 August 2006

5. Date of Loan Effectiveness – In Loan Agreement 24 November 2006 – Actual 16 November 2006 – Number of Extensions none

6. Closing Date – In Loan Agreement 31 December 2010 – Actual 30 June 2011 – Number of Extensions nonea

7. Terms of Loan – Interest Rate 1% per annum during the grace period of and 1.5% thereafter – Maturity (number of years) 32 years – Grace Period (number of years) 8 years

a The delay in actual closing was due to delayed refund of outstanding imprest balance. See para. 28 of main report.

vii

8. Disbursements a. Dates Initial Disbursement Final Disbursement Time Interval

3 August 2007 30 June 2011 47 months

Effective Date Original Closing Date Time Interval

16 November 2006 31 December 2010 49 months

b. Amount (SDR) Category or Last Net Amount Undis- Subloan Original Amount Revised Amount Disbursed bursed Allocation Canceled Allocation Available Balance 1. Training and capacity building 745,000 741,037 3,963 3,963 3,963 0 2. Workshop and seminars 173,000 157,284 15,716 15,716 15,716 0 3. Equipment, vehicle, surveys, and studies 510,000 485,300 24,700 24,700 24,700 0 4. Operating costs 507,000 499,960 7,040 7,040 7,040 0 5. Consulting services 1,365,000 1,329,605 35,395 35,395 35,395 0 6. Unallocated 166,000 166,000 0 0 0 0 7. Impress Account 0 0 0 0 0 Total SDR 3,466,000 3,379,186 86,814 86,814 86,814 0

$ Equivalent 5,000,000 126,154a 0 a At historical rate.

C. Project Data

1. Project Cost ($)

Cost Appraisal Estimate Actual

Foreign Exchange Cost 183,750 0 Local Currency Cost 6,066,250 296,355 Total 6,250,000 296,355a a Amount includes government counterpart share of $170,201 (as per government program completion report), at historical rate.

2. Project Schedule Item Actual Date of Contract with Consultants Contract 0004: Altaf Bhatti 10 May 2007 Contract 0005: Abdul Quayyum 31 May 2007 Contract 0006: Aqeel Taj 11 July 2008

viii

Item Actual Contract 0007: Noor Ahmed 20 September 2007 Contract 0008: Abdul Malik Sherani 11 July 2008 Equipment and Supplies Dates First Procurement 7 July 2008 Last Procurement 25 August 2008 Workshop and Seminars Workshops for midterm and annual plans 18 August 2008 Trainings and Capacity Building Trainings 7 July 2008 3. Project Performance Report Ratings Ratings

D. Data on Asian Development Bank Missions No. of No. of Specialization of Name of Mission Date Persons Person-Days Members Loan fact-finding 4–15 Jul 2005 10 120 a, b, c, e, f, g, i, j, k, l Loan appraisal 4–10 Sep 2005 3 21 b, c, l Loan negotiations 24–26 Oct 2005 5 15 c, l, m Review 1 13–18 Mar 2006 6 12 b, c Review 2 3–8 Jul 2006 4 24 a, b, f, I Review 3 21–26 April 2008 2 12 a, b Review 4 12–17 May 2008 2 12 a, b a = senior social sectors specialist, b = consultant, c = health specialist, d = project analyst, e = environment specialist, f = education specialist, g = senior governance specialist, h = education advisor, i = economist, j = project economist, k = program officer (Pakistan Resident Mission), l = counsel, m = country program specialist

Note: The Pakistan Resident Mission provided support to the missions in the field.

BASIC DATA (ASIAN DEVELOPMENT FUND GRANT)

A. Grant Identification

1. Country Pakistan 2. Grant Number 0028 3. Program Title Balochistan Devolved Social Services Program 4. Borrower Islamic Republic of Pakistan 5. Executing Agency Planning & Development Department, Balochistan 6. Amount of Grant US$5 million from ADB’s Special Funds resources 7. Program Completion Report PCR: PAK 1296 Number B. Grant Data 1. Appraisal – Date Started 5 September 2005 – Date Completed 10 September 2005

2. Negotiations – Date Started 24 October 2005 – Date Completed 26 October 2005

3. Date of Board Approval 08 December 2005

4. Date of Agreement 24 August 2006

5. Date of Effectiveness – In Agreement 24 November 2006 – Actual 16 November 2006 – Number of Extensions none

6. Closing Date – In Agreement 31 December 2009 – Actual 31 December 2009 – Number of Extensions none

7. Disbursements a. Dates Initial Disbursement Final Disbursement Time Interval

14 December 2006 14 December 2006 0

Effective Date Original Closing Date Time Interval

16 November 2006 31 December 2009 37 months

b. Amount (US$) Category or Last Net Amount Subgrant Original Revised Amount Available Amount Undisbursed Allocation Allocation Canceled Disbursed Balance 1. BDSSP 5,000,000 2,500,000 2,500,000 2,500,000 2,500,000 0 Total 2,500,000 0

BDSSP = Balochistan Devolved Social Services Program

C. Grant Data

1. Program Cost (US$) Cost Appraisal Estimate Actual

Foreign Exchange Cost 5,000,000 2,500,000 Local Currency Cost 0 0 Total 5,000,000 2,500,000

2. Grant Schedule ($) Item Appraisal Estimate Actual First tranche 2,500,000 2,500,000 Second tranche 2,500,000 0

3. Program Performance Report Ratings Ratings

I. PROGRAM DESCRIPTION

1. On 8 December 2005, Asian Development Bank (ADB) approved the Balochistan Devolved Social Services Program (Program) for $205 million to support the government of Balochistan Province in implementing reforms to improve the delivery of health, education, and water supply and sanitation (WSS) services by public and private providers.1 The Program included (i) a program loan of $130 million from ADB’s ordinary capital resources (OCR) to meet Program adjustment costs, (ii) a program loan of $65 million from ADB’s Special Fund resources to finance performance grants to districts, (iii) a technical assistance (TA) loan of $5 million from Special Fund resources to support Program implementation, and (iv) a grant of $5 million from Special Fund resources to finance the control of HIV/AIDS and other infectious diseases. The program loans and grant were to be disbursed in two tranches. The first tranche was valued at $135 million and second at $65 million (see para. 24). Noncompliance with most second tranche actions prevented the release of this tranche.

2. The education, health, and WSS indicators for Balochistan province were much lower than for the country as a whole, with wide variations between districts and rural–urban disparities.2 The Program aimed to accelerate the province’s progress towards the Millennium Development Goals (MDGs) and the social sector targets of the government of Balochistan’s poverty reduction strategy by utilizing opportunities for enhancing the role of local governments in social service delivery provided by the Balochistan Local Government Ordinance (BLGO), 2001.3

3. Program outputs were designed to strengthen (i) provincial and district government relationships through provincial efficiency incentives for local governments; (ii) the relationship between district governments and social service delivery providers by devolving managerial powers to district governments; (iii) relationships between citizens and provincial and district policy makers; and (iv) private sector participation in service delivery. The Program design and monitoring framework is in Appendix 1.

II. EVALUATION OF DESIGN AND IMPLEMENTATION

A. Relevance of Design and Formulation

4. The Program’s design was relevant at appraisal but diminished with weakened political commitment to devolution, which was inconsistent with Program design. The Program was aligned with the human development goals of the government of Balochistan’s poverty reduction strategy for education, health, and WSS. The Program design supported reforms to strengthen local governments and introduce stakeholder participation in managing social facilities in line with the BLGO.

5. The Program design was consistent with ADB’s country strategy and program updates for 2004 and 2005, which maintained the strategic focus on good governance and social sectors

1 ADB. 2005. Report and Recommendation of the President to the Board of Directors: Proposed Program Loans, Technical Assistance Loan, and Asian Development Fund Grant to the Islamic Republic of Pakistan for the Balochistan Devolved Social Services Program. Manila (Loans 2202, 2203[SF], 2204[SF], and grant 0028[SF]-PAK for $205 million equivalent, approved on 8 December 2005). 2 Government of Pakistan, Federal Bureau of Statistics. 2002. Pakistan Integrated Household Survey. Islamabad; Government of Balochistan, Bureau of Statistics. 2004. Multiple Indicator Cluster Survey. Quetta. 3 Government of Balochistan. 2003. Balochistan Poverty Reduction Strategy Paper. Quetta; Government of Balochistan. 2001. Balochistan Local Government Ordinance, 2001 (Ordinance XVIII of 2001). Quetta.

2 to support Pakistan’s progress toward the MDGs.4 A program preparatory TA was approved on 8 July 2004 to design support mechanisms through a program loan and a TA project for building government capacity in social service delivery. 5 The team processing the program and TA consulted closely with the program management units of the Balochistan Resource Management Program and Decentralization Support Program to ensure the Program’s complementarity, fiscal prudence, and efficient resource allocation. 6 The Program’s design benefitted from stakeholder consultation and lessons learned from performance grant mechanisms established by the Sindh and Punjab devolved social services programs.7

6. However, despite the planned synergies with existing strategies and programs of both ADB and the government of Balochistan, the risk of diminished political commitment to decentralization was unforeseen at Program design. Despite the constitutional protection provided to the local government system, the elected provincial assemblies brought amendments to the BLGO in 2005, tilting the balance of power toward the provincial executive.8 The Balochistan Local Government Act, 2010 substantially reversed the devolution of social sectors.9 This act abolished the Provincial Finance Commission and the Local Government Commission, among other new institutions, upon which Program reforms had relied for strengthening local governments. The Program’s design contained no mechanisms to address such changes.

7. The Program aimed to improve service delivery in multiple sectors, reform governance in multiple tiers of government, and add new reforms to the unevaluated reform agenda of the Balochistan Resource Management Program. Capacity constraints in Balochistan and limited experience in implementing a range of reforms through the program loan modality were not adequately considered in the multisector program design. The sequencing of reforms under the policy matrix was back loaded, with outputs to be realized at the end of the Program. The TA loan design envisaged recruiting consultants and establishing Program support units in provincial and local governments. The government’s commitment to implementing a comprehensive reform agenda wavered, as it decided not to recruit consultants or contract staff for the Program units under the TA loan (para. 37).

4 ADB. 2004. Country Strategy and Program Update: Pakistan, 2005–2006. Manila; ADB. 2005. Country Strategy and Program Update: Pakistan, 2006–2008. Manila. 5 ADB. 2004. Technical Assistance to Pakistan for Preparing the Balochistan Devolved Social Services Program. Manila. 6 ADB. 2004. Report and Recommendation of the President to the Board of Directors on Proposed Program Loans and Technical Assistance Grants to the Islamic Republic of Pakistan for the Balochistan Resource Management Program. Manila (Loans 2107 and 2108[SF]-PAK); ADB. 2002. Report and Recommendation of the President to the Board of Directors on Proposed Program and Technical Assistance Loans to the Islamic Republic of Pakistan for the Decentralization Support Program. Manila (Loan 1935 and1936-PAK). 7 ADB. 2003. Report and Recommendation of the President to the Board of Directors on Proposed Program and Technical Assistance Loans to the Islamic Republic of Pakistan for the Sindh Devolved Social Services Program. Manila (Loan 2049-PAK, for $220 million, approved 12 December); ADB. 2004. Report and Recommendation of the President to the Board of Directors on Proposed Program Loans and Technical Assistance Grant to the Islamic Republic of Pakistan for the Punjab Devolved Social Services Program. Manila (Loans 2144 and 2145[SF]-PAK, for $150 million equivalent, approved 20 December). 8 For example, see the 2005 amendment to section 25 of the BLGO, 2001. The power to suspend an order or decision of the elected chief executive of the district was transferred to the chief executive of the province from the local government commission, an institution with broad representation of the government, technocrats, and civil society. 9 Government of Balochistan. 2010. Balochistan Local Government Act, 2010 (Act No. V of 2010). Quetta.

B. Program Outputs

8. Program outputs were structured around four areas: (i) provincial efficiency incentives to strengthen and clarify intergovernmental relationships and roles; (ii) strengthened managerial powers in local governments to improve relationships between local governments and providers of social services; (iii) enhanced citizens’ voice and empowerment to strengthen relationships between citizens and provincial and district policy makers; and (iv) enhanced engagement of the private sector and nonprofit organizations in social service delivery. TA loan outputs comprising the development of sector strategies, strengthened capacity of social sector managers and service providers, and improved monitoring and evaluation tools, were envisioned to support related policy reforms.

9. As discussed below and noted in para. 1, most Program outputs were not achieved due to the government’s noncompliance with second tranche actions. Appendix 2 reflects the implementation status of reforms under the policy matrix.

1. Provincial Efficiency Incentives

10. Budget certainty for social sectors. The Program did not succeed in making the performance grants system a source of predictable transfers to local governments for social sectors. Under the first tranche of the Program, the Provincial Finance Commission (PFC) award and performance grant system were established, and performance grants amounting to PRs427.60 million were released to 19 of the 29 districts governments by May 2006. 10 Counterpart funds—amounting to PRs1,350.00 million generated from the proceeds of the Asian Development Fund (ADF) loan of $22.50 million equivalent and PRs135.00 million generated from the proceeds of the ADF grant of $2.50 million equivalent—were made available for transfer to local governments as performance grants during financial year (FY) 2008 and FY2009.

11. The release and utilization of performance grants were slowed by the limited absorptive capacity of local governments and inadequate technical support and coordination from Appendix 2 2 Appendix provincial sector departments toward preparing utilization proposals in accordance with grant objectives. 11 The Finance Department did not agree to include tehsil (subdistrict) or town municipal administrations (TMAs) in the performance grant system to sidestep capacity constraints and fiduciary risk from enhanced funding at that level. The government requested a

partial waiver of the related second tranche action, to which ADB did not agree. Reporting 3 requirements under rule 9 of the Balochistan Performance Grant Rules (BPGR), 2005 were not fully implemented as the districts did not provide key information to the provincial program office, despite requests. The BPGR required local governments to comply with eligibility criteria and contract conditions and submit applications to the PFC Secretariat. 12 Because local governments had difficulty understanding the application process, by the end of the FY2008, 12 of the 30 district governments had not applied for the second installment and, by the end of FY2009, only 6 district governments had accessed the second installment of performance grants. Subsequently, PRs1,051.72 million was released from counterpart funds to all the districts. As of the end of October 2010, utilization stood at only PRs603.60 million, only 44% of the counterpart funds available under the ADF loan. See Table A3.1 in Appendix 3 on the

10 These performance grants were financed from counterpart funds generated from the first tranche of the Balochistan Resource Management Program. Subsequent releases increased the amount of first installment of performance grants for fiscal year 2006 to PRs510.50 million. 11 ADB. 2008. Completion Report: Balochistan Resource Management Program in Pakistan. Manila 12 Schedule III to the BPGR, 2005.

4 release and utilization of performance grants by district. The achievement of this output was therefore only partly successful.

12. Autonomy in budget preparation and administrative actions. Program actions to redefine the roles and responsibilities of provincial departments and local governments for the social sectors in line with the BLGO were only partly implemented. Proposals for rationalizing the roles of the education, health, and public health engineering (PHE) departments were developed by departmental working groups and made available on government website during the first tranche. Education and health proposals were comprehensive and included job descriptions for district managers and supervisors, staffing requirements, and proposals for further devolving administrative powers in line with the BLGO. The devolution of the PHE Department was only partly achieved, and its proposal for rationalizing its role and staffing reflected incomplete devolution. These proposals remained unapproved. The government requested a partial waiver for the second tranche action, to which ADB did not agree. A draft of the revised rules of business of the government was prepared in 2008, but this, too, was not approved by the government because of impending changes in devolution under the new local government framework.13 The achievement of this output was therefore unsuccessful.

13. Improved quality of social services. The envisaged outputs of approved sector strategies for health, education, and WSS—complete with approved expenditure-allocation norms in line with minimum standards for social sectors—were only partly achieved for health and sanitation. Sector strategies for education and water supply were not finalized, and the expenditure allocation norms remained under revision. A component under this output requiring the notification of clinical laboratory regulatory rules and safe blood transfusion rules for Balochistan was completed with both the rules notified and published in the official gazette and on the government website. The first drafts of sector strategies were prepared as a prior action for the release of the first tranche. However, because specialists were not recruited, the provincial education strategy could not be finalized. Work on a provincial drinking water supply strategy was subsequently initiated with the support of the United Nations Children’s Fund. Awareness of the sector strategies remained limited, as did their budget implications for implementing expenditure norms in line departments. The achievement of this output was therefore unsuccessful.

2. Managerial Power

14. District governments’ authority to appoint and transfer staff. Policy actions aimed to provide local governments with managerial authority to appoint, transfer, provide incentives to, and otherwise manage staff in the education, health, and PHE sectors. However, the provincial cabinet did not approve sector departments’ proposals for amending the Balochistan Removal from Services Ordinance, 2001 and granting to district governments the power to appoint and transfer staff in basic scales 1–16. Draft proposals for an incentive system for each of the three sectors were developed and posted on the government website in November 2006, prior to the release of the first tranche. However, the financial implications of implementing the incentive system delayed the seeking of chief minister’s approval until January 2010. Only a few incentives were implemented in education and health, such as introducing a timescale for promoting teachers, providing transportation and residential facilities for female teachers, and, in some instances, providing residential accommodation to health staff. However, these incentives were not based on a comprehensive policy. The government requested the waiver of

13 A comparison of the roles of the health and education departments in the 2006 notified rules of business and the 2008 revised draft reveals few changes that could align the department’s role with the BLGO, 2001.

two of the three second tranche policy actions under this output, to which ADB did not agree as it considered the policy actions essential for improving devolved social services. The output was therefore not achieved.

15. Adequate deployment of staff. The Program intended to achieve adequate staffing through a system of contract appointments at the facility level in the three sectors to reflect demand and provide greater accountability to communities. While the education, health, and PHE departments made many appointments of service delivery staff at social service facilities, filling up to 90% of the vacant posts, guidelines for contractual appointments were not implemented. The output was therefore only partly achieved.

16. Improved capacity of district and facility managers. While the government developed draft management information system strategies and human resource action plans for the social sectors, it was unable to finalize the management support systems required to implement them. A draft strategy for developing a management information system for WSS was developed, but no evidence was provided regarding its approval or implementation. Draft strategies for strengthening the management information systems for education and health were uploaded to the government website before the release of the first tranche, but the government did not provide any evidence of implementation. Over 26% of the TA loan was allocated to training and workshops, of which only 2% was utilized. Only six trainings were organized: for developing district capacity in medium-term and annual plans, financial management, and school management. The output was therefore not achieved.

3. Citizens’ Voice and Empowerment

17. Grievance-redress mechanism. The Program did not succeed in institutionalizing a grievance-redress mechanism at facilities or in implementing a local government monitoring mechanism through elected local representatives. For 60% of the districts, the government provided lists with names and contact details of facility managers and, in some cases, certification by executive district officers regarding the setting up of complaint cells. For WSS

the complaint cells were set up at the tehsil (subdistrict) and district level. No evidence was 2 Appendix provided on how effective the grievance-redress system was. ADB’s April 2008 review mission noted that few service users were aware of the mechanism.

18. Program actions included the development, notification, and implementation of local

government monitoring committees’ rules. Completing a first tranche action, the government 5 notified the local government monitoring rules in August 2006. The rules comprehensively described procedures for electing monitoring committees, their functions and responsibilities, and reporting and follow-up actions for implementing their recommendations. The monitoring committees were notified, but no evidence of their functioning was provided before the Balochistan Local Government Act, 2010 reversed the monitoring committee system. The output was therefore not achieved.

19. Community management of facilities. The Program was unable to strengthen the role of communities in managing social service facilities. The communication strategy developed as a first tranche condition was not approved until Program closure. The provincial government used its own strategy to publicize development activities without strengthening communities. District governments were not provided with the resources or capacity to implement a communication strategy for enhancing the role of communities in managing health and education facilities. The Education Department established 4,234 parent–teacher school management committees in 24 districts in 2002 under a program funded by the United States

Agency for International Development. The intent of the Program action was to activate and train these school management committees. The lists of committees constituted in 2002 were provided as evidence, but they were not updated and there was no indication of their being operational. The Health Department issued notifications in June 2007 constituting hospital management committees for all district headquarters hospitals and two other 50-bed hospitals. However, no evidence was provided regarding their functioning or the selection of private members from civil society, nongovernment organizations, or private general practitioners. The output was therefore not achieved.

20. Civil society monitoring of social service delivery and citizen oversight. The Program was unable to institutionalize citizen oversight and monitoring programs through the implementation of public expenditure tracking surveys, quantitative service delivery surveys, and citizen report cards. While drafts of the surveys were developed under the first tranche, subsequently the government requested waivers for all the three actions, saying that the surveys and report cards were not required. ADB did not agree to the request, as the Program included these activities to pursue community empowerment under the BLGO. The BPGR included the utilization of the surveys and citizen report cards as tools for monitoring and verifying performance indicators for improvements in public service delivery.14 The government’s reasons for not conducting these surveys did not appear to be relevant.15 The provisions of the Balochistan Local Government Act, 2010 are indicative of the limited role envisaged for community monitoring and oversight of public service delivery. The output was therefore not achieved.

4. Private Sector Engagement

21. Private sector participation. The Program was unable to leverage the advantages that private sector participation in social services delivery could have contributed. Draft strategies for public-private partnerships (PPP) and strengthening private sector participation in social service delivery were developed and placed on the government website under the first tranche. However, there is no evidence that such strategies were approved by a competent forum or implemented. Line departments put forward arrangements pre-dating the Program as examples of PPPs. The Health Department provided examples of quasi-PPP programs, such as the Balochistan Health Foundation and federally funded President’s Primary Healthcare Initiative, which contracted out the management of some basic health units to a nongovernment organization. The Education Department reported the Balochistan Education Foundation’s operation of some community schools.16 The output was therefore not achieved.

22. Regulation of private social services delivery. The Program was unable to promote private sector self-regulation in health and education. While draft proposals for self-regulation in health and education were prepared under the first tranche, the April 2008 review found little awareness of these strategies in the departments. The preliminary proposals were basic and

14 Section 10 and schedule VIII of the BPGR, 2005. 15 In requesting waivers for the implementation of PETSs, the government stated that an expenditure tracking system between the Finance Department and the accountant general of Balochistan was already functional, so there was no need for introducing another such system at the district and facility level. The reason given for not conducting the QSDS was that a similar survey had already been conducted on the health sector back in 1999 and 2000, with the report published in a scientific journal. The pilot of the citizen report card for Quetta was for the WSS and street lighting services, which the government did not want to conduct, arguing that the acute shortage of water supply in Quetta was already publicly known. 16 The Balochistan health and education foundations were autonomous bodies set up under acts of the provincial parliament in 1994. Their activities include providing assistance to private institutions in the sector, but they remain funded by the government, including some direct assistance from development partners.

provided theoretical and definitional aspects of self-regulation. The situation on the ground did not reflect any inclusion of the concept of self-regulation in the social sectors under the Program. The provincial Health Regulatory Authority established under an ordinance promulgated in 2001 had started functioning, but it lacked the vigor envisaged under the proposed self-regulatory approach. The output was therefore not achieved.

5. Technical Assistance Loan

23. The Program design included a TA loan estimated at $6.25 million equivalent, with $5 million coming from ADB’s Special Fund resources and $1.25 million from the government. The TA loan was provided to support sector work on the following policy reforms: strengthening sector planning, management, and monitoring; developing local government and community capacity in social service governance; and implementing policy reforms and performance grants. Only $126,154, or 2.52% of the $5 million allocation, was utilized: to hire five contracted support staff for the provincial program office; train 190 government functionaries of the education, health, PHE, local government, and rural development departments; and procure some office equipment (see paras. 36–40). The intended outputs under the TA loan were therefore not achieved.

C. Program Costs

24. Total funding for the Program was $205 million, of which $200 million was a loan and $5 million a grant. The Program amount was to be released in two tranches once the government fulfilled certain policy actions. The first tranche consisted of $135 million ($110 million from OCR and $25 million from ADF), and the second tranche was $65 million ($20 million from OCR and $45 million from ADF). The $5 million ADF grant for controlling HIV/AIDS and other infectious diseases was to be released in two tranches of $2.5 million each. The $5 million TA loan was to be disbursed to cover eligible expenditures incurred.

25. Counterpart funds generated from the ADF program loan and grant were meant to support performance grants to finance local government expenditures in education, health, and Appendix 2 2 Appendix WSS until 30 June 2010. This was partly implemented through transfers to local governments (paras. 10–11). Counterpart funds generated from the entire OCR loan were paid by Balochistan to the federal government to refinance its high-interest debt. The fiscal space thus created was to be dedicated to providing current budget support in devolved social sectors and

transfers to the local governments under the PFC award. This was partly implemented until 7 FY2010, after which the PFC was abolished and grants to local governments stopped.

D. Disbursements

26. As appraised, the first tranche of $135 million was disbursed in accordance with ADB’s standard disbursement procedures. The release of the first tranche was originally scheduled for July 2006, but delayed compliance with certain prior actions delayed the release to December 2006. As per the original design, the provincial government paid $110 million out of the first tranche release to the federal government to retire high-interest debt, which freed up PRs815 million annually for the government beginning in FY2009. The ADF component of PRs1,350 million meant for performance grants was allocated by the Finance Department to all 30 districts of the province to finance their various social sector schemes. However, the actual amount released to the districts was PRs1,051 million, of which the districts were able to spend, as of October 2010, only PRs603.60 million, or 57%. Similarly, of the ADF grant funds of PRs135 million withdrawn by the government as part of the first tranche, PRs121 million was released to

8 the districts for preventing the spread of HIV/AIDS and other infectious diseases. Weak absorptive capacity meant the districts were able to utilize, as of October 2010, only PRs57.90 million, or 48% (see Table A3.2 in Appendix 3). The support provided as a TA loan was minimally utilized, with the government spending only $126,000, or 2.5% of the $5 million.

27. Second tranche release depended on the government fulfilling 21 policy actions. However, finding it difficult to meet certain policy actions, the government requested a number of waivers (para. 32). ADB did not agree, so the second tranche was not released. Subsequently, the government requested ADB to bring forward the closing date of the TA loan from 31 December 2010 to 24 May 2010, which was accordingly effected.

E. Program Schedule

28. The original Program period was from 1 July 2006 to 30 June 2009. Both program loans and the TA loan became effective on 16 November 2006, a week earlier than scheduled in the loan agreement. The first tranche was released on 14 December 2006, about 6 months later than planned. The second tranche was scheduled for release in July 2008, but noncompliance with policy actions prevented tranche release. The two program loans closed on 31 December 2009 as per the original plan, and the TA loan closed early, on 24 May 2010, at the government’s request. Financial closure of the TA loan, on 30 June 2011, was delayed by the government’s delayed refund of the outstanding imprest balance. There were considerable delays in the government’s disbursing of funds to districts under the performance grant mechanism and in districts’ utilization of funds to implement social sector schemes.

F. Implementation Arrangements

29. The Planning and Development Department was the executing agency for the Program. The structure of Program implementation arrangement consisted of (i) a program-specific mechanism; (ii) provincial arrangements; and (iii) arrangements at the district, tehsil, and community level (Appendix 4).

30. The provincial steering committee (PSC) was the highest Program authority and responsible for providing policy guidance to the Program, supervising Program activities, approving the annual work plan and budget for the TA component, monitoring the performance of the provincial program office (PPO) and program units, and ensuring coordination with other programs and projects. The PSC met 11 times from December 2005 to June 2008. It tried to expedite the completion of second tranche actions by approving a number of incomplete proposals, which the government did not approve.

31. The multiple layers of implementation units and supervisory bodies were too complex and ambitious. Because the government decided not to set up program units in districts and TMAs, districts achieved very little improvement in service delivery. Changes in key project staff, senior management’s lack of ownership or support, and the uncertain future of the devolution framework under the BLGO were major factors slowing implementation. Implementation arrangements were therefore unsatisfactory.

G. Conditions and Covenants

32. The Program had 41 tranche conditions, 20 for the release of the first tranche and 21 for the second. All the first tranche conditions were met prior to tranche release on 11 December

2006. At the time of the scheduled release of the second tranche in July 2008, the government had achieved full compliance with one policy condition, partial compliance with 13 policy actions, and noncompliance with 7 policy actions, which the government requested ADB to waive. ADB did not agree, as the actions were essential to achieving the Program outcome, so the second tranche was not released. Appendix 2 summarizes the status of compliance with policy actions.

33. The program and TA loans had 33 covenants, only 2 of which were fully met. Among the 14 covenants not complied with was the establishment of TMA program committees and program units, as envisaged under paras. 6 and 11 of schedule 5. Among the 17 covenants partly complied with was the setting up of the PPO, but without full staffing. Loan covenants were overambitious and unrealistic and generally incongruent with social, political, and economic conditions, as well as provincial government capacity. Appendix 5 provides an overview of the status of compliance with loan covenants.

H. Related Technical Assistance

34. A project preparatory TA grant for $400,000, approved by ADB on 8 July 2004 and signed with the government in October 2004, was to fund the designing of a program loan to strengthen devolved social services and improve service delivery to the poor. 17 The TA disbursed $380,522, or 95% of the approved amount, and succeeded in achieving the outputs of (i) designing support mechanisms for making social services devolved to local governments and communities more effective, efficient, equitable, and sustainable; (ii) planning a program loan to establish these support mechanisms and a related policy reform agenda; and (iii) planning technical support to improve social services to meet local needs. Consultant recruitment under the TA was timely, and a team of consultants mobilized in Quetta in February 2005 carried out stakeholder consultations, sector-specific workshops, briefings on the program for the districts heads, and a consultative workshop for senior policy makers.18 The TA team carried out participatory planning sessions for performance grants with representatives and officials of all three tiers of the local government in 13 districts outside Quetta. The feedback from these stakeholder interactions and sector work helped develop the program framework, policy matrix, and design. This TA was thus successful in achieving its outcome. Appendix 2 2 Appendix

35. An advisory TA grant of $300,000 for capacity building was approved on 26 October 2005 and signed with the government on 15 December 2005 to serve a bridging TA before the PPO was fully established.19 This TA was rated partly successful, as it provided the foundation

for policy reforms under the Program. The TA specifically supported the executing agency in 9 preparing compliance documents for the first tranche, developed a training program for local governments on accessing and implementing performance grants, and developing a hybrid survey of public expenditure and service delivery.20 Appendix 6 is the TA completion report.

I. Consultant Recruitment and Procurement

36. At appraisal, the TA loan envisaged the input of 8.3 person-months of international consultancy and 104 person-months of domestic consultancy. None of these consultants was

17 ADB. 2004. Technical Assistance to Pakistan for Preparing the Balochistan Devolved Social Services Program. Manila (TA-4356-PAK). 18 Quetta is the capital of Balochistan Province. 19 ADB. 2005. Technical Assistance to Pakistan for Capacity Building of Balochistan for Devolved Social Services. Manila (TA 4673–PAK). 20 ADB. 2007. Technical Assistance Completion Report for Capacity Building of Balochistan for Devolved Social Services. Manila (TA 4673–PAK).

10 mobilized. In addition, 4,242 person-months of contract staffing was to be hired to help the PPO and the provincial and district program units implement the Program.

37. The proposal for recruiting 7 domestic consultants instead of the originally planned 14 was initiated by the PPO in February 2008, 15 months after loan effectiveness. Recognizing the delay in consultant recruitment and slow progress in implementing second tranche conditions, ADB offered to help the government with recruitment under the TA loan if the government showed commitment to complying with all conditions by mid-2009. The government did not respond to the suggestion and decided not to recruit any consultants.

38. Of the 101 contracted staff intended for the provincial and local government program units, only 5 were recruited, to provide operational support at the PPO. The government decided not to recruit program staff for the program units at the Finance Department, Bureau of Statistics, or districts and tehsils. All contract staff were selected in accordance with ADB’s Guidelines on the Use of Consultants (2010, as amended from time to time). A total of $54,499 was disbursed for consultants, against the original allocation of $1,969,525. National consulting for contracted staff totaled 132 person-months, hired on an individual basis.

39. Procurement was restricted to office equipment for the PPO and sector line departments, with $30,644 disbursed to procure computers, printers, photocopiers, and fax machines, against the original allocation of $735,975. Procurement was carried out in accordance with ADB’s Procurement Guidelines (2010, as amended from time to time). Similarly, no substantial progress was made toward training and capacity building, and few workshop and seminars were held. A total of $30,183 was disbursed for both the categories, against the original allocation of $1,325,000. The limited recruitment of contract staff and procurement carried out under the TA loan was satisfactory.

J. Performance of Consultants, Contractors, and Suppliers

40. No substantial progress was made under the TA loan, as only 2.5% of the loan amount was utilized. Except for the recruitment of five domestic consultants as contract staff for the PPO, the procurement of some office equipment for the PPO and provincial line departments, and a few training and capacity-building events, the TA loan remained unutilized. No domestic and international consultants were recruited. However, the performance of contracted staff in carrying out day-to-day activities and coordinating for the PPO was satisfactory.

K. Performance of the Borrower and the Executing Agency

41. The performance of the borrower and the Planning and Development Department is rated partly satisfactory. First tranche actions were completed as per Program design but with some delay. Changed political commitment to the BLGO prevented the completion of second tranche actions and tranche release.

42. The PSC’s supervision, monitoring, and coordination of Program implementation was less effective (paras. 30–31). While the PPO facilitated program actions for the release of the first tranche, it was ineffective in completing second tranche actions as per Program design. At a critical period in implementing second tranche actions, the PPO was without a full-time program coordinator, which adversely affected the PPO’s ability to carry out effective coordination of program reforms.

43. The government did not effectively utilize the capacity-building opportunities provided by the TA. The executing agency did not recruit specialist consultants or set up program units in sector departments and local governments as envisaged under the program and TA loan design. The disbursement and utilization of performance grants remained slow and were not monitored in accordance with the BPGR. The PPO shared progress and implementation issues regularly with ADB, which helped ADB provide appropriate guidance to the executing agency. The Planning and Development Department did not submit the project completion report as required under the loan agreement.

L. Performance of the Asian Development Bank

44. The performance of ADB is rated partly satisfactory. ADB carried out detailed situation and stakeholder analysis when designing the Program. The planned multisector interventions and simultaneous introduction of governance reforms at the provincial and local level made program design complex. ADB did not, however, assess the executing agency’s capacity to implement multisector reforms through program loan modality and to administer a TA loan. The commitment and interventions of other development partners in the sectors were not adequately assessed.21 ADB fielded two missions to review compliance with first tranche actions and two missions to review progress on second tranche actions. ADB compliance assessment for the first tranche actions and the performance ratings in ADB’s project performance monitoring system were generally liberal, which failed to highlight implementation problems sufficiently. Issues vis-à-vis second tranche actions were adequately identified through the missions and the review of regular progress reports from the PPO. ADB took adequate steps to resolve outstanding issues (para. 37), but the changed political commitment to devolution—a key outcome of local government strengthening—was beyond ADB’s control or effective influence. Changes of ADB program officers during the first 2 years of the Program period contributed to the long gap between ADB review missions after the release of the first tranche. ADB regularly engaged with the executing agency in regard to completing the second tranche actions and provided guidance for resolving implementation challenges.

III. EVALUATION OF PERFORMANCE 2 Appendix

A. Relevance

45. The program is rated partly relevant. At appraisal the Program design was consistent

with ADB and government of Balochistan’s strategy for improving governance and social 11

indicators. However, the multisector design of the Program which relied on decentralization under the BLGO became partly relevant at completion, due to repeal of the BLGO by the Balochistan Local Government Act, 2010, substantially reversing devolution. The design underestimated capacity constraints and political commitment to the reform agenda. While the first tranche actions were successfully completed, the Program was able to complete only one of the 21 second tranche actions, resulting in unsatisfactory outputs. Program design recognized the risk of diminished political commitment to decentralization but assumed that sustained political commitment to human development could outweigh the risk (paras. 4–7).

21 The Program anticipated the provision of TA for strengthening management information systems for the education and health departments and developing one for PHE Department with support coming from other development partners. The United Nations Children’s Fund supported the management information system for WSS, and the Japanese International Cooperation Agency supported a district health information system. The development of a management information system for PHE was not completed, and the ones for the health and education departments were not strengthened by the program (para. 16).

B. Effectiveness in Achieving Outcome

46. The Program was ineffective in achieving its planned outcome. It aimed to make the delivery of devolved education, health, and WSS services by public and private providers broader and more equitable, mainly by strengthening district governments. Except for a few partly completed reforms under the first tranche, the Program was ineffective in implementing policy actions; completion of first tranche actions had limited bearing on the overall outcome of the Program. Changing political commitment to devolution and the lack of capacity to administer the TA loan caused the general failure of policy reforms critical to achieving the outcome. The effectiveness of the Program was diminished by several issues and implementation challenges: (i) the predictable financing of local governments and the implementation of a performance grant system were only partly implemented and subsequently reversed, (ii) the limited absorptive capacity of local governments stymied the utilization of grants, (iii) sector strategies and expenditure norms were not implemented, (iv) administrative and managerial devolution for social sectors was not implemented, (v) grievance-redress and local government monitoring mechanisms were not institutionalized, (vi) public expenditure and service delivery surveys intended to provide citizen feedback were not implemented, and (vii) PPP policies for the social sector were not approved or implemented.

C. Efficiency in Achieving Outcome and Outputs

47. Program management and the utilization of Program resources are rated inefficient. While the PSC and PPO facilitated the completion of the first tranche actions with TA support, the PSC was not able to subsequently supervise and guide Program activities as per design. After first tranche release and the closure of the bridging TA, the PPO was unable to facilitate Program implementation. The absence of local program units hampered program implementation. While the TA loan was available to support Program implementation, only 2.5% was utilized and no specialist consultant was recruited. These problems and noncompliance with policy actions caused cancellation of the second tranche. Additional resources provided under the first tranche successfully created fiscal space for the government through refinancing high-interest debt with counterpart funds generated from proceeds of the OCR loan. However, this fiscal space was not used to continually fund the social sectors through local governments. Only part of the first tranche of the ADF loan and grant was used to provide performance grants to local governments, and disbursement lagged behind schedule.

48. Returns to the public in terms of improved service delivery were minimal, and the intended outcome was not achieved. Given the non-compliance of second tranche actions, policy reversal of the government of Balochistan, and precarious security situation in the province, comprehensive financial and economic analyses were not undertaken. Even if they were computed it is unlikely that high rates of return were obtained.

D. Preliminary Assessment of Sustainability

49. The sustainability of the Program, as per its design, is unlikely. Social service delivery during the Program period was limited. Since the local government system was subsequently abolished, the sustainability of Program reform is irrelevant. The Balochistan Local Government Act, 2010 has substantially reversed devolution and limited social sector functions of local governments.

E. Institutional Development and Impact

50. The contribution of the Program to institutional development is negligible. With the cancellation of the undisbursed amount of the loan because second tranche actions were not completed, Program outputs were generally not achieved. As per the design of the policy matrix, outputs would have been able to contribute to better institutional arrangements only if second tranche actions had been fulfilled. Sector strategies and expenditure norms were not approved. Performance grants, though initially implemented through the PFC, became irrelevant after the new local government system was introduced in 2010. Although the increase in public expenditure for promoting gender equality across sectors was effected through performance grants, the implementation of the gender reform action plan was weak, marginally improving life for women. The net enrollment rate for girls in middle school increased slightly from 13% in 2004/05 to 15% in 2008/09.22

IV. OVERALL ASSESSMENT AND RECOMMENDATIONS

A. Overall Assessment

51. The program is rated unsuccessful with an overall score of 0.2 (Appendix 7). Program design was rated partly relevant as it aligned with the ADB country partnership strategy and the human development goals of the Balochistan poverty reduction strategy. However, its multisector approach and primary reliance on the BLGO as a mechanism for strengthening devolved social service delivery proved to be problematic. Changing political commitment to local government reform in line with the BLGO and capacity constraints were major factors contributing to the Program’s unsuccessful outcome. At the provincial level, this adversely affected the administration of the TA loan. At the district level, capacity constraints hindered the effective implementation of the local government framework and the efficient and timely utilization of additional financial resources provided under the Program.

52. Program oversight by the PSC and coordination by the PPO remained weak. Reporting on the release and utilization of performance grants by districts, the PPO, and the Finance Appendix 2 2 Appendix Department was irregular. The government’s commitments at the time of Program design and loan signing were not sustained during implementation. Policy reversals and failure to utilize the TA loan made achieving the outcome improbable. At Program closing, 7 of the 21 policy actions for the second tranche required waivers and 13 were only partly complied with.

B. Lessons

53. Executing agency capacity. Capacity in the executing and implementing agencies was expected to be built through the accompanying TA loan. Program outputs like developing sector strategies, implementing expenditure tracking and service delivery surveys, training communities to manage and monitor social service delivery, and other outputs critical to achieving Program outcome depended on effective TA loan administration. However, executing agency capacity to administer the TA loan was inadequate. Given the capacity constraints in the province, more frequent and regular TA review missions and direct support through ADB- administered TA could have improved Program implementation. However, security concerns made it difficult to attract and retain good consultants.

22 Government of Pakistan, Federal Bureau of Statistics. 2010. Pakistan Social and Living Standards Measurement Survey 2008-09. Islamabad.

54. Sector focus of policy reform. Given provincial and district capacity limitations, a Program focusing on a single sector would have had a better chance of success. Sustained support in one sector can be less demanding for both the government and ADB. Multi-sector policy reform interventions involving governance reform are difficult to manage, and a new system of local political governance imposed by the federal government added further complications.

55. Reform sequence. The sequence of reforms in the policy matrix was such that all substantial results could be achieved only after the completion of second tranche actions. In addition, liberal assessment of compliance allowed the release of the first tranche without achieving much toward sustainable outputs. More result-oriented sequencing of policy actions for the first tranche could have achieved some sustainable improvements in social service delivery.

C. Recommendations

1. Program Related

56. As agreed by the government, counterpart funds generated from the proceeds of the ADF program loan and grant were to be transferred as performance grants for investments in the social sectors. The government should obtain full utilization reports from the districts for the funds that were transferred as performance grants, providing complete details of activities undertaken using counterpart funds generated from the proceeds of the ADF loan and grant.

2. General

57. For programs involving governance reform, it is important to carry out a more comprehensive assessment of political commitment and dynamics through wider stakeholder consultation at the fact-finding stage. A comprehensive assessment of capacity in the executing and implementing agencies can improve the design of TA accompanying a program loan. The scope of program interventions should be adjusted in line with assessed capacity. Further, it is important that program reforms are scheduled in a sequence that reflects measureable and sustainable progress at each milestone, such as tranche release or subprogram.

DESIGN AND MONITORING FRAMEWORK

Data Sources/Reporting Comments Design Summary Performance Targets/Indicators Mechanisms Assumptions and Risks A. Program Impact Balochistan Province By 2010, in Balochistan Province: Economic and social sector Assumptions MICS 2010 final report is accelerating progress statistics not available. toward MDGs and Net primary school enrollment Macroeconomic stability and Comparison of PIHS Balochistan’s poverty rate increased from 24% in 2004 PIHS and other household surveys growth 2001/02 and PSLM to 40% including MICS reduction strategy Sustained political 2004/05 and 2008/09 relating to education, Infant mortality rate reduced from Third-party assessments & commitment to human show some improvement health, and WSS 104 per 1,000 live births in 2004 validation development in the following to 80 indicators: Risks (i) net primary school Total fertility rate reduced from 7 enrolment from 37% in 2004 to 4 Diminished political commitment to in PSLM 2004/05 to Full child immunization increased decentralization 44% in PSLM 2008– from 24% in 2004 to 70% 09; Attention diverted by security (ii) full child Delivery by skilled attendants situation immunization from increased from 21% in 2004 to 24% in PIHS 35% 2001/02 to 43% in PSLM 2008/09; and Access to excreta disposal (iii) access to improved increased from 40% in 2004 to sanitation from 67% 56% in PSLM 2004-05 to 80% in PSLM 2008- 09. B. Program Outcome Broader supply and By 2010 in Balochistan Province: PIHS and other household surveys Assumptions The MICS 2010 final more equitable delivery income, gender, ethnicity, and including MICS report is not available. Effective targeting of social of devolved education, geographic inequities in access to Education and health information services to the poor and health, and WSS social services reduced by 5% systems and reports of provincial females services by public and from baseline in 2004 line departments 1 Appendix private providers in Demand-side factors Local governments receiving Balochistan Province, Third-party assessments & adequately addressed predictable transfers of finances achieved mainly by validations strengthened district for social sector delivery in Adequate capacity of district accordance with the BLGO, 2001 Special surveys for some indicators sector managers and frontline

governments service providers Local governments with power to

PFC annual report on performance

post, transfer, and discipline grants Availability of qualified 15 frontline service providers in frontline providers in public

16 Data Sources/Reporting Comments Design Summary Performance Targets/Indicators Mechanisms Assumptions and Risks accordance with the BLGO, 2001 Annual budget documents and private sectors, particularly female providers 1 Appendix Private sector with increased role Notification of local governments’ in delivering publicly funded social transfer, posting, and disciplinary Active involvement of NGOs, services power CCBs, CBO, and SMCs Civil society (NGOs, CBOs, Annual reports of district Risks CCBs, SMCs) given active role in governments on actions taken social sector service delivery Fiduciary risks Provincial program office reports, Awareness of HIV/AIDS increases including program unit feedback Major natural disasters by 10% from baseline Performance grant Malaria incidence reduced by mechanism subjected to 10% from baseline bureaucratic wrangles and red tape Tuberculosis detection rate improved by 10% and cure rate improved by 5% from baseline C. Policy Reform Outputs PFC award, annual budget Assumptions Outputs are discussed in 1. Provincial efficiency PFC award announced consistent documents, and annual PSDP detail in the main report’s incentives to strengthen with fiscal needs of local documents Government committed to Program Outputs section and clarify the governments for the devolved strengthening devolved social II.B and in Appendix 2. relationship between social sectors, including salary service delivery provincial and local and non-salary recurrent and governments development expenditures

Financial allocations to high- priority social sector services increased

Performance grants to local

governments for social sectors Annual budget documents Adequate and timely

initiated and expanded provision of skilled staff and facilities

Overlapping functions of PFC annual report on performance Program ownership by provincial and local governments grants in devolved social service delivery government agencies and eliminated, and government civil society departments and devolved

agencies realigned with respect to devolved social services Progress reports from provincial Active dialogue with civil program office society and NGO support Roles and staffing of provincial government departments pertaining to social sectors

Data Sources/Reporting Comments Design Summary Performance Targets/Indicators Mechanisms Assumptions and Risks rationalized

Sector strategies, policy frameworks, and action plans

prepared by provincial departments and local governments to improve equity, efficiency, effectiveness, and quality of social service delivery Continued and consistent government, NGO, and civil Administrative orders and progress society support reports from provincial program office Timely implementation of effective, predictable Review of strategy papers performance grant flow, and smooth cash flow to local governments

Risks Political and bureaucratic instability Possible deterioration in security situation Major natural disasters 2. Managerial power Local governments authorized to Notification of rules and aligned to strengthen make postings and transfers in administrative orders the relationship between social sector the local governments Education and health staff better and providers of social Notification of disciplinary powers of

recruited and deployed; 1 Appendix services local governments accountability of social service providers enhanced; and steps taken to strengthen the monitoring and evaluation of social services

MIS for social sectors is improved

Performance grants provided for 17

in-service training of service

18 Data Sources/Reporting Comments Design Summary Performance Targets/Indicators Mechanisms Assumptions and Risks providers: teachers and supervisors in education; and 1 Appendix nurses, dispensers, paramedics, and technicians in health 3. Citizens’ voice and Grievance-redress mechanisms Notification of grievance-redress empowerment to developed at all tiers of procedures strengthen the government, especially at service relationship between delivery level citizens and policy makers (voice) and Stakeholders increasingly citizens and service involved in managing social sector facilities providers Complaint cells in all local (empowerment) PTSMCs, CCBs, and water user governments as required under the committees formed, empowered, BLGO; annual report of local and strengthened, and hospital government commission boards introduced in district and 50-bed hospitals 4. Private sector Provincial program office report engagement, including nonprofit organizations, in social service delivery Actions taken to promote self- regulation by private education and health sectors Provincial program office report Strategy and action plan prepared and implemented to promote public–private partnership and develop incentives for increased private sector participation in social service delivery

Innovative steps taken in social sector delivery, especially with the help of the private sector Provincial program office report PFC annual report on performance grants D. Performance Grant Outputs 5.1 Policy support MDGs: District governments’ monthly, Assumptions See comments against quarterly, and annual reports to the A. Program Impact 5.2 Municipal Fewer underweight children (MDG PFC and/or the Finance Department above. infrastructure grants 1) PFC annual report on performance Performance grants lead to

Data Sources/Reporting Comments Design Summary Performance Targets/Indicators Mechanisms Assumptions and Risks Higher net enrollment in primary grants more efficient use of 5.3 Capacity-building education and increased primary resources and effective social grants school completion (MDG 2) service delivery

5.4 ADF grant for Less gender disparity in primary Effective monitoring and controlling HIV/AIDS and secondary education and PIHS, MICS, and QSDS financial management of and other infectious increased middle school performance grants enrollment for girls (MDG 3) diseases Lower child mortality (MDG 4) Risk Improved maternal health (MDG PETS

5) Delayed release of funds to Improved access to safe drinking Third-party assessment & validation water and basic sanitation (MDG district governments 7) in the rural areas Improved access to safe drinking Reports of provincial program office water and basic sanitation (MDG

7) in urban areas

Training of permanent staff to improve capacity of district sector Audit reports managers and service delivery staff Systems improvements implemented HIV/AIDS, tuberculosis, malaria, Reports of provincial program office and other infectious diseases brought under control (MDG 6): (i) increased awareness of HIV/AIDS pandemic and measures to control its spread; (ii) improved tuberculosis case

detection and treatment 1 Appendix through DOTS; (iii) improved control of malaria vectors, and detection and treatment of malaria cases;

and

(iv) Childhood infectious diseases

controlled through improved 19 child immunization, treatment

20 Data Sources/Reporting Comments Design Summary Performance Targets/Indicators Mechanisms Assumptions and Risks of ARI, and control of childhood diarrhea 1 Appendix 6. Technical Assistance Outputs 6.1 Studies Sector strategies, rationalized PPO annual report on BDSSP Assumptions No output achieved, as roles and staffing, performance no consultant was criteria and incentives, recruitment Quarterly and annual progress High-quality work by recruited under the TA

and contract appointments, reports of TA regular review consultants hired under TA loan. missions integration of vertical programs, Timely hiring of TA consultant private sector self-regulation, and procurement expenditure allocation yardsticks, essential health services package, and strategy and action plan for public–private partnership and private sector 6.2 Other TA support Participation in delivering social Risk Only 190 social sector services and capacity-building and district staff were activities for social sector Delayed approval of PC II provided training under managers and service providers, the TA loan. expanded performance grants, MIS and monitoring evaluation systems, public expenditure tracking surveys, quantitative service delivery surveys, third- party assessment & validation, citizen report cards D. Activities and Milestones E. Inputs Provincial steering committee established and meeting at least quarterly ADB: $130 million OCR Only $110 million from OCR, $22.5 million from PPO (December 2005) ADB: $65 million ADF loan ADF loan, $0.126 million Program units in provincial Finance Department and Bureau of Statistics (December 2005) ADB: $5 million TA loan from TA loan, and $2.5 million from ADF grant Consultants start working with PPO; program units; and, where required, provincial departments of ADB: $5 million ADF grant disbursed. education, health and PHE (June 2006) Government resources (TA): Program units in 28 districts and selected TMAs (June 2006) $1.25 million 1st tranche conditions fulfilled & 1st tranche released (by July 2006) ADB: staff time for support and review (1 person-month 2nd tranche conditions fulfilled & 2nd tranche released (by July 2008) per year) Performance grants released to districts: Government of Balochistan (i) 1st year grants, annual installment for single-tranche grants, and quarterly installments counterpart staff support: (beginning July 2006) program coordinator and (ii) 2nd year grants, annual installment for single-tranche grants, and quarterly installments

Data Sources/Reporting Comments Design Summary Performance Targets/Indicators Mechanisms Assumptions and Risks (FY2008) other staff for 4 years. (iii) 3rd year grants, annual installment for single-tranche grants, and quarterly installments (FY2009) (iv) 4th year grants, annual installment for single-tranche grants, and quarterly installments (FY2010) TA activities (July 2006–June 2010) Program review: periodic reviews & midterm review 18 months after loan effectiveness Continued dialogue with stakeholders to effectively promote social service delivery (September 2005–June 2010) ADB = Asian Development Bank, ADF = Asian Development Fund, ARI = acute respiratory infection, BLGO = Balochistan Local Government Ordinance, BDSSP = Balochistan Devolved Social Services Program, CBO = community-based organization, CCB = citizen community board, DOTS = directly observed treatment strategy (for tuberculosis), FY = fiscal year, HIV/AIDS = human immunodeficiency virus/acquired immunodeficiency syndrome, MDG = Millennium Development Goal, MICS = multiple indicator cluster survey, MIS = management information system, NGO = nongovernment organization, OCR = ordinary capital resources, PC II = Planning Commission Form 2, PETS = public expenditure tracking survey, PFC = provincial finance commission, PHE = public health engineering, PIHS = Pakistan integrated household survey, PPO = provincial program office, PSDP = public sector development program, PSLM = Pakistan social and living standards measurement survey, PTSMC = parent–teacher school management committee, QSDS = quantitative service delivery survey, SMC = school management committee, TA = technical assistance, TMA = tehsil or town municipal administration, WSS = water supply and sanitation.

Appendix 1 Appendix

22 IMPLEMENTATION STATUS OF POLICY MATRIX UNDER THE PROGRAM

Appendix 2 Appendix Policy Reform Area First Tranche Actions Second Tranche Actions Documentation Provided & Remarks Policy Outcome 1: Provincial Efficiency Incentives

1.1. Budget A.1.1.1. The Government of B.1.1.1. The government disburses the T.1: Of the 29 program districts, performance grants certainty for Balochistan disburses the performance grants for the social sectors were disbursed to 19 districts (notification of the education, performance grants for the social in all district governments and enhances Finance Department on 16 May 2006). The health, and WSS sectors to most district governments. the scope of performance grants. performance grants include allocations for health, (social sectors) education, and water supply and sanitation sectors. T.2: Performance grants were disbursed to only 6 districts. Program units were asked to demonstrate that districts complied with the BPGR, 2005 on eligibility and access. The government requested a partial waiver, to which ADB did not agree. 1.2. Autonomy in A.1.2.1. The government develops B.1.2.1. The government (i) notifies the T.1: Working groups have been constituted budgets and proposals for rationalizing the roles amended government rules of business (notification of the secretary of health on 13 January administrative and staffing of social sector and (ii) rationalizes the roles and staffing 2006, secretary of the PHED on 11 January 2006, actions departments in line with the BLGO, of social sector departments in line with and secretary of education on 30 March 2006). First 2001. the BLGO, 2001. draft proposals from social sector departments finalized in line with the Balochistan Local Government Ordinance, 2001. Draft proposals are available on the government website. T.2: Rules of business amendment and the proposals for rationalizing the roles and staffing of social sector departments remained in draft form. 1.3. Improved A.1.3.1. Provincial education, health, B.1.3.1. The government approves T.1: Working groups are constituted (notification of quality of social and PHE departments develop, in education, health, and WSS sector the secretary of health on 5 April 2006, secretary of services consultation with district strategies. the PHED on 11 January 2006, and secretary of governments, sector strategies that education on 30 March 2006). A consultation will expand access to basic social workshop is undertaken with district governments, services for the poor and females. civil society, and key stakeholders on 7–8 June 2006. The first draft of sector strategies is finalized and available on the government website. T.2: Not complied with for education. ADB requested the PPO to send evidence of approval for Health and WSS sector strategies. A.1.3.2 The government prepares B.1.3.2 The government approves T.1: Working groups are constituted (notification of expenditure allocation norms in line expenditure allocation norms in line with the secretary of health on 4 April 2006 and the with minimum standards for social minimum standards for social sectors that secretary of education on 30 March 2006). First draft sectors that will have direct benefits will directly benefit the poor and females. expenditure allocation norms are prepared and for the poor and females. available on the government website. T.2: Approval submitted only through PSC minutes,

Policy Reform Area First Tranche Actions Second Tranche Actions Documentation Provided & Remarks which is insufficient. Government approval is not received. The Finance Department did not approve the norms due to quality concerns. A.1.3.3. The government notifies B.1.3.3. The government notifies safe Clinical Laboratory Rules, 2005 notified and clinical laboratory rules and develops blood transfusion rules. published in the Balochistan Gazette on 20 August safe blood transfusion rules. 2005. The first draft of safe blood transfusion rules is finalized, notified as the Blood Transfusion Rules, 2005, and published in the Balochistan Gazette on 24 September 2005 (and is available on the government website). T.2: The government notified safe blood transfusion rules. Policy Outcome 2: Managerial Power 2.1. Authority in A.2.1.1. The government undertakes B.2.1.1. The government, based on the T.1: The working group is constituted (notification of staff-related a review of district governments’ review, notifies powers for district the secretary of education on 30 March 2006). The matters devolved existing powers to take actions under governments to take actions under first draft of the amendments is finalized by the to district Balochistan Removal from Services Balochistan Removal from Services Services and General Administration Department governments Ordinance, 2001 for regular and Ordinance, 2001 for regular and contract and is available at the government website. contract employees in the education, employees in the education, health, and T.2: Waiver was requested by the government, but health, and PHE departments. PHE departments. ADB did not agree. B.2.1.2. The government notifies powers T.2: Waiver was requested by the government, but devolved to district governments for the ADB did not agree. posting and transfer of staff with basic scale 1–16 in the education, health, and PHE departments. A.2.1.3. The government, in B.2.1.3. The government approves the T.1: Working groups are constituted (notification of consultation with local governments, incentive system for sector managers and secretary of the PHED on 11 January 2006 and the develops an incentive system for service delivery staff, and the government secretary of education on 30 March 2006). sector managers and service delivery and local governments implement it. Consultation workshop with local governments staff, including residential benefits for conducted on 7–8 June 2006. The first draft female staff. proposal was finalized and is available on the government website.

T.2: Evidence of approval and implementation was 2 Appendix not provided. 2.2. Adequate A.2.2.1. The government establishes, B.2.2.1. The government and district T.1: Consultations with district governments were deployment of in consultation with district governments fill at least 25% of vacant conducted on 7–8 June 2006. The first draft staff governments, facility-specific non- positions identified as of July 2006 in the guidelines were finalized by the Services and regularizable contractual appointment education and health sectors through General Administration Department in April 2006

guidelines. these contractual appointment guidelines. and are available on the government website.

T.2: The PPO submitted documentation indicating 23 that the required number of positions had been filled but not through the newly established contractual

Policy Reform Area First Tranche Actions Second Tranche Actions Documentation Provided & Remarks

appointments mode. The government requested a 2 Appendix partial waiver, but ADB did not agree.

2.3. Improved A.2.3.1. The government develops, in B.2.3.1. The government approves and T.1: The working group is constituted (notification of capacity of consultation with district implements the MIS strategy for WSS and the secretary of education on 30 March 2006). district and governments, a strategy for a WSS strengthens the MISs for health and Consultations with district governments were facility managers MIS and initiates strengthening the education. conducted on 7–8 June 2006. The first draft strategy in the social MISs for health and education. for WSS and an action plan for health and education sector were finalized. The draft strategy is available on the government website. T.2: The approved MIS strategy for WSS was not provided, nor was evidence of implementation. Supporting documentation showing strengthened health and education sectors was not submitted. A.2.3.2. The government develops, in B.2.3.2. The government approves the T.1: Working groups are constituted (notification of consultation with district human resource development action plan the secretary of health on 4 April 2006, secretary of governments, a human resource for district and facility social service education on 30 March 2006, and secretary of the development action plan for district managers, and the government and most PHED on 11 January 2006). Consultations with and facility social service managers district governments initiate implementing district governments were conducted on 7–8 June and providers, both in-service and the action plan. 2006. The first draft action plan was finalized and is pre-service. available on the government website. T.2: Approved plans were not submitted.

Policy Outcome 3: Citizens’ Voice and Empowerment 3.1. Grievance- A.3.1.1. At least 50% of district B.3.1.1. All district governments establish T.1: Working groups are constituted (notification of redress governments establish complaint- complaint-redress mechanisms in social the secretary of health on 3 January 2006 and mechanism redress mechanisms in social service service facilities. secretary of education on 30 March 2006). facilities. Instructions to district governments were issued (letter of the director of education on 14 July 2006 and letter of the provincial coordinator on 28 April 2006). Complaint-redress mechanisms were established in facilities in most program districts, and compliance is being reported and monitored. Information is available at the government website. T.2: District officers provided certification for health and WSS, but supporting documentation for education was not provided. A.3.1.2. The government takes steps B.3.1.2. (i) Most local governments T.1: Monitoring rules were notified by the secretary to strengthen local government observe the monitoring rules introduced of the Balochistan Local Government Board through monitoring committees by (i) notifying by the government. (ii) Local government the Balochistan Local Government (Monitoring) monitoring rules including those councils approve the bylaws, and local Rules, 2006, published in the Balochistan Gazette relating to the inclusion of women government monitoring committees on 4 August 2006. Model bylaws were disseminated

Policy Reform Area First Tranche Actions Second Tranche Actions Documentation Provided & Remarks members and making their role observe the bylaws. to district governments for adoption through the effective and (ii) developing and BLGO, 2001, as amended from time to time. disseminating draft bylaws for local T.2: Evidence of approval of bylaws and their government monitoring committees. implementation was required. Meetings records and minutes were not submitted. 3.2. Community A.3.2.1. The government develops, in B.3.2.1 The government approves and, T.1: Consultation workshop with district management of consultation with district along with district governments, governments conducted on 7–8 Jun 2006. First draft social service governments, a communication implements the communication strategy to strategy finalized on 11 Feb 2006 and is available at facilities strategy to strengthen community strengthen community involvement in the the government website. involvement in the management of management of social sector facilities. T.2: Evidence on communication strategy approval social service facilities. by the provincial government and the districts was not provided. It was approved only by the PSC. A.3.2.2. In at least 30% of districts, B.3.2.2. In most districts, most PTSMCs T.1: The working group was constituted (notification most PTSMCs have been formed and have been formed and trained. of the secretary of education on 30 March 2006). trained. Committees have been formed and trained in nine districts. Information is available at the government website. T.2: Evidence of PTSMCs being formed predated the program period. A proposal for training was submitted, but ADB did not approve certain activities under the TA in response to limited compliance and government commitment to the program. A.3.2.3. The government develops, in B.3.2.3. The government notifies T.1: Consultation workshop with district consultation with district guidelines, and most district governments governments conducted on 7–8 June 2006. The governments, guidelines for hospital establish hospital management boards in Health Department finalized first draft guidelines, management boards for district and district and 50-bed hospitals. which are available at the government website. 50-bed hospitals. T.2: Generic instructions from the provincial Health Department for the constitution of hospital management boards and/or committees were provided as evidence. Evidence of district management boards and/or committees including electing the private sector or civil society members was not provided.

3.3. Civil society A.3.3.1. The government develops B.3.3.1. The government implements the T.1: The Finance Department finalized the public 2 Appendix monitoring of PETS as provided for in the BPGR, first PETS as provided for in the BPGR, expenditure tracking survey, and a draft is available social service 2005. 2005 and makes their results available to on the government website. delivery and the public. T.2: Waiver was requested by the government, but citizen oversight ADB did not agree.

Policy Reform Area First Tranche Actions Second Tranche Actions Documentation Provided & Remarks

A.3.3.2. The government develops a B.3.3.2. The government implements the T.1: The QSDS is finalized by the Planning and 2 Appendix QSDS. QSDS and makes the result available to Development Department’s Bureau of Statistics, and the public. a draft is available at the government website. T.2: Waiver was requested by the government, but ADB did not agree. A.3.3.3. The government and Quetta B.3.3.3. The Quetta city district T.1: The first draft of the citizen report card is district government develop citizen government implements the citizen report developed by Local Government Department and a report card in Quetta city district. card and makes results available to the draft is available on the government website. public. T.2: Waiver was requested by the government, but ADB did not agree. Policy Outcome 4: Private Sector Engagement 4.1. Private A.4.1.1. The government develops, in B.4.1.1. The government approves and, T.1: Working groups are constituted (notification of sector consultation with district along with district governments, initiates the secretary of education on 30 March 2006, participation in governments, a strategy to promote the implementation of the strategy to secretary of health on 4 April, and secretary of the social service public–private partnership in social promote public–private partnership in PHED on 18 March 2006). A consultation workshop delivery service delivery. social service delivery. with district governments and other stakeholders was conducted on 7–8 June 2006. The first draft proposals for government consideration were prepared and are available at the government website. T.2: The PPO submitted an unapproved strategy to promote PPPs in health but no evidence of implementation. No strategy linked to education or WSS was submitted. 4.2. Regulation A.4.2.1. The government develops, in B.4.2.1. The government approves and, T.1: Working groups in education and health of private social consultation with district along with district governments, initiates departments were constituted (notification of the services delivery governments, a strategy for the self- the implementation of the strategy for the secretary of education on 30 March 2006 and the regulation of the private sector in self-regulation of the private sector in secretary of health on 4 April 2006). A consultation education and health. education and health. workshop with district governments and other stakeholders was conducted on 7–8 June 2006. The first draft proposal for health has been prepared, and a draft proposal for education is being finalized. T.2: Evidence submitted for health predates the loan agreement. Nothing was submitted for the private sector in education. Asian Development Bank, ADF = Asian Development Fund, BLGO = Balochistan Local Government Ordinance, BPGR = Balochistan Performance Grant Rules, 2005, MIS = management information system, PETS = public expenditure tracking survey, PHED = public health engineering department, PPO = provincial program office, PTSMC = parent–teacher school management committee, QSDS = quantitative service delivery survey, T.1 = first tranche, T.2 = second tranche, TA = technical assistance, WSS = water supply and sanitation. Source: Asian Development Bank.

Appendix 3 27

RELEASE AND UTILIZATION OF PERFORMANCE GRANTS

Table A3.1: Details on the Release and Utilization of Performance Grants by District, as of 31 October 2010

Funds Financial Released Utilization Utilization District (PRs million) (PRs million) (%) Khuzdar 41.56 41.46 99.75 Kech 41.56 10.40 25.02 Nasirabad 20.78 20.39 98.13 Pishin 20.78 15.46 74.40 Killa Abdullah 39.28 19.94 50.75 Killa Saifullah 40.78 20.78 50.96 Zhob 29.06 13.88 47.76 Kohlu 40.78 12.98 31.83 Mastung 41.56 20.78 50.00 Kalat 41.56 41.56 100.00 Panjgur 41.56 17.53 42.18 Kharan 40.78 36.47 89.44 Awaran 41.56 20.82 50.09 Jaffarabad 20.78 20.46 98.47 Bolan/Kachhi 41.56 35.45 85.29 Changi/Dalbandin 41.56 20.78 50.00 Gawadar 41.56 38.86 93.50 Ziarat 21.56 16.30 75.60 Sibi 40.78 20.78 50.96 Shirani 40.78 20.78 50.96 Jhal Magsi 20.78 14.88 71.61 Nushki 21.56 14.86 68.91 Dera Bugti 41.56 20.49 49.30 Musakhail 20.78 8.41 40.49 Loralai 20.78 20.32 97.78 2 Appendix Lasbela 41.56 3.45 8.30 Harnai 43.12 17.06 39.56 Quetta 28.28 23.50 83.08 Washuk 41.56 … …

Barkhan 41.56 14.79 35.58

Total 1,051.72 603.60 57.39 27 … = data not available. Note: Counterpart funds generated from the proceeds of Loan 2203–PAK(SF) totaled PRs1,350 million. Source: Government of Balochistan. 2010. Completion Report: Balochistan Devolved Social Services Program in Pakistan. Manila.

28 Appendix 3

Table A3.2: Details on the Release and Utilization of Performance Grants for HIV/AIDS and Other Infectious Diseases, as of 31 October 2010

Funds Financial Released Utilization Utilization District (PRs million) (PRs million) (%) Khuzdar 5.00 5.00 100.00 Kech 5.00 1.90 38.00 Nasirabad 5.00 2.76 55.22 Pishin 3.00 2.99 99.97 Killa Abdullah 4.00 2.99 74.88 Killa Saifullah 2.00 ...... Zhob 3.00 ...... Kohlu 5.00 1.10 22.00 Mastung 5.00 3.00 60.00 Kalat 5.00 5.00 100.00 Panjgur 5.00 2.49 49.80 Kharan 5.00 5.00 100.00 Awaran 5.00 3.00 60.00 Jaffarabad 0.00 ...... Bolan/Kachhi 5.00 3.00 60.00 Changi/Dalbandin 5.00 3.00 60.00 Gawadar 4.00 1.96 49.00 Ziarat 3.00 ...... Sibi 5.00 3.00 60.00 Shirani 3.00 3.00 100.00 Jhal Magsi 0.00 ...... Nushki 3.00 3.00 100.00 Dera Bugti 5.00 ...... Harnai 5.00 2.71 54.14 Loralai 3.00 3.00 100.00 Lasbela 5.00 ...... Harnai 3.00 ...... Quetta 5.00 ...... Washuk 5.00 ...... Barkhan 5.00 ...... Total 121.00 57.91 47.86 … = data not available. Note: Counterpart funds generated from the proceeds of ADF Grant 0028–PAK totaled PRs135 million. Source: Government of Balochistan. 2010. Completion Report: Balochistan Devolved Social Services Program in Pakistan. Manila.

Appendix 4 29

IMPLEMENTATION ARRANGEMENTS

Figure A4: Program Organization

Appendix 2 Appendix

30 Appendix 5

STATUS OF COMPLIANCE WITH LOAN COVENANTS

Covenants Reference in Loan Status of Agreement Compliance1

Program Steering Committee. The Program Steering Committee Schedule 5, para. 2 Complied with. (PSC) shall be established to (i) provide policy guidance for the Program; (ii) provide oversight function for the Program activities; (iii) monitor the performance of the Provincial Program Office (PPO) and the Program Units; and (iv) ensure the Program coordination with DSP, BRMP and other programs and projects. The Provincial Program Office. Within two weeks of the Effective Schedule 5, para. 3 Partly complied Date, Balochistan shall have established the Provincial Program with. Office (PPO), which shall (i) plan, coordinate, facilitate and monitor the implementation of the Program; (ii) coordinate activities, particularly capacity building, with DSP, BRMP and other programs; (iii) interact, communicate and coordinate on (a) policy reforms with provincial departments, and (b) the implementation of the Performance Grants through the Finance Department, and as required, the PFC; (iv) institute appropriate monitoring mechanisms through the Bureau of Statistics, Finance Department and third party validation through contracted NGOs; and (v) prepare reports documenting, analyzing and reviewing the progress and results of the Program and make such reports publicly available. The PPO shall be headed by a Program coordinator who shall be an Schedule 5, para. 4 Partly complied officer with basic grade of 19 or 20. The Program coordinator shall be with. assisted by one Program officer with basic grade of 17 or 18, two Program assistants, each with basic grade of 15 and support staff, as required. The Program coordinator, the Program officer, the two Program assistants and the support staff shall be working full-time at the PPO. Program Committees. Within two weeks of the Effective Date, Schedule 5, para. 5 Partly complied Balochistan shall ensure that a District Program Committee (DPC) with. shall have been established in each district government to oversee all matters in connection with the implementation of (i) the policy reforms 1 Appendix under the Program as they relate to the district administration; and (ii) the Performance Grants. The committee shall meet at least once a month. Within two weeks of the date a TMA has been declared eligible for the Schedule 5, para. 6 Not complied with. Performance Grant, Balochistan shall ensure that a TMA Program

Committee shall have been established. PUs. Within two months of the Effective Date, Balochistan shall Schedule 5, para. 7 Partly complied 30 ensure that a PU shall have been established in (i) the Finance with. Department; (ii) the Bureau of Statistics; and (iii) each of the 28 district governments. In the Finance Department, the PU shall be headed by an additional Schedule 5, para. 8 Partly complied secretary who shall be assisted by a Program officer and a database with. operator. It shall have the functions to (i) implement the Performance Grants using the framework of the BPGR 2005 and in accordance with the Program design and agreed upon fund allocation, distribution and management procedure; (ii) prepare, according to the Program design and targets, an annual plan for the Performance Grants for the following fiscal year to be approved by the PSC by the end February of each year; (iii) periodically review the Performance Grants and make such improvements as are necessary; (iv) pilot and institutionalize, with the assistance of the contracted NGO, public expenditure tracking surveys with third party validation and make such survey publicly available; (v) liaise with the PFC with regards to the

1 Assessment of compliance with covenants is based on review of program files and progress reports

Appendix 5 31 Covenants Reference in Loan Status of Agreement Compliance1 administration of the Performance Grants; (vi) ensure timely release of funds under the Performance Grants to local governments; (vii) prepare and submit to PSC an annual report on the Performance Grants and make such report publicly available; and (viii) establish linkage of the Program with BRMP and DSP. In the Bureau of Statistics, the PU shall be headed by the director of Schedule 5, para. 9 Partly complied the Bureau of Statistics who shall be assisted by a Program officer with. and one account assistant. Its functions shall be to (i) strengthen survey, data collection and analysis, monitoring and evaluation processes at the provincial level and in the district governments and establish channels for reporting and feedback from grass root levels; and (ii) pilot and institutionalize, with the assistance of the contracted NGO, quantitative service delivery surveys with third-party validation and make such survey publicly available. In each district, the PU shall be headed by the EDO finance and Schedule 5, para. 10 Not complied with. planning who shall be assisted by a Program officer and a database operator. Its functions shall be to (i) utilize the Performance Grants in accordance with the specific criteria and conditions of the Program and follow the process and procedures laid down for monitoring accounting and reporting in the BPGR 2005 and the specific conditions of the terms of partnership; (ii) ensure transparency, equity and accountability in the participatory planning, allocation and use of the Performance Grants; and (iii) encourage and facilitate citizen participation and involvement and public-private partnership in planning, monitoring and implementation of service delivery. Within two months of the date, a TMA has been declared eligible for Schedule 5, para. 11 Not complied with. the Performance Grant, Balochistan shall ensure that a PU shall have been established. Policy Dialogue. The Borrower shall continue timely policy dialogue Schedule 5, para. 12 Not complied with. with ADB on problems and constraints encountered during implementation of the Program and on desirable changes to overcome or mitigate such problems and constraints, and on further reforms to strengthen the social services sector. The Borrower shall keep ADB informed of policy discussions with Schedule 5, para. 13 Not complied with. other multilateral and bilateral agencies that have implications for the implementation of the Program, and shall provide ADB with an opportunity to comment on any resulting policy proposals. The Borrower shall take ADB’s views into consideration before finalizing and implementing any such proposals. Continuity of Reforms. The Borrower shall ensure (a) that the Schedule 5, para. 14 Not complied with. objectives achieved, policies adopted, and actions taken prior to the date of the Agreements, as set forth in the development policy letter and accompanying Policy Matrix shall continue to be in full force and effect for the duration of the Program period and subsequently; (b) proper and timely achievement, adoption, compliance and implementation of the objectives, policies, and actions to be achieved, adopted, complied with, and implemented as set forth in the development policy letter and accompanying Policy Matrix; and (c) that once achieved, adopted, complied with, and implemented, such objectives, policies, and actions shall continue to be in full force and effect during and subsequent to the Program period. Counterpart Funds. The Borrower and Balochistan shall ensure that Schedule 5, para. 15 Complied with. the Counterpart Funds to be generated out of the proceeds of the Loan shall be used to restructure the Balochistan provincial debt through retirement of high-interest cash development loans. Reform Undertakings. Balochistan shall provide devolved social Schedule 5, para. 16 Not complied with. services appropriate administrative, financial, and operational autonomy and agrees that for areas where powers have not yet been

32 Appendix 5

Covenants Reference in Loan Status of Agreement Compliance1 devolved, within eighteen months of the Effective Date, a transition arrangement shall be formulated indicating when and how such powers shall be devolved. Within eighteen months of the Effective Date, Balochistan shall Schedule 5, para. 17 Not complied with. undertake appropriate actions to achieve clarity on the roles, responsibilities, and accountabilities of the provincial social sector departments in the context of the BLGO 2001 to ensure clear distinctions with the mandates of social sector departments in the district governments. Reform Commitments. Balochistan shall undertake long-term Schedule 5, para. 18 Partly complied commitment towards the improvement of the delivery of the social with. services in the Program area. Balochistan shall ensure that none of the functions and responsibilities Schedule 5, para. 19 Not complied with. devolved to the local government shall be recentralized. Balochistan shall ensure that the performance grant mechanisms and Schedule 5, para. 20 Partly complied procedures including on the eligibility of local governments, award of with. grant, and timely transfer of grant, monitoring, reporting, audit and other aspects shall be consistent with the BPGR 2005. Balochistan shall ensure continuous support and assistance to the Schedule 5, para. 21 Not complied with. local governments to improve their (a) capacity in budgeting and financial management; (b) planning, programming and design of development schemes and projects; (c) supervision and implementation of development works; (d) operation and maintenance; and (e) monitoring and evaluation. Program Implementation. Balochistan shall take all steps and Schedule 5, para. 22 Not complied with. actions, including provision of staff resources to ensure satisfactory completion in a timely manner of all Program outputs. Balochistan shall also implement the Program and monitor and evaluate its performance in line with the Program administration memorandum as agreed with ADB. Technical Assistance Loan Agreement. Balochistan shall ensure Schedule 5, para. 23 Not complied with. that in carrying out the policy measures and activities included under the Program, due consideration is given to the findings and recommendations of the activities under the Technical Assistance Loan Agreement. The Borrower and Balochistan shall allow ADB to provide comments on the findings and recommendations of the activities under the Technical Assistance Loan Agreement, for consideration in policy measures for this and upcoming programs in Balochistan or program of similar nature in Pakistan. Auditing, Reporting, Monitoring and Evaluation. Without limiting Schedule 5, para. 24 Partly complied the generality of Section 4.05 of this Agreement, Balochistan shall with. ensure that PPO submit to ADB quarterly and annual reports on the implementation of the Program, including the accomplishment of the targets and carrying out of the actions set out in the Policy Letter. Without limiting the generality of Section 4.05 of this Agreement, Schedule 5, para. 25 Partly complied within two (2) months of the Effective Date, Balochistan shall ensure with. that the PPO prepare and submit to ADB, the Program inception report which shall summarize actions so far taken and the further actions required for the implementation of the policy reforms under the Program. Within three (3) months of the completion of the Program, Balochistan Schedule 5, para. 25 Not complied with. shall ensure that the PPO prepare and submit the Program completion report to ADB. Within two months of the Effective Date, Balochistan shall set up a Schedule 5, para. 26 Partly complied Program performance management system to assess and evaluate with. the scope, implementation arrangements, benefits monitoring, progress and achievements of the Program as agreed with ADB and cause it to be operated within the Program period.

Appendix 5 33 Covenants Reference in Loan Status of Agreement Compliance1

The local governments that received the Performance Grants shall Schedule 5, para. 27 Partly complied maintain accounting, reporting and auditing procedures as prescribed with. in the BPGR 2005. Program Review. ADB and Balochistan shall undertake regular Schedule 5, para. 28 Partly complied review missions, at least on semi-annual basis, to (i) evaluate with. compliance with the policy reforms under the Program; (ii) advise on actions required to keep Program implementation on course; and (iii) review the coordination and collaborations of the Program implementation with other ongoing reforms including DSP and BRMP. ADB and Balochistan shall also conduct a comprehensive mid-term Schedule 5, para. 29 Not complied with. review in the second year of Program implementation to evaluate progress on policy reforms and the investments under the Performance Grants. Environment. Balochistan shall ensure that the Program shall be Schedule 5, para. 32 Partly complied carried out in accordance with the environmental rules and regulations with. applicable under the Borrower’s laws and regulations and ADB’s Environmental Policy (2002). Balochistan shall ensure that adequate environmental protection and safety measures shall be included in the design of the Program facilities and that the Program facilities shall be constructed, operated, and maintained in accordance with ADB’s Environmental Policy. Balochistan shall ensure that (i) the implementation of environmental monitoring and mitigation measures and the environmental plan; and (ii) the environmental criteria for the Performance Grants activities shall adhere to overall selection requirements and sector-specific requirements, shall be as agreed with ADB. Loan 2203 and Grant 0028 (in addition to above) The Borrower and Balochistan shall ensure that the Counterpart Schedule 5, para. 15 Partly complied Funds to be generated out of the proceeds of the Loan shall be used with. as Performance Grants in the social sectors until June 2010 in accordance with BPGR 2005. Balochistan further ensures that the performance Grant shall be transferred in addition to the regular PFC award transfers to the local governments, in which each eligible local government shall allocate a minimum of 20% of this Performance grant in each of the social sectors. The Borrower and Balochistan shall ensure that the Counterpart Schedule 5, para. 16 Partly complied Funds to be generated out of the proceeds of the Grant shall be used with. as Performance Grants until June 2010 to be utilized exclusively for any of the following activities (i) increase awareness of HIV/AIDS and measures to control its spread; (ii) improve case detection of tuberculosis and increase the access to tuberculosis treatment, mainly through directly observed therapy; (iii) control of malaria vectors, and detection and treatment of malaria cases; and (iv) control of childhood infectious diseases by increasing access to expanded immunization program. Loan 2204 (in addition to above) Balochistan shall ensure that certified copies of the audited accounts Schedule 6, para.16 Partly complied and financial statements and the report of the auditors (including the with. auditors’ opinion on the use of the proceeds of the Loan and compliance with the financial covenants of the Loan Agreement, as well as on the use of the procedures for imprest account/statement of expenditures) shall be submitted to ADB in English within six months after the end of each fiscal year.

34 Appendix 6

TECHNICAL ASSISTANCE COMPLETION REPORT OF TA 4673–PAK

Appendix 6 35

36 Appendix 7

OVERALL ASSESSMENT OF THE BALOCHISTAN DEVOLVED SOCIAL SERVICES PROGRAM

Weight Definition under ADB Rating Rating BDSSP Criterion (%) Guidelines Description Value Rating Score 1. Relevance 20 Relevance is the consistency Highly relevant 3 1 0.2 of a project’s impact and Relevant 2 outcome with the Partly relevant 1 government’s development Irrelevant 0 strategy and the ADB strategic objectives at the time of approval and evaluation, as well as the adequacy of the design. 2. Effectiveness 30 Effectiveness describes the Highly effective 3 0 0 extent to which the outcome, Effective 2 as specified in the design and Less effective 1 monitoring framework, either Ineffective 0 as agreed at approval or as subsequently modified, has been achieved. 3. Efficiency 30 Efficiency describes, ex-post, Highly efficient 3 0 0 how economically resources Efficient 2 have been converted into Less efficient 1 results, using the economic Inefficient 0 internal rate of return, or cost- effectiveness, of the investment or other indicators as measures and the resilience to risk of the net benefit flows over time. 4. Sustainability 20 Sustainability considers the Most likely 3 0 0 likelihood that human, Likely 2 institutional, financial, and Less likely 1 other resources are sufficient Unlikely 0 to maintain the outcome over its economic life. Overall Highly successful: Overall weighted average is greater than or equal to Unsuccessful 0.2 Assessment 2.7. (weighted Successful: Overall weighted average is greater than or equal to 1.6 average of the and less than 2.7. criteria above) Partly successful: Overall weighted average is greater than or equal to 0.8 and less than 1.6. Unsuccessful: Overall weighted average is less than 0.8 ADB = Asian Development Bank, BDSSP = Balochistan Devolved Social Services Program. Note: The performance rating follows ADB. 2006. Guidelines for Preparing Performance Evaluation Reports for Public Sector Operations. Manila.